rgw: fix zone delete message

Though this error message is hard to reach, it is still wrongly printed
as failed to create instead of failed to delete.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #8016 from ErwanAliasr1/evelu-15042

rpm,deb: remove conditional BuildRequires for btrfs-progs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8214 from athanatos/wip-feature-bits

Deprecate or free up a bunch of feature bits

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8162 from rjfd/wip-15142

rbd-mirror: use the mirroring directory to detect candidate images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8204 from rjfd/wip-15143

librbd: enable/disable image mirroring automatically for pool mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8172 from trociny/fix-test-rbd-mirror-position

test: rbd-mirror: compare positions using all fields

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8185 from trociny/wip-rbd-mirror-faster-exit

rbd-mirror: fix long termination due to 30sec wait in main loop

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8221 from jdurgin/wip-test-rbd-mirror

qa: add workunit to run ceph_test_rbd_mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: fix long termination due to 30sec wait in Mirror::run loop

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8191 from liewegas/wip-15180

qa/workunits/rest/test.py: don't use newfs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8101 from jjhuo/bluestore

os/bluestore: small fixes in bluestore StupidAllocator

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8178 from majianpeng/bluefs-closewriter

os/bluestore/BlueFS: Before reap ioct, it should wait io complete

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8220 from dillaman/wip-15202

journal: prevent race injecting new records into overflowed object

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: add workunit to run ceph_test_rbd_mirror

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8219 from jdurgin/wip-test-pool-create-cleanup

test_pool_create.sh: put test files in the test dir so they are cleaned up

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

journal: prevent race injecting new records into overflowed object

The recorded added after the overflow will not have been detached
from the object recorder.

Fixes: #15202
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8149 from trociny/test-rbd-mirror

test: rbd-mirror: add "switch to the next tag" test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8182 from trociny/wip-rbd-mirror-args

rbd-mirror: make remote context respect env and argv config params

Conflicts:
qa/workunits/rbd/rbd_mirror.sh (asok command format changed)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8159 from trociny/wip-rbd-mirror-asok

rbd-mirror: use pool/image names in asok commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test_pool_create.sh: put test files in the test dir so they are cleaned up

If they aren't cleaned up, tarball gitbuilders are unhappy.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8216 from ceph/wip-rbd-maint-ops

qa/workunits/rbd: new online maintenance op tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8183 from ceph/wip-rgw-fix-mdlog

Wip rgw fix mdlog

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

qa/workunits/rbd: new online maintenance op tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8212 from yehudasa/wip-15181

cls/rgw: fix use of timespan

Reviewed-by: Sage Weil <sage@redhat.com>

cls/rgw: fix use of timespan

Need to use make_timespan() when referring to seconds.

Fixes: #15181
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8209 from liewegas/wip-13992

ceph_test_rados_misc: shorten mount timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph_test_rados_misc: shorten mount timeout

This might fix #13992.

Signed-off-by: Sage Weil <sage@redhat.com>

os/filestore: fix warning

os/filestore/FileStore.cc: In member function ‘int FileStore::_zero(const coll_t&, const ghobject_t&, uint64_t, size_t)’:
os/filestore/FileStore.cc:3294:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       if (offset + len > st.st_size) {
                        ^

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7932 from theanalyst/mailmap/10.0.4

mailmap for 10.0.4

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa/workunits/rest/test.py: don't use newfs

It doesn't work anymore.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8121 from liewegas/wip-15136

mon: remove 'mds setmap'

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rbd: rbd-mirroring: Added unit tests to test image mirroring behaviour

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Fixed if clause in mirror_image_get_info

This fix targets the case when journaling is not enabled but
someone calls mirror_image_get_info.

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Disable image mirroring when journaling feature is disabled

Signed-off-by: Ricardo Dias <rdias@suse.com>
Fixes: #15143

rbd: rbd-mirroring: Enable image mirroring when journaling feature is enabled

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Enable image mirroring upon create with journaling feature

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8137 from ceph/wip-15146-ceph-dep

rpm: remove sub-package dependencies on "ceph"

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jun Huang <hjwsm1989@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #7838 from rldleblanc/pg_prime-enable

mon: enable 'mon osd prime pg temp' by default

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7939 from yangdongsheng/remove_v1

mon: standardize Ceph removal commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7721 from xiexingguo/xxg-wip-fixdc

os/filestore: fix result handling logic of destroy_collection

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7754 from xiexingguo/xxg-wip-14845

osd: fix inaccurate counter and skip over queueing an empty transaction

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7970 from Yan-waller/yj-wip-async

msg/async: _try_send trim already sent for outcoming_bl more efficient

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8127 from ceph/wip-15122

ceph_test_msgr: reduce test size to fix memory size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge branch 'wip-13990' of git://github.com/tchaikov/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8160 from yuyuyu101/jewel-spdk

os/bluestore/NVMEDevice: make IO thread using dpdk launch

Merge pull request #8163 from liewegas/wip-15151

osd/ReplicatedPG: clear watches on change after applying repops

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7359 from songbaisen/song9

mon: consider pool size when creating pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7674 from majianpeng/bluestore-sync-transaction

os/bluestore: make bluestore_sync_transaction = true can work.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7860 from majianpeng/bluestores-clone-overlay

os/bluestore: clone overlay data

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7930 from ceph/wip-debian-librgw-tests

debian: package librgw_file* tests

Merge remote-tracking branch 'gh/wip-strict-debian-deps'

Merge pull request #7957 from majianpeng/bluestore-read

os/filestore: require offset == length == 0 for full object read; add test

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #8039 from gregsfortytwo/wip-14962-greg

osd/ReplicatedPG: be more careful about calling publish_stats_to_osd()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8096 from liewegas/wip-15073

osd: ensure new osdmaps commit before publishing them to pgs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: use current period id when it is empty for mdlog list and shard lock/unlock

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8167 from theanalyst/mstart/rgw_ports

mstart: start rgw on different ports as well

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

osd: assert that we don't try to use an OSDMap > superblock.newest_map

handle_osd_map now adds OSDMaps to the cache as it receives them
but before they are stable.  Ensure that we don't try to use them.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: commit osdmaps before exposing them to PGs

handle_osd_map and the PGs use different sequencers when writing
their updates.  We therefore need to make sure new osdmaps are
committed to disk before we expose them to PGs, lest they update
their info to reference a new osdmap that hasn't actually
committed yet.

This doesn't happen with FileStore because transactions are
ordered when they are queued, but it does affect BlueStore.

Fix by splitting handle_osd_map into two phases, one that just
persists stuff, and the second half that publishes the new maps to
the rest of the OSD.

Fixes: #15073
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8170 from yehudasa/wip-rgw-sync-fixes-3

Wip rgw sync fixes 3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

os/bluestore/BlueFS: Before reap ioct, it should wait io complete.

When change write mode of FileWrite(in _flush_range) from buffer to
direct. osd met segment fault.
This because call _close_writer don't check related ios whether
complete.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

mon: remove 'mds setmap' command

This command has no discernable purpose, and is difficult and
dangerous to support anyway.

Fixes: #15136
Signed-off-by: Sage Weil <sage@redhat.com>

test: rbd-mirror: different log and asok location for local/remote contexts

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: make remote context respect env and argv config params

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8179 from zhjwpku/zhjwpku

doc/architecture.rst: remove redundant word "across"

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/architecture.rst: remove redundant word "across"

Signed-off-by: Zhao Junwang <zhjwpku@gmail.com>

rbd-mirror: command line options should override environment

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

os/bluestore: rename need_size to want_size in StupidAllocator.

Allocate() could return extent with size < need_size, so rename it per
Sage's suggestion.

Signed-off-by: Jianjian Huo <samuel.huo@gmail.com>

os/bluestore: avoid duplicated searches during allocating free extents

When StupidAllocator searches for free extents, if hint is not 0(most times),
it will search extents starting from hint and up in each applicable bin,
if not found, then search all extents in those bins again.

If the extent to find(>= needs) exists below hint and is not in the choose_bin(),
let's say it's in the bin_X; then during this twice searches, all extents from
hint and up in bins before bin_X will be searched twice. This patch will eliminate
those unnecessary searches.

Signed-off-by: Jianjian Huo <samuel.huo@gmail.com>

features: deprecate CEPH_FEATURE_OSD_PROXY_FEATURES

Can't be removed until CEPH_FeATURE_OSD_MIN_SIZE_RECOVERY is also
ready.

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_OSD_MIN_SIZE_RECOVERY

Can't be removed until CEPH_FEATUERE_OSD_PROXY_METADATA goes
as well.

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_OSD_TRANSACTION_MAY_LAYOUT

We can't remove this until the other 1<<46's also go.

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_OSD_OBJECT_DIGEST

We can't actually reuse this until all of the 1<<46 bits are
ready.

Signed-off-by: Samuel Just <sjust@redhat.com>

features,osd/: deprecate CEPH_FEATURE_OSD_REPOP feature bit

This allows us to rip out the code for supporting the old
repop format.  We can't actually reuse this bit until all
4 1<<46 bits are deprecated for one release.

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_OSD_SET_ALLOC_HINT

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_PACKED_RECOVERY

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_MON_SCRUB

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_OSD_FEATURE_SNAPMAPPER

Signed-off-by: Samuel Just <sjust@redhat.com>

features: free CEPH_FEATURE_OSD_HBMSGS, not used since before hammer

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_RECOVERY_RESERVATION

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_BACKFILL_RESERVATON

Signed-off-by: Samuel Just <sjust@redhat.com>

features: free CEPH_FEATURE_MON_GV bit, hasn't been used since before hammer

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_MON_NULLROUTE

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_CHUNKY_SCRUB

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_INDEP_PG_MAP

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_QUERY_T

Signed-off-by: Samuel Just <sjust@redhat.com>

features: free CEPH_FEATURE_OMAP bit, hasn't been used since before hammer

Signed-off-by: Samuel Just <sjust@redhat.com>

features: deprecate CEPH_FEATURE_MONCLOCKCHECK

Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: take a reference to the cr in async cr workers

and drop the cr when worker is done. The cr can be cleaned before
worker is done, so we should have it stick around as long as the
worker is alive.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: finalize reqs through a temporary set

can't call _finish_request() on iter->second when iterating, it clobbers the
map.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update data_log only when completing the op

It doesn't make sense to update it earlier, the zone that follows will not sync before
it sees the completion.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8169 from ceph/wip-15041

doc: initial draft of RBD mirroring admin documentation

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: initial draft of RBD mirroring admin documentation

Fixes: #15041
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: rbd-mirror: compare positions using all fields

Comparing by only entry_tid may give false positives.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: use pool/image names in asok commands

Now the commands look similar to rbd cache asok commands.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8168 from linuxbox2/rgw-ldap-fixes

Rgw ldap fixes

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw-ldap: conditional build

Permit building without LDAP support--support is enabled by default.
Tested with CMake and autotools standalone builds.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #8165 from tchaikov/wip-fix-rgw-encoding

rgw/rgw_common.h: fix the RGWBucketInfo decoding

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge remote-tracking branch 'gh/jewel'

osd: populate the trim_thru epoch using MOSDMap.oldest_map

instead of filling MOSDMap with the local oldest_map, we share
the maximum MOSDMap.oldest_map received so far with peers. That
way one OSD's failure to trim won't prevent it from sharing with
others that they are allowed to trim.

Fixes: #13990
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8158 from dillaman/wip-rbd-mirror-fixes

rbd-mirror: fix image replay test failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>