radosgw-admin: add explanation for EACCESS on 'realm pull'

References #15200

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: 'period commit' supplies user-readable error messages

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: parse error messages from failed requests

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8273 from wjin/fix

osd: fix log info

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8260 from liewegas/wip-ec-test

test/test-erasure-code.sh: disable pg_temp priming

Reviewed-by: Loic Dachary <ldachary@redhat.com>

osd: fix log info

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

Merge pull request #8267 from athanatos/wip-14309

osd: bail out of _committed_osd_maps if we are shutting down

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8266 from athanatos/wip-test-exit-value

Test exit values on test.sh, fix tier.cc

Reviewed-by: Sage Weil <sage@redhat.com>

test/librados/tier.cc: Fix Whiteout tests to force promotion

Otherwise, there may not actually be whiteouts created.

Fixes: 15177
Signed-off-by: Samuel Just <sjust@redhat.com>

qa/workunits/rados/test.sh: check subprocess return values

Fixes: #15165
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #8263 from tchaikov/wip-15031

msg/async: avoid log spam on throttle

Reviewed-by: Sage Weil <sage@redhat.com>

test/test-erasure-code.sh: disable pg temp priming

If we prime pg_temp, there is a race where this test expects an out OSD
to not be in the acting set but it will be there initially.

Fixes: #15211
Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: avoid log spam on throttle

increase the log level from 1 to 10, the log is printed when message
throttler kicks in.

Fixes: #15031
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8252 from xiexingguo/xxg-wip-15164

doc: fix wrong type of hyphen

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8239 from jcsp/wip-15223

mon: fix mixed-version MDSMonitor

Reviewed-by: Sage Weil <sage@redhat.com>

doc: fix wrong type of hyphen

Which makes picture of http://docs.ceph.com/docs/master/start/quick-start-preflight/
look sick.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7934 from ceph/wip-default-tcmalloc-cache-size

etc/sysconfig/ceph: set 128MB tcmalloc cache size

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #8244 from linuxbox2/jewel-ldap-fixes

rgw_ldap fixes

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8225 from SUSE/wip-radosgw-description

packaging: align radosgw package description

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8217 from ceph/wip-rbd-mirror-init

rbd-mirror: systemd and upstart init scripts

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8245 from theanalyst/fix/delete-zone-msg

rgw: fix error message for zone delete

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8243 from theanalyst/fix/zone-rm-msg

rgw: typo fix for zone delete message

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix error message for zone delete

Currently if a zone is not a part of a zonegroup, an error message
is printed that zone doesn't exist, and the zone gets deleted anyway.
Since this is exhibited only if the zone isn't a part of a zonegroup
and we allow creation of such zones, clarify that the zone wasn't a
part of zonegroup instead

Fixes: #14951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_ldap:  move update of s->perm_mask

This should happen on successful auth, not only when a new user
record is stored.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_ldap:  two bug fixes

1. only attempt auth on tokens which pass valid() check
2. ignore false positive search result (and fix result in that case)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #8240 from dillaman/wip-15224

cls_rbd: throw error if set_features mask included non-mutable feature

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: fix zone delete message

Though this error message is hard to reach, it is still wrongly printed
as failed to create instead of failed to delete.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #8130 from tchaikov/wip-15113

mon: do not panic at seeing requests without session

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

cls_rbd: throw error if set_features mask included non-mutable feature

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: fix mixed-version MDSMonitor

Encode old-style until mon quorum features indicate
jewel, forbid creating multiple filesystems, and
when creating a single filesystem use the legacy fscid.

Fixes: #15223
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #8016 from ErwanAliasr1/evelu-15042

rpm,deb: remove conditional BuildRequires for btrfs-progs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8214 from athanatos/wip-feature-bits

Deprecate or free up a bunch of feature bits

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8162 from rjfd/wip-15142

rbd-mirror: use the mirroring directory to detect candidate images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8204 from rjfd/wip-15143

librbd: enable/disable image mirroring automatically for pool mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

packaging: align radosgw package description

First, make the Debian package description mention that RGW aims to
implement the Swift API.

Second, replace the RPM package description with the Debian one, both for
consistency and because the Debian one is better.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8172 from trociny/fix-test-rbd-mirror-position

test: rbd-mirror: compare positions using all fields

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8185 from trociny/wip-rbd-mirror-faster-exit

rbd-mirror: fix long termination due to 30sec wait in main loop

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8221 from jdurgin/wip-test-rbd-mirror

qa: add workunit to run ceph_test_rbd_mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: fix long termination due to 30sec wait in Mirror::run loop

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8191 from liewegas/wip-15180

qa/workunits/rest/test.py: don't use newfs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8101 from jjhuo/bluestore

os/bluestore: small fixes in bluestore StupidAllocator

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8178 from majianpeng/bluefs-closewriter

os/bluestore/BlueFS: Before reap ioct, it should wait io complete

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8220 from dillaman/wip-15202

journal: prevent race injecting new records into overflowed object

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: add workunit to run ceph_test_rbd_mirror

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8219 from jdurgin/wip-test-pool-create-cleanup

test_pool_create.sh: put test files in the test dir so they are cleaned up

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

journal: prevent race injecting new records into overflowed object

The recorded added after the overflow will not have been detached
from the object recorder.

Fixes: #15202
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8149 from trociny/test-rbd-mirror

test: rbd-mirror: add "switch to the next tag" test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8182 from trociny/wip-rbd-mirror-args

rbd-mirror: make remote context respect env and argv config params

Conflicts:
qa/workunits/rbd/rbd_mirror.sh (asok command format changed)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8159 from trociny/wip-rbd-mirror-asok

rbd-mirror: use pool/image names in asok commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

packaging: added rbd-mirror startup scripts

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

upstart: new rbd-mirror daemon scripts

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

systemd: new ceph-rbd-mirror scripts

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

OSD: bail out of _committed_osd_maps if we are shutting down

Introduced in b839a06c1aba24ecf0780387a605192ff929f158
Fixes: 14309
Signed-off-by: Samuel Just <sjust@redhat.com>

test_pool_create.sh: put test files in the test dir so they are cleaned up

If they aren't cleaned up, tarball gitbuilders are unhappy.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8216 from ceph/wip-rbd-maint-ops

qa/workunits/rbd: new online maintenance op tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8183 from ceph/wip-rgw-fix-mdlog

Wip rgw fix mdlog

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

qa/workunits/rbd: new online maintenance op tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8212 from yehudasa/wip-15181

cls/rgw: fix use of timespan

Reviewed-by: Sage Weil <sage@redhat.com>

cls/rgw: fix use of timespan

Need to use make_timespan() when referring to seconds.

Fixes: #15181
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #8209 from liewegas/wip-13992

ceph_test_rados_misc: shorten mount timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph_test_rados_misc: shorten mount timeout

This might fix #13992.

Signed-off-by: Sage Weil <sage@redhat.com>

os/filestore: fix warning

os/filestore/FileStore.cc: In member function ‘int FileStore::_zero(const coll_t&, const ghobject_t&, uint64_t, size_t)’:
os/filestore/FileStore.cc:3294:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       if (offset + len > st.st_size) {
                        ^

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7932 from theanalyst/mailmap/10.0.4

mailmap for 10.0.4

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa/workunits/rest/test.py: don't use newfs

It doesn't work anymore.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8121 from liewegas/wip-15136

mon: remove 'mds setmap'

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rbd: rbd-mirroring: Added unit tests to test image mirroring behaviour

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Fixed if clause in mirror_image_get_info

This fix targets the case when journaling is not enabled but
someone calls mirror_image_get_info.

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Disable image mirroring when journaling feature is disabled

Signed-off-by: Ricardo Dias <rdias@suse.com>
Fixes: #15143

rbd: rbd-mirroring: Enable image mirroring when journaling feature is enabled

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Enable image mirroring upon create with journaling feature

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8137 from ceph/wip-15146-ceph-dep

rpm: remove sub-package dependencies on "ceph"

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jun Huang <hjwsm1989@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #7838 from rldleblanc/pg_prime-enable

mon: enable 'mon osd prime pg temp' by default

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7939 from yangdongsheng/remove_v1

mon: standardize Ceph removal commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7721 from xiexingguo/xxg-wip-fixdc

os/filestore: fix result handling logic of destroy_collection

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7754 from xiexingguo/xxg-wip-14845

osd: fix inaccurate counter and skip over queueing an empty transaction

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7970 from Yan-waller/yj-wip-async

msg/async: _try_send trim already sent for outcoming_bl more efficient

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8127 from ceph/wip-15122

ceph_test_msgr: reduce test size to fix memory size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge branch 'wip-13990' of git://github.com/tchaikov/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8160 from yuyuyu101/jewel-spdk

os/bluestore/NVMEDevice: make IO thread using dpdk launch

Merge pull request #8163 from liewegas/wip-15151

osd/ReplicatedPG: clear watches on change after applying repops

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7359 from songbaisen/song9

mon: consider pool size when creating pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7674 from majianpeng/bluestore-sync-transaction

os/bluestore: make bluestore_sync_transaction = true can work.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7860 from majianpeng/bluestores-clone-overlay

os/bluestore: clone overlay data

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7930 from ceph/wip-debian-librgw-tests

debian: package librgw_file* tests

Merge remote-tracking branch 'gh/wip-strict-debian-deps'

Merge pull request #7957 from majianpeng/bluestore-read

os/filestore: require offset == length == 0 for full object read; add test

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #8039 from gregsfortytwo/wip-14962-greg

osd/ReplicatedPG: be more careful about calling publish_stats_to_osd()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8096 from liewegas/wip-15073

osd: ensure new osdmaps commit before publishing them to pgs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: use current period id when it is empty for mdlog list and shard lock/unlock

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8167 from theanalyst/mstart/rgw_ports

mstart: start rgw on different ports as well

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

osd: assert that we don't try to use an OSDMap > superblock.newest_map

handle_osd_map now adds OSDMaps to the cache as it receives them
but before they are stable.  Ensure that we don't try to use them.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: commit osdmaps before exposing them to PGs

handle_osd_map and the PGs use different sequencers when writing
their updates.  We therefore need to make sure new osdmaps are
committed to disk before we expose them to PGs, lest they update
their info to reference a new osdmap that hasn't actually
committed yet.

This doesn't happen with FileStore because transactions are
ordered when they are queued, but it does affect BlueStore.

Fix by splitting handle_osd_map into two phases, one that just
persists stuff, and the second half that publishes the new maps to
the rest of the OSD.

Fixes: #15073
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8170 from yehudasa/wip-rgw-sync-fixes-3

Wip rgw sync fixes 3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

os/bluestore/BlueFS: Before reap ioct, it should wait io complete.

When change write mode of FileWrite(in _flush_range) from buffer to
direct. osd met segment fault.
This because call _close_writer don't check related ios whether
complete.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

mon: ignore msg without session attached

there is chance that a connection gets reset, and the session attached
to a message that was sent over this connection is reset. i.e.
con->set_priv(NULL) in Monitor::ms_handle_reset(), but we are about to
handle_command() this message after the message is somehow requeued. so,
not all MonOpRequests have an attached Session, especially those are
re-dispatched. we could check conn->is_connected() everywhere, but
that's error-prone and hard to maintain. so, in this change:

* we need to check for the session before looking at it closer.
* hold a reference to it when holding it, because in another thread,
  ms_handle_reset() could be freeing it.

Fixes: #15113
Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: remove 'mds setmap' command

This command has no discernable purpose, and is difficult and
dangerous to support anyway.

Fixes: #15136
Signed-off-by: Sage Weil <sage@redhat.com>

test: rbd-mirror: different log and asok location for local/remote contexts

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: make remote context respect env and argv config params

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8179 from zhjwpku/zhjwpku

doc/architecture.rst: remove redundant word "across"

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/architecture.rst: remove redundant word "across"

Signed-off-by: Zhao Junwang <zhjwpku@gmail.com>