Merge pull request #17470 from smithfarm/wip-21104-luminous

luminous: build/ops: ceph-fuse RPM should require fusermount

Reviewed-By: Ken Dreyer <ktdreyer@redhat.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #17477 from smithfarm/wip-21137-luminous

luminous: mgr: 500 error when attempting to view filesystem data

Reviewed-By: John Spray <jcsp@redhat.com>

Merge pull request #17495 from smithfarm/wip-21235-luminous

luminous: tests: thrashosds read error injection doesn't take live_osds into account

Merge pull request #17483 from smithfarm/wip-21183-luminous

luminous: mgr: Crash in MonCommandCompletion

Merge pull request #17486 from smithfarm/wip-21188-luminous

luminous: pybind: dashboard usage graph getting bigger and bigger

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17350 from liewegas/wip-pr-17334-luminous

os/bluestore: reshard-fix wrong shard length

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17473 from smithfarm/wip-21112-luminous

luminous: cephfs: get_quota_root sends lookupname op for every buffered write

Reviewed-By: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17465 from smithfarm/wip-21045-luminous

luminous: rbd: TestMirroringWatcher.ModeUpdated: periodic failure due to injected message failures

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17565 from theanalyst/wip-21278-luminous

luminous: the standbys are not updated via ceph tell mds.* command

Reviewed-By: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17566 from theanalyst/wip-21267-luminous

luminous: Incorrect grammar in FS message 1 filesystem is have a failed mds daemon

Reviewed-By: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #17557 from theanalyst/wip-21269-luminous

luminous: some generic options can not be passed by rbd-nbd

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17448 from smithfarm/wip-21118-luminous

luminous: rgw: need to stream metadata full sync init

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17464 from smithfarm/wip-20964-luminous

luminous: rbd: switch to new config option getter methods

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17446 from smithfarm/wip-21115-luminous

luminous: rgw multisite: objects encrypted with SSE-KMS are stored unencrypted in target zone

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17468 from smithfarm/wip-21052-luminous

luminous: build/ops: RHEL 7.3 Selinux denials at OSD start

Reviewed-By: Ken Dreyer <ktdreyer@redhat.com>

Merge pull request #17444 from smithfarm/wip-20968-luminous

luminous: rgw: bytes_send and bytes_recv in the msg of usage show returning is 0 in master branch

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17555 from theanalyst/wip-21270-luminous

luminous: rgw: shadow objects are sometimes not removed

Reviewed-By: Orit Wasserman <owasserm@redhat.com>

Merge pull request #17558 from theanalyst/wip-21277-luminous

luminous: [cls] metadata_list API function does not honor `max_return` parameter.

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17556 from theanalyst/wip-21265-luminous

luminous: [cli] rename of non-existent image results in seg fault

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17560 from theanalyst/wip-21288-luminous

luminous: [test] various teuthology errors

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17561 from theanalyst/wip-21289-luminous

luminous: [rbd] image-meta list does not return all entries

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17336 from tchaikov/wip-luminous-pr-16774

luminous: mgr/dashboard: fix rbd's pool sub menu

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17622 from liewegas/wip-more-lum-pi

osd: update info only if new_interval

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17564 from theanalyst/wip-21283-luminous

qa/objectstore/bluestore*: less debug output

osd: update info only if new_interval

refactor OSD::build_initial_pg_history() so we update the info only if a
new interval is identified. also, this is consistent with
OSD::build_past_intervals_parallel().

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4ae053439e526fab0451d1502576dce89eb6b44f)

Merge pull request #17496 from smithfarm/wip-21236-luminous

luminous: osd: build_initial_pg_history doesn't update up/acting/etc

Merge pull request #17601 from liewegas/wip-21171-luminous

os/bluestore: fix deferred write deadlock, aio short return handling

Merge pull request #17551 from dillaman/wip-21280

luminous: test/cls_rbd: mask newer feature bits to support upgrade tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #17467 from smithfarm/wip-21046-luminous

luminous: common: adding line break at end of some cli results

Merge pull request #17472 from smithfarm/wip-21108-luminous

luminous: mon: OSDMonitor: osd pool application get support

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17475 from smithfarm/wip-21133-luminous

luminous: osd: PrimaryLogPG: sparse read won't trigger repair correctly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17481 from smithfarm/wip-21182-luminous

luminous: core: osd crush rule rename not idempotent

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17494 from smithfarm/wip-21234-luminous

luminous: bluestore: asyn cdeferred_try_submit deadlock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17469 from smithfarm/wip-21103-luminous

luminous: cephfs: client: missing space in some client debug log messages

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #17562 from theanalyst/wip-21276-luminous

luminous: os/bluestore/BlueFS.cc: 1255: FAILED assert(!log_file->fnode.extents.empty())

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17345 from liewegas/wip-pr-17311-luminous

mon: use ceph_clock_now if message is self-generated

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17344 from liewegas/wip-pr-17330-luminous

mon/OSDMonitor: make 'osd crush class rename' idempotent

Merge pull request #17527 from liewegas/wip-pr-17510-luminous

crush: fix fast rule lookup when uniform

Merge pull request #17511 from liewegas/wip-stupid-luminous

os/bluestore: default to stupid allocator

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #17501 from smithfarm/wip-21242-luminous

luminous: osd: osd/PG: discard msgs from down peers

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17500 from smithfarm/wip-21240-luminous

luminous: mon: rate limit on health check update logging

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17498 from smithfarm/wip-21238-luminous

luminous: osd: subscribe to new osdmap while waiting_for_healthy

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17497 from smithfarm/wip-21237-luminous

luminous: bluestore: osd crash when change option bluestore_csum_type from none to CRC32

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17378 from jan--f/wip-21187-luminous

luminous: core: osd/PGLog: write only changed dup entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

os/bluestore/aio: handle short return from io_submit

io_submit may return a value less than nr, indicating that only some of
the provided iocbs were queued.  If that happens we should loop, not
return and silently drop those aios on the floor.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dc17dfd9ff05b5676488c2b1bca53026b2ca6244)

os/bluestore: more deferred debugging

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 11ac691dcc86aeed4ccb692a79ade4be04c5ef2f)

os/bluestore: drop deferred_submit_lock

This lock serves no purpose.  It only protects the call to the aio_submit
on the OpSequencer running batch, but we are the only caller who will do
that submission, and a new batch won't be moved into place until ours is
completed and retired (by the completion for the aio we are queueing).

More importantly, this fixes a deadlock:

- thread A submits aio, but the queue is full, and blocks
- thread B blocks taking deferred_submit_lock while holding deferred_lock
- aio completion thread blocks trying to take deferred_lock, and thus
no further aios are drained.

Fixes: http://tracker.ceph.com/issues/21171
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7a5ef62af194373c3a8c307b05ae6d467e49c9f0)

Merge pull request #17534 from liewegas/wip-pr-17459-luminous

os/bluestore: add bluestore_prefer_deferred_size_hdd/ssd to tracked keys

mds: fix "1 filesystem is have a..." message

Like "Health check failed: 1 filesystem is have a failed mds daemon (FS_WITH_FAILED_MDS)"

Fixes: http://tracker.ceph.com/issues/21153
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7d21119a47212140d121df99efdb838a3f078a82)

ceph: collect all mds in mdsids()

otherwise, only the active mds are returned.

Fixes: http://tracker.ceph.com/issues/21230
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit dd702cc94e32f533f0f2e76f9d202d1fa7c6a57f)

ceph: always populate `targets` with ids_by_service()

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b682e61ddc190b91da4c6cbaeba3c69d54652afb)

ceph: extract ids_by_service() so it can be reused

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 25639f66910aa8a188d5b4b8fff8803e169da62a)

qa/objectstore/bluestore*: less debug output

Let's see if this makes the spurious MON_DOWN failures go away?  (See
http://tracker.ceph.com/issues/20910)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 444f5aa08539cde827e7d93a514367a8ba39b122)

Conflicts:
 qa/objectstore/bluestore-bitmap.yaml
 Added bluestore-bitmap.yaml to Luminous as well

os/bluestore/BlueFS: prevent _compact_log_async reentry

_should_compact_log uses new_log != nullptr to tell whether compaction is
already in progress, but we don't set it until we are midway through the
process.  Set it at the top of the method to prevent reentry.

See 455cc6cea2e5e8bc1126b5047b6003fa583af1b5, which failed to implement
this properly.

Fixes: http://tracker.ceph.com/issues/21250
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6bd9db304f906e9e4a07ca5ce07ffffdbc1678f0)

rbd: support iterating over metadata items when listing

Fixes: http://tracker.ceph.com/issues/21179
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9c4f9856b034061aed052192b0ee09b9a9d09bc0)

qa/suites/rbd: reduce monitor requirements for mirroring tests

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 27194c30c50009eaf6391a21837f73b312fe5de6)

qa/suites/rbd: fixed cache override

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 98061bb3d7ce6309ddb04ea4d7e9d44a7ecd09c6)

qa/suites/rbd: reduce OSD debug log levels

The teuthology machines are periodically running out of space
due to the aggressive log settings.

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 763f1877ecda680e62a2b7eff5815bb6b7dd18f2)

cls/rbd: metadata_list not honoring max_return parameter

Fixes: http://tracker.ceph.com/issues/21247
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0130b85732849da8bb3e96f4afb7c023c958aef9)

rbd-nbd: fix generic option issue

Fixes: http://tracker.ceph.com/issues/20426
Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
(cherry picked from commit 16f64a4c01068603736af476f640268d0baa338d)

rbd-nbd: lower down the log level when return -EINVAL in aio_callback.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
(cherry picked from commit dcd7e5bb82524bf111a6d80d9e0c050ae55b26a1)

librbd: rename of non-existent image results in seg fault

Fixes: http://tracker.ceph.com/issues/21248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4a75ee43d3b108a9593a010160479fd62884eda8)

rgw: add tail tag to track tail instance

Use tail_tag instead of idtag. The tail_tag is modified when either
creating a new tail, or adding a refcount to the tail (e.g., when
copying object). When just modifying the object's head, tail_tag
should remain the same. The gc will then use the tail_tag (if
exists, idtag if not) as the refcount id.

Fixes: http://tracker.ceph.com/issues/20234
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f71a3e525d9a69fe537428f25930fb207a602afe)

test/cls_rbd: mask newer feature bits to support upgrade tests

Fixes: http://tracker.ceph.com/issues/21217
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit aeb8f29d21625f3570e2f3e6dd26594d1a6df810)

os/bluestore: add bluestore_prefer_deferred_size_hdd/ssd to tracked keys

To make these two keys can be changed online:

./bin/ceph tell osd.* injectargs "--bluestore_prefer_deferred_size_hdd 0"
osd.0: bluestore_prefer_deferred_size_hdd = '0' (not observed, change may require restart)
osd.1: bluestore_prefer_deferred_size_hdd = '0' (not observed, change may require restart)
osd.2: bluestore_prefer_deferred_size_hdd = '0' (not observed, change may require restart)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit cd1225634cf4d20e485f7c30d5a0438febc7ddcc)

crush: fix fast rule lookup when uniform

Older clients will search for the first rule with a matching ruleset,
type, and size.  The has_uniform_rules bool is only set if we have rule
ids and rulesets that line up, but we must also verify that the rest of the
mask matches or else we can get a different CRUSH mapping result because
the mask might not match and old clients will fail to find a rule and we
will find one.  We also can't just check the ruleset as the legacy clients
find the *first* (of potentially many) matching rules; hence we only do
the fast check if all rulesets == rule id.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f24095e0e9734531dbdbcd4bff1392c463188e8e)

os/bluestore: mark bluestore_allocator as DEV

We don't want users changing this without a good reason.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: put StupidAllocator btree into bluestore_alloc mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6f189d8ab3c549a5c5e01616310c6eb39119fbc1)

os/bluestore: default 'bluefs_allocator = stupid'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2b6eea80eacd65cbb1aa3702a3a3b642304c04f5)

os/bluestore: default to bluestore_allocator = stupid

Performance looks better, and code is simpler.

Also fix config option annotations.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3750cf9cafabd3296c22b8b8dff35e44d18b9dd8)

osd/PG: discard msgs from down peers

if a repop is replied after a replica goes down in a new osdmap, and
before the pg advances to this new osdmap, the repop replies before this
repop can be discarded by that replica OSD, because the primary resets the
connection to it when handling the new osdmap marking it down, and also
resets the messenger sesssion when the replica reconnects. to avoid the
out-of-order replies, the messages from that replica should be discarded.

Fixes: http://tracker.ceph.com/issues/19605
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e3fce6be44506168a7a138aab93f6a4d6776397b)

mon: rate limit on health check update logging

Fixes: http://tracker.ceph.com/issues/20888
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5b903c7b428c2c7024fdfc1dffacad0b1521b750)

mon: add helper for emitting logs by health status

We were starting to repeat our little "if(status==HEALTH_ERR"
blocks in too many places.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 618a27868d65aa152ac67cce8057a75c4d61def9)

osd: subscribe to new osdmap while waiting_for_healthy

If we are sitting around waiting until we are able to ping our "up" peers,
we need to be sure that our notion of "up" is still correct and we're not
just stuck on an old, stale OSDMap.

Fixes: http://tracker.ceph.com/issues/21121
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fbafa659dae94faba435ca449ee5e77b51108b4b)

os/bluestore: formatting nits

So we don't output anything if csum is currently off:

_do_alloc_write blob Blob(0x7fad7cc25950 blob([0x42a80000~10000] none/0x1)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 22740cc9b3e489ae82b4b3071a17793c11abce79)

os/bluestore: don't re-initialize csum-setting for existing blobs

The global checksum setting may change, e.g., from NONE to CRC32,
which can cause improper re-initialization of the csum-settings of
existing blobs(e.g., partial write/overwrite may turn out to shrink
'csum_data').

We could develop some complicated solutions but for now let's not
bother since the above scenario is rare.

Fixes: http://tracker.ceph.com/issues/21175
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8ec4b4960f9b93ff6d85a02308ccda0bccadb599)

osd: fix build_initial_pg_history

We need to update our info about the previous interval in order to
detect interval changes properly.

Fixes: http://tracker.ceph.com/issues/21203
Signed-off-by: w11979 <wang.wenfeng@h3c.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit afdacc6ed76dd288fdbc0f99857d9ffba23c5132)

qa/tasks/ceph_manager: revive osds before doing final rerr reset

We assume below that rerrosd is up, but it may not be when we exit the
loop.

Fixes: http://tracker.ceph.com/issues/21206
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 21027233b2a31b587e7caf95b1d85f5bcf0ae8ec)

os/bluestore: separate finisher for deferred_try_submit

Reusing finishers[0], which is used for completions back into the OSD,
is deadlock-prone: the OSD code might block trying to submit new IO or
while waiting for some other bluestore work to complete.

Fixes: http://tracker.ceph.com/issues/21207
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2b6a9d1ec0778f66b6e9d9f216df21a00caeba3f)

pybind: fix chart becomes bigger when refresh

Fixes: http://tracker.ceph.com/issues/20746
Signed-off-by: Yixing Yan <yanyx@umcloud.com>
(cherry picked from commit edf93427734ce156f5eaac7a25f4c16a92852b98)

mgr: fix crash in MonCommandCompletion

Fixes: http://tracker.ceph.com/issues/21157
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5fa61b2ed078a3cf333a3111d4f43584ce27ba9d)

mon/OSDMonitor: make 'osd crush rule rename' idempotent

Fixes: http://tracker.ceph.com/issues/21162
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit cd2191c3f192986e20beaab84b643dfaf49901f7)

mgr/dashboard: fix FS status on old MDS daemons

These may not have the version string.  We already handled
this in the equivalent code in the status module, so just
do the same thing in dashboard.

Fixes: http://tracker.ceph.com/issues/20692
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 800662c2fa3f040992987dc5c635dfbc14f0a0ea)

osd/PrimaryLogPG: fix sparse read won't trigger repair correctly

Normal reads instead of hole-reads should trigger auto-repair.
Also we shall quit on successfully triggering a repair, or else
we are at risk of doing the op multiple times.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4ad25c346215dc2ddc4b1a43948b6fe01316edf2)

client: skip lookupname if writing to unlinked file

When a client writes to an unlinked file, get_quota_root tries to
resolve the filename via an MDS lookupname op. The op always results in
-13 permission denied for path-restricted caps or -2 no such file or
directory otherwise. More importantly, the repeated lookupname ops slow
down buffered writes significantly.

Don't do the lookupname for unlinked files; use the root_ancentor's
quota instead.

Fixes: http://tracker.ceph.com/issues/20945
Backport: jewel, luminous
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 1d617eee2e8c32f7300cb2cb8dd37b3714a4699f)

mon/OSDMonitor: "osd pool application ls" support

It would be a pain if we have to call 'ceph osd dump --format=json-pretty'
to find out these each time...

Demo output:
(1) ceph osd pool application get
{
    "cephfs_data_b": {
        "cephfs": {}
    },
    "cephfs_metadata_a": {
        "cephfs": {}
    },
    "test_pool": {
        "rbd": {
            "test": "me"
        }
    }
}

(2) ceph osd pool application get test_pool
{
    "rbd": {
        "test": "me"
    }
}

(3) ceph osd pool application get test_pool rbd
{
    "test": "me"
}

(4) ceph osd pool application get test_pool rbd test
me

Fixes: http://tracker.ceph.com/issues/20976
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 46586b22882c0c08c099243e189f5a165063da0c)

rpm: ceph-fuse requires fuse package

The ceph-fuse(8) man page describes using the "fusermount" tool to mount
CephFS. This utility is in the "fuse" package. Prior to this change, the
ceph-fuse RPM only depends on "fuse-libs", not "fuse".

The Debian packaging has always depended on the full "fuse" (previously
"fuse-utils") package, so we should make the RPM packaging do the same.

Fixes: http://tracker.ceph.com/issues/21057
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 6fba9cc3f910442d3d09b41f01f454a39bfdbc48)

client: add blank space between function name and inode number for log

Fixes: http://tracker.ceph.com/issues/21069
Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>
(cherry picked from commit 0cb4d379fdd2b869bd765e1fcb424a8f8a8f6a04)

selinux: Allow nvme devices

This commit allows nvme devices which use a different label than
standard block devices.

Fixes: http://tracker.ceph.com/issues/19200
Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit a8af61c8da9028adc13f7aea611d5a5fee9dbfd1)

common: add new line break when get command descriptions

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 19dd4203eb930ffb7a321cb617c580d99ceca9c1)

rbd: adding newline operation when only '--format' used

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 992829d38cb89732f6994642c1bdaa2448e610b7)

rbd: '--pretty-format' should return error if '--format' not used

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 47e7549826b55905e7f298717fedf0e8f220940a)

Conflicts:
    src/test/cli/rbd/help.t (master has several new commands compared to
                            luminous: omit these)

test/librbd: TestMirroringWatcher unit tests should ignore duplicates

Fixes: http://tracker.ceph.com/issues/21029
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5d3d5ea5aca5807b09050fe8ad2245c4963a0f34)

rbd: switched from legacy to new-style configuration options

Fixes: http://tracker.ceph.com/issues/20737
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7dc82bc08cbfabd424dcd0b83ed618d28db5fe8e)

rgw: metadata full sync streams metadata keys

We don't read the entire list of all metadata entries in a single
request.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 4a509a669125a9a2d74de11cce555df2547103d1)

rgw: rest list metadata supports max-entries and marker

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit a45fd78d08f4d862c273a13abf787a93ce8e0906)

rgw-admin: add marker to metadata list

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit ae600c2c3f765f409cb16557870ceb91d6714760)

rgw: add marker to metadata api keys listing

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 84ece81bc3590b08e5565c5b96bcbb4cb97a4100)

rgw: marker for raw pool listing

We now have a marker (cursor) that can be used for raw pool
listing.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit a24b8aeb1e376ce9e5e50535d8ce1a287c99fa90)

test/rgw: add kms encryption key for teuthology

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f27ebabe551f62241d1f5b55add9ec6b5d782073)