rgw: add TODOs to split trim logic into separate source files

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ec0a655a80b20e9c76cf0d65cfea3fde66ce0a10)

rgw: use RGWShardCollectCR in MetaMasterTrimCR

limit the number of concurrent sync status requests to peers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 667ec9f3653cf40edcc475f66819684a26d77d61)

radosgw-admin: add 'mdlog autotrim' command

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 16462640a311fdd262a3ee74552fc2297a4f1db5)

rgw: RGWSyncLogTrimThread runs mdlog trim

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit aaab1ec2f736248b3557f42b4108e91b45f6b407)

rgw: add MetaTrimPollCR to coordinate polling and leases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8431f178004b952d9443a3acbd315d46514fadc2)

rgw: peer trims mdlog shards up to master's oldest entry

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0557a79612504646e9f195cacff839e6ad050957)

rgw: add MetaPeerTrimShardCR to trim mdlog shards

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3376c4b67c52f02111dea3bc032403f425a709c1)

rgw: peer purges mdlog periods before master's oldest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6cde60e954e60ff77e93b93e8c5bd7059f6a2644)

rgw: add MetaPeerTrimCR to query master mdlog info

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6e9296d49f4ce322a98651119f5dc66df7f5224c)

rgw: master trims mdlogs as peers make progress on current period

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1afabc1fdd1c36431c216ead8da8dfd799bdbe49)

rgw: add MetaMasterTrimShardCollectCR

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1765f869289e7bb2dd8cf4c89783b95f13a8fcdf)

rgw: master purges period mdlogs once all peers are done

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit eb140e38207dc43e16d803ee1174678cc9d5e39c)

rgw: add PurgePeriodLogsCR to purge entire mdlog periods

Conflicts: rgw_raw_obj
        src/rgw/rgw_sync.cc

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 32d16a9557fe61fe35b1670e86b20b0a4184e7ca)

rgw: master calculates minimum sync status of peers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 00d7b7b664c49bd476c060152e2738031cc4ba05)

rgw: add MetaMasterTrimCR to query sync status from peers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8bb106720160e42ae6e04f5dffc6af03794009ab)

rgw: add CRs for async mdlog history operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 70bcf452a2cfeafee3fb8a560391c50c3f52fedd)

Conflicts: rgw_raw_obj
        src/rgw/rgw_metadata.cc

rgw: use objv_tracker for mdlog history

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2c7897309617b29fc24a85ae21efffc387908380)

rgw: add RGWRadosRemoveCR

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 57b94899dccf0c0a63aabfd77af62608117f440e)

Conflicts: rgw_obj vs rgr_raw_obj
        src/rgw/rgw_sync.cc

rgw: add objv_tracker arg to RGWSimpleRadosRead/WriteCR

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ca462a3b9aef2be6422e420fc2482f831409cb72)

Conflicts: rgw_obj vs rgw_raw_obj
src/rgw/rgw_cr_rados.cc
src/rgw/rgw_cr_rados.h

rgw: add objv_tracker arg to RGWRados::put_system_obj_data

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2cee82f266335914103edf9673af494bfa39b7fe)

Conflicts: rgw_obj vs rgw_raw_obj
src/rgw/rgw_cache.h
src/rgw/rgw_rados.cc
src/rgw/rgw_rados.h

rgw: move coroutines out of anonymous namespace

anonymous namespaces do terrible things to name mangling, and this shows
up in our coroutine logging

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fadfc162ebf74b3bb1db4f0a67e7a21547f60514)

rgw: move timelog trim wrapper to header

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit db79bfabc87884ca5ae92a0685ec217a99af4597)

rgw: skip sync thread if current period is empty

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5deabd35796f483f7fd74ba042b7f81a3366422b)

rgw: DataLogTrimCR renamed zone to zone_id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f8d9ac6e740d6d1642da507747d2337000f0ab20)

rgw: hold lock for data log trimming

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0777fff4163588850a95433799d4e3ce1bd8b6e9)

rgw: add static function to generate lock cookie

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9a1b2359730df27e6688a77bfa3d5e801ad846a3)

rgw: use factory function to hide RGWDataLogTrimCR

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5ded7ee28baa478f94e6bfb84d0c0fc4e4af7ff8)

Conflicts:
src/rgw/rgw_data_sync.h (sync modules not backported)

rgw: pass cr registry to managers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ef4d7eab11fb5d2a41c9c28b9eb8b075aaff0d05)

Conflicts:
src/rgw/rgw_data_sync.cc (this is a re-cherry-pick that includes
                the RGWSyncLogTrimThread piece that was not available
                as of 87cb8474f523be1b281882c46a8a3597977a51c9)

rgw: add RGWSyncLogTrimThread to RGWRados

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3b674bbd865cb640266751480c86925a8c07e099)

rgw: add RGWDataLogTrimCR

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6a366f955a0caa4febdbcf855a9da482a27b7fb0)

rgw: add RGWRadosTimelogTrimCR

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 20f31a412d3479efe71d5745c55575173c3fe9e8)

rgw: enable async calls to time_log_trim

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d67436bb0d6cb9ae7757916547b8274990f8cc6b)

rgw: add rest handlers to query sync status

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a66b4cc9faaf003e9b3389994470048f1a317405)

Conflicts: src/rgw/rgw_rest_log.cc
   jewel/v10.2.10 already has "rgw: change metadata read_sync_status interface"
   so we need to use the read_sync_status() call in
   src/rgw/rgw_rest_log.cc.

rgw: expose sync managers through RGWRados

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ccef4b0f59c598eff2afc908f639ccf560782015)

rgw: add json decoders for data sync status

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ebbb70bd5165e3e835d8e528e038d512e7dedba9)

rgw: add dout_prefix for rgw_data_sync.cc

and took out the redundant 'rgw' from 'rgw meta sync:'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 821c70d710295ceb5fd12b3058df6a672ed50b0a)

rgw: use RGWShardCollectCR for RGWReadDataSyncStatusCoroutine

this allows us to limit the number of outstanding requests for shard
markers

there also appeared to be issues with spawning the shard CRs
from RGWReadDataSyncStatusCoroutine::handle_data(), because
handle_data() was returning before the shard CRs completed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 30823c5d4e98eb501ff1c2f7315705c3c5dd614d)

Merge pull request #19840 from joscollin/wip-jewel-backport-22569-2

jewel: doc: clarify Path Restriction instructions

Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc: hyperlink fix

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>
(cherry picked from commit f763bacab3ea66ab13b4d62ad883c365a8e0a604)

Merge pull request #19795 from joscollin/wip-jewel-backport-22569

jewel: doc: clarify Path Restriction instructions

Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc/mds: add generated files into the /etc/ceph/

Fixes: 16906
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
(cherry picked from commit d1277f116cd297bae8da7b3e1a7000d3f99c6a51)

Merge pull request #19656 from tchaikov/wip-install-deps-jewel

jewel: install-deps.sh: update g++ symlink also

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19603 from ceph/wip-jewel-ansible-branch

qa/tests: use 'stable-2.1' branch for jewel testing

install-deps.sh: update g++ symlink also

we need to update g++ symlink also, if it points to the wrong version

http://tracker.ceph.com/issues/22220
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: the libboost issue does not affect master. as master builds
     boost from source. so, it's not cherry-picked from master.

qa/tests: use 'stable-2.1' branch for jewel testing

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #19521 from tchaikov/wip-22220-jewel

jewel: install-deps.sh: readlink /usr/bin/gcc not /usr/bin/x86_64-linux-gnu-gcc

Signed-off-by: Nathan Cutler <ncutler@suse.com>

install-deps.sh: readlink /usr/bin/gcc not /usr/bin/x86_64-linux-gnu-gcc

See: http://tracker.ceph.com/issues/22220
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19461 from tchaikov/wip-22220-jewel

jewel: install-deps.sh: point gcc to the one shipped by distro

Reviewed-by: Nathan Cutler <ncutler@suse.com>

install-deps.sh: point gcc to the one shipped by distro

to define a struct in a method is legal in C++11, but it causes internal
compiler error due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82155
if we are using GCC-7. so we need to either workaround in our source
code by moving the struct definition out of the member method or revert
to a GCC without this bug. but if we go with the first route, the jewel
build still fails, because GCC-7 starts to use the new CXX11 ABI, which
is not compatible with the libboost we use in jewel. the libboost was
still built with the old ABI for backward compatibility. so let's just
fix the install-deps.sh to point gcc to the origin one.

See: http://tracker.ceph.com/issues/22220
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: the libboost issue does not affect master. as master builds
 boost from source. so, it's not cherry-picked from master.

Merge pull request #19214 from liewegas/wip-no-ec-upgrade-test

qa/workunitse/rados/test/test-upgrade-11.0.0-noec: more *EC* skipping

qa/workunitse/rados/test/test-upgrade-11.0.0-noec: more *EC* skipping

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19176 from liewegas/wip-no-ec-upgrade-test

qa/workunits/rados/test-upgrade-11.0.0-noec.yaml: skip ec tests

qa/workunits/rados/test-upgrade-11.0.0-noec.yaml: skip ec tests

This is needed for jewel-x point to point upgrade because earlier point
releases can't handle our ec profiles with ruleset-* (later ones can) and
the test races with the mon upgrades.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #19153 from smithfarm/wip-22236-jewel

jewel: ceph-disk flake8 test fails on very old, and very new, versions of flake8

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: ceph-disk: ignore E722 in flake8 test

Very old, and very new, versions of flake8 treat E722 as an error:

flake8 runtests: commands[0] | flake8 --ignore=H105,H405,E127 ceph_disk tests
ceph_disk/main.py:1575:9: E722 do not use bare except'
ceph_disk/main.py:1582:9: E722 do not use bare except'
ceph_disk/main.py:3252:5: E722 do not use bare except'
ceph_disk/main.py:3288:21: E722 do not use bare except'
ceph_disk/main.py:3296:17: E722 do not use bare except'
ceph_disk/main.py:4358:5: E722 do not use bare except'
tests/test_main.py:26:1: E722 do not use bare except'
ERROR: InvocationError: '/opt/j/ws/mkck/src/ceph-disk/.tox/flake8/bin/flake8 --ignore=H105,H405,E127 ceph_disk tests'

Fixes: http://tracker.ceph.com/issues/22207
References: https://gitlab.com/pycqa/flake8/issues/361

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 3600cd7bfdca76485d2998c5da1a0dd25816d1ce)

Merge pull request #19121 from liewegas/wip-jewel-x-fix

qa/workunits/rados/test-upgrade-11.0.0: skip LibRadosMiscECPP.CompareExtentRange

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/rados/test-upgrade-11.0.0: skip LibRadosMiscECPP.CompareExtentRange

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17082 from vumrao/wip-vumrao-scrub-recovery-false

jewel: osd: default osd_scrub_during_recovery=false

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17841 from smithfarm/wip-21478-jewel

jewel: systemd: Add explicit Before=ceph.target

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17847 from batrick/i21481

jewel: qa: use xfs instead of btrfs w/ filestore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17101 from vumrao/wip-vumrao-jewel

jewel: core: Addition of online osd 'omap'compaction command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: liuchang0812 <liuchang0812@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17893 from dzafman/wip-19140-19224

jewel: osd ops (sent and?) arrive at osd out of order

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18080 from linuxbox2/jewel-awsv4-sort

jewel: radosgw: fix awsv4 header line sort order.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #18165 from tchaikov/wip-no-btrfs-jewel

jewel: qa: use xfs instead of btrfs w/ filestore

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites: add objectstore facet where it was missing

There are places where we were testing xfs vs btrfs on
filestore and now want to cover both filestore and bluestore.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 29a19dce30ddd41ef4e7806b6970a6daa738f39b)

Conflicts: remove bluestore.yaml as jewel does not support it. and
  remove links to objectstore from where the tests do not exist in jewel
  yet, for instance, qa/suites/mgr/basic.

Merge pull request #17594 from tchaikov/wip-jewel-20616

jewel: librados: copy out data to users' buffer for xio

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18548 from dillaman/wip-21911-jewel

jewel: test/librbd: fixed metadata tests under upgrade scenarios

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18560 from dillaman/wip-21912-jewel

jewel: test/librbd: updated test_notify to handle new release lock semantics

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

test/librbd: updated test_notify to handle new release lock semantics

Fixes: http://tracker.ceph.com/issues/21912
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: fixed metadata tests under upgrade scenarios

Fixes: http://tracker.ceph.com/issues/21911
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17942 from smithfarm/wip-21522-jewel

jewel: ceph-disk: fix --runtime omission when enabling ceph-osd@$ID.service units for device-backed OSDs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17815 from dzafman/wip-21117

jewel: osd: osd_scrub_during_recovery only considers primary, not replicas

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/cephtool/test.sh: add sudo for daemon compact

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2f272ab4512bb2248ac2ef180ece6d122f8e7479)

test: add test of ceph compact command

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit d1f24d053f367511cf12a43350bad246c5f28c9f)

 Conflicts:
qa/workunits/cephtool/test.sh
         Removed all tests which are not part of this backport

osd: new command compact via tell/daemon

user could manual compact OSD's omap as following:
1. ceph tell osd.id compact
2. ceph daemon osd.id compact
user's requests will be impacted during compaction.

Fixes: http://tracker.ceph.com/issues/19592
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit b4ad4297652df2f6ebfadcdededc7a47607ab534)

 Conflicts:
src/osd/OSD.cc
         Removed all admin socket register and unregister commands
         which are not part of this backport
         Changed admin_command to command variable because in jewel
         we use command variable.

os: export compact interface in ObjectStore and ObjectMap

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit eb5723d3abd4e1b6caec477090761d7919efc7bd)

 Conflicts:
src/os/bluestore/BlueStore.h
         Removed declarations which are not part of this backport
          inject_data_error()
          inject_mdata_error()
          debug_data_eio()
          debug_mdata_eio()
          debug_oj_on_delete()
src/os/filestore/FileStore.h
         Removed declarations which are not part of this backport
          set<ghobject_t> data_error_set
          set<ghobject_t> mdata_error_set
          inject_data_error() override
          inject_mdata_error() override

Merge pull request #18169 from tchaikov/wip-bz1494543-jewel

jewel: ceph-disk: factor out the retry logic into a decorator

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #18166 from tchaikov/wip-jewel-sphinx

jewel: admin: bump sphinx to 1.6

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-disk: retry on OSError

we are likely to
1) create partition, for instance, sdc1
2) partprobe sdc
3) udevadm settle
4) check the device by its path: /dev/sdc1

but there is chance that the uevent sent from kernel fails to reach udev
before we call "udevadm", hence "/dev/sdc1" does not exist even after
"udevadm settle" returns. so we retry in case of OSError here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4f82dfb9e761a09484e6ba3bd027da535162783e)

Merge pull request #16294 from smithfarm/wip-18853-jewel

jewel: build/ops: upstart: radosgw-all does not start on boot if ceph-base is not installed

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: factor out the retry logic into a decorator

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 02a8620de7f200736900aafc944b2c1ac47d9910)

Conflicts:
src/ceph-disk/ceph_disk/main.py: jewel does not have PROCDIR,
so resolve it by using '/proc'. also, in jewel, unmount() does not
have `do_rm` parameter, so do not handle it.

admin/build-doc: support optional argument for specifying sphinx builders

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5b9ec53512cb8119db3af335283445ac972053b7)

Conflicts:
admin/build-doc: trivial resolution

doc,cmake: do not scan non-man pages when building man pages

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0e00d2f21bddd8ee811dec816cff5b4f05b1f891)

Conflicts:
doc/man/CMakeLists.txt: jewel does not have this file. so do
not cherry-pick changes of it.

doc: switch to upstream sphinx-ditaa

it works with setuptools and is now compatible with py3

the py3 branch is created to track the upstream's master branch

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fda079d5ed4c8a80c677f714a680233a568d512e)

admin: bump sphinx to 1.6

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88a1114cb3f90e52ba718dc832617fa004d63b3c)

qa: stop testing btrfs

This is too noisy and we don't recommend it anyway.

See http://tracker.ceph.com/issues/20169

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5a2d78ad85ef7adcb716f98b1ba28c1463d2ed53)

Conflicts:
qa/objectstore/filestore-btrfs.yaml: we add some notes in this
file in master, but didn't backport the commit(s) adding these notes to
jewel. we are removing this file anyway. so who cares!

qa/suites: remove 'fs' facet from all tests

The objectstore facet now covers bluestore, filestore(xfs),
and filestore(btrfs).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 73981ad8078cba5d6efec8e67796a1dbc007063e)

Conflicts:
qa/suites/fs/recovery/xfs.yaml: in master, this file is factored
into a facet: a/suites/fs/recovery/fs/xfs.yaml, but in jewel, it still a
plain xfs.yaml. but it's good enough for us, as what we need is just
xfs.

qa/objectstore: split filestore into xfs and btrfs variants

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 208675afb131b578912649d6dbf2a34ad7f3826e)

10.2.10

Merge pull request #18100 from ceph/wip-yuriw-fix-clients-jewel

tests: Added openstack.yaml bits to enable runs on ovh nodes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests - Added openstack.yaml bits to enable runs on ovh nodes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

radosgw: fix awsv4 header line sort order.

The awsv4 signature calculation includes a list of header lines, which
are supposed to be sorted.  The existing code sorts by header name, but
it appears that in fact it is necessary to sort the whole header *line*,
not just the field name.  Sorting by just the field name usually works,
but not always.  The s3-tests teuthology suite includes
s3tests.functional.test_s3.test_object_header_acl_grants
s3tests.functional.test_s3.test_bucket_header_acl_grants
which include the following header lines,

x-amz-grant-read-acp:id=56789abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234
x-amz-grant-read:id=56789abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234
x-amz-grant-write-acp:id=56789abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234
x-amz-grant-write:id=56789abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234

in this case, note that ':' needs to sort after '-'.

Fixes: http://tracker.ceph.com/issues/21607
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit c11485e1b3a58631628644152816d9b22a17d8bd)

Merge pull request #18044 from ceph/wip-yuriw-fix-clients-jewel

qa: Changed distros symlink to point to supported OSs

Reviewed-by: Sage Weil <sage@redhat.com>

Changed distros symlink to point to supported OSs

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #18027 from ceph/wip-yuriw-fix-clients-jewel

tests - Added suit to test upgraded clients against jewel ceph clusters

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests - Added suit to test upgraded clients against jewel ceph clusters

Replaces https://github.com/ceph/ceph/pull/17981
We need to run this suite using suite-branch option in
order to use jewel workloads agains ceph cluster luminous+ branches

Added 'libcephfs1' to exclude_packages in upgrade_workload

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

osd: default osd_scrub_during_recovery=false

This is a more friendly behavior.  Tell users in the release notes how to
get the old behavior.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8dca17c067b51050308c5f1cb5eddd400fd6f3f0)

 Conflicts:
PendingReleaseNotes "trivial resolution"

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

ceph-disk: fix '--runtime' omission for ceph-osd service

f425a127b introduces a regression that ceph-disk omits "--runtime" when
enabling ceph-osd@$ID.service units for device-backed OSDs.

Fixes: http://tracker.ceph.com/issues/21498
Signed-off-by: Carl Xiong <cxiong@suse.com>
(cherry picked from commit a385b5b0c35106c9b44d81655983b2f7566b21cd)

Merge pull request #17892 from smithfarm/wip-p2p-s3-test

jewel: tests: fix upgrade/jewel-x/point-to-point-x

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa: point-to-point-x: upgrade client.1 to -x along with cluster nodes

The client.1 rgw in workload_x had not been upgraded to -x.

Fixes: http://tracker.ceph.com/issues/21499
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit afc5a7d3208071b65c0d76d124bfc47a099a446c)