test/cls_rbd: mask newer feature bits to support upgrade tests

Fixes: http://tracker.ceph.com/issues/21217
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit aeb8f29d21625f3570e2f3e6dd26594d1a6df810)

Merge pull request #16703 from ddiss/wip-20837-jewel

jewel: ceph-disk: don't activate suppressed journal devices

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #17396 from smithfarm/wip-21113-jewel

jewel: cephfs: get_quota_root sends lookupname op for every buffered write

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #17009 from dreamhost/wip-16463-jewel

jewel: rgw: aws4: add rgw_s3_auth_aws4_force_boto2_compat conf option

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16963 from smithfarm/wip-20972-jewel

jewel: cephfs: ceph-fuse segfaults at mount time, assert in ceph::log::Log::stop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16952 from dreamhost/wip-20966-jewel

jewel: rgw: radosgw-admin: fix bucket limit check argparse, div(0)

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #16951 from linuxbox2/jewel-aws4-plus

jewel: rgw: replace '+' with "%20" in canonical query string for s3 v4 auth

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16880 from linuxbox2/jewel-rgw-dir-suggest

jewel: rgw: Fix up to 1000 entries at a time in check_bad_index_multipart

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16720 from prallabh/wip-jewel-20821

jewel: rgw: Do not decrement stats cache when the cache values are zero

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15966 from prallabh/jewel

jewel: rgw: Custom data header support

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16144 from smithfarm/wip-20364-jewel

jewel: mon: osd crush set crushmap need sanity check

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17210 from dzafman/wip-20730-jewel

jewel: core: disable skewed utilization warning by default

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16405 from dzafman/wip-20041-jewel

jewel: osd: scrub_to specifies clone ver, but transaction include head write ver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16169 from Vicente-Cheng/wip-20511-jewel

jewel: osd: ReplicatedPG: solve cache tier osd high memory consumption

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16167 from Vicente-Cheng/wip-20492-jewel

jewel: osd: omap threadpool heartbeat is only reset every 100 values

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16151 from Vicente-Cheng/wip-20349-jewel

jewel: cephfs: df reports negative disk "used" value when quota exceed

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16150 from Vicente-Cheng/wip-20403-jewel

jewel: mds: save  projected path into inode_t::stray_prior_path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16141 from smithfarm/wip-20270-jewel

jewel: tests: LibRadosMiscConnectFailure.ConnectFailure hang

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16285 from smithfarm/wip-18137-jewel

jewel: rbd-mirror: image sync should send NOCACHE advise flag

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #16295 from smithfarm/wip-20265-jewel

jewel: rbd: cli: ensure positional arguments exist before casting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16276 from smithfarm/wip-19563-jewel

jewel: rgw: folders starting with _ underscore are not in bucket index

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16268 from cbodley/wip-16072-jewel

jewel: rgw: rgw_common.cc: modify the end check in RGWHTTPArgs::sys_get

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16266 from cbodley/wip-20408

jewel: rgw: multipart copy-part remove '/' for s3 java sdk request header

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15988 from cbodley/wip-jewel-15983

jewel: rgw: fix infinite loop in rest api for log list

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15449 from smithfarm/wip-19720-jewel

jewel: rgw: rgw_file: fix size and (c|m)time unix attrs in write_finish

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #17351 from liewegas/wip-prune-past-intervals-jewel

osd: add osd_hack_prune_past_intervals

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: osd crush set crushmap need sanity check

The sanity check verifying the new crushmap does not remove crush rules
that are in use is not exclusive to ceph setcrushmap.

Fixes: http://tracker.ceph.com/issues/19302
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit ed760457bf154c10adf75c6df046eecab7eb8e4b)

 crush: when take place the crush map should consider the rule is in used

Signed-off-by: song baisen song.baisen@zte.com.cn
(cherry picked from commit 09bf6f2858d09c8c314a8c242a9c70d2834718dc)

client: skip lookupname if writing to unlinked file

When a client writes to an unlinked file, get_quota_root tries to
resolve the filename via an MDS lookupname op. The op always results in
-13 permission denied for path-restricted caps or -2 no such file or
directory otherwise. More importantly, the repeated lookupname ops slow
down buffered writes significantly.

Don't do the lookupname for unlinked files; use the root_ancentor's
quota instead.

Fixes: http://tracker.ceph.com/issues/20945
Backport: jewel, luminous
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 1d617eee2e8c32f7300cb2cb8dd37b3714a4699f)

Merge pull request #16059 from vumrao/wip-vumrao-20353

jewel: osd: Add heartbeat message for Jumbo Frames (MTU 9000)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15726 from asheplyakov/20314-bp-jewel

jewel: mon: fail to form large quorum; msg/async busy loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

osd: add osd_hack_prune_past_intervals

Last ditch (but dangerous) method of reducing memory usage for
past_intervals, which can help very very unhappy clusters recovery.
A properly implemented version of this is in luminous.  This hacky
version was used successfully to recover multiple jewel-based clusters,
but is still only recommended for use when the OSD is otherwise unable
to recover.

This change is not cherry-picked from master because luminous implements
a more sophisticated version of this that changes the past intervals
representation entirely, but it is too invasive to backport.  This
workaround to prune just on startup should be sufficient for emergencies.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15719 from Vicente-Cheng/wip-20146-jewel

jewel: rgw: 'gc list --include-all' command infinite loop the first 1000 items

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15477 from smithfarm/wip-19767-jewel

jewel: rgw: Delete non-empty bucket in slave zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16015 from asheplyakov/19208-bp-jewel

jewel: osd: osd_internal_types: wake snaptrimmer on put_read lock, too

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15762 from smithfarm/wip-17385-jewel

jewel: ceph cli: Rados object in state configuring race

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15474 from smithfarm/wip-19559-jewel

jewel: osd: objecter full_try behavior not consistent with osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15947 from asheplyakov/20428-bp-jewel

jewel: osd: unlock sdata_op_ordering_lock with sdata_lock hold to avoid missing wakeup signal

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15475 from tchaikov/wip-16239-jewel

qa: add a sleep after restarting osd before "tell"ing it

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #15473 from smithfarm/wip-20080-jewel

jewel: build/ops: rpm: set subman cron attributes in spec file

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Thomas Serlin <tserlin@redhat.com>

Merge pull request #15460 from smithfarm/wip-17843-jewel

jewel: rbd: object-map: batch updates during trim operation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15488 from dillaman/wip-20023-jewel

jewel: rbd: rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15602 from tchaikov/wip-20175-jewel

jewel: tests: decouple ceph_test_librbd_api from libceph-common

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16124 from smithfarm/wip-19174-jewel

jewel: rbd: rbd_clone_copy_on_read ineffective with exclusive-lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: add a field to store generic user data in the bucket index,
that can be populated/fetched via a configurable custom http header

Signed-off-by: Pavan Rallabhandi <prallabhandi@walmartlabs.com>
(cherry picked from commit abca7a86c3cfbb58fafb5d057d9d6f5017a53704)

Conflicts:
src/rgw/rgw_op.cc
Number of arguments to RGWPutObjProcessor_Multipart::do_complete has been changed in master with an extra field accounted_size added, which is not required in Jewel.
         Number of arguments for the processor->complete() has been changed in master, which is not required in Jewel.

src/rgw/rgw_op.h
A new member field `crypt_http_responses` is added in the class `RGWPutObj` in master, which is not required in Jewel.

src/rgw/rgw_rados.cc
         In the function RGWPutObjProcessor::complete() there is an extra argument to do_complete() in master, which is not required in Jewel.
         In the function RGWPutObjProcessor_Atomic::do_complete, a call to obj_op.write_meta has accounted_size, which is not required in Jewel.
         There is an extra argument accounted_size in RGWRados::Bucket::UpdateIndex::complete in master, which is not required in Jewel.
         In RGWRados::Bucket::UpdateIndex::complete, RGWObjEnt has been removed in master, which has to be retained in Jewel.
         In RGWRados::cls_obj_complete_op, user_data is added to the rgw_bucket_dir_entry_meta structure.
         In RGWRados::cls_bucket_list, the user_data field of RGWObjEnt is populated.

src/rgw/rgw_rados.h
         In UpdateIndex::complete(), remove_objs is of type rgw_obj_key in Jewel instead of rgw_obj_index_key
         Corrected various function declarations as appropriate.
         RGWPutObjProcessor_Multipart is not part of this file in Jewel.

src/rgw/rgw_rest_swift.cc
In RGWListBucket_ObjStore_SWIFT::send_response(), accounted_size is not required in Jewel.

src/rgw/rgw_common.h
Add user_data field in RGWObjEnt structure.

src/rgw/rgw_json_enc.cc
Add user_data field while dumping RGWObjEnt.

Merge pull request #15760 from smithfarm/wip-20325-jewel

libradosstriper silently fails to delete empty objects in jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16061 from weiqiaomiao/wqm-wip-ceph-disk-jewel

jewel: ceph-disk: remove the special check to bcache devices

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>

Merge pull request #15503 from Vicente-Cheng/wip-20011-jewel

jewel: tools: ceph-disk: separate ceph-osd --check-needs-* logs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw: Prevent overflow of stats cached values

Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>
Fixes: http://tracker.ceph.com/issues/20934
Signed-off-by: Aleksei Gutikov <aleksey.gutikov@synesis.ru>
(cherry picked from commit 634215eea1ddd4e4f5dc0066c4a2e745cfc20475)

Conflicts:
src/rgw/rgw_quota.cc
       Have the quota stats structures to reflect rounded values in KB and honor the signed integer values

rgw: Do not decrement stats cache when the cache values are zero

With RGWs configured in a load balancer, there is a possibility of
having the cached values going unbound, when PUT/DELETE operations
do not land up on the same RGW. To avoid such cases, make sure the
decrement of stats happen only when the cached values are sane.

Fixes: http://tracker.ceph.com/issues/20661
Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>
(cherry picked from commit 3903e213c7ac7624e3452f5f3b1ca1c339bf2ca2)

Conflicts:
src/rgw/rgw_quota.cc

rgw_file:  pre-compute unix attrs in write_finish()

New serialized Unix attrs need to reflect the change being made,
and should be reverted if the change fails.

Fixes: http://tracker.ceph.com/issues/19653
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ed91d23ccaaac4e72a4c28a58e77485395949f04)

Conflicts:
    src/rgw/rgw_file.cc (processor->complete() takes different arguments in
        jewel than in master)

Merge pull request #14874 from ceph/wip-19805-jewel

jewel: rbd: default features should be negotiated with the OSD

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14977 from dillaman/wip-19873-jewel

jewel: rbd: rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15461 from smithfarm/wip-19795-jewel

jewel: tests: test_notify.py: assert(not image.is_exclusive_lock_owner()) on line 147

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15463 from smithfarm/wip-20017-jewel

jewel: rbd: rbd-nbd: kernel reported invalid device size (0, expected 1073741824)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15464 from smithfarm/wip-20153-jewel

jewel: rbd: Potential IO hang if image is flattened while read request is in-flight

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15455 from smithfarm/wip-19838-jewel

jewel: rgw: reduce log level of 'storing entry at' in cls_log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15456 from smithfarm/wip-20003-jewel

jewel: rgw: rgw_file:  v3 write timer does not close open handles

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15457 from smithfarm/wip-20018-jewel

jewel: rgw: multisite: memory leak on failed lease in RGWDataSyncShardCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15459 from smithfarm/wip-20144-jewel

jewel: rgw: rgw_file: cannot delete bucket w/uxattrs

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15465 from smithfarm/wip-20155-jewel

jewel: rgw: fix crash caused by shard id out of range when listing data log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

librbd: clean up object map update interface, revisited

In master, the "batch update" change [1] was merged before the "order
concurrent updates" [2], while in jewel the latter is already
backported [3]. A backport of [1] to jewel was attempted, and was
necessarily applied on top of [3] - i.e. in the reverse order compared
to how the commits went into master. This reverse ordering caused the
automated cherry-pick to miss some parts of [1] which this commit is
adding manually.

[1] https://github.com/ceph/ceph/pull/11510
[2] https://github.com/ceph/ceph/pull/12420
[3] https://github.com/ceph/ceph/pull/12909

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

librbd: batch ObjectMap updations upon trim

Shrinking a clone which has snapshots and does not share
majority of objects with its parent (i.e., there are less
objects to be copied up) involves huge number of object
map updates -- two (pre, post) per object. This results
in lots of requests to be send to OSDs especially when
trimming a gigantus image. This situation can be optimized
by sending batch ObjectMap updates for an object range
thereby significantly cutting down OSD traffic resulting
in faster trim times.

Fixes: http://tracker.ceph.com/issues/17356
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 05653b7c512334533b801013f7e426363237301b)

Merge PR #16248 into jewel

* refs/remotes/upstream/pull/16248/head:
client: don't re-send interrupted flock request
mds/flock: properly remove item from global_waiting_locks
mds/flock: properly update ceph_lock_state_t::client_waiting_lock_counts
mds/flock: don't import ceph_lock_state_t::waiting_locks

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #15454 from smithfarm/wip-19775-jewel

jewel: rgw: multisite: realm rename does not propagate to other clusters

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15453 from smithfarm/wip-19768-jewel

jewel: rgw: multisite: operating bucket's acl&cors is not restricted on slave zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15452 from smithfarm/wip-19765-jewel

jewel: rgw: when uploading objects continuously into a versioned bucket, some objects will not sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15451 from smithfarm/wip-19764-jewel

jewel: rgw: set latest object's acl failed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15450 from smithfarm/wip-19758-jewel

jewel: rgw: multisite: after CreateBucket is forwarded to master, local bucket may use different value for bucket index shards

Reviewed-by: Casey Bodley <cbodley@redhat.com>

config: disable skewed utilization warning by default

This has a few problems:

1- It does not do it's analysis over CRUSH rule roots/classes, which
means that an innocent user of classes will see skewed usage (bc hdds are
more full than ssds, say)

2- It does not take degraded clusters into account, which means the warning
will appear when a fresh OSD is added.

See http://tracker.ceph.com/issues/20730

Based on master commit 7832c53 but we've decided not to remove the code
in the older releases so it can be used if appropriate for a
particular cluster configuration.  Also, 7832c53 won't cleanly
cherry-pick so this is easier.

Signed-off-by: David Zafman <dzafman@redhat.com>

qa/workunits/rados/test-upgrade-v11.0.0: skip ListObjectsError

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/rados/test-upgrade-v11.0.0: skip more tests

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16870 from liewegas/wip-20929-jewel

jewel: ceph-disk: dmcrypt cluster must default to ceph

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #14659 from asheplyakov/19473-jewel

jewel: rgw: add the remove-x-delete feature to cancel swift object expiration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15322 from gregsfortytwo/jewel-19931-snaptrim-shutdown

jewel: osd: Reset() snaptrimmer on shutdown and do not default-abort on leaked pg refs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15236 from asheplyakov/20036-bp-jewel

jewel: mon: factor mon_osd_full_ratio into MAX AVAIL calc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14346 from ceph/wip-ceph-disk-fix-jewel

jewel: build/ops: Add fix subcommand to ceph-disk, fix SELinux denials, and speed up upgrade from non-SELinux enabled ceph to an SELinux enabled one

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15083 from asheplyakov/19926-bp-jewel

jewel: mon: crash on shutdown, lease_ack_timeout event

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14663 from smithfarm/wip-19228-jewel

jewel: rbd: Enabling mirroring for a pool with clones may fail

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15050 from asheplyakov/19265-bp-jewel

jewel: core: an OSD was seen getting ENOSPC even with osd_failsafe_full_ratio passed

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #15065 from asheplyakov/19915-bp-jewel

jewel: osd: PrimaryLogPG: do not call on_shutdown() if (pg.deleting)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14943 from shinobu-x/wip-18293-jewel

jewel: osd: leaked MOSDMap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14699 from smithfarm/wip-19679-jewel

jewel: mds: damage reporting by ino number is useless

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #14673 from smithfarm/wip-19466-jewel

jewel: mds: log rotation doesn't work if mds has respawned

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/workunits/rados/test-upgrade-v11.0.0.sh: more tests skipped

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15442 from Vicente-Cheng/wip-20025-jewel

jewel: cephfs: osdc/Filer: truncate large file party by party

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14691 from smithfarm/wip-19571-jewel

jewel: tests: upgrade:client-upgrade/firefly-client-x: drop CentOS

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: default features should be negotiated with the OSD

Derived from f066ce8f80bce929edc209590efd47cce2196ae2

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: add get_all_features on client side

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 84f6d5c109911923c25414de639308921983e438)

Conflicts:
src/cls/rbd/cls_rbd_client.h: trivial resolution

Merge pull request #15448 from smithfarm/wip-19615-jewel

jewel: rgw: multisite: bucket zonegroup redirect not working

Reviewed-by: Casey Bodley <cbodley@redhat.com>

tests: upgrade:client-upgrade/firefly-client-x: drop CentOS

The RBD suite needs ceph-cm-ansible to install qemu-kvm on CentOS, but doing
that breaks the firefly install on CentOS because:

1. the qemu-kvm that gets installed is from hammer (0.94.5)
2. qemu-kvm brings in librados2, librbd1 as dependencies

As a result, the hammer librados2 and librbd1 are installed on the test nodes
even before the teuthology install task starts. When it does start and tries
to install firefly, it fails because firefly librados2 and librbd1 cannot be
installed over their hammer versions.

Fixes: http://tracker.ceph.com/issues/19571
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #15197 from smithfarm/wip-20007-jewel

jewel: rgw: remove unnecessary 'error in read_id for object name: default'

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15051 from asheplyakov/19910-bp-jewel

jewel: build/ops: extended, customizable systemd ceph-disk timeout

Reviewed-by: David Disseldorp <ddiss@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #15428 from smithfarm/wip-20162-jewel

jewel: build/ops: deb: Fix logrotate packaging

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15447 from smithfarm/wip-19613-jewel

jewel: rgw: multisite: rest api fails to decode large period on "period commit"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test: remove hard-coded image name from RBD metadata test

Fixes: http://tracker.ceph.com/issues/19798
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8f72e745e157cc12d76ca6babe956c5698ee297f)

librbd: relax "is parent mirrored" check when enabling mirroring for pool

If the parent is in the same pool and has the journaling feature enabled
we can assume the mirroring will eventually be enabled for it.

Fixes: http://tracker.ceph.com/issues/19130
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit fe31bca22f90ce02f461d6421a4f66539db888d3)

Merge pull request #15196 from smithfarm/wip-20006-jewel

jewel: build/ops: rpm: fix python-Sphinx package name for SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: aws4: add rgw_s3_auth_aws4_force_boto2_compat conf option

Runtime bugfix to handle presigned urls computed with canonical requests using
the port number once.

Boto2 computes canonical requests using the port number twice although it
should be used once only. This behaviour is a bug supported by AWS S3. Boto2 is
used in RGW S3 as reference implementation.

The client-side tools not supporting this boto2 bug will fail although they
should work too.

In order to support both presigned url implementations this patch adds a config
option to compute a second signature. With this option disabled, the code will
compute two signatures when the first signature is not valid. The aws4 auth
succeed if some of the two signatures is valid.

The config option rgw_s3_auth_aws4_force_boto2_compat, is enabled by default so
one signature, working with boto2, is computed only.

Fixes: http://tracker.ceph.com/issues/16463
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
(cherry picked from commit 078c513b6bc6b1d1da50db1d51fbbb65bddd44b9)

qa/workunits/rados/test/test-upgrade-v11.0.0: skip Quota test

This makes a pool go full which confounds the wait for healthy check
during restarts.

Signed-off-by: Sage Weil <sage@redhat.com>