Merge pull request #22128 from liewegas/wip-rbd-msgr-luminous

luminous: qa/suites/rbd/basic/msgr-failures: remove many.yaml

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd/basic/msgr-failures: remove many.yaml

Overkill, and triggers some failures, see
http://tracker.ceph.com/issues/23789

Removed in master by 4046f46d0e6a70d860d74945dfb95c2511394640

Fixes: http://tracker.ceph.com/issues/23789
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #21547 from VictorDenisov/backport

luminous: tests: filestore journal replay does not guard omap operations

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #21515 from tchaikov/wip-luminous-pr-21469

luminous: mon/LogMonitor: do not crash on log sub w/ no messages

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #21376 from pdvian/wip-23666-luminous

luminous: msg/async/AsyncConnection: Fix FPE in process_connection

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21405 from pdvian/wip-23672-luminous

luminous: os/bluestore: alter the allow_eio policy regarding kernel's error list.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21407 from tchaikov/wip-luminous-23246

luminous: os/bluestore: fix exceeding the max IO queue depth in KernelDevice.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #21514 from smithfarm/wip-posix-zfs-luminous

luminous: common: posix_fallocate on ZFS returns EINVAL

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #21818 from xiexingguo/wip-23925

luminous: osd/OSDMap: check against cluster topology changing before applying pg upmaps

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21575 from ceph/wip-cd-fix-pool-create

luminous: tests: ceph-deploy: create the rbd pool right after install

Merge pull request #21705 from smithfarm/wip-23913-luminous

luminous: rbd-nbd can deadlock in logging thread

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21939 from pdvian/wip-23986-luminous

luminous: osdc/Objecter: fix recursive locking in _finish_command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21965 from pdvian/wip-24014-luminous

luminous: mgr/influx: Only split string on first occurence of dot (.)

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #21378 from pdvian/wip-23668-luminous

luminous: mon: add  'ceph osd pool get erasure allow_ec_overwrites' command

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #21556 from tchaikov/wip-luminous-23763

luminous: test/librados: increase pgp_num along with pg_num

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21764 from pdvian/wip-23914-luminous

osd/PrimaryLogPG: avoid infinite loop when flush collides with write …

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21964 from pdvian/wip-23988-luminous

luminous: osd/PG: fix DeferRecovery vs AllReplicasRecovered race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #21993 from pdvian/wip-24015-luminous

luminous: osd/PG: fix uninit read in Incomplete::react(AdvMap&)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22012 from b-ranto/wip-bp-selinux

luminous: Backport recent SELinux changes

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21674 from pdvian/wip-23865-luminous

luminous: rgw: fix error handling for GET with ?torrent

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #19884 from pdvian/wip-22637-luminous

luminous: rgw:lc: RGWPutLC return ERR_MALFORMED_XML  when missing <Rule> tag in…

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #21382 from smithfarm/wip-23681-luminous

luminous: mg_read() call has wrong arguments

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21435 from pdvian/wip-23682-luminous

luminous: rgw: return EINVAL if max_keys can not convert correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21438 from pdvian/wip-23685-luminous

luminous: rgw_file: conditionally unlink handles when direct deleted

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21660 from pdvian/wip-23862-luminous

luminous: rgw: aws4 auth supports PutBucketRequestPayment

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #21661 from pdvian/wip-23861-luminous

luminous: rgw: admin rest api shouldn't return error when getting user's stats if

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21669 from oritwas/wip-rgw-luminous-22721

luminous: rgw: fix bi_list to reset is_truncated flag if it skips entires

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21675 from pdvian/wip-23866-luminous

luminous: rgw: move all pool creation into rgw_init_ioctx

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21735 from pdvian/wip-23868-luminous

luminous: rgw: do not reflect period if not current

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21736 from pdvian/wip-23869-luminous

luminous: rgw: es module: set compression type correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21758 from cbodley/wip-23864

luminous: rgw: add buffering filter to compression for fetch_remote_obj

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21792 from cbodley/wip-23977

luminous: rgw: raise log level on coroutine shutdown errors

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21908 from cbodley/wip-24060

luminous: radosgw-admin sync status improvements

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21917 from smithfarm/wip-24063-luminous

luminous: rgw: Misnamed S3 operation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21741 from pdvian/wip-23902-luminous

luminous: [rbd-mirror] local tag predecessor mirror uuid is incorrectly replaced with remote

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21869 from pdvian/wip-23945-luminous

luminous: qa/workunits/rbd: potential race in mirror disconnect test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21944 from dillaman/wip-23809-luminous

luminous: qa/workunits/rbd/import_export: update unit formatting

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #21995 from tchaikov/luminous-24051

luminous: os/bluestore: fix the demotion in StupidAllocator::init_rm_free

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
ifed01

selinux: Allow ceph to execute ldconfig

The ceph-volume testing showed that the ceph daemons can run ldconfig in
a corner case when they are forbidden access to some files. This patch
allows ceph to execute ldconfig in Enforcing mode.

Fixes: https://tracker.ceph.com/issues/22302
Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit fa5071b6d7182f54cd7b1ffe171a4b006f5255cb)

selinux: Allow ceph to block suspend

Ceph is now trying to block suspend in certain cases, we need to update
the policy accordingly.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit e3be372d854f9014ca711f14525ca9e510d34f1a)

os/bluestore: fix the demotion in StupidAllocator::init_rm_free

* we should update the _size field when demotion kicks in, otherwise the
  interval_set::m and interval_set::_size won't be consistent.
* also rename post_process to claim, as it "steals" the leftover(s) of
chopped segment.

Fixes: https://tracker.ceph.com/issues/24051
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d43c8da83329dd743c635869b1443366d03839fd)

Merge pull request #21872 from tchaikov/wip-luminous-jdk

luminous: cmake: fix the cepfs java binding build on Bionic

Reviewed-by: Robin H. Johnson <robin.johnson@dreamhost.com>

osd/PG: fix uninit read in Incomplete::react(AdvMap&)

If a PG is incomplete when the pool is deleted we'll dereference invalid
iterators here.

Fixes: http://tracker.ceph.com/issues/23980
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3187b036d83adae3646d411e6705e056189f33bb)

radosgw-admin: add pretty 'bucket sync status' command

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 650aef59cfaf429951a82a9d65a37b8c4bd1b942)

Conflicts:
src/rgw/rgw_admin.cc (no readonly_ops_list, no std::string_view)

rgw: expose struct bucket_index_marker_info in header

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6274368d9cc6566e85b52e53d6949a7814a5ea27)

rgw: rgw_bucket_sync_status takes bucket info

rgw_bucket_sync_status() no longer reads the bucket instance info, and
instead requires the caller to pass it in

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3640ee844a9d77f0fa6a6849c4d8b51cc932f4e2)

radosgw-admin: rename 'bucket sync status' to 'bucket sync markers'

renames the command to 'bucket sync markers' because it only returns the
local sync status markers for use in multisite tests

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2df2b0665d12dd9e47bf7155a1c6335c2d1a3bf4)

Conflicts:
src/rgw/rgw_admin.cc (no readonly_ops_list)

rgw: translate the state in rgw_data_sync_marker

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 7ddca6e50d85380505134481d37734b4f1433f1a)

doc: update radosgw-admin.rst and help.t about data sync status

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 33a0ec7164eff9fb66f05d984eb92b0de4ced9b1)

rgw: add --shard-id for data sync status

when specified --shard-id with data sync status, it'll
display the realated pending and recovering bucket shards, as well as sync
marker, besides this, we can specified --max-entries to limit the number
of datalog entries(for pending buckets) or omapkey entries(for recovering
buckets) be consumed.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 1dc7f5ea207dc17c743bfa1e06db446e16d31aa6)

rgw: read behind bucket shards of a specified data log shard

add read_shard_status to read pending bucket shards,
recovering bucket shards and sync marker

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 9c5ff46ad9c849eb0c99a4d78be3f9dd2299779c)

Conflicts:
rgw_data_sync.cc (make_move_iterator fails to build with gcc<5)

osd/PG: fix DeferRecovery vs AllReplicasRecovered race

- DeferRecovery event queued by AsyncReserver due to preemption
  event.  We are in Recovering state with RECOVERING bit set.
- We finish recovery, clear RECOVERING state bit, and queue
  AllReplicasRecovered from PrimaryLogPG::start_recovery_ops()
- DeferRecovery event arrives, moving us from Recovering -> NotRecovering
- AllReplciasRecovered event arrives, crashing us.

This is all hard to deal with because the events are queued and may
arrive later.  Solve the problem here by tolerating a delayed
DeferRecovery event: if the RECOVERING pg state bit isn't set, ignore
it (it's old).  The async reserver cancel events are unpredictable.

Fixes: http://tracker.ceph.com/issues/23860
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cfe59cf20c4b09aa7b25c3f9a724a01380699744)

mgr/influx: Only split string on first occurence of dot (.)

Service names are not always osd.X or mon.X, they might be
rgw.radosgw.rgw1

This would lead to:

  Unhandled exception from module 'influx' while running on mgr.mon01: too many values to unpack

Only split on the first dot as the rest is the service name

Fixes: http://tracker.ceph.com/issues/23996
Signed-off-by: Wido den Hollander <wido@42on.com>
(cherry picked from commit bd7d0fc16a90b3fefadc2d6904180a7fd04ef5eb)

global/global_init: close stdout in postfork_finish, not start

During the interval where we have done our prefork but not actually
detached from the parent to daemonize, we may still want to send output
to stdout.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 056d02b0649d28200170f729dc0d625209206cfb)

log: allow stop() when not started

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f3862d8849c014a4a863ebedc9719241ec9c2c46)

rbd-nbd: fix postfork_{start,finish} callers

_start is called right after the prefork().  _finish is called when we
do the logical daemonize() and say goodbye to the parent.

Fixes: http://tracker.ceph.com/issues/23143
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a350c8032af41452b8f9a200f86e7f0dc0db7345)

Merge pull request #21936 from ceph/wip-yuriw-24021-luminous

qa/test: added client-upgrade-luminous suite

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21871 from tchaikov/wip-luminous-18649

luminous: mgr/restful: A couple of restful fixes

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #21911 from dzafman/backport_23850-zafman

osd/ECBackend: only check required shards when finishing recovery reads

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa/test: added client-upgrade-luminous suite
(whitelisted CACHE_POOL_NO_HIT_SET)

Fixes https://tracker.ceph.com/issues/24021

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/workunits/rbd/import_export: update unit formatting

Mimic changed the unit formating output for rbd CLI commands, so
for upgrade tests we need to support both Luminous and Mimic style.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osdc/Objecter: fix recursive locking in _finish_command

The path

#9  Objecter::_finish_command (this=this@entry=0x7f76c00aeb30, c=c@entry=0x7f76b0000b10, r=<optimized out>, rs="osd down") at /build/ceph-13.0.2-1932-g458b4fb/src/osdc/Objecter.cc:4950
#10 0x00007f76d26de106 in Objecter::_check_command_map_dne (this=this@entry=0x7f76c00aeb30, c=c@entry=0x7f76b0000b10) at /build/ceph-13.0.2-1932-g458b4fb/src/osdc/Objecter.cc:1726
#11 0x00007f76d26e52e4 in Objecter::_scan_requests (this=this@entry=0x7f76c00aeb30, s=0x7f76c00af8a0, skipped_map=skipped_map@entry=false, cluster_full=cluster_full@entry=false, pool_full_map=0x7f76be7fb330, need_resend=..., need_resend_linger=..., need_resend_command=std::map with 0 elements, sul=...,
    gap_removed_snaps=0x7f76ac0016f8) at /build/ceph-13.0.2-1932-g458b4fb/src/osdc/Objecter.cc:1120
#12 0x00007f76d26eded5 in Objecter::handle_osd_map (this=this@entry=0x7f76c00aeb30, m=m@entry=0x7f76ac0014a0) at /build/ceph-13.0.2-1932-g458b4fb/src/osdc/Objecter.cc:1228

led to recursive lock of the session mutex (locked in _scan_requests,
and again in _finish_command).

Fix by making the callers for _finish_command (and
_check_command_map_dne) take the session lock.

Fixes: http://tracker.ceph.com/issues/23940
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 891f5192427a4a783d5d7194fc2556dfdc1a0ed2)

Merge pull request #21725 from smithfarm/wip-23631-luminous

luminous: rbd: python bindings fixes and improvements

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21726 from smithfarm/wip-23900-luminous

luminous: [rbd-mirror] asok hook for image replayer not re-registered after bootstrap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21870 from tchaikov/wip-luminous-24041

luminous: ceph-disk: write log to /var/log/ceph not to /var/run/ceph

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #21919 from smithfarm/wip-24070-luminous

luminous: build-integration-branch script

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados: add coverage for osd_recovery_max_single_start > 1

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 234d652317baefb28fd498f476f1db573974db4c)

osd/ECBackend: preserve requests for other objects when sending extra reads

When multiple objects are in flight for the same ReadOp, swap() on the
map<hobject_t, read_request_t> would remove requests for all objects.

We just want to replace the requests for the single object we're
dealing with in send_all_remaining_reads().

This prevents crashing trying to look up rop.to_read[hoid] when another
object in the same ReadOp gets an EIO and tries to send more requests.

Test this by using osd-recovery-max-single-start to bundle multiple
reads into one ReadOp. Save and restore CEPH_ARGS so custom settings
are reset for each test.

Fixes: http://tracker.ceph.com/issues/23195 (the 2nd crash there)
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit d4808256d25851f85ce5e86704cc4e867e2fc905)

osd/ECBackend: recover from EIO based on the minimum data necessary

Discount shards that already returned EIO, and use minimum_to_decode()
to request just what is necessary to recover or read the originally
requested extents of the object.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit b162a5478d6a907cc0a9ddd5ae8442e81f8d8fb3)

Conflicts:
src/osd/ECBackend.cc (Adjust for Luminous not having subchunks)
src/osd/ECBackend.h (trivial)

osd/ECBackend: only check required shards when finishing recovery reads

1235810c2ad08ccb7ef5946686eb2b85798f5bca allowed recovery to use
multiple passes of reads to handle EIO, but the end condition for
checking whether we finished reading requires the full data to be
decodable (this is what get_want_to_read_shards returns).

This is just a loss of efficiency normally, since when there is only
one object the subsequent read works, and grabs all the data
necessary. The crash comes from having multiple objects in the same
ReadOp - in this case the sequence of events is:

- start recovery of two objects (osd_recovery_max_single_start > 1)
- read object a shard 3
- read object b shard 3
- fail minimum_to_decode because shard 3 can't reconstruct all of object a
- re-read all of object a, marking more reads in progress
- fail minimum_to_decode because shard 3 can't reconstruct all of object b
- skip re-reading object because there are now reads in progress
- finish reading k shards of object a
- still fail minimum_to_decode for object b, so no extra data was read
- send_all_remaining_reads tries to lookup object b in ReadOp object
- crash dereferencing to_read[object b], since this was cleared after handling the original object b read reply

This patch fixes the immediate inefficiency and crash by only checking
for the missing shards that were requested, rather than the entire
object, for recovery reads.

Fixes: http://tracker.ceph.com/issues/23195 (first crash)
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 468ad4b41010488c8d48ef65ccbebfdb4270690f)

Conflicts:
src/osd/ECBackend.cc (trivial)

osd/ECBackend: store the original needed shards in a ReadOp

This avoids extra logic recomputing them due to EIO or redundant
reads.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 0f9474d4e8e54cb8784cc2dbaa50318c2ff8e667)

Conflicts:
src/osd/ECBackend.cc (trivial)

qa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/

it's a regression introduced by ac56a202

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fc43ae1724f6cdf3a00e1995281e1e4d414093da)

Brought into Luminous to make backport clean

qa/standalone: extract delete_pool()

some tests, like osd-backfill-stats.sh are using delete_pool(), but
they don't have this function defined. and this function is defined
in standalone tests separately, so would be simpler if we can
consolidate them in ceph-helper.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ac56a202fd57d6a18d3852d26a4e5cf4a214bf8a)

Brought into Luminous to make backport clean

script: build-integration-branch: avoid Unicode error

When run on PRs that have non-ASCII characters in their titles, the script
fails like this when run in a non UTF-8 environment:

UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2026' in
position 651: ordinal not in range(256)

Since failing like that is not useful, avoid the error by forcing ASCII
encoding and filtering out any non-ASCII characters.

Fixes: http://tracker.ceph.com/issues/24003
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 6dd31fdbec43e37bece24c0de886d2c4083901c8)

script/build-integration-branch: python3 compatible and pep8 clean

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4e8e3116949d11f62ac027e300c09dc5f929cfc5)

script/build-integration-branch: check errors

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 686f2ae7fa8050d6113674ea5570ad956792e49a)

script/build-integration-branch: print pr url list with titles

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5916683c52fe6c1d54e91415ba0c4af790d48485)

src/script/build-integration-branch

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b5c27826158f699960fcf887e901545c359ad68c)

rgw: policy: modify s3:ListBucketMultiPartUploads to s3:ListBucketMultipartUploads

according to AWS S3, the permission keyword should be s3:ListBucketMultipartUploads
rather than s3:ListBucketMultiPartUploads.

all operation permission list in AWS S3 as follows:
  https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html?shortFooter=true

Signed-off-by: xiangxiang <xiangxiang@xsky.com>
(cherry picked from commit 41ef62d9047ca7469023181f203e79c63ec30d64)

rgw: display data sync recovering shards in radosgw-admin sync status

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 8c3ea263b910db99dfe7e7aa5f17193941c70d1d)

rgw: add RGWReadDataSyncRecoveringShardsCR to read recovering shards

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit f4de5b58e27f320e8eed6a72684ad7621a7c1110)

rgw: display errors of object sync failed in sync error list

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 39fd90c8ad2ce5af8d074a7f52d12dea3b06c730)

rgw: add lagging shard ids in rgw sync status

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
(cherry picked from commit 99f3c3f8e97159cda401b280ea25f9822b14b90d)

rgw: RGWRadosGetOmapKeysCR uses omap_get_keys2

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0ccf50fc9df95bf2b125e4b3cb1b641aef8c23b5)

Conflicts: (no sync tracing)
src/rgw/rgw_data_sync.cc
src/rgw/rgw_sync.cc

rgw: RGWRadosGetOmapKeysCR uses completion return code

when this operates on a nonexistent object, the osd will reject the
request with ENOENT before trying to process the subops. so Objecter
will get back a subop return code of 0, try to decode an empty
bufferlist into the result and map that subop return code to EIO

by using the AioCompletion's return code, we get the correct result of
ENOENT instead

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2f94f63e705c33c4bf57923d96c747c5c6681e2d)

Merge pull request #21109 from pdvian/wip-23474-luminous

luminous: client: add ceph_ll_sync_inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #21346 from pdvian/wip-23632-luminous

luminous: mds: mds shutdown fixes and optimization

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21354 from pdvian/wip-23635-luminous

luminous: client: fix request send_to_auth was never really used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21638 from smithfarm/wip-23852-luminous

luminous: osd: do not crash on empty snapset

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21359 from xiexingguo/backport-pr-21315

luminous: crush/CrushWrapper: fix potential invalid use of iterator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21366 from pdvian/wip-23636-luminous

luminous: mds: fix unhealth heartbeat during rejoin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21372 from pdvian/wip-23637-luminous

luminous: mds: mark damaged if sessions' preallocated inos don't match inotable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21441 from pdvian/wip-23702-luminous

luminous: mds: properly dirty sessions opened by journal replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21447 from pdvian/wip-23703-luminous

luminous: MDSMonitor: fix mds health printed in bad format

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21448 from pdvian/wip-23750-luminous

luminous: mds: make rstat.rctime follow inodes' ctime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21512 from pdvian/wip-23791-luminous

luminous: MDSMonitor: initialize new Filesystem epoch from pending

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21542 from pdvian/wip-23802-luminous

luminous:client: flush the mdlog in _fsync before waiting on unstable reqs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21596 from joscollin/wip-23771-luminous

luminous: client: Fix the gid_count check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21732 from batrick/i23792

luminous: MDSMonitor: cleanup and protect fsmap access

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21796 from dalgaaf/luminous-issue-23873

luminous: client: use common interp of st_nlink for dirs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>