jewel: build/rgw: configure if curl is linked against openssl

Since the Luminous/master change was cmake only, making an explicit autoconf
test.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: reuse CURL* for keystone.

When using keystone with https (the recommended setting),
it is inefficient to start up a new SSL connection for
each and every operation.  Keeping a CURL* structure around
should reduce the cost of doing this.  This logic tries
to do so, but it also tries to free them fairly aggressively
(5-10 seconds).  This should still greatly reduce load on
keystone at peak times while not tying up excess resources.

Fixes: https://tracker.ceph.com/issues/23162
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 0c5cee1977eac232045571f4850e64be93271aec)

 Conflicts:
  src/rgw/rgw_common.h
  trivial conflict as lowercase_dash_http_attr isnt a part of L

build/link against curl w/ openssl not nss or gnutls. (debian)

curl + gnutls has some big significant performance hits at least
when doing keystone validation.  nss has long-term memory growth
issues.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit cb12e1afd5beedf1b8c0b85e24e412e34c2fc427)

Merge pull request #20061 from cbodley/wip-jewel-rgw-log-trim

jewel: rgw: automated trimming of datalog and mdlog

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #20557 from smithfarm/wip-really-disable-pidfile-test

test/CMakeLists: disable test_pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

test/CMakeLists: disable test_pidfile.sh

Too flaky, see http://tracker.ceph.com/issues/20975

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ff3de2304497544033837bb8d0c809a9e54a3e6e)

Conflicts:
    src/test/Makefile.am: applied change manually because jewel uses autotools

Merge pull request #20344 from pdvian/wip-22866-jewel

jewel: mon/OSDMonitor: fix dividing by zero in OSDUtilizationDumper

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20463 from dzafman/wip-23006

test: Adjust for Jewel quirk caused of differences with master

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20446 from smithfarm/wip-23010-jewel

jewel: Filestore rocksdb compaction readahead option not set by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20289 from smithfarm/wip-21239-jewel

jewel: test_health_warnings.sh can fail

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20284 from dzafman/wip-22912

jewel: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19611 from ukernel/jewel-22494

jewel: include/fs_types: fix unsigned integer overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #20520 from smithfarm/wip-disable-test-pidfile

test/CMakeLists: disable test-pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

test/CMakeLists: disable test-pidfile.sh

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c4936f4493d34341aab8f01488b45dc981d4ef3f)

Merge pull request #17707 from liewegas/wip-17445

jewel: osd: also check the exsistence of clone obc for "CEPH_SNAPDIR" requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17883 from andyli029/wip-jewel-signal-handler-typo

jewel: core: global/signal_handler.cc: fix typo

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19906 from jdurgin/jewel-filestore-split

jewel: HashIndex: randomize split threshold by a configurable amount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20331 from dzafman/wip-21036

jewel: snapset xattr corruption propagated from primary to other shards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18294 from smithfarm/wip-21796-jewel

jewel: Ubuntu amd64 client can not discover the ubuntu arm64 ceph cluster

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18850 from vumrao/vumrao-jewel-rocksdb-cache-size

jewel: common/config: set rocksdb_cache_size to OPT_U64

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #19330 from ovh/bp-21862-jewel

jewel: tools/ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19927 from jdurgin/wip-jewel-omap-recovery

jewel: config: lower default omap entries recovered at once

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19978 from pdvian/wip-22693-jewel

jewel: common: compute SimpleLRU's size with contents.size() instead of lru

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20108 from tchaikov/wip-18004-jewel

jewel: osd: update heartbeat peers when a new OSD is added

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #20146 from smithfarm/wip-22818-jewel

jewel: repair_test fails due to race with osd start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #20306 from tchaikov/wip-pr-16661-jewel

jewel: throttle: Minimal destructor fix for Luminous

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

test: Adjust for Jewel quirk caused of differences with master

Fixes: http://tracker.ceph.com/issues/23006
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #18780 from shinobu-x/wip-21053-jewel

jewel: RHEL 7.3 Selinux denials at OSD start

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #20143 from smithfarm/wip-21872-jewel

jewel: ObjectStore/StoreTest.FiemapHoles/3 fails with kstore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

common/options.cc: Set Filestore rocksdb compaction readahead option.

fixes: http://tracker.ceph.com/issues/21505

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 66567e573836c570040c8d8148c34b0cf9dc9ce2)

Conflicts:
src/common/options.cc (master commit made the change in
           src/common/options.cc, which doesn't exist in jewel)

mon/OSDMonitor: fix deviding by zero in OSDUtilizationDumper

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
(cherry picked from commit 9fe103233eaee965849c00c66c1b352f31932e01)

Merge pull request #19907 from pdvian/wip-22590-jewel

jewel: cephfs: ceph.in: pass RADOS inst to LibCephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20067 from smithfarm/wip-22762-jewel

jewel: mds: unbalanced auth_pin/auth_unpin in RecoveryQueue code

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #20074 from smithfarm/wip-22772-jewel

jewel: rgw: user creation can overwrite existing user even if different uid is given

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20057 from linuxbox2/jewel-rgw-latchfix

jewel: rgw: rgw_file: alternate fix deadlock on lru eviction

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20039 from oritwas/wip-rgw-jewel-22703

jewel: rgw: resharding needs to set back the bucket ACL after link

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #19908 from pdvian/wip-22582-jewel

jewel: rgw: dont log EBUSY errors in 'sync error list'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19783 from ceph/wip-rgw-22182

jewel: rgw: segfaults after running radosgw-admin data sync init

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18305 from smithfarm/wip-21447-jewel

jewel: rgw: multisite: Get bucket location which is located in another zonegroup, will return 301 Moved Permanently

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #18121 from linuxbox2/jewel-quota-multipart

jewel: RGW: Multipart upload may double the quota

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #18116 from linuxbox2/jewel-mpserializer

jewel: rgw: release cls lock if taken in RGWCompleteMultipart

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17731 from ceph/wip-rgw-jewel-15998

jewel: rgw: fix marker encoding problem

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Marcus Watts <mwatts@redhat.com>

Merge pull request #17151 from smithfarm/wip-20480-jewel

jewel: rgw: log includes zero byte sometimes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17149 from smithfarm/wip-20399-jewel

jewel: rgw: rgw_file: recursive lane lock can occur in LRU drain

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #16763 from linuxbox2/mb-jewel-rgw-delparts

jewel: rgw: fix the bug that part's index can't be removed after completing

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #16708 from linuxbox2/wip-jewel-15912-15673

jewel: rgw: Fix duplicate tag removal during GC, cls/refcount: store and use list of retired tags)

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19975 from pdvian/wip-22689-jewel

jewel: client: release revoking Fc after invalidate cache

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #18084 from batrick/i17935

jewel: ceph_volume_client: fix setting caps for IDs

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #17925 from batrick/i21519

jewel: client: set client_try_dentry_invalidate to false by default

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #17188 from smithfarm/wip-21067-jewel

jewel: mds: fix integer overflow

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20287 from smithfarm/wip-22191-jewel

jewel: class rbd.Image discard----OSError: [errno 2147483648] error discarding region

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20286 from smithfarm/wip-22186-jewel

jewel: abort in listing mapped nbd devices when running in a container

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #19801 from smithfarm/wip-21642-jewel

jewel: rbd ls -l crashes with SIGABRT

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20282 from smithfarm/wip-21971-jewel

jewel: [journal] tags are not being expired if no other clients are registered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20281 from smithfarm/wip-21290-jewel

jewel: [rbd] image-meta list does not return all entries

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20280 from smithfarm/wip-21266-jewel

jewel: [cli] rename of non-existent image results in seg fault

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19855 from pdvian/wip-22594-jewel

jewel: rbd: librbd: filter out potential race with image rename

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19644 from pdvian/wip-22498-jewel

jewel: rbd-mirror: cluster watcher should ensure it has latest OSD map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19186 from pdvian/wip-22209-jewel

jewel: rbd: disk usage on empty pool no longer returns an error message

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19115 from pdvian/wip-22173-jewel

jewel: [rbd-nbd] Fedora does not register resize events

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19098 from pdvian/wip-22170-jewel

jewel: librbd: set deleted parent pointer to null

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18843 from shinobu-x/wip-21866-jewel

jewel: rbd: rbd crashes during map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19194 from mdw-at-linuxbox/wip-jewel-rgw-swift-anon

jewel: rgw: fix swift anonymous access

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: be_select_auth_object() sanity check oi soid

Fixes: http://tracker.ceph.com/issues/18409
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9614ab556ca8e4e5daec1e71d9b6032633ba21a0)

osd: Primary shard needs to be at the front to be preferred

Caused by: cd0d8b0714d8684cf61b4650e170027ef46f489b

Fixes: http://tracker.ceph.com/issues/21907
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b7d2f27da1a68046a688749f95093a88e6362b28)

osd: Check snapset for validity when selecting authoritative shard

Fixes: http://tracker.ceph.com/issues/20186
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cd0d8b0714d8684cf61b4650e170027ef46f489b)

Conflicts:
src/osd/PGBackend.cc (trivial, jewel has BitwiseComparator)

throttle: Minimal destructor fix for Luminous

Get rid of the undefined behavior of destroying condition variables
while they're being waited on.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 0f52f486ef05edf8bf89e2bbb7b79f1bc1644cfa)

Conflicts:
src/common/Throttle.cc
src/common/Throttle.h
src/test/common/Throttle.cc: in jewel, we don't have perfconter
for throttles, neither do we have backport14.h back then, so we need to
resolve these conflicts, by removing perfcounter related change in
Throttle.cc, and add the make_unique helper for test/common/Throttle.cc,
add scope_guard to Throttle.cc.

rgw: resharding needs to set back the bucket ACL after link

Jewel only fix. New implementation of resharding in Luminous.
Fixes: http://tracker.ceph.com/issues/22703
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #19804 from smithfarm/wip-21691-jewel

jewel: tests: rbd_mirror_helpers.sh request_resync_image function saves image id to wrong variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd discard return -EINVAL if len > MAX_INT32

rbd discard use 'int' to return discarded length, but the 'len' user
passed is 'uint64', in some case, the ret value will be truncated
and return a negative value which means discard failed. ret -EINVAL
if len > MAX_INT32 to indicate support len <= MAX_INT32 only.

Fixes: http://tracker.ceph.com/issues/16465
Signed-off-by: Huan Zhang <zhanghuan@chinac.com>
(cherry picked from commit ec259dfde87ed9bc5e7247f311b629707c636895)

librbd: prevent overflow of discard API result code

Prevent discard/writesame lengths larger than 2GB.

Fixes: http://tracker.ceph.com/issues/21966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3effd324db181e625665be33b5c6529dca723cc5)
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Conflicts:
PendingReleaseNotes (adapted for jewel)
src/librbd/librbd.cc (no writesame in jewel)

osd: subscribe to new osdmap while waiting_for_healthy

If we are sitting around waiting until we are able to ping our "up" peers,
we need to be sure that our notion of "up" is still correct and we're not
just stuck on an old, stale OSDMap.

Fixes: http://tracker.ceph.com/issues/21121
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fbafa659dae94faba435ca449ee5e77b51108b4b)

rbd-nbd: fix unused nbd device search bug in container

In some container scenarios, the host may choose to
map a specific nbd device, for example, /dev/nbd6 into the
container, in that case, the nbd device available in the
container is not numbered from 0. The current unused
nbd device search function will return no result.
This patch fixes it.

Fixes: http://tracker.ceph.com/issues/22012
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
Reviewed-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>
(cherry picked from commit be0f9581f9727187ca03232e0b368e7da7a60609)

rbd-nbd: only try to load nbd module once when open nbd device, during do_map operation.

Signed-off-by: Pan Liu <liupan1111@gmail.com>
(cherry picked from commit 37aa18858b38cee61e122adc0a1834ca797ea84f)

cls/journal: ensure tags are properly expired

Previously, if only the local image was using the journal or if
a disconnected peer was attached, the tag entries could not be
expired even if unreferenced.

Fixes: http://tracker.ceph.com/issues/21960
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 19fa1c7f5b2809e9a223b7b196dfc031e97a5dcd)

Conflicts:
src/cls/journal/cls_journal.cc (jewel does not have d3de6f5e07cb30d81ce8cc539256c6cf754b28d5)

rbd: support iterating over metadata items when listing

Fixes: http://tracker.ceph.com/issues/21179
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9c4f9856b034061aed052192b0ee09b9a9d09bc0)

librbd: rename of non-existent image results in seg fault

Fixes: http://tracker.ceph.com/issues/21248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4a75ee43d3b108a9593a010160479fd62884eda8)

Conflicts:
        src/librbd/internal.cc (retain redundant "delete ictx" that has been
            dropped in master)

ceph-objectstore-tool: performance: Add --pgid for faster search for object by name

Fixes: http://tracker.ceph.com/issues/21327
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 14c5c834b9cfbf505197d999a776dd69d5e4d109)

Conflicts:
src/tools/ceph_objectstore_tool.cc (trivial)

Merge pull request #20070 from smithfarm/wip-22771-jewel

jewel: tools: ceph-objectstore-tool set-size should clear data-digest

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #19469 from linuxbox2/jewel-22410

jewel: rgw: fix chained cache invalidation to prevent cache size growth

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19747 from linuxbox2/jewel-rgw-21984

jewel: rgw: fix doubled underscore with s3/swift server-side copy

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Marcus Watts <mwatts@redhat.com>

Merge pull request #19769 from linuxbox2/jewel-rgw-22517

jewel: rgw: resolve Random 500 errors in Swift PutObject (22517)

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #18303 from smithfarm/wip-21632-jewel

jewel: doc: remove region from INSTALL CEPH OBJECT GATEWAY

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: use bucket marker for multipart complete oid

the use of bucket_id breaks multipart uploads if the bucket has been
resharded, because InitMultipart uses the old bucket_id and Complete
uses the new

Fixes: http://tracker.ceph.com/issues/22368
(this fixes a previous backport, so has no associated commit from master)

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0dd77b24819323808860c6fb98a0bc2f9f27e26f)

rgw: release cls lock if taken in RGWCompleteMultipart

Follows Casey's proposal to conditionally release the lock in
::complete(), in order to avoid duplicated code in various early
return cases.

Fixes: http://tracker.ceph.com/issues/21596
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 704f793f08a02760d23eb5778b738bb07be0e7cf)

RGW: Fix duplicate tag removal during GC

We need to remove all processed tags before we fetch a new batch of tags
for removal, otherwise some tags might get processed twice.

Fixes: http://tracker.ceph.com/issues/20107
Signed-off-by: Jens Rosenboom <j.rosenboom@x-ion.de>
(cherry picked from commit 8eeee2b78d6f4176fb20c2b1499e3237ee229bf4)

qa/tasks/reg11184: import run

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 59e3827be7ff4cbcb24b196125b25802ffd741c8)

qa/tasks/ceph_manager: wait for osd to start after objectstore-tool sequence

Fixes: http://tracker.ceph.com/issues/20705
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 583a38bca293495679d4372f7470e4e1f2a930fb)

Conflicts:
    qa/tasks/ceph_manager.py (bring in a053ce091e1aa910a1d01aec489203500e67efe5
        which has to be cherry-picked manually anyway, because it pre-dates the
        ceph-qa-suite move)

test: fix misc fiemap testing

1) Different filesystem will have different behavior to
allocate extents. Therefore, even if write 4000 extents,
the filesystem may not really allocate 4000 extents.

2) kstore always return [0, xxx] even if offset ~= 0. Therefore,
the whole non-zero offset FiemapHoles test should be skipped

3) enable fiemap test for filestore, bluestore, memstore again

Fixes: http://tracker.ceph.com/issues/21716
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 87f33376d977962ab7438c46873ea9b6292390d1)

Conflicts:
    src/test/objectstore/store_test.cc (master does not set
       filestore_op_thread_suicide_timeout; trivial resolution)

Merge pull request #18951 from dzafman/wip-21440

Jewel: Performance: Only scan for omap corruption once

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #18690 from dzafman/wip-1508935

jewel: osd: recover_replicas: object added to missing set for backfill, but is not in recovering, error!

Reviewed-by: Kefu Chai <kchai@redhat.com>

OSD: heartbeat peers need to be updated when a new OSD added into an already existed cluster

Fixes: http://tracker.ceph.com/issues/18004
Signed-off-by: Pan Liu <pan.liu@istuary.com>
(cherry picked from commit e95026f6be44f5aae93f005d249ffcf66bef15fd)

rgw: add TODOs to split trim logic into separate source files

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ec0a655a80b20e9c76cf0d65cfea3fde66ce0a10)

rgw: use RGWShardCollectCR in MetaMasterTrimCR

limit the number of concurrent sync status requests to peers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 667ec9f3653cf40edcc475f66819684a26d77d61)

radosgw-admin: add 'mdlog autotrim' command

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 16462640a311fdd262a3ee74552fc2297a4f1db5)

rgw: RGWSyncLogTrimThread runs mdlog trim

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit aaab1ec2f736248b3557f42b4108e91b45f6b407)

rgw: add MetaTrimPollCR to coordinate polling and leases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8431f178004b952d9443a3acbd315d46514fadc2)

rgw: peer trims mdlog shards up to master's oldest entry

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0557a79612504646e9f195cacff839e6ad050957)

rgw: add MetaPeerTrimShardCR to trim mdlog shards

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3376c4b67c52f02111dea3bc032403f425a709c1)

rgw: peer purges mdlog periods before master's oldest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6cde60e954e60ff77e93b93e8c5bd7059f6a2644)