osd: DBOjectMap::check: Dump complete mapping when inconsistency found

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fcf1e17c645e8fad5216c3e59627c817e5c858c7)

test_object_map: Use ASSERT_EQ() for check() so failure doesn't stop testing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 053a273cbc02d6902a4bb1f11db1ea946498df3a)

tools: Check for overlaps in internal "complete" table for DBObjectMap

Changed check to return an error count and fix tool error message

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e5e8eb962db6187ea19b96ba29ac83469c90b4ea)

Conflicts:
src/os/filestore/DBObjectMap.h (trivial)

tools: Add dump-headers command to ceph-osdomap-tool

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f4101591ad701a62fe027c4744ca8ea505f44bdc)

Conflicts:
src/os/filestore/DBObjectMap.h (trivial)

tools: Add --oid option to ceph-osdomap-tool

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2d94889e9ee3359017b1efd560f3557ce03ccee6)

osd: Remove unnecessary assert and assignment in DBObjectMap

Fix and add comment(s)

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 937e6a03ea4692cc44d53faa0615f8e808c9eb03)

Merge pull request #14809 from tchaikov/wip-18193-jewel

jewel: tests: test/librados/tmap_migrate: g_ceph_context->put() upon return

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14701 from smithfarm/wip-18193-jewel

jewel: core: transient jerasure unit test failures

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14776 from linuxbox2/jewel-pullup-civet-chunked

[DNM] jewel: pullup civet chunked

Merge pull request #14416 from smithfarm/wip-19557-jewel

jewel: tests: upgrade/hammer-x failing with OSD has the store locked when Thrasher runs ceph-objectstore-tool on down PG

Reviewed-by: Kefu Chai <kchai@redhat.com>

civetweb:  pullup chunked encoding by Marcus

Fixes: http://tracker.ceph.com/issues/19736
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

test/librados/tmap_migrate: g_ceph_context->put() upon return

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflict: test/librados/tmap_migrate.cc
  this change is not cherry-picked from master, because tmap_migrate was
removed in master. so we are applying the same change in
cb1cda96713b2ec0f6418c4cbe3d964c2020729c to this test.

Merge pull request #14686 from smithfarm/wip-19686-jewel

jewel: osd: Give requested scrubs a higher priority

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #14605 from asheplyakov/19476-jewel

jewel: rgw: don't return skew time in pre-signed url

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14660 from smithfarm/wip-19478-jewel

jewel: rgw: zonegroupmap set does not work

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14607 from asheplyakov/19607-jewel

jewel: rgw: multisite: fetch_remote_obj() gets wrong version when copying from remote

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14587 from asheplyakov/19617-jewel

jewel: mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Sage Weil <sage@redhat.com>

test/ceph_crypto: do not read ceph.conf in global_init()

ForkDeathTest.MD5 expect an empty output while global_init() complains
if ceph.conf is missing if 0 is passed in as the `flags`. this test
passes if ceph.conf is in current working directory, but jenkins does
not prepare this file for this test.

Fixes: http://tracker.ceph.com/issues/18128
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c72a2271a8012a66d7bbccf5766a73da5bb878d6)

Conflicts:
        src/test/ceph_crypto.cc - jewel does not have
             5af29540675b674c1985ff98b28a783ed124acf6

tests: fix erasure-code premature deallocation of cct

The setup function returns before the run function, the cct variable
must be a data member, not a local variable that gets de-allocated
before run() starts.

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit efa1e54362423d4cfd1541fb8c68237b7b9ebbe3)

rbd-nbd: no need create asok file for unmap and list-mapped commands.

Fixes: http://tracker.ceph.com/issues/17951
Signed-off-by: Pan Liu <liupan1111@gmail.com>
(cherry picked from commit 72352653d585ef89043a4ece371b5c0cb3f6f32a)

rbd-nbd: restart parent process logger after forking

Fixes: http://tracker.ceph.com/issues/18070
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 29baf254d72cc593572b5a6215360ba51e3be198)

crushtool: do not release g_ceph_context at exit

it is but a work around of occasionally timeout.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d305cc51b18cbf4b2757bbacb5d43324461306a9)

common,test: g_ceph_context->put() upon return

prior to this change, global_init() could create a new CephContext
and assign it to g_ceph_context. it's our responsibilty to release
the CephContext explicitly using cct->put() before the application
quits. but sometimes, we fail to do so.

in this change, global_init() will return an intrusive_ptr<CephContext>,
which calls `g_ceph_context->put()` in its dtor. this ensures that
the CephContext is always destroyed before main() returns. so the
log is flushed before _log_exp_length is destroyed.

there are two cases where global_pre_init() is called directly.
- ceph_conf.cc: g_ceph_context->put() will be called by an intrusive_ptr<>
  deleter.
- rgw_main.cc: global_init() is called later on on the success code
  path, so it will be taken care of.

Fixes: http://tracker.ceph.com/issues/17762
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb1cda96713b2ec0f6418c4cbe3d964c2020729c)

Conflicts:
src/ceph_fuse.cc
src/ceph_mgr.cc
src/global/global_init.cc
src/rgw/rgw_main.cc
src/test/compressor/test_compression.cc
src/test/compressor/test_compression_plugin.cc
src/test/compressor/test_compression_plugin_snappy.cc
src/test/compressor/test_compression_plugin_zlib.cc
src/test/compressor/test_compression_snappy.cc
src/test/compressor/test_compression_zlib.cc
src/test/erasure-code/TestErasureCode.cc
src/test/erasure-code/TestErasureCodeExample.cc
src/test/erasure-code/TestErasureCodeIsa.cc
src/test/erasure-code/TestErasureCodeJerasure.cc
src/test/erasure-code/TestErasureCodeLrc.cc
src/test/erasure-code/TestErasureCodePlugin.cc
src/test/erasure-code/TestErasureCodePluginIsa.cc
src/test/erasure-code/TestErasureCodePluginJerasure.cc
src/test/erasure-code/TestErasureCodePluginLrc.cc
src/test/erasure-code/TestErasureCodePluginShec.cc
src/test/erasure-code/TestErasureCodeShec.cc
src/test/erasure-code/TestErasureCodeShec_thread.cc
src/test/fio/fio_ceph_objectstore.cc
src/test/librados/misc.cc
src/test/mon/PGMap.cc
src/test/msgr/test_async_networkstack.cc
src/test/msgr/test_userspace_event.cc
src/test/objectstore/Allocator_test.cc
src/test/objectstore/BitAllocator_test.cc
src/test/objectstore/test_bluefs.cc
src/test/objectstore/test_bluestore_types.cc
src/test/objectstore/test_memstore_clone.cc
src/test/osd/TestPGLog.cc
src/test/rgw/test_http_manager.cc
src/test/rgw/test_rgw_compression.cc
src/test/test_mempool.cc
src/tools/rados/rados.cc

Merge pull request #14665 from smithfarm/wip-19610-jewel

jewel: [librados_test_stub] cls_cxx_map_get_XYZ methods don't return correct value

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14653 from smithfarm/wip-19662-jewel

jewel: rgw_file: fix event expire check, don't expire directories being read

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #14635 from smithfarm/wip-19690-jewel

jewel: doc: Improvements to crushtool manpage

Reviewed-by: Loic Dachary <ldachary@redhat.com>

crushtool: s/exit(EXIT_FAILURE)/return EXIT_FAILURE/

so the destructor(s) can be called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e01b89ed6be6b99fec5c725f4bc5769b42468cac)

Conflicts:
src/tools/crushtool.cc - jewel does not have 17feefbcb3105553b763cb7ce123b20b77b95857

global/signal_handler: reset injected segv after test

~CephContext() => ~TypedSingletonWrapper() => ~MempoolObs =>
unregister_command() => ldout() << "unregister_command" =>
Log::submit_entry() => *(volatile int *)(0) = 0xdead;

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d932c8f2f23263924103a900714db82ee87f6eef)

Conflicts:
        src/log/Log.cc - jewel has "namespace log" instead of "namespace
                         logging" (trivial resolution)

test_cors.cc: fix the mem leak

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c8a3777203482cabf6739a8ba69b127df8697628)

Merge pull request #13608 from smithfarm/wip-19063-jewel

jewel: tests: eliminate race condition in Thrasher constructor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #14195 from cbodley/wip-19353

jewel: rgw: use separate http_manager for read_sync_status

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14066 from asheplyakov/19321-bp-jewel

jewel: rgw: fix break inside of yield in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14064 from asheplyakov/19211-bp-jewel

jewel: rgw: "cluster [WRN] bad locator @X on object @X...." in cluster log

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13842 from smithfarm/wip-19145-jewel

jewel: rgw: a few cases where rgw_obj is incorrectly initialized

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13837 from smithfarm/wip-19048-jewel

jewel: rgw: multisite: some yields in RGWMetaSyncShardCR::full_sync() resume in incremental_sync()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13724 from asheplyakov/18626-bp-jewel

jewel: rgw: Use decoded URI when verifying TempURL

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14643 from smithfarm/wip-revert-14427

Wip revert 14427

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14602 from asheplyakov/19646-jewel

jewel: ceph-disk: enable directory backed OSD at boot time

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14449 from smithfarm/wip-test-doc-oversight

tests: fix oversight in yaml comment

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13606 from smithfarm/wip-19062-jewel

jewel: build/ops: enable build of ceph-resource-agents package on rpm-based os

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #14654 from smithfarm/wip-19461-jewel

jewel: admin ops: fix the quota section

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: Give requested scrub work a higher priority

Once started we now queue scrub work at higher priority than
scheduled scrubs.

Fixes: http://tracker.ceph.com/issues/15789
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ebab8b1f4f67fbdec1e147c580329c1e2b5cf7cd)

Conflicts:
        src/osd/OSD.h - in jewel, the PGScrub() call is enclosed within
            op_wq.queue(make_pair(...)) instead of enqueue_back()

Merge pull request #14680 from smithfarm/wip-19711-jewel

jewel: [test] test_notify.py: rbd.InvalidArgument: error updating features for image test_notify_clone2

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: rbd master/slave notify test should test active features

Fixes: http://tracker.ceph.com/issues/19692
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0dcba41cba96566d0b8da54cf0316d523b88ded2)

Merge pull request #14402 from shinobu-x/wip-17331-jewel

 jewel: ceph-disk: ceph-disk list reports mount error for OSD having mount options with SELinux context

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13865 from smithfarm/wip-19158-jewel

jewel: rgw: health check errors out incorrectly

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Fixes: http://tracker.ceph.com/issues/19597
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9ffd464dac102f684d6dfa78e58d2cb45e165ed6)

radosgw-admin: use zone id when creating a zone

Fixes: http://tracker.ceph.com/issues/19498
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 3fea36d635fcba8ca584a1c0ec9f07840009402c)

qa: rgw task uses period instead of region-map

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e3e3a71d1f1fb43bb4172ce2dfac9a28ca89df0f)

rgw-admin: remove deprecated regionmap commands

Fixes: http://tracker.ceph.com/issues/18725
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5830c1849a0c0110d17c37784808e456e6dcb7b3)

Conflicts:
src/rgw/rgw_admin.cc (trivial resolution)

doc: rgw: correct the quota section

Add the missing option and fix typo.

Fixes: http://tracker.ceph.com/issues/19397
Signed-off-by: Chu, Hua-Rong <hrchu@cht.com.tw>
(cherry picked from commit 51a88267f0d7f51aeb62092949b66b9f6c062e15)

rgw_file:  remove unused rgw_key variable

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 1100a1c26e76485569cfebcf863b18cf908f6161)

rgw_file: fix readdir after dirent-change

Also, fixes link count computation off-by-one, update of state.nlink
after computation, link computation reset at start, and a time print
in debug log.

Fixes: http://tracker.ceph.com/issues/19634
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
link count

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e0f80266ecd424bf9466579b3edc03911a7c5719)

rgw_file: don't expire directories being read

If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 007b7451c26716c51207c161dc347e9a00da53f1)

rgw_file:  rgw_readdir:  return dot-dirs only when *offset is 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 61482c2b85a07519f2256b1a3f2b6d8aa99d5f06)

rgw_file:  chunked readdir

Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e0191d74e3aef06bf300df045a53a3952a71f651)

rgw_file: fix missing unlock in unlink

Fixes: http://tracker.ceph.com/issues/19435
Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
(cherry picked from commit cb6808a6366a70f54d0cc16437d16aa1b7819c84)

rgw_file: implement reliable has-children check (unlink dir)

Bug report and discussion provided by
Gui Hecheng <guihecheng@cmss.chinamobile.com> in nfs-ganesha upstream
github.  Briefly, while a reliable check is potentially costly,
it is necessary.

Fixes: http://tracker.ceph.com/issues/19270
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b05f1c6d61aa4501a971e87de6dcaf3e58c3d9b4)

rgw_file: introduce rgw_lookup type hints

The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2e66c7a7cc763c5c0d6f5db04855f60f2b2ceed3)

Merge pull request #14383 from smithfarm/wip-19547-jewel

jewel: build/ops: rbdmap.service not included in debian packaging (jewel-only)

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #14143 from smithfarm/wip-19355-jewel

jewel: rgw: when converting region_map we need to use rgw_zone_root_pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14136 from smithfarm/wip-19330-jewel

jewel: rgw: upgrade to multisite v2 fails if there is a zone without zone info

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13872 from smithfarm/wip-19163-jewel

jewel: doc: radosgw-admin: add the 'object stat' command to usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13863 from smithfarm/wip-19155-jewel

jewel: rgw: typo in rgw_admin.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13779 from smithfarm/wip-18866-jewel

jewel: rgw: 'radosgw-admin sync status' on master zone of non-master zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Revert "osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag."

This reverts commit 2d68822c784eb4d62d3b0198ed4ec04404dbffb3.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "osdc/Objecter: resend RWORDERED ops on full"

This reverts commit f2474042ecd6560323673170c13f2cb964406e70.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: mention --show-mappings in crushtool manpage

Fixes: http://tracker.ceph.com/issues/19649
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b48b6f4ed8b5f5b5852cbbfd5b3d5b650efb7f1b)

Merge pull request #14596 from gregsfortytwo/wip-17916-divergent

Wip 17916 divergent

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: Thrasher: handle "OSD has the store locked" gracefully

On slower machines (VPS, OVH) it takes time for the OSD to go down.

Fixes: http://tracker.ceph.com/issues/19556
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit a5b19d2d73540b730392f8001c8601f2cecc1b51)

rgw: fix for null version_id in fetch_remote_obj()

commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f fixed the wrong use of
the dest_obj's version, but removed the check for "null" version

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 915370776df5b964c2ee8d9f9329562919eef8d5)

rgw: version id doesn't work in fetch_remote_obj

Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f)

Conflicts:
src/rgw/rgw_rados.cc: trivial: dest_obj.key.instance in master
  versus dest_obj.get_instance() in Jewel

rgw: don't return skew time in pre-signed url

Fixes: http://tracker.ceph.com/issues/18828
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit dd8b348f4aad0124e8a4457117bf3f5f76af7bdb)

ceph-disk: enable directory backed OSD at boot time

https://github.com/ceph/ceph/commit/539385b143feee3905dceaf7a8faaced42f2d3c6
introduced a regression preventing directory backed OSD from starting at
boot time.

For device backed OSD the boot sequence starts with ceph-disk@.service
and proceeds to

    systemctl enable --runtime ceph-osd@.service

where the --runtime ensure ceph-osd@12 is removed when the machine
reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at
boot time.

However directory backed OSD solely rely on the ceph-osd@.service unit
to start at boot time and will therefore fail to boot.

The --runtime flag is selectively set for device backed OSD only.

Fixes: http://tracker.ceph.com/issues/19628
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit f425a127b7487d2093c8c943f0bcdec3d673d601)

Conflicts:
src/ceph-disk/ceph_disk/main.py: trivial: Jewel does not support
   OpenRC and other inits, hence no corresponding 'elif'

pglog: require users set a config option before ignoring divergent_priors

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

osd: pglog: clean up divergent_priors off disk when running; don't assert on startup

Fixes: http://tracker.ceph.com/issues/17916
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #14492 from gregsfortytwo/wip-jewel-snaptrim

Backport snap trimming improvements to Jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'jewel' into wip-jewel-snaptrim

PendingReleaseNotes: discuss snap trim improvements

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

PrimaryLogPG: reimplement osd_snap_trim_sleep within the state machine

Rather than blocking the main op queue, just pause for that amount of
time between state machine cycles.

Also, add osd_snap_trim_sleep to a few of the thrasher yamls.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 2ed7759cfeb03e71f0fbd98fe7c2db2bb741861c)

Conflicts:
src/osd/PrimaryLogPG.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 67336454a4cee66522bc0ca01b2c58b8960f75ec)

Conflicts:
qa/suites/rados/thrash/thrashers/default.yaml
qa/suites/rados/thrash/thrashers/pggrow.yaml
src/osd/OSD.h
src/osd/ReplicatedPG.cc
src/osd/ReplicatedPG.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

rados: check that pool is done trimming before removing it

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 4aebf59d906fa3e03d21bdac182f89fe3cd4c802)
(cherry picked from commit 34398c29b3c57f00d932cf96570f882dce64a82b)

osd/ReplicatedPG: limit the number of concurrently trimming pgs

This patch introduces an AsyncReserver for snap trimming to limit the
number of pgs on any single OSD which can be trimming, as with backfill.
Unlike backfill, we don't take remote reservations on the assumption
that the set of pgs with trimming work to do is already well
distributed, so it doesn't seem worth the implementation overhead to get
reservations from the peers as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 21cc515adfb225ba70f1d80b1b76f0345c214c22)

Conflicts:
src/osd/PrimaryLogPG.cc
src/osd/PrimaryLogPG.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 68ea24396ca6450d4d8042a7c5f51306b7d199fa)
(cherry picked from commit c7176b869898c870b56b1762958652d801af4c4c)

Conflicts: Many. As evidenced by involving two distinct patches
in this one commit, it wasn't a clean backport.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

mon/MonClient: make get_mon_log_message() atomic

* LogClient: move reset_session() into get_mon_log_message() and add a
  "flush" param to the latter. so it can get_mon_log_message()
  atomically. otherwise another call changing the log queue could sneak
  in between reset_session() and get_mon_log_message().
* MonClient: add a "flush" param to do_send() so we can reset the
  LogClient session once we are connected to a monitor.

Fixes: http://tracker.ceph.com/issues/19427
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5215e291da2b527d85e129eda86043490843178e)

Conflicts:
src/mon/MonClient.cc: handle_auth: replaced 'log_client->reset_session();
send_log();' sequence with newly introduced 'send_log(true);' like
the original patch does

Merge pull request #13834 from smithfarm/wip-18969-jewel

jewel: rgw: Change loglevel to 20 for 'System already converted' message

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13833 from smithfarm/wip-18908-jewel

jewel: rgw: the swift container acl does not support field .ref

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13214 from ovh/bp-osd-updateable-throttles-jewel

jewel: osd: allow client throttler to be adjusted on-fly, without restart

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14326 from shinobu-x/wip-15025-jewel

jewel: osd: new added OSD always down when full flag is set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13874 from smithfarm/wip-19171-jewel

jewel: doc: rgw S3 create bucket should not do response in json

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #13492 from shinobu-x/wip-18516-jewel

jewel: build/ops: "osd marked itself down" will not recognised if host runs mon + osd on shutdown/reboot

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13254 from shinobu-x/wip-14609-jewel

jewel: common: radosstriper: protect aio_write API from calls with 0 bytes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13489 from shinobu-x/wip-18955-jewel

jewel: ceph-disk: bluestore --setgroup incorrectly set with user

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14070 from smithfarm/wip-19339-jewel

jewel: tests: dummy suite fails in OpenStack

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14329 from smithfarm/wip-19493-jewel

jewel: ceph-disk: Racing between partition creation & device node creation

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #14427 from smithfarm/wip-19140-jewel

jewel: osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14324 from shinobu-x/wip-19371-jewel

jewel: common: monitor creation with IPv6 public network segfaults

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14112 from shinobu-x/wip-19192-jewel

jewel: tools: brag fails to count "in" mds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14150 from smithfarm/wip-18823-jewel

jewel: tests: run-rbd-unit-tests.sh assert in lockdep_will_lock, TestLibRBD.ObjectMapConsistentSnap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14152 from smithfarm/wip-18893-jewel

jewel: librbd: Incomplete declaration for ContextWQ in librbd/Journal.h

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14154 from smithfarm/wip-18948-jewel

jewel: tests: additional rbd-mirror test stability improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14148 from smithfarm/wip-18778-jewel

jewel: rbd: rbd --pool=x rename y z does not work

Reviewed-by: Jason Dillaman <dillaman@redhat.com>