install-deps: nuke wheelhouse if it's stale

otherwise, if we update *requirements.txt, and the wheelhouse fails to
offer the newly added dependency, the setup-virtualenv.sh will fail the
build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ed0ca70707439f44779ca7093461182d73db1a77)

install-deps.sh: Fixes for RHEL 7

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit a1a3e7960cf2c792ac8c9c71e68caa0ebada8933)

Conflicts:
install-deps.sh

Merge pull request #29242 from smithfarm/wip-38277-mimic

mimic: core: mon,osd: limit MOSDMap messages by size as well as map count

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29249 from smithfarm/wip-38852-mimic

mimic: core: .mgrstat failed to decode mgrstat state; luminous dev version?

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29247 from smithfarm/wip-38437-mimic

mimic: core: crc cache should be invalidated when posting preallocated rx buffers

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #29593 from pdvian/wip-40884-mimic

mimic: mon/MgrMonitor: fix null deref when invalid formatter is specified

Merge pull request #29732 from pdvian/wip-40949-mimic

mimic: Better default value for osd_snap_trim_sleep

Merge pull request #29741 from smithfarm/wip-41287-mimic

mimic: core: ceph daemon mon.a config set mon_health_to_clog false cause leader mon assert

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29976 from dzafman/wip-38450

mimic: osd: make project_pg_history handle concurrent osdmap publish

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30081 from pdvian/wip-41595-mimic

mimic: ceph-objectstore-tool can't remove head with bad snapset

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30180 from smithfarm/wip-40083-mimic

mimic: osd: Better error message when OSD count is less than osd_pool_default_size

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30181 from smithfarm/wip-40732-mimic

mimic: mon/AuthMonitor: fix initial creation of rotating keys

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30182 from smithfarm/wip-41291-mimic

mimic: os/filestore: assure sufficient leaves in pre-split

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30183 from smithfarm/wip-41351-mimic

mimic: cmake,run-make-check.sh: disable SPDK by default

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30214 from smithfarm/wip-41490-mimic

mimic: osd/OSDCap: Check for empty namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30222 from smithfarm/wip-41502-mimic

mimic: osd/PeeringState: do not complain about past_intervals constrained by oldest epoch

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30243 from smithfarm/wip-41442-mimic

mimic: core: osd: clear PG_STATE_CLEAN when repair object

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30391 from pdvian/wip-41805-mimic

mimic: mgr: do not reset reported if a new metric is not collected

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29253 from smithfarm/wip-40464-mimic

mimic: core: osd beacon sometimes has empty pg list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29245 from smithfarm/wip-38351-mimic

mimic: tests: Limit loops waiting for force-backfill/force-recovery to happen

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29250 from smithfarm/wip-38856-mimic

mimic: core: should set EPOLLET flag on del_event()

Merge pull request #29251 from smithfarm/wip-40179-mimic

mimic: tests: qa/standalone/scrub/osd-scrub-snaps.sh sometimes fails

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29765 from smithfarm/wip-40130-mimic

mimic: doc: Document behaviour of fsync-after-close

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #30687 from smithfarm/wip-42104-mimic

mimic: build/ops: admin/build-doc: use python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

admin/build-doc: use python3 (follow-on fix)

Unfortunately, a5127318cf2feb12b5fd61aeed35b48bc31b4e34 (first attempt to
cherry-pick ace8cb15a600daf5261c0ab349b43010a238a1d9) was incomplete.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(partial manual cherry-pick of ace8cb15a600daf5261c0ab349b43010a238a1d9)

doc: fix urls in posix.rst

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit c5c5f031f77f778af73f8659b01695b88f158807)

doc: cephfs: add section on fsync error reporting to posix.rst

Fixes: http://tracker.ceph.com/issues/24641
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit b52d85c2a71950db222e3649d5265e5891368ba1)

Merge pull request #30663 from smithfarm/wip-fix-docs-build-check-mimic

mimic: build/ops: admin/build-doc: use python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30131 from smithfarm/wip-40736-mimic

mimic: doc/rgw: document use of 'realm pull' instead of 'period pull'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30099 from smithfarm/wip-40673-mimic

mimic: docs: fix rgw_ldap_dnattr username token

Reviewed-by: Casey Bodley <cbodley@redhat.com>

admin/build-doc: use python3

to address https://github.com/sphinx-doc/sphinx/issues/3620, we need to
use sphinx with its fix at
https://github.com/sphinx-doc/sphinx/commit/e049f86b2de1cfdf8a74c88dc9593d047c85d5cb
in other words, we need to use sphinx v2.0.0 and up. but sphinx 2.0
requires python >= 3.5, so we have to use python3 for building the
documents.

in this change:

* doc-requirements.txt: install python3 packages on debian derivatives
* build-doc: install python3.6 packages from EPEL7, and use python3
  venv for using sphinx2
* doc-requirements.txt: bump up all python packages to latest
  stable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ace8cb15a600daf5261c0ab349b43010a238a1d9)

Conflicts:
admin/doc-requirements.txt

Merge pull request #30070 from smithfarm/wip-40139-mimic

mimic: doc: describe metadata_heap cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29039 from dcasier/wip-40100-mimic

mimic: ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #29257 from smithfarm/wip-39602-mimic

mimic: doc: rgw: CreateBucketConfiguration for s3 PUT Bucket request

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30555 from alfredodeza/mimic-guits-41392

mimic ceph-volume: do not fail when trying to remove crypt mapper

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #30096 from jan--f/wip-41612-mimic

mimic: ceph-volume: minor optimizations related to class Volumes's use

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: do not fail when trying to remove crypt mapper

In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.

Closes: https://tracker.ceph.com/issues/41392
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 80d3e4ca054b4d1334c2fb99fede46ab54ca4305)

Merge pull request #30303 from alfredodeza/mimic-ceph-volume-zap-fix

mimic ceph-volume lvm.zap fix cleanup for db partitions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #30334 from jan--f/wip-disjoint-device-lists-mimic

mimic: ceph-volume: ensure device lists are disjoint

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #30333 from jan--f/wip-41762-mimic

mimic: ceph-volume: implement __format__ in Size to format sizes in py3

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr: do not reset reported if a new metric is not collected

Fixes: https://tracker.ceph.com/issues/41741
Signed-off-by: Ilsoo Byun <ilsoobyun@linecorp.com>
(cherry picked from commit 02cc60f6935a5005aa461da183c6c4332503be83)

Merge pull request #30295 from alfredodeza/mimic-rm41378-2

mimic ceph-volume: pre-install python-apt and its variants before test runs

Merge pull request #30301 from alfredodeza/mimic-rm41660

mimic ceph-volume: fix stderr failure to decode/encode when redirected

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #30306 from alfredodeza/mimic-rm40664

mimic ceph-volume  zap always skips block.db, leaves them around

ceph-volume: don't keep device lists as sets

This was introduced by #27754. The explicit device lists were cast to
sets but other parts of the code where not updated accordingly. To avoid
touching all code places, only cast to sets for disjoint test and keep
lists otherwise.

Fixes: https://tracker.ceph.com/issues/41292
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0534cf188a671096d5ddb9d48cdae3dccc6c0b18)

ceph-volume: batch ensure device lists are disjoint

Fixes: https://tracker.ceph.com/issues/41018
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8fcdb4d628f3a55e4783f0a3df7ac7c0a36eaa43)

ceph-volume: implement __format__ in Size to format sizes in py3

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7a741f201a078dfd9bce34e66ed16bcb37d99f7f)

ceph-volume tests: verify that wipefs tries several times

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 845e62cb4ba2636325304db2c2d65a185ace25cf)

ceph-volume lvm.zap: retry wipefs several times to prevent race condition failures

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b7b8b106fa485d3b8345b583fbc7f74e30cbc03e)

ceph-volume lvm.zap fix cleanup for db partitions

this uses the correct type 'db' for db type partitions, else
a block.db parition does not get cleaned up by ceph-volume zap

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
(cherry picked from commit 140af8a7eee8d230ac9d94ba100320a20c137a12)

ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 79196354aca3b3d30c7de1e2ac8b5a3c06ee8330)

ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 595e492783283359b613d199a75f3e7495bfb9ed)

ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ee18ebc078b2d380b88f3726737184f34345e2f3)

ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe66e01f6a4b19bb15695537713a629a951b28ce)

ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfb8422e262dcce49c50610a3977bfe1edc2a435)

ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 95b16b516c0736839fe50f12ecbeb7cd62c67335)

ceph-volume create a new tox.ini for shell-based tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4612ab3ae62a3e8e1cc9e51489feab0c938296d3)

qa/standalone/ceph-helpers: resurrect all OSD before waiting for health

address the regression introduced by e62cfceb
in e62cfceb, we wanted to test the newly introduced TOO_FEW_OSDS
warning, so we increased the number of OSD to the size of pool, so if
the number of OSD is less than pool size, monitor will send a warning
message.

but we need to bring all OSDs back if we are expecting a healthy
cluster. in this change, all OSDs are resurrect before
`wait_for_health_ok`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cdba0f14201a327e824b6ec48e014233281dee2d)

qa/standalone/ceph-helpers: fix health_ok test

Stopping the osd daemon won't reliably get you HEALTH_WARN or ERR; you have
to make sure it is also marked down.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dcdca44aa45e7dc5aab56da26caec644e32024a6)

ceph-volume tests pre-instrall python-apt to prevent auto-install failing later

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40cac7b9510f7904c2f9dfb1294d809e04bdcf7e)

Merge pull request #30188 from smithfarm/wip-41332-mimic

mimic: build/ops: rpm: always build ceph-test package

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30227 from smithfarm/wip-41530-mimic

mimic: doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30244 from smithfarm/wip-41701-mimic

mimic: build/ops: install-deps.sh: install `python*-devel` for python*rpm-macros

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: install `python*-devel` for python*rpm-macros

in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.

but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.

it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".

as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.

once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0b2be2b6710695867197ee84e411bdf0a843a06a)

Conflicts:
install-deps.sh
- VERSION_ID in mimic, MAJOR_VERSION in master

mon/PGMap: enable/disable TOO_FEW_OSDS warning with an option

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3b74fbc93d32d6cc367049d3bcbe83d3837abba5)

Conflicts
        src/mon/PGMap.cc
- cct->_conf->get_val in mimic

qa/standalone: remove osd_pool_default_size in test_wait_for_health_ok

Signed-off-by: zjh <jhzeng93@foxmail.com>
(cherry picked from commit e62cfceb9549e806f774ef3da549efd9f4d90171)

osd: Better error message when OSD count is less than osd_pool_default_size

Fixes: http://tracker.ceph.com/issues/38617
Signed-off-by: zjh <jhzeng93@foxmail.com>
(cherry picked from commit 94237d36932b27961afffbe65d433af26c3fe513)

Conflicts:
doc/rados/operations/health-checks.rst
- trivial
        src/mon/PGMap.cc
- cct->_conf->get_val in mimic

osd: clear PG_STATE_CLEAN when repair object

there is a race be found, when we repair object on clean state,
we queue a DoRecovery peering event, but before the peering event
dequeue,a snaptrim event on the missing object's snap dequeue,
then we will get pass the context< SnapTrimmer >().can_trim()
and go to get the context of the missing object(snapdir)

we can avoid this by clear clean state when we found missing..

Fixes: https://tracker.ceph.com/issues/41348
Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
(cherry picked from commit 521f095c6505bbee7570fb3c01b32436bdbf65a4)

Conflicts:
src/osd/PrimaryLogPG.cc
- assert() instead of ceph_assert(), and Feature PR
  https://github.com/ceph/ceph/pull/26942 ("Improvements to auto repair") is
  not being backported

osd/PeeringState: do not complain about past_intervals constrained by oldest epoch

The start of the required interval has a floor set by the oldest osdmap
epoch we have.  That can lead to an invalid/empty required interval
(because the start is >= the end), but the PG may still have past
intervals.  That can be cause by a slow PG deletion.

No need to complain about this harmless condition.

Fixes: https://tracker.ceph.com/issues/39546
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b1396774a40b7b3e54c9f52b6bb6f03d31c35401)

Conflicts:
src/osd/PeeringState.cc
- file does not exist in mimic; made the changes manually in src/osd/PG.cc
- mimic has a different way of getting the oldest osdmap

doc: default values for mon_health_to_clog_* were flipped

On a freshly installed nautilus cluster (i.e. 14.2.2), the default values are:

mon_health_to_clog_interval = 3600
mon_health_to_clog_tick_interval = 60.000000

Fixes: https://tracker.ceph.com/issues/41403
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
(cherry picked from commit 820ada1e58c3713b7de8a4ce669185d26860b84b)

osd/OSDCap: Check for empty namespace

Fixes: http://tracker.ceph.com/issues/40835
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 002ced8a5f25cfbecf5f34e147a0c863ecbf77c7)

rpm: always build ceph-test package

Fixes: https://tracker.ceph.com/issues/41296
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b9c744301051aa637f948be074be7b53c26cb504)

Merge pull request #30128 from smithfarm/wip-41645-mimic

mimic: build/ops: install-deps.sh: Remove CR repo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

cmake,run-make-check.sh,deb,rpm: disable SPDK by default

but we still enable it in `run-make-check.sh`

* cmake: disable SPDK by default
* run-make-check.sh: enable WITH_SPDK so at least we can ensure it
  builds
* deb,rpm: add uuid-dev / libuuid-devel as a "make check" dependency

Fixes: https://tracker.ceph.com/issues/41330
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3234fb3982483d43a23df902628178566128b7b4)

Conflicts:
        CMakeLists.txt
ceph.spec.in
debian/control
run-make-check.sh
- disable SPDK for mimic, even in run-make-check.sh, since the feature is not
  being used in production

filestore: assure sufficient leaves in pre-split

Fixes: http://tracker.ceph.com/issues/39390
Signed-off-by: Jeegn Chen <jeegnchen@tencent.com>
(cherry picked from commit 4e530a14b43b4a4767152e72bc5398bf4d532967)

mon/AuthMonitor: clear_secrets() in create_initial()

If we are creating the initial state and initial proposal, start with an
empty keyring.  Specifically, we want to clear out any rotating secrets
from a previously failed paxos round so that the subsequent call to
check_rotate() will correctly populate the initial proposal with new
rotating keys.  (When we don't do this, the leader OSD will have the
keys from an earlier round in memory but no other mons will.)

Fixes: http://tracker.ceph.com/issues/40634
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a346713516ed6d6935ad6894ffeac3bd41ac99a1)

auth/cephx/CephxKeyServer: make clear_secrets() clear rotating secrets too

Clear the rotating secrets.  And also reset the versions.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e1600b145277da0ed545aac2e106036ba4f1686c)

doc/rgw: document use of 'realm pull' instead of 'period pull'

'radosgw-admin period pull' fetches a period configuration, but does not
update the realm's current_period to use it. the 'realm pull' command
does both, and the difference is especially important in the failover
case

Fixes: http://tracker.ceph.com/issues/39655
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f0059614b8e7055e4f1d6dfdefe362e2e81815ef)

install-deps.sh: Remove CR repo

Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.

Fixes: https://tracker.ceph.com/issues/37335
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit f39d9b7d1f4f2b63233802211a9ed0e641446ca1)

docs: fix rgw ldap username token

Signed-off-by: Thomas Kriechbaumer <thomas@kriechbaumer.name>
(cherry picked from commit 009f404983558ac084aa6772954ac918db19d890)

ceph-volume: test number of times LVs list was created

Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 20d6264538dab444c0ef13f40c6c21ec3d2057f8)

ceph-volume: reuse list of LVs

Avoid creating of list of LVs in different methods of the same module
and resue them.

Fixes: https://tracker.ceph.com/issues/37490
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit ab3a58e78124a984d162b2f63992d01c5e59e2a7)

test: ceph-objectstore-tool add remove --force with bad snapset test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 87d80eb41774a95e6ff854e110233d36728b2817)

Conflicts:
qa/standalone/special/ceph_objectstore_tool.py : Resolved in test_removeall

ceph-objectstore-tool: Ignore snapset error if just removing head with --force

Fixes: https://tracker.ceph.com/issues/41522
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2e5fcf7b46d1f58fbb4e96e73bce543c90f86c65)

doc: describe metadata_heap cleanup

Fixes: http://tracker.ceph.com/issues/18174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit e76604224c0e74bbb3350743910d263c6591fd26)

osd: make project_pg_history handle concurrent osdmap publish

The class's osdmap may be updated while we are in our loop.  Pass it in
explicitly instead.

Fixes: http://tracker.ceph.com/issues/26970
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cfe6ca82ed800e81e286396b5b7783ca299325f0)

Correction to this commit required because of separating it from very large pull request.
Needed to include part of 4bc01379bbf946d2f5963dcca6b071914117ce4a which
changed endmap from const OSDMap& to const OSDMapRef&.

osdc/Objecter: invalidate crcs on preallocated rx buffers

Both simple and async messengers use c_str() when copying the data from
the socket into the receive buffer, going behind bufferlist's back.  If
the receive buffer is preallocated, we need to invalidate its crc cache
by hand to avoid possible data crc mismatches on the client side.

Fixes: https://tracker.ceph.com/issues/38416
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 3eeab975b604d9fc4290747f85a78d59a2452c2e)

Conflicts:
src/test/librados/io_cxx.cc
- In master, 3730d10623650ce8569be96b28cbba599a9a0db6 renamed this file from
  src/test/librados/io.cc but that commit is not being backported to mimic.
  Manually cherry-picked the test mods into src/test/librados/io.cc.

Merge pull request #29900 from alfredodeza/mimic-rm41378

mimic ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89231c9a60a46ba0691f787efbfe5ab35f05c679)

Merge pull request #29770 from alfredodeza/mimic-bz-1738379

mimic ceph-volume: use the OSD identifier when reporting success

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume devices.lvm zap use the identifier to report success

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d0d5750c1ba508152c1d5111f2003a039a27145f)

ceph-volume tests check success message when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88c7429a60fdab6222d14132bea93b9bf773c720)

Merge pull request #29661 from jan--f/wip-41246-mimic

mimic: simple functional tests drop test for lvm zap

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #29688 from jan--f/wip-41298-mimic

mimic: batch functional idempotency test fails since message is now on stderr

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mon: take the mon lock in handle_conf_change

Let the finisher thread to do the job, it can hold the mon lock
and not hold the config::lock, so can avoid dead lock with mon::tick thread.

Fixes: http://tracker.ceph.com/issues/39625
Signed-off-by: huangjun <huangjun@xsky.com>
(cherry picked from commit bf57e20476cf0266840db0edd495368e43f0abb8)

common/options.cc, doc: osd_snap_trim_sleep overrides other variants

A value > 0 for osd_snap_trim_sleep, will override the backend specific
variants of osd_snap_trim_sleep.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 733df09fe5111e7beca75f8be0afb8669ef9a625)

doc/rados/configuration/osd-config-ref.rst: document snap trim sleep

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit accf95e9dea257c3aaa64b7a36d077468d7c86ec)

osd: add hdd, ssd and hybrid variants for osd_snap_trim_sleep

This is better than the earlier default, which was set to 0.

Fixes: https://tracker.ceph.com/issues/40528
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 560fca12e695a817e1b7e46d365838ed871b64bd)

Conflicts:
src/osd/PrimaryLogPG.h : Resolved for osd_snap_trim_sleep
src/osd/OSD.cc : Resolved in get_osd_snap_trim_sleep

Merge pull request #29618 from pdvian/wip-40943-mimic

mimic: mon/OSDMonitor.cc: better error message about min_size

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29699 from pdvian/wip-41086-mimic

mimic: Change default for bluestore_fsck_on_mount_deep as false

Reviewed-by: Neha Ojha <nojha@redhat.com>