ceph-volume tests.util verify Disk objects don't change state with divisions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ca526336221ea2ad32e7daf3e5e46b07291a1d8)

ceph-volume util.disk fix an issue where Disk objects would mutate on div operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d1a3c74240e67621e993a304eaf566d4e2789d1)

ceph-volume util.prepare add a helper to get block.db sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860068ec5daca6d7087cf1ce0594a2fe36e91193)

ceph-volume lvm.batch.bluestore add TODOs for custom fast/slow devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0eb0bdfc55d8ceb27cd12bc3a7fb807a7e65c788)

-f  ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d117d66903fa740108c9948b499150b02523fdf4)

ceph-volume lvm.batch fix error reporting, Device objects aren't strings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 267a386cc270081b2cd284bf78bf94b52be113dc)

ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Reworks the bluestore validation and reporting to account for reusable
VGs from fast devices, and adds validation calls to ensure the new way
to calculate this process will work.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 27f47565a0d43ecee03dea4f6addc739e3e78655)

ceph-volume lvm.batch.filestore capture SizeAllocationErrors

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71fcd35c3d44795f92b17e9aedb7ca868b3750d3)

ceph-volume lvm.batch make sure data devices don't have existing LVs on bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d2ea49a4e64820fc3cd95c2a6ca8869309638e00)

Merge pull request #23503 from ukernel/mimic-26834

mimic: mds: calculate load by checking self CPU usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23672 from smithfarm/wip-26982-mimic

mimic: mds: crash when dumping ops in flight

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23700 from joscollin/wip-26984-mimic

mimic: client: requests that do name lookup may be sent to wrong mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23703 from pdvian/wip-26923-mimic

mimic: mds: use fast dispatch to handle MDSBeacon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23704 from pdvian/wip-26929-mimic

mimic: MDSMonitor: note ignored beacons/map changes at higher debug level

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23725 from joscollin/wip-25206-mimic

mimic: CephVolumeClient: delay required after adding data pool to MDSMap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23769 from pdvian/wip-26956-mimic

mimic: qa: cfuse_workunit_kernel_untar_build fails on Ubuntu 18.04

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23792 from pdvian/wip-26988-mimic

mimic: mds: explain delayed client_request due to subtree migration

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23818 from pdvian/wip-26989-mimic

mimic: cephfs-journal-tool: enable purge_queue journal's event commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23878 from smithfarm/wip-24863-mimic

mimic: ceph_volume_client: allow atomic update of RADOS objects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23880 from smithfarm/wip-26978-mimic

mimic: tools: cephfs-data-scan: print the max used ino

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23883 from smithfarm/wip-32086-mimic

mimic: mds: MDBalancer::try_rebalance() may stop prematurely

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #24047 from pdvian/wip-35706-mimic

mimic: mgr/dashboard: Display RGW user/bucket quota max size in human readable form

Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Display RGW user/bucket quota max size in human readable form

Fixes: https://tracker.ceph.com/issues/34315
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 8311e4b6f5db5391b535f7d19c60bbd621978239)

Merge PR #23659 into mimic

* refs/pull/23659/head:
qa: fix mon config section
qa: ignore normal slow IO warnings for cephfs
qa: create common conf for all cephfs suites

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24038 from ceph/wip-s3a-fix-mirror-mimic

mimic: qa/tasks: s3a fix mirror

Merge pull request #23956 from pdvian/wip-32129-mimic

mimic: docs: radosgw: ldap-auth: fixed option name 'rgw_ldap_searchfilter'

Reviewed-by: Nathan Cutler <ncutler@suse.com>

qa/tasks: update mirror link for maven, the original mirror no longer exists

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 13e100259e3658fcfd2123474cb8ea8498ac3fb6)

Merge pull request #23875 from smithfarm/wip-35079-mimic

mimic: mgr/dashboard: RestClient can't handle ProtocolError exceptions

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23966 from alfredodeza/mimic-rm35535

mimic: ceph-volume: batch tests for mixed-type of devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23744 from liewegas/wip-valgrind-distro-mimic

mimic: qa/suites/rados: move valgrind test to singleton-flat

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23951 from smithfarm/wip-pglog-mimic

mimic: osd/PrimaryLogPG: avoid dereferencing invalid complete_to

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23885 from smithfarm/wip-24905-mimic

mimic: build/ops: mimic 13.2.0 doesn't build in Fedora rawhide

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23886 from smithfarm/wip-25119-mimic

mimic: tests: cluster [WRN] 25 slow requests in powercycle

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23888 from smithfarm/wip-26933-mimic

mimic: osd: segv in OSDMap::calc_pg_upmaps from balancer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23893 from pdvian/wip-32079-mimic

mimic: test: Use pids instead of jobspecs which were wrong

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #23874 from smithfarm/wip-25120-mimic

mimic: mgr/dashboard: URL prefix is not working

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: RestClient can't handle ProtocolError exceptions

Fixes: https://tracker.ceph.com/issues/25190
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 446881f8fe26b5af5d535d72f912d16dbd01b500)

mgr/dashboard: Set timeout in RestClient calls

Set a default timeout of 45 seconds to all REST client calls. This can be customized via 'ceph dashboard set-rest-requests-timeout <seconds>'. Currently the REST client is only used by the RGW controller.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 23128391986bf89b29fad14e0f26e9cb6ecf5cc3)

Conflicts:
doc/mgr/dashboard.rst

Merge pull request #23831 from pdvian/wip-27059-mimic

mimic : debian/rules: fix ceph-mgr .pyc files left behind

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

ceph-volume lvm.batch use 'ceph' as the cluster name with filestore

Custom cluster names are currently broken on ceph-volume, should get
addressed with http://tracker.ceph.com/issues/27210 which is out of
scope for these changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1481799a254eceb05258313aafd532e062ea0bc)

ceph-volume tests/functional update filestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a096a016ccd0acc86ce816aba8ce95f6e3b68c11)

ceph-volume tests/functional update bluestore xenial test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89e52dd19792072919bef6ba2f2ab50d7b0ff4ed)

ceph-volume tests/functional update filestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a5ec54207a02c4bb85dc8226890def18da45e2ff)

ceph-volume tests/functional update bluestore centos7 test vars

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2549d5737283b39ecf5b0af98c73ffeb18033c36)

ceph-volume lvm.api use double -f flags when calling pvremove

Fairly destructive, just like everything else when zapping a device.
This is required in the case of double UUIDs detected, something that
surfaced when testing with a loop device to create an nvme (the loop
device ends up with the same UUID as the nvme).

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c1cbc407c095e0bc3e25a8ff1519b4909ec519cc)

ceph-volume tests/functional remove duplicate test playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a008fe1385e4440f510eab97470a5d4e56a69402)

ceph-volume tests/functional setup playbook for nvme devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71db509df0c4a14598010568f8bb078f6b2841ab)

ceph-volume tests/functional single (test) playbook to zap and redeploy batch operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 23330c0107a9f9ee436338c112b1852fd36e34a3)

ceph-volume tests/functional create a noop playbook to allow symlinking always to setup.yml

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860e66083fe769714a10076fa101b94507e47169)

ceph-volume tests/functional add mixed-type scenarios to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0d5b90bb1fbf30c6e33dbf7e10a3d0ba053297ae)

ceph-volume tests/functional add CentOS7 bluestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 799ce4508d9ed166ef6447daa21e0564262fbafe)

ceph-volume tests/functional add CentOS7 filestore mixed-type scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89bbcff659eed8f2cfa26fb93a0c12c7ff83e83b)

Merge pull request #23854 from smithfarm/wip-35078-mimic

mimic: doc: broken bash example in bluestore migration

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23868 from smithfarm/wip-27212-mimic

mimic: rpm: should change ceph-mgr package depency from py-bcrypt to python2-bcrypt

Reviewed-by: Sebastian Wagner <swagner@suse.com>

osd/PrimaryLogPG: avoid dereferencing invalid complete_to

For the auto-repair (EIO caused) case, we will not reinitialize
**complete_to** (because last_complete is equal to last_update!)
and hence there is chance that **complete_to** should aleady
point to **log.end()** before we call recover_got.

We could simply drop it here as we (already) logged the **complete_to**
iterator change in a more compatible way a few lines below.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 69a2cc35840939436da09691ca62476d7f599de4)

docs: radosgw: ldap-auth: fixed option name 'rgw_ldap_searchfilter'

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>
(cherry picked from commit 4e1d3d8772b7f07becc646b7598cefbc0bc71111)

Merge pull request #23860 from smithfarm/wip-25144-mimic

mimic: mon: Automatically set expected_num_objects for new pools with >=100 PGs per OSD

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23869 from smithfarm/wip-27213-mimic

mimic: libradosstriper conditional compile

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23676 from ifed01/wip-ifed-fix-get_fragmentation

mimic: os/bluestore: fix assertion in StupidAllocator::get_fragmentation

Merge pull request #23722 from pdvian/wip-26931-mimic

mimic: scrub livelock

Merge pull request #23861 from smithfarm/wip-25176-mimic

mimic: osd,mon: increase mon_max_pg_per_osd to 300

Merge pull request #23863 from smithfarm/wip-25204-mimic

mimic: rados python bindings use prval from stack

Merge pull request #23870 from smithfarm/wip-32108-mimic

mimic: object errors found in be_select_auth_object() aren't logged the same

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #23872 from smithfarm/wip-34532-mimic

mimic: core: force-create-pg broken

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23873 from smithfarm/wip-35068-mimic

mimic: core: deep scrub cannot find the bitrot if the object is cached

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23215 from b-ranto/wip-mgr-prom-bp-mimic

[mimic] Sync the prometheus module

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #23403 from neha-ojha/wip-24989-mimic

mimic: Limit pg log length during recovery/backfill so that we don't run out of memory

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23748 from pdvian/wip-26944-mimic

mimic: os/bluestore/BlueStore.cc: 1025: FAILED assert(buffer_bytes >= b->length) from ObjectStore/StoreTest.ColSplitTest2/2

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #23772 from ifed01/wip-ifed-perf-reset-mimic

mimic: os/bluestore: bluestore_buffer_hit_bytes perf counter doesn't reset

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23782 from smithfarm/wip-32082-mimic

mimic: mgr balancer does not save optimized plan but latest

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #23865 from smithfarm/wip-26907-mimic

mimic: kv: MergeOperator name() returns string, and caller calls c_str() on the temporary

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: fix mon config section

Fixes: fc6e683832125a1042638f21b623e29d0d4c34d8
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 47e60908a3c07fe262a50a8073dbd8e309e17261)

qa: ignore normal slow IO warnings for cephfs

Generally the slow warnings we get are just over the threshold. These warnings
are related to deploying multiple Ceph daemons side-by-side. Let's see how we
do with two minutes.

Ignoring the warnings entirely is unsatisfactory as they serve as a useful
canary in the coal mine when you see warnings for ops > some unreasonably large
amount of time.

Fixes: http://tracker.ceph.com/issues/26900
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fc6e683832125a1042638f21b623e29d0d4c34d8)

qa: create common conf for all cephfs suites

This will be followed by removing common CephFS configurations in the
ceph.conf.template in teuthology.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 73fa0efcbb15a47157615705043288f2913ff3bc)

Conflicts:
qa/suites/fs/32bits/overrides/debug.yaml
qa/suites/fs/basic_functional/overrides/debug.yaml
qa/suites/fs/basic_workload/overrides/debug.yaml
qa/suites/fs/multiclient/overrides/debug.yaml
qa/suites/fs/multifs/overrides/debug.yaml
qa/suites/fs/permission/overrides/debug.yaml
qa/suites/fs/snaps/overrides/debug.yaml
qa/suites/fs/thrash/overrides/debug.yaml
qa/suites/fs/traceless/overrides/debug.yaml
qa/suites/fs/upgrade/snaps/overrides/debug.yaml
qa/suites/fs/verify/overrides/debug.yaml
qa/suites/kcephfs/cephfs/overrides/debug.yaml
qa/suites/kcephfs/mixed-clients/overrides/debug.yaml
qa/suites/kcephfs/recovery/debug/.qa
qa/suites/kcephfs/recovery/overrides/debug.yaml
qa/suites/kcephfs/thrash/overrides/debug.yaml

All of the above files were deleted by the master commit, but showed up as
conflicts due to slightly different content in mimic. Resolved trivially by
"git rm".

        qa/suites/fs/bugs/conf

This file was deleted because sub-suite does not exist in Mimic. The QA test
for https://github.com/ceph/ceph/pull/22139 was chosen not to be backported.

Merge pull request #23559 from dillaman/wip-25121-mimic

mimic: rbd: improved trash snapshot namespace handling

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #23560 from dillaman/wip-24945-mimic

mimic: librbd: validate data pool for self-managed snapshot support

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #23858 from smithfarm/wip-27060-mimic

mimic: run-rbd-unit-tests.sh test fails to finish in jenkin's make check run

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #23521 from pdvian/wip-26845-mimic

mimic: rgw: Limit the number of lifecycle rules on one bucket.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #23686 from pdvian/wip-26921-mimic

mimic: rgw: Fix log level of gc_iterate_entries

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #23855 from smithfarm/wip-26849-mimic

mimic: rgw: civetweb fails on urls with control characters

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #23771 from pdvian/wip-26946-mimic

mimic: qa/rgw: patch keystone requirements.txt

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23856 from smithfarm/wip-26980-mimic

mimic: multisite: intermittent failures in test_bucket_sync_disable_enable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23857 from smithfarm/wip-35070-mimic

mimic: cls/rgw: add rgw_usage_log_entry type to ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test: Use pids instead of jobspecs which were wrong

Fixes: http://tracker.ceph.com/issues/27056
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit bc3317031057b5737710a095ce593135b62f9302)

TestOSDMap: Test with negative upmap values

Check we can handle negative pg_upmap and pg_upmap_items values.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit f76c889c2cb02539f5c5bdfc7bd5ab8f506bad86)

osd/OSDMap: Disallow negative values in _apply_upmap

A negative pg_upmap or pg_upmap_item value can cause a segfault so
ignore them.

Fixes: http://tracker.ceph.com/issues/22056
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 689e1cfba74e9866d411d5d21e0cf0b28343519c)

qa/tasks/thrashosds-health.yaml: whitelist slow requests

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c0d9f7177c47a2b613fb1a3e9fa56eb90b36973a)

cmake: do not depend on ${DPDK_LIBRARIES} if not using bundled dpdk

${DPDK_LIBRARIES} will be link flags instead of library targets if
libdpdk is not built using the bundled source.

Fixes: http://tracker.ceph.com/issues/24449
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1c41870184a3857d94ccf2c77958e2f0aad8dad8)

mds: fix premature stop of MDBalancer::try_rebalance()

When mds wants to export loads to multiple targets. The amount of single
export can easily be less than (0.2 * target_loads). For example:

6 active mds, mds.0's load is 11, other ranks' loads are 5. target_load
is 6. Export amount for each target is 1.

Fixes: http://tracker.ceph.com/issues/26973
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e68cbec6a647f0252d848620ce6e51b4f3eea1e3)

mds: don't queue straydir for split

straydir can not get splitted (for now at least)

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 959e63c1f88b6ce6b2492cfb98e5875d928558e4)

tools/cephfs: make cephfs-data-scan prints the max used ino

Fixes: http://tracker.ceph.com/issues/26925
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e943ff0b8f22ac346b82d3f45351695aa8944121)

Conflicts:
doc/cephfs/disaster-recovery-experts.rst
- no such file in mimic

cmake: add option WITH_LIBRADOSSTRIPER

otherwise this cmake variable is never defined, hence all tests
requiring libstriper are broken.

this is a regresssion introduced by 5513a90c62 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e17900d552aef1ac96e5d1d60ec3ff179ceb38e9)

tools/rados: include acconfig.h for picking up WITH_LIBRADOSSTRIPER

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9497775cb4426f319987cf2a03ca171281e4e6c7)

qa/ceph-volume: add a test for put_object_versioned()

Test if the version passed to put_object_versioned() is used to
crosscheck.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 8ab6f84d5799cf2f32fb2b08168ff1cfb82f7d15)

ceph-volume-client: allow atomic updates for RADOS objects

put_object_versioned() takes the version of the object and verifies if
the version of the object is the expected one before updating the data
in the object. This verification of version before actually writing
makes put_objcet_version() atomic.

Rest of the changes include adding get_object_and_version() so that
current version of the object can be obtained and modification of
get_object() and put_object() to use get_object_and_version() and
put_object_versioned() respectively.

Fixes: http://tracker.ceph.com/issues/24173
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit ca7253cff6cdac590bb14d0d297c02452bf75bf6)

mgr/dashboard: Fix cherrypy static content URL prefix config

Fixes: https://tracker.ceph.com/issues/25067
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit a36eff1e51928777f9efda62da723902257f9d48)

os/bluestore: fix deep-scrub operation againest disk silent errors

Say a object who has data caches, but in a while later, caches' underlying
physical device has silent disk erros accidentally, then caches and physical
data are not same. In such case, deep-scrub operation still tries to read
caches firstly and won't do crc checksum, then deep-scrub won't find such
data corruptions timely.

Here introduce a new flag 'CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE' which tells
deep-scrub to bypass object caches. Note that we only bypass cache who is in
STATE_CLEAN state. For STATE_WRITING caches, currently they are not written
to physical device, so deep-scrub operation can not read physical device and
can read these dirty caches safely. Once they are in STATE_CLEAN state(or not
added to bluestore cache), next round deep-scurb can check them correctly.

As to above discussions, I refactor BlueStore::BufferSpace::read sightly,
adding a new 'flags' argument, whose value will be 0 or:
     enum {
       BYPASS_CLEAN_CACHE = 0x1,     // bypass clean cache
     };

flags 0: normal read, do not bypass clean or dirty cache
flags BYPASS_CLEAN_CACHE: bypass clean cache, currently only for deep-scrube
                        operation

Test:
   I deliberately corrupt a object with cache, with this patch, deep-scrub
   can find data error very timely.

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@easystack.cn>
(cherry picked from commit a7f1af25dd2ba88a322ed21828f073a277b09d02)

os: Use ceph_assert for asserts

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit c3321f7633670ab84899d85da289109f3b56e886)

Conflicts:
src/os/bluestore/BlueFS.cc
src/os/bluestore/BlueStore.cc
src/os/bluestore/KernelDevice.cc
src/os/bluestore/StupidAllocator.cc
src/os/bluestore/bluestore_types.h
src/os/bluestore/fastbmap_allocator_impl.cc
src/os/bluestore/fastbmap_allocator_impl.h
src/os/filestore/FDCache.h

core: add missing flag name for CEPH_OSD_OP_FLAG_WITH_REFERENCE

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@easystack.cn>
(cherry picked from commit 2c862ce6e92bd28ef9e71fdf6a1f88a6e0173043)

mon/MonCommands: fix 'osd force-create-pg'

Fixes: 7797ed67d2f9140b7eb9f182b06d04233e1e309c
Fixes: http://tracker.ceph.com/issues/26940
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 84843422f402117f10a779b070bd19f30bd3b514)