Merge pull request #23130 from ivancich/wip-fix-dir-suggest-lumious

luminous: rgw: fail to recover index from crash luminous backport

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #22932 from smithfarm/wip-24814-luminous

luminous: rgw: REST admin metadata API paging failure bucket & bucket.instance: InvalidArgument

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #22930 from smithfarm/wip-24632-luminous

luminous: rgw performance regression for luminous 12.2.4

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22934 from smithfarm/wip-24810-luminous

luminous: Invalid Access-Control-Request-Request may bypass validate_cors_rule_method

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22984 from pdvian/wip-24808-luminous

luminous: rgw: fix gc may cause a large number of read traffic

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #23000 from pdvian/wip-24830-luminous

luminous: rgw: "radosgw-admin objects expire" always returns ok even if the pro…

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #23019 from pdvian/wip-24833-luminous

luminous: rgw: radogw-admin reshard status command should print text for reshar…

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #23144 from cbodley/wip-25032

luminous: rgw: add curl_low_speed_limit and curl_low_speed_time config to avoid the thread hangs in data sync.

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

Merge pull request #23207 from linuxbox2/luminous-rgw-22790

luminous: rgw: do not ignore EEXIST in RGWPutObj::execute

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22729 from pdvian/wip-24495-luminous

luminous : osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22918 from smithfarm/wip-24804-luminous

luminous: Python bindings use iteritems method which is not Python 3 compatible

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23054 from liewegas/wip-osdmap-warning-luminous

luminous: messages,mon,osd: silence gcc-8 warnings related to memset()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23113 from smithfarm/wip-24824-luminous

luminous: tests: test_ceph_argparse.py broken on py3-only system

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23155 from smithfarm/wip-25033-luminous

luminous: tests: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) in powercycle

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #23202 from smithfarm/wip-25074-luminous

luminous: Boost system library is no longer required to compile and link example librados program

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23313 from cernceph/dvanders_chown

luminous: rpm: silence osd block chown

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22546 from smithfarm/wip-24501-luminous

luminous: osd: eternal stuck PG in 'unfound_recovery'

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22708 from pdvian/wip-24474-luminous

luminous: librados: fix buffer overflow for aio_exec python binding

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22723 from liewegas/wip-hobject-hash-luminous

luminous: common/hobject: improve hobject hash

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #22773 from smithfarm/wip-24714-luminous

luminous: mon: Add option to view IP addresses of clients in output of 'ceph features'

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23014 from batrick/i24190

luminous: client: increase verbosity level for log messages in helper methods

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23169 from ukernel/luminous-24856

luminous: mds: handle discontinuous mdsmap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22885 from smithfarm/wip-24696-luminous

luminous: mds: low wrlock efficiency due to dirfrags traversal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22895 from smithfarm/wip-22386-luminous

luminous: ceph-fuse mount failed because no mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23013 from batrick/i24136

luminous: MDSMonitor: uncommitted state exposed to clients/mdss

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23015 from batrick/i23790

luminous: mds: unset deleted vars in shutdown_pass

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23212 from batrick/i25036

luminous: mds: print mdsmap processed at low debug level

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23213 from batrick/i25039

luminous: mds: dump recent events on respawn

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23214 from batrick/i25041

luminous: mds: increase debug level for dropped client cap msg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22680 from VictorDenisov/backport_23772

luminous: let "ceph status" use base 10 when printing numbers not sizes

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #22686 from pdvian/wip-24471-luminous

luminous: spdk: fix ceph-osd crash when activate SPDK

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22774 from theanalyst/wip-luminous-24717

luminous: qa/workunits/suites/blogbench.sh: use correct dir name

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #22848 from smithfarm/wip-24739-luminous

luminous: Bring back diff -y for non-FreeBSD

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #22860 from smithfarm/wip-24770-luminous

luminous: bluestore: set correctly shard for existed Collection

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #22862 from smithfarm/wip-24772-luminous

luminous: osd: may get empty info at recovery

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22864 from smithfarm/wip-24774-luminous

luminous: Mimic build fails with -DWITH_RADOSGW=0

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>

Merge pull request #22904 from ifed01/wip-ifed-backport-24260

luminous: os/bluestore: fix flush_commit locking

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22909 from ifed01/wip-ifed-fix-23540-luminous

luminous: os/bluestore: fix incomplete faulty range marking when doing compression

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23025 from smithfarm/wip-24864-luminous

luminous: common: Abort in OSDMap::decode() during qa/standalone/erasure-code/test-erasure-eio.sh

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23112 from smithfarm/wip-24979-luminous

luminous: tests: ceph-helpers.sh tries to use dirname without mandatory parameter

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #23374 from ceph/backport-luminous-22764

luminous: ceph-volume: refuse to zap mapper devices

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23395 from alfredodeza/backport-luminous-rm24020

luminous: ceph-volume expand on the LVM API to create multiple LVs at different sizes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

mds: update MDSRank::cluster_degraded before handling mds failure

Migrator checks MDSRank::is_cluster_degraded() to decide if it needs to
send message to other mds. MDSRank::cluster_degraded should get updated
before calling Migrator::handle_mds_failure_or_stop().

Introduced by commit 7de9da4a "mds: handle discontinuous mdsmap"

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 93458c7306c3822175ed3c048f0937af8f02c341)

Merge pull request #23394 from alfredodeza/backport-luminous-rm24152

luminous: ceph-volume enable the ceph-osd during lvm activation

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23397 from alfredodeza/backport-luminous-rm25216

luminous: ceph-volume lvm.activate conditional mon-config on prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

client: increase verbosity level for log messages in helper methods

Fixes: http://tracker.ceph.com/issues/21014
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit e276fdaaa8b737ea845e19388796796ca30d2162)

Conflicts:
src/client/Client.cc

ceph-volume tests patch __release__ to mimic always for stdin keys

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 99c17ecbaf443824ea8e155e06c11b20a8603d76)

ceph-volume lvm.activate conditional mon-config on prime-osd-dir

The --no-mon-config should only be done in Mimic releaes and later

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 020d6b1e5555e4547e8a0a19bd29eea89e1b7a31)

ceph-volume api.lvm fallback to 1 part if none are define when creating lvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 552d987233545f6f33b9220ea4ea98753f96962f)

ceph-volume tests verify create_lvs behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8c487124d795778dbbf461ffb95841b1c8025e08)

ceph-volume api.lvm default to null tags on lvs created

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 490e73eff1b5b7cc98db5d3df292759e1773c0ab)

ceph-deploy api.lvm allow multiple lv creation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e073c56d80d8f37e0f96093f8dd4a95aef81ea02)

ceph-volume api.lvm add a sizing method for VGs to calculate LV creation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e61b9e5c771e167990caad3d85c0bfd27682d643)

ceph-volume exceptions create an error for size allocations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b035bb0f54d4a7777ce242798bdeb18ad0dc28e0)

ceph-volume tests verify vg sizing behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 08832f21e52f28d65a16aadae54a10c8fd16259f)

ceph-volume api.lvm force vgs to display units in 'g'

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit da9f90fcb40e1e82c322f6020253b63da69238da)

ceph-volume: optional systemd enable --runtime

Allow units to be enabled but not persisted across a reboot,
and use this when enabling osds.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 3e6f387be13b209aa9f56f5069331e449615d316)

ceph-volume: enable ceph-osd during lvm activation

Enable the ceph-osd@<id> unit during lvm activate to link these
units to the ceph-osd.target.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: http://tracker.ceph.com/issues/24152
(cherry picked from commit 261d8ac94d0008f57ac194718e9277c126e096b5)

Merge pull request #23367 from alfredodeza/luminous-rm25173

luminous ceph-volume do not use stdin in luminous

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

lvm zap: refuse to zap mapper devices

Fixes: http://tracker.ceph.com/issues/24504
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 8e602d52657d50888a83963148334deb1f8d4e1c)

adds the util.disk.is_mapper_device method

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 852a466fdfa29c91f266ec122dcd5b01b8d1fca5)

Merge pull request #23369 from alfredodeza/luminous-cv-auto

luminous: ceph-volume initial take on auto sub-command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23236 from dzafman/wip-25085-luminous

luminous: Fix  25085 and 24949

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22806 from smithfarm/wip-24697-luminous

luminous: ceph osd safe-to-destroy crashes the mgr

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22753 from pdvian/wip-24498-luminous

luminous: librbd: force 'invalid object map' flag on-disk update

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

ceph-volume tests verify get_devices captures device metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8deb088f101894763d27a6c31a3e8a2e96fc8320)

ceph-volume tests.conftest allow custom dirs in tmpfile fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9d820e82ac6bc0f46890df1cdaa104e88cab2a7e)

ceph-volume tests ensure human_readable_size works up to terabytes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4ad42f4244062995de003d8141cddcdc1d2b80fc)

ceph-volume tests verify mapper and /dev devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9cfd37344c69ebf1f3d8440f295d79ae05b545e9)

ceph-volume tests add tests for get_block_devs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 672c4c4d1c31b4631b1238ed0efb98fef34f2a3d)

ceph-volume tests verify _map_dev_paths mappings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2c4d46955e8d7f52b090cd8b5b5561a2dba443e5)

ceph-volume devices.auto initial take on auto sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e3b04c9677c273f91e0a9de35dcd68c578d124db)

ceph-volume util.disk create a device metadata utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c6559538db28d6b8bc8529e6b2a62e8619c69bd6)

ceph-volume util.disk add dev mapping helpers for path resolution

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c06e63fed8a11e80444e8e7ec2e82373d75aa501)

ceph-volume tests verify get_file_contents utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0177f71caba3fd64de285d6808b5f859c55ed2fe)

ceph-volume util.system capture file contents

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 39792573dabac3b85a41760c965b60746ca1f059)

ceph-volume devices.auto initial take on auto sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5649bd724b5d300c7080052433f9b9d1c6774cff)

ceph-volume main include commented out auto subcommand

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad1b8cae055ee6c822370d91027613680ccd3dd3)

ceph-volume tests ensure is_lv detection on LVs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 47060a1ef4562b29771bdfb5a375a225f18c228c)

ceph-volume api.lvm create helpers to detect an lv from a device path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 68ca7a691e6e333209b67eb5cea8594f37fb93c1)

Merge pull request #22844 from smithfarm/wip-24735-luminous

luminous: order rbdmap.service before remote-fs-pre.target

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #22865 from pdvian/wip-24584-luminous

luminous: osdc: Fix the wrong BufferHead offset

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22767 from pdvian/wip-24514-luminous

luminous: rgw: fix the bug of radowgw-admin zonegroup set requires realm

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #21806 from ivancich/wip-rgw-unordered-luminous

luminous: rgw: ability to list bucket contents in unsorted order for efficiency

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22817 from smithfarm/wip-24690-luminous

luminous: rgw-multisite: endless loop in RGWBucketShardIncrementalSyncCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22820 from smithfarm/wip-24692-luminous

luminous: rgw: index complete miss zones_trace set

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22822 from smithfarm/wip-24693-luminous

luminous: rgw: meta and data notify thread miss stop cr manager

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22846 from smithfarm/wip-24737-luminous

luminous: add unit test for cls bi list command

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #22942 from pdvian/wip-24782-luminous

luminous: rgw: set cr state if aio_read err return in RGWCloneMetaLogCoroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22985 from joscollin/wip-24844-luminous

luminous: rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #23345 into luminous

* refs/pull/23345/head:
Revert "luminous: pybind/mgr/mgr_module: make rados handle available to all modules"

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

ceph-volume tests verify that luminous doesn't use stdin

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a97a63540290c93e05857ac175f3c9c45a4602c6)

ceph-volume util do not use stdin for luminous

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e03be24a4ff9b882d4329059fd1f04963d9ba2c5)

ceph-volume tests verify that mkfs filestore uses default flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fa9a8afe491445f622e95d32b1f822df8c5b3055)

ceph-volume lvm.prepare pass the cephx_secret for mkfs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b25c8f6897bd66a7156c3a59724cd85eb835c5b6)

ceph-volume util.prepare use keyfile from stdin for mkfs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 70b495ac06b1fa382654759764014d5391d67f9a)

Merge pull request #23137 from smithfarm/wip-24068-luminous

luminous: tests: osd sends op_reply out of order

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #23350 from ceph/backport-luminous-23348

luminous: ceph-volume: adds test for `ceph-volume lvm list /dev/sda`

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: adds a unit test for `lvm list /dev/sda`

This test is to prove that the issue from
http://tracker.ceph.com/issues/24957 was fixed
by http://tracker.ceph.com/issues/24784

When running lvm list against a raw device it should handle
gracefully the situation where there are multiple PVs with the
name of the given device.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit ef10886f1e061d4273d3c0763ab0827e7e58e9d7)

ceph-volume: move pvolumes fixture into conftest.py

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 37ed1be08b226a5b3a7a5bb70c4d5d1fd22db5d1)