ceph-volume: retry when 'vagrant up' fails

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2c88e70877dcc2a04a69be4ab9c4834d5ced6198)

ceph-volume: run tests without waiting on ceph repos

This provides a new playbook that installs ceph on all nodes and
generates the ceph config. Then it will rsync ceph-volume from the
control node, or jenkins slave, to all testing vms before completing
the deployment. This means we can run tests on PRs without waiting
for repos to be built for the branch in the PR.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 6d712ef959f7ea5385bc6bf9b9ade60662940369)

Merge pull request #23786 from ceph/backport-mimic-23532

mimic: ceph-volume: expand auto engine for single type devices on filestore

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23784 from ceph/backport-mimic-23489

mimic: ceph-volume tests/functional add mgrs daemons to lvm tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23780 from ceph/backport-mimic-23448

mimic: ceph-volume: `lvm batch` allow extra flags (like dmcrypt) for bluestore

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.zap loop over all vgs from a pv to destroy them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9066792764edcab1d0c2b3178d6f8f9784e30f85)

ceph-volume tests.util ensure dmcrypt_close will skip non-existent paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7a5ee22437a05fc809bffde97c8e3401f7ba564d)

ceph-volume util.encryption if crypt path does not exist, skip it

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 635f8e9e2ecdad5c54a599029bdad74e88fad600)

ceph-volume lvm.zap unmount lvs and close dmcrypt on partitions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 549109c1de2b43f31747828cb5703f9db221dd1a)

ceph-volume tests.functional batch add filestore sceanrios to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ac00e722551e759fc3c1c157ddc89933fb3ac275)

ceph-volume lvm.zap iterate over multiple lvs on a pv

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 21d4f4afcefb6b2510ae67d1ca5e3bf1d510cdfb)

ceph-volume tests.functional add xenial batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfd1bfc2422318edb5a87739b70f93bfc057dc02)

ceph-volume tests.functional add bluestore,filestore playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 26ca6c129a70cd28aa0ed2dae552fb91a251ad21)

ceph-volume tests.functional add centos7 filestore batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a4ff2bff502f1837f696e5f8ebe88f7586d1a83f)

ceph-volume tests.functional add centos7 bluestore batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 544e364e8c3c01633333ddc11c29302965e0616c)

ceph-volume tests.util assert new as_int,as_float helper methods

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d2c181fc70bf20fb17cf41437fb207afc5dd7656)

ceph-volume tests verify new utility for journal sizes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3adf13f868f270293fc307b58786155341470a9d)

ceph-volume util.prepare add a helper to get journal sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4c0b4df1df2db85209c13ff66ae7a67524856c2d)

ceph-volume util.disk add as_int and as_float helpers for Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ff7054116c50cb149de207280b0469bf51655728)

ceph-volume lvm.strategies fix a bug where incorrect template was used

The template wouldn't render the total OSDs and the removed section was
repeated

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f473bce7addbd31c7c976d2f5f1aef82696d4da1)

ceph-volume lvm.strategies use get_journal_size helper for filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 01aebfc6a075bc91d0dc345f9cd953e3a4259015)

ceph-volume lvm.strategies update filestore use of Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f29ec4c6077e9ab36a9eb5ac030b17cd3a47567c)

ceph-volume lvm.strategies update bluestore use of Size

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 75e9729b76594ce1b6504c10e613a91aea82477d)

ceph-volume lvm.strategies create single type filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit af99401cad2c25b02e6ea0df4554d80afbad2b9b)

ceph-volume tests.util verify that Size object don't change state

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5a54ab17602d159bdd29fe82a25f4f026b0fe6c8)

ceph-volume tests.util update to reflect Size object changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7816fa9f90985d03c17fda31093252fc66a48146)

ceph-volume util.disk do not modify self instance of Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 00dba2e83ad80cd41b6b29efa642e2da826c2124)

ceph-volume lvm.strategies use new template location in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f7bca7ba1f74de125a3789133e06c6cf0c083c1)

ceph-volume util create a templates module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 783e58472c55d378773404050d95470d3bb8bbc2)

doc/rados update journal size defaults to 5GB

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 35a48728ae160a347471cf5b461c552b85323a1b)

ceph-volume tests/functional add mgrs daemons to lvm tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2b7a22ef10e38a7bb4d08c9ebe9736e6452b005f)

Merge pull request #23109 from pdvian/wip-24931-mimic

mimic: client: add inst to asok status output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23180 from ukernel/mimic-24856

mimic: mds: handle discontinuous mdsmap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23196 from pdvian/wip-25035-mimic

mimic: mds: print mdsmap processed at low debug level

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23565 from pdvian/wip-26905-mimic

mimic: mds: mark beacons as high priority

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22670 from votdev/mimic_bug_24080

mimic: mgr/dashboard: Prevent RGW API user deletion

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #23111 from smithfarm/wip-18163-mimic

mimic: tools: ceph-detect-init: support SLED

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #23335 from smithfarm/wip-25178-mimic

mimic: rados: not all exceptions accept keyargs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-volume lvm.batch move import statement to the top

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c460f18d0930e3dbe1cfb97f2d399e4a8a2ceed9)

ceph-volume lvm.batch allow extra flags for bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4149d664fa51eb7b2f900370c2bd000b1a5b1edc)

Merge pull request #23777 from ceph/backport-mimic-23075

mimic: ceph-volume batch command

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.api update lvm sizing tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8d4525a59d1d7eb1d571848698c69cace29bfa53)

ceph-volume api.lvm convert and use extents for sizing calculations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 99e034d7eed9508ee016431c040e3fddca9bca69)

ceph-volume lvm.batch update help docs from auto to batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 226a5389ab3379943d3af35e3e1a86481a3980b2)

ceph-volume lvm.main rename auto to batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ae98cc3512d5e66609640066ccbca1a36acfa220)

ceph-volume lvm rename auto to batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d7707650a59855eac75e37ddb4053c62b0dc11e9)

ceph-volume lvm.strategies add stubs for filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fb6a2da7a13cc7edfe496583addbd29c21ecde1a)

ceph-volume lvm.strategies add bluestore with validators

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f13951414e5e4378a0473c9ddef7e641f4e9d3f)

ceph-volume lvm.auto create a strategies module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6043a2255dc8e9b4e3b09a4642a16718fd93370d)

ceph-volume lvm.prepare do not pass the name when creating a vg

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 30d17788a915a2f49856fbd20f61638d42553b74)

ceph-volume api.lvm create_vg should optionally name vgs on the fly

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 01aece2b9ab034e8d4419f262a33764664ad93a2)

ceph-volume lvm.main make auto a subcommand

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0cd5a698b163e46b5282fb50d3599fd9f6a4d2b1)

ceph-volume lvm.auto add --yes prompt to execute strategies

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d38e8d95a42357483689c5280c6c29888d14c58a)

ceph-volume lvm.auto help menu and dynamic strategy loading

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9a426d0666d35a8b47d00752c6930b675f8bafda)

ceph-volume lvm.auto move auto as a sub-command of lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7fa5317e2f911f94c867a344c2c5cc5441a63db1)

ceph-volume tests verify Size() behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 079e4c3f3126a00fc9a3036e3f426c07ecc0659c)

ceph-volume util.disk capture real size when inspecting disks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 165b944c6196c7b760e9ab6b9b2cf9eb6092190c)

ceph-volume util.disk create a Size class for sizing representations and calculations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c13b12c5bed2f7f2133ba2f4c4b29159fc410f53)

ceph-volume main remove auto as a top-level command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b9bc2eb628355a8f6d87a41eae2f7bcf83156a81)

ceph-volume api.lvm extract sizing method into its own utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2e9da026339df13e6a6cfb3b9c785d30e49cb1ff)

Merge pull request #23756 from ceph/backport-mimic-23443

mimic: ceph-volume: `lvm batch` documentation and man page updates

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23225 from pdvian/wip-25037-mimic

mimic: mds: scrub doesn't always return JSON results

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23275 from pdvian/wip-25040-mimic

mimic: mds: dump recent events on respawn

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23309 from pdvian/wip-25042-mimic

mimic: mds: increase debug level for dropped client cap msg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23343 from pdvian/wip-25045-mimic

mimic: mds: health warning for slow metadata IO

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23407 from ukernel/mimic-24820

mimic: mds: avoid using g_conf->get_val<...>(...) in hot path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23435 from pdvian/wip-25222-mimic

mimic: common: check completion condition before waiting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23603 from pdvian/wip-26914-mimic

mimic: client: check for unmounted condition before printing debug output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/man/ceph-volume add batch information

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e00d9bbf53f189ca8d2faff3c673ce61cfeaaa72)

doc/ceph-volume include batch doc in the toc tree

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3bad7fa766e2ae072aec053dc86393c519978eb)

doc/ceph-volume decument batch reporting and bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f965f9db2246df2359f8309719fc18613336b1d2)

Merge pull request #23541 from pdvian/wip-26847-mimic

mimic: rgw: The delete markers generated by object expiration should have owner

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23118 from ivancich/wip-fix-dir-suggest-mimic

mimic: rgw: fail to recover index from crash mimic backport

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23140 from pdvian/wip-24984-mimic

mimic: rgw: radosgw-admin: 'sync error trim' loops until complete

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23172 from pdvian/wip-24986-mimic

mimic: multisite: object metadata operations are skipped by sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23173 from pdvian/wip-25021-mimic

mimic: rgw: add curl_low_speed_limit and curl_low_speed_time config to avoid

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23206 from linuxbox2/mimic-rgw-22790

mimic: rgw: do not ignore EEXIST in RGWPutObj::execute

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23383 from pdvian/wip-25088-mimic

mimic: rgw: change default rgw_thread_pool_size to 512

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23498 from pdvian/wip-26842-mimic

mimic: rgw_file: deep stat handling

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #23226 from liewegas/wip-23220-mimic

mimic: osd/OSDMap: CRUSH_TUNABLES5 added in jewel, not kraken

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23259 from liewegas/wip-purge-new-mimic

mimic: mon: add purge-new

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23324 from cernceph/dvanders_mimic_chown

mimic: rpm: silence osd block chown

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23356 from wido/mimic-25191

mimic: mgr: Ignore daemon if no metadata was returned

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23378 from dzafman/wip-24875-mimic

mimic: osd: do_sparse_read(): Verify checksum earlier so we will try to repair

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23453 from vumrao/wip-vumrao-mimic-25225

mimic: filestore: add pgid in filestore pg dir split log message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23463 from pdvian/wip-26837-mimic

mimic: mgrc: enable disabling stats via mgr_stats_threshold

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23583 from pdvian/wip-26912-mimic

mimic: "balancer execute" only requires read permissions

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23175 from smithfarm/wip-25032-mimic

mimic: spdk: compile with -march=core2 instead of -march=native

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23203 from votdev/mimic_bug_25068

mimic: mgr/dashboard: RGW is not working if an URL prefix is defined

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #23358 from pdvian/wip-25083-mimic

mimic: librbd: deep-copy should not write to objects that cannot exist

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23490 from tchaikov/mimic-26880

mimic: link against libstdc++ statically

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23201 from smithfarm/wip-25073-mimic

mimic: Boost system library is no longer required to compile and link example librados program

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22339 from tchaikov/mimic-20924

mimic: mds,osd,mon,msg: use intrusive_ptr for holding Connection::priv

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23392 from alexandergraul/fix-ceph-disk

mimic: ceph-disk: return a list instead of an iterator

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23458 from badone/wip-OSDHealthMetric-race-mimic

mimic: mgr/MgrClient: Protect daemon_health_metrics

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23163 from smithfarm/wip-25055-mimic

mimic: doc: http://docs.ceph.com/docs/mimic/rados/operations/pg-states/

Reviewed-By: Jan Fajerski <jfajerski@suse.com>

client: check for unmounted condition before printing debug output

If the filesystem was unmounted before we attempt to call ceph_ll_close
on a Fh, then we can end up tripping over the dereferences in the
debug output. Check for unmounted filesystem first, and only print
the debug info if it's still mounted.

This patch only fixes up ceph_ll_close, which is the only call ganesha
makes after calling ceph_abort_conn. We may need to fix other paths in
the future though if that ever changes.

Tracker: http://tracker.ceph.com/issues/25213
Signed-off-by: Jeff Layton <jlayton@kernel.org>
(cherry picked from commit 3c94f23d3b10928615a0f1381eb758006db84b13)

mgr: tweaks to command permissions

Upgrade all selftest ops to rw, downgrade restful
cert list to r.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7602f256c47fd9782e86ba21e6ef2edc8925b87e)

Conflicts:
src/pybind/mgr/selftest/module.py : Removed self-test commads

mgr: fix permissions on `balancer execute`

Fixes: http://tracker.ceph.com/issues/25345
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f63ed1af4f4ed72419c21b1b1e39d1bd373f401d)

mds: mark beacons as high priority

The mons already mark beacon replies as high priority (via default mon message
priority). We should expect that the mons handle our beacons at the same
priority so the MDS doesn't wrongly get marked laggy.

Fixes: http://tracker.ceph.com/issues/26899
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ec3fa7b5658cc61706e17c8c56e0c7da5062aa34)