ceph-volume lvm.zap iterate over multiple lvs on a pv

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 21d4f4afcefb6b2510ae67d1ca5e3bf1d510cdfb)

ceph-volume tests.functional add xenial batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfd1bfc2422318edb5a87739b70f93bfc057dc02)

ceph-volume tests.functional add bluestore,filestore playbooks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 26ca6c129a70cd28aa0ed2dae552fb91a251ad21)

ceph-volume tests.functional add centos7 filestore batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a4ff2bff502f1837f696e5f8ebe88f7586d1a83f)

ceph-volume tests.functional add centos7 bluestore batch tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 544e364e8c3c01633333ddc11c29302965e0616c)

ceph-volume tests.util assert new as_int,as_float helper methods

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d2c181fc70bf20fb17cf41437fb207afc5dd7656)

ceph-volume tests verify new utility for journal sizes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3adf13f868f270293fc307b58786155341470a9d)

ceph-volume util.prepare add a helper to get journal sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4c0b4df1df2db85209c13ff66ae7a67524856c2d)

ceph-volume util.disk add as_int and as_float helpers for Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ff7054116c50cb149de207280b0469bf51655728)

ceph-volume lvm.strategies fix a bug where incorrect template was used

The template wouldn't render the total OSDs and the removed section was
repeated

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f473bce7addbd31c7c976d2f5f1aef82696d4da1)

ceph-volume lvm.strategies use get_journal_size helper for filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 01aebfc6a075bc91d0dc345f9cd953e3a4259015)

ceph-volume lvm.strategies update filestore use of Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f29ec4c6077e9ab36a9eb5ac030b17cd3a47567c)

ceph-volume lvm.strategies update bluestore use of Size

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 75e9729b76594ce1b6504c10e613a91aea82477d)

ceph-volume lvm.strategies create single type filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit af99401cad2c25b02e6ea0df4554d80afbad2b9b)

ceph-volume tests.util verify that Size object don't change state

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5a54ab17602d159bdd29fe82a25f4f026b0fe6c8)

ceph-volume tests.util update to reflect Size object changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7816fa9f90985d03c17fda31093252fc66a48146)

ceph-volume util.disk do not modify self instance of Size objects

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 00dba2e83ad80cd41b6b29efa642e2da826c2124)

ceph-volume lvm.strategies use new template location in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f7bca7ba1f74de125a3789133e06c6cf0c083c1)

ceph-volume util create a templates module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 783e58472c55d378773404050d95470d3bb8bbc2)

doc/rados update journal size defaults to 5GB

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 35a48728ae160a347471cf5b461c552b85323a1b)

Merge pull request #23109 from pdvian/wip-24931-mimic

mimic: client: add inst to asok status output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23180 from ukernel/mimic-24856

mimic: mds: handle discontinuous mdsmap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23196 from pdvian/wip-25035-mimic

mimic: mds: print mdsmap processed at low debug level

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23565 from pdvian/wip-26905-mimic

mimic: mds: mark beacons as high priority

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22670 from votdev/mimic_bug_24080

mimic: mgr/dashboard: Prevent RGW API user deletion

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #23111 from smithfarm/wip-18163-mimic

mimic: tools: ceph-detect-init: support SLED

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #23335 from smithfarm/wip-25178-mimic

mimic: rados: not all exceptions accept keyargs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23777 from ceph/backport-mimic-23075

mimic: ceph-volume batch command

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.api update lvm sizing tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8d4525a59d1d7eb1d571848698c69cace29bfa53)

ceph-volume api.lvm convert and use extents for sizing calculations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 99e034d7eed9508ee016431c040e3fddca9bca69)

ceph-volume lvm.batch update help docs from auto to batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 226a5389ab3379943d3af35e3e1a86481a3980b2)

ceph-volume lvm.main rename auto to batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ae98cc3512d5e66609640066ccbca1a36acfa220)

ceph-volume lvm rename auto to batch

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d7707650a59855eac75e37ddb4053c62b0dc11e9)

ceph-volume lvm.strategies add stubs for filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fb6a2da7a13cc7edfe496583addbd29c21ecde1a)

ceph-volume lvm.strategies add bluestore with validators

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f13951414e5e4378a0473c9ddef7e641f4e9d3f)

ceph-volume lvm.auto create a strategies module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6043a2255dc8e9b4e3b09a4642a16718fd93370d)

ceph-volume lvm.prepare do not pass the name when creating a vg

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 30d17788a915a2f49856fbd20f61638d42553b74)

ceph-volume api.lvm create_vg should optionally name vgs on the fly

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 01aece2b9ab034e8d4419f262a33764664ad93a2)

ceph-volume lvm.main make auto a subcommand

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0cd5a698b163e46b5282fb50d3599fd9f6a4d2b1)

ceph-volume lvm.auto add --yes prompt to execute strategies

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d38e8d95a42357483689c5280c6c29888d14c58a)

ceph-volume lvm.auto help menu and dynamic strategy loading

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9a426d0666d35a8b47d00752c6930b675f8bafda)

ceph-volume lvm.auto move auto as a sub-command of lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7fa5317e2f911f94c867a344c2c5cc5441a63db1)

ceph-volume tests verify Size() behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 079e4c3f3126a00fc9a3036e3f426c07ecc0659c)

ceph-volume util.disk capture real size when inspecting disks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 165b944c6196c7b760e9ab6b9b2cf9eb6092190c)

ceph-volume util.disk create a Size class for sizing representations and calculations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c13b12c5bed2f7f2133ba2f4c4b29159fc410f53)

ceph-volume main remove auto as a top-level command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b9bc2eb628355a8f6d87a41eae2f7bcf83156a81)

ceph-volume api.lvm extract sizing method into its own utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2e9da026339df13e6a6cfb3b9c785d30e49cb1ff)

Merge pull request #23756 from ceph/backport-mimic-23443

mimic: ceph-volume: `lvm batch` documentation and man page updates

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23225 from pdvian/wip-25037-mimic

mimic: mds: scrub doesn't always return JSON results

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23275 from pdvian/wip-25040-mimic

mimic: mds: dump recent events on respawn

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23309 from pdvian/wip-25042-mimic

mimic: mds: increase debug level for dropped client cap msg

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23343 from pdvian/wip-25045-mimic

mimic: mds: health warning for slow metadata IO

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23407 from ukernel/mimic-24820

mimic: mds: avoid using g_conf->get_val<...>(...) in hot path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23435 from pdvian/wip-25222-mimic

mimic: common: check completion condition before waiting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23603 from pdvian/wip-26914-mimic

mimic: client: check for unmounted condition before printing debug output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/man/ceph-volume add batch information

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e00d9bbf53f189ca8d2faff3c673ce61cfeaaa72)

doc/ceph-volume include batch doc in the toc tree

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3bad7fa766e2ae072aec053dc86393c519978eb)

doc/ceph-volume decument batch reporting and bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f965f9db2246df2359f8309719fc18613336b1d2)

Merge pull request #23541 from pdvian/wip-26847-mimic

mimic: rgw: The delete markers generated by object expiration should have owner

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23118 from ivancich/wip-fix-dir-suggest-mimic

mimic: rgw: fail to recover index from crash mimic backport

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23140 from pdvian/wip-24984-mimic

mimic: rgw: radosgw-admin: 'sync error trim' loops until complete

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23172 from pdvian/wip-24986-mimic

mimic: multisite: object metadata operations are skipped by sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23173 from pdvian/wip-25021-mimic

mimic: rgw: add curl_low_speed_limit and curl_low_speed_time config to avoid

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23206 from linuxbox2/mimic-rgw-22790

mimic: rgw: do not ignore EEXIST in RGWPutObj::execute

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23383 from pdvian/wip-25088-mimic

mimic: rgw: change default rgw_thread_pool_size to 512

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23498 from pdvian/wip-26842-mimic

mimic: rgw_file: deep stat handling

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #23226 from liewegas/wip-23220-mimic

mimic: osd/OSDMap: CRUSH_TUNABLES5 added in jewel, not kraken

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23259 from liewegas/wip-purge-new-mimic

mimic: mon: add purge-new

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23324 from cernceph/dvanders_mimic_chown

mimic: rpm: silence osd block chown

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23356 from wido/mimic-25191

mimic: mgr: Ignore daemon if no metadata was returned

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23378 from dzafman/wip-24875-mimic

mimic: osd: do_sparse_read(): Verify checksum earlier so we will try to repair

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23453 from vumrao/wip-vumrao-mimic-25225

mimic: filestore: add pgid in filestore pg dir split log message

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23463 from pdvian/wip-26837-mimic

mimic: mgrc: enable disabling stats via mgr_stats_threshold

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23583 from pdvian/wip-26912-mimic

mimic: "balancer execute" only requires read permissions

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23175 from smithfarm/wip-25032-mimic

mimic: spdk: compile with -march=core2 instead of -march=native

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23203 from votdev/mimic_bug_25068

mimic: mgr/dashboard: RGW is not working if an URL prefix is defined

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #23358 from pdvian/wip-25083-mimic

mimic: librbd: deep-copy should not write to objects that cannot exist

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23490 from tchaikov/mimic-26880

mimic: link against libstdc++ statically

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23201 from smithfarm/wip-25073-mimic

mimic: Boost system library is no longer required to compile and link example librados program

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22339 from tchaikov/mimic-20924

mimic: mds,osd,mon,msg: use intrusive_ptr for holding Connection::priv

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23392 from alexandergraul/fix-ceph-disk

mimic: ceph-disk: return a list instead of an iterator

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23458 from badone/wip-OSDHealthMetric-race-mimic

mimic: mgr/MgrClient: Protect daemon_health_metrics

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23163 from smithfarm/wip-25055-mimic

mimic: doc: http://docs.ceph.com/docs/mimic/rados/operations/pg-states/

Reviewed-By: Jan Fajerski <jfajerski@suse.com>

client: check for unmounted condition before printing debug output

If the filesystem was unmounted before we attempt to call ceph_ll_close
on a Fh, then we can end up tripping over the dereferences in the
debug output. Check for unmounted filesystem first, and only print
the debug info if it's still mounted.

This patch only fixes up ceph_ll_close, which is the only call ganesha
makes after calling ceph_abort_conn. We may need to fix other paths in
the future though if that ever changes.

Tracker: http://tracker.ceph.com/issues/25213
Signed-off-by: Jeff Layton <jlayton@kernel.org>
(cherry picked from commit 3c94f23d3b10928615a0f1381eb758006db84b13)

mgr: tweaks to command permissions

Upgrade all selftest ops to rw, downgrade restful
cert list to r.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7602f256c47fd9782e86ba21e6ef2edc8925b87e)

Conflicts:
src/pybind/mgr/selftest/module.py : Removed self-test commads

mgr: fix permissions on `balancer execute`

Fixes: http://tracker.ceph.com/issues/25345
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f63ed1af4f4ed72419c21b1b1e39d1bd373f401d)

mds: mark beacons as high priority

The mons already mark beacon replies as high priority (via default mon message
priority). We should expect that the mons handle our beacons at the same
priority so the MDS doesn't wrongly get marked laggy.

Fixes: http://tracker.ceph.com/issues/26899
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ec3fa7b5658cc61706e17c8c56e0c7da5062aa34)

Merge pull request #23337 from smithfarm/wip-qa-ceph-detect-init-alt

mimic: qa: fix ceph-disk suite and add coverage for ceph-detect-init

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

rgw: The delete markers generated by object expiration should have owner
attribute.

Fixes: http://tracker.ceph.com/issues/24568
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 63c37dfefbcfe87dca70237560483caa1fc67ef7)

rgw_file: deep stat handling

A change to the exact-match behavior in leaf stat operations is
required to support instantiation of a full path hierarchy when
one or more intervening directories has not been enumerated.

Fixes: http://tracker.ceph.com/issues/24915
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit bca1b099735dd74908ca9b1cf509c7afcbd7e3db)

debian: strip "-Wl,-Bsymbolic-functions" from LDFLAGS

Fixes: http://tracker.ceph.com/issues/25209
Fixes: http://tracker.ceph.com/issues/26880
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2063056f3ada7cf26d3c426385234e3a052193bf)

cmake: fix "WITH_STATIC_LIBSTDCXX"

- do not link libkv with ALLOC_LIBS, it turns out that if we link
tcmalloc *before* -static-libstdc++ -static-libgcc, libstdc++ and gcc
libs will show up in `ldd` output
- add `-static-libstdc++ -static-libgcc` to CMAKE_SHARED_LINKER_FLAGS
and CMAKE_EXE_LINKER_FLAGS instead of adding them to all shared
libraries and executable. simpler this way.
- link against libtcmalloc statically, because libtcmalloc is a C++
library, linking against it dynamically and linking against C++ runtime
statically will pull in depdencies on two versions of C++ runtime, which
will bring down the app at run-time.
- do not pass '-pie' to linker when building executable if
`WITH_STATIC_LIBSTDCXX` and tcmalloc is used, because the static tcmalloc
is not compiled with PIC.
- only apply '-pie' if ENABLE_SHARED is enabled.

Fixes: http://tracker.ceph.com/issues/26880
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d245ffb0e616d3f2dc954047389da69f7142c915)

mgrc: enable disabling stats via mgr_stats_threshold

Because we had a min_max setting with CRIT the maximum,
it wasn't possible to actually turn off stats entirely.

Fixes: http://tracker.ceph.com/issues/25197
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 911fe5ce4fb5d0a7282dad87c2bfe6528c4f2fcf)

mimic: mgr/MgrClient: Protect daemon_health_metrics

Without holding the lock update_daemon_health() can race with
send_report() corrupting the daemon_health_metrics vector.

Fixes: http://tracker.ceph.com/issues/23352
Signed-off-by: Kjetil Joergensen <kjetil@medallia.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 4996506a6b4ab309110039ea29a075f14d09a379)

mimic: filestore: add pgid in filestore pg dir split log message

Fixes: http://tracker.ceph.com/issues/25225
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit d62ea5cc05857bc1a1f9cf4a0e3d4df315bef8af)
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

common: check completion condition before waiting

Fixes: http://tracker.ceph.com/issues/25007
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7090ea24dd5d5538585a690e7cee268f32328c26)

Merge pull request #23400 from alfredodeza/backport-mimic-rm25216

mimic: ceph-volume lvm.activate conditional mon-config on prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23393 from alfredodeza/backport-mimic-rm24152

mimic: ceph-volume enable the ceph-osd during lvm activation

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

mds: avoid using g_conf->get_val<...>(...) in hot path

g_conf->get_val<>()'s overhead is high

Fixes: http://tracker.ceph.com/issues/24820
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d249581df98455ddc6d0ce21faa2db87b21fad85)

 Conflicts:
src/common/legacy_config_opts.h
src/mds/MDBalancer.cc
src/mds/MDBalancer.h
src/mds/MDCache.h

mds: update MDSRank::cluster_degraded before handling mds failure

Migrator checks MDSRank::is_cluster_degraded() to decide if it needs to
send message to other mds. MDSRank::cluster_degraded should get updated
before calling Migrator::handle_mds_failure_or_stop().

Introduced by commit 7de9da4a "mds: handle discontinuous mdsmap"

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 93458c7306c3822175ed3c048f0937af8f02c341)