ceph-volume: add tests for ceph-volume lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3d5e391693fc64747a4774287c968f842294eaa6)

ceph-volume: print success message if zap succeeds

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 6d70762dee0cd9047c291cf9869f666a375e956b)

ceph-volume: terminal.write only takes one argument

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a5454eba241abd8cc1b1a660b12a1aec7c3c16e0)

ceph-volume: the metadata field for the lv path is lv_path not path

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit e3a9113e7025f1e3a3130ec1e2d565f37bf3d2dc)

ceph-volume: update help text for ceph-volume lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit cbc38c4e4a7dcc24b31bcfa6af73eb8cf04f56ad)

ceph-volume: remove lvm metadata when zapping an lv

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 5f57fc87205c2d35da9a1f28c72e233ffb3fe4d9)

ceph-volume: adds Volume.clear_tags

Will remove all tags for the current lv

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3c9401f1618f7fcbea827fe7279e8dfde617d957)

ceph-volume: zap should leave lvs and partitions intact

This will remove filesystems and wipe data from any lv or partition
given to 'ceph-volume lvm zap' but still leave it intact for further
use.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit caba9fb80b358222192f736bd1b0ab31dca25cec)

ceph-volume: adds utilites to wipe the fs and clear data when zapping

These should eventually move to a disk api.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2f64d4a0abd41afbcb9eba6a237642502d9a9135)

ceph-volume: adds tests for api.lvm.remove_lv

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 937b57438147681c2c6e32c6db38d8bea68d4731)

ceph-volume: api.lvm.remove_lv should return True when successful

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7f055aa6deb3904cf0334e214c13a26098b08aa8)

ceph-volume: zap logical volumes

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1d083a2191315ee94301c2f5f102a0906dd05fa8)

ceph-volume: adds a remove_lv command to the lvm api

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit cf98b6971ced0caa29ff4f7ac346df652177fa29)

ceph-volume: stubs out the ceph-volume lvm zap command

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2e64b797ef6ae91623ffba8ae28d3f8ccc7d7b93)

Merge PR #18385 into luminous

* refs/pull/18385/head:
mds: fix race in PurgeQueue::wait_for_recovery()
mds: open purge queue when transitioning out of standby replay
mds: always re-probe mds log when standby replay done

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #18316 into luminous

* refs/pull/18316/head:
mds: prevent trim count from underflowing

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #18300 into luminous

* refs/pull/18300/head:
mds: keep CInode::STATE_QUEUEDEXPORTPIN state when exporting inode

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #17729 into luminous

* refs/pull/17729/head:
ceph.in: validate service glob

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #18299 into luminous

* refs/pull/18299/head:
mds: update client metadata for already open session

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #18298 into luminous

* refs/pull/18298/head:
osdc/ObjectCacher: limit memory usage of BufferHead

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #18085 into luminous

* refs/pull/18085/head:
ceph_volume_client: fix setting caps for IDs

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #18030 into luminous

* refs/pull/18030/head:
qa: relax cap expected value check
mds: improve cap min/max ratio descriptions
mds: fix whitespace
mds: cap client recall to min caps per client
mds: fix conf types
mds: fix whitespace
doc/cephfs: add client min cache and max cache ratio describe
mds: adding tunable features for caps_per_client

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17921 into luminous

* refs/pull/17921/head:
ceph_volume_client: perform snapshot operations in

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #18593 from ceph/luminous-wip-bz1499840

luminous ceph-volume lvm bluestore support

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18334 from kmroz/wip-21816-luminous

luminous: rgw: fix bilog entries on multipart complete

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17861 from smithfarm/wip-21441-luminous

luminous: rbd: [cli] mirror getter commands will fail if mirroring has never been enabled

Merge pull request #18416 from kmroz/wip-21855-luminous

luminous: librbd: object map batch update might cause OSD suicide timeout

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #18337 from kmroz/wip-21640-luminous

luminous: rbd-mirror: forced promotion can result in incorrect status

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #18336 from kmroz/wip-21639-luminous

luminous: librbd: snapshots should be created/removed against data pool

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17860 from smithfarm/wip-21299-luminous

luminous: rbd: [rbd-mirror] asok hook names not updated when image is renamed

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

ceph-volume lvm.activate remove links on activate before priming

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 634b5caaed7c9188e426727e83a0768bdbc51f0d)

ceph-volume lvm.activate ceph-bluestore-tool needs --cluster too

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fb36087bc3c7e7bb8358cbf8ad84409fa23fac68)

ceph-volume lvm.activate consume cluster_name from lvm metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2b7ff497fe9e0e167ab01bf0d8ce5e26f0da836f)

ceph-volume lvm.prepare persist cluster_name on lvm metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d38ad987e08791be379f6c006c1eb5bf9464415)

ceph-volume util.prepare bluestore db and wal paths are to devices, not osd dir paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fd616acd6ae0f184c1757e7de38fc746d526908c)

ceph-volume lvm.prepare bluestore wal and db links get created with mkfs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 80698d2309a221d2318b11be9b8f8a95b51416ea)

ceph-volume lvm.prepare use wal and db for mkfs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4970ab5c804111ce4b2b57817f1e29d58b18a893)

ceph-volume lvm.prepare ensure tags are set for ceph.type correctly for block and data

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 58dbefd3be7bad51e6c56c50fbfb450e205b3036)

ceph-volume lvm.activate check if lv device is mounted at path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1ead135bef801280e7cec21283c95c7ba440d84b)

ceph-volume lvm.prepare default to using bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 39dd0d1d10aba08c3e8299eb1702260274a756a7)

ceph-volume lvm.create default to using bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3145da6e643dedb2fbed72e7d4d0cddf397c5e3)

ceph-volume lvm.common remove boolean defaults for CLI flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a7d5965ab532a621f402bad55d45dfed61d90cc5)

ceph-volume lvm.prepare update to use create_osd_path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bafb72144666f891773bed95e9a0f757673af2e0)

Merge pull request #18138 from kmroz/wip-21684-luminous

luminous: rgw: stop/join TokenCache revoke thread only if started.

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17994 from ukernel/luminous-21337

luminous: mds: make sure snap inode's last matches its parent dentry's last

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>

Merge pull request #18431 from theanalyst/wip-21857-luminous

luminous: rgw: We cant't get torrents if  objects are encrypted using SSE-C

Reviewed-By: Casey Bodley <cbodley@redhat.com>

ceph-volume lvm.activate only prime-osd-dir when directory is empty

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9bb6cfd867c2d8e972093d023622f8c2e5b440d4)

ceph-volume lvm.activate safeguard against auto detect objectstore flag not being set

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c9b5f352a8f9e2743e5dca3a0479c8099a59fd12)

ceph-volume util.system use strings for comparison in py3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1b671a8602b036cb8d53bc40eb70198c13143051)

f ceph-volume tests update changes from the help menu in lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3ddcd3c2f19cc848a11a2882f150462ad2f26c65)

ceph-volume lvm.common --journal-size doesn't accept a volume group

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a8282e9a580bcfa74abdd33032e2ce2851b5cf29)

ceph-volume lvm.prepare filestore does not require a volume group

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3653f105927d455b8df143797af0cfe72f8d6834)

ceph-volume lvm.common volume groups are no longer required

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 526c0aa81ac8eed95c2b4664a00ded2118372fff)

doc/ceph-volume activate workflow mention tmpfs in osd dirs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8788fe3721034f390240c01493bc1d27d65a73a5)

doc/ceph-volume update activation to include bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 288a3124793d11d71b540555fd03c2f61a79dd16)

doc/ceph-volume update create to indicate bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 61ce8b4b844fcb64fc95373c5515de902a52fc8c)

Merge pull request #17858 from smithfarm/wip-21448-luminous

luminous: rgw: string_view instance points to expired memory in PrefixableSignatureHelper

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18442 from theanalyst/wip-21637-luminous

luminous: encryption: PutObj response does not include sse-kms headers

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18437 from theanalyst/wip-21698-luminous

luminous: radosgw-admin usage show loops indefinitly

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18434 from theanalyst/wip-21817-luminous

luminous: rgw: zone compression type is not validated

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18440 from theanalyst/wip-21652-luminous

luminous: policy checks missing from Get/SetRequestPayment operations

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18441 from theanalyst/wip-21651-luminous

luminous: rgw: avoid logging keystone revocation failures when no keystone is configured

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18443 from theanalyst/wip-21634-luminous

luminous:  s3:GetBucketLocation bucket policy fails with 403

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18444 from theanalyst/wip-21635-luminous

luminous: s3:GetBucketCORS/s3:PutBucketCORS policy fails with 403

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18430 from theanalyst/wip-21695-luminous

luminous: failed CompleteMultipartUpload request does not release lock

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18445 from theanalyst/wip-21633-luminous

luminous: s3:GetBucketWebsite/PutBucketWebsite fails with 403

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18436 from theanalyst/wip-21789-luminous

luminous: user creation can overwrite existing user even if different uid is given

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18435 from theanalyst/wip-21790-luminous

luminous: RGW: Multipart upload may double the quota

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18432 from theanalyst/wip-21856-luminous

disable dynamic resharding in multisite enviorment

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18429 from theanalyst/wip-21792-luminous

luminous: encryption: reject requests that don't provide all expected headers

Reviewed-By: Casey Bodley <cbodley@redhat.com>

doc/ceph-volume update prepare with bluestore workflow

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f1a5627399e2589fe706b158a32a9fb8642ac23)

doc/ceph-volume update filestore to indicate it is not the only/default objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c93603e5764e021305cfeb1c297ee16318126fe1)

doc/ceph-volume elaborate on env vars for interval and tries in activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1dc24d1a7065b4e23b1bfa0e7c6d263b5ebc6906)

ceph-volume lvm.common update cli flags help for bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c9693fb3f8386016aba5d7c8f53f5f71f948c88d)

ceph-volume util.disk st_mode is needed for stat()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e3b5663f7968e44ec0cd6db2bfc43acdb233e314)

ceph-volume decorators always log to the file with exceptions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cb3533a72571251d9f1b6171f783246462726857)

ceph-volume tests verify behavior of get_lv_from_argument

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit adabbd462bcb19a6da65211e24dfb2b9b5043ef4)

ceph-volume api.lvm return result of get_lv when using arguments

If the argument is an absolute path it is fine to just return whatever
get_lv finds since it is a "safe" call, it will return a None if nothing
is found

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f209c9d4d50bd7abe0f7f6b9fb29359cb7196718)

ceph-volume lvm.listing support bluestore, not only filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6a5b89a475c2a36022ca09dbb7f6b94cc3b44881)

ceph-volume api.lvm allow full paths to detect lv from arguments

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8d257101b1556ec5ce805622e9b7b6b74f6b000d)

ceph-volume tests add support for bluestore list verification

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4674da9c07f871bf1f25bb782c136fe082095a14)

ceph-volume tests activate with fsid works for bluestore and filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5b9f0853f780db3b0af6b4b7c1ad1d59184069c9)

ceph-volume tests arg validator can accept absolute paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 15586715f506505a57ec5122416541479df74bc8)

ceph-volume tests update changes from the help menu in lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 39bdc2e3f621a32bcbb3dfe64f156bbfd7e4f5ee)

ceph-volume tests update to get_lv from get_journal_lv

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9be1012072f36e21a3f3cef42a42f7f1eb83c64e)

ceph-volume tests.util new path/device mount interface

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d5984bad97347e58318793fc05da84832d323113)

ceph-volume tests update api.lvm tests

The create_lv signature changed to require full size description and
tags need to be an actual dictionary (vs. keyword args)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e8a18637531cde59d73a97d106958489c1d12117)

ceph-volume lvm.activate ceph-bluestore-tool should populate the osd dir

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f3bb616bcf8a4737a3701ebf812de831785d6341)

ceph-volume lvm.activate auto detect objectstore type

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 472aa0b1997a0b1301b699b7ce30dce728aaea0c)

ceph-volume lvm.trigger enable objectstore auto detection for activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f579ef07026564f1a94b2923a2e4640ade2da503)

ceph-volume lvm.activate add bluestore support for activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5696fed09b65044142b34f89492a6d91d3243e14)

ceph-volume util.system create mappings of mounts

This makes it easier for consumers to detect a path, or a device that
might be mounted or might have more than one mount, including tmpfs and
devtmpfs support, which was just not possible before.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f61007cc650f6964ef2ff3ea98a1debf02713198)

ceph-volume util.prepare separate filestore vs. bluestore prepare utils

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 85cdc783008ff0e0566c34aaab7cd5f12db1cbc7)

ceph-volume process allow obfuscating a key or index when logging commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bbcca70d962855d991f0aa8e86a19f1855345668)

ceph-volume util.prepare add other ln helpers for bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 555b664882db2cd247087f907bdf4d426fb652f8)

ceph-volume util.arg_validators allow to pass a group as --data for lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 211c38f591945dee53196068533784c385bae102)

ceph-volume util.disk add utilities for is_device and is_partition

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b23fc72240ced158cf5335f6e815dbfd804dfab)

ceph-volume api.lvm udpate create_lv for bluestore

Require full size notation for LVM (e.g. 50G or 100M) and do not
"translate" tags by pre-fixing them with 'ceph'. That is already how the
rest of the API works, no need to make that translation in one place

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ac0d1555d3d9caeef9d4fc3e46bc11b7e437413b)

Merge pull request #17857 from smithfarm/wip-21446-luminous

luminous: rgw:multisite: Get bucket location which is located in another zonegroup, will return 301 Moved Permanently

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17856 from smithfarm/wip-21444-luminous

luminous: rgw: setxattrs call leads to different mtimes for bucket index and object

Reviewed-By: Casey Bodley <cbodley@redhat.com>