mgr: add trivial OSDMap wrapper class

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2ef005196ba2eb49c34c32def624938c7a8beb03)

mgr/PyModules: add 'pg_dump' get

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit bfb9286f4212947183c46543d609b664ea13b489)

mgr/PyModules: add 'pg_status' dump

This is summary info, same as what's in 'ceph status'.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 85b5b80906d00e098d4b1af1354c60a357022dd2)

Merge pull request #18674 from ceph/wip-rgw-s3-branch

qa/tests: use ceph-luminous branch for s3tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa: use ceph-luminous branch for s3tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #18412 from kmroz/wip-21659-luminous

luminous: mgr: fix crashable DaemonStateIndex::get calls

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18113 from jcsp/wip-prometheus-port-backport

luminous: ceph-mgr: can not change prometheus port for mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18655 from ceph/wip-yuriw-bp-PR18634-luminous

qa: add "restful" to ceph_mgr_modules in ceph-ansible suite

qa: add "restful" to ceph_mgr_modules in ceph-ansible suite

backport of https://github.com/ceph/ceph/pull/18634

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit caf9ee5c60d999951979d0b67afda8d56e1cd91d)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #17889 from smithfarm/wip-21372-luminous

luminous: core: Improve OSD startup time by only scanning for omap corruption once

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #18004 from linuxbox2/luminous-rgwf-wr

luminous: rgw_file: fix write error when the write offset overlaps.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18438 from theanalyst/wip-21696-luminous

luminous: fix a bug about inconsistent unit of comparison

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #18410 from kmroz/wip-21732-luminous

luminous: qa/suites/rest/basic/tasks/rest_test: whitelisting

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18629 from ceph/wip-yuriw-21978-luminous

qa/suites/upgrade/jewel-x: Changed typo ('hammer' to 'jewel')

Changed typo ('hammer' to 'jewel')

Fixes http://tracker.ceph.com/issues/21978
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #18627 from ceph/backport-18513

ceph-volume lvm zap backport

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: set journal_uuid and journal_device when using a partition

This correctly sets the tags when a partition is used for a filestore
journal.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1a8561d38dd7363dc920ae82ec1343b2b75a1ca2)

docs for ceph-volume lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3e93a31deda96ed6fb23fa19ce3e273de05f7d88)

ceph-volume: add tests for ceph-volume lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3d5e391693fc64747a4774287c968f842294eaa6)

ceph-volume: print success message if zap succeeds

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 6d70762dee0cd9047c291cf9869f666a375e956b)

ceph-volume: terminal.write only takes one argument

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a5454eba241abd8cc1b1a660b12a1aec7c3c16e0)

ceph-volume: the metadata field for the lv path is lv_path not path

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit e3a9113e7025f1e3a3130ec1e2d565f37bf3d2dc)

ceph-volume: update help text for ceph-volume lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit cbc38c4e4a7dcc24b31bcfa6af73eb8cf04f56ad)

ceph-volume: remove lvm metadata when zapping an lv

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 5f57fc87205c2d35da9a1f28c72e233ffb3fe4d9)

ceph-volume: adds Volume.clear_tags

Will remove all tags for the current lv

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3c9401f1618f7fcbea827fe7279e8dfde617d957)

ceph-volume: zap should leave lvs and partitions intact

This will remove filesystems and wipe data from any lv or partition
given to 'ceph-volume lvm zap' but still leave it intact for further
use.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit caba9fb80b358222192f736bd1b0ab31dca25cec)

ceph-volume: adds utilites to wipe the fs and clear data when zapping

These should eventually move to a disk api.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2f64d4a0abd41afbcb9eba6a237642502d9a9135)

ceph-volume: adds tests for api.lvm.remove_lv

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 937b57438147681c2c6e32c6db38d8bea68d4731)

ceph-volume: api.lvm.remove_lv should return True when successful

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7f055aa6deb3904cf0334e214c13a26098b08aa8)

ceph-volume: zap logical volumes

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1d083a2191315ee94301c2f5f102a0906dd05fa8)

ceph-volume: adds a remove_lv command to the lvm api

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit cf98b6971ced0caa29ff4f7ac346df652177fa29)

ceph-volume: stubs out the ceph-volume lvm zap command

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2e64b797ef6ae91623ffba8ae28d3f8ccc7d7b93)

Merge PR #18385 into luminous

* refs/pull/18385/head:
mds: fix race in PurgeQueue::wait_for_recovery()
mds: open purge queue when transitioning out of standby replay
mds: always re-probe mds log when standby replay done

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #18316 into luminous

* refs/pull/18316/head:
mds: prevent trim count from underflowing

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #18300 into luminous

* refs/pull/18300/head:
mds: keep CInode::STATE_QUEUEDEXPORTPIN state when exporting inode

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #17729 into luminous

* refs/pull/17729/head:
ceph.in: validate service glob

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #18299 into luminous

* refs/pull/18299/head:
mds: update client metadata for already open session

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge PR #18298 into luminous

* refs/pull/18298/head:
osdc/ObjectCacher: limit memory usage of BufferHead

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #18085 into luminous

* refs/pull/18085/head:
ceph_volume_client: fix setting caps for IDs

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #18030 into luminous

* refs/pull/18030/head:
qa: relax cap expected value check
mds: improve cap min/max ratio descriptions
mds: fix whitespace
mds: cap client recall to min caps per client
mds: fix conf types
mds: fix whitespace
doc/cephfs: add client min cache and max cache ratio describe
mds: adding tunable features for caps_per_client

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #17921 into luminous

* refs/pull/17921/head:
ceph_volume_client: perform snapshot operations in

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #18593 from ceph/luminous-wip-bz1499840

luminous ceph-volume lvm bluestore support

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #18334 from kmroz/wip-21816-luminous

luminous: rgw: fix bilog entries on multipart complete

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17861 from smithfarm/wip-21441-luminous

luminous: rbd: [cli] mirror getter commands will fail if mirroring has never been enabled

Merge pull request #18416 from kmroz/wip-21855-luminous

luminous: librbd: object map batch update might cause OSD suicide timeout

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #18337 from kmroz/wip-21640-luminous

luminous: rbd-mirror: forced promotion can result in incorrect status

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #18336 from kmroz/wip-21639-luminous

luminous: librbd: snapshots should be created/removed against data pool

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #17860 from smithfarm/wip-21299-luminous

luminous: rbd: [rbd-mirror] asok hook names not updated when image is renamed

Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

ceph-volume lvm.activate remove links on activate before priming

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 634b5caaed7c9188e426727e83a0768bdbc51f0d)

ceph-volume lvm.activate ceph-bluestore-tool needs --cluster too

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fb36087bc3c7e7bb8358cbf8ad84409fa23fac68)

ceph-volume lvm.activate consume cluster_name from lvm metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2b7ff497fe9e0e167ab01bf0d8ce5e26f0da836f)

ceph-volume lvm.prepare persist cluster_name on lvm metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d38ad987e08791be379f6c006c1eb5bf9464415)

ceph-volume util.prepare bluestore db and wal paths are to devices, not osd dir paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fd616acd6ae0f184c1757e7de38fc746d526908c)

ceph-volume lvm.prepare bluestore wal and db links get created with mkfs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 80698d2309a221d2318b11be9b8f8a95b51416ea)

ceph-volume lvm.prepare use wal and db for mkfs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4970ab5c804111ce4b2b57817f1e29d58b18a893)

ceph-volume lvm.prepare ensure tags are set for ceph.type correctly for block and data

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 58dbefd3be7bad51e6c56c50fbfb450e205b3036)

ceph-volume lvm.activate check if lv device is mounted at path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1ead135bef801280e7cec21283c95c7ba440d84b)

ceph-volume lvm.prepare default to using bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 39dd0d1d10aba08c3e8299eb1702260274a756a7)

ceph-volume lvm.create default to using bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3145da6e643dedb2fbed72e7d4d0cddf397c5e3)

ceph-volume lvm.common remove boolean defaults for CLI flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a7d5965ab532a621f402bad55d45dfed61d90cc5)

ceph-volume lvm.prepare update to use create_osd_path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bafb72144666f891773bed95e9a0f757673af2e0)

Merge pull request #18138 from kmroz/wip-21684-luminous

luminous: rgw: stop/join TokenCache revoke thread only if started.

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #17994 from ukernel/luminous-21337

luminous: mds: make sure snap inode's last matches its parent dentry's last

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>

Merge pull request #18431 from theanalyst/wip-21857-luminous

luminous: rgw: We cant't get torrents if  objects are encrypted using SSE-C

Reviewed-By: Casey Bodley <cbodley@redhat.com>

ceph-volume lvm.activate only prime-osd-dir when directory is empty

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9bb6cfd867c2d8e972093d023622f8c2e5b440d4)

ceph-volume lvm.activate safeguard against auto detect objectstore flag not being set

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c9b5f352a8f9e2743e5dca3a0479c8099a59fd12)

ceph-volume util.system use strings for comparison in py3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1b671a8602b036cb8d53bc40eb70198c13143051)

f ceph-volume tests update changes from the help menu in lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3ddcd3c2f19cc848a11a2882f150462ad2f26c65)

ceph-volume lvm.common --journal-size doesn't accept a volume group

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a8282e9a580bcfa74abdd33032e2ce2851b5cf29)

ceph-volume lvm.prepare filestore does not require a volume group

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3653f105927d455b8df143797af0cfe72f8d6834)

ceph-volume lvm.common volume groups are no longer required

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 526c0aa81ac8eed95c2b4664a00ded2118372fff)

doc/ceph-volume activate workflow mention tmpfs in osd dirs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8788fe3721034f390240c01493bc1d27d65a73a5)

doc/ceph-volume update activation to include bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 288a3124793d11d71b540555fd03c2f61a79dd16)

doc/ceph-volume update create to indicate bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 61ce8b4b844fcb64fc95373c5515de902a52fc8c)

Merge pull request #17858 from smithfarm/wip-21448-luminous

luminous: rgw: string_view instance points to expired memory in PrefixableSignatureHelper

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18442 from theanalyst/wip-21637-luminous

luminous: encryption: PutObj response does not include sse-kms headers

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18437 from theanalyst/wip-21698-luminous

luminous: radosgw-admin usage show loops indefinitly

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18434 from theanalyst/wip-21817-luminous

luminous: rgw: zone compression type is not validated

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18440 from theanalyst/wip-21652-luminous

luminous: policy checks missing from Get/SetRequestPayment operations

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18441 from theanalyst/wip-21651-luminous

luminous: rgw: avoid logging keystone revocation failures when no keystone is configured

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18443 from theanalyst/wip-21634-luminous

luminous:  s3:GetBucketLocation bucket policy fails with 403

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18444 from theanalyst/wip-21635-luminous

luminous: s3:GetBucketCORS/s3:PutBucketCORS policy fails with 403

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18430 from theanalyst/wip-21695-luminous

luminous: failed CompleteMultipartUpload request does not release lock

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18445 from theanalyst/wip-21633-luminous

luminous: s3:GetBucketWebsite/PutBucketWebsite fails with 403

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18436 from theanalyst/wip-21789-luminous

luminous: user creation can overwrite existing user even if different uid is given

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18435 from theanalyst/wip-21790-luminous

luminous: RGW: Multipart upload may double the quota

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18432 from theanalyst/wip-21856-luminous

disable dynamic resharding in multisite enviorment

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge pull request #18429 from theanalyst/wip-21792-luminous

luminous: encryption: reject requests that don't provide all expected headers

Reviewed-By: Casey Bodley <cbodley@redhat.com>

doc/ceph-volume update prepare with bluestore workflow

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f1a5627399e2589fe706b158a32a9fb8642ac23)

doc/ceph-volume update filestore to indicate it is not the only/default objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c93603e5764e021305cfeb1c297ee16318126fe1)

doc/ceph-volume elaborate on env vars for interval and tries in activation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1dc24d1a7065b4e23b1bfa0e7c6d263b5ebc6906)

ceph-volume lvm.common update cli flags help for bluestore support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c9693fb3f8386016aba5d7c8f53f5f71f948c88d)

ceph-volume util.disk st_mode is needed for stat()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e3b5663f7968e44ec0cd6db2bfc43acdb233e314)

ceph-volume decorators always log to the file with exceptions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cb3533a72571251d9f1b6171f783246462726857)

ceph-volume tests verify behavior of get_lv_from_argument

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit adabbd462bcb19a6da65211e24dfb2b9b5043ef4)

ceph-volume api.lvm return result of get_lv when using arguments

If the argument is an absolute path it is fine to just return whatever
get_lv finds since it is a "safe" call, it will return a None if nothing
is found

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f209c9d4d50bd7abe0f7f6b9fb29359cb7196718)

ceph-volume lvm.listing support bluestore, not only filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6a5b89a475c2a36022ca09dbb7f6b94cc3b44881)

ceph-volume api.lvm allow full paths to detect lv from arguments

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8d257101b1556ec5ce805622e9b7b6b74f6b000d)

ceph-volume tests add support for bluestore list verification

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4674da9c07f871bf1f25bb782c136fe082095a14)

ceph-volume tests activate with fsid works for bluestore and filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5b9f0853f780db3b0af6b4b7c1ad1d59184069c9)