Merge pull request #25740 from pdvian/wip-37649-mimic

mimic: rgw: feature -- log successful bucket resharding events

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #25899 from ashishkumsingh/wip-37558-mimic

mimic:  rgw: handle S3 version 2 pre-signed urls with meta-data

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #26003 from smithfarm/wip-37948-mimic

mimic: rgw: debug logging for v4 auth does not sanitize encryption keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25607 from joscollin/wip-37684-mimic

mimic: doc/user-management: Remove obsolete reset caps command

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25501 from joscollin/wip-37601-mimic

mimic: doc: broken link on troubleshooting-mon page

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #26113 from alfredodeza/mimic-rm37963

mimic ceph-volume fix TypeError on dmcrypt when using Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests verify behavior of create_dmcrypt_key in python 3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 69613cc5a8b2e111e24b75ad5b69f924c89fd75b)

ceph-volume util.encryption ensure int is used for urandom (py3 compat)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0ab9fa62d6d46464b16f08d3a5bb3da50f5030e7)

Merge pull request #26035 from noahdesu/mimic-pr26015

mimic: cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26029 from alfredodeza/mimic-fix_raw_input

mimic ceph-volume: Adapt code to support Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26013 from alfredodeza/mimic-bz1644847

mimic ceph-volume zap devices associated with an OSD ID and/or OSD FSID

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

cli: dump osd-fsid as part of osd find <id>

Dumps the osd-fsid uuid as part of the `osd find <id>` command.
Currently this uuid is only available as part of `osd dump` but
ceph-ansible has a use case to interrogate a single osd without needing
the entire osdmap dump.

Fixes: http://tracker.ceph.com/issues/37966
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
(cherry picked from commit 4d01b2c96e0ea1122225c30da21bc39c40e15c0e)

ceph-volume: Adapt code to support Python3

- raw_input() has been renamed to input() in Python3
- Changed signature of prompt_bool. Variables that are named like built-ins must be named like xxx_ and not _xxx

Fixes: https://tracker.ceph.com/issues/37470
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit fe25a0ea625e75c598f6d0749e7259eef167fa8e)

ceph-volume tests.functional.batch symlink test_zap to all batch scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3c0058911fd0b3e273a7afb1a9f4f6072942d75)

ceph-volume tests.functional.batch add test_zap yaml to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 25843b762953e53c08e6475d1f9dbfd5102a283f)

ceph-volume tests.functional.batch create a separate test_zap playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b8f0d483a6203de204fcc605ade536749a9f6c8)

ceph-volume lvm.zap update success message for OSD IDs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 73bb17f79a5b67b28ceace9dc7dcc2dde38dad5e)

doc/man/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit acbf7c9b2ad2810621d7ed3987e814cd8c2a2f65)

doc/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f82304f4b5f5ecd865b6a208f894b59cec6386e2)

ceph-volume tests.lvm verify associated lvs by osd id+fsid behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40a414464727375d5c558532aed3b70766ec5b78)

ceph-volume lvm.zap initial take on zapping by OSD ID, FSID

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 59f6cec6dd0687a2528f75a518bd0d4617c27fe0)

Merge pull request #25554 from jan--f/25238-mimic

mimic ceph-volume: introduce class hierachy for strategies

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rgw: sanitize customer encryption keys from log output in v4 auth

Fixes: http://tracker.ceph.com/issues/37847
CVE-2018-16889

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 000797941fd303c3adc24f0089aeee0e902da205)

ceph-volume lvm.strategies remove unused import

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d127ae3e5901c6477ed44f4d50be834040c44b6f)

  Conflicts:
  src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py
          resolved by removing str_to_int import

ceph-volume: filestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 34df12ea6a8686f161b4bdfde92e557743f59cf4)

ceph-volume: bluestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 302334ac6a4375125b6c1e68bf7160f8bc0597c2)

ceph-volume: add strategies.py to for shared code

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b38987ee9652527e4845f2beb6406cbd398dac9b)

Merge pull request #25923 from alfredodeza/mimic-ceph-volume-fix-json

mimic ceph-volume: fix JSON output in `inventory`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: fix JSON output in `inventory`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a3e6f569b4fa0419dff4690a72e9be6fe0a255c1)

Merge pull request #25819 from batrick/mimic-pr24562

doc: removed warning for resolved errata

Reviewed-by: John Wilkins <jowilkin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw: handle S3 version 2 pre-signed urls with meta-data

Add CanonicalizedAmzHeaders to the v2 signing document, as required
in https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html

Also lift qs metadata into x_meta_map in RGWPutObj_ObjStore_S3::get_parms(),
so they are applied to the stored object.

Fixes: https://tracker.ceph.com/issues/23470
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 3b0480ed136cebd3fba2322a77c7d047ef6ad5e3)

rgw: distinguish parse_v4_credentials

The caller and callees of this method are v4-specific, and their
names contain a v4 infix.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit aa8c686da10ce1211a706f3519757045dfe34bed)

Conflicts:
src/rgw/rgw_auth_s3.cc
src/rgw/rgw_auth_s3.h
src/rgw/rgw_rest_s3.cc
  - Included v4 infix in 'parse_credentials()' and removed
    "session_token" param from definition, declaration and call
    of 'parse_v4_credentials()'.

Merge pull request #25868 from dillaman/wip-37842-mimic

mimic: test/pybind/test_rbd: filter out unknown list_children2 keys

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

test/pybind/test_rbd: filter out unknown list_children2 keys

Fixes: http://tracker.ceph.com/issues/37729
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 24cc60335d86ff625e0c5d98cab0c4449a24f296)

Merge pull request #25837 from alfredodeza/mimic-rm37805

mimic ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #25775 from alfredodeza/mimic-rm37442

mimic ceph-volume normalize comma to dot for string to int conversions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad6b924e2bf3b06ec66eb5bc4fea065f5babc512)

ceph-volume tests.util ensure ints and strings with commas can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 91bc3a1479c550be7b48128d5755228a2007e4b0)

ceph-volume test verify strings with a comma can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 818902e568adaefecc6da9afee58b3a804a0e4d0)

ceph-volume util normalize comma with dot for str-to-int conversion

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit aa5323fbcfdc6c21177cb800884712b0252b84a0)

Merge pull request #25777 from alfredodeza/mimic-rm37486

mimic ceph-volume: set permissions right before prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests.functional fix typo when stopping osd.0 in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fba43289a2ff8a94361cc016e7a4dfdc61f324e8)

ceph-volume tests.functional.lvm stop all osds before activating again to test permissions

Otherwise, ``ceph-volume lvm activate --all`` will skip OSDs that are
already running. So the test that verifies that permissions are
corrected will fail because ``activate`` will simply skip them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9be6b15d3316c8eab159bc5d374e95e23d082dde)

ceph-volume lvm.activate ensure filestore osd path is chowned always

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d6c32b5db0627d95c2ffb9428e86271daa4a336)

ceph-volume: include the ceph-facts role when needed during testing

The ceph-ansible stable-3.2 branch does not have the ceph-facts role
but master does. We need to make sure to import this role only when
it's supported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 569b4fb7b3894275a3b9f4ab603e32099301f9aa)

ceph-volume tests.functional set root:root in OSDs in filestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 503396f685f6a93e0c63e97477677b7145f96c26)

ceph-volume tests.functional set root:root in OSDs in bluestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 98631fdeeb6dbe765d3eb80c5c1800946735f2a2)

ceph-volume lvm.activate always chown osd path before osd-prime-dir in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0af31ec24533091aa4991b2343fe80da94777b4d)

removed warning for resolved errata

Signed-off-by: Scoots Hamilton <scoots@redhat.com>
(cherry picked from commit 93528996189f2eaf42bae18db428f3ea18f28b14)

Conflicts:
doc/cephfs/cache-size-limits.rst

Merge pull request #25749 from alfredodeza/mimic-cryptsetup-allow-discards

mimic ceph-volume: enable device discards

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #25551 from jan--f/25203-mimic

mimic ceph-volume: fix Batch object in py3 environments

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PRs after 13.2.3 and before 13.2.4 back into mimic branch

13.2.4

rgw: fix issues with 'enforce bounds' patch

The patch to enforce bounds on max-keys/max-uploads/max-parts had a few
issues that would prevent us from compiling it. Instead of changing the
code provided by the submitter, we're addressing them in a separate
commit to maintain the DCO.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 29bc434a6a81a2e5c5b8cfc4c8d5c82ca5bf538a)
mimic specific fixes:
As the largeish change from master g_conf() isn't in mimic yet, use the g_conf
global structure, also make rgw_op use the value from req_info ceph context as
we do for all the requests

rgw: enforce bounds on max-keys/max-uploads/max-parts

RGW S3 listing operations provided a way for authenticated users to
cause a denial of service against OMAPs holding bucket indices.

Bound the min & max values that a user could pass into the max-X
parameters, to keep the system safe. The default of 1000 is chosen to
match AWS S3 behavior.

Affected operations:
- ListBucket, via max-keys
- ListBucketVersions, via max-keys
- ListBucketMultiPartUploads, via max-uploads
- ListMultipartUploadParts, via max-parts

The Swift bucket listing codepath already enforced a limit, so is
unaffected by this issue.

Prior to this commit, the effective limit is the lower of
osd_max_omap_entries_per_request or osd_max_omap_bytes_per_request.

Backport: luminous, mimic
Fixes: http://tracker.ceph.com/issues/35994
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit d79f68a1e31f4bc917eec1b6bbc8e8446377dc6b)

 Conflicts:
src/common/options.cc:
  Conflicts due to options from master

mon/config-key: limit caps allowed to access the store

Henceforth, we'll require explicit `allow` caps for commands, or for the
config-key service. Blanket caps are no longer allowed for the
config-key service, except for 'allow *'.

(for luminous and mimic, we're also ensuring MonCap's parser is able to
understand forward slashes '/' when parsing prefixes)

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 5fff611041c5afeaf3c8eb09e4de0cc919d69237)

Merge pull request #25291 from smithfarm/wip-37414-mimic

mimic: mgr/balancer: add crush_compat_metrics param to change optimization keys

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25771 from alfredodeza/mimic-ceph-volume-fix-join

mimic ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 38f289b691ed1275bdf7342a4347434ed340c8fc)

Merge pull request #25351 from alfredodeza/mimic-rm37449

mimic ceph-volume  zap: improve zapping to remove all partitions and all LVs, encrypted or not

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24695 from vshankar/wip-35932

mimic : client: retry remount on dcache invalidation failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #25031 from smithfarm/wip-36576-mimic

mimic: cephfs: mgr/status: fix fs status subcommand did not show standby-replay MDS' perf info

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #25198 from pdvian/wip-37285-mimic

mimic: rgw: radosgw-admin: translate reshard status codes (trivial)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement

mimic: qa/ceph-ansible: Move "branch" out of "vars" section

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #25218 from k0ste/mimic_backports

mimic: mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #25227 from pdvian/wip-37288-mimic

mimic: tests: qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25242 from smithfarm/wip-37384-mimic

mimic: test: Start using GNU awk and fix archiving directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #25252 from pdvian/wip-37363-mimic

mimic: librbd: fix missing unblock_writes if shrink is not allowed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25153 from jonsger/wip-24827-mimic

mimic: build/ops: run-make-check.sh ccache tweaks

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25171 from pdvian/wip-36392-mimic

mimic: rpm: Use hardened LDFLAGS

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25293 from smithfarm/wip-37419-mimic

mimic: mgr/balancer: add cmd to list all plans

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25295 from rzarzynski/wip-bug-36248-mimic

mimic: common: fix memory leaks in WeightedPriorityQueue.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25507 from smithfarm/wip-36729-mimic

mimic: osd: Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #25753 from alfredodeza/mimic-better-devid

mimic common/blkdev, ceph-volume: improve get_device_id

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume: Micron SSDs don't include vendor name in ID_SERIAL

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3e4a638c931a5198aca74dc4037d0b03d5e563ba)

ceph-volume: update get_device_id to match in-tree implementation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c967dbf81f85602d2cd044ee5a91ba38cd76bdb9)

ceph-volume: enable device discards

When using SSDs as encrypted OSD device, discards do not pass the
encryption layer. This option activates discard requests.

Signed-off-by: Jonas Jelten <jj@stusta.net>
(cherry picked from commit 333330414730bf57c68efe6f1f13841b0a014dae)

13.2.3

rgw: feature -- log successful bucket resharding events

This change adds a single log entry at level 1 to indicate when a
bucket resharding successfully completes.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 1b9c762b7963cc91bccb79075933434bad7735db)

Conflicts:
src/rgw/rgw_reshard.cc : Resolved in RGWBucketReshard::execute

doc/user-management: Remove obsolete reset caps command

This command has not worked since hammer and makes no sense in an
environment where we do not allow empty caps

Fixes: http://tracker.ceph.com/issues/37663
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit e4e7f3ea6af8316cd46bb74c3f6d6819b99cea53)

ceph-volume: add Batch smoketest

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 936ee680f7633620c8a1127db9630a22326b9e2c)

ceph-volume: fix Batch object in py3 environments

Fixes: https://tracker.ceph.com/issues/37356
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 851c7693daa315b7972702bedf09dd66a37667d8)

osd: combine recovery/scrub/snap sleep timer into one.

One SafeTime on thread. And those events if timer don't take long time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 610d5196c0132739e1ae133db0c7d896063470ad)

Merge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes

mimic: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25285 from smithfarm/wip-37396-mimic

mimic: tests: /usr/bin/ld: cannot find -lradospp in rados mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>n

osd: Add support for osd_delete_sleep to throttle pg deletion

Fixes: http://tracker.ceph.com/issues/36474
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92f1f5859f142cb83cc1d1a367853ac91d6b01d4)

Conflicts:
    src/osd/OSD.cc
- use cct->_conf->get_val instead of cct->_conf.get_val

test: Set any value for osd_delete_sleep to guarantee we are testing even SSD

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f621a1190f45a780b71b916b31c1a01a20ece96)

Merge pull request #25332 from pdvian/wip-36503-mimic

mimic: tests: use timeout for fs asok operations

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc: broken link on troubleshooting-mon page
Fixes: https://tracker.ceph.com/issues/37451
This fix involves adding :ref: labels to the add-or-rm-mons.rst
page. I also added :ref: labels for other headings within
add-or-rm-mons.rst (for future reference).

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
(cherry picked from commit ca05e456969a721631f5e9882c896cab76a4a3dc)

Conflicts:
doc/rados/troubleshooting/troubleshooting-mon.rst
Fixed the :ref: sentence and removed the _replace line.

Merge pull request #25283 from jdurgin/wip-osd-memory-target-mimic

mimic: core: os/bluestore: cache autotuning and memory limit

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25362 from noahdesu/mimic-docs

mimic: docs: backport edit on github changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

The up-set of erasure-coded PGs may include CRUSH_ITEM_NONE,
which as a result causes mis-killing of valid upmap items.

Fixes: https://tracker.ceph.com/issues/37493
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: ningtao <ningtao@sangfor.com.cn>
(cherry picked from commit f043dcc6d8663ed960a354b2441fd3f8dd66c62d)

Conflicts:
        - *maybe_remove_pg_upmaps* input changed, in master
          we now have a passed in *nextmap* parameter

mon: auto clean up stale upmap items

See https://www.spinics.net/lists/ceph-users/msg49518.html

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4181aa86906886423af80e478e45554606fc7cf6)

Conflicts:
        - *nextmap* has been renamed to *tmpmap*
        - add the *have_pg_upmaps* wrapper which is necessary for unittest

osd/OSDMap: add pg-existence sanity check

The reason why __get_pg_pool_size(pg)__ or __get_pg_pool_crush_rule(pg)__ fails is
that the pg does not exist anymore. So it generally makes sense to check __pg_exists(pg)__
before moving further.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 79ef3bea7550559b7ea4e2b68dc407c7eb25defa)

Conflicts:
- *nextmap* has been renamed to *tmpmap*

osd/OSDMap.cc: move sanity checks to the proper place

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3dcd4eaf867369437720a21daf3c344cc4b9844e)

osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone

If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.

The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:

```
src/osd/OSDMap.cc: 4078: FAILED assert(target > 0)
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8a11ed479d7ea95f76170e74123ca2c48e3a3963)

Merge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id

mimic ceph-volume add device_id to inventory listing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

doc/_templates/page.html: add link to report doc bugs

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de30c4d3c5927bb58b49b44b391647e397e1fe4f)

luminous: doc: show edit on github links and version warnings

backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.

this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
 Conflicts:
doc/conf.py

ceph-volume tests patch udevadm so it doesn't actually call udevadm in a test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f440fb96694e38f69e174c01f284ab9a2066bc8)