ceph-volume: fix JSON output in `inventory`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a3e6f569b4fa0419dff4690a72e9be6fe0a255c1)

Merge pull request #25819 from batrick/mimic-pr24562

doc: removed warning for resolved errata

Reviewed-by: John Wilkins <jowilkin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25868 from dillaman/wip-37842-mimic

mimic: test/pybind/test_rbd: filter out unknown list_children2 keys

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

test/pybind/test_rbd: filter out unknown list_children2 keys

Fixes: http://tracker.ceph.com/issues/37729
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 24cc60335d86ff625e0c5d98cab0c4449a24f296)

Merge pull request #25837 from alfredodeza/mimic-rm37805

mimic ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #25775 from alfredodeza/mimic-rm37442

mimic ceph-volume normalize comma to dot for string to int conversions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad6b924e2bf3b06ec66eb5bc4fea065f5babc512)

ceph-volume tests.util ensure ints and strings with commas can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 91bc3a1479c550be7b48128d5755228a2007e4b0)

ceph-volume test verify strings with a comma can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 818902e568adaefecc6da9afee58b3a804a0e4d0)

ceph-volume util normalize comma with dot for str-to-int conversion

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit aa5323fbcfdc6c21177cb800884712b0252b84a0)

Merge pull request #25777 from alfredodeza/mimic-rm37486

mimic ceph-volume: set permissions right before prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests.functional fix typo when stopping osd.0 in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fba43289a2ff8a94361cc016e7a4dfdc61f324e8)

ceph-volume tests.functional.lvm stop all osds before activating again to test permissions

Otherwise, ``ceph-volume lvm activate --all`` will skip OSDs that are
already running. So the test that verifies that permissions are
corrected will fail because ``activate`` will simply skip them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9be6b15d3316c8eab159bc5d374e95e23d082dde)

ceph-volume lvm.activate ensure filestore osd path is chowned always

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d6c32b5db0627d95c2ffb9428e86271daa4a336)

ceph-volume: include the ceph-facts role when needed during testing

The ceph-ansible stable-3.2 branch does not have the ceph-facts role
but master does. We need to make sure to import this role only when
it's supported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 569b4fb7b3894275a3b9f4ab603e32099301f9aa)

ceph-volume tests.functional set root:root in OSDs in filestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 503396f685f6a93e0c63e97477677b7145f96c26)

ceph-volume tests.functional set root:root in OSDs in bluestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 98631fdeeb6dbe765d3eb80c5c1800946735f2a2)

ceph-volume lvm.activate always chown osd path before osd-prime-dir in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0af31ec24533091aa4991b2343fe80da94777b4d)

removed warning for resolved errata

Signed-off-by: Scoots Hamilton <scoots@redhat.com>
(cherry picked from commit 93528996189f2eaf42bae18db428f3ea18f28b14)

Conflicts:
doc/cephfs/cache-size-limits.rst

Merge pull request #25749 from alfredodeza/mimic-cryptsetup-allow-discards

mimic ceph-volume: enable device discards

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #25551 from jan--f/25203-mimic

mimic ceph-volume: fix Batch object in py3 environments

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge PRs after 13.2.3 and before 13.2.4 back into mimic branch

13.2.4

rgw: fix issues with 'enforce bounds' patch

The patch to enforce bounds on max-keys/max-uploads/max-parts had a few
issues that would prevent us from compiling it. Instead of changing the
code provided by the submitter, we're addressing them in a separate
commit to maintain the DCO.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 29bc434a6a81a2e5c5b8cfc4c8d5c82ca5bf538a)
mimic specific fixes:
As the largeish change from master g_conf() isn't in mimic yet, use the g_conf
global structure, also make rgw_op use the value from req_info ceph context as
we do for all the requests

rgw: enforce bounds on max-keys/max-uploads/max-parts

RGW S3 listing operations provided a way for authenticated users to
cause a denial of service against OMAPs holding bucket indices.

Bound the min & max values that a user could pass into the max-X
parameters, to keep the system safe. The default of 1000 is chosen to
match AWS S3 behavior.

Affected operations:
- ListBucket, via max-keys
- ListBucketVersions, via max-keys
- ListBucketMultiPartUploads, via max-uploads
- ListMultipartUploadParts, via max-parts

The Swift bucket listing codepath already enforced a limit, so is
unaffected by this issue.

Prior to this commit, the effective limit is the lower of
osd_max_omap_entries_per_request or osd_max_omap_bytes_per_request.

Backport: luminous, mimic
Fixes: http://tracker.ceph.com/issues/35994
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit d79f68a1e31f4bc917eec1b6bbc8e8446377dc6b)

 Conflicts:
src/common/options.cc:
  Conflicts due to options from master

mon/config-key: limit caps allowed to access the store

Henceforth, we'll require explicit `allow` caps for commands, or for the
config-key service. Blanket caps are no longer allowed for the
config-key service, except for 'allow *'.

(for luminous and mimic, we're also ensuring MonCap's parser is able to
understand forward slashes '/' when parsing prefixes)

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 5fff611041c5afeaf3c8eb09e4de0cc919d69237)

Merge pull request #25291 from smithfarm/wip-37414-mimic

mimic: mgr/balancer: add crush_compat_metrics param to change optimization keys

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25771 from alfredodeza/mimic-ceph-volume-fix-join

mimic ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 38f289b691ed1275bdf7342a4347434ed340c8fc)

Merge pull request #25351 from alfredodeza/mimic-rm37449

mimic ceph-volume  zap: improve zapping to remove all partitions and all LVs, encrypted or not

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24695 from vshankar/wip-35932

mimic : client: retry remount on dcache invalidation failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #25031 from smithfarm/wip-36576-mimic

mimic: cephfs: mgr/status: fix fs status subcommand did not show standby-replay MDS' perf info

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #25198 from pdvian/wip-37285-mimic

mimic: rgw: radosgw-admin: translate reshard status codes (trivial)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement

mimic: qa/ceph-ansible: Move "branch" out of "vars" section

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #25218 from k0ste/mimic_backports

mimic: mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #25227 from pdvian/wip-37288-mimic

mimic: tests: qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25242 from smithfarm/wip-37384-mimic

mimic: test: Start using GNU awk and fix archiving directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #25252 from pdvian/wip-37363-mimic

mimic: librbd: fix missing unblock_writes if shrink is not allowed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25153 from jonsger/wip-24827-mimic

mimic: build/ops: run-make-check.sh ccache tweaks

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25171 from pdvian/wip-36392-mimic

mimic: rpm: Use hardened LDFLAGS

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #25293 from smithfarm/wip-37419-mimic

mimic: mgr/balancer: add cmd to list all plans

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25295 from rzarzynski/wip-bug-36248-mimic

mimic: common: fix memory leaks in WeightedPriorityQueue.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25507 from smithfarm/wip-36729-mimic

mimic: osd: Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #25753 from alfredodeza/mimic-better-devid

mimic common/blkdev, ceph-volume: improve get_device_id

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume: Micron SSDs don't include vendor name in ID_SERIAL

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3e4a638c931a5198aca74dc4037d0b03d5e563ba)

ceph-volume: update get_device_id to match in-tree implementation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c967dbf81f85602d2cd044ee5a91ba38cd76bdb9)

ceph-volume: enable device discards

When using SSDs as encrypted OSD device, discards do not pass the
encryption layer. This option activates discard requests.

Signed-off-by: Jonas Jelten <jj@stusta.net>
(cherry picked from commit 333330414730bf57c68efe6f1f13841b0a014dae)

13.2.3

ceph-volume: add Batch smoketest

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 936ee680f7633620c8a1127db9630a22326b9e2c)

ceph-volume: fix Batch object in py3 environments

Fixes: https://tracker.ceph.com/issues/37356
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 851c7693daa315b7972702bedf09dd66a37667d8)

osd: combine recovery/scrub/snap sleep timer into one.

One SafeTime on thread. And those events if timer don't take long time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 610d5196c0132739e1ae133db0c7d896063470ad)

Merge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes

mimic: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25285 from smithfarm/wip-37396-mimic

mimic: tests: /usr/bin/ld: cannot find -lradospp in rados mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>n

osd: Add support for osd_delete_sleep to throttle pg deletion

Fixes: http://tracker.ceph.com/issues/36474
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92f1f5859f142cb83cc1d1a367853ac91d6b01d4)

Conflicts:
    src/osd/OSD.cc
- use cct->_conf->get_val instead of cct->_conf.get_val

test: Set any value for osd_delete_sleep to guarantee we are testing even SSD

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f621a1190f45a780b71b916b31c1a01a20ece96)

Merge pull request #25332 from pdvian/wip-36503-mimic

mimic: tests: use timeout for fs asok operations

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #25283 from jdurgin/wip-osd-memory-target-mimic

mimic: core: os/bluestore: cache autotuning and memory limit

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #25362 from noahdesu/mimic-docs

mimic: docs: backport edit on github changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

The up-set of erasure-coded PGs may include CRUSH_ITEM_NONE,
which as a result causes mis-killing of valid upmap items.

Fixes: https://tracker.ceph.com/issues/37493
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: ningtao <ningtao@sangfor.com.cn>
(cherry picked from commit f043dcc6d8663ed960a354b2441fd3f8dd66c62d)

Conflicts:
        - *maybe_remove_pg_upmaps* input changed, in master
          we now have a passed in *nextmap* parameter

mon: auto clean up stale upmap items

See https://www.spinics.net/lists/ceph-users/msg49518.html

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4181aa86906886423af80e478e45554606fc7cf6)

Conflicts:
        - *nextmap* has been renamed to *tmpmap*
        - add the *have_pg_upmaps* wrapper which is necessary for unittest

osd/OSDMap: add pg-existence sanity check

The reason why __get_pg_pool_size(pg)__ or __get_pg_pool_crush_rule(pg)__ fails is
that the pg does not exist anymore. So it generally makes sense to check __pg_exists(pg)__
before moving further.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 79ef3bea7550559b7ea4e2b68dc407c7eb25defa)

Conflicts:
- *nextmap* has been renamed to *tmpmap*

osd/OSDMap.cc: move sanity checks to the proper place

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3dcd4eaf867369437720a21daf3c344cc4b9844e)

osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone

If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.

The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:

```
src/osd/OSDMap.cc: 4078: FAILED assert(target > 0)
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8a11ed479d7ea95f76170e74123ca2c48e3a3963)

Merge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id

mimic ceph-volume add device_id to inventory listing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

doc/_templates/page.html: add link to report doc bugs

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de30c4d3c5927bb58b49b44b391647e397e1fe4f)

luminous: doc: show edit on github links and version warnings

backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.

this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
 Conflicts:
doc/conf.py

ceph-volume tests patch udevadm so it doesn't actually call udevadm in a test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f440fb96694e38f69e174c01f284ab9a2066bc8)

ceph-volume util.disk use device.path for error reporting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1670585ebed7c685b49a13c1066eedc1aa37d29b)

ceph-volume tests.functional add back sdc1 partitions with parted to redeploy after zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b0bc162b601c2dbc9827df0ab472ba2a1146076c)

ceph-volume tests.functional add back partitions with parted to redeploy after zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 945e8f56ce21411f664a58147afb343f0671b591)

ceph-volume tests.functional recreate partitions previously removed with --destroy

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e1d136ee3bcb2a7bbe0790b87e1d54a39b522e01)

ceph-volume tests.util ensure that both disk and device report is_device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit df8ba476d68a62f6e0449aa27c81f3effa1e0536)

ceph-volume tests.util ensure is_encrypted works for lv, partition, and mapper devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ff0166f9f445d229216113623f18244af188b57f)

ceph-volume tests.util ensure partitions are listed as top-level keys

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6dc01772245e557b73551111f7c48cc60aec0893)

ceph-volume tests add mapper test for dm paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ebf7d7631be362e6a26af19225bfc07ec48138d)

ceph-volume tests update is_mapper unit test when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 830cd7d638b70e97723eb41f0d33f6a60e3ece32)

ceph-volume util.device check both dm and mapper paths for is_mapper property

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ba83c1f37e6ba6fa695ff97a028479e88e2b762c)

ceph-volume lvm skip empty vg_name removals

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 14c74ede9a83975a3464c219cab0be3c06238ff9)

ceph-volume util.arg_validators optionally skip gpt header check

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 58be4d0e18c4b2f3560361345f91c5a1c7b6c5c4)

ceph-volume util.device if no vgs, set it to an empty list

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 822ca670eef57e009e356fc58d6732cdeecdecdc)

ceph-volume util.device set vgs to Device object if available

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 91c01e88ecc8f94f78b30463f13a51affe405d31)

ceph-volume util update is_device when reported as device or disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f74635d16d7ceefb8e0de02ded85a6e7be53457f)

ceph-volume util add method to detect encryption on devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1ee7435aef2f6475256e8482194d55e16b77eee8)

ceph-volume util.disk add a helper to remove partitions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 870f9e5b48a5fe13e213be86ee8947ca7207c3bf)

ceph-volume util.disk include partitions as top level devices in get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7f3c3598491e556ec1b93ea0a54f52144202fab6)

ceph-volume util.disk fix issue when capturing partition information

Before, the `if` condition meant that it would only work when the output
was '1', which is incorrect as that would only happen if a partition was
the first one, ignoring any other partition. The contents of that file
is the partition number, not a boolean to tell if it is a partition or
not.

It now includes the `holders` file contents which is needed for
dm-mapper work

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 28cb13a970909dc8efe508aee18f47e0fb3fdde3)

ceph-volume lvm.zap split zapping in methods per device type

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0d2a9a40d2c301039d571101d1e92a82058a51e0)

ceph-volume: add test for disk.udevadm_property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 58316e3f826f8bb91b375589aa3676b664ce3984)

ceph-volume: add device_id field to inventory output

This intends to mimic the C++ implementation in src/common/blkdev.cc.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 41da5e9a4334628db09e3166eaceff79b9b6c7e0)

qa: use timeout for fs asok operations

Otherwise a bug preventing an asok operation from completing will cause the
entire job to fail.

Fixes: http://tracker.ceph.com/issues/36335
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 296d89b0463ee3aad8dfb50b0e41a27e2c852ddb)

Merge pull request #25294 from smithfarm/wip-37422-mimic

mimic: build/ops: Can't compile Ceph on Fedora 29 as it doesn't recognize python*3*-tox as an install Tox

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #25040 from smithfarm/wip-36749-mimic

mimic: mgr: [restful] deep_scrub is not a valid OSD command

Reviewed-by: Boris Ranto <branto@redhat.com>

client: remove redundant abort logic during dentry invalidation test

commit d1471f070c added retry logic when remounting is
used to invalidate kernel dcache. ceph-fuse performs a
dcache invalidation test during startup, thereby logging
(and continuing) or aborting as per certain configuration
settings. the abort logic is pretty much redundant and
can be cleaned up preserving the existing behavior.

Also, assert is none of dcache invalidation mecahnisms
(`dentry_invalidate_cb` or `remount_cb`) are available.

Fixes: http://tracker.ceph.com/issues/35932
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 232d12f03612983336a2e90cb024deb79e5e6934)

 Conflicts:
src/client/Client.cc

client: retry remount on dcache invalidation failure

For some (unknown) reason, there have been reports of
ceph-fuse crash due to failure is remounting at the
time of invalidating kernel dentry cache. This issue
is not also reproducible yet.

Therefore, as suggested by Patrick and Zheng, for a
temporary workaround, client would ignore the failure
as the invalidation would be retried again. There is
a max cap on the number of consecutive remount failures
after which client would abort.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d1471f070cd1ad9c0f773e00d2552161d1ad5955)

 Conflicts:
src/client/Client.cc
src/client/Client.h
src/common/options.cc

common: fix memory leaks in WeightedPriorityQueue.

Fixes: http://tracker.ceph.com/issues/36248
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 61395d44010f0ac1c183d030b244ae1ec0c8b99a)

build/ops: rpm: stop install-deps.sh clobbering spec file Python build setting

Fedora 29 still ships a Python 2 binary, but some of Ceph's build
dependencies are only available in py3 versions there. In other
words, from F29 on, it is no longer possible to do a py2 Ceph build
on Fedora, even if a python2 binary exists on the system.

If that were not enough, the Python 2 that ships with Fedora 29 is
linked against a non-compatible version of OpenSSL.

Before this commit, install-deps.sh was overriding the spec file's
Python build setting based on the presence or absence of a python2
binary. As the bug cited below indicates, this was not a good idea.

It's better for the spec file to be explicit about which OS versions
are py2 and which are py3, and just stick to that.

Fixes: http://tracker.ceph.com/issues/37301
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7d4916618c419e4c85823cf259b96e49d9203035)

Conflicts:
ceph.spec.in
install-deps.sh
- no WITH_SEASTAR in mimic

build/ops: refrain from installing/using lsb_release in install-deps.sh

Unfortunately the mapping between release number and codename (which is only
relevant for Debian and Ubuntu btw) is not available from /etc/os-release.
In that one respect, lsb_release was "better".

However, when I weigh the advantages of obtaining that mapping from an external
tool, with the (substantial) risk that the external dependency might cause
trouble on one or more supported distros (to say nothing of the non- or
semi-/pseudo-supported ones), against the work involved in maintaining a
hard-coded mapping (negligible), the needle on my scale immediately swings
toward eliminating the dependency.

Also, I see this commit as part of the longer-term effort to completely expunge
lsb_release from our codebase. See git log --grep lsb_release.

For another example of an external distro-detection tool (albeit one that was
included in Python 2) gone awry, see http://tracker.ceph.com/issues/18163.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 38508b41dc7f33a0b35e3633be3da1d2dae844dc)

mgr/balancer: add cmd to list all plans

Signed-off-by: Yang Honggang <yanghonggang@umcloud.com>
(cherry picked from commit 99245c69829c53dab6790da5502420c7183c6bcd)

mgr/balancer: add crush_compat_metrics param

Add a crush_compat_metrics options to allow choosing between pgs,
objects, and bytes when rebalancing with crush-compat mode.

Multi-values are supported, so that the placement scoring can be
a function of several metrics. By default we score based on the
mean of pgs, objects, and bytes scores for all roots.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 1f6866c15307a2a681d0e5c0293797da7fa5d1de)

Merge pull request #25000 from MrStupnikov/mimic-docfix-1358

mimic: doc: Put command template into literal block

Reviewed-by: Nathan Cutler <ncutler@suse.com>