Merge PR #23334 into master

* refs/pull/23334/head:
pybind/rados/rados: do not pass prval from stack

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23217 from dzafman/wip-25085

osd: Allow repair of an object with a bad data_digest in object_info on all replicas

Reviewed-by: Sage Weil <sage@redhat.com>

pybind/rados/rados: do not pass prval from stack

The prval is a pointer to an int to write the final completion code of
the rados op.  This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).

We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).

Fixes: http://tracker.ceph.com/issues/25175
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23348 from ceph/wip-rm24957

ceph-volume: adds test for `ceph-volume lvm list /dev/sda`

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: adds a unit test for `lvm list /dev/sda`

This test is to prove that the issue from
http://tracker.ceph.com/issues/24957 was fixed
by http://tracker.ceph.com/issues/24784

When running lvm list against a raw device it should handle
gracefully the situation where there are multiple PVs with the
name of the given device.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: move pvolumes fixture into conftest.py

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23336 from noahdesu/vstart-dashboard-no-rbd

vstart: disable dashboard when rbd not built

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #23297 into master

* refs/pull/23297/head:
ceph_volume_client: add delay for MDSMap to be distributed

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #23308 into master

* refs/pull/23308/head:
doc: s/Ceph FS/CephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

vstart: disable dashboard when rbd not built

dashboard doesn't load correctly without the rbd module, which means
vstart commands that interact with dashboard fail and vstart exits.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>

Merge pull request #23292 from yuriw/wip-yuriw-25140-master

qa/tests: added 1st draft of mimic-x suite

Merge pull request #23302 from yuriw/wip-yuriw-crontab-master

qa/tests: added mimic-x to the schedule

Merge pull request #23321 from cernceph/dvanders_enable

ceph-volume: enable the ceph-osd during lvm activation

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23332 from alfredodeza/wip-rm25171

ceph-volume add a __release__ string, to help version-conditional calls

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23305 from smithfarm/wip-cleanup-upgrade

qa/upgrade: cleanup for nautilus

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/tests: added 1st draft of mimic-x suite

Fixes: https://tracker.ceph.com/issues/25140
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume add a __release__ string, to help version-conditional calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23276 from tchaikov/wip-config-diff-lock

common/config: fix the lock in ConfigProxy::diff()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23251 from neha-ojha/wip-25112

osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23249 from liewegas/wip-mon-cx-nautilus

osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23229 from rjfd/wip-dashboard-query-params-bug

mgr/dashboard: fix query parameters in task annotated endpoints

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #22607 into master

* refs/pull/22607/head:
common/options: convert many TYPE_[U]INT -> TYPE_SIZE
common/options: remove journal_max_corrupt_search

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-volume: enable ceph-osd during lvm activation

Enable the ceph-osd@<id> unit during lvm activate to link these
units to the ceph-osd.target.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: http://tracker.ceph.com/issues/24152

ceph-volume: optional systemd enable --runtime

Allow units to be enabled but not persisted across a reboot,
and use this when enabling osds.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #23287 from Devp00l/wip-duplicate-error-messages

mgr/dashboard: Fix duplicate error messages

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22669 from votdev/feature_24574

mgr/dashboard: Cleanup RGW config checks

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

qa/upgrade: cleanup for nautilus

Drop unused suites, which ATM means all of them except upgrade/luminous-x
which recently got a cleanup in https://github.com/ceph/ceph/pull/23162

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23162 from smithfarm/wip-upgrade-cleanup

tests: upgrade/luminous-x: fix order of final-workload directory

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tests: added mimic-x to the schedule

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

common/options: convert many TYPE_[U]INT -> TYPE_SIZE

Note that the _cost options are in fact in units of bytes.

Signed-off-by: Sage Weil <sage@redhat.com>

doc: s/Ceph FS/CephFS

Fixes: https://github.com/ceph/ceph/pull/22784#discussion_r200755460
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #23043 from tchaikov/wip-python-cephfs-dependencies

deb,rpm: fix python-cephfs dependencies

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph_volume_client: add delay for MDSMap to be distributed

Otherwise the setxattr will fail if the mds has not yet received the MDSMap
which adds the new data pool.

Fixes: https://tracker.ceph.com/issues/25141
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23288 from theanalyst/doc/releases/13.2.1

doc: releases: mimic 13.2.1 release notes

Reviewed-By: Sage Weil <sweil@redhat.com>

Merge pull request #23278 from b-ranto/wip-volume-selinux

ceph-volume: Restore SELinux context

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23106 from noahdesu/always-on-mgr-modules2

mgr: create always on class of modules

Merge pull request #23143 from joke-lee/qa-admin-api-user-info-through-access-key

qa: add test for https://github.com/ceph/ceph/pull/22790

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23289 from alfredodeza/wip-rm24993-2

ceph-volume ensure encoded bytes are always used

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: Restore SELinux context

We need to reset the root context of the file system after mounting it.
Otherwise, the SELinux policy rules will not be preserved.

Fixes: https://tracker.ceph.com/issues/24785
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #23277 from zy751713126/fix_note

cls/rbd: fix method comment

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23279 from tchaikov/wip-cmake-cleanup

cmake: cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23283 from tchaikov/wip-cmake-fmt

cmake: bump up the required fmt version

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: releases: mimic 13.2.1 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

ceph-volume tests.util verify as_bytes behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests catch utf-8 encoded stdin errors

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume process use the as_bytes util for stdin

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util create a helper for converting to bytes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22994 from tspmelo/wip-prettier-lint

mgr/dashboard: Improve prettier scripts and documentation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Fix duplicate error messages

Duplicate error messages currently appear if the task wrapper service is
used. It calls 'notifyTask' on a failed task, this would be fine if
we didn't have the API interceptor, which watches all API requests and
triggers 'notifyTask' itself if an error appears.

Fixes issue #25139

Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #23281 from tspmelo/wip-opaque

mgr/dashboard: Fix localStorage problem in Jest

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: bump up the required fmt version

seastar actually requires fmt 4.0.0 and up, as 3.0.2 does not offer
fmt/printf.h. see
https://github.com/fmtlib/fmt/blob/master/ChangeLog.rst#400---2017-06-27
.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix localStorage problem in Jest

Latest version of Jest was showing the following error:
"SecurityError: localStorage is not available for opaque origins"

Signed-off-by: Tiago Melo <tmelo@suse.com>

cmake: extract mgr into its own CMakeLists.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: extract common,msg out of src/CMakeLists.txt

* add arch as a static library.
* add crc32 as a static library. its only dependency is libarch.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cls/rbd: fix note

Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>

Merge pull request #23254 from tchaikov/wip-cmake-cleanup

cmake: modularize src/perfglue

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23255 from tchaikov/wip-seastar-yaml-cpp

seastar: lower the required yaml-cpp version to 0.5.1

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23253 from smithfarm/wip-doc-dev-toc

doc: make it easier to reach the old dev doc TOC

Reviewed-by: Dan Mick <dan.mick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common/config: fix the lock in ConfigProxy::diff()

it's a regression introduced by e406d8eb9e1deb801ecb346169eaaf96adbb4b53

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/internals: drop :orphan: directive

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23165 from smithfarm/wip-doc-snaptrim-nit

doc/rados/operations/pg-states: fix PG state names, part 2

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #23072 from fullerdj/wip-djf-24687

mon/OSDMonitor: Warnings for expected_num_objects

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23268 from ceph/simple-testinfra-version

ceph-volume: do not pin the testinfra version for the simple tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: do not pin the testinfra version for the simple tests

For testinfra to work with a 2.6.x version of ansible it must use the
latest version.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

mon/OSDMonitor: Warn if missing expected_num_objects

When creating a pool on filestore, warn if the user appears to be
creating a pool to store a large number of objects but omitted the
expected_num_objects parameter. Create the pool anyway.

Fixes: http://tracker.ceph.com/issues/24687
Signed-off-by: Douglas Fuller <dfuller@redhat.com>

osd,mon: increase mon_max_pg_per_osd to 250

https://bugzilla.redhat.com/show_bug.cgi?id=1603615 indicates
a case when pg calc conflicts with mon_max_pg_per_osd, and does not
allow pool creation when this limit is 200. Hence, increase this limit
to avoid this.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23263 from ceph/simple-ansible-version

ceph-volume: update version of ansible to 2.6.x for simple tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: update version of ansible to 2.6.x for simple tests

ceph-ansible now requires a 2.5.x or 2.6.x version of ansible if you're
using the master branch. This updates our functional tests for the
simple subcommand to use a 2.6.x version of ansible.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

test: Add test for repair of bad object info data_digest on all copies

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix pg repair to handle object info data_digest mismatched, but replicas match each other.

Fixes: https://tracker.ceph.com/issues/25085
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #23256 from trociny/wip-25114

test/pybind: fix test_rbd.TestClone.test_trash_snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Improve prettier scripts and documentation

Added prettier validation to the run-frontend-unittests script. This will make
sure we are always running prettier in our commits.

Added 2 new npm scripts:
- 'prettier', will run prettier formatter on all frontend files
- 'prettier:lint', will check all frontend files against prettier linter

Removed 'pretty-quick' and related scripts. Since we now have all files
prettified we can simply run prettier on them.

Remove 'tslint-eslint-rules' package and all related rules. Prettier can check
all the removed rules.

Updated HACKING.rst with some information about prettier.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Prettify all remaining files

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add reference to prettier in CodingStyle

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23168 from tchaikov/wip-dpdk-fixes

msg/async: fix FTBFS of dpdk

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23200 from votdev/bug_25068

mgr/dashboard: RGW is not working if an URL prefix is defined

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #22906 from tspmelo/wip-improve-summary

mgr/dashboard: Improve SummaryService and TaskWrapperService

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

do_cmake: fail if cmake fails

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add Findfmt.cmake

the cmake config provided by fmt-devel-3.0.2 does not offer fmt::fmt target,
so we need to offer a Findfmt.cmake to do this.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: make it easier to reach the old dev doc TOC

Signed-off-by: Nathan Cutler <ncutler@suse.com>

test/pybind: fix test_rbd.TestClone.test_trash_snapshot

Fixes: http://tracker.ceph.com/issues/25114
Signed-off-by: Mykola Golub <mgolub@suse.com>

cmake: make cpu_profiler a library target

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: make heap_profiler a library target

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: import allocator libraries in a better way

* instead expose the paths, it'd better to expose a library target.
* remove HAVE_LIBJEMALLOC, as it not used anywhere.

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: lower the required yaml-cpp version to 0.5.1

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23191 from dillaman/wip-23398

rbd: improved trash snapshot namespace handling

Reviewed-by: songweibin <song.weibin@zte.com.cn>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23228 from tchaikov/wip-ceph-dencoder

test: move ceph-dencoder to src/tools

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20366 from Synesis-LLC/master

doc: print pg peering in SVG instead of PNG

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools: Add testing feature "corrupt-info" to ceph-objectstore-tool

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Improve logic in be_compare_scrubmaps()

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #23248 from linuxbox2/rgw-fix-relaxed

rgw: fix build

Merge pull request #22758 from ricardoasmarques/wip-user-management

mgr/dashboard: Ceph dashboard user management from the UI

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

This was delayed and did not appear in 13.2.0.

Fixes: 1fdc85f224d52655d74eff9a29b6029b974661b3
Signed-off-by: Sage Weil <sage@redhat.com>

rgw: fix build

merge crossed with #23074

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge PR #23247 into master

* refs/pull/23247/head:
Revert "rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade"
Revert "rpm,deb: fix chown to use -h"
rpm,deb: fix chown to use -h
rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Revert "rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade"

This reverts commit 58cde3fd793c037beca6645cc30b2ae1c30e2af1.

This was only needed for luminous and mimic.  We are keeping these commits in master (and reverting them) only so that the cherry-pick -x references work.

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "rpm,deb: fix chown to use -h"

This reverts commit 92207c76b91c26c0558688a358ade7c0d94d1b59.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23234 from ceph/wip-rm24784

ceph-volume: PVolumes.get() should return one PV when using name or uuid

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23237 from neha-ojha/wip-whitelist-slow-requests

qa/tasks/thrashosds-health.yaml: whitelist slow requests

Merge pull request #23141 from alfredodeza/wip-rm24993

ceph-volume process.call with stdin in Python 3 fix

Reviewed-by: Andrew Schoen <aschoen@redhat.com>