qa: make test_volume_client.py py3 compatible

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa/ceph-volume: allow executing python pyloads using python3

Signed-off-by: Rishabh Dave <ridave@redhat.com>

cephfs.pyx: add py3 compatibility

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume-client: add py3 compatibility

Fixes: http://tracker.ceph.com/issues/17230
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge PR #23297 into master

* refs/pull/23297/head:
ceph_volume_client: add delay for MDSMap to be distributed

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #23308 into master

* refs/pull/23308/head:
doc: s/Ceph FS/CephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23292 from yuriw/wip-yuriw-25140-master

qa/tests: added 1st draft of mimic-x suite

Merge pull request #23302 from yuriw/wip-yuriw-crontab-master

qa/tests: added mimic-x to the schedule

Merge pull request #23321 from cernceph/dvanders_enable

ceph-volume: enable the ceph-osd during lvm activation

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23332 from alfredodeza/wip-rm25171

ceph-volume add a __release__ string, to help version-conditional calls

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23305 from smithfarm/wip-cleanup-upgrade

qa/upgrade: cleanup for nautilus

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/tests: added 1st draft of mimic-x suite

Fixes: https://tracker.ceph.com/issues/25140
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume add a __release__ string, to help version-conditional calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23276 from tchaikov/wip-config-diff-lock

common/config: fix the lock in ConfigProxy::diff()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23251 from neha-ojha/wip-25112

osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23249 from liewegas/wip-mon-cx-nautilus

osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23229 from rjfd/wip-dashboard-query-params-bug

mgr/dashboard: fix query parameters in task annotated endpoints

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #22607 into master

* refs/pull/22607/head:
common/options: convert many TYPE_[U]INT -> TYPE_SIZE
common/options: remove journal_max_corrupt_search

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-volume: enable ceph-osd during lvm activation

Enable the ceph-osd@<id> unit during lvm activate to link these
units to the ceph-osd.target.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: http://tracker.ceph.com/issues/24152

ceph-volume: optional systemd enable --runtime

Allow units to be enabled but not persisted across a reboot,
and use this when enabling osds.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #23287 from Devp00l/wip-duplicate-error-messages

mgr/dashboard: Fix duplicate error messages

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22669 from votdev/feature_24574

mgr/dashboard: Cleanup RGW config checks

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

qa/upgrade: cleanup for nautilus

Drop unused suites, which ATM means all of them except upgrade/luminous-x
which recently got a cleanup in https://github.com/ceph/ceph/pull/23162

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23162 from smithfarm/wip-upgrade-cleanup

tests: upgrade/luminous-x: fix order of final-workload directory

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tests: added mimic-x to the schedule

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

common/options: convert many TYPE_[U]INT -> TYPE_SIZE

Note that the _cost options are in fact in units of bytes.

Signed-off-by: Sage Weil <sage@redhat.com>

doc: s/Ceph FS/CephFS

Fixes: https://github.com/ceph/ceph/pull/22784#discussion_r200755460
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #23043 from tchaikov/wip-python-cephfs-dependencies

deb,rpm: fix python-cephfs dependencies

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph_volume_client: add delay for MDSMap to be distributed

Otherwise the setxattr will fail if the mds has not yet received the MDSMap
which adds the new data pool.

Fixes: https://tracker.ceph.com/issues/25141
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23288 from theanalyst/doc/releases/13.2.1

doc: releases: mimic 13.2.1 release notes

Reviewed-By: Sage Weil <sweil@redhat.com>

Merge pull request #23278 from b-ranto/wip-volume-selinux

ceph-volume: Restore SELinux context

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23106 from noahdesu/always-on-mgr-modules2

mgr: create always on class of modules

Merge pull request #23143 from joke-lee/qa-admin-api-user-info-through-access-key

qa: add test for https://github.com/ceph/ceph/pull/22790

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23289 from alfredodeza/wip-rm24993-2

ceph-volume ensure encoded bytes are always used

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: Restore SELinux context

We need to reset the root context of the file system after mounting it.
Otherwise, the SELinux policy rules will not be preserved.

Fixes: https://tracker.ceph.com/issues/24785
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #23277 from zy751713126/fix_note

cls/rbd: fix method comment

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23279 from tchaikov/wip-cmake-cleanup

cmake: cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23283 from tchaikov/wip-cmake-fmt

cmake: bump up the required fmt version

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: releases: mimic 13.2.1 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

ceph-volume tests.util verify as_bytes behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests catch utf-8 encoded stdin errors

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume process use the as_bytes util for stdin

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util create a helper for converting to bytes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22994 from tspmelo/wip-prettier-lint

mgr/dashboard: Improve prettier scripts and documentation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Fix duplicate error messages

Duplicate error messages currently appear if the task wrapper service is
used. It calls 'notifyTask' on a failed task, this would be fine if
we didn't have the API interceptor, which watches all API requests and
triggers 'notifyTask' itself if an error appears.

Fixes issue #25139

Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #23281 from tspmelo/wip-opaque

mgr/dashboard: Fix localStorage problem in Jest

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: bump up the required fmt version

seastar actually requires fmt 4.0.0 and up, as 3.0.2 does not offer
fmt/printf.h. see
https://github.com/fmtlib/fmt/blob/master/ChangeLog.rst#400---2017-06-27
.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix localStorage problem in Jest

Latest version of Jest was showing the following error:
"SecurityError: localStorage is not available for opaque origins"

Signed-off-by: Tiago Melo <tmelo@suse.com>

cmake: extract mgr into its own CMakeLists.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: extract common,msg out of src/CMakeLists.txt

* add arch as a static library.
* add crc32 as a static library. its only dependency is libarch.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cls/rbd: fix note

Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>

Merge pull request #23254 from tchaikov/wip-cmake-cleanup

cmake: modularize src/perfglue

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23255 from tchaikov/wip-seastar-yaml-cpp

seastar: lower the required yaml-cpp version to 0.5.1

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23253 from smithfarm/wip-doc-dev-toc

doc: make it easier to reach the old dev doc TOC

Reviewed-by: Dan Mick <dan.mick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common/config: fix the lock in ConfigProxy::diff()

it's a regression introduced by e406d8eb9e1deb801ecb346169eaaf96adbb4b53

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/internals: drop :orphan: directive

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23165 from smithfarm/wip-doc-snaptrim-nit

doc/rados/operations/pg-states: fix PG state names, part 2

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #23072 from fullerdj/wip-djf-24687

mon/OSDMonitor: Warnings for expected_num_objects

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23268 from ceph/simple-testinfra-version

ceph-volume: do not pin the testinfra version for the simple tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: do not pin the testinfra version for the simple tests

For testinfra to work with a 2.6.x version of ansible it must use the
latest version.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

mon/OSDMonitor: Warn if missing expected_num_objects

When creating a pool on filestore, warn if the user appears to be
creating a pool to store a large number of objects but omitted the
expected_num_objects parameter. Create the pool anyway.

Fixes: http://tracker.ceph.com/issues/24687
Signed-off-by: Douglas Fuller <dfuller@redhat.com>

osd,mon: increase mon_max_pg_per_osd to 250

https://bugzilla.redhat.com/show_bug.cgi?id=1603615 indicates
a case when pg calc conflicts with mon_max_pg_per_osd, and does not
allow pool creation when this limit is 200. Hence, increase this limit
to avoid this.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23263 from ceph/simple-ansible-version

ceph-volume: update version of ansible to 2.6.x for simple tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: update version of ansible to 2.6.x for simple tests

ceph-ansible now requires a 2.5.x or 2.6.x version of ansible if you're
using the master branch. This updates our functional tests for the
simple subcommand to use a 2.6.x version of ansible.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #23256 from trociny/wip-25114

test/pybind: fix test_rbd.TestClone.test_trash_snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Improve prettier scripts and documentation

Added prettier validation to the run-frontend-unittests script. This will make
sure we are always running prettier in our commits.

Added 2 new npm scripts:
- 'prettier', will run prettier formatter on all frontend files
- 'prettier:lint', will check all frontend files against prettier linter

Removed 'pretty-quick' and related scripts. Since we now have all files
prettified we can simply run prettier on them.

Remove 'tslint-eslint-rules' package and all related rules. Prettier can check
all the removed rules.

Updated HACKING.rst with some information about prettier.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Prettify all remaining files

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add reference to prettier in CodingStyle

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #23168 from tchaikov/wip-dpdk-fixes

msg/async: fix FTBFS of dpdk

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23200 from votdev/bug_25068

mgr/dashboard: RGW is not working if an URL prefix is defined

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #22906 from tspmelo/wip-improve-summary

mgr/dashboard: Improve SummaryService and TaskWrapperService

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

do_cmake: fail if cmake fails

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: add Findfmt.cmake

the cmake config provided by fmt-devel-3.0.2 does not offer fmt::fmt target,
so we need to offer a Findfmt.cmake to do this.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: make it easier to reach the old dev doc TOC

Signed-off-by: Nathan Cutler <ncutler@suse.com>

test/pybind: fix test_rbd.TestClone.test_trash_snapshot

Fixes: http://tracker.ceph.com/issues/25114
Signed-off-by: Mykola Golub <mgolub@suse.com>

cmake: make cpu_profiler a library target

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: make heap_profiler a library target

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: import allocator libraries in a better way

* instead expose the paths, it'd better to expose a library target.
* remove HAVE_LIBJEMALLOC, as it not used anywhere.

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: lower the required yaml-cpp version to 0.5.1

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23191 from dillaman/wip-23398

rbd: improved trash snapshot namespace handling

Reviewed-by: songweibin <song.weibin@zte.com.cn>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #23228 from tchaikov/wip-ceph-dencoder

test: move ceph-dencoder to src/tools

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20366 from Synesis-LLC/master

doc: print pg peering in SVG instead of PNG

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23248 from linuxbox2/rgw-fix-relaxed

rgw: fix build

Merge pull request #22758 from ricardoasmarques/wip-user-management

mgr/dashboard: Ceph dashboard user management from the UI

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

This was delayed and did not appear in 13.2.0.

Fixes: 1fdc85f224d52655d74eff9a29b6029b974661b3
Signed-off-by: Sage Weil <sage@redhat.com>

rgw: fix build

merge crossed with #23074

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge PR #23247 into master

* refs/pull/23247/head:
Revert "rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade"
Revert "rpm,deb: fix chown to use -h"
rpm,deb: fix chown to use -h
rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Revert "rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade"

This reverts commit 58cde3fd793c037beca6645cc30b2ae1c30e2af1.

This was only needed for luminous and mimic.  We are keeping these commits in master (and reverting them) only so that the cherry-pick -x references work.

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "rpm,deb: fix chown to use -h"

This reverts commit 92207c76b91c26c0558688a358ade7c0d94d1b59.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #23234 from ceph/wip-rm24784

ceph-volume: PVolumes.get() should return one PV when using name or uuid

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #23237 from neha-ojha/wip-whitelist-slow-requests

qa/tasks/thrashosds-health.yaml: whitelist slow requests

Merge pull request #23141 from alfredodeza/wip-rm24993

ceph-volume process.call with stdin in Python 3 fix

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #22533 from linuxbox2/wip-rgw-24507

rgw: add option for relaxed region enforcement

qa/tasks/thrashosds-health.yaml: whitelist slow requests

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23074 from tchaikov/wip-seastar-config

common/config: add ConfigProxy for crimson

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: Ceph dashboard user management

Fixes: https://tracker.ceph.com/issues/24446
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Skip tab focus on "show/hide password" button

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add 'onCancel' output for 'ConfirmationModal'

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add cd-select-badges component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

ceph-volume: PVolumes.get() should return one PV when using name or uuid

It is possible to get duplicated pv entires from the 'pvs' lvm command.
If we're using PVolumes.get() with either pv_name or pv_uuid we can
safely return a single PVolume object. However, if we use pv_tags with
PVolumes.get() we must still raise a MultiplePVsError if many pvs
are found that have the tags, because they are not guaranteed to be the
same pv, as would be the case with pv_name or pv_uuid.

Fixes: http://tracker.ceph.com/issues/24784
Signed-off-by: Andrew Schoen <aschoen@redhat.com>