qa/tasks/test_nfs: Add todo note for cluster update test

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs/test_nfs: Add test for get and list detailed export

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs/test_nfs: Add test for export user id

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs/test_nfs: Add description for each test

Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs: Add list and get export usage

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs: Add tests for export create with non-existing fsname and cluster id

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs: Add tests for invalid cluster id, export type and modify list cluster

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs: Add test for listing exports

Signed-off-by: Varsha Rao <varao@redhat.com>

pybind/mgr/volumes/nfs: Reorder FSExport and NFSCluster class methods

To maintain consistency and improve readability.

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Set pool_namespace and cluster_id through decorator

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Check cluster availability in export command methods

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Misc changes to export get and ls command

This patch does following changes:
* Add extra line before every class definition
* Reorder export dictionary
* Use _fetch_export() in get export command
* Remove cluster_id key from export dictionary if single export is deleted

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs: Add test for nfs cluster ls command

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Add nfs cluster ls command

This commands provides list of deployed nfs clusters.

Fixes: https://tracker.ceph.com/issues/45742
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Remove 'ganesha-' prefix from orch returned service id

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: get export in a nfs cluster

Fixes: https://tracker.ceph.com/issues/45741
Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes/nfs: list exports within a ganesha cluster

List the pseudo paths of the exports within a ganesha cluster
with `nfs export ls <cluster ID>` command.

List all the details of the exports within a ganesha cluster
with `nfs export ls <cluster ID> --detailed` command.

Partially-fixes: https://tracker.ceph.com/issues/45741
Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volume/nfs: allow only [A-Za-z0-9-_.] characters

... for ganesha cluster names.

Fixes: https://tracker.ceph.com/issues/45744
Signed-off-by: Ramana Raja <rraja@redhat.com>

qa/tasks/cephfs: Add test to check if export is available on restarting mgr

Signed-off-by: Varsha Rao <varao@redhat.com>

qa/tasks/cephfs: Add idempotency test for nfs commands

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Check if cluster exists before creating exports

Fixes: https://tracker.ceph.com/issues/45740
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/volumes/nfs: Fetch exports in persistent way if mgr is restarted

Fixes: https://tracker.ceph.com/issues/45740
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge PR #35784 into master

* refs/pull/35784/head:
mon/MDSMonitor: copy MDS info which may be removed

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #35038 into master

* refs/pull/35038/head:
qa/tasks/cephfs/cephfs_test_case.py: skip cleaning the core dumps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32493 from rishabh-d-dave/cv-use-dash-S-everywhere

ceph-volume: don't use container classes in api/lvm.py

Merge pull request #35657 from athanatos/sjust/wip-lba-replay

crimson/seastore: add extent replay support, implement for lba tree nodes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35370 from tspmelo/wip-ng-modal

mgr/dashboard: Use ng-bootstrap for Modal

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

ceph-volume: delete devices.lvm.prepare.Prepare.get_lv

Since it calls get_lv_from_argument() internally and use get_first_lv()
instead and update tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: don't use api.lvm.VolumeGroups

Use api.lvm.get_vgs() and api.lvm.get_first_vg() instead and update
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: don't use api.lvm.PVolumes

Update api.lvm.get_first_pv() and api.lvm.get_pvs() instead and update
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: don't use api.lvm.get_lv

Use api.lvm.get_lvs() and api.lvm.get_first_lv() instead and update the
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: don't use api.lvm.get_lv_from_argument

Use api.lvm.get_lvs() and api.lvm.get_first_lv() instead and update
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: don't use class api.lvm.Volumes

Replace code using class Volumes by methods get_lvs() and
get_first_lv(). Also, update the tests accordingly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34532 from rhcs-dashboard/wip-45068-fix-parse-error

mgr/dashboard: Prometheus query error in the metrics of Pools, OSDs and RBD images

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35576 from sebastian-philipp/cephadm-etc-ceph-ceph-conf

mgr/cephadm: Manage /etc/ceph/ceph.conf

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35565 from bk201/wip-44985

mgr/dashboard: fix wal/db slots controls in the OSD form

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35779 from votdev/show_yes_no

mgr/dashboard: Display check icon instead of true|false in various datatables

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35615 from ishanrai05/scss-linting

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #35826 into master

* refs/pull/35826/head:
mds: fix typo in MDSSocketHook

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35729 from rzarzynski/wip-bl-deprecate-claim

common/bl, *: deprecate bufferlist::claim() and switch to rvalue-taking operator=()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #35833 from smithfarm/wip-rn_14_2_10

doc/releases/nautilus: add 14.2.10 release notes

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

doc/releases/nautilus: add 14.2.10 release notes

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35786 from matthewoliver/cephadm_iscsi_remove_pool_cap

cephadm: ceph-iscsi remove pool from cap

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35800 from jan--f/drive-groups-tidy-interface

py-common/deployment/translate: drop unneeded ctor arg

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35781 from rhcs-dashboard/fix-bucket-list-columns-ui

mgr/dashboard: fix UI of the columns for rgw-bucket-list

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

common/bl, *: deprecate list::claim() in favor of operator=(list&&).

The motivation is that `claim(list&)` seems to actually be a pre-C++11
counterpart of the already available `operator=(list&&)`.
This commit deprecates the `claim()` method but doesn't drop it yet.

All occurrences of `buffer::list::claim(list&)` are switched to
  * `list::operator=(list&&)` or
  * reworked to use `list::list(list&&)` instead.

Changes are applied to: rgw, osdc, osd, os/memstore, os/filestore,
os/bluestore, os, msg, mgr, messages, mds, librbd, librados, crimson,
common, cls, mon.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mds: fix typo in MDSSocketHook

Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34311 from cyx1231st/wip-seastar-fix-heartbeat-race

crimson/osd: support 1 lossy connection between a heartbeat peer

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35815 from inspur-wyq/wip-doc-radosgw-archive-sync-module

doc/radosgw/archive-sync-module: fix url of "Multisite Configuration"

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: improve readability of Heartbeat::Peer classes

* Add rationales to introduce Heartbeat::Peer class series.
* Find better names for internal interfaces.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #35787 from tchaikov/wip-46208

install-deps,cmake: use boost v1.73

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35771 from sebastian-philipp/mypy-0.782

qa,src: update mypy to 0.782

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #35763 from rzarzynski/wip-test-unused-var-in-test_ino_release_cb

test: fix unused variable warning in test_ino_release_cb.cc.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35666 from inspur-wyq/wip-debian-cmake-version

debian: update cmake version to cmake_minimum_required(VERSION 3.10.2)

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/radosgw/archive-sync-module.rst: fix url of "Multisite Configuration"

Signed-off-by: wangyunqing <wangyunqing@inspur.com>

Merge pull request #35750 from rzarzynski/wip-common-config-dissect-boolcast

common, crimson: reuse the interpreter of Option::TYPE_BOOL

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35512 from ZhenLiu94/liuz02

src/mon/MonCommands.h: Remove the redundant right parenthesis in the help message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35760 from tchaikov/wip-mgr-track-pending-module

mgr: avoid false alarm of MGR_MODULE_ERROR

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

json_spirit: avoid using bind placeholders in global namespace

to silence the warning from boost v1.73, like

json_spirit/json_spirit_reader.cpp:7:
/opt/ceph/include/boost/bind.hpp:36:1: note: ‘#pragma message: The practice of declaring the Bind placeholders (_1, _2, ...) in the global namespace is deprecated. Please use <boost/bind/bind.hpp> +
using namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to retain the current behavior.’
   36 | BOOST_PRAGMA_MESSAGE(
      | ^~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

make-dist: redistribute boost 1.73

Fixes: https://tracker.ceph.com/issues/46208
Signed-off-by: Kefu Chai <kchai@redhat.com>

py-common/deployment/translate: drop unneeded ctor arg

The DriveGroupSpec is already part of the DriveSelection.

Fixes: https://tracker.ceph.com/issues/46231
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #35490 from sseshasa/wip-grace-interval-timer-reset

mon/OSDMonitor: Reset grace period if failure interval exceeds a threshold.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa,src: update mypy to 0.782

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35791 from tchaikov/wip-mock-rados

script/gen_static_command_descriptions: add mock for rados python binding

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

script/gen_static_command_descriptions: add mock for rados python binding

to address issue like

Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/doc/scripts/../../src/script/gen_static_command_descriptions.py", line 168, in <module>
    print(json.dumps(gen_commands_dicts(), indent=2, sort_keys=True))
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/doc/scripts/../../src/script/gen_static_command_descriptions.py", line 163, in gen_commands_dicts
    comms = from_mon_commands_h() + from_mgr_modules()
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/doc/scripts/../../src/script/gen_static_command_descriptions.py", line 85, in from_mgr_modules
    comms = sum([list_mgr_module(name) for name in names], [])
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/doc/scripts/../../src/script/gen_static_command_descriptions.py", line 85, in <listcomp>
    comms = sum([list_mgr_module(name) for name in names], [])
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/doc/scripts/../../src/script/gen_static_command_descriptions.py", line 65, in list_mgr_module
    mgr_mod = __import__(m_name, globals(), locals(), [], 0)
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/src/pybind/mgr/volumes/__init__.py", line 2, in <module>
    from .module import Module
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/src/pybind/mgr/volumes/module.py", line 8, in <module>
    from .fs.nfs import NFSCluster, FSExport
  File "/home/jenkins-build/build/workspace/ceph-pr-docs/src/pybind/mgr/volumes/fs/nfs.py", line 6, in <module>
    from rados import TimedOut
ImportError: cannot import name 'TimedOut'

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: download and build boost 1.73 if it is not installed

Fixes: https://tracker.ceph.com/issues/46208
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: update FindBoost.cmake for 1.73

adapted from
https://github.com/Kitware/CMake/blob/master/Modules/FindBoost.cmake
commit b76b4dea1cd60e9899fd641602901ded9e0d7a7b

Fixes: https://tracker.ceph.com/issues/46208
Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: use ceph-libboost1.73

Fixes: https://tracker.ceph.com/issues/46208
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34744 from ceph/rhbz-1827349

ceph-volume: handle idempotency with batch and explicit scenarios

Merge pull request #35468 from guits/guits-raw_dmcrypt

ceph-volume: add dmcrypt support in raw mode

mgr/dashboard: run lint and fix scripts in parallel

use npm-run-all package to run all the lint scripts and fix
scripts in parallel

Fixes: https://tracker.ceph.com/issues/46108
Signed-off-by: Ishan Rai <ishanrai05@gmail.com>

Merge pull request #35770 from tchaikov/wip-install-deps

install-deps.sh: drop el7 bits and uninstall python-sphinx if sphinx-build is in python2

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

cephadm: ceph-iscsi remove pool from cap

When we create a ceph-iscsi daemon/continer in cephadm we create a user
and set some caps. Turns out we were a little too restrictive.

We were locking down to only access the pool that was given in the spec,
which happens to be the pool the iscsi config is stored. But in reality
we need to be able  to attach any rbd images which could exist in other
pools.

So this patch removes the `pool=` from the osd cap, so from:

   osd = allow rwx pool={spec.pool}

To:

   osd = allow rwx

Fixes: https://tracker.ceph.com/issues/46138
Signed-off-by: Matthew Oliver <moliver@suse.com>

mon/MDSMonitor: copy MDS info which may be removed

Otherwise the log entry will contain garbage (and crash the mgr).

Fixes: https://tracker.ceph.com/issues/46216
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31726 into master

* refs/pull/31726/head:
mds: MDS-MDS message classes inherit from MMDSOp
mds: MMDSOp implementation and detect unversioned messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #34878 into master

* refs/pull/34878/head:
libcephfs: mark ceph_stat() family functions deprecated

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #35323 into master

* refs/pull/35323/head:
tool: change coding style, indent with two spaces

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #35420 into master

* refs/pull/35420/head:
mgr/volumes: Fix pool removal on volume deletion

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #35624 into master

* refs/pull/35624/head:
include: cleanup lru con/des

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #35664 into master

* refs/pull/35664/head:
qa: add omit_sudo=False for commands ran with sudo

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

ceph-volume: remove unused function

This function is never called in raw context, let's remove it.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: add raw testing coverage

This commit adds testing against `ceph-volume raw` subcommand.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: add dmcrypt support in raw mode

This commit adds the dmcrypt support in `ceph-volume raw` mode.

Note about `ceph-volume raw list` change:
Given `lsblk -J` (json output) option isn't available on all OS, I came up with
adding '--inverse' option to the existing command which allows us to get the
mapper devices list in that command output. Not listing root devices containing
partitions shouldn't have side effect since we are in `ceph-volume raw`
context.

example:
running `lsblk --paths --nodeps --output=NAME --noheadings` doesn't allow to
get the mapper list because the output is like following :

$ lsblk --paths --nodeps --output=NAME --noheadings
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd

the dmcrypt mappers are hidden because of the `--nodeps` given they are
displayed as a dependency.

$ lsblk --paths --output=NAME --noheadings
/dev/sda
|-/dev/mapper/ceph-3b52c90d-6548-407d-bde1-efd31809702f-sda-block-dmcrypt
`-/dev/mapper/ceph-3b52c90d-6548-407d-bde1-efd31809702f-sda-db-dmcrypt
/dev/sdb
/dev/sdc
/dev/sdd

adding `--inverse` is a trick to get around this issue, the counterpart is that
we can't list root devices if they contain at least one partition but this
shouldn't be an issue in `ceph-volume raw` context given we only deal with
raw devices.

Fixes: https://tracker.ceph.com/issues/45959
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge PR #35757 into master

* refs/pull/35757/head:
doc: update cephfs metadata pool recovery procedure

Reviewed-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #35537 from sebastian-philipp/cephadm-yaml-ordered-readable

mgr/orch: increase readability for yaml representation

Merge pull request #35274 from jmolmo/issue_45726

cephadm: error trying to get ceph auth entry for crash daemon

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35773 from theanalyst/master-rgw-cors-fixes

rgw: sanitize newlines in s3 CORSConfiguration's ExposeHeader

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: fix UI of the columns for rgw-bucket-list

Fixes: https://tracker.ceph.com/issues/46210
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

doc: update cephfs metadata pool recovery procedure

This is mostly clarifying a few things after personally experimenting
with this.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35695 from sebastian-philipp/mypy-rm-global-ignore-imports

mypy.ini: Remove global `ignore_missing_imports = True`

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #35762 from sebastian-philipp/doc-dev-rm-confusing.txt

doc/dev: remove confusing.txt

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35503 from sebastian-philipp/rm-daemon-ok-to-stop

mgr/cephadm: move ok_to_stop to CephadmService

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35761 from sebastian-philipp/qa-cephad-upgrade-no-rgw

qa/cephadm/upgrade: Exclude RGW for now

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33228 from galsalomon66/s3select_integration_into_RGW_2

rgw: add s3select support

mgr/dashboard: Display check icon instead of true|false in various datatables

- Convert the boolean values true|false to a check icon in datatable columns.
- Remove unused code in the user management list.

Fixes: https://tracker.ceph.com/issues/46209
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #35593 from p-se/wip-pse-cephadm-grafana-fix-version

mgr/cephadm: use fixed Grafana version

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Manage /etc/ceph/ceph.conf

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/tests: _ceph_get_module_option returns proper type now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: config_notify now provides a hook

reason is, we want to use this hook to schedule a
ceph.conf update for all hosts.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: refactor _run_cephadm

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>