pybind/mgr/mgr_module: extract load_func_metadata()

so it can be reused by Command class.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: annotate command handler

and use enum as the argument

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/cephadm: use desc of command for docstring

in the new command mechinary, the docstring of command handler
is used as the description of the command.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/mgr_module: drop arg parameter from CLICommand constructor

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/snap_schedule: use param annotation for argsdesc

use parameter annotation for CLICommand args desc, less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: use param annotation for args desc

use parameter annotation for CLICommand arg desc

less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/mgr_module: use 'not in' for checking membership

silences the warning from flake8 like

test for membership should be 'not in' [E713]

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/ceph_argparse: add argdesc() method to type classes

so they can be used to build argdesc from method annotations, these
argdesc strings will be consumed by ceph CLI to do client-side
validation of command arguments.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/devicehealth: add arg annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/devicehealth: PEP8 compliance cleanups

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr: translate argspec to argdesc

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybing/mgr/mgr_module: use annotations when collecting args

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr: do not check for kwargs if already collecting kwargs

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybing/mgr/mgr_module: make CLICommand._parse_args() a staticmethod

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybing/mgr/mgr_module: extract method for collecting kwargs

the existing method is using argdesc for collecting kwargs, by
extracting into a separated method, we can use the annotation as an
alternative.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybing/mgr/mgr_module: extract kwargs handling into separated method

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38850 from liewegas/mgr-handle-old-host-inventory

mgr/cephadm: tolerate old host inventory without 'hostname' key

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38866 from sebastian-philipp/cephadm-pyright

cephadm: Fix type errors reported by pyright

Reviewed-by: Joao Eduardo Luis <joao@suse.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #38851 from liewegas/fix-mailmap-sage

.mailmap: map myself to sweil@redhat.com

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38838 from orozery/rbd-encryption-cli

librbd: rbd encryption cli

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38864 from MahatiC/wip-ssd-fix

librbd/cache/pwl: Rename config options

Reviewed-by: Xiaoyan Li <xiaoyan.li@intel.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38751 from rhcs-dashboard/fix-48591-master

mgr/dashboard: Monitoring alert badge includes suppressed alerts

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38879 from Rethan/remove-osd-heartbeat-option

doc/rados: remove unused osd heartbeat option

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38564 from ronen-fr/wip-ronenf-clangcrims-1

rgw: fixing compilation errors

Reviewed-by: Willem Jan Withagen (@wjwithagen)
Reviewed-by: Matt Benjamin mbenjamin@redhat.com
Reviewed-by: Daniel Gryniewicz dang@redhat.com dang@fprintf.net
Reviewed-by: Adam C. Emerson aemerson@redhat.com
Reviewed-by: Yuval Lifshitz ylifshit@redhat.com

Merge PR #37095 into master

* refs/pull/37095/head:
mds: make purge_queue delete objects asynchronously and keep accepting pushes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #37876 into master

* refs/pull/37876/head:
systemd: cephfs-mirror systemd unit files
doc, man: man page for `cephfs-mirror` tool
doc: document mirror daemon internals
cephfs-mirror: switch to using PeerReplayer class
cephfs-mirror: cancel ongoing snapshot syncs on dir removal
cephfs-mirror: display peer snapshot sync stats
cephfs-mirror: carve out (and implement) mirroring snapshots to peers
cephfs-mirror: remove `cephfs_mirror_directory_choose_policy` option
cephfs-mirror: include helper routines to separate source
cephfs-mirror: remove peer only when peer is tracked
cephfs-mirror: typedef ceph_mount_info as MountRef shared pointer
cephfs-mirror: enclose json dump in object section
cephfs-mirror: note current peer set
cephfs-mirror: fix option typo and document certain options
cephfs-mirror: remove unnecessary command line options
cephfs-mirror: default log level 0/5

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #38820 into master

* refs/pull/38820/head:
include: use narrower ostream instead of iostream
mds: remove unused fstream header
mds: use narrower ostream header instead of sstream
mds: remove unused iostream header
mds: remove unused unix headers

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #38835 into master

* refs/pull/38835/head:
qa: add test for fs rm idempotency
mon/FSCommands: restore idempotent behavior of fs rm
qa: add ceph cmd helper
qa: allow kwargs for raw_cluster_cmd

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #38848 into master

* refs/pull/38848/head:
qa: ignore MDS_SLOW_METADATA_IO with osd thrasher

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #38886 from cbodley/wip-rgw-pubsub-warning

rgw: fix initialization of s3_event in pubsub

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #38882 from tchaikov/wip-no-print-nanoseconds

tests/test_bluestore_types: cast duration to ceph::timespan

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

librbd: add encryption cli

This commit adds rbd encryption cli (encryption format and nbd map)

Signed-off-by: Or Ozeri <oro@il.ibm.com>

Merge PR #38652 into master

* refs/pull/38652/head:
mgr/volumes: Filter inherited snapshots while listing snapshots

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38785 from sebastian-philipp/cephadm-move-create-daemon-to-serve

mgr/cephadm: move a bunch of methods from module.py to serve.py

Reviewed-by: Michael Fritch <mfritch@suse.com>

rgw: fix initialization of s3_event in pubsub

resolves compiler warning:

rgw_sync_module_pubsub.cc: In constructor ‘RGWPSHandleObjEventCR::RGWPSHandleObjEventCR(RGWDataSyncCtx*, PSEnvRef, const rgw_user&, EventRef<rgw_pubsub_event>&, EventRef<rgw_pubsub_s3_event>&, const TopicsRef&)’:
rgw_sync_module_pubsub.cc:1028:3: warning: ‘RGWPSHandleObjEventCR::s3_event’ is initialized with itself [-Winit-self]
 1028 |   RGWPSHandleObjEventCR(RGWDataSyncCtx* const _sc,
      |   ^~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Casey Bodley <cbodley@redhat.com>

tests/test_bluestore_types: cast duration to ceph::timespan

on FreeBSD + Clang + libc++ + amd64, std::chrono::nanoseconds::rep is `long
long` instead of `long`, so the explictly instantiated template
operator<< operator instances in `src/common/ceph_time.cc` are not able to
cover this use case. so in this change, the duration is casted to
ceph::timespan instead of nanoseconds. so we can ensure that `operator<<` is
always available in this case.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38860 from tchaikov/wip-48777

osd/pg_scrubber: do not print log in dtors

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #38724 from tchaikov/wip-install-deps-remove-mirror

install-deps.sh: do not use deb.rug.nl as a mirror anymore

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/rados: remove unused osd heartbeat option

Since commit a5f9ca, option osd_mon_ack_timeout was removed, change
doc accordingly.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>

mgr/volumes: Filter inherited snapshots while listing snapshots

Filter inherited snapshots resulted as part of a snapshot
at ancestor level while listing snapshots of a subvolume
and subvolumegroup

Also, fail the snapshot info on inherited snapshot.

Fixes: https://tracker.ceph.com/issues/48501
Signed-off-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #33692 from majianpeng/osd-remove-useless-code

osd: remove usless check condition.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38765 from tchaikov/wip-mutex-debug-atomic

common/mutex_debug: add memory barrier before load/store nlock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd/cache/pwl: Rename config options

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

Merge pull request #38696 from cyx1231st/wip-seastar-onode-tree-denc-delta

crimson/onode-staged-tree: implement delta encode, decode, replay and tests

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/onode-staged-tree: set indent-tabs-mode to nil

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/onode-staged-tree: implement get_type() for test extents

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson: implement seastore replay test for onode-staged-tree

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #38806 from yuriw/wip-yuriw-octopus-x-master

qa/tests: added rbd_import_export workload

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: added stress-split-no-cephadm tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge PR #38849 into master

* refs/pull/38849/head:
doc: document ms_mode option in the mount.ceph manpage

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #38793 into master

* refs/pull/38793/head:
qa: check for rank hole without racy joinable flag gymnastics
qa: skip client mounts for TestResize

Reviewed-by: Venky Shankar <vshankar@redhat.com>

qa: add test for fs rm idempotency

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mon/FSCommands: restore idempotent behavior of fs rm

Fixes: https://tracker.ceph.com/issues/48808
Fixes: 811e48d6ffd841ecd93a57eb00c6aceae476f4f8
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add ceph cmd helper

A more programmer friendly command to use.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: allow kwargs for raw_cluster_cmd

And refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd/pg_scrubber: do not print log in dtors

PG::recovery_state is defined after PG::m_scrubber, while
PG::gen_prefix() retrieves the osdmap from recovery_state. in
PgScrubber::~PgScrubber(), we print out the osdmap by dereferencing
recovery_state for the prefix of the logging message. in other words,
PgScrubber's destructor is referencing "recovery_state" which is
already destroyed.

in this change, the logging messages are removed from dtors. we can
print out the pgid in a follow-up PR if it helps.

Fixes: https://tracker.ceph.com/issues/48777
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38718 from tchaikov/wip-mgr-health-cleanup

mgr/devicehealth: use helpers to define options and commands

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

cephadm: Fix type errors reported by pyright

pyright is picky about `except ImportError` constructs
and also checks un-annotated code by default.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

systemd: cephfs-mirror systemd unit files

Signed-off-by: Venky Shankar <vshankar@redhat.com>

doc, man: man page for `cephfs-mirror` tool

Signed-off-by: Venky Shankar <vshankar@redhat.com>

doc: document mirror daemon internals

... fix some typos and reorganize sections to be reader
friendly. Also include a requirements section.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: switch to using PeerReplayer class

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: cancel ongoing snapshot syncs on dir removal

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: display peer snapshot sync stats

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: carve out (and implement) mirroring snapshots to peers

Remote file synchronization by @mchangir.

Signed-off-by: Milind Changire <mchangir@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: remove `cephfs_mirror_directory_choose_policy` option

Basically, the mirror daemon would process directory list in in a
round-robin fashion. Randomly choosing directories probably does not
make much sense right now.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: include helper routines to separate source

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: remove peer only when peer is tracked

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: typedef ceph_mount_info as MountRef shared pointer

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: enclose json dump in object section

So that mirroring module can decode the JSON object.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: note current peer set

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: fix option typo and document certain options

s/cephfs_mirror_mirror_action_update_interval/cephfs_mirror_action_update_interval/

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: remove unnecessary command line options

Signed-off-by: Venky Shankar <vshankar@redhat.com>

cephfs-mirror: default log level 0/5

Signed-off-by: Venky Shankar <vshankar@redhat.com>

.mailmap: map myself to sweil@redhat.com

Signed-off-by: Sage Weil <sweil@redhat.com>

Merge pull request #38863 from tchaikov/wip-osd-scrubber-tab

osd/pg_scrubber: use tab-width:8

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge PR #34859 into master

* refs/pull/34859/head:
test: Initialize timer
test: Avoid named semaphores on Windows
test: Update cmake files, allowing tests to run on Windows
test: port Ceph tests to Windows

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osd/pg_scrubber: use tab-width:8

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38781 from tchaikov/wip-googletest-gcc11

googletest: pick up change to silence error=maybe-uninitialized warning

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38809 from Rethan/fs-status-style

pybind/mgr: change format_bytes unit color from black to yellow

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38814 from tchaikov/wip-fix-throttle-test

common/Throttle: return new count in put() right away

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38811 from majianpeng/optimize-is_blackhole

msg: using switch-case in place of if-else.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38780 from majianpeng/fix-wrong-num_ios

os/bluestore: fix wrong num_ios for read op.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #38837 from vbashkirtsev/master

common/options.h: use uint64_t for TYPE_SIZE options

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: document ms_mode option in the mount.ceph manpage

Fixes: https://tracker.ceph.com/issues/48838
Signed-off-by: Jeff Layton <jlayton@redhat.com>

mgr/cephadm: tolerate old host inventory without 'hostname' key

Older cephadm clusters lack the 'hostname' key in the host spec.  e.g.,

 "cpach": {"addr": "cpach", "labels": ["mon"]}, "eutow": {"addr": "eutow", "labels": ["mon"]}, "stud": {"addr": "stud", "labels": ["mon"]}}

Populate hostname from the dict key if necessary for compatibility.

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tests: added rbd_import_export tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #38768 from votdev/fix_path_to_dash_devel_doc

mgr/dashboard: Fix path to dash_devel.rst

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #38790 into master

* refs/pull/38790/head:
common: remove dead option

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #38788 into master

* refs/pull/38788/head:
mount.ceph: collect v2 addresses for non-legacy ms_mode options

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #38640 into master

* refs/pull/38640/head:
qa: add test for reserved keyword feature
qa: use no client for required client feature tests
mds: do not allow setting a reserved feature by name
mds: return sv for efficiency

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

qa: ignore MDS_SLOW_METADATA_IO with osd thrasher

Fixes: https://tracker.ceph.com/issues/48834
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38766 from sebastian-philipp/cephadm-service-action-async

mgr/cephadm: make `ceph orch {restart|...}` asynchronous

Reviewed-by: Michael Fritch <mfritch@suse.com>

include: use narrower ostream instead of iostream

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove unused fstream header

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use narrower ostream header instead of sstream

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove unused iostream header

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove unused unix headers

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38255 from linuxbox2/wip-fio-librgw

rgw:fio: introduce libfio_rgw

Merge pull request #36120 from smithfarm/wip-rpath-ceph-diff-sorted

cmake: add empty RPATH to ceph-diff-sorted

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>