mds: Fix MDCache.h reorder compiler warnings

Fixes: https://tracker.ceph.com/issues/42642
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #30598 from smithfarm/wip-doc-releases-toc

doc/releases: access main releases page from top-level TOC

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #31291 from liewegas/bug-42578

mgr/ActivePyModules: behave if a module queries a devid that does not exist

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31230 from aclamk/fix-rados-put-truncate

tools/rados: prevent 'put' operation from recreating object when --offset=0

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31366 from smithfarm/wip-cbs-4

ceph-backport.sh: implement interactive setup routine and new options

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Fix iSCSI Target's validation (#30894)

mgr/dashboard: Fix iSCSI Target's validation

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Fix OSD's safe_to_destroy method (#30499)

mgr/dashboard: Fix OSD's safe_to_destroy method

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge PR #30835 into master

* refs/pull/30835/head:
mds: no assert on frozen dir when scrub path

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30915 into master

* refs/pull/30915/head:
mds: Reorganize class members in MDSAuthCaps header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30745 into master

* refs/pull/30745/head:
mds: Reorganize class members in MDCache header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31330 into master

* refs/pull/31330/head:
messages: protect CephFS messages get/put methods
messages: protect MClientCapRelease cons
messages: simplify MMDSCacheRejoin default cons
mds: simplify type declarations

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31374 from cbodley/wip-qa-rgw-multisite-sse

qa/rgw: use 'testing' kms backend for multisite tests

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

ceph-backport.sh: increment the version number

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph-backport.sh: delete deprecated config file after setup complete

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph-backport.sh: implement interactive setup routine and new options

This commit implements several new features:

* a --cherry-pick-only option
* a --force option
* an --existing-pr option
* an interactive setup routine

The --cherry-pick-only option can be used to test whether a backport
cherry-picks cleanly, for example. This is the same as the --prepare
functionality that was provided by an earlier version of the script, and
--prepare is re-introduced as a synonym for --cherry-pick-only.

The --force option can be used to make the script less careful (less
"cowardly"). For example, if the script refuses to do a backport because the
backport tracker issue is assigned to someone else, the script will "cowardly"
refuse to continue. Use --force to override. Be aware that --force will also
blow away an existing wip branch - the script asks for user confirmation in this
case.

The new --existing-pr option can be used to specify the number (ID) of an
existing backport PR that addresses the backport tracker issue given via the
positional argument.

The new "interactive setup routine" should make the setup process much simpler
for the user. If there is a setup issue, the script produces a report and starts
the interactive setup routine, which prompts the user for the needed
information.

Also, the script no longer requires the user to explicitly provide values for
github_user and redmine_user_id. Instead, it divines the correct values from the
GitHub token and the Redmine key, respectively.

Finally, the existing ~/bin/backport_common.sh file is deprecated in favor of
two files:

    ~/.github_token
    ~/.redmine_key

(The latter is already used by Sage's build-integration-branch tool and it
didn't make sense to have two different configuration files for a single
purpose.)

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/rgw: use 'testing' kms backend for multisite tests

a missing piece from https://github.com/ceph/ceph/pull/30940

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #31192 from sebastian-philipp/run_mypy-0.740

script/run_mypy: Support mypy 0.740

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge PR #31341 into master

* refs/pull/31341/head:
ceph-daemon: only set up /var/run/ceph/$fsid if it exists

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #31331 into master

* refs/pull/31331/head:
ceph-daemon: Use `shutil` to move/copy files

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #31353 into master

* refs/pull/31353/head:
doc/orchestrator: Fix various issues in Orchestrator CLI documentation

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #31329 into master

* refs/pull/31329/head:
ceph-daemon: Fix `ls` cmd for legacy confs

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-daemon: Fix `ls` cmd for legacy confs

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #31328 into master

* refs/pull/31328/head:
ceph-daemon: raise RuntimeError when CephContainer.run() fails

ceph-daemon: Use `shutil` to move/copy files

Signed-off-by: Michael Fritch <mfritch@suse.com>

ceph-daemon: raise RuntimeError when CephContainer.run() fails

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/dashboard: Automatically use correct chromedriver version (#31352)

mgr/dashboard: Automatically use correct chromedriver version

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #31292 into master

* refs/pull/31292/head:
mgr/ssh: fix remove_mds exception
doc/mgr/orchestrator_cli: remove extra characters after checkmarks
doc/mgr/orchestrator_cli: no {stop,start,restart} for all the random svcs
mgr/ssh: add 'redeploy' operation
ceph-daemon: stop + reset-failed existing unit on daemon deploy
ceph-daemon: make second call arg optional
ceph-daemon: make 'deploy' on mon only to mkfs step is mon doesn't yet exist
mgr/ssh,orchestrator_cli: add 'restart' service action
mgr/ssh: service[-instance] [start|stop]
mgr/ssh: simplify get_services call for 'mds update'
mgr/ssh: make _get_services take service *id* or service *name*
mgr/ssh: fix 'service ls' on stopped services
mgr/ssh: move some code around
doc/mgr/orchestrator_cli: 'service ls' works for ssh
doc/mgr/orchestrator_cli: we already completed mds ops for rook+ssh

Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/ssh: fix remove_mds exception

Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr/orchestrator_cli: remove extra characters after checkmarks

Not sure what those were.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr/orchestrator_cli: no {stop,start,restart} for all the random svcs

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: add 'redeploy' operation

This will stop, reprovision container, and restart. For most daemons, that
means we can

 - change the container image
 - refresh the config
 - refresh the keyring

For OSDs, it means we also re-run ceph-volume activate.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: stop + reset-failed existing unit on daemon deploy

Normally it won't exist, but it will on a re-deploy.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: make second call arg optional

If not specified, pull it from command[0].

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: make 'deploy' on mon only to mkfs step is mon doesn't yet exist

This allows us to (re)deploy a mon container like the other daemon types,
which means rewriting the config, keyring, and systemd unit(s).

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh,orchestrator_cli: add 'restart' service action

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: service[-instance] [start|stop]

Note that 'reload' is meaningless for ssh orch...

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #31266 from rhcs-dashboard/wip-42565-master

mgr/dashboard: fix LazyUUID4 not serializable

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: CephFS directory component (#30047)

mgr/dashboard: CephFS directory component

Reviewed-by: Volker Theile <vtheile@suse.com>

doc/orchestrator: Fix various issues in Orchestrator CLI documentation

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Automatically use correct chromedriver version

Every Chrome version requires a corresponding chromedriver version for the e2e
test to work.
Since we were using a fixed chromedriver version, each time chrome was updated
the e2e scripts stopped working.
With this new script we will check which version of Chrome is installed and
update chromedriver to the correct version.

Fixes: https://tracker.ceph.com/issues/42607
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Signed-off-by: Tiago Melo <tmelo@suse.com>

script/run_mypy: Support mypy 0.740

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-Authored-By: Ernesto Puerta <37327689+epuertat@users.noreply.github.com>

Merge pull request #31191 from sebastian-philipp/ceph-volume-pr-template

github: Add ceph-volume to list of jenkins commands

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

messages: protect CephFS messages get/put methods

Use SafeMessage inheritance so that only intrusive pointers can modify
the reference count. This is just a defensive programming refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

messages: protect MClientCapRelease cons

This should only be constructed via make_message.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

messages: simplify MMDSCacheRejoin default cons

Trivial initializations can be moved out.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: simplify type declarations

This is a trivial refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #31326 from cbodley/wip-cmake-mgr-cap

cmake: define mgr_cap_obj library when WITH_MGR=OFF

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31015 into master

* refs/pull/31015/head:
mgr/orchestrator: Fix raise_if_exception for Python 3

Reviewed-by: Tim Serong <tserong@suse.com>

ceph-daemon: only set up /var/run/ceph/$fsid if it exists

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31337 into master

* refs/pull/31337/head:
Fixing typo in ceph-daemon script for the output_pub_ssh_key argument

Reviewed-by: Sage Weil <sage@redhat.com>

Fixing typo in ceph-daemon script for the output_pub_ssh_key argument

Signed-off-by: John McGowan <john@steakfest.com>

Merge pull request #31335 from wjwithagen/wjw-fix-objecstore-bluestore

test: move bluestore dependent code under WITH_BLUESTORE

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31312 from changchengx/qa

qa: correct zap disk with ceph-deploy tool

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30791 from kshtsk/wip-python3-no-iteritems-ceph-deploy

tasks/ceph_deploy: get rid of iteritems for python3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31209 from wjwithagen/wjw-fix-snprintf

common: fix missing <stdio.h> include

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31184 from pdvian/wip-fix-sign-compare-warning

crush/CrushWrapper.cc: Fix sign compare compiler warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31306 from wjwithagen/wjw-fix-src_test_objectstore_CMakeLists.txt

test: only compile ceph_test_bmap_alloc_replay WITH_BLUESTORE

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31313 from tchaikov/wip/debian/rules/ceph-daemon

debian/rules: run dh_python2 with ceph-daemon

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31238 from tchaikov/wip/install-deps/el8

install-deps.sh: use sepia/lab-extra/8

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge PR #30751 into master

* refs/pull/30751/head:
doc: Document Export Process during Subtree Migrations

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30749 into master

* refs/pull/30749/head:
doc: document mds journal event types

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>

test: move bluestore dependant code under WITH_BLUESTORE

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #31152 from LongDuncan/feature/mgr_restful_rquest_api_support_multicommand

mgr/restful: requests api adds support multiple commands

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge PR #30938 into master

* refs/pull/30938/head:
mds: Reorganize class members in MDSCacheObject header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28077 into master

* refs/pull/28077/head:
client: optimize rename operation under different quota root

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30365 into master

* refs/pull/30365/head:
cephfs-shell: get rid of "return" at the end of the method
pybind/cephfs.pyx: add NotDirectory exception
cephfs-shell: don't expect both Error and OSEror together
cephfs-shell: print built-in error messages for exceptions
cephfs-shell: use poutput defined with the module
cephfs-shell: use perror() defined within the module
cephfs-shell: don't pass default values in calls to perror
cephfs-shell: align perror's signature with cmd2.Cmd.perror
pybind/cephfs: make actual error message part of strerror of OSError

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge PR #30054 into master

* refs/pull/30054/head:
doc: fs subvolume resize command
qa/tasks: tests for resize subvolume
mgr/volumes: fs subvolume resize command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30402 into master

* refs/pull/30402/head:
client: add warning when cap != in->auth_cap.

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #30990 into master

* refs/pull/30990/head:
mds: Reorganize class members in MDSDaemon header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31207 into master

* refs/pull/31207/head:
qa/tasks: Better handling of thrasher names and __init__ calls

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31137 into master

* refs/pull/31137/head:
msg/DispatchQueue: Set throttle stamp for local_delivery

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31319 into master

* refs/pull/31319/head:
ceph-daemon: absolute path for /bin/install
ceph-daemon: ExecStart=/bin/bash script

Reviewed-by: Alfredo Deza <adeza@redhat.com>

cmake: define mgr_cap_obj library when WITH_MGR=OFF

resolves the cmake error when WITH_MGR=OFF:

  Error evaluating generator expression:

    $<TARGET_OBJECTS:mgr_cap_obj>

  Objects of target "mgr_cap_obj" referenced but no such target exists.
Call Stack (most recent call first):
  src/mon/CMakeLists.txt:34 (add_library)

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #31318 into master

* refs/pull/31318/head:
ceph.spec.in: run pathfix.py
ceph-daemon: use /usr/bin/python, not /usr/bin/env python

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31226 into master

* refs/pull/31226/head:
mgr/telemetry: add stats about crush map
mgr/telemetry: add rgw metadata
mgr/telemetry: include fs size (files, bytes, snaps)
mds: report r{files,bytes,snaps} via perfcounters
mgr/telemetry: mds cache stats
mgr/telemetry: add some rbd metadata
mgr/telemetry: note whether osd cluster_network is in use
mgr/telemetry: add host counts
mgr/telemetry: add more pool metadata
mgr/telemetry: remove crush rule name
mgr/telemetry: include min_mon_release and msgr v1 vs v2 addr count
mgr/telemetry: add CephFS metadata

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #25374 into master

* refs/pull/25374/head:
osd: add flag to prevent truncate_seq copy in copy-from operation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #31256 into master

* refs/pull/31256/head:
mgr/MgrClient: fix open condition

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: AttributeError: 'NoneType' object has no attrib… (#31078)

mgr/dashboard: Fixed AttributeError: 'NoneType' in smart data integration

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

ceph.spec.in: run pathfix.py

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: absolute path for /bin/install

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31320 into master

* refs/pull/31320/head:
ceph-daemon: fix os.mkdir call

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-daemon: fix os.mkdir call

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31314 into master

* refs/pull/31314/head:
mgr/orchestrator_cli: pass default value to "req=False" params

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #31315 into master

* refs/pull/31315/head:
ceph-daemon: create ~/.ssh if not exist

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-daemon: ExecStart=/bin/bash script

Not sure why this matters, but it doesn't work on centos7 without the
/bin/bash part.

https://stackoverflow.com/questions/45776003/fixing-a-systemd-service-203-exec-failure-no-such-file-or-directory

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: create ~/.ssh if not exist

there is chance that /root/.ssh does not exist yet, when we are trying
to add key to authorized_keys.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/telemetry: add stats about crush map

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: use /usr/bin/python, not /usr/bin/env python

This works around noise from the rpm build:

*** ERROR: ambiguous python shebang in /usr/sbin/ceph-daemon: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #31045 from tchaikov/wip/librados/no-more-ack-commit

librados,test,rgw: cleanups to deprecate safe_cb related functions

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31311 from tchaikov/wip/rbd/cleanup

librbd: always initialize local variables

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30912 from dillaman/wip-42265

mgr: add 'rbd' profiles to support 'rbd_support' module commands

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31244 from changchengx/deploy_doc

doc: update with osd addition

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31196 from liewegas/wip-osdmon-iec

mon/OSDMonitor: expand iec_options for 'osd pool set'

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: use new aio_create_completion()

Signed-off-by: Kefu Chai <kchai@redhat.com>

journal: use new aio_create_completion()

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/librados_test_stub: implement aio_create_completion2() and friends

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: use new aio_create_completion()

to silence warnings like:

src/librbd/Utils.h:139:48: warning: 'static librados::v14_2_0::AioCompletion* librados::v14_2_0::Rados::aio_create_completion(void*, librados::callback_t, librados::callback_t)' is deprecated
[-Wdeprecated-declarations]
   return librados::Rados::aio_create_completion(
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
     obj, &detail::rados_callback<T, MF>, nullptr);
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/libradosstriper: use new aio_create_completion()

to silence -Wdeprecated-declarations warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

cls/journal: s/create_completion/create_completion2/

src/cls/journal/cls_journal_client.cc:154:72: warning: 'static librados::v14_2_0::AioCompletion* librados::v14_2_0::Rados::aio_create_completion(void*, librados::callback_t, librados::callback_t)' is
deprecated [-Wdeprecated-declarations]
       librados::Rados::aio_create_completion(this, rados_callback, NULL);
                                                                        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: s/create_completion/create_completion2/

to silence the warnings like

s_user_client.cc:151:77: warning: 'static librados::v14_2_0::AioCompletion* librados::v14_2_0::Rados::aio_create_completion(void*, librados::callback_t, librados::callback_t)' is deprecated
[-Wdeprecated-declarations]
   AioCompletion *c = librados::Rados::aio_create_completion(NULL, NULL, NULL);
                                                                             ^

Signed-off-by: Kefu Chai <kchai@redhat.com>