mgr/dashboard: fix image filter's query on rbd-details grafana panel

The image filter on the RBD Details grafana panel is using a query with a type "label_values(ceph_rbd_read_ops{cluster=~\"$cluster\", , pool=\"$pool\"}, image)". The extra comma needs to be removed.

Fixes: https://tracker.ceph.com/issues/70653
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 8d9fe8fd709fe2792dec5fdbf364e77e3446f7fb)

Merge pull request #62526 from cbodley/wip-69811-squid

squid: rgw/s3: remove local variable 'uri' that shadows member variable

Reviewed-by: Joseph Mundackal <jmundackal@bloomberg.net>

Merge pull request #62219 from idryomov/wip-66419-squid

squid: qa/workunits/rbd: wait for resize to be applied in rbd-nbd

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #61817 from zdover23/wip-doc-2025-02-14-backport-61688-to-squid

squid: doc/dev/release-process.rst: release builds cannot build containers

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #61952 from zdover23/wip-doc-2025-02-21-backport-61948-to-squid

squid: doc/cephadm/services: Re-improve osd.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #61664 from zdover23/wip-doc-2025-02-06-backport-61616-to-squid

squid: doc/cephadm: clarify "Monitoring OSD State"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #62290 from zdover23/wip-doc-2025-03-14-backport-62258-to-squid

squid: doc/rados/configuration/bluestore-config-ref: Fix lowercase typos

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #62377 from zdover23/wip-doc-2025-03-19-backport-61670-to-squid

squid: common/options: fix the description of osd_max_scrubs

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

rgw/s3: remove local variable 'uri' that shadows member variable

fixes "Invalid group uri" errors from `s3cmd setacl --acl-public s3://bucketname`
introduced by account changes for squid

Fixes: https://tracker.ceph.com/issues/69527
Reported-by: Yixin Jin <yjin@akamai.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4257d11936cfcf0c96e6fabda72514236bbeeb4a)

Merge pull request #62346 from phlogistonjohn/wip-70154-squid

squid: container: small container image improvements

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #62155 from cbodley/wip-70343-squid

squid: radosgw-admin: lower default thread pool size

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #61159 from ivancich/wip-69295-squid

squid: rgw: append query string to redirect URL if present

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61092 from shashalu/fix-data-corruption-ETIMEDOUT-squid

squid: rgw: fix data corruption when rados op return ETIMEDOUT

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62467 from ljflores/wip-squid-62422

squid: PendingReleaseNotes; doc/rados/operations: document "rm-pg-upmap-primary-{all}" commands

Merge pull request #62309 from adamemerson/wip-55179-squid

squid: rgw: Deleting an object with null version fix

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #62306 from adamemerson/wip-67207-squid

squid: qa/multisite: add extra checkpoints in datalog_autotrim testcase

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #61147 from cbodley/wip-69317-squid

squid: rgw/posix: std::ignore return value of write()

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61134 from cbodley/wip-69297-squid

squid: rgw/abortmp: Race condition on AbortMultipartUpload

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61104 from cbodley/wip-69260-squid

squid: <common> fix formatter buffer out-of-bounds

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61051 from cbodley/wip-69213-squid

squid: rgw: fix bucket link operation

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #60982 from cbodley/wip-69152-squid

squid: radosgw-admin: bucket link/unlink support accounts

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62327 from NitzanMordhai/wip-70496-squid

squid: OSDMonitor: exclude destroyed OSDs from "ceph node ls" output

PendingReleaseNotes; doc/rados/operations: document "rm-pg-upmap-primary-{all}" commands

Added a release note for `ceph osd rm-pg-upmap-primary` and documentation to
complement #62190.

Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit 4dee7917bf010dc6d8e24f0ef437bd8a41c355c9)

*Note: PendingReleaseNotes was modified to have the release note under the v19.2.2 section.

Merge pull request #62421 from ljflores/wip-70592-squid

squid: mon, osd: add command to remove invalid pg-upmap-primary entries

Merge pull request #62363 from athanatos/wip-70537-squid

squid: osd: fix osd mclock queue item leak

Merge pull request #62393 from rhcs-dashboard/wip-70556-squid

squid: mgr/dashboard: add xlmtodict import and fix lifecycle get request

Merge pull request #62449 from zdover23/wip-doc-2025-03-23-backport-62366-to-squid

squid: doc/radosgw: Improve layout.rst

doc/radosgw: Improve layout.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit a7e2fab008519416d5bd91f2163ea51e82c440af)

Merge pull request #62443 from zdover23/wip-doc-2025-03-22-backport-62422-to-squid

squid: doc/cephadm/services: Add mention of --zap for OSD removal

doc/cephadm/services: Add mention of --zap for OSD removal

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 20b621fceb865403e19e1d7876f81e670829ff59)

Merge pull request #62427 from zdover23/wip-doc-2025-03-21-backport-62426-to-squid

squid: doc/cephadm/services: Correct indentation in osd.rst

doc/cephadm/services: Correct indentation in osd.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 7b5f73f4c17acc7cd0aab87db528e5232c8f05e1)

Merge pull request #61832 from idryomov/wip-69911-squid

squid: librbd: fix mirror image status summary in a namespace

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>

mon, osd: add command to remove invalid pg-upmap-primary entries

The current rm-pg-upmap-primary command checks that the pgid exists
in the pgmap before continuing to remove it. Due to https://tracker.ceph.com/issues/66867,
some invalid pg-upmap-primary entires may exist for pools that have been removed.
Currently, these mappings are impossible to remove since the pgids no longer
exist in the pgmap.

This new command, rm-pg-upmap-primary-all, allows users the ability to remove
any and all pg-upmap-primary mappings in the osdmap at once, which includes
valid and invalid entries.

This command may also be helpful when upgrading from versions where users
are plagued by https://tracker.ceph.com/issues/61948. Users may use an upgraded
mon to remove all pg-upmap-primray entries (valid and invalid) so they continue
to upgrade to a safe version.

See manual testing for this patch here: https://tracker.ceph.com/issues/67179#note-12

Fixes: https://tracker.ceph.com/issues/67179
Fixes: https://tracker.ceph.com/issues/69760
Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit 6e9e2033bf0f4779bdfac9a3a4f29115459c8c0e)

Merge pull request #62131 from cbodley/wip-70312-squid

squid: rgw: /admin/account APIs accept negative values

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #60980 from cbodley/wip-69145-squid

squid: radosgw-admin: 'user create' rejects uids matching the account id format

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #60979 from cbodley/wip-69146-squid

squid: radosgw-admin: allow 'sync group pipe modify' with existing user

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #60952 from igomon-bloomberg/wip-65912-squid

squid: rgw/s3-notifications: use user-name/password topic attributes for SASL authentication

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #60783 from jzhu116-bloomberg/wip-68983-squid

squid: rgw/lc: delete expired delete-marker when processing Expiration lc action with Days specified

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #62154 from cbodley/wip-70340-squid

squid: rgw: use object ARN for InitMultipart permissions

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #62404 from zdover23/wip-doc-2025-03-20-backport-62394-to-squid

squid: doc/radosgw /notifications: fix topic details

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw /notifications: fix topic details

This commit fixes documentation about many-to-many topic relationship for notifications. The current sentence states the same fact twice instead of clarifying.

Fixes: https://tracker.ceph.com/issues/64956
Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>
(cherry picked from commit 058b95abe62893c5c13f02cdf7dd438b0a95a128)

Merge pull request #62137 from adamemerson/wip-68836-squid

[CVE-2024-48916] rgw/sts: fix to disallow unsupported JWT algorithms

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62134 from kchheda3/wip-70311-squid

squid: rgw/log: Fix crash during shutdown with ops-log enable.

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62093 from cbodley/wip-70264-squid

squid: rgw/rados: enable object deletion at rados pool quota

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62064 from cbodley/wip-70239-squid

squid: rgw: PutObjectLockConfiguration can enable object lock on existing buckets

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #62033 from cbodley/wip-70207-squid

squid: qa/rgw: configure 'iam root' accounts outside of rgw/verify

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #62340 from phlogistonjohn/wip-70507-squid

squid: build-with-container: two small fixes

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #62379 from zdover23/wip-doc-2025-03-19-backport-62367-to-squid

squid: doc/cephadm: Add PG autoscaler advice to upgrade.rst

mgr/dashboard: add xlmtodict import and fix lifecycle get request

Fixes: https://tracker.ceph.com/issues/70128
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 567c9e76674f490cdb88f77545da197d54e5fe85)

 Conflicts:
ceph.spec.in
debian/control
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-details/rgw-bucket-details.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-details/rgw-bucket-details.component.ts
src/pybind/mgr/dashboard/services/rgw_client.py
src/pybind/mgr/tox.ini

accepted incoming and removed jmespath dependency

Merge PR #62095 into squid

* refs/pull/62095/head:
include/cephfs: dump optmetadata
mds: set alternate_name for new fullbit dentries
qa: test alternate_name recovery during mds failover
client: avoid multiple calls to path_walk
libcephfs: getcwd after chdir for API constraint
client: unwrap dentries for getcwd
client: refactor / optimize chdir
test/libcephfs: test getcwd with case insensitive dir
test/libcephfs: refactor ManyNestedDirs
include/filepath: add iterators for components
cmake/cephfs: fix options to enable client and dependencies
pybind/cephfs: use legacy noexcept for cdefs for cython 3.Y.Z
pybind/cephfs: increment ref before calling out to c++
client: skip unexpected command replies
mgr: indicate map message is acked instead of unhandled
osdc/Objecter: convert to ms_dispatch2 for ack
client: indicate maps are acked not processed
msg: add alternate statuses for ms_dispatch2 handling
tools/cephfs_mirror: do not process maps with fast dispatch
doc: add docs for volumes interface for charmap
qa: add tests for subvolume charmap settings
pybind/mgr/volumes: wire up charmap for subvol/subvolgroup
pybind/mgr: send MDS commands through cephfs client
pybind/cephfs: wire up mds_command2
mgr: add module method to send notifications
libcephfs: add mds_command2 for asynchronous commands
mgr: excise CephFS client from mgr C++ base
mgr: use std namespace
doc: add docs for CephFS charmap config
qa: add charmap tests
qa: add helpful exceptions for attr changes
qa: ignore libicu leaks
client: add wrappings for charmap manipuluation of dentry names
client: add dir_result_t::dentry::print
win32: add libicu Windows build
CMakeLists: add boost::locale dependency for client
install-deps: unconditionally install boost libraries
test/libcephfs: update root operation return values
client: refactor all path traversals through path_walk
test/libcephfs: test parallel creates
test/libcephfs: add test for lookup failure after readdir
client: init dentry shared_gen with invalid value
client: add _lookup debugging
client: remove redundant check
client: dump InodeStat from mds
mds: encode optmetadata in InodeStat sent to clients
mds: check client features for charmap
mds: add client feature bit for charmap
mds: wire up vxattr for changing charmap
mds: inherit charmap on mkdir
mds,include: add charmap optmetadata
mds,include: add inode_t optional metadata
client: hide alternate_name from API
client: move alternate_name once
client: optimize alternate_name passing to helper
client: relocate definition
client: print dentry with alternate_name on dump
client: move inode dump to print method
mds: add debugging for encoding lease stat
mds: make encode_lease a proper method
mds: add fscrypt metadata for inode stat size
client: use DentryRef for ref counting in MetaRequest
client: add DentryRef
client: add helper for determining if a perm check is necessary
client: cache client_permissions config
client: add debugging for conf changes
client: sort configs
client/UserPerm: add print method
client: note mount parameters in debug log
client: print stat mode in octal
common: add missing op string
include/filepath: add empty path check
python-common: add a utils function to replace distutils.util.strtobool

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>

Merge pull request #62384 from aaSharma14/wip-70546-squid

squid: mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in pool.libsonnet to fix inconsistency in Line Graphs

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #62383 from aaSharma14/wip-70548-squid

squid: mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in osd.libsonnet to fix inconsistency in Line Graphs

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #62382 from aaSharma14/wip-70547-squid

squid: mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in host.libsonnet to fix inconsistency in Line Graphs

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #62381 from aaSharma14/wip-70549-squid

squid: mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in cephfs.libsonnet to fix inconsistency in Line Graphs

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in pool.libsonnet

Fixes:- https://tracker.ceph.com/issues/70229
Signed-off-by: Piyush Agarwal <piyushagarwal14.pa@gmail.com>
(cherry picked from commit 09807250719b566c3c20fe57d2f323eb81b38ee5)

mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in osd.libsonnet

Fixes:- https://tracker.ceph.com/issues/70227
Signed-off-by: Piyush Agarwal <piyushagarwal14.pa@gmail.com>
(cherry picked from commit 171954e7dd677d1b73db65555736d82cefefc62e)

mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in host.libsonnet

Fixes:- https://tracker.ceph.com/issues/70228
Signed-off-by: Piyush Agarwal <piyushagarwal14.pa@gmail.com>
(cherry picked from commit f155abe51879f219b2dc944851ef1bc5f4577fe9)

mgr/dashboard: Changing SimpleGraphPanel to TimeSeries Panel in cephfs.libsonnet

Tracker:- https://tracker.ceph.com/issues/69404
Signed-off-by: Piyush Agarwal <piyushagarwal14.pa@gmail.com>
(cherry picked from commit 1772dc1cdf0a5fd47bf4954c343b18f70d357304)

doc/cephadm: Add PG autoscaler advice to upgrade.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit ee69f52193996d528410a71c2c67fc27b47dee31)

common/options: fix the description of osd_max_scrubs

osd_max_scrubs takes effect even though mclock is used.

```
$ ceph config show-with-defaults osd.0 | grep '^osd_max_scrubs' | sed -E -e 's/( +)/ /g'
osd_max_scrubs 3 default
$ ceph config show-with-defaults osd.0 | grep '^osd.* override ' | sed -E -e 's/( +)/ /g'
osd_delete_sleep 0.000000 override
osd_delete_sleep_hdd 0.000000 override
osd_delete_sleep_hybrid 0.000000 override
osd_delete_sleep_ssd 0.000000 override
osd_pg_delete_cost 15728640 override
osd_recovery_sleep 0.000000 override
osd_recovery_sleep_hdd 0.000000 override
osd_recovery_sleep_hybrid 0.000000 override
osd_recovery_sleep_ssd 0.000000 override
osd_scrub_sleep 0.000000 override
osd_snap_trim_sleep 0.000000 override
osd_snap_trim_sleep_hdd 0.000000 override
osd_snap_trim_sleep_hybrid 0.000000 override
osd_snap_trim_sleep_ssd 0.000000 override
```

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit 153639ca7f47e58b00377d86ba6baadc45431a48)

Merge pull request #62353 from aaSharma14/wip-70524-squid

squid: mgr/dashboard: When configuring the RGW Multisite endpoints from the UI allow FQDN(Not only IP)

Reviewed-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #62375 from zdover23/wip-doc-2025-03-19-backport-62371-to-squid

squid: doc/dev/developer_guide/essentials: update mailing lists

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev/developer_guide/essentials: update mailing lists

Update information for subscribing to Ceph development mailing lists as current documentation is outdated.

Fixes: https://tracker.ceph.com/issues/64580
Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>
(cherry picked from commit e7bf607269335ac40d91cb4b8f265064ffaac402)

dmclock/.../dmclock_server: do not clean clients with requests

PriorityQueueBase::do_clean() shouldn't remove ClientRec instances which
still have queued requests.  Otherwise, very low priority clients might
end up having requests actually lost, which shouldn't be possible.

In the OSD, this resulted in PGRecovery items being lost if queued with
background_best_effort while expanding a cluster.  Such items can
legitimately sit in the queue for a long period of time as they
represent background data migration which is allowed to be starved by an
aggressive client workload.  Dropping the items broke an assumption in
the OSD that all items enqueued would eventually be dequeued resulting
in resources being leaked.

Fixes: https://tracker.ceph.com/issues/61594
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 35cf0f6c1355ff420c431340ac36d9a3c029d6c2)

test/osd/TestMClockScheduler: TestMultiClientOrderedEnqueueDequeue use client class

Otherwise, scheduler will yield a delay eventually.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 0938a592e922c284cb939561dd1bd1589e154ae3)

test/osd/TestMClockScheduler: create_item should pass prio < cutoff

Cutoff is set to 12, so let's pass something < 12 rather than 12.
Comments in some tests suggest that the intent is for create_item
to create things in the mclock queue rather than the high_queue.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 5562ecbcdb1b0ae43145cb7ea9c7a3c76e6449f4)

test/osd/TestMClockScheduler: add test for very slow dequeue

Related: https://tracker.ceph.com/issues/61594
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit b35589f7eb39e6bfabe7df1c55281f41925eca61)

osd/scheduler/mClockScheduler: add constructor to specify idle/erase/check times

Useful for unit test, default times are *really* long.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8b7b0463792d85575996e65752087a25c9a3a65a)

osd/scheduler/mClockScheduler: move constructor to header

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 35b6176d90e1c48ba899b9ed0fd807ad8b290e0d)

src/dmclock: remove superfluous license files

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit c95539ec3704c2372b4152bc8af5223a2fe7c55d)

src/dmclock: copy dmclock submodule into ceph.git

Copies dmclock.git at e4ccdcfa828 into ceph.git.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit c625008dae9fb067e5f567031d5c8c355f2672c2)

Merge pull request #61891 from k0ste/wip-70067-squid

squid: os/bluestore: fix the problem that _estimate_log_size_N calculates the log size incorrectly

mgr/dashboard: When configuring the RGW Multisite endpoints from the UI allow FQDN(Not only IP)

When configuring the RGW Multisite endpoints from the UI allow FQDN, at the moment when using a FQDN it's not allowed

Fixes: https://tracker.ceph.com/issues/69055
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 9f3619af9ae911955916195084d225928d4b2f43)

 Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json (conflicts
with typescript package version, kept the existing one)
src/pybind/mgr/dashboard/frontend/package.json (conflicts with
typescript package version, kept the existing one)
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-multisite-migrate/rgw-multisite-migrate.component.ts (conflicts with automated system user creation in main)
src/pybind/mgr/dashboard/frontend/src/app/shared/forms/cd-validators.ts (conflicts with oauthAddressTest validator)

include/cephfs: dump optmetadata

For inspection from `dump tree` MDS command.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 7ad1937547d0fe8c5302cd822d6cad57bdc6140f)

mds: set alternate_name for new fullbit dentries

Fixes: 9fbf37ade38bc1d057eda2459b0ccfd2ab40626a
Fixes: https://tracker.ceph.com/issues/70409
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 5fc94be121f8f99d0d46a2e7b41a05a9956e2fce)

Conflicts:
src/mds/journal.cc: code movement caused by referent inode changeset

qa: test alternate_name recovery during mds failover

Fixes: https://tracker.ceph.com/issues/70409
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit db2784581ecefd3c1a7042915e1549c4e87e3097)

client: avoid multiple calls to path_walk

When we wrap a dname after a path walk, we cannot do another path walk.
Originally this method was setup to permit that but with (upcoming) encryption
it cannot work. The dname from the path_walk is in the encrypted namespace so
we cannot do another lookup in a second path_walk.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit af8d267e5f8123eaf3cb311b7d608025468429f9)

libcephfs: getcwd after chdir for API constraint

libcephfs semantics require that the old path to the cwd be returned when
getcwd is encounters an unlinked directory in the current working directory.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit c7c5089ff95d6c68e009686bf4ff74772ad4ff67)

client: unwrap dentries for getcwd

This was missed in the path_walk refactor. readdir is not the only way to "get"
dentry names.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit cda2a6e6facb24eba96afb51341720336febdd54)

client: refactor / optimize chdir

In particular: there's no reason to do a getcwd after chdir.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 3bfd614a358dc664469f8d5f56e9248818495bb2)

test/libcephfs: test getcwd with case insensitive dir

Unfortunately, it's not easy to refactor this test into a shared method without
setting up an explicit test class which has been avoided up to this point. So
I'm going to just copy the code. Sorry.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 28bdb8e565f0b6b3090808f4759c70693d92eb05)

test/libcephfs: refactor ManyNestedDirs

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit d62a3795902ab5b61d7b4f605fd170ac9e78c639)

include/filepath: add iterators for components

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit a33faa7c413c4b166b267a1d4892d0ffdf01490c)

cmake/cephfs: fix options to enable client and dependencies

invoking cmake with -DWITH_LIBCEPHFS=OFF fails to configure the client target:

CMake Error at src/client/CMakeLists.txt:13 (target_link_libraries):
  Target "client" links to:

    Boost::locale

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

because the client target is not conditional on WITH_LIBCEPHFS in src/CMakeLists.txt:

add_subdirectory(client)

if(WITH_LIBCEPHFS)

because client is also needed for ceph-fuse, make the client and its
dependencies depend on WITH_LIBCEPHFS OR WITH_FUSE

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 70eb1d64ac1ea6e7202d42571f83c34d3758ad8f)

pybind/cephfs: use legacy noexcept for cdefs for cython 3.Y.Z

For some newer versions of cython, it appears it requires explicitly specifying
noexcept but old versions of Cython 0.29.Z do not understand that attribute.

See: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 90ac7409279d103f8811cf31f3a530d8fff7ba12)

pybind/cephfs: increment ref before calling out to c++

At the time this construction seemed safe since the caller should have a
reference but it could conveivably be the only ref. We don't want the ref count
to reach 0.

Additionally, catch errors so this callback is genuinely noexcept.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 395263ceb2f351d540359f52694b48b2824e1291)

client: skip unexpected command replies

Instead of marking the message as handled, give another component (or Client) a
chance to process.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 616fdd1ae394195a2628fef6f6808e035b71069d)

mgr: indicate map message is acked instead of unhandled

This avoids messages like:

    2025-02-18T05:31:17.738+0000 7f5206546640  0 ms_deliver_dispatch: unhandled message 0x5632d05f0700 fsmap(e 9) from mon.0 v2:172.21.3.230:40412/0

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 56cb05775a937159ee0e6e26e30fe204accb5223)

osdc/Objecter: convert to ms_dispatch2 for ack

Convert ms_dispatch to ms_dispatch2 to enable indicating that a map message is
acknowledged and instead of processed (or deliberately not processed).

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit f060ee867f5423fd040f658877ad7ef9f57e9ba6)

client: indicate maps are acked not processed

Ancillary change: do not do client upkeep after map processing.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 2810ed0f60833d1c1806d2c84db3116b11f72e37)

msg: add alternate statuses for ms_dispatch2 handling

Many dispatchers return false to allow other dispatchers also common messages
like MOSDMap or MFSMap. They implicitly depend on some dispatcher which is
always at the "tail" of the dispatcher queue to return "true" indicating the
msg was processed to avoid messages like:

    2025-02-18T05:31:17.738+0000 7f5206546640  0 ms_deliver_dispatch: unhandled message 0x5632d05f0700 fsmap(e 9) from mon.0 v2:172.21.3.230:40412/0

but this cannot always happen when some libraries like the RadosClient used standalone.

So, add a variant for encapsulating other indications for how the message was
processed by dispatch2.  For example, a message may be "acknowledged" but
explicitly allow other dispatchers to try processing the message.

Note: we're using a variant to avoid updating all of the ms_dispatch code to
use the sentinel classes.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit c9d0913f53b0c4d632746075c05c2f2187d77c02)

Conflicts:
src/nvmeof/NVMeofGwMonitorClient.cc: not present in squid
src/nvmeof/NVMeofGwMonitorClient.h: not present in squid

tools/cephfs_mirror: do not process maps with fast dispatch

This acquires a lock which is a no-no in the messenger.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 3b79bdbee04c270c227be3dd1cb6dfe392646f8d)

doc: add docs for volumes interface for charmap

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 404d7e3278119149660d8aa1a12bac6ce6396118)

qa: add tests for subvolume charmap settings

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 71c57121cf62b2b8b576e5d21bb71bdbccb6960d)

pybind/mgr/volumes: wire up charmap for subvol/subvolgroup

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit a6eeec80d77366fef71f1e13c037a3095823cbf5)

pybind/mgr: send MDS commands through cephfs client

To avoid linking to the CephFS client statically, use the dynamically-linked
`cephfs` module to send commands to the MDS.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit f7329814c917f290df54807d16a31b3108223b9f)

pybind/cephfs: wire up mds_command2

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 72d0a76e8c5cbb357e4f4c355e8630ec8aa02088)

mgr: add module method to send notifications

For use by MgrModule.send_command to signal MDS command completion.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit b3ec7385ab2755fa3fa98ab98ed03257c93d315c)

libcephfs: add mds_command2 for asynchronous commands

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 62601ef64202a3514cd805936535671eee836448)