]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
17 months agoqa: simplify run_shell argument processing
Patrick Donnelly [Thu, 7 Mar 2024 03:21:24 +0000 (22:21 -0500)]
qa: simplify run_shell argument processing

No functional changes.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c3707b99c38ab153149684488276c76be6186d8f)

17 months agodoc: add dev docs for quiesce protocol
Patrick Donnelly [Thu, 11 Jan 2024 03:08:13 +0000 (22:08 -0500)]
doc: add dev docs for quiesce protocol

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 51c9b86bf1f7be41f568ef1594b305133cd8b118)

17 months agopybind/mgr/volumes: block quiesce for critical .meta file
Patrick Donnelly [Fri, 26 Jan 2024 21:37:15 +0000 (16:37 -0500)]
pybind/mgr/volumes: block quiesce for critical .meta file

This file must remain accessible during quiesce to allow snapshots to be taken
on a subvolume.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0308f215ae7a6392de4dbd658c6d4334c1c98e9d)

17 months agomds: add vxattr to block quiesce on an inode
Patrick Donnelly [Fri, 26 Jan 2024 21:36:53 +0000 (16:36 -0500)]
mds: add vxattr to block quiesce on an inode

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e2529fc74808b102b1ca85a3ee6644160da32e60)

17 months agomds: convert encoded ephemeral dist pin to flags
Patrick Donnelly [Fri, 26 Jan 2024 21:34:41 +0000 (16:34 -0500)]
mds: convert encoded ephemeral dist pin to flags

To use space efficiently, convert this field to a 8 bit flags field that is
backward compatible.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fb05dd1806fc0edf2e4c0908aae114d57099a6d6)

17 months agomds: add counter to throttle quiesce
Patrick Donnelly [Wed, 21 Feb 2024 15:03:14 +0000 (10:03 -0500)]
mds: add counter to throttle quiesce

So a storm of quiesce operations do not affect normal MDS operations.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f4eec857fc8bda4980b9046cde6314fa005122c2)

17 months agomds: add quiesce set feature flag
Patrick Donnelly [Tue, 31 Oct 2023 17:27:16 +0000 (13:27 -0400)]
mds: add quiesce set feature flag

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/63664
(cherry picked from commit af073d2fba48a80a533e2a1be2f21716d56aa26c)

17 months agomds: skip non-head inodes for quiesce
Patrick Donnelly [Wed, 21 Feb 2024 19:08:13 +0000 (14:08 -0500)]
mds: skip non-head inodes for quiesce

These are accessible only via snapshots.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 92b5bcb44c0eacbc4c449a418350c0ff0506b7b5)

17 months agomds: add quiesce op
Patrick Donnelly [Tue, 14 Nov 2023 18:17:57 +0000 (13:17 -0500)]
mds: add quiesce op

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/63664
(cherry picked from commit 06916dd2a7f30874d29b03e0cd998370cf595f58)

17 months agomds: print all SimpleLock flags in debug output
Patrick Donnelly [Tue, 27 Feb 2024 20:18:21 +0000 (15:18 -0500)]
mds: print all SimpleLock flags in debug output

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3980454db49cbd70c42a502b832262552d9c9c74)

17 months agomds: pretty print mutation when dumping lock
Patrick Donnelly [Wed, 10 Jan 2024 19:08:11 +0000 (14:08 -0500)]
mds: pretty print mutation when dumping lock

Now looks like:

    ... (iquiesce xlock x=1 by request(mds.0:2785 nref=6)) ...

instead of:

    ... (iquiesce xlock x=1 by 0xdeadbeef)) ...

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7c59c3749021e9113dfb777c197f70709f416ce6)

17 months agomds: add new inode quiescelock
Patrick Donnelly [Tue, 9 Jan 2024 19:04:45 +0000 (14:04 -0500)]
mds: add new inode quiescelock

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a3e541f9992e5e264a8d9ecffd69787d722f88a0)

17 months agomds: use 128 bits for waiters on MDSCacheObject
Patrick Donnelly [Tue, 20 Feb 2024 22:08:32 +0000 (17:08 -0500)]
mds: use 128 bits for waiters on MDSCacheObject

Adding a new inode lock will overflow inode wait bits into the MDSCacheObject
wait bits. Make space for the quiescelock.

This includes a minor refactor to no longer attempt scoping the set of masks we
test in MDSCacheObject::waiting when calling MDSCacheObject::is_waiter_for.
This optimization wasn't worth the overhead and would be awkard to keep as
std::bitset cannot be used as a key for a std::multimap (easily).  Instead, we
use the sequence number as a key which helps us to avoid allocating another map
whenever we call MDSCacheObject::take_waiting.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 84b33ea9fe6268adfaca1e17b282e68788e38bf4)

17 months agomds: provide mechanism to authpin while freezing
Patrick Donnelly [Tue, 13 Feb 2024 16:07:26 +0000 (11:07 -0500)]
mds: provide mechanism to authpin while freezing

When a subtree is freezing, it's no longer possible to acquire new authpins.
This is a problem when a compound request like quiescing a subtree is trying to
acquire authpins for each sub-op. This creates a situation where some quiesce
sub-ops complete with authpins (thereby preventing the tree from becoming
"frozen") and new sub-ops cannot acquire authpins (because the tree is
"freezing"). To circumvent this, allow some authpin requests to proceed if
FLAG_BYPASSFREEZING is set.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a9964a7ccc4394f923fb0f1c76eb8fa03fe8733d)

17 months agomds: add command to get specific op
Patrick Donnelly [Tue, 9 Jan 2024 18:55:48 +0000 (13:55 -0500)]
mds: add command to get specific op

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3197a975a521654500173175c6e856c54df497ef)

17 months agomds: finish request before completing internal req
Patrick Donnelly [Tue, 13 Feb 2024 21:07:37 +0000 (16:07 -0500)]
mds: finish request before completing internal req

Importantly, this marks the request dead so it cannot be killed.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c944eadaf6987a812e738459214872af1ff7678a)

17 months agomds: complete internal op if killed
Patrick Donnelly [Tue, 14 Nov 2023 18:50:53 +0000 (13:50 -0500)]
mds: complete internal op if killed

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ba3dac94e6a90ed76970fef89b1a82dcedb4ec2b)

17 months agomds: avoid killing dead requests
Patrick Donnelly [Tue, 13 Feb 2024 20:58:19 +0000 (15:58 -0500)]
mds: avoid killing dead requests

This is important when you have:

* sub-op A finishes and resumes parent
* parent kills all sub-ops (including sub-op A)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b72e153d2b36956e97b8686aeb6aaacffe928750)

17 months agomds: add command to kill request
Patrick Donnelly [Mon, 13 Nov 2023 19:23:54 +0000 (14:23 -0500)]
mds: add command to kill request

Fixes: https://tracker.ceph.com/issues/63374
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9956f9ad78a3b6ebe520dcd10e33a59c361c4d91)

17 months agomds: add path argument to `ops` and `dump tree` to stream result to local file
Patrick Donnelly [Sat, 17 Feb 2024 15:23:43 +0000 (10:23 -0500)]
mds: add path argument to `ops` and `dump tree` to stream result to local file

This file can be collected during testing to avoid sending a large JSON result
over the `ceph tell`/asok interface.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a1303a52af7504c3748a75ade9619c7a6c454831)

17 months agomds: print internal_request filepaths if present
Patrick Donnelly [Wed, 31 Jan 2024 19:28:14 +0000 (14:28 -0500)]
mds: print internal_request filepaths if present

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit eaea93e05018294e893e0e5241b3293e10971f5e)

17 months agomds: add more information to debug message
Patrick Donnelly [Tue, 14 Nov 2023 18:19:47 +0000 (13:19 -0500)]
mds: add more information to debug message

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f8a26a61305ba8b9fbb9ce47b5221e3adb754c9b)

17 months agomds: remove redundant parenthesis
Patrick Donnelly [Sat, 14 Oct 2023 00:27:02 +0000 (20:27 -0400)]
mds: remove redundant parenthesis

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 1513327c1a55614ac7e2f1a6da0de06e5fd2214d)

17 months agomds: implement Mutation::dump method
Patrick Donnelly [Tue, 14 Nov 2023 14:38:03 +0000 (09:38 -0500)]
mds: implement Mutation::dump method

So we can dump it from asok commands.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ba7d1f5f2a2e4355155e00173b0d2ade562a8945)

17 months agomds: make LockType fields const
Patrick Donnelly [Tue, 9 Jan 2024 04:53:24 +0000 (23:53 -0500)]
mds: make LockType fields const

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9858e01abd57458b04f87fcaeb432ab7edb6ac07)

17 months agomds: annotate mdr with try_rdlock_snap_layout failure
Patrick Donnelly [Tue, 9 Jan 2024 19:06:39 +0000 (14:06 -0500)]
mds: annotate mdr with try_rdlock_snap_layout failure

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6fa38e4e4ff230ca2a1c6d2de63d561d7cfee826)

17 months agomds: refactor if into switch
Patrick Donnelly [Tue, 9 Jan 2024 19:01:17 +0000 (14:01 -0500)]
mds: refactor if into switch

To make it easier to read and match other constructs in this code.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d74d3aadd08ad37a74bf3a9ade18d0b6670c92a0)

17 months agomds: call Locker method using this
Patrick Donnelly [Wed, 10 Jan 2024 02:56:20 +0000 (21:56 -0500)]
mds: call Locker method using this

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 401e6f49fc2e1c725c8e89e23cdc8afcf5f92876)

17 months agomds: simplify assert
Patrick Donnelly [Thu, 4 Jan 2024 17:24:29 +0000 (12:24 -0500)]
mds: simplify assert

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 180979d7014a6c3799bb7282d90d6e6af860978a)

17 months agomds: dump locks passed to Locker::acquire_locks
Patrick Donnelly [Tue, 16 Jan 2024 19:03:29 +0000 (14:03 -0500)]
mds: dump locks passed to Locker::acquire_locks

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3057ec9572dd2c1805db09ffd5eaac8ab40a45b2)

17 months agomds: add LockOp::print method for debugging
Patrick Donnelly [Tue, 16 Jan 2024 19:02:41 +0000 (14:02 -0500)]
mds: add LockOp::print method for debugging

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8ddf631886ce4ac34f74e9e62eacc2c33c6e445f)

17 months agomds: use new insert template via print
Patrick Donnelly [Tue, 16 Jan 2024 19:01:54 +0000 (14:01 -0500)]
mds: use new insert template via print

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d1b066cdbd1ec65b620996cb7f4da7006dfcb7d0)

17 months agomds: add request result to mutation for analysis by tests
Patrick Donnelly [Wed, 24 Jan 2024 02:27:18 +0000 (21:27 -0500)]
mds: add request result to mutation for analysis by tests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d1765e621611fef5517c2f2227d7786ae41e2001)

17 months agomds: add comment on locking order rules
Patrick Donnelly [Fri, 5 Jan 2024 20:06:20 +0000 (15:06 -0500)]
mds: add comment on locking order rules

From a past commit.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 32a544f9a71aa68547db5bb0fdd517cafbf7d928)

17 months agomds: allow specifying rdlock position
Patrick Donnelly [Thu, 8 Feb 2024 17:30:38 +0000 (12:30 -0500)]
mds: allow specifying rdlock position

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 944d00ca1fcafb39e45cb431c63f00e71a7976df)

17 months agomds: remove dead method
Patrick Donnelly [Wed, 7 Feb 2024 19:59:06 +0000 (14:59 -0500)]
mds: remove dead method

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ed5987883a411d090d1820ef00453e8147bc1753)

17 months agocommon: provide a template for object dumps
Patrick Donnelly [Tue, 16 Jan 2024 18:59:47 +0000 (13:59 -0500)]
common: provide a template for object dumps

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 92df0fcc649345f438d0d7cc6fab658c02060f07)

17 months agocommon: support long running ops without slow warnings
Patrick Donnelly [Thu, 16 Nov 2023 17:36:25 +0000 (12:36 -0500)]
common: support long running ops without slow warnings

A quiesce op in the MDS may run for an indeterminate amount of time. Warnings
should not be generated.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c036b151465a2ecb8bf24943e1030988f7e6c6ce)

17 months agocommon: simplify loop
Patrick Donnelly [Thu, 16 Nov 2023 17:07:32 +0000 (12:07 -0500)]
common: simplify loop

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit cdef9b6dd0a4d0fe39843c5a8198295d14b1c2e7)

17 months agocommon: add JSONFormatterFile class
Patrick Donnelly [Sat, 17 Feb 2024 15:22:29 +0000 (10:22 -0500)]
common: add JSONFormatterFile class

To stream JSON to a file for collection (for testing).  The idea here that
trying to send a very large JSON result over `ceph tell` is best to be avoided.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8928a1266fc6bc492dac34253edb36c045e491e5)

17 months agocommon: use more efficient vector for stack
Patrick Donnelly [Sat, 17 Feb 2024 01:43:06 +0000 (20:43 -0500)]
common: use more efficient vector for stack

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 770ab517c2b05045c09af3d0a56b6446bd20bb58)

17 months agoinclude: use larger int for large gathers
Patrick Donnelly [Tue, 14 Nov 2023 19:15:14 +0000 (14:15 -0500)]
include: use larger int for large gathers

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 58923668320ba373312c68a8869a1089da41d804)

17 months agoMerge PR #56202 into squid
Patrick Donnelly [Fri, 22 Mar 2024 15:36:40 +0000 (11:36 -0400)]
Merge PR #56202 into squid

* refs/pull/56202/head:
mds/quiesce-db: keep the db thread alive until shutdown
mds/quiesce-db: incorporate review comments
mds/quiesce: declare QuiesceDbPeerListing and QuiesceDbPeerAck
mds/quiesce: resolve the quiesce cluster at the mds monitor
include/types: add an I/O helper for std::unordered_map
messages: avoid using mutable members in MMDSQuiesce*
mds/quiesce-db: incorporate review comments
doc/cephfs/fs-volumes: doc fixes and updates
pybind/mgr: correct type hints for `get_quiesce_leader_info`
mds/quiesce: only use ACTIVE daemons for the quiesce cluster
mds,messages: quiesce db inter-rank messaging
mds/quiesce: MDSRankQuiesce - integration of the quiesce db manager
doc/cephfs/fs-volumes: Add info about the quiesce command
doc: fixes for local dev builds
mgr/volumes: support for `fs subvolume quiesce`
mgr/volumes: use `volume_exception_to_retval` as a decorator
pybind/mgr: add a `one-shot` parameter to send_command
mds/quiesce: QuiesceAgent implementation and unit tests
mds/quiesce: QuiesceDb.h and QuiesceDbManager with tests
common/Timer.cc: improve debug messages from the timer_thread
mds: MDSRank.cc: return status from `send_message_mds`
encoding: add emplace variants for map dencoders
common/Cond: make C_SaferCond private members protected to facilitate inheritance
qa/tasks/cephfs: give the tests more time to run heavy fs workloads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
17 months agoMerge pull request #56365 from rhcs-dashboard/wip-65037-squid
Nizamudeen A [Fri, 22 Mar 2024 10:14:11 +0000 (15:44 +0530)]
Merge pull request #56365 from rhcs-dashboard/wip-65037-squid

squid: mgr/dashboard: fix M retention frequency display

Reviewed-by: Nizamudeen A <nia@redhat.com>
17 months agoMerge pull request #56371 from rhcs-dashboard/wip-65047-squid
Ivo Almeida [Fri, 22 Mar 2024 09:41:11 +0000 (09:41 +0000)]
Merge pull request #56371 from rhcs-dashboard/wip-65047-squid

squid: mgr/dashboard: fix retention add for subvolume

Reviewed-by: afreen23 <NOT@FOUND>
17 months agomgr/dashboard: fix retention add for subvolume 56371/head
Ivo Almeida [Wed, 21 Feb 2024 13:02:19 +0000 (13:02 +0000)]
mgr/dashboard: fix retention add for subvolume

- Added parameters for subvolume and subvolume group when adding a new
snap schedule.
- Added call to remove retention policies when removing a snap schedule
  in case it is the last one with same path

Fixes: https://tracker.ceph.com/issues/64524
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit 80e1207f4b536fe6edbc81e61cbf951e135eba54)

17 months agomgr/dashboard: fix M retention frequency display 56365/head
Ivo Almeida [Tue, 19 Mar 2024 13:13:17 +0000 (13:13 +0000)]
mgr/dashboard: fix M retention frequency display

Fix snap schedule list monthly retention frequency display

Fixes: https://tracker.ceph.com/issues/64982
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit 0bd8d3b6bc8ecbe99e22545549a63de3444e5f75)

17 months agoMerge pull request #56339 from zdover23/wip-doc-2024-03-20-backport-56333-to-squid
Zac Dover [Thu, 21 Mar 2024 02:03:17 +0000 (12:03 +1000)]
Merge pull request #56339 from zdover23/wip-doc-2024-03-20-backport-56333-to-squid

squid: doc/cephfs: fix architecture link to correct relative path

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agoMerge pull request #56267 from zdover23/wip-doc-2024-08-19-backport-56240-to-squid
Zac Dover [Thu, 21 Mar 2024 02:02:53 +0000 (12:02 +1000)]
Merge pull request #56267 from zdover23/wip-doc-2024-08-19-backport-56240-to-squid

squid: doc/rados: update common.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agoMerge pull request #56210 from joscollin/wip-64918-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:55:46 +0000 (08:55 -0700)]
Merge pull request #56210 from joscollin/wip-64918-squid

squid: qa: enhance labeled perf counters test for cephfs-mirror

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #56051 from lxbsz/wip-64585
Yuri Weinstein [Wed, 20 Mar 2024 15:54:03 +0000 (08:54 -0700)]
Merge pull request #56051 from lxbsz/wip-64585

squid: mds: allow lock state to be LOCK_MIX_SYNC in replica for filelock

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #56030 from vshankar/wip-64617-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:51:18 +0000 (08:51 -0700)]
Merge pull request #56030 from vshankar/wip-64617-squid

squid: mds: check file layout in mknod

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #56017 from vshankar/wip-64758-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:46:49 +0000 (08:46 -0700)]
Merge pull request #56017 from vshankar/wip-64758-squid

squid: mds: relax certain asserts in mdlog replay thread

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #55997 from dparmar18/wip-64741-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:46:08 +0000 (08:46 -0700)]
Merge pull request #55997 from dparmar18/wip-64741-squid

squid: client: do not proceed with I/O if filehandle is invalid

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #55989 from joscollin/wip-64744-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:45:32 +0000 (08:45 -0700)]
Merge pull request #55989 from joscollin/wip-64744-squid

squid: mds: fix `dump dir` indicate dir is not cached

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #55940 from neesingh-rh/wip-64701-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:44:50 +0000 (08:44 -0700)]
Merge pull request #55940 from neesingh-rh/wip-64701-squid

squid: mgr/volumes: support to reject CephFS clones if cloner threads are not available

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #55830 from kotreshhr/wip-64583-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:24:13 +0000 (08:24 -0700)]
Merge pull request #55830 from kotreshhr/wip-64583-squid

squid: qa: Fix fs/full suite

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #55805 from anoopcs9/wip-64566-squid
Yuri Weinstein [Wed, 20 Mar 2024 15:23:28 +0000 (08:23 -0700)]
Merge pull request #55805 from anoopcs9/wip-64566-squid

squid: client: Fix return in removexattr for xattrs from `system.` namespace

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agodoc/cephfs: fix architecture link to correct relative path 56339/head
molpako [Wed, 20 Mar 2024 09:43:42 +0000 (18:43 +0900)]
doc/cephfs: fix architecture link to correct relative path

Signed-off-by: molpako <35787116+molpako@users.noreply.github.com>
(cherry picked from commit ceeb1ca4b38dc307e493b04a59d5d07fed23971f)

17 months agoMerge pull request #56253 from rhcs-dashboard/wip-64929-squid
Pedro Gonzalez Gomez [Wed, 20 Mar 2024 07:47:25 +0000 (08:47 +0100)]
Merge pull request #56253 from rhcs-dashboard/wip-64929-squid

squid: mgr/dashboard: ceph authenticate user from fs

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
17 months agoMerge pull request #56229 from adk3798/squid-docs-typo-replace-osd
Adam King [Tue, 19 Mar 2024 20:55:36 +0000 (16:55 -0400)]
Merge pull request #56229 from adk3798/squid-docs-typo-replace-osd

squid: doc: cephadm/services/osd: fix typo

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>
17 months agoMerge pull request #55927 from ajarr/wip-64669-squid
Ilya Dryomov [Tue, 19 Mar 2024 13:47:57 +0000 (14:47 +0100)]
Merge pull request #55927 from ajarr/wip-64669-squid

squid: qa: add diff-continuous and compare-mirror-image tests to rbd and krbd suites respectively

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
17 months agoMerge pull request #56194 from vshankar/wip-64924-squid
Venky Shankar [Tue, 19 Mar 2024 13:21:28 +0000 (18:51 +0530)]
Merge pull request #56194 from vshankar/wip-64924-squid

squid: mds: disable `defer_client_eviction_on_laggy_osds' by default

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
17 months agoqa/suites: add diff-continuous and compare-mirror-image tests 55927/head
Ramana Raja [Thu, 29 Feb 2024 17:12:19 +0000 (12:12 -0500)]
qa/suites: add diff-continuous and compare-mirror-image tests

... to rbd and krbd suites respectively.

This allows the compare-mirror-image tests introduced in ea3a567
to be run against various kernel branches, e.g., testing branch.
And allows diff_continuous test in rbd_suite to run against distro
kernel.

Fixes: https://tracker.ceph.com/issues/64574
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 92b254138dd43755490392ffcf5ebca04872e116)

17 months agoqa/suites/rbd: rename nbd folder to device folder
Ramana Raja [Thu, 29 Feb 2024 16:55:08 +0000 (11:55 -0500)]
qa/suites/rbd: rename nbd folder to device folder

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit af43f6162464a20625319a2783a104f7032ba21e)

17 months agodoc/rados: update common.rst 56267/head
Zac Dover [Sun, 17 Mar 2024 09:27:11 +0000 (19:27 +1000)]
doc/rados: update common.rst

Clarify some overwritten paragraphs in
doc/rados/configuration/common.rst. Cf.
https://github.com/ceph/ceph/pull/50795/files#r1153897231.

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 233b670704d15c3a69b8d91885b60f3fb101ec58)

17 months agoMerge pull request #56261 from zdover23/wip-doc-2024-03-19-backport-56247-to-squid
Zac Dover [Mon, 18 Mar 2024 17:39:03 +0000 (03:39 +1000)]
Merge pull request #56261 from zdover23/wip-doc-2024-03-19-backport-56247-to-squid

squid: docs/rbd: fix typo in arg name

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
17 months agodocs/rbd: fix typo in arg name 56261/head
N Balachandran [Mon, 18 Mar 2024 04:02:39 +0000 (09:32 +0530)]
docs/rbd: fix typo in arg name

Replace "{image-}" with "{image-id}" in the "rbd trash rm"
command description.

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit f3eb489520fd4fae057e61275d16c6c8fd596f3f)

docs/rbd: replace introspect with inspect

Replace "introspect" with "inspect" in the rbd basic commands
description.

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit ebf2f60f784728c04d8ec59015d666bafcef8218)

docs/rbd: typo in "retrieving image information"

Replace "for the image" with "of the image".

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 4fd5c134536d652ae1f9e05ecf52cb81adb3b850)

17 months agoMerge pull request #56255 from zdover23/wip-doc-2024-03-18-backport-56248-to-squid
Anthony D'Atri [Mon, 18 Mar 2024 14:05:49 +0000 (10:05 -0400)]
Merge pull request #56255 from zdover23/wip-doc-2024-03-18-backport-56248-to-squid

squid: doc/rbd: minor changes to the rbd man page

17 months agodoc/rbd: minor changes to the rbd man page 56255/head
N Balachandran [Mon, 18 Mar 2024 12:22:47 +0000 (17:52 +0530)]
doc/rbd: minor changes to the rbd man page

Fixes typos and grammar for some commands. Adds
additional details for some commandds.

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 5dcff6a4b8d835fc55e454af977dc5ebad99d37f)

17 months agoMerge pull request #56251 from guits/wip-64931-squid
Guillaume Abrioux [Mon, 18 Mar 2024 10:38:23 +0000 (11:38 +0100)]
Merge pull request #56251 from guits/wip-64931-squid

squid: node-proxy: fix RedFishClient.logout() method

17 months agomgr/dashboard: ceph authenticate user from fs 56253/head
Pedro Gonzalez Gomez [Fri, 1 Mar 2024 12:43:12 +0000 (13:43 +0100)]
mgr/dashboard: ceph authenticate user from fs

Fixes: https://tracker.ceph.com/issues/64660
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 19a9f79b5f218d8cb757579bf8c0f5d610dde3a8)

17 months agonode-proxy: support more Location value formats 56251/head
Guillaume Abrioux [Fri, 15 Mar 2024 14:20:29 +0000 (14:20 +0000)]
node-proxy: support more Location value formats

After some tests, it turns out that depending on the hardware,
the header 'Location' which is returned by the server after logged can be different.
I could notice the following:

either:

Location: scheme://address:port/redfish/v1/SessionService/Session

or

Location: /redfish/v1/SessionService/Session

a previous tracker [1] was opened because I thought only the first one existed, which is wrong.

Fixes: https://tracker.ceph.com/issues/64951
[1] https://tracker.ceph.com/issues/64894

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit d7ccf26983c41344a12f33b2a30fc79b65cc548f)

17 months agonode-proxy: fix RedFishClient.logout() method
Guillaume Abrioux [Wed, 13 Mar 2024 13:32:59 +0000 (13:32 +0000)]
node-proxy: fix RedFishClient.logout() method

the endpoint passed down to util.query() is wrong:
is passes the full url (scheme://addr:port/path) where it should only
pass the path. The cause is that RedFishClient.login() basically stores
the value of the Location header in `self.location`.

The consequence of this is that it makes the client unable to properly logout.

Fixes: https://tracker.ceph.com/issues/64894
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit b1d828d1d2f31c02f225bb375d915353582d158a)

17 months agoMerge pull request #56125 from cbodley/wip-64834-squid
Casey Bodley [Sun, 17 Mar 2024 14:34:57 +0000 (14:34 +0000)]
Merge pull request #56125 from cbodley/wip-64834-squid

squid: RGW - Swift retarget needs bucket set on object

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
17 months agoMerge pull request #56144 from cbodley/wip-64876-squid
Casey Bodley [Sun, 17 Mar 2024 14:34:32 +0000 (14:34 +0000)]
Merge pull request #56144 from cbodley/wip-64876-squid

squid: rgw_lc:  replace strftime w/fmt and chrono:calendar

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
17 months agoMerge pull request #56156 from yuvalif/wip-64887-squid
Casey Bodley [Sun, 17 Mar 2024 14:34:08 +0000 (14:34 +0000)]
Merge pull request #56156 from yuvalif/wip-64887-squid

squid: rgw/kafka: set message timeout to 5 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
17 months agoMerge pull request #56181 from ivancich/wip-64909-squid
Casey Bodley [Sun, 17 Mar 2024 14:33:47 +0000 (14:33 +0000)]
Merge pull request #56181 from ivancich/wip-64909-squid

squid: rgw: rgw-restore-bucket-index -- sort uses specified temp dir

Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
17 months agoMerge pull request #56216 from idryomov/wip-63770-squid
Ilya Dryomov [Sat, 16 Mar 2024 18:41:57 +0000 (19:41 +0100)]
Merge pull request #56216 from idryomov/wip-63770-squid

squid: librbd: account for discards that truncate in ObjectListSnapsRequest

Reviewed-by: Ramana Raja <rraja@redhat.com>
17 months agoMerge pull request #55761 from ajarr/wip-64553-squid
Ilya Dryomov [Sat, 16 Mar 2024 18:07:05 +0000 (19:07 +0100)]
Merge pull request #55761 from ajarr/wip-64553-squid

squid: qa: Add tests to validate synced images on rbd-mirror

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
17 months agoMerge pull request #56234 from zdover23/wip-doc-2024-03-16-backport-56182-to-squid
Anthony D'Atri [Sat, 16 Mar 2024 01:32:47 +0000 (21:32 -0400)]
Merge pull request #56234 from zdover23/wip-doc-2024-03-16-backport-56182-to-squid

squid: doc/glossary: add "librados" entry

17 months agodoc/glossary: add "librados" entry 56234/head
Zac Dover [Thu, 14 Mar 2024 06:29:09 +0000 (16:29 +1000)]
doc/glossary: add "librados" entry

Add a "librados" entry to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2a93a8e837a42559f8a81c6fd9274b24f4fdf7f6)

17 months agodoc: cephadm/services/osd: fix typo 56229/head
Lorenz Bausch [Wed, 13 Mar 2024 16:26:51 +0000 (17:26 +0100)]
doc: cephadm/services/osd: fix typo

Signed-off-by: Lorenz Bausch <info@lorenzbausch.de>
(cherry picked from commit d513f1ca69d38f0d7eb40a2d68275caed322cc68)

17 months agoMerge pull request #55715 from cbodley/wip-64541-squid
Casey Bodley [Fri, 15 Mar 2024 14:27:23 +0000 (14:27 +0000)]
Merge pull request #55715 from cbodley/wip-64541-squid

squid: rgw: RGWSI_SysObj_Cache::remove() invalidates after successful delete

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
17 months agorgwlc: implement Filter size limit checks 56144/head
Matt Benjamin [Tue, 24 Oct 2023 21:59:34 +0000 (17:59 -0400)]
rgwlc: implement Filter size limit checks

i.e., this implements ObjectSizeGreaterThan and ObjectSizeLessThan
for supported expiration and transition actions.

Fixes: https://tracker.ceph.com/issues/63304
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit c9b19be15362c63526e92f4d968f942a40cab150)

17 months agorgwlc: implement NewerNoncurrentVersions
Matt Benjamin [Mon, 23 Oct 2023 18:57:33 +0000 (14:57 -0400)]
rgwlc: implement NewerNoncurrentVersions

Per AWS doc, this value controls "how many noncurrent versions
Amazon S3 will retain." [1]  We understand this to mean, retain
NewerNoncurrentVersions of any object, regardless of expiration.

Removed unused RGWLifecycleConfiguration::has_same_action() (cleanup)

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/API_NoncurrentVersionExpiration.html.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit a4313e77a31c5165254068b53dfac3d277b0a6a8)

17 months agoMerge pull request #56207 from zdover23/wip-doc-2024-03-15-backport-56188-to-squid
Zac Dover [Fri, 15 Mar 2024 11:03:48 +0000 (21:03 +1000)]
Merge pull request #56207 from zdover23/wip-doc-2024-03-15-backport-56188-to-squid

squid: doc/rbd: add clone mapping command

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
17 months agotest/librbd: add DiffIterateTruncate and DiffIterateWriteAndTruncate 56216/head
Ilya Dryomov [Tue, 5 Mar 2024 20:05:22 +0000 (21:05 +0100)]
test/librbd: add DiffIterateTruncate and DiffIterateWriteAndTruncate

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d84485ec534ccf42327242a9141906f4881c030e)

17 months agolibrbd: don't clip expanded diff on truncate in ObjectListSnapsRequest
Ilya Dryomov [Wed, 28 Feb 2024 13:20:16 +0000 (14:20 +0100)]
librbd: don't clip expanded diff on truncate in ObjectListSnapsRequest

If the diff was expanded due to LIST_SNAPS_FLAG_WHOLE_OBJECT, clipping
it when handling a truncate is wrong -- when subtracting that interval,
we either split the expanded extent into two or chop off a piece of it.
However the point of LIST_SNAPS_FLAG_WHOLE_OBJECT is to report a single
extent covering the entire object.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0ebca5fe22d976f1a17f8078f20a47c49a057edb)

17 months agolibrados/snap_set_diff: ignore truncates above size at start
Ilya Dryomov [Sun, 18 Feb 2024 10:46:15 +0000 (11:46 +0100)]
librados/snap_set_diff: ignore truncates above size at start

Because currently calc_snap_set_diff() only ever appends to the running
diff, an excessive (either too large or completely bogus) zero extent
is reported in cases where an object is first expanded (with a snapshot
taken at that point) and then truncated but still above the size of the
object as of the starting snapshot.

Fixes: https://tracker.ceph.com/issues/63770
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5b11fb314abefea390c5663c33e9e1ace6179974)

17 months agoMerge pull request #56155 from rhcs-dashboard/wip-64884-squid
Ivo Almeida [Fri, 15 Mar 2024 10:03:08 +0000 (10:03 +0000)]
Merge pull request #56155 from rhcs-dashboard/wip-64884-squid

squid: mgr/dashboard: fix snap schedule time format

Reviewed-by: Nizamudeen A <nia@redhat.com>
17 months agomgr/vol: don't import IndexException since it's unused 55940/head
Rishabh Dave [Mon, 4 Mar 2024 11:41:21 +0000 (17:11 +0530)]
mgr/vol: don't import IndexException since it's unused

Introduced-by: 079f722c37ef6cc8bd3cc26c49ae119dd83431f9
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit b4c1d552aa4066efdda3493192927cb780693469)

17 months agodoc/rbd: add clone mapping command 56207/head
Zac Dover [Thu, 14 Mar 2024 08:37:23 +0000 (18:37 +1000)]
doc/rbd: add clone mapping command

Add a command that explains how to map a formatted clone when the parent
image and the formatted clone have different encryption types.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d34c1879c8886ec7f19c7a93490c4736ae9a6d20)

17 months agoMerge pull request #56185 from zdover23/wip-doc-2024-03-14-backport-56160-to-squid
Zac Dover [Thu, 14 Mar 2024 19:44:55 +0000 (05:44 +1000)]
Merge pull request #56185 from zdover23/wip-doc-2024-03-14-backport-56160-to-squid

squid: doc/rbd: add map information for clone images to rbd-encryption.rst

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
17 months agomds/quiesce-db: keep the db thread alive until shutdown 56202/head
Leonid Usov [Wed, 6 Mar 2024 16:06:50 +0000 (18:06 +0200)]
mds/quiesce-db: keep the db thread alive until shutdown

With the change we can now avoid having to join it during the membership update, preventing potential deadlocks

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 8b896a9e145796119f0451201dc4d53ddac97db0)

17 months agomds/quiesce-db: incorporate review comments
Leonid Usov [Sun, 3 Mar 2024 22:45:07 +0000 (00:45 +0200)]
mds/quiesce-db: incorporate review comments

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 3e012f7ba5b8871d1bcf74d402c58553e18835dd)

17 months agomds/quiesce: declare QuiesceDbPeerListing and QuiesceDbPeerAck
Leonid Usov [Thu, 29 Feb 2024 12:08:18 +0000 (14:08 +0200)]
mds/quiesce: declare QuiesceDbPeerListing and QuiesceDbPeerAck

With these dedicated structs we can fully defer to QuiesceDbEncoding
when encoding/decoding quiesce db messages

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 205fd3388ef0ed3011bb21384e79b34b6a6611ec)

17 months agomds/quiesce: resolve the quiesce cluster at the mds monitor
Leonid Usov [Wed, 28 Feb 2024 13:11:17 +0000 (15:11 +0200)]
mds/quiesce: resolve the quiesce cluster at the mds monitor

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 759925748e8be355a743f6edd106052d5a026dfd)

17 months agoinclude/types: add an I/O helper for std::unordered_map
Leonid Usov [Wed, 28 Feb 2024 12:41:13 +0000 (14:41 +0200)]
include/types: add an I/O helper for std::unordered_map

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 42a5fb3b7562dac09dbf4575558820297317965e)

17 months agomessages: avoid using mutable members in MMDSQuiesce*
Leonid Usov [Tue, 27 Feb 2024 21:25:20 +0000 (23:25 +0200)]
messages: avoid using mutable members in MMDSQuiesce*

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 2fbe40e72e8352a3bf47190d7bc8c80bb60eb7bd)