]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Ilya Dryomov [Thu, 23 May 2024 16:15:08 +0000 (18:15 +0200)]
.github: expand tests label to all files under qa
The test job definition under qa/suites is an integral part of almost
any test. Often, the test logic is split between the task or workunit
and respective snippet(s) under qa/suites.
Other files under qa are less used, but still related to nothing but
testing, so just add the label on all of it.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
0543b9264f662cd25c059dff7252002fc6859d0d )
Ilya Dryomov [Tue, 23 Apr 2024 11:00:08 +0000 (13:00 +0200)]
.github: sync the list of paths for rbd label with CODEOWNERS
With https://github.com/actions/labeler/issues/483 still pending, do it
manually.
A couple of entries matching both files and directories were expanded
since, unlike fnmatch(3), minimatch treats slashes specially:
- src/include/rbd* -> src/include/rbd* src/include/rbd/**
- src/rbd* -> src/rbd* src/rbd*/**
- src/test/rbd* -> src/test/rbd* src/test/rbd*/**
A bunch of entries under src/tools were combined into:
- src/tools/rbd*/**
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
716996eff8ffde66f9b8404ee942189dbfeb5249 )
Ilya Dryomov [Tue, 23 Apr 2024 10:57:49 +0000 (12:57 +0200)]
CODEOWNERS: assign ceph-rbd-mirror systemd service to RBD
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
9d1302d780c64568c0aed0828217bcb5a8a5c575 )
Zac Dover [Mon, 27 May 2024 11:00:34 +0000 (21:00 +1000)]
Merge pull request #57706 from zdover23/wip-doc-2024-05-26-backport-57393-to-squid
squid: doc/man: add missing long option switches
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Matan Breizman [Mon, 27 May 2024 10:17:19 +0000 (13:17 +0300)]
Merge pull request #57600 from Matan-B/wip-56998-squid
squid: crimson/osd/pg: only kick off snap trimming when PGs are clean
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 27 May 2024 10:15:18 +0000 (13:15 +0300)]
Merge pull request #57596 from Matan-B/wip-56916-squid
squid: crimson/osd/replicated_backend: misc fixes
Reviewed-by: Samuel Just <sjust@redhat.com>
Avan [Sun, 26 May 2024 19:26:59 +0000 (00:56 +0530)]
Merge pull request #57636 from rhcs-dashboard/wip-66174-squid
squid: mgr/dashboard: add absolute path validation for pseudo path of nfs export
Reviewed-by: afreen23 <NOT@FOUND>
Anthony D'Atri [Sun, 26 May 2024 14:37:44 +0000 (10:37 -0400)]
Merge pull request #57703 from zdover23/wip-doc-2024-05-26-backport-57701-to-squid
squid: doc/cephfs: disambiguate two sentences
Patrick Donnelly [Fri, 10 May 2024 14:09:16 +0000 (10:09 -0400)]
doc/man: add missing long option switches
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
ad710c1f9395917310046a810176fca37b286fab )
Zac Dover [Sat, 25 May 2024 08:41:03 +0000 (18:41 +1000)]
doc/cephfs: disambiguate two sentences
Rewrite two sentences so that they are unambgiuously in the indicative
mood and not, as they previously seemed, in the imperative mood.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
32155e236085f64f7ea289f26be1274d5d2771fa )
Zac Dover [Sat, 25 May 2024 07:16:28 +0000 (17:16 +1000)]
Merge pull request #57668 from zdover23/wip-doc-2024-05-24-backport-57554-to-squid
squid: doc/cephfs: separate commands into sections
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Sat, 25 May 2024 07:12:52 +0000 (17:12 +1000)]
Merge pull request #57665 from zdover23/wip-doc-2024-05-24-backport-57567-to-squid
squid: doc/cephfs: edit "Cloning Snapshots" in fs-volumes.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Sat, 25 May 2024 07:08:07 +0000 (17:08 +1000)]
Merge pull request #57662 from zdover23/wip-doc-2024-05-24-backport-57574-to-squid
squid: doc/cephfs: edit "Pinning Subvolumes..."
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Sat, 25 May 2024 07:07:18 +0000 (17:07 +1000)]
Merge pull request #57646 from zdover23/wip-doc-2024-05-23-backport-57633-to-squid
squid: doc: clarify use of location: in host spec
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Adam King <adking@redhat.com>
Ilya Dryomov [Thu, 23 May 2024 20:14:57 +0000 (22:14 +0200)]
Merge pull request #57653 from petrutlucian94/wip-66204-squid
squid: qa: add ceph-rbd windows service restart test
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Zac Dover [Thu, 23 May 2024 15:05:46 +0000 (01:05 +1000)]
Merge pull request #57643 from zdover23/wip-doc-2024-05-23-backport-57642-to-squid
squid: doc/dev/release-process.rst: note new 'project' arguments
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Mon, 20 May 2024 06:29:44 +0000 (16:29 +1000)]
doc/cephfs: separate commands into sections
Separate commands so that each command has its own subsection in the
section "FS Subvolumes" in the file doc/cephfs/fs-volumes.rst.
Previously, the list of commands for manipulating subvolumes was one
long, unbroken list and the beginning of one section could easily be
mistaken for the end of the previous section.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
a84ec3a1c0f026f54d079509b5d6cafde032154c )
Zac Dover [Mon, 20 May 2024 11:55:16 +0000 (21:55 +1000)]
doc/cephfs: edit "Cloning Snapshots" in fs-volumes.rst
Edit the "Cloning Snapshots" section in doc/cephfs/fs-volumes.rst. This
commit represents only a grammar pass. A future commit (and future PR)
will separate this section into subsections by command.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
69941180e9b1ff5f7f0eddfba670028aae93e333 )
Zac Dover [Mon, 20 May 2024 15:07:14 +0000 (01:07 +1000)]
doc/cephfs: edit "Pinning Subvolumes..."
Edit the section "Pinning Subvolumes and Subvolume Groups" in
doc/cephfs/fs-volumes.rst. This is merely a grammar edit.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
b319ead00e17a3a6d734e305454c8cfc403619ea )
Matan Breizman [Thu, 23 May 2024 14:07:26 +0000 (17:07 +0300)]
Merge pull request #57599 from Matan-B/wip-55449-squid
squid: crimson/os/seastore: support extent checksum verification
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 23 May 2024 14:06:36 +0000 (17:06 +0300)]
Merge pull request #57515 from Matan-B/wip-56806-squid
squid: crimson/osd/pg_recovery: backoff if the recovery/backfill is deferred
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 23 May 2024 10:32:56 +0000 (13:32 +0300)]
Merge pull request #57598 from Matan-B/wip-57035-squid
squid: test/crimson/seastore/test_object_data_handler: remap pins through the transaction that created the pins
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 23 May 2024 10:32:40 +0000 (13:32 +0300)]
Merge pull request #57597 from Matan-B/wip-56025-squid
squid: crimson/osd/osd_operations/client_requests: we don't support rados locator keys
Reviewed-by: Samuel Just <sjust@redhat.com>
Lucian Petrut [Wed, 2 Aug 2023 13:08:01 +0000 (13:08 +0000)]
qa: update rbd-wnbd test, using MBR instead of GPT
We're getting the following error while initializing 64MB disks
on WS 2019: "The disk is not large enough to support a GPT
partition style.".
For this reason, we'll use MBR instead.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit
8d294f948a78d7f98392fe2a3bf123f583eb9757 )
Lucian Petrut [Thu, 12 Jan 2023 10:55:06 +0000 (12:55 +0200)]
qa: add ceph-rbd windows service restart test
We're adding a test that:
* maps a configurable number of images
* runs a specified test - we're reusing the ones from stress_test,
making just a few minor changes to allow running the same test
multiple times
* restarts the ceph-rbd Windows service
* waits for the images to be reconnected and refreshes the mount
information
* reruns the test
* repeats the above workflow for a specified number of times,
reusing the same images
This test ensures that:
* mounted images are still available after a service restart
* drive letters are retained
* the image content is retained
* there are no race conditions when connecting or disconnecting
a large number of images in parallel
* the driver is capable of mapping a specified number of images
simultaneously
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit
d6d36b535c86618aa754298f724854a6fde31140 )
Lucian Petrut [Tue, 10 Jan 2023 14:50:04 +0000 (16:50 +0200)]
qa: reorganize Windows python test
We're splitting the rbd-wnbd python test into separate files so
that the common code may easily be reused by other tests. This
also makes the code easier to read and maintain.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit
808d42d575c97b6d0db0e6c6d88ee5fda97fe1b1 )
Matthew Vernon [Wed, 22 May 2024 15:31:33 +0000 (16:31 +0100)]
doc: clarify use of location: in host spec
It wasn't clear that you can specify more than one element of the CRUSH hierarchy in a spec file, nor that it might be useful to do so (e.g. to ensure the host ends up beneath the default root).
So update the text to make it clearer, and similarly the example.
Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit
2366391ccec0fb6d8a1c159d6e3cdf5ff4f1d603 )
Dan Mick [Wed, 22 May 2024 22:25:51 +0000 (15:25 -0700)]
doc/dev/release-process.rst: note new 'project' arguments
Support added to the release scripts (from ceph-build.git) to
work for ceph-iscsi, so 'project' must be passed to these scripts,
and will appear in the prerelease pathnames. See also
https://github.com/ceph/ceph-build/pull/2243 and
https://github.com/ceph/ceph-container/pull/2210
Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit
7e64c6386e3571c657e0889ef12449e5b43f751a )
Patrick Donnelly [Thu, 23 May 2024 00:58:23 +0000 (20:58 -0400)]
Merge PR #57342 into squid
* refs/pull/57342/head:
PendingReleaseNotes: add note on the client incompatibility health warning and feature bit
doc/cephfs: add client_mds_auth_caps client feature bit
doc/cephfs: add missing client feature bits
doc/cephfs: document MDS_CLIENTS_BROKEN_ROOTSQUASH health error
qa: add tests for MDS_CLIENTS_BROKEN_ROOTSQUASH
mds: raise health warning if client lacks feature for root_squash
mon/MDSMonitor: add note about missing metadata inclusion
mds: check relevant caps for fs include root_squash
mds: refactor out fs_name match in MDSAuthCaps
qa: test for root_squash with multiple caps
qa: pass kwargs to mount from remount
qa: simplify update_attrs and only update relevant keys
client: allow overriding client features
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Patrick Donnelly [Wed, 22 May 2024 18:22:01 +0000 (14:22 -0400)]
Merge PR #57340 into squid
* refs/pull/57340/head:
qa: make quiesce ops dump world readable
qa: use specific ops/cache dump file names
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:21:18 +0000 (14:21 -0400)]
Merge PR #57179 into squid
* refs/pull/57179/head:
mds: encode flags for all inode types
qa: test file inode with F_QUIESCE_BLOCK is replicated
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:20:46 +0000 (14:20 -0400)]
Merge PR #57176 into squid
* refs/pull/57176/head:
mds: move drop_locks to directly after rdonly check
qa: test quiesce.block is replicated
qa: test that ceph.dir.subvolume is replicated properly
mds: add debug "lock path" command
qa: move reqid_tostr helper
qa: return run_shell process for waiters
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:20:19 +0000 (14:20 -0400)]
Merge PR #57175 into squid
* refs/pull/57175/head:
qa: extend rank 1 lockup for test_quiesce_authpin_wait
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:19:46 +0000 (14:19 -0400)]
Merge PR #57171 into squid
* refs/pull/57171/head:
qa: increase debugging for snap_schedule
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:19:17 +0000 (14:19 -0400)]
Merge PR #57063 into squid
* refs/pull/57063/head:
qa: do not iterate list being modified
qa: remove unnecessary background job cleanup
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:18:41 +0000 (14:18 -0400)]
Merge PR #57062 into squid
* refs/pull/57062/head:
mds: use mds_cache_quiesce_decay_rate to init quiesce_counter
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:18:07 +0000 (14:18 -0400)]
Merge PR #57061 into squid
* refs/pull/57061/head:
qa: add missing pg_health fragment links in fs suite
qa: ignore PG health warnings in CephFS QA
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:06:45 +0000 (14:06 -0400)]
Merge PR #57203 into squid
* refs/pull/57203/head:
mds: do not try fragmenting or exporting a quiesced directory
mds: set/test ALL_LOCKED on fragment_dir request
mds: pass bypassfreezing to parent auth pin req
qa: add quiesce tests during fragmentation
qa: translate empty output from rank_tell to empty dict
qa: move reqid_tostr helper
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Patrick Donnelly [Wed, 22 May 2024 18:06:04 +0000 (14:06 -0400)]
Merge PR #57202 into squid
* refs/pull/57202/head:
squid: mds/cache: don't assume non-auth xlocks to be remote locks
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 22 May 2024 18:04:24 +0000 (14:04 -0400)]
Merge PR #57013 into squid
* refs/pull/57013/head:
mds/quiesce: don't take mirrored cap-related locks on the replica
mds/quiesce: xlock the file to let clients keep their buffered writes
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 22 May 2024 18:03:59 +0000 (14:03 -0400)]
Merge PR #56984 into squid
* refs/pull/56984/head:
mds/quiesce: agent: avoid a race condition with rapid db updates
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
avanthakkar [Fri, 6 Oct 2023 13:40:48 +0000 (19:10 +0530)]
mgr/dashboard: add path validation for pseudo path of nfs export
Fixes: https://tracker.ceph.com/issues/63124
Signed-off-by: avanthakkar <avanjohn@gmail.com>
(cherry picked from commit
adeea099a57d34a63b483899a40c22b7bfb8c999 )
Yuri Weinstein [Wed, 22 May 2024 14:39:42 +0000 (07:39 -0700)]
Merge pull request #56478 from sseshasa/wip-65150-squid
squid: common/LogEntry: Add log level to str helper for fmt::formatter<LogEntry>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Yuri Weinstein [Wed, 22 May 2024 14:38:55 +0000 (07:38 -0700)]
Merge pull request #56477 from sseshasa/wip-65151-squid
squid: qa: Add benign cluster warning from ec-inconsistent-hinfo test to ignorelist
Reviewed-by: Laura Flores <lflores@redhat.com>
Matan Breizman [Wed, 22 May 2024 11:29:11 +0000 (14:29 +0300)]
Merge pull request #57514 from Matan-B/wip-56610-squid
squid: crimson/osd/replicated_recovery_backend: Fix recovery obc usage
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Wed, 22 May 2024 11:28:22 +0000 (14:28 +0300)]
Merge pull request #57511 from Matan-B/wip-56844-squid
squid: crimson/common/tri_mutex: make locking/promotion atomic if possible
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Wed, 22 May 2024 11:28:00 +0000 (14:28 +0300)]
Merge pull request #57510 from Matan-B/wip-56606-squid
squid: crimson/osd/ops_executer: fix snap overlap range error
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Wed, 22 May 2024 11:27:40 +0000 (14:27 +0300)]
Merge pull request #57507 from Matan-B/wip-56848-squid
squid: crimson/osd/recovery_backends: discard outdated recovery ops
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Wed, 22 May 2024 11:26:15 +0000 (14:26 +0300)]
Merge pull request #57505 from Matan-B/wip-56277-squid
squid: crimson: Add support for pool compression
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Wed, 22 May 2024 11:25:46 +0000 (14:25 +0300)]
Merge pull request #57502 from Matan-B/wip-56511-squid
squid: qa/suites/crimson-rados/thrash: enable chance_down
Reviewed-by: Samuel Just <sjust@redhat.com>
Zac Dover [Wed, 22 May 2024 06:04:53 +0000 (16:04 +1000)]
Merge pull request #57559 from ceph/wip-lusov-await-eperm-squid
squid: mds/quiesce: db: quiesce-await should EPERM if a set is past QS_QUIESCED
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Patrick Donnelly [Fri, 3 May 2024 00:45:43 +0000 (20:45 -0400)]
PendingReleaseNotes: add note on the client incompatibility health warning and feature bit
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
e70f005f1b2f4ba5466d254ec4a6432297d3fbf4 )
Patrick Donnelly [Fri, 3 May 2024 00:46:17 +0000 (20:46 -0400)]
doc/cephfs: add client_mds_auth_caps client feature bit
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
536b47cbfc669b5a3f04d93964408a2258d05ad0 )
Patrick Donnelly [Fri, 3 May 2024 00:38:19 +0000 (20:38 -0400)]
doc/cephfs: add missing client feature bits
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
79ba8970d7cbc714e160d5957bd849eede93e5a3 )
Patrick Donnelly [Thu, 2 May 2024 23:33:50 +0000 (19:33 -0400)]
doc/cephfs: document MDS_CLIENTS_BROKEN_ROOTSQUASH health error
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
b810bc9c54515b69aefffb36f1a47235b3c9125d )
Patrick Donnelly [Fri, 3 May 2024 00:52:29 +0000 (20:52 -0400)]
qa: add tests for MDS_CLIENTS_BROKEN_ROOTSQUASH
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
9d0ab233d822668e88c873bc1314e984feaf1296 )
Patrick Donnelly [Fri, 3 May 2024 00:50:37 +0000 (20:50 -0400)]
mds: raise health warning if client lacks feature for root_squash
Rather than evict all clients lacking this feature bit, raise a health error
that pushes the administrator to address it. This avoids the surprise of having
all affected clients suddenly evicted in the cluster.
Fixes: https://tracker.ceph.com/issues/65733
Fixes: 954ed30
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
66ff5c9fc8d4664f18b2fa462e96e5548c35951f )
Patrick Donnelly [Fri, 3 May 2024 00:49:22 +0000 (20:49 -0400)]
mon/MDSMonitor: add note about missing metadata inclusion
There is a "client_count" metadata on the health warning that apparently was
intended to be used for aggregating warnings but never was. Add a TODO item for
that.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
6517b704e311fd24dbf5bfbdec2ddd23b17d4092 )
Patrick Donnelly [Wed, 1 May 2024 01:41:14 +0000 (21:41 -0400)]
mds: check relevant caps for fs include root_squash
When denying client reconnects because the MDS caps include root_squash and the
client features do not include CEPHFS_FEATURE_MDS_AUTH_CAPS_CHECK, ensure those
caps are only for the file system the MDS is joined to.
Fixes: https://tracker.ceph.com/issues/65733
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
f79ae86f2c23388f6ecc3177764735e071998e09 )
Patrick Donnelly [Thu, 2 May 2024 12:55:36 +0000 (08:55 -0400)]
mds: refactor out fs_name match in MDSAuthCaps
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
247b4fed28617c104473d1586b66a8735bff0411 )
Patrick Donnelly [Thu, 2 May 2024 01:08:57 +0000 (21:08 -0400)]
qa: test for root_squash with multiple caps
Where the client has root_squash for one cap but not for another. The fs
without root_squash should not necessarily reject the client.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
bccc8ceb471c441ec04d7eb2c353630f8c5ce843 )
Conflicts:
qa/tasks/cephfs/test_admin.py: missing test
Patrick Donnelly [Thu, 2 May 2024 02:06:54 +0000 (22:06 -0400)]
qa: pass kwargs to mount from remount
So we can pass mntargs.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
afcbfc040b56779e58563f715f26a0fe25e9f916 )
Patrick Donnelly [Thu, 2 May 2024 02:04:57 +0000 (22:04 -0400)]
qa: simplify update_attrs and only update relevant keys
So we can just pass the caller's kwargs to update_attrs.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
597ff3cb15e7a7ce527b35eb01d9958b755bbf01 )
Patrick Donnelly [Thu, 2 May 2024 00:51:59 +0000 (20:51 -0400)]
client: allow overriding client features
For testing purposes.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
d9239f9375c1ae92a4990950f40078766bd912e8 )
Xuehan Xu [Thu, 25 Apr 2024 05:56:25 +0000 (13:56 +0800)]
crimson/osd/pg: SnapTrimEvent to support interrupts
SnapTrimEvent operations are scheduled from `PG::on_active_actmap()`
using a `seastar::do_until` loop. This commit replaces the loop type
into an `interruptor::repeat` and SnapTrimEvent are now scheduled by
`start_operation_may_interrupt`.
Previously, `SnapTrimEvent::start` handled interruptions by returning
a `crimson::ct_error::eagain::make();`. Now, the errorator is directly
returned via the `snap_trim_event_ret_t` and interrupts the loop
described above.
As a result, interruptions originated by interval changes are now
supported by SnapTrimEvent.
---
*** Manual conflict ***
The following PR is not included in S:
https://github.com/ceph/ceph/pull/55645/commits/
6d561092faf1af5158f438af2a585197bf44683e
This backported commit had to manually revert some of the changes introduced by #55645.
Specially, crimson/osd/osd_operations/snaptrim_event.cc
Cherry-picked from:
a4ddf6cfc88619f1f50f75a079af5f39f5167a46
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Xuehan Xu [Tue, 23 Apr 2024 01:47:35 +0000 (09:47 +0800)]
crimson/osd/pg: only kick off snap trimming when PGs are clean
Fixes: https://tracker.ceph.com/issues/65568
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Xuehan Xu [Wed, 20 Mar 2024 07:12:28 +0000 (15:12 +0800)]
test/crimson/seastore: add test profile for full integrity check
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
898efa78e2cde90a2bdf8e6f7eda1d4def736705 )
Xuehan Xu [Wed, 21 Feb 2024 08:09:59 +0000 (16:09 +0800)]
crimson/os/seastore/transaction_manager: fully load extents when
remapping extents with full extent integrity check enabled
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
361ac1f6937339d8e8e177f3e15cdb07c7376be7 )
Xuehan Xu [Wed, 21 Feb 2024 06:50:47 +0000 (14:50 +0800)]
test/crimson/seastore/test_object_data_handler: disable max_extent_size
for remap related cases
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
fd86b6976df28911a44e667d4795a25b635eaefb )
Xuehan Xu [Tue, 20 Feb 2024 10:06:25 +0000 (18:06 +0800)]
test/crimson/seastore/test_transaction_manager: disable max_extent_size for remap related tests
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
7c5c55f27b66f875857d931c329a4c884416b36a )
Xuehan Xu [Wed, 21 Feb 2024 06:53:33 +0000 (14:53 +0800)]
crimson/os/seastore/transaction_manager: add the max_data_allocation_size
configuration
Limit the max size of extents in seastore, which can avoid much read
amplification in case of remapping extents when extents integrity check
is mandatory
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
a2f6c7b2a8607298be18e8d4e06644da05b2bb03 )
Xuehan Xu [Wed, 21 Feb 2024 06:52:41 +0000 (14:52 +0800)]
crimson/os/seastore/lba_manager: add the alloc_extents interface
This is necessary because continuous data may be splitted into multiple
extents if the size of each data extent is constrained
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
ab916224e3e254e31025fb1723ca7e1147dee2c7 )
Xuehan Xu [Fri, 9 Feb 2024 06:34:03 +0000 (14:34 +0800)]
crimson/os/seastore/lba_manager: clean up lba_manager's alloc_extent related interfaces
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
03c8ae1ad04309b8d99b451f5e75e3c0b9233b32 )
Xuehan Xu [Thu, 29 Feb 2024 12:17:33 +0000 (20:17 +0800)]
crimson/os/seastore/cached_extent: rename get_crc32c to calc_crc32c
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
01776a081c041ee5b949de0f1a5841ca31baa8b2 )
Xuehan Xu [Mon, 5 Feb 2024 05:41:17 +0000 (13:41 +0800)]
crimson/os/seastore/transaction_manager: check checksums when loading
logical extents
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
c7764f397c1bbffa22d2a6b60f043709adfdc045 )
Xuehan Xu [Sun, 4 Feb 2024 09:54:26 +0000 (17:54 +0800)]
crimson/os/seastore: record extents' chksums in the lba tree
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
f4b570da86f77dfa08f33a0adc6135a1aa608233 )
Xuehan Xu [Sun, 4 Feb 2024 09:23:18 +0000 (17:23 +0800)]
crimson/os/seastore/transaction: change delayed_alloc_list/pre_alloc_list to CachedExtents
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
eb1cac71bc6a2905bd003fef2c16be61c604fa2b )
Xuehan Xu [Mon, 22 Apr 2024 10:11:55 +0000 (18:11 +0800)]
test/crimson/seastore/test_object_data_handler: remap pins through the
transaction that created the pins
Fixes: https://tracker.ceph.com/issues/65610
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
02acc0a71891ccadff68c60eaaf65bfa2c8bab4c )
Xuehan Xu [Mon, 22 Apr 2024 10:10:33 +0000 (18:10 +0800)]
crimson/os/seastore/btree: assert that the transaction getting logical
extents from pins are the one that created the pins
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
eae1b3172ee81a01403dfe8722ef6fcfee621032 )
Xuehan Xu [Sun, 7 Apr 2024 02:05:35 +0000 (10:05 +0800)]
test/neorados/misc: disable the locator key test for crimson
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
776c2b3e40d7e7c1819d56b02d38a87a1f4dbbf9 )
Xuehan Xu [Tue, 2 Apr 2024 10:13:54 +0000 (18:13 +0800)]
test/librados/misc_cxx: disable the locator key test for crimson
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
7a71f9a390774e868edb8c14c9cd03720f6e46da )
Xuehan Xu [Mon, 18 Mar 2024 06:45:21 +0000 (14:45 +0800)]
test/pybind/test_rados: disable the locator key test for crimson
clusters
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
b3754acf3533378f5c00ffaf8f0ffc9854aeb9b1 )
Xuehan Xu [Mon, 18 Mar 2024 06:40:50 +0000 (14:40 +0800)]
crimson/osd/osd_operations/client_requests: we don't support rados locator
keys
Fixes: https://tracker.ceph.com/issues/64782
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
44434b22628e10d229fd0ae552a80f2bc8de6307 )
Matan Breizman [Tue, 16 Apr 2024 08:33:21 +0000 (08:33 +0000)]
crimson/osd/replicated_backend: fix log _submit_transaction
```
ERROR 2024-04-16 07:44:02,484 [shard 1:main] osd - /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/
19.0.0-3109-g30b0aa00 /rpm/el9/BUILD/
ceph-19.0.0-3109-g30b0aa00 /src/crimson/osd/replicated_backend.cc:47 @virtual PGBackend::rep_op_fut_t ReplicatedBackend::_submit_transaction(std::set<pg_shard_t>&&, const hobject_t&, ceph::os::Transaction&&, osd_op_params_t&&, epoch_t, epoch_t, std::vector<pg_log_entry_t>&&): failed to log message: fmt='{} {}: object {}, {}': fmt::v9::format_error (argument not found)
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
aad993f2115d54c74d46d5ffa5c81f5deb3adc22 )
Matan Breizman [Sun, 14 Apr 2024 14:05:17 +0000 (14:05 +0000)]
crimson/osd/osd_operations/client_request_common: add version and reqid logs
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
006124a837a257df89d7cdff1f07ce968cfd5661 )
Matan Breizman [Thu, 11 Apr 2024 13:21:02 +0000 (13:21 +0000)]
crimson/osd/replicated_recovery_backend: push_info to use 'need' version
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
2e060f7daf5974bbc4f2933ed312b814428df223 )
Matan Breizman [Tue, 21 May 2024 12:37:28 +0000 (15:37 +0300)]
Merge pull request #57513 from Matan-B/wip-57129-squid
squid: crimson/os/seastore/transaction_manager: fix write pipeline phase leak
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:28:12 +0000 (15:28 +0300)]
Merge pull request #57509 from Matan-B/wip-56383-squid
squid: crimson/os/seastore/btree: clean up `FixedKVLeafNode::get_logical_child()`
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:27:48 +0000 (15:27 +0300)]
Merge pull request #57508 from Matan-B/wip-56875-squid
squid: crimson/osd/osd_meta: load incremental osdmap from "inc_osdmap.XXX"
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:26:28 +0000 (15:26 +0300)]
Merge pull request #57506 from Matan-B/wip-56353-squid
squid: crimson/os/seastore: avoid new allocation when overwriting data in RBM for performance
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:25:24 +0000 (15:25 +0300)]
Merge pull request #57504 from Matan-B/wip-56611-squid
squid: crimson/osd/replicated_recovery_backend: prepare_pull use pg_info
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:24:57 +0000 (15:24 +0300)]
Merge pull request #57503 from Matan-B/wip-56912-squid
squid: crimson/common/operation: fix and move exit() after entering the next phase
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:24:21 +0000 (15:24 +0300)]
Merge pull request #57501 from Matan-B/wip-56775-squid
squid: crimson/osd: implement basic reactor-utilization stats report to log
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:23:44 +0000 (15:23 +0300)]
Merge pull request #57500 from Matan-B/wip-56627-squid
squid: crimson/os/seastore: alloc mapping with refcount when rewriting logical extents
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 21 May 2024 12:23:11 +0000 (15:23 +0300)]
Merge pull request #57499 from Matan-B/wip-55847-squid
squid: crimson: convert some of client_request to use coroutines
Reviewed-by: Samuel Just <sjust@redhat.com>
Neha Ojha [Mon, 20 May 2024 21:35:42 +0000 (17:35 -0400)]
Merge pull request #57413 from neha-ojha/wip-squid-rc
squid: src/ceph_release: dev -> rc
Reviewed-by: Laura Flores <lflores@redhat.com>
Adam King [Mon, 20 May 2024 17:50:37 +0000 (13:50 -0400)]
Merge pull request #57532 from rhcs-dashboard/wip-66092-squid
squid: mgr/k8sevents: update V1Events to CoreV1Events
Reviewed-by: Laura Flores <lflores@ibm.com>
Neha Ojha [Mon, 20 May 2024 16:53:01 +0000 (16:53 +0000)]
doc/dev/release-checklists.rst: squid milestone added
Signed-off-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Fri, 10 May 2024 20:59:25 +0000 (20:59 +0000)]
.github/milestone.yml: add squid
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit
bca45c7df2c26b105401b299161f553ffe4e32c1 )
Leonid Usov [Fri, 26 Apr 2024 00:20:42 +0000 (03:20 +0300)]
mds/quiesce: db: quiesce-await should EPERM if a set is past QS_QUIESCED
Fixes: https://tracker.ceph.com/issues/65669
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit
fd88b52d6fb11c206b3a06e8533c4551c902f173 )
Fixes: https://tracker.ceph.com/issues/66034