]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
13 months agomgr/dashboard: fix readonly landingpage 57747/head
Pedro Gonzalez Gomez [Wed, 24 Apr 2024 08:48:11 +0000 (10:48 +0200)]
mgr/dashboard: fix readonly landingpage

Conflicts:
mgr/dashboard/frontend/src/app/core/layouts/workbench-layout/workbench-layout.component.ts
mgr/dashboard/frontend/src/app/ceph/dashboard-v3/dashboard/dashboard-v3.component.ts
- Hardware status and multi-cluster features are not included in squid, therefore this only affects the telemetry status

Fixes: https://tracker.ceph.com/issues/65643
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 7abae5c982ee1fd7de74653bdd5ad46dbc4ae388)

13 months agoMerge pull request #57512 from Matan-B/wip-57088-squid
Matan Breizman [Tue, 28 May 2024 09:07:57 +0000 (12:07 +0300)]
Merge pull request #57512 from Matan-B/wip-57088-squid

squid: crimson/os/seastore: only update onode sizes when necessary

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57719 from Matan-B/wip-56892-squid
Matan Breizman [Tue, 28 May 2024 08:14:12 +0000 (11:14 +0300)]
Merge pull request #57719 from Matan-B/wip-56892-squid

squid: crimson/osd/osd_operations/client_request: retrieve the correct version for objects to be recovered urgently

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57661 from Matan-B/wip-57128-squid
Matan Breizman [Tue, 28 May 2024 08:13:34 +0000 (11:13 +0300)]
Merge pull request #57661 from Matan-B/wip-57128-squid

squid: crimson/os/seastore: misc cleanups and fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57660 from Matan-B/wip-57135-squid
Matan Breizman [Tue, 28 May 2024 08:13:08 +0000 (11:13 +0300)]
Merge pull request #57660 from Matan-B/wip-57135-squid

squid: crimson/os/seastore/transaction_manager: remove incorrect assertions

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57728 from idryomov/wip-labeler-rbd-squid
Ilya Dryomov [Tue, 28 May 2024 07:25:53 +0000 (09:25 +0200)]
Merge pull request #57728 from idryomov/wip-labeler-rbd-squid

squid: .github: sync the list of paths for rbd label, expand tests label to qa/*

Reviewed-by: Ramana Raja <rraja@redhat.com>
13 months agoMerge pull request #57714 from rhcs-dashboard/dashboard-releasenotes-squid
Nizamudeen A [Tue, 28 May 2024 06:39:51 +0000 (12:09 +0530)]
Merge pull request #57714 from rhcs-dashboard/dashboard-releasenotes-squid

squid: doc: Update pendingreleasenotes for dashboard

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
13 months agoMerge pull request #57723 from zdover23/wip-doc-2024-05-27-backport-57628-to-squid
Anthony D'Atri [Mon, 27 May 2024 22:07:18 +0000 (18:07 -0400)]
Merge pull request #57723 from zdover23/wip-doc-2024-05-27-backport-57628-to-squid

squid: doc: document metrics exported by CephFS

13 months ago.github: expand tests label to all files under qa 57728/head
Ilya Dryomov [Thu, 23 May 2024 16:15:08 +0000 (18:15 +0200)]
.github: expand tests label to all files under qa

The test job definition under qa/suites is an integral part of almost
any test.  Often, the test logic is split between the task or workunit
and respective snippet(s) under qa/suites.

Other files under qa are less used, but still related to nothing but
testing, so just add the label on all of it.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0543b9264f662cd25c059dff7252002fc6859d0d)

13 months ago.github: sync the list of paths for rbd label with CODEOWNERS
Ilya Dryomov [Tue, 23 Apr 2024 11:00:08 +0000 (13:00 +0200)]
.github: sync the list of paths for rbd label with CODEOWNERS

With https://github.com/actions/labeler/issues/483 still pending, do it
manually.

A couple of entries matching both files and directories were expanded
since, unlike fnmatch(3), minimatch treats slashes specially:

- src/include/rbd* -> src/include/rbd* src/include/rbd/**
- src/rbd*         -> src/rbd* src/rbd*/**
- src/test/rbd*    -> src/test/rbd* src/test/rbd*/**

A bunch of entries under src/tools were combined into:

- src/tools/rbd*/**

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 716996eff8ffde66f9b8404ee942189dbfeb5249)

13 months agoCODEOWNERS: assign ceph-rbd-mirror systemd service to RBD
Ilya Dryomov [Tue, 23 Apr 2024 10:57:49 +0000 (12:57 +0200)]
CODEOWNERS: assign ceph-rbd-mirror systemd service to RBD

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 9d1302d780c64568c0aed0828217bcb5a8a5c575)

13 months agodoc: document metrics exported by CephFS 57723/head
Jos Collin [Wed, 22 May 2024 12:40:24 +0000 (18:10 +0530)]
doc: document metrics exported by CephFS

Fixes: https://tracker.ceph.com/issues/66173
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit f8f47852f309d2f8d6ea53a0650c7fbef2a70efa)

13 months agoMerge pull request #57706 from zdover23/wip-doc-2024-05-26-backport-57393-to-squid
Zac Dover [Mon, 27 May 2024 11:00:34 +0000 (21:00 +1000)]
Merge pull request #57706 from zdover23/wip-doc-2024-05-26-backport-57393-to-squid

squid: doc/man: add missing long option switches

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
13 months agocrimson/osd/replicated_recovery_backend: assert that the object to be pulled has... 57719/head
Xuehan Xu [Mon, 15 Apr 2024 13:58:02 +0000 (21:58 +0800)]
crimson/osd/replicated_recovery_backend: assert that the object to be pulled has a valid recovery source

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 453db346383936e4d4640fce6c1f0bf132cb7992)

13 months agocrimson/osd/osd_operations/client_request: retrieve the correct version
Xuehan Xu [Mon, 15 Apr 2024 13:30:07 +0000 (21:30 +0800)]
crimson/osd/osd_operations/client_request: retrieve the correct version
for objects to be recovered urgently

PG::is_unreadable_object() can't guarantee that it'll set the parameter
"v", so CommonClientRequest::do_recover_missing() has to retrieve it by
itself

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 7743d33136e7e2431cca1c3929adba6cd386e773)

13 months agoMerge pull request #57600 from Matan-B/wip-56998-squid
Matan Breizman [Mon, 27 May 2024 10:17:19 +0000 (13:17 +0300)]
Merge pull request #57600 from Matan-B/wip-56998-squid

squid: crimson/osd/pg: only kick off snap trimming when PGs are clean

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57596 from Matan-B/wip-56916-squid
Matan Breizman [Mon, 27 May 2024 10:15:18 +0000 (13:15 +0300)]
Merge pull request #57596 from Matan-B/wip-56916-squid

squid: crimson/osd/replicated_backend: misc fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agodoc: Update pendingreleasenotes for dashboard 57714/head
Nizamudeen A [Tue, 21 May 2024 05:11:26 +0000 (10:41 +0530)]
doc: Update pendingreleasenotes for dashboard

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit ef3e179b99f5816f5ecd55844337f54ef563a93f)

13 months agoMerge pull request #57636 from rhcs-dashboard/wip-66174-squid
Avan [Sun, 26 May 2024 19:26:59 +0000 (00:56 +0530)]
Merge pull request #57636 from rhcs-dashboard/wip-66174-squid

squid: mgr/dashboard: add absolute path validation for pseudo path of nfs export

Reviewed-by: afreen23 <NOT@FOUND>
13 months agoMerge pull request #57703 from zdover23/wip-doc-2024-05-26-backport-57701-to-squid
Anthony D'Atri [Sun, 26 May 2024 14:37:44 +0000 (10:37 -0400)]
Merge pull request #57703 from zdover23/wip-doc-2024-05-26-backport-57701-to-squid

squid: doc/cephfs: disambiguate two sentences

13 months agodoc/man: add missing long option switches 57706/head
Patrick Donnelly [Fri, 10 May 2024 14:09:16 +0000 (10:09 -0400)]
doc/man: add missing long option switches

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ad710c1f9395917310046a810176fca37b286fab)

13 months agodoc/cephfs: disambiguate two sentences 57703/head
Zac Dover [Sat, 25 May 2024 08:41:03 +0000 (18:41 +1000)]
doc/cephfs: disambiguate two sentences

Rewrite two sentences so that they are unambgiuously in the indicative
mood and not, as they previously seemed, in the imperative mood.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 32155e236085f64f7ea289f26be1274d5d2771fa)

13 months agoMerge pull request #57668 from zdover23/wip-doc-2024-05-24-backport-57554-to-squid
Zac Dover [Sat, 25 May 2024 07:16:28 +0000 (17:16 +1000)]
Merge pull request #57668 from zdover23/wip-doc-2024-05-24-backport-57554-to-squid

squid: doc/cephfs: separate commands into sections

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
13 months agoMerge pull request #57665 from zdover23/wip-doc-2024-05-24-backport-57567-to-squid
Zac Dover [Sat, 25 May 2024 07:12:52 +0000 (17:12 +1000)]
Merge pull request #57665 from zdover23/wip-doc-2024-05-24-backport-57567-to-squid

squid: doc/cephfs: edit "Cloning Snapshots" in fs-volumes.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
13 months agoMerge pull request #57662 from zdover23/wip-doc-2024-05-24-backport-57574-to-squid
Zac Dover [Sat, 25 May 2024 07:08:07 +0000 (17:08 +1000)]
Merge pull request #57662 from zdover23/wip-doc-2024-05-24-backport-57574-to-squid

squid: doc/cephfs: edit "Pinning Subvolumes..."

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
13 months agoMerge pull request #57646 from zdover23/wip-doc-2024-05-23-backport-57633-to-squid
Zac Dover [Sat, 25 May 2024 07:07:18 +0000 (17:07 +1000)]
Merge pull request #57646 from zdover23/wip-doc-2024-05-23-backport-57633-to-squid

squid: doc: clarify use of location: in host spec

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Adam King <adking@redhat.com>
14 months agoMerge pull request #57653 from petrutlucian94/wip-66204-squid
Ilya Dryomov [Thu, 23 May 2024 20:14:57 +0000 (22:14 +0200)]
Merge pull request #57653 from petrutlucian94/wip-66204-squid

squid: qa: add ceph-rbd windows service restart test

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
14 months agoMerge pull request #57643 from zdover23/wip-doc-2024-05-23-backport-57642-to-squid
Zac Dover [Thu, 23 May 2024 15:05:46 +0000 (01:05 +1000)]
Merge pull request #57643 from zdover23/wip-doc-2024-05-23-backport-57642-to-squid

squid: doc/dev/release-process.rst: note new 'project' arguments

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agodoc/cephfs: separate commands into sections 57668/head
Zac Dover [Mon, 20 May 2024 06:29:44 +0000 (16:29 +1000)]
doc/cephfs: separate commands into sections

Separate commands so that each command has its own subsection in the
section "FS Subvolumes" in the file doc/cephfs/fs-volumes.rst.
Previously, the list of commands for manipulating subvolumes was one
long, unbroken list and the beginning of one section could easily be
mistaken for the end of the previous section.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a84ec3a1c0f026f54d079509b5d6cafde032154c)

14 months agodoc/cephfs: edit "Cloning Snapshots" in fs-volumes.rst 57665/head
Zac Dover [Mon, 20 May 2024 11:55:16 +0000 (21:55 +1000)]
doc/cephfs: edit "Cloning Snapshots" in fs-volumes.rst

Edit the "Cloning Snapshots" section in doc/cephfs/fs-volumes.rst. This
commit represents only a grammar pass. A future commit (and future PR)
will separate this section into subsections by command.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 69941180e9b1ff5f7f0eddfba670028aae93e333)

14 months agodoc/cephfs: edit "Pinning Subvolumes..." 57662/head
Zac Dover [Mon, 20 May 2024 15:07:14 +0000 (01:07 +1000)]
doc/cephfs: edit "Pinning Subvolumes..."

Edit the section "Pinning Subvolumes and Subvolume Groups" in
doc/cephfs/fs-volumes.rst. This is merely a grammar edit.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit b319ead00e17a3a6d734e305454c8cfc403619ea)

14 months agocrimson/os/seastore: drop the impossible error eagain 57661/head
Yingxin Cheng [Mon, 29 Apr 2024 03:39:44 +0000 (11:39 +0800)]
crimson/os/seastore: drop the impossible error eagain

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 2a4b2e7d1401da185f59e2f73bf7b52fb8094a89)

14 months agocrimson/os/seastore/extent_placement_manager: make reserve and block atomic
Yingxin Cheng [Mon, 29 Apr 2024 03:03:02 +0000 (11:03 +0800)]
crimson/os/seastore/extent_placement_manager: make reserve and block atomic

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 71461dafdfb59df8e385f9b276576564cfeddb02)

14 months agocrimson/os/seastore: improve seastore_t logs
Yingxin Cheng [Mon, 29 Apr 2024 02:46:11 +0000 (10:46 +0800)]
crimson/os/seastore: improve seastore_t logs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 72472625e40e10558ab5f1a6d43ceb3d41edfcfd)

14 months agocrimson/os/seastore: add logs for background
Yingxin Cheng [Fri, 26 Apr 2024 07:59:55 +0000 (15:59 +0800)]
crimson/os/seastore: add logs for background

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 86041f1a3f639a4779c476aee1e32c0d35800c8e)

14 months agotest/crimson/gtest_seastar: add logs
Yingxin Cheng [Fri, 26 Apr 2024 06:23:28 +0000 (14:23 +0800)]
test/crimson/gtest_seastar: add logs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit f9352acca26872abe90039f5444386e704b68a90)

14 months agotest/crimson/gtest_seastar: cleanups
Yingxin Cheng [Fri, 26 Apr 2024 06:21:38 +0000 (14:21 +0800)]
test/crimson/gtest_seastar: cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 49cd76bebb8cd883476ebe3c665bef9f64a6fc4c)

14 months agocrimson/os/seastore/transaction_manager: remove incorrect assertions 57660/head
Xuehan Xu [Mon, 29 Apr 2024 08:56:09 +0000 (16:56 +0800)]
crimson/os/seastore/transaction_manager: remove incorrect assertions

Indirect mappings always hold the checksums of the corresponding intermediate
mappings, which may have non-zero

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit fca8efca527f44db63df068751f9e6c8bdc939e2)

14 months agoMerge pull request #57599 from Matan-B/wip-55449-squid
Matan Breizman [Thu, 23 May 2024 14:07:26 +0000 (17:07 +0300)]
Merge pull request #57599 from Matan-B/wip-55449-squid

squid: crimson/os/seastore: support extent checksum verification

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57515 from Matan-B/wip-56806-squid
Matan Breizman [Thu, 23 May 2024 14:06:36 +0000 (17:06 +0300)]
Merge pull request #57515 from Matan-B/wip-56806-squid

squid: crimson/osd/pg_recovery: backoff if the recovery/backfill is deferred

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57598 from Matan-B/wip-57035-squid
Matan Breizman [Thu, 23 May 2024 10:32:56 +0000 (13:32 +0300)]
Merge pull request #57598 from Matan-B/wip-57035-squid

squid: test/crimson/seastore/test_object_data_handler: remap pins through the transaction that created the pins

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57597 from Matan-B/wip-56025-squid
Matan Breizman [Thu, 23 May 2024 10:32:40 +0000 (13:32 +0300)]
Merge pull request #57597 from Matan-B/wip-56025-squid

squid: crimson/osd/osd_operations/client_requests: we don't support rados locator keys

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoqa: update rbd-wnbd test, using MBR instead of GPT 57653/head
Lucian Petrut [Wed, 2 Aug 2023 13:08:01 +0000 (13:08 +0000)]
qa: update rbd-wnbd test, using MBR instead of GPT

We're getting the following error while initializing 64MB disks
on WS 2019: "The disk is not large enough to support a GPT
partition style.".

For this reason, we'll use MBR instead.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 8d294f948a78d7f98392fe2a3bf123f583eb9757)

14 months agoqa: add ceph-rbd windows service restart test
Lucian Petrut [Thu, 12 Jan 2023 10:55:06 +0000 (12:55 +0200)]
qa: add ceph-rbd windows service restart test

We're adding a test that:

* maps a configurable number of images
* runs a specified test - we're reusing the ones from stress_test,
  making just a few minor changes to allow running the same test
  multiple times
* restarts the ceph-rbd Windows service
* waits for the images to be reconnected and refreshes the mount
  information
* reruns the test
* repeats the above workflow for a specified number of times,
  reusing the same images

This test ensures that:

* mounted images are still available after a service restart
* drive letters are retained
* the image content is retained
* there are no race conditions when connecting or disconnecting
  a large number of images in parallel
* the driver is capable of mapping a specified number of images
  simultaneously

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit d6d36b535c86618aa754298f724854a6fde31140)

14 months agoqa: reorganize Windows python test
Lucian Petrut [Tue, 10 Jan 2023 14:50:04 +0000 (16:50 +0200)]
qa: reorganize Windows python test

We're splitting the rbd-wnbd python test into separate files so
that the common code may easily be reused by other tests. This
also makes the code easier to read and maintain.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 808d42d575c97b6d0db0e6c6d88ee5fda97fe1b1)

14 months agodoc: clarify use of location: in host spec 57646/head
Matthew Vernon [Wed, 22 May 2024 15:31:33 +0000 (16:31 +0100)]
doc: clarify use of location: in host spec

It wasn't clear that you can specify more than one element of the CRUSH hierarchy in a spec file, nor that it might be useful to do so (e.g. to ensure the host ends up beneath the default root).

So update the text to make it clearer, and similarly the example.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit 2366391ccec0fb6d8a1c159d6e3cdf5ff4f1d603)

14 months agodoc/dev/release-process.rst: note new 'project' arguments 57643/head
Dan Mick [Wed, 22 May 2024 22:25:51 +0000 (15:25 -0700)]
doc/dev/release-process.rst: note new 'project' arguments

Support added to the release scripts (from ceph-build.git) to
work for ceph-iscsi, so 'project' must be passed to these scripts,
and will appear in the prerelease pathnames.  See also
https://github.com/ceph/ceph-build/pull/2243 and
https://github.com/ceph/ceph-container/pull/2210

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 7e64c6386e3571c657e0889ef12449e5b43f751a)

14 months agoMerge PR #57342 into squid
Patrick Donnelly [Thu, 23 May 2024 00:58:23 +0000 (20:58 -0400)]
Merge PR #57342 into squid

* refs/pull/57342/head:
PendingReleaseNotes: add note on the client incompatibility health warning and feature bit
doc/cephfs: add client_mds_auth_caps client feature bit
doc/cephfs: add missing client feature bits
doc/cephfs: document MDS_CLIENTS_BROKEN_ROOTSQUASH health error
qa: add tests for MDS_CLIENTS_BROKEN_ROOTSQUASH
mds: raise health warning if client lacks feature for root_squash
mon/MDSMonitor: add note about missing metadata inclusion
mds: check relevant caps for fs include root_squash
mds: refactor out fs_name match in MDSAuthCaps
qa: test for root_squash with multiple caps
qa: pass kwargs to mount from remount
qa: simplify update_attrs and only update relevant keys
client: allow overriding client features

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
14 months agoMerge PR #57340 into squid
Patrick Donnelly [Wed, 22 May 2024 18:22:01 +0000 (14:22 -0400)]
Merge PR #57340 into squid

* refs/pull/57340/head:
qa: make quiesce ops dump world readable
qa: use specific ops/cache dump file names

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57179 into squid
Patrick Donnelly [Wed, 22 May 2024 18:21:18 +0000 (14:21 -0400)]
Merge PR #57179 into squid

* refs/pull/57179/head:
mds: encode flags for all inode types
qa: test file inode with F_QUIESCE_BLOCK is replicated

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57176 into squid
Patrick Donnelly [Wed, 22 May 2024 18:20:46 +0000 (14:20 -0400)]
Merge PR #57176 into squid

* refs/pull/57176/head:
mds: move drop_locks to directly after rdonly check
qa: test quiesce.block is replicated
qa: test that ceph.dir.subvolume is replicated properly
mds: add debug "lock path" command
qa: move reqid_tostr helper
qa: return run_shell process for waiters

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57175 into squid
Patrick Donnelly [Wed, 22 May 2024 18:20:19 +0000 (14:20 -0400)]
Merge PR #57175 into squid

* refs/pull/57175/head:
qa: extend rank 1 lockup for test_quiesce_authpin_wait

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57171 into squid
Patrick Donnelly [Wed, 22 May 2024 18:19:46 +0000 (14:19 -0400)]
Merge PR #57171 into squid

* refs/pull/57171/head:
qa: increase debugging for snap_schedule

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57063 into squid
Patrick Donnelly [Wed, 22 May 2024 18:19:17 +0000 (14:19 -0400)]
Merge PR #57063 into squid

* refs/pull/57063/head:
qa: do not iterate list being modified
qa: remove unnecessary background job cleanup

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57062 into squid
Patrick Donnelly [Wed, 22 May 2024 18:18:41 +0000 (14:18 -0400)]
Merge PR #57062 into squid

* refs/pull/57062/head:
mds: use mds_cache_quiesce_decay_rate to init quiesce_counter

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57061 into squid
Patrick Donnelly [Wed, 22 May 2024 18:18:07 +0000 (14:18 -0400)]
Merge PR #57061 into squid

* refs/pull/57061/head:
qa: add missing pg_health fragment links in fs suite
qa: ignore PG health warnings in CephFS QA

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57203 into squid
Patrick Donnelly [Wed, 22 May 2024 18:06:45 +0000 (14:06 -0400)]
Merge PR #57203 into squid

* refs/pull/57203/head:
mds: do not try fragmenting or exporting a quiesced directory
mds: set/test ALL_LOCKED on fragment_dir request
mds: pass bypassfreezing to parent auth pin req
qa: add quiesce tests during fragmentation
qa: translate empty output from rank_tell to empty dict
qa: move reqid_tostr helper

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
14 months agoMerge PR #57202 into squid
Patrick Donnelly [Wed, 22 May 2024 18:06:04 +0000 (14:06 -0400)]
Merge PR #57202 into squid

* refs/pull/57202/head:
squid: mds/cache: don't assume non-auth xlocks to be remote locks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
14 months agoMerge PR #57013 into squid
Patrick Donnelly [Wed, 22 May 2024 18:04:24 +0000 (14:04 -0400)]
Merge PR #57013 into squid

* refs/pull/57013/head:
mds/quiesce: don't take mirrored cap-related locks on the replica
mds/quiesce: xlock the file to let clients keep their buffered writes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
14 months agoMerge PR #56984 into squid
Patrick Donnelly [Wed, 22 May 2024 18:03:59 +0000 (14:03 -0400)]
Merge PR #56984 into squid

* refs/pull/56984/head:
mds/quiesce: agent: avoid a race condition with rapid db updates

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
14 months agomgr/dashboard: add path validation for pseudo path of nfs export 57636/head
avanthakkar [Fri, 6 Oct 2023 13:40:48 +0000 (19:10 +0530)]
mgr/dashboard: add path validation for pseudo path of nfs export

Fixes: https://tracker.ceph.com/issues/63124
Signed-off-by: avanthakkar <avanjohn@gmail.com>
(cherry picked from commit adeea099a57d34a63b483899a40c22b7bfb8c999)

14 months agoMerge pull request #56478 from sseshasa/wip-65150-squid
Yuri Weinstein [Wed, 22 May 2024 14:39:42 +0000 (07:39 -0700)]
Merge pull request #56478 from sseshasa/wip-65150-squid

squid: common/LogEntry: Add log level to str helper for fmt::formatter<LogEntry>

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
14 months agoMerge pull request #56477 from sseshasa/wip-65151-squid
Yuri Weinstein [Wed, 22 May 2024 14:38:55 +0000 (07:38 -0700)]
Merge pull request #56477 from sseshasa/wip-65151-squid

squid: qa: Add benign cluster warning from ec-inconsistent-hinfo test to ignorelist

Reviewed-by: Laura Flores <lflores@redhat.com>
14 months agoMerge pull request #57514 from Matan-B/wip-56610-squid
Matan Breizman [Wed, 22 May 2024 11:29:11 +0000 (14:29 +0300)]
Merge pull request #57514 from Matan-B/wip-56610-squid

squid: crimson/osd/replicated_recovery_backend: Fix recovery obc usage

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57511 from Matan-B/wip-56844-squid
Matan Breizman [Wed, 22 May 2024 11:28:22 +0000 (14:28 +0300)]
Merge pull request #57511 from Matan-B/wip-56844-squid

squid: crimson/common/tri_mutex: make locking/promotion atomic if possible

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57510 from Matan-B/wip-56606-squid
Matan Breizman [Wed, 22 May 2024 11:28:00 +0000 (14:28 +0300)]
Merge pull request #57510 from Matan-B/wip-56606-squid

squid: crimson/osd/ops_executer: fix snap overlap range error

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57507 from Matan-B/wip-56848-squid
Matan Breizman [Wed, 22 May 2024 11:27:40 +0000 (14:27 +0300)]
Merge pull request #57507 from Matan-B/wip-56848-squid

squid: crimson/osd/recovery_backends: discard outdated recovery ops

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57505 from Matan-B/wip-56277-squid
Matan Breizman [Wed, 22 May 2024 11:26:15 +0000 (14:26 +0300)]
Merge pull request #57505 from Matan-B/wip-56277-squid

squid: crimson: Add support for pool compression

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57502 from Matan-B/wip-56511-squid
Matan Breizman [Wed, 22 May 2024 11:25:46 +0000 (14:25 +0300)]
Merge pull request #57502 from Matan-B/wip-56511-squid

squid: qa/suites/crimson-rados/thrash: enable chance_down

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57559 from ceph/wip-lusov-await-eperm-squid
Zac Dover [Wed, 22 May 2024 06:04:53 +0000 (16:04 +1000)]
Merge pull request #57559 from ceph/wip-lusov-await-eperm-squid

squid: mds/quiesce: db: quiesce-await should EPERM if a set is past QS_QUIESCED

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoPendingReleaseNotes: add note on the client incompatibility health warning and featur... 57342/head
Patrick Donnelly [Fri, 3 May 2024 00:45:43 +0000 (20:45 -0400)]
PendingReleaseNotes: add note on the client incompatibility health warning and feature bit

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e70f005f1b2f4ba5466d254ec4a6432297d3fbf4)

14 months agodoc/cephfs: add client_mds_auth_caps client feature bit
Patrick Donnelly [Fri, 3 May 2024 00:46:17 +0000 (20:46 -0400)]
doc/cephfs: add client_mds_auth_caps client feature bit

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 536b47cbfc669b5a3f04d93964408a2258d05ad0)

14 months agodoc/cephfs: add missing client feature bits
Patrick Donnelly [Fri, 3 May 2024 00:38:19 +0000 (20:38 -0400)]
doc/cephfs: add missing client feature bits

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 79ba8970d7cbc714e160d5957bd849eede93e5a3)

14 months agodoc/cephfs: document MDS_CLIENTS_BROKEN_ROOTSQUASH health error
Patrick Donnelly [Thu, 2 May 2024 23:33:50 +0000 (19:33 -0400)]
doc/cephfs: document MDS_CLIENTS_BROKEN_ROOTSQUASH health error

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b810bc9c54515b69aefffb36f1a47235b3c9125d)

14 months agoqa: add tests for MDS_CLIENTS_BROKEN_ROOTSQUASH
Patrick Donnelly [Fri, 3 May 2024 00:52:29 +0000 (20:52 -0400)]
qa: add tests for MDS_CLIENTS_BROKEN_ROOTSQUASH

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9d0ab233d822668e88c873bc1314e984feaf1296)

14 months agomds: raise health warning if client lacks feature for root_squash
Patrick Donnelly [Fri, 3 May 2024 00:50:37 +0000 (20:50 -0400)]
mds: raise health warning if client lacks feature for root_squash

Rather than evict all clients lacking this feature bit, raise a health error
that pushes the administrator to address it. This avoids the surprise of having
all affected clients suddenly evicted in the cluster.

Fixes: https://tracker.ceph.com/issues/65733
Fixes: 954ed30
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 66ff5c9fc8d4664f18b2fa462e96e5548c35951f)

14 months agomon/MDSMonitor: add note about missing metadata inclusion
Patrick Donnelly [Fri, 3 May 2024 00:49:22 +0000 (20:49 -0400)]
mon/MDSMonitor: add note about missing metadata inclusion

There is a "client_count" metadata on the health warning that apparently was
intended to be used for aggregating warnings but never was. Add a TODO item for
that.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6517b704e311fd24dbf5bfbdec2ddd23b17d4092)

14 months agomds: check relevant caps for fs include root_squash
Patrick Donnelly [Wed, 1 May 2024 01:41:14 +0000 (21:41 -0400)]
mds: check relevant caps for fs include root_squash

When denying client reconnects because the MDS caps include root_squash and the
client features do not include CEPHFS_FEATURE_MDS_AUTH_CAPS_CHECK, ensure those
caps are only for the file system the MDS is joined to.

Fixes: https://tracker.ceph.com/issues/65733
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f79ae86f2c23388f6ecc3177764735e071998e09)

14 months agomds: refactor out fs_name match in MDSAuthCaps
Patrick Donnelly [Thu, 2 May 2024 12:55:36 +0000 (08:55 -0400)]
mds: refactor out fs_name match in MDSAuthCaps

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 247b4fed28617c104473d1586b66a8735bff0411)

14 months agoqa: test for root_squash with multiple caps
Patrick Donnelly [Thu, 2 May 2024 01:08:57 +0000 (21:08 -0400)]
qa: test for root_squash with multiple caps

Where the client has root_squash for one cap but not for another. The fs
without root_squash should not necessarily reject the client.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bccc8ceb471c441ec04d7eb2c353630f8c5ce843)

Conflicts:
qa/tasks/cephfs/test_admin.py: missing test

14 months agoqa: pass kwargs to mount from remount
Patrick Donnelly [Thu, 2 May 2024 02:06:54 +0000 (22:06 -0400)]
qa: pass kwargs to mount from remount

So we can pass mntargs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit afcbfc040b56779e58563f715f26a0fe25e9f916)

14 months agoqa: simplify update_attrs and only update relevant keys
Patrick Donnelly [Thu, 2 May 2024 02:04:57 +0000 (22:04 -0400)]
qa: simplify update_attrs and only update relevant keys

So we can just pass the caller's kwargs to update_attrs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 597ff3cb15e7a7ce527b35eb01d9958b755bbf01)

14 months agoclient: allow overriding client features
Patrick Donnelly [Thu, 2 May 2024 00:51:59 +0000 (20:51 -0400)]
client: allow overriding client features

For testing purposes.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d9239f9375c1ae92a4990950f40078766bd912e8)

14 months agocrimson/osd/pg: SnapTrimEvent to support interrupts 57600/head
Xuehan Xu [Thu, 25 Apr 2024 05:56:25 +0000 (13:56 +0800)]
crimson/osd/pg: SnapTrimEvent to support interrupts

SnapTrimEvent operations are scheduled from `PG::on_active_actmap()`
using a `seastar::do_until` loop. This commit replaces the loop type
into an `interruptor::repeat` and SnapTrimEvent are now scheduled by
`start_operation_may_interrupt`.

Previously, `SnapTrimEvent::start` handled interruptions by returning
a `crimson::ct_error::eagain::make();`. Now, the errorator is directly
returned via the `snap_trim_event_ret_t` and interrupts the loop
described above.

As a result, interruptions originated by interval changes are now
supported by SnapTrimEvent.

---

*** Manual conflict ***
The following PR is not included in S:
https://github.com/ceph/ceph/pull/55645/commits/6d561092faf1af5158f438af2a585197bf44683e

This backported commit had to manually revert some of the changes introduced by #55645.
Specially, crimson/osd/osd_operations/snaptrim_event.cc

Cherry-picked from: a4ddf6cfc88619f1f50f75a079af5f39f5167a46

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
14 months agocrimson/osd/pg: only kick off snap trimming when PGs are clean
Xuehan Xu [Tue, 23 Apr 2024 01:47:35 +0000 (09:47 +0800)]
crimson/osd/pg: only kick off snap trimming when PGs are clean

Fixes: https://tracker.ceph.com/issues/65568
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
14 months agotest/crimson/seastore: add test profile for full integrity check 57599/head
Xuehan Xu [Wed, 20 Mar 2024 07:12:28 +0000 (15:12 +0800)]
test/crimson/seastore: add test profile for full integrity check

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 898efa78e2cde90a2bdf8e6f7eda1d4def736705)

14 months agocrimson/os/seastore/transaction_manager: fully load extents when
Xuehan Xu [Wed, 21 Feb 2024 08:09:59 +0000 (16:09 +0800)]
crimson/os/seastore/transaction_manager: fully load extents when
remapping extents with full extent integrity check enabled

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 361ac1f6937339d8e8e177f3e15cdb07c7376be7)

14 months agotest/crimson/seastore/test_object_data_handler: disable max_extent_size
Xuehan Xu [Wed, 21 Feb 2024 06:50:47 +0000 (14:50 +0800)]
test/crimson/seastore/test_object_data_handler: disable max_extent_size
for remap related cases

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit fd86b6976df28911a44e667d4795a25b635eaefb)

14 months agotest/crimson/seastore/test_transaction_manager: disable max_extent_size for remap...
Xuehan Xu [Tue, 20 Feb 2024 10:06:25 +0000 (18:06 +0800)]
test/crimson/seastore/test_transaction_manager: disable max_extent_size for remap related tests

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 7c5c55f27b66f875857d931c329a4c884416b36a)

14 months agocrimson/os/seastore/transaction_manager: add the max_data_allocation_size
Xuehan Xu [Wed, 21 Feb 2024 06:53:33 +0000 (14:53 +0800)]
crimson/os/seastore/transaction_manager: add the max_data_allocation_size
configuration

Limit the max size of extents in seastore, which can avoid much read
amplification in case of remapping extents when extents integrity check
is mandatory

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit a2f6c7b2a8607298be18e8d4e06644da05b2bb03)

14 months agocrimson/os/seastore/lba_manager: add the alloc_extents interface
Xuehan Xu [Wed, 21 Feb 2024 06:52:41 +0000 (14:52 +0800)]
crimson/os/seastore/lba_manager: add the alloc_extents interface

This is necessary because continuous data may be splitted into multiple
extents if the size of each data extent is constrained

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit ab916224e3e254e31025fb1723ca7e1147dee2c7)

14 months agocrimson/os/seastore/lba_manager: clean up lba_manager's alloc_extent related interfaces
Xuehan Xu [Fri, 9 Feb 2024 06:34:03 +0000 (14:34 +0800)]
crimson/os/seastore/lba_manager: clean up lba_manager's alloc_extent related interfaces

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 03c8ae1ad04309b8d99b451f5e75e3c0b9233b32)

14 months agocrimson/os/seastore/cached_extent: rename get_crc32c to calc_crc32c
Xuehan Xu [Thu, 29 Feb 2024 12:17:33 +0000 (20:17 +0800)]
crimson/os/seastore/cached_extent: rename get_crc32c to calc_crc32c

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 01776a081c041ee5b949de0f1a5841ca31baa8b2)

14 months agocrimson/os/seastore/transaction_manager: check checksums when loading
Xuehan Xu [Mon, 5 Feb 2024 05:41:17 +0000 (13:41 +0800)]
crimson/os/seastore/transaction_manager: check checksums when loading
logical extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit c7764f397c1bbffa22d2a6b60f043709adfdc045)

14 months agocrimson/os/seastore: record extents' chksums in the lba tree
Xuehan Xu [Sun, 4 Feb 2024 09:54:26 +0000 (17:54 +0800)]
crimson/os/seastore: record extents' chksums in the lba tree

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit f4b570da86f77dfa08f33a0adc6135a1aa608233)

14 months agocrimson/os/seastore/transaction: change delayed_alloc_list/pre_alloc_list to CachedEx...
Xuehan Xu [Sun, 4 Feb 2024 09:23:18 +0000 (17:23 +0800)]
crimson/os/seastore/transaction: change delayed_alloc_list/pre_alloc_list to CachedExtents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit eb1cac71bc6a2905bd003fef2c16be61c604fa2b)

14 months agotest/crimson/seastore/test_object_data_handler: remap pins through the 57598/head
Xuehan Xu [Mon, 22 Apr 2024 10:11:55 +0000 (18:11 +0800)]
test/crimson/seastore/test_object_data_handler: remap pins through the
transaction that created the pins

Fixes: https://tracker.ceph.com/issues/65610
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 02acc0a71891ccadff68c60eaaf65bfa2c8bab4c)

14 months agocrimson/os/seastore/btree: assert that the transaction getting logical
Xuehan Xu [Mon, 22 Apr 2024 10:10:33 +0000 (18:10 +0800)]
crimson/os/seastore/btree: assert that the transaction getting logical
extents from pins are the one that created the pins

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit eae1b3172ee81a01403dfe8722ef6fcfee621032)

14 months agotest/neorados/misc: disable the locator key test for crimson 57597/head
Xuehan Xu [Sun, 7 Apr 2024 02:05:35 +0000 (10:05 +0800)]
test/neorados/misc: disable the locator key test for crimson

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 776c2b3e40d7e7c1819d56b02d38a87a1f4dbbf9)

14 months agotest/librados/misc_cxx: disable the locator key test for crimson
Xuehan Xu [Tue, 2 Apr 2024 10:13:54 +0000 (18:13 +0800)]
test/librados/misc_cxx: disable the locator key test for crimson

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 7a71f9a390774e868edb8c14c9cd03720f6e46da)