]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
20 months agoclient: queue a delay cap flushing if there are ditry caps/snapcaps 54472/head
Xiubo Li [Thu, 21 Sep 2023 07:02:38 +0000 (15:02 +0800)]
client: queue a delay cap flushing if there are ditry caps/snapcaps

We should queue a cap release anyway if there are dirty caps/snapcaps
for the inodes when trimming caps.

Fixes: https://tracker.ceph.com/issues/62979
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit d14752ff1f56da93ad0d9c94fcf101945fdacadd)

20 months agoMerge pull request #54268 from ronen-fr/wip-63372-pacific
Yuri Weinstein [Thu, 9 Nov 2023 17:01:41 +0000 (09:01 -0800)]
Merge pull request #54268 from ronen-fr/wip-63372-pacific

pacific: osd: fix use-after-move in build_incremental_map_msg()

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
20 months agoMerge pull request #53662 from lxbsz/wip-62523
Yuri Weinstein [Thu, 9 Nov 2023 16:59:13 +0000 (08:59 -0800)]
Merge pull request #53662 from lxbsz/wip-62523

pacific: ceph: allow xlock state to be LOCK_PREXLOCK when putting it

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53645 from vshankar/wip-61803-pacific
Yuri Weinstein [Thu, 9 Nov 2023 16:58:43 +0000 (08:58 -0800)]
Merge pull request #53645 from vshankar/wip-61803-pacific

pacific: cephfs-journal-tool: disambiguate usage of all keyword (in tool help).

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53640 from vshankar/wip-62949-pacific
Yuri Weinstein [Thu, 9 Nov 2023 16:58:06 +0000 (08:58 -0800)]
Merge pull request #53640 from vshankar/wip-62949-pacific

pacific: cephfs-mirror: do not run concurrent C_RestartMirroring context

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53362 from k0ste/wip-57110-pacific
Yuri Weinstein [Thu, 9 Nov 2023 16:57:30 +0000 (08:57 -0800)]
Merge pull request #53362 from k0ste/wip-57110-pacific

pacific: mds: replacing bootstrap session only if handle client session message

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53270 from mchangir/wip-59001-pacific
Yuri Weinstein [Thu, 9 Nov 2023 16:57:03 +0000 (08:57 -0800)]
Merge pull request #53270 from mchangir/wip-59001-pacific

pacific: cephfs_mirror: correctly set top level dir permissions

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #53169 from leonid-s-usov/bp/cap-throttle-event/pacific
Yuri Weinstein [Thu, 9 Nov 2023 16:56:20 +0000 (08:56 -0800)]
Merge pull request #53169 from leonid-s-usov/bp/cap-throttle-event/pacific

pacific: mds/Server: mark a cap acquisition throttle event in the request

Reviewed-by: Rishabh Dave <ridave@redhat.com>
20 months agoMerge pull request #54294 from ajarr/wip-63385-pacific
Yuri Weinstein [Wed, 8 Nov 2023 21:37:59 +0000 (13:37 -0800)]
Merge pull request #54294 from ajarr/wip-63385-pacific

pacific: qa/suites/rbd: add test to check rbd_support module recovery

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
20 months agoMerge pull request #54293 from ajarr/wip-63382-pacific
Yuri Weinstein [Wed, 8 Nov 2023 21:37:19 +0000 (13:37 -0800)]
Merge pull request #54293 from ajarr/wip-63382-pacific

pacific: mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
20 months agoMerge pull request #54256 from pkalever/wip-63351-pacific
Yuri Weinstein [Wed, 8 Nov 2023 21:36:35 +0000 (13:36 -0800)]
Merge pull request #54256 from pkalever/wip-63351-pacific

pacific: rbd-nbd: fix stuck with disable request

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
20 months agomgr/rbd_support: remove CreateSnapshotRequests __del__() 54293/head
Ramana Raja [Mon, 30 Oct 2023 15:05:27 +0000 (11:05 -0400)]
mgr/rbd_support: remove CreateSnapshotRequests __del__()

There is no need for CreateSnapshotRequests.__del__() that calls
CreateSnapshotRequests.wait_for_pending().
MirrorSnapshotScheduleHandler.shutdown() already calls
CreateSnapshotRequests.wait_for_pending().

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit fed1e87685a698876cf167b3681327e5b0066ee6)

Conflicts:
       src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py
 - Above conflict was due to commit e4a16e2
   ("mgr/rbd_support: add type annotation") not in pacific

20 months agomgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock
Ramana Raja [Thu, 26 Oct 2023 17:18:52 +0000 (13:18 -0400)]
mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock

The MirrorSnapshotScheduleHandler's run thread issues asynchronous
create snapshot requests using a CreateSnapshotRequests instance. When
the thread invokes a CreateSnapshotRequests instance's get_ioctx(),
the instance's class variable lock is acquired. With the class
variable lock held, the garbage collection of a CreateSnapshotRequests
instance may race in the thread. The thread would then call
CreateSnapshotRequests __del__() that tries to acquire the class
variable lock that the thread already holds. Fix this
recursive deadlock by converting the CreateSnapshotRequests lock from
a class variable to an instance variable. There is no need to share
the lock across CreateSnapshotRequests instances.

Also convert MirrorSnapshotScheduleHandler, PerfHandler and
TrashPurgeScheduleHandler class variables to instance variables
that don't need to be shared across the instances.

Fixes: https://tracker.ceph.com/issues/62994
Signed-off-by: Ramana Raja <rraja@redhat.com>
Co-Authored-By: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 4452bc22d1c6c8499cf55d6e39090adf7ae1dcbf)

 Conflicts:
src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py
src/pybind/mgr/rbd_support/perf.py
src/pybind/mgr/rbd_support/trash_purge_schedule.py
 - Above conflicts were due to commit e4a16e2
   ("mgr/rbd_support: add type annotation") not in pacific

20 months agoMerge pull request #53970 from Matan-B/wip-63179-pacific
Matan Breizman [Sun, 5 Nov 2023 08:08:07 +0000 (10:08 +0200)]
Merge pull request #53970 from Matan-B/wip-63179-pacific

pacific: osd/OSD: introduce reset_purged_snaps_last

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
20 months agoMerge PR #53103 into pacific
Patrick Donnelly [Fri, 3 Nov 2023 20:32:01 +0000 (16:32 -0400)]
Merge PR #53103 into pacific

* refs/pull/53103/head:
libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #53984 from sseshasa/wip-63185-pacific
Yuri Weinstein [Thu, 2 Nov 2023 17:45:12 +0000 (10:45 -0700)]
Merge pull request #53984 from sseshasa/wip-63185-pacific

pacific: mon/ConfigMonitor: Show localized name in "config dump --format json" output

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #53430 from pdvian/wip-fix-scrubmsg
Yuri Weinstein [Thu, 2 Nov 2023 17:44:04 +0000 (10:44 -0700)]
Merge pull request #53430 from pdvian/wip-fix-scrubmsg

pacific: osd/scrub: Fix scrub starts messages spamming the cluster log

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
20 months agoqa/suites/rbd: add test to check rbd_support module recovery 54294/head
Ramana Raja [Mon, 18 Sep 2023 02:52:56 +0000 (22:52 -0400)]
qa/suites/rbd: add test to check rbd_support module recovery

... on repeated blocklisting of its client.

There were issues with rbd_support module not being able to recover
from its RADOS client being repeatedly blocklisted. This occured for
example in clusters with OSDs slow to process RBD requests while the
module's mirror_snapshot_scheduler was taking mirror snapshots by
requesting exclusive locks on the RBD images and workloads were running
on the snapshotted images via kernel clients.

Fixes: https://tracker.ceph.com/issues/62891
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 2f2cd3bcff82afc3a4d251143eb462e700e7fc60)

20 months agoosd: fix use-after-move in build_incremental_map_msg() 54268/head
Ronen Friedman [Wed, 25 Oct 2023 07:24:18 +0000 (02:24 -0500)]
osd: fix use-after-move in build_incremental_map_msg()

Fixes: https://tracker.ceph.com/issues/63310
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit 9e2b8b0e8235b36e55310aab49b8f760e8d57cad)

20 months agoMerge pull request #53135 from ifed01/wip-ifed-verbose-open-col-pac
Igor Fedotov [Tue, 31 Oct 2023 09:20:22 +0000 (12:20 +0300)]
Merge pull request #53135 from ifed01/wip-ifed-verbose-open-col-pac

pacific: osd,bluestore: gracefully handle a failure during meta collection load

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
20 months agoMerge pull request #53587 from ifed01/wip-ifed-vselector-53906-pac
Yuri Weinstein [Mon, 30 Oct 2023 18:12:03 +0000 (11:12 -0700)]
Merge pull request #53587 from ifed01/wip-ifed-vselector-53906-pac

pacific: bluestore: Fix problem with volume selector

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
20 months agoMerge pull request #52948 from ifed01/wip-ifed-fix-55260-pac
Yuri Weinstein [Mon, 30 Oct 2023 18:08:58 +0000 (11:08 -0700)]
Merge pull request #52948 from ifed01/wip-ifed-fix-55260-pac

pacific: os/bluestore: don't require bluestore_db_block_size when attaching new

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
20 months agotest/librbd/fsx: wait for resize to propagate in krbd_resize() 54256/head
Prasanna Kumar Kalever [Fri, 20 Oct 2023 10:11:05 +0000 (15:41 +0530)]
test/librbd/fsx: wait for resize to propagate in krbd_resize()

With this changes resize request will not be blocked until the resize is
completed. Because of this the fsx test fails as it assumes that the
request to resize immediately implies changes on the device size.

Hence we have to add a wait in resize handler of fsx for the device to
actually get resized.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 6f3d0f570f1a262b06d4c661582091d8ddb11bfa)

20 months agorbd-nbd: fix stuck with disable request
Prasanna Kumar Kalever [Tue, 12 Sep 2023 12:15:05 +0000 (17:45 +0530)]
rbd-nbd: fix stuck with disable request

Problem:
-------
Trying to disable any feature on an rbd image mapped with nbd leads to stuck
in rbd-nbd.

The rbd-nbd registers a watcher callback to detect image resize in
NBDWatchCtx::handle_notify(). The handle_notify calls image info method, which
calls refresh_if_required and it got stuck there.

It is getting stuck in ImageState::refresh_if_required() because
DisableFeaturesRequest issues update notifications while still holding onto
the exclusive lock with everything that has to do with it blocked.

Solution:
--------
Set only notify flag as part of NBDWatchCtx::handle_notify() and handle
the resize detection part as part of a different thread.

Fixes: https://tracker.ceph.com/issues/58740
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit dbb4daff404c5d2da32c33f4e852e84a257c0b8d)

20 months agoMerge pull request #54053 from idryomov/wip-63028-pacific
Yuri Weinstein [Mon, 30 Oct 2023 15:22:43 +0000 (08:22 -0700)]
Merge pull request #54053 from idryomov/wip-63028-pacific

pacific: pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
20 months agoMerge pull request #54144 from rishabh-d-dave/wip-62879-pacific
Anthony D'Atri [Tue, 24 Oct 2023 23:51:40 +0000 (19:51 -0400)]
Merge pull request #54144 from rishabh-d-dave/wip-62879-pacific

pacific: cephfs: upgrade cephfs-shell's path wherever necessary

20 months agoqa/tasks/ceph_manager: thrash - add reset_purged_snaps_last 53970/head
Matan Breizman [Thu, 31 Aug 2023 09:55:33 +0000 (09:55 +0000)]
qa/tasks/ceph_manager: thrash - add reset_purged_snaps_last

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 680e088b8d48e51dfb3aaa4207de67025a0fdabd)

20 months agoosd/OSD: introduce reset_purged_snaps_last
Matan Breizman [Thu, 21 Sep 2023 12:10:07 +0000 (12:10 +0000)]
osd/OSD: introduce reset_purged_snaps_last

When the OSD preboots it sends a MMonGetPurgedSnaps message to
the monitor (`_get_purged_snaps`).
The monitor will reply with all the purged snapshots that their purged_epoch_ is in the
range of superblock.purged_snaps_last + 1 up to the last superblock.current_epoch + 1.
When the OSD will handle the reply from the mon (`handle_get_purged_snaps_reply`)
it will call `record_purged_snaps` to write those purged snapshots in the
OSD store as well (PSN_ keys).

Once purged_snaps_last is reset, in the following OSD reboot, the snapshots that were marked as
purged (purged_snaps_ keys) in the mon's store will be also marked,
correspondingly, in the OSD store.
That way `scrub_purged_snaps` will be able to re-trim the snapshots that weren't
marked as purged in the OSD side (for some reason)

Fixes: https://tracker.ceph.com/issues/62981
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 120ed0f0e8f65c18bfcd1d649617770c2c5af663)
Manual conflict fixes: 'scrubdebug' command was removed since it's
                       not part of the original commit.

                       write_superblock() parameters were changed

20 months agoMerge pull request #53758 from cbodley/wip-63040-pacific
Yuri Weinstein [Mon, 23 Oct 2023 15:26:06 +0000 (08:26 -0700)]
Merge pull request #53758 from cbodley/wip-63040-pacific

pacific: [CVE-2023-43040] rgw: Fix bucket validation against POST policies

Reviewed-by: Casey Bodley <cbodley@redhat.com>
20 months agoMerge pull request #49477 from aaSharma14/wip-58299-pacific
Pedro Gonzalez Gomez [Mon, 23 Oct 2023 08:23:05 +0000 (10:23 +0200)]
Merge pull request #49477 from aaSharma14/wip-58299-pacific

pacific: mgr/dashboard: Fix CephPoolGrowthWarning alert

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
20 months agocephfs: upgrade cephfs-shell's path wherever necessary 54144/head
Rishabh Dave [Mon, 20 Feb 2023 04:47:10 +0000 (10:17 +0530)]
cephfs: upgrade cephfs-shell's path wherever necessary

Commit dc69033763cc116c6ccdf1f97149a74248691042 moves cephfs-shell from
"<CEPH-REPO-ROOT>/src/tools/cephfs/" to
"<CEPH-REPO-ROOT>/src/tools/cephfs/shell" but cephfs-shell's location in
src/vstart.sh and qa/tasks/cephfs/test_cephfs_shell.py is left
un-updated. This produces a broken vstart_environment.sh and broken
export command in test_cephfs_shell.py.

Introduced-by: dc69033763cc116c6ccdf1f97149a74248691042
Fixes: https://tracker.ceph.com/issues/58795
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 48ef0444774934dd6d0d3e026142d95e4098bebd)

 Conflicts:
qa/tasks/cephfs/test_cephfs_shell.py
- Comment present at the top of file was different in Pacific
  compared to main branch.

21 months agoMerge pull request #53808 from cfsnyder/wip-62945-pacific
Yuri Weinstein [Fri, 20 Oct 2023 14:53:53 +0000 (07:53 -0700)]
Merge pull request #53808 from cfsnyder/wip-62945-pacific

pacific: rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53784 from idryomov/wip-63010-pacific
Yuri Weinstein [Fri, 20 Oct 2023 14:53:04 +0000 (07:53 -0700)]
Merge pull request #53784 from idryomov/wip-63010-pacific

pacific: qa/suites/krbd: stress test for recovering from watch errors

Reviewed-by: Mykola Golub <mgolub@suse.com>
21 months agoMerge pull request #53562 from cfsnyder/wip-58787-pacific
Yuri Weinstein [Fri, 20 Oct 2023 14:51:43 +0000 (07:51 -0700)]
Merge pull request #53562 from cfsnyder/wip-58787-pacific

pacific: rgwlc: prevent lc for one bucket from exceeding time budget

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
21 months agoMerge pull request #53295 from ajarr/wip-62686-pacific
Yuri Weinstein [Fri, 20 Oct 2023 14:50:25 +0000 (07:50 -0700)]
Merge pull request #53295 from ajarr/wip-62686-pacific

pacific: librbd: kick ExclusiveLock state machine on client being blocklisted when waiting for lock

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
21 months agoMerge pull request #53274 from idryomov/wip-61707-pacific
Yuri Weinstein [Fri, 20 Oct 2023 14:49:22 +0000 (07:49 -0700)]
Merge pull request #53274 from idryomov/wip-61707-pacific

pacific: librbd: make CreatePrimaryRequest remove any unlinked mirror snapshots

Reviewed-by: Ramana Raja <rraja@redhat.com>
21 months agoMerge pull request #53978 from adk3798/wip-63115-pacific
Adam King [Tue, 17 Oct 2023 19:14:10 +0000 (15:14 -0400)]
Merge pull request #53978 from adk3798/wip-63115-pacific

pacific: mgr/cephadm: ceph orch add fails when ipv6 address is surrounded by square brackets.

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #53977 from adk3798/wip-62802-pacific
Adam King [Tue, 17 Oct 2023 19:13:25 +0000 (15:13 -0400)]
Merge pull request #53977 from adk3798/wip-62802-pacific

pacific: cephadm: run tcmu-runner through script to do restart on failure

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
21 months agoMerge pull request #53975 from adk3798/wip-62469-pacific
Adam King [Tue, 17 Oct 2023 19:12:14 +0000 (15:12 -0400)]
Merge pull request #53975 from adk3798/wip-62469-pacific

pacific: cephadm: add tcmu-runner to logrotate config

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
21 months agoMerge pull request #53974 from adk3798/wip-62448-pacific
Adam King [Tue, 17 Oct 2023 19:11:24 +0000 (15:11 -0400)]
Merge pull request #53974 from adk3798/wip-62448-pacific

pacific: mgr/cephadm: Add "networks" parameter to orch apply rgw

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
21 months agoMerge pull request #53469 from adk3798/pacific-tcmu-custom-configs
Adam King [Tue, 17 Oct 2023 19:10:24 +0000 (15:10 -0400)]
Merge pull request #53469 from adk3798/pacific-tcmu-custom-configs

pacific: cephadm: make custom_configs work for tcmu-runner container

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #52413 from adk3798/wip-61686-pacific
Adam King [Tue, 17 Oct 2023 19:09:29 +0000 (15:09 -0400)]
Merge pull request #52413 from adk3798/wip-61686-pacific

pacific: python-common/drive_group: handle fields outside of 'spec' even when 'spec' is provided

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
21 months agoMerge pull request #52412 from adk3798/wip-61683-pacific
Adam King [Tue, 17 Oct 2023 19:08:48 +0000 (15:08 -0400)]
Merge pull request #52412 from adk3798/wip-61683-pacific

pacific: python-common/drive_selection: lower log level of limit policy message

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
21 months agoMerge pull request #52411 from adk3798/wip-61544-pacific
Adam King [Tue, 17 Oct 2023 19:07:30 +0000 (15:07 -0400)]
Merge pull request #52411 from adk3798/wip-61544-pacific

pacific: cephadm: Adding support to configure public_network cfg section

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
21 months agoMerge pull request #52083 from adk3798/wip-61677-pacific
Adam King [Tue, 17 Oct 2023 19:06:04 +0000 (15:06 -0400)]
Merge pull request #52083 from adk3798/wip-61677-pacific

pacific: cephadm: allow ports to be opened in firewall during adoption, reconfig, redeploy

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
21 months agocephadm: make custom_configs work for tcmu-runner container 53469/head
Adam King [Mon, 21 Aug 2023 17:48:56 +0000 (13:48 -0400)]
cephadm: make custom_configs work for tcmu-runner container

This is intended to be a temporary workaround to make
custom config files be able to be mounted into
the tcmu-runner container. The hope is to refactor
cephadm's iscsi handling for squid, but a patch
like this could be useful for iscsi in older
releases where currently custom config files
are unusable for the tcmu-runner container

What this patch actually does is have us write the
custom config files to a dir for the tcmu-runner
container so that the rest of the logic works without
change. I thought this would be easier to remove later
than a patch that integrates more with the container
mounts or general deployment

The use case in mind is something like

service_type: iscsi
service_id: foo
service_name: iscsi.foo
placement:
  hosts:
  - host1
custom_configs:
  -  mount_path: /etc/tcmu/tcmu.conf
     content: |
       log_level = 4
spec:
  api_password: admin
  api_port: 5000
  api_user: admin
  pool: foo

which would allow users to modify the logging of the
tcmu-runner container for debugging purposes

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit de92392708bf456bba975cc18b3138035d79ae05)

21 months agomgr/rbd_support: make type hits on aio_mirror_image_*() callbacks better 54053/head
Ilya Dryomov [Thu, 12 Oct 2023 19:32:53 +0000 (21:32 +0200)]
mgr/rbd_support: make type hits on aio_mirror_image_*() callbacks better

Make it clear that mirror mode, mirror info and snap ID can be None if
the respective operation fails.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 01fff6a72a328459c1d153e5dc1de6a34e48a82f)

Conflicts:
src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py [ commit
  e4a16e261370 ("mgr/rbd_support: add type annotation") not in
  pacific ]

21 months agopybind/rbd: don't produce info on errors in aio_mirror_image_get_info()
Ilya Dryomov [Thu, 12 Oct 2023 17:03:10 +0000 (19:03 +0200)]
pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()

Check completion return value before attemting to decode c_info.
Otherwise we are guaranteed to access invalid memory in decode_cstr()
while trying to compute global_id string length when the client is
blocklisted for example.

Fixes: https://tracker.ceph.com/issues/63028
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a81bd2db3af4d7b53736be8e42a3eaa53028d60c)

21 months agoPendingReleaseNotes: Note change to 'ceph config dump' pretty-print output. 53984/head
Sridhar Seshasayee [Tue, 29 Aug 2023 04:29:15 +0000 (09:59 +0530)]
PendingReleaseNotes: Note change to 'ceph config dump' pretty-print output.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 401b30f19f51e86f1471447a6af788b94e283ff0)
Conflicts:
    PendingReleaseNotes
- Remove unrelated release note related to Cephfs
- Move related release note under the new ">=16.2.15" section

21 months agomon/ConfigMonitor: Show localized name in "config dump --format json" output
Sridhar Seshasayee [Wed, 9 Aug 2023 12:52:29 +0000 (18:22 +0530)]
mon/ConfigMonitor: Show localized name in "config dump --format json" output

The "ceph config dump" command without the json formatted output shows
the localized option names and their values. An example of a normalized
vs localized option is shown below:

Normalized: mgr/dashboard/ssl_server_port (maintaned within Option struct)
Localized: mgr/dashboard/x/ssl_server_port (maintained in mon store)

But the "ceph config dump --format json*" output showed the normalized
option names which was not consistent with the "config dump" output.
The output of the command along with variations for pretty printing must
show the same content.

This commit introduces a new member within the ConfigMap's MaskedOption
struct called "localized_name". This is initialized to the localized name
as part of ConfigMonitor::load_config() method.

The MaskedOption::dump() used for the json formatting is modified to
display the localized_name instead of the normalized name.

Fixes: https://tracker.ceph.com/issues/62379
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 3821722e5660437298a7c0f41e1061d363090103)

21 months agoceph orch add fails when ipv6 address is surrounded by square brackets. 53978/head
Teoman ONAY [Mon, 3 Jul 2023 14:00:20 +0000 (16:00 +0200)]
ceph orch add fails when ipv6 address is surrounded by square brackets.

fixes: https://tracker.ceph.com/issues/61885
fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2153448

Signed-off-by: Teoman ONAY <tonay@ibm.com>
(cherry picked from commit 1ea71bee6197ed0357b586498a43d9d726160a43)

21 months agocephadm: run tcmu-runner through script to do restart on failure 53977/head
Adam King [Tue, 13 Jun 2023 23:54:30 +0000 (19:54 -0400)]
cephadm: run tcmu-runner through script to do restart on failure

Currently, cephadm runs tcmu-runner as a background
process inside the unit file deployed for iscsi
(rbd-target-api is the primary process). This means
if tcmu-runner crashes for whatever reason, systemd
will not attempt to restart it. This commits sets
up a script to serve as the container entrypoint
for the tcmu-runner container that will run
tcmu-runner and also restart it on failure
(unless there are too many failures in a short
period, at which point it gives up).

The hope is to eventually drop use of this script
for a better solution in squid onward, but this
should be helpful on older releases (quincy and
pacific at least) where we won't be able to
bring that better solution

Fixes: https://tracker.ceph.com/issues/61667
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 47eb6b3f62afe993073429b02051ae0343d7aea3)

Conflicts:
src/cephadm/cephadm
src/cephadm/tests/test_cephadm.py

21 months agocephadm: Fix extra_container_args for iSCSI
Raimund Sacherer [Fri, 26 May 2023 15:52:57 +0000 (17:52 +0200)]
cephadm: Fix extra_container_args for iSCSI

extra_container_args where only applied for rbd_target_api container and not for
tcmu-runner container.

Signed-off-by: Raimund Sacherer <rsachere@redhat.com>
(cherry picked from commit ad60fc3db644b8bf44a582e79888e2fb15d7ce3a)

Conflicts:
src/cephadm/cephadm

21 months agocephadm: add tcmu-runner to logrotate config 53975/head
Adam King [Fri, 2 Jun 2023 00:06:35 +0000 (20:06 -0400)]
cephadm: add tcmu-runner to logrotate config

This process could be used to set up the tcmu-runner
to log to a file much like other ceph daemons

- create /etc/tcmu directory
- create /etc/tcmu/tcmu.conf directory with default options
- change dir to /var/log
- change log level to 4
- add -v /etc/tcmu:/etc/tcmu to tcmu-runner container podman line in unit.run

In order to support this (mostly for debugging) we should
add tcmu-runner to the logrotate config

Fixes: https://tracker.ceph.com/issues/61571
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit d5d40e07cae8a1d6a94029c4354d146b0baa3971)

21 months agomgr/cephadm: Add "networks" parameter to orch apply rgw 53974/head
Teoman ONAY [Wed, 26 Jul 2023 14:40:11 +0000 (16:40 +0200)]
mgr/cephadm: Add "networks" parameter to orch apply rgw

This parameter is available in specs but not available as a parameter.
Having it will ease its use in cephadm-adopt playbook in ceph-ansible.

fixes: https://tracker.ceph.com/issues/62185

Signed-off-by: Teoman ONAY <tonay@ibm.com>
(cherry picked from commit 7f33397c76540dff8ed724caf2aa14ac94c73e03)

21 months agolibrbd/ManagedLock: kickstart ExclusiveLock state machine 53295/head
Ramana Raja [Mon, 2 Oct 2023 16:39:26 +0000 (12:39 -0400)]
librbd/ManagedLock: kickstart ExclusiveLock state machine

... that is stalled waiting for lock. Do this when trying to reacquire
lock in the ImageWatcher's rewatch mechanism. This would enable the
ExclusiveLock state machine to propagate the blocklist error to the
caller trying to perform an image operation requiring an exclusive
lock.

Previous attempt, e66db763, to fix the hang due to exclusive lock
acquisiton (stuck waiting for lock) racing with client blocklisting
did not always work. e66db763 kickstarted the ExclusiveLock state
machine when the ImageWatcher tried to schedule a exclusive lock
request and the blocklisting was detected. However, there is a short
window between a watch getting deregistered and client blocklisting
getting detected as part of rewatching. If hit when trying to schedule
a lock request, the ExclusiveLock state machine wasn't kickstarted,
blocklist error wasn't propagated, and the hang resurfaced.

A more robust approach is taken to resume the ExclusiveLock state
machine stuck waiting for lock during client blocklisting. Whenever
a client's ImageWatcher loses connection to the cluster, as it happens
during blocklising, the ImageWatcher initiates a mechanism to rewatch
the image and tries to reacquire the lock. Piggyback on this rewatch
mechanism that gets triggered during client blocklisting. And when
trying to reacquire the lock, kickstart the ExclusiveLock state
machine stalled waiting for lock (STATE_WAITING_FOR_LOCK).

Fixes: https://tracker.ceph.com/issues/63009
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 18b018578cf8ac51a7e7a7d25f62d7bde345461a)

21 months agoMerge pull request #49478 from aaSharma14/wip-58301-pacific
Pedro Gonzalez Gomez [Tue, 10 Oct 2023 08:23:59 +0000 (10:23 +0200)]
Merge pull request #49478 from aaSharma14/wip-58301-pacific

pacific: mgr/dashboard: fix CephPGImbalance alert

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
21 months agoMerge pull request #48524 from k0ste/wip-57887-pacific
Avan [Thu, 5 Oct 2023 09:35:07 +0000 (15:05 +0530)]
Merge pull request #48524 from k0ste/wip-57887-pacific

pacific: mgr/prometheus: avoid duplicates and deleted entries for rbd_stats_pools

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
21 months agorgw: fix output formatting of bucket index check admin api 53808/head
Cory Snyder [Mon, 25 Sep 2023 10:06:41 +0000 (10:06 +0000)]
rgw: fix output formatting of bucket index check admin api

The bucket index check admin API was previously returning invalid
JSON.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 32fb6a1a68398a99324b2e64ebe3bcf3a9ccf02a)

21 months agorgw: fix radosgw-admin bucket check stat calculation bug
Cory Snyder [Fri, 22 Sep 2023 21:08:25 +0000 (21:08 +0000)]
rgw: fix radosgw-admin bucket check stat calculation bug

Fixes a regression with radosgw-admin bucket check stat
calculation and bucket reshard stat calculation when
there are objects that have transitioned from unversioned
to versioned. The bug was introduced in
152aadb71b61c53a4832a1c8cf82fce3d64b68d1.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 4728daa5557bfb79a608dd903b8630e2b15fcb2c)

21 months agorgw: add test case to reproduce bucket check stats bug for versioned bucket
Cory Snyder [Fri, 22 Sep 2023 21:00:46 +0000 (21:00 +0000)]
rgw: add test case to reproduce bucket check stats bug for versioned bucket

Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 340522f9aed50d65137568c1f9dcf4b1e7945a79)

21 months agorgw: radosgw-admin bucket check should only print index entries with --check-objects...
Cory Snyder [Fri, 22 Sep 2023 08:35:16 +0000 (08:35 +0000)]
rgw: radosgw-admin bucket check should only print index entries with --check-objects flag

Printing all index entries can be very time consuming for large
buckets and the inability to switch this behavior off makes it
cumbersome to use the command for fixing bucket stats. This was
also preventing the command from outputting recalculated bucket
stats when the --fix flag wasn't specified.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 6b057fe55413c0eaf9959f006584cba6cc4c192a)

Conflicts:
src/rgw/driver/rados/rgw_bucket.cc

Cherry-pick notes:
- Conflicts due to rgw_bucket.cc moved into driver dir in later versions

21 months agorgw: prevent another leftover bucket index olh entry scenario
Cory Snyder [Thu, 21 Sep 2023 19:27:51 +0000 (19:27 +0000)]
rgw: prevent another leftover bucket index olh entry scenario

If a call to bucket_index_link_olh or bucket_index_unlink_instance
fails, its associated pending xattr may have prevented the olh object
from being removed by another thread. We should do a best effort
cleanup attempt for this case by calling update_olh before returning
an error to the caller.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 570adec5bb8142f5baf1f05f0040e8afdb11ec05)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc

Cherry-pick notes:
- Conflicts due to rgw_rados file being moved into driver dir in later versions

21 months agorgw: fix rgw versioned bucket stat accounting during reshard and check index
Cory Snyder [Thu, 7 Sep 2023 17:23:14 +0000 (17:23 +0000)]
rgw: fix rgw versioned bucket stat accounting during reshard and check index

Fixes: https://tracker.ceph.com/issues/62760
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 152aadb71b61c53a4832a1c8cf82fce3d64b68d1)

Conflicts:
src/cls/rgw/cls_rgw.cc

Cherry-pick notes:
- Some function ordering changed

21 months agoqa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Cory Snyder [Thu, 7 Sep 2023 14:43:23 +0000 (14:43 +0000)]
qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit b79dcf640ac2cc3dacf1b87bbe351db823c445d0)

21 months agorgw: add radosgw-admin bucket check olh/unlinked commands
Cory Snyder [Thu, 25 May 2023 16:59:04 +0000 (16:59 +0000)]
rgw: add radosgw-admin bucket check olh/unlinked commands

Adds commands to radosgw-admin for checking for and fixing leftover
entries in the bucket index (and associated RADOS objects).

Fixes: https://tracker.ceph.com/issues/62075
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 9b2042a0750109f575b02c9c4b127dd941994662)

Conflicts:
src/rgw/driver/rados/rgw_bucket.cc
src/rgw/driver/rados/rgw_bucket.h
src/rgw/driver/rados/rgw_rados.cc
src/rgw/rgw_admin.cc

Cherry-pick notes:
- conflicts due to rados files being moved into driver directory
- conflicts due to rename of rados classes

21 months agoMerge pull request #53781 from rhcs-dashboard/wip-63070-pacific
Nizamudeen A [Tue, 3 Oct 2023 11:12:36 +0000 (16:42 +0530)]
Merge pull request #53781 from rhcs-dashboard/wip-63070-pacific

pacific: mgr/dashboard: allow tls 1.2 with a config option

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
21 months agoqa/suites/krbd: stress test for recovering from watch errors 53784/head
Ilya Dryomov [Wed, 27 Sep 2023 18:17:11 +0000 (20:17 +0200)]
qa/suites/krbd: stress test for recovering from watch errors

Fixes: https://tracker.ceph.com/issues/63010
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 237aa221ebad429457a621d2e38cfdf0025e38f9)

21 months agoqa/suites/krbd: rename singleton to singleton-msgr-failures
Ilya Dryomov [Thu, 28 Sep 2023 09:52:06 +0000 (11:52 +0200)]
qa/suites/krbd: rename singleton to singleton-msgr-failures

A "singleton without msgr-failures" is wanted in the next commit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 2094a0450d071ed5fc430a5cdfa524a69098fbd4)

21 months agomgr/dashboard: allow tls 1.2 with a config option 53781/head
Nizamudeen A [Wed, 27 Sep 2023 11:27:32 +0000 (16:57 +0530)]
mgr/dashboard: allow tls 1.2 with a config option

Provide the option to allow tls1.2

`ceph dashboard set-enable-unsafe-tls-v1-2 True` followed with a mgr
restart will enable tls 1.2.

With tls1.2 enabled
```
╰─$ nmap -sV --script ssl-enum-ciphers -p 11000 127.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-09-27 16:56 IST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00018s latency).

PORT      STATE SERVICE  VERSION
11000/tcp open  ssl/http CherryPy wsgiserver
|_http-server-header: Ceph-Dashboard
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 16.55 seconds
```

Without tls1.2 enabled (which defaults to tls 1.3)
```
╰─$ nmap -sV --script ssl-enum-ciphers -p 11000 127.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-09-27 16:54 IST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000075s latency).

PORT      STATE SERVICE  VERSION
11000/tcp open  ssl/http CherryPy wsgiserver
| ssl-enum-ciphers:
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A
|_http-server-header: Ceph-Dashboard
```

Fixes: https://tracker.ceph.com/issues/62940
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 219c62bea60083c0e59a86454b0cecf8afaf9780)

21 months agorgw: Fix bucket validation against POST policies 53758/head
Joshua Baergen [Wed, 17 May 2023 18:17:09 +0000 (12:17 -0600)]
rgw: Fix bucket validation against POST policies

It's possible that user could provide a form part as a part of a POST
object upload that uses 'bucket' as a key; in this case, it was
overriding what was being set in the validation env (which is the real
bucket being modified). The result of this is that a user could actually
upload to any bucket accessible by the specified access key by matching
the bucket in the POST policy in said POST form part.

Fix this simply by setting the bucket to the correct value after the
POST form parts are processed, ignoring the form part above if
specified.

Fixes: https://tracker.ceph.com/issues/63004
Signed-off-by: Joshua Baergen <jbaergen@digitalocean.com>
(cherry picked from commit 98bfb71cb38899333deb58dd2562037450fd7fa8)

Conflicts:
src/rgw/rgw_rest_s3.cc get_params() conflict with sse-s3 stuff

21 months agoceph: allow xlock state to be LOCK_SYNC when putting it 53662/head
Xiubo Li [Tue, 26 Sep 2023 04:55:41 +0000 (12:55 +0800)]
ceph: allow xlock state to be LOCK_SYNC when putting it

When the journal logs are successfully flushed the lockers' state
could be LOCK_SYNC during the xlock count is non-zero.

Fixes: https://tracker.ceph.com/issues/44565
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit cf42b377c4be665765f4e2136f5ee8e73a6289e5)

21 months agoceph: allow xlock state to be LOCK_PREXLOCK when putting it
Xiubo Li [Tue, 26 Sep 2023 04:54:58 +0000 (12:54 +0800)]
ceph: allow xlock state to be LOCK_PREXLOCK when putting it

When the journal logs are successfully flushed the lockers' state
could be LOCK_PREXLOCK if the inflight OP need to gather issued
caps.

Fixes: https://tracker.ceph.com/issues/44565
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 81890e8a57341b3a14db310ae1700394ecdbbad4)

21 months agocephfs-journal-tool: disambiguate usage of all keyword (in tool help). 53645/head
Manish M Yathnalli [Wed, 21 Jun 2023 07:59:41 +0000 (13:29 +0530)]
cephfs-journal-tool: disambiguate usage of all keyword (in tool help).

        The fs:all for rank option description was confusing. It seemd
like the fs was optional, but it is mandatory. This change modifies the
help message to reflect the correct way to use all in the --rank option.

Fixes: https://tracker.ceph.com/issues/61753
Signed-off-by: Manish M Yathnalli <myathnal@redhat.com>
(cherry picked from commit 52c033f85e274c86bd75f0eb902a32d86356094e)

21 months agoMerge pull request #52750 from mchangir/wip-61989-pacific
Venky Shankar [Mon, 25 Sep 2023 07:49:08 +0000 (13:19 +0530)]
Merge pull request #52750 from mchangir/wip-61989-pacific

pacific: mgr/snap_schedule: allow retention spec 'n' to be user defined

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agocephfs-mirror: restart failed/blocklisted replayer instances 53640/head
Venky Shankar [Wed, 2 Aug 2023 05:39:00 +0000 (01:39 -0400)]
cephfs-mirror: restart failed/blocklisted replayer instances

This was buggy right from the start. Start maintaining per replayer
blocklisted or failed timestamp and use that to check if a replayer
restart is required.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f2fb84ddd25ac66ca1e79339801913b35b597e83)

21 months agocephfs-mirror: do not run concurrent C_RestartMirroring context
Venky Shankar [Wed, 2 Aug 2023 05:37:58 +0000 (01:37 -0400)]
cephfs-mirror: do not run concurrent C_RestartMirroring context

Fixes: http://tracker.ceph.com/issues/62072
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b4d8f99c2aaef3b3dd74a4840c21dcb3ca9eb81a)

22 months agoos/bluestore/bluefs: Make volume selector operations atomic 53587/head
Adam Kupczyk [Wed, 2 Feb 2022 19:28:14 +0000 (20:28 +0100)]
os/bluestore/bluefs: Make volume selector operations atomic

Make all RocksDBBlueFSVolumeSelector files/extents/size tracking atomic.
It used to be synchronized by BlueFS global lock.
Now, in Fine Grain Locking era, it is necessary to prevent corruption.

Fixes: https://tracker.ceph.com/issues/53906
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 372bda350966624d5081635e659f7c46947980c2)

22 months agoos/bluestore/bluefs: Code for volume selector check
Adam Kupczyk [Thu, 20 Jan 2022 12:44:35 +0000 (13:44 +0100)]
os/bluestore/bluefs: Code for volume selector check

Adds ability to verify that volume selector properly tracks disk usage.
Creates options:
- bluefs_check_volume_selector_on_umount
- bluefs_check_volume_selector_often
that can be used to validate that vselector does not diverge from
values it should have.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit d233e3b1d23c135f0ec8d808c0961ddce8526bc8)

 Conflicts:
src/common/options/global.yaml.in
(new yaml config settings aren't present in Pacific)

22 months agoMerge pull request #53560 from batrick/wip-57776-pacific
zdover23 [Thu, 21 Sep 2023 05:01:51 +0000 (15:01 +1000)]
Merge pull request #53560 from batrick/wip-57776-pacific

pacific: doc: discuss the standard multi-tenant CephFS security model

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agorgwlc: prevent lc for one bucket from exceeding time budget 53562/head
Matt Benjamin [Mon, 31 Oct 2022 16:40:50 +0000 (12:40 -0400)]
rgwlc: prevent lc for one bucket from exceeding time budget

Fixes: https://tracker.ceph.com/issues/57951
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 617ffccbca0169ac0f1cd713962d44e8cc74a8af)

22 months agodoc: discuss the standard multi-tenant CephFS security model 53560/head
Greg Farnum [Fri, 30 Sep 2022 19:34:27 +0000 (19:34 +0000)]
doc: discuss the standard multi-tenant CephFS security model

Fixes: https://tracker.ceph.com/issues/57737
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 91e7c7de6a5ccb44e9cbf3fffe258c952f733fe8)

22 months agoMerge pull request #52949 from k0ste/wip-61723-pacific
Adam Kupczyk [Tue, 19 Sep 2023 13:29:44 +0000 (15:29 +0200)]
Merge pull request #52949 from k0ste/wip-61723-pacific

pacific: Bluestore: fix bluestore collection_list latency perf counter

22 months agoMerge pull request #52821 from rishabh-d-dave/wip-62337-pacific
Laura Flores [Sun, 17 Sep 2023 22:34:33 +0000 (18:34 -0400)]
Merge pull request #52821 from rishabh-d-dave/wip-62337-pacific

pacific: mds: use variable g_ceph_context directly in MDSAuthCaps

22 months agoMerge pull request #53416 from trociny/wip-62505-pacific
Casey Bodley [Fri, 15 Sep 2023 15:03:48 +0000 (16:03 +0100)]
Merge pull request #53416 from trociny/wip-62505-pacific

pacific: rgw: handle http options CORS with v4 auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoMerge PR #53447 into pacific
Patrick Donnelly [Wed, 13 Sep 2023 18:17:10 +0000 (14:17 -0400)]
Merge PR #53447 into pacific

* refs/pull/53447/head:
qa: remove duplicate import

Reviewed-by: Casey Bodley <cbodley@redhat.com>
22 months agoqa: remove duplicate import 53447/head
Patrick Donnelly [Wed, 13 Sep 2023 16:36:04 +0000 (12:36 -0400)]
qa: remove duplicate import

To fix tox error.

Fixes: c09558d8de0defb2695bb345c30387843e65cd93
Fixes: 469f3502d48d6ccfeba4d2ac2446ef4e47665f53
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
22 months agopacific: osd/scrub: Fix scrub starts messages spamming the cluster log 53430/head
Prashant D [Tue, 12 Sep 2023 18:58:23 +0000 (14:58 -0400)]
pacific: osd/scrub: Fix scrub starts messages spamming the cluster log

With the re-introduction of scrub starts message in commit-id e0c0b4f,
the cluster log is getting spammed by scrub *starts* for the same
PG. This is due to replicas rejecting the scrub reserve requests
resulting in scrub getting rescheduled for the same PG continuously.
Instead of logging the scrub *starts* message before scrub reservation
is done by all acting set OSDs, log *starts* message when active
scrubbing starts for the PG. The reservations period is expected
to take up to a few milliseconds and the scrubbing itself consumes the
most of the scrub period.

Fixes: https://tracker.ceph.com/issues/62669
Signed-off-by: Prashant D <pdhange@redhat.com>
22 months agoMerge pull request #53403 from rishabh-d-dave/wip-62807-pacific
zdover23 [Tue, 12 Sep 2023 10:39:13 +0000 (20:39 +1000)]
Merge pull request #53403 from rishabh-d-dave/wip-62807-pacific

pacific: doc/cephfs: write cephfs commands fully in docs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agorgw/auth: move http options v4 logic to get_v4_canonical_method() 53416/head
Tobias Urdin [Thu, 17 Aug 2023 11:45:06 +0000 (11:45 +0000)]
rgw/auth: move http options v4 logic to get_v4_canonical_method()

Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
(cherry picked from commit dbc0a4ed2dd6a92d06e585dda25b1325464efefb)

Conflicts:
src/rgw/rgw_auth_s3.cc (trivial)
src/rgw/rgw_auth_s3.h (trivial)

22 months agorgw/auth: handle HTTP OPTIONS with v4 auth
Tobias Urdin [Mon, 7 Aug 2023 20:34:43 +0000 (20:34 +0000)]
rgw/auth: handle HTTP OPTIONS with v4 auth

This adds code to properly verify the signature
for HTTP OPTIONS calls that is preflight CORS
requests passing the expected method in the
access-control-request-method header.

Fixes: https://tracker.ceph.com/issues/62033
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
(cherry picked from commit 814cae52baf04d9c3f0f3bd04b8e2daf16eb3874)

Conflicts:
src/rgw/rgw_cors.h (trivial)
src/rgw/rgw_op.cc (different signature for validate_cors_rule_method)
src/rgw/rgw_rest_s3.cc (trivial)

22 months agorgw/auth: check auth for http options requests
Tobias Urdin [Thu, 27 Jul 2023 14:59:45 +0000 (14:59 +0000)]
rgw/auth: check auth for http options requests

If we get a HTTP OPTIONS request we previously always
allowed the anonymous engine to handle the request even
if other auth was given in the request.

Fixes: https://tracker.ceph.com/issues/62033
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
(cherry picked from commit 135b6fc20e8ad094ed94e05bfb1975578abbbdab)

22 months agodoc/cephfs: write cephfs commands fully in docs 53403/head
Rishabh Dave [Mon, 11 Sep 2023 09:55:46 +0000 (15:25 +0530)]
doc/cephfs: write cephfs commands fully in docs

We write CephFS commands incompletely in docs. For example, "ceph tell
mds.a help" is simply written as "tell mds.a help". This might confuse
the reader and it won't harm to write the command in full.

Fixes: https://tracker.ceph.com/issues/62791
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit e63b573d3edc272d83ee1b5eb3dace037f762d87)

22 months agomds: replacing bootstrap session only if handle client session message 53362/head
Mer.xuanyi [Mon, 20 Jun 2022 08:37:50 +0000 (16:37 +0800)]
mds: replacing bootstrap session only if handle client session message

Fixes: https://tracker.ceph.com/issues/56116
Signed-off-by: Mer Xuanyi <xuanyi.meng@xtaotech.com>
(cherry picked from commit 74b56b17bf8d2f21687c29d42f4b30cfea559f7d)

22 months agoMerge pull request #53204 from idryomov/wip-62586-pacific
Ilya Dryomov [Fri, 8 Sep 2023 10:12:50 +0000 (12:12 +0200)]
Merge pull request #53204 from idryomov/wip-62586-pacific

pacific: qa/suites/upgrade/pacific-p2p: skip TestClsRbd.mirror_snapshot test

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
22 months agoMerge PR #51045 into pacific
Patrick Donnelly [Wed, 6 Sep 2023 00:45:57 +0000 (20:45 -0400)]
Merge PR #51045 into pacific

* refs/pull/51045/head:
qa: Add test for per-module finisher thread
qa: allow check_counter to look at nested keys
qa: allow specifying min for check-counter
mgr: Add one finisher thread per module

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
22 months agoMerge PR #52744 into pacific
Patrick Donnelly [Tue, 5 Sep 2023 19:42:17 +0000 (15:42 -0400)]
Merge PR #52744 into pacific

* refs/pull/52744/head:
qa: wait for file to have correct size

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge PR #53185 into pacific
Patrick Donnelly [Tue, 5 Sep 2023 19:41:42 +0000 (15:41 -0400)]
Merge PR #53185 into pacific

* refs/pull/53185/head:
mds: acquire inode snaplock in open

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge PR #53243 into pacific
Patrick Donnelly [Tue, 5 Sep 2023 19:40:57 +0000 (15:40 -0400)]
Merge PR #53243 into pacific

* refs/pull/53243/head:
mds: drop locks and retry when lock set changes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agolibrbd/ImageWatcher: kick-start ExclusiveLock state machine
Ramana Raja [Mon, 14 Aug 2023 15:27:13 +0000 (11:27 -0400)]
librbd/ImageWatcher: kick-start ExclusiveLock state machine

... that is stalled on waiting for lock and let it detect client
blocklisting. This would propagate the blocklist error to the caller
requesting an operation needing an exclusive lock.

When a caller requests a librbd operation that requires an exclusive
lock, the librbd client checks whether the exclusive lock is
held by another client. If the lock is held by another client, librbd
stalls its ExclusiveLock state machine and through its ImageWatcher
notifies the lock owner that it wants the exclusive lock. After
receiving the response from the lock owner, the ImageWatcher schedules
another lock request. Meanwhile if the client gets blocklisted, the
ImageWatcher fails to schedule another lock request and returns. The
ExclusiveLock state machine remains stalled and the blocklist error is
not propagated to the caller. Instead, when scheduling another lock
request, make the ImageWatcher call the ExclusiveLock state machine's
peer notification handler if the client is blocklisted. This allows
the Exclusive lock state machine to detect that the client has been
blocklisted in its send_acquire_lock() member function and propagate
the blocklist error to the caller.

Fixes: https://tracker.ceph.com/issues/61607
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit e66db76354093009c254aabce9e4654e27045375)