]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
20 months agomgr/rbd_support: remove CreateSnapshotRequests __del__() 54289/head
Ramana Raja [Mon, 30 Oct 2023 15:05:27 +0000 (11:05 -0400)]
mgr/rbd_support: remove CreateSnapshotRequests __del__()

There is no need for CreateSnapshotRequests.__del__() that calls
CreateSnapshotRequests.wait_for_pending().
MirrorSnapshotScheduleHandler.shutdown() already calls
CreateSnapshotRequests.wait_for_pending().

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit fed1e87685a698876cf167b3681327e5b0066ee6)

20 months agomgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock
Ramana Raja [Thu, 26 Oct 2023 17:18:52 +0000 (13:18 -0400)]
mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock

The MirrorSnapshotScheduleHandler's run thread issues asynchronous
create snapshot requests using a CreateSnapshotRequests instance. When
the thread invokes a CreateSnapshotRequests instance's get_ioctx(),
the instance's class variable lock is acquired. With the class
variable lock held, the garbage collection of a CreateSnapshotRequests
instance may race in the thread. The thread would then call
CreateSnapshotRequests __del__() that tries to acquire the class
variable lock that the thread already holds. Fix this
recursive deadlock by converting the CreateSnapshotRequests lock from
a class variable to an instance variable. There is no need to share
the lock across CreateSnapshotRequests instances.

Also convert MirrorSnapshotScheduleHandler, PerfHandler and
TrashPurgeScheduleHandler class variables to instance variables
that don't need to be shared across the instances.

Fixes: https://tracker.ceph.com/issues/62994
Signed-off-by: Ramana Raja <rraja@redhat.com>
Co-Authored-By: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 4452bc22d1c6c8499cf55d6e39090adf7ae1dcbf)

20 months agoMerge pull request #54265 from zdover23/wip-doc-2023-10-31-backport-54239-to-reef
Anthony D'Atri [Tue, 31 Oct 2023 17:31:34 +0000 (13:31 -0400)]
Merge pull request #54265 from zdover23/wip-doc-2023-10-31-backport-54239-to-reef

reef: doc/glossary: improve "BlueStore" entry

20 months agoMerge pull request #54270 from zdover23/wip-doc-2023-10-31-backport-54238-to-reef
Anthony D'Atri [Tue, 31 Oct 2023 14:55:12 +0000 (10:55 -0400)]
Merge pull request #54270 from zdover23/wip-doc-2023-10-31-backport-54238-to-reef

reef: doc/rados: improve "scrubbing" explanation

20 months agodoc/rados: improve "scrubbing" explanation 54270/head
Zac Dover [Mon, 30 Oct 2023 02:19:07 +0000 (12:19 +1000)]
doc/rados: improve "scrubbing" explanation

Improve the English describing the operation of scrubbing PGs in
doc/rados/configuration/osd-config-ref.rst.

Co-authored-by: Ronen Friedman <rfriedma@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 19b139932d8aef9b47e1fa9c400c3f73b5cb3d7b)

20 months agodoc/glossary: improve "BlueStore" entry 54265/head
Zac Dover [Mon, 30 Oct 2023 02:37:39 +0000 (12:37 +1000)]
doc/glossary: improve "BlueStore" entry

Initially s/backend/back end/ but then I added a little more information
about BlueStore's use of RocksDB to map object names to block locations
on disk.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8713cca328c9373636efdb92449d743b5bd56584)

20 months agoMerge pull request #54261 from aaSharma14/wip-63369-reef
Aashish Sharma [Tue, 31 Oct 2023 09:08:34 +0000 (14:38 +0530)]
Merge pull request #54261 from aaSharma14/wip-63369-reef

reef: mgr/dashboard: fix rgw multisite error when no rgw entity is present

Reviewed-by: Nizamudeen A <nia@redhat.com>
20 months agoMerge pull request #54260 from aaSharma14/wip-63368-reef
Aashish Sharma [Tue, 31 Oct 2023 09:08:14 +0000 (14:38 +0530)]
Merge pull request #54260 from aaSharma14/wip-63368-reef

reef: mgr/dashboard: Block Ui fails in angular with target es2022

Reviewed-by: Nizamudeen A <nia@redhat.com>
20 months agomgr/dashboard: fix rgw multisite error when no rgw entity is present 54261/head
Aashish Sharma [Mon, 30 Oct 2023 12:39:09 +0000 (18:09 +0530)]
mgr/dashboard: fix rgw multisite error when no rgw entity is present

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 7f57798b7f82b07eb69d87312114d41e44d3c7f3)

20 months agomgr/dashboard: Block Ui fails in angular with target es2022 54260/head
Aashish Sharma [Mon, 30 Oct 2023 07:39:25 +0000 (13:09 +0530)]
mgr/dashboard: Block Ui fails in angular with target es2022

The BlockUi element is failing in angular with target es2022..change it to es2020

Fixes: https://tracker.ceph.com/issues/63347
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 551281bdc2a049c58ff1d035014312d8eaa99920)

20 months agoMerge pull request #54253 from aaSharma14/wip-63358-reef
Aashish Sharma [Tue, 31 Oct 2023 04:48:08 +0000 (10:18 +0530)]
Merge pull request #54253 from aaSharma14/wip-63358-reef

reef: mgr/dashboard: update rgw multisite import form helper info

Reviewed-by: Nizamudeen A <nia@redhat.com>
20 months agoMerge pull request #54227 from zdover23/wip-doc-2023-10-28-backport-54219-to-reef
zdover23 [Tue, 31 Oct 2023 03:00:36 +0000 (13:00 +1000)]
Merge pull request #54227 from zdover23/wip-doc-2023-10-28-backport-54219-to-reef

reef: doc/rados: remove cache-tiering-related keys

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
20 months agomgr/dashboard: update rgw multisite import form helper info 54253/head
Aashish Sharma [Mon, 30 Oct 2023 07:47:37 +0000 (13:17 +0530)]
mgr/dashboard: update rgw multisite import form helper info

Change 'To obtain the token, generate it from your secondary Ceph cluster' to 'To obtain the token, generate it from your primary Ceph cluster' in rgw multisite import form helper

Fixes: https://tracker.ceph.com/issues/63348
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 6d066e3d7eefc3f4aceccde6b53fc5265668b3de)

20 months agoMerge pull request #54146 from vshankar/wip-63282-reef
Yuri Weinstein [Mon, 30 Oct 2023 15:30:32 +0000 (08:30 -0700)]
Merge pull request #54146 from vshankar/wip-63282-reef

reef: client: always refresh mds feature bits on session open

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #54094 from mchangir/wip-63242-reef
Yuri Weinstein [Mon, 30 Oct 2023 15:29:52 +0000 (08:29 -0700)]
Merge pull request #54094 from mchangir/wip-63242-reef

reef: mgr/snap_schedule: make fs argument mandatory if more than one filesystem exists

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #54235 from zdover23/wip-doc-2023-10-30-backport-54232-to-reef
Anthony D'Atri [Mon, 30 Oct 2023 01:17:01 +0000 (21:17 -0400)]
Merge pull request #54235 from zdover23/wip-doc-2023-10-30-backport-54232-to-reef

reef: doc/ceph-volume: improve front matter

20 months agodoc/ceph-volume: improve front matter 54235/head
Zac Dover [Sun, 29 Oct 2023 12:15:13 +0000 (22:15 +1000)]
doc/ceph-volume: improve front matter

Simplify and improve the readability of the front matter of
doc/ceph-volume/lvm/activate.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 23190dd91861d1dcb8d03b78fb378e042394dfad)

20 months agoMerge pull request #54233 from zdover23/wip-doc-2023-10-30-backport-54231-to-reef
Anthony D'Atri [Mon, 30 Oct 2023 00:29:23 +0000 (20:29 -0400)]
Merge pull request #54233 from zdover23/wip-doc-2023-10-30-backport-54231-to-reef

reef: doc/ceph-volume: explain idempotence

20 months agodoc/ceph-volume: explain idempotence 54233/head
Zac Dover [Sun, 29 Oct 2023 12:01:11 +0000 (22:01 +1000)]
doc/ceph-volume: explain idempotence

Improve the grammar of a note explaining that the execution of the call
that activates ceph-volume is idempotent.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e58bc3e4df97d950d96d006b6336e10dbe9327a2)

20 months agoMerge pull request #54228 from zdover23/wip-doc-2023-10-28-backport-54218-to-reef
Anthony D'Atri [Sat, 28 Oct 2023 14:45:32 +0000 (10:45 -0400)]
Merge pull request #54228 from zdover23/wip-doc-2023-10-28-backport-54218-to-reef

reef: doc/rados: edit troubleshooting-pg.rst

20 months agodoc/rados: edit troubleshooting-pg.rst 54228/head
Zac Dover [Fri, 27 Oct 2023 05:22:34 +0000 (15:22 +1000)]
doc/rados: edit troubleshooting-pg.rst

s/placement group/pool/ in a sentence that, prior to this change, was
confusing. Suitable for backports prior to Reef.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 97512a3e3c28d9aae7e61c30bbe66987298960a9)

20 months agodoc/rados: remove cache-tiering-related keys 54227/head
Zac Dover [Fri, 27 Oct 2023 06:58:28 +0000 (16:58 +1000)]
doc/rados: remove cache-tiering-related keys

Remove information related to cache-tiering-related keys from
doc/rados/operations/pools.rst. Cache-tiering is deprecated in Reef.
This PR is suitable for backporting to the Reef release branch, but not
to release branches prior to Reef.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d8e5b87a313f6b7b98b93b61327f94af4d8f2bed)

20 months agoMerge pull request #54055 from idryomov/wip-63028-reef
Yuri Weinstein [Fri, 27 Oct 2023 14:55:50 +0000 (07:55 -0700)]
Merge pull request #54055 from idryomov/wip-63028-reef

reef: pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
20 months agoMerge pull request #54223 from rhcs-dashboard/wip-63300-reef
Nizamudeen A [Fri, 27 Oct 2023 12:50:50 +0000 (18:20 +0530)]
Merge pull request #54223 from rhcs-dashboard/wip-63300-reef

reef: mgr/dashboard: fixed edit user quota form error

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
20 months agoMerge pull request #54220 from zdover23/wip-doc-2023-10-27-backport-53976-to-reef
zdover23 [Fri, 27 Oct 2023 10:23:29 +0000 (20:23 +1000)]
Merge pull request #54220 from zdover23/wip-doc-2023-10-27-backport-53976-to-reef

reef: cephadm: add a --dry-run option to cephadm shell

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
20 months agoMerge pull request #54217 from zdover23/wip-doc-2023-10-27-backport-54200-to-reef
Anthony D'Atri [Fri, 27 Oct 2023 08:44:38 +0000 (04:44 -0400)]
Merge pull request #54217 from zdover23/wip-doc-2023-10-27-backport-54200-to-reef

reef: doc/rados: remove HitSet-related key information

20 months agomgr/dashboard: fixed edit user quota form error 54223/head
Ivo Almeida [Mon, 23 Oct 2023 11:02:47 +0000 (12:02 +0100)]
mgr/dashboard: fixed edit user quota form error

Fixes: https://tracker.ceph.com/issues/63287
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit 1a281ccd71bb00606e4d317d46b0144e673f53a9)

20 months agocephadm: add a --dry-run option to cephadm shell 54220/head
John Mulligan [Wed, 11 Oct 2023 18:05:17 +0000 (14:05 -0400)]
cephadm: add a --dry-run option to cephadm shell

Instead of creating the shell, the --dry-run option prints the container
command that would be used. This can be used as a starting point for
creating custom container commands similar to what cephadm shell would
generate but with tweaks.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit db1e785b2c1a55f4d232b1642573e5672a136190)

doc/cephadm: clean up debugging section

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 4a033788f87859be4da706e38621009474ed3b82)

doc/cephadm: extend container debugging to cover live processes

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit b4e2b3991774fc15f677752a344181920fc1216c)

20 months agodoc/rados: remove HitSet-related key information 54217/head
Zac Dover [Wed, 25 Oct 2023 23:48:57 +0000 (09:48 +1000)]
doc/rados: remove HitSet-related key information

Remove HitSet-related key information from
doc/rados/operations/pools.rst. HitSet-related keys are relevant only to
releases of Ceph that support cache tiering. Only Quincy and earlier
(inclusive) releases of Ceph support cache tiering. Backport this commit
from main to Reef, but not to Quincy or to release branches earlier than
Quincy.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 78878511a3e782d5cde63bfb71409c46244402f7)

20 months agoMerge pull request #53688 from pdvian/wip-62993-reef
Yuri Weinstein [Wed, 25 Oct 2023 14:46:49 +0000 (07:46 -0700)]
Merge pull request #53688 from pdvian/wip-62993-reef

reef: osd/OpRequest: Add detail description for delayed op in osd log file

Reviewed-by: Neha Ojha <nojha@redhat.com>
20 months agoMerge pull request #53186 from batrick/wip-62611-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:22:03 +0000 (08:22 -0700)]
Merge pull request #53186 from batrick/wip-62611-reef

reef: mon/OSDMonitor: do not propose on error in prepare_update

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #53041 from pdvian/wip-62478-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:20:56 +0000 (08:20 -0700)]
Merge pull request #53041 from pdvian/wip-62478-reef

reef: mon, qa: issue pool application warning even if pool is empty

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #52554 from k0ste/wip-62093-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:19:25 +0000 (08:19 -0700)]
Merge pull request #52554 from k0ste/wip-62093-reef

reef: mon: fix iterator mishandling in PGMap::apply_incremental

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
20 months agoMerge pull request #52535 from idryomov/wip-61733-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:14:30 +0000 (08:14 -0700)]
Merge pull request #52535 from idryomov/wip-61733-reef

reef: mon/MonClient: resurrect original client_mount_timeout handling

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
20 months agoMerge pull request #54003 from kamoltat/wip-ksirivad-backport-reef-53979
Yuri Weinstein [Tue, 24 Oct 2023 14:49:07 +0000 (07:49 -0700)]
Merge pull request #54003 from kamoltat/wip-ksirivad-backport-reef-53979

reef: src/mon/Monitor: Fix set_elector_disallowed_leaders

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
20 months agoMerge pull request #53674 from kamoltat/wip-ksirivad-backport-reef-51923
Yuri Weinstein [Tue, 24 Oct 2023 14:46:32 +0000 (07:46 -0700)]
Merge pull request #53674 from kamoltat/wip-ksirivad-backport-reef-51923

reef: pybind/mgr/pg_autoscaler: fix warn when not too few pgs

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #53658 from kamoltat/wip-ksirivad-reef-backport-52442
Yuri Weinstein [Tue, 24 Oct 2023 14:45:16 +0000 (07:45 -0700)]
Merge pull request #53658 from kamoltat/wip-ksirivad-reef-backport-52442

reef: pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
20 months agoMerge pull request #53996 from idryomov/wip-63149-reef
Yuri Weinstein [Tue, 24 Oct 2023 14:40:28 +0000 (07:40 -0700)]
Merge pull request #53996 from idryomov/wip-63149-reef

reef: qa/suites/rbd: drop cache tiering workload tests

Reviewed-by: Mykola Golub <mgolub@suse.com>
20 months agoMerge pull request #54140 from Matan-B/wip-53284-reef
Matan Breizman [Tue, 24 Oct 2023 08:36:36 +0000 (11:36 +0300)]
Merge pull request #54140 from Matan-B/wip-53284-reef

reef: crimson/os/seastore/transaction_manager: move intermediate_key by "remap_offset" when remapping the "back" half of the original pin

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54139 from Matan-B/wip-53083-reef
Matan Breizman [Tue, 24 Oct 2023 08:36:10 +0000 (11:36 +0300)]
Merge pull request #54139 from Matan-B/wip-53083-reef

reef: crimson/osd/lsan_suppressions: add MallocExtension::Register

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54138 from Matan-B/wip-52857-reef
Matan Breizman [Tue, 24 Oct 2023 08:35:51 +0000 (11:35 +0300)]
Merge pull request #54138 from Matan-B/wip-52857-reef

reef: crimson/osd: cleanup and drop OSD::ShardDispatcher

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #51447 from k0ste/wip-59731-reef
Casey Bodley [Mon, 23 Oct 2023 18:48:58 +0000 (19:48 +0100)]
Merge pull request #51447 from k0ste/wip-59731-reef

reef: RGW: Solving the issue of not populating etag in Multipart upload result

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
20 months agoMerge pull request #53762 from cbodley/wip-63045-reef
Yuri Weinstein [Mon, 23 Oct 2023 18:25:07 +0000 (11:25 -0700)]
Merge pull request #53762 from cbodley/wip-63045-reef

reef: rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
20 months agoMerge pull request #53896 from cbodley/wip-62744-reef
Yuri Weinstein [Mon, 23 Oct 2023 18:24:25 +0000 (11:24 -0700)]
Merge pull request #53896 from cbodley/wip-62744-reef

reef: qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
20 months agoMerge pull request #54037 from vshankar/wip-61828-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:59:03 +0000 (07:59 -0700)]
Merge pull request #54037 from vshankar/wip-61828-reef

reef: qa: assign file system affinity for replaced MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #54035 from vshankar/wip-63174-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:57:29 +0000 (07:57 -0700)]
Merge pull request #54035 from vshankar/wip-63174-reef

reef: mds: adjust pre_segments_size for MDLog when trimming segments for st…

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53918 from kotreshhr/wip-63165-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:56:40 +0000 (07:56 -0700)]
Merge pull request #53918 from kotreshhr/wip-63165-reef

reef: pybind/mgr/volumes: log mutex locks to help debug deadlocks

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53906 from vshankar/wip-62797-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:55:59 +0000 (07:55 -0700)]
Merge pull request #53906 from vshankar/wip-62797-reef

reef: qa: move nfs (mgr/nfs) related tests to fs suite

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Adam King adking@redhat.com
20 months agoMerge pull request #53827 from rishabh-d-dave/wip-63094-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:52:54 +0000 (07:52 -0700)]
Merge pull request #53827 from rishabh-d-dave/wip-63094-reef

reef: qa/cephfs: fix build failure for mdtest project

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53825 from rishabh-d-dave/wip-63096-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:52:26 +0000 (07:52 -0700)]
Merge pull request #53825 from rishabh-d-dave/wip-63096-reef

reef: qa/cephfs: fix ior project build failure

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53666 from lxbsz/wip-62521
Yuri Weinstein [Mon, 23 Oct 2023 14:51:53 +0000 (07:51 -0700)]
Merge pull request #53666 from lxbsz/wip-62521

reef: client: move the Inode to new auth mds session when changing auth cap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53661 from lxbsz/wip-62524
Yuri Weinstein [Mon, 23 Oct 2023 14:51:18 +0000 (07:51 -0700)]
Merge pull request #53661 from lxbsz/wip-62524

reef: ceph: allow xlock state to be LOCK_PREXLOCK when putting it

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53624 from lxbsz/wip-62515
Yuri Weinstein [Mon, 23 Oct 2023 14:50:44 +0000 (07:50 -0700)]
Merge pull request #53624 from lxbsz/wip-62515

reef: qa/cephfs: switch to python3 for centos stream 9

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53572 from kotreshhr/wip-62405-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:50:13 +0000 (07:50 -0700)]
Merge pull request #53572 from kotreshhr/wip-62405-reef

reef:    mgr/volumes: Fix pending_subvolume_deletions in volume info

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53558 from batrick/wip-62733-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:49:41 +0000 (07:49 -0700)]
Merge pull request #53558 from batrick/wip-62733-reef

reef: mds: add event for batching getattr/lookup

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53553 from batrick/wip-62899-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:49:10 +0000 (07:49 -0700)]
Merge pull request #53553 from batrick/wip-62899-reef

reef: qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53548 from batrick/wip-62901-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:48:31 +0000 (07:48 -0700)]
Merge pull request #53548 from batrick/wip-62901-reef

reef: mds: log message when exiting due to asok command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
20 months agoMerge pull request #53497 from lxbsz/wip-62860
Yuri Weinstein [Mon, 23 Oct 2023 14:46:53 +0000 (07:46 -0700)]
Merge pull request #53497 from lxbsz/wip-62860

reef: mds: fix deadlock between unlinking and linkmerge

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoclient: always refresh mds feature bits on session open 54146/head
Venky Shankar [Mon, 16 Oct 2023 04:11:19 +0000 (00:11 -0400)]
client: always refresh mds feature bits on session open

Fixes: http://tracker.ceph.com/issues/63188
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 167fdd22e38d9febef8d7e0969403261314e3c98)

 Conflicts:
src/client/Client.cc

Minor conflict due to presence of

        cap_auths = std::move(m->cap_auths);

20 months agoMerge pull request #54118 from aaSharma14/wip-63272-reef
Nizamudeen A [Mon, 23 Oct 2023 05:09:36 +0000 (10:39 +0530)]
Merge pull request #54118 from aaSharma14/wip-63272-reef

reef: mgr/dashboard: add port and zone endpoints to import realm token form in rgw multisite

Reviewed-by: Nizamudeen A <nia@redhat.com>
20 months agoMerge pull request #54137 from zdover23/wip-doc-2023-10-22-backport-54136-to-reef
Anthony D'Atri [Sun, 22 Oct 2023 13:55:33 +0000 (09:55 -0400)]
Merge pull request #54137 from zdover23/wip-doc-2023-10-22-backport-54136-to-reef

reef: doc/rgw: pubsub capabilities reference was removed from docs

20 months agoMerge pull request #54105 from Matan-B/wip-46062-reef
Matan Breizman [Sun, 22 Oct 2023 10:05:06 +0000 (13:05 +0300)]
Merge pull request #54105 from Matan-B/wip-46062-reef

reef: crimson: Enable tcmalloc when using seastar

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #53966 from Matan-B/wip-53305-reef
Matan Breizman [Sun, 22 Oct 2023 10:04:43 +0000 (13:04 +0300)]
Merge pull request #53966 from Matan-B/wip-53305-reef

reef: crimson/os/seastore/onode_manager: populate value recorders of onodes to be erased

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agocrimson/os/seastore/transaction_manager: move intermediate_key by 54140/head
Xuehan Xu [Tue, 5 Sep 2023 08:03:29 +0000 (16:03 +0800)]
crimson/os/seastore/transaction_manager: move intermediate_key by
"remap_offset" when remapping the "back" half of the original pin

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit dc6c091fd3d9a100b1164946ecb9e02685683a08)

20 months agocrimson/osd/lsan_suppressions: add MallocExtension::Register 54139/head
Matan Breizman [Tue, 22 Aug 2023 13:14:06 +0000 (13:14 +0000)]
crimson/osd/lsan_suppressions: add MallocExtension::Register

```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)

SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bc190972fabdf6eb7e772bdc9df5eee6100bb8b6)

20 months agocrimson/osd: cleanup and drop OSD::ShardDispatcher 54138/head
Yingxin Cheng [Mon, 7 Aug 2023 08:58:12 +0000 (16:58 +0800)]
crimson/osd: cleanup and drop OSD::ShardDispatcher

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit e3d910a16c4b6239c7cf523c5510bc76fadcb832)

20 months agocrimson/os/seastore/onode_manager: populate value recorders of onodes to 53966/head
Xuehan Xu [Wed, 6 Sep 2023 06:56:51 +0000 (14:56 +0800)]
crimson/os/seastore/onode_manager: populate value recorders of onodes to
be erased

Otherwise, the following modification sequence with the same transaction
might lead to onode extents' crc inconsistency during journal replay:

1. modify the last mapping in an onode extent;
2. erase the last mapping in that onode extent.

During journal replay, if the first modification is not recorded in the
delta, the onode extent's content would be inconsistent with that before
the system reboot

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 6c236bb63a9a49c7f9e2dab03a5fe0bb016560f0)

20 months agoMerge pull request #54098 from Matan-B/wip-53455-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:39 +0000 (11:32 +0300)]
Merge pull request #54098 from Matan-B/wip-53455-reef

reef: crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54097 from Matan-B/wip-53501-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:25 +0000 (11:32 +0300)]
Merge pull request #54097 from Matan-B/wip-53501-reef

reef: crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54096 from Matan-B/wip-53433-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:08 +0000 (11:32 +0300)]
Merge pull request #54096 from Matan-B/wip-53433-reef

reef: crimson/tools/perf_staged_fltree: fix compile error

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54095 from Matan-B/wip-52896-reef
Matan Breizman [Sun, 22 Oct 2023 08:31:13 +0000 (11:31 +0300)]
Merge pull request #54095 from Matan-B/wip-52896-reef

reef: crimson/net: consolidate messenger implementations and enable multi-shard UTs

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54093 from Matan-B/wip-53046-reef
Matan Breizman [Sun, 22 Oct 2023 08:30:33 +0000 (11:30 +0300)]
Merge pull request #54093 from Matan-B/wip-53046-reef

reef: crimson/tools/perf_crimson_msgr: randomize client nonce

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54092 from Matan-B/wip-51141-reef
Matan Breizman [Sun, 22 Oct 2023 08:29:52 +0000 (11:29 +0300)]
Merge pull request #54092 from Matan-B/wip-51141-reef

reef: crimson/os/seastore: OP_CLONE in seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54091 from Matan-B/wip-52306-reef
Matan Breizman [Sun, 22 Oct 2023 08:29:03 +0000 (11:29 +0300)]
Merge pull request #54091 from Matan-B/wip-52306-reef

reef: crimson/osd: fixes and cleanups around multi-core OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agodoc/rgw: pubsub capabilities reference was removed from docs 54137/head
Yuval Lifshitz [Sun, 22 Oct 2023 07:09:22 +0000 (07:09 +0000)]
doc/rgw: pubsub capabilities reference was removed from docs

most pubsub docs were removed in
0fecbe2476afbb06de7ea731b7c92066ae40cfc4
however, the text removed described functionality that is only
supported bia pubsub

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit c479f2e6caa1f8f736eedb6f0036dbaa4649a204)

20 months agoMerge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef
Anthony D'Atri [Sat, 21 Oct 2023 03:39:08 +0000 (23:39 -0400)]
Merge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef

reef: .github: Clarify checklist details

20 months ago.github: Clarify checklist details 54130/head
Anthony D'Atri [Thu, 19 Oct 2023 15:28:07 +0000 (11:28 -0400)]
.github: Clarify checklist details

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 1ca356a916910a4016d5d3460d68ea3732f3bad9)

20 months agoMerge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef
Anthony D'Atri [Fri, 20 Oct 2023 19:26:15 +0000 (15:26 -0400)]
Merge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef

reef: doc/rados: edit troubleshooting-pg (2 of x)

20 months agoMerge pull request #53924 from adk3798/wip-63169-reef
Adam King [Fri, 20 Oct 2023 17:06:32 +0000 (13:06 -0400)]
Merge pull request #53924 from adk3798/wip-63169-reef

reef: mgr/cephadm: fix upgrades with nvmeof

Reviewed-by: John Mulligan <jmulligan@redhat.com>
20 months agoMerge pull request #53851 from batrick/wip-63082-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:57:55 +0000 (07:57 -0700)]
Merge pull request #53851 from batrick/wip-63082-reef

reef: mds/FSMap: allow upgrades if no up mds

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #53484 from batrick/wip-62852-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:57:25 +0000 (07:57 -0700)]
Merge pull request #53484 from batrick/wip-62852-reef

reef: qa: ignore expected cluster warning from damage tests

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #53101 from batrick/wip-62553-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:56:51 +0000 (07:56 -0700)]
Merge pull request #53101 from batrick/wip-62553-reef

reef: libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #52460 from batrick/wip-62022-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:56:10 +0000 (07:56 -0700)]
Merge pull request #52460 from batrick/wip-62022-reef

reef: pybind/mgr: reopen database handle on blocklist

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agomgr/dashboard: add port and zone endpoints to import realm token form in 54118/head
Aashish Sharma [Fri, 13 Oct 2023 08:23:23 +0000 (13:53 +0530)]
mgr/dashboard: add port and zone endpoints to import realm token form in
rgw multisite

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 84ec19442b2db4f4e389810efbc01674d9824408)

20 months agodoc/rados: edit troubleshooting-pg (2 of x) 54114/head
Zac Dover [Wed, 18 Oct 2023 22:25:38 +0000 (08:25 +1000)]
doc/rados: edit troubleshooting-pg (2 of x)

Edit doc/rados/troubleshooting/troubleshooting-pg.rst (1 of x).

Follows https://github.com/ceph/ceph/pull/54052.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d0cdfc1d72c4edefb060f8ba9ab130189617cab1)

21 months agoMerge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef
Anthony D'Atri [Thu, 19 Oct 2023 22:09:46 +0000 (18:09 -0400)]
Merge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef

reef: doc: Documentation about main Ceph metrics

21 months agodoc: Documentation about main Ceph metrics 54111/head
Juan Miguel Olmo Martínez [Wed, 11 Oct 2023 16:00:15 +0000 (18:00 +0200)]
doc: Documentation about main Ceph metrics

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Zac Dover <zac.dover@proton.me>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 9bb63bdc8969f2ecdfeedfc8396890ad59f0d796)

21 months agoMerge pull request #53922 from adk3798/reef-mgr-cephadm-image-base
Yuri Weinstein [Thu, 19 Oct 2023 16:18:32 +0000 (09:18 -0700)]
Merge pull request #53922 from adk3798/reef-mgr-cephadm-image-base

reef: mgr/cephadm: fix default image base in reef

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #53919 from ajarr/wip-63156-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:17:54 +0000 (09:17 -0700)]
Merge pull request #53919 from ajarr/wip-63156-reef

reef: librbd: kick ExclusiveLock state machine stalled waiting for lock from reacquire_lock()

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
21 months agoMerge pull request #53864 from adk3798/wip-61963-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:17:14 +0000 (09:17 -0700)]
Merge pull request #53864 from adk3798/wip-61963-reef

reef: Adding rollback mechanism to handle bootstrap failures

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
21 months agoMerge pull request #53861 from ljflores/wip-63125-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:15:39 +0000 (09:15 -0700)]
Merge pull request #53861 from ljflores/wip-63125-reef

reef: osd: fix: slow scheduling when item_cost is large

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
21 months agoMerge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136
Yuri Weinstein [Thu, 19 Oct 2023 16:15:10 +0000 (09:15 -0700)]
Merge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136

reef:pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #53093 from sseshasa/wip-62546-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:14:25 +0000 (09:14 -0700)]
Merge pull request #53093 from sseshasa/wip-62546-reef

reef: osd/scheduler/mClockScheduler: Use same profile and client ids for all clients to ensure allocated QoS limit consumption.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agocrimson: Enable tcmalloc when using seastar 54105/head
Mark Nelson [Wed, 27 Apr 2022 15:06:22 +0000 (15:06 +0000)]
crimson: Enable tcmalloc when using seastar

classic-osds have always caused significant memory fragmentation
when using the libc memory allocator due to the way that Ceph
tends to utilize memory.  In recent testing, crimson-osd was found
to use 25-27GB of RAM with the stock 3GB bluestore cache settings
(osd_memory_target is only used when tcmalloc is available).  Upon
further testing, it was found that the classic OSD is even worse,
using between 32-33GB of RAM after a 5 minute 4K sequential
write test when using libc malloc.

The good news is that it appears that crimson-osd is able to use
tcmalloc for alienstore without significant modification. Better
still, it drastically reduces memory usage.  In the same test that
resulted in 25GB RSS memory usage for crimson-osd with libc malloc,
a tcmalloc linked version took around 9GB (with an 8GB
osd_memory_target).  Since we do not yet (afaik) expose classic OSD
debugging in crimson it is tough to tell why we are still a little
over, but it's clear that for alienstore we are going to need to
use tcmalloc as we do in classic.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit d884a45aea8d5e0a5fa1689f2fd3167d68292893)

21 months agoRevert "[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar"
Matan Breizman [Tue, 6 Jun 2023 14:53:58 +0000 (14:53 +0000)]
Revert "[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar"

This reverts commit 380bc6d2e4ec62e724612dbac57ffdbd82511f9d.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 96c6c2681faa616314582ad3e9ccef71ee6d5f69)

21 months agoMerge pull request #54057 from Matan-B/wip-51875-reef
Matan Breizman [Thu, 19 Oct 2023 14:29:02 +0000 (17:29 +0300)]
Merge pull request #54057 from Matan-B/wip-51875-reef

reef: crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53957 from Matan-B/wip-52529-reef
Matan Breizman [Thu, 19 Oct 2023 12:28:26 +0000 (15:28 +0300)]
Merge pull request #53957 from Matan-B/wip-52529-reef

reef: crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54021 from Matan-B/wip-48052-reef
Matan Breizman [Thu, 19 Oct 2023 08:34:38 +0000 (11:34 +0300)]
Merge pull request #54021 from Matan-B/wip-48052-reef

reef: qa/suites/crimson-rados: bring backfill testing

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agocrimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it 53957/head
Xuehan Xu [Wed, 19 Jul 2023 07:52:22 +0000 (15:52 +0800)]
crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
with ReplicateRequest

This commit is a follow-up for e14142894922b403344d7f08315b67a291b9a27a
and f4d336a2dbf864fe798ca7f500d44576d3f626d5

Fixes: https://tracker.ceph.com/issues/61943
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 0b7c47db78cc22c7cf25c50b430674e17b01a601)

21 months agocrimson/os/seastore/cache: replace is_clean by is_stable_clean wherever 54098/head
Xuehan Xu [Tue, 26 Sep 2023 03:30:54 +0000 (11:30 +0800)]
crimson/os/seastore/cache: replace is_clean by is_stable_clean wherever
possible

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit f60f318450fd8118b96e3dbae29cccc68ed1ae19)