]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Yuri Weinstein [Thu, 2 Nov 2023 18:17:42 +0000 (11:17 -0700)]
Merge pull request #54289 from ajarr/wip-63384-reef
reef: mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Anthony D'Atri [Wed, 1 Nov 2023 13:12:47 +0000 (09:12 -0400)]
Merge pull request #54283 from zdover23/wip-doc-2023-11-01-backport-54281-to-reef
reef: doc/cephadm: edit troubleshooting.rst (1 of x)
Ramana Raja [Mon, 30 Oct 2023 15:05:27 +0000 (11:05 -0400)]
mgr/rbd_support: remove CreateSnapshotRequests __del__()
There is no need for CreateSnapshotRequests.__del__() that calls
CreateSnapshotRequests.wait_for_pending().
MirrorSnapshotScheduleHandler.shutdown() already calls
CreateSnapshotRequests.wait_for_pending().
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit
fed1e87685a698876cf167b3681327e5b0066ee6 )
Ramana Raja [Thu, 26 Oct 2023 17:18:52 +0000 (13:18 -0400)]
mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock
The MirrorSnapshotScheduleHandler's run thread issues asynchronous
create snapshot requests using a CreateSnapshotRequests instance. When
the thread invokes a CreateSnapshotRequests instance's get_ioctx(),
the instance's class variable lock is acquired. With the class
variable lock held, the garbage collection of a CreateSnapshotRequests
instance may race in the thread. The thread would then call
CreateSnapshotRequests __del__() that tries to acquire the class
variable lock that the thread already holds. Fix this
recursive deadlock by converting the CreateSnapshotRequests lock from
a class variable to an instance variable. There is no need to share
the lock across CreateSnapshotRequests instances.
Also convert MirrorSnapshotScheduleHandler, PerfHandler and
TrashPurgeScheduleHandler class variables to instance variables
that don't need to be shared across the instances.
Fixes: https://tracker.ceph.com/issues/62994
Signed-off-by: Ramana Raja <rraja@redhat.com>
Co-Authored-By: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
4452bc22d1c6c8499cf55d6e39090adf7ae1dcbf )
Zac Dover [Wed, 1 Nov 2023 01:53:59 +0000 (11:53 +1000)]
doc/cephadm: edit troubleshooting.rst (1 of x)
Edit doc/cephadm/troubleshooting.rst. This commit and the PR of which it
is a part was raised in response to
https://github.com/ceph/ceph/pull/53976. The limits of reStructuredText
are particularly visible here in every instance of a BASH for-loop and
in every instance of a command stretched over multiple lines.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
69472c26af5419faa9ed93c071ed5933d03fa67f )
Anthony D'Atri [Tue, 31 Oct 2023 17:31:34 +0000 (13:31 -0400)]
Merge pull request #54265 from zdover23/wip-doc-2023-10-31-backport-54239-to-reef
reef: doc/glossary: improve "BlueStore" entry
Anthony D'Atri [Tue, 31 Oct 2023 14:55:12 +0000 (10:55 -0400)]
Merge pull request #54270 from zdover23/wip-doc-2023-10-31-backport-54238-to-reef
reef: doc/rados: improve "scrubbing" explanation
Zac Dover [Mon, 30 Oct 2023 02:19:07 +0000 (12:19 +1000)]
doc/rados: improve "scrubbing" explanation
Improve the English describing the operation of scrubbing PGs in
doc/rados/configuration/osd-config-ref.rst.
Co-authored-by: Ronen Friedman <rfriedma@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
19b139932d8aef9b47e1fa9c400c3f73b5cb3d7b )
Zac Dover [Mon, 30 Oct 2023 02:37:39 +0000 (12:37 +1000)]
doc/glossary: improve "BlueStore" entry
Initially s/backend/back end/ but then I added a little more information
about BlueStore's use of RocksDB to map object names to block locations
on disk.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
8713cca328c9373636efdb92449d743b5bd56584 )
Aashish Sharma [Tue, 31 Oct 2023 09:08:34 +0000 (14:38 +0530)]
Merge pull request #54261 from aaSharma14/wip-63369-reef
reef: mgr/dashboard: fix rgw multisite error when no rgw entity is present
Reviewed-by: Nizamudeen A <nia@redhat.com>
Aashish Sharma [Tue, 31 Oct 2023 09:08:14 +0000 (14:38 +0530)]
Merge pull request #54260 from aaSharma14/wip-63368-reef
reef: mgr/dashboard: Block Ui fails in angular with target es2022
Reviewed-by: Nizamudeen A <nia@redhat.com>
Aashish Sharma [Mon, 30 Oct 2023 12:39:09 +0000 (18:09 +0530)]
mgr/dashboard: fix rgw multisite error when no rgw entity is present
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
7f57798b7f82b07eb69d87312114d41e44d3c7f3 )
Aashish Sharma [Mon, 30 Oct 2023 07:39:25 +0000 (13:09 +0530)]
mgr/dashboard: Block Ui fails in angular with target es2022
The BlockUi element is failing in angular with target es2022..change it to es2020
Fixes: https://tracker.ceph.com/issues/63347
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
551281bdc2a049c58ff1d035014312d8eaa99920 )
Aashish Sharma [Tue, 31 Oct 2023 04:48:08 +0000 (10:18 +0530)]
Merge pull request #54253 from aaSharma14/wip-63358-reef
reef: mgr/dashboard: update rgw multisite import form helper info
Reviewed-by: Nizamudeen A <nia@redhat.com>
zdover23 [Tue, 31 Oct 2023 03:00:36 +0000 (13:00 +1000)]
Merge pull request #54227 from zdover23/wip-doc-2023-10-28-backport-54219-to-reef
reef: doc/rados: remove cache-tiering-related keys
Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Aashish Sharma [Mon, 30 Oct 2023 07:47:37 +0000 (13:17 +0530)]
mgr/dashboard: update rgw multisite import form helper info
Change 'To obtain the token, generate it from your secondary Ceph cluster' to 'To obtain the token, generate it from your primary Ceph cluster' in rgw multisite import form helper
Fixes: https://tracker.ceph.com/issues/63348
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
6d066e3d7eefc3f4aceccde6b53fc5265668b3de )
Yuri Weinstein [Mon, 30 Oct 2023 15:30:32 +0000 (08:30 -0700)]
Merge pull request #54146 from vshankar/wip-63282-reef
reef: client: always refresh mds feature bits on session open
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Mon, 30 Oct 2023 15:29:52 +0000 (08:29 -0700)]
Merge pull request #54094 from mchangir/wip-63242-reef
reef: mgr/snap_schedule: make fs argument mandatory if more than one filesystem exists
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Anthony D'Atri [Mon, 30 Oct 2023 01:17:01 +0000 (21:17 -0400)]
Merge pull request #54235 from zdover23/wip-doc-2023-10-30-backport-54232-to-reef
reef: doc/ceph-volume: improve front matter
Zac Dover [Sun, 29 Oct 2023 12:15:13 +0000 (22:15 +1000)]
doc/ceph-volume: improve front matter
Simplify and improve the readability of the front matter of
doc/ceph-volume/lvm/activate.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
23190dd91861d1dcb8d03b78fb378e042394dfad )
Anthony D'Atri [Mon, 30 Oct 2023 00:29:23 +0000 (20:29 -0400)]
Merge pull request #54233 from zdover23/wip-doc-2023-10-30-backport-54231-to-reef
reef: doc/ceph-volume: explain idempotence
Zac Dover [Sun, 29 Oct 2023 12:01:11 +0000 (22:01 +1000)]
doc/ceph-volume: explain idempotence
Improve the grammar of a note explaining that the execution of the call
that activates ceph-volume is idempotent.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
e58bc3e4df97d950d96d006b6336e10dbe9327a2 )
Anthony D'Atri [Sat, 28 Oct 2023 14:45:32 +0000 (10:45 -0400)]
Merge pull request #54228 from zdover23/wip-doc-2023-10-28-backport-54218-to-reef
reef: doc/rados: edit troubleshooting-pg.rst
Zac Dover [Fri, 27 Oct 2023 05:22:34 +0000 (15:22 +1000)]
doc/rados: edit troubleshooting-pg.rst
s/placement group/pool/ in a sentence that, prior to this change, was
confusing. Suitable for backports prior to Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
97512a3e3c28d9aae7e61c30bbe66987298960a9 )
Zac Dover [Fri, 27 Oct 2023 06:58:28 +0000 (16:58 +1000)]
doc/rados: remove cache-tiering-related keys
Remove information related to cache-tiering-related keys from
doc/rados/operations/pools.rst. Cache-tiering is deprecated in Reef.
This PR is suitable for backporting to the Reef release branch, but not
to release branches prior to Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
d8e5b87a313f6b7b98b93b61327f94af4d8f2bed )
Yuri Weinstein [Fri, 27 Oct 2023 14:55:50 +0000 (07:55 -0700)]
Merge pull request #54055 from idryomov/wip-63028-reef
reef: pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Nizamudeen A [Fri, 27 Oct 2023 12:50:50 +0000 (18:20 +0530)]
Merge pull request #54223 from rhcs-dashboard/wip-63300-reef
reef: mgr/dashboard: fixed edit user quota form error
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
zdover23 [Fri, 27 Oct 2023 10:23:29 +0000 (20:23 +1000)]
Merge pull request #54220 from zdover23/wip-doc-2023-10-27-backport-53976-to-reef
reef: cephadm: add a --dry-run option to cephadm shell
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Anthony D'Atri [Fri, 27 Oct 2023 08:44:38 +0000 (04:44 -0400)]
Merge pull request #54217 from zdover23/wip-doc-2023-10-27-backport-54200-to-reef
reef: doc/rados: remove HitSet-related key information
Ivo Almeida [Mon, 23 Oct 2023 11:02:47 +0000 (12:02 +0100)]
mgr/dashboard: fixed edit user quota form error
Fixes: https://tracker.ceph.com/issues/63287
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit
1a281ccd71bb00606e4d317d46b0144e673f53a9 )
John Mulligan [Wed, 11 Oct 2023 18:05:17 +0000 (14:05 -0400)]
cephadm: add a --dry-run option to cephadm shell
Instead of creating the shell, the --dry-run option prints the container
command that would be used. This can be used as a starting point for
creating custom container commands similar to what cephadm shell would
generate but with tweaks.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit
db1e785b2c1a55f4d232b1642573e5672a136190 )
doc/cephadm: clean up debugging section
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit
4a033788f87859be4da706e38621009474ed3b82 )
doc/cephadm: extend container debugging to cover live processes
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit
b4e2b3991774fc15f677752a344181920fc1216c )
Zac Dover [Wed, 25 Oct 2023 23:48:57 +0000 (09:48 +1000)]
doc/rados: remove HitSet-related key information
Remove HitSet-related key information from
doc/rados/operations/pools.rst. HitSet-related keys are relevant only to
releases of Ceph that support cache tiering. Only Quincy and earlier
(inclusive) releases of Ceph support cache tiering. Backport this commit
from main to Reef, but not to Quincy or to release branches earlier than
Quincy.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
78878511a3e782d5cde63bfb71409c46244402f7 )
Yuri Weinstein [Wed, 25 Oct 2023 14:46:49 +0000 (07:46 -0700)]
Merge pull request #53688 from pdvian/wip-62993-reef
reef: osd/OpRequest: Add detail description for delayed op in osd log file
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 15:22:03 +0000 (08:22 -0700)]
Merge pull request #53186 from batrick/wip-62611-reef
reef: mon/OSDMonitor: do not propose on error in prepare_update
Reviewed-by: Laura Flores <lflores@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 15:20:56 +0000 (08:20 -0700)]
Merge pull request #53041 from pdvian/wip-62478-reef
reef: mon, qa: issue pool application warning even if pool is empty
Reviewed-by: Laura Flores <lflores@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 15:19:25 +0000 (08:19 -0700)]
Merge pull request #52554 from k0ste/wip-62093-reef
reef: mon: fix iterator mishandling in PGMap::apply_incremental
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 15:14:30 +0000 (08:14 -0700)]
Merge pull request #52535 from idryomov/wip-61733-reef
reef: mon/MonClient: resurrect original client_mount_timeout handling
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 14:49:07 +0000 (07:49 -0700)]
Merge pull request #54003 from kamoltat/wip-ksirivad-backport-reef-53979
reef: src/mon/Monitor: Fix set_elector_disallowed_leaders
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 14:46:32 +0000 (07:46 -0700)]
Merge pull request #53674 from kamoltat/wip-ksirivad-backport-reef-51923
reef: pybind/mgr/pg_autoscaler: fix warn when not too few pgs
Reviewed-by: Laura Flores <lflores@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 14:45:16 +0000 (07:45 -0700)]
Merge pull request #53658 from kamoltat/wip-ksirivad-reef-backport-52442
reef: pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs
Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
Yuri Weinstein [Tue, 24 Oct 2023 14:40:28 +0000 (07:40 -0700)]
Merge pull request #53996 from idryomov/wip-63149-reef
reef: qa/suites/rbd: drop cache tiering workload tests
Reviewed-by: Mykola Golub <mgolub@suse.com>
Matan Breizman [Tue, 24 Oct 2023 08:36:36 +0000 (11:36 +0300)]
Merge pull request #54140 from Matan-B/wip-53284-reef
reef: crimson/os/seastore/transaction_manager: move intermediate_key by "remap_offset" when remapping the "back" half of the original pin
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 24 Oct 2023 08:36:10 +0000 (11:36 +0300)]
Merge pull request #54139 from Matan-B/wip-53083-reef
reef: crimson/osd/lsan_suppressions: add MallocExtension::Register
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 24 Oct 2023 08:35:51 +0000 (11:35 +0300)]
Merge pull request #54138 from Matan-B/wip-52857-reef
reef: crimson/osd: cleanup and drop OSD::ShardDispatcher
Reviewed-by: Samuel Just <sjust@redhat.com>
Casey Bodley [Mon, 23 Oct 2023 18:48:58 +0000 (19:48 +0100)]
Merge pull request #51447 from k0ste/wip-59731-reef
reef: RGW: Solving the issue of not populating etag in Multipart upload result
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 18:25:07 +0000 (11:25 -0700)]
Merge pull request #53762 from cbodley/wip-63045-reef
reef: rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 18:24:25 +0000 (11:24 -0700)]
Merge pull request #53896 from cbodley/wip-62744-reef
reef: qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:59:03 +0000 (07:59 -0700)]
Merge pull request #54037 from vshankar/wip-61828-reef
reef: qa: assign file system affinity for replaced MDS
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:57:29 +0000 (07:57 -0700)]
Merge pull request #54035 from vshankar/wip-63174-reef
reef: mds: adjust pre_segments_size for MDLog when trimming segments for st…
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:56:40 +0000 (07:56 -0700)]
Merge pull request #53918 from kotreshhr/wip-63165-reef
reef: pybind/mgr/volumes: log mutex locks to help debug deadlocks
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:55:59 +0000 (07:55 -0700)]
Merge pull request #53906 from vshankar/wip-62797-reef
reef: qa: move nfs (mgr/nfs) related tests to fs suite
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Adam King adking@redhat.com
Yuri Weinstein [Mon, 23 Oct 2023 14:52:54 +0000 (07:52 -0700)]
Merge pull request #53827 from rishabh-d-dave/wip-63094-reef
reef: qa/cephfs: fix build failure for mdtest project
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:52:26 +0000 (07:52 -0700)]
Merge pull request #53825 from rishabh-d-dave/wip-63096-reef
reef: qa/cephfs: fix ior project build failure
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:51:53 +0000 (07:51 -0700)]
Merge pull request #53666 from lxbsz/wip-62521
reef: client: move the Inode to new auth mds session when changing auth cap
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:51:18 +0000 (07:51 -0700)]
Merge pull request #53661 from lxbsz/wip-62524
reef: ceph: allow xlock state to be LOCK_PREXLOCK when putting it
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:50:44 +0000 (07:50 -0700)]
Merge pull request #53624 from lxbsz/wip-62515
reef: qa/cephfs: switch to python3 for centos stream 9
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:50:13 +0000 (07:50 -0700)]
Merge pull request #53572 from kotreshhr/wip-62405-reef
reef: mgr/volumes: Fix pending_subvolume_deletions in volume info
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:49:41 +0000 (07:49 -0700)]
Merge pull request #53558 from batrick/wip-62733-reef
reef: mds: add event for batching getattr/lookup
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:49:10 +0000 (07:49 -0700)]
Merge pull request #53553 from batrick/wip-62899-reef
reef: qa: lengthen shutdown timeout for thrashed MDS
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:48:31 +0000 (07:48 -0700)]
Merge pull request #53548 from batrick/wip-62901-reef
reef: mds: log message when exiting due to asok command
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Yuri Weinstein [Mon, 23 Oct 2023 14:46:53 +0000 (07:46 -0700)]
Merge pull request #53497 from lxbsz/wip-62860
reef: mds: fix deadlock between unlinking and linkmerge
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Venky Shankar [Mon, 16 Oct 2023 04:11:19 +0000 (00:11 -0400)]
client: always refresh mds feature bits on session open
Fixes: http://tracker.ceph.com/issues/63188
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
167fdd22e38d9febef8d7e0969403261314e3c98 )
Conflicts:
src/client/Client.cc
Minor conflict due to presence of
cap_auths = std::move(m->cap_auths);
Nizamudeen A [Mon, 23 Oct 2023 05:09:36 +0000 (10:39 +0530)]
Merge pull request #54118 from aaSharma14/wip-63272-reef
reef: mgr/dashboard: add port and zone endpoints to import realm token form in rgw multisite
Reviewed-by: Nizamudeen A <nia@redhat.com>
Anthony D'Atri [Sun, 22 Oct 2023 13:55:33 +0000 (09:55 -0400)]
Merge pull request #54137 from zdover23/wip-doc-2023-10-22-backport-54136-to-reef
reef: doc/rgw: pubsub capabilities reference was removed from docs
Matan Breizman [Sun, 22 Oct 2023 10:05:06 +0000 (13:05 +0300)]
Merge pull request #54105 from Matan-B/wip-46062-reef
reef: crimson: Enable tcmalloc when using seastar
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 10:04:43 +0000 (13:04 +0300)]
Merge pull request #53966 from Matan-B/wip-53305-reef
reef: crimson/os/seastore/onode_manager: populate value recorders of onodes to be erased
Reviewed-by: Samuel Just <sjust@redhat.com>
Xuehan Xu [Tue, 5 Sep 2023 08:03:29 +0000 (16:03 +0800)]
crimson/os/seastore/transaction_manager: move intermediate_key by
"remap_offset" when remapping the "back" half of the original pin
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
dc6c091fd3d9a100b1164946ecb9e02685683a08 )
Matan Breizman [Tue, 22 Aug 2023 13:14:06 +0000 (13:14 +0000)]
crimson/osd/lsan_suppressions: add MallocExtension::Register
```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 8 byte(s) in 1 object(s) allocated from:
#0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
#1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)
SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
bc190972fabdf6eb7e772bdc9df5eee6100bb8b6 )
Yingxin Cheng [Mon, 7 Aug 2023 08:58:12 +0000 (16:58 +0800)]
crimson/osd: cleanup and drop OSD::ShardDispatcher
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
e3d910a16c4b6239c7cf523c5510bc76fadcb832 )
Xuehan Xu [Wed, 6 Sep 2023 06:56:51 +0000 (14:56 +0800)]
crimson/os/seastore/onode_manager: populate value recorders of onodes to
be erased
Otherwise, the following modification sequence with the same transaction
might lead to onode extents' crc inconsistency during journal replay:
1. modify the last mapping in an onode extent;
2. erase the last mapping in that onode extent.
During journal replay, if the first modification is not recorded in the
delta, the onode extent's content would be inconsistent with that before
the system reboot
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
6c236bb63a9a49c7f9e2dab03a5fe0bb016560f0 )
Matan Breizman [Sun, 22 Oct 2023 08:32:39 +0000 (11:32 +0300)]
Merge pull request #54098 from Matan-B/wip-53455-reef
reef: crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 08:32:25 +0000 (11:32 +0300)]
Merge pull request #54097 from Matan-B/wip-53501-reef
reef: crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 08:32:08 +0000 (11:32 +0300)]
Merge pull request #54096 from Matan-B/wip-53433-reef
reef: crimson/tools/perf_staged_fltree: fix compile error
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 08:31:13 +0000 (11:31 +0300)]
Merge pull request #54095 from Matan-B/wip-52896-reef
reef: crimson/net: consolidate messenger implementations and enable multi-shard UTs
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 08:30:33 +0000 (11:30 +0300)]
Merge pull request #54093 from Matan-B/wip-53046-reef
reef: crimson/tools/perf_crimson_msgr: randomize client nonce
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 08:29:52 +0000 (11:29 +0300)]
Merge pull request #54092 from Matan-B/wip-51141-reef
reef: crimson/os/seastore: OP_CLONE in seastore
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 22 Oct 2023 08:29:03 +0000 (11:29 +0300)]
Merge pull request #54091 from Matan-B/wip-52306-reef
reef: crimson/osd: fixes and cleanups around multi-core OSD
Reviewed-by: Samuel Just <sjust@redhat.com>
Yuval Lifshitz [Sun, 22 Oct 2023 07:09:22 +0000 (07:09 +0000)]
doc/rgw: pubsub capabilities reference was removed from docs
most pubsub docs were removed in
0fecbe2476afbb06de7ea731b7c92066ae40cfc4
however, the text removed described functionality that is only
supported bia pubsub
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit
c479f2e6caa1f8f736eedb6f0036dbaa4649a204 )
Anthony D'Atri [Sat, 21 Oct 2023 03:39:08 +0000 (23:39 -0400)]
Merge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef
reef: .github: Clarify checklist details
Anthony D'Atri [Thu, 19 Oct 2023 15:28:07 +0000 (11:28 -0400)]
.github: Clarify checklist details
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit
1ca356a916910a4016d5d3460d68ea3732f3bad9 )
Anthony D'Atri [Fri, 20 Oct 2023 19:26:15 +0000 (15:26 -0400)]
Merge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef
reef: doc/rados: edit troubleshooting-pg (2 of x)
Adam King [Fri, 20 Oct 2023 17:06:32 +0000 (13:06 -0400)]
Merge pull request #53924 from adk3798/wip-63169-reef
reef: mgr/cephadm: fix upgrades with nvmeof
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Yuri Weinstein [Fri, 20 Oct 2023 14:57:55 +0000 (07:57 -0700)]
Merge pull request #53851 from batrick/wip-63082-reef
reef: mds/FSMap: allow upgrades if no up mds
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Fri, 20 Oct 2023 14:57:25 +0000 (07:57 -0700)]
Merge pull request #53484 from batrick/wip-62852-reef
reef: qa: ignore expected cluster warning from damage tests
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Fri, 20 Oct 2023 14:56:51 +0000 (07:56 -0700)]
Merge pull request #53101 from batrick/wip-62553-reef
reef: libcephsqlite: fill 0s in unread portion of buffer
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Fri, 20 Oct 2023 14:56:10 +0000 (07:56 -0700)]
Merge pull request #52460 from batrick/wip-62022-reef
reef: pybind/mgr: reopen database handle on blocklist
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Aashish Sharma [Fri, 13 Oct 2023 08:23:23 +0000 (13:53 +0530)]
mgr/dashboard: add port and zone endpoints to import realm token form in
rgw multisite
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit
84ec19442b2db4f4e389810efbc01674d9824408 )
Zac Dover [Wed, 18 Oct 2023 22:25:38 +0000 (08:25 +1000)]
doc/rados: edit troubleshooting-pg (2 of x)
Edit doc/rados/troubleshooting/troubleshooting-pg.rst (1 of x).
Follows https://github.com/ceph/ceph/pull/54052.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
d0cdfc1d72c4edefb060f8ba9ab130189617cab1 )
Anthony D'Atri [Thu, 19 Oct 2023 22:09:46 +0000 (18:09 -0400)]
Merge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef
reef: doc: Documentation about main Ceph metrics
Juan Miguel Olmo Martínez [Wed, 11 Oct 2023 16:00:15 +0000 (18:00 +0200)]
doc: Documentation about main Ceph metrics
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Zac Dover <zac.dover@proton.me>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit
9bb63bdc8969f2ecdfeedfc8396890ad59f0d796 )
Yuri Weinstein [Thu, 19 Oct 2023 16:18:32 +0000 (09:18 -0700)]
Merge pull request #53922 from adk3798/reef-mgr-cephadm-image-base
reef: mgr/cephadm: fix default image base in reef
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Yuri Weinstein [Thu, 19 Oct 2023 16:17:54 +0000 (09:17 -0700)]
Merge pull request #53919 from ajarr/wip-63156-reef
reef: librbd: kick ExclusiveLock state machine stalled waiting for lock from reacquire_lock()
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Yuri Weinstein [Thu, 19 Oct 2023 16:17:14 +0000 (09:17 -0700)]
Merge pull request #53864 from adk3798/wip-61963-reef
reef: Adding rollback mechanism to handle bootstrap failures
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Yuri Weinstein [Thu, 19 Oct 2023 16:15:39 +0000 (09:15 -0700)]
Merge pull request #53861 from ljflores/wip-63125-reef
reef: osd: fix: slow scheduling when item_cost is large
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Yuri Weinstein [Thu, 19 Oct 2023 16:15:10 +0000 (09:15 -0700)]
Merge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136
reef:pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Yuri Weinstein [Thu, 19 Oct 2023 16:14:25 +0000 (09:14 -0700)]
Merge pull request #53093 from sseshasa/wip-62546-reef
reef: osd/scheduler/mClockScheduler: Use same profile and client ids for all clients to ensure allocated QoS limit consumption.
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Mark Nelson [Wed, 27 Apr 2022 15:06:22 +0000 (15:06 +0000)]
crimson: Enable tcmalloc when using seastar
classic-osds have always caused significant memory fragmentation
when using the libc memory allocator due to the way that Ceph
tends to utilize memory. In recent testing, crimson-osd was found
to use 25-27GB of RAM with the stock 3GB bluestore cache settings
(osd_memory_target is only used when tcmalloc is available). Upon
further testing, it was found that the classic OSD is even worse,
using between 32-33GB of RAM after a 5 minute 4K sequential
write test when using libc malloc.
The good news is that it appears that crimson-osd is able to use
tcmalloc for alienstore without significant modification. Better
still, it drastically reduces memory usage. In the same test that
resulted in 25GB RSS memory usage for crimson-osd with libc malloc,
a tcmalloc linked version took around 9GB (with an 8GB
osd_memory_target). Since we do not yet (afaik) expose classic OSD
debugging in crimson it is tough to tell why we are still a little
over, but it's clear that for alienstore we are going to need to
use tcmalloc as we do in classic.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit
d884a45aea8d5e0a5fa1689f2fd3167d68292893 )
Matan Breizman [Tue, 6 Jun 2023 14:53:58 +0000 (14:53 +0000)]
Revert "[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar"
This reverts commit
380bc6d2e4ec62e724612dbac57ffdbd82511f9d .
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
96c6c2681faa616314582ad3e9ccef71ee6d5f69 )
Matan Breizman [Thu, 19 Oct 2023 14:29:02 +0000 (17:29 +0300)]
Merge pull request #54057 from Matan-B/wip-51875-reef
reef: crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 12:28:26 +0000 (15:28 +0300)]
Merge pull request #53957 from Matan-B/wip-52529-reef
reef: crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
Reviewed-by: Samuel Just <sjust@redhat.com>