]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
20 months agomgr/dashboard: update rgw multisite import form helper info 54253/head
Aashish Sharma [Mon, 30 Oct 2023 07:47:37 +0000 (13:17 +0530)]
mgr/dashboard: update rgw multisite import form helper info

Change 'To obtain the token, generate it from your secondary Ceph cluster' to 'To obtain the token, generate it from your primary Ceph cluster' in rgw multisite import form helper

Fixes: https://tracker.ceph.com/issues/63348
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 6d066e3d7eefc3f4aceccde6b53fc5265668b3de)

20 months agoMerge pull request #54146 from vshankar/wip-63282-reef
Yuri Weinstein [Mon, 30 Oct 2023 15:30:32 +0000 (08:30 -0700)]
Merge pull request #54146 from vshankar/wip-63282-reef

reef: client: always refresh mds feature bits on session open

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #54094 from mchangir/wip-63242-reef
Yuri Weinstein [Mon, 30 Oct 2023 15:29:52 +0000 (08:29 -0700)]
Merge pull request #54094 from mchangir/wip-63242-reef

reef: mgr/snap_schedule: make fs argument mandatory if more than one filesystem exists

Reviewed-by: Xiubo Li <xiubli@redhat.com>
20 months agoMerge pull request #54235 from zdover23/wip-doc-2023-10-30-backport-54232-to-reef
Anthony D'Atri [Mon, 30 Oct 2023 01:17:01 +0000 (21:17 -0400)]
Merge pull request #54235 from zdover23/wip-doc-2023-10-30-backport-54232-to-reef

reef: doc/ceph-volume: improve front matter

20 months agodoc/ceph-volume: improve front matter 54235/head
Zac Dover [Sun, 29 Oct 2023 12:15:13 +0000 (22:15 +1000)]
doc/ceph-volume: improve front matter

Simplify and improve the readability of the front matter of
doc/ceph-volume/lvm/activate.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 23190dd91861d1dcb8d03b78fb378e042394dfad)

20 months agoMerge pull request #54233 from zdover23/wip-doc-2023-10-30-backport-54231-to-reef
Anthony D'Atri [Mon, 30 Oct 2023 00:29:23 +0000 (20:29 -0400)]
Merge pull request #54233 from zdover23/wip-doc-2023-10-30-backport-54231-to-reef

reef: doc/ceph-volume: explain idempotence

20 months agodoc/ceph-volume: explain idempotence 54233/head
Zac Dover [Sun, 29 Oct 2023 12:01:11 +0000 (22:01 +1000)]
doc/ceph-volume: explain idempotence

Improve the grammar of a note explaining that the execution of the call
that activates ceph-volume is idempotent.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e58bc3e4df97d950d96d006b6336e10dbe9327a2)

20 months agoMerge pull request #54228 from zdover23/wip-doc-2023-10-28-backport-54218-to-reef
Anthony D'Atri [Sat, 28 Oct 2023 14:45:32 +0000 (10:45 -0400)]
Merge pull request #54228 from zdover23/wip-doc-2023-10-28-backport-54218-to-reef

reef: doc/rados: edit troubleshooting-pg.rst

20 months agodoc/rados: edit troubleshooting-pg.rst 54228/head
Zac Dover [Fri, 27 Oct 2023 05:22:34 +0000 (15:22 +1000)]
doc/rados: edit troubleshooting-pg.rst

s/placement group/pool/ in a sentence that, prior to this change, was
confusing. Suitable for backports prior to Reef.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 97512a3e3c28d9aae7e61c30bbe66987298960a9)

20 months agoMerge pull request #54055 from idryomov/wip-63028-reef
Yuri Weinstein [Fri, 27 Oct 2023 14:55:50 +0000 (07:55 -0700)]
Merge pull request #54055 from idryomov/wip-63028-reef

reef: pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
20 months agoMerge pull request #54223 from rhcs-dashboard/wip-63300-reef
Nizamudeen A [Fri, 27 Oct 2023 12:50:50 +0000 (18:20 +0530)]
Merge pull request #54223 from rhcs-dashboard/wip-63300-reef

reef: mgr/dashboard: fixed edit user quota form error

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
20 months agoMerge pull request #54220 from zdover23/wip-doc-2023-10-27-backport-53976-to-reef
zdover23 [Fri, 27 Oct 2023 10:23:29 +0000 (20:23 +1000)]
Merge pull request #54220 from zdover23/wip-doc-2023-10-27-backport-53976-to-reef

reef: cephadm: add a --dry-run option to cephadm shell

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
20 months agoMerge pull request #54217 from zdover23/wip-doc-2023-10-27-backport-54200-to-reef
Anthony D'Atri [Fri, 27 Oct 2023 08:44:38 +0000 (04:44 -0400)]
Merge pull request #54217 from zdover23/wip-doc-2023-10-27-backport-54200-to-reef

reef: doc/rados: remove HitSet-related key information

20 months agomgr/dashboard: fixed edit user quota form error 54223/head
Ivo Almeida [Mon, 23 Oct 2023 11:02:47 +0000 (12:02 +0100)]
mgr/dashboard: fixed edit user quota form error

Fixes: https://tracker.ceph.com/issues/63287
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit 1a281ccd71bb00606e4d317d46b0144e673f53a9)

20 months agocephadm: add a --dry-run option to cephadm shell 54220/head
John Mulligan [Wed, 11 Oct 2023 18:05:17 +0000 (14:05 -0400)]
cephadm: add a --dry-run option to cephadm shell

Instead of creating the shell, the --dry-run option prints the container
command that would be used. This can be used as a starting point for
creating custom container commands similar to what cephadm shell would
generate but with tweaks.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit db1e785b2c1a55f4d232b1642573e5672a136190)

doc/cephadm: clean up debugging section

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 4a033788f87859be4da706e38621009474ed3b82)

doc/cephadm: extend container debugging to cover live processes

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit b4e2b3991774fc15f677752a344181920fc1216c)

20 months agodoc/rados: remove HitSet-related key information 54217/head
Zac Dover [Wed, 25 Oct 2023 23:48:57 +0000 (09:48 +1000)]
doc/rados: remove HitSet-related key information

Remove HitSet-related key information from
doc/rados/operations/pools.rst. HitSet-related keys are relevant only to
releases of Ceph that support cache tiering. Only Quincy and earlier
(inclusive) releases of Ceph support cache tiering. Backport this commit
from main to Reef, but not to Quincy or to release branches earlier than
Quincy.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 78878511a3e782d5cde63bfb71409c46244402f7)

20 months agoMerge pull request #53688 from pdvian/wip-62993-reef
Yuri Weinstein [Wed, 25 Oct 2023 14:46:49 +0000 (07:46 -0700)]
Merge pull request #53688 from pdvian/wip-62993-reef

reef: osd/OpRequest: Add detail description for delayed op in osd log file

Reviewed-by: Neha Ojha <nojha@redhat.com>
20 months agoMerge pull request #53186 from batrick/wip-62611-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:22:03 +0000 (08:22 -0700)]
Merge pull request #53186 from batrick/wip-62611-reef

reef: mon/OSDMonitor: do not propose on error in prepare_update

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #53041 from pdvian/wip-62478-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:20:56 +0000 (08:20 -0700)]
Merge pull request #53041 from pdvian/wip-62478-reef

reef: mon, qa: issue pool application warning even if pool is empty

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #52554 from k0ste/wip-62093-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:19:25 +0000 (08:19 -0700)]
Merge pull request #52554 from k0ste/wip-62093-reef

reef: mon: fix iterator mishandling in PGMap::apply_incremental

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
20 months agoMerge pull request #52535 from idryomov/wip-61733-reef
Yuri Weinstein [Tue, 24 Oct 2023 15:14:30 +0000 (08:14 -0700)]
Merge pull request #52535 from idryomov/wip-61733-reef

reef: mon/MonClient: resurrect original client_mount_timeout handling

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
20 months agoMerge pull request #54003 from kamoltat/wip-ksirivad-backport-reef-53979
Yuri Weinstein [Tue, 24 Oct 2023 14:49:07 +0000 (07:49 -0700)]
Merge pull request #54003 from kamoltat/wip-ksirivad-backport-reef-53979

reef: src/mon/Monitor: Fix set_elector_disallowed_leaders

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
20 months agoMerge pull request #53674 from kamoltat/wip-ksirivad-backport-reef-51923
Yuri Weinstein [Tue, 24 Oct 2023 14:46:32 +0000 (07:46 -0700)]
Merge pull request #53674 from kamoltat/wip-ksirivad-backport-reef-51923

reef: pybind/mgr/pg_autoscaler: fix warn when not too few pgs

Reviewed-by: Laura Flores <lflores@redhat.com>
20 months agoMerge pull request #53658 from kamoltat/wip-ksirivad-reef-backport-52442
Yuri Weinstein [Tue, 24 Oct 2023 14:45:16 +0000 (07:45 -0700)]
Merge pull request #53658 from kamoltat/wip-ksirivad-reef-backport-52442

reef: pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
20 months agoMerge pull request #53996 from idryomov/wip-63149-reef
Yuri Weinstein [Tue, 24 Oct 2023 14:40:28 +0000 (07:40 -0700)]
Merge pull request #53996 from idryomov/wip-63149-reef

reef: qa/suites/rbd: drop cache tiering workload tests

Reviewed-by: Mykola Golub <mgolub@suse.com>
20 months agoMerge pull request #54140 from Matan-B/wip-53284-reef
Matan Breizman [Tue, 24 Oct 2023 08:36:36 +0000 (11:36 +0300)]
Merge pull request #54140 from Matan-B/wip-53284-reef

reef: crimson/os/seastore/transaction_manager: move intermediate_key by "remap_offset" when remapping the "back" half of the original pin

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54139 from Matan-B/wip-53083-reef
Matan Breizman [Tue, 24 Oct 2023 08:36:10 +0000 (11:36 +0300)]
Merge pull request #54139 from Matan-B/wip-53083-reef

reef: crimson/osd/lsan_suppressions: add MallocExtension::Register

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54138 from Matan-B/wip-52857-reef
Matan Breizman [Tue, 24 Oct 2023 08:35:51 +0000 (11:35 +0300)]
Merge pull request #54138 from Matan-B/wip-52857-reef

reef: crimson/osd: cleanup and drop OSD::ShardDispatcher

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #51447 from k0ste/wip-59731-reef
Casey Bodley [Mon, 23 Oct 2023 18:48:58 +0000 (19:48 +0100)]
Merge pull request #51447 from k0ste/wip-59731-reef

reef: RGW: Solving the issue of not populating etag in Multipart upload result

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
20 months agoMerge pull request #53762 from cbodley/wip-63045-reef
Yuri Weinstein [Mon, 23 Oct 2023 18:25:07 +0000 (11:25 -0700)]
Merge pull request #53762 from cbodley/wip-63045-reef

reef: rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
20 months agoMerge pull request #53896 from cbodley/wip-62744-reef
Yuri Weinstein [Mon, 23 Oct 2023 18:24:25 +0000 (11:24 -0700)]
Merge pull request #53896 from cbodley/wip-62744-reef

reef: qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
20 months agoMerge pull request #54037 from vshankar/wip-61828-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:59:03 +0000 (07:59 -0700)]
Merge pull request #54037 from vshankar/wip-61828-reef

reef: qa: assign file system affinity for replaced MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #54035 from vshankar/wip-63174-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:57:29 +0000 (07:57 -0700)]
Merge pull request #54035 from vshankar/wip-63174-reef

reef: mds: adjust pre_segments_size for MDLog when trimming segments for st…

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53918 from kotreshhr/wip-63165-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:56:40 +0000 (07:56 -0700)]
Merge pull request #53918 from kotreshhr/wip-63165-reef

reef: pybind/mgr/volumes: log mutex locks to help debug deadlocks

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53906 from vshankar/wip-62797-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:55:59 +0000 (07:55 -0700)]
Merge pull request #53906 from vshankar/wip-62797-reef

reef: qa: move nfs (mgr/nfs) related tests to fs suite

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Adam King adking@redhat.com
20 months agoMerge pull request #53827 from rishabh-d-dave/wip-63094-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:52:54 +0000 (07:52 -0700)]
Merge pull request #53827 from rishabh-d-dave/wip-63094-reef

reef: qa/cephfs: fix build failure for mdtest project

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53825 from rishabh-d-dave/wip-63096-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:52:26 +0000 (07:52 -0700)]
Merge pull request #53825 from rishabh-d-dave/wip-63096-reef

reef: qa/cephfs: fix ior project build failure

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53666 from lxbsz/wip-62521
Yuri Weinstein [Mon, 23 Oct 2023 14:51:53 +0000 (07:51 -0700)]
Merge pull request #53666 from lxbsz/wip-62521

reef: client: move the Inode to new auth mds session when changing auth cap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53661 from lxbsz/wip-62524
Yuri Weinstein [Mon, 23 Oct 2023 14:51:18 +0000 (07:51 -0700)]
Merge pull request #53661 from lxbsz/wip-62524

reef: ceph: allow xlock state to be LOCK_PREXLOCK when putting it

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53624 from lxbsz/wip-62515
Yuri Weinstein [Mon, 23 Oct 2023 14:50:44 +0000 (07:50 -0700)]
Merge pull request #53624 from lxbsz/wip-62515

reef: qa/cephfs: switch to python3 for centos stream 9

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53572 from kotreshhr/wip-62405-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:50:13 +0000 (07:50 -0700)]
Merge pull request #53572 from kotreshhr/wip-62405-reef

reef:    mgr/volumes: Fix pending_subvolume_deletions in volume info

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53558 from batrick/wip-62733-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:49:41 +0000 (07:49 -0700)]
Merge pull request #53558 from batrick/wip-62733-reef

reef: mds: add event for batching getattr/lookup

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53553 from batrick/wip-62899-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:49:10 +0000 (07:49 -0700)]
Merge pull request #53553 from batrick/wip-62899-reef

reef: qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoMerge pull request #53548 from batrick/wip-62901-reef
Yuri Weinstein [Mon, 23 Oct 2023 14:48:31 +0000 (07:48 -0700)]
Merge pull request #53548 from batrick/wip-62901-reef

reef: mds: log message when exiting due to asok command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
20 months agoMerge pull request #53497 from lxbsz/wip-62860
Yuri Weinstein [Mon, 23 Oct 2023 14:46:53 +0000 (07:46 -0700)]
Merge pull request #53497 from lxbsz/wip-62860

reef: mds: fix deadlock between unlinking and linkmerge

Reviewed-by: Venky Shankar <vshankar@redhat.com>
20 months agoclient: always refresh mds feature bits on session open 54146/head
Venky Shankar [Mon, 16 Oct 2023 04:11:19 +0000 (00:11 -0400)]
client: always refresh mds feature bits on session open

Fixes: http://tracker.ceph.com/issues/63188
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 167fdd22e38d9febef8d7e0969403261314e3c98)

 Conflicts:
src/client/Client.cc

Minor conflict due to presence of

        cap_auths = std::move(m->cap_auths);

20 months agoMerge pull request #54118 from aaSharma14/wip-63272-reef
Nizamudeen A [Mon, 23 Oct 2023 05:09:36 +0000 (10:39 +0530)]
Merge pull request #54118 from aaSharma14/wip-63272-reef

reef: mgr/dashboard: add port and zone endpoints to import realm token form in rgw multisite

Reviewed-by: Nizamudeen A <nia@redhat.com>
21 months agoMerge pull request #54137 from zdover23/wip-doc-2023-10-22-backport-54136-to-reef
Anthony D'Atri [Sun, 22 Oct 2023 13:55:33 +0000 (09:55 -0400)]
Merge pull request #54137 from zdover23/wip-doc-2023-10-22-backport-54136-to-reef

reef: doc/rgw: pubsub capabilities reference was removed from docs

21 months agoMerge pull request #54105 from Matan-B/wip-46062-reef
Matan Breizman [Sun, 22 Oct 2023 10:05:06 +0000 (13:05 +0300)]
Merge pull request #54105 from Matan-B/wip-46062-reef

reef: crimson: Enable tcmalloc when using seastar

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53966 from Matan-B/wip-53305-reef
Matan Breizman [Sun, 22 Oct 2023 10:04:43 +0000 (13:04 +0300)]
Merge pull request #53966 from Matan-B/wip-53305-reef

reef: crimson/os/seastore/onode_manager: populate value recorders of onodes to be erased

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agocrimson/os/seastore/transaction_manager: move intermediate_key by 54140/head
Xuehan Xu [Tue, 5 Sep 2023 08:03:29 +0000 (16:03 +0800)]
crimson/os/seastore/transaction_manager: move intermediate_key by
"remap_offset" when remapping the "back" half of the original pin

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit dc6c091fd3d9a100b1164946ecb9e02685683a08)

21 months agocrimson/osd/lsan_suppressions: add MallocExtension::Register 54139/head
Matan Breizman [Tue, 22 Aug 2023 13:14:06 +0000 (13:14 +0000)]
crimson/osd/lsan_suppressions: add MallocExtension::Register

```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)

SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bc190972fabdf6eb7e772bdc9df5eee6100bb8b6)

21 months agocrimson/osd: cleanup and drop OSD::ShardDispatcher 54138/head
Yingxin Cheng [Mon, 7 Aug 2023 08:58:12 +0000 (16:58 +0800)]
crimson/osd: cleanup and drop OSD::ShardDispatcher

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit e3d910a16c4b6239c7cf523c5510bc76fadcb832)

21 months agocrimson/os/seastore/onode_manager: populate value recorders of onodes to 53966/head
Xuehan Xu [Wed, 6 Sep 2023 06:56:51 +0000 (14:56 +0800)]
crimson/os/seastore/onode_manager: populate value recorders of onodes to
be erased

Otherwise, the following modification sequence with the same transaction
might lead to onode extents' crc inconsistency during journal replay:

1. modify the last mapping in an onode extent;
2. erase the last mapping in that onode extent.

During journal replay, if the first modification is not recorded in the
delta, the onode extent's content would be inconsistent with that before
the system reboot

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 6c236bb63a9a49c7f9e2dab03a5fe0bb016560f0)

21 months agoMerge pull request #54098 from Matan-B/wip-53455-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:39 +0000 (11:32 +0300)]
Merge pull request #54098 from Matan-B/wip-53455-reef

reef: crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54097 from Matan-B/wip-53501-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:25 +0000 (11:32 +0300)]
Merge pull request #54097 from Matan-B/wip-53501-reef

reef: crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54096 from Matan-B/wip-53433-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:08 +0000 (11:32 +0300)]
Merge pull request #54096 from Matan-B/wip-53433-reef

reef: crimson/tools/perf_staged_fltree: fix compile error

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54095 from Matan-B/wip-52896-reef
Matan Breizman [Sun, 22 Oct 2023 08:31:13 +0000 (11:31 +0300)]
Merge pull request #54095 from Matan-B/wip-52896-reef

reef: crimson/net: consolidate messenger implementations and enable multi-shard UTs

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54093 from Matan-B/wip-53046-reef
Matan Breizman [Sun, 22 Oct 2023 08:30:33 +0000 (11:30 +0300)]
Merge pull request #54093 from Matan-B/wip-53046-reef

reef: crimson/tools/perf_crimson_msgr: randomize client nonce

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54092 from Matan-B/wip-51141-reef
Matan Breizman [Sun, 22 Oct 2023 08:29:52 +0000 (11:29 +0300)]
Merge pull request #54092 from Matan-B/wip-51141-reef

reef: crimson/os/seastore: OP_CLONE in seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54091 from Matan-B/wip-52306-reef
Matan Breizman [Sun, 22 Oct 2023 08:29:03 +0000 (11:29 +0300)]
Merge pull request #54091 from Matan-B/wip-52306-reef

reef: crimson/osd: fixes and cleanups around multi-core OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agodoc/rgw: pubsub capabilities reference was removed from docs 54137/head
Yuval Lifshitz [Sun, 22 Oct 2023 07:09:22 +0000 (07:09 +0000)]
doc/rgw: pubsub capabilities reference was removed from docs

most pubsub docs were removed in
0fecbe2476afbb06de7ea731b7c92066ae40cfc4
however, the text removed described functionality that is only
supported bia pubsub

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit c479f2e6caa1f8f736eedb6f0036dbaa4649a204)

21 months agoMerge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef
Anthony D'Atri [Sat, 21 Oct 2023 03:39:08 +0000 (23:39 -0400)]
Merge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef

reef: .github: Clarify checklist details

21 months ago.github: Clarify checklist details 54130/head
Anthony D'Atri [Thu, 19 Oct 2023 15:28:07 +0000 (11:28 -0400)]
.github: Clarify checklist details

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 1ca356a916910a4016d5d3460d68ea3732f3bad9)

21 months agoMerge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef
Anthony D'Atri [Fri, 20 Oct 2023 19:26:15 +0000 (15:26 -0400)]
Merge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef

reef: doc/rados: edit troubleshooting-pg (2 of x)

21 months agoMerge pull request #53924 from adk3798/wip-63169-reef
Adam King [Fri, 20 Oct 2023 17:06:32 +0000 (13:06 -0400)]
Merge pull request #53924 from adk3798/wip-63169-reef

reef: mgr/cephadm: fix upgrades with nvmeof

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #53851 from batrick/wip-63082-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:57:55 +0000 (07:57 -0700)]
Merge pull request #53851 from batrick/wip-63082-reef

reef: mds/FSMap: allow upgrades if no up mds

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53484 from batrick/wip-62852-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:57:25 +0000 (07:57 -0700)]
Merge pull request #53484 from batrick/wip-62852-reef

reef: qa: ignore expected cluster warning from damage tests

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53101 from batrick/wip-62553-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:56:51 +0000 (07:56 -0700)]
Merge pull request #53101 from batrick/wip-62553-reef

reef: libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #52460 from batrick/wip-62022-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:56:10 +0000 (07:56 -0700)]
Merge pull request #52460 from batrick/wip-62022-reef

reef: pybind/mgr: reopen database handle on blocklist

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agomgr/dashboard: add port and zone endpoints to import realm token form in 54118/head
Aashish Sharma [Fri, 13 Oct 2023 08:23:23 +0000 (13:53 +0530)]
mgr/dashboard: add port and zone endpoints to import realm token form in
rgw multisite

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 84ec19442b2db4f4e389810efbc01674d9824408)

21 months agodoc/rados: edit troubleshooting-pg (2 of x) 54114/head
Zac Dover [Wed, 18 Oct 2023 22:25:38 +0000 (08:25 +1000)]
doc/rados: edit troubleshooting-pg (2 of x)

Edit doc/rados/troubleshooting/troubleshooting-pg.rst (1 of x).

Follows https://github.com/ceph/ceph/pull/54052.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d0cdfc1d72c4edefb060f8ba9ab130189617cab1)

21 months agoMerge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef
Anthony D'Atri [Thu, 19 Oct 2023 22:09:46 +0000 (18:09 -0400)]
Merge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef

reef: doc: Documentation about main Ceph metrics

21 months agodoc: Documentation about main Ceph metrics 54111/head
Juan Miguel Olmo Martínez [Wed, 11 Oct 2023 16:00:15 +0000 (18:00 +0200)]
doc: Documentation about main Ceph metrics

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Zac Dover <zac.dover@proton.me>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 9bb63bdc8969f2ecdfeedfc8396890ad59f0d796)

21 months agoMerge pull request #53922 from adk3798/reef-mgr-cephadm-image-base
Yuri Weinstein [Thu, 19 Oct 2023 16:18:32 +0000 (09:18 -0700)]
Merge pull request #53922 from adk3798/reef-mgr-cephadm-image-base

reef: mgr/cephadm: fix default image base in reef

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #53919 from ajarr/wip-63156-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:17:54 +0000 (09:17 -0700)]
Merge pull request #53919 from ajarr/wip-63156-reef

reef: librbd: kick ExclusiveLock state machine stalled waiting for lock from reacquire_lock()

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
21 months agoMerge pull request #53864 from adk3798/wip-61963-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:17:14 +0000 (09:17 -0700)]
Merge pull request #53864 from adk3798/wip-61963-reef

reef: Adding rollback mechanism to handle bootstrap failures

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
21 months agoMerge pull request #53861 from ljflores/wip-63125-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:15:39 +0000 (09:15 -0700)]
Merge pull request #53861 from ljflores/wip-63125-reef

reef: osd: fix: slow scheduling when item_cost is large

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
21 months agoMerge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136
Yuri Weinstein [Thu, 19 Oct 2023 16:15:10 +0000 (09:15 -0700)]
Merge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136

reef:pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #53093 from sseshasa/wip-62546-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:14:25 +0000 (09:14 -0700)]
Merge pull request #53093 from sseshasa/wip-62546-reef

reef: osd/scheduler/mClockScheduler: Use same profile and client ids for all clients to ensure allocated QoS limit consumption.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agocrimson: Enable tcmalloc when using seastar 54105/head
Mark Nelson [Wed, 27 Apr 2022 15:06:22 +0000 (15:06 +0000)]
crimson: Enable tcmalloc when using seastar

classic-osds have always caused significant memory fragmentation
when using the libc memory allocator due to the way that Ceph
tends to utilize memory.  In recent testing, crimson-osd was found
to use 25-27GB of RAM with the stock 3GB bluestore cache settings
(osd_memory_target is only used when tcmalloc is available).  Upon
further testing, it was found that the classic OSD is even worse,
using between 32-33GB of RAM after a 5 minute 4K sequential
write test when using libc malloc.

The good news is that it appears that crimson-osd is able to use
tcmalloc for alienstore without significant modification. Better
still, it drastically reduces memory usage.  In the same test that
resulted in 25GB RSS memory usage for crimson-osd with libc malloc,
a tcmalloc linked version took around 9GB (with an 8GB
osd_memory_target).  Since we do not yet (afaik) expose classic OSD
debugging in crimson it is tough to tell why we are still a little
over, but it's clear that for alienstore we are going to need to
use tcmalloc as we do in classic.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit d884a45aea8d5e0a5fa1689f2fd3167d68292893)

21 months agoRevert "[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar"
Matan Breizman [Tue, 6 Jun 2023 14:53:58 +0000 (14:53 +0000)]
Revert "[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar"

This reverts commit 380bc6d2e4ec62e724612dbac57ffdbd82511f9d.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 96c6c2681faa616314582ad3e9ccef71ee6d5f69)

21 months agoMerge pull request #54057 from Matan-B/wip-51875-reef
Matan Breizman [Thu, 19 Oct 2023 14:29:02 +0000 (17:29 +0300)]
Merge pull request #54057 from Matan-B/wip-51875-reef

reef: crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53957 from Matan-B/wip-52529-reef
Matan Breizman [Thu, 19 Oct 2023 12:28:26 +0000 (15:28 +0300)]
Merge pull request #53957 from Matan-B/wip-52529-reef

reef: crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54021 from Matan-B/wip-48052-reef
Matan Breizman [Thu, 19 Oct 2023 08:34:38 +0000 (11:34 +0300)]
Merge pull request #54021 from Matan-B/wip-48052-reef

reef: qa/suites/crimson-rados: bring backfill testing

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agocrimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it 53957/head
Xuehan Xu [Wed, 19 Jul 2023 07:52:22 +0000 (15:52 +0800)]
crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
with ReplicateRequest

This commit is a follow-up for e14142894922b403344d7f08315b67a291b9a27a
and f4d336a2dbf864fe798ca7f500d44576d3f626d5

Fixes: https://tracker.ceph.com/issues/61943
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 0b7c47db78cc22c7cf25c50b430674e17b01a601)

21 months agocrimson/os/seastore/cache: replace is_clean by is_stable_clean wherever 54098/head
Xuehan Xu [Tue, 26 Sep 2023 03:30:54 +0000 (11:30 +0800)]
crimson/os/seastore/cache: replace is_clean by is_stable_clean wherever
possible

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit f60f318450fd8118b96e3dbae29cccc68ed1ae19)

21 months agocrimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru
Xuehan Xu [Thu, 14 Sep 2023 07:50:20 +0000 (15:50 +0800)]
crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 88ca65776bd7248152b55360d4621107ce0caeb9)

21 months agocrimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent 54097/head
Zhang Song [Tue, 19 Sep 2023 06:08:51 +0000 (14:08 +0800)]
crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit 07577e3bc14423acec391a85ee79ddc6b1f269a1)

21 months agocrimson/tools/perf_staged_fltree: fix compile error 54096/head
Myoungwon Oh [Wed, 13 Sep 2023 05:16:16 +0000 (14:16 +0900)]
crimson/tools/perf_staged_fltree: fix compile error

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 251288410c3ad8c763e386bec92eedb327801a4d)

21 months agotest/crimson: increase smp settings for test_socket and test_messenger 54095/head
Yingxin Cheng [Thu, 3 Aug 2023 07:51:31 +0000 (15:51 +0800)]
test/crimson: increase smp settings for test_socket and test_messenger

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit e44ce1a892b721ae4a5a8d9e3525ff2c95cb1448)

21 months agotest/crimson/test_messenger: add delay to suite markdown
Yingxin Cheng [Thu, 3 Aug 2023 07:45:16 +0000 (15:45 +0800)]
test/crimson/test_messenger: add delay to suite markdown

markdown() needs delay to be propagated to the primary core.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit ec8722f2128a92f5ef76f1c0cd14e33ec8a5c295)

21 months agotest/crimson/test_messenger: enable multi-core messenger in FailoverSuite
Yingxin Cheng [Thu, 3 Aug 2023 07:43:37 +0000 (15:43 +0800)]
test/crimson/test_messenger: enable multi-core messenger in FailoverSuite

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 290dd2d68df8289352438ca9167ed1d18ddcdd9a)

21 months agotest/crimson/test_messenger: support cross-core protocol-level tests
Yingxin Cheng [Thu, 3 Aug 2023 07:42:45 +0000 (15:42 +0800)]
test/crimson/test_messenger: support cross-core protocol-level tests

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit d65ba2d018b14d87252f50baa890350a6e7b38e3)

21 months agocrimson/net: support cross-core blocking for interceptor
Yingxin Cheng [Thu, 3 Aug 2023 07:30:10 +0000 (15:30 +0800)]
crimson/net: support cross-core blocking for interceptor

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit bcf20d6771f5adf608fcf44b47073c133c6b0b35)

21 months agocrimson/net: should not check shutdown from a different core
Yingxin Cheng [Thu, 3 Aug 2023 07:28:08 +0000 (15:28 +0800)]
crimson/net: should not check shutdown from a different core

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 74e9098833e578d7514b8ab1f7a4588a052885a4)

21 months agocrimson/net: rework interceptor to support cross-core interceptions
Yingxin Cheng [Thu, 3 Aug 2023 07:26:54 +0000 (15:26 +0800)]
crimson/net: rework interceptor to support cross-core interceptions

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 6be5e630ed79e09e0b77c481856f56a6b27ea562)

21 months agotest/crimson/test_messenger: relax the ordering checks to tracked_conn
Yingxin Cheng [Wed, 26 Jul 2023 04:52:57 +0000 (12:52 +0800)]
test/crimson/test_messenger: relax the ordering checks to tracked_conn

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 4515c34977f8c54d459b0140305e24c9bb29a455)

21 months agotest/crimson/test_messenger: move all connection state checks to the protocol level
Yingxin Cheng [Fri, 21 Jul 2023 03:17:32 +0000 (11:17 +0800)]
test/crimson/test_messenger: move all connection state checks to the protocol level

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 933745cd815f7cc81438a89b3ba9ac5fc2ef61b7)

21 months agocrimson/net: don't miss keepalive during message sweeping
Yingxin Cheng [Fri, 21 Jul 2023 03:12:25 +0000 (11:12 +0800)]
crimson/net: don't miss keepalive during message sweeping

The flags need to be updated atomically to prevent missing them during
message sweeping.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 9bf4eeaa923351dc8f53b7cc3a02c2e8483c054e)