]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
20 months agocrimson/osd/lsan_suppressions: add MallocExtension::Register 54139/head
Matan Breizman [Tue, 22 Aug 2023 13:14:06 +0000 (13:14 +0000)]
crimson/osd/lsan_suppressions: add MallocExtension::Register

```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)

SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bc190972fabdf6eb7e772bdc9df5eee6100bb8b6)

20 months agoMerge pull request #54098 from Matan-B/wip-53455-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:39 +0000 (11:32 +0300)]
Merge pull request #54098 from Matan-B/wip-53455-reef

reef: crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54097 from Matan-B/wip-53501-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:25 +0000 (11:32 +0300)]
Merge pull request #54097 from Matan-B/wip-53501-reef

reef: crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54096 from Matan-B/wip-53433-reef
Matan Breizman [Sun, 22 Oct 2023 08:32:08 +0000 (11:32 +0300)]
Merge pull request #54096 from Matan-B/wip-53433-reef

reef: crimson/tools/perf_staged_fltree: fix compile error

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54095 from Matan-B/wip-52896-reef
Matan Breizman [Sun, 22 Oct 2023 08:31:13 +0000 (11:31 +0300)]
Merge pull request #54095 from Matan-B/wip-52896-reef

reef: crimson/net: consolidate messenger implementations and enable multi-shard UTs

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54093 from Matan-B/wip-53046-reef
Matan Breizman [Sun, 22 Oct 2023 08:30:33 +0000 (11:30 +0300)]
Merge pull request #54093 from Matan-B/wip-53046-reef

reef: crimson/tools/perf_crimson_msgr: randomize client nonce

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54092 from Matan-B/wip-51141-reef
Matan Breizman [Sun, 22 Oct 2023 08:29:52 +0000 (11:29 +0300)]
Merge pull request #54092 from Matan-B/wip-51141-reef

reef: crimson/os/seastore: OP_CLONE in seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54091 from Matan-B/wip-52306-reef
Matan Breizman [Sun, 22 Oct 2023 08:29:03 +0000 (11:29 +0300)]
Merge pull request #54091 from Matan-B/wip-52306-reef

reef: crimson/osd: fixes and cleanups around multi-core OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
20 months agoMerge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef
Anthony D'Atri [Sat, 21 Oct 2023 03:39:08 +0000 (23:39 -0400)]
Merge pull request #54130 from zdover23/wip-doc-2023-10-21-backport-54107-to-reef

reef: .github: Clarify checklist details

20 months ago.github: Clarify checklist details 54130/head
Anthony D'Atri [Thu, 19 Oct 2023 15:28:07 +0000 (11:28 -0400)]
.github: Clarify checklist details

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 1ca356a916910a4016d5d3460d68ea3732f3bad9)

20 months agoMerge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef
Anthony D'Atri [Fri, 20 Oct 2023 19:26:15 +0000 (15:26 -0400)]
Merge pull request #54114 from zdover23/wip-doc-2023-10-20-backport-54084-to-reef

reef: doc/rados: edit troubleshooting-pg (2 of x)

21 months agoMerge pull request #53924 from adk3798/wip-63169-reef
Adam King [Fri, 20 Oct 2023 17:06:32 +0000 (13:06 -0400)]
Merge pull request #53924 from adk3798/wip-63169-reef

reef: mgr/cephadm: fix upgrades with nvmeof

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #53851 from batrick/wip-63082-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:57:55 +0000 (07:57 -0700)]
Merge pull request #53851 from batrick/wip-63082-reef

reef: mds/FSMap: allow upgrades if no up mds

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53484 from batrick/wip-62852-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:57:25 +0000 (07:57 -0700)]
Merge pull request #53484 from batrick/wip-62852-reef

reef: qa: ignore expected cluster warning from damage tests

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53101 from batrick/wip-62553-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:56:51 +0000 (07:56 -0700)]
Merge pull request #53101 from batrick/wip-62553-reef

reef: libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #52460 from batrick/wip-62022-reef
Yuri Weinstein [Fri, 20 Oct 2023 14:56:10 +0000 (07:56 -0700)]
Merge pull request #52460 from batrick/wip-62022-reef

reef: pybind/mgr: reopen database handle on blocklist

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agodoc/rados: edit troubleshooting-pg (2 of x) 54114/head
Zac Dover [Wed, 18 Oct 2023 22:25:38 +0000 (08:25 +1000)]
doc/rados: edit troubleshooting-pg (2 of x)

Edit doc/rados/troubleshooting/troubleshooting-pg.rst (1 of x).

Follows https://github.com/ceph/ceph/pull/54052.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d0cdfc1d72c4edefb060f8ba9ab130189617cab1)

21 months agoMerge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef
Anthony D'Atri [Thu, 19 Oct 2023 22:09:46 +0000 (18:09 -0400)]
Merge pull request #54111 from zdover23/wip-doc-2023-10-20-backport-54099-to-reef

reef: doc: Documentation about main Ceph metrics

21 months agodoc: Documentation about main Ceph metrics 54111/head
Juan Miguel Olmo Martínez [Wed, 11 Oct 2023 16:00:15 +0000 (18:00 +0200)]
doc: Documentation about main Ceph metrics

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Zac Dover <zac.dover@proton.me>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 9bb63bdc8969f2ecdfeedfc8396890ad59f0d796)

21 months agoMerge pull request #53922 from adk3798/reef-mgr-cephadm-image-base
Yuri Weinstein [Thu, 19 Oct 2023 16:18:32 +0000 (09:18 -0700)]
Merge pull request #53922 from adk3798/reef-mgr-cephadm-image-base

reef: mgr/cephadm: fix default image base in reef

Reviewed-by: John Mulligan <jmulligan@redhat.com>
21 months agoMerge pull request #53919 from ajarr/wip-63156-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:17:54 +0000 (09:17 -0700)]
Merge pull request #53919 from ajarr/wip-63156-reef

reef: librbd: kick ExclusiveLock state machine stalled waiting for lock from reacquire_lock()

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
21 months agoMerge pull request #53864 from adk3798/wip-61963-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:17:14 +0000 (09:17 -0700)]
Merge pull request #53864 from adk3798/wip-61963-reef

reef: Adding rollback mechanism to handle bootstrap failures

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
21 months agoMerge pull request #53861 from ljflores/wip-63125-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:15:39 +0000 (09:15 -0700)]
Merge pull request #53861 from ljflores/wip-63125-reef

reef: osd: fix: slow scheduling when item_cost is large

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
21 months agoMerge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136
Yuri Weinstein [Thu, 19 Oct 2023 16:15:10 +0000 (09:15 -0700)]
Merge pull request #53534 from kamoltat/wip-ksirivad-reef-backport-54136

reef:pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #53093 from sseshasa/wip-62546-reef
Yuri Weinstein [Thu, 19 Oct 2023 16:14:25 +0000 (09:14 -0700)]
Merge pull request #53093 from sseshasa/wip-62546-reef

reef: osd/scheduler/mClockScheduler: Use same profile and client ids for all clients to ensure allocated QoS limit consumption.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #54057 from Matan-B/wip-51875-reef
Matan Breizman [Thu, 19 Oct 2023 14:29:02 +0000 (17:29 +0300)]
Merge pull request #54057 from Matan-B/wip-51875-reef

reef: crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53957 from Matan-B/wip-52529-reef
Matan Breizman [Thu, 19 Oct 2023 12:28:26 +0000 (15:28 +0300)]
Merge pull request #53957 from Matan-B/wip-52529-reef

reef: crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54021 from Matan-B/wip-48052-reef
Matan Breizman [Thu, 19 Oct 2023 08:34:38 +0000 (11:34 +0300)]
Merge pull request #54021 from Matan-B/wip-48052-reef

reef: qa/suites/crimson-rados: bring backfill testing

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agocrimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it 53957/head
Xuehan Xu [Wed, 19 Jul 2023 07:52:22 +0000 (15:52 +0800)]
crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
with ReplicateRequest

This commit is a follow-up for e14142894922b403344d7f08315b67a291b9a27a
and f4d336a2dbf864fe798ca7f500d44576d3f626d5

Fixes: https://tracker.ceph.com/issues/61943
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 0b7c47db78cc22c7cf25c50b430674e17b01a601)

21 months agocrimson/os/seastore/cache: replace is_clean by is_stable_clean wherever 54098/head
Xuehan Xu [Tue, 26 Sep 2023 03:30:54 +0000 (11:30 +0800)]
crimson/os/seastore/cache: replace is_clean by is_stable_clean wherever
possible

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit f60f318450fd8118b96e3dbae29cccc68ed1ae19)

21 months agocrimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru
Xuehan Xu [Thu, 14 Sep 2023 07:50:20 +0000 (15:50 +0800)]
crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 88ca65776bd7248152b55360d4621107ce0caeb9)

21 months agocrimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent 54097/head
Zhang Song [Tue, 19 Sep 2023 06:08:51 +0000 (14:08 +0800)]
crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit 07577e3bc14423acec391a85ee79ddc6b1f269a1)

21 months agocrimson/tools/perf_staged_fltree: fix compile error 54096/head
Myoungwon Oh [Wed, 13 Sep 2023 05:16:16 +0000 (14:16 +0900)]
crimson/tools/perf_staged_fltree: fix compile error

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 251288410c3ad8c763e386bec92eedb327801a4d)

21 months agotest/crimson: increase smp settings for test_socket and test_messenger 54095/head
Yingxin Cheng [Thu, 3 Aug 2023 07:51:31 +0000 (15:51 +0800)]
test/crimson: increase smp settings for test_socket and test_messenger

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit e44ce1a892b721ae4a5a8d9e3525ff2c95cb1448)

21 months agotest/crimson/test_messenger: add delay to suite markdown
Yingxin Cheng [Thu, 3 Aug 2023 07:45:16 +0000 (15:45 +0800)]
test/crimson/test_messenger: add delay to suite markdown

markdown() needs delay to be propagated to the primary core.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit ec8722f2128a92f5ef76f1c0cd14e33ec8a5c295)

21 months agotest/crimson/test_messenger: enable multi-core messenger in FailoverSuite
Yingxin Cheng [Thu, 3 Aug 2023 07:43:37 +0000 (15:43 +0800)]
test/crimson/test_messenger: enable multi-core messenger in FailoverSuite

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 290dd2d68df8289352438ca9167ed1d18ddcdd9a)

21 months agotest/crimson/test_messenger: support cross-core protocol-level tests
Yingxin Cheng [Thu, 3 Aug 2023 07:42:45 +0000 (15:42 +0800)]
test/crimson/test_messenger: support cross-core protocol-level tests

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit d65ba2d018b14d87252f50baa890350a6e7b38e3)

21 months agocrimson/net: support cross-core blocking for interceptor
Yingxin Cheng [Thu, 3 Aug 2023 07:30:10 +0000 (15:30 +0800)]
crimson/net: support cross-core blocking for interceptor

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit bcf20d6771f5adf608fcf44b47073c133c6b0b35)

21 months agocrimson/net: should not check shutdown from a different core
Yingxin Cheng [Thu, 3 Aug 2023 07:28:08 +0000 (15:28 +0800)]
crimson/net: should not check shutdown from a different core

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 74e9098833e578d7514b8ab1f7a4588a052885a4)

21 months agocrimson/net: rework interceptor to support cross-core interceptions
Yingxin Cheng [Thu, 3 Aug 2023 07:26:54 +0000 (15:26 +0800)]
crimson/net: rework interceptor to support cross-core interceptions

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 6be5e630ed79e09e0b77c481856f56a6b27ea562)

21 months agotest/crimson/test_messenger: relax the ordering checks to tracked_conn
Yingxin Cheng [Wed, 26 Jul 2023 04:52:57 +0000 (12:52 +0800)]
test/crimson/test_messenger: relax the ordering checks to tracked_conn

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 4515c34977f8c54d459b0140305e24c9bb29a455)

21 months agotest/crimson/test_messenger: move all connection state checks to the protocol level
Yingxin Cheng [Fri, 21 Jul 2023 03:17:32 +0000 (11:17 +0800)]
test/crimson/test_messenger: move all connection state checks to the protocol level

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 933745cd815f7cc81438a89b3ba9ac5fc2ef61b7)

21 months agocrimson/net: don't miss keepalive during message sweeping
Yingxin Cheng [Fri, 21 Jul 2023 03:12:25 +0000 (11:12 +0800)]
crimson/net: don't miss keepalive during message sweeping

The flags need to be updated atomically to prevent missing them during
message sweeping.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 9bf4eeaa923351dc8f53b7cc3a02c2e8483c054e)

21 months agotest/crimson/test_messenger: drop test_concurrent_dispatch()
Yingxin Cheng [Mon, 10 Jul 2023 07:47:58 +0000 (15:47 +0800)]
test/crimson/test_messenger: drop test_concurrent_dispatch()

The only purpose of test_concurrent_dispatch() is to verify that 2
messages are all received. Revise test_echo() for the same purpose, and
drop test_concurrent_dispatch().

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit a4f6c82a9a0897aa1cc5ff02c10b660fcbd6eba0)

21 months agotest/crimson/test_messenger: implement multi-shard test_echo::test_state::Client
Yingxin Cheng [Mon, 10 Jul 2023 07:18:22 +0000 (15:18 +0800)]
test/crimson/test_messenger: implement multi-shard test_echo::test_state::Client

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 9b5182779d76043c428e664f201acca5986aab7c)

21 months agotest/crimson: add missing calls to ConfigProxy::start()
Yingxin Cheng [Mon, 10 Jul 2023 07:16:11 +0000 (15:16 +0800)]
test/crimson: add missing calls to ConfigProxy::start()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 9d817d33d564a4240ba576c378046edee54a599c)

21 months agocrimson/net: adjust dispatcher interface about cross-core notifications
Yingxin Cheng [Mon, 10 Jul 2023 01:51:39 +0000 (09:51 +0800)]
crimson/net: adjust dispatcher interface about cross-core notifications

Due to that we aren't able to determine cross-core ordering:
* Move ms_handle_connect/accept() to be called in the new shard, so it
  will notify before ms_dispatch() in the same core;
* Introduce another ms_handle_shard_change() when the current core is
  changed;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 6bab7e698db8b7f3bec240952ed33a2bb9918a20)

21 months agotest/crimson/test_messenger: implement multi-shard test_echo::test_state::Server
Yingxin Cheng [Fri, 7 Jul 2023 02:13:18 +0000 (10:13 +0800)]
test/crimson/test_messenger: implement multi-shard test_echo::test_state::Server

Also introduces ShardedGates.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit ec6acfc352b1bd57b7e0d87114bbf51f2c0190b9)

21 months agocrimson/tools/perf_crimson_msgr: randomize client nonce 54093/head
Yingxin Cheng [Thu, 17 Aug 2023 03:28:37 +0000 (11:28 +0800)]
crimson/tools/perf_crimson_msgr: randomize client nonce

In case we want to start multiple clients simultaneously.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 91c65e22387aa4b7c572712f4a3028ed6e8d40f9)

21 months agocrimson/os/seastore: debug outputs 54092/head
xuxuehan [Fri, 26 May 2023 08:03:59 +0000 (16:03 +0800)]
crimson/os/seastore: debug outputs

Signed-off-by: xuxuehan <xuxuehan@qianxin.com>
(cherry picked from commit 6a583aad13e5fc26bbfc7374b6c292f0dc77b662)

21 months agocrimson/os/seastore/object_data_handler: take indirect offset into
Xuehan Xu [Wed, 12 Jul 2023 10:35:17 +0000 (18:35 +0800)]
crimson/os/seastore/object_data_handler: take indirect offset into
account when reading object data

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 9cdcd06caadef6cdca64a974c6525ba300c1e289)

21 months agocrimson/os/seastore/transaction_manager: don't remove intermediate
Xuehan Xu [Tue, 11 Jul 2023 10:30:03 +0000 (18:30 +0800)]
crimson/os/seastore/transaction_manager: don't remove intermediate
mapping when remapping pins

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 4b14549e6ebd6aa63e9006d3c487e139731817fc)

21 months agocrimson/os/seastore/transaction_manager: remap indirect mappings
Xuehan Xu [Tue, 11 Jul 2023 09:48:53 +0000 (17:48 +0800)]
crimson/os/seastore/transaction_manager: remap indirect mappings

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit e2c0ebec76aec12e8af6558fff22657acf64bd5d)

21 months agocrimson/os/seastore/transaction_manager: remove extents for physical lba mappings...
xuxuehan [Thu, 25 May 2023 03:02:29 +0000 (11:02 +0800)]
crimson/os/seastore/transaction_manager: remove extents for physical lba mappings if necessary
when removing indirect lba mappings

Signed-off-by: xuxuehan <xuxuehan@qianxin.com>
(cherry picked from commit d6bfd9455daa4ea5f0c86d0a931b808c5c60b3f5)

21 months agocrimson/os/seastore/lba_manager: decrease refcount of the physical lba
Xuehan Xu [Wed, 17 May 2023 03:35:20 +0000 (03:35 +0000)]
crimson/os/seastore/lba_manager: decrease refcount of the physical lba
mappings when removing indirect lba mappings

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 31204c2e1cb05ac1476abbcc5d5198f38e326095)

21 months agocrimson/os/seastore/onode_manager: erase onodes from the OnodeTree at the moment...
xuxuehan [Fri, 26 May 2023 07:55:44 +0000 (15:55 +0800)]
crimson/os/seastore/onode_manager: erase onodes from the OnodeTree at the moment of erase_onode

Signed-off-by: xuxuehan <xuxuehan@qianxin.com>
(cherry picked from commit e5ee75ee85a7c6f5de6823986eeb30fe44349c47)

21 months agocrimson/os/seastore/object_data_handler: don't reserve lba space when truncating...
Xuehan Xu [Sun, 23 Apr 2023 06:22:52 +0000 (06:22 +0000)]
crimson/os/seastore/object_data_handler: don't reserve lba space when truncating objects to zero size

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 826bb50fb3c4d279fa06a648940aebf78303f82d)

21 months agotest/crimson/seastore: add OP_CLONE unittest case
Xuehan Xu [Sun, 2 Apr 2023 06:10:59 +0000 (14:10 +0800)]
test/crimson/seastore: add OP_CLONE unittest case

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 9993b99c1c86bbfd1248db6f621cde7dc966b5d5)

21 months agocrimson/os/seastore/lba_manager: take lba entry indirection into account when getting...
Xuehan Xu [Sat, 6 May 2023 03:01:58 +0000 (03:01 +0000)]
crimson/os/seastore/lba_manager: take lba entry indirection into account when getting lba mappings

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 48023c7f67600cbbccc49f6499f0df3526e1b882)

21 months agocrimson/os/seastore: implement OP_CLONE
Xuehan Xu [Wed, 29 Mar 2023 10:07:20 +0000 (18:07 +0800)]
crimson/os/seastore: implement OP_CLONE

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 9f303cde7b9d7aca14a2024948079b4280168a2d)

21 months agocrimson/os/seastore/lba_manager: lba map value may contain laddr
Xuehan Xu [Fri, 17 Mar 2023 10:15:20 +0000 (18:15 +0800)]
crimson/os/seastore/lba_manager: lba map value may contain laddr

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit f11504cd5d3fbba01cf9dbf11345e2c15001f947)

21 months agocrimson/osd: cleanup ShardDispatcher 54091/head
Yingxin Cheng [Tue, 4 Jul 2023 02:31:55 +0000 (10:31 +0800)]
crimson/osd: cleanup ShardDispatcher

ShardDispatcher logically equals to seastar::sharded<PGShardManager>,
drop unnecessary members.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 4ecc58d0ff372420a780c41bcaa975a25719c207)

21 months agocrimson/osd: avoid unnecessary foreign copy in ms_dispatch()
Yingxin Cheng [Tue, 4 Jul 2023 02:24:22 +0000 (10:24 +0800)]
crimson/osd: avoid unnecessary foreign copy in ms_dispatch()

Foreign-copy also implies that requests may be out-of-order, which is
incorrect in the data path.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 436706c4612bada876682df967cd40187805e823)

21 months agocrimson/osd: make sure OSD is constructing in the primary core
Yingxin Cheng [Tue, 4 Jul 2023 02:18:41 +0000 (10:18 +0800)]
crimson/osd: make sure OSD is constructing in the primary core

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit d6c5f05933e279607576dc2d6968c67248528813)

21 months agoMerge pull request #54020 from Matan-B/wip-52784-reef
Matan Breizman [Thu, 19 Oct 2023 07:11:10 +0000 (10:11 +0300)]
Merge pull request #54020 from Matan-B/wip-52784-reef

reef: qa/suites/crimson-rados: add centos9 to supported distros

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54019 from Matan-B/wip-52598-reef
Matan Breizman [Thu, 19 Oct 2023 07:10:39 +0000 (10:10 +0300)]
Merge pull request #54019 from Matan-B/wip-52598-reef

reef: qa/suites/crimson-rados: Use centos8 for testing

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54063 from Matan-B/wip-52592-reef
Matan Breizman [Thu, 19 Oct 2023 07:09:26 +0000 (10:09 +0300)]
Merge pull request #54063 from Matan-B/wip-52592-reef

reef: crimson/net: set TCP_NODELAY according to ms_tcp_nodelay

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54062 from Matan-B/wip-52420-reef
Matan Breizman [Thu, 19 Oct 2023 07:07:46 +0000 (10:07 +0300)]
Merge pull request #54062 from Matan-B/wip-52420-reef

reef: crimson/os/seastore/zbd: zbdsegmentmanager write path fixes.

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54061 from Matan-B/wip-52390-reef
Matan Breizman [Thu, 19 Oct 2023 07:07:02 +0000 (10:07 +0300)]
Merge pull request #54061 from Matan-B/wip-52390-reef

reef: crimson/os/object_data_handler: splitting right side doesn't mean splitting only one extent

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54060 from Matan-B/wip-52223-reef
Matan Breizman [Thu, 19 Oct 2023 07:06:34 +0000 (10:06 +0300)]
Merge pull request #54060 from Matan-B/wip-52223-reef

reef: crimson/osd/osd_operations: consistent naming to pipeline users

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54059 from Matan-B/wip-52091-reef
Matan Breizman [Thu, 19 Oct 2023 07:05:40 +0000 (10:05 +0300)]
Merge pull request #54059 from Matan-B/wip-52091-reef

reef: crimson/tools/perf_crimson_msgr: integrate multi-core msgr with various improvements

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54058 from Matan-B/wip-51147-reef
Matan Breizman [Thu, 19 Oct 2023 07:03:09 +0000 (10:03 +0300)]
Merge pull request #54058 from Matan-B/wip-51147-reef

reef: Crimson/osd: support multicore osd

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #54069 from rhcs-dashboard/wip-63231-reef
Nizamudeen A [Thu, 19 Oct 2023 04:45:24 +0000 (10:15 +0530)]
Merge pull request #54069 from rhcs-dashboard/wip-63231-reef

reef: mgr/dashboard: disable hosts field while editing the filesystem

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
21 months agoMerge pull request #53921 from ljflores/wip-63168-reef
Zack Cerza [Wed, 18 Oct 2023 20:57:35 +0000 (14:57 -0600)]
Merge pull request #53921 from ljflores/wip-63168-reef

reef: qa/suites/upgrade: fix env indentation in stress-split upgrade tests

21 months agoMerge pull request #54079 from zdover23/wip-doc-2023-10-19-backport-54051-to-reef
Anthony D'Atri [Wed, 18 Oct 2023 20:04:55 +0000 (16:04 -0400)]
Merge pull request #54079 from zdover23/wip-doc-2023-10-19-backport-54051-to-reef

reef: doc: Update mClock QOS documentation to discard osd_mclock_cost_per_*

21 months agodoc: Update mClock QOS documentation to discard osd_mclock_cost_per_* 54079/head
tanchangzhi [Tue, 17 Oct 2023 08:48:51 +0000 (16:48 +0800)]
doc: Update mClock QOS documentation to discard osd_mclock_cost_per_*

The cost parameters (osd_mclock_cost_per_*) have been removed.
The cost of an operation is now determined using the random IOPS
and maximum sequential bandwidth capability of the OSD's underlying device.

Fixes: https://tracker.ceph.com/issues/58529
Signed-off-by: tanchangzhi <544463199@qq.com>
(cherry picked from commit a7502cb5125dda8d052dc3991ed5d7013606aca2)

21 months agoMerge pull request #53654 from mcv21/wip-62970-reef
Yuri Weinstein [Wed, 18 Oct 2023 18:50:17 +0000 (11:50 -0700)]
Merge pull request #53654 from mcv21/wip-62970-reef

reef: Debian: update to dh compat 12, fix more serious packaging errors, correct copyright syntax

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
21 months agoMerge pull request #53823 from cfsnyder/wip-62944-reef
Casey Bodley [Wed, 18 Oct 2023 17:52:36 +0000 (18:52 +0100)]
Merge pull request #53823 from cfsnyder/wip-62944-reef

reef: rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #54071 from zdover23/wip-doc-2023-10-18-backport-54028-to-reef
Anthony D'Atri [Wed, 18 Oct 2023 16:49:42 +0000 (12:49 -0400)]
Merge pull request #54071 from zdover23/wip-doc-2023-10-18-backport-54028-to-reef

reef: doc/start: Modernize and clarify hardware-recommendations.rst

21 months agoMerge pull request #54073 from zdover23/wip-doc-2023-10-18-backport-54052-to-reef
Anthony D'Atri [Wed, 18 Oct 2023 16:49:00 +0000 (12:49 -0400)]
Merge pull request #54073 from zdover23/wip-doc-2023-10-18-backport-54052-to-reef

reef: doc/rados: edit troubleshooting-pg.rst (1 of x)

21 months agoMerge pull request #53229 from ifed01/wip-ifed-snapdiff-ref
Yuri Weinstein [Wed, 18 Oct 2023 16:22:31 +0000 (09:22 -0700)]
Merge pull request #53229 from ifed01/wip-ifed-snapdiff-ref

reef: cephfs: implement snapdiff

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
21 months agoMerge pull request #52850 from lxbsz/wip-61165
Yuri Weinstein [Wed, 18 Oct 2023 16:21:22 +0000 (09:21 -0700)]
Merge pull request #52850 from lxbsz/wip-61165

reef: client: issue a cap release immediately if no cap exists

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #52847 from lxbsz/wip-62192
Yuri Weinstein [Wed, 18 Oct 2023 16:20:35 +0000 (09:20 -0700)]
Merge pull request #52847 from lxbsz/wip-62192

reef: mds: do not send split_realms for CEPH_SNAP_OP_UPDATE msg

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agorgw: fix output formatting of bucket index check admin api 53823/head
Cory Snyder [Mon, 25 Sep 2023 10:06:41 +0000 (10:06 +0000)]
rgw: fix output formatting of bucket index check admin api

The bucket index check admin API was previously returning invalid
JSON.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 32fb6a1a68398a99324b2e64ebe3bcf3a9ccf02a)

Conflicts:
src/rgw/driver/rados/rgw_bucket.cc

Cherry-pick notes:
- Conflicts due to check_bad_index_multipart taking optional_yield param on main

21 months agorgw: fix radosgw-admin bucket check stat calculation bug
Cory Snyder [Fri, 22 Sep 2023 21:08:25 +0000 (21:08 +0000)]
rgw: fix radosgw-admin bucket check stat calculation bug

Fixes a regression with radosgw-admin bucket check stat
calculation and bucket reshard stat calculation when
there are objects that have transitioned from unversioned
to versioned. The bug was introduced in
152aadb71b61c53a4832a1c8cf82fce3d64b68d1.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 4728daa5557bfb79a608dd903b8630e2b15fcb2c)

21 months agorgw: add test case to reproduce bucket check stats bug for versioned bucket
Cory Snyder [Fri, 22 Sep 2023 21:00:46 +0000 (21:00 +0000)]
rgw: add test case to reproduce bucket check stats bug for versioned bucket

Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 340522f9aed50d65137568c1f9dcf4b1e7945a79)

21 months agorgw: radosgw-admin bucket check should only print index entries with --check-objects...
Cory Snyder [Fri, 22 Sep 2023 08:35:16 +0000 (08:35 +0000)]
rgw: radosgw-admin bucket check should only print index entries with --check-objects flag

Printing all index entries can be very time consuming for large
buckets and the inability to switch this behavior off makes it
cumbersome to use the command for fixing bucket stats. This was
also preventing the command from outputting recalculated bucket
stats when the --fix flag wasn't specified.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 6b057fe55413c0eaf9959f006584cba6cc4c192a)

21 months agorgw: prevent another leftover bucket index olh entry scenario
Cory Snyder [Thu, 21 Sep 2023 19:27:51 +0000 (19:27 +0000)]
rgw: prevent another leftover bucket index olh entry scenario

If a call to bucket_index_link_olh or bucket_index_unlink_instance
fails, its associated pending xattr may have prevented the olh object
from being removed by another thread. We should do a best effort
cleanup attempt for this case by calling update_olh before returning
an error to the caller.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 570adec5bb8142f5baf1f05f0040e8afdb11ec05)

21 months agorgw: fix rgw versioned bucket stat accounting during reshard and check index
Cory Snyder [Thu, 7 Sep 2023 17:23:14 +0000 (17:23 +0000)]
rgw: fix rgw versioned bucket stat accounting during reshard and check index

Fixes: https://tracker.ceph.com/issues/62760
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 152aadb71b61c53a4832a1c8cf82fce3d64b68d1)

21 months agoqa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Cory Snyder [Thu, 7 Sep 2023 14:43:23 +0000 (14:43 +0000)]
qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit b79dcf640ac2cc3dacf1b87bbe351db823c445d0)

21 months agorgw: add radosgw-admin bucket check olh/unlinked commands
Cory Snyder [Thu, 25 May 2023 16:59:04 +0000 (16:59 +0000)]
rgw: add radosgw-admin bucket check olh/unlinked commands

Adds commands to radosgw-admin for checking for and fixing leftover
entries in the bucket index (and associated RADOS objects).

Fixes: https://tracker.ceph.com/issues/62075
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 9b2042a0750109f575b02c9c4b127dd941994662)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc

Cherry-pick notes:
- Conflicts due to multiple methods takeing optional_yield param on main

21 months agoMerge pull request #53329 from vshankar/tr-62583
Yuri Weinstein [Wed, 18 Oct 2023 14:55:26 +0000 (07:55 -0700)]
Merge pull request #53329 from vshankar/tr-62583

reef: mds: blocklist clients with "bloated" session metadata

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53271 from mchangir/wip-59408-reef
Yuri Weinstein [Wed, 18 Oct 2023 14:54:48 +0000 (07:54 -0700)]
Merge pull request #53271 from mchangir/wip-59408-reef

reef: cephfs_mirror: correctly set top level dir permissions

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53241 from batrick/wip-62661-reef
Yuri Weinstein [Wed, 18 Oct 2023 14:54:19 +0000 (07:54 -0700)]
Merge pull request #53241 from batrick/wip-62661-reef

reef: mds: drop locks and retry when lock set changes

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53187 from batrick/wip-62609-reef
Yuri Weinstein [Wed, 18 Oct 2023 14:52:36 +0000 (07:52 -0700)]
Merge pull request #53187 from batrick/wip-62609-reef

reef: mgr: register OSDs in ms_handle_accept

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
21 months agoMerge pull request #53183 from batrick/wip-62518-reef
Yuri Weinstein [Wed, 18 Oct 2023 14:51:41 +0000 (07:51 -0700)]
Merge pull request #53183 from batrick/wip-62518-reef

reef: mds: acquire inode snaplock in open

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #53138 from mihalicyn/wip-62570-reef
Yuri Weinstein [Wed, 18 Oct 2023 14:51:19 +0000 (07:51 -0700)]
Merge pull request #53138 from mihalicyn/wip-62570-reef

reef: ceph_fs.h: add separate owner_{u,g}id fields

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agoMerge pull request #52975 from batrick/wip-62422-reef
Yuri Weinstein [Wed, 18 Oct 2023 14:50:01 +0000 (07:50 -0700)]
Merge pull request #52975 from batrick/wip-62422-reef

reef: mds: dump locks when printing mutation ops

Reviewed-by: Xiubo Li <xiubli@redhat.com>
21 months agodoc/rados: edit troubleshooting-pg.rst (1 of x) 54073/head
Zac Dover [Tue, 17 Oct 2023 09:13:01 +0000 (19:13 +1000)]
doc/rados: edit troubleshooting-pg.rst (1 of x)

Edit doc/rados/troubleshooting/troubleshooting-pg.rst (1 of x).

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit fbde0a441ee753c06afe3723584d6c9686ceef21)

21 months agodoc/start: Modernize and clarify hardware-recommendations.rst 54071/head
Anthony D'Atri [Mon, 16 Oct 2023 01:27:44 +0000 (21:27 -0400)]
doc/start: Modernize and clarify hardware-recommendations.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 83bd3a8dfb83c46a2ba84d9f309a6dbe72c64ea9)