]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Matan Breizman [Thu, 19 Oct 2023 12:28:26 +0000 (15:28 +0300)]
Merge pull request #53957 from Matan-B/wip-52529-reef
reef: crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 08:34:38 +0000 (11:34 +0300)]
Merge pull request #54021 from Matan-B/wip-48052-reef
reef: qa/suites/crimson-rados: bring backfill testing
Reviewed-by: Samuel Just <sjust@redhat.com>
Xuehan Xu [Wed, 19 Jul 2023 07:52:22 +0000 (15:52 +0800)]
crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
with ReplicateRequest
This commit is a follow-up for
e14142894922b403344d7f08315b67a291b9a27a
and
f4d336a2dbf864fe798ca7f500d44576d3f626d5
Fixes: https://tracker.ceph.com/issues/61943
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
0b7c47db78cc22c7cf25c50b430674e17b01a601 )
Matan Breizman [Thu, 19 Oct 2023 07:11:10 +0000 (10:11 +0300)]
Merge pull request #54020 from Matan-B/wip-52784-reef
reef: qa/suites/crimson-rados: add centos9 to supported distros
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:10:39 +0000 (10:10 +0300)]
Merge pull request #54019 from Matan-B/wip-52598-reef
reef: qa/suites/crimson-rados: Use centos8 for testing
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:09:26 +0000 (10:09 +0300)]
Merge pull request #54063 from Matan-B/wip-52592-reef
reef: crimson/net: set TCP_NODELAY according to ms_tcp_nodelay
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:07:46 +0000 (10:07 +0300)]
Merge pull request #54062 from Matan-B/wip-52420-reef
reef: crimson/os/seastore/zbd: zbdsegmentmanager write path fixes.
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:07:02 +0000 (10:07 +0300)]
Merge pull request #54061 from Matan-B/wip-52390-reef
reef: crimson/os/object_data_handler: splitting right side doesn't mean splitting only one extent
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:06:34 +0000 (10:06 +0300)]
Merge pull request #54060 from Matan-B/wip-52223-reef
reef: crimson/osd/osd_operations: consistent naming to pipeline users
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:05:40 +0000 (10:05 +0300)]
Merge pull request #54059 from Matan-B/wip-52091-reef
reef: crimson/tools/perf_crimson_msgr: integrate multi-core msgr with various improvements
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 19 Oct 2023 07:03:09 +0000 (10:03 +0300)]
Merge pull request #54058 from Matan-B/wip-51147-reef
reef: Crimson/osd: support multicore osd
Reviewed-by: Samuel Just <sjust@redhat.com>
Nizamudeen A [Thu, 19 Oct 2023 04:45:24 +0000 (10:15 +0530)]
Merge pull request #54069 from rhcs-dashboard/wip-63231-reef
reef: mgr/dashboard: disable hosts field while editing the filesystem
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
Zack Cerza [Wed, 18 Oct 2023 20:57:35 +0000 (14:57 -0600)]
Merge pull request #53921 from ljflores/wip-63168-reef
reef: qa/suites/upgrade: fix env indentation in stress-split upgrade tests
Anthony D'Atri [Wed, 18 Oct 2023 20:04:55 +0000 (16:04 -0400)]
Merge pull request #54079 from zdover23/wip-doc-2023-10-19-backport-54051-to-reef
reef: doc: Update mClock QOS documentation to discard osd_mclock_cost_per_*
tanchangzhi [Tue, 17 Oct 2023 08:48:51 +0000 (16:48 +0800)]
doc: Update mClock QOS documentation to discard osd_mclock_cost_per_*
The cost parameters (osd_mclock_cost_per_*) have been removed.
The cost of an operation is now determined using the random IOPS
and maximum sequential bandwidth capability of the OSD's underlying device.
Fixes: https://tracker.ceph.com/issues/58529
Signed-off-by: tanchangzhi <544463199@qq.com>
(cherry picked from commit
a7502cb5125dda8d052dc3991ed5d7013606aca2 )
Yuri Weinstein [Wed, 18 Oct 2023 18:50:17 +0000 (11:50 -0700)]
Merge pull request #53654 from mcv21/wip-62970-reef
reef: Debian: update to dh compat 12, fix more serious packaging errors, correct copyright syntax
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Casey Bodley [Wed, 18 Oct 2023 17:52:36 +0000 (18:52 +0100)]
Merge pull request #53823 from cfsnyder/wip-62944-reef
reef: rgw: add radosgw-admin bucket check olh/unlinked commands
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Anthony D'Atri [Wed, 18 Oct 2023 16:49:42 +0000 (12:49 -0400)]
Merge pull request #54071 from zdover23/wip-doc-2023-10-18-backport-54028-to-reef
reef: doc/start: Modernize and clarify hardware-recommendations.rst
Anthony D'Atri [Wed, 18 Oct 2023 16:49:00 +0000 (12:49 -0400)]
Merge pull request #54073 from zdover23/wip-doc-2023-10-18-backport-54052-to-reef
reef: doc/rados: edit troubleshooting-pg.rst (1 of x)
Yuri Weinstein [Wed, 18 Oct 2023 16:22:31 +0000 (09:22 -0700)]
Merge pull request #53229 from ifed01/wip-ifed-snapdiff-ref
reef: cephfs: implement snapdiff
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 16:21:22 +0000 (09:21 -0700)]
Merge pull request #52850 from lxbsz/wip-61165
reef: client: issue a cap release immediately if no cap exists
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 16:20:35 +0000 (09:20 -0700)]
Merge pull request #52847 from lxbsz/wip-62192
reef: mds: do not send split_realms for CEPH_SNAP_OP_UPDATE msg
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Cory Snyder [Mon, 25 Sep 2023 10:06:41 +0000 (10:06 +0000)]
rgw: fix output formatting of bucket index check admin api
The bucket index check admin API was previously returning invalid
JSON.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
32fb6a1a68398a99324b2e64ebe3bcf3a9ccf02a )
Conflicts:
src/rgw/driver/rados/rgw_bucket.cc
Cherry-pick notes:
- Conflicts due to check_bad_index_multipart taking optional_yield param on main
Cory Snyder [Fri, 22 Sep 2023 21:08:25 +0000 (21:08 +0000)]
rgw: fix radosgw-admin bucket check stat calculation bug
Fixes a regression with radosgw-admin bucket check stat
calculation and bucket reshard stat calculation when
there are objects that have transitioned from unversioned
to versioned. The bug was introduced in
152aadb71b61c53a4832a1c8cf82fce3d64b68d1 .
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
4728daa5557bfb79a608dd903b8630e2b15fcb2c )
Cory Snyder [Fri, 22 Sep 2023 21:00:46 +0000 (21:00 +0000)]
rgw: add test case to reproduce bucket check stats bug for versioned bucket
Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
340522f9aed50d65137568c1f9dcf4b1e7945a79 )
Cory Snyder [Fri, 22 Sep 2023 08:35:16 +0000 (08:35 +0000)]
rgw: radosgw-admin bucket check should only print index entries with --check-objects flag
Printing all index entries can be very time consuming for large
buckets and the inability to switch this behavior off makes it
cumbersome to use the command for fixing bucket stats. This was
also preventing the command from outputting recalculated bucket
stats when the --fix flag wasn't specified.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
6b057fe55413c0eaf9959f006584cba6cc4c192a )
Cory Snyder [Thu, 21 Sep 2023 19:27:51 +0000 (19:27 +0000)]
rgw: prevent another leftover bucket index olh entry scenario
If a call to bucket_index_link_olh or bucket_index_unlink_instance
fails, its associated pending xattr may have prevented the olh object
from being removed by another thread. We should do a best effort
cleanup attempt for this case by calling update_olh before returning
an error to the caller.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
570adec5bb8142f5baf1f05f0040e8afdb11ec05 )
Cory Snyder [Thu, 7 Sep 2023 17:23:14 +0000 (17:23 +0000)]
rgw: fix rgw versioned bucket stat accounting during reshard and check index
Fixes: https://tracker.ceph.com/issues/62760
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
152aadb71b61c53a4832a1c8cf82fce3d64b68d1 )
Cory Snyder [Thu, 7 Sep 2023 14:43:23 +0000 (14:43 +0000)]
qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
b79dcf640ac2cc3dacf1b87bbe351db823c445d0 )
Cory Snyder [Thu, 25 May 2023 16:59:04 +0000 (16:59 +0000)]
rgw: add radosgw-admin bucket check olh/unlinked commands
Adds commands to radosgw-admin for checking for and fixing leftover
entries in the bucket index (and associated RADOS objects).
Fixes: https://tracker.ceph.com/issues/62075
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
9b2042a0750109f575b02c9c4b127dd941994662 )
Conflicts:
src/rgw/driver/rados/rgw_rados.cc
Cherry-pick notes:
- Conflicts due to multiple methods takeing optional_yield param on main
Yuri Weinstein [Wed, 18 Oct 2023 14:55:26 +0000 (07:55 -0700)]
Merge pull request #53329 from vshankar/tr-62583
reef: mds: blocklist clients with "bloated" session metadata
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:54:48 +0000 (07:54 -0700)]
Merge pull request #53271 from mchangir/wip-59408-reef
reef: cephfs_mirror: correctly set top level dir permissions
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:54:19 +0000 (07:54 -0700)]
Merge pull request #53241 from batrick/wip-62661-reef
reef: mds: drop locks and retry when lock set changes
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:52:36 +0000 (07:52 -0700)]
Merge pull request #53187 from batrick/wip-62609-reef
reef: mgr: register OSDs in ms_handle_accept
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:51:41 +0000 (07:51 -0700)]
Merge pull request #53183 from batrick/wip-62518-reef
reef: mds: acquire inode snaplock in open
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:51:19 +0000 (07:51 -0700)]
Merge pull request #53138 from mihalicyn/wip-62570-reef
reef: ceph_fs.h: add separate owner_{u,g}id fields
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:50:01 +0000 (07:50 -0700)]
Merge pull request #52975 from batrick/wip-62422-reef
reef: mds: dump locks when printing mutation ops
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Zac Dover [Tue, 17 Oct 2023 09:13:01 +0000 (19:13 +1000)]
doc/rados: edit troubleshooting-pg.rst (1 of x)
Edit doc/rados/troubleshooting/troubleshooting-pg.rst (1 of x).
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
fbde0a441ee753c06afe3723584d6c9686ceef21 )
Anthony D'Atri [Mon, 16 Oct 2023 01:27:44 +0000 (21:27 -0400)]
doc/start: Modernize and clarify hardware-recommendations.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit
83bd3a8dfb83c46a2ba84d9f309a6dbe72c64ea9 )
Matan Breizman [Wed, 18 Oct 2023 07:04:47 +0000 (10:04 +0300)]
Merge pull request #53954 from Matan-B/wip-52266-reef
reef: crimson/mgr: Fix config show command
Reviewed-by: Samuel Just <sjust@redhat.com>
Nizamudeen A [Fri, 13 Oct 2023 07:47:05 +0000 (13:17 +0530)]
mgr/dashboard: disable hosts field while editing the filesystem
Even though the Placement field was disabled, the Host field was still
showing up in UI while Editing. That option is not possible in fs form.
Fixes: https://tracker.ceph.com/issues/63193
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit
b18a1441bebdb5c997ed3970d0a53dd39cebd277 )
Xuehan Xu [Sun, 23 Jul 2023 09:33:16 +0000 (17:33 +0800)]
crimson/net: set TCP_NODELAY according to ms_tcp_nodelay
There are cases in which we need low latency of an individual op, like
updating RBD image's object map which would block other ops for a
object not created yet. We need to make sure the rbd-object-map-update
like ops have as low latency as possible
Fixes: https://tracker.ceph.com/issues/62098
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
e21cfa770570f7e7410b830b11edee17e80dead2 )
Aravind Ramesh [Tue, 11 Jul 2023 04:09:36 +0000 (06:09 +0200)]
crimson/os/seastore/zbd: Split iovs in case of bigger buffer sizes.
In Seastore's ZBDSegmentManager, during long running write workloads
if write buffer size exceeds max buf size(IOV_MAX, 16MB),
it was resulting in an assert:
ceph/src/include/buffer.h:1189 : In function 'void ceph::buffer::v15_2_0::list::prepare_iov(VectorT*) const [with VectorT = std::vector<iovec>]', ceph_assert(%s)
This case is handled in BlockSegmentManager by splitting the
IO vectors to appropriate sizes and assert does not happen.
Updated the ZBDSegmentManager to similarly split IO vectors to
appropriate size.
Signed-off-by: Aravind Ramesh <Aravind.Ramesh@wdc.com>
(cherry picked from commit
c15c0de4886a839ffebfd22d8204e7d1ca8a65b2 )
Aravind Ramesh [Wed, 5 Jul 2023 16:58:24 +0000 (18:58 +0200)]
crimson/os/seastore/zbd: increase padding byte buffer size.
Before closing a segment, seastore writes a tail information
at the end of the segment, since ZBD zones are serially write
required, if a zone is not fully written to the end, it writes
padding bytes till the tail section.
Increase the size of write chunks to reduce the number of writes.
Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit
181e6394014dcdca3d1d0cea5f35e92545cb84b9 )
Xuehan Xu [Tue, 11 Jul 2023 08:22:59 +0000 (16:22 +0800)]
crimson/os/object_data_handler: splitting right side doesn't mean
splitting only one extent
Fixes: https://tracker.ceph.com/issues/61952
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
735357bcf5c1b765da661e4934d53d78661795a6 )
Matan Breizman [Tue, 27 Jun 2023 07:05:28 +0000 (07:05 +0000)]
crimson/osd/osd_operations/pg_advance_map: peering_request_pg_pipeline getter
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
9f8159d9e017f5e36db0eb440284bc742a1fd5a5 )
Matan Breizman [Wed, 28 Jun 2023 09:07:26 +0000 (09:07 +0000)]
crimson/osd/osd_operations: consistent naming to pipeline users
Rename pp/bp to client_pp or peering_pp.
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
9f3dc8db7195b6a42499b46c7dbea872d30788b9 )
Yingxin Cheng [Mon, 3 Jul 2023 08:19:51 +0000 (16:19 +0800)]
crimson/tools/perf_crimson_msgr: client don't skip core 0 if specified explicitly
Usually, we skip core 0 to avoid unwanted performance impacts, but that
may not be always the case. Add the configuration not to skip core 0.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
a2df1d0942b4f756a78de4608a464948415c92a1 )
Yingxin Cheng [Mon, 26 Jun 2023 08:49:36 +0000 (16:49 +0800)]
crimson/tools/perf_crimson_msgr: don't create client/server if not necessary
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
a07a02d0bc5e65bf9bd38aba3536c394450340aa )
Yingxin Cheng [Fri, 16 Jun 2023 07:34:04 +0000 (15:34 +0800)]
crimson/tools/perf_crimson_msgr: report server-side conns/iops/throughput
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
5c14be46c8d9f2920c6e2e22dee82070e9a7f092 )
Yingxin Cheng [Thu, 15 Jun 2023 02:49:46 +0000 (10:49 +0800)]
crimson/tools/perf_crimson_msgr: use lowres clock in the hot path
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
237e92d3dd076e65b95917e348b55126da6f0bcc )
Yingxin Cheng [Wed, 31 May 2023 07:38:15 +0000 (15:38 +0800)]
crimson/tools/perf_crimson_msgr: integrate multi-core messenger as server
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
b04daeadf5671911f83ccbbea7bf791ced5f7a51 )
Yingxin Cheng [Wed, 31 May 2023 07:31:59 +0000 (15:31 +0800)]
crimson/tools/perf_crimson_msgr: configure clients and client connections
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
9aab9143aeee94363e1bbaab311bc93ffdf45856 )
Yingxin Cheng [Wed, 31 May 2023 07:07:49 +0000 (15:07 +0800)]
crimson/tools/perf_crimson_msgr: change depth to depth-per-job
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
938b82ab4f4abdbdc57f01086edc1d1288ac15fd )
Yingxin Cheng [Thu, 25 May 2023 06:18:58 +0000 (14:18 +0800)]
crimson/tools/perf_crimson_msgr: add server multi core option
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
0d170bd9df853180908de79258f4ad0eeb0bbc6e )
Yingxin Cheng [Thu, 25 May 2023 06:17:38 +0000 (14:17 +0800)]
crimson/tools/perf_crimson_msgr: decrease sampling rate
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
1d4e9678feeb6e02eb5dd8bf7bf715dd1f770884 )
Yingxin Cheng [Thu, 25 May 2023 06:16:16 +0000 (14:16 +0800)]
crimson/tools/perf_crimson_msgr: support stop signal in server mode
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
6d6043fa70c60e0e0d37c12fe3b05439246155f3 )
Yingxin Cheng [Thu, 25 May 2023 06:12:41 +0000 (14:12 +0800)]
crimson/tools/perf_crimson_msgr: collect server reactor utilizations without client
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
430bcaa5e52e68feb0794fad668cb471eec60ccf )
Yingxin Cheng [Thu, 25 May 2023 06:09:26 +0000 (14:09 +0800)]
crimson/tools/perf_crimson_msgr: collect server reactor utilizations with client
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
291cc302a92d215b3a20d3a4efd8e4df0e65b6a5 )
Yingxin Cheng [Thu, 25 May 2023 05:58:46 +0000 (13:58 +0800)]
crimson/tools/perf_crimson_msgr: report client reactor utilizations
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
50a80470afb0d63c623ae0f52ebfa803dcd37451 )
Yingxin Cheng [Thu, 25 May 2023 05:48:12 +0000 (13:48 +0800)]
crimson/tools/perf_crimson_msgr: report aggregated client summary
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
c3f08a757fca12d89e5825b014589f13a8e85a2d )
Yingxin Cheng [Thu, 25 May 2023 05:42:26 +0000 (13:42 +0800)]
crimson/tools/perf_crimson_msgr: rearrange client cores
Move client cores to the largest smp shards, and reserve core-0 to
reduce performance impacts.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
18bc828f3ec9be843e48eae2630ebe9be3c924a0 )
Yingxin Cheng [Thu, 25 May 2023 04:11:47 +0000 (12:11 +0800)]
crimson/tools/perf_crimson_msgr: misc cleanups
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
5371e9bcef10059c1887d7ef3dace25bf0c32552 )
Yingxin Cheng [Thu, 25 May 2023 03:53:43 +0000 (11:53 +0800)]
crimson/tools/perf_async_msgr: print configurations
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
45586840ea56c32e6d4b7c55305b6500fae72415 )
Yingxin Cheng [Thu, 25 May 2023 03:52:05 +0000 (11:52 +0800)]
crimson/tools/perf_async_msgr: support configure ms_async_op_threads
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
8580dcbf4ed33468778bcad1b71308a6e8d26c88 )
chunmei [Fri, 23 Jun 2023 04:35:38 +0000 (04:35 +0000)]
crimson/osd: modify the interface to integrate with milticore-msg
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
00a5eaea4661e6fb23d174b2f367222541b24f21 )
chunmei [Fri, 23 Jun 2023 04:25:53 +0000 (04:25 +0000)]
crimson/osd: remove seastar::now() to gurantee sequence in pipline
in debug build, if wait_barrier is seastar::now, will cause reactor
schedule the following continuation away, but can't schedule them back
in the same order as by which they are scheduled away, so the continuations
are out of order here.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
c7628de4d230cd08d536d7e0c35bf6822cc7146e )
chunmei [Fri, 23 Jun 2023 03:35:09 +0000 (03:35 +0000)]
crimson/osd: make OSDState and OSDMap_Gate sharded
put them into PerShardState to avoid cross core calling for normal cases.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
324191130ab77a06777f3a71b4c6bf995a4e5fe7 )
chunmei [Sat, 8 Apr 2023 08:58:15 +0000 (08:58 +0000)]
crimson/osd: change OSD::ms_dispatch to take an ConnectionFRef
Shortly, we will need to be able to pass a ConnectionRef from another core.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
d657afc7a76d14882152d6f38da5adeaa39e55e1 )
chunmei [Fri, 31 Mar 2023 04:11:27 +0000 (04:11 +0000)]
crimson/osd: relax requirement that PGShardManager methods be invoked on PRIMARY_CORE
Proxy uses of osd_singleton_state to PRIMARY_CORE as needed.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
6bb4f00a9d006f99b21104bf46b78dff269a116f )
chunmei [Fri, 31 Mar 2023 02:49:33 +0000 (02:49 +0000)]
crimson/osd: enable PGShardMapping access from all cores
Previously, all accesses (incuding loookups) had to occur on core 0. Now that
we want to be able to dispatch from all cores, we need PGShardManager to be
accessible from all cores. To that end, we now proxy updates to core 0, but maintain
local copies of the map so that cores can perform local lookups.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
a3052969bfcfd9329b2baf7883765642d09ff038 )
chunmei [Fri, 23 Jun 2023 03:18:15 +0000 (03:18 +0000)]
crimson/osd: make osd sharded
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
5f82fbbc72f840fa8642015a120485648aaf507c )
Matan Breizman [Tue, 17 Oct 2023 16:00:33 +0000 (19:00 +0300)]
Merge pull request #53956 from Matan-B/wip-52221-reef
reef: crimson/os/seastore/collection_manager: fill CollectionNode::decoded on clean reads
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 15:59:03 +0000 (18:59 +0300)]
Merge pull request #53942 from Matan-B/wip-51034-reef
reef: crimson/osd: fix ENOENT on accessing RadosGW user's index of buckets
Reviewed-by: Samuel Just <sjust@redhat.com>
Leonid S. Usov [Tue, 17 Oct 2023 13:44:00 +0000 (16:44 +0300)]
Merge pull request #53168 from leonid-s-usov/bp/cap-throttle-event/reef
reef: mds/Server: mark a cap acquisition throttle event in the request
Leonid S. Usov [Tue, 17 Oct 2023 13:43:42 +0000 (16:43 +0300)]
Merge pull request #52899 from leonid-s-usov/backport/bulk-data-pool/reef
reef: Consider setting "bulk" autoscale pool flag when automatically creating a data pool for CephFS
Matan Breizman [Tue, 17 Oct 2023 13:10:42 +0000 (16:10 +0300)]
Merge pull request #53969 from Matan-B/wip-53432-reef
reef: crimson/vstart: default seastore_device_size will be out of space f…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:10:06 +0000 (16:10 +0300)]
Merge pull request #53968 from Matan-B/wip-53230-reef
reef: crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:09:49 +0000 (16:09 +0300)]
Merge pull request #53967 from Matan-B/wip-53232-reef
reef: test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:09:16 +0000 (16:09 +0300)]
Merge pull request #53965 from Matan-B/wip-52204-reef
reef: crimson/osd/object_context: consider clones found as long as they're in SnapSet::clones
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:52 +0000 (16:08 +0300)]
Merge pull request #53964 from Matan-B/wip-53082-reef
reef: crimson/admin/admin_socket: remove path file if it exists
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:34 +0000 (16:08 +0300)]
Merge pull request #53963 from Matan-B/wip-53031-reef
reef: crimson/osd/pg: check if backfill_state exists when judging objects'
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:10 +0000 (16:08 +0300)]
Merge pull request #53962 from Matan-B/wip-53051-reef
reef: crimson/os/seastore/omap_manager: fix the entry leak issue in BtreeOMapManager::omap_list()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:07:43 +0000 (16:07 +0300)]
Merge pull request #53961 from Matan-B/wip-52128-reef
reef: crimson/os/seastore/journal/cbj: generalize scan_valid_records()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:07:14 +0000 (16:07 +0300)]
Merge pull request #53960 from Matan-B/wip-53023-reef
reef: crimson/os/seastore: fix daggling reference of oid in SeaStore::Shard::stat()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:06:51 +0000 (16:06 +0300)]
Merge pull request #53959 from Matan-B/wip-52757-reef
reef: crimson/os/seastore: retire_extent_addr clean up
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:05:48 +0000 (16:05 +0300)]
Merge pull request #53958 from Matan-B/wip-52655-reef
reef: crimson/ertr: assert on invocability of func provided to safe_then()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:05:01 +0000 (16:05 +0300)]
Merge pull request #53955 from Matan-B/wip-52446-reef
reef: crimson/os/seastore/transaction_manager: data loss issues
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:04:31 +0000 (16:04 +0300)]
Merge pull request #53951 from Matan-B/wip-51179-reef
reef: crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:04:05 +0000 (16:04 +0300)]
Merge pull request #53949 from Matan-B/wip-51916-reef
reef: crimson/net: support connections in multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:46:25 +0000 (15:46 +0300)]
Merge pull request #53947 from Matan-B/wip-52217-reef
reef: crimson/os/seastore/omap_manager: correct editor settings
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:46:08 +0000 (15:46 +0300)]
Merge pull request #53946 from Matan-B/wip-52090-reef
reef: crimson/os/seastore/btree: should add left's size when merging levels…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:45:48 +0000 (15:45 +0300)]
Merge pull request #53937 from Matan-B/wip-50598-reef
reef: crimson/osd: add embedded suppression ruleset for LSan
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:38:25 +0000 (14:38 +0300)]
Merge pull request #53953 from Matan-B/wip-52369-reef
reef: crimson/os/seastore: add metrics about total invalidated transactions
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:37:59 +0000 (14:37 +0300)]
Merge pull request #53952 from Matan-B/wip-51770-reef
reef: crimson/os/seastore/rbm: make rbm support multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:36:49 +0000 (14:36 +0300)]
Merge pull request #53950 from Matan-B/wip-51852-reef
reef: crimson/osd/watch: Add logs around Watch/Notify
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:34:49 +0000 (14:34 +0300)]
Merge pull request #53948 from Matan-B/wip-52032-reef
reef: crimson/osd: don't ignore start_pg_operation returned future
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:29:58 +0000 (14:29 +0300)]
Merge pull request #53945 from Matan-B/wip-51980-reef
reef: crimson/os/seastore: fix in check_node
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:28:09 +0000 (14:28 +0300)]
Merge pull request #53944 from Matan-B/wip-51961-reef
reef: Crimson/osd: Disable concurrent MOSDMap handling
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:27:31 +0000 (14:27 +0300)]
Merge pull request #53943 from Matan-B/wip-51945-reef
reef: crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack
Reviewed-by: Samuel Just <sjust@redhat.com>