]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
chunmei [Fri, 23 Jun 2023 04:35:38 +0000 (04:35 +0000)]
crimson/osd: modify the interface to integrate with milticore-msg
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
00a5eaea4661e6fb23d174b2f367222541b24f21 )
chunmei [Fri, 23 Jun 2023 04:25:53 +0000 (04:25 +0000)]
crimson/osd: remove seastar::now() to gurantee sequence in pipline
in debug build, if wait_barrier is seastar::now, will cause reactor
schedule the following continuation away, but can't schedule them back
in the same order as by which they are scheduled away, so the continuations
are out of order here.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
c7628de4d230cd08d536d7e0c35bf6822cc7146e )
chunmei [Fri, 23 Jun 2023 03:35:09 +0000 (03:35 +0000)]
crimson/osd: make OSDState and OSDMap_Gate sharded
put them into PerShardState to avoid cross core calling for normal cases.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
324191130ab77a06777f3a71b4c6bf995a4e5fe7 )
chunmei [Sat, 8 Apr 2023 08:58:15 +0000 (08:58 +0000)]
crimson/osd: change OSD::ms_dispatch to take an ConnectionFRef
Shortly, we will need to be able to pass a ConnectionRef from another core.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
d657afc7a76d14882152d6f38da5adeaa39e55e1 )
chunmei [Fri, 31 Mar 2023 04:11:27 +0000 (04:11 +0000)]
crimson/osd: relax requirement that PGShardManager methods be invoked on PRIMARY_CORE
Proxy uses of osd_singleton_state to PRIMARY_CORE as needed.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
6bb4f00a9d006f99b21104bf46b78dff269a116f )
chunmei [Fri, 31 Mar 2023 02:49:33 +0000 (02:49 +0000)]
crimson/osd: enable PGShardMapping access from all cores
Previously, all accesses (incuding loookups) had to occur on core 0. Now that
we want to be able to dispatch from all cores, we need PGShardManager to be
accessible from all cores. To that end, we now proxy updates to core 0, but maintain
local copies of the map so that cores can perform local lookups.
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
a3052969bfcfd9329b2baf7883765642d09ff038 )
chunmei [Fri, 23 Jun 2023 03:18:15 +0000 (03:18 +0000)]
crimson/osd: make osd sharded
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
5f82fbbc72f840fa8642015a120485648aaf507c )
Matan Breizman [Tue, 17 Oct 2023 16:00:33 +0000 (19:00 +0300)]
Merge pull request #53956 from Matan-B/wip-52221-reef
reef: crimson/os/seastore/collection_manager: fill CollectionNode::decoded on clean reads
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 15:59:03 +0000 (18:59 +0300)]
Merge pull request #53942 from Matan-B/wip-51034-reef
reef: crimson/osd: fix ENOENT on accessing RadosGW user's index of buckets
Reviewed-by: Samuel Just <sjust@redhat.com>
Leonid S. Usov [Tue, 17 Oct 2023 13:44:00 +0000 (16:44 +0300)]
Merge pull request #53168 from leonid-s-usov/bp/cap-throttle-event/reef
reef: mds/Server: mark a cap acquisition throttle event in the request
Leonid S. Usov [Tue, 17 Oct 2023 13:43:42 +0000 (16:43 +0300)]
Merge pull request #52899 from leonid-s-usov/backport/bulk-data-pool/reef
reef: Consider setting "bulk" autoscale pool flag when automatically creating a data pool for CephFS
Matan Breizman [Tue, 17 Oct 2023 13:10:42 +0000 (16:10 +0300)]
Merge pull request #53969 from Matan-B/wip-53432-reef
reef: crimson/vstart: default seastore_device_size will be out of space f…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:10:06 +0000 (16:10 +0300)]
Merge pull request #53968 from Matan-B/wip-53230-reef
reef: crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:09:49 +0000 (16:09 +0300)]
Merge pull request #53967 from Matan-B/wip-53232-reef
reef: test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:09:16 +0000 (16:09 +0300)]
Merge pull request #53965 from Matan-B/wip-52204-reef
reef: crimson/osd/object_context: consider clones found as long as they're in SnapSet::clones
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:52 +0000 (16:08 +0300)]
Merge pull request #53964 from Matan-B/wip-53082-reef
reef: crimson/admin/admin_socket: remove path file if it exists
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:34 +0000 (16:08 +0300)]
Merge pull request #53963 from Matan-B/wip-53031-reef
reef: crimson/osd/pg: check if backfill_state exists when judging objects'
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:10 +0000 (16:08 +0300)]
Merge pull request #53962 from Matan-B/wip-53051-reef
reef: crimson/os/seastore/omap_manager: fix the entry leak issue in BtreeOMapManager::omap_list()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:07:43 +0000 (16:07 +0300)]
Merge pull request #53961 from Matan-B/wip-52128-reef
reef: crimson/os/seastore/journal/cbj: generalize scan_valid_records()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:07:14 +0000 (16:07 +0300)]
Merge pull request #53960 from Matan-B/wip-53023-reef
reef: crimson/os/seastore: fix daggling reference of oid in SeaStore::Shard::stat()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:06:51 +0000 (16:06 +0300)]
Merge pull request #53959 from Matan-B/wip-52757-reef
reef: crimson/os/seastore: retire_extent_addr clean up
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:05:48 +0000 (16:05 +0300)]
Merge pull request #53958 from Matan-B/wip-52655-reef
reef: crimson/ertr: assert on invocability of func provided to safe_then()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:05:01 +0000 (16:05 +0300)]
Merge pull request #53955 from Matan-B/wip-52446-reef
reef: crimson/os/seastore/transaction_manager: data loss issues
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:04:31 +0000 (16:04 +0300)]
Merge pull request #53951 from Matan-B/wip-51179-reef
reef: crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:04:05 +0000 (16:04 +0300)]
Merge pull request #53949 from Matan-B/wip-51916-reef
reef: crimson/net: support connections in multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:46:25 +0000 (15:46 +0300)]
Merge pull request #53947 from Matan-B/wip-52217-reef
reef: crimson/os/seastore/omap_manager: correct editor settings
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:46:08 +0000 (15:46 +0300)]
Merge pull request #53946 from Matan-B/wip-52090-reef
reef: crimson/os/seastore/btree: should add left's size when merging levels…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:45:48 +0000 (15:45 +0300)]
Merge pull request #53937 from Matan-B/wip-50598-reef
reef: crimson/osd: add embedded suppression ruleset for LSan
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:38:25 +0000 (14:38 +0300)]
Merge pull request #53953 from Matan-B/wip-52369-reef
reef: crimson/os/seastore: add metrics about total invalidated transactions
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:37:59 +0000 (14:37 +0300)]
Merge pull request #53952 from Matan-B/wip-51770-reef
reef: crimson/os/seastore/rbm: make rbm support multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:36:49 +0000 (14:36 +0300)]
Merge pull request #53950 from Matan-B/wip-51852-reef
reef: crimson/osd/watch: Add logs around Watch/Notify
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:34:49 +0000 (14:34 +0300)]
Merge pull request #53948 from Matan-B/wip-52032-reef
reef: crimson/osd: don't ignore start_pg_operation returned future
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:29:58 +0000 (14:29 +0300)]
Merge pull request #53945 from Matan-B/wip-51980-reef
reef: crimson/os/seastore: fix in check_node
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:28:09 +0000 (14:28 +0300)]
Merge pull request #53944 from Matan-B/wip-51961-reef
reef: Crimson/osd: Disable concurrent MOSDMap handling
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:27:31 +0000 (14:27 +0300)]
Merge pull request #53943 from Matan-B/wip-51945-reef
reef: crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:21:04 +0000 (14:21 +0300)]
Merge pull request #53941 from Matan-B/wip-51948-reef
reef: crimson/os/seastore/cached_extent: add prepare_commit interface
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:20:38 +0000 (14:20 +0300)]
Merge pull request #53940 from Matan-B/wip-51380-reef
reef: crimson/osd/heartbeat: Improve maybe_share_osdmap behavior
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:19:53 +0000 (14:19 +0300)]
Merge pull request #53939 from Matan-B/wip-51750-reef
reef: crimson/os/seastore/backref_manager: scan backref entries by journal seq
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:19:10 +0000 (14:19 +0300)]
Merge pull request #53938 from Matan-B/wip-51475-reef
reef: crimson/os/seastore: fix ceph_assert in segment_manager.h
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:15:39 +0000 (14:15 +0300)]
Merge pull request #53935 from Matan-B/wip-51355-reef
reef: crimson/os/seastore: enable SMR HDD
Reviewed-by: Samuel Just <sjust@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:37:47 +0000 (12:37 -0700)]
Merge pull request #53797 from smanjara/ragweed-test-reef
reef: rgw/upgrade: point upgrade suites to ragweed ceph-reef branch
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:37:03 +0000 (12:37 -0700)]
Merge pull request #53773 from cbodley/wip-63057-reef
reef: rgw: improve buffer list utilization in the chunkupload scenario
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:35:12 +0000 (12:35 -0700)]
Merge pull request #53770 from cbodley/wip-63054-reef
reef: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:34:05 +0000 (12:34 -0700)]
Merge pull request #53767 from cbodley/wip-63051-reef
reef: rgw: s3website doesn't prefetch for web_dir() check
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:33:06 +0000 (12:33 -0700)]
Merge pull request #53765 from cbodley/wip-63048-reef
reef: rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:32:15 +0000 (12:32 -0700)]
Merge pull request #53760 from cbodley/wip-63060-reef
reef: rgw: fix unwatch crash at radosgw startup
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:31:29 +0000 (12:31 -0700)]
Merge pull request #53756 from cbodley/wip-63042-reef
reef: [CVE-2023-43040] rgw: Fix bucket validation against POST policies
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:30:28 +0000 (12:30 -0700)]
Merge pull request #53681 from mkogan1/wip-t62141-reef
reef: valgrind: UninitCondition under __run_exit_handlers suppression
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Anthony D'Atri [Sun, 15 Oct 2023 23:54:09 +0000 (19:54 -0400)]
Merge pull request #54026 from zdover23/wip-doc-2023-10-16-backport-54022-to-reef
reef: doc/rados: Edit troubleshooting-osd (3 of x)
Zac Dover [Sun, 15 Oct 2023 12:43:39 +0000 (22:43 +1000)]
doc/rados: Edit troubleshooting-osd (3 of x)
Edit doc/rados/troubleshooting/troubleshooting-osd.rst (3 of x).
Follows https://github.com/ceph/ceph/pull/53986.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
01b5aa5d27a97a02cef03b9eb0ee9dbcd55684f1 )
Anthony D'Atri [Fri, 13 Oct 2023 20:31:28 +0000 (16:31 -0400)]
Merge pull request #54008 from zdover23/wip-doc-2023-10-14-backport-54006-to-reef
doc/rados: remove FileStore material (in Reef)
Zac Dover [Fri, 13 Oct 2023 18:40:26 +0000 (04:40 +1000)]
doc/rados: remove FileStore material (in Reef)
Remove FileStore-related material because FileStore is not supported in
Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
eab0be5ca513ca2dfaeb31911c175e88bcf4f9e4 )
Anthony D'Atri [Fri, 13 Oct 2023 20:15:24 +0000 (16:15 -0400)]
Merge pull request #54007 from zdover23/wip-doc-2023-10-14-backport-54005-to-reef-2
reef: doc/dev: remove cache-pool
Zac Dover [Fri, 13 Oct 2023 18:17:44 +0000 (04:17 +1000)]
doc/dev: remove cache-pool
Remove cache-pool, which is deprecated in Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
073b0919ed9e413a04a7e12b7a0c23b5e058cd52 )
zdover23 [Fri, 13 Oct 2023 14:01:03 +0000 (00:01 +1000)]
Merge pull request #54000 from zdover23/wip-doc-2023-10-13-backport-53986-to-reef
reef: doc/rados: Edit troubleshooting-osd (2 of x)
Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Zac Dover [Thu, 12 Oct 2023 08:33:58 +0000 (18:33 +1000)]
doc/rados: Edit troubleshooting-osd (2 of x)
Edit doc/rados/troubleshooting/troubleshooting.rst (2 of x).
Follows https://github.com/ceph/ceph/pull/53936.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
37e7099267996a3075b4902a10a19d94fc738c08 )
Nizamudeen A [Fri, 13 Oct 2023 05:35:08 +0000 (11:05 +0530)]
Merge pull request #53458 from rhcs-dashboard/wip-62602-reef
reef: mgr/dashboard: remove unnecessary failing hosts e2e
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Anthony D'Atri [Thu, 12 Oct 2023 09:02:48 +0000 (05:02 -0400)]
Merge pull request #53982 from zdover23/wip-doc-2023-10-12-backport-53936-to-reef
reef: doc/rados: edit troubleshooting-osd (1 of x)
Zac Dover [Wed, 11 Oct 2023 10:38:55 +0000 (20:38 +1000)]
doc/rados: edit troubleshooting-osd (1 of x)
Edit doc/rados/troubleshooting/troubleshooting-osd.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
0fc663d793fe34e42e19b251fcc79b5af52581c6 )
chunmei [Thu, 14 Sep 2023 06:21:05 +0000 (06:21 +0000)]
crimson/vstart: add --seastore-device-size option in vstart.sh command line
default seastore_device_size will be out of space for smp >28
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
cf69c4f4946884266d6a42c92eef20e74771c3fa )
Myoungwon Oh [Thu, 31 Aug 2023 02:43:24 +0000 (11:43 +0900)]
crimson/os/seastore/cbj: fix a potential overflow bug
CircularBoundedJournal is similar to circular queue, so segment_seq
increases after rolling. However, current implementation always
increases segment_seq_t when rolling occurs, resulting in
the overflow if segment_seq_t hits MAX_SEG_SEQ.
To mitigate this, this commit changes the type of the segment_seq_t
to uint64_t.
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
36982b66574ff277e04a02d5b9a28e80d0790778 )
Myoungwon Oh [Fri, 25 Aug 2023 08:55:36 +0000 (17:55 +0900)]
crimson/os/seastore/object_data_handler: consider a RBM case when checking if write can be merged
RBM's paddr always indicates physical address, which means it doesn't have the dealayed.
So, this commit adds a condition that checks if given paddr is used for ongoing write.
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
73de8937f65706c9c3dd2e36f6504196d0796b01 )
Myoungwon Oh [Thu, 31 Aug 2023 05:13:07 +0000 (14:13 +0900)]
test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
670a19f27761c937ad8d07abf9bb7c94ed4a3b00 )
Xuehan Xu [Sun, 6 Aug 2023 07:45:24 +0000 (15:45 +0800)]
crimson/osd/osd_operations/snaptrim_event: don't process snaptrim events
of the same PG concurrently
Concurrently triming snaps of the same PG may lead to the following
problem:
1. fiber 1 get objects to trim for PG 1.x and snapid 1.d, which involves
object OBJ;
2. fiber 2 get objects to trim for PG 1.x and snapid 1.c, which also
involes object OBJ;
3. fiber 1 removes snap 1.c since it has been removed in the osdmap
4. fiber 2 try to get obc for object OBJ and find that OBJ doesn't have
snap for id 1.c
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
267babe435a54130eda5770a537420fb516e2a8f )
Xuehan Xu [Tue, 18 Jul 2023 06:14:50 +0000 (14:14 +0800)]
crimson/osd/object_context: consider clones found as long as they're in
SnapSet::clones
Fixes: https://tracker.ceph.com/issues/61825
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
957fd43e4b09a7607d4db52fe9674be905f76de8 )
Xuehan Xu [Tue, 18 Jul 2023 06:12:59 +0000 (14:12 +0800)]
crimson/osd/object_context_loader: rename with_head_and_clone_obc to
with_clone_obc_direct
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
ec646eb22bfd30f6cf19c86fe24c233001272047 )
Matan Breizman [Wed, 23 Aug 2023 08:10:32 +0000 (08:10 +0000)]
crimson/admin/admin_socket: remove path file if it exists
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
374de2a7622bf3fc75eceb24e39cf36a27ed8d1b )
Xuehan Xu [Thu, 17 Aug 2023 03:09:05 +0000 (11:09 +0800)]
crimson/osd/pg: check if backfill_state exists when judging objects'
degradation
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
04764838b5395a4e4d0c872f2d2a433fc4aec074 )
Xuehan Xu [Fri, 18 Aug 2023 11:56:50 +0000 (19:56 +0800)]
crimson/os/seastore/omap_manager: fix the entry leak issue in
BtreeOMapManager::omap_list()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
cf105932a7fadf67ec494587b078bbfb4caf4d26 )
myoungwon oh [Wed, 16 Aug 2023 11:58:49 +0000 (11:58 +0000)]
crimson/os/seastore/journal/cbj: remove return_record and read_record
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
39ea5add3a81c32263533ec8f01590efeceb2c86 )
myoungwon oh [Mon, 14 Aug 2023 08:10:04 +0000 (08:10 +0000)]
crimson/os/seastore/journal/cbj: check if batched records are written at once before rolling
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
ef1a6a90e0131cdf3bb96ace835f845ae23ba0ce )
myoungwon oh [Fri, 11 Aug 2023 14:47:21 +0000 (14:47 +0000)]
crimson/os/seastore: genralize read_validate_record_metadata and read_validate_data
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
15ae64c3311378f6055d312c2b5f64b46912d899 )
myoungwon oh [Fri, 11 Aug 2023 04:52:23 +0000 (04:52 +0000)]
crimson/os/seastore/journal/cbj: check if the first record in CBJ is written after rolling or not
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
a10f2855da3bc1c25f664158af1cc79d70bdef57 )
myoungwon oh [Sat, 22 Jul 2023 14:52:14 +0000 (14:52 +0000)]
crimson/os/seastore/journal/cbj: introduce magic value to identify that written records are correct
The cbj would replay invalid records if the same write pattern (mkfs -> write -> mkfs -> write) repeats.
On the second mkfs(), current cbj wrongly recognizes the old records are valid beceause they have
correct seq. number at appropriate offset even though the records are expired.
To solve this, this commit introduces random value to identify the journal during mkfs()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
942ea6487f59e52d88050fb8deb8a2e6c0ca83c3 )
myoungwon oh [Sat, 22 Jul 2023 06:03:04 +0000 (06:03 +0000)]
crimson/os/seastore: introduce generalized scan_valid_records for RBM
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
a580e516965ff8ae57dbaab202854f9ebe38ba5f )
myoungwon oh [Thu, 20 Jul 2023 02:50:48 +0000 (02:50 +0000)]
crimson/os/seastore: introduce RecordScanner to generalize scan_valid_records()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
22c747826b398cc0a0d68aab9bba1fd096a2de9d )
Xuehan Xu [Wed, 16 Aug 2023 11:37:59 +0000 (19:37 +0800)]
crimson/os/seastore: fix daggling reference of oid in
SeaStore::Shard::stat()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
0ee9bb21be89bdcfb3e6cfb224c256d6eccd9f38 )
Xinyu Huang [Wed, 2 Aug 2023 02:09:52 +0000 (10:09 +0800)]
crimson/os/seastore: retire_extent_addr clean up
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
b99a1956bdcb879a3e5440e617593bae43fafd62 )
Radosław Zarzyński [Wed, 26 Jul 2023 10:42:45 +0000 (12:42 +0200)]
crimson/ertr: assert on invocability of func provided to safe_then()
The goal is to fail with an easier-to-understand error on failed
`static_assert()` than cryptic template substitution errors when
the function provided to e.g. `safe_then()` doesn't match the type
carried within an errorated future.
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit
83687f2cc86897d60a460575a7ab2acbd2f9170c )
Xuehan Xu [Wed, 28 Jun 2023 07:18:27 +0000 (15:18 +0800)]
crimson/os/seastore/collection_manager: fill CollectionNode::decoded on
clean reads
Otherwise, deltas of collection nodes can't be replayed correctly,
because the data of nodes is not loaded to CollectionNode::decoded
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
5eb473b99c1ea75fd6c3807d3f54eaec1018d383 )
Xuehan Xu [Fri, 14 Jul 2023 03:59:06 +0000 (11:59 +0800)]
crimson/os/seastore/object_data_handler: add ObjectDataHandler(extent_len_t)
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
c1fdb2a8a41c6f0f204c980c4f3af7cfac28723d )
Xuehan Xu [Fri, 14 Jul 2023 03:56:12 +0000 (11:56 +0800)]
crimson/os/seastore/cache: correct log prefix for get_absent_extent()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
5a563d63e754d5163424486dbd0d665165615794 )
Zhang Song [Sun, 9 Jul 2023 04:22:58 +0000 (12:22 +0800)]
crimson/os/seastore: add metrics about total invalidated transactions
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit
6265452c50c2bbd0283a66d03d6a8538c2285293 )
myoungwon oh [Sat, 17 Jun 2023 08:27:17 +0000 (08:27 +0000)]
src/test/crimson/seastore/nvmedevice: update a test case based on rbm multi shard change
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
7a15d484c552a177585954cdb4e2e226a43c1035 )
myoungwon oh [Fri, 26 May 2023 08:29:50 +0000 (08:29 +0000)]
crimson/os/seastore/rbm: make rbm support multiple shards
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
1d6beef975144210ae29626e61aeb284aff30d53 )
Xinyu Huang [Fri, 26 May 2023 11:05:08 +0000 (11:05 +0000)]
test/crimson/seastore: test if lazy read work correctly
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
9d973efe32c089351ec756232d3699b4c9fe1f64 )
Xinyu Huang [Fri, 21 Apr 2023 14:22:16 +0000 (14:22 +0000)]
test/crimson/seastore: test overwrite pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
efd3cdc71056baabba291d7a359dda6dbd9121b1 )
Xinyu Huang [Fri, 21 Apr 2023 14:19:34 +0000 (14:19 +0000)]
test/crimson/seastore: add test for TransactionManager::remap_pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
b5931d2795cfe44b96c6ee9347514b002197bb5b )
Xinyu Huang [Fri, 21 Apr 2023 14:42:00 +0000 (14:42 +0000)]
crimson/os/seastore: check if length exist in mark_space_used and mark_space_free
to avoid misuse of buffer length and data length when there is a no actual data extent
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
dc4a23cef3787099dcc17f99b05f4ec1a63eca9a )
Xinyu Huang [Wed, 19 Apr 2023 01:46:31 +0000 (01:46 +0000)]
crimson/os/seastore: deepcopy the buffer of exist clean extent when duplicate_for_write
Beacuse exist clean extent shares buffer with original clean extent.
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
026066a7e5c4cc1bc04ed91003c553146a0b60ef )
Xinyu Huang [Fri, 7 Apr 2023 00:54:36 +0000 (00:54 +0000)]
crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
5d00ec0d00ddefe5b82ce82d2acfa134d964a883 )
Xinyu Huang [Fri, 21 Apr 2023 15:52:24 +0000 (15:52 +0000)]
crimson/os/seastore/transaction_manager: implement remap_pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
55d33b69b004adc6fe6740872122311ca9372c0c )
Xinyu Huang [Fri, 21 Apr 2023 15:10:49 +0000 (15:10 +0000)]
crimson/os/seastore/transaction_manager: implement alloc_remapped_extent
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
cf6e46e57afc44b46d7855dcb5849724f6ddfcbb )
Xinyu Huang [Sat, 10 Jun 2023 02:16:29 +0000 (10:16 +0800)]
crimson/os/seastore: support no data extent in read path
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
f19de7c77f7d6b783802f6c18e2e9fe106c90243 )
Xinyu Huang [Tue, 21 Feb 2023 06:18:29 +0000 (06:18 +0000)]
crimson/os/seastore: let CachedExtent::ptr optional
separate buffer length and data length, since no buffer extent is allowed
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
7cb44d350b68002123c3c2f97a888cc42260f522 )
Matan Breizman [Wed, 31 May 2023 11:47:45 +0000 (11:47 +0000)]
crimson/osd/watch: don't print bufferlists
The bufferlists clutter the logs and don't infrom of anything useful:
```
DEBUG 2023-05-26 23:36:29,780 [shard 2] osd - send_completion sending notify replies: {notify_reply_t{watcher_gid=4220, watcher_cookie=
139786421930752 , bl=buffer::list(len=0,
)}, notify_reply_t{watcher_gid=4220, watcher_cookie=
139786422032096 , bl=buffer::list(len=0,
)}}
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
3548489f622c50ccaa1d8c351e420873743ed980 )
Matan Breizman [Wed, 31 May 2023 08:45:53 +0000 (08:45 +0000)]
crimson/osd/watch: Add logs around Watch/Notify
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
b1c05f9acecfd2dffb57d63f42fec9da156ff366 )
Yingxin Cheng [Thu, 29 Jun 2023 05:25:28 +0000 (13:25 +0800)]
crimson/osd/heartbeat: relax the order of replacement reset and accept
With the new implementation in messenger, the order of replacement reset
and accept events cannot be determined because they are from different
connections.
Modify the heatbeat logic to tolerate the both cases.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
a8709464232fc5cc28032716f67c754971420e6b )
Yingxin Cheng [Tue, 27 Jun 2023 02:31:11 +0000 (10:31 +0800)]
crimson/net: cleanup, rename is_fixed_cpu
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
2fed9397d5f1a93e01d02b5914931420c4f28c37 )
Yingxin Cheng [Fri, 16 Jun 2023 07:35:22 +0000 (15:35 +0800)]
crimson/net: dispatch the initial acceptance from the socket core
It's meaningless to dispatch the initial acceptance always from the
msgr core.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
1e8a39f359d8481bb3038b461063c87384bf3ee8 )