]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Cory Snyder [Mon, 25 Sep 2023 10:06:41 +0000 (10:06 +0000)]
rgw: fix output formatting of bucket index check admin api
The bucket index check admin API was previously returning invalid
JSON.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
32fb6a1a68398a99324b2e64ebe3bcf3a9ccf02a )
Conflicts:
src/rgw/driver/rados/rgw_bucket.cc
Cherry-pick notes:
- Conflicts due to check_bad_index_multipart taking optional_yield param on main
Cory Snyder [Fri, 22 Sep 2023 21:08:25 +0000 (21:08 +0000)]
rgw: fix radosgw-admin bucket check stat calculation bug
Fixes a regression with radosgw-admin bucket check stat
calculation and bucket reshard stat calculation when
there are objects that have transitioned from unversioned
to versioned. The bug was introduced in
152aadb71b61c53a4832a1c8cf82fce3d64b68d1 .
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
4728daa5557bfb79a608dd903b8630e2b15fcb2c )
Cory Snyder [Fri, 22 Sep 2023 21:00:46 +0000 (21:00 +0000)]
rgw: add test case to reproduce bucket check stats bug for versioned bucket
Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
340522f9aed50d65137568c1f9dcf4b1e7945a79 )
Cory Snyder [Fri, 22 Sep 2023 08:35:16 +0000 (08:35 +0000)]
rgw: radosgw-admin bucket check should only print index entries with --check-objects flag
Printing all index entries can be very time consuming for large
buckets and the inability to switch this behavior off makes it
cumbersome to use the command for fixing bucket stats. This was
also preventing the command from outputting recalculated bucket
stats when the --fix flag wasn't specified.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
6b057fe55413c0eaf9959f006584cba6cc4c192a )
Cory Snyder [Thu, 21 Sep 2023 19:27:51 +0000 (19:27 +0000)]
rgw: prevent another leftover bucket index olh entry scenario
If a call to bucket_index_link_olh or bucket_index_unlink_instance
fails, its associated pending xattr may have prevented the olh object
from being removed by another thread. We should do a best effort
cleanup attempt for this case by calling update_olh before returning
an error to the caller.
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
570adec5bb8142f5baf1f05f0040e8afdb11ec05 )
Cory Snyder [Thu, 7 Sep 2023 17:23:14 +0000 (17:23 +0000)]
rgw: fix rgw versioned bucket stat accounting during reshard and check index
Fixes: https://tracker.ceph.com/issues/62760
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
152aadb71b61c53a4832a1c8cf82fce3d64b68d1 )
Cory Snyder [Thu, 7 Sep 2023 14:43:23 +0000 (14:43 +0000)]
qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
b79dcf640ac2cc3dacf1b87bbe351db823c445d0 )
Cory Snyder [Thu, 25 May 2023 16:59:04 +0000 (16:59 +0000)]
rgw: add radosgw-admin bucket check olh/unlinked commands
Adds commands to radosgw-admin for checking for and fixing leftover
entries in the bucket index (and associated RADOS objects).
Fixes: https://tracker.ceph.com/issues/62075
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit
9b2042a0750109f575b02c9c4b127dd941994662 )
Conflicts:
src/rgw/driver/rados/rgw_rados.cc
Cherry-pick notes:
- Conflicts due to multiple methods takeing optional_yield param on main
Yuri Weinstein [Wed, 18 Oct 2023 14:55:26 +0000 (07:55 -0700)]
Merge pull request #53329 from vshankar/tr-62583
reef: mds: blocklist clients with "bloated" session metadata
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:54:48 +0000 (07:54 -0700)]
Merge pull request #53271 from mchangir/wip-59408-reef
reef: cephfs_mirror: correctly set top level dir permissions
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:54:19 +0000 (07:54 -0700)]
Merge pull request #53241 from batrick/wip-62661-reef
reef: mds: drop locks and retry when lock set changes
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:52:36 +0000 (07:52 -0700)]
Merge pull request #53187 from batrick/wip-62609-reef
reef: mgr: register OSDs in ms_handle_accept
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:51:41 +0000 (07:51 -0700)]
Merge pull request #53183 from batrick/wip-62518-reef
reef: mds: acquire inode snaplock in open
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:51:19 +0000 (07:51 -0700)]
Merge pull request #53138 from mihalicyn/wip-62570-reef
reef: ceph_fs.h: add separate owner_{u,g}id fields
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Yuri Weinstein [Wed, 18 Oct 2023 14:50:01 +0000 (07:50 -0700)]
Merge pull request #52975 from batrick/wip-62422-reef
reef: mds: dump locks when printing mutation ops
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Matan Breizman [Wed, 18 Oct 2023 07:04:47 +0000 (10:04 +0300)]
Merge pull request #53954 from Matan-B/wip-52266-reef
reef: crimson/mgr: Fix config show command
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 16:00:33 +0000 (19:00 +0300)]
Merge pull request #53956 from Matan-B/wip-52221-reef
reef: crimson/os/seastore/collection_manager: fill CollectionNode::decoded on clean reads
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 15:59:03 +0000 (18:59 +0300)]
Merge pull request #53942 from Matan-B/wip-51034-reef
reef: crimson/osd: fix ENOENT on accessing RadosGW user's index of buckets
Reviewed-by: Samuel Just <sjust@redhat.com>
Leonid S. Usov [Tue, 17 Oct 2023 13:44:00 +0000 (16:44 +0300)]
Merge pull request #53168 from leonid-s-usov/bp/cap-throttle-event/reef
reef: mds/Server: mark a cap acquisition throttle event in the request
Leonid S. Usov [Tue, 17 Oct 2023 13:43:42 +0000 (16:43 +0300)]
Merge pull request #52899 from leonid-s-usov/backport/bulk-data-pool/reef
reef: Consider setting "bulk" autoscale pool flag when automatically creating a data pool for CephFS
Matan Breizman [Tue, 17 Oct 2023 13:10:42 +0000 (16:10 +0300)]
Merge pull request #53969 from Matan-B/wip-53432-reef
reef: crimson/vstart: default seastore_device_size will be out of space f…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:10:06 +0000 (16:10 +0300)]
Merge pull request #53968 from Matan-B/wip-53230-reef
reef: crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:09:49 +0000 (16:09 +0300)]
Merge pull request #53967 from Matan-B/wip-53232-reef
reef: test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:09:16 +0000 (16:09 +0300)]
Merge pull request #53965 from Matan-B/wip-52204-reef
reef: crimson/osd/object_context: consider clones found as long as they're in SnapSet::clones
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:52 +0000 (16:08 +0300)]
Merge pull request #53964 from Matan-B/wip-53082-reef
reef: crimson/admin/admin_socket: remove path file if it exists
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:34 +0000 (16:08 +0300)]
Merge pull request #53963 from Matan-B/wip-53031-reef
reef: crimson/osd/pg: check if backfill_state exists when judging objects'
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:08:10 +0000 (16:08 +0300)]
Merge pull request #53962 from Matan-B/wip-53051-reef
reef: crimson/os/seastore/omap_manager: fix the entry leak issue in BtreeOMapManager::omap_list()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:07:43 +0000 (16:07 +0300)]
Merge pull request #53961 from Matan-B/wip-52128-reef
reef: crimson/os/seastore/journal/cbj: generalize scan_valid_records()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:07:14 +0000 (16:07 +0300)]
Merge pull request #53960 from Matan-B/wip-53023-reef
reef: crimson/os/seastore: fix daggling reference of oid in SeaStore::Shard::stat()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:06:51 +0000 (16:06 +0300)]
Merge pull request #53959 from Matan-B/wip-52757-reef
reef: crimson/os/seastore: retire_extent_addr clean up
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:05:48 +0000 (16:05 +0300)]
Merge pull request #53958 from Matan-B/wip-52655-reef
reef: crimson/ertr: assert on invocability of func provided to safe_then()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:05:01 +0000 (16:05 +0300)]
Merge pull request #53955 from Matan-B/wip-52446-reef
reef: crimson/os/seastore/transaction_manager: data loss issues
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:04:31 +0000 (16:04 +0300)]
Merge pull request #53951 from Matan-B/wip-51179-reef
reef: crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 13:04:05 +0000 (16:04 +0300)]
Merge pull request #53949 from Matan-B/wip-51916-reef
reef: crimson/net: support connections in multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:46:25 +0000 (15:46 +0300)]
Merge pull request #53947 from Matan-B/wip-52217-reef
reef: crimson/os/seastore/omap_manager: correct editor settings
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:46:08 +0000 (15:46 +0300)]
Merge pull request #53946 from Matan-B/wip-52090-reef
reef: crimson/os/seastore/btree: should add left's size when merging levels…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:45:48 +0000 (15:45 +0300)]
Merge pull request #53937 from Matan-B/wip-50598-reef
reef: crimson/osd: add embedded suppression ruleset for LSan
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:38:25 +0000 (14:38 +0300)]
Merge pull request #53953 from Matan-B/wip-52369-reef
reef: crimson/os/seastore: add metrics about total invalidated transactions
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:37:59 +0000 (14:37 +0300)]
Merge pull request #53952 from Matan-B/wip-51770-reef
reef: crimson/os/seastore/rbm: make rbm support multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:36:49 +0000 (14:36 +0300)]
Merge pull request #53950 from Matan-B/wip-51852-reef
reef: crimson/osd/watch: Add logs around Watch/Notify
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:34:49 +0000 (14:34 +0300)]
Merge pull request #53948 from Matan-B/wip-52032-reef
reef: crimson/osd: don't ignore start_pg_operation returned future
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:29:58 +0000 (14:29 +0300)]
Merge pull request #53945 from Matan-B/wip-51980-reef
reef: crimson/os/seastore: fix in check_node
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:28:09 +0000 (14:28 +0300)]
Merge pull request #53944 from Matan-B/wip-51961-reef
reef: Crimson/osd: Disable concurrent MOSDMap handling
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:27:31 +0000 (14:27 +0300)]
Merge pull request #53943 from Matan-B/wip-51945-reef
reef: crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:21:04 +0000 (14:21 +0300)]
Merge pull request #53941 from Matan-B/wip-51948-reef
reef: crimson/os/seastore/cached_extent: add prepare_commit interface
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:20:38 +0000 (14:20 +0300)]
Merge pull request #53940 from Matan-B/wip-51380-reef
reef: crimson/osd/heartbeat: Improve maybe_share_osdmap behavior
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:19:53 +0000 (14:19 +0300)]
Merge pull request #53939 from Matan-B/wip-51750-reef
reef: crimson/os/seastore/backref_manager: scan backref entries by journal seq
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:19:10 +0000 (14:19 +0300)]
Merge pull request #53938 from Matan-B/wip-51475-reef
reef: crimson/os/seastore: fix ceph_assert in segment_manager.h
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:15:39 +0000 (14:15 +0300)]
Merge pull request #53935 from Matan-B/wip-51355-reef
reef: crimson/os/seastore: enable SMR HDD
Reviewed-by: Samuel Just <sjust@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:37:47 +0000 (12:37 -0700)]
Merge pull request #53797 from smanjara/ragweed-test-reef
reef: rgw/upgrade: point upgrade suites to ragweed ceph-reef branch
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:37:03 +0000 (12:37 -0700)]
Merge pull request #53773 from cbodley/wip-63057-reef
reef: rgw: improve buffer list utilization in the chunkupload scenario
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:35:12 +0000 (12:35 -0700)]
Merge pull request #53770 from cbodley/wip-63054-reef
reef: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:34:05 +0000 (12:34 -0700)]
Merge pull request #53767 from cbodley/wip-63051-reef
reef: rgw: s3website doesn't prefetch for web_dir() check
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:33:06 +0000 (12:33 -0700)]
Merge pull request #53765 from cbodley/wip-63048-reef
reef: rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:32:15 +0000 (12:32 -0700)]
Merge pull request #53760 from cbodley/wip-63060-reef
reef: rgw: fix unwatch crash at radosgw startup
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:31:29 +0000 (12:31 -0700)]
Merge pull request #53756 from cbodley/wip-63042-reef
reef: [CVE-2023-43040] rgw: Fix bucket validation against POST policies
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:30:28 +0000 (12:30 -0700)]
Merge pull request #53681 from mkogan1/wip-t62141-reef
reef: valgrind: UninitCondition under __run_exit_handlers suppression
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Anthony D'Atri [Sun, 15 Oct 2023 23:54:09 +0000 (19:54 -0400)]
Merge pull request #54026 from zdover23/wip-doc-2023-10-16-backport-54022-to-reef
reef: doc/rados: Edit troubleshooting-osd (3 of x)
Zac Dover [Sun, 15 Oct 2023 12:43:39 +0000 (22:43 +1000)]
doc/rados: Edit troubleshooting-osd (3 of x)
Edit doc/rados/troubleshooting/troubleshooting-osd.rst (3 of x).
Follows https://github.com/ceph/ceph/pull/53986.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
01b5aa5d27a97a02cef03b9eb0ee9dbcd55684f1 )
Anthony D'Atri [Fri, 13 Oct 2023 20:31:28 +0000 (16:31 -0400)]
Merge pull request #54008 from zdover23/wip-doc-2023-10-14-backport-54006-to-reef
doc/rados: remove FileStore material (in Reef)
Zac Dover [Fri, 13 Oct 2023 18:40:26 +0000 (04:40 +1000)]
doc/rados: remove FileStore material (in Reef)
Remove FileStore-related material because FileStore is not supported in
Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
eab0be5ca513ca2dfaeb31911c175e88bcf4f9e4 )
Anthony D'Atri [Fri, 13 Oct 2023 20:15:24 +0000 (16:15 -0400)]
Merge pull request #54007 from zdover23/wip-doc-2023-10-14-backport-54005-to-reef-2
reef: doc/dev: remove cache-pool
Zac Dover [Fri, 13 Oct 2023 18:17:44 +0000 (04:17 +1000)]
doc/dev: remove cache-pool
Remove cache-pool, which is deprecated in Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
073b0919ed9e413a04a7e12b7a0c23b5e058cd52 )
zdover23 [Fri, 13 Oct 2023 14:01:03 +0000 (00:01 +1000)]
Merge pull request #54000 from zdover23/wip-doc-2023-10-13-backport-53986-to-reef
reef: doc/rados: Edit troubleshooting-osd (2 of x)
Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Zac Dover [Thu, 12 Oct 2023 08:33:58 +0000 (18:33 +1000)]
doc/rados: Edit troubleshooting-osd (2 of x)
Edit doc/rados/troubleshooting/troubleshooting.rst (2 of x).
Follows https://github.com/ceph/ceph/pull/53936.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
37e7099267996a3075b4902a10a19d94fc738c08 )
Nizamudeen A [Fri, 13 Oct 2023 05:35:08 +0000 (11:05 +0530)]
Merge pull request #53458 from rhcs-dashboard/wip-62602-reef
reef: mgr/dashboard: remove unnecessary failing hosts e2e
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Anthony D'Atri [Thu, 12 Oct 2023 09:02:48 +0000 (05:02 -0400)]
Merge pull request #53982 from zdover23/wip-doc-2023-10-12-backport-53936-to-reef
reef: doc/rados: edit troubleshooting-osd (1 of x)
Zac Dover [Wed, 11 Oct 2023 10:38:55 +0000 (20:38 +1000)]
doc/rados: edit troubleshooting-osd (1 of x)
Edit doc/rados/troubleshooting/troubleshooting-osd.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
0fc663d793fe34e42e19b251fcc79b5af52581c6 )
chunmei [Thu, 14 Sep 2023 06:21:05 +0000 (06:21 +0000)]
crimson/vstart: add --seastore-device-size option in vstart.sh command line
default seastore_device_size will be out of space for smp >28
Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit
cf69c4f4946884266d6a42c92eef20e74771c3fa )
Myoungwon Oh [Thu, 31 Aug 2023 02:43:24 +0000 (11:43 +0900)]
crimson/os/seastore/cbj: fix a potential overflow bug
CircularBoundedJournal is similar to circular queue, so segment_seq
increases after rolling. However, current implementation always
increases segment_seq_t when rolling occurs, resulting in
the overflow if segment_seq_t hits MAX_SEG_SEQ.
To mitigate this, this commit changes the type of the segment_seq_t
to uint64_t.
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
36982b66574ff277e04a02d5b9a28e80d0790778 )
Myoungwon Oh [Fri, 25 Aug 2023 08:55:36 +0000 (17:55 +0900)]
crimson/os/seastore/object_data_handler: consider a RBM case when checking if write can be merged
RBM's paddr always indicates physical address, which means it doesn't have the dealayed.
So, this commit adds a condition that checks if given paddr is used for ongoing write.
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
73de8937f65706c9c3dd2e36f6504196d0796b01 )
Myoungwon Oh [Thu, 31 Aug 2023 05:13:07 +0000 (14:13 +0900)]
test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
670a19f27761c937ad8d07abf9bb7c94ed4a3b00 )
Xuehan Xu [Sun, 6 Aug 2023 07:45:24 +0000 (15:45 +0800)]
crimson/osd/osd_operations/snaptrim_event: don't process snaptrim events
of the same PG concurrently
Concurrently triming snaps of the same PG may lead to the following
problem:
1. fiber 1 get objects to trim for PG 1.x and snapid 1.d, which involves
object OBJ;
2. fiber 2 get objects to trim for PG 1.x and snapid 1.c, which also
involes object OBJ;
3. fiber 1 removes snap 1.c since it has been removed in the osdmap
4. fiber 2 try to get obc for object OBJ and find that OBJ doesn't have
snap for id 1.c
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
267babe435a54130eda5770a537420fb516e2a8f )
Xuehan Xu [Tue, 18 Jul 2023 06:14:50 +0000 (14:14 +0800)]
crimson/osd/object_context: consider clones found as long as they're in
SnapSet::clones
Fixes: https://tracker.ceph.com/issues/61825
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
957fd43e4b09a7607d4db52fe9674be905f76de8 )
Xuehan Xu [Tue, 18 Jul 2023 06:12:59 +0000 (14:12 +0800)]
crimson/osd/object_context_loader: rename with_head_and_clone_obc to
with_clone_obc_direct
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
ec646eb22bfd30f6cf19c86fe24c233001272047 )
Matan Breizman [Wed, 23 Aug 2023 08:10:32 +0000 (08:10 +0000)]
crimson/admin/admin_socket: remove path file if it exists
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
374de2a7622bf3fc75eceb24e39cf36a27ed8d1b )
Xuehan Xu [Thu, 17 Aug 2023 03:09:05 +0000 (11:09 +0800)]
crimson/osd/pg: check if backfill_state exists when judging objects'
degradation
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
04764838b5395a4e4d0c872f2d2a433fc4aec074 )
Xuehan Xu [Fri, 18 Aug 2023 11:56:50 +0000 (19:56 +0800)]
crimson/os/seastore/omap_manager: fix the entry leak issue in
BtreeOMapManager::omap_list()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
cf105932a7fadf67ec494587b078bbfb4caf4d26 )
myoungwon oh [Wed, 16 Aug 2023 11:58:49 +0000 (11:58 +0000)]
crimson/os/seastore/journal/cbj: remove return_record and read_record
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
39ea5add3a81c32263533ec8f01590efeceb2c86 )
myoungwon oh [Mon, 14 Aug 2023 08:10:04 +0000 (08:10 +0000)]
crimson/os/seastore/journal/cbj: check if batched records are written at once before rolling
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
ef1a6a90e0131cdf3bb96ace835f845ae23ba0ce )
myoungwon oh [Fri, 11 Aug 2023 14:47:21 +0000 (14:47 +0000)]
crimson/os/seastore: genralize read_validate_record_metadata and read_validate_data
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
15ae64c3311378f6055d312c2b5f64b46912d899 )
myoungwon oh [Fri, 11 Aug 2023 04:52:23 +0000 (04:52 +0000)]
crimson/os/seastore/journal/cbj: check if the first record in CBJ is written after rolling or not
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
a10f2855da3bc1c25f664158af1cc79d70bdef57 )
myoungwon oh [Sat, 22 Jul 2023 14:52:14 +0000 (14:52 +0000)]
crimson/os/seastore/journal/cbj: introduce magic value to identify that written records are correct
The cbj would replay invalid records if the same write pattern (mkfs -> write -> mkfs -> write) repeats.
On the second mkfs(), current cbj wrongly recognizes the old records are valid beceause they have
correct seq. number at appropriate offset even though the records are expired.
To solve this, this commit introduces random value to identify the journal during mkfs()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
942ea6487f59e52d88050fb8deb8a2e6c0ca83c3 )
myoungwon oh [Sat, 22 Jul 2023 06:03:04 +0000 (06:03 +0000)]
crimson/os/seastore: introduce generalized scan_valid_records for RBM
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
a580e516965ff8ae57dbaab202854f9ebe38ba5f )
myoungwon oh [Thu, 20 Jul 2023 02:50:48 +0000 (02:50 +0000)]
crimson/os/seastore: introduce RecordScanner to generalize scan_valid_records()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
22c747826b398cc0a0d68aab9bba1fd096a2de9d )
Xuehan Xu [Wed, 16 Aug 2023 11:37:59 +0000 (19:37 +0800)]
crimson/os/seastore: fix daggling reference of oid in
SeaStore::Shard::stat()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
0ee9bb21be89bdcfb3e6cfb224c256d6eccd9f38 )
Xinyu Huang [Wed, 2 Aug 2023 02:09:52 +0000 (10:09 +0800)]
crimson/os/seastore: retire_extent_addr clean up
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
b99a1956bdcb879a3e5440e617593bae43fafd62 )
Radosław Zarzyński [Wed, 26 Jul 2023 10:42:45 +0000 (12:42 +0200)]
crimson/ertr: assert on invocability of func provided to safe_then()
The goal is to fail with an easier-to-understand error on failed
`static_assert()` than cryptic template substitution errors when
the function provided to e.g. `safe_then()` doesn't match the type
carried within an errorated future.
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit
83687f2cc86897d60a460575a7ab2acbd2f9170c )
Xuehan Xu [Wed, 28 Jun 2023 07:18:27 +0000 (15:18 +0800)]
crimson/os/seastore/collection_manager: fill CollectionNode::decoded on
clean reads
Otherwise, deltas of collection nodes can't be replayed correctly,
because the data of nodes is not loaded to CollectionNode::decoded
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
5eb473b99c1ea75fd6c3807d3f54eaec1018d383 )
Xuehan Xu [Fri, 14 Jul 2023 03:59:06 +0000 (11:59 +0800)]
crimson/os/seastore/object_data_handler: add ObjectDataHandler(extent_len_t)
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
c1fdb2a8a41c6f0f204c980c4f3af7cfac28723d )
Xuehan Xu [Fri, 14 Jul 2023 03:56:12 +0000 (11:56 +0800)]
crimson/os/seastore/cache: correct log prefix for get_absent_extent()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
5a563d63e754d5163424486dbd0d665165615794 )
Aishwarya Mathuria [Wed, 28 Jun 2023 11:25:03 +0000 (16:55 +0530)]
crimson/mgr: Fix config show command
Update the MMgrOpen message to include config_bl and config_defaults_bl
Fixes: https://tracker.ceph.com/issues/61846
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
(cherry picked from commit
249a6ddedb3840a91cc17708eba3a72f8b22d267 )
Zhang Song [Sun, 9 Jul 2023 04:22:58 +0000 (12:22 +0800)]
crimson/os/seastore: add metrics about total invalidated transactions
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit
6265452c50c2bbd0283a66d03d6a8538c2285293 )
myoungwon oh [Sat, 17 Jun 2023 08:27:17 +0000 (08:27 +0000)]
src/test/crimson/seastore/nvmedevice: update a test case based on rbm multi shard change
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
7a15d484c552a177585954cdb4e2e226a43c1035 )
myoungwon oh [Fri, 26 May 2023 08:29:50 +0000 (08:29 +0000)]
crimson/os/seastore/rbm: make rbm support multiple shards
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
1d6beef975144210ae29626e61aeb284aff30d53 )
Xinyu Huang [Fri, 26 May 2023 11:05:08 +0000 (11:05 +0000)]
test/crimson/seastore: test if lazy read work correctly
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
9d973efe32c089351ec756232d3699b4c9fe1f64 )
Xinyu Huang [Fri, 21 Apr 2023 14:22:16 +0000 (14:22 +0000)]
test/crimson/seastore: test overwrite pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
efd3cdc71056baabba291d7a359dda6dbd9121b1 )
Xinyu Huang [Fri, 21 Apr 2023 14:19:34 +0000 (14:19 +0000)]
test/crimson/seastore: add test for TransactionManager::remap_pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
b5931d2795cfe44b96c6ee9347514b002197bb5b )
Xinyu Huang [Fri, 21 Apr 2023 14:42:00 +0000 (14:42 +0000)]
crimson/os/seastore: check if length exist in mark_space_used and mark_space_free
to avoid misuse of buffer length and data length when there is a no actual data extent
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
dc4a23cef3787099dcc17f99b05f4ec1a63eca9a )
Xinyu Huang [Wed, 19 Apr 2023 01:46:31 +0000 (01:46 +0000)]
crimson/os/seastore: deepcopy the buffer of exist clean extent when duplicate_for_write
Beacuse exist clean extent shares buffer with original clean extent.
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
026066a7e5c4cc1bc04ed91003c553146a0b60ef )