J. Eric Ivancich [Thu, 23 Dec 2021 21:25:26 +0000 (16:25 -0500)]
rgw: `radosgw-admin bucket stats` on indexless bucket crashes
The new bucket layout code didn't check whether the bucket is
indexless prior to asking for the last entry in the layout log. The
layout log appears to be empty for an indexless bucket, thereby
putting the runtime in an undefined state that later may cause a
failed assertion.
This commit adds two safety checks and returns -EINVAL along with
putting useful information on stderr when either stats are requested
on an indexless bucket or when the layout log is empty.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Yuval Lifshitz [Wed, 8 Dec 2021 19:35:25 +0000 (21:35 +0200)]
rgw: fix reshard cancelling race condition
this is happening when resharding while objects are uploaded
tests steps are here:
https://gist.github.com/yuvalif/
060f66f03511bff881e952287df3087b
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Casey Bodley [Wed, 8 Dec 2021 15:11:22 +0000 (10:11 -0500)]
rgw: preserve 'bucket sync disable' over reshard
if bucket sync is disabled, apply that flag to new index objects on
bucket reshard
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 22 Nov 2021 19:23:01 +0000 (14:23 -0500)]
rgw/multisite: handle shard_progress correctly in RunBucketSources
we run bucket sync on each of the sync pipes, so size the vector
accordingly
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 22 Nov 2021 18:51:10 +0000 (13:51 -0500)]
Revert "rgw: cr: add prealloc_stack()"
This reverts commit
7970f355497f48ee5a18bf3a0bc034226c6d225c.
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 22 Nov 2021 18:50:07 +0000 (13:50 -0500)]
Revert "rgw: bucket sync: track progress by stack id"
This reverts commit
c0baf3eb34c6c1de7e4de2e35cb62e219c174b0b.
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Conflicts:
src/rgw/rgw_data_sync.cc no longer loops over num_shards
Casey Bodley [Mon, 22 Nov 2021 18:05:40 +0000 (13:05 -0500)]
rgw/multisite: RunBucketSourcesSync no longer takes optional target
RGWDataSyncSingleEntryCR is the only caller of RGWRunBucketSourcesSyncCR
it always provides a source_bs, and never provides a target_bs. so remove
all the complexity related to target_bs, and the idea that we'd need to
sync several source bucket shards related to the target bucket
we now just have the single loop over the target buckets that use the
given bucket as a source
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 15 Oct 2021 14:57:55 +0000 (10:57 -0400)]
radosgw-admin: allow reshard commands in multisite on secondary
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 18 Oct 2021 18:15:33 +0000 (14:15 -0400)]
rgw: fix for uninitialized oldest_gen/latest_gen
when data sync queries RGWOp_BILog_Info from an un-upgraded gateway, it
doesn't include the oldest_gen/latest_gen fields. so initialize these
variables to 0 by default
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 12 Oct 2021 16:45:29 +0000 (12:45 -0400)]
rgw: enable RGWReshard thread on any zone that supports it
enable the background dynamic resharding thread based on
RGWSI_Zone::can_reshard(), which takes the zonegroup features into
account
Fixes: https://tracker.ceph.com/issues/52877
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 28 Sep 2021 14:45:27 +0000 (10:45 -0400)]
rgw: prevent reshard from creating too many log generations
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Shilpa Jagannath [Fri, 23 Jul 2021 06:58:24 +0000 (12:28 +0530)]
rgw: remove per-shard sync status object after incremental sync finishes
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 20 Aug 2021 14:39:34 +0000 (10:39 -0400)]
radosgw-admin: bucket sync status guards against shard count mismatch
if the remote gives us more shards than we expect, just count those
shards as 'behind' and avoid out-of-bounds access of shard_status
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 19 Aug 2021 21:21:37 +0000 (17:21 -0400)]
radosgw-admin: bucket sync status handles missing full status
if the full sync status object is missing, it's possible that we just
haven't started syncing it again after upgrading from just the per-shard
status objects
in this case, as long as we have a log generation 0, assume that we just
haven't initialized the full status object and try to read the gen=0
per-shard incremental status for comparison
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 19 Aug 2021 20:12:21 +0000 (16:12 -0400)]
rgw: rgw_read_bucket_inc_sync_status doesn't need bucket info
all we need to construct the per-shard bucket sync status object names
are the bucket names themselves, which we already have from
rgw_sync_bucket_pipe
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 19 Aug 2021 20:30:49 +0000 (16:30 -0400)]
rgw: resize status vector before reading inc_sync_status
rgw_read_bucket_inc_sync_status() uses the size of this vector as the
'num_shards', so we need to resize it appropriately beforehand
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 19 Aug 2021 20:21:55 +0000 (16:21 -0400)]
rgw: RGWOp_BILog_Status reads full status unconditionally
the calls to rgw_read_bucket_inc_sync_status() depend on
sync_status.incremental_gen, which we need to read via
rgw_read_bucket_full_sync_status() regardless of whether
we're returning it to the client (version > 1)
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Adam C. Emerson [Fri, 10 Sep 2021 16:28:28 +0000 (12:28 -0400)]
rgw: RGWCollectBucketSyncStatusCR doesn't need the shard count
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 10 Sep 2021 16:06:53 +0000 (12:06 -0400)]
rgw: RunBucketSourceSync uses num_shards from remote bilog info
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 10 Sep 2021 15:38:05 +0000 (11:38 -0400)]
rgw: RGWListBucketIndexesCR only needs zero shard
We only need to check one shard, and everything has shard zero.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 10 Sep 2021 15:00:07 +0000 (11:00 -0400)]
rgw: sync checkpoint gets num_shards from remote bilog info
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Thu, 2 Sep 2021 21:58:58 +0000 (17:58 -0400)]
rgw: RGWRemoteBucketManager constructor takes num_shards
The logic for getting it was moved to its caller.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Thu, 2 Sep 2021 21:36:09 +0000 (17:36 -0400)]
rgw: InitBucketFullSyncStatusCR gets num shards from remote
As specified in rgw_bucket_index_marker_info, unless we're doing the
compatibility check, in which case we look at generation 0.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Shilpa Jagannath [Thu, 10 Jun 2021 16:51:21 +0000 (22:21 +0530)]
rgw: read shard count using remote bilog info during bucket sync
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Casey Bodley [Mon, 8 Mar 2021 17:48:13 +0000 (12:48 -0500)]
doc/rgw: document zone features
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 3 Mar 2021 21:58:33 +0000 (16:58 -0500)]
rgw: RGWSI_Zone::can_reshard() respects zonegroup 'resharding' feature
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 3 Mar 2021 21:55:23 +0000 (16:55 -0500)]
radosgw-admin: 'sync status' shows enabled/disabled zonegroup features
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 3 Mar 2021 21:54:58 +0000 (16:54 -0500)]
radosgw-admin: zone/zonegroup commands support --enable-feature=x --disable-feature=y
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 3 Mar 2021 19:13:05 +0000 (14:13 -0500)]
rgw: add set of 'features' to zone and zonegroup
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 3 Mar 2021 19:05:24 +0000 (14:05 -0500)]
json: encoding for flat_set accepts all template params
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Yuval Lifshitz [Wed, 16 Jun 2021 09:32:25 +0000 (12:32 +0300)]
rgw/multisite: don't delete per shard status on init
and pass correct generation and num shards when deleting
per shard status objects when disabling during incremental sync
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Yuval Lifshitz [Mon, 14 Jun 2021 14:03:35 +0000 (17:03 +0300)]
rgw/multisite: support enable right after disable
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Yuval Lifshitz [Thu, 27 May 2021 15:54:31 +0000 (18:54 +0300)]
rgw/multisite: remove the retry mechanism
when writign the sync status object
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Yuval Lifshitz [Tue, 18 May 2021 15:59:54 +0000 (18:59 +0300)]
rgw/multisite: allow bucket sync disable/enable
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Yuval Lifshitz [Tue, 25 May 2021 18:11:25 +0000 (21:11 +0300)]
rgw/multisite: track shard sync status objects per generation
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Casey Bodley [Wed, 9 Jun 2021 16:26:54 +0000 (12:26 -0400)]
rgw: remove destination shard id from rgw_bucket_sync_pair_info
the sync_pair is used as input to RGWBucketPipeSyncStatusManager::status_oid()
to generate the per-shard sync status object names
this sync status tracks incremental bucket sync, which reads changes
from a source bucket's bilog shard, and copies objects from the remote
source bucket to the local destination bucket
this doesn't require sync to know anything about the destination bucket
shards, so rgw_bucket_sync_pair_info and status_oid() now only track the
the destination's rgw_bucket instead of rgw_bucket_shard
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Adam C. Emerson [Fri, 14 May 2021 22:59:48 +0000 (18:59 -0400)]
rgw: Trim old generations in BucketTrimInstanceCR
Only one generation per call.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Tue, 18 May 2021 22:34:43 +0000 (18:34 -0400)]
rgw: Add RGWRadosRemoveOidCR
A more generally applicable way of removing objects in coroutines.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Tue, 18 May 2021 21:56:27 +0000 (17:56 -0400)]
rgw: Add RGWAsyncPutBucketInstanceInfoCR
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Mon, 26 Apr 2021 22:45:09 +0000 (18:45 -0400)]
rgw: Trim bilog with generation
From the REST interface and radosgw-admin. Assume Generation 0 if none
provided and error if it doesn't exist.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Tue, 27 Apr 2021 23:31:57 +0000 (19:31 -0400)]
rgw: Bilog trim takes markers as string view
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Tue, 27 Apr 2021 23:31:15 +0000 (19:31 -0400)]
common: get_str_vec takes std::string_view
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 14 May 2021 19:44:01 +0000 (15:44 -0400)]
rgw: BucketInstanceTrimCR knows about generations
Fetch the current generation from remote peers and trim the minimum
marker on the minimum generation.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 14 May 2021 19:26:42 +0000 (15:26 -0400)]
rgw: Add cast from bucket_index_log_layout
To bucket_index_layout_generation
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Thu, 13 May 2021 21:11:52 +0000 (17:11 -0400)]
rgw: add sync_info to BILog_Status output
Needed so we can get the incremental generation.
Guard this behind a version check and return the original output if
less than 2.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Shilpa Jagannath [Wed, 2 Jun 2021 17:57:06 +0000 (23:27 +0530)]
rgw: fix incremental sync by using the right generation for bilog listing
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Fri, 21 May 2021 06:47:49 +0000 (12:17 +0530)]
rgw: on bucket reshard, write datalog entries for each shard of the previous generation
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Casey Bodley [Tue, 25 May 2021 18:37:26 +0000 (14:37 -0400)]
qa/rgw: temporarily disable multisite reshard tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 30 Apr 2021 20:22:33 +0000 (16:22 -0400)]
test/rgw: add multisite test for full sync after reshard
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 30 Apr 2021 19:08:39 +0000 (15:08 -0400)]
test/rgw: add simple multisite reshard test
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 18 May 2021 19:55:28 +0000 (15:55 -0400)]
radosgw-admin: 'bucket sync checkpoint' waits for generation to catch up
poll on rgw_read_bucket_full_sync_status() until
full_status.incremental_gen catches up to the latest_gen we got from
rgw_read_remote_bilog_info()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 19 May 2021 18:03:48 +0000 (14:03 -0400)]
rgw: get_bucket_instance_ids() uses num_shards from layout
knock out a TODO that was causing this assertion failure in
RGWRados::get_bucket_stats() after a reshard:
ceph_assert(headers.size() == bucket_instance_ids.size());
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 20 May 2021 15:15:26 +0000 (11:15 -0400)]
rgw: rgw_read_remote_bilog_info() returns rgw_bucket_index_marker_info
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Shilpa Jagannath [Mon, 15 Feb 2021 14:46:29 +0000 (20:16 +0530)]
rgw: adding generation number to async notification
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Casey Bodley [Tue, 23 Feb 2021 19:55:50 +0000 (14:55 -0500)]
rgw: add custom json encode/decode for the v1 notify API
this adds wrapper structs rgw_data_notify_v1_encoder and
rgw_data_notify_v1_decoder that can encode/decode the v1 json format
directly on the v2 data structure
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 14 May 2021 17:49:09 +0000 (13:49 -0400)]
test/rgw: fix python error on test failure
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 14 May 2021 16:30:21 +0000 (12:30 -0400)]
rgw: fix handling of bucket layout in metadata sync
clear the bucket layout we get from the metadata master, and overwrite it
with our zone's defaults
without clearing the layout, init_default_bucket_layout() was adding another
log layout in addition to the one from the master. this caused the bilog
list API to provide a 'next_log' when only gen=0 exists
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Yuval Lifshitz [Fri, 14 May 2021 09:27:09 +0000 (12:27 +0300)]
rgw/multisite: fix bucket shard state init function
* make sure src/dest shard ids are the same in sync pair
* copy sync pair by value in coroutine loop
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Shilpa Jagannath [Mon, 5 Apr 2021 20:15:45 +0000 (01:45 +0530)]
rgw: update bucket sync status after bucket shards finishes current gen
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Casey Bodley [Fri, 26 Mar 2021 15:00:57 +0000 (11:00 -0400)]
rgw: reshard preserves old index in multisite
if the old index is still referenced by an InIndex log layout, we can't
call clean_index() to remove the index objects yet. log trimming will do
that later, once the bilogs are no longer needed
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 25 Feb 2021 20:39:26 +0000 (15:39 -0500)]
rgw: BILog_List handles requests for generation=0
previous logic treated requests for generation=0 as the latest gen
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 9 Feb 2021 23:40:29 +0000 (18:40 -0500)]
rgw: zero-initialize rgw_bucket_sync_status::incremental_gen
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 9 Feb 2021 23:00:14 +0000 (18:00 -0500)]
rgw: handle older/newer generations after reading bucket sync status
wait until we've read the bucket sync status and found that we're in
incremental sync before we start using incremental_gen for comparison
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Adam C. Emerson [Mon, 14 Dec 2020 05:56:23 +0000 (00:56 -0500)]
rgw: Handle entries of the wrong generation
Drop entries from past generations.
Send entries of future generations to the error repo for retry.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Casey Bodley [Wed, 10 Feb 2021 00:04:19 +0000 (19:04 -0500)]
rgw: rgw_data_change can decode v1 format if gen was 0
but if gen>0, require decoders to understand the v2 format. this way,
old clients can't decode entries with gen>0, so they won't be able to
serve them to other zones
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 10 Feb 2021 00:03:43 +0000 (19:03 -0500)]
rgw: rename rgw_data_change::gen_id
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Adam C. Emerson [Mon, 14 Dec 2020 02:13:44 +0000 (21:13 -0500)]
rgw: add gen parameter to RGWDataChangesLog::add_entry
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Mon, 14 Dec 2020 01:30:52 +0000 (20:30 -0500)]
rgw: Add gen_id to rgw_data_change
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Sun, 13 Dec 2020 23:54:28 +0000 (18:54 -0500)]
rgw: vector<rgw_data_change_log_entry> not list
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Casey Bodley [Mon, 1 Feb 2021 19:39:39 +0000 (14:39 -0500)]
rgw: add json encoding of bucket layout types
adds a "layout" section to RGWBucketInfo
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 1 Feb 2021 19:34:54 +0000 (14:34 -0500)]
json: add encode_json() overload for string_view
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 2 Feb 2021 17:51:14 +0000 (12:51 -0500)]
rgw: refactor per-entry reshard logic into separate function
this cuts down on nesting and avoids the need for goto
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 1 Feb 2021 17:04:36 +0000 (12:04 -0500)]
rgw: reshard adds a log layout for the new index
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 1 Feb 2021 17:02:44 +0000 (12:02 -0500)]
radosgw-admin: try reshard even if bucket is resharding
allow reshard in case a previous reshard failed. if the reshard is
actually still in progress, we'll fail to get the reshard lock
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 19 Jan 2021 14:27:13 +0000 (09:27 -0500)]
test: fix threading for FaultInjector death tests
addresses test timeout and warning message:
[WARNING] /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest-death-test.cc:1121:: Death tests use fork(), which is unsafe particularly in a threaded context. For this test, Google Test detected 3 threads. See https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#death-tests-and-threads for more explanation and suggested solutions, especially if this is the last message you see before your test times out.
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 18 Dec 2020 21:01:15 +0000 (16:01 -0500)]
radosgw-admin: remove fault injection options from usage
these are only used for testing, not administration
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 18 Dec 2020 16:19:55 +0000 (11:19 -0500)]
rgw: RGWBucketReshard doesn't need a friend
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 18 Dec 2020 15:46:49 +0000 (10:46 -0500)]
rgw: fix decode of cls_rgw reshard types
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 18 Dec 2020 15:23:55 +0000 (10:23 -0500)]
rgw: BucketReshardManager takes layouts
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 18 Dec 2020 15:22:23 +0000 (10:22 -0500)]
rgw: clean up uses of BucketShard::init() without info
the rgw_bucket overload of BucketShard::init() has to look up the bucket
info. use the RGWBucketInfo overload when we have one
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Thu, 17 Dec 2020 17:19:12 +0000 (12:19 -0500)]
qa/rgw: disable coredumps for reshard fault injection
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 16 Dec 2020 23:17:29 +0000 (18:17 -0500)]
test/rgw: add test_bucket_reshard() for fault injection testing
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 16 Dec 2020 23:16:41 +0000 (18:16 -0500)]
test/rgw: test_rgw_reshard.py exec_cmd() can return error code
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 16 Dec 2020 23:13:43 +0000 (18:13 -0500)]
radosgw-admin: 'bucket reshard' returns positive error codes
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 16 Dec 2020 18:57:28 +0000 (13:57 -0500)]
rgw: reshard first cleans up old-style reshards
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 15 Dec 2020 18:57:31 +0000 (13:57 -0500)]
rgw: refactor reshard init/cleanup with fault injection
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 15 Dec 2020 17:55:03 +0000 (12:55 -0500)]
rgw: add typedef for ReshardFaultInjector
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 15 Dec 2020 17:07:14 +0000 (12:07 -0500)]
rgw: BucketReshardManager stores BucketReshardShards by value
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 15 Dec 2020 17:06:24 +0000 (12:06 -0500)]
rgw: remove unused attrs from RGWBucketReshard
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Shilpa Jagannath [Mon, 12 Oct 2020 12:51:16 +0000 (18:21 +0530)]
rgw: reshard tests with fault injection
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Casey Bodley [Wed, 19 Aug 2020 19:15:56 +0000 (15:15 -0400)]
common: add generic FaultInjector
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Shilpa Jagannath [Mon, 21 Sep 2020 07:31:44 +0000 (13:01 +0530)]
rgw: Better error handling
- function update_bucket() handles updating bucket state
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Fri, 11 Sep 2020 06:28:55 +0000 (11:58 +0530)]
rgw: use a helper function to handle repetitive num_shards check
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Tue, 1 Sep 2020 18:25:42 +0000 (23:55 +0530)]
rgw: code organization fixes and error handling
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Fri, 28 Aug 2020 06:00:04 +0000 (11:30 +0530)]
rgw: clean up fixes for extra spaces and indentation.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Tue, 21 Jul 2020 13:10:09 +0000 (18:40 +0530)]
rgw: helper function to set reshard status from rgw::BucketReshardState.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Tue, 21 Jul 2020 08:08:24 +0000 (13:38 +0530)]
rgw: minor fixes
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Tue, 14 Jul 2020 12:26:05 +0000 (17:56 +0530)]
rgw: failing to reshard, restore old indexes
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Tue, 14 Jul 2020 07:33:31 +0000 (13:03 +0530)]
rgw: allow clean_index to take const bucket_index_layout_generation&
fix bi_get() to get objects after being resharded
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Shilpa Jagannath [Fri, 10 Jul 2020 08:05:25 +0000 (13:35 +0530)]
rgw: - take non-optional gen_id in svc_bi_rados functions
- call init_index() on target layout during reshard process.
Takes const rgw::bucket_index_layout_generation&
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>