]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Yehuda Sadeh [Mon, 5 Jun 2017 17:05:24 +0000 (10:05 -0700)]
rgw: fix a race
We referred to complete_op_data after aio call that could have released
it.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 2 Jun 2017 22:15:27 +0000 (15:15 -0700)]
test/cli/radosgw-admin: update reference help.t
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 2 Jun 2017 21:42:13 +0000 (14:42 -0700)]
rgw: fix index completion
Was missing bucket complete op, only had the guard (bad merge?).
Pass zones_trace as pointer and treat it as optional param (which
it is).
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 2 Jun 2017 00:21:07 +0000 (17:21 -0700)]
rgw: increase max shard prime, consolidate logic
Handle larger num of shards (max ~64k now). Use the same hash logic for shard ids
everywhere.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 1 Jun 2017 23:55:35 +0000 (16:55 -0700)]
rgw: new num shards proportional to number of objects per shard
and also limit number of shards to not exceed the max.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 30 May 2017 23:35:17 +0000 (16:35 -0700)]
rgw: handle zones_trace in bi async completion handler
needed following a rebase
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 25 May 2017 21:11:01 +0000 (14:11 -0700)]
rgw: remove debug helper code
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:57:28 +0000 (16:57 -0700)]
rgw: ignore -ENOENT on bi_list()
just treat it as success with empty result
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Orit Wasserman [Sun, 28 May 2017 07:42:54 +0000 (10:42 +0300)]
rgw: run resharding thread only on master zone
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 23 May 2017 15:45:00 +0000 (18:45 +0300)]
rgw: add check to see if out is null
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 23 May 2017 10:37:08 +0000 (13:37 +0300)]
rgw: update entry in resharding log when resharding has started
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 21 May 2017 10:26:37 +0000 (13:26 +0300)]
rgw: start resharding theard
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Yehuda Sadeh [Thu, 18 May 2017 18:51:21 +0000 (11:51 -0700)]
rgw: replace reshard blocking sleep with interruptible condition wait
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Orit Wasserman [Fri, 19 May 2017 11:02:17 +0000 (14:02 +0300)]
rgw: add bucket to resharding queue if needs resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Yehuda Sadeh [Thu, 25 May 2017 00:00:02 +0000 (17:00 -0700)]
rgw-admin: radosgw-admin reshard execute -> reshard process
to be in line with other radosgw-admin commands.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:01:25 +0000 (16:01 -0700)]
rgw: update bucket instance with resharding details
can be useful info for later in a cross zone sync
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 20:21:43 +0000 (13:21 -0700)]
radosgw-admin: reshard execute output adjustments
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 18:47:05 +0000 (11:47 -0700)]
cls/rgw: remove unused field in reshard entry
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 18:21:46 +0000 (11:21 -0700)]
rgw: reshard: renew lease and handle marker when listing reshard repo
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 18:24:20 +0000 (11:24 -0700)]
cls/rgw: fix reshard_list truncation identification
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 18:23:46 +0000 (11:23 -0700)]
cls/rgw: expose api to generate reshard indexing key
So that we could use it as a marker when doing reshard_list.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 23 May 2017 00:10:14 +0000 (17:10 -0700)]
rgw: remove unneeded and duplicate code
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 23 May 2017 00:10:07 +0000 (17:10 -0700)]
rgw: initialize bucket structure
after reading bucket info
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:58:47 +0000 (16:58 -0700)]
rgw-admin: revive reshard execute command
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:58:19 +0000 (16:58 -0700)]
rgw: bucket_id does not include bucket name
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:55:21 +0000 (16:55 -0700)]
rgw: bi_list() initialize is_truncated if -ENOENT
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:54:30 +0000 (16:54 -0700)]
rgw: internal api cleanup
rename method, minor change
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 24 May 2017 23:53:34 +0000 (16:53 -0700)]
cls/rgw: reshard add command doesn't read existing status
No need to read existing status.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 19 May 2017 22:44:06 +0000 (15:44 -0700)]
cls/rgw: fix remove_reshard objclass op
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 19 May 2017 16:22:29 +0000 (09:22 -0700)]
cls/rgw: modify logshard key
The logshard key shouldn't include bucket id.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 18 May 2017 23:21:19 +0000 (16:21 -0700)]
rgw: multiple fixes and adjustments related to resharding scheduler
still wip, but:
- get rid of some unneeded rados locking
- rename shards to logshards where relevant to avoid confusion
- remove unused code
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 18 May 2017 21:48:07 +0000 (14:48 -0700)]
rgw: fix compilation following rebase
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 18 May 2017 18:51:21 +0000 (11:51 -0700)]
rgw: replace reshard blocking sleep with interruptible condition wait
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Orit Wasserman [Fri, 19 May 2017 13:48:12 +0000 (16:48 +0300)]
rgw: actually reshard an entry from the resharding queue
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 19 May 2017 12:20:18 +0000 (15:20 +0300)]
rgw: check if buckets needs resharding on all ops that add new objects
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 19 May 2017 11:02:17 +0000 (14:02 +0300)]
rgw: add bucket to resharding queue if needs resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 18 May 2017 19:48:28 +0000 (22:48 +0300)]
rgw: check for bucket resharding when adding an object
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 18 May 2017 19:47:12 +0000 (22:47 +0300)]
rgw: add a function to add a bucket to the resharding queue
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 18 May 2017 04:01:35 +0000 (07:01 +0300)]
rgw: add a check_bucket_shards to the quota
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 14 May 2017 08:03:22 +0000 (11:03 +0300)]
rgw: moved reshard_bucket into RGWReshard
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 14 May 2017 06:05:49 +0000 (09:05 +0300)]
rgw: remove cct from RGWReshard and BucketIndexLockGuard
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 14 May 2017 05:34:04 +0000 (08:34 +0300)]
rgw: add resharding thread
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Yehuda Sadeh [Wed, 17 May 2017 23:40:16 +0000 (16:40 -0700)]
rgw: store new num shards in bucket index reshard entry
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 17 May 2017 21:50:53 +0000 (14:50 -0700)]
rgw: fix deletion of index completion object
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 17 May 2017 01:10:15 +0000 (18:10 -0700)]
rgw: reshard complete handling, use multiple locks
to reduce contention. Also, fix completions cleanup.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 16 May 2017 23:33:17 +0000 (16:33 -0700)]
rgw: clean up index completions on shutdown
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 16 May 2017 22:30:32 +0000 (15:30 -0700)]
rgw: update data log after index completion reshard handling
Also rearrange code a bit.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Sat, 13 May 2017 00:07:04 +0000 (17:07 -0700)]
rgw: guard resharding in async completions
Catch return value from async index completions, resend operations
synchronously on a separate worker thread if needed.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 12 May 2017 21:44:06 +0000 (14:44 -0700)]
rgw: reshard guard for versioned operations
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Fri, 12 May 2017 18:47:54 +0000 (11:47 -0700)]
cls/rgw: changes to support reshard guard on olh bi ops
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 11 May 2017 22:41:15 +0000 (15:41 -0700)]
rgw: handle reshard wait logic in cancel() too
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 11 May 2017 17:39:26 +0000 (10:39 -0700)]
rgw: fix update_bucket_id, reshard blocking logic
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 10 May 2017 23:58:41 +0000 (16:58 -0700)]
rgw_admin: add rgw-admin reshard status command and initial blocking logic
A command that returns the reshard status field on the bucket index objects (all
shards).
Also, initial implementation of identifying and blocking while reshard in
progress.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 10 May 2017 18:46:57 +0000 (11:46 -0700)]
rgw: more detailed resharding status kept in bucket header
So that we can differentiate between in-progress and complete status.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Wed, 10 May 2017 16:39:55 +0000 (09:39 -0700)]
rgw: a few fixes
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 9 May 2017 22:38:57 +0000 (15:38 -0700)]
rgw: mark source bucket shards when starting reshard
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 9 May 2017 22:19:10 +0000 (15:19 -0700)]
rgw: move resharding code to BucketReshard class
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Tue, 9 May 2017 20:41:30 +0000 (13:41 -0700)]
rgw: split per-bucket resharding logic out from general resharding
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Mon, 8 May 2017 22:42:11 +0000 (15:42 -0700)]
rgw: revert some reshard blocking logic
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Mon, 8 May 2017 21:43:37 +0000 (14:43 -0700)]
cls/rgw: create an objclass method to guard in-progress resharding
This method will be called on the bucket index shard.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Orit Wasserman [Mon, 8 May 2017 13:41:35 +0000 (16:41 +0300)]
rgw: RGWReshard update to new cls API
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Yehuda Sadeh [Thu, 4 May 2017 22:11:09 +0000 (15:11 -0700)]
cls/rgw: adjust types and ops
- don't keep data bufferlist in struct cls_rgw_bucket_instance_entry
- remove rgw_get_bucket_instance
We need to keep the reshard info as part of the bucket index header (for
each source bucket shard), and not in a separate object. Otherwise it'll
be impossible to have the resharding check atomic as part of the regular
bucket index updates.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 4 May 2017 20:50:26 +0000 (13:50 -0700)]
cls/rgw: minor changes
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 4 May 2017 20:39:54 +0000 (13:39 -0700)]
cls/rgw: generate_reshard_key, change output param
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Yehuda Sadeh [Thu, 4 May 2017 20:36:38 +0000 (13:36 -0700)]
cls/rgw: return default header if not initialized
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Orit Wasserman [Mon, 8 May 2017 05:06:48 +0000 (08:06 +0300)]
rgw: cls_reshard_get remove unneeded check for bufferlist length
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Wed, 3 May 2017 08:48:49 +0000 (11:48 +0300)]
rgw: get_bucket_header will return -ENOENT when there is no header
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Mon, 1 May 2017 06:24:19 +0000 (09:24 +0300)]
rgw: add guard class for bucket index lock
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Mon, 1 May 2017 06:23:34 +0000 (09:23 +0300)]
rgw: add reshard_pool_ctx
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 30 Apr 2017 07:16:18 +0000 (10:16 +0300)]
rgw: check for exisiting entries in cls_rgw_reshard_add
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 30 Apr 2017 05:31:30 +0000 (08:31 +0300)]
rgw: use pointer for is_truncated
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 28 Apr 2017 13:59:52 +0000 (16:59 +0300)]
rgw: cls_rgw_reshard use helper function the generate key
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 28 Apr 2017 10:13:07 +0000 (13:13 +0300)]
rgw: limit op.max and add extra entry to handle tuncation in cls_rgw_reshard_list
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 28 Apr 2017 10:03:11 +0000 (13:03 +0300)]
rgw: use .rgw.log:reshard pool for resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 27 Apr 2017 20:53:37 +0000 (23:53 +0300)]
rgw: remove RGWReshard::get_head
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 27 Apr 2017 20:40:08 +0000 (23:40 +0300)]
rgw: replace dout with ldout
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 27 Apr 2017 20:24:32 +0000 (23:24 +0300)]
rgw: remove cls_reshard_get_head
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 27 Apr 2017 20:13:52 +0000 (23:13 +0300)]
rgw: store resharding information in bucket header
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 20 Apr 2017 14:50:09 +0000 (17:50 +0300)]
rgw: handle correctly when there is no bucket or it doesn't have resharding flag
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 18 Apr 2017 15:50:32 +0000 (18:50 +0300)]
rgw: block on bucket index operation when resharding the index
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 18 Apr 2017 09:37:23 +0000 (12:37 +0300)]
rgw: add RGWReshard object to RGWRados
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 18 Apr 2017 09:30:07 +0000 (12:30 +0300)]
rgw: Add lock/unlock bucket instance and block_while_resharding methods
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 14 Apr 2017 11:35:54 +0000 (14:35 +0300)]
rgw: add cls_rgw_get_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 13 Apr 2017 09:47:33 +0000 (12:47 +0300)]
radosgw-admin: implement reshard execute command
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 13 Apr 2017 08:36:34 +0000 (11:36 +0300)]
radsogw-admin add create_new_bucket_instance
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Fri, 14 Apr 2017 10:00:39 +0000 (13:00 +0300)]
rgw: use const string for oid in get_bucket_instance_from_oid
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Mon, 10 Apr 2017 09:14:07 +0000 (12:14 +0300)]
rgw: add RGWReshard::clear_bucket_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 9 Apr 2017 18:29:17 +0000 (21:29 +0300)]
rgw: add RGWReshard::set_bucket_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 13 Apr 2017 15:44:33 +0000 (18:44 +0300)]
rgw: add missing handles for set_resharding, clear_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 6 Apr 2017 18:58:37 +0000 (21:58 +0300)]
rgw: add cls _bucket_resharding_set and cls_bucket_resharding_clear
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 6 Apr 2017 19:48:28 +0000 (22:48 +0300)]
rgw: Add resharding flag and the new bucket instance id to RGWBucketInfo
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 21 Mar 2017 13:57:58 +0000 (15:57 +0200)]
radosgw-admin: Add reshard add, list, execute and cancel commands
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Thu, 20 Oct 2016 14:30:54 +0000 (16:30 +0200)]
radosgw-admin: move resharding into a separate function
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Wed, 8 Mar 2017 15:56:39 +0000 (17:56 +0200)]
rgw: add RGWReshard class
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Sun, 12 Mar 2017 12:49:20 +0000 (14:49 +0200)]
rgw: add reshard_pool
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 28 Feb 2017 14:09:51 +0000 (16:09 +0200)]
rgw: add cls_rgw_reshard ops
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Casey Bodley [Fri, 2 Jun 2017 18:09:33 +0000 (14:09 -0400)]
Merge pull request #15119 from ceph/wip-rgw-config-docs
doc: mention certain conf vars should be in global
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yehuda Sadeh [Fri, 2 Jun 2017 16:16:07 +0000 (09:16 -0700)]
Merge pull request #14351 from yehudasa/wip-rgw-mdsearch
rgw: metadata search part 2
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Sage Weil [Fri, 2 Jun 2017 15:34:45 +0000 (10:34 -0500)]
Merge pull request #15427 from tchaikov/wip-vstart-osd-id
vstart: print "start osd.$id" instead of "start osd$id"
Reviewed-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 2 Jun 2017 15:32:15 +0000 (10:32 -0500)]
Merge pull request #15217 from badone/wip-async-sleep-timer-fix
osd: Move scrub sleep timer to osdservice
Reviewed-by: Josh Durgin <jdurgin@redhat.com>