rgw: framework for per-rgw sync status reporting

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

ceph_json: add deque encode/decode

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

radosgw-admin: replace region with zonegroup in the help file

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix assert on shutdown by keeping g_ceph_context around longer

RGWRealmWatcher and friends are on the stack in main(), so we can't drop
the last ref on g_ceph_context until their destructors have fired. use a
boost::intrusive_ptr to claim g_ceph_context's initial ref so that its
destructor fires last

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove epoch and duplicates from list_periods

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix compilation error in RGWPeriodPusher::handle_notify

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix compilation error on i386

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: call drain_all() before returning from some crs

spawned crs are still using marker_tracker, need to drain_all() as
we destroy it in the destructor.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: allocate a larger buffer for full sync omap oid

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: register RGWPeriodPusher with RGWRealmWatcher

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWPeriodPusher shares periods between zones/groups

RGWPeriodPusher implements the RGWRealmWatcher interface to get
notifications for new periods.  when it discovers that it needs to push
a period to other zones, it spawns a thread to send them (and keep
retrying) until all of those zones successfully acknowledge the period

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWRealmWatcher supports multiple watch types

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rename RGWRealmReloader::FrontendPauser to Pauser

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: period push succeeds even if it isn't a new period

when pushing periods between zones, we need a successful result to
signal that we can stop sending the period

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: refactor reconfigure out of RGWRealmWatcher

add a RGWRealmReloader class to handle reconfiguration, and use a
Watcher interface to get notifications from RGWRealmWatcher

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: keep Context for reconfig timer cancellation

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add RGWRealm::notify_new_period()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix typo in RGWPeriod::update

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix for decode of ZoneGroup id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: cr call() doesn't return a value

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: data sync marker tracker, don't assert

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: we need to rename the zone inside the zonegroup too

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix RGWSystemMetaObj::rename

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: add --zone-new-name to usage

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: init num shards if sync status did not exist

in data sync process

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: use correct connection for certain data sync operations

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: translate source_zone param into source zone id

operations are done using the zone id, not the zone name

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: start marker tracker sync entry can fail

We might hit duplicate entry, just skip it, don't assert.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: clarify log message

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fetch bucket instance info if doesn't exist

when doing data sync. This handles a race in which we try to sync data
of bucket instance before fetched its metadata

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: data sync fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't force pool names to start with periods

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: more backoff related changes and fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move code around

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: backoff mechanism for data sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: data sync CR retry after specific errors

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: initialize data sync if needed

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: full data sync, pass correct op

Need to specify appropriate op (depending on whether object is
versioned or not versioned).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: special handling of 'null' object version

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: initialize data member

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: keep timestamp stats in incremental sync position data

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: init REST connection by zone name

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: coroutine manager returns ECANCELED when going down

RGWRemoteMetaLog::run_sync() was getting stuck in an infinite loop after
a call to RGWCoroutinesManager::stop().  instead of returning success
when we see 'going_down', fail the coroutine with ECANCELED

the other RGWCoroutinesManager::run() overload also needed modification
to prevent us from overwriting this error code

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix compilation with older boost versions that don't have asio coroutines

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: remove include to <boost/asio/coroutine.h>

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: remove include to <boost/asio/coroutine.h>

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: show data full sync stats

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: keep sync progress info for full sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update the correct shards when doing full sync

The shard we use for each entry (either meta, or data) needs to be selected
correctly to match the different logs we keep. Otherwise async notifications
don't work correctly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: shutdown lease CR on mdlog inc sync failure

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: use boost::intrusive_ptr for http_op ref counting

this fixes a segfault in the RGWPostRESTResourceCR::request_complete()
error path, which accessed http_op after calling put()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: apply RGWRESTReadResource::wait_bl() fix to PostResource as well

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWRESTPostResource takes param_list_t for RGWPostRESTResourceCR

RGWPostRESTResourceCR needs to make a copy of the parameters given to
its constructor, because it doesn't use them until send_request(). this
results in a segfault when the given parameter list is on the stack

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add make_param_list() helper

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add typedef for rest parameter lists

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: use insert to copy containers in rgw_rest_conn

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: move RGWPostRESTResourceCR to rgw_cr_rest.h

for use by RGWRealmPusher

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWRestConn uses vector instead of map

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: workaround to clear location constrain when using default zonegroup

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: bucket ops, obj create and obj delete should check to see if bucket exists

rgw_build_policies set the bucket_exists flag

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: realm watcher locks mutex for SafeTimer::shutdown()

this fixes an assert() on shutdown

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix bucket creation and deletion when there is no realm or period configured

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: use default zonegroup and zone in init_complete

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: allow creating a bucket without current period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix data sync initialization

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move data sync init into the processing thread

that way we don't depend on remote peer to be available

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rework incremental md sync error handling

similar to what we do with the full md sync. Identify transient
errors, and if so return -EAGAIN so that caller would retry.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move children collection to other method

and call it again instead of just draining at the end

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: handle transient error in md full sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: coroutine drain fixes

Init drain_cr() before draining, otherwise we wouldn't be able to
call it a second time

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: RGWCoroutine::spawn() returns stack

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: a coroutine util function to collect a single stack

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: rework error handling when syncing a single md entry

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: get some errors from req_data

When reading request status, we need to check with req_data first,
since it might have failed before we even got the chance to set our
own status.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: multiple sync initialization and error handling fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: return -EAGAIN on a connection related http failure

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: cleanup

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: error out if failed to build md sync index

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: init sync env

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: mdlog full sync, yield for each entry

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move sync_oid() and shard_object_name() to sync env

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: create RGWMetaSyncEnv, use it as container to sync env

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: RGWPeriod stores sync_status as vector of strings

rgw_meta_sync_status keeps the shards in a std::map, but a std::vector
is more appropriate when all keys are present. the period also doesn't
need to store the full/incremental status in rgw_meta_sync_marker

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: store metadata sync status on period commit

store the new master zone's sync status with each new period

Signed-off-by: Casey Bodley <cbodley@redhat.com>

make: fix link of ceph_test_cls_rgw_meta

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add rgw_rest_realm.h to the Makefile

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add missing rgw_meta_sync_status.h to the Makefile

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: account total number of metadata entries for sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add backoff and restart to meta sync

also fix lease cr lifecycle

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix the mdlog sync lock control flow

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: use set_cr_error() and set_cr_done() instead of set_state()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: md full sync backoff

Still need to work this out correctly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: create a realm in case of pervious regions configuration

The realm name is concatanation of master region and master zone and its
id is the mds sume of the name

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add support for region format in RGWOp_ZoneGroupMap_Get

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update current_period after adding converted zonegroup

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add RGWRegionMap for backward compatability

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update realm_id when converting a region to zonegroup

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add converted regions into the current period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: RGWRealmWatcher to reload RGWRados on changes

adds RGWRealmWatcher, which watches the RGWRealm control object for
changes.  in a separate thread, all frontends are paused while the
RGWRados object is replaced with a new one

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: period commit calls notify_zone

Signed-off-by: Casey Bodley <cbodley@redhat.com>