rgw: update realm_id when converting a region to zonegroup

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add converted regions into the current period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: RGWRealmWatcher to reload RGWRados on changes

adds RGWRealmWatcher, which watches the RGWRealm control object for
changes.  in a separate thread, all frontends are paused while the
RGWRados object is replaced with a new one

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: period commit calls notify_zone

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add realm control object for watch/notify

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add pause/unpause to RGWFrontend interface

RGWProcessFrontend implements pause/unpause through the ThreadPool
interface. RGWMongooseFrontend implements them with a reader/writer
lock that blocks the civetweb_callback() until unpause

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rwlock: add prioritize_write flag to prevent write starvation

RWLock's constructor takes an additional 'prioritize_write' flag, which
defaults to false.  when enabled, this adds the rwlock attribute
PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP, which will block calls to
pthread_rwlock_rdlock() if there are any writers waiting.  this avoids
starvation of pthread_rwlock_wrlock() in read-mostly workloads

also made the destructor non-virtual; this was adding a virtual function
table to the class, even though there are no other virtual functions.
added the 'final' keyword to document that it's not meant to be derived
from

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWHTTPManager dtor stops background thread

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: remove period activate in favor of commit

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add --commit flag for period update

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add 'period commit' command

on the master zone, store the new period locally and notify the zone for
restart.  on other zones, send a 'period push' to the master zone with
an empty period id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: period push handler supports commit

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add RGWPeriod::commit()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: period push handler updates current period/epoch

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWPeriod accessors are const

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add realm pull command

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add send_to_remote_or_url() helper

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add op for GET /admin/realm

this is for the radosgw-admin 'realm pull' command, which gets the realm
object by name or id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add realm params to period pull

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: move period Ops to rgw_rest_realm.cc

this removes the handling of /admin/config?type=period

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: period allows setting id, epoch, latest_epoch

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: omit epoch in oid for staging period

there should only be one staging period object, so the epoch is
an unnecessary part of its oid. and for period commit, we need to
remember the epoch of the predecessor period to make sure we're not
based on an old epoch

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add 'period update' to usage

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWRemoteMetaLog ctor sets async_rados=nullptr

RGWRemoteMetaLog::init() is not called in all error paths, so we need to
start with nullptr

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: fix comparison typo in RGWRados::init_complete

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWHandler_Config should assume empty type is zonegroupmap

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: guard metadata full/incremental sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: guard metadata sync initialization step

Guard the fetch-meta stage

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: a bit better coroutine related logging

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: initialize rest connection if no period

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: multiple init metadata sync locking fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: backward compatability update_name for existing zones

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update new name after converting region to zonegroup

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: fix zone get command

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: use default zone in case there is no period or zone name

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: RGWRados zonegroup and zone_public_config should not be accessed directly

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: RGWRados internal zone should be protected and not accessed directly

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix RGWSystemMetaObj::init by passing old_format flag

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: convert regionmap

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add support to zonegroupmap set

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: zonegroupmap get should no error if no realm or period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: no need to store or update zonegroupmap

Structure and command will be kept for backward compatability

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: remove zonegroup_map from RGWRados

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: move link dependencies to rgw_a

Signed-off-by: Casey Bodley <cbodley@redhat.com>

CMake - add check for libcurl

Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>

rgw: don't read/init sync status if meta master

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix log message

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: can set zone read-only, can set zonegroup id

and other fixes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update rest connections from the current period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: read local zonegroup and zone only when needed

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: use current_period zonegroup and zone when exists

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: read current period during init

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: zonegroup and zone should not be accessed directly

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: get_predefined_name should get the ceph context

so it can be called before object init

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: can override master zone config

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: update zone set

zone set was broken, now it can update the zone params correctly. Also
inheriting the id from the current zone if exists, or uses the zone
name.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

radosgw-admin: zone add command should be zonegroup add

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: run sync thread only when number of zones > 2 and master is configured

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update master zone

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: stuff more info into the period

and don't update the current period when adding zone, this
will be done through the staging period.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: period update command

A command that creates staging period data out of current local data

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: auto create placement target params

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: create default zonegroup for first time run of the gateway

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: --rgw-zone and --rgw-zonegroup use names not id

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw_admin: can config access/secret key for zone

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: can set zonegroup api name

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: can update zonegroup endpoints, is_master

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: set first created zone as default

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: prefix default pools with zone name

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: set first created realm as default

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: period oid doesn't include realm id

So that realm id is not a dependency for reading period.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: period get does not require specifying period id

Can return the current period

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: use --rgw-realm instead of --realm

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: realm set-default -> realm default

to be in line with the other default commands

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: zone delete doesn't require zonegroup

iterates through all the zonegroups and remove it

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: zone name need to be provided on zone create

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: don't require zonegroup id on some commands

should be able to get the default one

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't use --zonegroup, use --rgw-zonegroup

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: a zone default command, and other changes

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: don't use --zone, use --rgw-zone

--rgw-zone is tied to ceph.conf, and what we've always been using

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: update zonegroup endpoints when creating it

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update zone endpoints when creating a new zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update rest_conn_master when adding a new master zonegroup

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: zone get and rename commands should work without zonegroup id or name

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: check for valid input in zone delete command

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: check for valid input in zone add command

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: check for valid input for zonegroup modify command

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: check for legal input for zonegroup default command

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: check legal input for zone modify command

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: use realm_id from zonegroup if not provided

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix RGWZone::dump

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add zone modify command to change master zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add zonegroup modify command to allow setting/clearing zonegroup master status

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: fix zonegroup and master usage text

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: remove is_master flag from zoneparams

make it easier to change master zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: print zone name and id in the begining of the dump

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: fix zone create to check for zonegroup name

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: store latest sync status when activating a period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add decode_json method to rgw_meta_sync_status structures

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: move rgw_meta_sync_status, rgw_meta_sync_marker and rgw_meta_sync_info into a sperate h file

Signed-off-by: Orit Wasserman <owasserm@redhat.com>