rgw: set default value for env->get() call

Fixes: #13239
This fixes a regression introduced at commit abe4ec293d08b0314bf5c081ace2456073f3a22c.
The host var is a string, env->get() returns a char pointer, shouldn't
pass in NULL.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 0d5730efc0054334897315dc23ba04f30548e36e)
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

rgw: remove trailing :port from host for purposes of subdomain matching

Some clients (ahem, CrossFTP) include the :port in the HTTP_HOST header.
Strip it out.

Switch req_info field to a std::string and avoid copying it in preprocess.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit abe4ec293d08b0314bf5c081ace2456073f3a22c)

Merge pull request #5930 from liewegas/wip-11798-hammer

upstart: limit respawn to 3 in 30 mins (instead of 5 in 30s)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5908 from liewegas/wip-simple-connect-hammer

Pipe: Drop connect_seq increase line

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

upstart: limit respawn to 3 in 30 mins (instead of 5 in 30s)

It may take tens of seconds to restart each time, so 5 in 30s does not stop
the crash on startup respawn loop in many cases.  In particular, we'd like
to catch the case where the internal heartbeats fail.

This should be enough for all but the most sluggish of OSDs and capture
many cases of failure shortly after startup.

Fixes: #11798
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit eaff6cb24ef052c54dfa2131811758e335f19939)

Pipe: Drop connect_seq increase line

Revert commit 0fc47e267b6f8dcd4511d887d5ad37d460374c25.

When accepting and "connect.connect_seq == existing->connect_seq",
existing->state maybe STATE_OPEN, STATE_STANDBY or STANDY_CONNECTING.
This commit only fix partial problem and want to assert
"(existing->state == STATE_CONNECTING)".

So later we added codes to catch
"(existing->state == STATE_OPEN || existing->state == STATE_STANDBY)"
before asserting.

Backport: dumpling, firefly, giant

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
(cherry picked from commit 67225cb3ee1f6d274a02293724942bdb25cec8ca)

Merge pull request #5767 from dachary/wip-12597-hammer

Crash during shutdown after writeback blocked by IO errors

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5687 from liewegas/wip-hammer-feature-hammer

include/ceph_features: define HAMMER_0_94_4 feature

Merge pull request #5892 from ceph/wip-13060-hammer

osd: allow peek_map_epoch to return an error

Reviewed-by: David Zafman <dzafman@redhat.com>

osd/PG: peek_map_epoch: skip legacy PGs if infos object is missing

- pg is removed
- osd is stopped before pg is fully removed
- on restart, we ignore/skip the pg because its epoch is too old
- we upgrade to hammer and convert other pgs, skipping this one, and then
  remove the legacy infos object
- hammer starts, tries to parse the legacy pg, and fails because the infos
  object is gone, crashing.

The fix is to continue ignoring the zombie pg.

Fixes: #16030
Signed-off-by: Sage Weil <sage@redhat.com>

osd: allow peek_map_epoch to return an error

Allow PG::peek_map_epoch to return an error indicating the PG
should be skipped.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f15d9585edc5a12ac2d076951076247253b897c2)

[fixed *pepoch default of 0]
[fixed other return paths in peek_map_epoch]

Merge pull request #5769 from dachary/wip-12850-hammer

Crash during TestInternal.MultipleResize

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5768 from dachary/wip-12849-hammer

[  FAILED  ] TestLibRBD.BlockingAIO

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5697 from tchaikov/wip-12638-hammer

mon: add a cache layer over MonitorDBStore

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5381 from dachary/wip-12499-hammer

ceph-fuse 0.94.2-1trusty segfaults / aborts

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5757 from dachary/wip-12836-hammer

WBThrottle::clear_object: signal on cond when we reduce throttle values

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5759 from dachary/wip-12841-hammer

recursive lock of md_config_t (0)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5761 from dachary/wip-12843-hammer

long standing slow requests: connection->session->waiting_for_map->connection ref cycle

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5762 from dachary/wip-12844-hammer

osd suicide timeout during peering - search for missing objects

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5763 from dachary/wip-12846-hammer

osd/PGLog.cc: 732: FAILED assert(log.log.size() == log_keys_debug.size())

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5764 from dachary/wip-12847-hammer

common: do not insert emtpy ptr when rebuild emtpy bufferlist

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5373 from dachary/wip-12489-hammer

pg_interval_t::check_new_interval - for ec pool, should not rely on min_size to determine if the PG was active at the interval

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5383 from dachary/wip-12504-hammer

rest-bench common/WorkQueue.cc: 54: FAILED assert(_threads.empty())

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5765 from dachary/wip-12883-hammer

cache agent is idle although one object is left in the cache

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5754 from dachary/wip-12588-hammer

Change radosgw pools default crush ruleset

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5377 from dachary/wip-12396-hammer

register_new_pgs() should check ruleno instead of its index

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5758 from dachary/wip-12839-hammer

Mutex Assert from PipeConnection::try_get_pipe

tests: tiering agent and proxy read

Verify that an object promoted to a cache tier because of a proxy read
is evicted as expected.

http://tracker.ceph.com/issues/12673 Refs: #12673

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 7924231930732bd297d3bd034c8295e96cb81088)

osd: trigger the cache agent after a promotion

When a proxy read happens, the object promotion is done in parallel. The
agent_choose_mode function must be called to reconsider the situation
to protect against the following scenario:

  * proxy read
  * agent_choose_mode finds no object exists and the agent
    goes idle
  * object promotion happens
  * the agent does not reconsider and eviction does not happen
    although it should

http://tracker.ceph.com/issues/12673 Fixes: #12673

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit e1f58feb9b1d20b72f2eb2eefdea5982e0cddccd)

Merge pull request #5276 from dachary/wip-11824-hammer

implicit erasure code crush ruleset is not validated

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5382 from dachary/wip-12500-hammer

segfault launching ceph-fuse with bad --name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5367 from dachary/wip-12311-hammer

read on chunk-aligned xattr not handled

Reviewed-by: Sage Weil <sage@redhat.com>

lockdep: allow lockdep to be dynamically enabled/disabled

librbd test cases attempt to enable lockdep coverage via the librados
API.  Use a configuration observer to register/unregister lockdep
support.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit af0cade0293be22e74a3293765676c7b227b06c4)

tests: librbd API test cannot use private md_config_t struct

Remove all depencencies on md_config_t and instead use librados API
methods to get/set configuration values.

Fixes: #12479
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2c51aadb447d3447b5dace19c5cb6268e94247c8)

tests: ensure old-format RBD tests still work

Override the RBD default image format back to version 1
to ensure tests properly cover the old format.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3b95edb5bd373bd05f83516eccfd30a6b5cf4693)

librados_test_stub: implement conf get/set API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4d03c664f2a9362009c7e3077ed65e2989f0f21b)

crypto: use NSS_InitContext/NSS_ShutdownContex to avoid memory leak

Switched to context-aware NSS init/shutdown functions to avoid conflicts
with parent application.  Use a reference counter to properly shutdown the
NSS crypto library when the last CephContext is destroyed.  This avoids
memory leaks with the NSS library from users of librados.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9fa0112dd9286178da1d76020158503b2062d252)

auth: use crypto_init_mutex to protect NSS_Shutdown()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit cf058174fe6be26129eb52f4fb24d0f7146ef7af)

auth: reinitialize NSS modules after fork()

Fixes: #11128
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 4c24d0cc074462ae258b5cf901cd884bb4f50a53)

librbd: prevent race condition between resize requests

It was possible that the same resize request could be sent twice
if a completed resize op started a newly created resize op while
it was also being concurrently started by another thread.

Fixes: #12664
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6ec431bac1cf9446f3af90209d193dfcf003f2c5)

librbd: Add a paramter:purge_on_error in ImageCtx::invalidate_cache().

If bh_write met error, it will try again. For closing image, if met this
issue, it will trigger a assert:
>>2015-02-03 15:22:49.198292 7ff62d537800 -1 osdc/ObjectCacher.cc: In
function 'ObjectCacher::~ObjectCacher()' thread 7ff62d537800 time
>>2015-02-03 15:22:49.195927osdc/ObjectCacher.cc: 551: FAILED
>>assert(i->empty())

Now add purge_on_error, when shutdown_cache it set true.
In ImageCtx::invalidate_cache, if met error and purge_on_error is true,
purge the dirty bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 35def5c81f7fc83d55d18320e4860c6a63d4c7f5)

Conflicts:
src/librbd/ImageCtx.cc : trivial resolution

librbd: Remvoe unused func ImageCtx::read_from_cache.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 101440a41253680770f94bc380af7072c7adaebf)

osdc: clean up code in ObjectCacher::Object::map_write

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 9f80c2909ace09cd51c24b49c98a093e0e864dca)

osdc: Don't pass mutex into ObjectCacher::_wait_for_write.

Because the mutex is the same as ObjectCacher::lock.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit d7cf7aeea5cba1ffa8e51ff1ad2424b1ec161a12)

osdc: After write try merge bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 1a48a8a2b222e41236341cb1241f0885a1b0b9d8)

osdc: Make last missing bh to wake up the reader.

Avoid wakeup early and wait again.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit d582bda090b3339d03e25b0e6d0971ad0267f476)

osdc: For trust_enoent is true, there is only one extent.

Now the judgement only in conditon which will return -ENOENT.
But o->exists don't depend on the extent size. It only depend on
trust_enoent. So move this judgement at the first of _readx().
Make this bug ASAP occur.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 2449fddc13b5ce8bbf1bb4ecaa5d6937f54e54d1)

osdc: In _readx() only no error can tidy read result.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 540346d4a901d8041c9fd74641c98cdfd2e1ab32)

Merge pull request #5223 from SUSE/wip-12305-hammer

ceph.spec.in: running fdupes unnecessarily

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5716 from dachary/wip-12851-hammer

Ensure that swift keys don't include backslashes

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #5717 from dachary/wip-12591-hammer

rgw: create a tool for orphaned objects cleanup

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #5755 from dachary/wip-12589-hammer

ceph-disk zap should ensure block device

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5732 from ceph/wip-11455-hammer

rgw: init some manifest fields when handling explicit objs

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5721 from dachary/wip-12853-hammer

RGW Swift API: X-Trans-Id header is wrongly formatted

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #5498 from ceph/wip-12432-hammer

rgw: HTTP return code is not being logged by CivetWeb

Reviewed-by: Loic Dachary <ldachary@redhat.com>

OSD: break connection->session->waiting message->connection cycle

Fixes: #12338
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 7f15e22be291df139c85909b96b2366f6dbadf37)

osd/PGLog: dirty_to is inclusive

There are only two callers of mark_dirty_to who do not pass max,
and they are both in the merge_log extending tail path.  In that
case, we want to include the last version specified in the log
writeout.  Fix the tail extending code to always specify the
last entry added, inclusive.

Fixes: #12652
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f0ca14df0641daa04eee39d98d8bd0faf46e4e6d)

common: fix code format

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit 15e5ebe14787bbbc6b906d72ccd2d418d762cfad)

test: add test case for insert empty ptr when buffer rebuild

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit 2d2f0eb338906742d516a2f8e6d5d05505be8f47)

common: fix insert empty ptr when bufferlist rebuild

Fixes: #12775
Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
(cherry picked from commit fb1b6ddd7bfe40d4a38d5ec59b1c3f59f2a83f23)

Conflicts:
src/common/buffer.cc : because invalidate_crc did not exist
        in hammer

Merge pull request #5527 from SUSE/wip-12585-hammer

OSD crash creating/deleting pools

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: copy the RecoveryCtx::handle when creating a new RecoveryCtx instance from another one

Fixes: 12523
Signed-off-by: Guang Yang <yguang@yahoo-inc.com>
(cherry picked from commit 6ab9efe799b947f2ef6113df8cc542c73760107d)

config: skip lockdep for intentionally recursive md_config_t lock

lockdep can't handle recursive locks, resulting in false positive
reports for certain set_val_or_die() calls, like via
md_config_t::parse_argv() passed "-m".

Fixes: #12614
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 415226995ebe841cc9e8639a3233150a75aecaed)

osd: Keep a reference count on Connection while calling send_message()

Fixes: #12437
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a140085f467889f2743294a3c150f13b62fcdf51)

WBThrottle::clear_object: signal if we cleared an object

Fixes: #12223
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8447b08d60d5cf7dce20f8b839cc2cb5a93782de)

ceph-disk: always check zap is applied on a full device

zap only makes sense on a full device and this is verified for

    ceph-disk prepare --zap-disk

but not for

    ceph-disk zap

Unify the two and add a test to check that it fails as expected when
trying to zap a directory.

http://tracker.ceph.com/issues/11272 Fixes: #11272

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 8e1bcf21f090fc98ac9b495d76614e3012f5872a)

Conflicts:
src/test/ceph-disk.sh : trivial resolution

librados: Make librados pool_create respect default_crush_ruleset

Make pool_create in librados use the osd_pool_default_crush_replicated_ruleset

Fixes: #11640
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
(cherry picked from commit da96a89033590277460aef1c80f385bd93d625e1)

Merge pull request #5551 from ceph/wip-corpus-hammer

ceph-object-corpus: add 0.94.2-207-g88e7ee7 hammer objects

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph-object-corpus: add 0.94.2-207-g88e7ee7 hammer objects

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: init some manifest fields when handling explicit objs

Fixes: #11455
When dealing with old manifest that has explicit objs, we also
need to set the head size and head object correctly so that
code that relies on this info doesn't break.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit d7bf8cb594e5276d1c80544f5ec954d52b159750)

mon: test the crush ruleset when creating a pool

We want to fix the following scenario:

 * an erasure code plugin (or another part of the code) creates a
   ruleset
 * the ruleset crashes during mapping (for whatever reason)
 * ceph osd pool create uses the bugous ruleset
 * the monitors try to do mapping a crash

Having a bugous ruleset in the crush map is very difficult prevent. The
catastrophic event of using it with a newly created pool can however be
prevented by calling the CrushTester just before creating the pool and
after all implicit or explicit crush ruleset creation happened.

http://tracker.ceph.com/issues/11814 Fixes: #11814

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit f1e86be589803596e86acc964ac5c5c03b4038d8)

Conflicts:
src/test/mon/osd-crush.sh
          removed the run_mon because in hammer it is shared
          between all tests
        src/mon/OSDMonitor.cc
          prepare_new_pool changed stringstream to *ostream

erasure-code: set max_size to chunk_count() instead of 20 for shec

See 8b64fe93b088a3a33d357869c47e6bf928c3f0e4 for a detailed explanation.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit a44499f7e6609d9a32254dc6f4303122de74db31)

vstart.sh: set PATH to include pwd

ceph-mon needs crushtool to be in PATH. Don't set if it is run
from ceph_vstart_wrapper, which already sets it as it needs.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit e08bccf377292d72463796ae1dff5d8be40d88e9)

Merge pull request #5365 from dachary/wip-12491-hammer

buffer: critical bufferlist::zero bug

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #5369 from dachary/wip-12498-hammer

get pools health'info have error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5370 from dachary/wip-12496-hammer

pgmonitor: wrong at/near target max“ reporting

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5378 from dachary/wip-12394-hammer

Memory leak in Mutex.cc, pthread_mutexattr_init without pthread_mutexattr_destroy

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5372 from dachary/wip-12493-hammer

the output is wrong when runing ceph osd reweight

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: rework X-Trans-Id header to be conform with Swift API.

Fixes: #12108
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
(cherry picked from commit d1735a4e8ea2056d333a5001a7a410fc03fe1b9e)

Transaction Id added in response

Signed-off-by: Abhishek Dixit dixitabhi@gmail.com
(cherry picked from commit b711e3124f8f73c17ebd19b38807a1b77f201e44)

Conflicts:
src/rgw/rgw_rest.cc
             the prototype of the end_header( function
             in the context of the diff changed

rgw: api adjustment following a rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 7cf1f376e7dbc98a44f4a8e78bfa11f84291a941)

rgw: orphans, fix check on number of shards

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 47edec359be2a4d230a4a54c0c670d0aa711fa1e)

rgw: orphans, change default number of shards

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5528f21d0861d42378a8eb10759a1e55ddd66c09)

rgw: change error output related to orphans

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit cac57caf639236adc9a641a85352eb729b58725a)

rgw: orphan, fix truncated detection

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit d8ce04db8542dae0027423f3c50355e5de74dc9a)

radosgw-admin: simplify orphan command

No need for --init-search

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 4980cbd04ff650d79a033bc5298e6c1abe985133)

radosgw-admin: stat orphan objects before reporting leakage

We don't want to report new objects as leaked, because they might just
be written, and waiting to be linked to a new object. The number of
seconds prior to the job initialization can be cofigured through
--orphan-stale-secs (by default it's 24h).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 66b0090023859d97da2bbaca698b6a50c225abd0)

radosgw-admin: orphans finish command

A command to remove orphans data

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 55d6f5ecf18f532c7f056f8b60c101843594b00c)

rgw: cannot re-init an orphan scan job

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f8993102731518153bb18eb74c3e6e4943a5dbac)

rgw: stat_async() sets the object locator appropriately

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit d7d117121858155acc8548908e9459a48c676dd1)

rgw: list_objects() sets namespace appropriately

list_objects() now uses parse_raw_oid(), so that it can set the correct
namespace. It only affects users of the function that want to get all
objects in bucket, regardless to the namespace associated with it. This
makes it so that the orphan code actually works now with namespaced
objects, and with special named objects (namely, start with underscore).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 20bd490bebf9be3553d4e25322c9544b0b090086)

rgw: modify orphan search fingerprints

Now works with multipart uploads.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit fa61ea2ccc94f04afb39868ac34abc264806d8d4)

rgw: compare oids and dump leaked objects

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1bc63d98ff8a3aa180065153690b4f8a73658b79)

rgw: keep accurate state for linked objects orphan scan

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f19b2f087cfb5a89edf3b19487c0df3a214e350f)

rgw: iterate over linked objects, store them

only keep part of the oid name if it is in a namespace.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 75902fdb12ce0413d3beb455df47134f3d7386e7)

rgw: add rgw_obj::parse_raw_oid()

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 7c1aa8373a090d5c1b2aa121582547723120a079)

rgw: iterate asynchronously over linked objects

Read objects manifest. So that we could keep the relevant info later.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 05a953d965a0a6a0e9feddaa62f7429e154e828e)

rgw: async object stat functionality

An async functionality that stats object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 13adf3cf4f5fd8a0feb7cc1d7a4ee4ded9f573db)

rgw-admin: build index of bucket indexes

for the orphan search tool

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 16a2dbd838b69fb445c06122a68fc65ec3a7b3de)

rgw: initial work of orphan detection tool implementation

So far doesn't do much, iterate through all objects in a specific pool
data, store it in a sharded index.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 767fc29aa62a0a4e4dd8a78785576e8f1b44ee5b)