mgr/cephadm: avoid repeated calls to get_module_option

We already stash these as MgrModule members.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #43936 into master

* refs/pull/43936/head:
qa/tasks/cephadm: pull image to all hosts in parallel
qa/tasks/cephadm: add hosts via mon remote
qa/tasks/cephadm: use shortname for remote directory
qa/tasks/cephadm: deploy no more than 5 mons in roleless mode
qa/tasks/radosbench: default clients to all clients (not client.0)
qa/tasks/ceph_manager: parallelize flush_pg_stats()
qa/suites/big: remove thrasher
qa/suites/big: update for cephadm

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge PR #44080 into master

* refs/pull/44080/head:
mgr/cephadm: record when finished with scheduled daemon action

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #44106 from sebastian-philipp/mgr-tox-37

mgr/tox.ini: Add python 3.7 environment

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #43943 from sebastian-philipp/osd-memeory-hyperconverged

doc/cephadm: OSD memory autotuning for hyperconverged

Reviewed-by: Adam King <adking@redhat.com>

qa/tasks/cephadm: pull image to all hosts in parallel

This doesn't affect bootstrap, but it does mean we avoid any delay
the first time we cephadm.shell on some non-boostrap host.

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/cephadm: add hosts via mon remote

If we use a new remote for each shell command, we end up waiting
for the image to pull on every host in sequence.

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/cephadm: use shortname for remote directory

This aligns with what the ceph and syslog tasks do.

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/cephadm: deploy no more than 5 mons in roleless mode

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/radosbench: default clients to all clients (not client.0)

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/ceph_manager: parallelize flush_pg_stats()

Signed-off-by: Sage Weil <sage@newdream.net>

qa/suites/big: remove thrasher

This doesn't work with roleless (yet)

Signed-off-by: Sage Weil <sage@newdream.net>

qa/suites/big: update for cephadm

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm/tests: remove `_deploy_cephadm_binary`

(not needed)

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/tox.ini: Add python 3.7 environment

Plus fixes.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #43774 from aclamk/fix-bluefs-truncate

Fix data corruption in bluefs truncate()

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #43875 from liewegas/ceph-cli-better-help

ceph: make -h/--help show match when some args are supplied

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #44092 from sebastian-philipp/cephadm-docs-deployment-scenarios

doc/cephadm: Cephadm docs deployment scenarios

Reviewed-by: Adam King <adking@redhat.com>

doc/cephadm: deployment scenarios single host and isolated environment

This PR adds a deployment scenarios section to the cephadm docs to document the single-host-defaults flag, and explain how to deploy in an isolated environment.

Signed-off-by: Melissa Li <melissali@redhat.com>

doc/cephadm: isolated environment and other deployment scenarios

This PR adds a section to the cephadm docs to describe how to install cephadm in different deployment scenarios (set cluster on single host, and deployment in an isolated environment or private network).

Signed-off-by: Melissa Li <melissali@redhat.com>

Merge pull request #43905 from rhcs-dashboard/fix-53242-master

mgr/dashboard: dashboard does not show degraded objects if they are less than 0.5% under "Dashboard->Capacity->Objects block

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44023 from rhcs-dashboard/kcli-expanded-monitoring

mgr/dashboard: cephadm e2e start script: --expanded: deploy monitoring stack

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44064 from MrFreezeex/fix-statusupdater-utest

rbd-mirror: make RemoveImmediateUpdate test synchronous

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #44045 from rhcs-dashboard/upgrade-cypress

mgr/dashboard: upgrade Cypress to the latest stable version

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44019 from liu-chunmei/crimson-background-recovery

crimson/osd: add delay for background_recovery

reviewed-by:  Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #43989 from cfsnyder/wip-53308

osd/OSDMap.cc: clean up pg_temp for nonexistent pgs

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #44036 into master

* refs/pull/44036/head:
.github/pull_request_template: drop teuthology reference
.github/pull_request_template: add cleanup option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

mgr/cephadm: record when finished with scheduled daemon action

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #44061 from Matan-B/wip-matanb-doc-teuthology

doc/dev: adding Teuthology suggested resources

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #43996 from rhcs-dashboard/predefined-labels

mgr/dashboard: Predefine labels in create host form

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

rbd-mirror: make RemoveImmediateUpdate test synchronous

Try fixing sporadic failure linked in the tracker in
TestMockMirrorStatusUpdater.RemoveImmediateUpdate by making it
synchronous.

Fixes: https://tracker.ceph.com/issues/53375
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>

mgr/dashboard: upgrade Cypress to the latest stable version

- Remove unneeded dependency that was causing UI performance issues: zone.js
- Ignore 'ResizeObserver loop limit exceeded' error.
- run-frontend-e2e-tests.sh refactoring: create rgw dashboard user through
  'ceph dashboard set-rgw-credentials' and use it on rgw buckets' tests.

Fixes: https://tracker.ceph.com/issues/53357
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #43244 from ronen-fr/wip-rf-scrub-command

osd: make 'pg deep-scrub' command initiate a scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #43722 from lxbsz/caps_doc

doc: update the capabilities doc for cephfs

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/dev: adding Teuthology suggested resources

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #43982 from guits/refactor_cv_human_readable_func

ceph-volume: human_readable_size() refactor

Merge pull request #44007 from tchaikov/wip-cmake-python3.10

cmake: check for python(\d)\.(\d+) when building boost

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43992 from rhcs-dashboard/flaky-inventory-test-fix

mgr/dashboard: fix flaky inventory e2e test

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43958 from rhcs-dashboard/daemon-event-padding

mgr/dashboard: Daemon Events listing using bootstrap class

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #43866 from rhcs-dashboard/add-hint-provisioned-images

mgr/dashboard: provisioned values is misleading in RBD image table

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #43843 from cbodley/wip-test-cls-rgw-stats

test/cls/rgw: add index transaction simulator to model bucket stats

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #43888 from mgfritch/cephadm-expect-hostname

cephadm: fixup expect-hostname message

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #43873 from guits/add_shared_folder_shell_cmd

cephadm: add --shared_ceph_folder to shell cmd

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #43876 from sebastian-philipp/all-osd-at-once

mgr/cephadm: create osds at all hosts at once

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/dashboard: dashboard does not show degraded objects if they are less than 0.5% under "Dashboard->Capacity->Objects block

This PR is intended to fix this issue

Fixes: https://tracker.ceph.com/issues/53242
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #44033 from ljflores/wip-update-email-id

mailmap: add Laura Flores

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: ljflores <NOT@FOUND>
Reviewed-by: neha-ojha <NOT@FOUND>

Merge pull request #43795 from myoungwon/wip-paddr-split

seastore: generalize paddr_t

Reviewed-by: Samuel Just <sjust@redhat.com>

mgr/dashboard: Predfine labels in create host form

Also retains the previously created labels by user in the form

Fixes: https://tracker.ceph.com/issues/53315
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix flaky inventory e2e test

When `inventory.getTableCount('total').should('be.eq', totalDiskCount);`
this line is executed the table was not loaded properly and hence the
getTableCount returns 0 on the first try but on second try it passes
since the table is loaded. But in orch e2es the retries are set to 0. I
am not sure if it makes sense to set it to 1. Anyway I am adapting the
test a bit to expect the count to be equal to totalDiskCount so that the
test will wait a bit.

Fixes: https://tracker.ceph.com/issues/53353
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #43524 from Rethan/feat-expiration-time

rbd: when trash mv, show expiration time if it's not now

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #43852 from hualongfeng/show_feature

tools/rbd: make rbd info display dirty-cache feature

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #43907 from cybozu/rbd-correct-encoding-of-snap-protection-record-in-exporting

rbd: correct encoding of snap protection record in exporting image

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>

.github/pull_request_template: drop teuthology reference

It is not clear what role this has relative to the needs-qa label.

Signed-off-by: Sage Weil <sage@newdream.net>

.github/pull_request_template: add cleanup option

Signed-off-by: Sage Weil <sage@newdream.net>

seastore: generalize paddr_t

Due to necessity to support both segment and rbm addrs,
paddr_t needs to be generlized.
To do so, this commits introduces seg_paddr_t---
seg_paddr_t inherits paddr_t, and deals with common_addr
according to the semantic of segment.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #43999 from kamoltat/wip-autoscale-profile-scale-up-default

pybind/mgr/pg_autoscale: revert to default profile scale-up

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #43974 into master

* refs/pull/43974/head:
qa: disable metrics on kernel client during upgrade

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

mailmap: add Laura Flores

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #43987 from rhcs-dashboard/53123-dashboard-nfs-cleanup

mgr/dashboard: NFS non-existent files cleanup

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: ljflores <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43983 from rhcs-dashboard/rgw-add-realm-column

mgr/dashboard: rgw daemon list: add realm column

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>

Merge pull request #43824 from cbodley/wip-qa-rgw-upgrade-octopus-multisite-cv

qa/upgrade: rgw multisite upgrade test excludes ceph-volume

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #43764 from gregsfortytwo/wip-rbd-crash-consistency

doc: fix up rbd snapshot docs around crash consistency

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

pybind/mgr/pg_autoscale: revert to default profile scale-up

pg_autoscale module will now start out all the pools
with a scale-up profile by default.

Added tests in workunits/mon/pg_autoscaler.sh
to evaluate if the default pool creation is
a scale-up profile

Updated documentation and release notes to
reflect the change in the default behavior
of the pg_autoscale profile.

Fixes: https://tracker.ceph.com/issues/53309
Signed-off-by: Kamoltat <ksirivad@redhat.com>

qa: disable metrics on kernel client during upgrade

v16.2.4 MDS triggers an assert from these messages.

Also: add latest pacific for extra coverage.

Fixes: https://tracker.ceph.com/issues/53293
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/rgw: run ceph_test_cls_rgw_stats in rgw/verify suite

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: model multipart uploads in ceph_test_cls_rgw_stats

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw: add ceph_test_cls_rgw_stats simulator

a simulator that generates lots of create/delete operations against a
set of object names, and tracks the expected bucket stats for comparison

each operation has a 70% chance to complete successfully, a 10% chance to
cancel, 10% chance to suggest update, and 10% chance to suggest removal

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: add some missing equality comparisons

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: RGWObjCategory to string

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: RGWModifyOp <-> string conversions

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: move stream operator from rgw_common.h

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: input parameters are const

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: add rgw_bucket_dir_stats alias

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41557 from ifed01/wip-ifed-better-daemonperf

os/bluestore: improve usability for bluestore/bluefs perf counters

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Laura Flores lflores@redhat.com

mgr/dashboard: cephadm e2e start script: deploy monitoring stack when --expanded

Fixes: https://tracker.ceph.com/issues/53333
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #43886 from nmshelke/doc-fix-53054

doc: prerequisites fix for cephFS mount

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #44008 from rzarzynski/wip-crimson-leaky-objectcontextregistry

crimson/osd: fix leaks of ObjectContext in the registry.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #43326 from pdvian/wip-doc-config-correction

doc/dev/config: Replace invalid config debug-pg

Reviewed-by: Neha Ojha <nojha@redhat.com>

crimson/osd: add delay for background_recovery

to avoid interuptor nested.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

Merge pull request #43980 from liu-chunmei/crimson-peerevent-nested

crimson: add delay for peering_event start when nested

reviewed-by : Samuel Just <sjust@redhat.com>,
                       Kefu Chai <tchaikov@gmail.com>,
                       Xuehan Xu <xxhdx1985126@gmail.com>,
                       Radoslaw Zarzynski <rzarzyns@redhat.com>.

crimson/osd: fix leaks of ObjectContext in the registry.

The patch is supposed to fix the following problems (extra
debugs onboard):

```
NFO  2021-11-16 01:18:38,713 [shard 0] osd - ~OSD: OSD dtor called
INFO  2021-11-16 01:18:38,713 [shard 0] osd - Heartbeat::Peer: osd.6 removed
INFO  2021-11-16 01:18:38,714 [shard 0] osd - Heartbeat::Peer: osd.5 removed
INFO  2021-11-16 01:18:38,714 [shard 0] osd - Heartbeat::Peer: osd.2 removed
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ShardServices: ShardServices dtor called
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: ShardServices dtor called; unref_size=3, size=3
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: unreferenced p=0x619000115380
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: unreferenced p=0x619000114980
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: unreferenced p=0x619000112680
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: set p=0x619000114980
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: set p=0x619000115380
INFO  2021-11-16 01:18:38,714 [shard 0] osd - ~ObjectContextRegistry: set p=0x619000112680
INFO  2021-11-16 01:18:38,738 [shard 0] osd - crimson shutdown complete

=================================================================
==33351==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2808 byte(s) in 3 object(s) allocated from:
    #0 0x7fe10c0327b0 in operator new(unsigned long) (/lib64/libasan.so.5+0xf17b0)
    #1 0x55accbe8ffc4 in ceph::common::intrusive_lru<ceph::common::intrusive_lru_config<hobject_t, crimson::osd::ObjectContext, crimson::osd::obc_to_hoid<crimson::osd::ObjectContext> > >::get_or_create(hobject_t const&) (/usr/bin/ceph-osd+0x3b000fc4)

Objects leaked above:
0x619000112680 (936 bytes)
0x619000114980 (936 bytes)
0x619000115380 (936 bytes)
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: check for python(\d)\.(\d+) when building boost

now that python3.10 has two digits as its minor version, we should be
able to match it.

this change fixes the build with python3.10. without this change, we'd
have

error: wrong library name 'python310' in the --with-<library> option.

when building with python3.10

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

mgr/cephadm: use asyncio.gather to parallelize things

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: make OSDService.create_from_spec_one async

And gather all results. Plus a lot of mechanical adjustments

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: Add CephadmOrchestrator.wait_async()

less typing

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #43409 from linuxbox2/wip-rgwadmin-logtest

qa/rgw: use local runner with cmdline radosgw_admin.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35100 from soumyakoduri/cloudtiering

rgw/CloudTransition: Transition objects to cloud endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42780 from ronen-fr/wip-ronenf-unique-scrub

osd/scrub: mark PG as being scrubbed, from scrub initiation to Inactive state

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #43880 into master

* refs/pull/43880/head:
mgr/cephadm: turn off asyncssh debug output

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #43093 from zdover23/wip-doc-2021-09-09-rados-bootstrap-options-2

doc/rados: update mon_host & friends options

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Daemon Events listing using bootstrap class

Fixes: https://tracker.ceph.com/issues/53282
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #43969 from sebastian-philipp/core

doc/cephadm: core dumps

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

rgw/CloudTransition: Replace Coroutines with RGWRestConn APIs

To avoid the overhead of using coroutines during lifecycle transition,
RGWRESTStream* APIs are used to transition objects to remote cloud.

Also handled few optimizations and cleanup stated below:
* Store the list of cloud target buckets as part of LCWorker instead
  of making it global. This list is maintained for the duration of
  RGWLC::process(), post which discarded.
* Refactor code to remove coroutine based class definitions which are no
  longer needed and use direct function calls instead.
* Check for cloud transitioned objects using tier-type and return error if
  accessed in RGWGetObj, RGWCopyObj and RGWPutObj ops.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/RGWRESTConn: Define a wrapper to send PUT/POST stream request

Similar to "get_resource()", add an API "send_resource()" to send
PUT/POST/DELETE Stream request on RGWRestConn

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/CloudTransition: Include aws region name for remote endpoint

With commit#81ad226, aws auth v4 rquires region name for remote
endpoint connection. Include the same in the tier parameters.

& misc fixes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/CloudTransition: Handle versioned objects

For versioned and locked objects, similar semantics as that of LifecycleExpiration are applied as stated below -

If the bucket versioning is enabled and the object transitioned to cloud is
 - current version, irrespective of what the config option "retain_object" value is, the object is not deleted but instead delete marker is created on the source rgw server.
 - noncurrent version, it is deleted or retained based on the config option "retain_object" value.

If the object is locked, and is
 - current version, it is transitioned to cloud post which it is made noncurrent with delete marker created.
 - noncurrent version, transition is skipped.

Also misc rebase fixes and cleanup -

* Rename config option to "retain_head_object"

to reflect its functionality to keep head object post transitioning
to cloud if enabled

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/CloudTransition: Skip transition to cloud if the object is locked

If an object is locked, skip its transition to cloud.

@todo: Do we need special checks for bucket versioning too?
If current, instead of deleting the data, do we need to create
a delete marker? What about the case if retain_object is set to true.

& misc rebase fixes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/CloudTransition: Change tier-type to cloud-s3

Currently the transition is supported to cloud providers
that are compatible with AWS/S3. Hence change the tier-type to
cloud-s3 to configure the S3 style endpoint details.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/CloudTransition: handle versioned objects

If the object is versioned, to avoid objects getting overwritten
post transition to cloud, append object versionID to the target
object name

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/CloudTransition: Add documentation

Also to avoid object name collisions across various buckets
post cloud transition, add bucket name to the object prefix.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>