mgr/volumes: add `ceph fs subvolumegroup getpath` command

... to return the path of a FS subvolume group.

Fixes: http://tracker.ceph.com/issues/40617
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 4ac6c2acc2b8d08dd00cf1463a62246bb8128ff4)

Merge pull request #29060 from sebastian-philipp/nautilus-orchestrator_cache

nautilus: mgr/orchestrator: Cache and DeepSea iSCSI + NFS

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #28187 from pdvian/wip-39516-nautilus

nautilus: tests: osd-backfill-space.sh test failed in TEST_backfill_multi_partial()

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #28573 from smithfarm/wip-40281-nautilus

nautilus: bluestore: 50-100% iops lost due to bluefs_preextend_wal_files = false

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29007 from smithfarm/wip-40750-nautilus

nautilus: build/ops: rpm: drop SuSEfirewall2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #29102 from tspmelo/wip-40733-nautilus

nautilus: mgr/dashboard: Fix npm vulnerabilities

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29162 from smithfarm/wip-40837-nautilus

nautilus: bluestore: Set concurrent max_background_compactions in rocksdb to 2

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29204 from smithfarm/wip-40319-nautilus

nautilus: tests: make: *** [hello_world_cpp] Error 127 in rados

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29391 from pdvian/wip-40625-nautilus

nautilus: osd/OSD: auto mark heartbeat sessions as stale and tear them down

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #29464 from jan--f/wip-41058-nautilus

nautilus: ceph-volume does not recognize wal/db partitions created by ceph-disk

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: fall back to PARTTYPE if PARTLABEL is empty

In some cases ceph-disk does not populate PARTLABEL for wal and db
partitions. This commit adds the assumption that the empty string is a
valid label and falls back to identifying those disks by PARTTYPE.

Fixes: https://tracker.ceph.com/issues/40917
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 82d2ae7ffdf62ca93c7848fb62e506db4cda893c)

ceph-volume: adjust tests for empty PARTLABEL fields

It seems like ceph-disk sometimes creates OSDs where the wal and db
partitions have no PARTLABEL set. This commit alters tests so that the
empty string is considered a valid label.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit bb29110c3c6e081568231131c70522dee6d12bb5)

ceph-volume: refactor ceph-disk_member unittests

This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f42287c3416ad09c112e64756f5051c5c13aa344)

Merge pull request #29453 from neha-ojha/wip-41052-nautilus

nautilus: qa/tasks/cbt.py: change port to work with client_endpoints

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa/tasks/cbt.py: change port to work with client_endpoints

hack to work with https://github.com/ceph/cbt/pull/160/

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ee02416c654581d77d5f976da89b4a33dd1172e3)

Merge pull request #28740 from smithfarm/wip-40546-nautilus

nautilus: common: Keyrings created by ceph auth get are not suitable for ceph auth import

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28810 from ifed01/wip-ifed-db-stats-nau

nautilus: tools: ceph-kvstore-tool: print db stats

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29252 from smithfarm/wip-40180-nautilus

nautilus: tests: qa/standalone/scrub/osd-scrub-snaps.sh sometimes fails

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29309 from smithfarm/wip-40216-nautilus

nautilus: rgw_file:  fix invalidation of top-level directories

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29254 from smithfarm/wip-40465-nautilus

nautilus: core: osd beacon sometimes has empty pg list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28528 from pdvian/wip-39743-nautilus

nautilus: mon: paxos: introduce new reset_pending_committing_finishers for safety

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28575 from smithfarm/wip-40381-nautilus

nautilus: rbd: tests: add "rbd diff" coverage to suite

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28576 from smithfarm/wip-40382-nautilus

nautilus: tests: expected MON_CLOCK_SKEW but got none

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28870 from pdvian/wip-40235-nautilus

nautilus: rbd/action: fix error getting positional argument

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29193 from pdvian/wip-40272-nautilus

nautilus: rbd-mirror: link against the specified alloc library

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29244 from pdvian/wip-40274-nautilus

nautilus: core: librados: move buffer free functions to inline namespace

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29246 from pdvian/wip-40276-nautilus

nautilus: rbd: librbd: do not unblock IO prior to growing object map during resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29315 from pdvian/wip-40293-nautilus

nautilus: rbd-nbd: sscanf return 0 mean not-match.

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/orchestrator: add --refresh to `ceph orchestrator service ls`

`ceph orchestrator device ls` already supports --refresh to force
a reload, we should support this for `ceph orchestrator service ls`
as well.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit d5e320ae541cabb2bfdfe20e68fa88233999b63c)

mgr/orchestrator: allow listing iscsi services

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit e2bebee712a0716c6d009e8f5559fbe247123161)

mgr/deepsea: return ganesha and iscsi endpoint URLs

This updates describe_service() to include nfs and iscsi services
(deepsea internally refers to these as "ganesha" and "igw" roles).
Additionally, if deepsea sets any of container_id, service, version,
rados_config_location, service_url, status or status_desc, these will
come through now too.

This relies on https://github.com/SUSE/DeepSea/pull/1606 for the new
functionality, but if run against an older version of deepsea, it will
continue to operate as it did before, it just won't include
rados_config_location or service_url data.

I've also updated the caching in describe_service() so that if the
cache needs filling, it just requests all services from DeepSea, caches
everything, then does filtering on the results.  Previously, if the
cache was empty and only one service type was requested, the cache
would then *only* include that service and no other items, so subsequent
requests for different service types would return nothing until the
cache expired.

Fixes: https://tracker.ceph.com/issues/39095
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 1af7c08a96ed82b8824e644b2884d963793299b2)

Merge pull request #28962 from ifed01/wip-ifed-kv-prefetch-nau

nautilus: os/bluestore: support RocksDB prefetch in buffered read mode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28639 from xiexingguo/wip-pr-28404-for-n

nautilus: osd/OSD: keep synchronizing with mon if stuck at booting

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28891 from ifed01/wip-ifed-fix-alloc-dump-nau

nautilus: os/bluestore: more smart allocator dump when lacking space for bluefs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28892 from ifed01/wip-ifed-fix-no-compress-nau

nautilus: os/bluestore: load OSD all compression settings unconditionally.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28893 from ifed01/wip-ifed-remove-assert-bs-tool-nau

nautilus: os/bluestore/bluestore-tool: minor fixes around migrate

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28963 from ifed01/wip-ifed-add-omap-tail-nau

nautilus: os/bluestore: create the tail when first set FLAG_OMAP

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28966 from liewegas/wip-set-aio-write-max-nautilus

nautilus: os/bluestore: fix >2GB bluefs writes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29023 from ifed01/wip-ifed-fix-stuplid-alloc-len0-nau

nautilus: os/bluestore: avoid length overflow in extents returned by Stupid Allocator

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #29147 from pdvian/wip-40267-nautilus

nautilus: common: OutputDataSocket retakes mutex on error path

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29168 from b-ranto/wip-n-mgr-metadata

nautilus: mgr: Add mgr metdata to prometheus exporter module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #29173 from neha-ojha/wip-40583-nautilus

nautilus:common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29194 from pdvian/wip-40273-nautilus

nautilus: mgr: use ipv4 default when ipv6 was disabled

Merge pull request #29207 from smithfarm/wip-40904-nautilus

nautilus: mgr/influx: module fails due to missing close() method

Merge pull request #29227 from ifed01/wip-ifed-slow-ops2-nau

nautilus: bluestore: add slow op detection for collection_listing

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28756 from smithfarm/wip-40231-nautilus

nautilus: core: maybe_remove_pg_upmap can be super inefficient for large clusters

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #28869 from dzafman/wip-backport-40073

nautilus: osd: Fix the way that auto repair triggers after regular scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28993 from smithfarm/wip-40730-nautilus

nautilus: mon: auth mon isn't loading full KeyServerData after restart

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29115 from liewegas/wip-39693-nautilus

nautilus: osd/PG: fix cleanup of pgmeta-like objects on PG deletion

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29140 from liewegas/wip-40441-nautilus

nautilus: msg/async, v2: make the reset_recv_state() unconditional

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #29141 from liewegas/wip-msg-async-1-nautilus

nautilus: msg/async: avoid unnecessary costly wakeups for outbound messages

Merge pull request #29142 from liewegas/wip-msg-noneed-set-connection-nautilus

nautilus: msg/async: no-need set connection for Message.

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #29143 from liewegas/wip-secure-nautilus

nautilus: msg/async: enable secure mode by default, no longer experimental

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29159 from smithfarm/wip-40845-nautilus

nautilus: core: MDSMonitor: use stringstream instead of dout for mds repaired

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29188 from badone/wip-nautilus-lazy-omap-stats-backport-tracker-40744

nautilus: core: lazy omap stat collection

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28550 from smithfarm/wip-40007-nautilus

nautilus: rgw: fix prefix handling in LCFilter

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28648 from pdvian/wip-40107-nautilus

nautilus: rgw: Librgw doesn't GC deleted object correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28712 from pdvian/wip-40134-nautilus

nautilus: rgw: TempURL should not allow PUTs with the X-Object-Manifest.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28713 from smithfarm/wip-40505-nautilus

nautilus: rgw: fix miss get ret in STSService::storeARN

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28714 from smithfarm/wip-40515-nautilus

nautilus: rgw: multisite: DELETE Bucket CORS is not forwarded to master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28715 from smithfarm/wip-40508-nautilus

nautilus: rgw: conditionally allow builtin users with non-unique email addresses

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28728 from joke-lee/nautilus

nautilus: rgw, Policy should be url_decode when assume_role

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28729 from joke-lee/nautilus-backport-28172

nautilus: rgw: fix rgw crash and set correct error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28735 from smithfarm/wip-40125-nautilus

nautilus: rgw: hadoop-s3a suite failing with more ansible errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28736 from smithfarm/wip-40129-nautilus

nautilus: rgw: Swift interface: server side copy fails if object name contains "?"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28737 from smithfarm/wip-40137-nautilus

nautilus: rgw: the Multi-Object Delete operation of S3 API wrongly handles the Code response element

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28739 from smithfarm/wip-40142-nautilus

nautilus: rgw: multisite: 'radosgw-admin bucket sync status' should call syncs_from(source.name) instead of id

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28751 from pdvian/wip-40150-nautilus

nautilus: rgw: add 'GET /admin/realm?list' api to list realms

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28854 from pdvian/wip-40226-nautilus

nautilus: rgw_file: include tenant when hashing bucket names

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #28886 from pdvian/wip-40263-nautilus

nautilus: rgw_file: all directories are virtual with respect to contents

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29154 from smithfarm/wip-40591-nautilus

nautilus: rgw: deleting bucket can fail when it contains unfinished multipart uploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29163 from smithfarm/wip-40851-nautilus

nautilus: rgw: multisite: radosgw-admin commands should not modify metadata on a non-master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29205 from smithfarm/wip-40760-nautilus

nautilus: rgw: Save an unnecessary copy of RGWEnv

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29265 from smithfarm/wip-40124-nautilus

nautilus: rgw: hadoop-s3a suite failing with more ansible errors

Merge pull request #29286 from smithfarm/wip-40355-nautilus

nautilus: rgw: provide admin-friendly reshard status output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29287 from smithfarm/wip-40358-nautilus

nautilus: rgw: set null version object issues

Merge pull request #29310 from smithfarm/wip-40349-nautilus

nautilus: rgw/OutputDataSocket: append_output(buffer::list&) says it will (but does not) discard output at data_max_backlog

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29311 from smithfarm/wip-40352-nautilus

nautilus: rgw: multisite: RGWListBucketIndexesCR for data full sync needs pagination

Merge pull request #29313 from smithfarm/wip-40512-nautilus

nautilus: rgw: Put LC doesn't clear existing lifecycle

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29325 from smithfarm/wip-40450-nautilus

nautilus: core: s3tests-test-readwrite failed in rados run (Connection refused)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28549 from smithfarm/wip-39749-nautilus

nautilus: rgw: multisite: fix --bypass-gc flag for 'radosgw-admin bucket rm'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd/OSD: auto mark heartbeat sessions as stale and tear them down

The primary benefit is that the OSD doesn't need to keep a flood of
blocked heartbeat messages around in memory.
This prevents OSDs from accumulating heartbeat messages due to a
broken switch and then exhausting the whole node's memory:

Jun 11 04:19:26 host-192-168-9-12 kernel: [409881.137077] Out of memory:
Kill process 1471476 (ceph-osd) score 47 or sacrifice child
Jun 11 04:19:26 host-192-168-9-12 kernel: [409881.146054] Killed process
1471476 (ceph-osd) total-vm:4822548kB, anon-rss:3097860kB,
file-rss:2556kB, shmem-rss:0kB

Fixes: http://tracker.ceph.com/issues/40586
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 6cc90f363b8096d2d5fad30e57426d0cea9e3478)

Merge pull request #29040 from dcasier/wip-40100-nautilus

nautilus: ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

qa/rgw: update default port in perl workunits

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1643879218638915073d5310b859a94d10ffeac6)

qa/rgw: extra s3tests tasks use rgw endpoint configuration

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 09e992ff01b4ce286540e1230a30df67103f5968)

rgw_file: pretty-print fh_key

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 84a2c22108a3d15b836680894ba750b637e49cd2)

rgw_lc: use a new bl while encoding RGW_ATTR_LC

The current code seems to append the value to the bucket attr which isn't what
we want

Fixes: https://tracker.ceph.com/issues/39654
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit e23af175aff1d80a7857406f7c17f5a609ad7e5e)

rgw/multisite:RGWListBucketIndexesCR for data full sync pagination

RGWListBucketIndexesCR uses the 'marker' from remote zone's RGWOp_Metadata_List
admin api to list all keys for data sync.

Fixes http://tracker.ceph.com/issues/39551

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 708ba6dcc90525c8c4316760e01ee9b1e6da21b7)

rgw/OutputDataSocket: actually discard data on full buffer

A dout message in OutputDataSocket::append_output() states that
data will be dropped when appending would cause data_max_backlog
to be exceeded--but the method appends it anyway.

Log output discards at level 0, as messages will be lost.  Suppress
repeated warnings mod 100.  Switch to vector.

Fixes: http://tracker.ceph.com/issues/40178
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit c806b825dae649829de8847d36cb21ffd2bbee8e)

rgw_file: permit lookup_handle to lookup root_fh

Originally, root_fh was reachable from RGWLibFS::fh_cache, but
this was problematic as it has infinite lifetime and so is not
present in RGWLibFS::fs_lru.

To fix the above, RGWLibFS::root_fh came to live unhooked from the
handle caches which works because nfs-ganesha/the application
always has an instance handle.  However, this silently breaks
READDIR event invalidation.

Fixes: https://tracker.ceph.com/issues/40196
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 547cb9f87b72440cb8a1d496ac5cfb37b2d83cc7)

rbd-nbd: sscanf return 0 mean not-match.

When exec:  rbd-nbd map rbd/image --device /dev/image
The error message is:
rbd-nbd: failed to open device: /dev/image.

In fact, it should print:
rbd-nbd: invalid device path: /dev/image (expected /dev/nbd{num})

Meantime change help which make more understand.

Fixes: http://tracker.ceph.com/issues/39269
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 5b5c21be7138dc29f190887061f3a1de457a2e24)

Merge pull request #29260 from jan--f/wip-40921-nautilus

nautilus: ceph-volume: missing string substitution when reporting mounts

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rgw: set null version object acl issues

1.set null version object acl will create empty index
RGWRados::set_attrs did not clear instance, so index prepare, complete got instance=null,
which lead to empty index 1000_<obj>_i_null.
there is no harm to create empty index, but listomapkeys to find that key.

2.if object is exist with versioned key, we can set none exists null version object
order:
1) enable bucket version
2) put obj
3) disable bucket version
4) set versoned_id=null acl will succeed which should not

Fixes: http://tracker.ceph.com/issues/36763
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit ca6e5c89e8d6da498da15f0eea4bcd4d271b44e9)

rgw: provide admin friendly reshard status output

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit ba4167c2f30fbb868482d48aaada6388f58de920)

rgw: update the "radosgw-admin reshard status"
command documentation with expected output examples

Fixes: http://tracker.ceph.com/issues/37615
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 8565ec0db69223ae71766ba000eecc0d2f7d14c2)

qa/rgw: add dnsmasq back to s3a-hadoop

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8c8a706b6f4cb3b2a5b89aa1eb06b7e47909852d)

qa/rgw: remove ceph-ansible from s3a-hadoop suite

Fixes: http://tracker.ceph.com/issues/39706
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0fc2c8ecee2b6233292b9fd1325347fd0fdf9171)

Conflicts:
qa/tasks/s3a_hadoop.py
- mimic lacks the "if hadoop_ver == 'trunk'" conditional block

qa/rgw: use default ports (80 or 443) unless overridden

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3ff5f0c2406e40d1024d8152e0ac9400302a757a)

qa/rgw: rgw task can override --rgw-dns-name on the command line

the value for rgw_dns_name isn't known until a machine is assigned, so
it can't be set via 'override: conf:'. add a per-client config option
to the rgw task so it can add the endpoint's hostname and/or s3website
hostname on the radosgw command line

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 658e5932fb79e2d33b73363b6ce76ff299809e16)

qa/rgw: allow rgw client config to override port

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 921faebb723c11686bf790ca424c952a786f358a)