mgr/cephadm: use fixed config file path as argument

Fixes other alertmanager containers which do not default to look for a
configuration file in `/etc/alertmanager/alertmanager.yml`.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit cdd0ba13a5109e4319ffdaff8e8a17212e78dd15)

cephadm: fix custom alertmanager

When alertmanager image has been customized and does not have a
/etc/alertmanager directory, the uid/gid extraction fails. This change
assumes that instead the /etc/prometheus directory exists and can be
used to extract the uid and gid.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit d218de4cde8d37f7f69a2a2a9474c0ac9afca41d)

cephadm: Allow users to use a custom dashboard ssl port

This modification allows the user to create a new bootstrap cluster using a predefined SSl port for the dashboard.
If firewall is enabled, any new manager daemon deployed in new hosts will take care of open the required ports for all the services enabled in the manager.

Two new parameters for cephadm tool (aka binary or standalone):
Command bootstrap:
--ssl-dashboard-port SSL_DASHBOARD_PORT
                      Port number used to connect with dashboard using SSL
Command deploy:
--tcp-ports TCP_PORTS
                      List of tcp ports to open in the host firewall

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 08c4a53e79a7ed63912f765fbbf86e3364d8f9be)

mgr/cephadm: mgr or mds scale-down should prefer non-active daemons

When removing daemons during a mgr/mds scale-down, prefer to remove
standby daemons so the active daemon is not killed

Fixes: https://tracker.ceph.com/issues/44252
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0fbf12c9d17a98bf4315b0bff81c0aeb38458e0f)

doc/cephadm: remove ssh user limitation troubleshooting

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit b59b480c3d6f0bcceff155c02770acb2ea842561)

python-common: ban paths from non-data-devices

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit ac5839b9f5d4ff2c4269b9da0314837cdafa5360)

docs/mgr: remove traces of ceph orch osd spec

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit f8b8e6f22900848ed12d166592f504e2d6b70554)

mgr/orchestrator: remove traces of ceph orch osd spec

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 4109e981d969c5c61f6dbf66e1e8c3c17724f5f8)

mgr/cephadm: add default osdspec_affinity to module fixture

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit eb1b37cfc34db90db86ef8449f645d5a192656b9)

mgr/cephadm: save osdspec_affinity to dd store

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 0feae8893364729f2e9b9ee9bc53b6eba66cbc10)

mgr/cephadm: add by_type daemon retrieval method

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 1f8965f8cc48c9e901f3531259d446e3d9211135)

mgr/cephadm: also mock `ceph vesions`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1c8cb204ffdb654a2b954a3d39a1c1712c151684)

mgr/cephadm: add test_upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d71f3b42738eef2d234ec752bc27d3e8e53e744c)

mgr/cephadm/tests: move with_service to fixtures.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 50ecd89d790fb33a7dc297ae1eeb7751dd58cba2)

pybind/mgr: Also mock the config store in pytest

This will be used to dry-run mgr/cephadm/upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f44c1e71033f8f62e3bca17d274049b858e7b015)

mgr/cephadm: check existance of the RADOS pool

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4419c18f9fb961b54715c355194b85e57804adec)

mgr/cephadm: convert py2 -> py3 type checking

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit e05bf091724e318b547967c6ec81ed5cf5807b16)

mgr/cephadm: clean-up nfs service module imports

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4ad0636e81d1005c116ef6aae211e6fcead23ed4)

qa/tasks/cephfs/nfs: Add tests for cluster config set and reset

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit bba2d66176fa30b934aaa6fb7670cd0fecc0af78)

doc/cephfs/nfs: Update the doc about 'reset' and 'set' config interfaces

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 287da119215671fe24b371f9df01059569db4b3e)

mgr/volumes/nfs: Add interface for adding user defined configuration

Code related to creating and removing rados object has been rearranged abd
modified. This patch introduces interface to set and reset user config.

nfs cluster config set <clusterid> -i <config_file>
nfs cluster config reset <clusterid>

Fixes: https://tracker.ceph.com/issues/45747
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 9535154794c1d6d116b81c1317c48ba1aecd7581)

mgr/orch: _interface.py: PEP8tify

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b7caeed9b694a3191f06d96339a81f5d99b0ab88)

mgr/orch: _interface.py: Convert type annotations to py3

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0bb09324e7b4cffc957a89c3c5188bdb823176aa)

doc/cephadm: Remove shell alias

Turns out, the alias doesn't work for files.

Fixes: https://tracker.ceph.com/issues/46701
Fixes:

```
$ ceph orch apply -i myfile.yaml
ERROR: no such file or directory: myfile.yaml
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 75baae35288cca79a6e958c3370dba3caab813f8)

mgr/cephadm: _get_config_and_keyring: `host` is required now

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6105d1ba4c94e1278bf38896ad467a5cf7e5b92b)

mgr/cephadm: Some more typing

Make sure we don't mix daemon names, config entities and
auth entities. Cause all of them don't mix well together.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 01e2dafb6e5b7cf3f26a63c2c66d346a1ab0bffa)

mgr/cephadm: Prevent unnecessary SSH reconfiguration

When 'set-user', 'set-pub-key' or 'set-priv-key' are called two
times with the same value, we can ignore the second call.

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 4cbf587b686d3b539f98b5c4cab1f0320efbe68c)

mgr/orchestrator: Update service_action() docstring

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 4d6365b6a9a9bda4375b84ad3661e43e5de9abf8)

mgr/cephadm: module.py: PEP8tify `ceph orch * ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit be3f513239d45feb8557f30e05fc562ce9594f51)

mgr/cephadm: Make `ceph orch * --refresh` asynchronous

There is just no way we can allow those calls to be synchornous:

* All CLI commands are blocked, till the refresh completes.
* ^C doesn't help at all, as it just kills the client

Therefore they have to be scheduled in the bg.

Fixes: https://tracker.ceph.com/issues/46813
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7189575404262c0fbb4717f824c022d48ab58fcd)

mgr/cephadm: Refactor upgrade to call _daemon_action with new image

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 289f37d29674b9a26415147a964b1fcb26a7eaf7)

mgr/cephadm, Add --image to `ceph orch daemon redeploy <what>`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 9d0480d18b6f30f929e8599ee8b32fdc6f91aa27)

mgr/cephadm: rgw update period after realm, zone creation

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit 70b11a1a83d2d03062d7a2db81de8675b2f0b0f4)

mgr/cephadm: PEP8tify  upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ffc1fd65c6b82a5565953f0da184a0b218d7ca24)

mgr/cephadm: Add UpgradeState class

My resons are:

* This increase mypy coverage
* Makes the code more discoverable.
* Allowes us to have intelligent UpgradeState properties

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3072b1d8cbc8b5ac83697007767e12ff805a693d)

cephadm: don't add `ceph-volume lvm activate` for adopted simple OSDs

This changes the logic in deploy_daemon_units() to add either `chown` calls
for simple (ceph-disk style) OSDs, or to add `ceph-volume lvm activate` calls
for LVM OSDs, rather than always adding both.  When I was working on
https://github.com/ceph/ceph/pull/34703, I'd originally added an "osd_simple"
flag to figure out what type of OSD was being adopted/deployed, but passing
that around was kinda ugly, so was removed from that PR.  This time around
I'm checking if /etc/ceph/osd/$OSD_ID-$OSD_FSID.json.adopted-by-cephadm
exists, which seems pretty safe IMO.  My only concern with this method is:
what happens if someone adopts a simple OSD, then later wants to migrate it
to LVM.  Presumably that's a destroy and recreate, keeping the same OSD ID?
If that's true, then the JSON file probably still exists, so the subsequent
create will do the wrong thing, i.e. will add `chown` calls, not `ceph-volume
lvm activate` calls.  Any/all feedback appreciated...

Fixes: https://tracker.ceph.com/issues/46833
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 8112949851eb4849f0d559a118739d92051dcf8e)

mgr/caphadm/tests: Add `test_daemon_check_post`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7d3deb2e23266ee5e206d76bb4ce33b436ac5ec2)

mgr/cephadm: Don't run `daemons_post` if _create_daemon fail

We can't run daemon_check_post for the type, if a single daemon failed.
Mainly cause `daemon_check_post` is run by service type.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit fdaa00977e3d9ee3bd1c6f5be69a0e93d6825136)

mgr/cephadm: Add test_daemon_action_fail

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f500b8fc747f1767a31e54e6fb4a9a596889d1a1)

mgr/cephadm/tests: Split `test_daemon_action`

Into:

* `test_list_daemons`
* `test_daemon_action` (now without listing daemons)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6fd599bd4d66c290ba94accd76c869a50a9dd5ce)

mgr/cephadm: guard _check_daemons

Continue with other daemons, if one fails

Fixes: https://tracker.ceph.com/issues/46748
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 313e091fee6b3399852683575f7431c1ba235133)

mgr/cephadm: PEP8tify

Those files should generate fewer conflicts for now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 17db7ba7dfb5ad617ec2b3f81a37c6fdc69f9222)

mgr/cephadm: Enhance AlertManagerSpec to allow adding additional webhook receiver URLs

Fixes: https://tracker.ceph.com/issues/46775
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 4b87eeb97861929130c4aa7e2c99e29d2a1652ac)

Merge pull request #36260 from rhcs-dashboard/wip-46672-octopus

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #36655 from smithfarm/wip-46968-octopus

octopus: rgw: introduce safe user-reset-stats

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36654 from smithfarm/wip-46966-octopus

octopus: rgw: fix double slash (//) killing the gateway

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36653 from smithfarm/wip-46955-octopus

octopus: rgw: fix shutdown crash in RGWAsyncReadMDLogEntries

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36652 from smithfarm/wip-46949-octopus

octopus: cls/rgw: preserve olh entry's name on last unlink

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36651 from smithfarm/wip-46938-octopus

octopus: qa/tasks/ragweed: always set ragweed_repo

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #36650 from smithfarm/wip-46929-octopus

octopus: rgw: Empty reqs_change_state queue before unregistered_reqs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36649 from smithfarm/wip-46640-octopus

octopus: rgw: add abort multipart date and rule-id header to init multipart upload response

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36648 from smithfarm/wip-46593-octopus

octopus: rgw: add negative cache to the system object

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36647 from smithfarm/wip-46518-octopus

octopus: rgw: fix boost::asio::async_write() does not return error...

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36646 from smithfarm/wip-46510-octopus

octopus: rgw: Adding data cache and CDN capabilities

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36645 from smithfarm/wip-46475-octopus

octopus: rgw/iam: correcting the result of get role policy.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36636 from smithfarm/wip-46953-octopus

octopus: rgw: policy: reuse eval_principal to evaluate the policy principal

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36026 from smithfarm/wip-46457-octopus

octopus: selinux: allow ceph_t amqp_port_t:tcp_socket

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36018 from smithfarm/wip-45922-octopus

octopus: rgw: lc: add lifecycle perf counters

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #35929 from smithfarm/wip-46343-octopus

octopus: rgw: orphan-list timestamp fix

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36638 from smithfarm/wip-46945-octopus

octopus: librbd:  global and pool-level config overrides require image refresh to apply

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>/

Merge pull request #36414 from shyukri/wip-46322-octopus

octopus: osd/OSDCap: rbd profile permits use of "rbd_info"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36411 from shyukri/wip-46087-octopus

octopus: mgr/prometheus: automatically discover RBD pools for stats gathering

Reviewed-by: Mykola Golub <mgolub@mirantis.com>/
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36331 from dillaman/wip-librbd-aio-completion-octopus

octopus: librbd: potential race conditions handling API IO completions

Reviewed-by: Mykola Golub <mgolub@mirantis.com>/

Merge pull request #36660 from smithfarm/wip-46721-octopus

octopus: rbd: immutable-object-cache: fixed crashes on start up

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36659 from smithfarm/wip-46719-octopus

octopus: rbd: librbd: don't resend async_complete if watcher is unregistered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36658 from smithfarm/wip-46711-octopus

octopus: rbd: librbd: flush all queued object IO from simple scheduler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>/

Merge pull request #36657 from smithfarm/wip-46309-octopus

octopus: tests/rbd_mirror: fix race on test shut down

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36656 from smithfarm/wip-46150-octopus

octopus: rbd: librbd: race when disabling object map with overlapping in-flight writes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #36662 from smithfarm/wip-46595-octopus

octopus: crush/CrushWrapper: rebuild reverse maps after rebuilding crush map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #35772 from s0nea/wip-46205-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #36350 from s0nea/wip-46751-octopus

octopus: mgr/dashboard: Add hosts page unit tests

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

librbd: generic helper for tracking in-flight IO and managing flush requests

Layers that potentially queue IOs but not all IOs will need to track all
in-flight IOs to properly ensure that a flush cannot complete while
older IO is still stuck in a queue.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 728057e492a697191d72049ce64b337fa4c80078)

Conflicts:
src/librbd/CMakeLists.txt
- no "io/ImageDispatch.cc" in octopus

Merge pull request #36416 from shyukri/wip-46639-octopus

octopus: doc/rbd: add rbd-target-gw enable and start

Reviewed-by: Nathan Cutler <ncutler@suse.com>

crush/CrushWrapper: rebuild reverse maps after rebuilding crush map

The Objecter will crash when localized reads are enabled and two threads
attempt to rebuild the (invalidated) reverse maps concurrently. This
should address the issue for the Objecter use-case without the need to
add additional locking.

Fixes: https://tracker.ceph.com/issues/44311
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8b866794f5b3674c5e3ad9adceb5e3230d55a0e0)

doc/rbd: add note for the immutable-object-cache domain socket

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d44cb50670c20613c63c249595e43650ec29b13a)

common/options: changed default immutable object cache path

Switch the default path for the immutable object cache from "/tmp" to
"/tmp/ceph_immutable_object_cache" to prevent the deletion of all
files in the "/tmp" directory and sub-directories for unconfigured
daemons.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit eeecc1860f81129347c648a6cbd4a8a78ff4c636)

immutable-object-cache: fix error handling during start up

Previously the daemon would crash if it couldn't remove all the
files from the specified cache directory. It would also crash if
it could not access its specified domain socket file.

Fixes: https://tracker.ceph.com/issues/45169
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f6dcb6c3dbd45574e1a7ca3f5e8f1f52a0d7ae1e)

librbd: properly unregister failed async_complete request

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 143179ffa92ce64d89fc98d6d6c9e7a3722a12f2)

librbd: don't resend async_complete if watcher is unregistered

Also wait for pending async_complete after unregistering the
watcher.

Fixes: https://tracker.ceph.com/issues/45268
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 5b6804e19a8f524ab1528a638eb286482e12fe48)

librbd: flush all queued object IO from simple scheduler

Normally IO is tracked via the AioCompletion's async_op but the
scheduler will "complete" writes while the IO might be still
executing. Therefore, prior to shutting down this dispatch layer
we need to wait for all IO to complete.

Fixes: https://tracker.ceph.com/issues/46668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 75ff8fd14dccaa7d2f11ba8a561ad3c0f410585c)

test/rbd_mirror: fix race on test shut down

in TestMockImageReplayerSnapshotReplayer.UnlinkRemoteSnapshot

Fixes: https://tracker.ceph.com/issues/46076
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit c214785fa59ace2eb68a54b4354864ff707335ca)

librbd: race when disabling object map with overlapping in-flight writes

The block guard that protects against overlapping updates to the object
map needs to be flushed prior to closing the object map instance.

Fixes: https://tracker.ceph.com/issues/46083
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ee69323cd27263cb7f9dd97dcbfb1c36f1cc0837)

rgw: introduce safe user-reset-stats

Defines cls_user_reset_stats2, a value-returning cls operation
that sets new stats via progressive calls with an accumulator,
avoiding risk of excessive call runtime.

Fixes: https://tracker.ceph.com/issues/41080
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 25a82ed3795ecf6395e3d16fcbb4c29e478aa065)

rgw/tools: add flags to rgw_rados_operate api

currently flags are hardcoded to zero. this prevents some usecases, e.g.
passing OPERATION_RETURNVEC flag

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 6474b77e37fd4da69b4bb5b2f171f3afa16898eb)

rgw: fix double slash (//) killing the gateway

When a bucket is inialized as a static website, a curl request on the bucket with double slash kills the gateway.
The problem is on the URL handling of the subdirectory, which tries to remove the last slash of any URL, so when only / is given as a sub-directory, this results to an empty string.

Fixes: https://tracker.ceph.com/issues/41225
Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
(cherry picked from commit 626330c504b1af8393986bd0553786c3031422cd)

rgw: fix shutdown crash in RGWAsyncReadMDLogEntries

RGWAsyncReadMDLogEntries must not store pointers into coroutine memory,
because it's not guaranteed to outlive our call. store these by-value
instead, and have RGWReadMDLogEntriesCR::request_complete() copy/move
them back on completion

Fixes: https://tracker.ceph.com/issues/45771
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 13bf06dbe961132ca99f470ac026674e45fecc38)

cls/rgw: preserve olh entry's name on last unlink

When rgw_bucket_unlink_instance removes the last instance of a name, it
also clears the value of rgw_bucket_olh_entry.key. However, bucket index
resharding uses this key when choosing its shard placement, so an empty
key causes all of these olh entries to be misplaced in shard 0. After
reshard, all of the olh recovery/cleanup logic would be sent to the
correct shard, and these misplaced olh entries would never be cleaned
up.

Preserving the key's name on last unlink allows the olh entry to be
resharded correctly and cleaned up normally.

Fixes: https://tracker.ceph.com/issues/46456
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit acf8f3cad9f55e34c703fdaef684853a3fb3b369)

qa/tasks/ragweed: always set ragweed_repo

* extract get_ragweed_branch() out of download() task, for better
  readablity.
* use a loop for retry when the first clone fails
* drop the `raise ValueError()` clause as it never happens. we could use
  an assert() here, but i don't think it is necessary anyway.
* use sh() instead of run() for better readablity.
* always set ragweed_repo. before this change this variable is
  unbounded if `force-branch` is set.

Fixes: https://tracker.ceph.com/issues/46771
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 87e61e1d6ece9874cd8776dde92ca815e8002166)

rgw: Empty reqs_change_state queue before unregistered_reqs

In RGWHTTPManager::manage_pending_request(), before unregistering
or unlinking the http requests, empty the reqs_change_state list
to avoid use after free.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit b210437e5f28d53e770dd6938ce3c4be443da055)

rgw:  abort multipart date should be round up to midnight next day like expiration one

Signed-off-by: zhangshaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit c314d322f8a0fc628f9d19b69491b54a82c069f0)

rgw: use std::optional instead of boost::optional

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 7c85903cbe4321a6a8dff5254225b33e05e3bbd5)

rgw: add abort multipart date and rule-id header to init multipart upload response.

Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 4bb2a0164be3dd69e28972bcb39f0892a4360bba)

rgw: add negative cache to the system object

add negative cache to the system object

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Fixes: https://tracker.ceph.com/issues/45816
(cherry picked from commit 0900bd8cf90babd6fafcb398854a4ddb071a27ee)

rgw: fix boost::asio::async_write() does not return error...

although remote has closed the connection

Fixes: https://tracker.ceph.com/issues/46332
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit c997eb6ad77deebd8e903fe84da7af6fcf50d528)

rgw: fix nginx-rgw docs

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit d73b879ac169c46f2dfeba0f4ca7f3a8af272a53)

rgw: create rgw-cache example directory

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 94c81f3094222d49a9c12d088667f58926b1cbf5)

rgw: fix nginx cache docs

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 892e9fa5fe2a8759da50e62d48632dff6ad81c30)

use ceph::split

Signed-off-by: Or Friedmann <orfriedma@redhat.com>
(cherry picked from commit 737e1aa95686d8c4a429cac9ea5f388bcda04bac)

rgw: fixes to override_range_hdr

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit e6e1dd868140e7ed356e007d4bf95a5bd2415035)

fix docs

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit a4ae27242a2426cefd602ba053ae565e3c96ff0b)