rgw/rest: enable iam UserPolicy apis against account users

when the authenticated user belongs to an account:
* operate only on that account's users
* match UserName to user's display_name instead of user_id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3841906eeb2e64a07c74b746ffe79649627b8d55)

rgw/rest: simplify RGWRestUserPolicy hierarchy

base class constructor takes `uint64_t action` instead of overriding
the virtual `get_op()` on each subclass

constructor takes `uint32_t perm` instead of deriving separate base
classes RGWUserPolicyRead/Write for check_caps() permission

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6d61c397a84efde7e99ed7c00d2216990f0d9884)

rgw/rest: iam user policy api cleanup

make get_params() virtual and protected. base class always validates
UserName

add common init_processing() function that calls get_params() and loads
the user by UserName. this step happens before verify_permission()

set s->err.message in several error paths

add the xmlns="https://iam.amazonaws.com/doc/2010-05-08/" part to the
responses

return ERR_LIMIT_EXCEEDED instead of ERR_INVALID_REQUEST when
RGWPutUserPolicy exceeds the policy limit

where RGW_ATTR_USER_POLICY is missing, treat it the same way we treat an
empty map of policies. this avoids separate error paths

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 93428aa6e49da8cdd602d761eaff693449bf57f2)

rgw/auth: Identity matches account user principals

when a user belongs to an account, they match Principal ARNs by account
id instead of tenant name, and by user name instead user id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8e24a611db7a951c2523503e922c9fda4fb68f2e)

rgw/auth: Identity matches paths in user principals

when RGWUserInfo::path is present, use it when matching user principals

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ea33bfb784c72cbc4d198c9f5139e54504466f54)

rgw/auth: Identity::is_identity() takes one Principal

take a single Principal instead flat_set<Principal>, and iterate over
calls to is_identity() instead

why?
* it simplifies the logic of each is_identity() function because they
  can use early returns to avoid visiting all of the cases
* Statement::eval_principal() no longer has to allocate a flat_set
  with a single element when the Identity is a role
* rgw::auth::Identity no longer depends on rgw::iam's choice of
  container type

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 05c15502e8cd7f8a7d279d562e2c7abadcaaeafa)

rgw/iam: rename rgw::auth::Principal::Tenant to Account

just changes the name to match its use in AWS, without changing any
behavior in rgw policy parsing/evaluation

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2cc488e9847afafa1e6b7dc283eca3cc6d74d156)

rgw/sal: add interfaces for account roles

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e47d08efa84bbb9cd99cfdd53b814786d9025bdd)

rgw/rados: add rgwrados::roles namespace abstraction for cls_user

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 02702b26bdf6859949bb16c3b9da97ff4250b09c)

rgw: init_quota() loads owner quota unconditionally

now that owners can be accounts, don't default to s->user when s->owner
matches s->bucket_owner

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d4c0d615eb13356512bc31fc89e2819cf03c6cdc)

radosgw-admin: quota commands can set account quota

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d473305b2fd8760b55c6c7e8149b3cebf49b6e7c)

rgw/iam: add IAM AccessKey apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 927d533308359f506eba1ee56a560692d9049d62)

rgw/user: add 'create_date' to RGWAccessKey

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c51b910e47ce784f086714ea8179b10620d78801)

rgw/user: expose functions to generate access/secret keys

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 21d0ccb69663518d56d7d1dd9b8f983dd2225871)

rgw/iam: add initial IAM User apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a9c49a5ce7a2eb74e50cde11f6a8aab32764aa89)

rgw/rest: wrap iam/sns/sts Error responses with ErrorResponse

all iam/sns/sts requests wrap the s3 <Error> xml response in another
<ErrorResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">

without this, boto3 fails to fully parse error responses, leading to
generic Unknown ClientError exceptions of the form:
    botocore.exceptions.ClientError: An error occurred (Unknown) when calling the PutUserPolicy operation: Unknown

with the ErrorResponse part, boto3 throws more specific exceptions that
include the error Code and Message:
    botocore.errorfactory.NoSuchEntityException: An error occurred (NoSuchEntity) when calling the PutUserPolicy operation: No such UserName in the account

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 195ecb3732e0d7f78dc74cc9b7f2ac3d7ed919a0)

rgw/rados: user metadata links to account index

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7fb80b0048279853595f160e7f2b0fb4c18b63a3)

rgw/sal: add interfaces for account users

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8971465a35616cc9b848b7943abd167a59abdda1)

rgw/rados: add rgwrados::users namespace abstraction for cls_user

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 22c19222c484a0b8dfc157418790ab72c3d58b12)

rgw/auth: use switch for is_non_s3_op()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 31f4c6231a6d6328570d913572b6aa6496dfef9a)

rgw/op: separate IAM ops in enum RGWOpType

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fdf9f8b8d4e49e960a9fc2b528a3e2a9acedc652)

rgw/iam: use enum for action values

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d48c9713abbbd074b158bb3418cd2ff4a759eeaf)

rgw/iam: add iam user metadata (path, create_date, tags)

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit de9feeb32ca71c67b29e753c3164cd778c08c280)

test/cls: add ceph_test_cls_user

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a49757e5c3e7580399802345babd2dde4e3e661b)

cls/user: add interfaces to index user account resources

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b56fc946eac20d982cd0743305a630948d5b12d6)

cmake: move WITH_RADOSGW checks up to test/CMakeLists.txt

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9f0f44e17f3d810197a0dc6c628968bedb21466c)

rgw/multisite: RGWBucketMetadataHandler updates linkage on owner change

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d304285d21590908f385bda2b3177ca2b7024ef8)

rgw: update bucket ownership when adding user to account

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ac8226ad64f4c03bc9ce7d37f1d2aa7d9afa9e24)

rgw/rados: Bucket::chown() updates owner on bucket instance

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c44c493aed60d22950e17e71024e74c5fcc4e2e1)

rgw: add RGWIdentityType::TYPE_ROOT for account root user

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 828aa90a2b0070d043c67464a03529105d835db2)

qa/rgw: create accounts for random s3test users

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ff81a31ad678472e6847ad39f57e14efd89b0ead)

rgw/acl: create_policy_from_headers() can yield

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e8f078188d22c3ceb55fb6893f314bb651ba4829)

rgw/acl: s3 CanonicalUser grants support account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d083e946d376a946854a4a49278f5e1d64393f8e)

rgw/sal: add load_aclowner_by_email()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 01e51d8240db17b45c5325df601f14cb647cc0e1)

rgw/sal: move list_buckets() to Driver

move User::list_buckets() to Driver and take rgw_owner to serve bucket
listings for account owners

also unifies the user/account stats interfaces around rgw_owner in
Driver

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d76ef3294735a42253a3718dd519f55283bdd3b4)

rgw: use rgw_owner in RGWBucketInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 65c80d7e63f12a76857726bab929261717adb75b)

rgw/acl: use rgw_owner for ACLOwner

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e88859e81a36b2b0a21d6e469adcb3c97b51f6c4)

rgw/pubsub: use rgw_owner for topic ownership

allow topics to be owned by accounts instead of users. radosgw-admin
topic list can now filter by --account-id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 13d1c803f6a90ad1747f21faa7a6e89d7e8af8c1)

rgw: use rgw_owner in RGWBucketEntryPoint

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 469b7e83d723021555886dc7c46d178190f59829)

rgw/rados: add rgwrados::buckets namespace abstraction for cls_user

move cls_user stuff out of RGWSI_User_RADOS into namespace
rgwrados::buckets that doesn't depend on rgw_user, so it can also be
used for account bucket owners

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 92f04d8637f78d916a3bcf955fe7cff56f053283)

rgw/rados: add account metadata handler

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit eb86cd410df93fa9edb959e9d2db9a56410dd0f3)

rgw/rados: implement account metadata operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8c3fc16b3d9ef866c6b9cee558b30d78b357c8cc)

rgw/rados: generalize RGWUID for accounts

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c72a51e08c5fb570a270efc3e1a7aff1e0383d16)

rgw/rados: add .rgw.meta:accounts pool

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e3418a9c4f2e2e00c3c066d8f8bb2db1d149745a)

rgw/quota: generalize quota to rgw_owners

use rgw_owner instead of rgw_user as a key in the quota cache stats.
only fetch_stats_from_storage() and init_refresh() need to differentiate
between user and account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f1835cd78770ccd0d4ff84f68ecf2a9750182a65)

rgw/auth: Identity::is_owner/admin_of(rgw_owner)

is_owner_of() and is_admin_of() take rgw_owner instead of rgw_user so
that identities associated with an account share ownership of that
account's resources

LocalApplier is the only Identity type that supports accounts, based on
comparison with RGWUserInfo::account_id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a1c675da7cf571457898d799206e911e23cdc711)

rgw/auth: account users also match ACL grants to their account id

ACL grants can now specify an account id for the CanonicalUser to
grant access to the entire account. this is implemented only for
LocalApplier

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1698784e2eb72e99e3eaa477ae8e53b35a6ab377)

rgw: add rgw_owner variant with json encoding

existing buckets and objects use `struct rgw_user` for the owner. with
the addition of accounts, we need to be able to represent ownership by
the account rather than its users

add a `rgw_owner` variant that preserves the existing json encoding of
`rgw_user` while adding a new representation for account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 87a74f8603bdeccdea3fdfa7b4e0fd344fba8aae)

rgw: enable 'user modify --account-id'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e69ce01774be57259bef76fb3267c607513f28e7)

rgw: add /admin/account rest apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cce372dc2ebd2b0d7bdb06b930fcc35ad27147d1)

radosgw-admin: add account admin commands

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 07d5ed956275e408fc429cbef821ad01b085a2c3)

rgw: add account admin ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6c834d6d38128dc03bb088f133f4bef6c0a711b9)

rgw/sal: add account interfaces to Driver

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 91a58853a6265f19175586c4543a41109dc3a791)

rgw: add account_id to RGWUserInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3f1cd1b44a8ef60c6f8fd460df5559145a059930)

rgw: add struct RGWAccountInfo

initial design and prototype by Abhishek

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 59bbd65bac0ea41ec8cca844b9b65b9def026467)

rgw: define account ids and names

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f678c780195c81d86533740456d843c08c7db1c0)

common: add gen_rand_numeric()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cac60663f553cf6a985050c068d906778e66214e)

rgw/sal: pass in ACLOwner for object writes

`ACLOwner` contains both the user id and display name. the bucket index
needs both values for:
```
struct rgw_bucket_dir_entry_meta {
  ...
  std::string owner;
  std::string owner_display_name;
```
`RGWRados::Bucket::UpdateIndex::complete()` relied on the parsing of
`RGW_ATTR_ACL` to get those values. but object write operations already
had that information earlier in the call stack, so we might as well pass
them in directly

for other operations like the copy/rewrite/transition of existing objects,
we decode the owner from the source object's `RGW_ATTR_ACL`

the existing `owner` param was confusing, as it represented the bucket
owner for quota stats updates. this get renamed to `bucket_owner` inside
of `RGWRados`, and hidden from the outside. bucket stats are attributed to
`RGWBucketInfo::owner`, not the `ACLOwner`, so we use that instead of
`s->bucket_owner`

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1d29b7056983825e7095c518edc7b3d4dcdf0b05)

Conflicts:
src/test/rgw/test_d4n_filter.cc removed on main

rgw/acl: use ACLOwners for create_default()

initialize RGWAccessControlPolicy with ACLOwners from the auth identity

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a2b41a27ebc2f14f3d2079fc4a2c2a682e871bcc)

rgw/auth: initialize s->owner in Strategy::apply()

for a common location that applies to all rest handlers that implement
authorize()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9989a86dd7a586d292cae0de32f7723b93147edc)

rgw/auth: Identity::get_aclowner() for resources it creates

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4b8cea086a44019486418d818687cc1502b1ed2d)

rgw/auth: generalize Identity::get_role_tenant() as get_tenant()

all identities can return a tenant. rgw ops should consult the auth
identity for this instead of a rgw_user or rgw::sal::User

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d82ff164505cd7a7abe6937879052ad03f1da939)

rgw/iam: zero-pad milliseconds for role and oidc-provider CreateDate

make sure that tv_usec=41000 gets rendered as '.041Z' instead of '.41Z'

Fixes: https://tracker.ceph.com/issues/64381
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit efb2a91138e76c5e163916093d5e37f776479bcf)

Merge pull request #56670 from cbodley/wip-65292-squid

squid: rgw/pubsub: CreateTopic validates topic name

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #56069 from cbodley/wip-64818-squid

squid: rgw: replicate v2 topic/notification metadata

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #56856 from guits/update-cv-func-tests-squid

ceph-volume: update functional testing

Merge pull request #56850 from zdover23/wip-doc-2024-04-12-backport-56831-to-squid

squid: doc/cephfs: refine client-auth (3 of 3)

Merge pull request #56750 from ronen-fr/wip-rf-unreserved-squid

squid:osd/scrub: disable scrub reservation queuing

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph-volume: update functional testing

various changes for fixing ceph-volume functional testing.
cleaning up deploy.yml (drop py2 references)

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 90bfaaa2f0ab7e28c0f78189ccb83635e944e758)

squid:osd/scrub: disable scrub reservation queuing

as a temporary measure for Squid RC0. There are known problems in the
existing code, with patch PRs being tested. This change disables the
Reserver functionality, until such time as the patches are merged.

Note the following trackers:
- https://tracker.ceph.com/issues/65044
- https://tracker.ceph.com/issues/65072

Once these are solved, this commit should be reverted.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #56766 from ronen-fr/wip-65374-squid

squid: tests/scripts: use 'tell pg deep-scrub pgid' instead of 'tell pgid deep-scrub'

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/cephfs: refine client-auth (3 of 3)

Refine the final third of doc/cephfs/client-auth.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 44d1eeed703e968b374ec4d12fff3a2466a4c276)

Merge pull request #56841 from zdover23/wip-doc-2024-04-12-backport-56784-to-squid

doc/cephfs: refine client-auth (2 of 3)

doc/cephfs: refine client-auth (2 of 3)

Refine the second third of doc/cephfs/client-auth.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 083f56352b4fff662dff3c78b6a0c455605d02ff)

test/rgw/notification: fix threading issue in test

test_ps_s3_persistent_topic_stats may delete boto client threads
before they are done.
also, no need to run the http server twice

Fixes: https://tracker.ceph.com/issues/63909
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 51e365927361db19b2ae2b270bab7f30fe3e51ee)

test/rgw/notification: use polling instead of sleep

when checking persistent stats queue length

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 5581bc7d14f4c5814a7d43597aa77cacd6a2d125)

test/rgw/notification: cleanup leftover topics

this may happen when a test fails, and does not cleanup topics
it created. other tests that verify the number of topics may fail
because of that.
all tests that verify number of topics, should delete all topics at the
start of the test.

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 9ae0de17e1a8c3a0527b04981eeb7e632da985ac)

test/rgw/notification: use simpler multithreaded http server

Fixes: https://tracker.ceph.com/issues/63909
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 673adcbdbd01e64c8b76c3176e062571fb8710ac)

test/rgw/notifications: do not delay http responses to notifications

we also increase the wating time after rerunning the http server
to allow fo the persistent notifications retries to finish

Fixes: https://tracker.ceph.com/issues/63909
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit ca6688696b9ac9f838ffe21951331b6a1974dae8)

rgw/notification: Load bucket attrs before calling publish_reserve.

As part of PR# 55657, publish_reserve would reload bucket to ensure bucket_attrs are loaded. However for lc events, where the bucket attrs were already loaded, the reloading was causing crash but there was no obvious root cause, so to avoid the crashes, remove reloading of bucket in publish_reserve and put the onus on callers to load the bucket before calling publish_reserve.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit fa5d370663804db9752c99e79659aae031a7bf85)

test/rgw/noitifications: run basic/http/migration tests with 2 RGWs

this is useful to:
* cover the cls_lock code with persistent notifications
* cover v1 to v2 migration collisions between RGWs

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit c0e540e6f11aa32b347ea6e7e9f03cad5d1913e8)

rgw/notifications: run v2 migration tests in teuthology

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 67d43ae2538a611373159346855dab8701e176fb)

RGW/Rados: Migrate topics to data path v2

also add migration tests

Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
(cherry picked from commit 1a7d1454801e3d7b888aa734c4e3b609febf417f)

rgw/notification: Support generating multisite sync delete events.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit 37069ac96b53d2d4927b9d6f743790847861ec8e)

rgw/notification: Make the Replication events to be aws compatible.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit 3ea110bb89a7f666de25f8a128fa8bbb231e304f)

rgw/notification: Make the Lifecycle events AWS compatible

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit ea6611327c9aa678c524e0d71e954bd735d60789)

PendingReleaseNotes: announce the notification_v2 feature and its migration

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4f0ff0f80ba0ef6273c31345e93244ccc37cfa69)

doc/radosgw: document the notification_v2 zone feature

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 30d97250f945e0afdfe9ca52a97a42922091d4dd)

doc/radosgw: generalize description of Zone Features

so they make sense outside of multisite configurations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0a97eb89e1d9c99e96fd215e9dc31febc2e44de5)

doc/radosgw: move the Zone Features section to its own page

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d5ac275694bce35d37d7226b2f63934e3c277cbd)

qa/rgw: notification suite creates a realm

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7ef94cdb1c97f537561a7c8bb226d9d22ff709a8)

qa/rgw: teach rgw.py to create an optional realm

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4ac0b7df3c15140bc9d904a1e12fac482c7130be)

rgw/notifications: delete persistent queue only if topic is deleted

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 666e79f1fb78fe8128791e9e23159571f76cfe70)

rgw/notifications: handle migration state between v1 and v2

test instructions:
https://gist.github.com/yuvalif/21449e301732b719cd1ed97c3eeeabb2

* during migration all topic and notification operations must fail with HTTP error code 503
* read operations should return the values of the v1 topics and notifications
* sending notifications should continue based on v1 values

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit bcd79d23df63e72839c652cbc20ebb67a024ba85)

rgw: move rgw_rest_pubsub.h out of driver/rados/

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5b2ae726a95c9922a5c5ba984d6e170dd115128b)

rgw/pubsub: topic apis forward post body

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9d46c21bc6e522b1d85056e1e545125743aace65)

rgw/topic: metadata handler adds/removes persistent queues

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 633a5bef314b5fbc3087271d684f22f9beafb21a)

rgw/topic: remove unused RGWSI_Topic_RADOS

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit be9cc0c7bc163f4e578bd5459866a2c1e5c8a2dd)

rgw/topic: add rgwrados::topic interface for topic metadata

add a new interface for topic metadata that doesn't depend on metadata
backends. this low-level interface is used by both RadosStore and the
topic metadata handler

remove Driver::delete_bucket_topic_mapping() from sal because the omap
object is deleted internally by rgwrados::topic::remove()

remove the RGWRados::topics_pool_ctx member

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6d1d036afb2d1624674fef43f2e70ef3b3ae2859)

rgw/topic: metadata key format in rgw_pubsub.h

the format of topic metadata keys is agnostic to the backend, so the
parsing/formatting functions should be in rgw_pubsub.h

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1caf954dab136e7d5cff97f76ce362c1bfb8d3f4)