doc/radosgw: document iam managed policies

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/radosgw: start on iam/account docs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: load and evaluate group policies

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rename iam_user_policies to iam_identity_policies

identity policies can come from iam groups and roles too

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add Group/GroupPolicy APIs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: ListUserPolicies supports Marker/MaxItems

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: add backend interfaces for group metadata

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add struct RGWGroupInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: OpenIDConnectProvider apis support account users

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: remove virtual class RGWOIDCProvider

class RGWOIDCProvider was doing a lot of different things, so i've split
out its responsibilities:

* move data members and encoding into new struct RGWOIDCProviderInfo,
  and add ceph-dencoder hooks for regression testing
* remove RGWOIDCProvider class and add load/store/delete/list functions
  to the sal::Driver interface
* rgw_rest_oidc_provider.cc handles most of the parameter validation,
  ARN parsing, and json formatting

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: refactor OIDC ops

rearrange the RGWRESTOp subclasses so that the base RGWRestOIDCProvider
can provide a simple verify_permission() that works the same for all
derived ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>

vstart/rgw: add account users for s3-tests

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add commands for managed policy

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: AttachRolePolicy adds managed role policy

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: AttachUserPolicy adds managed user policy

implement iam apis AttachUserPolicy, DetachUserPolicy, and
ListAttachedUserPolicies to manipulate managed user policy

the set of managed policy ARNs is stored in the user attr
RGW_ATTR_MANAGED_POLICY

for incoming requests, the policies from RGW_ATTR_MANAGED_POLICY are
added to s->iam_user_policies at the same time as RGW_ATTR_USER_POLICY

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add get_managed_policy() factory function

add definitions for the following managed policy ARNs:

* arn:aws:iam::aws:policy/IAMFullAccess
* arn:aws:iam::aws:policy/IAMReadOnlyAccess
* arn:aws:iam::aws:policy/AmazonSNSFullAccess
* arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess
* arn:aws:iam::aws:policy/AmazonS3FullAccess
* arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

factory function get_managed_policy() returns a parsed Policy for the
requested ARN if available

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add lots of actions needed for managed policies

in order to parse managed policies, we have to recognize all of the
actions and wildcards they use

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: Policy() takes string instead of bufferlist

the constructor immediately called bufferlist::to_str() to convert it
into a string; just take string so callers don't have to convert it

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: evaluate_iam_policies() handles account root user

> By default, all requests are implicitly denied with the exception of
> the AWS account root user, which has full access.

the account root user turns an implicit deny from identity policy into
an allow, though other policies can still deny explicitly

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: account users match account arns

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add cross-account policy evaluation

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add generic evaluate_iam_policies()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: verify_permission logs acl grants

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: adapt verify_user_permission() for account users

Signed-off-by: Casey Bodley <cbodley@redhat.com>

vstart/rgw: add default config for sts

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/role: support Description for Create/Get/UpdateRole

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add s3:Get/PutBucketOwnershipControls

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: remove load_account_role_by_name()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/role: role APIs support account users

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/role: separate dump_iam_role() for iam api

create a new dump_iam_role() for iam api responses that dumps the subset
of role information presented by the apis

RGWRoleInfo::dump() and decode_json() are used by metadata sync to
transfer role metadata between zones, so must contain all information
about the role

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add pagination to ListRoles

rename sal::Driver::get_roles() to list_roles() and add pagination
support for the RGWListRoles op and 'radosgw-admin role list'

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: enable Role apis against account users

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: role apis override init_processing/verify_permission

replace get_params() with init_processing() override which runs before
verify_permission(). use this to validate request parameters and load
the existing role if necessary. simplify verify_permission() by
forwarding to RGWRESTOp::verify_permission() which calls check_caps()

simplify inheritence hierarchy by taking cap perm and iam action as
constructor arguments

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: split RGWRestRole member variables

move member variables into the subclasses they're needed for

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: RGWUntagRole uses lower/upper bounds for iteration

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: RGWRestRole::parse_tags() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: AccessKey apis call forward_iam_request_to_master()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: User apis call forward_iam_request_to_master()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: move forward_iam_request_to_master() to rgw_rest_iam.*

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rest: enable iam UserPolicy apis against account users

when the authenticated user belongs to an account:
* operate only on that account's users
* match UserName to user's display_name instead of user_id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rest: simplify RGWRestUserPolicy hierarchy

base class constructor takes `uint64_t action` instead of overriding
the virtual `get_op()` on each subclass

constructor takes `uint32_t perm` instead of deriving separate base
classes RGWUserPolicyRead/Write for check_caps() permission

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rest: iam user policy api cleanup

make get_params() virtual and protected. base class always validates
UserName

add common init_processing() function that calls get_params() and loads
the user by UserName. this step happens before verify_permission()

set s->err.message in several error paths

add the xmlns="https://iam.amazonaws.com/doc/2010-05-08/" part to the
responses

return ERR_LIMIT_EXCEEDED instead of ERR_INVALID_REQUEST when
RGWPutUserPolicy exceeds the policy limit

where RGW_ATTR_USER_POLICY is missing, treat it the same way we treat an
empty map of policies. this avoids separate error paths

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: Identity matches account user principals

when a user belongs to an account, they match Principal ARNs by account
id instead of tenant name, and by user name instead user id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: Identity matches paths in user principals

when RGWUserInfo::path is present, use it when matching user principals

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: Identity::is_identity() takes one Principal

take a single Principal instead flat_set<Principal>, and iterate over
calls to is_identity() instead

why?
* it simplifies the logic of each is_identity() function because they
  can use early returns to avoid visiting all of the cases
* Statement::eval_principal() no longer has to allocate a flat_set
  with a single element when the Identity is a role
* rgw::auth::Identity no longer depends on rgw::iam's choice of
  container type

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: rename rgw::auth::Principal::Tenant to Account

just changes the name to match its use in AWS, without changing any
behavior in rgw policy parsing/evaluation

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: add interfaces for account roles

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: add rgwrados::roles namespace abstraction for cls_user

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: init_quota() loads owner quota unconditionally

now that owners can be accounts, don't default to s->user when s->owner
matches s->bucket_owner

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: quota commands can set account quota

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add IAM AccessKey apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/user: add 'create_date' to RGWAccessKey

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/user: expose functions to generate access/secret keys

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add initial IAM User apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rest: wrap iam/sns/sts Error responses with ErrorResponse

all iam/sns/sts requests wrap the s3 <Error> xml response in another
<ErrorResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">

without this, boto3 fails to fully parse error responses, leading to
generic Unknown ClientError exceptions of the form:
    botocore.exceptions.ClientError: An error occurred (Unknown) when calling the PutUserPolicy operation: Unknown

with the ErrorResponse part, boto3 throws more specific exceptions that
include the error Code and Message:
    botocore.errorfactory.NoSuchEntityException: An error occurred (NoSuchEntity) when calling the PutUserPolicy operation: No such UserName in the account

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: user metadata links to account index

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: add interfaces for account users

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: add rgwrados::users namespace abstraction for cls_user

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: use switch for is_non_s3_op()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/op: separate IAM ops in enum RGWOpType

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: use enum for action values

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/iam: add iam user metadata (path, create_date, tags)

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/cls: add ceph_test_cls_user

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/user: add interfaces to index user account resources

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: move WITH_RADOSGW checks up to test/CMakeLists.txt

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/multisite: RGWBucketMetadataHandler updates linkage on owner change

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update bucket ownership when adding user to account

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: Bucket::chown() updates owner on bucket instance

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add RGWIdentityType::TYPE_ROOT for account root user

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: create accounts for random s3test users

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/acl: create_policy_from_headers() can yield

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/acl: s3 CanonicalUser grants support account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: add load_aclowner_by_email()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: move list_buckets() to Driver

move User::list_buckets() to Driver and take rgw_owner to serve bucket
listings for account owners

also unifies the user/account stats interfaces around rgw_owner in
Driver

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: use rgw_owner in RGWBucketInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/acl: use rgw_owner for ACLOwner

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: use rgw_owner for topic ownership

allow topics to be owned by accounts instead of users. radosgw-admin
topic list can now filter by --account-id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: use rgw_owner in RGWBucketEntryPoint

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: add rgwrados::buckets namespace abstraction for cls_user

move cls_user stuff out of RGWSI_User_RADOS into namespace
rgwrados::buckets that doesn't depend on rgw_user, so it can also be
used for account bucket owners

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: add account metadata handler

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: implement account metadata operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: generalize RGWUID for accounts

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: add .rgw.meta:accounts pool

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/quota: generalize quota to rgw_owners

use rgw_owner instead of rgw_user as a key in the quota cache stats.
only fetch_stats_from_storage() and init_refresh() need to differentiate
between user and account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: Identity::is_owner/admin_of(rgw_owner)

is_owner_of() and is_admin_of() take rgw_owner instead of rgw_user so
that identities associated with an account share ownership of that
account's resources

LocalApplier is the only Identity type that supports accounts, based on
comparison with RGWUserInfo::account_id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: account users also match ACL grants to their account id

ACL grants can now specify an account id for the CanonicalUser to
grant access to the entire account. this is implemented only for
LocalApplier

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw_owner variant with json encoding

existing buckets and objects use `struct rgw_user` for the owner. with
the addition of accounts, we need to be able to represent ownership by
the account rather than its users

add a `rgw_owner` variant that preserves the existing json encoding of
`rgw_user` while adding a new representation for account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: enable 'user modify --account-id'

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add /admin/account rest apis

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: add account admin commands

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add account admin ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: add account interfaces to Driver

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add account_id to RGWUserInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add struct RGWAccountInfo

initial design and prototype by Abhishek

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: define account ids and names

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: add gen_rand_numeric()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: pass in ACLOwner for object writes

`ACLOwner` contains both the user id and display name. the bucket index
needs both values for:
```
struct rgw_bucket_dir_entry_meta {
  ...
  std::string owner;
  std::string owner_display_name;
```
`RGWRados::Bucket::UpdateIndex::complete()` relied on the parsing of
`RGW_ATTR_ACL` to get those values. but object write operations already
had that information earlier in the call stack, so we might as well pass
them in directly

for other operations like the copy/rewrite/transition of existing objects,
we decode the owner from the source object's `RGW_ATTR_ACL`

the existing `owner` param was confusing, as it represented the bucket
owner for quota stats updates. this get renamed to `bucket_owner` inside
of `RGWRados`, and hidden from the outside. bucket stats are attributed to
`RGWBucketInfo::owner`, not the `ACLOwner`, so we use that instead of
`s->bucket_owner`

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/acl: use ACLOwners for create_default()

initialize RGWAccessControlPolicy with ACLOwners from the auth identity

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: initialize s->owner in Strategy::apply()

for a common location that applies to all rest handlers that implement
authorize()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: Identity::get_aclowner() for resources it creates

Signed-off-by: Casey Bodley <cbodley@redhat.com>