crimson: fix crimson pg coll usage error

coll in pg isn't initialized, should use coll_ref->get_cid() insead.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #33066 from gregsfortytwo/wip-specfile

rpm: fix up a specfile syntax error

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #30033 from theanalyst/rgw/public-buckets

rgw: add PublicAccessBlock set of APIs on buckets

Reviewed-By: Casey Bodley <cbodley@redhat.com>

rpm: fix up a specfile syntax error

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #33049 from yuvalif/wip-yuval-fix-43768

qa/rgw/pubsub: fix tests to sync from master

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: add Pending Release Notes entry on public access config

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #32119 from joke-lee/rgw-sts-crash-duration-invalid

rgw: fix rgw crash when duration is invalid in sts request

Merge pull request #31987 from linuxbox2/rgw-putacls-no-body

rgw: s3: don't require a body in S3 put-object-acl

Merge pull request #30684 from theanalyst/rgw/qa/rgw-admin-user-stats

qa: radosgw_admin: validate a simple user stats output

Reviewed-By: Casey Bodley <cbodley@redhat.com>

Merge PR #33048 into master

* refs/pull/33048/head:
cephadm: fix error handling in `command_check_host()`
cephadm: increase default retry_max value

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32995 into master

* refs/pull/32995/head:
cephadm: add group 'disk' to privileged container
cephadm: adopt: disable ceph-volume unit on host
cephadm: refactor privileged arg handling
cephadm: chown block symlink targets
cephadm-adoption-corpus: add stud-mon.tgz
cephadm: adopt: rename leveldb *.ldb -> *.sst
cephadm: adopt: by default, pull first
cephadm: adopt: chown data content

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33020 into master

* refs/pull/33020/head:
osdc/Objecter: inline pool full check
osdc/Objecter: remove duplicated pause check code
osdc/Objecter: only pause if respects_full()
osdc/Objecter: move respects_full() to op_target_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #32831 into master

* refs/pull/32831/head:
common, include: drop the copy{_in} from bufferlist entirely.
os/bluestore: switch copy_in() users to bufferlist::iterator.
osdc: switch users of bufferlist::copy{_in} to iterators.
osd: switch users of bufferlist::copy{_in} to iterators.
rgw: switch copy{_in} users to bufferlist::iterator.
ec: switch users of bufferlist::copy{_in} to iterators.
cls/queue: switch users of bufferlist::copy{_in} to iterators.
client: switch users of bufferlist::copy{_in} to iterators.
*: switch trivial users of bufferlist::copy{_in} to iterators.
test/bl: switch copy{_in} users to bufferlist::iterator.
common, include: kill the bl::last_p member.
common: encode for std::list<T> doesn't use bl::copy_in() anymore.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32531 from zdover23/wip-doc-landing-page-update

doc: Added the crisp getting started guide to index.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cephadm: add group 'disk' to privileged container

This lets the osd read block devs that are group rw disk even after they
drop root privs.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32977 into master

* refs/pull/32977/head:
qa/workunits/cephadm/test_cephadm.sh: add missing monitoring tests
cephadm: simplify Monitoring.components structure
cephadm: add proper tox type for monitoring components

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

Merge PR #33012 into master

* refs/pull/33012/head:
mgr/cephadm: prefix daemon ids with hostname
cephadm: bootstrap: name mgr with $hostname.$random

Reviewed-by: Sebastian Wagner <swagner@suse.com>

cephadm: adopt: disable ceph-volume unit on host

This might be a simple or lvm unit.  Disable it so that the host doesn't
try to start this OSD after a reboot.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: refactor privileged arg handling

Pass a bool if we want a privileged container instead of explicitly
passing --privileged.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: fix error handling in `command_check_host()`

`find_program()` raises `ValueError` when the executable hasn't been
found. It means we need to catch `ValueError` exception in
`command_check_host()` and raise `Error` instead of `RuntimeError` since
only `Error` is caught at the end.

Typical failure:

```
INFO:cephadm:/usr/bin/ceph:stderr Error ENOENT: New host mon1 failed check: ['INFO:cephadm:podman|docker (/bin/podman) is present', 'INFO:cephadm:systemctl is present', 'Traceback (most recent call last):', '  File "<stdin>", line 2820, in <module>', '  File "<stdin>", line 2434, in command_check_host', '  File "<stdin>", line 796, in find_program', 'ValueError: lvcreate not found']
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

rgw: move public access conf to perm_state_base

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

test: rgw_iam_policy update tests with new Get Actions

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: iam_policy: add all the actions to actpairs map

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: public access: drop unused function

Also cleanup the comment to mention why we've deviated from the spec here

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: rename ACL & policy IsPublic to is_public

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: PublicAccessConfiguration -> PublicAccessBlockConfiguration

also drop iam namespace

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_op: get_public_access_from_attr indent fixes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_rest_s3: use formatter->dump_bool

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

public_access: reuse formatter->dump_bool

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: Fix IgnorePublicACLs for bucket ACLs

Currently Bucket ACLs with IgnorePublicACLs were broken this should fix that

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

drop redundant bucket policy status in rgw_common

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: implement IgnorePublicACLs

This allows for ignoring bucket/object acls that are configured to be public

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: move PublicAccessConfiguration to req_state

This allows for evaluation of more complex use cases where IgnorePublicACLs and
the like are set which need to be evaluated for GET/HEAD requests as well

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: op: Get Policy Status checks if there is a policy first

before evaluating its public nature

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_op: reuse function to get public access conf

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: honor PublicAccessBlockConfiguration for put object

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: block public access for Policies

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: enforce BlockPublicPolicy on put bucket policy

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: public_access: rename the getters to reflect what the conf actually does

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw_op: use ldpp_dout version wherever logs are required

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: move IsPublic to RGWAccessControlPolicy class

This helps reusing when evaluating for PutACLs and Put Policy

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: implement get/put/delete public access block for buckets

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

add RGWPublicAccess for configuring public access settings for a resource

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: both princ and nonprinc will not coexist in a single statement

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: initial implementation of a public policy tester

doesn't cover all the cases involving a nonprinc user yet

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: s3: implement GetBucketPolicyStatus API

This API returns whether the Bucket Policies/ACLs are public. There are a couple
of caveats:
- AWS currently returns PolicyNotFound error in case a bucket policy doesn't
exist, though a non existant bucket policy would mean the default ACLs apply
where the bucket is private, so error return here seems like an error
- the API spec mentions TRUE and FALSE as the response IsPublic element value,
however in practice both boto/aws clients and AWS S3 return/expect a lowercase
response.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Conflicts:
src/rgw/rgw_rest_s3.h
merge conflict after zipper rework, dropped a spurious newline in rgw_rest_s3.h
after get_obj_op decl.
src/rgw/rgw_common.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h:
merge conflict after bucket replication merge, trivial conflicts

rgw: acl: drop unused function & make get_group_perm const

Drop the unused RGWAccessControlPolicy::get_group_perm, make the ACL
get_group_perm as a const member function

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

cephadm: increase default retry_max value

When playing with cephadm, at multiple times, I've reached the max
number of attempt in `is_available()`

Increasing the `retry_max` helps to avoid failure like following:

```
INFO:cephadm:mgr not available, waiting (1/5)...
INFO:cephadm:mgr not available, waiting (2/5)...
INFO:cephadm:mgr not available, waiting (3/5)...
INFO:cephadm:mgr not available, waiting (4/5)...
INFO:cephadm:mgr not available, waiting (5/5)...
ERROR: mgr not available after 5 tries

```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #32155 from yuvalif/wip-yuval-multidelete-notifications

rgw/pubsub: send notifications from multi-delete op

mgr/cephadm: prefix daemon ids with hostname

This is friendlier to a human operator since they can immediately see
where an instance is located, as with the legacy scheme, while still
keeping the unique random suffix.  Use a . to separate so that we can
set per-host options.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32859 from liewegas/wip-cephadm-services

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33030 from smithfarm/wip-luminous-12.2.13-release-notes

doc: release notes for Luminous v12.2.13

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #33031 from smithfarm/wip-nautilus-14.2.7-release-notes

doc: release notes for Nautilus 14.2.7

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #33038 from tchaikov/wip-librados-reinterpret-cast

librados: prefer reinterpret_cast over c-style cast

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #32933 from DimStar77/obs-mini

ceph.spec.in: Use pkgconfig() style BuildRequires for udev/libudev-devel

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32553 from tspmelo/wip-ts-strict

mgr/dashboard: Enable compiler options used by Angular --strict flag

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

librados: prefer reinterpret_cast over c-style cast

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33016 from tchaikov/wip-do-cmake

do_cmake.sh: bail out if something goes wrong

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/workunits/cephadm/test_cephadm.sh: add missing monitoring tests

add tests for node-exporter, prometheus, and grafana

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: simplify Monitoring.components structure

by removing the nested image/metadata dict

Signed-off-by: Michael Fritch <mfritch@suse.com>

More commits to the landing page project.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

rgw/pubsub: fix tests to sync from master

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

cephadm: bootstrap: name mgr with $hostname.$random

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32570 into master

* refs/pull/32570/head:
cephfs-shell: Add tests for setxattr, getxattr and listxattr
cephfs-shell: Add listxattr command
cephfs-shell: Add getxattr command
cephfs-shell: Add setxattr command
doc: Update about extended attributes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31633 into master

* refs/pull/31633/head:
cephfs-shell: Instead of assert use stat for tests in rmdir
cephfs-shell: Add function for common rmdir test code
cephfs-shell: Add rmdir test for non empty directory
cephfs-shell: Add rmdir -p test for non empty directory
cephfs-shell: Add rmdir -p test for non existing dir
cephfs-shell: Add rmdir -p test to delete all dirs in given path
cephfs-shell: Add rmdir -p test for root directory with empty directories
cephfs-shell: Add rmdir test for valid file
cephfs-shell: Add rmdir test for invalid directory
cephfs-shell: Add rmdir test for valid directory
cephfs-shell: Fix rmdir '-p' issues

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32986 into master

* refs/pull/32986/head:
qa/tasks/ceph_manager: fix movement of cot exports with cephadm

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: release notes for Nautilus 14.2.7

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: release notes for Luminous v12.2.13

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32030 from vshankar/wip-mgr-volumes-clone

mgr/volumes: clone from snapshot

Merge PR #33014 into master

* refs/pull/33014/head:
mgr/dashboard: pass a list of drive_group to create_osds

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32973 into master

* refs/pull/32973/head:
mgr/insights: make 'insights prune-health-history 0' zap current state

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #32987 into master

* refs/pull/32987/head:
qa/tasks/ceph_manager: make fix_pgp_num behave when no pool is found

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #32989 into master

* refs/pull/32989/head:
qa/tasks/ceph_manager: add --log-early to raw_cluster_cmd

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #32830 into master

* refs/pull/32830/head:
global: ensure CEPH_ARGS is decoded before early arg processing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32806 into master

* refs/pull/32806/head:
common/bl: fix accessibility of bptr's _off and _len fields.
common/bl: drop get_raw() from the public buffer::ptr interface.
common: drop sharing of buffer::raw outside bufferlist.

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph_manager: fix movement of cot exports with cephadm

I think this will finally work...

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: chown block symlink targets

Again, these need to match the container's uid/gid, not the host's.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm-adoption-corpus: add stud-mon.tgz

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: adopt: rename leveldb *.ldb -> *.sst

We may be adopting from ubuntu, which means we need to rename these files.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: adopt: by default, pull first

This just bit me: I had a months-old container revision locally and ended
up accidentally starting up ancient monitor code.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: adopt: chown data content

The ceph uid/gid vary between distros.  Chown everything to match the
target distro.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: inline pool full check

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: remove duplicated pause check code

_calc_target() already pauses and unpauses based on the
target_should_be_paused() helper, and we just called _calc_target at the
top of this method.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: only pause if respects_full()

If the FULL_TRY or _FORCE flag is set on an op, then we should not pause
it due to a full flag.

Fixes: https://tracker.ceph.com/issues/43813
Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: move respects_full() to op_target_t

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33017 from linuxbox2/wip-rgw-beastc

rgw: improve beast

Merge pull request #33010 from tchaikov/wip-cmd-getval

src/: drop cct from cmd_getval()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

do_cmake.sh: bail out if something goes wrong

so we can be aware if some updated submodule reference is missing in our
repo

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: pass a list of drive_group to create_osds

as orchestrator backends expect a list of drive_groups.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Unable to remove an iSCSI gateway that is alrea… (#32984)

mgr/dashboard: Unable to remove an iSCSI gateway that is already in use

Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add Modified filter to Configuration table (#32923)

mgr/dashboard: Add Modified filter to Configuration table

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

rgw: improve beast

Avoid leaking connections that had partially-consumed
client data on unexpected disconnect.

Resolves CVE-2020-1700 (moderate impact flaw).

Fixes: https://tracker.ceph.com/issues/42531
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

cephadm: add proper tox type for monitoring components

instead of `type: ignore`

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge branch 'octopus'

mgr/dashboard: Enable noImplicitAny TS compiler option

Fixes: https://tracker.ceph.com/issues/43525
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enable noImplicitReturns TS compiler option

Fixes: https://tracker.ceph.com/issues/43525
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enable noImplicitThis TS compiler option

Fixes: https://tracker.ceph.com/issues/43525
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Enable noFallthroughCasesInSwitch TS compiler option

Fixes: https://tracker.ceph.com/issues/43525
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Move TS compile rules into tsconfig.json

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #32999 from liewegas/fix-found

doc/foundation: fix amihan