doc: reef 18.2.4 release note

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #56626 from rhcs-dashboard/fix-prometheus-conf-mgr-failover

mgr/dashboard: fix duplicate grafana panels when on mgr failover

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56694 from afreen23/wip-features-to-advanced

mgr/dashboard: Move features to advanced section and expand by default rbd config section

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56847 from rhcs-dashboard/provide-hub-HA-multi-cluster

mgr/dashboard: provide hub Cluster HA for multi-cluster setup

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56907 from ivoalmeida/snap-schedule-remove-minutely-retention-policy

mgr/dashboard: remove minutely from retention

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #56905 from zdover23/wip-doc-2024-04-16-security-gpg-info-remove

doc/security: remove old GPG information

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mgr/dashboard: provide hub Cluster HA for multi-cluster setup

Fixes: https://tracker.ceph.com/issues/65499
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #56885 from Svelar/asan_OSDScrub

test/osd/TestOSDScrub: free res_ninit/res_nquery  and MonClient resource

Merge pull request #56686 from rhcs-dashboard/fix-prometheus-federation--target-ip

mgr/dashboard: fetch prometheus api host with ip addr

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: remove minutely from retention dd

Removed "minutely" from retention policy dropdown

Fixes: https://tracker.ceph.com/issues/65493
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

doc/security: remove old GPG information

Remove a GPG key that was lost in a server upgrade, and remove the text
that describes it.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #56767 from adk3798/cephadm-log-only-match

qa/cephadm: only fail on CEPHADM_ error in logs

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #56821 from phlogistonjohn/jjm-tox-no-basepython

python: remove some vestigial tox basepython directives

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #56879 from zdover23/wip-doc-2024-04-15-cephfs-client-auth

doc/cephfs: improve ceph-fuse command

Merge pull request #56609 from Svelar/fix_tox_cephadm

test/cephadm: fix timeout issue

test/osd/TestOSDScrub: free MonClient resource

2. free MonClient using Monclient::shutdown

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

common/dns_resolve: free res_ninit/res_nquery

When sanitizer is enabled, unittest_osdscrub shows

```
=================================================================
==1633952==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 28 byte(s) in 1 object(s) allocated from:
    #0 0xaaaab4e108e0 in malloc (/root/ceph/build/bin/unittest_osdscrub+0x1ed08e0) (BuildId: b3cfa2137be96d75535beecf0f2500cec10c7550)
    #1 0xffffa8cac2f8 in __res_context_send resolv/./resolv/res_send.c:334:9
    #2 0xffffa8ca9c54 in __res_context_query resolv/./resolv/res_query.c:216:6
    #3 0xffffa8caa4a8 in __res_context_querydomain resolv/./resolv/res_query.c:625:9
    #4 0xffffa8caa4a8 in __res_context_search resolv/./resolv/res_query.c:381:9
    #5 0xffffa8caaa20 in context_search_common resolv/./resolv/res_query.c:550:16
    #6 0xffffa8caaa20 in res_nsearch resolv/./resolv/res_query.c:563:10
    #7 0xffffabbf1f64 in ceph::ResolvHWrapper::res_nsearch(__res_state*, char const*, int, int, unsigned char*, int) /root/ceph/src/common/dns_resolve.cc:37:10
    #8 0xffffabbf6574 in ceph::DNSResolver::resolve_srv_hosts(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ceph::DNSResolver::SRV_Protocol, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ceph::DNSResolver::Record, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ceph::DNSResolver::Record> > >*) /root/ceph/src/common/dns_resolve.cc:295:19
    #9 0xffffac8edaf0 in MonMap::init_with_dns_srv(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, std::ostream&) /root/ceph/src/mon/MonMap.cc:935:36
    #10 0xffffac8eeec8 in MonMap::build_initial(ceph::common::CephContext*, bool, std::ostream&) /root/ceph/src/mon/MonMap.cc:1014:20
    #11 0xffffac85beb0 in MonClient::build_initial_monmap() /root/ceph/src/mon/MonClient.cc:93:18
    #12 0xaaaab4e50d98 in TestOSDScrub_scrub_time_permit_Test::TestBody() /root/ceph/src/test/osd/TestOSDScrub.cc:73:6
    #13 0xaaaab4f655b0 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #14 0xaaaab4f16264 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #15 0xaaaab4ec6ca8 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #16 0xaaaab4ec8bec in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #17 0xaaaab4eca1ec in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #18 0xaaaab4ee5fb0 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #19 0xaaaab4f6f4c4 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #20 0xaaaab4f1d4bc in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #21 0xaaaab4ee5428 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #22 0xaaaab4e4b790 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #23 0xaaaab4e49dbc in main /root/ceph/src/test/unit.cc:45:10
    #24 0xffffa8bc73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #25 0xffffa8bc74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #26 0xaaaab4d9972c in _start (/root/ceph/build/bin/unittest_osdscrub+0x1e5972c) (BuildId: b3cfa2137be96d75535beecf0f2500cec10c7550)

-----------------------------------------------------
Suppressions used:
  count      bytes template
      1         45 ^MallocExtension::Initialize
-----------------------------------------------------

SUMMARY: AddressSanitizer: 28 byte(s) leaked in 1 allocation(s).
```

1. 'res_ninit/res_nquery' memory should be freed.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #52650 from guits/cv-refactor-osd-objectstore

ceph-volume: osd objectstore refactor

doc/cephfs: improve ceph-fuse command

Instruct readers to use "mkdir /mnt/cephfs1" to create a mountpoint
before using "ceph-fuse" to mount a filesystem, if "/mnt/cephfs1"
doesn't already exist. cf.
https://github.com/ceph/ceph/pull/56831#discussion_r1561102227

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #56775 from cyx1231st/wip-crimson-osd-report-stats

crimson/osd: implement basic reactor-utilization stats report to log

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #56870 from zdover23/wip-doc-2024-04-13-cephfs-client-auth

doc/cephfs: disambiguate "Reporting Free Space"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/cephfs: disambiguate "Reporting Free Space"

Make a few sentences clearer in doc/cephfs/ceph-auth.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #56859 into main

* refs/pull/56859/head:
script/ptl-tool: add switch for tags
script/ptl-tool: conditionally add "QA Release" field

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #54333 from cbodley/wip-rgw-account-v3

rgw: user accounts implementation

Reviewed-by: Adam Emerson <aemerson@redhat.com>

mgr/dashboard: Move features to advanced section

Fixes https://tracker.ceph.com/issues/65207

- Moves "features" section in rbd image create form to "Advanced" section.
- makes rbd configuration section to be expanded by default rather than
  being collapsed as it has only single section. This will improve user experience as it will not
require two clicks.
- updates e2e test

Signed-off-by: Afreen <afreen23.git@gmail.com>

Merge pull request #56282 from ivoalmeida/snapshot-schedule-repeat-frequency-validation

mgr/dashboard: snapshot schedule repeat frequency validation

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56801 from ascheglov/feature/notask/movable-rbd-image

librbd: make librbd::Image moveable

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

script/ptl-tool: add switch for tags

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: conditionally add "QA Release" field

Use the redmine default.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #56845 from Svelar/asan_rgw_crypto

test/test_rgw_crypto: free allocated test_in

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55320 from rishabh-d-dave/mdscaps-update-issues

cephfs,mon: fix bugs related to updating MDS caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #56854 from Svelar/asan_bluestore_type

bluestore/bluestore_types: check 'it' valid before using

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #56758 from ivoalmeida/snapshot-schedule-fix-rentention-policy-delete-arg

mgr/dashboard: fix snap schedule delete retention

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: fetch prometheus api host with ip addr

Check if the PROMETHEUS_API_HOST value has the ip addr of the host if
not replace hostnmame with ip addr

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #56827 from guits/cv-func-tests

ceph-volume: update functional testing

Merge pull request #56829 from Svelar/asan_fix_mds

test/TestQuiesceAgent: free quiesce_requests Context

ceph-volume: update functional testing

various changes for fixing ceph-volume functional testing.
cleaning up deploy.yml (drop py2 references)

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #56831 from zdover23/wip-doc-2024-04-11-cephfs-client-auth-3-of-3

doc/cephfs: refine client-auth (3 of 3)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

bluestore/bluestore_types: check 'it' valid before using

When sanitizer is enabled, unittest_bluestore_types fails as following
```
[ RUN      ] sb_info_space_efficient_map_t.basic
=================================================================
==143714==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xffff99f8b7f4 at pc 0xaaaab50bde18 bp 0xffffebefcdb0 sp 0xffffebefcda8
READ of size 8 at 0xffff99f8b7f4 thread T0
    #0 0xaaaab50bde14 in sb_info_t::get_sbid() const /root/ceph/src/os/bluestore/bluestore_types.h:1337:30
    #1 0xaaaab50a5908 in sb_info_space_efficient_map_t::find(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1385:10
    #2 0xaaaab50bd638 in sb_info_space_efficient_map_t::_add(long) /root/ceph/src/os/bluestore/bluestore_types.h:1424:15
    #3 0xaaaab50a52bc in sb_info_space_efficient_map_t::add_maybe_stray(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1358:12
    #4 0xaaaab4fec03c in sb_info_space_efficient_map_t_basic_Test::TestBody() /root/ceph/src/test/objectstore/test_bluestore_types.cc:113:11
    #5 0xaaaab51e9a40 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #6 0xaaaab5197040 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #7 0xaaaab51488a4 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #8 0xaaaab514a7e8 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #9 0xaaaab514bde8 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #10 0xaaaab5167bac in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #11 0xaaaab51f3940 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #12 0xaaaab519e5d8 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #13 0xaaaab5167024 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #14 0xaaaab50b4d6c in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #15 0xaaaab50a1080 in main /root/ceph/src/test/objectstore/test_bluestore_types.cc:2847:10
    #16 0xffff9d6c73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #17 0xffff9d6c74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #18 0xaaaab4f3812c in _start (/root/ceph/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)

0xffff99f8b7f4 is located 0 bytes to the right of 20-byte region [0xffff99f8b7e0,0xffff99f8b7f4)
allocated by thread T0 here:
    #0 0xaaaab4fe636c in operator new[](unsigned long) (/root/ceph/build/bin/unittest_bluestore_types+0xef636c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)
    #1 0xaaaab50c0d2c in mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t>::allocate(unsigned long, void*) /root/ceph/src/include/mempool.h:375:33
    #2 0xaaaab50c0c0c in std::allocator_traits<mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::allocate(mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t>&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:318:20
    #3 0xaaaab50c044c in std::_Vector_base<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20
    #4 0xaaaab50bf954 in void std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::_M_realloc_insert<long&>(__gnu_cxx::__normal_iterator<sb_info_t*, std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> > >, long&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:440:33
    #5 0xaaaab50be0d8 in sb_info_t& std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::emplace_back<long&>(long&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:121:4
    #6 0xaaaab50bd760 in sb_info_space_efficient_map_t::_add(long) /root/ceph/src/os/bluestore/bluestore_types.h:1429:24
    #7 0xaaaab50a5e78 in sb_info_space_efficient_map_t::add_or_adopt(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1361:15
    #8 0xaaaab4feb07c in sb_info_space_efficient_map_t_basic_Test::TestBody() /root/ceph/src/test/objectstore/test_bluestore_types.cc:103:11
    #9 0xaaaab51e9a40 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0xaaaab5197040 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0xaaaab51488a4 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #12 0xaaaab514a7e8 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #13 0xaaaab514bde8 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #14 0xaaaab5167bac in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #15 0xaaaab51f3940 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #16 0xaaaab519e5d8 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #17 0xaaaab5167024 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #18 0xaaaab50b4d6c in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #19 0xaaaab50a1080 in main /root/ceph/src/test/objectstore/test_bluestore_types.cc:2847:10
    #20 0xffff9d6c73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #21 0xffff9d6c74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #22 0xaaaab4f3812c in _start (/root/ceph/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)

SUMMARY: AddressSanitizer: heap-buffer-overflow /root/ceph/src/os/bluestore/bluestore_types.h:1337:30 in sb_info_t::get_sbid() const
Shadow bytes around the buggy address:
  0x200ff33f16a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f16e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x200ff33f16f0: fa fa fa fa fa fa fa fa fa fa fa fa 00 00[04]fa
  0x200ff33f1700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1710: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1720: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1730: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x200ff33f1740: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==143714==ABORTING
```

'it' might be invalid, so before using 'it', need to figure validity out

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge PR #53564 into main

* refs/pull/53564/head:
qa: add split/merge dirfrag tests for scrub
mds: Add fragment to scrub

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

test/test_rgw_crypto: free allocated test_in

When sanitizer is enabled, unittest__rgw_crypto shows

```
=================================================================
==136464==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 75023 byte(s) in 22 object(s) allocated from:
    #0 0xaaaabf7fb86c in operator new[](unsigned long) (/root/ceph/build/bin/unittest_rgw_crypto+0x48b86c) (BuildId: 8023dc30820215da92d6d4883620bedd8ac1190d)
    #1 0xaaaabf81db48 in TestRGWCrypto_verify_Encrypt_Decrypt_Test::TestBody() /root/ceph/src/test/rgw/test_rgw_crypto.cc:780:24
    #2 0xaaaabf9018ac in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0xaaaabf8b08a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0xaaaabf861f88 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0xaaaabf863ecc in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0xaaaabf8654cc in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0xaaaabf881290 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0xaaaabf90b7ac in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0xaaaabf8b7ac0 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0xaaaabf880708 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0xaaaabf823d70 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0xaaaabf81f390 in main /root/ceph/src/test/rgw/test_rgw_crypto.cc:822:10
    #13 0xffff878673f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0xffff878674c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0xaaaabf74d62c in _start (/root/ceph/build/bin/unittest_rgw_crypto+0x3dd62c) (BuildId: 8023dc30820215da92d6d4883620bedd8ac1190d)

SUMMARY: AddressSanitizer: 75023 byte(s) leaked in 22 allocation(s).
```

test_in should be freed to address the warning.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #56784 from zdover23/wip-doc-2024-04-09-cephfs-client-auth-2-of-3

doc/cephfs: refine client-auth (2 of 3)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

Merge pull request #56734 from zmc/cmake-sccache

cmake: Support sccache

Merge PR #56839 into main

* refs/pull/56839/head:
script/ptl-tool: push branch to shaman for qa runs
script/ptl-tool: add release name to branch with switch
script/ptl-tool: alphabetize arguments
script/ptl-tool: avoid repo specific remotes entirely
script/ptl-tool: improve help text for envvars

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

librbd: make librbd::Image moveable

Adds move constructor and move assignment operator to the librbd::Image.
Also marks copy ctor/assign op as deleted, and makes them public for better compiler diagnostics.

Signed-off-by: Anatoly Scheglov <finch@mts.ru>

test/rgw/pubsub: wait_for_queue_to_drain() supports tenant/account topics

Signed-off-by: Casey Bodley <cbodley@redhat.com>

script/ptl-tool: push branch to shaman for qa runs

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: add release name to branch with switch

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: alphabetize arguments

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: avoid repo specific remotes entirely

So we don't need to configure the remote name or paths at all. Just use
universal ssh remote URLs instead.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Revert "RGW: a subuser with no permission can still list buckets and create buckets"

This reverts commit 3cc27f0676c7ba2677f92969339b18b665c53c02.

Signed-off-by: Casey Bodley <cbodley@redhat.com>

script/ptl-tool: improve help text for envvars

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #56835 into main

* refs/pull/56835/head:
script/ptl-tool: create qa trackers for test branches
script/ptl-tool: add switch for debugging
script/ptl-tool: add --stop-at-built flag

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

script/ptl-tool: create qa trackers for test branches

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: add switch for debugging

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/ptl-tool: add --stop-at-built flag

To make modifications to the branch before it is tagged and a branch name
created.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #55381 from linuxbox2/wip-rgw-sigxfsz

rgw: ignore SIGXFSZ, which apparently can triggered by heavy ops-log …

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #56794 from adk3798/cephadm-raw-osd-teuthology-test

qa/cephadm: teuthology test for deploying raw OSDs

Merge pull request #56785 from cbodley/wip-uninstall-motr

install-deps: remove cortx-motr integration

Reviewed-by: Zack Cerza <zack@redhat.com>

doc/cephfs: refine client-auth (3 of 3)

Refine the final third of doc/cephfs/client-auth.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

test/TestQuiesceAgent: free quiesce_requests Context

When sanitizer is enabled, unittest_mds_quiesce_agent fails as following

```
[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from QuiesceAgentTest
[ RUN      ] QuiesceAgentTest.ThreadManagement
[       OK ] QuiesceAgentTest.ThreadManagement (3 ms)
[ RUN      ] QuiesceAgentTest.DbUpdates
[       OK ] QuiesceAgentTest.DbUpdates (1 ms)
[ RUN      ] QuiesceAgentTest.QuiesceProtocol
[       OK ] QuiesceAgentTest.QuiesceProtocol (3 ms)
[ RUN      ] QuiesceAgentTest.DuplicateQuiesceRequest
[       OK ] QuiesceAgentTest.DuplicateQuiesceRequest (2 ms)
[ RUN      ] QuiesceAgentTest.TimeoutBeforeComplete
[       OK ] QuiesceAgentTest.TimeoutBeforeComplete (2 ms)
[----------] 5 tests from QuiesceAgentTest (11 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test suite ran. (11 ms total)
[  PASSED  ] 5 tests.

=================================================================
==3975692==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0xaaaadd81c7c8 in operator new(unsigned long) (/root/ceph/build/bin/unittest_mds_quiesce_agent+0x1fc7c8) (BuildId: 7d45344ba1e43661d9de484f0a5d129377c4d4ae)
    #1 0xaaaadd8878d8 in QuiesceAgent::agent_thread_main() /root/ceph/src/mds/QuiesceAgent.cc:136:68
    #2 0xaaaadd86de38 in QuiesceAgent::AgentThread::entry() /root/ceph/src/mds/QuiesceAgent.h:244:24
    #3 0xffff83d6b554 in Thread::entry_wrapper() /root/ceph/src/common/Thread.cc:87:10
    #4 0xffff83d6b314 in Thread::_entry_func(void*) /root/ceph/src/common/Thread.cc:74:29
    #5 0xffff8154d5c4 in start_thread nptl/./nptl/pthread_create.c:442:8
    #6 0xffff815b5ed8  misc/../sysdeps/unix/sysv/linux/aarch64/clone.S:79

Indirect leak of 120 byte(s) in 1 object(s) allocated from:
    #0 0xaaaadd81c7c8 in operator new(unsigned long) (/root/ceph/build/bin/unittest_mds_quiesce_agent+0x1fc7c8) (BuildId: 7d45344ba1e43661d9de484f0a5d129377c4d4ae)
    #1 0xaaaadd8af4f4 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0xaaaadd8af3d8 in std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0xaaaadd8af3d8 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #4 0xaaaadd8aef00 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, (__gnu_cxx::_Lock_policy)2> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocated_ptr.h:98:21
    #5 0xaaaadd8aec14 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(QuiesceAgent::TrackedRoot*&, std::_Sp_alloc_shared_tag<std::allocator<QuiesceAgent::TrackedRoot> >, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:648:19
    #6 0xaaaadd8ae988 in std::__shared_ptr<QuiesceAgent::TrackedRoot, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(std::_Sp_alloc_shared_tag<std::allocator<QuiesceAgent::TrackedRoot> >, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1342:14
    #7 0xaaaadd8ae70c in std::shared_ptr<QuiesceAgent::TrackedRoot>::shared_ptr<std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(std::_Sp_alloc_shared_tag<std::allocator<QuiesceAgent::TrackedRoot> >, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:409:4
    #8 0xaaaadd8ae484 in std::shared_ptr<QuiesceAgent::TrackedRoot> std::allocate_shared<QuiesceAgent::TrackedRoot, std::allocator<QuiesceAgent::TrackedRoot>, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(std::allocator<QuiesceAgent::TrackedRoot> const&, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:862:14
    #9 0xaaaadd88ff0c in std::shared_ptr<QuiesceAgent::TrackedRoot> std::make_shared<QuiesceAgent::TrackedRoot, QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&>(QuiesceState&, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:878:14
    #10 0xaaaadd884a6c in QuiesceAgent::db_update(QuiesceMap&) /root/ceph/src/mds/QuiesceAgent.cc:60:26
    #11 0xaaaadd84a840 in QuiesceAgentTest::update(QuiesceDbVersion, std::initializer_list<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, QuiesceMap::RootInfo> >) /root/ceph/src/test/mds/TestQuiesceAgent.cc:156:18
    #12 0xaaaadd84985c in QuiesceAgentTest::update(unsigned long, std::initializer_list<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, QuiesceMap::RootInfo> >) /root/ceph/src/test/mds/TestQuiesceAgent.cc:165:14
    #13 0xaaaadd8288a8 in QuiesceAgentTest_DbUpdates_Test::TestBody() /root/ceph/src/test/mds/TestQuiesceAgent.cc:213:16
    #14 0xaaaadd977230 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #15 0xaaaadd924590 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #16 0xaaaadd8d4a40 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
    #17 0xaaaadd8d6984 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
    #18 0xaaaadd8d7f84 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
    #19 0xaaaadd8f3d48 in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
    #20 0xaaaadd981130 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
    #21 0xaaaadd92bb64 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
    #22 0xaaaadd8f31c0 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
    #23 0xaaaadd820710 in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #24 0xaaaadd81ed3c in main /root/ceph/src/test/unit.cc:45:10
    #25 0xffff814f73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #26 0xffff814f74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
    #27 0xaaaadd76e6ac in _start (/root/ceph/build/bin/unittest_mds_quiesce_agent+0x14e6ac) (BuildId: 7d45344ba1e43661d9de484f0a5d129377c4d4ae)

SUMMARY: AddressSanitizer: 184 byte(s) leaked in 2 allocation(s).
```

quiesce_requests Context should be freed.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

test/cephadm: extend timeout limit to fix timeout issue

When run make check(arm64) on omani series servers, eg.
https://jenkins.ceph.com/job/ceph-pull-requests-arm64/54751/.
Shows:
python: timeout after 30 seconds
INFO     root:call_wrappers.py:284 Non-zero exit code 124 from /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/cephadm/.tox/py3/bin/python -c for i in range(1000000): print(i, flush=True)

Fixes: https://tracker.ceph.com/issues/65355
Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #56561 from phlogistonjohn/jjm-issue65122-maint-cmd

cephadm: fix host-maintenance command always exiting with a failure

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #56716 from adk3798/test_cephadm_images

qa/cephadm: update images for test_cephadm workunit

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56481 from adk3798/test-cephadm-idmap-conf

cephadm: add idmap.conf to nfs sample file

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56791 from adk3798/nvmeof-mon-setting

mgr/cephadm: make enable_monitor_client configurable for nvmeof

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56613 from NitzanMordhai/wip-nitzan-osd-down-ignore-cephadm-suites

suites/rados/cephadm: adding OSD_DOWN to the log-ignorelist

Reviewed-by: Adam King <adking@redhat.com>

qa/rgw/s3tests: remove 'client.0' from bucket prefix

new sns test cases are using this for topic names, but the '.' is not
allowed there:

> api_params = {'Name': 'test-client.0-n3bdgre5el2jk8v-606'}
> botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the CreateTopic operation: Name must be made up of only uppercase and lowercase ASCII letters, numbers, underscores, and hyphens

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/notify: populate event userIdentity with account ids

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw/pubsub: test persistent notifications with account user

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/notify: support cross-tenant and cross-account notifications

a bucket's notification configuration may refer to topics from several
different tenants or accounts. when publishing to a given topic, look in
the correct namespace for each topic instead of defaulting to the
requesting user's tenant namespace

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: forward requests as s->owner instead of s->user

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/notify: publish functions use rgw_pubsub_dest::persistent_queue

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/rgw: warn about topics under account migration

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: use rgw_pubsub_dest::persistent_queue for queue oid

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: RGWPubSub::remove_topic() removes persistent queue

move the persistent queue removal into remove_topic() where we have
access to the topic metadata. avoid trying to remove the queue if it
isn't enabled

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: rgw_pubsub_dest stores persistent queue oid

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: add/remove_persistent_topic() takes topic queue, not name

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw/pubsub: topic policy doesn't deny access to owner

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: CreateTopic consults existing topic policy for overwrite

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: fix DeleteTopic permissions

non-account users now consult identity policies with
verify_user_permission() when the topic doesn't exist

account users now consult topic policy when it does exist

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: CreateTopic consults identity policies when topic doesn't exist

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: CreateTopic requires notification_v2 for accounts

the account's topic index is only updated by writes/deletes to v2 topic
metadata

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: avoid allocating hash set of strings for attr search

this unordered_set was not static, so we reinitialized it on every call

replace with a constexpr array of string_views so we can search through
sequential memory that's laid out at compile time

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: add ERR_AUTHORIZATION -> AuthorizationError

sns docs specify AuthorizationError as the 403 error code rather than
s3's AccessDenied:

    https://docs.aws.amazon.com/sns/latest/api/API_CreateTopic.html#API_CreateTopic_Errors

boto3 sns clients can catch this as AuthorizationErrorException

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: return 404 NotFound instead of NoSuchKey

repurpose the ERR_NOT_FOUND define which was otherwise unused to
customize the error response for sns apis, which return the NotFound
error code instead of NoSuchKey from s3:

    https://docs.aws.amazon.com/sns/latest/api/API_GetTopicAttributes.html#API_GetTopicAttributes_Errors

this allows boto3 sns clients to catch the NotFoundException as expected

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: notifications can refer to topics in other accounts/tenants

accounts can use topic policy to grant sns:Publish permissions to other
accounts. the PutBucketNotification op should expect TopicArns from
other accounts. the account name from each TopicArn should be used as
the 'tenant' argument for RGWPubSub's constructor so we look for the
topic in the right namespace

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: customize permissions for account users

for account users, CreateTopic and ListTopics permissions come from
identity policy alone, ignoring the ownership/policy of existing topics

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: when present, use account id instead of tenant

RGWPubSub provides topic namespace isolation for tenants by adding
prefixes to rados object names and topic metadata keys. accounts use
this the same way

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: add index for account topics

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: verify_topic_permission handles cross-account access

refactor verify_topic_owner_or_policy() to share the same interface
as similar functions like verify_user/bucket/object_permission()
from rgw_common.cc

in addition to the topic resource policy, this now also consults iam
identity policies like user, group, or role policy

for account users, this now implements cross-account policy evaluation.
this only comes into play for sns:Publish permissions though, because
the topics themselves are scoped to the account

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: rgw_common.h exposes evaluate_iam_policies()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: do init/validation in init_processing()

verify_permission() should do permission checks and nothing else!

admin/system users ignore errors from verify_permission() and go on to
call execute() regardless. that means that execute() can't rely on any
initialization that happened during verify_permission(), at risk of
crashing on admin/system requests. it also means that any permission
checks in execute() won't get overridden for admin/system users,
breaking their superuser access

by moving all parameter validation and initialization into
init_processing(), we can prepare all the state that verify_permission()
will need to do it's thing

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: replace log messages with error response

parameter validation errors should be returned to the client instead of
written to the rgw log

also raises the log level for lots of error messages. very few of them
should require admin attention

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: use existing s->bucket for notification ops

s->bucket is already initialized during rgw_build_bucket_policies(),
called from RGWHandler::do_init_permissions()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: make v2-specific calls private

Signed-off-by: Casey Bodley <cbodley@redhat.com>

PendingReleaseNotes: announce the rgw user account feature

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: reject user tenant that looks like an account id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/auth: log each policy that returns Allow or Deny

makes it much easier to debug authorization issues when you can see
exactly which policies led to success/failure

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/admin: 'user modify' won't change existing account id

Signed-off-by: Casey Bodley <cbodley@redhat.com>