crimson/.../client_request: work around gcc bz98401

See included comment for details.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: work around gcc bz101244 and bz102217

See included comment for details.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: convert ClientRequest::do_process to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: convert process_op to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: make reply_op_error return interruptible_future<>

co_await can't implicitely convert it.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: convert ClientRequest::process_pg_op to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: convert with_pg_process_interruptible coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: factor out with_pg_interruptible

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: don't pass Ref<PG> by reference

If we only need a reference to the PG, pass a PG&.  Passing Ref<PG>&
makes it easy to inadvertently std::move() the passed value from
a caller.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request: rename with_pg_int to with_pg_process

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../client_request.cc: move message decode check to with_pg

We only need to do this once, no need to recheck on requeue.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #55979 from athanatos/sjust/wip-64546-max-creating-pgs

mon: always create pgs in the epoch in which the pool was created

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #56533 from phlogistonjohn/jjm-cpatch-before

cpatch: add --run-before command line option

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #56136 from NitzanMordhai/wip-nitzan-init-hash-in-chunk-ref-by-hash

cls/cas/cls_cas_internal: Initialize 'hash' value before decoding

Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>

Merge pull request #54991 from ifed01/wip-ifed-some-recover-stuff

tools/ceph-objectstore-tool: tricks to tolerate disk errors for "pg export" command.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #56512 from rhcs-dashboard/remove-token-authentication-multi-cluster

mgr/dashboard: Remove token authentication multi cluster

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: remove the option for authentication with token in
multi-cluster form

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #55860 from ceph/add-multi-cluster-token-ttl

mgr/dashboard: Add multi cluster token ttl

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56600 from zdover23/wip-doc-2024-04-01-dev-essentials-leads-list

doc/dev: update leads list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: update leads list

Remove Joao Luis from the list of component leads.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #56591 from tchaikov/wip-unittest_rocksdb_option-fix-leak

tests: unittest_rocksdb_option: do not leak RocksDBStore

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #56592 from nh2/crush-location-doc-fixes

doc/rados/operations: Improve crush_location docs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/rados/operations: Improve crush_location docs

* Fix incorrect syntax
* Use underscores for config options, like other ceph docs did
* Fix incorrect statement that crush_location_hook adds fiels; it replaces
* Explain `root=default host=HOSTNAME` is not set if `crush_location` is given
* Remove duplication across sections
* Point out that `root=default` is important

Signed-off-by: Niklas Hambüchen <mail@nh2.me>

Merge pull request #56572 from ceph/wip-yuriw-release-process-main

doc: added jammy to the supported distro list for quincy

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

tests: unittest_rocksdb_option: do not leak RocksDBStore

before this change, we allocate an instance of `RocksDBStore` with
`new`, but we never free it. and LeanSanitizer points this out:

```
Direct leak of 952 byte(s) in 1 object(s) allocated from:
    #0 0x55f31440bc2d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_rocksdb_option+0xaebc2d) (BuildId: 81b849dbc41cbc6b05d5e603d9ba8a002dab2d24)
    #1 0x55f3144132fd in RocksDBOption_simple_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/TestRocksdbOptionParse.cc:17:22
    #2 0x55f3144ecf26 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x55f3144a4312 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x55f314453ccc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x55f314455d02 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x55f31445733b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x55f3144747c8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x55f3144f5576 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x55f3144ab1a2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x55f314473b52 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x55f31440f690 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x55f31440e4c3 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/unit.cc:45:10
    #13 0x7f0d32551d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we manage the life cycle of `RocksDBStore` using
a smart pointer. this should address the leak.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #56587 from tchaikov/wip-bluefs-use-after-free

test/objectstore/test_bluefs: fix heap-use-after-free

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #56581 from zdover23/wip-doc-2024-03-29-dev-essentials-leads-list

doc/dev: update leads list

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #55436 from tchaikov/mgr-python-3.12

mgr: use un-deprecated APIs to initialize Python interpretor

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

test/objectstore/test_bluefs: fix heap-use-after-free

in BlueFS.test_shared_alloc, we keep the return value of
`fs.get_perf_counters()`, and deference it after umounting the fs,
but the `PerfCounters*` pointer returned from `fs.get_perf_counters()`
is destroyed in `BlueFS::_shutdown_logger()` which is in turn called
by `BlueFS::umount()`. so ASan points this out:
```
==1662613==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000b2d80 at pc 0x7f0eefc30644 bp 0x7ffcdbab6430 sp 0x7ffcdbab6428
READ of size 8 at 0x6110000b2d80 thread T0
    #0 0x7f0eefc30643 in ceph::common::PerfCounters::get(int) const /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:246:8
    #1 0x557595ddfc15 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1182:3
    #2 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #13 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0x7f0eed083e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0x557595cd46a4 in _start (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x2856a4) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)

0x6110000b2d80 is located 0 bytes inside of 208-byte region [0x6110000b2d80,0x6110000b2e50)
freed by thread T0 here:
    #0 0x557595d92b1d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x343b1d) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x557595f31c43 in BlueFS::_shutdown_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:462:3
    #2 0x557595f54ab5 in BlueFS::umount(bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:1076:3
    #3 0x557595ddfbd7 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1180:6
    #4 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #5 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #6 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #7 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #8 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #9 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #10 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #13 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #14 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #15 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

previously allocated by thread T0 here:
    #0 0x557595d922bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x3432bd) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x7f0eefc33180 in ceph::common::PerfCountersBuilder::PerfCountersBuilder(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:537:21
    #2 0x557595f30ac9 in BlueFS::_init_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:221:23
    #3 0x557595f42bc6 in BlueFS::mount() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:977:3
    #4 0x557595ddd339 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1139:3
    #5 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #6 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #7 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #8 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #9 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #10 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #11 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #12 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #13 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #14 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #15 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #16 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of keeping `logger` across the `umount()` and
`mount()` calls, we get another instance of `logger`, query it for
the perf counter that we are interested, and compare the value
to see if it is unchanged.

this should address the ASan warning above.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #56544 from baum/fix_f68248944815acbf5fb08ef66f36da6d5baef824

pybind: support RHEL8, whose default python is python 3.6.8

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #56584 from phlogistonjohn/jjm-rpmbuild-fix

Merged without waiting for tests.  Obvious fix.

ceph.spec.in: remove command-with-macro line

A comment clearly left as a breadcrumb for a node-proxy manpage is
causing (intermittent) build failures. Remove the line and hope
the manpage is added if/when appropriate.

Fixes: 0dd73643649ddc2366e60de4fe6c078b6e112091
Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56540 from tchaikov/wip-fix-test-leaks-ceph-context

test: : do not increase ref when creating intrusive_ptr<CephContext>

Reviewed-by: Casey Bodley <cbodley@redhat.com>

pybind: use syntax supported by python3.6

Fixes: f68248944815acbf5fb08ef66f36da6d5baef824
It seems the shaman CentOS8 builders, which serve as the base for container images, aren't current with the ":=" syntax. Amend to support RHEL8, whose default python is python 3.6.8.

Error example from https://shaman.ceph.com/builds/ceph/ceph-nvmeof-mon/27188b78bdbaa3c33de518b8514906de629e774a/default/388913/

  File "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2505-g27188b78/rpm/el8/BUILD/ceph-19.0.0-2505-g27188b78/src/pybind/cephfs/setup.py", line 120
    if ldflags := os.environ.get('LDFLAGS'):
                ^
SyntaxError: invalid syntax

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>

doc: added jammy to the supported distro list for quincy

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc/dev: update leads list

Update the list of component leads in
doc/dev/developer_guide/essentials.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: allow the user to add ttl for the connected cluster's
token expiration

Fixes: https://tracker.ceph.com/issues/65055
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #56190 from rhcs-dashboard/disable-multi-cluster-non-hub

mgr/dashboard: disable multi-cluster feature for non-hub clusters

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #54633 from joscollin/wip-F61334-use-snapdiff-api

cephfs_mirror: use snapdiff api for incremental syncing

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #54301 from lxbsz/wip-sparse-read-test

ceph_test_rados_api_io_pp: add multiple extents test for sparse-read

Merge pull request #56570 from batrick/release-checklist-docs-2

doc: reconcile release checklist

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc: reconcile release checklist

Updating the releases documentation existed already but that section should be
moved to after dev freeze when the branch is forked.

Fixes: 7b5017913a9ecf27ca7281367a5472a9b9b043ec
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #56568 from batrick/release-checklist-docs

doc: update release checklist

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc: update release checklist

To include note that release notes must be removed after forking the release
branch.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #55337 into main

* refs/pull/55337/head:
mds: explain clients retaining Fb during xlock

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

Merge PR #56536 into main

* refs/pull/56536/head:
mds: do not dispatch aborted internal requests
qa: add test for killing quiesce_inode with outstanding remote authpin requests
mds: add lockup debugging command

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

Merge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2

crimson: introduce INTERNAL_PG_LOCAL_NS, use for snapmapper

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #56516 from afreen23/wip-ceph-logo

mgr/dashboard: make ceph logo redirect to dashboard

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56543 from jrchyang/main

os/bluestore: remove unnecessary std::hex in dout

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs

mgr/dashboard: s/active_mds/active_nfs in fs attach form

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix

mgr/dashboard: fix clone unique validator for name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw

mgr/dashboard: fix rgw display name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

os/bluestore: remove unnecessary std::hex in dout

Signed-off-by: Jrchyang Yu <yuzhiqiang_yewu@cmss.chinamobile.com>

test: : do not increase ref when creating intrusive_ptr<CephContext>

before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x558d341d837d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 1b7e7e5abfc2b58ce2334712e4c00b2441c25870)
    #1 0x7fd74c957559 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fd74c956933 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7fd74c956933 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7fd74c956690 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7fd74c950562 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circ
ular_buffer/base.hpp:1039:9
    #6 0x7fd74c9435b4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7fd74bc1891d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7fd74bc178eb in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x558d341f97e9 in pick_address_filtering_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:774:47
    #10 0x558d3430c4f6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x558d342c3fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x558d342749dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x558d34276a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x558d3427804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x558d342954d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x558d34314d26 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x558d342ca932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x558d34294862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x558d34218d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x558d34218d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #21 0x7fd749331d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

mds: do not dispatch aborted internal requests

Matching the behavior of Server::dispatch_client_request.

This logic can be unified in a future refactor but keeping this as a targetted
fix for the purposes of the bug.

Fixes: https://tracker.ceph.com/issues/65182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add test for killing quiesce_inode with outstanding remote authpin requests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers

cmake: enable building with Sanitizers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mds: add lockup debugging command

To artificially make the rank unresponsive for a time.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #56378 from guojidan/snap-stats

crimson/osd/osd_operations/snaptrim_event: update PG's stats

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

script/cpatch.py: run container build with host networking

Applies host networking to building the image. This is handy when using
custom networking hacks that redirect repos, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/cpatch.py: add a --run-before argument to run commands

Add a `--run-before` command line argument to cpatch.py that allows
commands to be executed before doing any other layer generation.
This supports actions like customizing packages installed in the
image. Something that can be handy when developing new features.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56274 from phlogistonjohn/jjm-mypy-bump

python: mypy version bump

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #56304 from phlogistonjohn/jjm-tox-default-envs

python: allow unit tests to use tox default envs

Reviewed-by: Adam King <adking@redhat.com>

mgr/dashboard: disable multi-cluster feature for non-hub clusters

Fixes: https://tracker.ceph.com/issues/65056
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: make ceph logo redirect to dashboard

Fixes https://tracker.ceph.com/issues/64734

Clicking on ceph logo in navigation takes back to overview dashboard.

Signed-off-by: Afreen <afreen23.git@gmail.com>

Merge pull request #56449 from afreen23/wip-security

mgr/dashboard:Update encryption and tags in bucket form

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: s/active_mds/active_nfs in fs attach form

Fixes: https://tracker.ceph.com/issues/65173
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-follow-redirects-1.15.6

mgr/dashboard: bump follow-redirects from 1.15.3 to 1.15.6 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56463 from rhcs-dashboard/cephfs-name-validation

mgr/dashboard: fix cephfs name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

crimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run

Fixes: https://tracker.ceph.com/issues/64975
Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../scrub_events: convert ScrubScan::run to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../scrub_events: convert ScrubFindRange::run to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

common/hobject: make hobject_t formatter methods const

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: move snapmapper object into INTERNAL_PG_LOCAL_NS

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #56384 from Svelar/drop_fmt_version

run-make: do not pass '-DWITH_FMT_VERSION' to cmake

qa/lsan.supp: suppress MallocExtension::Initialize

LeakSanitizer reports
```
==688591==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x55f8dd9969dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_fastbmap_allocator+0x1f89dd) (BuildId: cac39eac8ef1e8774f9dd48e6e3f677fdd864776)
    #1 0x55f8dd99c730 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x55f8dd99c690 in std::allocator<char>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x55f8dd99c690 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #4 0x55f8dd99c393 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:153:14
    #5 0x55f8dda96a6c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:307:21
    #6 0x55f8dda96852 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:395:8
    #7 0x7f4a751ab6f0 in
    MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId:
    eeef3d1257388a806e122398dbce3157ee568ef4)
```

this is a global object allocated by the allocator, so we can suppress this report.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: suppress LeakSanitizer reports of known leaks

there are known leaks, which are tracked by qa/lsan.suppr, in Ceph.
so let's reuse it so we don't see them when running unit test with
ASan enabled.

see also https://clang.llvm.org/docs/AddressSanitizer.html#issue-suppression

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: prevent ASAN_OPTIONS from detect_odr_violation

turns out we have multiple copies of following symbol defined by rbd executable:
```
AddressSanitizer: odr-violation: global 'ceph::buffer::list::always_empty_bptr' at /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/buffer.cc:1267:34
```
before addressing it. let's disable this warning.

Refs https://tracker.ceph.com/issues/65098

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

pybind: use LDFLAGS in env variable when check for building env

when building python bindings extensions using Cython, we first perform
sanity checks by building an executable and linking it against the
shared library of the C language binding. if the executable builds,
we consider the sanity test passes.

before this change, we don't add `LDFLAGS` specified in environmental
variable to the ldflags when building the executable. but we need to
link against libasan, so that the symbols used by, for instance,
librados.so, can be found by the executable.

in this change, we always check `LDFLAGS`, and add it to the ldflags
when building the executable for testing, if it's set.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: add ${CMAKE_SHARED_LINKER_FLAGS} to LDFLAGS when building python extensions

if sanitizers are enabled, we have to populate the required link flags
to python extensions's building workflow. otherwise ld would fail to
link like:
```
/usr/bin/ld: /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libceph-common.so.2: undefined reference to `__asan_stack_free_10'
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: do not override CMAKE_EXE_LINKER_FLAGS

instead of overriding CMAKE_EXE_LINKER_FLAGS, let's append to it.
so that the existing `CMAKE_EXE_LINKER_FLAGS` is not overriden.

this should enable us to build with Clang and with sanitizer(s) enabled.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: error out on UBSan error

so we can be alerted if UBSan identify something wrong.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: quote a list using quotes

otherwised we'd have

```
CMake Error at cmake/modules/FindSanitizers.cmake:17 (if):
  if given arguments:

    "address" "IN_LIST" "address" "thread" "undefined_behavior" "OR" "leak" "IN_LIST" "address" "thread" "undefined_behavior"

  Unknown arguments specified
```

when enabling TSan with WITH_TSAN=ON.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #56461 from tchaikov/wip-test-test_config_map-fix-leaks

test/mon/test_config_map: fix leaks identified by LeakSanitizer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #56450 from tahajahangir/patch-2

docs/rados: remove incorrect ceph command

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #56458 from adk3798/rgw-bootstrap-hostnames

mgr/cephadm: update rgw zonegroup w/ zonegroup_hostnames from rgw spec

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge PR #56454 into main

* refs/pull/56454/head:
vstart: require debugging switch for all mds debug configs

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard: fix rgw display name validation

the dot(.) character is allowed in full name

Fixes: https://tracker.ceph.com/issues/65146
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56351 from afreen23/wip-advanced-sec

mgr/dashboard: Add advanced fieldset component

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix cephfs name validation

allow volume name to start with dot (.)

Fixes: https://tracker.ceph.com/issues/65143
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56462 from zhscn/fix-uaf

crimson/os/seastore: fix use after free error

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: fix clone unique validator for name validation

Fixes: https://tracker.ceph.com/issues/65145
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #56436 from zdover23/wip-doc-2024-03-25-start-intro

doc/start: improve MDS explanation

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #56328 from rhcs-dashboard/cors-through-cli

mgr/dashboard: introduce cli commands to set cors urls

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #55783 from rhcs-dashboard/multi-cluster-imrovements

mgr/dashboard: Multi cluster improvements

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard:Update encryption and tags in bucket form

Fixes https://tracker.ceph.com/issues/65110

- renamed Security to Encryption
- aligned security fieldset with rest of the form using `cd-help-text`
  and aligning it with object locking in UX
- aligned security fieldset top use `cd-help-text`
- changed help text of Tags

Signed-off-by: Afreen <afreen23.git@gmail.com>