pybind/rbd: adjust error message in deep_copy()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

pybind/rbd: expose RBD_IMAGE_OPTION_FLATTEN option

It takes effect with deep_copy() and migration_prepare().

Fixes: https://tracker.ceph.com/issues/65624
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

pybind/rbd: expose RBD_IMAGE_OPTION_CLONE_FORMAT option

It takes effect with clone(), deep_copy() and migration_prepare().

Fixes: https://tracker.ceph.com/issues/65624
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #55978 from xxhdx1985126/wip-librbd-unittest-sparsify

test/librbd/test_internal: change to check for 4KB-aligned sparse read

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #55823 from ceph/wip-lusov-quiescer

qa: cephfs quiesce thrasher

Merge pull request #56806 from xxhdx1985126/wip-65399

crimson/osd/pg_recovery: backoff if the recovery/backfill is deferred

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #56998 from xxhdx1985126/wip-65568

crimson/osd/pg: only kick off snap trimming when PGs are clean

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #57128 from cyx1231st/wip-fix-ut-seastore-timeout

crimson/os/seastore: misc cleanups and fixes

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #56892 from xxhdx1985126/wip-crimson-urgent-recovery-version

crimson/osd/osd_operations/client_request: retrieve the correct version for objects to be recovered urgently

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #56610 from Matan-B/wip-matanb-crimson-replicated-head-obc

crimson/osd/replicated_recovery_backend: Fix recovery obc usage

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #57135 from xxhdx1985126/wip-seastore-checksum-assertion-fix

crimson/os/seastore/transaction_manager: remove incorrect assertions

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #57129 from xxhdx1985126/wip-65585

crimson/os/seastore/transaction_manager: fix write pipeline phase leak

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

qa/suites/fs: add quiescer to the fs suite

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

qa/tasks: the quiescer task and a waiter task to test it

Fixes: https://tracker.ceph.com/issues/63669
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

Merge pull request #57136 from zdover23/wip-doc-2024-04-29-radosgw-bucketpolicy

doc/radosgw: disambiguate version-added remarks

doc/radosgw: disambiguate version-added remarks

Rewrite two sentences so that they explain precisely what was added to
Ceph and when it was added to Ceph.

This commit is made in response to a request made by Dr Per Mildner on
https://pad.ceph.com/p/Report_Documentation_Bugs.

Signed-off-by: Zac Dover <zac.dover@proton.me>

crimson/osd/osd_operation: yield the cpu before another round the
"with_throttle"

This is to avoid infinite recursive calling to "with_throttle_while"

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/osd/pg_recovery: backoff if the recovery/backfill is deferred

Fixes: https://tracker.ceph.com/issues/65399
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: drop the impossible error eagain

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/extent_placement_manager: make reserve and block atomic

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: improve seastore_t logs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: remove incorrect assertions

Indirect mappings always hold the checksums of the corresponding intermediate
mappings, which may have non-zero

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: add logs for background

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/gtest_seastar: add logs

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/gtest_seastar: cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #55449 from xxhdx1985126/wip-seastore-extent-chksum

crimson/os/seastore: support extent checksum verification

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

crimson/os/seastore/transaction_manager: fix write pipeline phase leak

At present, if a transaction gets interrupted right after it enters
WritePipeline::ReserveProjectedUsage and before any later continuations
get executed, WritePipeline::ReserveProjectedUsage will be locked
forever.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #57088 from xxhdx1985126/wip-seastore-update-onode-size

crimson/os/seastore: only update onode sizes when necessary

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #57118 from xxhdx1985126/wip-65673

test/ceph_crypto: define __has_feature if the compiler doesn't have it

Reviewed-by: Kefu Chai <kefu.chai@scylladb.com>

Merge pull request #57119 from zdover23/wip-doc-2024-04-28-cephfs-client-auth

doc/cephfs: edit front matter in client-auth.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/cephfs: edit front matter in client-auth.rst

Clarify the front matter in doc/cephfs/client-auth.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #57112 from tchaikov/erasure-code-free

erasure-code/shec: use free() to release alloc()'ed memory chunk

Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #57113 from tchaikov/erasure-code-free-crush-wrapper

test//erasure-code/TestErasureCodeLrc: free allocated CrushWrapper

Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #57111 from tchaikov/erasure-code-nullptr

erasure-code/shec: replace 0 with nullptr when appropriate

Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>

crimson/osd/pg: SnapTrimEvent to support interrupts

SnapTrimEvent operations are scheduled from `PG::on_active_actmap()`
using a `seastar::do_until` loop. This commit replaces the loop type
into an `interruptor::repeat` and SnapTrimEvent are now scheduled by
`start_operation_may_interrupt`.

Previously, `SnapTrimEvent::start` handled interruptions by returning
a `crimson::ct_error::eagain::make();`. Now, the errorator is directly
returned via the `snap_trim_event_ret_t` and interrupts the loop
described above.

As a result, interruptions originated by interval changes are now
supported by SnapTrimEvent.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

test/ceph_crypto: define __has_feature if the compiler doesn't have it

Refer to https://gcc.gnu.org/onlinedocs/cpp/_005f_005fhas_005ffeature.html
and https://clang.llvm.org/docs/LanguageExtensions.html#has-feature-and-has-extension
for further information

Fixes: https://tracker.ceph.com/issues/65673
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

test//erasure-code/TestErasureCodeLrc: free allocated CrushWrapper

in TestErasureCodeLrc.cc, we create a `CrushWrapper` without
destroying and freeing it. and ASan points this out:
```
Indirect leak of 72000 byte(s) in 1000 object(s) allocated from:
    #0 0x559e00acc2bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_lrc+0x1ed2bd) (BuildId: 1cd93d0f231006242239eb0d81a8d677e36aeba2)
    #1 0x559e00b2bb76 in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x559e00b2bb00 in std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x559e00b2bb00 in std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > > >::allocate(std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #4 0x559e00b2b9d1 in std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_get_node() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:561:16
    #5 0x559e00b2aae4 in std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >* std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_create_node<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:611:23
    #6 0x559e00b29520 in std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> >, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:2431:19
    #7 0x559e00b255c2 in std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_map.h:501:15
    #8 0x559e00b188da in CrushWrapper::set_item_name(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/crush/CrushWrapper.h:466:7
    #9 0x7feb54a4dcc2 in CrushWrapper::insert_item(ceph::common::CephContext*, int, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/crush/CrushWrapper.cc:1119:5
    #10 0x559e00ada922 in ErasureCodeTest_create_rule_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/TestErasureCodeLrc.cc:127:5
    #11 0x559e00c0e0a6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
```

so, in this change, let's manage the lifecycle of the `CrushWrapper`
instance with a smart pointer, so that it is destroyed and free'd
properly, and this should silence the ASan warning.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

erasure-code/shec: use free() to release alloc()'ed memory chunk

ASan warns
```
==445793==ERROR: AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete) on 0x602000039b10
    #0 0x5604a544112d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_shec_all+0x1e012d) (BuildId: 8cfc74d22471b6905f9b23304aed2af945265a13)
    #1 0x7fc14752f588 in ErasureCodeShecTableCache::~ErasureCodeShecTableCache() /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShecTableCache.cc:61:19
    #2 0x5604a544ccbe in ParameterTest_parameter_all_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/erasure-code/TestErasureCodeShec_all.cc:263:1
...
0x602000039b10 is located 0 bytes inside of 4-byte region [0x602000039b10,0x602000039b14)
allocated by thread T0 here:
    #0 0x5604a5405afe in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_erasure_code_shec_all+0x1a4afe) (BuildId: 8cfc74d22471b6905f9b23304aed2af945265a13)
    #1 0x7fc1474c9617 in reed_sol_vandermonde_coding_matrix /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/jerasure/jerasure/src/reed_sol.c:86:10
    #2 0x7fc147528634 in ErasureCodeShec::shec_reedsolomon_coding_matrix(int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShec.cc:514:12
    #3 0x7fc147526cd8 in ErasureCodeShecReedSolomonVandermonde::prepare() /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShec.cc:390:14
    #4 0x7fc1475187aa in ErasureCodeShec::init(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&, std::ostream*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/shec/ErasureCodeShec.cc:57:3
```

where we use `delete` to free the encoder matrix allocated using
`malloc()`. as jerasure is a library implemented in C language,
unless we want to reimplment it in C++, we should use `free()` to
free the memory chunk allocated by
`reed_sol_vandermonde_coding_matrix()`. also, please note,
jerasure does not provide a function to free the memory allocated
by this function, we have to explore its implementation, and use
`malloc()` directly. this should silence the ASan warning.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

erasure-code/shec: replace 0 with nullptr when appropriate

0 fails to send the message to human readers that the variable is
a pointer, but nullptr does. for improving the readability, let's
use nullptr when the variable in question is a pointer.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #57105 from zdover23/wip-doc-2024-04-26-mgr-dashboard

doc/mgr: explain error message - dashboard.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/mgr: explain error message - dashboard.rst

Explain that an error message received in response to
"redirect_resolve_ip_addr True" might be caused by having an
insufficiently recent release of Ceph running in your cluster.

Fixes: https://tracker.ceph.com/issues/61312
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52024 from lxbsz/wip-61660

mds: fix the description for inotable testing only options

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #57035 from xxhdx1985126/wip-65610

test/crimson/seastore/test_object_data_handler: remap pins through the transaction that created the pins

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #56844 from cyx1231st/wip-crimson-fix-tri-mutex

crimson/common/tri_mutex: make locking/promotion atomic if possible

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #57093 from tchaikov/wip-ceph_crypto-matcher

test: remove extraneous ")" after literal string for matcher

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/tasks/cephfs: don't create a new CephManager if there is one in the context

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

qa/tasks: vstart_runner: introduce --config-mode

The new mode of the vstart_runner allows for passing
paths to yaml configs that will be merged and then
run just as the teuthology would do it.
Building on the standard run method we can even
pass "-" as the config name and provide one on the stdin like

    python3 ../qa/tasks/vstart_runner.py --config-mode "-" << END
    tasks:
      - quiescer:
          quiesce_factor: 0.5
          min_quiesce: 10
          max_quiesce: 10
          initial_delay: 5
          cancelations_cap: 2
          paths:
            - a
            - b
            - c
      - waiter:
          on_exit: 100
    END

This commit does the minimum to allow testing of the quiescer,
but it also lays the groundwork for running arbitrary configs.

The cornerstone of the approach is to inject our local implementations
of the main fs suite classes. To be able to do that, some minor
refactoring was required in the corresponding modules:
the standard classes were renamed to have a *Base suffix, and the
former class name without the suffix is made a module level variable
initialized with the *Base implementation. This refactoring
is meant to be backward compatible.

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

qa/tasks: introduce ThrasherGreenlet

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

Merge pull request #56926 from samarahu/rgw-update-boost-redis

RGW: boost/redis: Point to 1.85 tag

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #56486 from soumyakoduri/wip-skoduri-lc-stall

rgw/lc: advance head if the current entry doesn't exist

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

test: remove extraneous ")" after literal string for matcher

this is a follow-up of fbbb336de961e433fc796dbdf6db650a836e9ef9,
which added an extraneous ")" at the end of the string literal.
and it breaks the build if ASan is enabled.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #56860 from clwluvw/rgw-crypt-ssl

rgw: eliminate ssl enforcement for sse-s3 encryption

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #56868 from clwluvw/request-payer

rgw: exclude logging of request payer for 403 requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #56394 from cbodley/wip-rgw-cache-size

rgw: increase default metadata cache size for accounts

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #56606 from guojidan/num_bytes

crimson/osd/ops_executer: fix snap overlap range error

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: only update onode sizes when necessary

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #56942 from Svelar/fix_tox_cephadm

test/cephadm: extend timeout again

Merge pull request #57057 from adk3798/no-downgrade

doc/cephadm: remove downgrade reference from upgrade docs

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #56644 from tchaikov/wip-warning-not-fatal

test: do not consider warning from Sanitizer as fatal

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55950 from ifed01/wip-ifed-omap-pcounters

os/bluestore: add omap_set* perf counters

Reviewed-by: akupczyk@ibm.com
Reviewed-by: rfriedma@ibm.com

Merge pull request #57071 from zdover23/wip-doc-2024-04-24-rados-troubleshooting-pg

doc/rados: improve t-shooting pg

Merge pull request #57051 from zdover23/wip-doc-2024-04-23-rados-dual-stack

doc/rados: remove dual-stack docs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: improve t-shooting pg

Incorporate Anthony D'Atri's suggestions from
https://github.com/ceph/ceph/pull/57022 into the text in
doc/rados/troubleshooting/troubleshooting-pg.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #56745 from ronen-fr/wip-rf-pgtell

tests/scripts: use 'tell pg deep-scrub pgid' instead of 'tell pgid deep-scrub'

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #56420 from rzarzynski/wip-tools-rados-get-with-offset

tools: rados get takes --offset now

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #56067 from myoungwon/wip-ceph-dedup-daemon

tool: separating ceph-dedup-tool into tool and daemon

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #55949 from NitzanMordhai/wip-nitzan-safe-to-destroy-test-build-vars

test/osd/safe-to-destroy.sh: Add checks for the test

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #54979 from pdvian/wip-mgr-optracker

mgr: integrate optracker in ceph-mgr

Merge pull request #57053 from idryomov/wip-labeler-rbd

.github: sync the list of paths for rbd label with CODEOWNERS

Reviewed-by: Laura Flores <lflores@redhat.com>

doc/cephadm: remove downgrade reference from upgrade docs

This has been in here for years, but cephadm will block
attempted upgrades to lower versions and we generally
don't want people to think this is supported or safe.

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #56986 from cbodley/wip-65551

rgw/account: add bucket_quota to RGWAccountInfo

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>

Merge pull request #57038 from adk3798/mixin-max-nvme-gws-per-group

monitoring/ceph-mixin: set NVMeoFMaxGatewaysPerGroup to 4

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #56990 from cbodley/wip-doc-rgw-realm-change

doc/rgw: add release note for changes to rgw_realm init

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #55983 from lxbsz/wip-64572

qa/fsx: use the master branch to build the xfstest-dev

rgw/lc: cleanup duplicate code

Cleaning up duplicate code around updating head

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/lc: advance head if the current entry doesn't exist

This is extension to https://github.com/ceph/ceph/pull/47595.

When skipping the non-exist LC entry, instead of get_next_entry(),
use advance_head() to fetch the next entry. In case the cycle
is finished for that shard, head should be reset to avoid lc process
being stuck in indefinite loop.

Fixes: https://tracker.ceph.com/issues/65160
Signed-off-by: Soumya Koduri <skoduri@redhat.com>

.github: sync the list of paths for rbd label with CODEOWNERS

With https://github.com/actions/labeler/issues/483 still pending, do it
manually.

A couple of entries matching both files and directories were expanded
since, unlike fnmatch(3), minimatch treats slashes specially:

- src/include/rbd* -> src/include/rbd* src/include/rbd/**
- src/rbd*         -> src/rbd* src/rbd*/**
- src/test/rbd*    -> src/test/rbd* src/test/rbd*/**

A bunch of entries under src/tools were combined into:

- src/tools/rbd*/**

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

CODEOWNERS: assign ceph-rbd-mirror systemd service to RBD

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados: remove dual-stack docs

Remove references to dual-stack mode in
doc/rados/configuration/network-config-ref.rst and
doc/rados/configuration/msgr2.rst. This feature seems to have been
planned but never to have been completely implemented.

See the tracker issue listed below for an email exchange detailing the
confusion caused by the presence in the documentation of this
now-removed information.

Fixes: https://tracker.ceph.com/issues/65631
Signed-off-by: Zac Dover <zac.dover@proton.me>

test/cephadm: extend timeout limit again to fix timeout issue

When run make check(arm64) on omani series servers,
Shows:
python: timeout after 60 seconds
INFO     root:call_wrappers.py:284 Non-zero exit code 124 from /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/cephadm/.tox/py3/bin/python -c for i in range(1000000): print(i, flush=True)

Seems like 60s is not enough, extend to 300s would be fine.

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

crimson/osd/pg: only kick off snap trimming when PGs are clean

Fixes: https://tracker.ceph.com/issues/65568
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #57033 from gbregman/main

mgr/cephadm: Some changes to nvmeof conf file defaults

Merge PR #54067 into main

* refs/pull/54067/head:
mds: add dispatch killpoint and delay configs

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

Merge PR #56047 into main

* refs/pull/56047/head:
mgr: process map before notifying clients

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #56638 into main

* refs/pull/56638/head:
qa: do not iterate list being modified
qa: remove unnecessary background job cleanup

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #56669 into main

* refs/pull/56669/head:
qa: make teardown behavior consistent for kclient/fuse mounts

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #56723 into main

* refs/pull/56723/head:
mds: use mds_cache_quiesce_decay_rate to init quiesce_counter

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

Merge PR #57010 into main

* refs/pull/57010/head:
qa: update quiesce tests to expect ipolicy lock
mds: add missing policylock to test F_QUIESCE_BLOCK

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

Merge pull request #57022 from zdover23/wip-doc-2024-04-22-rados-operations-pg-troubleshooting

doc/rados: remove redundant pg repair commands

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

rgw: apply default quota config on account creation

add new default quota config options for accounts analogous to
rgw_user_default_quota_max_objects/size. apply the default bucket quota
config options as-is

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #57032 from stackhpc/doc-update-ms-bind-port-max

doc/rados: fix outdated value for ms_bind_port_max

Merge pull request #57036 from stackhpc/fix-typo-radosgw-metrics

doc/radosgw: fix typo in metrics.rst

monitoring/ceph-mixin: set NVMeoFMaxGatewaysPerGroup to 4

Recommendation from the nvmeof team

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #56448 from Svelar/arm_pass_check_shard_select

test/test_mempool: skip check_shard_select test on ARM64

doc/radosgw: fix typo in metrics.rst

Signed-off-by: Pierre Riteau <pierre@stackhpc.com>

test/crimson/seastore: add test profile for full integrity check

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

test/crimson/seastore/test_object_data_handler: remap pins through the
transaction that created the pins

Fixes: https://tracker.ceph.com/issues/65610
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/btree: assert that the transaction getting logical
extents from pins are the one that created the pins

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

mgr/cephadm: some changes to nvmeof conf file defaults
Fixes https://tracker.ceph.com/issues/65566

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

python-common/ceph/deployment: some changes to nvmeof conf file defaults
Fixes https://tracker.ceph.com/issues/65566

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>