Merge pull request #58284 from pponnuvel/wip-66700-quincy

quincy: mgr/nfs: Don't crash ceph-mgr if NFS clusters are unavailable.

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>

Merge pull request #56201 from ifed01/wip-ifed-fix-64443-quincy

quincy: test/store_test: fix DeferredWrite test when prefer_deferred_size=0

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #55776 from ifed01/wip-ifed-fix-expand-with-ncb-qui

quincy: os/bluestore: fix free space update after bdev-expand in NCB mode.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #54692 from Matan-B/wip-63650-quincy

quincy: tools/ceph_objectstore_tool: action_on_all_objects_in_pg to skip pgmeta

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #59057 from yuvalif/wip-67365-quincy

quincy: common/dout: fix FTBFS on GCC 14

Reviewed-By: Yuri Weinstein <yweinste@ibm.com>, Radoslaw Zarzynski <Radoslaw.Adam.Zarzynski@ibm.com>, Pere Diaz Bou <pdiazbou@ibm.com>

Merge pull request #58704 from kshtsk/wip-67064-quincy

quincy: common: fix compilation warnings in numa.cc

Merge pull request #59042 from NitzanMordhai/wip-67350-quincy

quincy: Test: osd-recovery-space.sh extends the wait time for "recovery toofull"

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #58742 from ifed01/wip-ifed-async-compact-qui

quincy: os/bluestore: enable async manual compactions

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #59216 from zdover23/wip-doc-2024-08-14-backport-59168-to-quincy

quincy: doc/cephfs: improve cache-configuration.rst

doc/cephfs: improve cache-configuration.rst

Improve the text in the section about dealing with cache-pressure alerts
that was added in https://github.com/ceph/ceph/pull/59077. The changes
in this commit were suggested by Anthony D'Atri.

Co-authored-by: Patrick Donnelly <pdonnelly@redhat.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit aa3bdae2314fef2fca8fc12dca006af657235e17)

Merge pull request #59150 from zdover23/wip-doc-2024-08-11-backport-59077-to-quincy

quincy: doc/cephfs: add cache pressure information

doc/cephfs: add cache pressure information

Add information to doc/cephfs/cache-configuration.rst about how to deal
with a message that reads "clients failing to respond to cache
pressure". This procedure explains how to slow the growth of the
recall_caps value so that it does not exceed the
mds_recall_warning_threshold.

The information in this commit was developed by Eugen Block. See
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/5ROH5CWKKOEIQMVXOVRT5OO7CWK2HPM3/#J65DFUPP4BY57MICPANXKI7KAXSZ5Z5P
and https://www.spinics.net/lists/ceph-users/msg73188.html.

Fixes: https://tracker.ceph.com/issues/57115
Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit bf26274ae4737417193f8c2b56bea20eb2a358aa)

Merge pull request #51136 from RaminNietzsche/wip-cloud-sync-fixes-to-the-quincy

cloud sync: fix crash due to objs on cr stack

Merge pull request #56166 from vshankar/wip-64249-quincy

quincy: qa: correct usage of DEBUGFS_META_DIR in dedent

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53626 from lxbsz/wip-62513

quincy: qa/cephfs: switch to python3 for centos stream 9

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53629 from lxbsz/wip-62867

quincy: qa: increase the http postBuffer size and disable sslVerify

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #55392 from dparmar18/wip-64067-quincy

quincy: qa: fix error reporting string in assert_cluster_log

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #55745 from lxbsz/wip-64221

quincy: qa/tasks/cephfs/test_misc: switch duration to timeout

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #56404 from batrick/wip-65062-quincy

quincy: qa/cephfs: ignorelist clog of MDS_UP_LESS_THAN_MAX

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #59101 from zdover23/wip-doc-2024-08-09-backport-59807-to-quincy

quincy: docs/rados/operations/stretch-mode: warn device class is not supported

docs/rados/operations/stretch-mode: warn device class is not supported

Signed-off-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
(cherry picked from commit aa1d8cf4fa321e24e850bd5f687a6ddad3ce05e3)

Merge pull request #54015 from k0ste/wip-59362-quincy

quincy: rgw: fix rgw cache invalidation after unregister_watch() error

Merge pull request #53471 from k0ste/wip-57636-quincy

quincy: rgw: Drain async_processor request queue during shutdown

Merge pull request #53462 from k0ste/wip-55678-quincy

quincy: rgw: remove potentially conficting definition of dout_subsys

Merge pull request #53438 from k0ste/wip-62824-quincy

quincy: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage

Merge pull request #53374 from jzhu116-bloomberg/wip-59694-quincy

quincy: rgw/notification: remove non x-amz-meta-* attributes from bucket notifications

Merge pull request #52995 from k0ste/wip-59025-quincy

quincy: rgw: use unique_ptr for flat_map emplace in BucketTrimWatche

Merge pull request #52937 from k0ste/wip-58904-quincy

quincy: rgw: Fix Browser POST content-length-range min value

Merge pull request #52935 from k0ste/wip-54610-quincy

quincy: RGW - Get quota on OPs with a bucket

Merge pull request #52606 from cbodley/wip-62139-quincy

quincy: rgw: s3 object lock avoids overflow in retention date

Merge pull request #52054 from trociny/wip-61431-quincy

quincy: rgw: multisite data log flag not used

Merge pull request #51446 from k0ste/wip-59730-quincy

quincy: RGW: Solving the issue of not populating etag in Multipart upload result

Merge pull request #51257 from k0ste/wip-55062-quincy

quincy: radosgw-admin: fix segfault on pipe modify without source/dest zone specified

Merge pull request #50539 from k0ste/wip-55151-quincy

quincy: rgw: Update "CEPH_RGW_DIR_SUGGEST_LOG_OP" for remove entries

Merge pull request #50520 from k0ste/wip-54227-quincy

quincy: rgw: Erase old storage class attr when the object is rewrited using r…

Merge pull request #50519 from k0ste/wip-54201-quincy

quincy: rgw: CopyObject works with x-amz-copy-source-if-* headers

Merge pull request #48082 from pritha-srivastava/wip-57509-quincy

quincy: rgw/sts: modify max_session_duration using update role REST API/ radosgw-admin command.

Merge pull request #57488 from ljflores/wip-66040-quincy

quincy: qa/suites/rados/singleton: add POOL_APP_NOT_ENABLED to ignorelist

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #50551 from rzarzynski/wip-59084-quincy

quincy: mon: add proxy to cache tier options

nReviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #58753 from adk3798/quincy-only-bind-to-network

quincy: mgr/cephadm: add ability for haproxy, prometheus, grafana to bind on specific ip

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #59080 from zdover23/wip-doc-2024-08-07-backport-59040-to-quincy

quincy: doc: documenting the feature that scrub clear the entries from damage…

Merge pull request #58762 from adk3798/wip-66477-quincy

quincy: qa/suites: add "mon down" log variations to ignorelist

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58761 from adk3798/wip-66957-quincy

quincy: cephadm: turn off cgroups_split setting  when bootstrapping with --no-cgroups-split

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58756 from adk3798/wip-65726-quincy

quincy: python-common: handle "anonymous_access: false" in to_json of Grafana spec

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56308 from adk3798/quincy-timemaster

quincy: cephadm: add timemaster to timesync services list

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57442 from joscollin/wip-65984-quincy

quincy: mgr/stats: initialize mx_last_updated in FSPerfStats

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #54712 from mchangir/wip-63589-quincy

quincy: qa: run kernel_untar_build with newer tarball

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #56393 from batrick/wip-65059-quincy

quincy: qa/suites/fs/nfs: use standard health ignorelist

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #56417 from batrick/wip-65089-quincy

quincy: qa/cephfs: add mgr debugging

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #56667 from batrick/wip-65107-quincy

quincy: qa/cephfs: add probabilistic ignorelist for pg_health

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #52986 from leonid-s-usov/backport/strip-nofail/quincy

quincy: nofail option in fstab not supported

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #55238 from joscollin/wip-64074-quincy

quincy: pybind/mgr/mirroring: drop mon_host from peer_list

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #56168 from vshankar/wip-64046-quincy

quincy: qa: set mds config with `config set` for a particular test

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #58316 from cbodley/wip-66725

quincy: build: Make boost_url a list

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #56815 from Matan-B/wip-65307-quincy

quincy: osd/SnapMapper: fix _lookup_purged_snap

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #55868 from trociny/wip-64649-quincy

quincy: mon/OSDMonitor: fix get_min_last_epoch_clean()

Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>

doc: documenting the feature that scrub clear the entries from damage table on repair

Fixes: https://tracker.ceph.com/issues/67337
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit 140e77638dc95fefcf62f01fe7e45bc0729439be)

Merge pull request #58760 from adk3798/wip-66790-quincy

quincy: cephadm: disable ms_bind_ipv4 if we will enable ms_bind_ipv6

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58759 from adk3798/wip-65967-quincy

quincy: mgr/cephadm: make setting --cgroups=split configurable for adopted daemons

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58757 from adk3798/wip-65950-quincy

quincy: mgr/cephadm: cleanup iscsi keyring upon daemon removal

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58755 from adk3798/wip-65419-quincy

quincy: cephadm: fix host-maintenance command always exiting with a failure

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58754 from adk3798/wip-65380-quincy

quincy: mgr/cephadm: make client-keyring deploying ceph.conf optional

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #56930 from adk3798/wip-65513-quincy

quincy: mgr/dashboard: fix duplicate grafana panels when on mgr failover

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

cephadm.py: add timemaster to timesync services list

On debian/ubuntu, if you need PTP, it's possible to use the linuxptp package for time-synchonization.
In that case the systemd service is called timemaster and is a wrapper for chrony/ntpd/phc2sys/ptp4l.

Signed-off-by: Florent Carli <florent.carli@rte-france.com>
(cherry picked from commit 4ee1c761e50e3c3feed89ccc5c450ac96681b778)

Conflicts:
src/cephadm/cephadm.py

Merge pull request #56837 from rishabh-d-dave/wip-65441-quincy

quincy: qa/cephfs: CephFSTestCase.create_client() must keyring

Merge pull request #56753 from dparmar18/quincy-backport-fix-test-cluster-enonent

quincy: qa/test_nfs: fix test failure when cluster does not exist

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #57445 from mchangir/wip-65900-quincy

quincy: mgr/snap_schedule: restore yearly spec to lowercase y

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #58818 from Svelar/wip-67136-quincy

quincy: bluestore/bluestore_types: avoid heap-buffer-overflow in another way to keep code uniformity

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #57404 from k0ste/wip-65914-quincy

quincy: kv/RocksDBStore: Configure compact-on-deletion for all CFs

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #57196 from k0ste/wip-64592-quincy

quincy: os/bluestore: fix the problem of l_bluefs_log_compactions double recording

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #59064 from zdover23/wip-doc-2024-08-07-backport-59060-to-quincy

quincy: doc/dev/peering: Change acting set num

doc/dev/peering: Change acting set num

This commit updates the peering doc page with the changed [2,1,2] to [3,1,2]

Signed-off-by: qn2060 <qn2060@users.noreply.github.com>
(cherry picked from commit 103d65ea6787ab16d1824f71578264e0abb9820c)

rgw/sts: code changes to replace modify role with
modify role trust policy, to better reflect
what the command does.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 875da7f9041b54ee507bff00f40b65f6810b939a)

Merge pull request #50538 from k0ste/wip-55502-quincy

quincy: rgw: address crash and race in RGWIndexCompletionManager

Merge pull request #55357 from cbodley/wip-64227-quincy

quincy: rgw/rest: fix url decode of post params for iam/sts/sns

Merge pull request #58313 from cbodley/wip-66711-quincy

quincy: rgw/notifications/test: fix rabbitmq and kafka issues in centos9

Merge pull request #57907 from adamemerson/wip-62292-quincy

quincy: rgw: modify string match_wildcards with fnmatch

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #57405 from ivancich/wip-65891-quincy

quincy: rgw: make incomplete multipart upload part of bucket check efficient

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #56119 from mkogan1/wip-64767-quincy

quincy: rgw/beast: Enable SSL session-id reuse speedup mechanism

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #51806 from soumyakoduri/wip-skoduri-quincy

[quincy] RGW backports

Reviewed-by: Mark Kogan <mkogan@redhat.com>

Merge pull request #51797 from yuvalif/wip-61478-quincy

quincy: rgw/kafka: remove potential race condition between creation and deletion of endpoint

Reviewed-by: Adam Emerson <aemerson@redhat.com>

common/dout: fix FTBFS on GCC 14

The following problem has been reported by Kaleb Keithley:

```
/builddir/build/BUILD/ceph-18.2.1/src/osd/osd_types.h: In lambda function:
/builddir/build/BUILD/ceph-18.2.1/src/common/dout.h:184:73: error: call to non-‘constexpr’ function ‘virtual unsigned int DoutPrefixProvider::get_subsys() const’
  184 |     dout_impl(pdpp->get_cct(), ceph::dout::need_dynamic(pdpp->get_subsys()), v) \
      |                                                         ~~~~~~~~~~~~~~~~^~
/builddir/build/BUILD/ceph-18.2.1/src/common/dout.h:155:58: note: in definition of macro ‘dout_impl’
  155 |       return (cctX->_conf->subsys.template should_gather<sub, v>());    \
      |                                                          ^~~
/builddir/build/BUILD/ceph-18.2.1/src/osd/osd_types.h:3617:3: note: in expansion of macro ‘ldpp_dout’
 3617 |   ldpp_dout(dpp, 10) << "build_prior all_probe " << all_probe << dendl;
      |   ^~~~~~~~~
```

For details of the problem and the idea behind the fix,
please refer to the comment this commit brings to `dout.h`.

The minimized replicator that the facilitated Goldbot-based
investigation:

```cpp
namespace ceph::dout {

template<typename T>
struct dynamic_marker_t {
  T value;
  // constexpr ctor isn't needed as it's an aggregate type
  constexpr operator T() const { return value; }
};

template<typename T>
constexpr dynamic_marker_t<T> need_dynamic(T&& t) {
  return dynamic_marker_t<T>{ std::forward<T>(t) };
}

template<typename T>
struct is_dynamic : public std::false_type {};

template<typename T>
struct is_dynamic<dynamic_marker_t<T>> : public std::true_type {};

} // ceph::dout

struct subsys_t {
  template <unsigned SubV, int LvlV>
  bool should_gather() const {
    return true;
  }
  bool should_gather(const unsigned sub, int level) const {
    return false;
  }
};

static subsys_t subsys;

  do {                                                                  \
  const bool should_gather = [&](const auto cctX) {                     \
    if constexpr (ceph::dout::is_dynamic<decltype(sub)>::value ||       \
                  ceph::dout::is_dynamic<decltype(v)>::value) {         \
      std::cout << "the dynamic path" << std::endl;                     \
      return subsys.should_gather(sub, v);                              \
    } else {                                                            \
      /* The parentheses are **essential** because commas in angle      \
       * brackets are NOT ignored on macro expansion! A language's      \
       * limitation, sorry. */                                          \
      std::cout << "the static path" << std::endl;                      \
      /*return subsys.should_gather(sub, v);*/                              \
      return (subsys.template should_gather<sub, v>());             \
    }                                                                   \
  }(cct);                                                               \
  } while (0)

  if (decltype(auto) pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for 'this' */ \
    dout_impl(42, sub, v)

  if (decltype(auto) pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for 'this' */ \
    dout_impl(42, ceph::dout::need_dynamic(42), v)

int main() {
    std::random_device dev;
    std::mt19937 rng(dev());
    std::uniform_int_distribution<std::mt19937::result_type> dist6(1,6); // distribution in range [1, 6]

    int sub = dist6(rng);
    ldpp_dout("mocked out", sub);
    //ldpp_subdout("mocked out", 4, 3);
}
```

Fixes: https://tracker.ceph.com/issues/64050
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 0eace4ea9ea42412d4d6a16d24a8660642e41173)

Test: osd-recovery-space.sh extends the wait time for "recovery toofull".

The osd-recovery-space test involves writing objects and expecting to receive
the "toofull" flag.
If we don't wait long enough, we might check the "toofull" flag before all objects
have completed writing, and the "toofull" status hasn't been activated yet.

The change will extend the waiting time and will also incorporate additional
checks for the return code from the status wait.

Fixes: https://tracker.ceph.com/issues/44510
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
(cherry picked from commit a7bd91dafb5736d2e3cae987e6164cdc14a0792a)

Merge pull request #59033 from zdover23/wip-doc-2024-08-05-backport-57905-to-quincy

quincy: doc/mgr/dashboard: fix TLS typo

doc/mgr/dashboard: fix TLS typo

Change "SSL/TSL" to "SSL/TLS"

Signed-off-by: Mindy Preston <mcpreston@icecube.wisc.edu>
(cherry picked from commit e63b967aeb00a667ab5c91b9b526c7a19a9539ec)

Merge pull request #59009 from zdover23/wip-doc-2024-08-04-backport-59006-to-quincy

quincy: doc/glossary: add "ceph-ansible"

doc/glossary: add "ceph-ansible"

Add a "ceph-ansible" entry to the glossary.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 484e9d2b9d81b6de498ac08125c9a458730dd444)

Merge pull request #58852 from idryomov/wip-67051-quincy

quincy: qa/workunits/rbd: avoid caching effects in luks-encryption.sh

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #58565 from idryomov/wip-rbd-c9-on-quincy

quincy: centos 9 related backports for RBD

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58208 from idryomov/wip-66300-quincy

quincy: librbd: disallow group snap rollback if memberships don't match

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58979 from zdover23/wip-doc-2024-08-01-backport-58899-to-quincy

quincy: doc/glossary: add "Prometheus"

Merge pull request #58975 from zdover23/wip-doc-2024-08-01-backport-58972-to-quincy

quincy: doc: Improve doc/radosgw/placement.rst

doc/glossary: add "Prometheus"

Add a glossary entry for "Prometheus".

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1a76464999cb2d2cc748d69e1bb0567544a0bb8b)

doc: Improve doc/radosgw/placement.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 5890a156524b8d1fe3a51ea1c76d800e5903a01c)

Merge pull request #58949 from zdover23/wip-doc-2024-07-31-backport-58946-to-quincy

quincy: doc/rados/operations: remove vanity cluster name reference from crush…

doc/rados/operations: remove vanity cluster name reference from crush-map.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 62562ec65e3c1cc040a57ab77fc219e0aad25651)

Merge pull request #58942 from zdover23/wip-doc-2024-07-31-backport-58922-to-quincy

quincy: doc/cephadm: edit "Using Custom Images"

Merge pull request #58945 from zdover23/wip-doc-2024-07-31-backport-58908-to-quincy

quincy: doc/glossary: add "ceph-fuse" entry

Merge pull request #58939 from zdover23/wip-doc-2024-07-31-backport-58918-to-quincy

quincy: doc/dev: improve basic-workflow.rst

doc/glossary: add "ceph-fuse" entry

Add a "ceph-fuse" entry to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d1c049d24befedc396221dafcc66db8c40acf5ab)