Merge branch 'revert-54865-wip-63782-reef' of https://github.com/ceph/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-64073-reef' of https://github.com/joscollin/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-62842-reef' of https://github.com/mchangir/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63922-reef' of https://github.com/vshankar/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'bp/renameat2/reef' of https://github.com/leonid-s-usov/ceph-ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63809' of https://github.com/lxbsz/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63810-reef' of https://github.com/neesingh-rh/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63416-reef' of https://github.com/mchangir/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-53418-reef' of https://github.com/vshankar/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63676-reef' of https://github.com/batrick/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63575-reef' of https://github.com/mchangir/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'bp/balance-after-stop/reef' of https://github.com/leonid-s-usov/ceph-ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63273' of https://github.com/lxbsz/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-63474-reef' of https://github.com/joscollin/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Merge branch 'wip-62004-reef' of https://github.com/k0ste/ceph into wip-vshankar-testing1-reef-2024-02-02-1152

Revert "reef: qa: add support/qa for cephfs-shell on CentOS 9 / RHEL9"

Merge pull request #55282 from zdover23/wip-doc-2024-01-24-backport-55278-to-reef

reef: doc: specify correct fs type for mkfs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

doc: specify correct fs type for mkfs

The default value is ext2, which is not supported (anymore?).

Signed-off-by: Vladislav Glagolev <vladislav.glagolev@devexpress.com>
(cherry picked from commit 886af37744847246b3e70f54b8577ed4f9815c20)

Merge pull request #55271 from zdover23/wip-doc-2024-01-23-backport-55269-to-reef

reef: doc/radosgw: edit "read/write global rate limit" admin.rst

doc/radosgw: edit "read/write global rate limit" admin.rst

Edit "Reading/Writing Global Rate Limit Configuration" in
doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c67a5e5d4bad17e7ae799dd62a66d1e23ec18942)

PendingReleaseNotes: add note about 'm' and 'M' periods

Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 7e2546f9454abb28a435bf803607447e6a301faf)

doc/cephfs: add note about monthly scheduling and retention period

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 7431b813a52a80fccf60cf14234d38b61907ecaf)

qa: update minute period multiplier to m

Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 9c882347912b039fdb6c4f8a6ac7ec601793d8eb)

qa: change minute duration specifier from M to m

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit e7218e03ae88c933738c94d63a0de6a7ce2cfe5b)

mgr/snap_schedule: change minute duration specifier from M to m

Problem:
As per the issue tracker, the period spec specifier 'M' is not
consistent with what is used elsewhere, like the period specifiers
displayed in the 'ceph status' command output.
The 'M' period specifier is used as a 'minute' level period specifier by
the cephfs team.
The issue reporter suggests to use 'M' as a 'month' period specifier.

Solution:
Since the 'minute' level period specifer, 'M', is used internally by
the development team, it is failrly easy to swap the 'minute' ('M')
level and 'month' ('m') level period specifers to finally mean that 'm'
implies 'minute' level period and 'M' implies 'month' level period.
Also, since this is the first time that somebody has ever reported that
neither the 'M' nor the 'm' level specifiers work in production, it is a
good idea to fix them once and for all.

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 2f6d87d1811b745c4efcac72c03ddb42751ad634)

Merge pull request #55263 from zdover23/wip-doc-2024-01-22-backport-54993-to-reef

reef: doc/rados/operations: document `ceph balancer status detail`

doc/rados/operations: document `ceph balancer status detail`

Document change in https://github.com/ceph/ceph/pull/54801

Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit 159751b68085fbe0fe10a881ff8bedecda11142f)

Merge pull request #55260 from zdover23/wip-doc-2024-01-21-backport-55190-to-reef

reef: doc/radosgw: edit "Enable/Disable Bucket Rate Limit"

doc/radosgw: edit "Enable/Disable Bucket Rate Limit"

Edit "Enable/Disable Bucket Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a1fca50dfdcb30e58ebcc4ef60ef921b977efc81)

Merge pull request #55253 from zdover23/wip-doc-2024-01-20-backport-55177-to-reef

reef: doc/radosgw: edit admin.rst "Get Bucket Rate Limit"

doc/radosgw: edit admin.rst "Get Bucket Rate Limit"

Edit "Get Bucket Rate Limit" in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7c4fdd3762831cb86993b1b4a814ea68a9ea2401)

Merge pull request #55242 from zdover23/wip-doc-2024-01-19-backport-55170-to-reef

reef: doc/radosgw: edit admin.rst "set bucket rate limit"

Merge pull request #55244 from zdover23/wip-doc-2024-01-19-backport-55168-to-reef

reef: doc/rados/operations: add EC overhead table to erasure-code.rst

Merge pull request #55246 from zdover23/wip-doc-2024-01-19-backport-54915-to-reef

reef: doc/cephfs/client-auth.rst: correct ``fs authorize cephfs1 /dir1 clie…

Merge pull request #54764 from rhcs-dashboard/wip-63652-reef

reef: mgr/dashboard: get rgw port from ssl_endpoint

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #55186 from rhcs-dashboard/wip-63725-reef

reef: mgr/dashboard: subvolume snapshot management

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

doc/cephfs/client-auth.rst: correct ``fs authorize cephfs1 /dir1 client.x rw``

(cherry picked from commit 223848a08f58fc20ee4afc008fabbec2d448be68)

doc/rados/operations: add EC overhead table to erasure-code.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 9e1999c13d5cab65e88200aa2bfc5ce385d98679)

doc/radosgw: edit admin.rst "set bucket rate limit"

Edit "Set Bucket Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e9b3ef4cb11760392ff7f766d652289636fcf5eb)

qa: drop mon_host check from test_cephfs_mirror_peer_bootstrap

Fixes: https://tracker.ceph.com/issues/63614
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 1c241e49e5a97361b42408ec5bb5ec72eadcc54b)

pybind/mgr/mirroring: drop mon_host from peer_list

Drop mon_host from peer_list to make it similar to rbd_mirror peer_list.

Fixes: https://tracker.ceph.com/issues/63614
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 686ef1d4d3e09a364396ec668ac97b08844b166b)

Merge pull request #55231 from zdover23/wip-doc-2024-01-18-backport-55161-to-reef

reef: doc/rados/operations: Fix off-by-one errors in control.rst

doc/rados/operations: Fix off-by-one errors in control.rst

Description of examples was off by one

Signed-off-by: tobydarling <anothercoffee@gmail.com>
(cherry picked from commit c636e3ed2d459a063ae6ab0e2dc876db5c58c7e6)

Merge pull request #55212 from afreen23/wip-64036-reef

reef: mgr/dashboard: Fixes multisite topology page breadcrumb

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55222 from aaSharma14/wip-64062-reef

reef: mgr/dashboard: add frontend unit tests for rgw multisite sync status card

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55221 from aaSharma14/wip-62575-reef

reef: mgr/dashboard: Create realm sets to default

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: add frontend unit tests for rgw multisite sync status
card

Fixes: https://tracker.ceph.com/issues/64039
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit d169a206d11283428f7eb74a928525bc93f58915)

mgr/dashboard: Create realm sets to default

In Multisite page, When we create a realm the realm sets to default even if some other realm is already default and default checkbox in unchecked as well while creating.

Fixes: https://tracker.ceph.com/issues/62453
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 00d3ead5ee52cd9865b3f8b2df8416f6446d81f4)

Merge pull request #55153 from rzarzynski/wip-64005-reef

reef:  common/weighted_shuffle: don't feed std::discrete_distribution with all-zero weights

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #55112 from k0ste/wip-63974-reef

reef: Tools/rados: Improve Error Messaging for Object Name Resolution

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #55078 from amathuria/wip-63947-reef

reef: mgr/pg_autoscaler: add check for norecover flag

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #55046 from ljflores/wip-reef-backport-pr-54907

reef: osd: fix Incremental decode for new/old_pg_upmap_primary

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #55077 from amathuria/wip-63946-reef

reef: DaemonServer.cc: fix config show command for RGW daemons

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54783 from ifed01/wip-ifed-better-osd-robust-reef

reef: osd: improve OSD robustness.

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #54693 from Matan-B/wip-63651-reef

reef: tools/ceph_objectstore_tool: action_on_all_objects_in_pg to skip pgmeta

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #53888 from sseshasa/wip-63128-reef

reef: mon/ConfigMonitor: Show localized name in "config dump --format json" output

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Fixes multisite topology page breadcrumb

The multi-site topology page just says object in breadcrumb.
The fix adds the missing "multi-site" breadcrumb.

Fixes https://tracker.ceph.com/issues/63635

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit f75a9da2871113a156092ff28f29b9d12f465975)

Merge pull request #55104 from rkachach/fix_issue_63957_63958

reef: backport of fixes for 63678 and 63694

Merge pull request #55194 from zdover23/wip-doc-2024-01-16-backport-55169-to-reef

reef: docs/radosgw: edit admin.rst "enable/disable user rate limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #55191 from rhcs-dashboard/reef-docs-build-fix

reef: admin/doc-requirements: bump Sphinx to 5.0.2

Reviewed-by: Matan-B <NOT@FOUND>

Merge pull request #55124 from rhcs-dashboard/wip-59470-reef

reef: mgr/dashboard: fix e2e failure related to landing page

docs/radosgw: edit admin.rst "enable/disable user rate limit"

Edit "Enable/Disable User Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1cd996b6fe2e3c923001a69772c7b7635d058f2b)

admin/doc-requirements: bump Sphinx to 5.0.2

```
Running Sphinx v4.5.0

Sphinx version error:
The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0; it therefore cannot be built with this version.
```

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a916feeee757e4e196967944feeb1e9b1f92c398)

mgr/dashboard: delete cephfs snapshot

Fixes: https://tracker.ceph.com/issues/63990
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit c3d7f70b7a7d594050e3e231cd7d6544179fdfeb)

mgr/dashboard: subvolume snapshot creation form

Fixes: https://tracker.ceph.com/issues/63934
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit f5d1b7df2ed090b45dd74634e3c92d4df2c4017c)

mgr/dashboard: small fixes to directories view

Increased the precedence for Directories
Expand the node by default and select them as well

Fixes: https://tracker.ceph.com/issues/63754
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 4eb42ff15db14680c0798ceb752fa6578b76fb92)

mgr/dashboard: fs snapshots e2e

Fixes: https://tracker.ceph.com/issues/63237
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 4689c3d65eb758cecf944da0fe2c5686fa2918ee)

mgr/dashboard: cephfs subvolume list snapshots

Added a tab for displaying the subvolume snapshots
- this tab will show an info alert when there are no subvolumes present
- if the subvolume is present, then it'll be auto-selected by default

Implemented a filter to search the groups and subvolumes by its name.
Also added a scrollbar when there are too many items in the nav list

Modified the REST APIs to fetch only the names of the resources and
fetch the info when an API call is requesting for it.

Added unit tests

Fixes: https://tracker.ceph.com/issues/63237
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit b35be54ed9f23b7fc7859f054902e37cb88cefd8)

Merge pull request #55163 from zdover23/wip-doc-2024-01-13-backport-54530-to-reef

reef: ceph-menv:fix typo in README

ceph-menv:fix typo in README

Signed-off-by: yu.wang <wangyu@wyu.space>
(cherry picked from commit 4efd9317b83eca48c1a92724e7ebe304650fbc04)

Merge pull request #55157 from zdover23/wip-doc-2024-01-12-backport-55146-to-reef

reef: doc/radosgw: edit admin.rst "get user rate limit"

doc/radosgw: edit admin.rst "get user rate limit"

Edit "Get User Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1c561e2beac67c85c40f2adf387c42aba8175ecb)

Merge pull request #55150 from zdover23/wip-doc-2024-01-12-backport-55142-to-reef

reef: doc/radosgw - edit admin.rst "set user rate limit"

test/test_weighted_shuffle: verify weights containing zeros

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit d02b17ff84c61123ed27d79dc177c2cfbbe6a72f)

common/weighted_shuffle: don't feed std::discrete_distribution with all-zero weights

This flaw results is the assertions like the following one:

```
/usr/include/c++/11/bits/random.tcc:2667: void std::discrete_distribution<_IntType>::param_type::_M_initialize() [with _IntType = int]: Assertion '__sum > 0' failed.
Aborted (core dumped)
```

The reason behind is that `std::discrete_distribution` sums
the weights and uses the result as a divisor.

Fixes: https://tracker.ceph.com/issues/62645
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 1b7a7a8df88ffac007dbafdecc131807de66c046)

doc/radosgw - edit admin.rst "set user rate limit"

Edit "Set User Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8660b4edc71113d5d2d51438c9fd6f388b35a214)

Merge pull request #54296 from yuvalif/wip-59498-reef

reef: rgw/lua: fix CopyFrom crash

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Tools/rados: Improve Error Messaging for Object Name Resolution

The current implementation of 'rados clearomap' exhibits a behavior where
an error message is generated without the associated object name or,
in the case of a non-existent object name, may result in a segmentation fault.

The proposed fix addresses this issue by enhancing the error message.
After applying the fix, error messages will consistently display the correct
object name, providing users with more accurate and actionable information.

Fixes: https://tracker.ceph.com/issues/63541
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
(cherry picked from commit 313bfca056ef7f83c0e8c70a3bc4ff9d070b63d9)

Merge pull request #54127 from BBoozmen/wip-63255-reef

reef: RGW/STS: when generating keys, take the trailing null character into account

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54862 from idryomov/wip-63654-reef

reef: librbd: fix regressions in ObjectListSnapsRequest

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #54852 from mkogan1/wip-63753-reef

reef: rgw: d3n: fix valgrind reported leak related to libaio worker threads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54844 from cbodley/wip-63775-reef

reef: add checking for rgw frontend init

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54843 from cbodley/wip-63773-reef

reef: rgw/iam: admin/system users ignore iam policy parsing errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54794 from k0ste/wip-62412-reef

reef: rgw/auth: Fix the return code returned by AuthStrategy,

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54791 from k0ste/wip-63626-reef

reef: rgw: SignatureDoesNotMatch for certain RGW Admin Ops endpoints w/v4 auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54736 from yuvalif/wip-63701-reef

reef: rgw/kafka/amqp: fix race conditionn in async completion handlers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54674 from cbodley/wip-63628-reef

reef: rgw: object lock avoids 32-bit truncation of RetainUntilDate

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54328 from petrutlucian94/wip-63430-reef

reef: rgw: fix cloud-sync multi-tenancy scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54129 from ivancich/wip-63247-reef

reef: rgw: only buckets with reshardable layouts need to be considered for resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>

encoding: use concepts for chrono encoding

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b33e08657ecb7949505c6eae920c83e1e84096bb)

rgw: object lock uses 64-bit encoding for RetainUntilDate

the default encoding of ceph::real_time truncates seconds to uint32_t,
so stores the wrong timestamp for object lock enforcement

Fixes: https://tracker.ceph.com/issues/63537
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1fd4309fbbebf0d3e67aa2800d5fb3c7de19dcc7)

Merge pull request #55128 from zdover23/wip-doc-2024-01-10-backport-55106-to-reef

reef: doc/rgw: edit admin.rst - rate limit management

doc/rgw: edit admin.rst - rate limit management

Edit the "Rate Limit Management" section in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 47fabe757cff630aaf23a85d268b7406057244e2)

mgr/dashboard: fix e2e failure related to landing page

Fixes: https://tracker.ceph.com/issues/59142
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 6ec48eeb261bc8aceb1bfa3ee3732acfb0974478)

Merge pull request #55091 from cbodley/wip-63964-reef

reef: make-dist: don't use --continue option for wget

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #55084 from petrutlucian94/wip-63955-reef

reef: win32_deps_build.sh: change Boost URL

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

reef: backport of fixes for 63678 and 63694
Fixes: https://tracker.ceph.com/issues/63957
Fixes: https://tracker.ceph.com/issues/63958
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

make-dist: don't use --continue option for wget

the boost jfrog mirror is broken and returns an HTML error page instead
of the archive. the file size of this page is 11534 bytes

when download_from() retries the download from download.ceph.com, the -c
option tells it to resume the download of the existing file. the
resulting boost_1_82_0.tar.bz2 ends up with the correct total file size
of 121325129 bytes, but the first 11534 bytes still correspond to the
HTML from jfrog. that causes the sha256sum mismatch

remove the -c option so that wget fetches the archive in its entirety

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2ba7d6e59e8750867059c29a33c22bcb3e358b65)

Merge pull request #54818 from baergj/wip-63746-reef

reef: librbd: Append one journal event per image request

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #54807 from trociny/wip-63733-reef

reef:  pybind/rbd: fix compilation with cython3

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>