Merge pull request #55400 from zdover23/wip-doc-2024-01-31-backport-55396-to-quincy

quincy: doc/architecture: improve some paragraphs

doc/architecture: improve some paragraphs

Improve paragraphs under the heading "The Ceph Storage Cluster". Remove
a sentence that was pleonastic in its context in the paragraph.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2f0542d66901295cf875893de0ac15304578d917)

Merge pull request #54706 from k0ste/wip-63313-quincy

quincy: ceph-volume: fix a bug in _check_generic_reject_reasons

Merge pull request #54630 from guits/wip-63598-quincy

quincy: ceph-volume: fixes fallback to stat in is_device and is_partition

Merge pull request #55385 from zdover23/wip-doc-2024-01-31-backport-55372-to-quincy

quincy: doc/architecture.rst - fix typo

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/architecture.rst - fix typo

s/requies/requires

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6c0417fbcbe6b9760b3836e5166d6bd929578096)

Merge pull request #55248 from rhcs-dashboard/wip-63653-quincy

quincy: mgr/dashboard: get rgw port from ssl_endpoint

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #55344 from zdover23/wip-doc-2024-01-29-backport-55341-to-quincy

quincy: doc/architecture.rst: improve rados definition

doc/architecture.rst: improve rados definition

Improve the definition of RADOS, and link to information about RADOS.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 763f6b7a22e846962f388d58fd7e699cbf16ffe7)

Merge pull request #55339 from zdover23/wip-doc-2024-01-28-backport-55333-to-quincy

quincy: doc/radosgw: fix verb disagreement - index.html

doc/radosgw: fix verb disagreement - index.html

Fix a tricky verb disagreement and rewrite a few sentences for what I
hope is greater clarity.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9f271093f4331381dc024cb4309f9f486d366818)

Merge pull request #55322 from zdover23/wip-doc-2024-01-26-backport-55307-to-quincy

quincy: doc/radosgw: edit "Usage" admin.rst

doc/radosgw: edit "Usage" admin.rst

Edit "Usage" in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d8df6f61e817a34c2c3282224cff117ae43e3f98)

Merge pull request #53459 from rhcs-dashboard/wip-62603-quincy

quincy: mgr/dashboard: remove unnecessary failing hosts e2e

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53809 from rhcs-dashboard/wip-63090-quincy

quincy: mgr/dashboard: rbd image hide usage bar when disk usage is not provided

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #55283 from zdover23/wip-doc-2024-01-24-backport-55278-to-quincy

quincy: doc: specify correct fs type for mkfs

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #54411 from mihalicyn/wip-63477-quincy

quincy: MClientRequest: properly handle ceph_mds_request_head_legacy for ext_num_retry, ext_num_fwd, owner_uid, owner_gid

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #54244 from vshankar/wip-63284-quincy

quincy: client: always refresh mds feature bits on session open

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #53363 from k0ste/wip-57111-quincy

quincy: mds: replacing bootstrap session only if handle client session message

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #53358 from k0ste/wip-61992-quincy

quincy: mds/MDSRank: Add set_history_slow_op_size_and_threshold for op_tracker

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #52976 from batrick/wip-62423-quincy

quincy: mds: dump locks when printing mutation ops

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #52820 from rishabh-d-dave/wip-62336-quincy

quincy: mds: use variable g_ceph_context directly in MDSAuthCaps

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #52271 from dparmar18/wip-61840-quincy

quincy: do not evict clients if OSDs are laggy

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>

doc: specify correct fs type for mkfs

The default value is ext2, which is not supported (anymore?).

Signed-off-by: Vladislav Glagolev <vladislav.glagolev@devexpress.com>
(cherry picked from commit 886af37744847246b3e70f54b8577ed4f9815c20)

Merge pull request #55272 from zdover23/wip-doc-2024-01-23-backport-55269-to-quincy

quincy: doc/radosgw: edit "read/write global rate limit" admin.rst

doc/radosgw: edit "read/write global rate limit" admin.rst

Edit "Reading/Writing Global Rate Limit Configuration" in
doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c67a5e5d4bad17e7ae799dd62a66d1e23ec18942)

Merge pull request #55243 from zdover23/wip-doc-2024-01-19-backport-55170-to-quincy

quincy: doc/radosgw: edit admin.rst "set bucket rate limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #55264 from zdover23/wip-doc-2024-01-22-backport-54993-to-quincy

quincy: doc/rados/operations: document `ceph balancer status detail`

doc/rados/operations: document `ceph balancer status detail`

Document change in https://github.com/ceph/ceph/pull/54801

Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit 159751b68085fbe0fe10a881ff8bedecda11142f)

Merge pull request #55261 from zdover23/wip-doc-2024-01-21-backport-55190-to-quincy

quincy: doc/radosgw: edit "Enable/Disable Bucket Rate Limit"

doc/radosgw: edit "Enable/Disable Bucket Rate Limit"

Edit "Enable/Disable Bucket Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a1fca50dfdcb30e58ebcc4ef60ef921b977efc81)

Merge pull request #55254 from zdover23/wip-doc-2024-01-20-backport-55177-to-quincy

quincy: doc/radosgw: edit admin.rst "Get Bucket Rate Limit"

doc/radosgw: edit admin.rst "Get Bucket Rate Limit"

Edit "Get Bucket Rate Limit" in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7c4fdd3762831cb86993b1b4a814ea68a9ea2401)

Merge pull request #55245 from zdover23/wip-doc-2024-01-19-backport-55168-to-quincy

quincy: doc/rados/operations: add EC overhead table to erasure-code.rst

Merge pull request #55247 from zdover23/wip-doc-2024-01-19-backport-54915-to-quincy

quincy: doc/cephfs/client-auth.rst: correct ``fs authorize cephfs1 /dir1 clie…

mgr/dashboard: get port from ssl_endpoint

Fixes: https://tracker.ceph.com/issues/63564
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit cacc2eb70fde37475f0d30ca61209d9162fdcc85)

Merge pull request #54176 from rhcs-dashboard/wip-63309-quincy

quincy: mgr/dashboard: fix rgw port manipulation error in dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: jliebl-git <NOT@FOUND>

Merge pull request #53385 from rhcs-dashboard/wip-62716-quincy

quincy: mgr/dashboard: remove green tick on old password field

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

doc/cephfs/client-auth.rst: correct ``fs authorize cephfs1 /dir1 client.x rw``

(cherry picked from commit 223848a08f58fc20ee4afc008fabbec2d448be68)

doc/rados/operations: add EC overhead table to erasure-code.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 9e1999c13d5cab65e88200aa2bfc5ce385d98679)

doc/radosgw: edit admin.rst "set bucket rate limit"

Edit "Set Bucket Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e9b3ef4cb11760392ff7f766d652289636fcf5eb)

Merge pull request #55060 from cbodley/wip-63937-quincy

quincy: test/pybind: replace nose with pytest

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #55232 from zdover23/wip-doc-2024-01-18-backport-55161-to-quincy

quincy: doc/rados/operations: Fix off-by-one errors in control.rst

doc/rados/operations: Fix off-by-one errors in control.rst

Description of examples was off by one

Signed-off-by: tobydarling <anothercoffee@gmail.com>
(cherry picked from commit c636e3ed2d459a063ae6ab0e2dc876db5c58c7e6)

Merge pull request #55195 from zdover23/wip-doc-2024-01-16-backport-55169-to-quincy

quincy: docs/radosgw: edit admin.rst "enable/disable user rate limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #55204 from zdover23/wip-doc-2024-01-17-backport-55185-to-quincy

quincy: admin/doc-requirements: bump Sphinx to 5.0.2

Reviewed-by: Nizamudeen A <nia@redhat.com>

admin/doc-requirements: bump Sphinx to 5.0.2

```
Running Sphinx v4.5.0

Sphinx version error:
The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0; it therefore cannot be built with this version.
```

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a916feeee757e4e196967944feeb1e9b1f92c398)

Merge pull request #54894 from mchangir/wip-62841-quincy

quincy: mgr/snap_schedule: add support for monthly snapshots

Merge pull request #54717 from batrick/wip-63675-quincy

quincy: mds: revert standby-replay trimming changes

Merge pull request #53824 from rishabh-d-dave/wip-63097-quincy

quincy: qa/cephfs: fix ior project build failure

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #54892 from mchangir/wip-63417-quincy

quincy: mds: do not simplify fragset

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #55058 from vshankar/wip-63921-quincy

quincy: mds: change priority of mds rss perf counter to useful

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #55010 from leonid-s-usov/bp/renameat2/quincy

quincy: client/fuse: handle case of renameat2 with non-zero flags

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #54910 from lxbsz/wip-63808

quincy: mds: set the loner to true for LOCK_EXCL_XSYN

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #54494 from leonid-s-usov/bp/balance-after-stop/quincy

quincy: mds/MDBalancer: ignore queued callbacks if MDS is not active

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #54038 from vshankar/wip-61830-quincy

quincy: qa: assign file system affinity for replaced MDS

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #53826 from rishabh-d-dave/wip-63095-quincy

quincy: qa/cephfs: fix build failure for mdtest project

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #53857 from rhcs-dashboard/wip-63119-quincy

quincy: qa/workunits: fix test_dashboard_e2e.sh: no spec files found

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #55123 from rhcs-dashboard/wip-59469-quincy

quincy: mgr/dashboard: fix e2e failure related to landing page

Reviewed-by: Nizamudeen A <nia@redhat.com>

docs/radosgw: edit admin.rst "enable/disable user rate limit"

Edit "Enable/Disable User Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1cd996b6fe2e3c923001a69772c7b7635d058f2b)

Merge pull request #55164 from zdover23/wip-doc-2024-01-13-backport-54530-to-quincy

quincy: ceph-menv:fix typo in README

ceph-menv:fix typo in README

Signed-off-by: yu.wang <wangyu@wyu.space>
(cherry picked from commit 4efd9317b83eca48c1a92724e7ebe304650fbc04)

Merge pull request #55098 from cbodley/wip-63840-quincy

quincy: qa: remove vstart runner from radosgw_admin task

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54851 from mkogan1/wip-63752-quincy

quincy: rgw: d3n: fix valgrind reported leak related to libaio worker threads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54842 from cbodley/wip-63772-quincy

quincy: rgw/iam: admin/system users ignore iam policy parsing errors

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #54737 from yuvalif/wip-63702-quincy

quincy: rgw/kafka/amqp: fix race conditionn in async completion handlers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54675 from cbodley/wip-63627-quincy

quincy: rgw: object lock avoids 32-bit truncation of RetainUntilDate

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #54445 from tobias-urdin/service-token-quincy

quincy: rgw/auth: Add service token support for Keystone auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54166 from cbodley/wip-58237-quincy

quincy: rgw: beast frontend checks for local_endpoint() errors

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #54165 from cbodley/wip-59132-quincy

quincy: rgw/s3: DeleteObjects response uses correct delete_marker flag

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #54162 from cbodley/wip-61353-quincy

quincy: rgw/s3: ListObjectsV2 returns correct object owners

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #53763 from cbodley/wip-63044-quincy

quincy: rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #53473 from k0ste/wip-55700-quincy

quincy: radosgw-admin: don't crash on --placement-id without --storage-class

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52795 from cbodley/wip-62310-quincy

quincy: rgw/crypt: apply rgw_crypt_default_encryption_key by default

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #52049 from trociny/wip-59612-quincy

quincy: rgw/sts: AssumeRole no longer writes to user metadata

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #51976 from trociny/wip-59065-quincy

quincy: rgw: fix multipart upload object leaks due to re-upload

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #55158 from zdover23/wip-doc-2024-01-12-backport-55146-to-quincy

quincy: doc/radosgw: edit admin.rst "get user rate limit"

doc/radosgw: edit admin.rst "get user rate limit"

Edit "Get User Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1c561e2beac67c85c40f2adf387c42aba8175ecb)

Merge pull request #54594 from idryomov/wip-53593-quincy

quincy: osd: don't require RWEXCL lock for stat+write ops.

Merge pull request #55151 from zdover23/wip-doc-2024-01-12-backport-55142-to-quincy

quincy: doc/radosgw - edit admin.rst "set user rate limit"

doc/radosgw - edit admin.rst "set user rate limit"

Edit "Set User Rate Limit" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8660b4edc71113d5d2d51438c9fd6f388b35a214)

rgw: object lock uses 64-bit encoding for RetainUntilDate

the default encoding of ceph::real_time truncates seconds to uint32_t,
so stores the wrong timestamp for object lock enforcement

Fixes: https://tracker.ceph.com/issues/63537
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1fd4309fbbebf0d3e67aa2800d5fb3c7de19dcc7)

encoding: add round_trip_encode()/decode() for chrono types

the default encodings for chrono types were made to be
backward-compatible with utime_t, so truncated seconds to 32 bits

adds new functions that encode these chrono types using their underlying
representation, which for ceph::real_time and ceph::timespan is
'nanoseconds as uint64_t'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b7aa16da4bbb444f6361e4f14be7ef64b0ea3bb1)

qa/suites: install pytest for pybind tasks

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cbdd5209957646f497b5e0df2f1a8dc6d918267f)

Conflicts:
qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests.yaml
does not exist on quincy

test/pybind/rgwfs: convert from nose to pytest

* use a fixture for setup_test()
* replace nose assertions

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 136b3ec5323eb392e1017fffba1a538b87b77a58)

test/pybind/rados: convert from nose to pytest

* replace nose.Skiptest with pytest.skip()
* replace nose attrs with @pytest.mark
* replace nose assertions
* replace setUp/tearDown with setup/teardown_method

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0d0128f50edd16dd257a4b897bb63d9168497f80)

Conflicts:
qa/suites/crimson-rados/basic/tasks/rados_python.yaml
src/test/pybind/test_rados.py
no rollback or wait markers, added snap marker

test/pybind/cephfs: convert from nose to pytest

* use a fixture for setup_test()
* invoke test_cephfs.py with pytest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0bd813f069ed7a3ee691fbfb33da017bb00c1a55)

Conflicts:
src/test/pybind/test_cephfs.py
missing https://github.com/ceph/ceph/pull/43546
missing https://github.com/ceph/ceph/pull/46905

test/pybind/rbd: convert from nose to pytest

* use fixtures for temporary images and groups
* use pytest.skip instead of nose.SkipTest
* replace setUp/tearDown with setup/teardown_method
* add @pytest.mark.skip_if_crimson
* replace nose assertions

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit af04457a43921bcac85b572098245a4c8a79452e)

Conflicts:
qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests.yaml
does not exist on quincy
qa/suites/crimson-rados/rbd/tasks/rbd_python_api_tests_old_format.yaml
qa/workunits/rbd/test_librbd_python.sh
src/test/pybind/test_rbd.py
skip_if_crimson filter not present on quincy

Merge pull request #55129 from zdover23/wip-doc-2024-01-10-backport-55106-to-quincy

quincy: doc/rgw: edit admin.rst - rate limit management

doc/rgw: edit admin.rst - rate limit management

Edit the "Rate Limit Management" section in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 47fabe757cff630aaf23a85d268b7406057244e2)

mgr/dashboard: fix e2e failure related to landing page

Fixes: https://tracker.ceph.com/issues/59142
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 6ec48eeb261bc8aceb1bfa3ee3732acfb0974478)

Merge pull request #55092 from cbodley/wip-63965-quincy

quincy: make-dist: don't use --continue option for wget

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #55085 from petrutlucian94/wip-63956-quincy

quincy: win32_deps_build.sh: change Boost URL

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #55025 from idryomov/wip-63607-quincy

quincy: librados: make querying pools for selfmanaged snaps reliable

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #54123 from batrick/wip-63276-quincy

quincy: cmake: use or turn off liburing for rocksdb

Reviewed-by: Laura Flores <lflores@redhat.com>

qa: remove vstart runner from radosgw_admin task

Fixes: https://tracker.ceph.com/issues/63583
Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit a44fd94b9681441a3251d0988d1ddd159f3733f8)

Merge pull request #52461 from batrick/wip-62023-quincy

quincy: pybind/mgr: reopen database handle on blocklist

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #53973 from Matan-B/wip-63181-quincy

quincy: osd/OSD: introduce reset_purged_snaps_last

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #53718 from leonid-s-usov/bp/msg-wait/quincy

quincy: msg/AsyncMessenger: re-evaluate the stop condition when woken up in 'wait()'

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #52458 from kamoltat/wip-ksirivad-backport-quincy-48209

quincy: osd/OSDMap: Check for uneven weights & != 2 buckets post stretch mode

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>