Merge pull request #58461 from cbodley/wip-cls-rgw-index-async

cls/rgw: warn about CLSRGWConcurrentIO use in asio threads

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #59726 from avanthakkar/earmark-nfs

mgr/nfs: earmark resolver for subvolume

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge PR #58936 into main

* refs/pull/58936/head:
mds: do not duplicate journaler write heads
mds: use Journaler getters
osdc: properly acquire locks for getters
osdc: add print method for Journaler::Header
mds: do not trim segments after open file table commit
mds: delay expiry if LogSegment is ahead of committed oft seq
mds: do not write journal head twice on trim
mds: simplify and explain expiry finisher ctx
mds: add mds_lock asserts for journal flush
mds: skip second wait_for_safe
mds: trim only to the LogSegment created for flush
mds: allow passing explicit seq to trim to
mds: quiet unhelpful debug message
mds: add C_IO_Wrapper completion debugging
mds: add dout for new segment

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59864 from cbodley/wip-68134

rgw/rados: RGWRados::delete_objs_inline() uses AioThrottle

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #59536 from smanjara/rgw-full-sync-cr

rg/multisite: handle errors properly in RGWDataFullSyncSingleEntryCR()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #57146 from kamoltat/wip-ksirivad-fix-connection-score-json

src/mon/ConnectionTracker.cc: Fix dump function
Reviewed-by Kamoltat Sirivadhna <ksirivad@redhat.com>

Merge pull request #58472 from YiteGu/reinforce-mon-scrub-db-get-error

mon/scrub: log error details of store access failures
Reviewed-by Kamoltat Sirivadhna <ksirivad@redhat.com>

Merge pull request #59609 from cbodley/wip-rgw-dir-suggest-reshard

rgw/rados: guard against dir suggest during reshard

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #59441 from ceph/wip-oozmen-67656-integration-test

test/rgw/notifications: integration test for the procedure of migrating an existing v2 topic to an account

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59244 from cbodley/wip-rgw-notification-token-waiter

rgw/notify: process_queue() constructs wait tokens when spawning

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59981 from guits/fix_68231

node-proxy: fix a regression when processing the RedFish API

Merge pull request #59946 from rhcs-dashboard/rbd-snapshot

mgr/dashboard: Forbid snapshot name "." and any containing "/".

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

Merge pull request #59986 from nbalacha/wip-nbalacha-doc-typo

doc: fix typo

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #59846 from rhcs-dashboard/usermanagment-roles

mgr/dashboard: carbonize user management-role form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: nmunet <NOT@FOUND>

Merge pull request #59194 from rhcs-dashboard/add-ssl-prometheus-federate

mgr/dashboard: Add ssl prometheus federate

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

doc: fix typo

Fixed a typo in the jaeger traces documentation.

Signed-off-by: N Balachandran <nibalach@redhat.com>

Merge pull request #58719 from rhcs-dashboard/add-progress-multisite-automation

mgr/dashboard: Add progress step to multisite automation wizard

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge PR #56776 into main

* refs/pull/56776/head:
client: fix d_reclen for readdir

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #59467 into main

* refs/pull/59467/head:
qa: correct daemon for warning conf
qa: print config dump after setting

Reviewed-by: Adam King <adking@redhat.com>

Merge PR #59504 into main

* refs/pull/59504/head:
mds: dump next_snap when checking dentry corruption

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #59810 into main

* refs/pull/59810/head:
client: remove hypertable

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mds: do not duplicate journaler write heads

The MDS can unnecessarily write out the journaler head multiple times. Check
the last_written Header to see if it's necessary to write and gather waiters.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: use Journaler getters

To access the Journaler::Header safely.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osdc: properly acquire locks for getters

This was left as a TODO. : /

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osdc: add print method for Journaler::Header

For dout prints.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: do not trim segments after open file table commit

Previously, the trimming of expired segments would only occur if the open file
table's committed sequence number is past the segment to be trimmed. This is
now part of the expiry checks so it's no longer necessary to restart trimming
when the open file table commit completes.

Furthermore, this was confusing to `flush journal` as it was waiting for the
journal head write but the open file table commit already triggered the trim.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: delay expiry if LogSegment is ahead of committed oft seq

And remove the misplaced conditional in ::trim_expiring_segments.

This is necessary as the `flush journal` command gets confused by missing a
wait_for_expiry on a LogSegment that is not actually expired.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: do not write journal head twice on trim

Add context to wait for MDLog::trim_expired_segments to write the journal head.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: simplify and explain expiry finisher ctx

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add mds_lock asserts for journal flush

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: skip second wait_for_safe

Now that we only trim up to the segment created for the flush, this second wait
should no longer be necessary.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: trim only to the LogSegment created for flush

This neatly avoids a situation where the MDS trims past the segment created for
the `flush journal` command.

Here's what it looks like:

    2024-07-30T20:15:51.679+0000 7f70a809a640  1 mds.d asok_command: flush journal {prefix=flush journal} (starting...)
    2024-07-30T20:15:51.679+0000 7f70a809a640 10 mds.1.26 handle_asok_command: flush journal
    2024-07-30T20:15:51.679+0000 7f70a809a640 20 mds.1.26 send
    2024-07-30T20:15:51.679+0000 7f70a809a640 20 mds.1.26 flush_mdlog
    2024-07-30T20:15:51.679+0000 7f70a809a640 10 mds.1.cache create_subtree_map 106 subtrees, 105 fullauth
    ...
    2024-07-30T20:15:51.680+0000 7f70a809a640 20 mds.1.log _submit_entry ESubtreeMap 105 subtrees , 0 ambiguous [metablob 0x101, 107 dirs]
    2024-07-30T20:15:51.680+0000 7f70a809a640 20 mds.1.log _start_new_segment: seq = 522843 starting new segment LogSegment(522843/0xffffffffffffffff events=0)
    ...
    2024-07-30T20:15:51.686+0000 7f709e887640 20 mds.1.26 handle_clear_mdlog: r=0
    2024-07-30T20:15:51.686+0000 7f709e887640  5 mds.1.26 trim_mdlog: beginning segment expiry
    2024-07-30T20:15:51.686+0000 7f709e887640 10 mds.1.log trim_to: 522843 23/0/0
    ...
    2024-07-30T20:15:51.858+0000 7f709e887640 20 mds.1.26 expire_segments
    2024-07-30T20:15:51.858+0000 7f709e887640  5 mds.1.26 expire_segments: waiting for 22 segments to expire
    ...
    2024-07-30T20:15:52.386+0000 7f70a008a640 20 mds.1.26 handle_expire_segments: r=0
    2024-07-30T20:15:52.386+0000 7f70a008a640 20 mds.1.26 trim_segments
    2024-07-30T20:15:52.386+0000 7f709e887640  5 mds.1.26 trim_expired_segments: expiry complete, expire_pos/trim_pos is now 22f4c774/22c00000
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(503314/0x22f4c774 events=649)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(503314/0x22f4c774 events=649)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(503963/0x230019e6 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(503963/0x230019e6 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(504987/0x23119d93 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(504987/0x23119d93 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(506011/0x23234d24 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(506011/0x23234d24 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(507035/0x23392678 events=397)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(507035/0x23392678 events=397)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(507432/0x23400db4 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(507432/0x23400db4 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(508456/0x2351c0af events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(508456/0x2351c0af events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(509480/0x23639df0 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(509480/0x23639df0 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(510504/0x2375912d events=607)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(510504/0x2375912d events=607)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(511111/0x2380290f events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(511111/0x2380290f events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(512135/0x2391b7f3 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(512135/0x2391b7f3 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(513159/0x23a6a15f events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(513159/0x23a6a15f events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(514183/0x23b855e7 events=442)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(514183/0x23b855e7 events=442)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(514625/0x23c01325 events=979)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(514625/0x23c01325 events=979)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(515604/0x23d0e086 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: expiring up to this major segment seq=515604
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(503314/0x22f4c774 events=649)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(503963/0x230019e6 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(504987/0x23119d93 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(506011/0x23234d24 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(507035/0x23392678 events=397)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(507432/0x23400db4 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(508456/0x2351c0af events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(509480/0x23639df0 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(510504/0x2375912d events=607)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(511111/0x2380290f events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(512135/0x2391b7f3 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(513159/0x23a6a15f events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(514183/0x23b855e7 events=442)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(514625/0x23c01325 events=979)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(515604/0x23d0e086 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(516628/0x23e2a879 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(516628/0x23e2a879 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(517652/0x23f47dbf events=665)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(517652/0x23f47dbf events=665)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(518317/0x240007be events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(518317/0x240007be events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(519341/0x2411af83 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(519341/0x2411af83 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(520365/0x24234a83 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(520365/0x24234a83 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(521389/0x24344292 events=691)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(521389/0x24344292 events=691)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(522080/0x24400e24 events=763)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: maybe expiring LogSegment(522080/0x24400e24 events=763)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: examining LogSegment(522843/0x244c4fda events=1)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments: expiring up to this major segment seq=522843
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(515604/0x23d0e086 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(516628/0x23e2a879 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(517652/0x23f47dbf events=665)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(518317/0x240007be events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(519341/0x2411af83 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(520365/0x24234a83 events=1024)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(521389/0x24344292 events=691)
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.log _trim_expired_segments: expiring LogSegment(522080/0x24400e24 events=763)
    2024-07-30T20:15:52.386+0000 7f709e887640 10 mds.1.log _trim_expired_segments waiting for expiry LogSegment(522843/0x244c4fda events=1)
    2024-07-30T20:15:52.386+0000 7f709e887640  1 -- [v2:127.0.0.1:6872/1197350146,v1:127.0.0.1:6873/1197350146] --> [v2:127.0.0.1:6850/2027314847,v1:127.0.0.1:6851/2027314847] -- osd_op(unknown.0.26:131573 2.7 2:ea85266a:::201.00000000:head [writefull 0~90 [fadvise_dontneed] in=90b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e76) -- 0x555daee1ac00 con 0x555da2bc7180
    2024-07-30T20:15:52.386+0000 7f709e887640  5 mds.1.26 trim_expired_segments: trim complete, expire_pos/trim_pos is now 244c4fda/22c00000
    2024-07-30T20:15:52.386+0000 7f709e887640 20 mds.1.26 write_journal_head

Related-to: https://tracker.ceph.com/issues/59119
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: allow passing explicit seq to trim to

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: quiet unhelpful debug message

looks like

    2024-08-15T06:08:36.056+0000 7f29379f4640 10 mds.0.journal try_to_expire waiting for purge of []

Nothing to purge!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add C_IO_Wrapper completion debugging

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: add dout for new segment

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Forbid snapshot name "." and any containing "/".

Fixes: https://tracker.ceph.com/issues/68198
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #59959 from idryomov/wip-librbd-supports-group-snap-get-info-define

librbd: add LIBRBD_SUPPORTS_GROUP_SNAP_GET_INFO define

Reviewed-by: N Balachandran <nibalach@redhat.com>

Merge pull request #59958 from adk3798/drop-py37-mgr-tox

pybind/mgr: drop py37 test from  tox.ini

Reviewed-by: John Mulligan <jmulligan@redhat.com>

node-proxy: fix a regression when processing the RedFish API

ce360a4a5f6 introduced a regression.

`Endpoint()` doesn't take a `EndpointMgr` object as parameter.
The call `e = Endpoint(self, _url, self.client)` obviously throws the
following error:

```
TypeError: __init__() takes 3 positional arguments but 4 were given
```

This commit fixes it.

Fixes: https://tracker.ceph.com/issues/68231
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #57809 from kchheda3/wip-dont-print-passwords

rgw/notification: add logic to prevent logging password associated with topics

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>

mgr/dashboard: adjust federate ssl code with mTLS

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

rgw/rados: RGWRados::delete_objs_inline() uses AioThrottle

use concurrent deletes to speed up inline garbage collection

Fixes: https://tracker.ceph.com/issues/68134
Signed-off-by: Casey Bodley <cbodley@redhat.com>

cls/rgw: cls_rgw_obj_chain uses vector instead of list

Signed-off-by: Casey Bodley <cbodley@redhat.com>

earmarking: improve error handling when getxattr fails

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

mgr/nfs: earmark resolver for subvolume

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #58952 from YiteGu/add-perfcounter-for-blk-discard

blk/kerneldevice: add perfcounter for block async discard

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

mgr/dashboard: Add progress bar to rgw multisite automation wizard

Fixes: https://tracker.ceph.com/issues/67829
Fixes: https://tracker.ceph.com/issues/67830
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #59918 from rhcs-dashboard/fix-68176-main

mgr/dashboard: fix setting compression type while editing rgw zone

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59925 from baum/NvmeofServiceSpec-service_id

pybind/mgr/orchestrator/module.py: NvmeofServiceSpec service_id

Merge pull request #59850 from aclamk/wip-aclamk-fix-67911-bdev-multi-label

os/bluestore: Fix BlueFS allocating bdev label reserved location.

Merge pull request #58509 from sseshasa/wip-hdd-osd-shard-params-for-mclock

common/options: Change HDD OSD shard configuration defaults for mClock

Reviewed-by: Mark Nelson <mark.a.nelson@gmail.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #59742 from guits/wip-guits-cv-cbt-zap-device

ceph-volume: add call to `ceph-bluestore-tool zap-device`

mgr/dashboard: carbonize user management-role form

Fixes: https://tracker.ceph.com/issues/68123
Signed-off-by: Puja Shahu <pshahu@redhat.com>
Signed-off-by: pujashahu <pujashahu@li-37e364cc-2d5a-11b2-a85c-fd7027b8f826.ibm.com>

Merge pull request #59944 from ronen-fr/wip-rf-standalone-query

qa/standalone/scrub: increase status updates frequency

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #59762 from aclamk/wip-aclamk-cbt-combined

ceph-bluestore-tool: Fixes for multilple bdev label

Merge pull request #59820 from rhcs-dashboard/rgw-multisite-sync-policy-improvements

mgr/dashboard: multisite sync policy improvements

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>

librbd: add LIBRBD_SUPPORTS_GROUP_SNAP_GET_INFO define

Make it easier to test if rbd_group_snap_get_info() and by extension
also rbd_group_snap_list2() APIs are available at compile time.  This
wasn't done in commit e5ccce14c4b0 ("rbd: add group snap info command")
because backporting wasn't intended.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #59387 from avanthakkar/earmark-smb

mgr/smb: earmark resolver for subvolume

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

pybind/mgr: drop py37 test from  tox.ini

Prior to this patch if you ran `tox` in src/pybind/mgr
it would include a test that runs the units tests using
python 3.7. These no longer pass and there is no longer
any reason to support python 3.7 on main.

Signed-off-by: Adam King <adking@redhat.com>

mgr/dashboard: multisite sync policy improvements

https://tracker.ceph.com/issues/68097

Changes for this PR includes:
1) Populating the destination zones select option with a set of options to choose from, for flow and pipe so that user can't enter any invalid zone name
2) Provided zone option as 'All Zones (*)' in pipe, if user want to select all zones for source and destination zones
3) We are hiding the UniqueId column on sync policy table as we do not want to show it as this column is introduced just to uniquely identify a row in the table and should not be displayed to users as it is part of the internal logic to work.

Signed-off-by: Naman Munet <nmunet@redhat.com>

Merge pull request #59891 from rhcs-dashboard/fix-68162-main

mgr/dashboard: fix handling NaN values in dashboard charts

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59890 from rhcs-dashboard/fix-68161-main

mgr/dashboard: update period after migrating to multi-site

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix setting compression type while editing rgw zone

Fixes: https://tracker.ceph.com/issues/68176
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

doc: update doc for subvolumes earmarking mentioning valid formats

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

mgr/smb: earmark resolver for subvolume

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #59858 from anoopcs9/fix-unversioned-obsoletes

ceph.spec.in: Fix unversioned Obsoletes: rpmbuild warning

Reviewed-by: Venky Shankar <vshankar@redhat.com>

qa/standalone/scrub: increase status updates frequency

To prevent test timeouts.
Also - remove a failing assertion on a specific 'pg query'
output, as it is not central to the test.

Fixes: https://tracker.ceph.com/issues/61385
Fixes: https://tracker.ceph.com/issues/64346
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #59728 from rhcs-dashboard/replace-hostname-addr-rgw

mgr/cephadm: use host address while updating rgw zone endpoints

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #59716 from rhcs-dashboard/fix-67897-main

mgr/rgwam: use realm/zonegroup/zone method arguments for period update

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #59723 from myoungwon/wip-remove-copy-overhead-ool-write

crimson/os/seastore: remove unnecessary memory copy during ool write

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #59746 from arm7star/main

test/crimson/test_messenger_thrash: fix a local variable scope issue

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #59691 from rhcs-dashboard/cephadm-e2e-endless-fix

mgr/dashboard: fix indefinite loop in cephadm dashboard e2e

Reviewed-by: afreen23 <NOT@FOUND>

Merge pull request #59595 from rhcs-dashboard/2269104-Clone-subgroup

mgr/dashboard: Cloning subvolume not listing _nogroup if no subvolume

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: nmunet <NOT@FOUND>

Merge pull request #59854 from avanthakkar/fix-nfs-doc

doc: update nfs doc for Kerberos setup of ganesha in Ceph

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #59640 from kshtsk/wip-jsonnetfmt-command-not-found

ceph.spec.in: we need jsonnet for all distroes for make check

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #59915 from guits/activate-lvm-tpm2

ceph-volume: fix OSD lvm/tpm2 activation

Merge pull request #59912 from guits/drive-group-spec-bool-args-followup-qa-nvme_loop-fix

qa: remove all bluestore signatures on devices

Merge pull request #59843 from adk3798/cephadm-oauth2-image-variable-name

mgr/cephadm: update oauth2 proxy image variable name

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
Reviewed-by: Ken Dreyer <kdreyer@ibm.com>

Merge pull request #59860 from adk3798/cephadm-no-exception-cert-store-get

mgr/cephadm: add --no-exception-when-missing flag to cert-store cert/key get

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #54334 from adk3798/cephadm-iscsi-teuth-extension

qa/cephadm: extend iscsi teuth test

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #59335 from mkogan1/wip-qat-logs

warn when QAT switches to software [de]compression

Merge pull request #59883 from idryomov/wip-group-snap-namespace-type-enum

librbd: introduce rbd_group_snap_namespace_type_t enum

Reviewed-by: N Balachandran <nibalach@redhat.com>

pybind/mgr/orchestrator/module.py: NvmeofServiceSpec service_id

- make service_id better alligned with default/empty group
  (https://github.com/ceph/ceph/commit/f6d552d7c777f1160545188dcffa6b685b05ca8a)
- fix service_id in nvmeof daemon add

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>

mgr/dashboard: fix handling NaN values in dashboard charts

Fixes: https://tracker.ceph.com/issues/68162
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgr/rgwam: use realm/zonegroup/zone method arguments for period update

period update method currently uses default zonegroup/zone for period
update commit. This PR is to use realm/zg/zone provided in the arguments
for period update. Also make the realm default that is created using realm pull
if it is the only realm present

Fixes: https://tracker.ceph.com/issues/67897
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #59737 from Suyashd999/fix-tkm1

rgw_log_backing: error code not returned

Reviewed-By: Adam Emerson <aemerson@ibm.com>, Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #59731 from Suyashd999/fix-ndf1

rgw/rgw_iam_policy: Checking for dereference of a null pointer (loaded from variable 't')

Reviewed-By: Adam Emerson <aemerson@ibm.com>, Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #57272 from ronen-fr/wip-rf-cv

osd/scrub: modify deep scrub interval randomization

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #59641 from ronen-fr/wip-rf-reduce-qprio

osd/scrub: disable high work-queue priority for high-prio scrubs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #59437 from ronen-fr/wip-rf-early-command

test/scrub: only instruct clean PGs to scrub

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #59832 from NitzanMordhai/wip-nitzan-crimson-enable-gate-assertions

common/gated: enable ceph_assert on shard id

osd/scrub: disable high work-queue priority for h.p. scrub

Prior to this fix, scrub-related messages were queued with
one of four priorities:

- if part of a regular, periodic scrub, the messages were queued
  with a very low priority (osd_scrub_priority. Default: 5), unless:
- if the pool configuration specified a higher priority for the
  scrub, the messages were queued with that priority
  (the SCRUB_PRIORITY pool configuration option);
- if a client operation was waiting for the scrub to complete,
  the messages were queued with the higher priority of the
  client ops (osd_client_op_priority. Default: 63);
  and if the scrub was a high-priority one, e.g. initiated by the
  operator, the messages were queued with the highest priority
  (osd_requested_scrub_priority. Default: 120).

That last priority is too high, as scrubs should not have
precedence over client ops. It is even more disruptive if the
cluster is set to have only manually initiated scrubs, and
all scrubs are high-priority ones.

This patch removes the osd_requested_scrub_priority. Instead,
all scrub messages are queued according to the first three
rules above.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

ceph-volume: fix OSD lvm/tpm2 activation

After an OSD is successfully prepared, the activation step fails
because the mapper is left open which makes `systemd-cryptsetup attach`
complain about that and prompt for password.
In order to avoid any other potential issue that would make activation
step hang for ever, I'm adding `headless=true`.

Fixes: https://tracker.ceph.com/issues/68150
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

qa: remove all bluestore signatures on devices

We must ensure BlueStore signatures are removed altogether since
multiple BlueStore label replications have been introduced.

Fixes: https://tracker.ceph.com/issues/68171
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59904 from mcv21/patch-2

Doc: Update "Getting Started" to link to start not install

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc: Update "Getting Started" to link to start not install

Currently the "Getting Started" link points at the install page, not the "Intro to Ceph" section, which seems more appropriate.

If you're happy with this, can it get backported to the still-supported releases, please?

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>

Merge PR #59566 into main

* refs/pull/59566/head:
cephfs-top: fix exception on small sized windows
cephfs-top: fix exception on large sized windows

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

librbd: introduce rbd_group_snap_namespace_type_t enum

Commit e5ccce14c4b0 ("rbd: add group snap info command") added
a commented out field in rbd_group_snap_info2_t struct but didn't
define the corresponding enum, expecting it to be brought in with the
patchset that introduces mirror group snapshots.  That work is still
in progress, but there is already interest in the new command and APIs
that use rbd_group_snap_info2_t struct and even a backport request.

Finalize the struct definition to avoid a breaking change for those
users in the future.  RBD_GROUP_SNAP_NAMESPACE_TYPE_USER can be treated
as dummy in terms of meaning until RBD_GROUP_SNAP_NAMESPACE_TYPE_MIRROR
is added.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #59074 from rhcs-dashboard/NFS-export-form-fixes

mgr/dashboard: NFS Export form fixes

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>