test/rgw/kafka: fix kafka relase to more recent one

Fixes: https://tracker.ceph.com/issues/75323
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

Merge pull request #66383 from bluikko/src-mgmt-gateway-spec-descs-python-common

python-common: Improve mgmt-gateway descriptions in service_spec.py

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #66758 from ajarr/wip-stagger-schedules

mgr/rbd_support: Stagger mirror snapshot and trash purge schedules

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #67590 from idryomov/wip-75200

qa/tasks: make rbd_mirror_thrash inherit from ThrasherGreenlet

Reviewed-by: Miki Patel <miki.patel132@gmail.com>

Merge pull request #65911 from kotreshhr/mirror-json-pretty

mgr/mirroring: json pretty formatting

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #67599 from bluikko/wip-doc-start-get-involved-fixes

doc/start: Update and fix get-involved.rst

python-common: Improve mgmt-gateway descriptions in service_spec.py

Fix one description that referred to time while the nginx config doesn't
configure time.

Use consistent language. Remove reference to "multioption" which is a
simple bool.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

mgr/rbd_support: Stagger mirror snapshot and trash purge schedules

Previously, multiple images or namespaces scheduled with the same
interval ran mirror snapshots or trash purges at around the same time,
creating spikes in cluster activity.

This change staggers scheduled jobs by:

- Adding a deterministic phase offset per image or namespace when no
  start-time is set.
- Picking a random element from the queue at each scheduled time, rather
  than always the first.

Together, these changes spread snapshot and trash purge operations more
evenly over time and improve cluster stability.

Fixes: https://tracker.ceph.com/issues/74288
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge PR #67563 into main

* refs/pull/67563/head:
script/redmine-upkeep: do not warn for backport PRs
script/redmine-upkeep: do not raise backport warnings for backports

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #67613 from ronen-fr/wip-rf-75270by1

osd/scrub: extract_crc_from_bufferlist: fix off-by-1 error

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>

Merge pull request #67400 from vmackie/victoriam-stretch-cluster

dashboard: add location field to NVMeoF namespace and gateway group APIs

Merge pull request #67301 from rhcs-dashboard/nfs-toggle-snapshot

mgr/dashboard: NFS: Toggle visibility of CephFS snapshots

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/mirroring: json pretty formatting

The 'daemon status' and 'peer_list' command
out don't support json-pretty format and isn't reader
friendly. This patch adds support of 'json-pretty'
when format='json-pretty' is passed.

Fixes: https://tracker.ceph.com/issues/73516
Signed-off-by: Kotresh HR <khiremat@redhat.com>

osd/scrub: extract_crc_from_bufferlist: fix off-by-1 error

Fixes: https://tracker.ceph.com/issues/75270
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #65878 from edwinzrodriguez/ceph-wip-73424

mds: Refactor cache dump path formatting

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #67335 from kotreshhr/fs_mirror_daemon_status

mgr/mirroring: Display mon_host and fsid in daemon status command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #66551 from salieri11/wip-igolikov-new-metrics-74135-73700

mds: add new performance and subvolume utilization metrics

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #67346 from rhcs-dashboard/custom-image-to-bootstrap

mgr/dashboard: support custom image for bootstrapping

Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #67136 from JonBailey1993/stats_fix_part_1

osd: Fix for num_bytes mismatch occurring from snapshot workloads with partial writes in fast_ec

dashboard: add location field to NVMeoF namespace and gateway group APIs

Namespace location:
- Add location field to Namespace model in nvmeof.py
- Add location parameter to PATCH /api/nvmeof/subsystem/{nqn}/namespace/{nsid}
- Location can now be retrieved via GET and set via PATCH

Gateway group locations:
- Add locations array to gateway group endpoint response
- Extract locations from all gateways in a service group
- Add _get_gateway_locations() helper method using nvme-gw show command
- Locations appear in placement.locations for each service

Signed-off-by: Victoria Mackie <victoriam@uk.ibm.com>

Merge pull request #67626 from tchaikov/wip-qa-remove-unused-var

qa/tasks/keycloak: drop unused local variable

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #67604 from amathuria/wip-amat-update-seastar-march-26

seastar: bump up seastar submodule

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #67227 from rhcs-dashboard/isolate-cherrypy

mgr: isolated CherryPy to prevent global state sharing

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

qa/tasks/keycloak: drop unused local variable

Regression introduced by ee710390d277784ddac3d70c9e11e427f46f363d
("rgw/tests: add os-specific java 1.7 install commands to keycloak
task").

```
283/322 Test #301: run-tox-qa ................................***Failed 92.31 sec
...
flake8: install_deps /ceph/qa> python -I -m pip install flake8
  flake8: commands[0] /ceph/qa> flake8 --select=F,E9 --exclude=venv,.tox
  ./tasks/keycloak.py:51:5: F841 local variable 'os_version' is assigned to but never used
```
Remove the unused os_version assignment to fix flake8 F841 in run-tox-qa.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #67291 from rhcs-dashboard/add-smb-share-qos

mgr/dashboard: add SMB share QoS rate limiting

Reviewed-by: Dnyaneshwari talwekar <dtalweka@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #67602 from bluikko/wip-doc-radosgw-adminops-fixes

doc/radosgw: Fix markup in adminops.rst

Merge pull request #67595 from bluikko/wip-doc-remove-markup-from-cli-commands

doc: Remove markup from CLI example commands

Merge pull request #66578 from ceph/dvanders_mds_set_balanced_budget

mds: enable osdc op throttles

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/start: Update and fix get-involved.rst

Remove not existing Planet Ceph, Wiki, Commit List rows.
Update Kernel Client, QA, Community mailing list links to working ones.
Use https instead of http.
Fix Ceph calendar link and split the old contribute guide link to a
separate table row.
Remove not working lists.ceph.com external link definition now that it
is unused.
Sort external link definitions in order of use.
Fix invalid space after a hyphen by rewrapping text.
Update Slack invite link.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67444 from rhcs-dashboard/fix-osd-form-alert

mgr/dashboard: Misleading warning when no eligible devices are available for OSD creation

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #66799 from aainscow/missing-list-corruption-fix

osd: Change rmissing map key from version_t to eversion_t

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #67578 from ivancich/wip-install-java-17

rgw/tests: add os-specific java 1.7 install commands to keycloak task

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #67523 from shraddhaag/wip-shraddhaag-improve-crimson-docs

doc: improve crimson docs

Merge PR #67608 into main

* refs/pull/67608/head:
Revert "Merge pull request #67167 from Hezko/nvme-module-two"

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #66111 from fa-at-pulsit/wip-73702-rgw-keystone-scope

rgw: Add Keystone scope information to ops logging

Merge pull request #67126 from gardran/wip-gardran-fix-mclock-options

osd: handle all the updated keys in OSD::maybe_override_options_for_qos

Reviewed-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>

Merge pull request #67414 from rhcs-dashboard/add-config-params-cephadm-certmgr

mgr/dashboard : Add cephadm config params to dashboard

Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

doc: reformat crimson docs

This commit rearranges crimson docs so the deployment steps are inorder
to how they are supposed to be executed. Also, it removed `crimson-osd`
referneces as that is an internal detail that users don't need to be
aware of.

Revert "Merge pull request #67167 from Hezko/nvme-module-two"

This PR broke the ceph-mgr [1] and was not QA'd more broadly.

[1] https://github.com/ceph/ceph/pull/67167#issuecomment-3984274659

This reverts commit 884bbcfd38d88b08213c3106dc89cd6c956d4b95, reversing
changes made to 472e267f03705145ee306d15f3d566ec4ac3f726.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

mgr/dashboard: add SMB share QoS rate limiting

Fixes: https://tracker.ceph.com/issues/74856
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

mgr/dashboard: Misleading warning when no eligible devices are available for OSD creation

Fixes: https://tracker.ceph.com/issues/75050
Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge PR #67476 into main

* refs/pull/67476/head:

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #67554 from rkachach/fix_splitting_services_ut

mgr/cephadm: splitting services big UT file into smaller per-service files

Reviewed-by: Adam King <adking@redhat.com>

crimson/os/seastore: fix experimental_list_directory iterator dereference

See: https://github.com/scylladb/seastar/commit/0046c83f652945f0060bf0db9383481623889cf8

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

crimson/common: fix metric label dumping

Seastar's escaped_string wrapper no longer implicitly converts to a string type.
Update dump_metric_value to call .value() to access the underlying string.

See: https://github.com/scylladb/seastar/commit/49c34b81bcae5bec3bf7f12bb91bdfd859326de0

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

cmake: explicitly support and enable vptr sanitizer

As of Clang 21, -fsanitize=undefined no longer implies vptr.
This adds vptr as an explicit component and provides the necessary compile options to satisfy Seastar's build requirements.
See: https://github.com/scylladb/seastar/commit/c1060ea7d4676df23ce62af96ef2daa768f5de8a for more details

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

seastar: bump up seastar submodule

Update branch to wip-amat-seastar-march26
https://github.com/ceph/seastar/tree/wip-amat-seastar-march26

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

Merge pull request #66832 from nbalacha/wip-nbalacha-lua-74219

Wip rgw/lua: performance improvements

doc/radosgw: Fix markup in adminops.rst

Add blank lines so that markup is not rendered verbatim.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67480 from afreen23/subsystem-integration

mgr/dashboard: Allow adding authentication to subsystem flow

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #67158 from sujay-d07/automated-kafka-security-testing

qa: Extend Kafka SASL Security Tests

Merge pull request #67569 from gbregman/main

mgr/dashboard: Add DHCHAP controller key to NVME host commands

doc: Remove markup from CLI example commands

Remove all the remaining stars in the preformatted block CLI example
commands.
Several example CLI commands had stars around placeholder text in
addition to the standard angle brackets. The stars could make sense in
man pages or in text to make the placeholder italic but it will just be
rendered as-is in preformatted text blocks.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67591 from bluikko/wip-doc-sphix-warnings-202603

doc: Fix Sphinx warnings

mgr/dashboard: Fix auth logic in subsystem and remove traffic encryption columns

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65877 from edwinzrodriguez/ceph-wip-73421

mds: Avoid use of moved 'fs' in FSMap

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65879 from edwinzrodriguez/ceph-wip-73425

mds: MDSCap Use default impl for copy and assign op

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65883 from edwinzrodriguez/ceph-wip-73429

mds: Refactor Mantle::balance to use std::string

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #66288 from chrisphoffman/wip-73825

client: Only use known fields when comparing policies

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc: Fix Sphinx warnings

Add missing blank lines in two files to fix Sphinx warnings.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67517 from rhcs-dashboard/backup-storage-consumption

mgr/dashboard: add storage consumption card

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: NFS: Toggle visibility of CephFS snapshots

Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
Fixes: https://tracker.ceph.com/issues/74875

Merge pull request #67276 from rhcs-dashboard/74636-service-modal-with-certmgr

mgr/dashboard : Update create/edit service modal to support certmg

Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>

mgr/dashboard: add storage consumption card

Fixes: https://tracker.ceph.com/issues/75181
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

qa/tasks: make rbd_mirror_thrash inherit from ThrasherGreenlet

Commit 21b4b89e5280 ("qa/tasks: watchdog terminate thrasher") made it
required for a thrasher to have stop_and_join() method, but the
preceding commit a035b5a22fb8 ("thrashers: standardize stop and join
method names") missed to add it to rbd_mirror_thrash (whether as an
ad-hoc implementation or by way of inheriting from ThrasherGreenlet).
Later on, commit 783f0e3a9903 ("qa: Adding a new class for the
daemonwatchdog to monitor") worsened the issue by expanding the use
of stop_and_join() to all watchdog barks rather than just the case of
a thrasher throwing an exception which is something that practically
never happens.

Fixes: https://tracker.ceph.com/issues/75200
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rgw/tests: add os-specific java 1.7 install commands to keycloak task

Add commands to keycloak task specific for rocky, rhel, centos, and
ubuntu. Also, clean-up installed package(s) after test is run.

This is necessary as rocky can't install the same package(s) that the
other os types currently can.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: Add DHCHAP controller key to NVME host commands
Fixes: https://tracker.ceph.com/issues/74939
Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

mgr/dashboard: Allow adding authentication to subsystem flow

- integrates third step with overall tearsheet

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67535 from rhcs-dashboard/final-merged-branch

mgr/dashboard: Final merged branch for subsystem complete flow

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #64718 from VallariAg/wip-nvmeof-top-tool

mgr/dashboard: Add nvmeof_top_cli service

Merge pull request #67345 from rhcs-dashboard/nfs-pyobj-serialze-err

mgr/dashboard: nfs export creation fails with obj deserialization

Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

script/redmine-upkeep: do not warn for backport PRs

Do not warn that a backport PR is claiming to fix a non-backport tracker
ticket which is not linked to the backport PR.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: do not raise backport warnings for backports

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #67095 from cbodley/wip-74591

qa/radosgw_admin: replace boto2 with boto3

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #66879 from Hezko/nvme-cli-responses

mgr/dashboard: introduce NvmeofCLICommand's success_message_template …

Merge pull request #67167 from Hezko/nvme-module-two

mgr/nvmeof: add nvmeof module which will create .nvmeof rbd pool

Merge pull request #67524 from idryomov/wip-75161

librbd: don't complete ImageUpdateWatchers::shut_down() prematurely

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #67482 from idryomov/wip-68279

librbd/mirror: detect trashed snapshots in UnlinkPeerRequest

Reviewed-by: Ramana Raja <rraja@redhat.com>

mgr/cephadm: spliting test_services UT into per-service UT files

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: removed big test_services file

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #66262 from ceph/updates_for_lf_csc_charter

CONTRIBUTING: link to Governance doc

Merge pull request #67486 from cbodley/wip-qa-rgw-multisite-boto2-decoder

qa/rgw/multisite: remove boto2 BotoJSONEncoder

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #67169 from kchheda3/wip-fix-notification-queue-full

rgw/notification: Prevent reserved_size leak by decrementing overhead on commit/abort.

 reviewed-by: yuvalif

Merge pull request #67083 from smanjara/wip-fix-multisite-sigv4

rgw/multisite: fix forwarded requests using awsv4 with UNSIGNED-PAYLOAD

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Remove tabs under subsystem
Fixes: https://tracker.ceph.com/issues/74904
Signed-off-by: pujaoshahu <pshahu@redhat.com>

rgw: Add Keystone scope information to ops logging

Fixes: https://tracker.ceph.com/issues/73702
On-behalf-of: SAP <filipp.akinfiev@clyso.com>
Signed-off-by: Filipp Akinfiev <filipp.akinfiev@clyso.com>

src/pybind/mgr: Add nvmeof-top tool

Add src/pybind/mgr/dashboard/services/nvmeof_top_cli.py
which adds commands:
"ceph nvmeof top io <subsystem>"
"ceph nvmeof top cpu"

This file is moved from the original work in:
https://github.com/pcuzner/ceph-nvmeof-top

Fixes: https://tracker.ceph.com/issues/64578
Co-authored-by: Paul Cuzner <pcuzner@ibm.com>
Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>

mgr/dashboard: introduce NvmeofCLICommand's success_message_template and success_message_map parameters to allow meaningful success messages

Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/dashboard: add-reviewstep-in-subsystem

Fixes: https://tracker.ceph.com/issues/75085
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>>

mgr/dashboard: NVme- Add namespace in subsystem resource page

Fixes: https://tracker.ceph.com/issues/74338
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

mgr/dashboard: NVmeof Add listeners in subsytem and resource page.

Fixes: https://tracker.ceph.com/issues/74339
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

mgr/dashboard: Nvmeof add hosts(Add Initiator) in subsystem details

Fixes: https://tracker.ceph.com/issues/75006
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #67472 from rhcs-dashboard/removetab-subsystem

mgr/dashboard: Remove tabs under subsystem

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67503 from afreen23/data-resiliency

mgr/dashboard: Fixes for overview page

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>

mgr/nvmeof: add nvmeof module introduction to pending release notes

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/nvmeof: add unittests

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

Merge pull request #67123 from kshtsk/wip-qa-tasks-keystone-rocky

qa/tasks/keystone: restart mariadb for rocky and alma linux too

Merge pull request #67088 from rzarzynski/wip-crimson-ctwrn-unnused

crimson/os: fix multiple compile-time warnings on unused variables

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

mgr/nvmeof: use nvmeof module during orch nvmeof apply

Added a call to create_pool_if_not_exists during the execution of ceph orch apply nvmeof command.

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/nvmeof: set nvmeof module to be enabled by default

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>