qa: drop creating huge files in test_cephfs_mirror_cancel_sync

Fixes: https://tracker.ceph.com/issues/71701
Fixes: https://tracker.ceph.com/issues/72869
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #65236 from leonidc/delete_all_gws

nvmeofgw: fix issue of delete all gws from the pool/group

Merge pull request #64176 from tchaikov/wip-pybind-cb-types

pybind: fix callback function type compatibility for GCC 14/15

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Nitzan Mordhai <nmordech@ibm.com>

Merge pull request #65934 from tchaikov/cephadm-detect-pip

cephadm/build: Fix _has_python_pip() function check

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #65919 from phlogistonjohn/jjm-bwc-variants

build-with-container: build image variants

Merge pull request #65930 from cloudbehl/filesystem-grafana-fixes

monitoring: Fix Filesystem grafana dashboard units

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65065 from ifed01/wip-ifed-fix-70390

os/bluestore: complete fix for extent spanning over shard boundaries

Merge pull request #65468 from Kushal-deb/kill-daemon

mgr/cephadm: configurable per-service stop timeout

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

cephadm/build: Fix _has_python_pip() function check

The _has_python_pip() function was incorrectly checking for the venv
module instead of pip, causing it to always return the wrong result.
This would prevent proper detection of whether pip is available during
the cephadm build process.

Fix by changing the module check from 'venv' to 'pip'.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #65848 from Hezko/nvmeof-cli-def-max-ns-num-on-subsys-create

mgr/dashboard: change the default max namespace from 4096 to None in subsystem add command

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65791 from aainscow/fix_compiler_warnings

crimson/rgw: Fix clang compiler warnings.

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Edwin Rodriguez <erodriguez@lazyk-ranch.com>

Merge pull request #65829 from rhcs-dashboard/73379-fix-secure-monitoring-stack

mgr/dashboard : Fix secure-monitoring-stack creds issue

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #60361 from rhcs-dashboard/converting-badges-to-carbon-tag

mgr/dashboard: replace bootstrap badges with carbon tags

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

script/build-with-container: add build image variants

Allow the user to control the content of the build image with a
high-level `--image-variant=` switch. Currently the supported values are
`default` (the same maximal image we have been generating) and
`packages` a slimmer image that avoids installing certain test-only
dependencies.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Dockerfile.build: make FOR_MAKE_CHECK a build argument

Set it only during install time.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

install-deps.sh: let FOR_MAKE_CHECK variable take precedence

Previously, the FOR_MAKE_CHECK variable could only enable installing
extra (test) dependencies when install-deps.sh was used and it was
ignored if `tty -s` exited true. This change allows FOR_MAKE_CHECK to
take precedence over the tty check and to specify one of true, 1, yes to
enable extra "for make check" deps or false, 0, no to explicitly disable
the extra deps.

Based-on-work-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #65731 from koch-synyx/doc/dashboard-rsa-note

doc/mgr/dashboard: add note that only RSA keys are supported for TLS

Merge pull request #65531 from ShwetaBhosale1/fix_issue_72906_haproxy_changes_for_nfs_active_active_support

mgr/cephadm: Add stick table and haproxy peers in haproxy.cfg for NFS to support nfs active-active cluster

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #65851 from guits/cv-fix-db-pre-created-lvs

ceph-volume: lvm.Lvm.setup_metadata_devices refactor

Merge pull request #65898 from gbregman/main

mgr/cephadm: Limit the value of max-namespaces-per-subsystem.

Merge pull request #65909 from bluikko/doc-sphinx-warnings-plus-cephfs

doc/cephfs: Fix Sphinx warnings and other minor improvements

Merge pull request #65375 from ifed01/wip-ifed-bluefs-ratio

os/bluestore: report metadata/data ratio in BlueFS perf counter

Reviewed-by: Mark Nelson <mark.a.nelson@gmail.com>

Merge pull request #64682 from ifed01/wip-ifed-fix-zstd

compressor/zstd: compressor correct buffer end detection using get_remaining()

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

monitoring: Fix Filesystem grafana dashboard units

Fixes: https://tracker.ceph.com/issues/73521
Signed-off-by: Ankush Behl <cloudbehl@gmail.com>

Merge branch 'ceph:main' into main

Merge pull request #64009 from rzarzynski/wip-bug-70869

osd: prevent OSDMap::check_health() from asserting due to new OSDs found in subtree

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@ibm.com>

Merge pull request #65910 from anoopcs9/fix-mgr-smb-doc

doc/mgr: Fix a copy-paste error in smb.rst

doc/mgr: Fix a copy-paste error in smb.rst

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>

doc/cephfs: Fix Sphinx warnings and other minor improvements

Fix 2 Sphinx warnings about problematic syntax in troubleshooting.rst:
add empty lines between paragraphs.

In fs-volumes.rst:

Fix 1 Sphinx warning about problematic syntax:
Fix indentation and change to ordered list instead of manually
hardcoding item numbers.

Use privileged bash prompt consistently for privileged commands.

Use the "[...] run a command of the following form:" for CLI examples
with placeholders.

Fix accidental emphasis on a list item by adding empty lines around
sub-list.

Use title case in section titles and trim overly long underline.

Capitalize UID, GID, Unicode, Ceph in text.

Use double backticks consistently for command line switches, data, etc.

Start with capital case and end in full stop if full sentence.

Linkify mention of quiesce dev doc and add a label for it in
doc/dev/mds_internals/quiesce.rst.

Other very minor changes like typo, remove space at end of line.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #65745 from guits/cv-drop-subprocess-call-get-devices

ceph-volume: use udev data instead of LVM subprocess in get_devices()

Merge pull request #65554 from matteopt/fix/udevdata-init-empty-file

ceph-volume: fix UdevData initialisation from empty /run/udev/data/* file

Merge pull request #65893 from thuvh/doc-m1-getopt

docs: add package on macos to run ceph-backport.sh

Merge pull request #65783 from benhanokh/dedup_permissions

rgw/dedup: Grant dedup process full RGW permissions.

mgr/cephadm: Limit the value of max-namespaces-per-subsystem.

Fixes: https://tracker.ceph.com/issues/73505
Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #64908 from Jakesquelch/wip-jakes-default-chunk-size

mon: default chunk size change

Merge pull request #65131 from kamoltat/wip-ksirivad-fix-72647

mgr/progress: compare up set instead of acting set

Merge pull request #65422 from NitzanMordhai/wip-nitzan-destroyed-osds-remove-health-msgs

mgr: remove health msgs from the metrics if osd destroyed

Merge pull request #65305 from rhcs-dashboard/upgrade-carbon-components-pkg

mgr/dashboard: upgrade carbon components package@5.59.2

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65841 from Hezko/nvmeof-cli-ns-list-no-nqn-fix

mgr/dashboard: ns list now support not passing nqn param

Reviewed-by: Afreen Misbah <afreen@ibm.com>

docs: add package on macos to run ceph-backport.sh

Signed-off-by: Hoai-Thu Vuong <thuvh87@gmail.com>

Merge pull request #64709 from dparmar18/fix-qa-fuse-mount-infinite-loop

qa/tasks/cephfs: raise exception if incorrect fstype found

Reviewed-by: Igor Golikov <igolikov@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #64345 from dparmar18/i71740

mds, client, mgr/volumes: toggle snapshot visibility for subvolume based paths

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #62971 from thuvh/feat_rgw_pubsub_push_metric

rgw: add metric when send message with kafka and ampq

Merge PR #65892 into main

* refs/pull/65892/head:
.github/workflows: update qa-symlink to use pull_request_target
script/verify-qa: print commit on run

Merge pull request #65855 from rhcs-dashboard/bump-monitoring-stack

mgr/cephadm: bump monitoring stack versions

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

.github/workflows: update qa-symlink to use pull_request_target

This should allow the script to run even when a PR is submitted by an
external contributor.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/verify-qa: print commit on run

To allow sanity checking what the script is running against.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #65723 from mheler/rgw_cloud_transition_header_fix

rgw/s3: Always include x-amz-content-sha256 header in AWS v4 signatures

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #65839 into main

* refs/pull/65839/head:
pybind/cephfs: fix including of platform_errno.h

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #65525 from ethanwu-syno/fix-snapper-parse-m

osd/SnapMapper: fix SnapMapper::Scrubber shard parsing

Merge pull request #65450 from tchaikov/wip-fix-out-of-bound-access

cpp-btree: fix array bounds warning in btree_node::swap()

Merge pull request #65033 from Nordix/fix-72380-sunnat

mgr/prometheus: Handle empty JSON from orch get-security-config

Merge pull request #65757 from phlogistonjohn/jjm-qa-task-smb

qa/tasks: move smb related task funcs out of cephadm.py to smb.py

Reviewed-by: Adam King <adking@redhat.com>

pybind/cephfs: fix including of platform_errno.h

Introduced-by: 2439bcb87398
Fixes: https://tracker.ceph.com/issues/73435
Signed-off-by: Rishabh Dave <ridave@redhat.com>

doc/mgr/dashboard: add note that only RSA keys are supported for TLS

The dashboard module fails to start when configured with ECDSA/EC
private keys due to pyOpenSSL limitations ("key type unsupported").
Add a note to the SSL/TLS documentation advising users to use
RSA keys until ECDSA is supported.

References: https://tracker.ceph.com/issues/73324

Signed-off-by: Martin Koch <koch@synyx.de>

mgr/dashboard: replace bootstrap badges with carbon tags

Fixes: https://tracker.ceph.com/issues/68252
Signed-off-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #65852 from rhcs-dashboard/73447-fix-usage-bar-for-secondary-cluster

mgr/dashboard : Fixed usage bar for secondary site in rbd mirroing

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #63190 from adamemerson/wip-gcc15-rocksdb-cstdint

build: Modify rocksdb submodule

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

mgr/dashboard : Fix secure-monitoring-stack creds issue
Fixes : https://tracker.ceph.com/issues/73379

Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

Merge pull request #65840 from rhcs-dashboard/fix-permissionsrt

mgr/dashboard: fix dashboard freeze on missing smb permissions

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #65734 from ArbitCode/wip-raja-get-caller-identity-issue

rgw/sts :  get-caller-identity issue fix

Merge pull request #65866 from samarahu/d4n-suite-redis-fix

qa/d4n: Fix redis scanning in workunit

Merge pull request #65857 from rhcs-dashboard/table-title-template

mgr/dashboard: support templates in table component title

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #65558 from rhcs-dashboard/rbd-form-fixes

mgr/dashboard: fix rbd form mirroring toggle

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #65833 from rhcs-dashboard/custom-message-validator

mgr/dashboard: support custom validator messages in inline edit

Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #65865 from jmundack/doc-Remove-unsupposed-OS-from-packages-and-container-hosts-table

doc: Update os-recommendations.rst to remove empty rows

Merge pull request #64587 from SundownRises/NotificationArea-component

mgr/dashboard: Carbonised Notification area component

Reviewed-by: Afreen Misbah <afreen@ibm.com>

qa/d4n: Fix redis scanning in workunit

Signed-off-by: Samarah <samarah.uriarte@ibm.com>

mgr/dashboard: change the default max namespace from 4096 to None in subsystem add command to take gw default

Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

doc: Update os-recommendations.rst to remove empty rows

- Removing rows in tables without any supported versions

Signed-off-by: Joseph Mundackal <jmundackal@bloomberg.net>

mgr/cephadm: bump monitoring stack versions

bumping prometheus, node-exporter, alertmanager, grafana.

Fixes: https://tracker.ceph.com/issues/73467
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: Remove unused css

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge PR #65713 into main

* refs/pull/65713/head:

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #62360 from cloudbehl/node-exporter-dashboard

mgr/dashboard: added node-exporter dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: Fix color issue of notification panel

The global header css of navigation panel was overriding the color of notifications.

Signed-off-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: support templates in table component title

fixes: https://tracker.ceph.com/issues/73456

Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #65787 from Hezko/nvmeof-cli-missing-cmnds2

mgr/dashboard: support gw get_stats and listener info

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard : Fixed usage bar for secondary site in rbd mirroing
fixes : https://tracker.ceph.com/issues/73447
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

ceph-volume: lvm.Lvm.setup_metadata_devices refactor

This commit refactors setup_metadata_devices into smaller helper methods.
It keeps the distinction between existing logical volumes and raw devices
explicit, centralizes tag handling and path assignment to make the
control flow obvious and separates responsibilities for checking, creating,
and tagging devices.

Fixes: https://tracker.ceph.com/issues/73445
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

mgr/dashboard: support custom validator messages in inline edit

we can add custom messages to the customTemplate like
```

        name: $localize`Name`,
        prop: 'name',
        flexGrow: 2,
        cellTemplate: this.removingStatTpl,
        cellTransformation: CellTemplate.editing,
        customTemplateConfig: {
          validators: [Validators.required, Validators.pattern(/^[A-Za-z ]+$/)],
          errorMessages: {
            required:  $localize`This field is required.`,
            pattern: $localize`The field format is invalid.`
          }
        }
      },
```

also extend support for async validator

Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/cephadm: configurable per-service stop timeout

Introduce a termination_grace_period field in service spec to define how long the
orchestrator should wait for a service to shut down gracefully before forcefully terminating it.
The value is plumbed mgr -> cephadm and written into 'unit.stop' as 'podman stop -t <N>'

Signed-off-by: Kushal Deb <Kushal.Deb@ibm.com>

mgr/dashboard: added node-exporter dashboard

Fixes: https://tracker.ceph.com/issues/70534
Signed-off-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #65704 from rhcs-dashboard/rename-sidenav-items

mgr/dashboard: Rename side-nav panel items

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Puja Shahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari talwekar <dtalweka@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #65743 from edwinzrodriguez/ceph-clang-format-remainder

Update misc file tab settings, based on the discussions in this PR
and https://github.com/ceph/ceph/pull/65049

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #65742 from edwinzrodriguez/ceph-clang-format-cc

Update cc file tab settings, based on the discussions in this PR
and https://github.com/ceph/ceph/pull/65049

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #65741 from edwinzrodriguez/ceph-clang-format-h

Update header file tab settings, based on the discussions in this PR
and https://github.com/ceph/ceph/pull/65049

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #65835 from phlogistonjohn/jjm-bwc-naming-cleanup

build-with-container: naming cleanup

script/build-with-container: improve error handling for invalid distros

Instead of throwing a long obnoxious traceback at the user if the value
supplied to -d/--distro is invalid do something nicer. For example:
```
$ ./src/script/build-with-container.py -d trixy  -e build
usage: build-with-container.py [-h] [--help-build-steps]
build-with-container.py: error: argument --distro/-d: unknown distro: 'trixy' not in centos10, centos10stream, centos8, centos9, centos9stream, rocky9, rockylinux9, rocky10, rockylinux10, fedora41, fc41, fedora42, fc42, fedora43, fc43, ubuntu20.04, ubuntu-focal, focal, ubuntu22.04, ubuntu-jammy, jammy, ubuntu24.04, ubuntu-noble, noble, debian12, debian-bookworm, bookworm, debian13, debian-trixie, trixie

```

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: add debian 13 (trixie)

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: add ubuntu 20.04 (focal)

Add ubuntu 20.04 (focal) to the available list of distro kinds.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: add a pair of fedora distro versions

Add fedora 42 and the soon-to-be-released fedora 43.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: lightly organize the distro kind aliases

Do a tiny reorg of the distro kind aliases and container images to keep
the EL distros together and comment out each "section".

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: be consistent with naming in distro kinds

Update the DistroKind enum and related items so that the naming is
applied consistently. That is: the canonical (no pun indented) form
of the name is "<name><version>" and codenames, such as "jammy" or
"bookworm" are aliases. This matches the previously existing code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/dashboard: ns list now support not passing nqn param

Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/dashboard: fix dashboard freeze on missing smb permissions

Fixes: https://tracker.ceph.com/issues/73436
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

mgr/cephadm: Add stick table and haproxy peers in haproxy.cfg for NFS to support nfs active-active cluster

Fixes: https://tracker.ceph.com/issues/72906
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>

mgr/cephadm: add tcp mode for ingress over RGW

In order to allow TLS for the full path client -> haproxy
-> RGW, give the option for the ingress service to just
serve in tcp mode and pass received data directly
to the backend servers. Makes no difference for the
ingress over nfs case that already used tcp mode.

Signed-off-by: Adam King <adking@redhat.com>

mgr/cephadm: change balance in haproxy cfg to roundrobin in tcp mode

This was recommended to better support large deployments
that are trying to send lots of data per client and want
multiple backend gateways serving each client

Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #65797 from ivancich/wip-bump-arrow

rgw: use later versions of Apache Arrow at utf8proc submodules

Reviewed-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #65644 from ShwetaBhosale1/fix_issue_73035_nfs_ganesha_tls_support

mgr/cephadm: Cephadm support for NFS-Ganesha TLS configuration

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>