Merge pull request #64009 from rzarzynski/wip-bug-70869

osd: prevent OSDMap::check_health() from asserting due to new OSDs found in subtree

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@ibm.com>

Merge pull request #65910 from anoopcs9/fix-mgr-smb-doc

doc/mgr: Fix a copy-paste error in smb.rst

doc/mgr: Fix a copy-paste error in smb.rst

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>

Merge pull request #65745 from guits/cv-drop-subprocess-call-get-devices

ceph-volume: use udev data instead of LVM subprocess in get_devices()

Merge pull request #65554 from matteopt/fix/udevdata-init-empty-file

ceph-volume: fix UdevData initialisation from empty /run/udev/data/* file

Merge pull request #65893 from thuvh/doc-m1-getopt

docs: add package on macos to run ceph-backport.sh

Merge pull request #65783 from benhanokh/dedup_permissions

rgw/dedup: Grant dedup process full RGW permissions.

Merge pull request #64908 from Jakesquelch/wip-jakes-default-chunk-size

mon: default chunk size change

Merge pull request #65131 from kamoltat/wip-ksirivad-fix-72647

mgr/progress: compare up set instead of acting set

Merge pull request #65422 from NitzanMordhai/wip-nitzan-destroyed-osds-remove-health-msgs

mgr: remove health msgs from the metrics if osd destroyed

Merge pull request #65305 from rhcs-dashboard/upgrade-carbon-components-pkg

mgr/dashboard: upgrade carbon components package@5.59.2

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65841 from Hezko/nvmeof-cli-ns-list-no-nqn-fix

mgr/dashboard: ns list now support not passing nqn param

Reviewed-by: Afreen Misbah <afreen@ibm.com>

docs: add package on macos to run ceph-backport.sh

Signed-off-by: Hoai-Thu Vuong <thuvh87@gmail.com>

Merge pull request #64709 from dparmar18/fix-qa-fuse-mount-infinite-loop

qa/tasks/cephfs: raise exception if incorrect fstype found

Reviewed-by: Igor Golikov <igolikov@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #64345 from dparmar18/i71740

mds, client, mgr/volumes: toggle snapshot visibility for subvolume based paths

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #62971 from thuvh/feat_rgw_pubsub_push_metric

rgw: add metric when send message with kafka and ampq

Merge PR #65892 into main

* refs/pull/65892/head:
.github/workflows: update qa-symlink to use pull_request_target
script/verify-qa: print commit on run

Merge pull request #65855 from rhcs-dashboard/bump-monitoring-stack

mgr/cephadm: bump monitoring stack versions

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

.github/workflows: update qa-symlink to use pull_request_target

This should allow the script to run even when a PR is submitted by an
external contributor.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/verify-qa: print commit on run

To allow sanity checking what the script is running against.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #65723 from mheler/rgw_cloud_transition_header_fix

rgw/s3: Always include x-amz-content-sha256 header in AWS v4 signatures

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #65839 into main

* refs/pull/65839/head:
pybind/cephfs: fix including of platform_errno.h

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #65525 from ethanwu-syno/fix-snapper-parse-m

osd/SnapMapper: fix SnapMapper::Scrubber shard parsing

Merge pull request #65450 from tchaikov/wip-fix-out-of-bound-access

cpp-btree: fix array bounds warning in btree_node::swap()

Merge pull request #65033 from Nordix/fix-72380-sunnat

mgr/prometheus: Handle empty JSON from orch get-security-config

Merge pull request #65757 from phlogistonjohn/jjm-qa-task-smb

qa/tasks: move smb related task funcs out of cephadm.py to smb.py

Reviewed-by: Adam King <adking@redhat.com>

pybind/cephfs: fix including of platform_errno.h

Introduced-by: 2439bcb87398
Fixes: https://tracker.ceph.com/issues/73435
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #65852 from rhcs-dashboard/73447-fix-usage-bar-for-secondary-cluster

mgr/dashboard : Fixed usage bar for secondary site in rbd mirroing

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #63190 from adamemerson/wip-gcc15-rocksdb-cstdint

build: Modify rocksdb submodule

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #65840 from rhcs-dashboard/fix-permissionsrt

mgr/dashboard: fix dashboard freeze on missing smb permissions

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #65734 from ArbitCode/wip-raja-get-caller-identity-issue

rgw/sts :  get-caller-identity issue fix

Merge pull request #65866 from samarahu/d4n-suite-redis-fix

qa/d4n: Fix redis scanning in workunit

Merge pull request #65857 from rhcs-dashboard/table-title-template

mgr/dashboard: support templates in table component title

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #65558 from rhcs-dashboard/rbd-form-fixes

mgr/dashboard: fix rbd form mirroring toggle

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #65833 from rhcs-dashboard/custom-message-validator

mgr/dashboard: support custom validator messages in inline edit

Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #65865 from jmundack/doc-Remove-unsupposed-OS-from-packages-and-container-hosts-table

doc: Update os-recommendations.rst to remove empty rows

Merge pull request #64587 from SundownRises/NotificationArea-component

mgr/dashboard: Carbonised Notification area component

Reviewed-by: Afreen Misbah <afreen@ibm.com>

qa/d4n: Fix redis scanning in workunit

Signed-off-by: Samarah <samarah.uriarte@ibm.com>

doc: Update os-recommendations.rst to remove empty rows

- Removing rows in tables without any supported versions

Signed-off-by: Joseph Mundackal <jmundackal@bloomberg.net>

mgr/cephadm: bump monitoring stack versions

bumping prometheus, node-exporter, alertmanager, grafana.

Fixes: https://tracker.ceph.com/issues/73467
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: Remove unused css

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge PR #65713 into main

* refs/pull/65713/head:

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #62360 from cloudbehl/node-exporter-dashboard

mgr/dashboard: added node-exporter dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: Fix color issue of notification panel

The global header css of navigation panel was overriding the color of notifications.

Signed-off-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: support templates in table component title

fixes: https://tracker.ceph.com/issues/73456

Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #65787 from Hezko/nvmeof-cli-missing-cmnds2

mgr/dashboard: support gw get_stats and listener info

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard : Fixed usage bar for secondary site in rbd mirroing
fixes : https://tracker.ceph.com/issues/73447
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

mgr/dashboard: support custom validator messages in inline edit

we can add custom messages to the customTemplate like
```

        name: $localize`Name`,
        prop: 'name',
        flexGrow: 2,
        cellTemplate: this.removingStatTpl,
        cellTransformation: CellTemplate.editing,
        customTemplateConfig: {
          validators: [Validators.required, Validators.pattern(/^[A-Za-z ]+$/)],
          errorMessages: {
            required:  $localize`This field is required.`,
            pattern: $localize`The field format is invalid.`
          }
        }
      },
```

also extend support for async validator

Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: added node-exporter dashboard

Fixes: https://tracker.ceph.com/issues/70534
Signed-off-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #65704 from rhcs-dashboard/rename-sidenav-items

mgr/dashboard: Rename side-nav panel items

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Puja Shahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari talwekar <dtalweka@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #65743 from edwinzrodriguez/ceph-clang-format-remainder

Update misc file tab settings, based on the discussions in this PR
and https://github.com/ceph/ceph/pull/65049

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #65742 from edwinzrodriguez/ceph-clang-format-cc

Update cc file tab settings, based on the discussions in this PR
and https://github.com/ceph/ceph/pull/65049

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #65741 from edwinzrodriguez/ceph-clang-format-h

Update header file tab settings, based on the discussions in this PR
and https://github.com/ceph/ceph/pull/65049

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #65835 from phlogistonjohn/jjm-bwc-naming-cleanup

build-with-container: naming cleanup

script/build-with-container: improve error handling for invalid distros

Instead of throwing a long obnoxious traceback at the user if the value
supplied to -d/--distro is invalid do something nicer. For example:
```
$ ./src/script/build-with-container.py -d trixy  -e build
usage: build-with-container.py [-h] [--help-build-steps]
build-with-container.py: error: argument --distro/-d: unknown distro: 'trixy' not in centos10, centos10stream, centos8, centos9, centos9stream, rocky9, rockylinux9, rocky10, rockylinux10, fedora41, fc41, fedora42, fc42, fedora43, fc43, ubuntu20.04, ubuntu-focal, focal, ubuntu22.04, ubuntu-jammy, jammy, ubuntu24.04, ubuntu-noble, noble, debian12, debian-bookworm, bookworm, debian13, debian-trixie, trixie

```

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: add debian 13 (trixie)

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: add ubuntu 20.04 (focal)

Add ubuntu 20.04 (focal) to the available list of distro kinds.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: add a pair of fedora distro versions

Add fedora 42 and the soon-to-be-released fedora 43.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: lightly organize the distro kind aliases

Do a tiny reorg of the distro kind aliases and container images to keep
the EL distros together and comment out each "section".

Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: be consistent with naming in distro kinds

Update the DistroKind enum and related items so that the naming is
applied consistently. That is: the canonical (no pun indented) form
of the name is "<name><version>" and codenames, such as "jammy" or
"bookworm" are aliases. This matches the previously existing code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/dashboard: ns list now support not passing nqn param

Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/dashboard: fix dashboard freeze on missing smb permissions

Fixes: https://tracker.ceph.com/issues/73436
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

Merge pull request #65797 from ivancich/wip-bump-arrow

rgw: use later versions of Apache Arrow at utf8proc submodules

Reviewed-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #65644 from ShwetaBhosale1/fix_issue_73035_nfs_ganesha_tls_support

mgr/cephadm: Cephadm support for NFS-Ganesha TLS configuration

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #65804 from dmick/add-bookworm-to-bwc

src/script: add bookworm to build scripts

rgw/dedup: Grant dedup process full RGW permissions.
This is necessary to allow for the creation of intermediate SLAB objects on systems configured with Ceph authentication.

Remove bogus assert

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2401399
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>

rgw: add metric when send message with kafka and ampq

- l_rgw_pubsub_push_pending
- l_rgw_pubsub_push_failed

Fixes: https://tracker.ceph.com/issues/70256
Signed-off-by: Hoai-Thu Vuong <thuvh87@gmail.com>

mgr/dashboard: support gw get_stats and listener info

Update nvmeof/gateway submodule to have the relevant protobuf objects and calls.

Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>

mgr/dashboard: Rename side-nav panel items

Fixes: https://tracker.ceph.com/issues/73252
Commit includes changes:
1) Renaming Topic to Notification destination
2) Renaming Tiering to Storage class
3) Renaming Users to User Management
4) fix storage class table refresh after delete
5) Also made changes to internal routing for topic and storage class

Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #65482 from rhcs-dashboard/table-refresh-pagination

mgr/dashboard: server side table rendering improvements

Reviewed-by: Ankush Behl <ankush.behl@ibm.com>
Reviewed-by: Pedro Gonzalez <pedro.gonzalez@redhat.com>

Merge pull request #65809 from bluikko/doc-scrub-admonition-typo-cephfs

doc/cephfs: Typos and small improvements in scrub.rst

Merge pull request #65811 from bluikko/doc-cache-config-label-warnings-cephfs

doc/cephfs: Add missing labels in cache-configuration.rst

Merge pull request #65135 from rishabh-d-dave/libcephfs-platform-errno

libcephfs: convert ceph errno to host-based errno

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #65699 from phlogistonjohn/jjm-py-cmake-labels

pybind/mgr: add default set of cmake/ctest labels to tests

Merge pull request #65819 from ceph/wip-yuriw-release-doc

doc/dev: Fix typo in release-process.rst

docs: Fixed typo

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

src/test/fs: add more --copy-dt-needed-entries

See comment in src/test/libcephfs/CMakeLists.txt.  I don't
know why bookworm tweaks this issue so that it occurs on two
more test binaries, but this fix works around the link error
in the previously-discovered way

Signed-off-by: Dan Mick <dan.mick@redhat.com>

make-debs.sh: use ID instead of NAME for workdir

NAME is "for the user", and as such, Debian's is "Debian GNU/Linux",
which isn't friendly for making a pathname.  ID is more like what
we want (lowercase, no spaces, limited special characters),
in the two cases we care most about, 'ubuntu' and 'debian'.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

src/script: add bookworm to build-with-container.py

..and its friend buildcontainer-setup.sh

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #65803 from zmc/sccache-flex

Dockerfile.build: Make sccache repo configurable

rgw: update cmake files to refer to system Apach Arrow version 9

When built using WITH_SYSTEM_ARROW refer to version 9.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #63169 from andreasstieger/fuse3

ceph.spec.in: build with fuse 3

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #65703 from connorfawcett/pool-migration-design-doc

docs/dev: Add design document with information on proposed design for pool migration

Merge pull request #63917 from rishabh-d-dave/vols-purge-trash-entry

pybind/cephfs, mgr/volumes: introduce non-recurisve rmtree(), refactor purge() to use it and add MDS optimizations

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/cephfs: Add missing labels in cache-configuration.rst

Fix two Sphinx warnings about :ref: links in troubleshooting.rst to
undefined labels.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/cephfs: Typos and small improvements in scrub.rst

Don't use space in "subdirectory" or "kick-start".

Change command syntax introduction text to the common "To ..., run a
command of the following form:" when the syntax has placeholders.

Use double backticks consistently (same as sections in the beginning)
for CLI commands, parameters etc.

Linkify "cluster health warnings" to the CephFS health warnings doc.

Use admonition and fix a typo in article.

Use title case for section titles and trim overly long underline.

Minor improvements to language.

Don't use unordered lists in two cases when there's no real need,
especially when there is only a single item in the latter case.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #65676 from Hezko/nvmeof-cli-ns-add-size-params2

mgr/dashboard: raise exception if both size and rbd_image_size are being passed in ns add

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #64874 from bluikko/doc-manual-freebsd-deployment-improvements-install

doc/install: Small improvements in manual-freebsd-deployment.rst

Dockerfile.build: Make sccache repo configurable

CEPH-BUILD-BRANCH: sccache-stats
DISTROS: centos9 noble
ARCHS: x86_64
FLAVORS: default
CI-CONTAINER: false
Signed-off-by: Zack Cerza <zack@cerza.org>

rgw: bump up versions of Apache Arrow and utf8proc submodules

Arrow goes up to 17.0.0 and utf8proc to 2.11.0.

arrow submodule
utf8proc submodule

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #64744 from bluikko/doc-services-formatting-cephadm

doc/cephadm/services: Improve formatting and small fixes in index.rst

Merge pull request #64729 from bluikko/doc-ceph-dokan-warnings-plus-cephfs

doc: Fix Sphinx warnings&errors and use labels in 4 files for links

doc: Fix a Sphinx warning&error and use labels for links in 4 files

Fix Sphinx warning about missing empty line after a label in
cephfs/ceph-dokan.rst.

Fix Sphinx error about invalid indentation in releases/squid.rst.

Fix Sphinx warning about invalid bash prompt block in
dev/crimson/crimson.rst.
Also fix warnings about too short section title text underline and fix
an incorrect link name (and use "above" instead of "below").

Fix Sphinx warning about missing strong emphasis closure, put the
problematic star inside inline preformatted/code in dev/blkin.rst.

Fix Sphinx warning about explicit markup, add the required empty line in
cephfs/mds-config-ref.rst.

Fix Sphinx error about invalid table in radosgw/s3/bucketops.rst.

Use the label at the beginning of the document in cephfs/ceph-dokan.rst
for hyperlinking with :ref: instead of using "external links" feature
from cephfs/createfs.rst cephfs/mount-prerequisites.rst
install/windows-install.rst.

Also use existing label in cephfs/mount-using-kernel-driver.rst
similarly from cephfs/createfs.rst cephfs/mount-prerequisites.rst.

Also use existing label in man/8/cephfs-shell.rst similarly from
cephfs/index.rst.

Add a label in cephfs/mount-using-fuse.rst and use it from
cephfs/createfs.rst cephfs/mount-prerequisites.rst.

Remove space at the end of line in cephfs/createfs.rst
cephfs/mount-prerequisites.rst cephfs/mount-using-fuse.rst
cephfs/cephfs-fuse.rst cephfs/index.rst.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

docs/dev: Add design document with information on proposed design for pool migration

Signed-off-by: Connor Fawcett <connorfa@uk.ibm.com>

ceph-dencoder: add `SnapRealmInfoNew` in ceph-dencoder types

Signed-off-by: Venky Shankar <vshankar@redhat.com>

snap_types: fix encode/decode for the newly added fiels in SnapRealmInfoNew

Signed-off-by: Venky Shankar <vshankar@redhat.com>

test/libcephfs: add test cases for Client::mksnap and Client::rmsnap

they exist as libceohfs APIs but aren't being called by FUSE/pybind clients,
make sure they work as intended.

Fixes: https://tracker.ceph.com/issues/71740
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

mgr/volumes: disable client_respect_subvolume_snapshot_visibility in CephfsConnectionPool::connect

To prevent any type of config change (via mons, args, envvar, tell command) on the ceph-mgr from breaking the volume plugin.

Fixes: https://tracker.ceph.com/issues/71740
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

doc/cephfs/fs-volumes: add documentation for controlling snapshot visibility

for subvolume based paths

Fixes: https://tracker.ceph.com/issues/71740
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

qa/tasks/cephfs: add test cases for subvolume config snapshot_visibility

Fixes: https://tracker.ceph.com/issues/71740
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>