Merge pull request #61397 from amathuria/wip-amat-test-osdmap-pruning

mon/test_mon_osdmap_prune: Use first_pinned instead of first_committed

Merge pull request #61365 from Matan-B/wip-matanb-snapmapper-logs

osd/SnapMapper: Improve logging

Merge pull request #61328 from adamemerson/wip-64191

test/neorados: Silence mismatched new/delete warning

Merge pull request #60945 from NitzanMordhai/wip-nitzan-crushwrapper-corpus-squid

dencoder tests fix type backwards incompatible checks

Merge pull request #61577 from ronen-fr/wip-rf-just-me

osd/scrub: remove unnecessary loop

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61538 from leonidc/fix-duplicated-optimized

nvmeofgw* : fix duplicated optimized host's pathes

Merge pull request #61590 from ronen-fr/wip-rf-noinfo-repair

osd/scrub: discard repair_oinfo_oid()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #61394 from ronen-fr/wip-rf-cacher-v2

common: modify md_config_obs_impl API

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #60426 from ronen-fr/wip-rf-svwperf

common/perf_counters: enabling 'find()' by logger name

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61613 from zdover23/wip-doc-2025-02-02-architecture

doc/architecture: remove sentence

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/architecture: remove sentence

Remove a sentence that is more marketing than reference.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61561 from athanatos/sjust/wip-crimson-recovery-69412

crimson: take obc lock during push commit on primary

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61001 from MaxKellermann/common_includes

common: add missing includes

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61598 from idryomov/wip-rbd-migration-https-doc

doc/rbd: use https links in live import examples

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

crimson/.../replicated_recovery_backend: take excl lock while pushes commit

Fixes: https://tracker.ceph.com/issues/69412
Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: route pushes earlier

Let ReplicatedRecoveryBackend::handle_recovery_op route pushes
between handle_push and handle_pull_response instead of
ReplicatedRecoveryBackend::handle_push.

Signed-off-by: Samuel Just <sjust@redhat.com>

doc/rbd: use https links in live import examples

Even though it's explicitly said that "http" stream can be used to
import via both HTTP and HTTPS, it can still be confusing that "type":
"http" is expected to go with "url": "https://...".  Switch example
URLs from HTTP to HTTPS to make it more obvious.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #57551 from linuxbox2/wip-rgwlc-66111

rgwlc: send pool transition notifications too

Merge pull request #60250 from aainscow/interval_set_enhancements

include: interval_set: Relax requirements and enhance performance of interval sets

Merge pull request #61135 from rkachach/fix_issue_cephadm_services_registry

mgr/cephadm: using service registry pattern for cephadm services

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #59480 from bill-scales/ec_partial_read

Further EC partial stripe read fixes

Merge pull request #61591 from gbregman/main

mgr/cephadm/nvmeof: Add verify_listener_ip field to NVMeOF configuration

Merge PR #61537 into main

* refs/pull/61537/head:
libcephfs_proxy: implement ceph_readdir_r()

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>

mgr/cephadm/nvmeof: Add verify_listener_ip field to NVMeOF configuration and remove obsolete enable_key_encryption
Fixes https://tracker.ceph.com/issues/69731

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #61540 from idryomov/wip-69679

mon/OSDMonitor: relax cap enforcement for unmanaged snapshots

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd/scrub: discard repair_oinfo_oid()

repair_oinfo_oid(), called every scrub, has a very specific
functionality: fix the object ID specified in the Object Info
attribute, if different from the ID of the owning object.

This fix was added in 2017, as a response to a unique failure
scenario that was observed in Sepia - probably following a
filesystem bug. See https://tracker.ceph.com/issues/18409 &
https://tracker.ceph.com/issues/20471.

The limited functionality of repair_oinfo_oid() -
only repairing this one specific issue, and only if the OI_ATTR
exists and is decodable - does not justify the overhead of
running it every scrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: remove unnecessary loop

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #61471 from idryomov/wip-65889-more

cls/rbd: don't use read API for write-like methods

Reviewed-by: Ramana Raja <rraja@redhat.com>

common: ceph_context: make use of get_tracked_keys()

modify some configuration object registrations
in common/ceph_context to use the updated
md_config_obs_t::get_tracked_keys() API

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #61545 from Hezko/nvmeof-cli-add-cmnds

mgr/dashboard: Add additional API and CLI endpoints

Merge pull request #61465 from ArbitCode/wip-raja-fix-multipart-upload-cant-get-obj-tag

RGW:fix obj by multipart upload cant get tag

Merge pull request #61510 from dmick/container-no-repo-creds

container/build.sh: don't require repo creds on NO_PUSH

Merge pull request #61566 from zdover23/wip-doc-2025-01-30-cephadm-services-osd

doc/cephadm: simplify confusing math proposition

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #61530 from aza547/ssl_cert

cephadm: rgw: allow specifying the ssl_certificate by filepath

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61511 from phlogistonjohn/jjm-ctr-label-ceph

container: add label ceph=True back

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61565 from AliMasarweh/wip-alimasa-bn-policy-with-tenant

RGW | bucket notifications: support cross tenant operations

Reviewed-by: yuvalif<ylifshit@redhat.com>

Merge pull request #61559 from linuxbox2/wip-new-zppbits

rgw: update to latest zpp_bits.h to compile w/gcc-14 & clang 19

Merge pull request #60246 from jamiepryde/SIMD-align-64

erasure-code: Increase SIMD_ALIGN from 32 to 64

Merge pull request #59679 from ceph/add-new-ec-plugins-for-qa

qa/erasure-code: add new teuthology isa configs

doc/cephadm: simplify confusing math proposition

s/This means that the exact device size is 3.64 * 1000, or 3640GB"/This
means that the exact device size is 3.64TB, or 3640 GB"/

In the original text, the number "3.64" appears to refer to a quantity
(and indeed, it is a quantity of Terabytes), but it is unlabeled. Also,
on repeated recent readings of this sentence I found it more puzzling
than enlightening. So I made this commit.

Signed-off-by: Zac Dover <zac.dover@proton.me>

RGW | bucket notifications: support cross tenant operations

Signed-off-by: Ali Masarwa <amasarwa@redhat.com>

Merge pull request #61123 from rhcs-dashboard/raise-smb-msg-exception

mgr/dashboard: smb raise exception for unsucessful resource update

Reviewed-by: Afreen Misbah <afreen@ibm.com>

libcephfs_proxy: implement ceph_readdir_r()

Signed-off-by: Xavi Hernandez <xhernandez@gmail.com>

Merge pull request #61317 from rhcs-dashboard/add-smb-service-msg

mgr/dashboard: add warning message on smb service management

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61033 from rhcs-dashboard/rgw-user-accounts-ui

mgr/dashboard: RGW user accounts UI

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #60819 from rhcs-dashboard/smb-cluster-form

mgr/dashboard: create smb cluster

Reviewed-by: Afreen Misbah <afreen@ibm.com>

crimson/.../replicate_recovery_backend: remove unnecessary check

Already checked in handle_recovery_op.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../object_context_loader: add lock_excl_sync method

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../object_context_loader: add obc get_obc_manager variant

Avoids extra lookup.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: move do_transaction to _handle_pull_response

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: only call _committed_pushed_object if complete

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: convert handle_pull_response to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: convert _handle_pull_response to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: convert submit_push_data to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: convert prep_push_target to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../recovery_backend: convert to logging macros, some formatting changes

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../replicated_recovery_backend: convert to logging macros, some formatting changes

Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: update to latest zpp_bits.h to compile w/gcc-14 & clang 19

Fixes: https://tracker.ceph.com/issues/69696
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

nvmeofgw*: 2 fixes - for duplicated optimized  pathes and fix for GW startup
 1. fix duplicated optimized host's pathes - trigger process_gw_down upon
   fast-gw reboot, removed old fast-reboot handlers
 2. fix GW startup - trigger process_gw_down when expired WAIT_BLOCKLIST timer

Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>

Merge pull request #61549 from zdover23/2025-01-28-radosgw-multisite

doc/radosgw: s/zonegroup/pools/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #60735 from adk3798/cephadm-remove-daemon-service-name

mgr/cephadm: set service name for DaemonDescription object used during daemon removal

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61215 from ArbitCode/wip-rgw-raja-feature-64526

RGW:support x-amz-expected-bucket-owner to verify bucket ownership wi…

mgr/dashboard: smb raise exception for unsucessful resource update

Adds a decorator to raise a DashboardException with the msg error of an unsucessful smb resource update

Fixes: https://tracker.ceph.com/issues/69286
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

doc/radosgw: s/zonegroup/pools/

s/zonegroup/pools/, where this change makes the text clearer.

This change was made in reponse to an upstream comment on
https://pad.ceph.com/p/Report_Documentation_Bugs.

Fixes: https://tracker.ceph.com/issues/69689
Signed-off-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: Add additional cli endpoints to align with existing nvmeof cli

Added new endpoints to ceph cli and dashboard API to align with cli commands that already exist in existing nvmeof cli.

fixes: https://tracker.ceph.com/issues/62705

Signed-off-by: Tomer Haskalovitch <il033030@Tomers-MBP.lan>

Merge pull request #61551 from leonidc/level_of_critical_mon_ev

nvmeofgw*: change log level of critical nvmeof monitor events to 1

mgr/cephadm: using service registry pattern for cephadm services

This change includes mainly the following enhancements:
- Introduced a centralized `CephadmServiceRegistry` to manage service registration and initialization.
- Added dynamic discovery of service modules in the same directory using `pkgutil` and `importlib`.
- Implemented a decorator `@service_registry_decorator` for automatic registration of service classes.

Fixes: https://tracker.ceph.com/issues/69021
Signed-off-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #61449 from mohit84/issue_69405

neorados_pool_test: Call create_pool at the end of NeoRados delete_pool related test

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

osd/OSDCap: fix misleading grammar comments

The restrictions on pool name and namespace have been independent of
each other for ages.  Specifying namespace[=]<namespace> doesn't require
specifying pool[=]<pool> like is currently suggested -- neither for
regular "allow" grants nor for "profile" grants.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mon/OSDMonitor: relax cap enforcement for unmanaged snapshots

Since commit 4972e054b32c ("mon/OSDMonitor: enforce caps when
creating/deleting unmanaged snapshots"), a) write access to the MON
service, b) write access to the OSD service for a pool or c) permission
for "osd pool op unmanaged-snap" command for a pool is required.  For
"profile rbd" we configure read-only access to the MON service and rely
on write access to the OSD service, however the corresponding check in
is_osd_writable() is too strict.

A OSD cap like "profile rbd namespace=myns" or "allow w namespace=myns"
allows write access to myns namespace of any pool, but is_osd_writable()
disallows operations with unmanaged snapshots with such a cap because
its match.pool_namespace.pool_name.empty() is true.  This condition
appears to serve as the "doesn't include support for the application
tag" guard, but it should actually be match.pool_tag.is_match_all()
(or match.pool_tag.application.empty() if open-coded) -- no restriction
on the pool name doesn't automatically mean that there is a restriction
on the application tag.

Fixes: https://tracker.ceph.com/issues/69679
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: create smb cluster

Fixes: https://tracker.ceph.com/issues/69156
Signed-off-by: Dnyaneshwari Talwekar <dtalwekar@redhat.com>

nvmeofgw*: change log level of critical nvmeof monitor events to 1

Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>

Allow passing ssl_certificate as an rgw_frontend_extra_arg.

Fixes: https://tracker.ceph.com/issues/69567
Signed-off-by: Alexander Hussein-Kershaw <alexhus@microsoft.com>
fix markup

Merge pull request #61392 from Hezko/nvmeof-gw-info-cli

Dashboard: Introduce nvmeof cli commands

Merge PR #60996 into main

* refs/pull/60996/head:
mds/SimpleLock: add is_xlocked_by()
mds/SimpleLock: add has_xlock_by()

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #61250 into main

* refs/pull/61250/head:
mds: avoid acquiring the wrlock twice for a single request
mds: add 'mds_allow_async_dirops' opt to allow/disable async dirop

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #61327 into main

* refs/pull/61327/head:
client: Fix signed comparison warning

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

interval_set: Improve comments

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

Merge pull request #60935 from rhcs-dashboard/smb-cluster-delete

mgr/dashboard: delete smb cluster

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

interval_set: Fix test_interval_set.cc

Here we duplicate all the original tests which used "insert" to verify they also pass with "union_insert". All new tests are modified to use "union_of" and "union_insert"

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: Add back insert()

A reviewer (see github) was concerned that the policing provided by insert() may have been required for some applications of insert_set. As such, I have re-instated the old insert method and instead refactored "union_insert". IU have also enhanced the comments.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: Review comments

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_map: Review comments

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: tests to cover interval set changes

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: align method to extend all intervals to be aligned with a specified alignment

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: erase and subtract enhancements

The old erase would only allow intervals which exist to be erased. It is often useful to erase any interval, even if it does not exist or partially overlaps one or many intervals.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: Enhancements to improve efficiency of insert() and to allow more flexibility.

The old insert was restrictive in ranges that could be added in. The new interface allows for a range to be added, whether it extends or joins other intervals.

Also change a number of interfaces to use the new insert.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_set: contains() returns true if a specified interval is entirely contained within the set.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_map: tests.  Tests to cover new function in interval map

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_map: contains() to determine if range is entirely contained within range.

Determine, for a particular interval iterator, whether the specified range is entirely contained inside the interval.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

interval_map: Add interfaces for getting start/end and lower ranges.

 These new utilities add the ability to:
   get_lower_range: Return the lowest interval iterator which covers the specified range
   get_start_off(): return the first offset in the interval.
   get_end_off(): Return the end offset of the last interval
   contains(): Return true if specified is entirely contained within the interval map.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

Merge pull request #60993 from MaxKellermann/types__backward_backward

include/types.h, tools/cephfs: remove _BACKWARD_BACKWARD_WARNING_H

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: Introduce nvmeof cli

Extends ceph cli with nvmeof commands

fixes: https://tracker.ceph.com/issues/62705

Signed-off-by: Tomer Haskalovitch <il033030@Tomers-MBP.lan>

mgr/dashboard: delete smb cluster
Fixes: https://tracker.ceph.com/issues/69191
       https://tracker.ceph.com/issues/69605

Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #57895 from rhcs-dashboard/kcli-multi-cluster

mgr/dashboard: adapt kcli plan to improve multi-cluster deployment

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #61382 from nbalacha/dashboard-typo

mgr/dashboard: Typo in NVMe-oF create service

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #61525 from zdover23/wip-doc-2025-01-26-cephadm-host-management

doc/cephadm: correct note

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #61506 from ronen-fr/wip-rf-scrub-formatters

osd/scrub: use ObjectSection & ArraySection
Reviewed-by: Samuel Just <sjust@redhat.com>

doc/cephadm: correct note

Correct the English in a note in doc/cephadm/host-management.rst.

This commit was made in response to
https://github.com/ceph/ceph/pull/50340, which corrected some of this
note in Quincy.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61520 from idryomov/wip-osdcap-syntax-doc

doc/rados: pool and namespace are independent osdcap restrictions

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>