crimson/os/seastore: prevent incorrect space calculation at maximum capacity

This commit updates seastore to increment used_bytes only when
 allocation is successful.
Without this commit, the seatore reports incorrect size if alloc_paddr fails
(e.g., due to insufficient space)

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

Merge pull request #63292 from tchaikov/test_not_before_queue-fix-leaks

test/test_not_before_queue: Fix memory leak by freeing ineligible items in destructor

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #63369 from zdover23/wip-doc-2025-05-20-mgr-diskprediction-62782-followup

doc/mgr: edit diskpredictor.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #63308 from Hezko/nvmeof-cli-align-subsys-list

mgr/dashboard: Add missing fields to subsytem list API/CLI

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #63048 from JoshuaGabriel/joshua_71135

orch/module: Make orch unpause an alias for orch resume

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #63371 from bluikko/doc-cephfs-mirroring-formatting-cephfs

doc/cephfs: Cosmetic changes and small fixes in cephfs-mirroring.rst

Merge pull request #63379 from ronen-fr/wip-rf-noloadavg

osd/scrub: remove OsdScrub::LoadTracker

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #63368 from zdover23/wip-doc-2025-05-20-mgr-dashboard_plugins-motd-inc-62782-followup

doc/mgr: edit motd.inc.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #62906 from rhcs-dashboard/fix-70276-main

mgr/dashboard: Allow the user to re-use existing realm/zg/zone and setup replication

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>

doc/mgr: edit motd.inc.rst

Edit doc/mgr/dashboard_plugins/motd.inc.rst.

This commit is part of a project to separate out the twenty-five files
that were committed to https://github.com/ceph/ceph/pull/62782.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #63367 from zdover23/wip-doc-2025-05-20-mgr-dashboard_plugins-feature_toggles-inc-62782-followup

doc/mgr: edit feature_toggles.inc.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #63365 from zdover23/wip-doc-2025-05-15-mgr-dashboard_plugins-debug-inc-rst-62782-followup

doc/mgr: edit debug.inc.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge PR #62934 into main

* refs/pull/62934/head:
libcephfs / pybind: use uid_t and gid_t instead of int

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #63331 from Matan-B/wip-matanb-crimson-tentacle-nightlies

qa/crontab: run Crimson tentacle nightly

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>

doc/cephfs: Cosmetic changes and small fixes in cephfs-mirroring.rst

Don't format an unordered list as preformatted block.

Fix (not linked) reference to a section name that doesn't exist.

Don't call systemd unit files "systemctl unit files", add an "and"
between them.

Clarify by means of inline formatting and whitespace that config value
is "60" and the unit for that is seconds (as opposed to value being
"60s").

Use double backticks consistently when listing commands, file names or
paths, command parameters, syntaxes, etc.

Use star for inline italic instead of single backtick (to differentiate
from the previous).

Don't use double backticks for "cephadm" when talking about the concept.

Don't capitalize a few words that didn't need it.
Capitalize table data for consistency.
All caps for "MDS".

Small updates to language in text.

Call it "cephfs-mirror" everywhere and not "cephfs_mirror" at one place.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

mgr/dashboard: Allow the user to re-use existing r
ealm/zg/zone and setup replication

1. Currently, we just allow the user to create a new realm/zg/zone and setup replication using the multi-site replication wizard. The ask is to allow the user to select the pre-existing realm/zg/zone and setup replication via automatic export and import of token as well.

2. Enable rgw module automatically in the selected cluster if its not
   enabled

Fixes: https://tracker.ceph.com/issues/70276
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #62071 from zhscn/wip-lba-mapping-refactor

crimson/os/seastore: introduce BtreeCursor to simplify the implementation of LBAMapping/BackrefMapping

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #62684 from mchangir/mds-use-already-avaialble-CInode-in-uninline-path

mds: use available CInode* for uninline data

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #63099 from phlogistonjohn/jjm-dash-node-chown

dashboard: ensure nodeenv downloaded content is owned by current user

Merge pull request #62507 from NVShawn/main

[Packaging]: Use system-defined location for home directories

Merge pull request #63370 from zdover23/wip-doc-2025-05-20-mgr-hello-62782-followup

doc/mgr: edit hello.rst

Merge pull request #63181 from shraddhaag/wip-shraddhaag-availability-docs

docs: add release notes and docs for availability score feature

Merge pull request #62946 from MaxKellermann/rgw_cksum__unreachable

rgw/rgw_cksum: work around -Wsometimes-uninitialized

Reviewed-by: Casey Bodley <cbodley@redhat.com>

osd: move load avg units conversion to the client

The OSD calls OsdScrub::update_load_average() to find out the load
average, and notes it down in a performance counter. The system
load average is multipled by 100 (to improve precision). That
multiplication should be on the side of the client, not the
scrub queue service.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: remove OsdScrub::LoadTracker

As we no longer maintain a 'daily average', and as the interaction
between the load tracker and the scrub scheduler is now much simplified,
we can remove the load tracker entirely.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

doc: address review comments

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

crimson/os/seastore/FixedKVBtree: correct MAX_DEPTH

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/FixedKVBtree: remove with_btree_ret helper function

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/lba: flatten lba namespace to keep consistent with backref

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BtreeLBAManager: support refresh LBAMapping

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore: simplify the processing of indirect LBAMapping

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore: fix LBAMapping/BackrefMapping usages

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BackrefManager: adpat new BackrefMapping

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BtreeLBAManager: cleanup the impl of alloc mapping and remap

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BtreeLBAManager: refactor update_mapping_ret_bare_t

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BtreeLBAManager: refactor with LBACursorRef

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BtreeLBAManager: split _alloc_extents implementation

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/BtreeLBAManager: cleanup alloc_mapping_info_t

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore: rename type remap_entry to remap_entry_t

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore: refactor LBAMapping and BackrefMapping

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/FixedKVBtree: return BtreeCursor instead of BtreeMapping

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/btree: introduce BtreeCursor

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/btree: move lba/backref_map_val_t to the common header file

rename btree_range_pin.h to btree_types.h

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore: remove child_pos in LBAMapping

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: remove PhysicalNodeMapping and BtreeNodeMapping

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore: move BtreeNodeMapping::is_viewable_by_trans to CachedExtent

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/btree: remove the unused template argument of op_context_t

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

crimson/os/seastore/FixedKVBtree: use min_max_t<>::min to build begin iterator

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

doc/mgr: edit hello.rst

Edit doc/mgr/hello.rst. This includes the rewriting of sentences so that
they now appear in grammatically-correct English.

This is part of a project to separate out the twenty-five files that
were committed to https://github.com/ceph/ceph/pull/62782.

Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/mgr: edit diskpredictor.rst

Edit doc/mgr/diskpredictor.rst. This includes the rewriting of sentences
so that they now appear in grammatically-correct English.

This is part of a project to separate out the twenty-five files that
were committed to https://github.com/ceph/ceph/pull/62782.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #63219 from xxhdx1985126/wip-seastore-omap-balance-pivot

crimson/os/seastore/linked_tree_node: correct balance pivots calculation for both fixed-kv tree and omap tree

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

doc/mgr: edit feature_toggles.inc.rst

Edit doc/mgr/dashboard_plugins/features_toggles.inc.rst.

This commit is part of a project to separate out the twenty-five files
that were committed to https://github.com/ceph/ceph/pull/62782.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #62631 into main

* refs/pull/62631/head:
cephfs: session tracker accounts for killing sessions

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

doc/mgr: edit debug.inc.rst

Edit doc/mgr/dashboard_plugins/debug.inc.rst.

This commit is part of a project to separate out the twenty-five files
that were committed to https://github.com/ceph/ceph/pull/63255.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #62632 into main

* refs/pull/62632/head:
libcephfs: increment library minor version
test: add test to fetch perf counters via libcephfs API
libcephfs: add API to get client perf counters
client: fix total write operations perf counter name

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #62698 into main

* refs/pull/62698/head:
mds: add some perf counters about cache trimming

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #63151 from xxhdx1985126/wip-rgw-location-https

rgw/rgw_rest: determine the domain uri prefix by rgw_transport_is_secure

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62510 from badone/wip-tracker-70645-mon_memory_target-fix

OSDMonitor: Make sure pcm is initialised

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #63266 from cbodley/wip-71291

rgw/zone: remove duplicated startup logic in RGWSI_Zone

Reviewed-by: Ali Masarwa <amasarwa@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #63357 from zdover23/wip-doc-2025-05-20-revert-merge-of-63351

doc: Revert "Merge pull request #63351 from anthonyeleven/document_HTTP_X_…

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: Revert "Merge pull request #63351 from anthonyeleven/document_HTTP_X_AMZ_DELETE_IF_UNMODIFIED_SINCE"

This reverts commit 6a054b06a6e60a9ded51b8be5357232b4177edb1, reversing
changes made to ff1d7c9e4efa195939b3d46a570f0d0242d54b97.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #63351 from anthonyeleven/document_HTTP_X_AMZ_DELETE_IF_UNMODIFIED_SINCE

WIP DNM doc/radosgw/s3: Document delete-if-unmodified-since

Reviewed-by: Zac Dover <zac.dover@proton.me>

qa/crontab: run Crimson nightly

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

doc/radosgw/s3: Document delete-if-unmodified-since

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #63325 from ronen-fr/wip-rf-load2

osd/scrub: remove the 2'nd option for determining 'low load' for scrub

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #63319 from rhcs-dashboard/accept-dot-in-uid

mgr/dashboard: accept dot(.) in user_id & restrict tenant validation in user form

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge PR #63281 into main

* refs/pull/63281/head:
Revert "mgr/volumes: handle bad arguments during subvolume create"

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #63289 from tchaikov/ceph-dedup-cleanups

tools/ceph_dedup: improve const-ness and use ranged-for loop when appropriate

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

crimson/os/seastore/btree: remove unused methods

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/omap_manager: limit the max size of kv inserted to
the omap tree

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: adjust and fix rebalance logic in lba/backref/omap tree

* For fixed_kv_btree, drop `prefer_left`.
* For omap btree, tolerate underflow nodes if rebalance has the same results.

Fixes: https://tracker.ceph.com/issues/71307
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #63294 from bluikko/doc-architecture-ref

doc: Use existing labels and ref for hyperlinks in architecture.rst

Merge pull request #63330 from bluikko/doc-fix-bluestore-man-warning

doc/man: Fix inline formatting in ceph-bluestore-tool.rst

Merge pull request #63329 from bluikko/doc-fix-blank-line-warnings

doc: Fix missing blank line Sphinx warnings

Merge pull request #63300 from tchaikov/assert-fix-leaks

common: fix backtrace leak in __ceph_abort and friends

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #60963 from k0ste/mgr_DaemonServer_mistype

mgr/DaemonServer: fixed mistype for mgr_osd_messages

Merge pull request #63166 from Matan-B/wip-matanb-crimson-seastore-recovery

qa/suites/crimson-rados: Seastore recovery tests

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

doc: Fix missing blank line Sphinx warnings

Fix four warnings from Sphinx about missing blank line after explicit
markup.

Indent content in list items correctly, fixing formatting errors.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/man: Fix inline formatting in ceph-bluestore-tool.rst

A space is missing between a token with emphasis and the following
token:
- Not consistent with other commands like "show-label" (has space).
- Inline formatting is rendered verbatim in the second occurrence, without
  the formatting being applied.
- Warning from Sphinx.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc: Use existing labels and ref for hyperlinks in architecture.rst

Use validated ":ref:" hyperlinks instead of "external links" in "target
definitions" when linking within the Ceph docs:
- Update to use existing labels when linkin from architecture.rst.
- Remove unused "target definitions".

Also use title case for section titles in
doc/start/hardware-recommendations.rst because change to use link text
generated from section title.

Other than generated link texts the rendered PR should look the same as
the old docs, only differing in the source RST.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

mgr/dashboard: add missing fields to subsytem list command

Signed-off-by: Tomer Haskalovitch <il033030@Tomers-MBP.lan>
add a test

osd/scrub: minimize calls to sysconf() in scrub_load_below_threshold()

Return an 'all is OK' value if the 1min CPU load - even before being
divided by the number of CPUs - is below the configured threshold.
This is a very common case, and avoids the need to call sysconf()
to get the number of CPUs.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: remove the 2'nd option for determining 'low load' for scrubbing

Previously, there were two conditions under which the CPU load was
considered
low enough to allow scrubbing:
- the CPU load was below the configured threshold, or
- the load was below a calculated "daily" average, and lower than the
  15-min average.
That second condition was confusing and surprising, and is now removed.

As the scrubber logic no longer requires the 5m & 15m load averages,
scrub_load_below_threshold() can use the data gathered by the
periodic LoadTracker::update_load_average().

Fixes: https://tracker.ceph.com/issues/71347
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #63318 from bluikko/doc-fix-crush-map-ref-rados

doc/rados/operations: Fix invalid hyperlink in crush-map-edits.rst

Merge pull request #63320 from bluikko/doc-fix-health-checks-formatting-rados

doc/rados/operations: Fix unordered list in health-checks.rst

Merge pull request #63321 from bluikko/doc-fix-cloud-transition-formatting-radosgw

doc/radosgw: Fix indentation in cloud-transition.rst

Merge pull request #62186 from linuxbox2/wip-rgw-cb409

rgw: introduce rgw_bucket_eexist_override

Merge pull request #63317 from bluikko/doc-fix-mclock-conf-ref-rados

doc/rados/configuration: Fix invalid hyperlinks in mclock-config-ref.rst

Merge pull request #63309 from JoshuaGabriel/joshua/cephfs-docs

doc/cephfs: specify mount path in example command

tools/ceph_dedup: Add const qualifiers and reference parameters

Improve code quality in ceph_dedup tool by:

- Adding const qualifiers to member functions and parameters where appropriate
- Converting parameter passing to use references instead of value copies
  for complex objects

These changes enhance code readability, better express intent through
const-correctness, and improve performance by avoiding unnecessary deep
copies.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

tools/ceph_dedup: Replace iterator loops with range-based loops

Refactor the code to use modern C++ range-based for loops instead of
traditional iterator-based loops, improving code readability

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

mgr/dashboard: accept dot(.) in user_id & (alphanumeric chars, _) in tenant in user form

fixes: https://tracker.ceph.com/issues/71341

Signed-off-by: Naman Munet <naman.munet@ibm.com>

doc/rados/configuration: Fix invalid hyperlinks in mclock-config-ref.rst

Fix two intradocument links that pointed to the same nonexisting section
title.
These were rendered "as-is" with backticks and everything, while still
linkified pointing to invalid "#idN" anchors.

Modify them to point to the correct section title text.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/radosgw: Fix indentation in cloud-transition.rst

Indent the second paragraph of a list item at the same level as the
previous paragraph. The unexpected indentation resulted in an ERROR from
Sphinx but it was still rendered with increased indentation looking
rather out of place.

Capitalize the first letter similarly to the previous paragraph.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/rados/operations: Fix unordered list in health-checks.rst

Fix missing empty line before an unordered list that caused it to be
rendered wrong.

Also, the sentences in each list item start with a capital case letter
so add the missing full stop at the end.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/rados/operations: Fix invalid hyperlink in crush-map-edits.rst

Fix attempted use of underscores for inline emphasis which resulted in
the text being emphasized to be considered a link. The text was rendered
partially as a link to an invalid anchor "#id3".

Instead use inline italic for formatting emphasis.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #63293 from bluikko/doc-multisite-ref-radosgw

doc/radosgw: Use ref for hyperlinking to multisite

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

doc/cephfs: specify mount path in example command

The example command for mounting CephFS was missing the local mount path.

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>

tools/ceph_dedup: add modelines for emacs and vim

add modelines for emacs and vim so that the source code is formatted
with expected alignment even if the editor's default setting is not
match with the expected settings.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #63279 from rhcs-dashboard/fix-promql-query-tests

mgr/dashboard: fix flaky promql query test