test/crimson/seastore/test_seastore: add clone removal test

Signed-off-by: Samuel Just <sjust@redhat.com>

test/crimson/.../test_object_data_handler: add multiple clone/overwrite test case

Signed-off-by: Samuel Just <sjust@redhat.com>

test/crimson/.../test_object_data_handler.cc: add support for clones

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: add logging to remap_mappings

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: add formatter for remap_entry_t

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../object_data_handler: fix LOG_PREFIX for do_clone

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert remap_mappings to use cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert reserve_region to use cursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: simplify _update_mapping_ret, remove update_mapping_ret_bare_t

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert _update_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: remove update_refcount, simplify _update_mapping return value

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert remap_mappings to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert remap_mappings to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert clone_mapping to use cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert clone_pin to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert clone_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_end_mapping to return LBACursorRef

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_end_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: remove LBAMapping update_mapping variant

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: remove get_mapping interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_pins to use get_cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_extents_if_live and helpers to use cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_extents_if_live to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert alloc_extents_ret to use LBACursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert alloc_data_extents to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert alloc_extents to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert rewrite_logical_extent to use get_cursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: switch alloc_extent_ret to LBACursorRef

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert reserve_region to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert reserve_region to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert alloc_extent to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert rewrite_logical_extent to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_pin to use get_cursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: remove remove_mappings

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: rework _remove in terms of LBACursor interfaces

Removes the need for _remove_direct_mapping and _remove_indirect_mapping.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: avoid using LBAManager::ref_update_result_t

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: replace LBAManager::complete_indirect_lba_mapping with TransactionManager::complete_mapping

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: simplify get_extent_if_linked

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert _remove_indirect_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert _remove_direct_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../remove: convert to use cursor interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert remove to use cursor interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_types: add LBACursor::is_direct

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert _remove_indirect_mapping_only to use new interfaces

Also removes now unused LBAMapping::remove_indirect_mapping_only.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: remove unused get_ref_update_result

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: replace unused incref_extent with update_mapping_refcount

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: remove unused _incref_extent

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert remove_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: switch update_mapping to take a cursor, add adapter

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert update_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: introduce resolve_cursor_to_mapping

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: add get_cursor interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_mappings to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_cursors to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../fixed_kv_btree: add get_btree helper

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert mkfs to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert remove to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert read_extent to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_pin to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #67639 from rhcs-dashboard/empty-state-bugfix

mgr/dashboard: Use illustration image for empty state table

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #66254 from afreen23/remove-e2e-yaml

qa: Remove cephadm e2e tests from teuthology

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Laura Flores <lflores@ibm.com>

Merge pull request #67589 from xxhdx1985126/wip-seastore-background-trans-cc-opt-new

crimson/os/seastore/cache: TRIM_DIRTY/CLEANER_* transactions won't invalidate other transactions anymore

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #67257 from tchaikov/wip-doc-mgr-cli

doc: update mgr module command documentation for per-module registries

Reviewed-by: John Mulligan <jmulligan@redhat.com>

crimson/os/seastore/transaction: should consider non-aligned remapped
extents when updating paddrs for TRIM_DIRTY/CLEANER transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/lba: TRIM/CLEANER trans to adjust deltas of
LBALeafNodes when committing them.

This is to deal with the following scenario:
1. A client transaction modifies the value of the LBALeafNode, but not
   the pladdr but other field;
2. A TRIM/CLEANER transaction modifies the pladdr for the same laddr_t
   concurrently

In the old approach, the client trans may override the pladdr with the
outdated value after the TRIM/CLEANER transaction commits

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: correct the exception condition when merging
rewritten fixed kv nodes

Fixes: https://tracker.ceph.com/issues/74798
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/btree: make updates of lba leaf nodes ptrs
synchronous with contents updates

Since we need merge content of lba leaf nodes when committing
trim/cleaner transactions, and we rely on the child ptrs to determine
whether to modify mappings of pending leaf nodes. We must make sure
the ptr updates and node content updates are synchronous.

See LBALeafNode::merge_content_to() for detail

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/extent_pinboard: reset 2q_state when removing
extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/transaction_manager: block client transactions if
they conflict with rewriting transactions until the rewriting
transactions finishes

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cached_extent: treat extents under rewrite io as
stable too

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: disable linked tree node operations when committing
rewriting transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: rewrite transactions don't invalidate other
transactions anymore

Fixes: https://tracker.ceph.com/issues/73070
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: drop unused last_commit

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: since extent committer will also set
CachedExtent::prior_poffset, remove invalid asserts

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: unlink mutated extents from the stable
extents' transaction views when committing or invalidating the
transaction

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: add facilities to synchronize data and states
between rewriting trasactions and others when committing

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/seastore_types: define rewriting transactions

These are the transactions that only rewrite extents and mutation lba
nodes, e.g. TRIM_DIRTY and CLEANER transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/async_cleaner: renew backref cursors when they are
generated by backref retrieval transactions and used by reclaim
transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/lba_manager: make sure alloc_extents return viewable
mappings

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/async_cleaner: avoid its header dependence on
backref_manager.h

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/btree_types: BtreeCursors don't hold local copies of
lba/backref values

Since lba mapping values might change during the executions of
client transactions once we allow background transactions to be
submitted without invalidating client ones, we want to avoid other
components using lba/backref mappings from keep local copies to prevent
petential problem

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/lba_mapping: don't allow classes above
TransactionManager to retrieve lba mappings' paddrs.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #67457 from Matan-B/wip-matanb-reactor-type

crimson/admin/osd_admin: introduce reactor_backend command

Reviewed-by: Mohit Agrawal <moagrawa@redhat.com>

Merge pull request #67689 from kotreshhr/cephfs-mirror-remove-extra-wait

tools/cephfs_mirror: Remove additional wait in pop_dataq_entry

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65858 from sajibreadd/wip-71167-scrub-improvement

mds: scrub pins more inodes than the mds_cache_memory_limit

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

doc: update mgr module command documentation for per-module registries

Update documentation to reflect the new per-module command registry
pattern introduced in PR #66467. The old global CLICommand decorators
have been replaced with module-specific registries.

Changes:
- doc/mgr/modules.rst: Rewrite CLICommand section with setup guide,
  update all examples to use AntigravityCLICommand pattern
- src/pybind/mgr/object_format.py: Add note explaining per-module
  registries and update all decorator examples
- doc/dev/developer_guide/dash-devel.rst: Update dashboard plugin
  examples to use DBCLICommand

All examples now correctly show:
- Creating registry with CLICommandBase.make_registry_subtype()
- Using module-specific decorator names (e.g., @StatusCLICommand.Read)
- Setting CLICommand class attribute for framework registration

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

crimson/admin/osd_admin: introduce reactor_backend command

follow-up to: https://github.com/ceph/ceph/pull/67165

Blocked by: https://github.com/scylladb/seastar/pull/3266

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #65423 from benhanokh/split_head_simple

rgw/dedup: split-head mechanism

Merge pull request #67541 from afreen23/resiliency-card

mgr/dashboard: Add resiliency card

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #67435 from avanthakkar/qos-clusterwide

mgr/smb: QoS bandwidth pass-through and burst_mult parameter

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #66181 from anshuman-agarwala/ppc64-ci

[run-make] Added flag for Dashboard and WError

Merge PR #67630 into main

* refs/pull/67630/head:
.github: limit what CI checks run for only doc/qa changes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #67682 into main

* refs/pull/67682/head:
qa: remove ceph-deploy configs with no effect
qa: remove long retired ceph-deploy

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

.github: limit what CI checks run for only doc/qa changes

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

Merge pull request #67115 from ShreeJejurikar/wip-74491

rgw: Add bucket logging pytest test suite

Merge pull request #65513 from gardran/wip-gardran-plogpg-optimize

osd/PrimaryLogPG: avoid redundant container clones and lookups

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #66368 from adamemerson/wip-neorados-leak

neorados: Fix Neorados CephContext leak and prevent future ones

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

rgw/dedup split-head
Simplified check for shared-tail-objects.
Added test for copy after dedup
Use tail-ioctx when removing newly created tail-head

Signed-off-by: benhanokh <gbenhano@redhat.com>

rgw/dedup split-head
Limit Split-Head to RGW-Objects without existing tail-objects (i.e.
obj_size <= 4MB)

Signed-off-by: benhanokh <gbenhano@redhat.com>

rgw/dedup: split-head mechanism
Split head object into 2 objects - one with attributes and no data and
a new tail-object with only data.
The new-tail object will be deduped (unlike the head objects which can't
be dedup)
We will split head for objects with size 16MB or less

A few extra improvemnts:
Skip objects created by server-side-copy
Use reftag for comp-swap instead of manifest
Skip shared-manifest objects after readint attributes
Made max_obj_size_for_split and min_obj_size_for_dedup config value in
rgw.yaml.in

refined test: validate size after dedup
TBD: add rados ls -l to report object size on-bulk to speedup the process
improved tests - verify refcount are working, validate objects, remove
duplicates and then verify the last remaining object making sure it was
not deleted

Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>