mgr/dashboard: Add restore events in notification screen

Fixes: https://tracker.ceph.com/issues/72887
Signed-off-by: pujashahu <pshahu@redhat.com>
Signed-off-by: pujaoshahu <pshahu@redhat.com>

mgr/nvmeof: add nvmeof module introduction to pending release notes

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
(cherry picked from commit 166fb04c1251bc2df6aa68cbd4e303005f8f08e7)

mgr/nvmeof: add unittests

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
(cherry picked from commit eecbff76fa6401edaf2abbee9d86e08162f752eb)

mgr/nvmeof: use nvmeof module during orch nvmeof apply

Added a call to create_pool_if_not_exists during the execution of ceph orch apply nvmeof command.

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
(cherry picked from commit f5734cf41b18add5e54efa13c4519359705dae57)

mgr/nvmeof: set nvmeof module to be enabled by default

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
(cherry picked from commit eccffe57c5a0cf8a762351fe26e6f631108fb849)

mgr/nvmeof: intergrate module into build and debian pkg

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
(cherry picked from commit 901ec98b4146b9e2f2d2b4ab257a2d1a5b903d9f)

mgr/nvmeof: introduce the new nvmeof module

Introduce a new NVMe-oF mgr module and which create the pool
used for storing NVMe-related metadata ceph orch nvmeof apply command.
This removes the need for users to manually create and configure the
metadata pool before using the NVMe-oF functionality, simplifying
setup and reducing the chance of misconfiguration.

Fixes: https://tracker.ceph.com/issues/74702
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
(cherry picked from commit 15fcbb5e3eac2153c51d16b96e32d86038eb0569)

rgw/lua: create fresh VM for each background script execution

Previously, the background thread reused the same Lua VM across
iterations, causing stale state to persist. This made operations
like 'pairs(RGW)' fail to iterate properly.

Now we create a fresh VM on each iteration, which:
- Fixes the iteration bug
- Simplifies the code (no need to update limits on existing VM)
- Ensures clean state for each script execution

Verified with unit tests:
- TableIterateBackground
- TableIterateBackgroundBreak
- TableIterateStepByStep

Fixes: https://tracker.ceph.com/issues/74839
Signed-off-by: Rotem Shapira <rotem.rs@gmail.com>

test/crimson/seastore/test_seastore: add clone removal test

Signed-off-by: Samuel Just <sjust@redhat.com>

test/crimson/.../test_object_data_handler: add multiple clone/overwrite test case

Signed-off-by: Samuel Just <sjust@redhat.com>

test/crimson/.../test_object_data_handler.cc: add support for clones

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: add logging to remap_mappings

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: add formatter for remap_entry_t

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../object_data_handler: fix LOG_PREFIX for do_clone

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert remap_mappings to use cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert reserve_region to use cursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: simplify _update_mapping_ret, remove update_mapping_ret_bare_t

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert _update_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: remove update_refcount, simplify _update_mapping return value

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert remap_mappings to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert remap_mappings to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert clone_mapping to use cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert clone_pin to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert clone_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_end_mapping to return LBACursorRef

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_end_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: remove LBAMapping update_mapping variant

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: remove get_mapping interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_pins to use get_cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_extents_if_live and helpers to use cursors

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_extents_if_live to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: convert alloc_extents_ret to use LBACursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert alloc_data_extents to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert alloc_extents to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert rewrite_logical_extent to use get_cursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: switch alloc_extent_ret to LBACursorRef

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert reserve_region to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert reserve_region to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert alloc_extent to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert rewrite_logical_extent to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_pin to use get_cursor

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: remove remove_mappings

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: rework _remove in terms of LBACursor interfaces

Removes the need for _remove_direct_mapping and _remove_indirect_mapping.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: avoid using LBAManager::ref_update_result_t

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: replace LBAManager::complete_indirect_lba_mapping with TransactionManager::complete_mapping

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: simplify get_extent_if_linked

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert _remove_indirect_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert _remove_direct_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../remove: convert to use cursor interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert remove to use cursor interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_types: add LBACursor::is_direct

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert _remove_indirect_mapping_only to use new interfaces

Also removes now unused LBAMapping::remove_indirect_mapping_only.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: remove unused get_ref_update_result

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: replace unused incref_extent with update_mapping_refcount

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: remove unused _incref_extent

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert remove_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../lba_manager: switch update_mapping to take a cursor, add adapter

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert update_mapping to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: introduce resolve_cursor_to_mapping

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #66203 from BBoozmen/wip-oozmen-73799

RGW/multisite: fix bucket-full-sync infinite loop caused by stale bucket_list_result reuse

doc/dev/cputrace.rst: Documentation for groups

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #67640 from NitzanMordhai/wip-nitzan-suite-rados-singleton-bluestore-missing-mds

test: rados singleton-bluestore missing mds for cephtool tests

Merge pull request #67649 from NitzanMordhai/wip-nitzan-self-test-influx-set-hostname-plugin-test

qa/tasks/mgr: test_module_selftest set influx hostname to avoid warni…

Merge pull request #67097 from anthonyeleven/more-osd-metadata-metrics

src/pybind/mgr/prometheus: Add five OSD metadata metrics to module.py

mgr/dashboard: mgr/dashboard: Namespace size should accept GiB in NVMe/TCP

Fixes: https://tracker.ceph.com/issues/75433
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #67567 from xxhdx1985126/wip-seastore-objectdatahandler-zero-bug

crimson/os/seastore/object_data_handler: avoid reserving zero-length regions

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #67697 from xxhdx1985126/wip-73820

crimson/osd/pg: drop inappropriate assertions

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #67667 from rhcs-dashboard/consumption-chart-fixes

mgr/dashboard: fix consumption chart units

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #65881 from edwinzrodriguez/ceph-wip-73427

mds: Remove unnecessary std::move in MDSRank

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #66639 from bluikko/wip-doc-radosgw-dedup-fixes

doc/radosgw: Improve language, formatting in s3_objects_dedup.rst

Merge pull request #67726 from shraddhaag/wip-shraddhaag-enable-crimson-basic

qa/suites/crimson-rados: enable cephadm tests

crimson/.../lba_manager: add get_cursor interfaces

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_mappings to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert get_cursors to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../fixed_kv_btree: add get_btree helper

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../btree_lba_manager: convert mkfs to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert remove to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert read_extent to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../transaction_manager: convert get_pin to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>

auth/cephx/CephxKeyServer: un-inline methods to reduce header dependencies

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

auth/Auth: un-inline methods to reduce header dependencies

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

auth/AuthMethodList: include cleanup

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

auth/CMakeLists.txt: add cephx/CephxKeyServer.cc to auth_srcs

Everybody who includes CephxKeyServer.h must link with
CephxKeyServer.cc.

Currently, only ceph-mon and ceph-monstore-tool do that.  All the
others (e.g. ceph_dencoder.cc) get away with that mistake because many
of the KeyServer methods are inline, and if they just happen to use
only the inline symbols, there is no (visible) problem.

However, during my attempts to un-inline some of these methods (to
reduce header dependencies and reduce binary bloat due to having many
copies of the same code), there were runtime linker failures in
"ceph-dencoder".

The simplest solution is to just add cephx/CephxKeyServer.cc to
auth_srcs.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

Merge pull request #67639 from rhcs-dashboard/empty-state-bugfix

mgr/dashboard: Use illustration image for empty state table

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #66254 from afreen23/remove-e2e-yaml

qa: Remove cephadm e2e tests from teuthology

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Laura Flores <lflores@ibm.com>

qa: fixing cephadm mgmt-gateway test to remove openssl dependency

openssl cmd is not available in the container anymore. Let's use
cephadm-signed certificates instead.

Fixes: https://tracker.ceph.com/issues/75400
Signed-off-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #67589 from xxhdx1985126/wip-seastore-background-trans-cc-opt-new

crimson/os/seastore/cache: TRIM_DIRTY/CLEANER_* transactions won't invalidate other transactions anymore

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

mgr/dashboard: fix add host API path for namespace mapping in NVMe/TCP

fixes: https://tracker.ceph.com/issues/75407

Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

Merge pull request #67257 from tchaikov/wip-doc-mgr-cli

doc: update mgr module command documentation for per-module registries

Reviewed-by: John Mulligan <jmulligan@redhat.com>

mgr/dashboard: Fix breaking overview page

Fixes https://tracker.ceph.com/issues/75405

- removed cssgrid usage from notification page
- The css of css grid overriding overview page's css

Signed-off-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: Fix scrubbing state

Fixes https://tracker.ceph.com/issues/75401

-  not shown correct title "data cleanup in progress"
- when all okay, scrubbing state percent not shown
- - adds "unknown" state to PG_STATES

Signed-off-by: Afreen Misbah <afreen@ibm.com>

crimson/os/seastore/transaction: should consider non-aligned remapped
extents when updating paddrs for TRIM_DIRTY/CLEANER transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/lba: TRIM/CLEANER trans to adjust deltas of
LBALeafNodes when committing them.

This is to deal with the following scenario:
1. A client transaction modifies the value of the LBALeafNode, but not
   the pladdr but other field;
2. A TRIM/CLEANER transaction modifies the pladdr for the same laddr_t
   concurrently

In the old approach, the client trans may override the pladdr with the
outdated value after the TRIM/CLEANER transaction commits

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: correct the exception condition when merging
rewritten fixed kv nodes

Fixes: https://tracker.ceph.com/issues/74798
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/btree: make updates of lba leaf nodes ptrs
synchronous with contents updates

Since we need merge content of lba leaf nodes when committing
trim/cleaner transactions, and we rely on the child ptrs to determine
whether to modify mappings of pending leaf nodes. We must make sure
the ptr updates and node content updates are synchronous.

See LBALeafNode::merge_content_to() for detail

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/extent_pinboard: reset 2q_state when removing
extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/transaction_manager: block client transactions if
they conflict with rewriting transactions until the rewriting
transactions finishes

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cached_extent: treat extents under rewrite io as
stable too

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore: disable linked tree node operations when committing
rewriting transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/os/seastore/cache: rewrite transactions don't invalidate other
transactions anymore

Fixes: https://tracker.ceph.com/issues/73070
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>