doc: add notable/security changes in v18.2.8

AI-Assisted: scan of changeset to indicate notable changes
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

doc: add known issues for v18.2.8

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

doc: add reef release notes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script: fix argument change for github module

    Traceback (most recent call last):
        File "/home/batrick/scm/ceph/./src/script/ceph-release-notes", line 400, in <module>
            gh = github.GitHub(
            access_token=args.token)
    TypeError: GitHub.__init__() got an unexpected keyword argument 'access_token'

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #67632 into main

* refs/pull/67632/head:
pybind/mgr: call new _ceph_exit for killpoints
mgr: add ceph binding for exit

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #67643 from rhcs-dashboard/75288-bugfix

mgr/dashboard: Breadcrumb should allow going back to subsystem tab

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #64439 from myoungwon/wip-logstoremanager

crimson/os/seastore: Introduce LogStoreManager

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #66026 from lvshuo2016/featurecrc-new

src/common: add crc32c support for riscv64

Reviewed-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #67815 from rhcs-dashboard/helper-spacing-fix

mgr/dashboard: add helper text to bucket form > policy and other spacing fixes

Reviewed-by: Dnyaneshwari talwekar <dtalweka@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #67847 from rhcs-dashboard/fix-services-page

mgr/dashboard: services page breaking for test_orchestrator module

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #67544 from tchaikov/wip-fix-unittest-peeringstate-leaks

test/osd: Fix unittest_peeringstate message leaks:

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>

Merge pull request #67807 from afreen23/overview-content

mgr/dashboard: Stabilizing the storage overview card

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: add helper text to bucket form > policy and other spacing fixes

Fixes: https://tracker.ceph.com/issues/75525
Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #65509 from MaxKellermann/mon_includes

mon: reduce header dependencies

Reviewed-by: Kefu Chai <k.chai@proxmox.com>

Merge pull request #67865 from tchaikov/doc-runtime-updatable

doc: show runtime-updatable config metadata

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: show runtime-updatable config metadata

Extend the confval Sphinx extension to render whether an option
can be updated at runtime.

Use the same policy as Option::can_update_at_runtime() so the
documentation reflects both native config options and mgr module
options consistently.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>

mgr/dashboard: Add capacity thresholds

Signed-off-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: services page breaking for test_orchestrator module

Fixes: https://tracker.ceph.com/issues/75561
Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #67124 from batrick/i74652

 mds: improve debugging for snaprealms marked subvolume

Venky Shankar <vshankar@redhat.com>

Merge pull request #67125 from batrick/multifs-auth-fix

qa: clean up and simplify client 2 caps

Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #67377 from chrisphoffman/wip-74934

client: During encryption of short case-insensitive file names, store raw ciphertext

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #67496 from batrick/i75143

mds: use SimpleLock::WAIT_ALL for wait mask

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #67556 from batrick/i70580

qa: allow multiple mgr sessions during eviction test

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #67610 from kginonredhat/75267-service-is-not-getting-up-when-being-added-with-ipv6-vip-address

Fixing issue of service mgmt-gateway not coming up when configured to…

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

crimson/os/seastore: handle duplicate keys in LogNode::remove_entry

Previously, LogNode::remove_entry returned early when a log_key was
found, assuming uniqueness. However, duplicate keys can exist in the
node if an older entry was previously removed.
This commit also adds a unit test to verify this scenario.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: reload head if modified

This commit also fixes the test case to verify that
the head is correctly allocated and updated
during omap_set_keys operations involving multiple keys.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore, osd/PGLog: handle omap_iterate retry to avoid duplicate entries

Seastore omap_iterate may retry internally on conflicts, which can
cause PGLog to process the same entries multiple times when entries
are handled directly in the iteration callback.

Introduce a conflict hook in omap_iterate so callers can reset
iteration state on retry. PGLog now buffers entries during iteration and
applies process_entry() only after a successful pass, clearing the buffer
on retry to avoid duplicates.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: ensure data integrity with deep copy in omap_get_value

Previously, omap_get_value could return a bufferlist pointing to
memory without guaranteed lifetime. This patch introduces LogNode::copy_t
to distinguish between DEEP and SHALLOW copies.

- Default get_value to DEEP copy for external safety.
- Use SHALLOW copy in internal paths (e.g., remove_kv) to maintain performance.
- Refactor LogManager::omap_get_value to simplify coroutine flow.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: ensure extent is loaded if missing from cache

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: correct UPPER_BOUND behavior to start looking up items exceeding the target

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: fix crash by checking for nullopt in search input

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: return correct result for non-logkey entries

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: use do_with to avoid stack-use-after-return

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

osd/PGLog: set dirty_to to max to avoid partial writes

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: add test for log_manager

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: support for large kv pair in LogNode

Each log_key_t contains a chunk_idx field to manage values
that span multiple LogNodes when the value size exceeds the
maximum capacity of a single LogNode.
See detailed description in log_manager.h.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: optimize handling of batched requests

During 4KB random write workloads, SeaStore  receives
batched dup_* entries in both omap_set_keys.
This change enables efficient batch processing of these
requests to reduce overhead.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: introduce omap_rm_keys interface in omap_manager

Deletion of pg_log_entry_t entries is performed by omap_rm_keys using a set.
For example, omap_rm_keys might be called with a set containing
pg_log_entry_t entries ranging from 0011.0001 to 0011.0010.
In this case, calling omap_rm_key individually for each entry is inefficient,
because each call triggers a traversal of the entire list.

To avoid this, omap_rm_keys with a set is introduced in omap_manager
to handle removal request more efficiently.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: adjust select_log_omap_root to return a single root

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: invoke managers depending types based if branch

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: remove duplicate keys for non-log entries

When writing a non-log key, remove any existing duplicate key
before inserting the new KV pair. With this change, full list
traversal is no longer required during remove_kv.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: make _fastinfo overwritable to minimize space overhead

This commit forces _fastinfo to be stored at the last position of a LogNode.
By doing so, _fastinfo can be overwritten by the next pg_log_entry.

Since _fastinfo has a fixed key with varying contents and is included in
every write transaction, placing it at the tail enables efficient overwrites.

As a result, this change reduces LogNode allocation and deallocation,
thereby lowering space overhead. Moreover, garbage collection for obsolete
key-value pairs is unnecessary due to overwrite semantics.

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

crimson/os/seastore: add log_manager to handle pgmeta_oid's key-value pairs separately

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

Merge pull request #65541 from MaxKellermann/auth_includes2

auth: reduce header dependencies

Reviewed-by: Kefu Chai <k.chai@proxmox.com>

crimson/os/seastore: remove log_root to maintain a single root for OMAP and LOG

Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>

Merge pull request #66261 from aclamk/aclamk-bs-cputrace-for-write-path

os/bluestore: Cputrace measurements for core BlueStore functions

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Jaya Prakash <jayaprakash@ibm.com>

Merge pull request #67716 from rhcs-dashboard/fix-namespace-issue

mgr/dashboard: fix add host API path for namespace mapping in NVMe/TCP

Reviewed-by: Puja Shahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67769 from rhcs-dashboard/fix-nvmeof-subsystem-create-firefox-next

mgr/dashboard: fix-nvmeof-subsystem-create-firefox-next

Reviewed-by: Puja Shahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67810 from gbregman/main

mgr/dashboard: Add "connection get_io_statistics" to NVMeoF CLI

Merge pull request #67742 from VallariAg/wip-nvmeof-upgrade-fix

qa: Add nvmeof upgrade from v20.2.0 and tentacle

mgr/dashboard: Fix loading states in storage overview card

Fixes https://tracker.ceph.com/issues/75321
Fixes https://tracker.ceph.com/issues/75299

- removes storage type
- stabilizes overview card for loading data
- raw capcity shown when promethues not there
- multiple refresh intervals which may vcause sync issues and bugs hence moved the logic to parent - overview component
- Now all queries are updated at 5 s interval except data consumption - using promethues interval. This needs more refactor hence would do in a later PR

Signed-off-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: Add tootltip to storage overview

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67812 from bluikko/wip-doc-cephadm-spelling2

doc/cephadm: Fix more spelling errors

Merge pull request #67783 from phlogistonjohn/jjm-bwc-configure-args

script/build-with-container: add CONFIGURE_ARGS env var to configure step

common,arch,cmake: add RISC-V crc32c support

This adds hardware-accelerated crc32c support for the RISC-V
architecture. It includes the feature implementation, necessary
CMake configuration, and plumbing in src/arch/riscv.c to correctly
detect and select the optimized instructions.

Signed-off-by: lvshuo2016 <lv.shuo@sanechips.com.cn>

mgr/dashboard: Add "connection get_io_statistics" to NVMeoF CLI

Fixes: https://tracker.ceph.com/issues/74964
Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #67122 from Naveenaidu/wip-naveen-fix-RemoteReservationRevoked-race-condition

osd/PeeringState: handle race condition of RemoteReservationRevoked event for Backfilling state

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #67754 from idryomov/wip-kclient-features-update

include/ceph_features: note more kernel versions

Reviewed-by: Miki Patel <miki.patel132@gmail.com>

Merge pull request #67244 from bill-scales/issue73891

osd: Avoid pwlc spanning intervals

Reviewed-by: Samuel Just <sjust@ibm.com>
Reviewed-by: Radoslaw Zarzynski<rzarzyns@redhat.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>

Merge pull request #62227 from atta/fix-lua-mkdtemp-permissions

rgw: fix luarocks directory permissions

mgr/cephadm: fix mgmt-gateway startup on IPv6 VIP

Ensure mgmt-gateway service starts correctly when it is configured
to listen on an IPv6 VIP address.

Fixes: https://tracker.ceph.com/issues/75267
Signed-off-by: Kobi Ginon <kginon@redhat.com>

Merge pull request #67436 from adk3798/cephadm-fix-mon-crush-loc-text

qa/cephadm: fix test_set_mon_crush_locations test

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #67776 from bluikko/wip-doc-radosgw-spelling

doc/radosgw: Fix spelling errors

doc/cephadm: Fix more spelling errors

And other such minor obvious issues, including a spelling error
introduced in the previous commit 2565579caa1a118e9032283b55e969f9badcd6b6

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67619 from tchaikov/wip-cmake-cephfs-journal-tool-without-liburing

cmake: link uring::uring to Boost::boost and fix cephfs tools linkage

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>

Merge pull request #67708 from rhcs-dashboard/fix-miscellaneous

mgr/dashboard: show System metadata used

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #66450 from rkachach/fix_issue_74015

mgr/cephadm: Fix mgmt-gateway default port in get_port_start()

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #67530 from rkachach/fix_ipv6_with_brackets

cephadm: fix EndPoint to handle bracketed IPv6 addresses

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #67690 from rhcs-dashboard/75322-Updating-empty-state-message-on-performance-chart

mgr/dashboard: Fixing message when prometheus is disabled in performa…

Reviewed-by: Afreen Misbah <afreen@ibm.com>

doc/radosgw: Fix spelling errors

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #67421 from ifed01/wip-ifed-raise-min-comp-size-hdd

options/global: raise bluestore_compression_min_blob_size_hdd to 64K

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #67695 from rkachach/fix_issue_74860

qa: fixing cephadm mgmt-gateway test to remove openssl dependency

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>

Merge pull request #67433 from ifed01/wip-ifed-fix-min-blob-size-tracking

os/bluestore: track compression_*blob_size* parameters for online update

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

mgr/cephadm: Fix mgmt-gateway default port in get_port_start()

The mgmt-gateway port was already defaulted to 443 in most places, but
get_port_start() did not apply this default. Since the output of
get_port_start() is used both to configure the daemon ports which are
later used to to open them in firewalld, this inconsistency meant the
HTTPS port was not opened when firewalld service was active.

This change makes get_port_start() also default to port 443, ensuring
the daemon is configured correctly and the corresponding firewalld port
is opened as expected.

Fixes: https://tracker.ceph.com/issues/74015
Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/dashboard: show miscellaneous data used

Fixes: https://tracker.ceph.com/issues/75320
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #64975 from liu-chunmei/shard_osd_final

osd shard changes upon restart

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #65912 from MattyWilliams22/wip-dont-delete-objects

test: Add option to not delete created objects

qa: Add nvmeof upgrade from v20.2.0

Also update install task for nvmeof/upgrade test

Fixes: https://tracker.ceph.com/issues/75453
Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>

mgr/dashboard: Fixing message when prometheus is disabled in performance charts
Fixes: https://tracker.ceph.com/issues/75322
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>

Merge pull request #67734 from rhcs-dashboard/fix-namespace-size

mgr/dashboard: Namespace size should accept GiB in NVMe/TCP

Reviewed-by: Puja Shahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67782 from rkachach/fix_issue_75492

mgr/nvmeof: Adding missing CLICommand field to nvmeof mgr module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #66580 from kamoltat/wip-ksirivad-fix-72994

mon [stretch-mode]: Allow a max bucket weight diff threshold
Reviewed-by: Ronen Friedman <rfriedma@ibm.com>

Merge PR #67780 into main

* refs/pull/67780/head:
Revert "Merge PR #67630 into main"

Reviewed-by: Shraddha Agrawal <shraddhaag@ibm.com>

mgr/nvmeof: Adding missing CLICommand file to nvmeof mgr module

Fixes: https://tracker.ceph.com/issues/75492
Signed-off-by: Redouane Kachach <rkachach@ibm.com>

script/build-with-container: add CONFIGURE_ARGS env var to configure step

Add a new optional CONFIGURE_ARGS environment variable to the configure
step so that there's a mechanism to pass custom cmake options that
aren't handled elsewhere in the run-make.sh script.

Because configure is a rather fundamental build step it's probably
preferable to set this via an env file so that it persists across
rebuilds. Using an environment var here also avoids both needing to
change run-make.sh or add another CLI option to BWC which already has
too many.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Revert "Merge PR #67630 into main"

This reverts commit 3a5e4524aa56de4c26400ccf994baa6ba8e16d9e, reversing
changes made to d334ff531c563bb7d0e37777f606322ec91b7453.

To everyone's surprise, skipping a workflow does not make it less
required. Well done Github!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #67275 from ifed01/wip-ifed-fix-bluefs-expand-test

qa/standalone: fix/improve bluefs tests

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #67609 from ifed01/wip-ifed-bluefs-stats-reset

os/bluestore: add 'bluefs stats reset' admin socket command.

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #67770 from bluikko/wip-doc-cephadm-spelling

doc/cephadm: Fix spelling errors

Merge pull request #67718 from rhcs-dashboard/fix-subsystem-create-layout-issue

mgr/dashboard: Footer actions shift upward instead of staying pinned at modal bottom in NVMe/TCP subsystem create wizard

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>

doc/cephadm: Fix spelling errors

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>

Merge pull request #65405 from rhcs-dashboard/notification-store-events

mgr/dashboard: Add restore events in notification screen

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

mgr/dashboard: Breadcrumb should allow going back to subsystem tab

Fixes: https://tracker.ceph.com/issues/75288
Signed-off-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #67760 from gbregman/main

mgr/dashboard: Add secure and verify-host-name to "listener add" on NVMeoF CLI

Merge pull request #67647 from rhcs-dashboard/fix-75317-main

mgr/dashboard: update onboarding screen as per design

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #67713 from rhcs-dashboard/fix-nvmeof-initiator-add-visibility

mgr/dashboard: Initiator add shows success but host is not added/displayed in Subsystem Initiators table

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: pujaoshahu <pshahu@redhat.com>

mgr/dashboard: fix-nvmeof-subsystem-create-firefox-next

Fixes: https://tracker.ceph.com/issues/75434
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>

mgr/dashboard: rename expand-cluster to add-storage

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: update onboarding screen as per design

Fixes: https://tracker.ceph.com/issues/75317
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

qa/suites/crimson-rados: add fio test case for osd shard number changes upon restart for 3 osd

Signed-off-by: Chunmei Liu <chunmei.liu@ibm.com>