doc/dev/developer_guide/essentials: update mailing lists

Update information for subscribing to Ceph development mailing lists as current documentation is outdated.

Fixes: https://tracker.ceph.com/issues/64580
Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>

Merge pull request #57629 from clwluvw/policy-remove-self-access

rgw: implement ConfirmRemoveSelfBucketAccess header for bucket policy

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge PR #62359 into main

* refs/pull/62359/head:
doc: add clarification note on how charmap is applied

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #62140 from rzarzynski/wip-nitzan-pg_pool_t-encode-decode-versions-compability

osd_type: decouple is_stretch_pool() from versioning of pg_pool_t's schema

Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #62165 from athanatos/sjust/wip-recovery-ops-leak-61594-with-history

osd: fix osd mclock queue item leak

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61821 from pritha-srivastava/wip-rgw-sts-oidc-user

rgw/sts: replacing load_stats with list_buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #62335 into main

* refs/pull/62335/head:
crypto: remove unused include

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

doc: add clarification note on how charmap is applied

There was confusion about whether setting the charmap on a directory also
affects the name of the directory itself.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #62031 into main

* refs/pull/62031/head:
qa: test fio with multiple versions of NFS

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #62253 from adamemerson/wip-rgw-shutdown-notify-crash

rgw: Try to handle unwatch errors sensibly

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #62156 from cbodley/wip-70270

rgw: sync fairness watcher reconnects on any error

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-By: Oguzhan Ozmen <oozmen@bloomberg.net>

Merge pull request #61803 from clwluvw/duplicate-prio

rgw: check for duplicate rule priorities in replication configuration

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #60843 from rhcs-dashboard/fix-69055-main

mgr/dashboard: When configuring the RGW Multisite endpoints from the UI allow FQDN(Not only IP)

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #62336 from tchaikov/librbd-std-variant

librbd: migrate from boost::variant to std::variant

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #62328 from ronen-fr/wip-rf-fin1-keys

common: md_config_obs_impl: remove deprecated get_tracked_conf_keys()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #62343 from VallariAg/wip-nvmeof-prometheus-singlegateway-rename

monitoring: rename NVMeoFSingleGatewayGroup alert

Merge pull request #61004 from rishabh-d-dave/mgr-vol-EINVAL

mgr/vol: use error number instead of error message

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #62301 from rishabh-d-dave/mgr-vol-fix-flake8-err

mgr/volumes: fix issues pointed by flake8

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #62203 from rishabh-d-dave/mgr-vol-fix-logger-name

mgr/vol: correct logger names

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #62334 from tchaikov/cmake-mark-cap-required

cmake: make libcap REQUIRED

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

librbd: migrate from boost::variant to std::variant

This change is part of a broader effort to reduce dependencies on
third-party libraries by leveraging C++ standard library alternatives.

Migrating from boost::variant to std::variant improves code readability
and maintainability while reducing external dependencies.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #62190 from ljflores/wip-tracker-69760

mon, osd: add command to remove invalid pg-upmap-primary entries

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Josh Salomon <josh.salomon@gmail.com>

mon, osd: add command to remove invalid pg-upmap-primary entries

The current rm-pg-upmap-primary command checks that the pgid exists
in the pgmap before continuing to remove it. Due to https://tracker.ceph.com/issues/66867,
some invalid pg-upmap-primary entires may exist for pools that have been removed.
Currently, these mappings are impossible to remove since the pgids no longer
exist in the pgmap.

This new command, rm-pg-upmap-primary-all, allows users the ability to remove
any and all pg-upmap-primary mappings in the osdmap at once, which includes
valid and invalid entries.

This command may also be helpful when upgrading from versions where users
are plagued by https://tracker.ceph.com/issues/61948. Users may use an upgraded
mon to remove all pg-upmap-primray entries (valid and invalid) so they continue
to upgrade to a safe version.

See manual testing for this patch here: https://tracker.ceph.com/issues/67179#note-12

Fixes: https://tracker.ceph.com/issues/67179
Fixes: https://tracker.ceph.com/issues/69760
Signed-off-by: Laura Flores <lflores@ibm.com>

Merge pull request #62110 from cbodley/wip-69842

rgw: rgw_init_ioctx() adds set_pool_full_try()

Reviewed-by: Adam Emerson <aemerson@redhat.com>

monitoring: rename NVMeoFSingleGatewayGroup alert

Rename the alert to NVMeoFSingleGateway.

The original name was confusing because it
accidently might convey that alert would be
triggered if there is a single gateway group.
Though 'NVMeoFSingleGatewayGroup' alert means that
there is single gateway in a group.

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>

Merge pull request #62276 from Matan-B/wip-matanb-radosbench

qa/tasks/radosbench: fix missing concurrency usage (on write)

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #62331 from baum/nvmeof_1_5

nvmeof: bump image version to 1.5

Merge pull request #61990 from aclamk/wip-aclamk-write-v2-dirty-reshard

os/bluestore: write_v2 resharding misplaced

crypto: remove unused include

openssl_crypto_accel.cc does not need the declarations included by
openssl/engine.h, also, openssl/engine.h was deprecated in favor of
the provider API, the engine support was removed in fedora 41.

so, let's avoid including it. please note, the "ENGINE" struct's
definition is available in openssl/types.h.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cmake: make libcap REQUIRED

this header is required when building the extended block device plugins
on linux, without it the build fails like:

```
/home/kefu/dev/ceph/src/extblkdev/ExtBlkDevInterface.h:38:10: fatal error: 'sys/capability.h' file not found
   38 | #include <sys/capability.h>
      |          ^~~~~~~~~~~~~~~~~~
```

so, let's mark this package REQUIRED, so that we can fail early
when generating the building system.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

nvmeof: bump image version to 1.5

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>

Merge pull request #62079 from myoungwon/seastore-omap-log-cleanup

crimson/os/seastore: cleanups related to managing omap and log tree

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60575 from steven-chien/systemd-prefix

systemd: use CMake install prefix in templates

common: remove deprecated get_tracked_conf_keys()

This is the last PR in a set of PRs that
replaced the deprecated get_tracked_conf_keys()
with the new get_tracked_keys().

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #62185 from ronen-fr/wip-rf-more8-keys

kernel/KernelDevice: replace obsolete get_tracked_conf_keys()

Reviewed-by: Matt Vandermeulen <matt@reenigne.net>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61672 from Matan-B/wip-matanb-cmake-crimson-v3

src: Cleanup WITH_ALIENSTORE

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62243 from NitzanMordhai/wip-nitzan-osd-destroyed-in-nodes-ls

OSDMonitor: exclude destroyed OSDs from "ceph node ls" output

Merge pull request #62182 from ronen-fr/wip-rf-more7cr-keys

test/crimson/test_config: replace obsolete get_tracked_conf_keys()

Merge pull request #60491 from soumyakoduri/wip-skoduri-restore-vers

rgw/cloud-restore: Handle versioned objects

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Reviewed-by: Shreyansh Sancheti <ssanchet@redhat.com>

Merge pull request #62319 from anthonyeleven/rt

doc/rados/troubleshooting: Improve troubleshooting-pg.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/rados/troubleshooting: Improve troubleshooting-pg.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #62316 from anthonyeleven/cme

doc/rados/operations: improve crush-map-edits.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/rados/operations: improve crush-map-edits.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #61816 from ljflores/wip-tracker-62776

qa/suites/rados/basic: ignore POOL_APP_NOT_ENABLED detail in cluster log

Merge pull request #57046 from clwluvw/vault-del-bucketkey

rgw: false assumption on vault bucket key deletion

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #62231 into main

* refs/pull/62231/head:
mds: set alternate_name for new fullbit dentries
qa: test alternate_name recovery during mds failover

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>

Merge pull request #62194 from rhcs-dashboard/fix-token-polling

mgr/dashboard: fix token polling req in multi-cluster UI

Merge pull request #61772 from rhcs-dashboard/fix-multisite-wizard-forms

mgr/dashboard: fix multisite wizard form not showing

Merge pull request #61781 from rhcs-dashboard/xml-fix

mgr/dashboard: fix lifecycle get request

Merge PR #61877 into main

* refs/pull/61877/head:
libcephfs_proxy: implement client side async rw operation
libcephfs_proxy: implement server side async rw operation
libcephfs_proxy: implement the async rw callback
libcephfs_proxy: define async rw opcode and structures
libcephfs_proxy: define some helper macros for callbacks
libcephfs_proxy: move pointer obfuscation functions to proxy_helpers.h
libcephfs_proxy: negotiate and use the async cbk feature
libcephfs_proxy: implement asynchronous callbacks
libcephfs_proxy: gracefully handle connection close
libcephfs_proxy: add negotiation documentation
libcephfs_proxy: replace legacy handshake by negotation in server side
libcephfs_proxy: replace legacy handshake by negotation in client side
libcephfs_proxy: implement server side negotiation
libcephfs_proxy: implement client side negotiation
libcephfs_proxy: implement receiving of negotiate structure
libcephfs_proxy: add negotiation structures
libcephfs_proxy: add support for control messages
libcephfs_proxy: reuse proxy_link_{read|write}

Reviewed-by: Sachin Prabhu <sp@spui.uk>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/volumes: fix issues pointed by flake8

Fixes: https://tracker.ceph.com/issues/70480
Signed-off-by: Rishabh Dave <ridave@redhat.com>

mgr/volumes: correct logger names

Fixes: https://tracker.ceph.com/issues/70376
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge PR #62228 into main

* refs/pull/62228/head:
libcephfs: getcwd after chdir for API constraint
client: unwrap dentries for getcwd
client: refactor / optimize chdir
test/libcephfs: test getcwd with case insensitive dir
test/libcephfs: refactor ManyNestedDirs
include/filepath: add iterators for components

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #62296 from afreen23/wip-sso-fix

mgr/dashboard: fix OAuth2 SSO when roles_path is empty

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

rgw/cloud-restore: Handle "null" instance non-current objects

During restore, if the object instance is "null", clear it while updating
bi entry.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

rgw/cloud-restore: Handle versioned objects

While restoring non-current object versions, ensure they remain non-current.

Read `olh_epoch` from the restored object's metadata into a new attr
"RGW_ATTR_RESTORE_VERSIONED_EPOCH". This attr/olh_epoch is used while
updating bi entry and also to reset HEAD object post expiry of temporary
copies.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

mgr/dashboard: add basic e2e to verify wizard

and topology viewer

Fixes: https://tracker.ceph.com/issues/69925
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix multisite wizard form not showing

Fixes: https://tracker.ceph.com/issues/69925
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #62119 from MartinNowak/patch-2

doc/radosgw: Improve bucketpolicy.rst #58689 - document SSE-C IAM condition key

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/dashboard: fix OAuth2 SSO when roles_path is empty

Fixes https://tracker.ceph.com/issues/70454

Due to PR linter updates, this place remained not updated hence failing

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #62258 from dvanders/dvanders_doc_fix

doc/rados/configuration/bluestore-config-ref: Fix lowercase typos

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #62232 into main

* refs/pull/62232/head:
include/cephfs: dump optmetadata

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard: add xlmtodict import and fix lifecycle get request

Fixes: https://tracker.ceph.com/issues/70128
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #62274 from phlogistonjohn/jjm-more-bwc-fixes

build-with-container: two small fixes

Merge pull request #62252 from ronen-fr/wip-rf-fix-62188

librbd/ConfigWatcher: fix a missing 'override'

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #54396 into main

* refs/pull/54396/head:
PendingReleaseNote: add note for pause_purging and pause_cloninig
doc/cephfs: add note for config option pause_purging and pause_cloning
qa/cephfs: add tests for mgr/vol config pause_cloning
qa/cephfs: extend wait for trash empty
qa/cephfs: add tests for config option pause_purging
qa/cephfs: don't strip any whitespace for get_shell_stdout
mgr/vol: add pause/resume mechanism for async jobs

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #62240 into main

* refs/pull/62240/head:
qa/cephfs: require-osd-release should now be tentacle

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

qa/tasks/radosbench: make use of concurrency flag

This affects mainly radosbench-high-concurrency.yaml
The concurrency flag was only used in non-write scenarios to
populate the initial data.
We should use this flag with `write` mode as well.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge PR #62200 into main

* refs/pull/62200/head:
common/include: Add map type election and fmt::format to interval_map

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #59780 from irq0/wip/osd-asok-messenger-dump

Add Asok Command: Dump Messenger Status (Connections, TCP stats, ..)

Merge PR #62230 into main

* refs/pull/62230/head:
client: avoid multiple calls to path_walk

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #61791 from MaxKellermann/common_includes

common: add missing includes

Reviewed-By: Adam Emerson <aemerson@ibm.com>

qa/rgw: suppress "pool(s) full" cluster warning for rados-pool-quota

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: test_rgw_rados_pool_quota expects InsufficientCapacity

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/s3: map ENOSPC to 507 InsufficientCapacity

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rgw_init_ioctx() adds set_pool_full_try()

add librados::OPERATION_FULL_TRY globally so we never hang waiting for
space when the pool reaches its quota limit

Fixes: https://tracker.ceph.com/issues/69842
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWRadosRemoveCR uses rgw_init_ioctx()

avoid direct calls to ioctx_create() so rgw_init_ioctx() can set
appropriate flags globally

Signed-off-by: Casey Bodley <cbodley@redhat.com>

include/cephfs: dump optmetadata

For inspection from `dump tree` MDS command.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

mds: set alternate_name for new fullbit dentries

Fixes: 9fbf37ade38bc1d057eda2459b0ccfd2ab40626a
Fixes: https://tracker.ceph.com/issues/70409
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

qa: test alternate_name recovery during mds failover

Fixes: https://tracker.ceph.com/issues/70409
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

libcephfs: getcwd after chdir for API constraint

libcephfs semantics require that the old path to the cwd be returned when
getcwd is encounters an unlinked directory in the current working directory.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: unwrap dentries for getcwd

This was missed in the path_walk refactor. readdir is not the only way to "get"
dentry names.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: refactor / optimize chdir

In particular: there's no reason to do a getcwd after chdir.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

test/libcephfs: test getcwd with case insensitive dir

Unfortunately, it's not easy to refactor this test into a shared method without
setting up an explicit test class which has been avoided up to this point. So
I'm going to just copy the code. Sorry.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

test/libcephfs: refactor ManyNestedDirs

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

include/filepath: add iterators for components

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #62160 from cbodley/wip-qa-vault-list

qa/vault: create_secrets() cleanup checks for orphaned keys

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

script: ensure curl is always available in build containers

Ensure that curl is installed in all build containers regardless of
ceph's dependencies or other factors. This allows us to use curl in
any subsequent build steps/scripts.

Fixes: https://tracker.ceph.com/issues/70451
Signed-off-by: John Mulligan <jmulligan@redhat.com>

script/build-with-container: fix incorrect build invocation for tests

The `tests` build target was set to build the "normal" target but it
needs to build the tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61945 from kotreshhr/snapshot-referent-inodes

mds: snapshot referent inodes feature

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #60014 from rhcs-dashboard/cephadm-e2e-endless-fix-1

mgr/dashboard: fix cephadm dashboard e2e and a combobox issue

Reviewed-by: Afreen Misbah <afreen@ibm.com>

crimson/os/seastore: introduce rename_omap_root() with src and dest onodes

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: prevent omaptree_rm_key() to be called concurrently in the same transaction

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: introduce and use omaptree_initialize()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: reuse get_omap_root()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

os/bluestore: Improve bluestore_blob_t::add_tail

It could happen that target blob already had empty mapping at the end.
In such case, do not add more empty mappings, just expand exising one.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

os/bluestore: Make write_v2 not need to use compress_extents, addendum

There was possible case that affected rightmost changed extent.
It could be that 2 mergeable extents were next to each other.
This change melds them together.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

os/bluestore: Make write_v2 calculate affected range

This reduces size for dirty_range and need_reshard range.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

os/bluestore: Make write_v2 not need to use compress_extents

Remove need to call compress_extents.
The only place that could produce compressable extents are
_try_reuse_allocated_[l/r] functions.

Create _place_extent_in_blob() function that adds mapping to blob.
If possible, it expands the extent; if not, creates the new one.
The function is used whenever we need new mapping to an existing Blob.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #62245 from afreen23/wip-disable-rgw-test

mgr/dashboard: Disable MFA test