crimson: Create the shared promise before waited upon

RecoveryBackend::pushes map creates each shared_promise
in wait_for_pushes call. There can be a situation where
set_pushed is called due to handled push reply (handle_push_reply)
before the shared_promise was even constructed due to backfill progress
is stuck.

Fixes: https://tracker.ceph.com/issues/70502
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>

Merge pull request #62795 from laimis9133/main

doc/radosgw/cloud-transition: use reStructuredText for code

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #62798 from zdover23/wip-doc-2025-04-14-squid-upgrade-warning

doc/releases: Direct users to upgrade to 19.2.2

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #62792 from prik73/70719-fix-user-form-login-typo

mgr/dashboard: fix typo in User Management form (logon → login)

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #62812 from Matan-B/wip-matanb-crimson-only-backfill-stuck

crimson/osd: fix do_recovery returned boolean

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Mohit Agrawal <moagrawa@redhat.com>

Merge pull request #62744 from rhcs-dashboard/accounts-e2e

mgr/dashboard: rgw accounts UI e2e tests

crimson/osd: fix stuck recovery

https://github.com/ceph/ceph/pull/62080 tested version was **different**
from the one that got merged.
The untested change was changing the boolean returned from start_recovery_ops.
While the seastar::repeat loop in BackgroundRecoveryT<T>::start() was changed accordingly,
other do_recovery() return cases were not considered.

See Tested / Merged here: https://github.com/Matan-B/ceph/pull/2/files

start_recovery_ops used by do_recovery should return whether the iteration (i.e recovery) keep going.

_Note: This has caused a regression in our suite_

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61992 from dparmar18/ib61357

qa: add parallelized cleanup step in _rebuild_metadata()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #62259 into main

* refs/pull/62259/head:
qa/tasks/cephfs/mount: introduce _default_gateway method
qa/tasks/cephfs/mount: use 'ip r' instead 'route'

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #62553 into main

* refs/pull/62553/head:
mds: fix issues with use-after-free in C_Flush_Journal
mds: fix issues with use-after-free in C_Flush_Journal

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #62764 from rhcs-dashboard/bucket-ratelimit-fix

mgr/dashboard: bucket ratelimit API failing while changing owner with ratelimit config

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #59533 from ifed01/wip-ifed-no-async-compact-in-splillover-tests

test/store_test: enforce sync compaction for spillover test cases.

Reviewed-by: Md Mahamudur Rahaman Sajib mahamudur.sajib@croit.io

Merge pull request #62807 from ceph/wip-yuriw-release.pocess-main

doc: changed refs from quincy to squid

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #61786 from Hezko/ns-create-size-fix

mgr/dashboard: Ns create size fix

doc/releases: Direct users to upgrade to 19.2.2

Direct users to upgrade only to Squid v19.2.2, and warn readers not to
upgrade to Squid 19.2.1. This PR is raised in response to a request from
Neha Ojha.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #62799 from anthonyeleven/fix-formatting

doc/cephadm/services: Fix formatting in osd.rst

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61610 from rzarzynski/wip-test-bl-lastp-on

test/bufferlist: turn the old last_p fix verification on

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #56922 from ifed01/wip-ifed-store-test-no-sleeps

test/store_test: get rid off sleep() call wherever possible

Reviewed-by: Pere Diaz Bou <pere-altea@hotmail.com>

Merge pull request #62804 from Matan-B/wip-matanb-rgw-spec.in

ceph.spec.in: package man/rgw-gap-list

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #62783 into main

* refs/pull/62783/head:
libcephfs_proxy: fully initialize async I/O structure

Reviewed-by: Sachin Prabhu <sp@spui.uk>
Reviewed-by: Shwetha Acharya <sacharya@redhat.com>

doc: changed refs from quincy to squid

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #62692 from ronen-fr/wip-rf-watch-ratiob

osd/scrub: additional configuration parameters to trigger scrub reschedule

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

libcephfs_proxy: fully initialize async I/O structure

The ceph_ll_io_info structure has recently been extended to support
zerocopy operations. The proxy was initializing just the known members,
so, after the zerocopy support, it was passing garbage in some fields,
causing failures.

This patch completely clears the whole structure to be sure that
everything is initialized to its default value.

Signed-off-by: Xavi Hernandez <xhernandez@gmail.com>

Merge pull request #62761 from rhcs-dashboard/tiering-fix

mgr/dashboard: RGW - Reatin Head Object Fix

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: fix bucket rate limit API on owner change

Fixes: https://tracker.ceph.com/issues/70874
PR covers & fixes below scenarios:
Whenever we change the owner of bucket from non-tenanted to tenanted and
vice-versa with the rate-limit changes, there was issue in sending bucket name
Scenario 1: Changing the bucket owner from tenanted to non-tenanted
Scenario 2: Changing the bucket owner from non-tenanted to tenanted
Scenario 3: Keeping the owner(tenanted) same and changing only rate limit

Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #61273 from rhcs-dashboard/topicmanagment-ui

mgr/dashboard: Add RGW topic listing in dashboard

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

ceph.spec.in: add man/rgw-gap-list

Introduced in https://github.com/ceph/ceph/pull/62749
Centos builds fail with:
```
error: Installed (but unpackaged) file(s) found:
   /usr/share/man/man8/rgw-gap-list.8.gz

RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/share/man/man8/rgw-gap-list.8.gz
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

doc/radosgw/cloud-transition: fix details

Fix documentation on Ceph cloud storage transitions - broken link urls, returning back reStructuredText and some other details.

Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>
Co-authored-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #62776 from rhcs-dashboard/tiering-secretKey-fix

mgr/dashboard: RGW - Tiering form - secret key fix

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

doc/cephadm/services: Fix formatting in osd.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #62784 from anthonyeleven/osd-restart-warn

doc/cephadm: Add admonition re restarting an OSD service

Reviewed-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: fix typo in User Management form
Fixes: https://tracker.ceph.com/issues/70719
-Corrected the label from 'logon' to 'login' in the User Management form

Signed-off-by: prik73 <prinovac@gmail.com>

osd/scrub: additional configuration params to trigger scrub reschedule

Adding the following parameters to the (small) set of configuration
options that, if changed, trigger re-computation of the next scrub
schedule:
- osd_scrub_interval_randomize_ratio,
- osd_deep_scrub_interval_cv, and
- osd_deep_scrub_interval (which was missing in the list of
  parameters watched by the OSD).

Fixes: https://tracker.ceph.com/issues/70806
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #62732 from Hezko/nvmeof-cli-gw-addr

mgr/dashboard: allow selecting target gw in nvmeof cli/api endpoints

Merge pull request #62786 from ceph/wip-yuriw-release.pocess-main

doc: minor additions based on the latest mods

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #62490 from bill-scales/ec_select_primary

osd: Restrict choice of primary shard for ec_optimizations pools

Merge pull request #62757 from rhcs-dashboard/fix-70866-main

mgr/dashboard: add description for default Pool Application select options

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

doc: minor additions based on the latest mods and updated example to squid

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #62704 from sseshasa/wip-fix-recovery-counters-calc

osd/scheduler/OpSchedulerItem: Fix calculation of recovery latency counters

Merge pull request #62749 from ivancich/wip-document-gap-list

rgw/doc: add man documentation for the rgw-gap-list tool

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rgw: add man documentation for the rgw-gap-list tool

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #62780 from aainscow/sv_notify_fix

rgw: Fix compilation issue with clang on centos

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/cephadm: Add admonition re restarting an OSD service

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #62771 from idryomov/wip-drop-remote-namespace-cls-err

cls/rbd: drop overzealous CLS_ERR message in mirror_remote_namespace_get()

Reviewed-by: N Balachandran <nithya.balachandran@ibm.com>

Merge pull request #62601 from rhcs-dashboard/smb-tabs-route

mgr/dashboard: make smb tabs routable

rgw: Fix compilation issue with clang on centos

We are getting a compilation issue with accessing an incomplete type.

This fix moves some code around to avoid this issue.

I don't fully understand why we don't see this issue everywhere!

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

mgr/dashboard: RGW - Reatin Head Object Fix

Fixes: https://tracker.ceph.com/issues/70871
Signed-off-by: Dnyaneshwari Talwekar <dtalwekar@redhat.com>

mgr/dashboard: RGW - Tiering form - secret key fix

Fixes: https://tracker.ceph.com/issues/70888
Signed-off-by: Dnyaneshwari Talwekar <dtalwekar@redhat.com>

Merge pull request #62697 from Hezko/nvmeof-response-fix

mgr/dashboard: fix subsystems list api response

test: Add unittests for pgtemp_primaryfirst/pgtemp_undo_primaryfirst

Add unittests for pgtemp_primaryfirst and pgtemp_undo_primaryfirst
to prove the later is a reverse transform and that neither has any
effect until an optimized EC pool configures non-primary shards.

Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>

osd: Restrict choice of primary shard for ec_optimizations pools

Pools with ec_optimizations enabled have restrictions on which
shards are permitted to become the primary because not all shards
are updated for every I/O.

To preserve backwards compatibility with downlevel clients
pg_temp is used as the method to override the selection of
primary by OSDMap. Directly changing the logic in OSDMap
would have meant that all clients need to be upgraded to
tentacle before using optimized EC pools, so was discounted.
Using primary_temp to set the primary for an EC pool is
not reliable because under error conditions an OSD can store
multiple shards for the same PG and primary_temp cannot
define which of these shards will be choosen.

For optimized EC pools pg_temp is shuffled so that the
non-primary shards are listed last. This means that the
existing logic in OSDMap that picks the first available
shard as the primary will avoid selecting a non-primary
shard. OSDMonitor applies the shuffle when pg_temp is set,
this is then reverted in PeeringState when initializing the
acting set after OSDMap has selected the primary.

PeeringState::choose_acting is modified to set pg_temp if
OSDMap has selected a non-primary shard, this will cause
a new OSDMAP to be published which will persuade
OSDMap to select a primary shard instead.

Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>

mgr/dashboard: rgw accounts UI e2e tests

Fixes: https://tracker.ceph.com/issues/69674
Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #62527 from rhcs-dashboard/smb-file-uploader

mgr/dashboard: add file uploader for smb usersandgroups

Reviewed-by: nizamial09 <nia@redhat.com>

Merge pull request #62673 from adamemerson/wip-test-osd-signed-comparison

test/osd: Fix signed comparison warning

Merge pull request #62163 from zmc/bwc-debs

Enabling containerized debian pipeline builds

mgr/dashboard: add traddr param to nvmeof cli/api endpoints

Signed-off-by: Tomer Haskalovitch <il033030@Tomers-MBP.lan>

Merge pull request #62626 from aainscow/have_significant_feature

osd: Introduce macro to police use of significant features.

cls/rbd: drop overzealous CLS_ERR message in mirror_remote_namespace_get()

Currently it unnecessarily floods the log of the OSD which hosts
rbd_mirroring object with "No such file or directory" errors.  Just
drop it as read_key() already logs all errors except ENOENT.

Credit to N Balachandran <nibalach@redhat.com> for spotting this.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #62589 from ceph/wip-yuriw-18.2.5-release.notes-main

doc: reef 18.2.5 release notes

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #62734 from ceph/wip-yuriw-19.2.2-release.notes-main

doc: squid 19.2.2 release notes

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

buildcontainer-setup.sh: Fix curl package conflict

It looks like at some point the centos9 image started shipping with
curl-minimal, which conflicts with the regular curl package. Asking dnf to find
the binary avoids this, since both packages provide it. Since we were already
doing this with rpmbuild, we can go ahead and loop wget into that in case
something similar happens there.

Signed-off-by: Zack Cerza <zack@cerza.org>

build-with-container.py: Pass distro to make-debs

Signed-off-by: Zack Cerza <zack@cerza.org>

make-debs.sh: Optionally take debian version

Our existing CI builds have names like:
  ceph-base_20.0.0-194-g6efaea33-1jammy_amd64.deb
Before this change, they are like:
  ceph-base_20.0.0-158-gb0de3a42-1_amd64.deb
This way we can pass e.g. "jammy" to end up with names compatible with our CI
builds.

Signed-off-by: Zack Cerza <zack@cerza.org>

debian/rules: Optionally enable sccache

Signed-off-by: Zack Cerza <zack@cerza.org>

debian/rules: Disable dh_dwz if DWZ=false

Signed-off-by: Zack Cerza <zack@cerza.org>

make-debs.sh: Don't use ccache if using sccache

Signed-off-by: Zack Cerza <zack@cerza.org>

build-with-container.py: Pass version to make-debs

Signed-off-by: Zack Cerza <zack@cerza.org>

make-debs.sh: Optionally avoid make-dist

Signed-off-by: Zack Cerza <zack@cerza.org>

doc: squid 19.2.2 release notes

https://tracker.ceph.com/issues/70822
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #62759 from bluikko/doc-prompts-ceph-volume

doc/ceph-volume: Promptify commands and fix formatting

Merge pull request #62663 from adamemerson/wip-tbb-O0

build: Pull TBB into `$EXTRALIBS` if it is available

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #62669 from adamemerson/wip-allocsim-replayer-assert

test/objectstore/allocsim: Fix link error

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #62753 from gbregman/main

mgr/cephadm/nvmeof: Add OMAP read lock parameters to NVMEoF configuration

Merge pull request #62762 from bluikko/patch-23

doc/cephfs: Fix double prompts in commands in multimds.rst

doc/cephfs: Fix double prompts in commands in multimds.rst

Some example commands had double prompts: one `#`
from the `.. prompt` and after that another hardcoded
`$` character.

Remove the extra user prompt and leave only the root
prompt from the `.. prompt` block.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/ceph-volume: Promptify commands and fix formatting

Use the more modern prompt block for CLI
commands, fix missing newline and messed up
line breaks.
Also change existing prompts to all indent with
same amount of spaces.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

mgr/dashboard: add description for default Pool Application select options

Add a description for the default pool application values in the pool creation form i.e cephfs <-> FileSystem, rbd <-> Block, rgw <-> Object.

Fixes: https://tracker.ceph.com/issues/70866
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #62633 from cyx1231st/wip-seastore-le-map-cleanup

crimson/os/seastore: cleanups related to lba and cache

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Zhang Song <zhangsong02@qianxin.com>

Merge pull request #62662 from adamemerson/wip-amqp-deprecation

rgw: Fix deprecated amqp includes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62580 from Matan-B/wip-matanb-findboost-warning

cmake/modules/FindBoost.cmake: fix Boost_NO_WARN_NEW_VERSIONS

Reviewed-by: Adam Emerson <aemerson@redhat.com>

doc/releases: fix formatting for backticks

Signed-off-by: Laura Flores <lflores@ibm.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #62451 from bill-scales/ec_data_structs

osd: EC optimizations: new types and additions to data structures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61652 from ifed01/wip-ifed-print-extra-level

os/bluestore: print the first RocksDB level which doesn't fit into fast

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>

mgr/cephadm/nvmeof: Add OMAP read lock parameters to NVMEoF configuration

Fixes https://tracker.ceph.com/issues/70861

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #57206 from clwluvw/restrict-public-buckets

rgw: implement RestrictPublicBuckets from PublicAccessBlock

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62538 from cbodley/wip-rgw-control-watch-par

rgw: send concurrent watch/unwatch operations

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62159 from cbodley/wip-qa-23264

qa/rgw: use storage classes for rgw/crypt

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62706 from cbodley/wip-70729

test/rgw/multisite: fix parsing of 'topic list' output

Reviewed-by: Ali Masarwa <amasarwa@redhat.com>

Merge pull request #62686 from Cybertinus/main

documentation:remove deprecated usage apt-key

documentation: Update get-packages.rst

The usage of apt-key is deprecated, thus updated the documentation to reference the new way of installing a repo key.

Signed-off-by: Cybertinus <github@cybertinus.nl>

Merge pull request #62727 from idryomov/wip-70831

test/librbd/test_notify.py: force line-buffered output

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>

Merge pull request #57448 from aclamk/wip-aclamk-bs-recompression-segmented-data

os/bluestore: Recompression, part 3. Segmented onode.

Merge pull request #62224 from aclamk/wip-aclamk-pere-wal-fsync

os/bluestore: Fast WAL for RocksDB

Merge pull request #62588 from aclamk/wip-aclamk-bluefs-remove-truncate

os/bluestore: Fix race in BlueFS truncate / remove

mgr/dashboard: Add RGW topic listing in dashboard

Fixes: https://tracker.ceph.com/issues/69143
Signed-off-by: pujaoshahu <pshahu@redhat.com>

Merge pull request #61909 from bazaah/chore/deprecated-iterator-impl

include,dpdk: fix two usages of std::iterator (deprecated)

qa/rgw: use storage classes for rgw/crypt

enable compression on one storage class so we can test copy to/from
compressed objects

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: storage classes specify extra args in yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #60670 from cbodley/wip-cls-rgw-shard-io

rgw/rados: add async algorithms for concurrent index operations

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #62720 from tchaikov/wip-kefu-mailmap

mailmap: update affiliation for Kefu Chai

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>