Merge pull request #62851 from cbodley/wip-rgw-beast-async-disable

rgw: frontend reads/writes respect rgw_beast_enable_async

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge PR #63083 into main

* refs/pull/63083/head:
doc/cephfs: add snapshot name note

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>

doc/cephfs: add snapshot name note

The charmap is not applied to snapshot names!

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #63056 from ronen-fr/wip-rf-fadvise

osd/scrub: making Scrub's fadvide flags a constant

Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>

Merge pull request #62805 from Matan-B/wip-matanb-rep-read-perfcount

osd/PrimaryLogPG: Relax replicated reads when recovering

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #62866 from ifed01/wip-ifed-osd-bench-more-logging

osd: be more verbose in OSD::run_osd_bench_test

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #63041 from cbodley/wip-qa-rgw-upgrade-goodbye-quincy

qa/rgw: prepare for tentacle testing

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #62827 from afreen23/bump-version

mgr/dashboard: Bump grafana version to 11.6.0

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #63057 from bluikko/doc-rgw-capitalization-radosgw

doc/radosgw: Improve language, capitalization and use config database

doc/radosgw: Improve language, capitalization and use config database

Use "RADOS Gateway" instead of "Rados Gateway", "rados gateway" etc.
I am aware of the term "Ceph Object Gateway" but this change intends to
be an uncontroversial low hanging fruit fix of obviously incorrectly
capitalized terms.

Use "RGW daemon" instead of "Gateway", "Rados Gateway" etc.
Use "RGW instance" instead of "rados gateway" for consistency with
exactly similar other instance.
If referring obviously clearly to an instance of the daemon with an
obviously not preferred term, change it to "RGW daemon"; for example
when talking about restarting the RGW.
Do not touch other instances that are not 100% clear.

The files touched mostly do not use "Ceph Object Gateway" so changing
the term to it would create inconsistency, or several more changes
would need to be done to update all instances to use this terminology.

Use configuration database instead of ceph.conf in d3n_datacache.rst.

Improve language in d3n_datacache.rst.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #63064 from bluikko/doc-metrics-inlinecode-radosgw

doc/radosgw: Use inline code for label syntax in metrics.rst

Merge pull request #60515 from ideepika/fix-68327

rgw: make keystone work without admin token(service ac requirement)

Reviewed-by: Tobias Urdin <tobias.urdin@binero.com>

Merge pull request #62980 from cbodley/wip-71083

rgw/lc: stop using merge_and_store_attrs in remove_bucket_config

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

doc/radosgw: Language and use inline code for labels in metrics.rst

Use inline code formatting for label syntax examples inside text.

Small improvements to language. Use all lower case for labels.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge PR #63046 into main

* refs/pull/63046/head:
qa/crontab: add tentacle nightlies

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

osd/scrub: making Scrub's fadvide flags a constant

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #63060 from zdover23/wip-doc-2025-04-30-cephadm-services-rgw-markup

doc/cephadm: correct markup in rgw.rst

Merge pull request #63062 from bluikko/doc-radogw-fix-radosgw

doc/radosgw: Use Ceph Object Gateway as term in metrics.rst

Merge pull request #62843 from rishabh-d-dave/vols-user-pool

mgr/vol: don't delete user-created pool in "volume create" command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #62760 from mohit84/pg_stuck_backfill

crimson: PG backfill is not showing any progress

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

doc/radosgw: Use Ceph Object Gateway as term in metrics.rst

The document uses mostly the term Ceph Object Gateway except in three
places where "Radosgw", "rgw" or obviously typod "radogw" is used.

Change those instances to Ceph Object Gateway consistent with the rest
of the document.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/cephadm: correct markup in rgw.rst

Correct the presentation of an example string in doc/cephadm/rgw.rst in
order to obviate an error reading "rgw.rst:202: WARNING: Inline emphasis start-string without end-string."

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #62894 from rzarzynski/wip-crimson-unused-errors

crimson: fix unused variable warnings due to assert() and NDEBUG builds

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #62826 from sseshasa/wip-doc-max-iops-cap-override

doc/rados: Update mClock doc on steps to override OSD IOPS capacity config

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

doc/rados: Update mClock doc on steps to override OSD IOPS capacity config

Describe the steps involved to
 - Specify a global value for osd_mclock_max_capacity_iops_{ssd,hdd}, and
 - Override existing individually scoped values for OSDs determined during
   start-up for osd_mclock_max_capacity_iops_{ssd,hdd}.

The above is to help with the following:
 - Steps to override existing setting with a global value.
 - reduce the number of entries in the mon store and instead use a single
   global specification for all OSDs in the cluster in case the underlying
   hardware is the same for all OSDs.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Fixes: https://tracker.ceph.com/issues/70774

Merge pull request #62972 from laimis9133/laimis9133-compression-docs

doc/radosgw/compression: separate RGW and RADOS pool level compression

Merge pull request #62836 from athanatos/sjust/wip-crimson-repop-reply-ordering-69439

crimson: osd_operation cleanups and fix for MOSDRepOpReply ordering

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge branch 'main' into laimis9133-compression-docs

Signed-off-by: Laimis Juzeliūnas <58551069+laimis9133@users.noreply.github.com>

Merge pull request #59206 from ygtzf/bugfix-compress-use-isal

compressor: compressor_zlib_isal did not take effect in compression

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

qa/crontab: add tentacle nightlies

And delete reef nightlies. This is primarily because we do not have capacity to also test reef.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #62904 into main

* refs/pull/62904/head:
pybind/mgr/volumes: make casesensitive attr uniform in interface

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #63035 from Matan-B/wip-matanb-crimson-scan_for_backfill-fix

crimson/osd/recovery_backend: scan_for_backfill_primary correctly handle missing object

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #63033 from bluikko/doc-placement-formatting-radosgw

doc/radosgw: Promptify cmds and improve formatting in placement.rst

Merge pull request #63028 from bluikko/doc-d3n-formatting-radosgw

doc/radosgw: Improve formatting in d3n_datacache.rst

Merge pull request #63029 from bluikko/doc-admin-privprompts-radosgw

doc/radosgw: Use privileged prompt for CLI commands in admin.rst

Merge pull request #63032 from bluikko/doc-compression-typo-radosgw

doc/radosgw: Remove stray full stop mid-sentence in compression.rst

Merge pull request #62672 from adamemerson/wip-test-common-signed-comparison

test/common: Fix signed comparison

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #62670 from adamemerson/wip-not-before-queue-signed-comparison

common/not_before_queue: Fix signed comparison warning

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

qa/rgw/upgrade: add upgrade from tentacle

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw/upgrade: add upgrade from squid

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw/upgrade: remove upgrade from reef

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #61962 from clwluvw/replication-perms

rgw: add support replication actions in policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/rgw/upgrade: remove upgrade from quincy

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62922 from saif-0987/refactor/testid-update-01

mgr/dashboard: Replace data-cy with data-testid for cypress IDs

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #63010 from ronen-fr/wip-rf-repair-62451

qa/standalone/scrub: fix expected outputs in repair tests

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

crimson/osd/recovery_backend: scan_for_backfill_primary correctly handle
missing object

scan_for_backfill was seperated to scan_for_backfill_primary and
scan_for_backfill_replica.
The fix from:
https://github.com/ceph/ceph/pull/62837/commits/88432ebd7432c513ccd495e77425401beddb9953
was only copied to the replica version.

Fixes: https://tracker.ceph.com/issues/71124
Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #62978 from afreen23/main

mgr/dashboard: Update translations

Reviewed-by: Nizamudeen A <nia@redhat.com>

doc/radosgw: Promptify cmds and improve formatting in placement.rst

Use preformatted blocks with a privileged bash prompt instead of
hardcoding prompts in the beginning of each line for CLI commands.

Indent continuation lines of multi-line CLI example commands the same
way they are indented elsewhere.

Use inline code formatting consistently, add double-backticks for
inside text references to CLI commands, configuration data, etc.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/radosgw: Remove stray full stop mid-sentence in compression.rst

Remove a full stop that seems to be a typo in the middle of a sentence.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/radosgw: Use privileged prompt for CLI commands in admin.rst

Instead of not defining a prompt to use in CLI commands and falling back
to the default unprivileged prompt, use explicit privileged bash prompt
for CLI commands that require privileges.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

doc/radosgw: Improve formatting in d3n_datacache.rst

Change to a full stop one comma that is followed by capital
case and looks like a separate sentence otherwise too.

Add missing inline code formatting consistently for file
names, config data, etc.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

qa/standalone/scrub: fix expected output in snaps repair tests

Specifically - TEST_corrupt_snapset_scrub_rep in osd-scrub-repair.sh.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

crimson: add operation wrapper for MOSDRepOpReply

This should avoid reordering between cores.

Fixes: https://tracker.ceph.com/issues/69439
Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: convert cross-core operations to use RemoteOperation

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: fix DynamicPerfStats usage in ClientRequest

ClientRequest::get_connection() return l_conn, which will be
null by the time PG::add_client_request_lat is called in
ClientRequest::do_process.  Modify get_connection() to
return a Connection& from whichever of l_conn or r_conn
isn't null.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../osd_operation.h: add RemoteOperation

Subsequent commits will switch various ops to inherit from
this thereby removing some boilerplate.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../osd.cc: convert active ops to start_pg_operation_active

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/.../pg_shard_manager: add start_pg_operation_active

Messages between OSDs for PGs that have already completed peering
require fewer checks than otherwise.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge PR #62872 into main

* refs/pull/62872/head:
qa: add test for cloning with charmap
pybind/mgr/volumes: fix typo in casesensitive vxattr

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Xavi Hernandez <xhernandez@gmail.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #62089 into main

* refs/pull/62089/head:
doc/dev/release-checklists: add vX.3.0 checklist item
doc/dev/release-checklist: check v20.0.0 tag

Reviewed-by: Adam King <adking@redhat.com>

Merge PR #63011 into main

* refs/pull/63011/head:
.github: run verify-qa from base branch
 .github: run verify-qa when PR HEAD is updated

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: utilize is_impersonating for forwarded sts requests

With the introduction of is_impersonating in SysReqApplier,
RoleApplier can now use the same mechanism to mark when a request
has been forwarded by a system user on behalf of another role (e.g.,
through STS) to mark it as a system request (s->system_request).

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

.github: run verify-qa from base branch

If the PR does not have the script, perhaps it does not run?

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

 .github: run verify-qa when PR HEAD is updated

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

rgw: dont rate limit forwarded requests

rely on s->system_request to skip rate limiting on forwarded requests
as well as normal system user requests.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: use is_admin() for permission checks

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: override perms for admin on data sync

If pipe is in user mode and the user is admin, don't check for perms
and let it go.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: change is_admin_of() to is_admin()

As admin propery of a user is something global and nothing related
to any other owner, we don't need any comparision.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: make rgw_sync_pipe_params::user optional

In rgw_sync_pipe_params, the mode can be either system or user.
When in system mode, no user is involved, but the current
implementation holds an empty rgw_user, which can cause confusion
in pipe_rules::find_basic_info_without_tags().

With this change, rgw_user is now optional, ensuring that when no
user is involved, it is explicitly nullopt rather than an empty object.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

qa/rgw: add perm check test for copy obj between zonegroups

Make sure perms are evaluated properly for the source object.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

doc: add release note for new policy actions on replication

Fixes: https://tracker.ceph.com/issues/70093
Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: remote copy obj pass rgwx-perm-check-uid for perm evaluation

When copying object from remote source (bucket from another zonegroup)
the perms of the source is not evaluated resulting in reading from
unauthorized buckets.
passing `rgwx-perm-check-uid` will let the source zone evaluates the
perm and close this bug.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: RGWRadosPutObj evals source bucket perm for backward compatibility

As of a3f40b4 we no longer evaluate perms locally for source bucket,
this could cause broken permission evaluation dusring upgrade as one
zone is not respecting the perm evaluation based on the `rgwx-perm-check-uid`
arg.

This can be dropped in T+2 release.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: make verify_bucket_permission functions const

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: give hint via header for perm evaluation in GetObj

Return `Rgwx-Perm-Checked` header as a hint for the destination zone
to know whether the perms where considered or not.
This is just a backward compatibility for upgrade and can be dropped
in T+2 release.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: rest client callback when all headers are passed

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: pass rgwx-perm-check-uid for multisite fetch object

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: GetObject(Version) not allowed to replicate sse-kms objects

To replicate objects encrypted via sse-kms objects,
s3:GetObjectVersionForReplication is required.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: take account GetObject(Version)Tagging when replicating

In case the uid has no permission to read tagging, the tags should
not be replicated.
Ref. https://docs.aws.amazon.com/AmazonS3/latest/userguide/setting-repl-config-perm-overview.html

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

qa/rgw: add test for source object perm check in multisite

Check whether the policies are honored on source object in source
zone when replicating.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: replication require lock perm if enabled

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: check source object replication by replication actions

Check for permissions of `s3:GetObjectVersionForReplication` in
addition to `s3:GetObject` and `s3:GetObjectVersion` when fetching
the object for multisite.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: export action_bit_string through header file

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: only allow system override if identity is not impersonating

Since multisite now delegates permission checks for source objects
to the source zone (a3f40b4), we need to avoid allowing system-level
overrides when the request is impersonating another identity.

SysReqApplier should only grant override permission if the request
is truly system-authenticated and not acting on behalf of another
user or role (i.e., no rgwx-perm-check-uid or rgwx-perm-check-role
in the request).

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: SysReqApplier overrides is_admin_of based on impersonation

SysReqApplier now returns true for is_admin_of() when the requester
was a system user and was not impersonating any user/role using
rgwx-perm-check-uid or rgwx-perm-check-role.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

qa/rgw: add test for new replication actions

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: support s3ReplicateTags perm on destination bucket for replication

Check for tag replication permission on destination bucket, so if
there was an explicit deny, donot include tags in the replicated
object.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: check for s3ReplicateObject perm on destination bucket for replication

Instead of s3:PutObject rely on s3:s3ReplicateObject permission to
check whether the user can replicate to the destination bucket.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: verify perm on delete replication

Check for s3:ReplicateDelete for replicating object deletes and
delete markers when pipe is set to user mode.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: move RGWUserPermHandler to header

So it can be used by others.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: weaning off RGWUserPermHandler from RGWDataSyncEnv

So it can be called by RGWAsyncRadosRequest classes not holding
sync_env.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: send bucket sync structs to bucket_sync.h

So it can be imported by headers like rgw_cr_rados.h that already
has dependency to rgw_data_sync.h.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

rgw: drop unused params passed to RGWStatRemoteObjCR by RGWObjFetchCR

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

Merge pull request #56576 from pritha-srivastava/wip-rgw-assume-role-multisite

rgw/sts: correcting authentication in case s3 ops are directed to a primary from secondary after assumerole.

qa/standalone/scrub: fix expected output in replicated repair tests

Specifically - TEST_corrupt_scrub_replicated in osd-scrub-repair.sh.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #63004 from dasJ/fix/ceph-volume-split

ceph-volume: Fix splitting with too many parts

Merge pull request #63008 from bluikko/doc-compression-promptify-radosgw

doc/radosgw: Promptify CLI commands in compression.rst

Merge pull request #63007 from bluikko/doc-keystone-formatting-radosgw

doc/radosgw: Promptify commands and improve formatting in keystone.rst

Merge pull request #63006 from bluikko/doc-bucketpolicy-formatting-radosgw

doc/radosgw: Improve formatting in bucketpolicy.rst

doc/radosgw/compression: separate RGW and RADOS pool level compression

Add notes indicating a difference between compressions done by RGW and on RADOS pool level for better understanding.

Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>
Co-authored-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>