when a forwarded request fails on the master zone, the local zone should
return that same error response back to the client. this means
reproducing both the http error and the aws xml <Error> response
rgw_forward_request_to_master() stores these errors in s->err, and
set_req_state_err() now avoids overwriting existing an error
rgw/rest: RGWRESTConn::forward() prefers to return http errors
callers need to distinguish between transport errors (a failure to
forward the request) and http errors (successfully forwarded and got a
response). forward() was losing this information by mapping any http
errors to errnos
use tl::expected to differentiate between transport errors and http
errors, with the latter being the successful/expected case
Patrick Donnelly [Mon, 28 Apr 2025 13:58:02 +0000 (09:58 -0400)]
Merge PR #61796 into main
* refs/pull/61796/head:
rgw/sts: Use client_id for assumerolewithwebidentityresponse
rgw/sts: adding validation of jwks_uri cert according
rgw/sts: fix to pick jwk which is of type
rgw/sts: adding code for JWT signature validation
Ville Ojamo [Mon, 28 Apr 2025 10:02:32 +0000 (17:02 +0700)]
doc/radosgw: Promptify commands and improve formatting in keystone.rst
Use blocks with bash privileged command prompt for CLI
command examples. Separate example command output to a
preformatted block. Previously a hard-coded prompt in
some place inconsistently while no prompts in others.
Janne Heß [Mon, 28 Apr 2025 09:04:25 +0000 (11:04 +0200)]
ceph-volume: Fix splitting with too many parts
The data can be anything and also contain a `=`, causing the line to
fail with `Too many values to unpack`. In my case, it failed with
`ID_FS_LABEL=pvc_name=rook-ceph-lvm-data-44f2gc`.
Regression was introduced here: https://github.com/ceph/ceph/pull/60006
Fixes: https://tracker.ceph.com/issues/71101 Signed-off-by: Janne Heß <janne@hess.ooo>
Replace `boost::variant` with `std::variant` as part of our effort to reduce
third-party dependencies in favor of C++ standard library alternatives.
Benefits include:
- Improved code readability and maintainability
- Reduced external dependency surface
- More consistent API usage with other components
rgw/qa: added test case to assume a role after role creation
syncs, and then creating a bucket on both primary and secondary.
The test name is test_assume_role_after_sync.
rgw/sts: by-passing authentication using temp creds
in case the request is forwarded from secondary in
a multi-site setup. authenticating with the system
user creds of which are used to sign the request.
Permissions are still derived from the role.
rgw/sts: adding validation of jwks_uri cert according
to https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html
for n&e which can be later used for all key types
(x5c, n&e).
Ville Ojamo [Sat, 26 Apr 2025 03:54:12 +0000 (10:54 +0700)]
doc/radosgw: Improve formatting in STS.rst
Some text uses hardcoded ordered list numbers and as a result
have duplicate list numbers. Move to use RST ordered lists and
indent the contents of each list item correctly. Fix referenced
list item number in text that pointed to a wrong list item.
Format references to parameter names, user IDs, file names etc
as inline code.
Add prompts to example commands.
Change instances of "rgw" or "Radosgw" in text to "RGW".
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ville Ojamo [Sat, 26 Apr 2025 04:17:16 +0000 (11:17 +0700)]
doc/radosgw: Fix RST syntax rendeded as text in oidc.rst
Empty line after starting a pre-formatted block with the double-colon
syntax is required, otherwise the double-colon does nothing and is just
rendered as-is as "::" and there would be no following pre-formatted
block.
Add empty lines after the double-colon syntax so that the following
block is rendered pre-formatted.
Also add bash privileged prompts to a block with 2 example CLI commands.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ville Ojamo [Sat, 26 Apr 2025 03:30:57 +0000 (10:30 +0700)]
doc/radosgw: Fix length of section header underlines in oidc.rst
Set the length of the RST syntax underlining for section headers to be
as long as the section header text.
Also it seems common to indent the document title with one space at both
the beginning and the end so do that.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Patrick Donnelly [Fri, 25 Apr 2025 19:00:39 +0000 (15:00 -0400)]
Merge PR #62833 into main
* refs/pull/62833/head:
qa: test charmap changes with dir and snaps
mds: check for snapshots on parent snaprealms
mds: use strict_strtobool for parsing bools
common: take string_view for strict_tobool
this change is created in the same spirit of bb1fa818.
when building the tree with clang-21, following warning was raised:
```
/home/kefu/dev/ceph/src/libcephfs_proxy/proxy_async.c:43:9: warning: arithmetic on a pointer to void is a GNU extension [-Wgnu-pointer-arith]
43 | data += iov->iov_len;
| ~~~~ ^
1 warning generated.
```
this change should address this warning by casting a `void*` pointer to
`char*` pointer before performing arithmetic operation on it.
Ville Ojamo [Fri, 25 Apr 2025 09:00:20 +0000 (16:00 +0700)]
doc/radosgw: Fix section header level in config-ref.rst
The section named "QoS Settings" looks like it should not be a child of
the section "SSE-S3 Settings". Move it to be a sibling to it, on the
same level instead.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ville Ojamo [Fri, 25 Apr 2025 07:16:52 +0000 (14:16 +0700)]
doc/radosgw: Fix section header levels in multisite-sync-policy.rst
The section header levels are reversed so the hierarchy in the TOC is
incorrect. Switch around the section header levels to make the TOC
hierarchy correct, for example individual examples are children of the
"Examples" section.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
src/mon/PGMap.cc: check unfound obejcts in `get_unavailable_pg_in_pool_map`
If a pool has any PG with unfound objects, we should consider
it unavailable for the availability score. If a PG has unfound
objects, it will be recorded in PGMap.
In `get_unavailable_pg_in_map`, if a PG has unfound obejcts,
we add it to `pool_pg_unavailable_map`.
When we update the `pg_stat` we don't
check whether the pg state is in `stale`.
Therefore, the attribute `last_unstale`
will always get updated even if the pg
state actually contains `stale`.
Solution:
Place a condition to only update
the attribute `last_unstale` when
we the pg truly doesn't have `stale`
in its state.
Kamoltat [Thu, 26 Oct 2023 19:08:37 +0000 (19:08 +0000)]
src/mon/PGMap.cc: init pool_availability
Added PoolAvailability Struct
Modified PGMap.cc to include a k,v map:
`pool_availability`.
The key being the `poolid` and value
is `PoolAvailability`
Init the function:
`PGMap::get_unavailable_pg_in_pool_map()`
to identify and aggregate all the PGs we
mark as `unavailable` as well as the pool
that associates with the unavailable PG.
Also, included `pool_availability`
to `PGMapDigest::dump()`.
Adam Kupczyk [Mon, 10 Jun 2024 16:03:24 +0000 (16:03 +0000)]
os/bluestore: Add admin socket commands to inspect onode metadata
Add admin socket commands:
1) bluestore collections
Lists collections.
2) bluestore list <coll> [start object] [max count]
Lists collection coll starting from object (optional). Default 100 entries. 0 = unlimited.
3) bluestore onode metadata <object>
Prints onode metadata as seen by BlueStore.
It might happen (usually in tests) that 2 BlueStore instances are created at the same time.
Since admin commands are unique, it fails to register.
Use first register to detect whether we can register at all.
Adam Kupczyk [Tue, 8 Apr 2025 08:36:21 +0000 (08:36 +0000)]
os/bluestore: Add do_write_v2_compressed()
Modify do_write_v2() to branch into do_write_v2_compressed().
Segmented and regular cases are recognized and handled properly.
New do_write_v2_compressed() oversees compression / recompression.
Make one Estimator per Collection.
It makes possible for estimator to learn in collection specific compressibility.
In write_v2_compressed use compressor already selected in choose_write_options.
Make Collection create Estimator on first use.
Adam Kupczyk [Tue, 8 Apr 2025 11:03:22 +0000 (11:03 +0000)]
os/bluestore/compression: Main part of recompression feature
Add feature of recompression scanner that looks around write region to see how much
would be gained, if we read some more around and wrote more.
Added Compression.h / Compression.cc.
Added debug_bluestore_compression dout.
Created Scanner class.
Provides write_lookaround() for scanning loaded extents.
Adam Kupczyk [Wed, 9 Apr 2025 16:03:52 +0000 (16:03 +0000)]
os/bluestore/compression: Estimator class
Add CMake rules to compile.
Add bluestore_compression dout subsys.
Created Estimator class.
It is used by Scanner to decide if specific extent is to be recompressed.
Prepare for future machine learning / adaptive algorithm for estimation.
So far logic of Estimator is relatively simple.
It learns expected recompression values and uses them in next iterations to predict.
projects / ceph.git / log