Matt Benjamin [Tue, 24 Oct 2023 21:59:34 +0000 (17:59 -0400)]
rgwlc: implement Filter size limit checks
i.e., this implements ObjectSizeGreaterThan and ObjectSizeLessThan
for supported expiration and transition actions.
Fixes: https://tracker.ceph.com/issues/63304 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit c9b19be15362c63526e92f4d968f942a40cab150)
Matt Benjamin [Mon, 23 Oct 2023 18:57:33 +0000 (14:57 -0400)]
rgwlc: implement NewerNoncurrentVersions
Per AWS doc, this value controls "how many noncurrent versions
Amazon S3 will retain." [1] We understand this to mean, retain
NewerNoncurrentVersions of any object, regardless of expiration.
Ilya Dryomov [Wed, 28 Feb 2024 13:20:16 +0000 (14:20 +0100)]
librbd: don't clip expanded diff on truncate in ObjectListSnapsRequest
If the diff was expanded due to LIST_SNAPS_FLAG_WHOLE_OBJECT, clipping
it when handling a truncate is wrong -- when subtracting that interval,
we either split the expanded extent into two or chop off a piece of it.
However the point of LIST_SNAPS_FLAG_WHOLE_OBJECT is to report a single
extent covering the entire object.
Ilya Dryomov [Sun, 18 Feb 2024 10:46:15 +0000 (11:46 +0100)]
librados/snap_set_diff: ignore truncates above size at start
Because currently calc_snap_set_diff() only ever appends to the running
diff, an excessive (either too large or completely bogus) zero extent
is reported in cases where an object is first expanded (with a snapshot
taken at that point) and then truncated but still above the size of the
object as of the starting snapshot.
J. Eric Ivancich [Mon, 11 Mar 2024 21:19:40 +0000 (17:19 -0400)]
rgw: rgw-restore-bucket-index -- sort uses specified temp dir
The sort command sometimes makes use of temporary files. When the user
specifies a directory to be used for temp files, have the sort command
use that same directory.
Matt Benjamin [Mon, 19 Feb 2024 14:01:48 +0000 (09:01 -0500)]
rgw_lc: replace strftime w/fmt and chrono:calendar
It's reliably claimed that std::strftime is not
mt-safe, and this would be a likely root cause of
intermittent scrambled expiration header output cases
that have been reported.
Fixes: https://tracker.ceph.com/issues/63973 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 96a5cbb9aa47ed1a2b51e720475ebc08012cb4cc)
Seena Fallah [Tue, 20 Feb 2024 14:54:06 +0000 (15:54 +0100)]
rgw: return 204 on bucket public access block deletion
According to AWS doc (https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeletePublicAccessBlock.html#API_DeletePublicAccessBlock_ResponseSyntax) 204 shall be returned on delete request.
Zac Dover [Thu, 7 Mar 2024 03:01:47 +0000 (13:01 +1000)]
doc/start: add Slack invite link
Add a link to the ceph-storage Slack invitation page. Previously the
link went to a plain old "this is the ceph-storage Slack" page that did
not direct the reader to sign up.
Mark Kogan [Tue, 5 Mar 2024 14:48:56 +0000 (16:48 +0200)]
rgw/beast: enablment of SSL session-id reuse speedup mechanism
Enable the OpenSSL session-id reuse acceleration mechanism that is described in:
https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_session_id_context.html
SSL_CTX_set_session_id_context, SSL_set_session_id_context - set context
within which session can be reused (server side only)
This experimental tool write a series of temporary files, the combined
size of which is roughly proportional to some combination of the
number of objects in the bucket and their respective sizes. Previously
the script stored all temporary files in /tmp, which may not be
sufficiently large in many cases.
There are now two enhancements. First, using the "-t" command-line
argument, the directory for temporary files can be specified. Second,
if the script consumes all available space on the file system
containing the temporary directory, an error message to that effect is
displayed, so the user understands what corrective action to take.
Although this seems to be a bug from RedFish, we need to handle
the case when it happens otherwise it makes the mgr orchestrator module
throw an error.
The idea here is to create a new status "unknown" when we can't fetch the
real status of a component.
Zac Dover [Mon, 4 Mar 2024 10:41:16 +0000 (20:41 +1000)]
doc/rados: link to pg setting commands
Link to the instructions for manually setting the number of PGs per
pool, from the mention of placement groups. These instructions are
included here in response to a request from Ronen Friedman on the
occasion of the removal of links to the PGcalc (see
https://github.com/ceph/ceph/pull/55899#pullrequestreview-1912940118).
Zac Dover [Sun, 3 Mar 2024 10:28:00 +0000 (20:28 +1000)]
doc/rados: remove PGcalc from docs
Remove mention of the "PG calc" tool from the documentation. I have
removed all mention of this in one fell swoop to help posterity restore
mention of this tool if we decide we need to do so.
The rbd-wnbd daemon currently caches one rados context per cluster.
However, it's registering hooks against the global context
admin socket, which won't be available. For this reason,
the "rbd-wnbd stats" command no longer works.
To address this issue, we'll ensure that rbd-wnbd sets command hooks
against the right admin socket instance, leveraging the image
context.
The "rbd-wnbd unmap" command is currently telling the WNBD driver
to remove the mapping without contacting the rbd-wnbd daemon
and waiting for it to perform its cleanup.
For this reason, attempting to delete the image immediately after
unmapping it can fail due to existing watchers.
As a temporary solution, we'll retry the image remove operation.
At a later time, we'll update the "rbd-wnbd unmap" command to go
through the rbd-wnbd daemon, ensuring that all the necessary
cleanup is performed before returning.
While at it, we're dropping a redundant LOG.error call so that we
won't print expected exceptions.
This commit will store the mapping config in the Windows registry
only after initializing the mapping. This ensures that we aren't
replacing the registry settings for already mapped images.
We'll also check if the registry setting was added by us before
cleaning it up.
Lucian Petrut [Mon, 12 Jun 2023 13:16:39 +0000 (13:16 +0000)]
rbd-wnbd: use one daemon process per host
We're currently using one rbd-wnbd process per image mapping.
Since OSD connections aren't shared across those processes,
we end up with an excessive amount of TCP sessions, potentially
exceeding Windows limits:
https://ask.cloudbase.it/question/3598/ceph-for-windows-tcp-session-count/
In order to improve rbd-wnbd's scalability, we're going to use
a single process per host (unless "-f" is passed when mapping the
image, in which case the daemon will run as part of the same
process). This allows OSD sessions to be shared across image
mappings.
Another advantage is that the "ceph-rbd" service starts faster,
especially when having a large number of image mappings.
Zac Dover [Fri, 1 Mar 2024 12:11:14 +0000 (22:11 +1000)]
doc/install: add manual RADOSGW install procedure
Add a manual RADOSGW installation procedure to
doc/install/manual-deployment.rst. This procedure was developed by Janne
Johansson and reported to the ceph-users mailing list on 29 Jan 2024
here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/LB3YRIKAPOHXYCW7MKLVUJPYWYRQVARU/
Co-authored-by: Janne Johansson <icepic.dz@gmail.com> Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 565bc9503838906995fa48f59debcd2843775b18)
daijufang [Mon, 4 Dec 2023 07:46:29 +0000 (07:46 +0000)]
src/rgw: fix for the multipart interface in the WORM function
1. Save the WORM configuration information in the initialization chunk information for use when merging chunks.
2. Support x-amz-bypass-governance-retention when merging chunks.
Fixes: https://tracker.ceph.com/issues/63724 Signed-off-by: daijufang <daijufang_yewu@cmss.chinamobile.com>
(cherry picked from commit 3a65a0ad14fb3c4d836441a417d594826fb6e711)
Samuel Just [Fri, 16 Feb 2024 00:04:05 +0000 (00:04 +0000)]
unittest-seastar-socket: debug to error on unexpected return from dispatch_rw_bounded
Related: https://tracker.ceph.com/issues/64457 Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 66969c07dc0cd5e0f01685ec19de26dae09279f5)
projects / ceph.git / log