Ilya Dryomov [Sun, 2 Mar 2025 08:24:52 +0000 (09:24 +0100)]
librbd: fix a deadlock on image_lock caused by Mirror::image_disable()
With Mirror::image_disable() taking image_lock for write and calling
list_children() under it, the following deadlock is possible:
1. Mirror::image_disable() takes image_lock for write and calls
list_children()
2. AbstractWriteLog::periodic_stats() timer fires (it runs every
5 seconds) and ImageCacheState::write_image_cache_state() is called
under a global timer_lock
3. ImageCacheState::write_image_cache_state() successfully takes
owner_lock and blocks attempting to take image_lock for read because
it's already held for write by Mirror::image_disable()
4. list_children() blocks inside of a call to ImageState::close() on
a descendant image
5. The descendant image close can't proceed because TokenBucketThrottle
requires a global timer_lock to complete QosImageDispatch shutdown
6. safe_timer thread which is holding timer_lock can't proceed because
ImageCacheState::write_image_cache_state() is effectively blocked on
the descendant image close through Mirror::image_disable()
Until commit 281a64acf920 ("librbd: remove snapshot mirror image-meta
when disabling"), Mirror::image_disable() was taking image_lock only for
read meaning that this deadlock wasn't possible. The only other change
that commit 281a64acf920 made to the code block protected by image_lock
was using child_mirror_image_internal for cls_client::mirror_image_get()
call on descendant images instead of mirror_image_internal to preserve
the value of mirror_image_internal for later. Both are local variables
that have nothing to do with image_lock, so I'm going back and making
Mirror::image_disable() take image_lock only for read again.
as the system monotonic clock is used when the container is used
in Scrub implementation, and on some kernels there are rare cases
where the monotonic clock can go backwards, we need to tolerate
such events.
Patrick Donnelly [Fri, 28 Feb 2025 01:01:03 +0000 (20:01 -0500)]
Merge PR #61682 into main
* refs/pull/61682/head:
mds/snap: include cleanup
mds/MetricsHandler: include cleanup
mds/LogSegment: add missing includes
mds/JournalPointer: include cleanup
mds/cephfs_features: include cleanup
mds/MDSMap: include cleanup
mds/Migrator: include cleanup
mds/MDLog: include cleanup
mds/SnapServer: include cleanup
mds/MDSTableServer: include cleanup
mds/MetricAggregator: include cleanup
mds/SnapClient: include cleanup
mds/MDSPinger: include cleanup
mds/flock: include cleanup
mds/MDSAuthCaps: include cleanup
mds/QuiesceAgent: include cleanup
mds/QuiesceDb: include cleanup
mds/mdstypes: include cleanup
msg/Locker: include cleanup
mds/journal: add missing includes
mds/Server: include cleanup
mds/LogEvent: include cleanup
mds/Capability: include cleanup
mds/DamageTable: include cleanup
mds/StrayManager: include cleanup
mds/Mutation: include cleanup
mds/SnapRealm: include cleanup
mds/InoTable: add missing include
mds/ScatterLock: add missing include
mon/MDSMonitor: add missing include
mon/MDSDaemon: add missing include
Dan Mick [Thu, 27 Feb 2025 00:16:26 +0000 (16:16 -0800)]
container/build.sh: remove local container images
Optionally, for those that want to run build.sh locally and
use the images. The default is to remove, for Jenkins builders,
which will build, push, and rmi.
Fixes: https://tracker.ceph.com/issues/70196 Signed-off-by: Dan Mick <dan.mick@redhat.com>
Patrick Donnelly [Tue, 25 Feb 2025 21:36:12 +0000 (16:36 -0500)]
Merge PR #61889 into main
* refs/pull/61889/head:
msg/async/Stack: include cleanup
msg/Message: include cleanup
msg/async/crypto_onwire: include cleanup
msg/async/ProtocolV1: add missing includes
msg/async/frames_v2: add missing includes
msg/msg_types: add missing includes
Casey Bodley [Wed, 19 Feb 2025 15:13:04 +0000 (10:13 -0500)]
rgw/cksum: GetObject omits checksum headers for Range requests
if we send response headers containing full-object checksums, clients
will compare them with the returned data and fail with:
> botocore.exceptions.FlexibleChecksumError: Expected checksum 4AAr8A== did not match calculated checksum: WbxXZw==
the only thing aws docs[1] say about Range requests refers to multipart
uploads:
> For completed uploads, you can get an individual part's checksum by using the GetObject or HeadObject operations and specifying a part number or byte range that aligns with a single part.
Casey Bodley [Wed, 8 Jan 2025 19:52:03 +0000 (14:52 -0500)]
rgw/s3: CreateBucket extension for layout type and shard count
extend s3's CreateBucketConfiguration [1] with a custom BucketIndex element
that can override rgw's default bucket index type and shard count
to create an indexless bucket:
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<BucketIndex>
<Type>Indexless</Type>
</BucketIndex>
</CreateBucketConfiguration>
to create a normal pre-sharded bucket:
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<BucketIndex>
<Type>Normal</Type>
<NumShards>1023</NumShards>
</BucketIndex>
</CreateBucketConfiguration>
Dhairya Parmar [Tue, 25 Feb 2025 10:58:06 +0000 (16:28 +0530)]
qa: remove unreacheable/redundant code from test_data_scan.py
call to self.fs.journal_tool(["journal", "reset", "--force", "--yes-i-really-really-mean-it"], 0)
is already made below it, the same call under if False is unreachable.
Zac Dover [Tue, 25 Feb 2025 04:57:11 +0000 (14:57 +1000)]
doc/releases: correct squid release order
Put the releases of Squid in descending order. This change alters the
order of the Squid releases so that it is the same as the order of the
other Ceph releases.
1.Added option kmip in KMS dropdown in configuration page RGW
2.Made API dynamic for setting form data for Vault, kmip both for (S3 and
KMS)
3.Both Getdata and setdata API for RGW config page is modified
Casey Bodley [Mon, 24 Feb 2025 21:50:39 +0000 (16:50 -0500)]
doc/rgw: clarify path-style vs virtual-hosted-style access
instead of referring to "vhost-style", copy the "path-style" and
"virtual-hosted-style" language from https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
expand the FQDN acronym to avoid potential confusion
"The second method is deprecated by AWS" had incorrectly referred to the
vhost-style method - clarify that it refers to path-style access
Patrick Donnelly [Mon, 24 Feb 2025 21:41:00 +0000 (16:41 -0500)]
Merge PR #61697 into main
* refs/pull/61697/head:
osdc/ObjectCacher: include cleanup
osdc/Objecter: include cleanup
osdc/Journaler: include cleanup
osdc/Striper: add missing includes
osdc/Filer: include cleanup
crush/CrushWrapper: un-inline methods to fix linker errors
client/Client: add missing include and forward declaration
These changes allow the pool to be configured to disable mirroring on
the default namespace if required (init-only mode). It also allows the
default namespace to be mirrored to a non-default namespace on the
remote pool.
Sachin Prabhu [Sun, 9 Feb 2025 12:38:14 +0000 (12:38 +0000)]
cephadm/smb: add ctdb ports to smb daemon endpoint
When setting up a clustered SMB service, make sure that the port for the
ctdb service is opened up on the firewall. This port is used by the ctdb
servers to communicate with each other.
projects / ceph.git / log