common/async: fold async_spawn_throttle_impl into spawn_throttle_impl
spawn_throttle_impl was a virtual base class for the sync_ and async_
variants. with the sync variant removed, combine the classes into one
and remove the use of virtual functions
common/async: remove null_yield support for spawn_throttle
null_yield support was provided by sync_spawn_throttle_impl, which
itself stores the boost::asio::io_context that it runs on. however,
valgrind reports issues with object lifetimes
each spawn_throttle_handler holds a reference count on the
spawn_throttle_impl to prevent it from going away while coroutines are
in flight. on completion, this spawn_throttle_handler gets invoked on
its associated executor. this invocation may drop the last reference
to spawn_throttle_impl while that associated executor is still trying
to use the io_context that was destroyed with sync_spawn_throttle_impl
finding no good way for a custom executor to preserve the lifetime of
the spawn_throttle_impl, we instead remove support for this flawed
design. the null_yield callers must first spawn a parent coroutine
before using spawn_throttle
rgw: RGWSI_Notify spawns a coroutine on null_yield
init_watch() and finalize_watch() use spawn_throttle for concurrent
operations, so need to spawn a parent coroutine unless init_watch()
already has one
rgw: RGWDeleteMultiObj spawns a coroutine on null_yield
if called with null_yield, RGWDeleteMultiObj::execute() creates an
io_context and spawns a coroutine to run on it. this ensures that
spawn_throttle always gets a valid yield context
In the EnableRequest state machine, clean up the handling of the async
request to fetch the mirror image, particularly when a non-primary image
is being created by the rbd-mirror daemon.
John Mulligan [Tue, 17 Jun 2025 19:09:20 +0000 (15:09 -0400)]
cephadm: add support for specific network binds to smb service
Add a bunch of code to support specific IP address (and/or interface -
see below) binds for the smb service. When the smb service is not
clustered it is using container networking - in this case we use
publish options for the container manager to only listen on the supplied
addresses.
When the smb service is clustered we need to jump through a bunch of
hoops to configure each service individually. Many are easy with just
a short set of CLI options. CTDB only listens on the (first) node
address that it can bind to and only that. smbd has complex interactions
based on the `interfaces` and `bind interfaces only` config parameters.
Because these parameters may be unique to a node (addresses certainly
will be - and interfaces names could be) we can not store this in
the registry based conf. Instead, we take the slightly hacky approach
of generating a stub conf file with just the interfaces related params
in them and telling sambacc to generate a config that includes this
stub config.
IMPORTANT: When using ctdb with public addresses smbd doesn't know what
additional IPs it may need to listen to, so instead of binding to
a fixed IP we configure it to use an interface. This does have a
downside of possibly listening to another address on the same interface
we don't want it to. Additionally, I have observed that as addresses
are added or removed from the interface by ctdb, smbd doesn't
consistently start listening to those addresses.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 18 Jun 2025 21:18:30 +0000 (17:18 -0400)]
mgr/cephadm: teach ctdb nodes logic about bind_addrs
Within the cephadm smb service class we have logic to help manage CTDB's
nodes. Ensure that this node handling logic also conforms to the recent
addition of the smb service's bind_addrs field.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 16 Jun 2025 20:05:22 +0000 (16:05 -0400)]
mgr/cephadm: add filter_host_candidates method to smb service class
Add a filter_host_candidates method to the smb service class allowing
that class to act as a HostSelector. The HostSelector was added in an
earlier commit to allow classes like this one to make specific host
selections based on unique to that class (or it's spec) criteria.
This method uses the newly added `bind_addrs` field of the smb service
spec to ensure only hosts that meet the desired set of
networks/addresses get used in placement.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 16 Jun 2025 20:04:35 +0000 (16:04 -0400)]
python-common/deployment: add bind_addrs and related type for smb
Add a `bind_addrs` field and `SMBClusterBindIPSpec` to the smb service
spec. If specified the `bind_addrs` field can contain one or more
SMBClusterBindIPSpec value. In JSON these values can contain either an
address `{"address": "192.168.76.10"}` or network `{"network":
"192.168.76.0/24"}`.
These specs will be used by cephadm to place the smb service only on
hosts that have IPs matching the supplied IP Address/Network values. It
will also instruct the smb services to only bind to these addresses.
A suggested future enhancement may be include an IP address range
representation for the SMBClusterBindIPSpec.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 16 Jun 2025 20:05:14 +0000 (16:05 -0400)]
mgr/cephadm: teach serve.py about host selector support
A previous commit added a HostSelector protocol type to the schedule
code. This change makes it so the function calling upon the
HostAssignment class detects if a CephService provides a
filter_host_candidates method - meaning the service class can act as a
HostSelector. If the class can be a HostSelector pass it to the
HostAssignment so that the custom selection operation can be run.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 16 Jun 2025 20:05:01 +0000 (16:05 -0400)]
mgr/cephadm: prepare schedule.py for per-service-type host filtering
Prepare schedule.py for per-service-type host filtering based on allowed
host addresses/networks. Add a new HostSelector protocol type to the
module defining what the filtering interface looks like.
This interface is intended allows CephService classes to "take over" the
network based filtering of nodes prior to placement and customize the
behavior of this step in cephamd's placement algorithm.
Note that the type must be passed in to the HostAssignment class as an
optional argument. If nothing is passed the class behaves as it did
before.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Edit the section "Data Pool Damage" in doc/cephfs/disaster-recovery.rst.
This commit is part of the project of improving the data-recovery parts
of the CephFS documentation, as requested in the Ceph Power Users
Feedback Summary in mid-2025.
auth: remove unused AuthTicket::renew_after member variable
The AuthTicket::renew_after field is only set in init_timestamps() and
read by dump() for debugging purposes. It has no functional use cases
and causes encoding/decoding inconsistencies.
During decoding, this field remains unchanged, creating discrepancies
between original and decoded values. This issue was masked because
check-generated.sh and readable.sh reused struct instances, preserving
stale field values across encode/decode cycles.
An upcoming change will allocate fresh instances for each decode
operation, which would expose these inconsistent values.
Remove the unused field to eliminate the encoding inconsistency and
simplify the codebase.
crimson/os/seastore/rbm: improve large sequential write by issuing writes in parallel
In large chunk workload, for instance, when 4MB data is divided into 64KB chunks,
the current implementation writes sequentially one by one. On general NVMe SSD
writing large chunk (>=64KB) can lead to relatively higher latency compared to a small write case,
resulting in performance degradation. Although NVMe SSD shows the increased
latency in such cases, this does not mean that thay reach their performance limit,
as they can handle multiple concurrent requests.
This commit improves parallelism by using parallel_for_each() to issue writes in parallel.
Alex Ainscow [Mon, 21 Jul 2025 07:17:57 +0000 (08:17 +0100)]
osd: Replace deprecated std::align_storage_t with alignas
C++23 has been enabled, causing deprecated warnings. Following the
"possible implementation" in the C++ docs, I have replaced the last
remaining aligned_storage_t.
* refs/pull/63214/head:
release note: add a note that "subvolume info" cmd output can also...
doc/cephfs: update docs since "subvolume info" cmd output can also...
qa/cephfs: add test to check clone source info's present in...
mgr/vol: show clone source info in "subvolume info" cmd output
mgr/vol: keep clone source info even after cloning is finished
* refs/pull/58564/head:
client: clamp sizes to INT_MAX in sync i/o code paths
client: restrict bufferlist to total write size
src/test: test sync/async i/o code paths with huge (4GiB) buffers
- for arm64 hitting (Use `node --trace-warnings ...` to show where the warning was created)
NX Cannot find module @rollup/rollup-linux-arm64-gnu. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.
Pass --verbose to see the stacktrace.
- due this this make check arm64 failing
- added the fix as per https://github.com/vitejs/vite/discussions/15532#discussioncomment-13369584
- its failing then NX Falling back to ts-node for local typescript execution. This may be a little slower.
NX Cannot find module '@rspack/binding-linux-arm64-gnu'
- the above fix failed asking more deps from rollup, so added whole rollup package
projects / ceph.git / log