Matan Breizman [Wed, 5 Feb 2025 17:35:38 +0000 (17:35 +0000)]
os: Introduce WITH_KSTORE
Previosuly, we used WITH_SEASTAR to not include KStore with Crimson.
As now Crimson's Alienstore is not defined WITH_SEASTAR We need to be able
to not include KStore to alien crimson target as well.
Instead, let Classic define PRIVATE WITH_KSTORE that will not
be used by the alien targets.
This doesn't change anything of non-crimson targets.
Matan Breizman [Thu, 6 Feb 2025 11:06:33 +0000 (11:06 +0000)]
crimson/os/alienstore/CMakeLists: move alien::cflags
alien::cflags is now only relevant to crimson-alienstore.
Remove empty INTERFACE_COMPILE_DEFINITIONS.
Note: alien::cflags can be probably entirely removed and
replaced by target_include_directories. However, this causes
```
'seastar::continuation_base<>' has virtual functions but non-virtual destructor
```
for alien_*.cc users.
Can be addressed seperatly.
Matan Breizman [Wed, 5 Feb 2025 10:41:16 +0000 (10:41 +0000)]
crimson/os/alienstore/CmakeLists: Drop WITH_SEASTAR/ALIEN
Removing WITH_SEASTAR/WITH_ALIEN will allow us to remove
WITH_ALIEN all together as now "alien" targets
are not defined WITH_SEASTAR.
Mere WITH_SEASTAR is enough to differentiate between "alien"
targets to ones used by Crimson.
Note: The cmake subdirectory of "crimson" will pull WITH_SEASTAR
to crimson-alienstore. That is why "crimson" is replaced by "seastar".
Ideally, we would not need to use "seastar" as well but the pure
alien sources (e.g alien_*.cc) must use seastar's concepts.
This "workaround" results in an crimson-alienstore to not be defined
WITH_SEASTAR even though it uses seastar. While this is not trivial,
it will help with removing WITH_ALIEN which makes Crimson's cmake
harder to understand. Moreover, WITH_SEASTAR trully represents a
Crimson target - and as crimson-alienstore is not trully Crimson,
his "workaround" seems to fit best.
Adam Kupczyk [Mon, 10 Mar 2025 08:58:50 +0000 (09:58 +0100)]
Merge pull request #60753 from aclamk/wip-aclamk-more-deferred-overwrite-fix
os/bluestore: Fix problem with deferred writes replay
This PR had gone through testing, including teuthology, but not in its final form.
But the issue its fixed is urgent.
This PR is merged before final teuthology run.
ceph-volume: allow zapping partitions on multipath devices
ceph-volume refuses to zap a device if it is a partition on a multipath
device due to an overly strict condition. This change ensures that only
full mapper devices (excluding partitions) are blocked from being zapped,
allowing partitions on multipath devices to be processed correctly.
Alex Ainscow [Tue, 11 Feb 2025 10:51:40 +0000 (10:51 +0000)]
common: bitset_set and mini_flat_map
The bitset_set is compile-time-fixed size bitmap, which can be accessed using a std::set-like iterator.
The mini_flat_map is similar to a boost::flat_map, except with more restrictions on the size and key, allowing it to make
better use of vectors.
Both have the restriction that the key must be unambiguously castable to/from a small integer. The mini_flat_map has the additional restriction, that the size must be known at construction time.
* refs/pull/62108/head:
cmake/cephfs: fix options to enable client and dependencies
Reviewed-by: Krunal Chheda <kchheda3@bloomberg.net> Reviewed-by: Adam C. Emerson <aemerson@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Matan Breizman [Wed, 5 Mar 2025 15:58:12 +0000 (15:58 +0000)]
crimson/osd/pg: Handle peer replies handling once received
Instead of updating last complete on disk and calling complete_write
in PG::submit_transacion (chained to all_completed).
Move the completion handling earlier as soon as all the peers acked.
This essentially means that we move the handling to
ReplicatedBackend::get_rep_op_reply, unless replication size is 1 -
then we would mark completion in ReplicatedBackend::submit_transaction.
Matan Breizman [Wed, 5 Mar 2025 15:28:16 +0000 (15:28 +0000)]
crimson/osd/pg: use rep_op_fut_t
rep_op_fut_t is already used in mutiple files.
Instead of intoducing common types in each relevant file,
future work shoud move this type (and similar) to "crimson_types" file.
pybind/cephfs: use legacy noexcept for cdefs for cython 3.Y.Z
For some newer versions of cython, it appears it requires explicitly specifying
noexcept but old versions of Cython 0.29.Z do not understand that attribute.
pybind/cephfs: increment ref before calling out to c++
At the time this construction seemed safe since the caller should have a
reference but it could conveivably be the only ref. We don't want the ref count
to reach 0.
Additionally, catch errors so this callback is genuinely noexcept.
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Jon Bailey [Wed, 15 Jan 2025 13:12:40 +0000 (13:12 +0000)]
test/osd: Added support for new plugins
Added new command line options for specifying plugins along with optional readers for them. Added arguments to JSON structures for some optional arguments to encode and decode more information about profiles. Added more choices for random selections of different plugin values and changed structure so profiles are created as part of a pool specifically, encapsulating them.
Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>
Jon Bailey [Thu, 9 Jan 2025 16:06:47 +0000 (16:06 +0000)]
src/test/osd: Move ceph_test_rados_io_sequence to its own subfolder
Move files for setting up the cpeh_test_rados_io_sequence to their own subfolder of test/osd so they can be grouped together as more files are added in the future.
Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>
Jamie Pryde [Tue, 26 Nov 2024 10:00:25 +0000 (10:00 +0000)]
erasure-code: Add minimum granularity interface to EC plugins
Add get_minimum_granularity() function to the EC plugin interface,
which returns the minimum size in bytes that can be used for partial writes.
This is based on the minimum size of coding chunk update
that the particular technique supports.
Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com> Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>
Matan Breizman [Sun, 2 Mar 2025 14:33:07 +0000 (14:33 +0000)]
crimson/mgr/client:Introduce Client::send()
Client::reconnect nullifies the connection used by the mgr client
before setting a new one.
In this time, we might re-use the nullptr connection due to tasks
that are being run in the background (See: dispatch_in_background).
To avoid this, we had multiple `if (!conn)` checks, some methods
even checked this condition twice to reduce the possibilty of using
undefined the connection.
Instead of introducing an additional check in Client::_send_report,
Introduce Client::send which would be responsible for:
a) Veryfing the connection is set
b) Trying to get a shared access to conn_lock
Client::reconnect will lock conn_lock exclusivly until the
connection is set. If we send is called while reconnecting,
sending will be dropped - same as before.
New sequence - write object initially with gaps and fill in the holes
until the object is fully populated. Permutate the step (gap between
writes), start at randomized offsets to produce gaps at the start of
the object as well as between writes
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
Bill Scales [Fri, 29 Nov 2024 11:12:40 +0000 (11:12 +0000)]
test: ceph_test_rados_io_sequence - add append with gaps sequence
New I/O sequence - appends to objects by writing past the old
object length so there is a gap before the new written data.
Tests permutations of write length and gap length.
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
Soumya Koduri [Wed, 29 Jan 2025 16:17:45 +0000 (21:47 +0530)]
rgw/cloudtier: Correct option ordering in RGWZoneGroupPlacementTier
Two tier-config options (related to `cloud-restore`) were incorrectly added
in the middle of the encoding and decoding methods of RGWZoneGroupPlacementTier.
This modification can cause compatibility issues with older decoders when
attempting to read v2-encoded REST objects.
The fix is to correct the option order and update the decode() function to
properly interpret the structure based on the encoded version.
projects / ceph-ci.git / log