Jamie Pryde [Thu, 23 Jan 2025 15:21:27 +0000 (15:21 +0000)]
erasure-code: Rewrite the encode_chunks interface for all plugins
We have changed the encode_chunks interface to take an in map and an out map.
The in map contains the data shards to be encoded. The out map contains the
empty buffers that the plugin will write the parity data to.
The old non-optimized EC path has been updated to call the new encode_chunks
interface from erasurecode.cc
The new EC optimizations path calls the encode_chunks interface from ECUtils.
Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com> Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Jamie Pryde [Tue, 26 Nov 2024 10:00:25 +0000 (10:00 +0000)]
erasure-code: Add minimum granularity interface to EC plugins
Add get_minimum_granularity() function to the EC plugin interface,
which returns the minimum size in bytes that can be used for partial writes.
This is based on the minimum size of coding chunk update
that the particular technique supports.
Jamie Pryde [Tue, 19 Nov 2024 12:11:37 +0000 (12:11 +0000)]
erasure-code: Add parity delta write optimization to SHEC
This commit adds FLAG_EC_PLUGIN_PARITY_DELTA_OPTIMIZATION to the list of
optimizations supported by the SHEC plugin, and encode_delta and
apply_delta functions.
Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com> Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Jamie Pryde [Tue, 5 Nov 2024 22:21:05 +0000 (22:21 +0000)]
erasure-code: Add parity delta write optimization to Jerasure
This commit adds FLAG_EC_PLUGIN_PARITY_DELTA_OPTIMIZATION to the list of
optimizations supported by the Jerasure plugin, and encode_delta and
apply_delta functions for all Jerasure techniques.
Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com> Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Jamie Pryde [Mon, 28 Oct 2024 14:37:46 +0000 (14:37 +0000)]
erasure-code: Add parity delta write optimization to ISA
This commit adds FLAG_EC_PLUGIN_PARITY_DELTA_OPTIMIZATION to the list of
optimizations supported by the ISA plugin, and encode_delta and
apply_delta functions to ISA. Stubs for these functions are added to
the other plugins.
Two new tests have been added to TestErasureCodePlugins.cc to test the
encode_delta and apply_delta functions. The first new test updates a
single parity chunk using a single delta. The second new test updates
all parity chunks with a delta for every data chunk.
Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com> Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Jamie Pryde [Tue, 15 Oct 2024 14:30:13 +0000 (14:30 +0000)]
erasure-code: Add optimization flags for plugins
Add flags to erasure code plugins to report supported optimizations
and new unittest_erasure_code_plugins to test these flags for many
permutations of plugins and profiles.
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
[Taken from Bill's fork with permission] Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com>
Laura Flores [Fri, 7 Mar 2025 06:22:00 +0000 (06:22 +0000)]
mon, osd: add command to remove invalid pg-upmap-primary entries
The current rm-pg-upmap-primary command checks that the pgid exists
in the pgmap before continuing to remove it. Due to https://tracker.ceph.com/issues/66867,
some invalid pg-upmap-primary entires may exist for pools that have been removed.
Currently, these mappings are impossible to remove since the pgids no longer
exist in the pgmap.
This new command, rm-pg-upmap-primary-all, allows users the ability to remove
any and all pg-upmap-primary mappings in the osdmap at once, which includes
valid and invalid entries.
This command may also be helpful when upgrading from versions where users
are plagued by https://tracker.ceph.com/issues/61948. Users may use an upgraded
mon to remove all pg-upmap-primray entries (valid and invalid) so they continue
to upgrade to a safe version.
See manual testing for this patch here: https://tracker.ceph.com/issues/67179#note-12
Fixes: https://tracker.ceph.com/issues/67179 Fixes: https://tracker.ceph.com/issues/69760 Signed-off-by: Laura Flores <lflores@ibm.com>
Vallari Agrawal [Mon, 17 Mar 2025 16:28:19 +0000 (21:58 +0530)]
monitoring: rename NVMeoFSingleGatewayGroup alert
Rename the alert to NVMeoFSingleGateway.
The original name was confusing because it
accidently might convey that alert would be
triggered if there is a single gateway group.
Though 'NVMeoFSingleGatewayGroup' alert means that
there is single gateway in a group.
Kefu Chai [Mon, 17 Mar 2025 09:52:25 +0000 (17:52 +0800)]
crypto: remove unused include
openssl_crypto_accel.cc does not need the declarations included by
openssl/engine.h, also, openssl/engine.h was deprecated in favor of
the provider API, the engine support was removed in fedora 41.
so, let's avoid including it. please note, the "ENGINE" struct's
definition is available in openssl/types.h.
Venky Shankar [Fri, 14 Mar 2025 14:07:28 +0000 (19:37 +0530)]
Merge PR #61877 into main
* refs/pull/61877/head:
libcephfs_proxy: implement client side async rw operation
libcephfs_proxy: implement server side async rw operation
libcephfs_proxy: implement the async rw callback
libcephfs_proxy: define async rw opcode and structures
libcephfs_proxy: define some helper macros for callbacks
libcephfs_proxy: move pointer obfuscation functions to proxy_helpers.h
libcephfs_proxy: negotiate and use the async cbk feature
libcephfs_proxy: implement asynchronous callbacks
libcephfs_proxy: gracefully handle connection close
libcephfs_proxy: add negotiation documentation
libcephfs_proxy: replace legacy handshake by negotation in server side
libcephfs_proxy: replace legacy handshake by negotation in client side
libcephfs_proxy: implement server side negotiation
libcephfs_proxy: implement client side negotiation
libcephfs_proxy: implement receiving of negotiate structure
libcephfs_proxy: add negotiation structures
libcephfs_proxy: add support for control messages
libcephfs_proxy: reuse proxy_link_{read|write}
Patrick Donnelly [Fri, 14 Mar 2025 12:29:12 +0000 (08:29 -0400)]
Merge PR #62228 into main
* refs/pull/62228/head:
libcephfs: getcwd after chdir for API constraint
client: unwrap dentries for getcwd
client: refactor / optimize chdir
test/libcephfs: test getcwd with case insensitive dir
test/libcephfs: refactor ManyNestedDirs
include/filepath: add iterators for components
Soumya Koduri [Fri, 3 Jan 2025 08:20:48 +0000 (13:50 +0530)]
rgw/cloud-restore: Handle versioned objects
While restoring non-current object versions, ensure they remain non-current.
Read `olh_epoch` from the restored object's metadata into a new attr
"RGW_ATTR_RESTORE_VERSIONED_EPOCH". This attr/olh_epoch is used while
updating bi entry and also to reset HEAD object post expiry of temporary
copies.
Venky Shankar [Thu, 13 Mar 2025 16:57:57 +0000 (22:27 +0530)]
Merge PR #54396 into main
* refs/pull/54396/head:
PendingReleaseNote: add note for pause_purging and pause_cloninig
doc/cephfs: add note for config option pause_purging and pause_cloning
qa/cephfs: add tests for mgr/vol config pause_cloning
qa/cephfs: extend wait for trash empty
qa/cephfs: add tests for config option pause_purging
qa/cephfs: don't strip any whitespace for get_shell_stdout
mgr/vol: add pause/resume mechanism for async jobs
Matan Breizman [Thu, 13 Mar 2025 14:29:31 +0000 (14:29 +0000)]
qa/tasks/radosbench: make use of concurrency flag
This affects mainly radosbench-high-concurrency.yaml
The concurrency flag was only used in non-write scenarios to
populate the initial data.
We should use this flag with `write` mode as well.
Patrick Donnelly [Tue, 11 Mar 2025 16:13:33 +0000 (12:13 -0400)]
test/libcephfs: test getcwd with case insensitive dir
Unfortunately, it's not easy to refactor this test into a shared method without
setting up an explicit test class which has been avoided up to this point. So
I'm going to just copy the code. Sorry.
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
projects / ceph-ci.git / log