Zac Dover [Wed, 20 Dec 2023 05:00:38 +0000 (15:00 +1000)]
doc/radosgw: edit compression.rst
Improve the grammar and simplify the sentence structure of
doc/radosgw/compression.rst. This commit is made in anticipation of a
near-future commit that will list the compression algorithms available
to users of Ceph.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
Ronen Friedman [Thu, 14 Dec 2023 14:27:13 +0000 (08:27 -0600)]
osd/scrub: add a basic set of performance counters
Add a labeled set of performance counters, with the labels selecting
one of four copies of the counters - one per each scrub level and
pool type combination.
Inside the Scrubber, the relevant set of counters is selected
when the scrub is initialized.
NitzanMordhai [Wed, 28 Jun 2023 09:57:11 +0000 (09:57 +0000)]
ceph-dencoder: osd - Add missing types
Currently, ceph-dencoder lacks certain osd types, preventing us from accurately checking the ceph corpus for encode-decode mismatches.
This pull request aims to address this issue by adding the missing types to ceph-dencoder.
To successfully incorporate these types into ceph-dencoder, we need to introduce the necessary `dump` and `generate_test_instances`
functions that was missing in some types. These functions are essential for proper encode and decode of the added types.
This PR will enhance the functionality of ceph-dencoder by including the missing types, enabling a comprehensive analysis of encode-decode consistency.
With the addition of these types, we can ensure the robustness and correctness of the ceph corpus.
This update will significantly contribute to improving the overall reliability and accuracy of ceph-dencoder.
It allows for a more comprehensive assessment of the encode-decode behavior, leading to enhanced data integrity and stability within the ceph ecosystem.
Zac Dover [Tue, 19 Dec 2023 09:15:57 +0000 (19:15 +1000)]
doc/install: update "update submodules"
Remove misleading material that would give readers the wrong idea about
when stale submodules are present. This commit is made in response to
information given to me by Ilya Dryomov here: https://github.com/ceph/ceph/pull/54929#issuecomment-1859237986.
Matan Breizman [Thu, 16 Nov 2023 10:19:51 +0000 (10:19 +0000)]
osd/OSD: optimize send_incremental_map
Previosuly, if `since` was earlier than the oldest_map,
only the lastest full osdmap would be sent.
However, we could actually send inc/full maps from oldest_map
up to the current epoch.
Moreover, build_incremental_map_msg is suited to handle this case as well.
if `since` is earlier than `cluster_osdmap_trim_lower_bound` it
will be adjusted accordingly.
At any point, the cluster_osdmap_trim_lower_bound is later (or equal) to
the superblock's oldest_map. See: past_intervals.rst (OSDSuperblock::maps)
Hence, if since < oldest_map, and oldest_map <= clutser_lower_bound
then since < cluster_lower_bound.
Note: oldest_map stands as a mark to indicate up until which epoch
the current OSD trimmed its osdmaps. See OSD::trim_maps().
Meaning, the OSDMaps in the range of [oldest_map, trim_lower_bound]
may be at hand to be shared. Nevertheless, we should actually use
the cluster_trim_lower_bound to set the correct range.
Matan Breizman [Wed, 30 Aug 2023 08:57:18 +0000 (08:57 +0000)]
osd/OSD: introduce trim_stale_maps
```
/**
* trim_stale_maps
*
* trim_maps had a possible (rare) leak which resulted in stale osdmaps.
* This method will cleanup any existing osdmaps from the store
* with an epoch earlier than the superblock's oldest_map epoch.
* See: https://tracker.ceph.com/issues/61962
*/
```
Kefu Chai [Fri, 15 Dec 2023 10:18:01 +0000 (18:18 +0800)]
cmake: differentiate WITH_CEPHFS from WITH_LIBCEPHFS
WITH_CEPHFS is used for enabling the server side of cephfs, i.e., mds,
while WITH_LIBCEPHFS is for the client side of it, namely libcephfs
and its python bindings. since we have these two different options,
in theory, user is allowed to build with WITH_CEPHFS=ON and
WITH_LIBCEPHFS=OFF, and in that case, `vstart` should not depend on
the non-existent target of cephfs and cython_cephfs. the same applies to
the build of cython_cephfs. this is an unusual combination, but
it is a valid one.
in this change, we
* build the python binding only if WITH_LIBCEPHFS is ON
* add cephfs and cython_cephfs as dependencies of vstart only if
WITH_LIBCEPHFS is ON
- Modify ClientRequest::snaps_need_to_recover() to return all
relevant snaps including the operation target.
- Update ClientRequest::snaps_need_to_recover() and
CommonClientRequest::recover_missings to use set::set
for snaps.
- Remove special handling for soid from
CommonClientRequest::recover_missings
- Simplify CommonClientRequest::recover_missings.
Patrick Donnelly [Fri, 15 Dec 2023 13:18:28 +0000 (08:18 -0500)]
Merge PR #52196 into main
* refs/pull/52196/head:
qa: configure balancer for multi-mds workloads
qa: create qa subvolumes in named subvolumegroup
qa: do not rely on default max_mds value
qa: add automate_balance to dashboard qa schema
doc/cephfs: add docs for balance_automate
doc/cephfs: use bash prompt for shell code
mds: add balance_automate fs setting
Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Xiubo Li <xiubli@redhat.com>
Kefu Chai [Fri, 15 Dec 2023 09:28:37 +0000 (17:28 +0800)]
cmake: add 1.82 and 1.83 to known versions
this change should silence the warning when building Ceph with Boost 1.82:
```
CMake Warning at cmake/modules/FindBoost.cmake:1384 (message):
New Boost version may have incorrect or missing dependencies and imported
targets
Call Stack (most recent call first):
cmake/modules/FindBoost.cmake:1507 (_Boost_COMPONENT_DEPENDENCIES)
cmake/modules/FindBoost.cmake:2125 (_Boost_MISSING_DEPENDENCIES)
CMakeLists.txt:687 (find_package)
```
despite that 244c5ebb added boost to `_Boost_KNOWN_VERSIONS`, it was
not enough. would be better to sync up the CMake upstream for an updated
version of this file like we did before.
CMake allows us to customize `CMAKE_CXX_FLAGS` by setting CXXFLAGS
environmental variable. and Debian's debhelper also sets CXXFLAGS
when it builds cmake projects for customizing the building flags.
but we fail to populate this setting down when building external
projects. this is important when it comes to the projects which
is critical to the performance. RocksDB is one of them.
in this change, we pass the `CMAKE_CXX_FLAGS` down in
`BuildRocksDB.cmake` so that its `CMAKE_CXX_FLAGS` contains
the same set of `CMAKE_CXX_FLAGS` used by its parent project.
this should help with the performance in the bluestore, where
RocksDB is used.
Ilya Dryomov [Sun, 10 Dec 2023 16:01:24 +0000 (17:01 +0100)]
test/pybind/rbd: don't ignore from_snapshot in check_diff()
Despite the test in test_diff_iterate() being correct, it started
failing:
> check_diff(self.image, 0, IMG_SIZE, 'snap1', [(0, 512, False)])
...
a = [], b = [(0, 512, False)]
...
> assert a == b
E AssertionError
This is because check_diff() drops 'snap1' argument on the floor and
passes None to image.diff_iterate() instead. This goes back to 2013,
see commit e88fe3cbbc8f ("rbd.py: add some missing functions").
- beginning of time -> HEAD, through intermediate snap
- snap -> snap, directly
- snap -> HEAD, directly
But coverage is too weak: none of the weird OBJECT_PENDING cases and
only a single diff-iterate vs deep-copy case is tested, for example.
Coverage is missing completely for:
- beginning of time -> HEAD, directly
- beginning of time -> snap, directly
- beginning of time -> snap, through intermediate snap
- snap -> snap, through intermediate snap
- snap -> HEAD, through intermediate snap
Kefu Chai [Fri, 15 Dec 2023 10:04:53 +0000 (18:04 +0800)]
cmake: only apply workaround for boost < 1.84
https://github.com/boostorg/phoenix/issues/111 was fixed in master.
see the fix which addresses the issue:
https://github.com/boostorg/phoenix/commit/8913607a3788cb82d48ed461ea59c919b7bad3df
projects / ceph.git / log