Adam Kupczyk [Thu, 8 Feb 2024 22:28:22 +0000 (22:28 +0000)]
os/bluestore: Add fsck procedure for bdev multi labels
Now fsck can properly detect collision between labels and object data / bluefs files.
Additional labels have lower precedence, they never overwrite other data.
If collision label - object data happens, the object is moved somewhere else.
If collision label - bluefs file happens, it is left unsolved.
Adam Kupczyk [Tue, 30 Jan 2024 07:01:34 +0000 (07:01 +0000)]
os/bluestore: Give label multiple positions to replicate to
Bdev label for main device can now be present in multiple locations.
The locations of valid labels are memorized and only those locations are used.
This is to preserve from overwriting data, should collision label - object or bluefs occur.
Adam Kupczyk [Tue, 30 Jan 2024 06:54:38 +0000 (06:54 +0000)]
os/bluestore: Create read_bdev_main_label function
Duplicate read_bdev_label into dedicated read_bdev_main_label.
New function reads multiple labels.
Also created check_or_set_main_bdev_label in similiar way.
Modify read_bdev_label and write_bdev_label functions to operate on any disk location.
Default falls back to original position 0, which is now named BDEV_LABEL_POSITION.
myoungwon oh [Thu, 13 Jun 2024 06:07:56 +0000 (06:07 +0000)]
crimson/os/seastore: remove multistream related codes
Current codes allow the device to allocate multiple namespace without specific policy
if the nvme device report that it is capable of mutistream functionality.
So, this commit removes the multistream related code, leaving it as a TODO.
Zac Dover [Sat, 17 Aug 2024 03:37:58 +0000 (13:37 +1000)]
doc/cephfs: s/mountpoint/mount point/
Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).
cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486
in which page 345 of The IBM Style Guide is referenced to back up this
change.
Zac Dover [Sat, 17 Aug 2024 03:44:30 +0000 (13:44 +1000)]
doc/cephfs: s/mountpoint/mount point/
Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).
cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486 in
which page 345 of The IBM Style Guide is referenced to back up this
change.
This commit alters only English-language text and example commands in
which the string "{mount point}" is meant to be replaced. No commands
meant for cutting-and-pasting have been altered in this commit.
Laura Flores [Mon, 15 Jul 2024 22:04:41 +0000 (17:04 -0500)]
qa/suites/rados/thrash-old-clients: test with N-2 releases on centos 9
It was recently decided to stop building and releasing ubuntu focal
packages for squid. This decision extended to the Shaman builds.
When we stopped building focal for squid in Shaman, this failure
started happening, because the test was looking for nonexistent
squid focal packages:
```
no results found at https://shaman.ceph.com/api/search/?project=ceph&distros=ubuntu%2F20.04%2Fx86_64&flavor=default&sha1=81127b728ce57cc8b876f0f2dd3e436633549a67
```
After a discussion in Slack, we agreed the best option going forward
would be to test on centos 9 and drop pacific from the mix, since pacific
does not have centos 9 packages. To later incorprate pacific, we will work
on a contanierized solution.
-----
Slack thread (may be expired):
https://ceph-storage.slack.com/archives/C1HFJ4VTN/p1721078395083699
Laura Flores
4:19 PM
@Dan Mick
I see we stopped building focal for squid on Shaman via Jenkins. I
know this is intended since we no longer plan to release squid focal
packages, but now the thrash-old-clients tests are failing on squid:
https://pulpito.ceph.com/teuthology-2024-07-14_21:00:02-rados-squid-distro-default-smithi/7801302/
These tests use an older client, i.e. reef, in a squid cluster. These
older clients go as far back as N-3 (so we test pacific, reef, and
quincy clients against a squid cluster). We need a distro that is shared
between all these releases in order to do that, which up until recently
was focal. Can we reintroduce focal shaman builds? We can put a note in
https://docs.ceph.com/en/latest/start/os-recommendations/#platforms to
explain that these packages are not released for squid, but are used to
test old clients.
Laura Flores
4:21 PM
In the above scenario, we could consider switching to centos 9 since squid,
reef and quincy share these. But we also test against pacific clients, and
pacific of course does not build c9.
Casey Bodley
41 minutes ago
it would be nice if those tests could eventually use containers for the upgraded
servers
Josh Durgin
4:46 PM
centos 9 is the easiest path for now, for quincy and reef
4:46
agree with casey containerized servers would be better going forward anyway
Zac Dover [Mon, 12 Aug 2024 12:47:08 +0000 (22:47 +1000)]
doc/cephfs: improve cache-configuration.rst
Improve the text in the section about dealing with cache-pressure alerts
that was added in https://github.com/ceph/ceph/pull/59077. The changes
in this commit were suggested by Anthony D'Atri.
Co-authored-by: Patrick Donnelly <pdonnelly@redhat.com> Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit aa3bdae2314fef2fca8fc12dca006af657235e17)
ceph-volume: refactor device path handling for LVM lookups
This consolidates the conditional checks for device paths to
reduce redundancy and improve readability and adds logic to
handle both '/dev/mapper' and '/dev/dm-' paths uniformly by
introducing a utility function `get_lvm_mapper_path_from_dm()`.
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1756:3: error: static_assert failed due to requirement 'formattable' "Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt"
static_assert(
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1777:10: note: in instantiation of function template specialization 'fmt::detail::make_value<fmt::basic_format_context<fmt::appender, char>, const hobject_t &>' requested here
return make_value<Context>(val);
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1899:23: note: in instantiation of function template specialization 'fmt::detail::make_arg<true, fmt::basic_format_context<fmt::appender, char>, fmt::detail::type::custom_type, const hobject_t &, 0>' requested here
data_{detail::make_arg<
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1918:10: note: in instantiation of function template specialization 'fmt::format_arg_store<fmt::basic_format_context<fmt::appender, char>, hobject_t, unsigned int, unsigned int, bool, unsigned long>::format_arg_store<const hobject_t &, unsigned int &, unsigned int &, const bool &, unsigned long &>' requested here
return {FMT_FORWARD(args)...};
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:3206:28: note: in instantiation of function template specialization 'fmt::make_format_args<fmt::basic_format_context<fmt::appender, char>, const hobject_t &, unsigned int &, unsigned int &, const bool &, unsigned long &>' requested here
return vformat(fmt, fmt::make_format_args(args...));
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/common/tri_mutex.h:136:14: note: in instantiation of function template specialization 'fmt::format<const hobject_t &, unsigned int, unsigned int, const bool &, unsigned long>' requested here
os << fmt::format("tri_mutex {} writers {} readers {}"
^
```
Zac Dover [Wed, 7 Aug 2024 13:11:11 +0000 (23:11 +1000)]
doc/cephfs: add cache pressure information
Add information to doc/cephfs/cache-configuration.rst about how to deal
with a message that reads "clients failing to respond to cache
pressure". This procedure explains how to slow the growth of the
recall_caps value so that it does not exceed the
mds_recall_warning_threshold.
The information in this commit was developed by Eugen Block. See
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/5ROH5CWKKOEIQMVXOVRT5OO7CWK2HPM3/#J65DFUPP4BY57MICPANXKI7KAXSZ5Z5P
and https://www.spinics.net/lists/ceph-users/msg73188.html.
Fixes: https://tracker.ceph.com/issues/57115 Co-authored-by: Eugen Block <eblock@nde.ag> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit bf26274ae4737417193f8c2b56bea20eb2a358aa)
osd/scrub: exempt only operator scrubs from max_scrubs limit
Existing code exempts all 'high priority' scrubs, including for example
'after_repair' and 'mandatory on invalid history' scrubs from the limit.
PGs that do not have valid last-scrub data (which is what we have when
a pool is first created) - are set to shallow-scrub immediately.
Unfortunately - this type of scrub is (in the low granularity implemented
in existing code) is 'high priority'.
Which means that a newly created pool will have all its PGs start
scrubbing, regardless of concurrency (or any other) limits.
Fixes: https://tracker.ceph.com/issues/67253 Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit babd65e412266f5c734f7a2b57d87657d3470c47)
conflict resolution:
- eliminating irrelevant 'main' code that was picked into this branch.
- the code to set the scrub_job's flag moved to osd_scrub_sched.cc,
where the corresponding function is.
(cherry picked from commit a3f16627fde5426b19b932b9ef41c167e029d30f)
do { \
const bool should_gather = [&](const auto cctX) { \
if constexpr (ceph::dout::is_dynamic<decltype(sub)>::value || \
ceph::dout::is_dynamic<decltype(v)>::value) { \
std::cout << "the dynamic path" << std::endl; \
return subsys.should_gather(sub, v); \
} else { \
/* The parentheses are **essential** because commas in angle \
* brackets are NOT ignored on macro expansion! A language's \
* limitation, sorry. */ \
std::cout << "the static path" << std::endl; \
/*return subsys.should_gather(sub, v);*/ \
return (subsys.template should_gather<sub, v>()); \
} \
}(cct); \
} while (0)
if (decltype(auto) pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for 'this' */ \
dout_impl(42, sub, v)
if (decltype(auto) pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for 'this' */ \
dout_impl(42, ceph::dout::need_dynamic(42), v)
int main() {
std::random_device dev;
std::mt19937 rng(dev());
std::uniform_int_distribution<std::mt19937::result_type> dist6(1,6); // distribution in range [1, 6]
int sub = dist6(rng);
ldpp_dout("mocked out", sub);
//ldpp_subdout("mocked out", 4, 3);
}
```
Test: osd-recovery-space.sh extends the wait time for "recovery toofull".
The osd-recovery-space test involves writing objects and expecting to receive
the "toofull" flag.
If we don't wait long enough, we might check the "toofull" flag before all objects
have completed writing, and the "toofull" status hasn't been activated yet.
The change will extend the waiting time and will also incorporate additional
checks for the return code from the status wait.
osd/scrub: exempt only operator scrubs from max_scrubs limit
Existing code exempts all 'high priority' scrubs, including for example
'after_repair' and 'mandatory on invalid history' scrubs from the limit.
PGs that do not have valid last-scrub data (which is what we have when
a pool is first created) - are set to shallow-scrub immediately.
Unfortunately - this type of scrub is (in the low granularity implemented
in existing code) is 'high priority'.
Which means that a newly created pool will have all its PGs start
scrubbing, regardless of concurrency (or any other) limits.
Fixes: https://tracker.ceph.com/issues/67253 Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit babd65e412266f5c734f7a2b57d87657d3470c47)
conflict resolution:
- eliminating irrelevant 'main' code that was picked into this branch.
- the code to set the scrub_job's flag moved to osd_scrub_sched.cc,
where the corresponding function is.
projects / ceph.git / log