myoungwon oh [Thu, 13 Jun 2024 06:07:56 +0000 (06:07 +0000)]
crimson/os/seastore: remove multistream related codes
Current codes allow the device to allocate multiple namespace without specific policy
if the nvme device report that it is capable of mutistream functionality.
So, this commit removes the multistream related code, leaving it as a TODO.
Zac Dover [Sat, 17 Aug 2024 03:37:58 +0000 (13:37 +1000)]
doc/cephfs: s/mountpoint/mount point/
Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).
cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486
in which page 345 of The IBM Style Guide is referenced to back up this
change.
Zac Dover [Sat, 17 Aug 2024 03:44:30 +0000 (13:44 +1000)]
doc/cephfs: s/mountpoint/mount point/
Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).
cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486 in
which page 345 of The IBM Style Guide is referenced to back up this
change.
This commit alters only English-language text and example commands in
which the string "{mount point}" is meant to be replaced. No commands
meant for cutting-and-pasting have been altered in this commit.
Zac Dover [Mon, 12 Aug 2024 12:47:08 +0000 (22:47 +1000)]
doc/cephfs: improve cache-configuration.rst
Improve the text in the section about dealing with cache-pressure alerts
that was added in https://github.com/ceph/ceph/pull/59077. The changes
in this commit were suggested by Anthony D'Atri.
Co-authored-by: Patrick Donnelly <pdonnelly@redhat.com> Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit aa3bdae2314fef2fca8fc12dca006af657235e17)
ceph-volume: refactor device path handling for LVM lookups
This consolidates the conditional checks for device paths to
reduce redundancy and improve readability and adds logic to
handle both '/dev/mapper' and '/dev/dm-' paths uniformly by
introducing a utility function `get_lvm_mapper_path_from_dm()`.
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1756:3: error: static_assert failed due to requirement 'formattable' "Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt"
static_assert(
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1777:10: note: in instantiation of function template specialization 'fmt::detail::make_value<fmt::basic_format_context<fmt::appender, char>, const hobject_t &>' requested here
return make_value<Context>(val);
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1899:23: note: in instantiation of function template specialization 'fmt::detail::make_arg<true, fmt::basic_format_context<fmt::appender, char>, fmt::detail::type::custom_type, const hobject_t &, 0>' requested here
data_{detail::make_arg<
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:1918:10: note: in instantiation of function template specialization 'fmt::format_arg_store<fmt::basic_format_context<fmt::appender, char>, hobject_t, unsigned int, unsigned int, bool, unsigned long>::format_arg_store<const hobject_t &, unsigned int &, unsigned int &, const bool &, unsigned long &>' requested here
return {FMT_FORWARD(args)...};
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/fmt/include/fmt/core.h:3206:28: note: in instantiation of function template specialization 'fmt::make_format_args<fmt::basic_format_context<fmt::appender, char>, const hobject_t &, unsigned int &, unsigned int &, const bool &, unsigned long &>' requested here
return vformat(fmt, fmt::make_format_args(args...));
^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/common/tri_mutex.h:136:14: note: in instantiation of function template specialization 'fmt::format<const hobject_t &, unsigned int, unsigned int, const bool &, unsigned long>' requested here
os << fmt::format("tri_mutex {} writers {} readers {}"
^
```
Zac Dover [Wed, 7 Aug 2024 13:11:11 +0000 (23:11 +1000)]
doc/cephfs: add cache pressure information
Add information to doc/cephfs/cache-configuration.rst about how to deal
with a message that reads "clients failing to respond to cache
pressure". This procedure explains how to slow the growth of the
recall_caps value so that it does not exceed the
mds_recall_warning_threshold.
The information in this commit was developed by Eugen Block. See
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/5ROH5CWKKOEIQMVXOVRT5OO7CWK2HPM3/#J65DFUPP4BY57MICPANXKI7KAXSZ5Z5P
and https://www.spinics.net/lists/ceph-users/msg73188.html.
Fixes: https://tracker.ceph.com/issues/57115 Co-authored-by: Eugen Block <eblock@nde.ag> Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit bf26274ae4737417193f8c2b56bea20eb2a358aa)
do { \
const bool should_gather = [&](const auto cctX) { \
if constexpr (ceph::dout::is_dynamic<decltype(sub)>::value || \
ceph::dout::is_dynamic<decltype(v)>::value) { \
std::cout << "the dynamic path" << std::endl; \
return subsys.should_gather(sub, v); \
} else { \
/* The parentheses are **essential** because commas in angle \
* brackets are NOT ignored on macro expansion! A language's \
* limitation, sorry. */ \
std::cout << "the static path" << std::endl; \
/*return subsys.should_gather(sub, v);*/ \
return (subsys.template should_gather<sub, v>()); \
} \
}(cct); \
} while (0)
if (decltype(auto) pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for 'this' */ \
dout_impl(42, sub, v)
if (decltype(auto) pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for 'this' */ \
dout_impl(42, ceph::dout::need_dynamic(42), v)
int main() {
std::random_device dev;
std::mt19937 rng(dev());
std::uniform_int_distribution<std::mt19937::result_type> dist6(1,6); // distribution in range [1, 6]
int sub = dist6(rng);
ldpp_dout("mocked out", sub);
//ldpp_subdout("mocked out", 4, 3);
}
```
Test: osd-recovery-space.sh extends the wait time for "recovery toofull".
The osd-recovery-space test involves writing objects and expecting to receive
the "toofull" flag.
If we don't wait long enough, we might check the "toofull" flag before all objects
have completed writing, and the "toofull" status hasn't been activated yet.
The change will extend the waiting time and will also incorporate additional
checks for the return code from the status wait.
osd/scrub: exempt only operator scrubs from max_scrubs limit
Existing code exempts all 'high priority' scrubs, including for example
'after_repair' and 'mandatory on invalid history' scrubs from the limit.
PGs that do not have valid last-scrub data (which is what we have when
a pool is first created) - are set to shallow-scrub immediately.
Unfortunately - this type of scrub is (in the low granularity implemented
in existing code) is 'high priority'.
Which means that a newly created pool will have all its PGs start
scrubbing, regardless of concurrency (or any other) limits.
Fixes: https://tracker.ceph.com/issues/67253 Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit babd65e412266f5c734f7a2b57d87657d3470c47)
conflict resolution:
- eliminating irrelevant 'main' code that was picked into this branch.
- the code to set the scrub_job's flag moved to osd_scrub_sched.cc,
where the corresponding function is.
some tests are currently failing when `lvm2` isn't installed:
```
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestList::test_empty_device_json_zero_exit_status - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestList::test_empty_device_zero_exit_status - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_no_ceph_lvs - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_ceph_data_lv_reported - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_ceph_journal_lv_reported - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_ceph_wal_lv_reported - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_physical_2nd_device_gets_reported[journal] - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_physical_2nd_device_gets_reported[db] - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestFullReport::test_physical_2nd_device_gets_reported[wal] - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_not_a_ceph_lv - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_a_ceph_lv - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_a_ceph_journal_device - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_by_osd_id_for_just_block_dev - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_by_osd_id_for_just_data_dev - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_by_osd_id_for_just_block_wal_and_db_dev - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_by_osd_id_for_data_and_journal_dev - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_by_nonexistent_osd_id - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_listing.py::TestSingleReport::test_report_a_ceph_lv_with_no_matching_devices - FileNotFoundError: [Errno 2] No such file or directory: 'pvs'
FAILED ceph_volume/tests/devices/lvm/test_migrate.py::TestNew::test_newdb_not_target_lvm - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
FAILED ceph_volume/tests/devices/lvm/test_zap.py::TestEnsureAssociatedLVs::test_nothing_is_found - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
FAILED ceph_volume/tests/devices/lvm/test_zap.py::TestEnsureAssociatedLVs::test_multiple_journals_are_found - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
FAILED ceph_volume/tests/devices/lvm/test_zap.py::TestEnsureAssociatedLVs::test_multiple_dbs_are_found - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
FAILED ceph_volume/tests/devices/lvm/test_zap.py::TestEnsureAssociatedLVs::test_multiple_wals_are_found - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
FAILED ceph_volume/tests/devices/lvm/test_zap.py::TestEnsureAssociatedLVs::test_multiple_backing_devs_are_found - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
FAILED ceph_volume/tests/objectstore/test_lvmbluestore.py::TestLvmBlueStore::test_activate_all_osd_is_active - FileNotFoundError: [Errno 2] No such file or directory: 'lvs'
```
Everything should be actually mocked. This commit addresses that.
ceph-volume: do not convert LVs's symlink to real path
This commit:
- Adds a new function `get_lvm_mappers` in `ceph_volume/util/disk.py`
to retrieve a list of LVM device mappers.
- Updates the `is_lv` property in `ceph_volume/util/device.py`
to use the new `get_lvm_mappers` function for better accuracy.
- Modifies the symlink handling in `Device` class to properly
identify LVM logical volumes.
- Adds a new test `test_reject_lv_symlink_to_device` to ensure
LVM symlinks are correctly identified and handled.
- Updates relevant tests to cover the changes in LVM device detection.
These changes improve the reliability and accuracy of LVM device detection
and handling, ensuring that symlinks to LVM logical volumes are
correctly processed.
Fixes: https://tracker.ceph.com/issues/61597 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com> Co-Authored-by: Jerry Pu <jerrypu@qnap.com>
(cherry picked from commit 729c5de4f852f1a1ee90e76b71157e9070af7d99)
Igor Fedotov [Fri, 31 May 2024 14:05:29 +0000 (17:05 +0300)]
ceph-volume: do source devices zapping if they're detached.
One needs to zap source device(s) after DB/WAL migration.
Original imlementation removes LVM tags only which leaves device(s) in a
state where "ceph-volume raw activate" still reconginizes them as
attached to OSD due to information preserved in bdev label.
Hence the need to do more zapping. Fixes: https://tracker.ceph.com/issues/66315 Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit ae5ef432845dcf9b061258357ffd97f4eae59a63)
projects / ceph.git / log