crimson/osd: fix life-time management of OSDConnectionPriv
Before the patch there was a possibility that `OSDConnectionPriv`
gets destructed before a `PipelineHandle` instance that was using
it. The reason is our remote-handling operations store `conn` directly
while `handle` is defined in a parent class. Due to the language rules
the former gets deinitialized earlier.
```
==756032==ERROR: AddressSanitizer: heap-use-after-free on address 0x615000039684 at pc 0x0000020bdfa2 bp 0x7ffd3abfa370 sp 0x7ffd3abfa360
READ of size 1 at 0x615000039684 thread T0
Reactor stalled for 261 ms on shard 0. Backtrace: 0x45d9d 0xe90f6d1 0xe6b8a1d 0xe6d1205 0xe6d16a8 0xe6d1938 0xe6d1c03 0x12cdf 0xccebf 0x7f6447161b1e 0x7f644714aee8 0x7f644714eed6 0x7f644714fb36 0x7f64471420b5 0x 7f6447143f3a 0xd61d0 0x32412 0xbd8a7 0xbd134 0xbdc1a 0x20bdfa1 0x20c184e 0x352eb7f 0x352fa28 0x20b04a5 0x1be30e5 0xe694bc4 0xe6ebb8a 0xe843a11 0xe845a22 0xe29f497 0xe2a3ccd 0x1ab1841 0x3aca2 0x175698d
#0 0x20bdfa1 in seastar::shared_mutex::unlock() ../src/seastar/include/seastar/core/shared_mutex.hh:122
#1 0x20c184e in crimson::OrderedExclusivePhaseT<crimson::osd::ConnectionPipeline::GetPG>::exit() ../src/crimson/common/operation.h:548
#2 0x20c184e in crimson::OrderedExclusivePhaseT<crimson::osd::ConnectionPipeline::GetPG>::ExitBarrier::exit() ../src/crimson/common/operation.h:533
#3 0x20c184e in crimson::OrderedExclusivePhaseT<crimson::osd::ConnectionPipeline::GetPG>::ExitBarrier::cancel() ../src/crimson/common/operation.h:539
#4 0x20c184e in crimson::OrderedExclusivePhaseT<crimson::osd::ConnectionPipeline::GetPG>::ExitBarrier::~ExitBarrier() ../src/crimson/common/operation.h:543
#5 0x20c184e in crimson::OrderedExclusivePhaseT<crimson::osd::ConnectionPipeline::GetPG>::ExitBarrier::~ExitBarrier() ../src/crimson/common/operation.h:544
#6 0x352eb7f in std::default_delete<crimson::PipelineExitBarrierI>::operator()(crimson::PipelineExitBarrierI*) const /opt/rh/gcc-toolset-11/root/usr/include/c++/11/bits/unique_ptr.h:85
#7 0x352eb7f in std::unique_ptr<crimson::PipelineExitBarrierI, std::default_delete<crimson::PipelineExitBarrierI> >::~unique_ptr() /opt/rh/gcc-toolset-11/root/usr/include/c++/11/bits/unique_ptr.h:361
#8 0x352eb7f in crimson::PipelineHandle::~PipelineHandle() ../src/crimson/common/operation.h:457
#9 0x352eb7f in crimson::osd::PhasedOperationT<crimson::osd::ClientRequest>::~PhasedOperationT() ../src/crimson/osd/osd_operation.h:152
#10 0x352eb7f in crimson::osd::ClientRequest::~ClientRequest() ../src/crimson/osd/osd_operations/client_request.cc:64
#11 ...
```
Zac Dover [Wed, 29 Jun 2022 12:57:13 +0000 (22:57 +1000)]
doc/index.rst: add link to Dev Guide basic workfl.
This PR adds a link to the "Basic Workflow" section of the
Developer Guide on the landing page of docs.ceph.com.
This PR is meant to improve the documentation for developers
new to Ceph and to guide them to instructions that will allow
them to become full-fledged contributors to the Ceph project
as quickly as possible.
The "Basic Workflow" page of the Developer Guide contains
information that answers almost all of the questions that I had
about contributing to the Ceph project when I was new to it,
and I am finally acting on my long-held conviction that the
"Basic Workflow" page of the Developer Guide should have a more
prominent position in the documentation suite than it has had.
1. If data or metadata pool is already in-use by filesystem
then it is not allowed to reuse the same pool for another
filesystems.
2. Test is failing because above(1) restrictions/checks comes
before checking erasure-code pools. Hence test is failing
and not finding expected error string in output.
4. Also adding new tests to verify string 'erasure-code'
by passing --force option so that check for pools reuse(1)
will be skipped and check for 'erasure-code' will be hit.
Yingxin Cheng [Fri, 24 Jun 2022 03:04:50 +0000 (11:04 +0800)]
crimson/os/seastore/segment_cleaner: increase avaliable ratio limit
Journal trimming may consume unexpected number of segments when the
available ratio limit is reached with user transactions blocked, causing
ceph_abort(). So increase the limit as a simple workaround.
Yingxin Cheng [Fri, 24 Jun 2022 05:25:51 +0000 (13:25 +0800)]
crimson/os/seastore: improve GC policies with modify-time
* record_header_t to store the average modify time for dirty extents.
* Drop tracking rewrite-time.
* Drop the last-modify field in extent_info_t.
* Maintain modify-time during rewriting.
* Introduce 3 GC policies: greedy, benefit, and cost-benefit.
Yingxin Cheng [Fri, 27 May 2022 09:13:06 +0000 (17:13 +0800)]
crimson/os/seastore: implement generational GC
Place extents into the dedicated RecordSubmitter by their data-category
and reclaimed-count. Segments of different data-category or
reclaimed-count should have different locality in the access patterns,
which is the foundation to form a desired bimodal distribution of
segment utilizations, so that GC can be more efficient.
Alex Handy [Tue, 28 Jun 2022 17:27:23 +0000 (10:27 -0700)]
doc/cephadm: osd: minor fixes in config examples
- Corrected drive counts (8 to 10) and rotational flag use (not consistent with HDD) in the "Multiple OSD specs for a single host" example
- Adjusted vendor names in "Multiple hosts with the same disk layout" for consistency of examples
Kefu Chai [Mon, 27 Jun 2022 13:18:57 +0000 (21:18 +0800)]
ceph.spec.in: use %enable_devtoolset11 to enable GTS-11
%enable_devtoolset11 redefines %___build_pre by appending
`source scl_source enable gcc-toolset-11` to it. `___build_pre` should
be able to populate this setting to both %build and %install. and hence
address the FTBFS where we need to use the tool chain from GTS-11.
This can be surprising but we actually compile things during
the `install` stage of `rpm-build`. The example is the pybind's
`setup.py` which builds `rados_dummy.c`.
Ilya Dryomov [Sun, 26 Jun 2022 11:05:09 +0000 (13:05 +0200)]
librbd: update progress for non-existent objects on deep-copy
As a side effect of commit e5a21e904142 ("librbd: deep-copy image copy
state machine skips clean objects"), handle_object_copy() stopped being
called for non-existent objects. This broke progress_object_no logic,
which expects to "see" all object numbers so that update_progress()
callback invocations can be ordered. Currently update_progress() based
progress reporting gets stuck after encountering a hole in the image.
To fix, arrange for handle_object_copy() to be called for all object
numbers, even if ObjectCopyRequest isn't created. Defer the extra call
to the image work queue to avoid locking issues.
Yin Congmin [Sat, 25 Jun 2022 09:43:52 +0000 (17:43 +0800)]
cmake: rename a series of pmem libraries to pmdk
At first, libpmem was the only library. Later, pmem related libraries
such as libpmemobj and libpmem2 were gradually added. These libraries
were also integrated into one named pmdk. So rename to pmdk.
Kefu Chai [Sat, 25 Jun 2022 14:27:02 +0000 (22:27 +0800)]
cmake: use CMAKE_<LANG>_COMPILER_LAUNCHER for configuring ccache
ccache only works for c and c++, so instead of using the universal
`RULE_LAUNCH_COMPILE` use `CMAKE_<LANG>_COMPILER_LAUNCHER` instead,
so ccache is only configured for c and c++ compilation. this is a better
solution for integrating ccache into our building system.
Yin Congmin [Fri, 13 May 2022 12:44:53 +0000 (20:44 +0800)]
install-deps: install pmdk libraries
Install libpmem and libpmemobj under focal ubuntu. the version of apt
list can meet the current requirements. libpmemobj require >=1.8.
Libpmem has no version requirements.
Yin Congmin [Sat, 25 Jun 2022 09:04:44 +0000 (17:04 +0800)]
cmake: lower the required version of libpmem to 1.8
The upgrade of pmemobj in https://github.com/ceph/ceph/pull/40493
is to introduce new API. The minimum version requirement is 1.8.
Therefore, the requirements for find_package can be lowered.
David Galloway [Fri, 24 Jun 2022 16:27:43 +0000 (12:27 -0400)]
.github: Add labels while PR is open
I think https://github.com/tibdex/backport will only create backport PRs if our doc/releases PRs are labelled *and then* closed. This action currently labels after the PR is closed.
Signed-off-by: David Galloway <dgallowa@redhat.com>
Ronen Friedman [Mon, 20 Jun 2022 12:47:57 +0000 (12:47 +0000)]
scrub/osd: disable blocked-scrub warnings during some tests
As some Teuthology tests seem to block objects for long minutes,
we must not issue the "scrub is blocked for too long" warning
(that warning causes the tests to fail).
A new configuration parameter now controls the grace period before
the warning is issued. Some tests were modified to set this
configuration parameter to a large value.
projects / ceph.git / log