]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Radoslaw Zarzynski [Wed, 22 Sep 2021 16:42:19 +0000 (16:42 +0000)]
crimson/os/alienstore: fix nullptr deref in OnCommit::finish().
`seastar::engine()` is available only for Seastar's threads;
it shouldn't be called outside of a reactor thread.
Unfortunately, this assumption is violated in `AlienStore`
where `OnCommit::finish()`, executed from a finisher thread
of `BlueStore`, calls `alien()` on `seastar::engine()`.
The net effect are crashes like the following one:
```
INFO 2021-09-22 14:26:33,214 [shard 0] osd - operator() writing superblock cluster_fsid
1d8f7908 -2ebf-4a91-ae70-
f445668c126b osd_fsid
4da9fe9a -1da5-4ea9-aa79-
a1178165ede5 [381/1839]
Segmentation fault.
Backtrace:
0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
4# (anonymous namespace)::OnCommit::finish(int) at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:53
5# Context::complete(int) at /home/rzarzynski/ceph1/build/../src/include/Context.h:100
6# Finisher::finisher_thread_entry() at /home/rzarzynski/ceph1/build/../src/common/Finisher.cc:65
7# 0x00007F16BBA0915A in /lib64/libpthread.so.0
8# clone in /lib64/libc.so.6
Dump of siginfo:
...
si_addr: 0x10
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:45 +0000 (21:28 +0000)]
crimson/common: add safe_then_unpack() to errorated futures.
It was a prerequisite for another commit I finally thrown
away. However, this little bit can be still be useful even
for the sake of compliance with the interruptible variant.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:02 +0000 (21:28 +0000)]
crimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Mon, 20 Sep 2021 13:59:30 +0000 (13:59 +0000)]
crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part.
This is a follow-up to commit
480273082718230d64d2c068db1b22f44997bdc8 .
`seastar::do_with()` doesn't extend the life-time of the callable's
```cpp
auto
do_with_impl(T1&& rv1, T2&& rv2, More&&... more) {
// ...
auto&& just_values = cherry_pick_tuple(idx(), std::move(all));
auto&& just_func = std::move(std::get<nr>(std::move(all)));
// ...
auto task = std::apply(
[](auto&&... x) {
return std::make_unique<internal::do_with_state<value_tuple, ret_type>>(std::forward<decltype(x)>(x)...);
},
std::move(just_values));
auto fut = std::apply(just_func, task->data());
// ...
}
```
closure. This means the `addrs` needs to be captured by value also
by the lambda passed to `repeat_until_value()`.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
David Galloway [Fri, 17 Sep 2021 13:38:54 +0000 (09:38 -0400)]
Merge pull request #43154 from ceph/wip-yuriw-pacific-16.2.6-RN
doc: 16.2.6 Release Notes
zdover23 [Fri, 17 Sep 2021 05:52:34 +0000 (15:52 +1000)]
Merge pull request #43197 from anthonyeleven/anthonyeleven/stale-read
doc: clarify stale_read and placement-groups wording
Reviewed-by: Zac Dover <zac.dover@gmail.com>
Patrick Donnelly [Fri, 17 Sep 2021 01:47:40 +0000 (21:47 -0400)]
Merge PR #43122 into master
* refs/pull/43122/head:
qa: add test for standby-replay marking rank damaged
MDSMonitor: handle damaged from standby-replay
mds: add config to mark rank damaged in standby-replay
include: unset std::hex after printing CompatSet
mds: refactor iterator lookup
mds: harden rank lookup
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Anthony D'Atri [Thu, 16 Sep 2021 23:40:33 +0000 (16:40 -0700)]
doc: clarify stale_read and placement-groups wording
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
Kefu Chai [Thu, 16 Sep 2021 23:23:14 +0000 (07:23 +0800)]
Merge pull request #43011 from rzarzynski/wip-crimson-opsequencer-ooo-lesser-footprint
crimson/osd: bring OpRegistry-based OpSequencer to handle out-of-order ClientRequests
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Samuel Just [Thu, 16 Sep 2021 19:22:18 +0000 (12:22 -0700)]
Merge pull request #43192 from rzarzynski/wip-crimson-net-dangling-bindvec
crimson/net: fix dangling addrvec in SocketMessenger::bind().
Reviewed-by: Samuel Just <sjust@redhat.com>
David Galloway [Thu, 16 Sep 2021 17:57:16 +0000 (13:57 -0400)]
doc/releases/pacific.rst: Add note about Bullseye
Radoslaw Zarzynski [Thu, 16 Sep 2021 17:04:28 +0000 (17:04 +0000)]
crimson/net: fix dangling addrvec in SocketMessenger::bind().
`SocketMessenger::bind()` takes the address vector by `const&`.
while the callers in `OSD` are passing a temporary:
```cpp
cluster_msgr->bind(pick_addresses(CEPH_PICK_ADDRESS_CLUSTER))
// ...
public_msgr->bind(pick_addresses(CEPH_PICK_ADDRESS_PUBLIC))
// ...
```
```cpp
entity_addrvec_t pick_addresses(int what) {
// ...
return addrs;
}
```
The net result is a life-time mismatch and crashes like the following
one:
```
WARN 2021-09-10 20:01:57,791 [shard 0] ms - [osd.0(client) v2:172.17.0.1:6800/
3172806564 @59988 >> mgr.? v2:172.17.0.7:6800/
3846062660 ] waiting 3.2 seconds ...
/opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/stl_iterator.h:820:17: runtime error: reference binding to misaligned address 0x000041b58ab3 for type 'const struct entity_addr_t', which requires 4 byte alignment
0x000041b58ab3: note: pointer points here
<memory cannot be printed>
/opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/stl_vector.h:1132:16: runtime error: reference binding to misaligned address 0x000041b58ab3 for type 'const struct value_type', which requires 4 byte alignment
0x000041b58ab3: note: pointer points here
<memory cannot be printed>
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-7486-g27cb19ed /rpm/el8/BUILD/
ceph-17.0.0-7486-g27cb19ed /src/msg/msg_types.h:561:22: runtime error: reference binding to misaligned address 0x000041b58ab3 for type 'const struct entity_addr_t', which requires 4 byte alignment
0x000041b58ab3: note: pointer points here
<memory cannot be printed>
Segmentation fault on shard 0.
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Neha Ojha [Thu, 16 Sep 2021 16:49:38 +0000 (16:49 +0000)]
doc/releases/pacific.rst: add core release notes
Signed-off-by: Neha Ojha <nojha@redhat.com>
Radoslaw Zarzynski [Wed, 1 Sep 2021 12:57:54 +0000 (12:57 +0000)]
crimson/osd: rework OpSequencer for out-of-order op execution.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Samuel Just [Thu, 16 Sep 2021 03:59:33 +0000 (20:59 -0700)]
Merge pull request #43180 from athanatos/sjust/wip-52630
crimson/net/SocketMessenger: include sleep.hh
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Samuel Just [Thu, 16 Sep 2021 03:59:09 +0000 (20:59 -0700)]
Merge pull request #43178 from athanatos/sjust/wip-52623
crimson/os/seastore/cache: Cache::get_root check for invalid
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Sage Weil [Thu, 16 Sep 2021 02:50:56 +0000 (22:50 -0400)]
Merge PR #43044 into master
* refs/pull/43044/head:
mgr/rook: add unit tests to Rook module
mgr/rook: translate placement spec to node selector and vice versa
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Sage Weil [Thu, 16 Sep 2021 02:48:36 +0000 (22:48 -0400)]
Merge PR #43172 into master
* refs/pull/43172/head:
qa/tasks/kubeadm: modify (do not clobber) daemon.json
Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
Samuel Just [Thu, 16 Sep 2021 02:31:52 +0000 (02:31 +0000)]
crimson/net/SocketMessenger: include sleep.hh
4a00a145 added a call to seastar::sleep.
Fixes: https://tracker.ceph.com/issues/52630
Signed-off-by: Samuel Just <sjust@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:37:26 +0000 (22:37 -0400)]
Merge PR #43022 into master
* refs/pull/43022/head:
doc/cephfs: add mds_dir_max_entries option into mds-config-ref.rst
mds: add mds_dir_max_entries config option
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:36:18 +0000 (22:36 -0400)]
Merge PR #42943 into master
* refs/pull/42943/head:
mon: do not quickly mark mds laggy when MON_DOWN
mon: refactor quorum age calculation
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:35:37 +0000 (22:35 -0400)]
Merge PR #42846 into master
* refs/pull/42846/head:
mds: Add new flag to MClientSession
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:34:58 +0000 (22:34 -0400)]
Merge PR #42737 into master
* refs/pull/42737/head:
mds: ensure that we send the btime in cap messages
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:34:23 +0000 (22:34 -0400)]
Merge PR #42719 into master
* refs/pull/42719/head:
mgr/volumes: Fix permission during subvol creation with mode
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Samuel Just [Wed, 15 Sep 2021 21:23:29 +0000 (21:23 +0000)]
crimson/os/seastore/cache: Cache::get_root check for invalid
As with Cache::get_extent, we need to check that the ref
is still valid after wait_io since it's not in the
read set yet.
Fixes: https://tracker.ceph.com/issues/52623
Signed-off-by: Samuel Just <sjust@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:33:18 +0000 (22:33 -0400)]
Merge PR #42584 into master
* refs/pull/42584/head:
doc: fix `daemon status` interface (exclude file system name)
test: adjust mirroring tests for `daemon status` change
mgr/mirroring: `daemon status` command does not require file system name
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 16 Sep 2021 02:32:31 +0000 (22:32 -0400)]
Merge PR #42541 into master
* refs/pull/42541/head:
client: switch to use ceph_assert instead of assert
mds: switch to use ceph_assert() instead of assert()
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Samuel Just [Thu, 16 Sep 2021 01:37:31 +0000 (18:37 -0700)]
crimson/os/seastore/cache: rename invalidate(Transaction&, CachedExtent&)
This particular overload is really only declaring the Transaction
conflicted -- it assumes the extent is already invalid. Rename to
mark_transaction_conflicted instead to avoid confusion with
invalidate(CachedExtent&).
Signed-off-by: Samuel Just <sjust@redhat.com>
Sage Weil [Wed, 15 Sep 2021 15:53:32 +0000 (10:53 -0500)]
qa/tasks/kubeadm: modify (do not clobber) daemon.json
Otherwise we blow away the mirror config.
Signed-off-by: Sage Weil <sage@newdream.net>
Yuri Weinstein [Wed, 15 Sep 2021 15:42:33 +0000 (15:42 +0000)]
doc: updated the change log with three additional PRs merged on top
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Ernesto Puerta [Tue, 14 Sep 2021 17:18:00 +0000 (19:18 +0200)]
doc: dashboard motd change
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Patrick Donnelly [Tue, 14 Sep 2021 15:12:10 +0000 (11:12 -0400)]
doc: note simpler MDS upgrade procedure for v16.2.6
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Mon, 13 Sep 2021 21:11:15 +0000 (21:11 +0000)]
doc: 16.2.6 Release Notes
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Kefu Chai [Wed, 15 Sep 2021 15:17:29 +0000 (23:17 +0800)]
Merge pull request #43146 from jianwei1216/fix-partial-recovery-become-whole-object-recovery-for-master
osd: fix partial recovery become whole object recovery after restart osd
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Kefu Chai [Wed, 15 Sep 2021 13:52:42 +0000 (21:52 +0800)]
Merge pull request #42465 from tchaikov/wip-ceph-volume
debian, rpm: split ceph-volume into a separated package
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Ernesto Puerta [Wed, 15 Sep 2021 13:40:58 +0000 (15:40 +0200)]
Merge pull request #43019 from rhcs-dashboard/fix-52028-master
mgr/dashboard: Incorrect MTU mismatch warning
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
Sebastian Wagner [Wed, 15 Sep 2021 06:38:48 +0000 (08:38 +0200)]
Merge pull request #43115 from mgfritch/cephadm-rm-repo-no-podman
cephadm: skip podman check during `rm-repo`
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Samuel Just [Wed, 15 Sep 2021 04:44:22 +0000 (21:44 -0700)]
Merge pull request #43129 from cyx1231st/wip-seastore-optimize-onode
crimson/os/seastore: minimize the static size of onode_layout_t
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Wed, 15 Sep 2021 04:41:46 +0000 (21:41 -0700)]
Merge pull request #43106 from rzarzynski/wip-crimson-msgr-bind
crimson: add support for ms_bind_retry_{delay,count}
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Samuel Just [Wed, 15 Sep 2021 04:41:15 +0000 (21:41 -0700)]
Merge pull request #43157 from liu-chunmei/seastore-binary-search
crimson/seastore:: add binary search for lba and omap iterator search
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Yingxin Cheng [Fri, 10 Sep 2021 08:37:05 +0000 (16:37 +0800)]
crimson/os/seastore: minimize the static size of onode_layout_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 10 Sep 2021 08:34:10 +0000 (16:34 +0800)]
crimson: exclude oid when encode/decode object_info_t to/from object store
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
chunmei-liu [Mon, 13 Sep 2021 23:02:57 +0000 (16:02 -0700)]
crimson/seastore:: add binary search for lba and omap iterator search
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Samuel Just [Tue, 14 Sep 2021 20:36:49 +0000 (13:36 -0700)]
Merge pull request #43158 from athanatos/sjust/wip-seastore-omap
crimson/os/seastore: SeaStore::get_omap_iterator should seek_to_first
Reviewed-by: Kefu Chai <kchai@redhat.com>
Samuel Just [Tue, 14 Sep 2021 20:36:24 +0000 (13:36 -0700)]
Merge pull request #43161 from rzarzynski/wip-crimson-fix-no-bs
crimson/os: allow to build crimson when WITH_BLUESTORE=OFF
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 4 Aug 2021 11:20:15 +0000 (19:20 +0800)]
rpm, debian: drop xmlstarlet from build deps
xmlstarlet is not used anymore while performing "make check", so drop it
from the build dependency list.
it is still used by the teuthology test though, so keep it as a
dependency of ceph-test.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 4 Aug 2021 11:00:01 +0000 (19:00 +0800)]
ceph.spec: remove parted from ceph-base deps
parted is only used by ceph-volume
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 28 Jul 2021 07:32:58 +0000 (15:32 +0800)]
ceph.spec, debian: move deps from base to ceph-volume
this change makes util-linux, xfsprogs and e2fsprogs runtime deps of ceph-volume
ceph-volume uses blkid and lsblk, which are in turn packaged by
util-linux.
util-linux were added as build dependency to fulfill the needs of
ceph-disk. and we tested ceph-disk as part of "make check", since
ceph-disk was dropped, there is no need to have util-linux as
build dependency anymore.
the same applies to e2fsprogs and xfsprogs
see also
a99177834120e7a2c4592054f6a8b8736e0ffb92
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 28 Jul 2021 07:16:19 +0000 (15:16 +0800)]
debian/control: make cryptsetup-bin a ceph-volume runtime dep
was originally added by
6a45b8ed041514816d09b2808d2d390eadbd0dd9 and
cfcaceac44d6b7b7c55e81d0bfb05f4893f3b1d0 as a dependency of ceph-disk.
although, ceph-disk was replaced by ceph-volume, cryptsetup-bin is still
used.
but it should be a runtime dependency of ceph-volume. and it is not a
build dependency of ceph, so it's removed from the build dependency list
as well.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 28 Jul 2021 07:14:42 +0000 (15:14 +0800)]
ceph.spec.in: make cryptsetup a ceph-volume runtime dep.
cryptsetup is solely used by ceph-volume, so it should be a runtime
dependency of ceph-volume, not ceph-base.
and remove it fomr build-dependency list, as we don't use this tool when
building ceph.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 23 Jul 2021 11:19:43 +0000 (19:19 +0800)]
ceph.spec.in: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 23 Jul 2021 10:55:18 +0000 (18:55 +0800)]
debian: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.
in this change
* ceph-volume is extracted out into a separated package
* ceph-volume depends on ceph-osd, as it deploys it and relies on
an already-installed ceph-osd in the system.
* ceph-osd recommends ceph-volume. as ceph-osd can be used as a
standalone package. but ceph-volume enhances it. also, to ensure
the existing users to get ceph-volume installed along with
ceph-osd.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 23 Jul 2021 10:36:37 +0000 (18:36 +0800)]
rpm,deb: make parted runtime dependency of ceph-osd
we don't run parted when building ceph, instead, it is a runtime
dependency of ceph-volume, which is packaged in ceph-osd.
so in this change
* parted is removed from build dependency list
* parted is added as a runtime depedency of ceph-osd.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Radoslaw Zarzynski [Tue, 14 Sep 2021 15:05:32 +0000 (15:05 +0000)]
crimson/os: add the preamble to futurized_store.cc.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Tue, 14 Sep 2021 13:48:58 +0000 (13:48 +0000)]
crimson/os: allow to build crimson when WITH_BLUESTORE=OFF.
This lets to ensure nobody is accidentally linking with the alienized
version of `common`.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Thu, 9 Sep 2021 06:16:09 +0000 (06:16 +0000)]
crimson: switch try_bind() to bind().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Thu, 9 Sep 2021 06:11:59 +0000 (06:11 +0000)]
crimson/net: bind() respects now ms_bind_retry_{delay,count}.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Thu, 9 Sep 2021 06:09:54 +0000 (06:09 +0000)]
crimson/net: make try_bind() aware about address_not_available.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Wed, 8 Sep 2021 19:30:17 +0000 (19:30 +0000)]
crimson/net: decouple listen_ertr from bind_ertr.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Wed, 8 Sep 2021 19:27:44 +0000 (19:27 +0000)]
crimson/net: decouple listen_ertr from FixedCPUServerSocket.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Guillaume Abrioux [Tue, 14 Sep 2021 08:00:14 +0000 (10:00 +0200)]
Merge pull request #43109 from guits/guits-cv-24044
ceph-volume: util/prepare fix osd_id_available()
Samuel Just [Tue, 14 Sep 2021 04:26:33 +0000 (21:26 -0700)]
crimson/os/seastore: SeaStore::get_omap_iterator should seek_to_first
Add debugging as well.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Tue, 14 Sep 2021 04:27:24 +0000 (21:27 -0700)]
vstart.sh: default to msgr2 for crimson
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Tue, 14 Sep 2021 03:33:31 +0000 (20:33 -0700)]
Merge pull request #43123 from liu-chunmei/lba_metrics_iter
crimson/seastore: add lba tree metrics for lba tree profiling
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
chunmei-liu [Thu, 9 Sep 2021 05:25:45 +0000 (22:25 -0700)]
crimson/seastore: add lba tree metrics for lba tree profiling
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Samuel Just [Tue, 14 Sep 2021 01:31:47 +0000 (18:31 -0700)]
Merge pull request #43152 from athanatos/sjust/wip-simplify-delayed
crimson/sesatore: simplify epm related transaction structures, fix some bugs
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Samuel Just [Sat, 11 Sep 2021 04:28:24 +0000 (21:28 -0700)]
crimson/os/seastore/transaction: add trace debugging for write_set
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Sat, 11 Sep 2021 05:44:26 +0000 (05:44 +0000)]
crimson/os/seastore/cached_extent: fix unused variable warning
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Sat, 11 Sep 2021 04:11:24 +0000 (21:11 -0700)]
crimson/os/seastore/transaction_manager: fix delayed alloc addr in rewrite_logical_extent
Setting nlextent's to the addr of the original extent violates invariants
in the intrusive_set we're using for ExtentIndex and generally breaks the
assumption in cache that paddrs are unique until reuse. This means we'll
do an extra update_mapping for now -- we can refactor SegmentCleaner to
avoid it later.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Sat, 11 Sep 2021 04:08:57 +0000 (21:08 -0700)]
crimson/os/seastore/scanner: fix budget_used ref in scan_valid_records
Fixes: https://tracker.ceph.com/issues/52582
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 10 Sep 2021 22:41:55 +0000 (22:41 +0000)]
crimson/os/seastore/cache: more get_extent debugging
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 10 Sep 2021 22:12:46 +0000 (15:12 -0700)]
crimson/os/seastore/extent_placement_manager: clarify else branch delayed_alloc_or_ool_write
Also shorten long line.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 10 Sep 2021 01:11:05 +0000 (18:11 -0700)]
crimson/os/seastore: replace fresh_block_list with inline/ool_block_list
With this change, Transaction::write_set is the union of mutually exclusive
lists
- delayed_alloc_list, inline_block_list, and mutated_block_list prior to
calling ExtentPlacementManager::delayed_alloc_or_ool_write
and
- inline_block_list, ool_block_list, and mutated_block_list after
calling ExtentPlacementManager::delayed_alloc_or_ool_write
Signed-off-by: Samuel Just <sjust@redhat.com>
Sage Weil [Mon, 13 Sep 2021 22:28:43 +0000 (18:28 -0400)]
Merge PR #43136 into master
* refs/pull/43136/head:
qa/tasks/kubeadm: change calico encap to IPIPCrossSubnet
qa/suites/orch/rook/smoke: add host networking to matrix
qa/tasks/rook: fix shadowing of config arg in rook_cluster()
Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
Ilya Dryomov [Mon, 13 Sep 2021 21:00:17 +0000 (23:00 +0200)]
Merge pull request #42417 from rzarzynski/wip-bl-optimize-emptybptrs
common/bl, tests: optimize carriage handling in bufferlist::c_str()
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Sage Weil [Mon, 13 Sep 2021 18:43:20 +0000 (13:43 -0500)]
qa/tasks/kubeadm: change calico encap to IPIPCrossSubnet
Signed-off-by: Sage Weil <sage@newdream.net>
Samuel Just [Fri, 10 Sep 2021 01:07:52 +0000 (18:07 -0700)]
crimson/os/seastore/cached_extent: assert !parent_index in ExtentIndex::insert
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 10 Sep 2021 01:07:31 +0000 (18:07 -0700)]
crimson/so/seastore: clarify ExtentIndex clear()
ExtentIndex::clear() needs to handle clearing parent_index.
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Fri, 3 Sep 2021 20:22:46 +0000 (20:22 +0000)]
crimson/os/seastore/transaction: remove delayed_set
delayed_set isn't necessary. Delayed extents have distinct paddr segments
already, so it's simpler to have a single index for all pending extents.
Signed-off-by: Samuel Just <sjust@redhat.com>
Radoslaw Zarzynski [Tue, 20 Jul 2021 10:32:36 +0000 (10:32 +0000)]
tests/bl: verify the carriage handling in bufferlist::c_str().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Thu, 5 Aug 2021 00:04:57 +0000 (00:04 +0000)]
common/bl: abstract from _carriage checking in c_str().
This is intended to enchance readability and sligthly
generalize the previous, `_carriage`-aware optimization.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Mon, 19 Jul 2021 14:06:20 +0000 (14:06 +0000)]
common/bl: c_str() considers _carriage now.
It could be that `_carriage` points on empty bptr which is
also stored in the `_buffers` container -- this happens
when we'are trying to preserve the appendable space to not
waste memory and avoid new allocation.
Before the change this approach was imposing unncessary
rebuilding on `c_str()`. Now we take into consideration
the special case when a bufferlist has the extra, empty
bptr at its end.
Fixes: https://tracker.ceph.com/issues/51725
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Guillaume Abrioux [Thu, 9 Sep 2021 08:23:43 +0000 (10:23 +0200)]
ceph-volume: util/prepare fix osd_id_available()
The current check only allows to request an OSD id that exists but
marked as 'destroyed'.
With this small fix, we can now use `--osd-id` with an id that doesn't
exist at all.
Fixes: https://tracker.ceph.com/issues/50880
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Joseph Sawaya [Fri, 3 Sep 2021 14:28:57 +0000 (10:28 -0400)]
mgr/rook: add unit tests to Rook module
This commit creates a unit test folder for rook, configurates tox
and creates a unit test to test the translation of placement specs
to node selectors and vice versa.
Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
Joseph Sawaya [Fri, 3 Sep 2021 14:24:24 +0000 (10:24 -0400)]
mgr/rook: translate placement spec to node selector and vice versa
This commit creates methods for translating PlacementSpecs to NodeSelectors
and vice versa, this is a general method that would be used by orch commands that
have a placement option. It's able to translate specs that specify a label, hosts
and the host_pattern '*'.
Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
Sebastian Wagner [Mon, 13 Sep 2021 14:38:01 +0000 (16:38 +0200)]
Merge pull request #43141 from sebastian-philipp/doc-orch-rm
doc/cephadm: Removing a service
Reviewed-by: Michael Fritch <mfritch@suse.com>
Sage Weil [Mon, 13 Sep 2021 14:21:32 +0000 (10:21 -0400)]
Merge PR #42998 into master
* refs/pull/42998/head:
qa/tasks/rook: add testing for host label add/rm
mgr/rook: host add/rm label in rook orchestrator
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Kefu Chai [Mon, 13 Sep 2021 13:45:11 +0000 (21:45 +0800)]
Merge pull request #43147 from krunerge/fix_typo
doc/dev/crimson/poseidonstore: fix typo
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Dimitri Savineau [Mon, 13 Sep 2021 13:13:07 +0000 (09:13 -0400)]
Merge pull request #43063 from guits/guits-trailing-spaces
ceph-volume: remove trailing spaces in code
Ronen Friedman [Mon, 13 Sep 2021 11:23:52 +0000 (14:23 +0300)]
Merge pull request #42951 from ronen-fr/wip-ronenf-pg-dtor
osd/scrub: destruct the scrubber shortly before the PG is destructed
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>-
Reviewed-by: Neha Ojha <nojha@redhat.com>
krunerge [Mon, 13 Sep 2021 11:12:36 +0000 (19:12 +0800)]
doc/dev/crimson/poseidonstore: fix typo
Signed-off-by: krunerge <krunerge@tencent.com>
Jianwei Zhang [Mon, 13 Sep 2021 10:13:18 +0000 (18:13 +0800)]
osd: fix partial recovery become whole object recovery after restart osd
support SERVER_OCTOPUS feature for pg_missing_item::encode()
Fixes: https://tracker.ceph.com/issues/52583
Signed-off-by: Jianwei Zhang <jianwei1216@qq.com>
Ilya Dryomov [Mon, 13 Sep 2021 10:04:34 +0000 (12:04 +0200)]
Merge pull request #42555 from hualongfeng/retire_error_change
librbd/cache/pwl/ssd: remove correct m_blocks_to_log_entries entry
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Ilya Dryomov [Mon, 13 Sep 2021 09:44:09 +0000 (11:44 +0200)]
Merge pull request #43038 from majianpeng/librbd-pwl-exclusive-lock
librbd: require exclusive lock for reads if pwl cache is enabled
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Feng Hualong [Wed, 28 Jul 2021 03:42:44 +0000 (11:42 +0800)]
librbd/cache/pwl/ssd: remove correct m_blocks_to_log_entries entry
When retiring, m_blocks_to_log_entries doesn't remove
corresponding write_entry (should be `*it` not `entry`)
that will be retired. It leads to read error. And
there should also consider discard entries.
Fixes: https://tracker.ceph.com/issues/52579
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
Sebastian Wagner [Mon, 13 Sep 2021 07:39:16 +0000 (09:39 +0200)]
Merge pull request #43143 from sebastian-philipp/further-reading
doc/cephadm: Add lots of links to other chapters
Reviewed-by: Zac Dover <zac.dover@gmail.com>
Ronen Friedman [Sun, 12 Sep 2021 13:32:44 +0000 (16:32 +0300)]
Merge pull request #41869 from ronen-fr/wip-ronenf-fmt-scrub
common, osd: add fmtlib formatting of some OSD types
Reviewed-by: Kefu Chai <kchai@redhat.com>
Ronen Friedman [Sun, 12 Sep 2021 12:47:52 +0000 (15:47 +0300)]
Merge pull request #43056 from Matan-B/wip-matanb-scrub-passkey
osd: Replacing friend declarations for passkey interface
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Ronen Friedman [Sun, 12 Sep 2021 12:40:22 +0000 (15:40 +0300)]
Merge pull request #42684 from ronen-fr/wip-ronenf-scrub-token
osd/scrub: tag replica scrub messages to identify stale events
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Sebastian Wagner [Sat, 11 Sep 2021 18:02:44 +0000 (20:02 +0200)]
doc/cephadm: Add lots of links to other chapters
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>