ceph.git
2 years agoMerge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit wip-rgw-zipper-standalone 43173/head
Samuel Just [Thu, 23 Sep 2021 05:59:57 +0000 (22:59 -0700)]
Merge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit

crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agocrimson/os/alienstore: fix nullptr deref in OnCommit::finish(). 43262/head
Radoslaw Zarzynski [Wed, 22 Sep 2021 16:42:19 +0000 (16:42 +0000)]
crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

`seastar::engine()` is available only for Seastar's threads;
it shouldn't be called outside of a reactor thread.
Unfortunately, this assumption is violated in `AlienStore`
where `OnCommit::finish()`, executed from a finisher thread
of `BlueStore`, calls `alien()` on `seastar::engine()`.
The net effect are crashes like the following one:

```
INFO  2021-09-22 14:26:33,214 [shard 0] osd - operator() writing superblock cluster_fsid 1d8f7908-2ebf-4a91-ae70-f445668c126b osd_fsid 4da9fe9a-1da5-4ea9-aa79-a1178165ede5         [381/1839]
Segmentation fault.
Backtrace:
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
 4# (anonymous namespace)::OnCommit::finish(int) at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:53
 5# Context::complete(int) at /home/rzarzynski/ceph1/build/../src/include/Context.h:100
 6# Finisher::finisher_thread_entry() at /home/rzarzynski/ceph1/build/../src/common/Finisher.cc:65
 7# 0x00007F16BBA0915A in /lib64/libpthread.so.0
 8# clone in /lib64/libc.so.6
Dump of siginfo:
  ...
  si_addr: 0x10
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #40728 from guits/doc_fix_pattern_rgw_nfs
Sébastien Han [Wed, 22 Sep 2021 15:08:31 +0000 (17:08 +0200)]
Merge pull request #40728 from guits/doc_fix_pattern_rgw_nfs

doc/rgw-nfs: use same pattern for keyring name

2 years agoMerge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order
Samuel Just [Wed, 22 Sep 2021 14:05:01 +0000 (07:05 -0700)]
Merge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order

crimson/onode-staged-tree: convert hash to the reversed version

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint
Kefu Chai [Wed, 22 Sep 2021 13:47:37 +0000 (21:47 +0800)]
Merge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint

crimson/os/seastore: add missing hints in omap tree

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #43055 from soumyakoduri/wip-skoduri-lua
Daniel Gryniewicz [Wed, 22 Sep 2021 12:12:23 +0000 (08:12 -0400)]
Merge pull request #43055 from soumyakoduri/wip-skoduri-lua

rgw/lua: Install the packages only for RadosStore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Daniel Gryniewicz <dang@redhat.com>

2 years agoMerge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart
Daniel Gryniewicz [Wed, 22 Sep 2021 12:11:30 +0000 (08:11 -0400)]
Merge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart

rgw: Add option to configure backend store

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object
Daniel Gryniewicz [Wed, 22 Sep 2021 12:10:52 +0000 (08:10 -0400)]
Merge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object

rgw/dbstore object APIs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #43241 from sebastian-philipp/suites-orch-labelere
Sebastian Wagner [Wed, 22 Sep 2021 09:34:35 +0000 (11:34 +0200)]
Merge pull request #43241 from sebastian-philipp/suites-orch-labelere

.github: fix path to cephadm suite

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agocrimson/onode-staged-tree: consolidate laddr hint calculation 43254/head
Yingxin Cheng [Wed, 22 Sep 2021 07:43:23 +0000 (15:43 +0800)]
crimson/onode-staged-tree: consolidate laddr hint calculation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/onode-staged-tree: convert hash to the reversed version
Yingxin Cheng [Wed, 22 Sep 2021 07:36:22 +0000 (15:36 +0800)]
crimson/onode-staged-tree: convert hash to the reversed version

Store the reversed version of object hash to make sure that onodes in
the same PG are sorted together.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/onode-staged-tree: print cursh-hash in hex mode
Yingxin Cheng [Wed, 22 Sep 2021 06:52:18 +0000 (14:52 +0800)]
crimson/onode-staged-tree: print cursh-hash in hex mode

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore: add missing hints in omap tree 43249/head
Yingxin Cheng [Sat, 18 Sep 2021 07:10:38 +0000 (15:10 +0800)]
crimson/os/seastore: add missing hints in omap tree

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack
Samuel Just [Wed, 22 Sep 2021 00:12:58 +0000 (17:12 -0700)]
Merge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack

crimson/common: add safe_then_unpack() to errorated futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2 years agocrimson/common: add safe_then_unpack() to errorated futures. 43247/head
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:45 +0000 (21:28 +0000)]
crimson/common: add safe_then_unpack() to errorated futures.

It was a prerequisite for another commit I finally thrown
away. However, this little bit can be still be useful even
for the sake of compliance with the interruptible variant.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.
Radoslaw Zarzynski [Tue, 21 Sep 2021 21:28:02 +0000 (21:28 +0000)]
crimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2
Samuel Just [Tue, 21 Sep 2021 18:56:12 +0000 (11:56 -0700)]
Merge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2

crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot
Samuel Just [Tue, 21 Sep 2021 18:55:30 +0000 (11:55 -0700)]
Merge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot

tests/crimson: don't be so verbose when run by the 'make check' bot.

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge PR #43219 into master
Patrick Donnelly [Tue, 21 Sep 2021 18:03:08 +0000 (14:03 -0400)]
Merge PR #43219 into master

* refs/pull/43219/head:
doc: clarify MDS failed state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoMerge PR #43156 into master
Sage Weil [Tue, 21 Sep 2021 15:37:50 +0000 (11:37 -0400)]
Merge PR #43156 into master

* refs/pull/43156/head:
qa/suites/orch/rook/smoke: add flannel back in

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2 years agocrimson/net: fix dangling addrvec in bind(), the repeat_until_value() part. 43243/head
Radoslaw Zarzynski [Mon, 20 Sep 2021 13:59:30 +0000 (13:59 +0000)]
crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part.

This is a follow-up to commit 480273082718230d64d2c068db1b22f44997bdc8.

`seastar::do_with()` doesn't extend the life-time of the callable's

```cpp
auto
do_with_impl(T1&& rv1, T2&& rv2, More&&... more) {
    // ...
    auto&& just_values = cherry_pick_tuple(idx(), std::move(all));
    auto&& just_func = std::move(std::get<nr>(std::move(all)));
    // ...
    auto task = std::apply(
        [](auto&&... x) {
            return std::make_unique<internal::do_with_state<value_tuple, ret_type>>(std::forward<decltype(x)>(x)...);
        },
        std::move(just_values));
    auto fut = std::apply(just_func, task->data());
    // ...
}
```

closure. This means the `addrs` needs to be captured by value also
by the lambda passed to `repeat_until_value()`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #43225 from guits/fix-iscsi-doc
Ilya Dryomov [Tue, 21 Sep 2021 12:35:59 +0000 (14:35 +0200)]
Merge pull request #43225 from guits/fix-iscsi-doc

doc/iscsi: fix some typos

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years ago.github: fix path to cepham suite 43241/head
Sebastian Wagner [Tue, 21 Sep 2021 11:35:03 +0000 (13:35 +0200)]
.github: fix path to cepham suite

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2 years agoMerge pull request #43237 from javicacheiro/fix_upgrade_now_uses_quay.io
Sebastian Wagner [Tue, 21 Sep 2021 11:29:09 +0000 (13:29 +0200)]
Merge pull request #43237 from javicacheiro/fix_upgrade_now_uses_quay.io

doc:  from v16.2.6 you have to point to the quay.io image

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2 years agoMerge pull request #43142 from sebastian-philipp/c-v-cephadm-osd-activate
Sebastian Wagner [Tue, 21 Sep 2021 11:12:07 +0000 (13:12 +0200)]
Merge pull request #43142 from sebastian-philipp/c-v-cephadm-osd-activate

doc/ceph-volume/activate: Users should call cephadm instead

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agoMerge pull request #43162 from sebastian-philipp/cephadm-service-mgmt-reoder
Sebastian Wagner [Tue, 21 Sep 2021 11:03:08 +0000 (13:03 +0200)]
Merge pull request #43162 from sebastian-philipp/cephadm-service-mgmt-reoder

doc/cephadm/service-management: Move "Updating Service Specifications"

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #43210 from rhcs-dashboard/config-e2e-failure
Ernesto Puerta [Tue, 21 Sep 2021 09:53:41 +0000 (11:53 +0200)]
Merge pull request #43210 from rhcs-dashboard/config-e2e-failure

mgr/dashboard: Fix failing config dashboard e2e check

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agodoc: from v16.2.6 you have to point to the quay.io image if using docker 43237/head
Javier Cacheiro [Tue, 21 Sep 2021 09:37:42 +0000 (11:37 +0200)]
doc: from v16.2.6 you have to point to the quay.io image if using docker

New images are no longer uploaded to the Docker Hub registry, so Docker users should point the upgrade
command to the quay.io image.

Fixes: https://tracker.ceph.com/issues/52683
Signed-off-by: Javier Cacheiro <javier.cacheiro.lopez@cesga.es>
2 years agoMerge pull request #43233 from badone/wip-install-xmlstarlet
Brad Hubbard [Tue, 21 Sep 2021 05:29:28 +0000 (15:29 +1000)]
Merge pull request #43233 from badone/wip-install-xmlstarlet

Revert "rpm, debian: drop xmlstarlet from build deps"

Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoRevert "rpm, debian: drop xmlstarlet from build deps" 43233/head
Brad Hubbard [Tue, 21 Sep 2021 03:14:17 +0000 (13:14 +1000)]
Revert "rpm, debian: drop xmlstarlet from build deps"

This reverts commit 707edc0b2d5283104aeb472131ee94dd59544290.

This commit was somewhat premature in removing the xmlstarlet
dependency.

Fixes: https://tracker.ceph.com/issues/52681
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2 years agoMerge PR #43179 into master
Patrick Donnelly [Tue, 21 Sep 2021 00:48:00 +0000 (20:48 -0400)]
Merge PR #43179 into master

* refs/pull/43179/head:
qa: lengthen grace for fs map showing dead MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2 years agoMerge PR #43164 into master
Patrick Donnelly [Tue, 21 Sep 2021 00:47:19 +0000 (20:47 -0400)]
Merge PR #43164 into master

* refs/pull/43164/head:
test/libcephfs: put inodes after lookup

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge PR #42763 into master
Patrick Donnelly [Tue, 21 Sep 2021 00:46:25 +0000 (20:46 -0400)]
Merge PR #42763 into master

* refs/pull/42763/head:
mon/FSCommands: add 'recover' flag in `fs new` command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agorgw/dbstore: Update README 42911/head
Soumya Koduri [Wed, 15 Sep 2021 19:29:35 +0000 (00:59 +0530)]
rgw/dbstore: Update README

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #43222 from sebastian-philipp/rook-ceph-client-before-3.8
Kefu Chai [Mon, 20 Sep 2021 16:16:35 +0000 (00:16 +0800)]
Merge pull request #43222 from sebastian-philipp/rook-ceph-client-before-3.8

mgr/rook: update rook-client-python to 2737c7467b7

Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
2 years agoMerge PR #43213 into master
Sage Weil [Mon, 20 Sep 2021 15:00:23 +0000 (11:00 -0400)]
Merge PR #43213 into master

* refs/pull/43213/head:
mon/OSDMonitor: make weight set create safer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoMerge PR #43211 into master
Sage Weil [Mon, 20 Sep 2021 14:59:56 +0000 (10:59 -0400)]
Merge PR #43211 into master

* refs/pull/43211/head:
qa/suites/upgrade/pacific-x: skip c-v install on pacific start
qa/packages: install ceph-volume

Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agodoc/iscsi: fix some typos 43225/head
Guillaume Abrioux [Mon, 20 Sep 2021 14:02:33 +0000 (16:02 +0200)]
doc/iscsi: fix some typos

This fixes some typos in the iscsi documentation.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2 years agorgw/dbstore: Object APIs
Soumya Koduri [Wed, 1 Sep 2021 12:10:59 +0000 (17:40 +0530)]
rgw/dbstore: Object APIs

Support for simple Put, Get, Delete, List Ops of Regular Objects on dbstore.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agorgw/dbstore: Insert testid user on startup 43054/head
Soumya Koduri [Wed, 8 Sep 2021 19:10:10 +0000 (00:40 +0530)]
rgw/dbstore: Insert testid user on startup

Its temporary change to insert testid user while
starting up rgw server on dbstore.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agorgw: Add option to configure backend store
Soumya Koduri [Sun, 5 Sep 2021 07:21:18 +0000 (12:51 +0530)]
rgw: Add option to configure backend store

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agorgw: Install the lua packages only for RadosStore 43055/head
Soumya Koduri [Thu, 9 Sep 2021 08:26:16 +0000 (13:56 +0530)]
rgw: Install the lua packages only for RadosStore

Installation, remove and listing of lua packages is supported
only on RadosStore. Verify the same.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #43193 from dang/wip-dang-zipper-lifecycle
Daniel Gryniewicz [Mon, 20 Sep 2021 13:12:52 +0000 (09:12 -0400)]
Merge pull request #43193 from dang/wip-dang-zipper-lifecycle

RGW Zipper - clean up and clarify lifecycle of s->object, s->bucket, and friends

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2 years agoqa/suites/upgrade/pacific-x: skip c-v install on pacific start 43211/head
Sage Weil [Mon, 20 Sep 2021 12:47:45 +0000 (08:47 -0400)]
qa/suites/upgrade/pacific-x: skip c-v install on pacific start

Signed-off-by: Sage Weil <sage@newdream.net>
2 years agomgr/dashboard: Fix failing config dashboard e2e check 43210/head
Nizamudeen A [Fri, 17 Sep 2021 16:07:09 +0000 (21:37 +0530)]
mgr/dashboard: Fix failing config dashboard e2e check

Recently a new osd config has been added in
6ca32bde2e1d0dd58df168126582a570ac09aad6 and this is getting Modified.
So on our dashboard e2e config check which checks for the Modified
filter, this is also coming on the entry. So we need to increase the
count.

Fixes: https://tracker.ceph.com/issues/52649
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agomgr/rook: update rook-client-python to 82673cd7c7a 43222/head
Sebastian Wagner [Mon, 20 Sep 2021 08:22:01 +0000 (10:22 +0200)]
mgr/rook: update rook-client-python to 82673cd7c7a

Fixes: https://tracker.ceph.com/issues/52659
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2 years agoqa/packages: install ceph-volume
Sage Weil [Fri, 17 Sep 2021 15:16:36 +0000 (11:16 -0400)]
qa/packages: install ceph-volume

Signed-off-by: Sage Weil <sage@newdream.net>
2 years agoMerge pull request #43025 from TRYTOBE8TME/wip-rgw-amqp-timeout
Yuval Lifshitz [Sun, 19 Sep 2021 16:26:29 +0000 (19:26 +0300)]
Merge pull request #43025 from TRYTOBE8TME/wip-rgw-amqp-timeout

Wip rgw amqp timeout

2 years agoMerge pull request #42815 from tchaikov/wip-rocksdb
Kefu Chai [Sun, 19 Sep 2021 13:57:41 +0000 (21:57 +0800)]
Merge pull request #42815 from tchaikov/wip-rocksdb

kv/rocksdb_cache: implement methods required by rocksdb v6.22.1

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2 years agodoc: clarify MDS failed state 43219/head
胡玮文 [Sun, 19 Sep 2021 13:12:44 +0000 (21:12 +0800)]
doc: clarify MDS failed state

Since 58eaa237b0a1, an MDS is only promoted if it is compatible with the
file system compat. The users may see persistent failed state even they
have enough standby daemons.

Signed-off-by: 胡玮文 <huww98@outlook.com>
2 years agoMerge pull request #43195 from Thingee/update-rbd-lead
Deepika Upadhyay [Sun, 19 Sep 2021 10:03:16 +0000 (15:33 +0530)]
Merge pull request #43195 from Thingee/update-rbd-lead

doc: updating RBD lead

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoMerge pull request #43208 from abcinje/librados-examples
Kefu Chai [Sat, 18 Sep 2021 14:09:43 +0000 (22:09 +0800)]
Merge pull request #43208 from abcinje/librados-examples

examples/librados: avoid a memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoexamples/librados: avoid a memory leak 43208/head
Injae Kang [Fri, 17 Sep 2021 15:42:41 +0000 (00:42 +0900)]
examples/librados: avoid a memory leak

Avoid a memory leak by deallocating the pre-allocated aio completion.

Signed-off-by: Injae Kang <abcinje@gmail.com>
2 years agoMerge pull request #43212 from ceph/crn-span
Josh Durgin [Sat, 18 Sep 2021 00:20:58 +0000 (17:20 -0700)]
Merge pull request #43212 from ceph/crn-span

src/script/ceph-release-notes: replace breaklink w span

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2 years agoMerge pull request #43206 from Matan-B/wip-matanb-installdeps-pip
Kefu Chai [Fri, 17 Sep 2021 23:27:34 +0000 (07:27 +0800)]
Merge pull request #43206 from Matan-B/wip-matanb-installdeps-pip

install-deps.sh: install newer version of pip

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #42991 from benhanokh/suppress_bogus_errors
Yuri Weinstein [Fri, 17 Sep 2021 19:13:58 +0000 (12:13 -0700)]
Merge pull request #42991 from benhanokh/suppress_bogus_errors

BUG-FIX: NCB code was reporting Bogus error when we had an overlapped…

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2 years agomon/OSDMonitor: make weight set create safer 43213/head
Sage Weil [Fri, 17 Sep 2021 17:51:08 +0000 (12:51 -0500)]
mon/OSDMonitor: make weight set create safer

We cannot return early success based on uncommitted state.

Signed-off-by: Sage Weil <sage@newdream.net>
2 years agosrc/script/ceph-release-notes: replace breaklink w span 43212/head
David Galloway [Fri, 17 Sep 2021 17:47:29 +0000 (13:47 -0400)]
src/script/ceph-release-notes: replace breaklink w span

Not sure why but the <!-- breaklink > was getting rendered as plaintext.

Putting <span></span> breaks the auto hyperlinking.

Signed-off-by: David Galloway <dgallowa@redhat.com>
2 years agoMerge PR #43144 into master
Patrick Donnelly [Fri, 17 Sep 2021 17:47:25 +0000 (13:47 -0400)]
Merge PR #43144 into master

* refs/pull/43144/head:
doc/cephfs: fix typos

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agotests/crimson: don't be so verbose when run by the 'make check' bot. 43209/head
Radoslaw Zarzynski [Fri, 17 Sep 2021 14:09:28 +0000 (14:09 +0000)]
tests/crimson: don't be so verbose when run by the 'make check' bot.

This commit is a try to balance crimson's debug facilities with overwhelming
some hosts the `make check` bot runs on with huge number of log entries;
see https://jenkins.ceph.com/job/ceph-pull-requests-arm64/17127/ for an
exemplification of the problem.

This is achieved by differentiating debug levels depending upon the
`FOR_MAKE_CHECK` env variable. It's a hack / a makeshift solution
only. IMHO ultimately we should introduce a dedicated target for
the bot while leaving `ninja test` for humans which would allow for more
debugs and maybe also for running `vstart.sh`-dependent tests.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #43071 from pritha-srivastava/wip-rgw-sts-op-logs
Matt Benjamin [Fri, 17 Sep 2021 14:18:52 +0000 (10:18 -0400)]
Merge pull request #43071 from pritha-srivastava/wip-rgw-sts-op-logs

rgw/sts: add check for s->auth.identity pointer

2 years agoMerge pull request #43165 from linuxbox2/wip-perl-semicolon
Matt Benjamin [Fri, 17 Sep 2021 14:16:45 +0000 (10:16 -0400)]
Merge pull request #43165 from linuxbox2/wip-perl-semicolon

workunits/rgw: semicolon terminates perl statements

2 years agoRGW Zipper - clean up setting bucket on s->object 43193/head
Daniel Gryniewicz [Thu, 16 Sep 2021 16:53:50 +0000 (12:53 -0400)]
RGW Zipper - clean up setting bucket on s->object

s->object is always created wihtout a bucket, and the bucket is added
later.  The proper place for this is in rgw_build_bucket_policies(), in
the permissions handling callpaths.  Remove all the other places where
bucket is set, ensuring that s->object has a stable bucket pointer.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoRGW Zipper - comment setting s->object in website code
Daniel Gryniewicz [Mon, 13 Sep 2021 17:03:18 +0000 (13:03 -0400)]
RGW Zipper - comment setting s->object in website code

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoRGW - Remove extra src_object from Copy OP
Daniel Gryniewicz [Tue, 31 Aug 2021 17:41:09 +0000 (13:41 -0400)]
RGW - Remove extra src_object from Copy OP

req_state has a src_object, and it's used from places that don't have
access to the Copy OP.  Remove the version from the Copy OP, and
consolidate on the one in req_state.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoRGW - Zipper don't overwrite s->bucket
Daniel Gryniewicz [Tue, 31 Aug 2021 13:22:09 +0000 (09:22 -0400)]
RGW - Zipper don't overwrite s->bucket

Undo an overwrite of s->bucket that was added as part of the zipper
work.  s->bucket should never be overwritten.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoRGW - Zipper bucket/object lifecycle in librgw
Daniel Gryniewicz [Tue, 31 Aug 2021 13:17:55 +0000 (09:17 -0400)]
RGW - Zipper bucket/object lifecycle in librgw

s->bucket and s->object need to have a single lifecycle across an entire
op, never replaced, since lots of things use pointers to them.

Make sure s->bucket is created in read_permissions(), and that s->object
is created with the correct bucket.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #43154 from ceph/wip-yuriw-pacific-16.2.6-RN
David Galloway [Fri, 17 Sep 2021 13:38:54 +0000 (09:38 -0400)]
Merge pull request #43154 from ceph/wip-yuriw-pacific-16.2.6-RN

doc: 16.2.6 Release Notes

2 years agoinstall-deps.sh: install newer version of pip 43206/head
Matan Breizman [Fri, 17 Sep 2021 13:35:28 +0000 (13:35 +0000)]
install-deps.sh: install newer version of pip

although pip comes with virtualenv, having a recent version of pip matters to use wheel packages.

Signed-off-by: Matan Breizman <Matan.Brz@gmail.com>
2 years agoMerge pull request #43197 from anthonyeleven/anthonyeleven/stale-read
zdover23 [Fri, 17 Sep 2021 05:52:34 +0000 (15:52 +1000)]
Merge pull request #43197 from anthonyeleven/anthonyeleven/stale-read

doc: clarify stale_read and placement-groups wording

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2 years agoMerge PR #43122 into master
Patrick Donnelly [Fri, 17 Sep 2021 01:47:40 +0000 (21:47 -0400)]
Merge PR #43122 into master

* refs/pull/43122/head:
qa: add test for standby-replay marking rank damaged
MDSMonitor: handle damaged from standby-replay
mds: add config to mark rank damaged in standby-replay
include: unset std::hex after printing CompatSet
mds: refactor iterator lookup
mds: harden rank lookup

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agodoc: clarify stale_read and placement-groups wording 43197/head
Anthony D'Atri [Thu, 16 Sep 2021 23:40:33 +0000 (16:40 -0700)]
doc: clarify stale_read and placement-groups wording

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #43011 from rzarzynski/wip-crimson-opsequencer-ooo-lesser-footprint
Kefu Chai [Thu, 16 Sep 2021 23:23:14 +0000 (07:23 +0800)]
Merge pull request #43011 from rzarzynski/wip-crimson-opsequencer-ooo-lesser-footprint

crimson/osd: bring OpRegistry-based OpSequencer to handle out-of-order ClientRequests

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2 years agoMerge pull request #43192 from rzarzynski/wip-crimson-net-dangling-bindvec
Samuel Just [Thu, 16 Sep 2021 19:22:18 +0000 (12:22 -0700)]
Merge pull request #43192 from rzarzynski/wip-crimson-net-dangling-bindvec

crimson/net: fix dangling addrvec in SocketMessenger::bind().

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agodoc: updating RBD lead 43195/head
Mike Perez [Thu, 16 Sep 2021 18:45:35 +0000 (11:45 -0700)]
doc: updating RBD lead

Signed-off-by: Mike Perez thingee@gmail.com
2 years agodoc/releases/pacific.rst: Add note about Bullseye 43154/head
David Galloway [Thu, 16 Sep 2021 17:57:16 +0000 (13:57 -0400)]
doc/releases/pacific.rst: Add note about Bullseye

2 years agocrimson/net: fix dangling addrvec in SocketMessenger::bind(). 43192/head
Radoslaw Zarzynski [Thu, 16 Sep 2021 17:04:28 +0000 (17:04 +0000)]
crimson/net: fix dangling addrvec in SocketMessenger::bind().

`SocketMessenger::bind()` takes the address vector by `const&`.
while the callers in `OSD` are passing a temporary:

  ```cpp
  cluster_msgr->bind(pick_addresses(CEPH_PICK_ADDRESS_CLUSTER))
    // ...
  public_msgr->bind(pick_addresses(CEPH_PICK_ADDRESS_PUBLIC))
    // ...
  ```

  ```cpp
  entity_addrvec_t pick_addresses(int what) {
    // ...
    return addrs;
  }
  ```

The net result is a life-time mismatch and crashes like the following
one:

  ```
  WARN  2021-09-10 20:01:57,791 [shard 0] ms - [osd.0(client) v2:172.17.0.1:6800/3172806564@59988 >> mgr.? v2:172.17.0.7:6800/3846062660] waiting 3.2 seconds ...
  /opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/stl_iterator.h:820:17: runtime error: reference binding to misaligned address 0x000041b58ab3 for type 'const struct entity_addr_t', which requires 4 byte alignment
  0x000041b58ab3: note: pointer points here
  <memory cannot be printed>
  /opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/stl_vector.h:1132:16: runtime error: reference binding to misaligned address 0x000041b58ab3 for type 'const struct value_type', which requires 4 byte alignment
  0x000041b58ab3: note: pointer points here
  <memory cannot be printed>
  /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-7486-g27cb19ed/rpm/el8/BUILD/ceph-17.0.0-7486-g27cb19ed/src/msg/msg_types.h:561:22: runtime error: reference binding to misaligned address 0x000041b58ab3 for type 'const struct entity_addr_t', which requires 4 byte alignment
  0x000041b58ab3: note: pointer points here
  <memory cannot be printed>
  Segmentation fault on shard 0.
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agodoc/releases/pacific.rst: add core release notes origin/wip-yuriw-pacific-16.2.6-RN
Neha Ojha [Thu, 16 Sep 2021 16:49:38 +0000 (16:49 +0000)]
doc/releases/pacific.rst: add core release notes

Signed-off-by: Neha Ojha <nojha@redhat.com>
2 years agocrimson/osd: rework OpSequencer for out-of-order op execution. 43011/head
Radoslaw Zarzynski [Wed, 1 Sep 2021 12:57:54 +0000 (12:57 +0000)]
crimson/osd: rework OpSequencer for out-of-order op execution.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #43180 from athanatos/sjust/wip-52630
Samuel Just [Thu, 16 Sep 2021 03:59:33 +0000 (20:59 -0700)]
Merge pull request #43180 from athanatos/sjust/wip-52630

crimson/net/SocketMessenger: include sleep.hh

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #43178 from athanatos/sjust/wip-52623
Samuel Just [Thu, 16 Sep 2021 03:59:09 +0000 (20:59 -0700)]
Merge pull request #43178 from athanatos/sjust/wip-52623

crimson/os/seastore/cache: Cache::get_root check for invalid

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge PR #43044 into master
Sage Weil [Thu, 16 Sep 2021 02:50:56 +0000 (22:50 -0400)]
Merge PR #43044 into master

* refs/pull/43044/head:
mgr/rook: add unit tests to Rook module
mgr/rook: translate placement spec to node selector and vice versa

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2 years agoMerge PR #43172 into master
Sage Weil [Thu, 16 Sep 2021 02:48:36 +0000 (22:48 -0400)]
Merge PR #43172 into master

* refs/pull/43172/head:
qa/tasks/kubeadm: modify (do not clobber) daemon.json

Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
2 years agocrimson/net/SocketMessenger: include sleep.hh 43180/head
Samuel Just [Thu, 16 Sep 2021 02:31:52 +0000 (02:31 +0000)]
crimson/net/SocketMessenger: include sleep.hh

4a00a145 added a call to seastar::sleep.

Fixes: https://tracker.ceph.com/issues/52630
Signed-off-by: Samuel Just <sjust@redhat.com>
2 years agoMerge PR #43022 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:37:26 +0000 (22:37 -0400)]
Merge PR #43022 into master

* refs/pull/43022/head:
doc/cephfs: add mds_dir_max_entries option into mds-config-ref.rst
mds: add mds_dir_max_entries config option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge PR #42943 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:36:18 +0000 (22:36 -0400)]
Merge PR #42943 into master

* refs/pull/42943/head:
mon: do not quickly mark mds laggy when MON_DOWN
mon: refactor quorum age calculation

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2 years agoMerge PR #42846 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:35:37 +0000 (22:35 -0400)]
Merge PR #42846 into master

* refs/pull/42846/head:
mds: Add new flag to MClientSession

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge PR #42737 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:34:58 +0000 (22:34 -0400)]
Merge PR #42737 into master

* refs/pull/42737/head:
mds: ensure that we send the btime in cap messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2 years agoMerge PR #42719 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:34:23 +0000 (22:34 -0400)]
Merge PR #42719 into master

* refs/pull/42719/head:
mgr/volumes: Fix permission during subvol creation with mode

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agocrimson/os/seastore/cache: Cache::get_root check for invalid 43178/head
Samuel Just [Wed, 15 Sep 2021 21:23:29 +0000 (21:23 +0000)]
crimson/os/seastore/cache: Cache::get_root check for invalid

As with Cache::get_extent, we need to check that the ref
is still valid after wait_io since it's not in the
read set yet.

Fixes: https://tracker.ceph.com/issues/52623
Signed-off-by: Samuel Just <sjust@redhat.com>
2 years agoMerge PR #42584 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:33:18 +0000 (22:33 -0400)]
Merge PR #42584 into master

* refs/pull/42584/head:
doc: fix `daemon status` interface (exclude file system name)
test: adjust mirroring tests for `daemon status` change
mgr/mirroring: `daemon status` command does not require file system name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge PR #42541 into master
Patrick Donnelly [Thu, 16 Sep 2021 02:32:31 +0000 (22:32 -0400)]
Merge PR #42541 into master

* refs/pull/42541/head:
client: switch to use ceph_assert instead of assert
mds: switch to use ceph_assert() instead of assert()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoqa: lengthen grace for fs map showing dead MDS 43179/head
Patrick Donnelly [Thu, 16 Sep 2021 02:21:03 +0000 (22:21 -0400)]
qa: lengthen grace for fs map showing dead MDS

Fixes: https://tracker.ceph.com/issues/52625
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agocrimson/os/seastore/cache: rename invalidate(Transaction&, CachedExtent&)
Samuel Just [Thu, 16 Sep 2021 01:37:31 +0000 (18:37 -0700)]
crimson/os/seastore/cache: rename invalidate(Transaction&, CachedExtent&)

This particular overload is really only declaring the Transaction
conflicted -- it assumes the extent is already invalid.  Rename to
mark_transaction_conflicted instead to avoid confusion with
invalidate(CachedExtent&).

Signed-off-by: Samuel Just <sjust@redhat.com>
2 years agoqa/tasks/kubeadm: modify (do not clobber) daemon.json 43172/head
Sage Weil [Wed, 15 Sep 2021 15:53:32 +0000 (10:53 -0500)]
qa/tasks/kubeadm: modify (do not clobber) daemon.json

Otherwise we blow away the mirror config.

Signed-off-by: Sage Weil <sage@newdream.net>
2 years agodoc: updated the change log with three additional PRs merged on top
Yuri Weinstein [Wed, 15 Sep 2021 15:42:33 +0000 (15:42 +0000)]
doc: updated the change log with three additional PRs merged on top

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2 years agodoc: dashboard motd change
Ernesto Puerta [Tue, 14 Sep 2021 17:18:00 +0000 (19:18 +0200)]
doc: dashboard motd change

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2 years agodoc: note simpler MDS upgrade procedure for v16.2.6
Patrick Donnelly [Tue, 14 Sep 2021 15:12:10 +0000 (11:12 -0400)]
doc: note simpler MDS upgrade procedure for v16.2.6

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agodoc: 16.2.6 Release Notes
Yuri Weinstein [Mon, 13 Sep 2021 21:11:15 +0000 (21:11 +0000)]
doc: 16.2.6 Release Notes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2 years agoMerge pull request #43146 from jianwei1216/fix-partial-recovery-become-whole-object...
Kefu Chai [Wed, 15 Sep 2021 15:17:29 +0000 (23:17 +0800)]
Merge pull request #43146 from jianwei1216/fix-partial-recovery-become-whole-object-recovery-for-master

osd: fix partial recovery become whole object recovery after restart osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>