]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoMerge pull request #41222 from tchaikov/wip-crimson-cleanups
Kefu Chai [Fri, 7 May 2021 15:39:32 +0000 (23:39 +0800)]
Merge pull request #41222 from tchaikov/wip-crimson-cleanups

crimson/os: cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #41223 from rzarzynski/wip-crimson-alienstore-sighup
Kefu Chai [Fri, 7 May 2021 15:38:32 +0000 (23:38 +0800)]
Merge pull request #41223 from rzarzynski/wip-crimson-alienstore-sighup

crimson/alienstore: block SIGHUP to coexist with Seastar's signal handling

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41177 from dsavineau/cv_remove_legacy_release_check
Guillaume Abrioux [Fri, 7 May 2021 15:33:47 +0000 (17:33 +0200)]
Merge pull request #41177 from dsavineau/cv_remove_legacy_release_check

ceph-volume: remove legacy release check

4 years agoMerge pull request #41178 from dsavineau/cv_tox_py3
Guillaume Abrioux [Fri, 7 May 2021 15:31:55 +0000 (17:31 +0200)]
Merge pull request #41178 from dsavineau/cv_tox_py3

ceph-volume: remove duplicate py3 env

4 years agoMerge pull request #40016 from neha-ojha/wip-default-mclock
Neha Ojha [Fri, 7 May 2021 15:08:39 +0000 (08:08 -0700)]
Merge pull request #40016 from neha-ojha/wip-default-mclock

use mclock_scheduler as the default scheduler

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
4 years agocrimson/alienstore: block SIGHUP to coexist with Seastar's signal handling. 41223/head
Radoslaw Zarzynski [Thu, 6 May 2021 17:21:28 +0000 (17:21 +0000)]
crimson/alienstore: block SIGHUP to coexist with Seastar's signal handling.

In `crimson/osd/main.cc` we instruct Seastar to handle `SIGHUP`.

```
        // just ignore SIGHUP, we don't reread settings
        seastar::engine().handle_signal(SIGHUP, [] {})
```

This happens using the Seastar's signal handling infrastructure
which is incompliant with the alien world.

```
void
reactor::signals::handle_signal(int signo, noncopyable_function<void ()>&& handler) {
    // ...
    struct sigaction sa;
    sa.sa_sigaction = [](int sig, siginfo_t *info, void *p) {
        engine()._backend->signal_received(sig, info, p);
    };
    // ...
}
```

```
 extern __thread reactor* local_engine;
extern __thread size_t task_quota;

inline reactor& engine() {
    return *local_engine;
}
```

The low-level signal handler above assumes `local_engine._backend`
is not null which stays true only for threads from the S*'s world.
Unfortunately, as we don't block the `SIGHUP` for alien threads,
kernel is perfectly authorized to pick up one them to run the handler
leading to weirdly-looking segfaults like this one:

```
INFO  2021-04-23 07:06:57,807 [shard 0] bluestore - stat
DEBUG 2021-04-23 07:06:58,753 [shard 0] ms - [osd.1(client) v2:172.21.15.100:6802/30478@51064 >> mgr.4105 v2:172.21.15.109:6800/29891] --> #7 === pg_stats(0 pgs seq 55834574872 v 0) v2 (87)
...
INFO  2021-04-23 07:06:58,813 [shard 0] bluestore - stat
DEBUG 2021-04-23 07:06:59,753 [shard 0] osd - AdminSocket::handle_client: incoming asok string: {"prefix": "get_command_descriptions"}
INFO  2021-04-23 07:06:59,753 [shard 0] osd - asok response length: 2947
INFO  2021-04-23 07:06:59,817 [shard 0] bluestore - stat
DEBUG 2021-04-23 07:06:59,865 [shard 0] osd - AdminSocket::handle_client: incoming asok string: {"prefix": "get_command_descriptions"}
INFO  2021-04-23 07:06:59,866 [shard 0] osd - asok response length: 2947
DEBUG 2021-04-23 07:07:00,020 [shard 0] osd - AdminSocket::handle_client: incoming asok string: {"prefix": "get_command_descriptions"}
INFO  2021-04-23 07:07:00,020 [shard 0] osd - asok response length: 2947
INFO  2021-04-23 07:07:00,820 [shard 0] bluestore - stat
...
Backtrace:
 0# 0x00005600CD0D6AAF in ceph-osd
 1# FatalSignal::signaled(int) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F5877C7EB20 in /lib64/libpthread.so.0
 4# 0x00005600CD830B81 in ceph-osd
 5# 0x00007F5877C7EB20 in /lib64/libpthread.so.0
 6# pthread_cond_timedwait in /lib64/libpthread.so.0
 7# crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) in ceph-osd
 8# 0x00007F5877999BA3 in /lib64/libstdc++.so.6
 9# 0x00007F5877C7414A in /lib64/libpthread.so.0
10# clone in /lib64/libc.so.6
daemon-helper: command crashed with signal 11
```

Ultimately, it turned out the thread came out from a syscall (`futex`)
and started crunching the `SIGHUP` handler's code in which a nullptr
dereference happened.

This patch blocks `SIGHUP` for all threads spawned by `AlienStore`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/os: use this explicitly 41222/head
Kefu Chai [Fri, 7 May 2021 13:39:36 +0000 (21:39 +0800)]
crimson/os: use this explicitly

to silence the warning from clang. it fails to figure out that this is
actually used, and complains that this is captured but not used.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: use map::merge() to merge maps
Kefu Chai [Fri, 7 May 2021 13:36:48 +0000 (21:36 +0800)]
crimson/os/seastore: use map::merge() to merge maps

C++17's std::map allows us to merge two maps, and in this case, we can
even consume `child_result`. so map::merge() is used instead of insert()
in hope to avoid the memcpy and allocation of pair<> nodes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson.os/seastore: do not capture unused variables
Kefu Chai [Fri, 7 May 2021 13:36:10 +0000 (21:36 +0800)]
crimson.os/seastore: do not capture unused variables

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: do not start identifier with "__"
Kefu Chai [Fri, 7 May 2021 13:14:30 +0000 (21:14 +0800)]
crimson/os/seastore: do not start identifier with "__"

avoid starting identifiers with two underscores, these names are
reserved for C/C++ compiler and standard library.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41217 from petrutlucian94/boost_url
Kefu Chai [Fri, 7 May 2021 12:22:42 +0000 (20:22 +0800)]
Merge pull request #41217 from petrutlucian94/boost_url

win*.sh,cmake: Fix Windows build

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41129 from athanatos/sjust/wip-seastore-osd
Kefu Chai [Fri, 7 May 2021 11:51:33 +0000 (19:51 +0800)]
Merge pull request #41129 from athanatos/sjust/wip-seastore-osd

crimson: add initial osd support for seastore

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41215 from CloudFerro/new_boost_url
Kefu Chai [Fri, 7 May 2021 10:48:05 +0000 (18:48 +0800)]
Merge pull request #41215 from CloudFerro/new_boost_url

cmake: Replace boost download url

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agowin*.sh,cmake: Fix Windows linking errors 41217/head
Lucian Petrut [Fri, 7 May 2021 09:23:30 +0000 (09:23 +0000)]
win*.sh,cmake: Fix Windows linking errors

The Windows build is hitting linking errors after
bumping the Boost version to 1.75. The issue is that Boost
is now setting the zlib dependecy using INTERFACE_LINK_LIBRARIES,
which means that it's no longer located using the standard
"find_package" mechanism.

In order for the linker to locate zlib, we'll add it to the
linker search path.

[1] https://github.com/boostorg/boost_install/issues/47

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agocmake: Replace boost download url 41215/head
Rafał Wądołowski [Fri, 7 May 2021 08:12:43 +0000 (10:12 +0200)]
cmake: Replace boost download url

Boost has moved downloads to JFrog Artifactory
https://www.boost.org/users/news/boost_has_moved_downloads_to_jfr.html

Signed-off-by: Rafał Wądołowski <rwadolowski@cloudferro.com>
4 years agocrimson/tools/store-nbd: fix help message for path 41129/head
Samuel Just [Thu, 6 May 2021 07:13:54 +0000 (00:13 -0700)]
crimson/tools/store-nbd: fix help message for path

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agoseastore: add comment to do_transaction outlining ordering TODO
Samuel Just [Wed, 5 May 2021 05:22:18 +0000 (22:22 -0700)]
seastore: add comment to do_transaction outlining ordering TODO

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/segment_manager/block: create block device during mkfs
Samuel Just [Mon, 3 May 2021 21:26:09 +0000 (14:26 -0700)]
crimson/os/seastore/segment_manager/block: create block device during mkfs

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: refactor segment_cleaner to init segment_manager params after...
Samuel Just [Tue, 27 Apr 2021 22:00:11 +0000 (15:00 -0700)]
crimson/os/seastore: refactor segment_cleaner to init segment_manager params after mount

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agovstart.sh: add --seastore
Samuel Just [Mon, 26 Apr 2021 20:52:46 +0000 (13:52 -0700)]
vstart.sh: add --seastore

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: add seastore to FuturizedStore::create
Samuel Just [Thu, 22 Apr 2021 23:57:19 +0000 (16:57 -0700)]
crimson/os/seastore: add seastore to FuturizedStore::create

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/segment_manager/block: DSYNC not needed
Samuel Just [Mon, 26 Apr 2021 20:49:54 +0000 (13:49 -0700)]
crimson/os/seastore/segment_manager/block: DSYNC not needed

We are expressly flushing, so this shouldn't be needed.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: refactor SegmentManager reference ownership
Samuel Just [Thu, 22 Apr 2021 23:56:08 +0000 (16:56 -0700)]
crimson/os/seastore: refactor SegmentManager reference ownership

Gives SeaStore ownership over SegmentManager and rearranges mkfs/mount.
Replaces mkfs_config_t/mount_config_t with config params.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agotest/crimson/seastore/transaction_manager_test_state: remove duplicate setup
Samuel Just [Thu, 22 Apr 2021 23:53:56 +0000 (16:53 -0700)]
test/crimson/seastore/transaction_manager_test_state: remove duplicate setup

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agotest/crimson/gtest_seastar: init config and perf counters for crimson tests
Samuel Just [Thu, 22 Apr 2021 22:11:28 +0000 (15:11 -0700)]
test/crimson/gtest_seastar: init config and perf counters for crimson tests

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/tools/store-nbd: don't use detailed space tracker
Samuel Just [Tue, 27 Apr 2021 21:56:00 +0000 (14:56 -0700)]
crimson/tools/store-nbd: don't use detailed space tracker

Intended for debugging rather than performance testing.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/journal: close open segment and reset soft state in close()
Samuel Just [Wed, 28 Apr 2021 07:22:16 +0000 (00:22 -0700)]
crimson/os/seastore/journal: close open segment and reset soft state in close()

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/journal: use SegmentManager methods for block size, etc
Samuel Just [Wed, 28 Apr 2021 07:21:19 +0000 (00:21 -0700)]
crimson/os/journal: use SegmentManager methods for block size, etc

Otherwise, initing those values needs to be done after the SegmentManager
instance is mounted.  This is simpler for now.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: only create handle in create_new_collection
Samuel Just [Wed, 28 Apr 2021 07:25:02 +0000 (00:25 -0700)]
crimson/os/seastore: only create handle in create_new_collection

FuturizedStore::create_new_collection isn't supposed to actually
create the collection.  See OSD::mkfs for a usage example.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: clean up meta implementation
Samuel Just [Wed, 28 Apr 2021 07:24:00 +0000 (00:24 -0700)]
crimson/os/seastore: clean up meta implementation

There's really no reason to cache the decoded representation here since
the meta keys are only accessed during startup, mkfs.  This approach is
much simpler.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/.../transaction_manager: skip zero mappings in mount
Samuel Just [Wed, 28 Apr 2021 08:43:02 +0000 (01:43 -0700)]
crimson/.../transaction_manager: skip zero mappings in mount

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: fix read() to use onode.size for len=0
Samuel Just [Wed, 28 Apr 2021 08:42:35 +0000 (01:42 -0700)]
crimson/os/seastore: fix read() to use onode.size for len=0

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: fix do_transaction -- transactions may be empty
Samuel Just [Fri, 30 Apr 2021 21:12:27 +0000 (14:12 -0700)]
crimson/os/seastore: fix do_transaction -- transactions may be empty

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: wire up get_fsid
Samuel Just [Fri, 30 Apr 2021 21:11:43 +0000 (14:11 -0700)]
crimson/os/seastore: wire up get_fsid

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: wire up stat
Samuel Just [Fri, 30 Apr 2021 21:10:57 +0000 (14:10 -0700)]
crimson/os/seastore: wire up stat

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/seastore.cc: update to use new debug macros
Samuel Just [Fri, 30 Apr 2021 07:31:45 +0000 (07:31 +0000)]
crimson/os/seastore/seastore.cc: update to use new debug macros

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: convert cache and transaction_manager to use new debugging macros
Samuel Just [Thu, 29 Apr 2021 22:15:40 +0000 (15:15 -0700)]
crimson/os/seastore: convert cache and transaction_manager to use new debugging macros

The goal here is to capture the transaction address and to standardize the
prefix format.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agowin32*.sh: ensure that the build dir exists
Lucian Petrut [Fri, 7 May 2021 07:18:39 +0000 (07:18 +0000)]
win32*.sh: ensure that the build dir exists

The Windows build scripts try to use the build dir before
actually creating it.

We'll have to move up the "mkdir" command a few lines.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agoMerge pull request #41214 from tchaikov/wip-crimson-clang-cleanups
Kefu Chai [Fri, 7 May 2021 06:59:41 +0000 (14:59 +0800)]
Merge pull request #41214 from tchaikov/wip-crimson-clang-cleanups

crimson: clang related cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/common/config_proxy: add a helper for get_val<>() 41214/head
Kefu Chai [Fri, 7 May 2021 05:14:33 +0000 (13:14 +0800)]
crimson/common/config_proxy: add a helper for get_val<>()

otherwise we have to put something like

local_conf().template get_val<T>(name)

which is not quite convenient or readable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: do not redefine default argument
Kefu Chai [Fri, 7 May 2021 05:03:24 +0000 (13:03 +0800)]
crimson/os/seastore: do not redefine default argument

we should not redefine a default argument of a method of templated class.

this change also address following error from clang:

../src/crimson/os/seastore/onode_manager/staged-fltree/node.cc:621:30: error: template parameter redefines default argument
template <bool FORCE_MERGE = false>
                             ^
../src/crimson/os/seastore/onode_manager/staged-fltree/node.h:438:32: note: previous default template argument defined here
  template <bool FORCE_MERGE = false>
                               ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: do not capture non-variables
Kefu Chai [Fri, 7 May 2021 04:59:36 +0000 (12:59 +0800)]
crimson/os/seastore: do not capture non-variables

merge_stage and merge_size are structured bindings, they are not
variables. so cannot be captured without defining variables in
capture list.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: do not capture unused variable
Kefu Chai [Fri, 7 May 2021 04:59:14 +0000 (12:59 +0800)]
crimson/os/seastore: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41211 from neha-ojha/wip-remove-mon-election
Neha Ojha [Fri, 7 May 2021 02:10:28 +0000 (19:10 -0700)]
Merge pull request #41211 from neha-ojha/wip-remove-mon-election

qa/suites/rados/standalone: remove mon_election symlink

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoqa/suites/rados/standalone: remove mon_election symlink 41211/head
Neha Ojha [Fri, 7 May 2021 00:35:35 +0000 (00:35 +0000)]
qa/suites/rados/standalone: remove mon_election symlink

The standalone tests need parameters to be passed as ceph_args to
override defaults.

This was just doubling the number of standalone tests being run in each rados
run with no effect!

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge PR #41201 into master
Sage Weil [Thu, 6 May 2021 20:55:16 +0000 (16:55 -0400)]
Merge PR #41201 into master

* refs/pull/41201/head:
doc/releases: 16.2.3

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
4 years agoMerge PR #41179 into master
Sage Weil [Thu, 6 May 2021 20:01:57 +0000 (16:01 -0400)]
Merge PR #41179 into master

* refs/pull/41179/head:
qa/tasks/cephadm_cases: longer wait for osd to start

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agodoc/releases: 16.2.3 41201/head
Sage Weil [Thu, 6 May 2021 15:26:51 +0000 (10:26 -0500)]
doc/releases: 16.2.3

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/suites/rados/mgr/tasks/progress: use high_recovery_ops for faster recovery 40016/head
Sridhar Seshasayee [Thu, 6 May 2021 08:27:02 +0000 (13:57 +0530)]
qa/suites/rados/mgr/tasks/progress: use high_recovery_ops for faster recovery

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
4 years agoPendingReleaseNotes: mclock_scheduler is the default scheduler for quincy
Neha Ojha [Tue, 6 Apr 2021 19:56:06 +0000 (19:56 +0000)]
PendingReleaseNotes: mclock_scheduler is the default scheduler for quincy

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoqa/standalone: use osd op queue = wpq
Neha Ojha [Mon, 3 May 2021 19:28:27 +0000 (19:28 +0000)]
qa/standalone: use osd op queue = wpq

mclock_scheduler is now the default and some of these tests need to be modified
to run well with it. Continue using wpq until
https://tracker.ceph.com/issues/50574 is addressed.

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agocommon/options/global.yaml.in: use mclock_scheduler as the default scheduler
Neha Ojha [Mon, 3 May 2021 18:35:35 +0000 (18:35 +0000)]
common/options/global.yaml.in: use mclock_scheduler as the default scheduler

The aim is to default to mclock_scheduler in quincy, so start early and
get more testing.

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #41192 from petrutlucian94/boost_url
Kefu Chai [Thu, 6 May 2021 16:34:25 +0000 (00:34 +0800)]
Merge pull request #41192 from petrutlucian94/boost_url

win32*.sh: Update boost URL, fix windows build

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41198 from tchaikov/wip-doc-mgr-options
Kefu Chai [Thu, 6 May 2021 16:33:16 +0000 (00:33 +0800)]
Merge pull request #41198 from tchaikov/wip-doc-mgr-options

doc/_ext: load options defined by mgr modules

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #41163 into master
Sage Weil [Thu, 6 May 2021 15:24:38 +0000 (11:24 -0400)]
Merge PR #41163 into master

* refs/pull/41163/head:
doc/releases/releases.yml: add 16.2.2
doc/releases/pacific: note cepahdm ingress+rgw support
doc: 16.2.2 Release Notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41183 into master
Sage Weil [Thu, 6 May 2021 15:10:49 +0000 (11:10 -0400)]
Merge PR #41183 into master

* refs/pull/41183/head:
cephadm: normalize image digest in 'ls' output too

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge pull request #40311 from rhcs-dashboard/refactor-crushmap-component
Ernesto Puerta [Thu, 6 May 2021 14:36:26 +0000 (16:36 +0200)]
Merge pull request #40311 from rhcs-dashboard/refactor-crushmap-component

mgr/dashboard: avoid data processing in crush-map component

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge PR #41107 into master
Sage Weil [Thu, 6 May 2021 14:33:00 +0000 (10:33 -0400)]
Merge PR #41107 into master

* refs/pull/41107/head:
mgr/cephadm: apply hostname/addr checks to 'orch host set-addr' too
mgr/cephadm: make 'host add' idempotent
mgr/cephadm: set host crush location based on HostSpec
python-common: add location property to HostSpec, + tests

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agodoc/_ext: rewrite directive using ObjectDescription 41198/head
Kefu Chai [Thu, 6 May 2021 07:36:27 +0000 (15:36 +0800)]
doc/_ext: rewrite directive using ObjectDescription

which allows us to use different scheme when defining an option,

without this change, if two options in different mgr module share the
same name we cannot differentiate them, after this change, their id
would prefixed with the module name.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/general-config-ref: use describe to define non-options
Kefu Chai [Thu, 6 May 2021 14:14:29 +0000 (22:14 +0800)]
doc/rados/configuration/general-config-ref: use describe to define non-options

max_open_files is not generated from yaml files, so define it using
"describe" directive instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/_ext: load options defined by mgr modules
Kefu Chai [Thu, 6 May 2021 04:51:23 +0000 (12:51 +0800)]
doc/_ext: load options defined by mgr modules

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agowin32*.sh: bump boost version to 1.75 41192/head
Lucian Petrut [Thu, 6 May 2021 12:39:00 +0000 (12:39 +0000)]
win32*.sh: bump boost version to 1.75

We're updating the Windows build scripts to use Boost 1.75, the same
as the Linux build.

We can now drop the custom Boost patches that were needed in order
to use mingw. Those have been merged upstream in the meantime.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agocephadm: normalize image digest in 'ls' output too 41183/head
Sage Weil [Wed, 5 May 2021 18:42:36 +0000 (14:42 -0400)]
cephadm: normalize image digest in 'ls' output too

We were previously only normalizing the 'inspect' output.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #41186 from zdover23/wip-doc-cephadm-upgrade-starting-the-upgrade...
Sebastian Wagner [Thu, 6 May 2021 11:18:09 +0000 (13:18 +0200)]
Merge pull request #41186 from zdover23/wip-doc-cephadm-upgrade-starting-the-upgrade-2021-05-06

doc/cephadm: rewriting "starting the upgrade"

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #41187 from zdover23/wip-doc-cephadm-upgrade-monitoring-the-upgrad...
Sebastian Wagner [Thu, 6 May 2021 11:17:21 +0000 (13:17 +0200)]
Merge pull request #41187 from zdover23/wip-doc-cephadm-upgrade-monitoring-the-upgrade-2021-05-06

doc/cephadm: rewrite "monitoring the upgrade"

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #41188 from zdover23/wip-doc-cephadm-upgrade-cancelling-the-upgrad...
Sebastian Wagner [Thu, 6 May 2021 11:16:04 +0000 (13:16 +0200)]
Merge pull request #41188 from zdover23/wip-doc-cephadm-upgrade-cancelling-the-upgrade-2021-05-06

doc/cephadm: rewrite "cancelling the upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #41189 from zdover23/wip-doc-cephadm-upgrade-potential-problems...
Sebastian Wagner [Thu, 6 May 2021 11:15:27 +0000 (13:15 +0200)]
Merge pull request #41189 from zdover23/wip-doc-cephadm-upgrade-potential-problems-2021-05-06

doc/cephadm: rewrite "potential problems"

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #41190 from zdover23/wip-doc-cephadm-using-custom-containers-2021...
Sebastian Wagner [Thu, 6 May 2021 11:14:33 +0000 (13:14 +0200)]
Merge pull request #41190 from zdover23/wip-doc-cephadm-using-custom-containers-2021-05-06

doc/cephadm: rewrite "using customized con..."

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #41050 from rhcs-dashboard/fix-bucket-versioning-edit
Ernesto Puerta [Thu, 6 May 2021 09:32:51 +0000 (11:32 +0200)]
Merge pull request #41050 from rhcs-dashboard/fix-bucket-versioning-edit

mgr/dashboard: fix bucket versioning when locking is enabled

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
4 years agoMerge pull request #41126 from idryomov/wip-rbd-qemu-iotests-focal
Ilya Dryomov [Thu, 6 May 2021 09:14:46 +0000 (11:14 +0200)]
Merge pull request #41126 from idryomov/wip-rbd-qemu-iotests-focal

qa/workunits/rbd: use bionic version of qemu-iotests for focal

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
4 years agomgr/dashboard: avoid data processing in crush-map component 40311/head
Avan Thakkar [Mon, 22 Mar 2021 14:40:20 +0000 (20:10 +0530)]
mgr/dashboard: avoid data processing in crush-map component

Fixes: https://tracker.ceph.com/issues/49236
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
4 years agowin32*.sh: improve logging
Lucian Petrut [Thu, 6 May 2021 07:18:56 +0000 (07:18 +0000)]
win32*.sh: improve logging

We'll use a log file when building the Ceph dependencies. Also, we'll
log some details about the operation that's about to be performed.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agowin32*.sh: fix boost url
Lucian Petrut [Thu, 6 May 2021 07:10:36 +0000 (07:10 +0000)]
win32*.sh: fix boost url

The boost download location has changed, breaking the Windows build.
Ths commit updates the boost download url.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agodoc/mgr/modules: should use Option class for defining options
Kefu Chai [Thu, 6 May 2021 03:57:27 +0000 (11:57 +0800)]
doc/mgr/modules: should use Option class for defining options

it is the encouraged way to define options nowadays.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/cephadm: rewrite "using customized con..." 41190/head
Zac Dover [Thu, 6 May 2021 00:54:24 +0000 (10:54 +1000)]
doc/cephadm: rewrite "using customized con..."

This PR rewrites the text in "Using Customized
Container Images" so that it is just a bit
clearer, and it also formats the prompt in the
text correctly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc/cephadm: rewrite "potential problems" 41189/head
Zac Dover [Thu, 6 May 2021 00:44:23 +0000 (10:44 +1000)]
doc/cephadm: rewrite "potential problems"

This PR adds correct prompts and slightly
reduces the cognitive load of the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc/cephadm: rewrite "cancelling the upgrade" 41188/head
Zac Dover [Thu, 6 May 2021 00:34:26 +0000 (10:34 +1000)]
doc/cephadm: rewrite "cancelling the upgrade"

This PR corrects a single prompt in the section
"Cancelling the Upgrade".

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc/cephadm: rewrite "monitoring the upgrade" 41187/head
Zac Dover [Thu, 6 May 2021 00:30:02 +0000 (10:30 +1000)]
doc/cephadm: rewrite "monitoring the upgrade"

This PR adds correctly-formatted prompts and
makes slight alterations to the readability
of the text.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc/cephadm: rewriting "starting the upgrade" 41186/head
Zac Dover [Thu, 6 May 2021 00:03:21 +0000 (10:03 +1000)]
doc/cephadm: rewriting "starting the upgrade"

This PR fixes the prompts in this section and
makes the sentences a tiny bit less onerous for
the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc/releases/releases.yml: add 16.2.2 41163/head
Sage Weil [Wed, 5 May 2021 20:14:13 +0000 (15:14 -0500)]
doc/releases/releases.yml: add 16.2.2

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #40937 from MrFreezeex/snap-remove-fix
Ilya Dryomov [Wed, 5 May 2021 20:42:42 +0000 (22:42 +0200)]
Merge pull request #40937 from MrFreezeex/snap-remove-fix

librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoMerge PR #41148 into master
Sage Weil [Wed, 5 May 2021 18:53:22 +0000 (14:53 -0400)]
Merge PR #41148 into master

* refs/pull/41148/head:
Add Rocky Linux to supported DISTRO_NAMES

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge PR #41147 into master
Sage Weil [Wed, 5 May 2021 18:08:11 +0000 (14:08 -0400)]
Merge PR #41147 into master

* refs/pull/41147/head:
ceph_test_rados_api_service: stop threads before asserting
ceph_test_rados_api_service: more retries for servicemkap

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #40551 from alimaredia/wip-rgw-dpp-update-master
J. Eric Ivancich [Wed, 5 May 2021 16:28:53 +0000 (12:28 -0400)]
Merge pull request #40551 from alimaredia/wip-rgw-dpp-update-master

rgw,test: Add even more DoutPrefixProvider's to the RGW codebase

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #40063 from rhcs-dashboard/cyclomatic-complexity
Ernesto Puerta [Wed, 5 May 2021 16:11:03 +0000 (18:11 +0200)]
Merge pull request #40063 from rhcs-dashboard/cyclomatic-complexity

mgr/dashboard: reduce cyclomatic complexity

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #40715 from pcuzner/pool-overview-enhancement
Ernesto Puerta [Wed, 5 May 2021 16:08:58 +0000 (18:08 +0200)]
Merge pull request #40715 from pcuzner/pool-overview-enhancement

mgr/dashboard:include compression stats on pool dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoqa/tasks/cephadm_cases: longer wait for osd to start 41179/head
Sage Weil [Wed, 5 May 2021 16:03:02 +0000 (11:03 -0500)]
qa/tasks/cephadm_cases: longer wait for osd to start

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #41149 into master
Sage Weil [Wed, 5 May 2021 15:57:28 +0000 (11:57 -0400)]
Merge PR #41149 into master

* refs/pull/41149/head:
mgr/cephadm: remove haproxy+keepalived container image from service spec

Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #41081 from votdev/issue_49718_disable_nfs_v3
Ernesto Puerta [Wed, 5 May 2021 15:47:58 +0000 (17:47 +0200)]
Merge pull request #41081 from votdev/issue_49718_disable_nfs_v3

mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoceph-volume: remove legacy release check 41177/head
Dimitri Savineau [Wed, 5 May 2021 14:52:41 +0000 (10:52 -0400)]
ceph-volume: remove legacy release check

Those conditions don't make sense anymore so we can remove them.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
4 years agoceph-volume: remove duplicate py3 env 41178/head
Dimitri Savineau [Wed, 5 May 2021 15:08:08 +0000 (11:08 -0400)]
ceph-volume: remove duplicate py3 env

We don't need to have py36 and py3 env in the tox configuration.
Currently we're running pytest twice on the same python 3 release.
Using py3 only is more generic and reflects was is defined in the
other tox.ini file for other components.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
4 years agoMerge pull request #41133 from nh2/patch-1
Kefu Chai [Wed, 5 May 2021 14:54:52 +0000 (22:54 +0800)]
Merge pull request #41133 from nh2/patch-1

cephfs.pyx: Fix docstring of get_layout

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/releases/pacific: note cepahdm ingress+rgw support
Sage Weil [Wed, 5 May 2021 14:28:39 +0000 (09:28 -0500)]
doc/releases/pacific: note cepahdm ingress+rgw support

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/dashboard: fix bucket versioning when locking is enabled 41050/head
Avan Thakkar [Tue, 27 Apr 2021 19:21:28 +0000 (00:51 +0530)]
mgr/dashboard: fix bucket versioning when locking is enabled

Fixes: https://tracker.ceph.com/issues/50545
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
4 years agopybind/cephfs: Fix docstring of get_layout 41133/head
Niklas Hambüchen [Tue, 4 May 2021 02:58:06 +0000 (04:58 +0200)]
pybind/cephfs: Fix docstring of get_layout

It was copy-pasted from another function.
Fixed by copy-pasting the docstring from the C docs.

Signed-off-by: Niklas Hambuechen <mail@nh2.me>
4 years agoMerge pull request #41165 from tchaikov/wip-cephfs-shell-manpage
Kefu Chai [Wed, 5 May 2021 09:43:26 +0000 (17:43 +0800)]
Merge pull request #41165 from tchaikov/wip-cephfs-shell-manpage

doc/man: build and install cephfs-shell manpage

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40977 from xxhdx1985126/wip-seastore-xattr
Kefu Chai [Wed, 5 May 2021 07:23:00 +0000 (15:23 +0800)]
Merge pull request #40977 from xxhdx1985126/wip-seastore-xattr

crimson/os/seastore: formal extended attributes support

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41109 from markhpc/wip-bluefs_buffered_io-desc
Kefu Chai [Wed, 5 May 2021 06:45:40 +0000 (14:45 +0800)]
Merge pull request #41109 from markhpc/wip-bluefs_buffered_io-desc

src/common: Update bluefs_buffered_io descrption.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agodoc/man: build and install cephfs-shell manpage 41165/head
Kefu Chai [Tue, 4 May 2021 02:59:03 +0000 (10:59 +0800)]
doc/man: build and install cephfs-shell manpage

otherwise the manpage of cephfs-shell is not built and installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41142 from tchaikov/wip-cmake-global
Kefu Chai [Wed, 5 May 2021 04:52:53 +0000 (12:52 +0800)]
Merge pull request #41142 from tchaikov/wip-cmake-global

cmake: let libglobal_obj depend on legacy-option-headers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>