]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agodoc/dev: update unit test declaration 38150/head
Zac Dover [Wed, 18 Nov 2020 08:56:14 +0000 (18:56 +1000)]
doc/dev: update unit test declaration

This commit updates the text in the "How
Unit Tests Are Declared" section of
tests-unit-tests.rst. This commit breaks
long sentences into shorter sentences and
breaks a sentence into a bulleted list that
might reduce the reader's cognitive load.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #38102 from tchaikov/wip-mgr-boost-cleanup
Kefu Chai [Wed, 18 Nov 2020 02:02:08 +0000 (10:02 +0800)]
Merge pull request #38102 from tchaikov/wip-mgr-boost-cleanup

mgr: do not include unused headers

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #38139 from batrick/doc-label
Kefu Chai [Wed, 18 Nov 2020 02:00:28 +0000 (10:00 +0800)]
Merge pull request #38139 from batrick/doc-label

.github: add labeler for documentation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38004 from tchaikov/wip-48058
Kefu Chai [Wed, 18 Nov 2020 01:58:47 +0000 (09:58 +0800)]
Merge pull request #38004 from tchaikov/wip-48058

qa/tasks/ceph: update_archive_setting() only if ctx.archive is valid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37488 from kamoltat/wip-mgr-progress-turn-off-option
Josh Durgin [Tue, 17 Nov 2020 21:37:30 +0000 (13:37 -0800)]
Merge pull request #37488 from kamoltat/wip-mgr-progress-turn-off-option

mgr/progress: progress module on/off option

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge remote-tracking branch 'wip-resurrect-authorizer-challenges'
Josh Durgin [Tue, 17 Nov 2020 20:24:07 +0000 (15:24 -0500)]
Merge remote-tracking branch 'wip-resurrect-authorizer-challenges'

Bring back CEPHX_V2 authorizer challenges

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years ago.github: add labeler for documentation 38139/head
Patrick Donnelly [Tue, 17 Nov 2020 16:24:26 +0000 (08:24 -0800)]
.github: add labeler for documentation

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #38084 from WarriorXK/disable-sso-without-python3-saml
Ernesto Puerta [Tue, 17 Nov 2020 16:05:52 +0000 (17:05 +0100)]
Merge pull request #38084 from WarriorXK/disable-sso-without-python3-saml

mgr/dashboard: Disable sso without python3-saml

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge PR #35749 into master
Patrick Donnelly [Tue, 17 Nov 2020 15:33:20 +0000 (07:33 -0800)]
Merge PR #35749 into master

* refs/pull/35749/head:
Update Release notes for multimds scrub
qa/cephfs: log-ignorelist scrub errors
qa/cephfs: Add more tests for multimds scrub
qa/cephfs: add tests for multimds scrub
qa/cephfs: update existing scrub test cases
mds: don't skip validating disk state of symlink
mds: abort/pause/resume scrubs in multiple mds
mds: track scrub status in multiple mds
mds: remove on_finish from {CInode,CDir}::scrub_info_t
Continuation: don't delete self while there are in-processing stages
mds: auth pin CInode when validating its disk state
mds: rdlock file/nest lock when accumulating stats of subtree dirfrags
mds: multiple mds scrub support
include/frag: add encode/decode functions for fragset_t
mds: remove object can't be scrubbed immediately from scrub stack
mds: prevent dirfrag scrub/fragment from running at the same time
mds: change scrub traverse from post-order to breadth-first search
mds: make both CInode and CDir as entities of scrub
mds: remove ScrubStack::scrubstack

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoUpdate Release notes for multimds scrub 35749/head
Yan, Zheng [Tue, 17 Nov 2020 05:10:51 +0000 (13:10 +0800)]
Update Release notes for multimds scrub

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoMerge pull request #38119 from neha-ojha/wip-core-labels 37676/head
Kefu Chai [Tue, 17 Nov 2020 02:52:43 +0000 (10:52 +0800)]
Merge pull request #38119 from neha-ojha/wip-core-labels

github/labeler.yml: add bluestore, cephadm labels and more core rules

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38096 from tchaikov/wip-cmake-boost-1.74
Kefu Chai [Tue, 17 Nov 2020 02:48:59 +0000 (10:48 +0800)]
Merge pull request #38096 from tchaikov/wip-cmake-boost-1.74

cmake: add 1.74 to known versions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #38097 from tchaikov/wip-clang-cleanup
Kefu Chai [Tue, 17 Nov 2020 02:47:45 +0000 (10:47 +0800)]
Merge pull request #38097 from tchaikov/wip-clang-cleanup

message,log:  mark final classes "final", fix clang related warnings

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #38099 from tchaikov/wip-mgr-python3.9
Kefu Chai [Tue, 17 Nov 2020 02:47:07 +0000 (10:47 +0800)]
Merge pull request #38099 from tchaikov/wip-mgr-python3.9

mgr/PyModuleRegistry: do not call PyEval_InitThreads() on python3.9

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #38052 from mgfritch/cephadm-validate-ssh-config
Michael Fritch [Tue, 17 Nov 2020 02:32:43 +0000 (19:32 -0700)]
Merge pull request #38052 from mgfritch/cephadm-validate-ssh-config

mgr/cephadm: validate user provided ssh_config

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge pull request #38053 from mgfritch/cephadm-placement-err
Michael Fritch [Tue, 17 Nov 2020 02:31:22 +0000 (19:31 -0700)]
Merge pull request #38053 from mgfritch/cephadm-placement-err

mgr/cephadm: show failure cause during placement

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge pull request #37901 from votdev/issue_48041_device_light
Michael Fritch [Tue, 17 Nov 2020 02:21:06 +0000 (19:21 -0700)]
Merge pull request #37901 from votdev/issue_48041_device_light

mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd per host

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years ago.github/labeler.yml: add cephadm 38119/head
Neha Ojha [Tue, 17 Nov 2020 01:23:19 +0000 (01:23 +0000)]
.github/labeler.yml: add cephadm

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years ago.github/labeler.yml: add bluestore and more core rules
Neha Ojha [Tue, 17 Nov 2020 01:19:13 +0000 (01:19 +0000)]
.github/labeler.yml: add bluestore and more core rules

more to come

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37939 from trociny/wip-rbd-nbd-wait-for-terminate
Jason Dillaman [Mon, 16 Nov 2020 22:10:17 +0000 (17:10 -0500)]
Merge pull request #37939 from trociny/wip-rbd-nbd-wait-for-terminate

rbd-nbd: fixes and improvements for unmap/detach wait for process terminate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #38081 from wjwithagen/wjw-fix-test_mock_CryptoObjectDispatch.cc
Jason Dillaman [Mon, 16 Nov 2020 22:09:25 +0000 (17:09 -0500)]
Merge pull request #38081 from wjwithagen/wjw-fix-test_mock_CryptoObjectDispatch.cc

test/librbd/crypto: Fixup include order to prevent implicit definitions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #38105 from ofriedma/wip-barbican-qa-issue
Casey Bodley [Mon, 16 Nov 2020 14:53:08 +0000 (09:53 -0500)]
Merge pull request #38105 from ofriedma/wip-barbican-qa-issue

qa/rgw: fix "cannot create secret" on barbican test

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agomgr/dashboard: Disable sso without python3-saml 38084/head
Kevin Meijer [Sat, 14 Nov 2020 18:44:07 +0000 (19:44 +0100)]
mgr/dashboard: Disable sso without python3-saml

Removed the requirement for the python3-saml package when wanting to disable SSO for the dashboard, this is currently relevant since the official container that runs Ceph mgr does not have this package installed.
So when upgrading from an older, non-containerized version, you would be stuck using a non-functional dashboard.

This pull requests changes that and allows the ceph dashboard sso disable command without the requirement of the library so that we SSO can always be disabled again.

Fixes: https://tracker.ceph.com/issues/48237 Signed-off-by: Kevin Meijer <admin@kevinmeijer.nl>
4 years agoMerge pull request #38072 from tchaikov/wip-github-labeler-crimson
Kefu Chai [Mon, 16 Nov 2020 13:22:03 +0000 (21:22 +0800)]
Merge pull request #38072 from tchaikov/wip-github-labeler-crimson

github/labeler: add "crimson" to labeler.yml

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #38041 from tchaikov/wip-mon-trim-osdmap
Kefu Chai [Mon, 16 Nov 2020 11:04:16 +0000 (19:04 +0800)]
Merge pull request #38041 from tchaikov/wip-mon-trim-osdmap

mon/OSDMonitor: remove osd_epochs if osd is marked in or out

Reviewed-by: Joao Eduardo Luis <joao@suse.com>
4 years agoMerge pull request #38103 from ybwang0211/link-to-balancer
Kefu Chai [Mon, 16 Nov 2020 10:30:53 +0000 (18:30 +0800)]
Merge pull request #38103 from ybwang0211/link-to-balancer

doc: Fix the broken link to the "balancer"

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoqa/rgw: fix "cannot create secret" on barbican test 38105/head
Or Friedmann [Sun, 15 Nov 2020 16:43:47 +0000 (18:43 +0200)]
qa/rgw: fix "cannot create secret" on barbican test

fix "cannot create secret" on barbican test

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Fixes: https://tracker.ceph.com/issues/47799
4 years agolog/Log: cast typed pointer to integer before printing it 38097/head
Kefu Chai [Mon, 16 Nov 2020 07:21:19 +0000 (15:21 +0800)]
log/Log: cast typed pointer to integer before printing it

libfmt does not print pointer if it's not "void*", but "thread_t" is
defined as a pointer pointing to "struct pthread" on FreeBSD, so we need
to cast it either to "void*" or an integer". let's cast it to an integer
so it's more consistent with the output on Linux where thread_t is
defined as an integer.

this change addresses the FTBFS on FreeBSD likee:

In file included from /home/jenkins/workspace/ceph-master-compile/src/log/Log.cc:27:
In file included from /usr/local/include/fmt/format.h:44:
/usr/local/include/fmt/core.h:1043:20: error: invalid application of 'sizeof' to an incomplete type 'pthread'
    static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
                   ^~~~~~~~~
/usr/local/include/fmt/core.h:1259:32: note: in instantiation of function template specialization
'fmt::v7::detail::arg_mapper<fmt::v7::basic_format_context<std::__1::back_insert_iterator<fmt::v7::detail::buffer<char> >, char> >::map<pthread>' requested here
  return arg_mapper<Context>().map(val);
                               ^
/usr/local/include/fmt/core.h:1408:23: note: in instantiation of function template specialization 'fmt::v7::detail::make_arg<true,
fmt::v7::basic_format_context<std::__1::back_insert_iterator<fmt::v7::detail::buffer<char> >, char>, fmt::v7::detail::type::int_type, pthread *, 0>' requested here
        data_{detail::make_arg<
                      ^
/usr/local/include/fmt/core.h:1764:10: note: in instantiation of member function 'fmt::v7::format_arg_store<fmt::v7::basic_format_context<std::__1::back_insert_iterator<fmt::v7::detail::buffer<char>
>, char>, pthread *const, char *>::format_arg_store' requested here
  return {args...};
         ^
/usr/local/include/fmt/core.h:1835:31: note: in instantiation of function template specialization 'fmt::v7::detail::make_args_checked<pthread *const &, char *, char [10], char>' requested here
  const auto& vargs = detail::make_args_checked<Args...>(format_str, args...);
                              ^
/home/jenkins/workspace/ceph-master-compile/src/log/Log.cc:376:23: note: in instantiation of function template specialization 'fmt::v7::format<char [10], pthread *const &, char *, char>' requested
here
    _log_message(fmt::format("  {} / {}", pthread_id, (char*)pthread_name), true);
                      ^
/usr/include/sys/_pthreadtypes.h:46:8: note: forward declaration of 'pthread'
struct pthread;
       ^
1 error generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc:Fix the web link that jumps to the "balancer" 38103/head
ybwang0211 [Mon, 16 Nov 2020 08:59:24 +0000 (16:59 +0800)]
doc:Fix the web link that jumps to the "balancer"

In "https://docs.ceph.com/en/latest/rados/operations/upmap/#upmap", the link to "balancer" was not set correctly, so we fixed it.

Signed-off-by: wangyingbin <wangyingbin@inspur.com>
4 years agomgr: do not include unused header files 38102/head
Kefu Chai [Mon, 16 Nov 2020 08:36:35 +0000 (16:36 +0800)]
mgr: do not include unused header files

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr: only include necessary headers
Kefu Chai [Mon, 16 Nov 2020 08:16:38 +0000 (16:16 +0800)]
mgr: only include necessary headers

for two reasons:

* faster compilation with less source files to process
* avoid including "boost/detail/iterator.hpp", so we can
  silence the warning like:

boost/detail/iterator.hpp:13:1: note: ‘#pragma message: This header is
deprecated. Use <iterator> instead.’
   13 | BOOST_HEADER_DEPRECATED("<iterator>")
      | ^~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/PyModuleRegistry: do not call PyEval_InitThreads() on python3.9 38099/head
Kefu Chai [Mon, 16 Nov 2020 07:33:42 +0000 (15:33 +0800)]
mgr/PyModuleRegistry: do not call PyEval_InitThreads() on python3.9

PyEval_InitThreads() and PyEval_ThreadsInitialized() are deprecated
since python3.9, and GIL is initialized by Py_Initialize() in python3.9.
so do not call them unless we are using python < 3.9.

see https://docs.python.org/3/c-api/init.html#c.PyEval_InitThreads

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agolog/Log: cast parameter of syslog() from unsigned to int
Kefu Chai [Mon, 16 Nov 2020 07:18:33 +0000 (15:18 +0800)]
log/Log: cast parameter of syslog() from unsigned to int

silences warning like:

../src/log/Log.cc:325:34: warning: field precision should have type 'int', but argument has type 'std::basic_string_view::size_type' (aka 'unsigned long') [-Wformat]
    syslog(LOG_USER|LOG_INFO, "%.*s", s.size(), s.data());
                               ~~^~   ~~~~~~~~
1 warning generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomessage: mark message classes "final"
Kefu Chai [Mon, 16 Nov 2020 06:58:22 +0000 (14:58 +0800)]
message: mark message classes "final"

silences warnings like:

src/messages/MOSDPGUpdateLogMissingReply.h:74:34: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
  ~MOSDPGUpdateLogMissingReply() final {}
                                 ^
../src/messages/MOSDPGUpdateLogMissingReply.h:21:7: note: mark 'MOSDPGUpdateLogMissingReply' as 'final' to silence this warning
class MOSDPGUpdateLogMissingReply : public MOSDFastDispatchOp {
      ^
1 warning generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: add 1.74 to known versions 38096/head
Kefu Chai [Mon, 16 Nov 2020 06:06:40 +0000 (14:06 +0800)]
cmake: add 1.74 to known versions

sync with
https://github.com/Kitware/CMake/blob/685fa8bec064e5776319c8a42c02578bd85b6a7d/Modules/FindBoost.cmake

for boost 1.74 support

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: enable CMP0074
Kefu Chai [Mon, 16 Nov 2020 06:04:15 +0000 (14:04 +0800)]
cmake: enable CMP0074

to search prefixes specified by the <PackageName>_ROOT CMake
variable and the <PackageName>_ROOT environment variable. find_package()
in CMake 3.12 started to use these prefixes if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38007 from sseshasa/wip-qa-rados-perf-scheduler
Sridhar Seshasayee [Mon, 16 Nov 2020 07:26:09 +0000 (12:56 +0530)]
Merge pull request #38007 from sseshasa/wip-qa-rados-perf-scheduler

qa/suite/rados: Introduce "scheduler" directory within rados perf suite.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agomgr/progress: introduce turn off/on feature 37488/head
Kamoltat [Wed, 30 Sep 2020 05:44:23 +0000 (05:44 +0000)]
mgr/progress: introduce turn off/on feature

progress module can be turned off/on by using
the commands: 'progress off' and 'progress on'

As well as refractoring teuthology test suite
to prevent future bugs that can possibly occur

fixes: https://tracker.ceph.com/issues/47238

Signed-off-by: kamoltat <ksirivad@redhat.com>
4 years agoqa/cephfs: log-ignorelist scrub errors
Yan, Zheng [Mon, 16 Nov 2020 01:09:13 +0000 (09:09 +0800)]
qa/cephfs: log-ignorelist scrub errors

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoqa/cephfs: Add more tests for multimds scrub
Sidharth Anupkrishnan [Tue, 25 Aug 2020 13:33:47 +0000 (19:03 +0530)]
qa/cephfs: Add more tests for multimds scrub

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
4 years agoqa/cephfs: add tests for multimds scrub
Yan, Zheng [Wed, 1 Jul 2020 02:20:38 +0000 (10:20 +0800)]
qa/cephfs: add tests for multimds scrub

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoqa/cephfs: update existing scrub test cases
Yan, Zheng [Mon, 29 Jun 2020 13:26:35 +0000 (21:26 +0800)]
qa/cephfs: update existing scrub test cases

Now scrub is always async.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: don't skip validating disk state of symlink
Yan, Zheng [Wed, 24 Jun 2020 10:46:41 +0000 (18:46 +0800)]
mds: don't skip validating disk state of symlink

we can check if backtrace is valid and check if inode number is in-use

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: abort/pause/resume scrubs in multiple mds
Yan, Zheng [Tue, 23 Jun 2020 02:25:14 +0000 (10:25 +0800)]
mds: abort/pause/resume scrubs in multiple mds

Limit scrub abort/pause/resume commands to mds.0. mds.0 sends messages
to other mds, asks them to abort/pause/resume scrubs.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: track scrub status in multiple mds
Yan, Zheng [Fri, 19 Jun 2020 02:52:52 +0000 (10:52 +0800)]
mds: track scrub status in multiple mds

Scrubs are always initialized from mds.0. So mds.0 can ensure that scrub
tags are unique globally. mds.0 periodically gathers scrubs running in
itself and in other mds. A scrub is finished only if it's not running in
any mds.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: remove on_finish from {CInode,CDir}::scrub_info_t
Yan, Zheng [Fri, 19 Jun 2020 02:37:01 +0000 (10:37 +0800)]
mds: remove on_finish from {CInode,CDir}::scrub_info_t

A CInode/CDir is scrubbed no longer means corresponding subtree is fully
scrubbed. The on_finish in {CInode,CDir}::scrub_info_t become useless.
This patch also removes code that flushs journal if scrub has repaired
anything.  Later patch will add the code back at different place.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoContinuation: don't delete self while there are in-processing stages
Yan, Zheng [Wed, 10 Jun 2020 16:15:28 +0000 (00:15 +0800)]
Continuation: don't delete self while there are in-processing stages

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: auth pin CInode when validating its disk state
Yan, Zheng [Wed, 3 Jun 2020 03:51:48 +0000 (11:51 +0800)]
mds: auth pin CInode when validating its disk state

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: rdlock file/nest lock when accumulating stats of subtree dirfrags
Simon Gao [Wed, 3 Jun 2020 03:04:25 +0000 (11:04 +0800)]
mds: rdlock file/nest lock when accumulating stats of subtree dirfrags

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: multiple mds scrub support
Simon Gao [Thu, 28 May 2020 08:22:12 +0000 (16:22 +0800)]
mds: multiple mds scrub support

If a non-auth object is encountered during scrubbing, forward scrub
to the object's auth mds.

Fixes: https://tracker.ceph.com/issues/12274
Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoinclude/frag: add encode/decode functions for fragset_t
Yan, Zheng [Sat, 23 May 2020 15:09:10 +0000 (23:09 +0800)]
include/frag: add encode/decode functions for fragset_t

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: remove object can't be scrubbed immediately from scrub stack
Yan, Zheng [Fri, 22 May 2020 08:46:38 +0000 (16:46 +0800)]
mds: remove object can't be scrubbed immediately from scrub stack

This avoid checking objects that can't be scrubbed repeatly.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: prevent dirfrag scrub/fragment from running at the same time
Yan, Zheng [Wed, 20 May 2020 03:38:51 +0000 (11:38 +0800)]
mds: prevent dirfrag scrub/fragment from running at the same time

When a CDir is in scrub stack, mds should not split/merge it.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: change scrub traverse from post-order to breadth-first search
Simon Gao [Fri, 15 May 2020 09:19:14 +0000 (17:19 +0800)]
mds: change scrub traverse from post-order to breadth-first search

After using breadth-first search, scrubing a dir inode does not need
to wait until all of its descendant difrags/inodes are scrubbed. This
simplfies scrub code a lot. The downside is that a scrubbed dir inode
no longer implies corresponding subtree has been fully scrubbed. It
makes later scrub (without force option) less efficient.

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: make both CInode and CDir as entities of scrub
Yan, Zheng [Sat, 2 May 2020 08:24:17 +0000 (16:24 +0800)]
mds: make both CInode and CDir as entities of scrub

Making CDir as entity of scrub is preparetion for scrubbing across
multiple mds. When subtree bound is encountered, scrub should be
forwarded to subtree's auth mds. The auth mds adds CDir to scrub stack.

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: remove ScrubStack::scrubstack
Yan, Zheng [Sun, 3 May 2020 15:17:51 +0000 (23:17 +0800)]
mds: remove ScrubStack::scrubstack

comment says it's hack for dout. I don't see any reason it's needed.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoMerge PR #34842 into master
Patrick Donnelly [Sun, 15 Nov 2020 20:39:11 +0000 (12:39 -0800)]
Merge PR #34842 into master

* refs/pull/34842/head:
qa/tasks/vstart_runner.py: disable the ns unsharing if using localhost

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #37746 into master
Patrick Donnelly [Sun, 15 Nov 2020 20:38:05 +0000 (12:38 -0800)]
Merge PR #37746 into master

* refs/pull/37746/head:
client: add a dedicated thread for the Client tick
client: try to renew caps and flush old cap releases

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #37982 into master
Patrick Donnelly [Sun, 15 Nov 2020 20:37:02 +0000 (12:37 -0800)]
Merge PR #37982 into master

* refs/pull/37982/head:
qa/cephfs: add code for when config is None in __init__

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge PR #38016 into master
Patrick Donnelly [Sun, 15 Nov 2020 20:35:58 +0000 (12:35 -0800)]
Merge PR #38016 into master

* refs/pull/38016/head:
qa: switch to 'osdop_read' instead of 'op_r'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #38056 into master
Patrick Donnelly [Sun, 15 Nov 2020 20:34:00 +0000 (12:34 -0800)]
Merge PR #38056 into master

* refs/pull/38056/head:
pybind/mgr/volumes: use = for equality comparision

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #38077 from tchaikov/wip-doc-scrub-max
Brad Hubbard [Sun, 15 Nov 2020 20:19:09 +0000 (06:19 +1000)]
Merge pull request #38077 from tchaikov/wip-doc-scrub-max

doc/rados/operation: add link to scrub_max_interval

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38038 from bk201/wip-48209
Kefu Chai [Sun, 15 Nov 2020 17:29:46 +0000 (01:29 +0800)]
Merge pull request #38038 from bk201/wip-48209

mgr/dashboard: fix cephadm e2e test failure on deleting OSDs

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #37908 from dzafman/wip-47930
Kefu Chai [Sun, 15 Nov 2020 17:00:56 +0000 (01:00 +0800)]
Merge pull request #37908 from dzafman/wip-47930

test: Fix race in TEST_recovery_scrub test

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #38043 from jhonxue/wip-fix-bluestore-bitmap-allocator
Kefu Chai [Sun, 15 Nov 2020 16:38:35 +0000 (00:38 +0800)]
Merge pull request #38043 from jhonxue/wip-fix-bluestore-bitmap-allocator

osd: fix bluestore bitmap allocator calculate wrong last_pos with hint

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #37963 from aclamk/wip-bluestore-no-debug-misc
Kefu Chai [Sun, 15 Nov 2020 16:37:51 +0000 (00:37 +0800)]
Merge pull request #37963 from aclamk/wip-bluestore-no-debug-misc

os/bluestore: Removed bluestore_debug_misc option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #37999 from trociny/wip-48153
Kefu Chai [Sun, 15 Nov 2020 16:34:12 +0000 (00:34 +0800)]
Merge pull request #37999 from trociny/wip-48153

os/bluestore: fix "end reached" check in collection_list_legacy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37630 from zjcmszh/aio_rmxattr
Kefu Chai [Sun, 15 Nov 2020 16:32:22 +0000 (00:32 +0800)]
Merge pull request #37630 from zjcmszh/aio_rmxattr

pybind/rados: add Ioctx::aio_rmxattr() and test aio_rmxattr()

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37976 from batrick/msg-destructor
Kefu Chai [Sun, 15 Nov 2020 16:31:21 +0000 (00:31 +0800)]
Merge pull request #37976 from batrick/msg-destructor

messages: use final specifier for destructor

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37992 from rishabh-d-dave/dashboard-qa-skipTest
Kefu Chai [Sun, 15 Nov 2020 16:30:42 +0000 (00:30 +0800)]
Merge pull request #37992 from rishabh-d-dave/dashboard-qa-skipTest

qa/mgr: mgr_test_case: raise SkipTest instead of calling skipTest()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
4 years agoMerge pull request #37998 from tchaikov/wip-log-mon-cleanup
Kefu Chai [Sun, 15 Nov 2020 16:18:17 +0000 (00:18 +0800)]
Merge pull request #37998 from tchaikov/wip-log-mon-cleanup

log,mon: use libfmt to format string and use range-based loop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge pull request #38026 from wjwithagen/wjw-fix-test-mon-build_initial
Kefu Chai [Sun, 15 Nov 2020 16:09:15 +0000 (00:09 +0800)]
Merge pull request #38026 from wjwithagen/wjw-fix-test-mon-build_initial

test/mon: FreeBSD returns ENOENT on resolv error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agotest/mon: Accept ENOENT as returned value on FreeBSD 38026/head
Willem Jan Withagen [Wed, 11 Nov 2020 14:09:12 +0000 (15:09 +0100)]
test/mon: Accept ENOENT as returned value on FreeBSD

FreeBSD returns ENOENT on resolv error

Tracker: https://tracker.ceph.com/issues/48183
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agodoc/rados/operation: add link to scrub_max_interval 38077/head
Kefu Chai [Sat, 14 Nov 2020 06:59:04 +0000 (14:59 +0800)]
doc/rados/operation: add link to scrub_max_interval

add the reference to the per-pool setting when explaing
PG_NOT_DEEP_SCRUBBED.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38082 from adamemerson/wip-anti-nihilism
J. Eric Ivancich [Sun, 15 Nov 2020 03:35:23 +0000 (22:35 -0500)]
Merge pull request #38082 from adamemerson/wip-anti-nihilism

Asyncening 0: The Yielding Anti-Nihilist

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge PR #38071 into master
Patrick Donnelly [Sat, 14 Nov 2020 18:04:12 +0000 (10:04 -0800)]
Merge PR #38071 into master

* refs/pull/38071/head:
.github: add labeler for cephfs

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agotest/librbd/crypto: Fixup include order to prevent implicit definitions 38081/head
Willem Jan Withagen [Sat, 14 Nov 2020 17:11:05 +0000 (18:11 +0100)]
test/librbd/crypto: Fixup include order to prevent implicit definitions

Clang complains a lot, part of it:
```
/home/jenkins/workspace/ceph-master-compile/src/test/librbd/crypto/test_mock_CryptoObjectDispatch.cc:35:8: error: explicit specialization of 'librbd::io::CopyupRequest<librbd::MockImageCtx>' after instantiation
struct CopyupRequest<librbd::MockImageCtx> {
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins/workspace/ceph-master-compile/src/librbd/io/ObjectRequest.cc:560:10: note: implicit instantiation first required here
  if (it == image_ctx->copyup_list.end()) {
         ^
/home/jenkins/workspace/ceph-master-compile/src/test/librbd/crypto/test_mock_CryptoObjectDispatch.cc:54:32: error: template specialization requires 'template<>'
        librbd::MockImageCtx>::s_instance = nullptr;
```

fixes: https://github.com/ceph/ceph/pull/37916
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #38067 from dillaman/wip-github-rbd-label
Kefu Chai [Sat, 14 Nov 2020 16:53:38 +0000 (00:53 +0800)]
Merge pull request #38067 from dillaman/wip-github-rbd-label

github: add 'rbd' auto-labelling

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38013 from drakonstein/master
Kefu Chai [Sat, 14 Nov 2020 11:56:40 +0000 (19:56 +0800)]
Merge pull request #38013 from drakonstein/master

rpm,deb: change sudoers file mode to 440

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
4 years agoqa/tasks/ceph: use Cluster.sh() and Remote.sh() when appropriate 38004/head
Kefu Chai [Tue, 10 Nov 2020 07:03:21 +0000 (15:03 +0800)]
qa/tasks/ceph: use Cluster.sh() and Remote.sh() when appropriate

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoqa/tasks/ceph: update_archive_setting() only if ctx.archive is valid
Kefu Chai [Tue, 10 Nov 2020 06:40:25 +0000 (14:40 +0800)]
qa/tasks/ceph: update_archive_setting() only if ctx.archive is valid

When running teuthology interactively, ctx.archive might not be set.
If it's not set, there is no point trying to access files there.

Fixes: https://tracker.ceph.com/issues/48058
Signed-off-by: Marcus Watts <mwatts@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #38073 from badone/wip-use-right-tunable-for-pg-scrub-doc 38145/head
Brad Hubbard [Sat, 14 Nov 2020 04:49:27 +0000 (14:49 +1000)]
Merge pull request #38073 from badone/wip-use-right-tunable-for-pg-scrub-doc

doc: Use correct tunable for PG_NOT_SCRUBBED

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agogithub/labeler: add more rules to labeler.yml 38072/head
Kefu Chai [Sat, 14 Nov 2020 02:43:20 +0000 (10:43 +0800)]
github/labeler: add more rules to labeler.yml

mainly for the maintainers from the RADOS team.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37883 from diwilli/patch-1
Michael Fritch [Sat, 14 Nov 2020 02:27:59 +0000 (19:27 -0700)]
Merge pull request #37883 from diwilli/patch-1

cephadm: Set listen-addresses on alertmanager container

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
4 years agoMerge pull request #38011 from mgfritch/cephadm-sudo-log-dir
Michael Fritch [Sat, 14 Nov 2020 02:23:34 +0000 (19:23 -0700)]
Merge pull request #38011 from mgfritch/cephadm-sudo-log-dir

cephadm: verify root privileges before log dir create

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agogithub/labeler: add "crimson" to labeler.yml
Kefu Chai [Sat, 14 Nov 2020 02:14:56 +0000 (10:14 +0800)]
github/labeler: add "crimson" to labeler.yml

so crimson related changes can be labeled by the labler action.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years ago.github: add labeler for cephfs 38071/head
Patrick Donnelly [Fri, 13 Nov 2020 23:07:12 +0000 (15:07 -0800)]
.github: add labeler for cephfs

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agogithub: add 'rbd' auto-labelling 38067/head
Jason Dillaman [Fri, 13 Nov 2020 19:43:21 +0000 (14:43 -0500)]
github: add 'rbd' auto-labelling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agorgw: No null_yield in rgw_rest 38082/head
Adam C. Emerson [Wed, 11 Nov 2020 22:12:08 +0000 (17:12 -0500)]
rgw: No null_yield in rgw_rest

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw: No null_yield in rgw_role
Adam C. Emerson [Wed, 11 Nov 2020 20:55:41 +0000 (15:55 -0500)]
rgw: No null_yield in rgw_role

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw: no null_yield in rgw_tools
Adam C. Emerson [Wed, 11 Nov 2020 19:06:07 +0000 (14:06 -0500)]
rgw: no null_yield in rgw_tools

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw: No null_yield in rgw_torrent
Adam C. Emerson [Wed, 11 Nov 2020 17:13:50 +0000 (12:13 -0500)]
rgw: No null_yield in rgw_torrent

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw: No null_yield in rgw_user
Adam C. Emerson [Wed, 11 Nov 2020 16:25:38 +0000 (11:25 -0500)]
rgw: No null_yield in rgw_user

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw: No null_yield in rgw_zone
Adam C. Emerson [Wed, 11 Nov 2020 14:36:23 +0000 (09:36 -0500)]
rgw: No null_yield in rgw_zone

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw: No null_yield in RGWOp
Adam C. Emerson [Wed, 11 Nov 2020 06:03:12 +0000 (01:03 -0500)]
rgw: No null_yield in RGWOp

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agorgw/services: No null_yield
Adam C. Emerson [Wed, 11 Nov 2020 00:33:35 +0000 (19:33 -0500)]
rgw/services: No null_yield

Null yield is fine at top levels where we know what kind of thread
we're running into, but do not belong in general use functions.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
4 years agoMerge pull request #38057 from zdover23/wip-doc-dev-guide-essentials-source-clean...
Neha Ojha [Fri, 13 Nov 2020 17:42:20 +0000 (09:42 -0800)]
Merge pull request #38057 from zdover23/wip-doc-dev-guide-essentials-source-clean-update

doc/dev: update essentials.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #38036 from zdover23/wip-doc-dev-guide-basic-workflow-make-check...
Neha Ojha [Fri, 13 Nov 2020 17:38:28 +0000 (09:38 -0800)]
Merge pull request #38036 from zdover23/wip-doc-dev-guide-basic-workflow-make-check-nov-2020

doc/dev: improve make check text

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37977 from dillaman/wip-48145
Mykola Golub [Fri, 13 Nov 2020 14:47:46 +0000 (16:47 +0200)]
Merge pull request #37977 from dillaman/wip-48145

librbd: update hidden global config when removing pool config override

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37975 from dang/wip-dang-webdir-empty-object
Daniel Gryniewicz [Fri, 13 Nov 2020 14:02:04 +0000 (09:02 -0500)]
Merge pull request #37975 from dang/wip-dang-webdir-empty-object

RGW - Don't create empty obj in webdir