]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agomgr/cephadm: Introduce tox and autopep8 36348/head
Volker Theile [Wed, 29 Jul 2020 10:13:01 +0000 (12:13 +0200)]
mgr/cephadm: Introduce tox and autopep8

This PR introduces tox and autopep8 to format the Python code according to PEP8. It does NOT include any lint related things.

Signed-off-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #36002 from david-z/wip-fix-slow-req
Kefu Chai [Fri, 31 Jul 2020 12:34:33 +0000 (20:34 +0800)]
Merge pull request #36002 from david-z/wip-fix-slow-req

mgr: don't update osd stat which is already out

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36101 from ifed01/wip-ifed-fix-store-test-with-db
Kefu Chai [Fri, 31 Jul 2020 12:33:24 +0000 (20:33 +0800)]
Merge pull request #36101 from ifed01/wip-ifed-fix-store-test-with-db

test/objectstore_store_test: fix test cases broken due to additional …

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
4 years agoMerge pull request #36337 from tchaikov/wip-qa-ceph-waiting
Kefu Chai [Fri, 31 Jul 2020 12:32:13 +0000 (20:32 +0800)]
Merge pull request #36337 from tchaikov/wip-qa-ceph-waiting

qa/tasks/ceph: do not print out empty list of pg

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #36360 from matthewoliver/cephadm_add_ipv6_unwrap
Sebastian Wagner [Fri, 31 Jul 2020 11:31:41 +0000 (13:31 +0200)]
Merge pull request #36360 from matthewoliver/cephadm_add_ipv6_unwrap

cephadm: add unwrap_ipv6 helper method

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36151 from jschmid1/osd_support_reweight_impl
Sebastian Wagner [Fri, 31 Jul 2020 11:28:18 +0000 (13:28 +0200)]
Merge pull request #36151 from jschmid1/osd_support_reweight_impl

mgr/cephadm: rework osd removals/replacements

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36338 from pcuzner/fix-daemon-actions
Sebastian Wagner [Fri, 31 Jul 2020 11:05:18 +0000 (13:05 +0200)]
Merge pull request #36338 from pcuzner/fix-daemon-actions

mgr/cephadm: fix call to cephadm for daemon restarts etc

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #36284 from sebastian-philipp/cephadm_service_id_migration
Sebastian Wagner [Fri, 31 Jul 2020 10:54:51 +0000 (12:54 +0200)]
Merge pull request #36284 from sebastian-philipp/cephadm_service_id_migration

mgr/cephadm: Add migration to keep the service names consistent

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36374 from varshar16/wip-doc-orchestrator-update-create-osd
Sebastian Wagner [Fri, 31 Jul 2020 10:41:33 +0000 (12:41 +0200)]
Merge pull request #36374 from varshar16/wip-doc-orchestrator-update-create-osd

doc/mgr/orchestrator: Fix formatting and examples in create OSD

Reviewed-by: Joshua Schmid <jschmid@suse.de>
4 years agoMerge pull request #36376 from tchaikov/wip-crimson-log
Kefu Chai [Fri, 31 Jul 2020 09:45:35 +0000 (17:45 +0800)]
Merge pull request #36376 from tchaikov/wip-crimson-log

qa/tasks/ceph: redirect stderr to log file

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agobuild/debian: remove osd_support 36151/head
Joshua Schmid [Tue, 28 Jul 2020 11:43:11 +0000 (13:43 +0200)]
build/debian: remove osd_support

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agodoc/cephadm: adjust osd removal docs
Joshua Schmid [Tue, 28 Jul 2020 07:35:56 +0000 (09:35 +0200)]
doc/cephadm: adjust osd removal docs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/cephadm: add tests for new remove/replace logic
Joshua Schmid [Fri, 24 Jul 2020 13:25:31 +0000 (15:25 +0200)]
mgr/cephadm: add tests for new remove/replace logic

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/cephadm: implement osd remove/replace logic in cephadm
Joshua Schmid [Fri, 24 Jul 2020 13:24:38 +0000 (15:24 +0200)]
mgr/cephadm: implement osd remove/replace logic in cephadm

Fixes: https://tracker.ceph.com/issues/44548
Fixes: https://tracker.ceph.com/issues/45594
Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/osd_support: remove module and all traces
Joshua Schmid [Fri, 24 Jul 2020 13:23:52 +0000 (15:23 +0200)]
mgr/osd_support: remove module and all traces

Signed-off-by: Joshua Schmid <jschmid@suse.de>
4 years agoMerge pull request #36367 from tchaikov/wip-46771
Kefu Chai [Fri, 31 Jul 2020 07:39:44 +0000 (15:39 +0800)]
Merge pull request #36367 from tchaikov/wip-46771

qa/tasks/ragweed: always set ragweed_repo

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agoqa/tasks/ragweed: always set ragweed_repo 36367/head
Kefu Chai [Thu, 30 Jul 2020 08:45:40 +0000 (16:45 +0800)]
qa/tasks/ragweed: always set ragweed_repo

* extract get_ragweed_branch() out of download() task, for better
  readablity.
* use a loop for retry when the first clone fails
* drop the `raise ValueError()` clause as it never happens. we could use
  an assert() here, but i don't think it is necessary anyway.
* use sh() instead of run() for better readablity.
* always set ragweed_repo. before this change this variable is
  unbounded if `force-branch` is set.

Fixes: https://tracker.ceph.com/issues/46771
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36159 from trociny/wip-46581
Jason Dillaman [Thu, 30 Jul 2020 19:43:20 +0000 (15:43 -0400)]
Merge pull request #36159 from trociny/wip-46581

librbd: handle DNE from immutable-object-cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36319 from orozery/librados_test_stub_add_objver
Jason Dillaman [Thu, 30 Jul 2020 16:55:15 +0000 (12:55 -0400)]
Merge pull request #36319 from orozery/librados_test_stub_add_objver

test/librados_test_stub: add version tracking of objects

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36317 from orozery/librados_test_stub_exclusive_create
Jason Dillaman [Thu, 30 Jul 2020 16:54:51 +0000 (12:54 -0400)]
Merge pull request #36317 from orozery/librados_test_stub_exclusive_create

test/librados_test_stub: handle exclusive create

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoqa/tasks/ceph: redirect stderr to log file 36376/head
Kefu Chai [Thu, 30 Jul 2020 15:20:35 +0000 (23:20 +0800)]
qa/tasks/ceph: redirect stderr to log file

crimson write log to stderr, let's redirect it to log file for a more
peaceful teuthology.log.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36363 from sebastian-philipp/p-common-pyyaml
Kefu Chai [Thu, 30 Jul 2020 16:20:43 +0000 (00:20 +0800)]
Merge pull request #36363 from sebastian-philipp/p-common-pyyaml

python-common, spec.in: Add missing pyyaml dependency

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36344 from wjwithagen/wjw-fix-_1
Kefu Chai [Thu, 30 Jul 2020 16:09:14 +0000 (00:09 +0800)]
Merge pull request #36344 from wjwithagen/wjw-fix-_1

test: fix ambiguous reference to _1

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36370 from tchaikov/wip-crimson-scrub2
Kefu Chai [Thu, 30 Jul 2020 15:24:11 +0000 (23:24 +0800)]
Merge pull request #36370 from tchaikov/wip-crimson-scrub2

crimson/osd: handle MOSDScrub2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #36286 from sebastian-philipp/cephadm-notify_config-ceph-conf...
Sebastian Wagner [Thu, 30 Jul 2020 14:20:44 +0000 (16:20 +0200)]
Merge pull request #36286 from sebastian-philipp/cephadm-notify_config-ceph-conf-race

mgr/cephadm: revamp ceph.conf distribution scheduling

Reviewed-by: Ricardo Marques <rimarques@suse.com>
4 years agoMerge pull request #36231 from tspmelo/wip-fix-overflow
Kiefer Chang [Thu, 30 Jul 2020 14:02:37 +0000 (22:02 +0800)]
Merge pull request #36231 from tspmelo/wip-fix-overflow

mgr/dashboard: Configure overflow of popover in health page

Reviewed-by: Ni-Feng Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agodoc/mgr/orchestrator: Fix formatting and examples in create OSDs 36374/head
Varsha Rao [Thu, 30 Jul 2020 13:28:47 +0000 (15:28 +0200)]
doc/mgr/orchestrator: Fix formatting and examples in create OSDs

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #36243 from Devp00l/wip-46660
Volker Theile [Thu, 30 Jul 2020 13:00:59 +0000 (15:00 +0200)]
Merge pull request #36243 from Devp00l/wip-46660

mgr/dashboard: Fix regression on table error handling

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agomgr/cephadm: Add migration to keep the service names consistent 36284/head
Sebastian Wagner [Fri, 24 Jul 2020 14:20:22 +0000 (16:20 +0200)]
mgr/cephadm: Add migration to keep the service names consistent

After 15.2.4, we unified some service IDs: MONs, MGRs etc no longer have a service id.
Which means, the service names changed:

mon.foo -> mon
mgr.foo -> mgr

This fixes the data structure consistency

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm: minor cleanup in test_migration
Sebastian Wagner [Fri, 24 Jul 2020 14:17:45 +0000 (16:17 +0200)]
mgr/cephadm: minor cleanup in test_migration

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36324 from dillaman/wip-46737
Mykola Golub [Thu, 30 Jul 2020 12:04:10 +0000 (15:04 +0300)]
Merge pull request #36324 from dillaman/wip-46737

librbd: ensure image cannot be closed until in-flight IO callbacks complete

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agocrimson/osd: handle MOSDScrub2 36370/head
Kefu Chai [Thu, 30 Jul 2020 11:45:23 +0000 (19:45 +0800)]
crimson/osd: handle MOSDScrub2

MOSDScrub2 is sent from mgr for serving "ceph pg
{scrub|deep-scrub|repair}' commands when it's talking to a mimic and newer OSD.

ceph task checks if all pgs are scrubbed by looking at the `last_scrub_stamp` fields
in the `ceph pg dump` output. and it request the not-yet-scrubbed pgs a
deep scrub to ensure they are scrub before timeout.

in this change, crimson handles MOSDScrub2 by starting a remote peering
request, and the underlying peering_state will notify the corresponding
PG to start scrub. to get the test pass, a minimal implmentation is
added to update the scrub timestamp to `now` upon request of
peering_state.

we will need to add the correct scrubbing support later. but this is
enough for passing the thrasher test and for preparing for more tests
which uses the "ceph" task.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36281 from sebastian-philipp/mgr-tox-rm-cov
Sebastian Wagner [Thu, 30 Jul 2020 11:42:53 +0000 (13:42 +0200)]
Merge pull request #36281 from sebastian-philipp/mgr-tox-rm-cov

pybind/mgr: remove coverage from tox.ini

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36335 from mgfritch/cephadm-vstart-mon-cidr
Sebastian Wagner [Thu, 30 Jul 2020 10:27:08 +0000 (12:27 +0200)]
Merge pull request #36335 from mgfritch/cephadm-vstart-mon-cidr

vstart: infer the mon public_network

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #36334 from mgfritch/cephadm-event-multiline
Sebastian Wagner [Thu, 30 Jul 2020 10:13:05 +0000 (12:13 +0200)]
Merge pull request #36334 from mgfritch/cephadm-event-multiline

mgr/orch: allow for multiline OrchestratorEvent message

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36162 from Daniel-Pivonka/cephadm-43681
Sebastian Wagner [Thu, 30 Jul 2020 09:55:33 +0000 (11:55 +0200)]
Merge pull request #36162 from Daniel-Pivonka/cephadm-43681

mgr/cephadm: streamline rgw deployment

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agotest: fix ambiguous reference to _1 36344/head
Willem Jan Withagen [Wed, 29 Jul 2020 12:01:10 +0000 (12:01 +0000)]
test: fix ambiguous reference to _1

FreeBSD/Clang/lib complains:
/home/jenkins/workspace/ceph-master/src/test/librados_test_stub/TestRadosClient.cc:295:55: error: reference to '_1' is ambiguous
    &TestRadosClient::finish_aio_completion, this, c, _1));
                                                      ^
/usr/local/include/boost/bind/placeholders.hpp:46:38: note: candidate found by name lookup is 'boost::placeholders::_1'
BOOST_STATIC_CONSTEXPR boost::arg<1> _1;
                                     ^
/usr/include/c++/v1/functional:2448:46: note: candidate found by name lookup is 'std::__1::placeholders::_1'
/* _LIBCPP_INLINE_VAR */ constexpr __ph<1>   _1{};

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #36362 from tchaikov/wip-crimson-read-write-file
Kefu Chai [Thu, 30 Jul 2020 09:16:14 +0000 (17:16 +0800)]
Merge pull request #36362 from tchaikov/wip-crimson-read-write-file

crimson: extract read_file()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agopython-common, spec.in: Add missing pyyaml dependency 36363/head
Sebastian Wagner [Thu, 30 Jul 2020 07:55:08 +0000 (09:55 +0200)]
python-common, spec.in: Add missing pyyaml dependency

Fixes: cd6a488ab2ca036dd4fb36751b938f605e97e1c8
Fixes: https://tracker.ceph.com/issues/46759
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agocrimson: move read_file() into common/buffer_io 36362/head
Kefu Chai [Wed, 29 Jul 2020 08:01:20 +0000 (16:01 +0800)]
crimson: move read_file() into common/buffer_io

so it can be reused by other components in crimson

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/common: move write_file into crimson namespace
Kefu Chai [Wed, 29 Jul 2020 07:52:00 +0000 (15:52 +0800)]
crimson/common: move write_file into crimson namespace

it simply does not belong to ceph::buffer

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36355 from batrick/ptl-tool-paging
Kefu Chai [Thu, 30 Jul 2020 04:17:36 +0000 (12:17 +0800)]
Merge pull request #36355 from batrick/ptl-tool-paging

script/ptl-tool: page through github response

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocephadm: add unwrap_ipv6 helper method 36360/head
Matthew Oliver [Thu, 30 Jul 2020 03:18:48 +0000 (13:18 +1000)]
cephadm: add unwrap_ipv6 helper method

When we pass in a mon-ip that is ipv6 we want it wrapped, this is so it
can be properly inserted into the mon_addrv address.
But there are times we need to unwrap it to test it's a valid ipv6
address.

This patch adds a helper method `unwrap_ipv6` which takes a string and
returns it either unwrapped or as it is, so it's harmless to other types
of ips.

This allows us to check a wrapped ipv6 base_ip with the networks on the
host.

Signed-off-by: Matthew Oliver <moliver@suse.com>
4 years agoMerge pull request #36093 from sebastian-philipp/build-integration-branch-stable...
Kefu Chai [Thu, 30 Jul 2020 03:15:38 +0000 (11:15 +0800)]
Merge pull request #36093 from sebastian-philipp/build-integration-branch-stable-branch-name

build-integration-branch: Append stable branch name

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #36288 into master
Patrick Donnelly [Thu, 30 Jul 2020 03:13:10 +0000 (20:13 -0700)]
Merge PR #36288 into master

* refs/pull/36288/head:
mds/CInode: Optimize only pinned by subtrees check

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoscript/ptl-tool: page through github response 36355/head
Patrick Donnelly [Tue, 14 Jul 2020 02:50:00 +0000 (19:50 -0700)]
script/ptl-tool: page through github response

This fixes the script to go through the response pages from GitHub.
Previously it would only look at the first page and potentially miss
some reviews/comments.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #36249 into master
Patrick Donnelly [Thu, 30 Jul 2020 03:06:14 +0000 (20:06 -0700)]
Merge PR #36249 into master

* refs/pull/36249/head:
client: expose ceph.quota.max_bytes xattr within snapshots

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #35977 into master
Patrick Donnelly [Thu, 30 Jul 2020 03:00:06 +0000 (20:00 -0700)]
Merge PR #35977 into master

* refs/pull/35977/head:
doc/cephfs: Update about cephfs-shell custom exit codes
cephfs-shell: Define cephfs-shell exit code

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #36064 into master
Patrick Donnelly [Thu, 30 Jul 2020 02:58:16 +0000 (19:58 -0700)]
Merge PR #36064 into master

* refs/pull/36064/head:
mgr/volumes: Fix traceback of ops when volume doesn't exist

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #36200 into master
Patrick Donnelly [Thu, 30 Jul 2020 02:56:57 +0000 (19:56 -0700)]
Merge PR #36200 into master

* refs/pull/36200/head:
mds: fix mds peer request 'no_available_op_found'

Reviewed-by: Zheng Yan <zyan@redhat.com>
4 years agoMerge PR #36233 into master
Patrick Donnelly [Thu, 30 Jul 2020 02:55:28 +0000 (19:55 -0700)]
Merge PR #36233 into master

* refs/pull/36233/head:
client: fix extra open ref decrease

Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agomgr/cephadm: add qa tests for daemon restart 36338/head
Paul Cuzner [Wed, 29 Jul 2020 21:02:04 +0000 (09:02 +1200)]
mgr/cephadm: add qa tests for daemon restart

Added tests to cove daemon stop/start and restart

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
4 years agomgr/cephadm: streamline rgw deployment 36162/head
Daniel-Pivonka [Thu, 16 Jul 2020 12:24:47 +0000 (08:24 -0400)]
mgr/cephadm: streamline rgw deployment

cephadm will create realm, zonegroup, and zone if needed before creating rgw service

fixes: https://tracker.ceph.com/issues/43681
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
4 years agoMerge PR #24068 into master
Patrick Donnelly [Wed, 29 Jul 2020 18:05:02 +0000 (11:05 -0700)]
Merge PR #24068 into master

* refs/pull/24068/head:
mds: rename {CDir,Migrator}::cache to mdcache
mds: make MDSCacheObject::is_ambiguous_auth() virtual
mds: make sure rename old inode's parent dirfrag is projected.
mds: track projected inode/fnode in Mutation
mds: use smart pointer to manager CDir::fnode
mds: use smart pointer to manage CInode::{inode,xattrs,old_inodes}
osdc/Filer: make layout pointer const

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agomgr/orch: allow for multiline OrchestratorEvent message 36334/head
Michael Fritch [Tue, 28 Jul 2020 19:57:17 +0000 (13:57 -0600)]
mgr/orch: allow for multiline OrchestratorEvent message

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #36053 from tchaikov/wip-mkdir
Kefu Chai [Wed, 29 Jul 2020 14:48:04 +0000 (22:48 +0800)]
Merge pull request #36053 from tchaikov/wip-mkdir

kv: replace compat_mkdir with fs::create_directory

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agomgr/dashboard: Fix regression on table error handling 36243/head
Stephan Müller [Wed, 22 Jul 2020 14:00:32 +0000 (16:00 +0200)]
mgr/dashboard: Fix regression on table error handling

The regression was introduced by #35290 through the use of the new tab module
the pools and host listing got wrapped into the new usage, however they needed
to use the table as Viewchild and the table was static before, but it's now
dynamic. This resulted in an empty variable that wasn't filled with the
right table object. The calling of the ".reset()" was not possible
during an error case and produced an error in console trying to access
"reset" of undefined, by not calling "reset" the table get's stuck with an
rotating reload symbol.

Fixes: https://tracker.ceph.com/issues/46660
Signed-off-by: Stephan Müller <smueller@suse.com>
4 years agomds: rename {CDir,Migrator}::cache to mdcache 24068/head
Yan, Zheng [Tue, 23 Jul 2019 01:41:30 +0000 (09:41 +0800)]
mds: rename {CDir,Migrator}::cache to mdcache

make it be consistant with CInode::mdcache

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: make MDSCacheObject::is_ambiguous_auth() virtual
Yan, Zheng [Tue, 23 Jul 2019 01:14:41 +0000 (09:14 +0800)]
mds: make MDSCacheObject::is_ambiguous_auth() virtual

CInode overrides is_ambiguous_auth(). Locker calls is_ambiguous_auth()
from base class.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: make sure rename old inode's parent dirfrag is projected.
Yan, Zheng [Thu, 7 May 2020 02:33:12 +0000 (10:33 +0800)]
mds: make sure rename old inode's parent dirfrag is projected.

if rename dest dentry is remote dentry, Server::_rename_prepare() only
pre dirty old inode, but does not project fnode for old inode's parent
dirfrag. This will trigger a assertion (introduced by previous commit)
in CDir::mark_dirty().

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: track projected inode/fnode in Mutation
Yan, Zheng [Tue, 9 Jul 2019 10:15:35 +0000 (18:15 +0800)]
mds: track projected inode/fnode in Mutation

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: use smart pointer to manager CDir::fnode
Yan, Zheng [Sat, 14 Jul 2018 08:33:19 +0000 (16:33 +0800)]
mds: use smart pointer to manager CDir::fnode

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agomds: use smart pointer to manage CInode::{inode,xattrs,old_inodes}
Yan, Zheng [Thu, 16 Jul 2020 03:19:10 +0000 (11:19 +0800)]
mds: use smart pointer to manage CInode::{inode,xattrs,old_inodes}

this avoid copying whole inode_t and xattr map when journaling inodes.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
4 years agoMerge pull request #36307 from petrutlucian94/rocksdb_lz4
Kefu Chai [Wed, 29 Jul 2020 12:35:13 +0000 (20:35 +0800)]
Merge pull request #36307 from petrutlucian94/rocksdb_lz4

cmake: fix lz4 params when building rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36295 from majianpeng/bluefs-reduce-unnecessary-flush
Kefu Chai [Wed, 29 Jul 2020 12:30:11 +0000 (20:30 +0800)]
Merge pull request #36295 from majianpeng/bluefs-reduce-unnecessary-flush

os/bluestore/BlueFS: Don't flush unused device.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #36232 from mgfritch/cephadm-ok-to-stop
Kefu Chai [Wed, 29 Jul 2020 12:29:01 +0000 (20:29 +0800)]
Merge pull request #36232 from mgfritch/cephadm-ok-to-stop

mgr/cephadm: add `orch ok-to-stop` commands

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
4 years agoMerge pull request #34848 from changchengx/protocolv2
Kefu Chai [Wed, 29 Jul 2020 12:25:56 +0000 (20:25 +0800)]
Merge pull request #34848 from changchengx/protocolv2

refine class member function implementation in ProtocolV2

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agotest/librados_test_stub: handle exclusive create 36317/head
Or Ozeri [Tue, 28 Jul 2020 09:08:30 +0000 (12:08 +0300)]
test/librados_test_stub: handle exclusive create

The exclusive boolean of the create method was unused so far.
This commit adds the proper handling by checking if an object already exists, and returning EEXISTS appropriately.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
4 years agoMerge pull request #36007 from votdev/issue_46448_hosts_unit_tests
Tatjana Dehler [Wed, 29 Jul 2020 11:33:14 +0000 (13:33 +0200)]
Merge pull request #36007 from votdev/issue_46448_hosts_unit_tests

mgr/dashboard: Add hosts page unit tests

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
4 years agolibrbd: potentially delay completion of image dispatcher spec 36324/head
Jason Dillaman [Wed, 29 Jul 2020 11:30:28 +0000 (07:30 -0400)]
librbd: potentially delay completion of image dispatcher spec

If an AioCompletion is being completed for an external API user, ensure
that the completion of image dispatcher finalizer does not race with the
potential to close the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36320 from s0nea/wip-dashboard-46573
Volker Theile [Wed, 29 Jul 2020 11:24:14 +0000 (13:24 +0200)]
Merge pull request #36320 from s0nea/wip-dashboard-46573

mgr/dashboard: wait longer for health status to be cleared

Reviewed-by: Ni-Feng Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoclient: expose ceph.quota.max_bytes xattr within snapshots 36249/head
Shyamsundar Ranganathan [Wed, 22 Jul 2020 19:21:50 +0000 (15:21 -0400)]
client: expose ceph.quota.max_bytes xattr within snapshots

For directories within snapshots, expose the ceph.quota.max_bytes
extended attribute information. This enables fetching quota
information when the snapshot was taken and is particularly useful
when cloning subvolume snapshots, to enforce the quota on the
clone subvolume as well.

Fixes: https://tracker.ceph.com/issues/46278
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
4 years agomgr: don't update osd stat which is already out 36002/head
Zhi Zhang [Wed, 29 Jul 2020 10:47:42 +0000 (18:47 +0800)]
mgr: don't update osd stat which is already out

Ceph status still reports slow requests on the OSD which is already out.
When orignal PG monitor handled PGSTATS msg, it wouldn't update osd stat
if this OSD is not in OSD map, but current MGR had no checks on that.

Fixes: https://tracker.ceph.com/issues/46440
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
4 years agoMerge pull request #36323 from tchaikov/wip-crimson-msgr-v1-v2
Kefu Chai [Wed, 29 Jul 2020 09:05:36 +0000 (17:05 +0800)]
Merge pull request #36323 from tchaikov/wip-crimson-msgr-v1-v2

crimson: picking peer addr of the compatible type

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #35728 from jan--f/c-v-add-subcommand-parse-drive-groups
Jan Fajerski [Wed, 29 Jul 2020 08:56:57 +0000 (10:56 +0200)]
Merge pull request #35728 from jan--f/c-v-add-subcommand-parse-drive-groups

ceph-volume: add drive-group subcommand

4 years agoMerge pull request #36342 from tchaikov/wip-crimson-heartbeat-erase
Kefu Chai [Wed, 29 Jul 2020 08:48:23 +0000 (16:48 +0800)]
Merge pull request #36342 from tchaikov/wip-crimson-heartbeat-erase

crimson/osd: erase an element by iterator instead

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #36217 from sebastian-philipp/cephadm-common-mypy-ini
Sebastian Wagner [Wed, 29 Jul 2020 08:19:42 +0000 (10:19 +0200)]
Merge pull request #36217 from sebastian-philipp/cephadm-common-mypy-ini

cephadm: use src/mypy.ini instead

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #36235 from matthewoliver/cephadm_iscsi_tcmu_runner
Sebastian Wagner [Wed, 29 Jul 2020 08:15:07 +0000 (10:15 +0200)]
Merge pull request #36235 from matthewoliver/cephadm_iscsi_tcmu_runner

cephadm: Add tcmu-runner container when deploying ceph-iscsi

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agocrimson/osd: implement cls_get_pool_stripe_width 36342/head
Kefu Chai [Wed, 29 Jul 2020 07:33:59 +0000 (15:33 +0800)]
crimson/osd: implement cls_get_pool_stripe_width

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: erase an element by iterator instead
Kefu Chai [Wed, 29 Jul 2020 06:37:57 +0000 (14:37 +0800)]
crimson/osd: erase an element by iterator instead

we should not remove an element while iterating it in a map, as erasing
the element invalidates the iterator, which causes segmfault when we are
advancing it after erasing the dereferenced element.

in this change, an iterator is used for walking through the map, in
comparision with creating a to-be-removed list, this one is more
efficient and more idiomatic.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoqa/tasks/ceph: do not print out empty list of pg 36337/head
Kefu Chai [Wed, 29 Jul 2020 01:27:28 +0000 (09:27 +0800)]
qa/tasks/ceph: do not print out empty list of pg

we could have following logging messages:

tasks.ceph:Waiting for all PGs to be active+clean and split+merged, waiting on ['2.6', '2.5', '1.0', '2.4'] to go clean and/or [] to split/merge

if the cluster has non-active+clean pgs when the "ceph" is about to
end. but this message is a little bit confusing in the sense it
lists "[]" in it.

in this change, only PGs being waited are listed. also, added some
cleanups:

* use "else" to check if the loop is terminated by a break
* remove "0" from the range() call

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36341 from tchaikov/wip-crimson-cls
Kefu Chai [Wed, 29 Jul 2020 06:18:56 +0000 (14:18 +0800)]
Merge pull request #36341 from tchaikov/wip-crimson-cls

crimson/osd: correct the function name of cls_cxx_map_get_vals_by_keys()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: correct the function name of cls_cxx_map_get_vals_by_keys() 36341/head
Kefu Chai [Wed, 29 Jul 2020 04:32:26 +0000 (12:32 +0800)]
crimson/osd: correct the function name of cls_cxx_map_get_vals_by_keys()

it was an oversight in 7a4c6359e483f8c71276ece5cde16eb0771ac5d2

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/cephadm: fix call to cephadm for daemon restarts etc
Paul Cuzner [Wed, 29 Jul 2020 04:00:37 +0000 (16:00 +1200)]
mgr/cephadm: fix call to cephadm for daemon restarts etc

The call currently passes the daemon_type as the first
parameter - but the function expects the hostname. This
results in failures when attempting daemon restarts through the ceph orch daemon <action> <daemon_id>
command.

Fixes: https://tracker.ceph.com/issues/46740
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
4 years agoMerge pull request #36079 from winndows/superfluous_break6
Kefu Chai [Wed, 29 Jul 2020 01:44:46 +0000 (09:44 +0800)]
Merge pull request #36079 from winndows/superfluous_break6

msg: Remove superfluous breaks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36297 from dvanders/dvanders_46443
Neha Ojha [Wed, 29 Jul 2020 01:11:12 +0000 (18:11 -0700)]
Merge pull request #36297 from dvanders/dvanders_46443

osd: fix crash in _committed_osd_maps if incremental osdmap crc fails

Reviewed-by: Xiaoxi Chen <xiaoxchen@ebay.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agomgr/cephadm: add `orch host ok-to-stop` command 36232/head
Michael Fritch [Tue, 21 Jul 2020 21:06:19 +0000 (15:06 -0600)]
mgr/cephadm: add `orch host ok-to-stop` command

$ ceph orch host ok-to-stop host1
It is presumed safe to stop host host1

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/cephadm: return HandleCommandResult from ok_to_stop
Michael Fritch [Tue, 21 Jul 2020 21:26:43 +0000 (15:26 -0600)]
mgr/cephadm: return HandleCommandResult from ok_to_stop

- return output from the result of the ok_to_stop command
- log ok-to-stop result during all invocations

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/orch: add errno to OrchestratorError
Michael Fritch [Wed, 22 Jul 2020 23:43:05 +0000 (17:43 -0600)]
mgr/orch: add errno to OrchestratorError

add errno to OrchestratorError and ServiceSpecValidationError exceptions

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agoqa/suites/rados/thrash/crc-failures: randomly inject bad incremental osdmap crc 36297/head
Neha Ojha [Tue, 28 Jul 2020 17:36:09 +0000 (10:36 -0700)]
qa/suites/rados/thrash/crc-failures: randomly inject bad incremental osdmap crc

Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agoosd: don't write transaction when inc crc failed
Dan van der Ster [Mon, 27 Jul 2020 15:40:27 +0000 (17:40 +0200)]
osd: don't write transaction when inc crc failed

80da5f9a987c6a48b93f25228fdac85890013520 exposed a flaw in how
handle_osd_map falls back to a full osdmap if the crc of an incremental
failed.

If the first message in a map message had a crc error, then the
loop would exit with last < start, which would then cause a null
dereference in _committed_osd_maps.

Fixes: https://tracker.ceph.com/issues/46443
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
4 years agoqa/standalone/osd: add bad-inc-map.sh
Dan van der Ster [Mon, 27 Jul 2020 12:23:54 +0000 (14:23 +0200)]
qa/standalone/osd: add bad-inc-map.sh

Test that the osd doesn't crash when it gets a bad incremental osdmap.

Related-to: https://tracker.ceph.com/issues/46443
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
4 years agovstart: infer the mon public_network 36335/head
Michael Fritch [Tue, 28 Jul 2020 17:24:21 +0000 (11:24 -0600)]
vstart: infer the mon public_network

set the mon public_network when deploying with the cephadm flag

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #36287 from dillaman/wip-librbd-close
Mykola Golub [Tue, 28 Jul 2020 18:30:07 +0000 (21:30 +0300)]
Merge pull request #36287 from dillaman/wip-librbd-close

librbd: use task finisher thread for image open/close callbacks

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #36253 from changchengx/exclusive
Jason Dillaman [Tue, 28 Jul 2020 17:33:17 +0000 (13:33 -0400)]
Merge pull request #36253 from changchengx/exclusive

doc: specify RBD_LOCK_MODE_EXCLUSIVE for exclusive-lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36328 from tchaikov/wip-crimson-cls_cxx_map_get_vals
Kefu Chai [Tue, 28 Jul 2020 15:22:14 +0000 (23:22 +0800)]
Merge pull request #36328 from tchaikov/wip-crimson-cls_cxx_map_get_vals

crimson/osd: implement cls_cxx_map_get_vals()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: implement cls_cxx_map_get_vals() 36328/head
Kefu Chai [Tue, 28 Jul 2020 14:39:21 +0000 (22:39 +0800)]
crimson/osd: implement cls_cxx_map_get_vals()

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc: specify RBD_LOCK_MODE_EXCLUSIVE for exclusive-lock 36253/head
Changcheng Liu [Thu, 23 Jul 2020 03:09:46 +0000 (11:09 +0800)]
doc: specify RBD_LOCK_MODE_EXCLUSIVE for exclusive-lock

The exclusive-lock could be transited transparently between clients
after finishing write operation. To disable "transparent" transition,
it needs to acquire the lock with RBD_LOCK_MODE_EXCLUSIVE.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
4 years agolibrbd: ensure image cannot be closed until in-flight IO callbacks complete
Jason Dillaman [Tue, 28 Jul 2020 13:07:49 +0000 (09:07 -0400)]
librbd: ensure image cannot be closed until in-flight IO callbacks complete

If a librbd client attempts to close the image while it still has in-flight IO
pending, it's possible for the AsyncOperation tracker which prevents the image
from being closed to be completed before the actual AioCompletion callback
fires. This can result in the now destructed ImageCtx being de-referenced by
the AioCompletion.

Fixes: https://tracker.ceph.com/issues/46737
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #34928 from p-se/wip-pse-revise-monitoring-doc
Kefu Chai [Tue, 28 Jul 2020 12:49:58 +0000 (20:49 +0800)]
Merge pull request #34928 from p-se/wip-pse-revise-monitoring-doc

mgr/dashboard: revise monitoring documentation

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>