ceph-ci.git
9 months agoMerge PR #36773 into master 20201217.205941 fs-wip-pdonnell-testing-20201217 fs-wip-pdonnell-testing-20201217.205941 kotresh multimds-wip-pdonnell-testing-20201007.214100 wip-sage-testing
Patrick Donnelly [Wed, 26 Aug 2020 17:46:07 +0000 (10:46 -0700)]
Merge PR #36773 into master

* refs/pull/36773/head:
mgr/volumes: Prevent subvolume recreate if trash is not-empty
mgr/volumes: Disallow subvolume group level snapshots
mgr/volumes: Add test case to ensure subvolume is marked
mgr/volumes: handle idempotent subvolume marks
mgr/volumes: Tests amended and added to ensure subvolume trash functionality
mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
mgr/volumes: maintain per subvolume trash directory
mgr/volumes: make subvolume_v2::_is_retained() object property

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agomgr/volumes: Prevent subvolume recreate if trash is not-empty
Shyamsundar Ranganathan [Wed, 26 Aug 2020 12:41:11 +0000 (08:41 -0400)]
mgr/volumes: Prevent subvolume recreate if trash is not-empty

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
9 months agomgr/volumes: Disallow subvolume group level snapshots
Shyamsundar Ranganathan [Tue, 25 Aug 2020 22:46:12 +0000 (18:46 -0400)]
mgr/volumes: Disallow subvolume group level snapshots

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
9 months agomgr/volumes: Add test case to ensure subvolume is marked
Shyamsundar Ranganathan [Tue, 25 Aug 2020 15:44:50 +0000 (11:44 -0400)]
mgr/volumes: Add test case to ensure subvolume is marked

with the vxattr ceph.dir.subvolume set to true.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
9 months agomgr/volumes: handle idempotent subvolume marks
Venky Shankar [Tue, 25 Aug 2020 06:57:50 +0000 (02:57 -0400)]
mgr/volumes: handle idempotent subvolume marks

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
9 months agomgr/volumes: Tests amended and added to ensure subvolume trash functionality
Shyamsundar Ranganathan [Mon, 24 Aug 2020 19:10:44 +0000 (15:10 -0400)]
mgr/volumes: Tests amended and added to ensure subvolume trash functionality

Amended a few test cases to ensure created subvolumes and snaps
are removed, and trash stays empty at the end of the test.

Further added one test case for create errors in a retained
v2 subvolume, to ensure metadata is sane, and created incarnation
is not present.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
9 months agomgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
Shyamsundar Ranganathan [Mon, 24 Aug 2020 13:33:10 +0000 (09:33 -0400)]
mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume

This enables snapshot scaling of subvolumes, and hence prevents
renaming, hardlinking etc. outside the subvolume.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
9 months agomgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
Shyamsundar Ranganathan [Sun, 23 Aug 2020 16:31:01 +0000 (12:31 -0400)]
mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash

For v2 subvolumes without any snapshots or pending purges
move the subvolume to trash, for the purge operation.

When removing only incarnations, leverage symlink based purge
job creation, to avoid renaming the incarnation out of the
subvolume.

For create errors, where the subvolume was retained, just remove
the created incarnation directory.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
9 months agomgr/volumes: maintain per subvolume trash directory
Venky Shankar [Fri, 21 Aug 2020 14:07:37 +0000 (10:07 -0400)]
mgr/volumes: maintain per subvolume trash directory

PR https://github.com/ceph/ceph/pull/36472 introduces changes
that disallow nested nested snapshots in a subtree (subvolume)
and renames across subvolumes. This effect asynchronous purge
in mgr/volumes as subvolume are moved to a trash directory for
asynchronous deletion by purge threads.

To workaround this, start maintaining a subvolume specific
trash directory. Use the trash directory as an index to the
subvolume specific trash directory entry.

This changes subvolume deletion logic which currently relies
on `--retain-snapshots` flag to decide if the subvolume user
directory should get purged or the subvolume base directory
itself. Deleting a subvolume moves the user facing directory
to its specific trash directory. Purge threads take care of
deleting user facing directories (in trash) and the subvolume
base directory if required (when certain conditions are met).

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
9 months agomgr/volumes: make subvolume_v2::_is_retained() object property
Venky Shankar [Fri, 21 Aug 2020 05:18:45 +0000 (01:18 -0400)]
mgr/volumes: make subvolume_v2::_is_retained() object property

... and drop the leading underscore.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge PR #36805 into master
Patrick Donnelly [Wed, 26 Aug 2020 15:27:43 +0000 (08:27 -0700)]
Merge PR #36805 into master

* refs/pull/36805/head:
pybind/mgr/volumes: add global lock debug

Reviewed-by: Venky Shankar <vshankar@redhat.com>
9 months agoMerge pull request #36774 from dillaman/wip-46669-2
Mykola Golub [Wed, 26 Aug 2020 15:22:34 +0000 (18:22 +0300)]
Merge pull request #36774 from dillaman/wip-46669-2

librbd: drop deferred AsioEngine destruction

Reviewed-by: Mykola Golub <mgolub@suse.com>
9 months agoMerge pull request #36582 from rhcs-dashboard/fix-redirected-URL
Lenz Grimmer [Wed, 26 Aug 2020 14:25:56 +0000 (16:25 +0200)]
Merge pull request #36582 from rhcs-dashboard/fix-redirected-URL

mgr/dashboard: redirect to original URL after successful login

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
9 months agopybind/mgr/volumes: add global lock debug
Patrick Donnelly [Wed, 5 Aug 2020 22:54:33 +0000 (15:54 -0700)]
pybind/mgr/volumes: add global lock debug

Fixes: https://tracker.ceph.com/issues/47149
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agoMerge pull request #36817 from sebastian-philipp/cephadm-pep8-tests
Sebastian Wagner [Wed, 26 Aug 2020 12:27:57 +0000 (14:27 +0200)]
Merge pull request #36817 from sebastian-philipp/cephadm-pep8-tests

mgr/cephadm: PEP8tify some tests

Reviewed-by: Adam King <adking@redhat.com>
9 months agoMerge pull request #36315 from p-se/fix-monitoring-images-ceph-salt
Sebastian Wagner [Wed, 26 Aug 2020 11:28:00 +0000 (13:28 +0200)]
Merge pull request #36315 from p-se/fix-monitoring-images-ceph-salt

cephadm: deploying of monitoring images partially broken

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
9 months agoMerge pull request #36571 from pcuzner/cephadm-tox-update
Sebastian Wagner [Wed, 26 Aug 2020 10:56:26 +0000 (12:56 +0200)]
Merge pull request #36571 from pcuzner/cephadm-tox-update

cephadm: remove py2 from tox tests

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
9 months agomgr/cephadm: PEP8tify some tests
Sebastian Wagner [Wed, 26 Aug 2020 09:45:56 +0000 (11:45 +0200)]
mgr/cephadm: PEP8tify some tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
9 months agoMerge pull request #36794 from sebastian-philipp/cephadm-pep8-cephadmservice.py
Sebastian Wagner [Wed, 26 Aug 2020 09:37:51 +0000 (11:37 +0200)]
Merge pull request #36794 from sebastian-philipp/cephadm-pep8-cephadmservice.py

mgr/cephadm: PEP8tify cephadmservice.py

Reviewed-by: Adam King <adking@redhat.com>
9 months agoMerge pull request #36761 from wjwithagen/wjw-fix-dashboard-package.json
Lenz Grimmer [Wed, 26 Aug 2020 08:24:18 +0000 (10:24 +0200)]
Merge pull request #36761 from wjwithagen/wjw-fix-dashboard-package.json

dashboard: Fix building build shell script

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
9 months agoMerge pull request #36527 from neha-ojha/wip-whitelist-blacklist
Neha Ojha [Tue, 25 Aug 2020 19:24:36 +0000 (12:24 -0700)]
Merge pull request #36527 from neha-ojha/wip-whitelist-blacklist

Replace 'whitelist' and 'blacklist' language

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
9 months agoMerge pull request #36509 from pcuzner/cephadm-add-host-facts
Sebastian Wagner [Tue, 25 Aug 2020 18:29:18 +0000 (20:29 +0200)]
Merge pull request #36509 from pcuzner/cephadm-add-host-facts

cephadm: add gather_facts command

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
9 months agoMerge pull request #36793 from sebastian-philipp/cephadm-pep8-schedule.py
Sebastian Wagner [Tue, 25 Aug 2020 18:27:27 +0000 (20:27 +0200)]
Merge pull request #36793 from sebastian-philipp/cephadm-pep8-schedule.py

mgr/cephadm: PEP8tify schedule.py

Reviewed-by: Adam King <adking@redhat.com>
9 months agoMerge pull request #36757 from mgfritch/cephadm-raw-str-warn
Sebastian Wagner [Tue, 25 Aug 2020 16:31:33 +0000 (18:31 +0200)]
Merge pull request #36757 from mgfritch/cephadm-raw-str-warn

mgr/cephadm: use raw string during `re.search`

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
9 months agoMerge pull request #36787 from tchaikov/wip-47031
Kefu Chai [Tue, 25 Aug 2020 14:14:42 +0000 (22:14 +0800)]
Merge pull request #36787 from tchaikov/wip-47031

crimson/osd: drop misdirected ops

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agomgr/cephadm: PEP8tify cephadmservice.py
Sebastian Wagner [Tue, 25 Aug 2020 13:33:06 +0000 (15:33 +0200)]
mgr/cephadm: PEP8tify cephadmservice.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
9 months agomgr/cephadm: PEP8tify schedule.py
Sebastian Wagner [Tue, 25 Aug 2020 13:30:51 +0000 (15:30 +0200)]
mgr/cephadm: PEP8tify schedule.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
9 months agoMerge pull request #36738 from ShraddhaAg/cephadm-archive-logs
Sebastian Wagner [Tue, 25 Aug 2020 13:18:58 +0000 (15:18 +0200)]
Merge pull request #36738 from ShraddhaAg/cephadm-archive-logs

qa/tasks/cephadm.py: add ceph logs directory in job's info.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
9 months agomgr/dashboard: redirect to original URL after successful login
Avan Thakkar [Wed, 12 Aug 2020 09:22:22 +0000 (14:52 +0530)]
mgr/dashboard: redirect to original URL after successful login

Fixes: https://tracker.ceph.com/issues/45067

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b5359a155c6ff7e92ab6c6ed16f7ff9fc76c76ea)

9 months agocrimson/osd: drop misdirected ops
Kefu Chai [Tue, 25 Aug 2020 09:30:00 +0000 (17:30 +0800)]
crimson/osd: drop misdirected ops

see also `PrimaryLogPG::do_op()`, we should ignore the ops hitting us if
we are not supposed to serve them. this happens when the client is using
a stale osdmap.

Fixes: https://tracker.ceph.com/issues/47031
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agoMerge pull request #36778 from tchaikov/wip-cmake-fio
Kefu Chai [Tue, 25 Aug 2020 09:16:22 +0000 (17:16 +0800)]
Merge pull request #36778 from tchaikov/wip-cmake-fio

cmake: silence "detached HEAD" warning

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
9 months agocmake: silence "detached HEAD" warning
Kefu Chai [Tue, 25 Aug 2020 04:34:15 +0000 (12:34 +0800)]
cmake: silence "detached HEAD" warning

git complains when checking out a tag in "detached HEAD", like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them
...

but this does not help, as, in general, we don't hack fio in Ceph,
so disable this warning. and also clone the repo in shallow mode
for the same reason -- we don't care about the whole history of
fio repo. we just use it for testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agoMerge pull request #36336 from pcuzner/ceph-volume-lsm
pcuzner [Mon, 24 Aug 2020 23:23:28 +0000 (11:23 +1200)]
Merge pull request #36336 from pcuzner/ceph-volume-lsm

ceph-volume: add libstoragemgmt support

9 months agocephadm: remove py2 from test_adoption
Paul Cuzner [Mon, 24 Aug 2020 22:51:26 +0000 (10:51 +1200)]
cephadm: remove py2 from test_adoption

Since cephadm is py3 based, and py2 is EOL this patch
removes the py2 test iteration from test_adoption.sh

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
9 months agoMerge PR #36684 into master
Patrick Donnelly [Mon, 24 Aug 2020 20:18:13 +0000 (13:18 -0700)]
Merge PR #36684 into master

* refs/pull/36684/head:
qa/tasks/nfs: Test mounting of export created with nfs command
qa/tasks/nfs: Add helper method to check nfs cluster status
qa/tasks/nfs: Cleanup created filesystem
qa/tasks/nfs: Remove unused port status function and 'stdin' keyword argument

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agolibrbd: support reverting to legacy blacklist op via neorados API
Jason Dillaman [Mon, 24 Aug 2020 15:24:31 +0000 (11:24 -0400)]
librbd: support reverting to legacy blacklist op via neorados API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 months agomgr/dashboard: remove racially insensitive terms
Ernesto Puerta [Thu, 20 Aug 2020 12:22:57 +0000 (14:22 +0200)]
mgr/dashboard: remove racially insensitive terms

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
9 months agolibrbd: RESTORE_SOURCE_WHITELIST -> ALLOWED_RESTORE_SOURCES
Neha Ojha [Wed, 19 Aug 2020 17:30:38 +0000 (17:30 +0000)]
librbd: RESTORE_SOURCE_WHITELIST -> ALLOWED_RESTORE_SOURCES

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agojournal, librbd: whitelisted_laggy_clients -> ignored_laggy_clients
Neha Ojha [Wed, 19 Aug 2020 17:24:03 +0000 (17:24 +0000)]
journal, librbd: whitelisted_laggy_clients -> ignored_laggy_clients

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months ago*: convert more blacklist to blocklist
Patrick Donnelly [Thu, 13 Aug 2020 20:29:46 +0000 (13:29 -0700)]
*: convert more blacklist to blocklist

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agolibrbd: convert WHITELIST to ALLOWLIST
Patrick Donnelly [Thu, 13 Aug 2020 20:29:23 +0000 (13:29 -0700)]
librbd: convert WHITELIST to ALLOWLIST

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
9 months ago*: convert EBLACKLISTED to EBLOCKLISTED
Patrick Donnelly [Thu, 13 Aug 2020 20:16:43 +0000 (13:16 -0700)]
*: convert EBLACKLISTED to EBLOCKLISTED

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agotest/librados_test_stub/TestMemIoCtxImpl.cc: blacklist -> blocklist
Neha Ojha [Fri, 7 Aug 2020 19:19:55 +0000 (19:19 +0000)]
test/librados_test_stub/TestMemIoCtxImpl.cc: blacklist -> blocklist

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agoqa/suites/rados/thrash: log-whitelist -> log-ignorelist
Neha Ojha [Fri, 7 Aug 2020 17:12:28 +0000 (17:12 +0000)]
qa/suites/rados/thrash: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agoqa/suites/krbd/thrash: log-whitelist -> log-ignorelist
Neha Ojha [Fri, 7 Aug 2020 16:37:57 +0000 (16:37 +0000)]
qa/suites/krbd/thrash: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agoqa/suites/multimds: log-whitelist -> log-ignorelist
Neha Ojha [Fri, 7 Aug 2020 16:35:30 +0000 (16:35 +0000)]
qa/suites/multimds: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agotest/librbd/managed_lock/test_mock_BreakRequest.cc: blacklist -> blocklist
Neha Ojha [Tue, 28 Jul 2020 14:37:51 +0000 (14:37 +0000)]
test/librbd/managed_lock/test_mock_BreakRequest.cc: blacklist -> blocklist

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agotest/librados_test_stub/TestRadosClient.cc: blacklist -> blocklist
Neha Ojha [Mon, 27 Jul 2020 21:50:02 +0000 (21:50 +0000)]
test/librados_test_stub/TestRadosClient.cc: blacklist -> blocklist

Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agomon/ConfigMonitor: handle renamed config options
Sage Weil [Fri, 26 Jun 2020 22:05:31 +0000 (17:05 -0500)]
mon/ConfigMonitor: handle renamed config options

Signed-off-by: Sage Weil <sage@newdream.net>
9 months agolibrados: fall back to legacy blacklist command if blocklist gets EINVAL
Sage Weil [Fri, 26 Jun 2020 22:04:51 +0000 (17:04 -0500)]
librados: fall back to legacy blacklist command if blocklist gets EINVAL

I would have expected ENOSYS or EOPNOTSUPP or similar, but the mon returns
EINVAL on an unrecognized command

Signed-off-by: Sage Weil <sage@newdream.net>
9 months agomisc language changes: whitelist -> ignore etc
Sage Weil [Wed, 17 Jun 2020 19:43:47 +0000 (14:43 -0500)]
misc language changes: whitelist -> ignore etc

Signed-off-by: Sage Weil <sage@newdream.net>
9 months agolibrbd: whitelisted_laggy_clients -> allowlisted_laggy_clients
Sage Weil [Wed, 17 Jun 2020 19:44:55 +0000 (14:44 -0500)]
librbd: whitelisted_laggy_clients -> allowlisted_laggy_clients

Signed-off-by: Sage Weil <sage@newdream.net>
9 months agoosd/OSDCap: whitelisted -> allowed
Sage Weil [Wed, 17 Jun 2020 19:43:29 +0000 (14:43 -0500)]
osd/OSDCap: whitelisted -> allowed

Signed-off-by: Sage Weil <sage@newdream.net>
9 months agoqa: log-whitelist -> log-ignorelist
Sage Weil [Wed, 17 Jun 2020 19:33:57 +0000 (14:33 -0500)]
qa: log-whitelist -> log-ignorelist

Signed-off-by: Sage Weil <sage@newdream.net>
9 months agoblacklist -> blocklist
Sage Weil [Mon, 15 Jun 2020 15:27:03 +0000 (10:27 -0500)]
blacklist -> blocklist

Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agoMerge PR #36394 into master
Patrick Donnelly [Mon, 24 Aug 2020 18:33:03 +0000 (11:33 -0700)]
Merge PR #36394 into master

* refs/pull/36394/head:
qa: add method run ceph cluster command with better interface
cephfs: allow only "good" characters

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agoMerge pull request #36772 from neha-ojha/wip-47119
Yuri Weinstein [Mon, 24 Aug 2020 18:28:21 +0000 (11:28 -0700)]
Merge pull request #36772 from neha-ojha/wip-47119

qa/suites/perf-basic/objectstore: remove filestore

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
9 months agolibrbd: drop deferred AsioEngine destruction
Jason Dillaman [Mon, 24 Aug 2020 17:31:17 +0000 (13:31 -0400)]
librbd: drop deferred AsioEngine destruction

Commit 90bd1d7a857c0f3c57bda60975f58f9859940185 removed the need to
defer deletion of the AsioEngine but missed removing the original
code.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 months agoqa/suites/perf-basic/objectstore: remove filestore
Neha Ojha [Mon, 24 Aug 2020 16:45:49 +0000 (09:45 -0700)]
qa/suites/perf-basic/objectstore: remove filestore

There is no point testing performance on filestore anymore.
It has already been removed from objectstore backends in rados/perf.

Fixes: https://tracker.ceph.com/issues/47119
Signed-off-by: Neha Ojha <nojha@redhat.com>
9 months agoMerge pull request #36760 from tchaikov/wip-qa-optional-args
Kefu Chai [Mon, 24 Aug 2020 12:03:04 +0000 (20:03 +0800)]
Merge pull request #36760 from tchaikov/wip-qa-optional-args

qa/tasks/workunit: allow passing optional args to workunit

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #36677 from bk201/wip-46986
Lenz Grimmer [Mon, 24 Aug 2020 11:34:50 +0000 (13:34 +0200)]
Merge pull request #36677 from bk201/wip-46986

mgr/dashboard: adjust cross button color in table filter chips

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
9 months agoqa/tasks/nfs: Test mounting of export created with nfs command
Varsha Rao [Mon, 17 Aug 2020 08:37:06 +0000 (14:07 +0530)]
qa/tasks/nfs: Test mounting of export created with nfs command

Fixes: https://tracker.ceph.com/issues/46989
Signed-off-by: Varsha Rao <varao@redhat.com>
9 months agoqa/tasks/nfs: Add helper method to check nfs cluster status
Varsha Rao [Mon, 17 Aug 2020 07:42:11 +0000 (13:12 +0530)]
qa/tasks/nfs: Add helper method to check nfs cluster status

Signed-off-by: Varsha Rao <varao@redhat.com>
9 months agoqa/tasks/nfs: Cleanup created filesystem
Varsha Rao [Mon, 17 Aug 2020 07:37:33 +0000 (13:07 +0530)]
qa/tasks/nfs: Cleanup created filesystem

Signed-off-by: Varsha Rao <varao@redhat.com>
9 months agoqa/tasks/nfs: Remove unused port status function and 'stdin' keyword argument
Varsha Rao [Mon, 17 Aug 2020 07:20:36 +0000 (12:50 +0530)]
qa/tasks/nfs: Remove unused port status function and 'stdin' keyword argument

Signed-off-by: Varsha Rao <varao@redhat.com>
9 months agoMerge pull request #36737 from tchaikov/wip-crimson-more-ops
Kefu Chai [Mon, 24 Aug 2020 11:28:35 +0000 (19:28 +0800)]
Merge pull request #36737 from tchaikov/wip-crimson-more-ops

crimson/osd: implement append and omap-clear ops

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoMerge pull request #36347 from rhcs-dashboard/wip-40767-master
Lenz Grimmer [Mon, 24 Aug 2020 11:22:08 +0000 (13:22 +0200)]
Merge pull request #36347 from rhcs-dashboard/wip-40767-master

mgr/dashboard: Placeholders for Documenting REST API

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
9 months agoMerge pull request #36765 from sebastian-philipp/man-cephadm-list-no-detail
Sebastian Wagner [Mon, 24 Aug 2020 10:25:39 +0000 (12:25 +0200)]
Merge pull request #36765 from sebastian-philipp/man-cephadm-list-no-detail

doc/man/cephadm: Add missing parameters for `ls`

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
9 months agocrimson/os/cyanstore: handle OP_RMATTR
Kefu Chai [Fri, 21 Aug 2020 08:15:08 +0000 (16:15 +0800)]
crimson/os/cyanstore: handle OP_RMATTR

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: avoid creating a throw-away object
Kefu Chai [Fri, 21 Aug 2020 07:44:59 +0000 (15:44 +0800)]
crimson/osd: avoid creating a throw-away object

for better readabilty and performnce

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: implement rm-xattr op
Kefu Chai [Fri, 21 Aug 2020 07:43:28 +0000 (15:43 +0800)]
crimson/osd: implement rm-xattr op

it is tested by test_rados.py:TestIoctx.test_xattrs_op, so let's
support this op in crimson as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: check for DNE object and return ENOENT in read ops
Kefu Chai [Fri, 21 Aug 2020 04:31:16 +0000 (12:31 +0800)]
crimson/osd: check for DNE object and return ENOENT in read ops

* omap_get_keys()
  this change addresses the failure of
  test_rados.py:TestIoctx.test_get_omap_keys
* omap_get_vals_by_keys()
  this change addresses the failure of
  test_rados.py:TestIoctx.test_get_omap_vals_by_keys
* read()
  this change addresses the failure of
  test_rados.py:TestIoctx.test_write_ops

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: refactor PGBackend::read() to pass os and op
Kefu Chai [Fri, 21 Aug 2020 06:27:56 +0000 (14:27 +0800)]
crimson/osd: refactor PGBackend::read() to pass os and op

prepare for the table based op lookup/execution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agotest/pybind: remove object for testing after done with it
Kefu Chai [Fri, 21 Aug 2020 05:30:24 +0000 (13:30 +0800)]
test/pybind: remove object for testing after done with it

* add vertical spacing for better readability
* remove object for testing after done with it

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/os/cyanstore: handle OP_ZERO
Kefu Chai [Fri, 21 Aug 2020 05:28:15 +0000 (13:28 +0800)]
crimson/os/cyanstore: handle OP_ZERO

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: update oi.size after truncating an object
Kefu Chai [Fri, 21 Aug 2020 05:08:42 +0000 (13:08 +0800)]
crimson/osd: update oi.size after truncating an object

* update oi.size if object size changes after the object is truncated
* do not add a truncate op to trasaction of the size of object does
  not change because of truncate op.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: return rval which is negative
Kefu Chai [Thu, 20 Aug 2020 17:29:51 +0000 (01:29 +0800)]
crimson/osd: return rval which is negative

a less-than-zero rval indicates an error, and should not be normalized
to 0 if allows_returnvec() evaluates to false. probably we need a better
way to return a negative error code which does not fall into any known
error. but at this moment, grab the last rval and return it if it is
less than zero, can be used as a short term solution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: implement cmp-ext op
Kefu Chai [Thu, 20 Aug 2020 17:28:07 +0000 (01:28 +0800)]
crimson/osd: implement cmp-ext op

it is tested by test_rados.py:TestIoctx.test_cmpext, so let's
support this op in crimson as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/os/cyanstore: handle OP_OMAP_CLEAR
Kefu Chai [Thu, 20 Aug 2020 13:06:24 +0000 (21:06 +0800)]
crimson/os/cyanstore: handle OP_OMAP_CLEAR

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: implement omap-clear op
Kefu Chai [Thu, 20 Aug 2020 12:52:11 +0000 (20:52 +0800)]
crimson/osd: implement omap-clear op

it is tested by test_rados.py:TestIoctx.test_clear_omap, so let's support
this op in crimson as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agocrimson/osd: implement append op
Kefu Chai [Thu, 20 Aug 2020 11:40:41 +0000 (19:40 +0800)]
crimson/osd: implement append op

it is tested by test_rados.py:TestIoctx.test_append, so let's support
this op in crimson as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agoCODEOWNERS: Add cephadm man page to @ceph/orchestrators
Sebastian Wagner [Mon, 24 Aug 2020 08:30:12 +0000 (10:30 +0200)]
CODEOWNERS: Add cephadm man page to @ceph/orchestrators

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
9 months agodoc/man/cephadm: Add missing parameters for `ls`
Sebastian Wagner [Mon, 24 Aug 2020 08:28:07 +0000 (10:28 +0200)]
doc/man/cephadm: Add missing parameters for `ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
9 months agoMerge pull request #36755 from dillaman/wip-47050
Mykola Golub [Mon, 24 Aug 2020 08:08:57 +0000 (11:08 +0300)]
Merge pull request #36755 from dillaman/wip-47050

librbd: flush op was not properly waiting for queued IO

Reviewed-by: Mykola Golub <mgolub@suse.com>
9 months agodashboard: Fix building build shell script
Willem Jan Withagen [Sun, 23 Aug 2020 23:25:17 +0000 (01:25 +0200)]
dashboard: Fix building build shell script

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
9 months agoMerge pull request #36521 from tchaikov/wip-rgw-optional
Kefu Chai [Sat, 22 Aug 2020 18:02:23 +0000 (02:02 +0800)]
Merge pull request #36521 from tchaikov/wip-rgw-optional

rgw: hold reloader using unique_ptr

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
9 months agoMerge pull request #36707 from dillaman/wip-47007
Mykola Golub [Sat, 22 Aug 2020 10:12:05 +0000 (13:12 +0300)]
Merge pull request #36707 from dillaman/wip-47007

rbd-mirror: peer setup can still race and fail creation of peer

Reviewed-by: Mykola Golub <mgolub@suse.com>
9 months agoMerge pull request #36471 from dillaman/wip-41394
Mykola Golub [Sat, 22 Aug 2020 10:11:19 +0000 (13:11 +0300)]
Merge pull request #36471 from dillaman/wip-41394

librbd: using migration abort can result in the loss of data

Reviewed-by: Mykola Golub <mgolub@suse.com>
9 months agoMerge pull request #36380 from dillaman/wip-46754
Mykola Golub [Sat, 22 Aug 2020 10:10:17 +0000 (13:10 +0300)]
Merge pull request #36380 from dillaman/wip-46754

rbd: retrieve global config overrides from the MONs

Reviewed-by: Mykola Golub <mgolub@suse.com>
9 months agoMerge pull request #36751 from tchaikov/wip-redirect-stderr
Kefu Chai [Sat, 22 Aug 2020 08:59:14 +0000 (16:59 +0800)]
Merge pull request #36751 from tchaikov/wip-redirect-stderr

qa/tasks/ceph: create a log file before redirecting to it

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
9 months agoqa/workunits/rados: quote optional args
Kefu Chai [Sat, 22 Aug 2020 08:11:28 +0000 (16:11 +0800)]
qa/workunits/rados: quote optional args

otherwise they will be escaped by shell

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agoqa/tasks/workunit: allow passing optional args to workunit
Kefu Chai [Sat, 22 Aug 2020 05:59:19 +0000 (13:59 +0800)]
qa/tasks/workunit: allow passing optional args to workunit

* add comment to _run_tests()
* use `os.path.commonpath()` instead using string matching directly
  for matching given workunit spec with executables.
* allow passing optional args to workunit

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agoMerge pull request #36696 from tchaikov/wip-cmake-boost-zlib
Kefu Chai [Sat, 22 Aug 2020 04:49:15 +0000 (12:49 +0800)]
Merge pull request #36696 from tchaikov/wip-cmake-boost-zlib

cmake: link against ZLIB if statically linked against Boost

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
9 months agoMerge pull request #36710 from tchaikov/wip-ceph-pypath
Kefu Chai [Sat, 22 Aug 2020 04:48:34 +0000 (12:48 +0800)]
Merge pull request #36710 from tchaikov/wip-ceph-pypath

ceph: correct additional paths added to sys.path

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
9 months agoMerge pull request #36722 from tchaikov/wip-test-pybind
Kefu Chai [Sat, 22 Aug 2020 04:30:20 +0000 (12:30 +0800)]
Merge pull request #36722 from tchaikov/wip-test-pybind

test/pybind: label test_rados tests with attributes

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
9 months agoqa/tasks/ceph: create a log file before redirecting to it
Kefu Chai [Fri, 21 Aug 2020 12:22:23 +0000 (20:22 +0800)]
qa/tasks/ceph: create a log file before redirecting to it

as it is shell who interprets ">>" and redirect the stderr to given
file, but the shell process is launched by ubuntu:ububunt without using
sudo, so the command fails with "Permission denied" failure. to address
this issue, in this change, a file with proper priviledges is created
beforehand using `install`, so shell is able to write to it.

also, instead of creating this file in `maybe_redirect_stderr()`, it
returns the command to create the log file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 months agoMerge PR #36560 into master
Patrick Donnelly [Sat, 22 Aug 2020 01:44:46 +0000 (18:44 -0700)]
Merge PR #36560 into master

* refs/pull/36560/head:
client: choose a random replica mds to send the request

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agoMerge PR #36681 into master
Patrick Donnelly [Sat, 22 Aug 2020 01:44:06 +0000 (18:44 -0700)]
Merge PR #36681 into master

* refs/pull/36681/head:
mds: don't track change of config 'mds_replay_unsafe_with_closed_session'
mds: fix 'forward loop' when forward_all_requests_to_auth is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agoMerge PR #36673 into master
Patrick Donnelly [Sat, 22 Aug 2020 01:42:50 +0000 (18:42 -0700)]
Merge PR #36673 into master

* refs/pull/36673/head:
mds: don't recover files after normal session close

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
9 months agoMerge PR #36632 into master
Patrick Donnelly [Sat, 22 Aug 2020 01:41:35 +0000 (18:41 -0700)]
Merge PR #36632 into master

* refs/pull/36632/head:
mds: standy-replay mds remained in the "resolve" state after restart the active mds

Reviewed-by: Zheng Yan <zyan@redhat.com>