]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoclient: do not return EEXIST for mkdirs 29079/head
Patrick Donnelly [Tue, 16 Jul 2019 13:52:23 +0000 (06:52 -0700)]
client: do not return EEXIST for mkdirs

Behavior should be similar to `mkdir -p`.

Introduced-by: 26905ca82776e8119a347a9a2e8d65c272c7ed62
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b049eb9bbcdaef547c3559e9dbe5a3ca272f60ec)

6 years agopybind/mgr/volumes: print exceptions in purge thread
Patrick Donnelly [Tue, 16 Jul 2019 01:27:31 +0000 (18:27 -0700)]
pybind/mgr/volumes: print exceptions in purge thread

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ecf85023954f464c328671c897d610c8ebc8c789)

6 years agopybind/mgr/volumes: refactor trash readdir
Patrick Donnelly [Mon, 15 Jul 2019 23:56:32 +0000 (16:56 -0700)]
pybind/mgr/volumes: refactor trash readdir

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e71e9187ec69ce80e9d7a2fc2c58946dd3db88e1)

6 years agopybind/mgr/volumes: use existing client provided recursive mkdir
Patrick Donnelly [Mon, 15 Jul 2019 23:39:50 +0000 (16:39 -0700)]
pybind/mgr/volumes: use existing client provided recursive mkdir

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e9c314abd06e2d91a58cbd681978a3abb6134952)

6 years agopybind/mgr/volumes: cleanup fs removal
Patrick Donnelly [Mon, 15 Jul 2019 20:09:06 +0000 (13:09 -0700)]
pybind/mgr/volumes: cleanup fs removal

In Nautilus, the simplest sequence is:

    fs fail name
    fs rm name --yes-i-really-mean-it

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7cc4b72305a895e99ae79e6a4d09f45c23a6257c)

6 years agopybind/mgr/subvolumes: use bytes for paths
Patrick Donnelly [Wed, 10 Jul 2019 23:08:29 +0000 (16:08 -0700)]
pybind/mgr/subvolumes: use bytes for paths

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3d63cd947f55a8508f972ee249ea6a04836429ae)

6 years agopybind/mgr/volumes: remove unused property
Patrick Donnelly [Wed, 10 Jul 2019 23:10:45 +0000 (16:10 -0700)]
pybind/mgr/volumes: remove unused property

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f7987f5fcba322879046e86720f1db55d076d881)

6 years agotest: cleanup removing all subvolumes before removing subvolume group
Venky Shankar [Fri, 5 Jul 2019 15:09:18 +0000 (11:09 -0400)]
test: cleanup removing all subvolumes before removing subvolume group

Test `test_subvolume_create_with_desired_mode_in_group()` creates three
subvolume in a subvolume group. During cleanup, it only removed two of
the three subvolumes. This causes failure when removing the subvolume
group since it's not empty.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit ae470a64ecf8ded8bbe3dbe02b5a470fecb7e2c3)

6 years agomgr / volumes: wrap rmtree() call within try..except block
Venky Shankar [Fri, 5 Jul 2019 09:51:54 +0000 (05:51 -0400)]
mgr / volumes: wrap rmtree() call within try..except block

This will be invoked by purge threads, so be sure to catch all
exceptions and return proper error codes appropriately.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 04547c92c1424c3c17aa84c49bfbfe0ffe4ac781)

6 years agomgr / volumes: use negative error codes everywhere
Venky Shankar [Fri, 5 Jul 2019 09:50:42 +0000 (05:50 -0400)]
mgr / volumes: use negative error codes everywhere

cephfs python binding returns positive error code. mgr/volumes
incorrectly does error code checks assuming the error codes to
be negative.

this was not an issue till now since mgr/volumes mostly does a
`raise VolumeException()` for the most part followed by the exception
being displayed to the operator (one exception is catching cephfs
ObjectNotFound error, in which case -errno.ENOENT is returned
(and checked whereever required)).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit dd569e22e8d1b6287207fb2d1a4269d0bef20784)

6 years agotest: add basic purge queue validation test
Venky Shankar [Thu, 20 Jun 2019 10:00:40 +0000 (06:00 -0400)]
test: add basic purge queue validation test

.. and since we have async subvolume deletes now, check
trash directory for emptiness in other tests.

Fixes: http://tracker.ceph.com/issues/40036
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit aec1d90b3611fec19556c9e4c639cb8abf90e4b6)

6 years agomgr / volumes: schedule purge job for volumes on init
Venky Shankar [Tue, 18 Jun 2019 04:37:59 +0000 (00:37 -0400)]
mgr / volumes: schedule purge job for volumes on init

So as to kickstart leftover subvolume purges.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 06e44be9986e7598f557d9646d7f93c9db4bc89d)

6 years agomgr / volumes: purge queue for async subvolume delete
Venky Shankar [Mon, 17 Jun 2019 12:21:43 +0000 (08:21 -0400)]
mgr / volumes: purge queue for async subvolume delete

Support asynchronous subvolume deletes by handing off the delete
operation to a dedicated set of threads. A subvolume delete operation
renames the subvolume (subdirectory) to a unique trash path entry
and signals the set of worker threads to pick up entries from the
trash directory for background removal.

This commit implements a `thread pool` strategy as a class mixin.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 483a2141fe8c9a58bc25a544412cdf5b047ad772)

6 years agomgr / volumes: maintain connection pool for fs volumes
Venky Shankar [Mon, 17 Jun 2019 09:36:54 +0000 (05:36 -0400)]
mgr / volumes: maintain connection pool for fs volumes

Right now every [sub]volume call does a connect/disconnect to the
cephfs filesystem. This is unnecessary and can be optimized by
caching the filesystem handle in a connection pool and (re)using
the handle for subsequent [sub]volume operations.

This would be useful for implementing features such as purge queue
for asynchronous subvolume deletes.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5c41e949af9acabd612b0644de0603e374b4b42a)

6 years agomgr/volumes: do not import unused module
Kefu Chai [Thu, 4 Jul 2019 07:01:28 +0000 (15:01 +0800)]
mgr/volumes: do not import unused module

cephfs is not used in this module, so drop its `import`

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bc13b7941dc06072d103dd50e00f8089dfacfb13)

6 years agoMerge PR #28589 into nautilus
Sage Weil [Wed, 17 Jul 2019 01:16:03 +0000 (20:16 -0500)]
Merge PR #28589 into nautilus

* refs/pull/28589/head:
msg/async/ProtocolV[12]: add ms_learn_addr_from_peer

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #28646 into nautilus
Sage Weil [Wed, 17 Jul 2019 01:15:20 +0000 (20:15 -0500)]
Merge PR #28646 into nautilus

* refs/pull/28646/head:
os/bluestore: BitmapAllocator::allocate: log new extents only
os/bluestore: distinguis request and results in bitmap alloc log
os/bluestore: fix duplicate allocations in bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29032 into nautilus
Sage Weil [Wed, 17 Jul 2019 01:14:59 +0000 (20:14 -0500)]
Merge PR #29032 into nautilus

* refs/pull/29032/head:
qa/tasks/mgr/dashboard/test_health: fix test_full_health test
mon: use per-pool stats only when all OSDs are reporting
osd: report whether we have per-pool stats
osd/osd_types: osd_stat_t: include num_per_pool_osds member

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29028 from ceph/backport-nautilus-28060
Andrew Schoen [Tue, 16 Jul 2019 16:59:35 +0000 (11:59 -0500)]
Merge pull request #29028 from ceph/backport-nautilus-28060

nautilus: ceph-volume: use the Device.rotational property instead of sys_api

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoqa/tasks/mgr/dashboard/test_health: fix test_full_health test 29032/head
Sage Weil [Fri, 12 Jul 2019 13:15:15 +0000 (08:15 -0500)]
qa/tasks/mgr/dashboard/test_health: fix test_full_health test

New fields!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e0a66c564ee0ec72c20d891ca695d1df76f58692)

6 years agomon: use per-pool stats only when all OSDs are reporting
Sage Weil [Fri, 12 Jul 2019 22:25:49 +0000 (17:25 -0500)]
mon: use per-pool stats only when all OSDs are reporting

Previously, we would start using the per-pool stat sums as soon as *any*
OSDs were reporting.  For a legacy cluster, that meant that as soon as
one bluestore instance is updated or one new bluestore OSD is created, the
usage stats per pool would become useless.

Instead, only use the per-pool stats once *all* OSDs are reporting the new
values.  This mostly aligns with the health warning when one more bluestore
OSDs are not reporting; once they are update the warning goes away.  (That
does not factor in fielstore OSDs, though; all OSDs need to be new *and*
bluestore.)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5dcb6d81bbc2a4c4d0da4a33d9f6bbba5065a1ad)

# Conflicts:
# src/messages/MGetPoolStatsReply.h
- ctor arguments are all weird
# src/mon/PGMap.h
- lots of std:: everywhre
# src/osdc/Objecter.h
- std:: everywhere

6 years agoosd: report whether we have per-pool stats
Sage Weil [Wed, 10 Jul 2019 21:16:57 +0000 (16:16 -0500)]
osd: report whether we have per-pool stats

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5f99db7efaa3ec929ca6125709a7838fb1ca773a)

6 years agoosd/osd_types: osd_stat_t: include num_per_pool_osds member
Sage Weil [Wed, 10 Jul 2019 21:16:46 +0000 (16:16 -0500)]
osd/osd_types: osd_stat_t: include num_per_pool_osds member

Count OSDs that are reporting per-pool stats.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6a0dc2bd35448fcfe3dc67fd6fa0b3e6dbce622b)

6 years agoMerge pull request #29022 from smithfarm/wip-40762-nautilus
Yuri Weinstein [Fri, 12 Jul 2019 21:04:54 +0000 (14:04 -0700)]
Merge pull request #29022 from smithfarm/wip-40762-nautilus

nautilus: rgw: list bucket with start marker and delimiter will miss next object with char '0'

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoceph-volume: skip missing interpreters when running tox tests 29028/head
Andrew Schoen [Fri, 10 May 2019 16:04:35 +0000 (11:04 -0500)]
ceph-volume: skip missing interpreters when running tox tests

This will skip tests for versions of python that don't exist on the node
that's running the tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f1db0a864814d9b81192d89b6eebb6c624ddec3b)

6 years agoceph-volume: use the Device.rotational property instead of sys_api
Andrew Schoen [Fri, 10 May 2019 14:02:08 +0000 (09:02 -0500)]
ceph-volume: use the Device.rotational property instead of sys_api

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1666822

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 8ebff475dfa136c88101ef7364820fe3c5cee547)

6 years agorgw: always generate after delimiter char to skip directory 29022/head
Tianshan Qu [Tue, 11 Jun 2019 04:27:12 +0000 (12:27 +0800)]
rgw: always generate after delimiter char to skip directory

the following code also need after_delim_s to skip directory after list
1000 objects

Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 8c6678387b00c862d71cffd6fed6ca6c236e3fb7)

6 years agorgw: fix list bucket with start maker and delimiter '/' will miss next object with...
Tianshan Qu [Tue, 21 May 2019 10:50:06 +0000 (18:50 +0800)]
rgw: fix list bucket with start maker and delimiter '/' will miss next object with char '0'

'/' + 1 = '0', and the start marker will be skipped, so it will miss the
'0' object.

Fixes: http://tracker.ceph.com/issues/39989
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 43b5264c7d61ae3be56d68f131b544edee307a46)

Conflicts:
src/rgw/rgw_rados.cc
- nautilus lacks "unsigned" in the line "unsigned char e = delim.back();"

6 years agoMerge pull request #27723 from ceph/backport-nautilus-26957
Alfredo Deza [Wed, 10 Jul 2019 12:33:14 +0000 (08:33 -0400)]
Merge pull request #27723 from ceph/backport-nautilus-26957

nautilus: ceph-volume: look for rotational data in lsblk

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume: look for rotational data in lsblk 27723/head
Andrew Schoen [Thu, 14 Mar 2019 20:14:40 +0000 (15:14 -0500)]
ceph-volume: look for rotational data in lsblk

Also ask lsblk if a device is rotational if no information
is found in /sys/block, default to True if nothing is found.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1666822

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit de748f9aef894b247622ee5dc49c1fbc31097ad0)

6 years agoMerge pull request #28923 from ceph/backport-nautilus-28294
Alfredo Deza [Mon, 8 Jul 2019 17:27:16 +0000 (13:27 -0400)]
Merge pull request #28923 from ceph/backport-nautilus-28294

nautilus: tests: pass --ssh-config to pytest to resolve hosts when connecting

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #28925 from ceph/backport-nautilus-28866
Alfredo Deza [Mon, 8 Jul 2019 17:26:01 +0000 (13:26 -0400)]
Merge pull request #28925 from ceph/backport-nautilus-28866

nautilus: ceph-volume broken assertion errors after pytest changes

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #28924 from ceph/backport-nautilus-28836
Alfredo Deza [Mon, 8 Jul 2019 17:24:52 +0000 (13:24 -0400)]
Merge pull request #28924 from ceph/backport-nautilus-28836

nautilus: ceph-volume tests add a sleep in tox for slow OSDs after booting

6 years agoceph-volume api.lvm catch IndexError when parsing dmmapper output 28925/head
Alfredo Deza [Wed, 3 Jul 2019 18:59:18 +0000 (14:59 -0400)]
ceph-volume api.lvm catch IndexError when parsing dmmapper output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfff70d07a1ccf7fb11a96dd7c63335cfc9b5e26)

6 years agoceph-volume tests update to use error.value instead of str(error)
Alfredo Deza [Wed, 3 Jul 2019 18:58:31 +0000 (14:58 -0400)]
ceph-volume tests update to use error.value instead of str(error)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 484be7ffb1e0865ff840090a73036567a1f4f5d2)

6 years agoceph-volume tests add a sleep in tox for slow OSDs after booting 28924/head
Alfredo Deza [Tue, 2 Jul 2019 12:34:01 +0000 (08:34 -0400)]
ceph-volume tests add a sleep in tox for slow OSDs after booting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b398f99f810d97c94af6d943aed4c11a5fd69c19)

6 years agotests: pass --ssh-config to pytest to resolve hosts when connecting 28923/head
Alfredo Deza [Wed, 29 May 2019 12:56:37 +0000 (08:56 -0400)]
tests: pass --ssh-config to pytest to resolve hosts when connecting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ed708e520999c145c3cf8a0c51e4605f275a1771)

6 years agoMerge pull request #28922 from cbodley/wip-qa-rgw-swift-server-nautilus
Yuri Weinstein [Mon, 8 Jul 2019 15:03:48 +0000 (08:03 -0700)]
Merge pull request #28922 from cbodley/wip-qa-rgw-swift-server-nautilus

nautilus: qa/rgw: more fixes for swift task

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoqa/rgw: clean up arguments for swift task 28922/head
Casey Bodley [Mon, 1 Jul 2019 16:09:14 +0000 (12:09 -0400)]
qa/rgw: clean up arguments for swift task

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 89fa4e71de3cdbe13cd7a5bc6162babe3016cba4)

6 years agoqa/rgw: swift task filters out config for skipped clients
Casey Bodley [Mon, 1 Jul 2019 16:08:11 +0000 (12:08 -0400)]
qa/rgw: swift task filters out config for skipped clients

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ba3c17c4fad8672d95af0f3cb9e0fb4802d712e6)

6 years agoqa/rgw: swift task looks for rgw_server endpoint
Casey Bodley [Mon, 1 Jul 2019 16:07:16 +0000 (12:07 -0400)]
qa/rgw: swift task looks for rgw_server endpoint

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5e667c4f73b1f064cc324bcaff9fe35ca9a4abc6)

6 years agoMerge pull request #28872 from badone/wip-40670-nautilus
Yuri Weinstein [Fri, 5 Jul 2019 15:17:16 +0000 (08:17 -0700)]
Merge pull request #28872 from badone/wip-40670-nautilus

nautilus: qa/ceph-ansible: cephfs_pools variable pgs should be pg_num

6 years agoMerge pull request #28871 from badone/wip-40669-nautilus
Yuri Weinstein [Fri, 5 Jul 2019 15:16:58 +0000 (08:16 -0700)]
Merge pull request #28871 from badone/wip-40669-nautilus

nautilus: qa/ceph-ansible: ceph-ansible requires ansible 2.8

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoqa/ceph-ansible: Move to ansible 2.8 28871/head
Brad Hubbard [Mon, 1 Jul 2019 02:31:10 +0000 (12:31 +1000)]
qa/ceph-ansible: Move to ansible 2.8

Fixes: http://tracker.ceph.com/issues/40602
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit cb9bc299df3ff5d022ccdbab0e7421d561e02c7f)

6 years agoqa/ceph-ansible: Replace pgs with pg_num 28872/head
Brad Hubbard [Mon, 1 Jul 2019 06:10:54 +0000 (16:10 +1000)]
qa/ceph-ansible: Replace pgs with pg_num

ceph-ansible expects the variable pg_num, not pgs.

Fixes: http://tracker.ceph.com/issues/40605
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge PR #28767 into nautilus
Patrick Donnelly [Fri, 28 Jun 2019 06:09:33 +0000 (23:09 -0700)]
Merge PR #28767 into nautilus

* refs/pull/28767/head:
mgr/volumes: fix incorrect VolumeException arguments
mgr/volumes: allow setting mode
mgr/volumes: allow setting data pool layout

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomgr/volumes: fix incorrect VolumeException arguments 28767/head
Ramana Raja [Sun, 23 Jun 2019 16:23:16 +0000 (21:53 +0530)]
mgr/volumes: fix incorrect VolumeException arguments

Fixes: http://tracker.ceph.com/issues/40429
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 695c493257c7e9ae4c2229926393f6487a622616)

6 years agomgr/volumes: allow setting mode
Ramana Raja [Sun, 23 Jun 2019 12:09:36 +0000 (17:39 +0530)]
mgr/volumes: allow setting mode

... of fs subvolumes and subvolume groups during their creation.

Fixes: https://tracker.ceph.com/issues/40299
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 5c00cc30a1ad49b01e4ab84a17b62def88fc83c4)

6 years agomgr/volumes: allow setting data pool layout
Ramana Raja [Thu, 20 Jun 2019 12:15:14 +0000 (17:45 +0530)]
mgr/volumes: allow setting data pool layout

... of fs subvolumes and subvolume groups during their creation.

Fixes: https://tracker.ceph.com/issues/40431
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 22ed8915281033c39df5e6ecceb5eebdc4e1c594)

6 years agoMerge pull request #27043 from sebastian-philipp/wip-38808-nautilus
Yuri Weinstein [Tue, 25 Jun 2019 14:29:40 +0000 (07:29 -0700)]
Merge pull request #27043 from sebastian-philipp/wip-38808-nautilus

nautilus: mgr/orchestrator: Remove "(add|test|remove)_stateful_service_rule"

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #28444 from pdvian/wip-39735-nautilus
Yuri Weinstein [Tue, 25 Jun 2019 14:28:04 +0000 (07:28 -0700)]
Merge pull request #28444 from pdvian/wip-39735-nautilus

nautilus: rgw: multisite: mismatch of bucket creation times from List Buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28537 from smithfarm/wip-39740-nautilus
Yuri Weinstein [Tue, 25 Jun 2019 14:27:21 +0000 (07:27 -0700)]
Merge pull request #28537 from smithfarm/wip-39740-nautilus

nautilus: rgw: swift object expiry fails when a bucket reshards

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28538 from smithfarm/wip-39745-nautilus
Yuri Weinstein [Tue, 25 Jun 2019 14:26:11 +0000 (07:26 -0700)]
Merge pull request #28538 from smithfarm/wip-39745-nautilus

nautilus: rgw: fix default_placement containing "/" when storage_class is standard

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28723 from yuriw/wip-yuriw-40452-nautilus
Yuri Weinstein [Mon, 24 Jun 2019 18:58:35 +0000 (11:58 -0700)]
Merge pull request #28723 from yuriw/wip-yuriw-40452-nautilus

qa/tests: added `bluestore_warn_on_legacy_statfs: false` setting

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoqa/tests: added `bluestore_warn_on_legacy_statfs: false` setting 28723/head
Yuri Weinstein [Mon, 24 Jun 2019 15:11:49 +0000 (08:11 -0700)]
qa/tests: added `bluestore_warn_on_legacy_statfs: false` setting

Fixes: http://tracker.ceph.com/issues/40467
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28584 from rhcs-dashboard/nautilus-object-chart
Yuri Weinstein [Fri, 21 Jun 2019 22:35:52 +0000 (15:35 -0700)]
Merge pull request #28584 from rhcs-dashboard/nautilus-object-chart

nautilus: mgr/dashboard: show degraded/misplaced/unfound objects.

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
6 years agoMerge pull request #28635 from b-ranto/wip-nautilus-dashboard-push
Yuri Weinstein [Fri, 21 Jun 2019 22:35:13 +0000 (15:35 -0700)]
Merge pull request #28635 from b-ranto/wip-nautilus-dashboard-push

nautilus: mgr/dashboard: Push Grafana dashboards on startup

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #28698 from cbodley/wip-qa-rgw-swift-skip-client-nautilus
Yuri Weinstein [Fri, 21 Jun 2019 19:57:23 +0000 (12:57 -0700)]
Merge pull request #28698 from cbodley/wip-qa-rgw-swift-skip-client-nautilus

nautilus: qa/rgw: fix swift warning message

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28661 from yuriw/wip-yuriw-40452-nautilus
Yuri Weinstein [Fri, 21 Jun 2019 19:33:15 +0000 (12:33 -0700)]
Merge pull request #28661 from yuriw/wip-yuriw-40452-nautilus

qa/tests: added ragweed coverage to stress-split* upgrade suites

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoqa/rgw: fix swift warning message 28698/head
Casey Bodley [Fri, 21 Jun 2019 19:18:55 +0000 (15:18 -0400)]
qa/rgw: fix swift warning message

  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/swift.py", line 245, in task
    log.warning('Swift tests cannot run on rhel 7.6+, skipping client {client}'.format(client))
KeyError: 'client'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9b8be0584dee616af9df01b9622bf53dfde00fad)

6 years agoMerge pull request #28681 from batrick/shell-py3-fixes
Yuri Weinstein [Fri, 21 Jun 2019 18:43:50 +0000 (11:43 -0700)]
Merge pull request #28681 from batrick/shell-py3-fixes

nautilus: cephfs-shell: misc. cephfs-shell backports

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
6 years agocephfs-shell: use consistent naming for paths 28681/head
Patrick Donnelly [Wed, 19 Jun 2019 19:46:26 +0000 (12:46 -0700)]
cephfs-shell: use consistent naming for paths

And some small changes to make some commands take variable arguments and fix
defaults.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b2e8a02f5953fac5722a25f24783b33de502c013)

6 years agocephfs-shell: remove unnecessary instances of to_bytes
Patrick Donnelly [Wed, 19 Jun 2019 04:12:33 +0000 (21:12 -0700)]
cephfs-shell: remove unnecessary instances of to_bytes

cephfs binding already automatically converts paths and some arguments to
bytes.

Fixes: http://tracker.ceph.com/issues/40455
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ba9f0eefa5466fd0e4bcac03582e8a538a897c26)

6 years agocephfs-shell: refactor list_items
Patrick Donnelly [Wed, 19 Jun 2019 03:57:29 +0000 (20:57 -0700)]
cephfs-shell: refactor list_items

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2a32752981235989bd3ef7158664fb7e77d7f093)

6 years agopybind/cephfs: improve user-friendliness of DirResult
Patrick Donnelly [Wed, 19 Jun 2019 03:50:28 +0000 (20:50 -0700)]
pybind/cephfs: improve user-friendliness of DirResult

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit dff5b1a5d09bf8481cb4e2a0cd0bbbd44b8f20cf)

6 years agopybind/cephfs: avoid unicode check
Patrick Donnelly [Wed, 19 Jun 2019 03:38:26 +0000 (20:38 -0700)]
pybind/cephfs: avoid unicode check

py3 does not have the unicode built-in. Instead, simply try to do the encoding
and catch failures appropriately. If it quacks like a duck...

(Note that cython apparently cheats and allows the unicode check but this is
simpler.)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9bc49fefcd81fbb1c0ce62b1b81f3be420e81663)

6 years agocephfs-shell: Remove unnecessary empty line
Varsha Rao [Thu, 6 Jun 2019 15:47:19 +0000 (21:17 +0530)]
cephfs-shell: Remove unnecessary empty line

This patch fixes flake8 too many blank lines error.

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit d141adac9ae8b704884d160b5458a431321a0b2a)

6 years agocephfs-shell: Fix flake8 bare 'except' warnings
Varsha Rao [Thu, 6 Jun 2019 15:44:02 +0000 (21:14 +0530)]
cephfs-shell: Fix flake8 bare 'except' warnings

Instead of bare 'except', catch OSError and libcephfs exceptions.

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit d063e8b1e05a23f3f15fe2080ec57fe6242e64f4)

6 years agocephfs-shell: Remove unused variable 'e'
Varsha Rao [Thu, 6 Jun 2019 15:24:21 +0000 (20:54 +0530)]
cephfs-shell: Remove unused variable 'e'

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 6a0a5ed83d6d3e2bb9977bda2c773d5d5b6accf7)

6 years agocephfs-shell: Fix flake8 use != to compare str literals warning
Varsha Rao [Thu, 6 Jun 2019 15:21:31 +0000 (20:51 +0530)]
cephfs-shell: Fix flake8 use != to compare str literals warning

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit db7b9306fb94efbe0c12283c614554727fbf6f1e)

6 years agocephfs-shell: Fixe flake8 line too long error
Varsha Rao [Thu, 6 Jun 2019 15:08:15 +0000 (20:38 +0530)]
cephfs-shell: Fixe flake8 line too long error

Break the line into multiple lines and add new variable wherever necessary, to
fix the line too long error.

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit e41d467d4f482665fb09bb00edde7c03d6376b82)

6 years agocephfs-shell: Remove whitespace after '('
Varsha Rao [Mon, 13 May 2019 12:15:35 +0000 (17:45 +0530)]
cephfs-shell: Remove whitespace after '('

This patch fixes flake8 whitespace error.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 2136530dab1d8894c10ec63bb84752604cf6065f)

6 years agocephfs-shell: Remove unused 'flag' variable
Varsha Rao [Mon, 13 May 2019 12:11:06 +0000 (17:41 +0530)]
cephfs-shell: Remove unused 'flag' variable

This patch fixes flake8 unused local variable error.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 20c31ea2397545790154140822cc2668d85b913f)

6 years agocephfs-shell: Fix flake8 invalid escape sequence warning
Varsha Rao [Mon, 13 May 2019 12:00:01 +0000 (17:30 +0530)]
cephfs-shell: Fix flake8 invalid escape sequence warning

Backslash-character pair is not a valid escape sequence, since Python 3.6
version. Prefixing character 'r' to the escape sequence, fixes the warning.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit c7d217c2afcb9cdc64356f6a6b1f8f701c7814d9)

6 years agocephfs-shell: Remove unnecessary empty lines
Varsha Rao [Mon, 13 May 2019 11:52:07 +0000 (17:22 +0530)]
cephfs-shell: Remove unnecessary empty lines

This patch fixes flake8 too many blank lines error.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit f322affda07e41996189b43ec227856076d39d26)

6 years agocephfs-shell: Fix flake8 bare 'except' warnings
Varsha Rao [Mon, 13 May 2019 11:43:39 +0000 (17:13 +0530)]
cephfs-shell: Fix flake8 bare 'except' warnings

Instead of bare 'except', catch just libcephfs exceptions.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit e6b6cf4ba5814e80da22b3b95b3686831202e483)

6 years agoqa: elide python version config
Patrick Donnelly [Tue, 18 Jun 2019 22:14:22 +0000 (15:14 -0700)]
qa: elide python version config

This test doesn't actually use the config and cephfs-shell is py3 only.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ff1f04f4d538d3c713d34fa4a77c4ff3eb58e01d)

6 years agocephfs-shell: assert python3
Patrick Donnelly [Tue, 18 Jun 2019 22:11:58 +0000 (15:11 -0700)]
cephfs-shell: assert python3

There's little reason to support this new tool with python2.

Fixes: http://tracker.ceph.com/issues/40418
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit de429c48c45add1fcf16b1e019e9598af48bb8f5)

6 years agocephfs-shell: Catch OSError exceptions in lcd
Varsha Rao [Mon, 10 Jun 2019 11:12:58 +0000 (16:42 +0530)]
cephfs-shell: Catch OSError exceptions in lcd

This patch adds try-except statement to catch the OSError exceptions.

Fixes: http://tracker.ceph.com/issues/40243
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 4b958532e6c9648b73bdd78e0758ca6ddc4cf344)

6 years agocephfs-shell: Fix 'lls' command errors
Varsha Rao [Mon, 10 Jun 2019 13:33:53 +0000 (19:03 +0530)]
cephfs-shell: Fix 'lls' command errors

This patch fixes following:
* Not printing complete path when '..' is passed as argument.
* Printing of path for current working directory.
* No need to set default value for add_argument. As it returns empty list with
  nargs='*'.
* No need to check for absolute path. os.listdir takes both relative and
  absolute path.
* Use try-catch instead of if-else statement to catch exceptions.

Fixes: http://tracker.ceph.com/issues/40244
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit d1ddd4c50f56331c7036964f78d4e399e10431b8)

6 years agocephfs-shell: Add stat command
Varsha Rao [Wed, 24 Apr 2019 14:18:09 +0000 (19:48 +0530)]
cephfs-shell: Add stat command

This patch adds stat command to cephfs-shell.

Fixes: https://tracker.ceph.com/issues/38829
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 07263fe0f2dbefc936a987eb0379221a28eb42e7)

6 years agocephfs-shell: add quota management
Milind Changire [Thu, 9 May 2019 06:25:05 +0000 (11:55 +0530)]
cephfs-shell: add quota management

Quotas can be managed by:
$ quota get dir
$ quota set {--max_files MAX_FILES} {--max_bytes MAX_BYTES} dir

Fixes: http://tracker.ceph.com/issues/39165
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit d187e9b0a917b828530bda89866d6a4867e9f9ff)

6 years agocephfs-shell: fix string decoding for ls command
Milind Changire [Mon, 29 Apr 2019 10:59:27 +0000 (16:29 +0530)]
cephfs-shell: fix string decoding for ls command

* fix string decoding for ls command
* fix string decoding for is_dir_exists() and is_file_exists()
* fix minor flake8-3.7 warnings about indentation and line length

Fixes: http://tracker.ceph.com/issues/39404
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit b9445a496782db73a6ee21d95100dc16069dd986)

6 years agoqa/tests: added ragweed coverage to stress-split* upgrade suites 28661/head
Yuri Weinstein [Wed, 19 Jun 2019 19:31:44 +0000 (12:31 -0700)]
qa/tests: added ragweed coverage to stress-split* upgrade suites

added ceph config set global bluestore_warn_on_legacy_statfs fals
chnaged user "foo" to "ragweed" in ragweed.py

Fixes: http://tracker.ceph.com/issues/40452
Fices: http://tracker.ceph.com/issues/40467
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28580 from ukernel/nautilus-40344
Yuri Weinstein [Thu, 20 Jun 2019 16:39:06 +0000 (09:39 -0700)]
Merge pull request #28580 from ukernel/nautilus-40344

nautilus: mds: fix corner case of replaying open sessions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28582 from pdvian/wip-40040-nautilus
Yuri Weinstein [Thu, 20 Jun 2019 16:38:37 +0000 (09:38 -0700)]
Merge pull request #28582 from pdvian/wip-40040-nautilus

nautilus: mds: avoid sending too many osd requests at once after mds restarts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #28611 from batrick/i40223
Yuri Weinstein [Thu, 20 Jun 2019 16:37:54 +0000 (09:37 -0700)]
Merge pull request #28611 from batrick/i40223

nautilus: mds: reset heartbeat during long-running loops in recovery

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28618 from ukernel/nautilus-40236
Yuri Weinstein [Thu, 20 Jun 2019 16:37:19 +0000 (09:37 -0700)]
Merge pull request #28618 from ukernel/nautilus-40236

nautilus:  mds: fix 'is session in blacklist' check in Server::apply_blacklist()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28641 from batrick/i39260-2
Yuri Weinstein [Thu, 20 Jun 2019 16:36:59 +0000 (09:36 -0700)]
Merge pull request #28641 from batrick/i39260-2

nautilus: cephfs-shell: Revert "cephfs.pyx: add py3 compatibility"

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28555 from s0nea/wip-40059-nautilus
Yuri Weinstein [Thu, 20 Jun 2019 16:36:28 +0000 (09:36 -0700)]
Merge pull request #28555 from s0nea/wip-40059-nautilus

nautilus: mgr/dashboard: Add custom dialogue for configuring PG scrub parameters

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoMerge pull request #28552 from sebastian-philipp/wip-40037-nautilus
Yuri Weinstein [Wed, 19 Jun 2019 16:21:46 +0000 (09:21 -0700)]
Merge pull request #28552 from sebastian-philipp/wip-40037-nautilus

nautilus: dashboard: orchestrator mgr modules assert failure on iscsi service request

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #28516 from smithfarm/wip-40301-nautilus
Yuri Weinstein [Wed, 19 Jun 2019 16:09:58 +0000 (09:09 -0700)]
Merge pull request #28516 from smithfarm/wip-40301-nautilus

nautilus: build/ops: cmake: set empty-string RPATH for ceph-osd

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: BitmapAllocator::allocate: log new extents only 28646/head
Igor Fedotov [Tue, 11 Jun 2019 17:31:32 +0000 (20:31 +0300)]
os/bluestore: BitmapAllocator::allocate: log new extents only

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit a90d57ca05c902eab8ca19090b856ea0e08f5d2b)

6 years agoos/bluestore: distinguis request and results in bitmap alloc log
Igor Fedotov [Tue, 11 Jun 2019 17:23:08 +0000 (20:23 +0300)]
os/bluestore: distinguis request and results in bitmap alloc log

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 84c2d4148574373c678ad8f00e35f9e209ba1f18)

6 years agoos/bluestore: fix duplicate allocations in bmap allocator
Igor Fedotov [Tue, 11 Jun 2019 17:05:09 +0000 (20:05 +0300)]
os/bluestore: fix duplicate allocations in bmap allocator

Fixes: http://tracker.ceph.com/issues/40080
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit eac81bc85970287fce60178e80c37628b3df0e7d)

6 years agocephfs-shell: Revert "cephfs.pyx: add py3 compatibility" 28641/head
Varsha Rao [Thu, 11 Apr 2019 07:03:12 +0000 (12:33 +0530)]
cephfs-shell: Revert "cephfs.pyx: add py3 compatibility"

This reverts commit 5106582fc7edae7f39161cf89e566c020fcfa0ce.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 417836de308a8b642fec5f03d819800142155b34)

This revert was missing from #27531

6 years agoMerge pull request #28233 from pdvian/wip-39670-nautilus
Yuri Weinstein [Wed, 19 Jun 2019 00:42:53 +0000 (17:42 -0700)]
Merge pull request #28233 from pdvian/wip-39670-nautilus

nautilus: mds: output lock state in format dump

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28595 from ajarr/wip-nautilus-refactor-volume-module
Yuri Weinstein [Wed, 19 Jun 2019 00:42:31 +0000 (17:42 -0700)]
Merge pull request #28595 from ajarr/wip-nautilus-refactor-volume-module

nautilus: mgr/volumes: refactor volume module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28610 from batrick/i40164
Yuri Weinstein [Wed, 19 Jun 2019 00:42:03 +0000 (17:42 -0700)]
Merge pull request #28610 from batrick/i40164

nautilus: cephfs: mount: key parsing fail when doing a remount

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28612 from batrick/i40161
Yuri Weinstein [Wed, 19 Jun 2019 00:41:12 +0000 (17:41 -0700)]
Merge pull request #28612 from batrick/i40161

nautilus: cephfs: FSAL_CEPH assertion failed in Client::_lookup_name: "parent->is_dir()"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>