]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoceph-volume: remove stderr in has_bluestore_label() 33239/head
Guillaume Abrioux [Wed, 5 Feb 2020 01:15:17 +0000 (02:15 +0100)]
ceph-volume: remove stderr in has_bluestore_label()

We don't want to generate this log when a call to
`has_bluestore_label()` fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7f8371c32b3f613b0d094f6f26ffbeb19ea0d25b)

5 years agoceph-volume: fix has_bluestore_label() function
Guillaume Abrioux [Tue, 4 Feb 2020 21:02:26 +0000 (22:02 +0100)]
ceph-volume: fix has_bluestore_label() function

When using vg/lv, this function throws an error like following:

```
 stderr: unable to read label for test_group/data-lv2: (2) No such file or directory
 stderr: 2020-02-04T21:03:32.153+0000 7fe091af4200 -1 bluestore(test_group/data-lv2) _read_bdev_label failed to open test_group/data-lv2: (2) No such file or directory
```

using `self.abspath` fixes this error.

Fixes: https://tracker.ceph.com/issues/43970
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 148069a20fef79ef8fe510f06879a0de02987eac)

5 years agoMerge pull request #33238 from jan--f/wip-31700-notracker-nautilus
Jan Fajerski [Wed, 12 Feb 2020 12:13:54 +0000 (13:13 +0100)]
Merge pull request #33238 from jan--f/wip-31700-notracker-nautilus

nautilus: ceph-volume: refactor listing.py + fixes

5 years agoceph-volume: fix various lvm list issues 33238/head
Jan Fajerski [Thu, 6 Feb 2020 15:49:12 +0000 (16:49 +0100)]
ceph-volume: fix various lvm list issues

A single report on a non-lvm device now works.
Format was cleaned up, report lvm journal,wal, db only once.

Fixes: https://tracker.ceph.com/issues/44009
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 000bf2ffff57701952e2aa1a67a04e519c4d07a6)

5 years agoceph-volume: add get_device_lvs to easily retrieve all lvs per device
Jan Fajerski [Thu, 6 Feb 2020 15:47:08 +0000 (16:47 +0100)]
ceph-volume: add get_device_lvs to easily retrieve all lvs per device

Also drop the sep argument from get_lvs and siblings, unused.
Introduce LV_CMD_OPTIONS to unify options to lvs.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ffe5b5732a46bd5ff888696adbfe53a38c982448)

5 years agoceph-volume: fix lvm list
Guillaume Abrioux [Wed, 5 Feb 2020 01:29:14 +0000 (02:29 +0100)]
ceph-volume: fix lvm list

17957d9beb42a04b8f180ccb7ba07d43179a41d3 introduced a regression in `lvm
list`.

When passing a vg/lv path for generating a single report, it fails
because the filter used in the `lvs` command isn't right. It uses the lv
name instead of the vg name because `os.path.basename(device)` is used
while it should be `os.path.dirname(device)`

Fixes: https://tracker.ceph.com/issues/43969
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0179fed3ab624830ba77349531763c3e116c82e5)

5 years agoceph-volume: delete test_lvs_list_is_created_just_once
Rishabh Dave [Fri, 6 Dec 2019 07:40:35 +0000 (13:10 +0530)]
ceph-volume: delete test_lvs_list_is_created_just_once

lisitng.py doesn't call api.Volumes anymore. Therefore, this test is
redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 665ed2424b7bef4160289f0135acc015f8ea9980)

5 years agoceph-volume: update tests since listing.py got heavily modified
Rishabh Dave [Wed, 4 Dec 2019 07:28:19 +0000 (12:58 +0530)]
ceph-volume: update tests since listing.py got heavily modified

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d1ae6d1a8b495adfc0c512f08359e0db1590272d)

5 years agoceph-volume: refactor devices/lvm/listing.py
Rishabh Dave [Thu, 21 Nov 2019 12:34:25 +0000 (18:04 +0530)]
ceph-volume: refactor devices/lvm/listing.py

Get rid of duplicate and redundant code and use get_lvs, get_vgs and
get_pvs to simplify the module as much as possible.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d02bd7dd581a4bd4041eb397fae540a18f16a88b)

5 years agoceph-volume: add new method in api/lvm.py
Rishabh Dave [Thu, 23 Jan 2020 14:17:21 +0000 (19:47 +0530)]
ceph-volume: add new method in api/lvm.py

The method determines whether given LV is managed by Ceph or not.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 876244b6ab2cf1fbd724fd33966501a3366c6d3f)

5 years agoceph-volume: add helper methods to get only first LVM devs
Rishabh Dave [Fri, 3 Jan 2020 10:14:04 +0000 (15:44 +0530)]
ceph-volume: add helper methods to get only first LVM devs

These convenience methods shortens following phrase to
"lv = get_first_lv()" -

lvs = get_lvs()
if len(lvs) >= 1:
lvs = lv[0]

These methods do the same things as above phrase internall. Rewrite
listing.py to use these new helper methods.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 17957d9beb42a04b8f180ccb7ba07d43179a41d3)

5 years agoceph-volume: filter based on tags for api.lvm.get_* methods
Rishabh Dave [Mon, 30 Dec 2019 07:10:49 +0000 (12:40 +0530)]
ceph-volume: filter based on tags for api.lvm.get_* methods

get_pvs, get_vgs and get_lvs must accept tags and filter volumes based
on tags.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit fb1390964fdfd10815ab4a4302ad454169bc0d5f)

5 years agoMerge pull request #33232 from jan--f/wip-43871-nautilus
Jan Fajerski [Wed, 12 Feb 2020 11:28:39 +0000 (12:28 +0100)]
Merge pull request #33232 from jan--f/wip-43871-nautilus

nautilus: ceph-volume: batch bluestore fix create_lvs call

5 years agoMerge pull request #33231 from jan--f/wip-43849-nautilus
Jan Fajerski [Wed, 12 Feb 2020 11:28:22 +0000 (12:28 +0100)]
Merge pull request #33231 from jan--f/wip-43849-nautilus

nautilus: ceph-volume: add sizing arguments to prepare

5 years agoMerge pull request #33116 from batrick/i43137
Ramana Raja [Wed, 12 Feb 2020 10:08:08 +0000 (15:38 +0530)]
Merge pull request #33116 from batrick/i43137

nautilus: pybind/mgr/volumes: idle connection drop is not working

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoceph-volume: batch bluestore fix create_lvs call 33232/head
Jan Fajerski [Tue, 28 Jan 2020 08:25:39 +0000 (09:25 +0100)]
ceph-volume: batch bluestore fix create_lvs call

Fixes: https://tracker.ceph.com/issues/43844
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit df18497bc9eaf1922e5c885e8cc124e439c59364)

5 years agoceph-volume: remove redefinition of [LV,PV,VG]_FIELDS 33231/head
Jan Fajerski [Tue, 7 Jan 2020 14:29:24 +0000 (15:29 +0100)]
ceph-volume: remove redefinition of [LV,PV,VG]_FIELDS

This was introduced in https://github.com/ceph/ceph/pull/32242
erroneously.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 6f107f7de9afc4fb200063410ec347b437d5b60a)

5 years agotests: fix tests after batch sizing was fixed
Jan Fajerski [Thu, 12 Dec 2019 12:45:44 +0000 (13:45 +0100)]
tests: fix tests after batch sizing was fixed

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 405112970ec62fca345d10040c1648b7e629d11a)

5 years agolvm/batch: adjust devices for byte based size calculation
Jan Fajerski [Thu, 12 Dec 2019 12:44:07 +0000 (13:44 +0100)]
lvm/batch: adjust devices for byte based size calculation

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 9e61b80d2ce1d9ff5139dace00807f5a0cceb125)

5 years agolvm: add sizing arguments to prepare and create.
Jan Fajerski [Tue, 10 Dec 2019 13:56:37 +0000 (14:56 +0100)]
lvm: add sizing arguments to prepare and create.

This adds options to size to-be-created LVs in the prepare and create
subcommands. Sizing can be done explicitly by passing a sizes or
implicitly by specifying the number of slots per [data|journal|wal|db]
device. The former will try to create a LV of the specified size and use
that to create OSDs if it succeeds. The latter will carve up the device
size into $n slots and use one of those slots for the to-be-created OSD.
If partitions or LVs are passed these options are ignored.
This also creates the foundation to move to byte-based sizing, by moving
VolumeGroup lvm querying and size calculation to bytes as the base unit.

Fixes: https://tracker.ceph.com/issues/43299
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8b8913ad3c0b8ceae9f458fd8d3ec292c5ff5eb1)

5 years agoutil/disk: extend Size class
Jan Fajerski [Tue, 10 Dec 2019 13:57:02 +0000 (14:57 +0100)]
util/disk: extend Size class

The Size class can now parse strings and has support for arithmetic
operations and comparisons with numbers.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit dd89f467a0b89a436371c314bc69cf71a6e9ce72)

5 years agoMerge pull request #33220 from yuriw/wip-yuriw-clients-upgrades-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 21:41:31 +0000 (13:41 -0800)]
Merge pull request #33220 from yuriw/wip-yuriw-clients-upgrades-nautilus

qa/tests: added client-upgrade-nautilus suite to be used on octopus …

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoqa/tests: added client-upgrade-nautilus suite to be used on octopus release 33220/head
Yuri Weinstein [Tue, 11 Feb 2020 19:45:06 +0000 (11:45 -0800)]
qa/tests: added client-upgrade-nautilus suite to be used on octopus release

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #32863 from shyukri/wip-43341-nautilus
Jan Fajerski [Tue, 11 Feb 2020 19:39:08 +0000 (20:39 +0100)]
Merge pull request #32863 from shyukri/wip-43341-nautilus

nautilus: ceph-volume: lvm/deactivate: add unit tests, remove --all

5 years agoMerge pull request #33217 from jan--f/wip-32242-notrack-nautilus
Jan Fajerski [Tue, 11 Feb 2020 18:11:58 +0000 (19:11 +0100)]
Merge pull request #33217 from jan--f/wip-32242-notrack-nautilus

nautilus: ceph-volume: add methods to pass filters to pvs, vgs and lvs commands

5 years agoceph-volume: add methods to pass filters to pvs, vgs and lvs commands 33217/head
Rishabh Dave [Thu, 21 Nov 2019 14:33:32 +0000 (20:03 +0530)]
ceph-volume: add methods to pass filters to pvs, vgs and lvs commands

Filters can be passed to these commands by using option '-S'.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit a4f2fcefb80df524c472782e8c1d18b3c2d1c1fc)

5 years agoMerge pull request #32912 from smithfarm/wip-43509-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 16:41:52 +0000 (08:41 -0800)]
Merge pull request #32912 from smithfarm/wip-43509-nautilus

nautilus: mon: print FSMap regardless of file system count

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agolvm/deactivate: add unit tests, remove --all 32863/head
Jan Fajerski [Mon, 16 Dec 2019 15:57:58 +0000 (16:57 +0100)]
lvm/deactivate: add unit tests, remove --all

Remove the --all flag until its actually implemented.

Fixes: https://tracker.ceph.com/issues/43330
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c13901f963d515fd34bd0ff9df67bcf214507b70)

Conflicts:
src/ceph-volume/ceph_volume/devices/lvm/deactivate.py

5 years agoMerge pull request #33209 from jan--f/wip-deactivate-nautilus
Jan Fajerski [Tue, 11 Feb 2020 16:14:12 +0000 (17:14 +0100)]
Merge pull request #33209 from jan--f/wip-deactivate-nautilus

nautilus: ceph-volume: lvm deactivate command

5 years agoMerge pull request #32915 from smithfarm/wip-43628-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 16:03:05 +0000 (08:03 -0800)]
Merge pull request #32915 from smithfarm/wip-43628-nautilus

nautilus: client: disallow changing fuse_default_permissions option at runtime

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #32914 from smithfarm/wip-43624-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 16:02:21 +0000 (08:02 -0800)]
Merge pull request #32914 from smithfarm/wip-43624-nautilus

nautilus: mds: note client features when rejecting client

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32913 from smithfarm/wip-43573-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 16:01:45 +0000 (08:01 -0800)]
Merge pull request #32913 from smithfarm/wip-43573-nautilus

nautilus: cephfs-journal-tool: fix crash and usage

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32909 from smithfarm/wip-43343-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 16:00:25 +0000 (08:00 -0800)]
Merge pull request #32909 from smithfarm/wip-43343-nautilus

nautilus: mds: fix revoking caps after after stale->resume circle

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32602 from batrick/i43558
Yuri Weinstein [Tue, 11 Feb 2020 15:59:47 +0000 (07:59 -0800)]
Merge pull request #32602 from batrick/i43558

nautilus: mds: reject forward scrubs when cluster has multiple active MDS (more than one rank)

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32600 from batrick/i43506
Yuri Weinstein [Tue, 11 Feb 2020 15:59:19 +0000 (07:59 -0800)]
Merge pull request #32600 from batrick/i43506

nautilus: MDSMonitor: warn if a new file system is being created with an EC default data pool

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #30843 from smithfarm/wip-41853-nautilus
Yuri Weinstein [Tue, 11 Feb 2020 15:58:33 +0000 (07:58 -0800)]
Merge pull request #30843 from smithfarm/wip-41853-nautilus

nautilus: mds: reject sessionless messages

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agolvm: add deactivate subcommand 33209/head
Jan Fajerski [Wed, 11 Dec 2019 13:54:16 +0000 (14:54 +0100)]
lvm: add deactivate subcommand

This new subcommand unmounts and OSDs tmpfs mount and closes crypt
devices if there are any.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 9797f6b83d727933b1a0daab239f3034675a1d0e)

5 years agoutil/system: add unmount_tmpfs helper
Jan Fajerski [Thu, 12 Dec 2019 08:28:27 +0000 (09:28 +0100)]
util/system: add unmount_tmpfs helper

Thsi unmounts a path if and only if it's a tmpfs mount.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 705ed1110138b1e7c77768f97d5acda4a76d868f)

5 years agoapi/lvm: add get_lv_by_osd_id method
Jan Fajerski [Wed, 11 Dec 2019 12:17:19 +0000 (13:17 +0100)]
api/lvm: add get_lv_by_osd_id method

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2558b55b0ecbcc83acccbf8412c19f01e8dafb3e)

5 years agoapi/lvm: add deactivate method to Volume class
Jan Fajerski [Wed, 11 Dec 2019 12:17:02 +0000 (13:17 +0100)]
api/lvm: add deactivate method to Volume class

Simply calls lvchange -an to deactivate a logical volume.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8087600d4853554794b709291a08d97befd07ed1)

5 years agoMerge pull request #32868 from shyukri/wip-42945-nautilus
Jan Fajerski [Tue, 11 Feb 2020 13:23:27 +0000 (14:23 +0100)]
Merge pull request #32868 from shyukri/wip-42945-nautilus

nautilus: ceph-volume: allow raw block devices everywhere

5 years agodoc: update ceph-volume lvm prepare 32868/head
Jan Fajerski [Tue, 5 Nov 2019 12:39:27 +0000 (13:39 +0100)]
doc: update ceph-volume lvm prepare

Add option to pass raw physical devices everywhere, restructure a little
(bluestore section before filestore) and reword a few things.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f2018d76eb8936aed8643328f0d5f111c87514e4)

5 years agoceph-volume: make lvm report fields into constants
Jan Fajerski [Tue, 5 Nov 2019 07:31:44 +0000 (08:31 +0100)]
ceph-volume: make lvm report fields into constants

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 01a603f6e952d2c5931b89699a86503e281fb376)

5 years agoceph-volume: api/lvm create or reuse a vg
Jan Fajerski [Tue, 22 Oct 2019 12:32:11 +0000 (14:32 +0200)]
ceph-volume: api/lvm create or reuse a vg

This changes create_lv so one can pass the desired device and either a
VG with a name starting with ceph is re-used or a new one is created.
This commit also adds two new lvm primitives, making use of lvm's select
feature. The goal is to eventually avoid keeping a full list of lv's (or
vg's) around and query the lvm system as needed.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit bb4de1a3fc238eaf9f717dc59c6bdf338ef6d657)

5 years agoMerge pull request #33202 from jan--f/wip-43853-nautilus
Jan Fajerski [Tue, 11 Feb 2020 12:08:01 +0000 (13:08 +0100)]
Merge pull request #33202 from jan--f/wip-43853-nautilus

nautilus: ceph-volume/batch: fail on filtered devices when non-interactive

5 years agoMerge pull request #33200 from jan--f/wip-42898-nautilus
Jan Fajerski [Tue, 11 Feb 2020 12:03:59 +0000 (13:03 +0100)]
Merge pull request #33200 from jan--f/wip-42898-nautilus

nautilus: ceph-volume: make get_devices fs location independent

5 years agoMerge pull request #32870 from shyukri/wip-43117-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:34:53 +0000 (12:34 +0100)]
Merge pull request #32870 from shyukri/wip-43117-nautilus

nautilus: ceph-volume: import mock.mock instead of unittest.mock (py2)

5 years agoMerge pull request #32877 from shyukri/wip-43570-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:32:01 +0000 (12:32 +0100)]
Merge pull request #32877 from shyukri/wip-43570-nautilus

nautilus: ceph-volume: Dereference symlink in lvm list

5 years agoMerge pull request #32873 from shyukri/wip-43201-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:31:08 +0000 (12:31 +0100)]
Merge pull request #32873 from shyukri/wip-43201-nautilus

nautilus: ceph-volume: fix the integer overflow

5 years agoMerge pull request #32874 from shyukri/wip-43321-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:19:50 +0000 (12:19 +0100)]
Merge pull request #32874 from shyukri/wip-43321-nautilus

nautilus: ceph-volume: use correct extents if using db-devices and >1 osds_per_device

5 years agoMerge pull request #32864 from shyukri/wip-43462-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:09:23 +0000 (12:09 +0100)]
Merge pull request #32864 from shyukri/wip-43462-nautilus

nautilus: ceph-volume/lvm/activate.py: clarify error message: fsid refers to osd_fsid

5 years agoMerge pull request #32860 from shyukri/wip-43281-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:08:22 +0000 (12:08 +0100)]
Merge pull request #32860 from shyukri/wip-43281-nautilus

nautilus: ceph-volume: util: look for executable in $PATH

5 years agoMerge pull request #31616 from jan--f/wip-42800-nautilus
Jan Fajerski [Tue, 11 Feb 2020 11:07:20 +0000 (12:07 +0100)]
Merge pull request #31616 from jan--f/wip-42800-nautilus

nautilus: ceph-volume: assume msgrV1 for all branches containing mimic

5 years agoceph-volume/batch: fail on filtered devices when non-interactive 33202/head
Jan Fajerski [Tue, 3 Dec 2019 12:44:00 +0000 (13:44 +0100)]
ceph-volume/batch: fail on filtered devices when non-interactive

When batch is called non-interactively and a user explicitly specifies,
say a db-device, this will be filtered when unavailable. This can cause
the resulting OSD to be very different from the users intention
(standalone vs external db when the db-device was filtered). If devices
get filtered in non-interactive mode, ceph-volume should fail.

Fixes: https://tracker.ceph.com/issues/43105
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2e985053deec6c4cf60c0b85aec3df16cd77ceeb)

5 years agoceph-volume: refactor tests for refactored get_devices 33200/head
Jan Fajerski [Mon, 11 Nov 2019 13:35:43 +0000 (14:35 +0100)]
ceph-volume: refactor tests for refactored get_devices

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 4749f4c59dc4edb1fd689e18e8d9f358f95cb945)

Conflicts:
src/ceph-volume/ceph_volume/tests/conftest.py
        resolved by importing PropertyMock

5 years agoceph-volume: refactor get_devices, don't use os.path.realpath
Jan Fajerski [Fri, 8 Nov 2019 15:36:30 +0000 (16:36 +0100)]
ceph-volume: refactor get_devices, don't use os.path.realpath

Fixes: https://tracker.ceph.com/issues/42777
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b35e8c485f73b5b65e0be29444eb06172d6df3e3)

5 years agoMerge pull request #32558 from shyukri/wip-43275-nautilus
Jan Fajerski [Tue, 11 Feb 2020 10:40:48 +0000 (11:40 +0100)]
Merge pull request #32558 from shyukri/wip-43275-nautilus

nautilus: ceph-volume/test: patch VolumeGroups

5 years agoMerge pull request #32556 from shyukri/wip-43022-nautilus
Jan Fajerski [Tue, 11 Feb 2020 10:39:40 +0000 (11:39 +0100)]
Merge pull request #32556 from shyukri/wip-43022-nautilus

nautilus: ceph-volume: minor clean-up of "simple scan" subcommand help

5 years agoMerge pull request #32998 from neha-ojha/wip-min-alloc-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 21:46:00 +0000 (13:46 -0800)]
Merge pull request #32998 from neha-ojha/wip-min-alloc-nautilus

nautilus: common/options: bluestore 4k min_alloc_size for SSD

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge pull request #32846 from smithfarm/wip-43256-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 21:37:08 +0000 (13:37 -0800)]
Merge pull request #32846 from smithfarm/wip-43256-nautilus

nautilus: common/config: update values when they are removed via mon

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32997 from neha-ojha/wip-32939-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 20:46:12 +0000 (12:46 -0800)]
Merge pull request #32997 from neha-ojha/wip-32939-nautilus

nautilus: mon/MgrMonitor.cc: add always_on_modules to the output of "ceph mgr module ls"

Reviewed-by: David Zafman <dzafman@redhat.com>
5 years agoMerge pull request #32901 from smithfarm/wip-43631-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 20:44:42 +0000 (12:44 -0800)]
Merge pull request #32901 from smithfarm/wip-43631-nautilus

nautilus: common/util: use ifstream to read from /proc files

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #32858 from smithfarm/wip-43473-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 20:43:33 +0000 (12:43 -0800)]
Merge pull request #32858 from smithfarm/wip-43473-nautilus

nautilus: common: fix deadlocky inflight op visiting in OpTracker.

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #32845 from smithfarm/wip-43245-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 20:41:21 +0000 (12:41 -0800)]
Merge pull request #32845 from smithfarm/wip-43245-nautilus

nautilus: os/bluestore/BlueStore.cc: set priorities for compression stats

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge pull request #32774 from dzafman/wip-43726-nautilus
Yuri Weinstein [Mon, 10 Feb 2020 20:38:35 +0000 (12:38 -0800)]
Merge pull request #32774 from dzafman/wip-43726-nautilus

nautilus: test: Fix wait_for_state() to wait for a PG to get into a state

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoceph-volume: Dereference symlink in lvm list 32877/head
Benoît Knecht [Thu, 2 Jan 2020 07:07:21 +0000 (08:07 +0100)]
ceph-volume: Dereference symlink in lvm list

This allows for a symlink to be passed to

```
ceph-volume lvm list <path>
```

which makes it possible to use `/dev/disk/by-path/*` devices, for
instance.

Fixes: https://tracker.ceph.com/issues/43497
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 09fa3df8a39d361f2af11bded8aab9e0da334d51)

5 years agoceph-volume: use correct extents when using db-devices and >1 osds_per_device 32874/head
Fabian Niepelt [Wed, 11 Dec 2019 13:19:14 +0000 (14:19 +0100)]
ceph-volume: use correct extents when using db-devices and >1 osds_per_device

Actual data size depending on osds_per_device needs to be calculated here. Otherwise, if osds_per_device is greater than 1, ceph-volume will allocate 100% of the device to the first osd and then fail to create the LV for the second because the volume group is already full.

Fixes: https://tracker.ceph.com/issues/39442
Signed-off-by: Fabian Niepelt <f.niepelt@mittwald.de>
(cherry picked from commit ecde6cd619605003706f545b0b5ad4e2e8932f28)

 Conflicts:
src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py

I've removed `data_uuid` since it's not in nautilus already

5 years agoceph-volume: fix the type mismatch, covert the tries and interval to int from string. 32873/head
taodd [Sun, 8 Dec 2019 02:10:59 +0000 (10:10 +0800)]
ceph-volume: fix the type mismatch, covert the tries and interval to int from string.
Fixes: https://tracker.ceph.com/issues/43186
Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
(cherry picked from commit 81ff4be4d13ce034d7a8b921113138a942e3051a)

5 years agoceph-volume: import mock.mock instead of unittest.mock (py2) 32870/head
Jan Fajerski [Fri, 22 Nov 2019 15:57:11 +0000 (16:57 +0100)]
ceph-volume: import mock.mock instead of unittest.mock (py2)

Fixes: bb4de1a3fc238eaf9f717dc59c6bdf338ef6d657
Fixes: https://tracker.ceph.com/issues/42970
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c1bd09f2591d983f1988e3dadcf3c65c59afb15d)

Conflicts:
src/ceph-volume/ceph_volume/tests/api/test_lvm.py

5 years agolvm/activate.py: clarify error message: fsid refers to osd_fsid 32864/head
Yaniv Kaul [Thu, 19 Dec 2019 14:54:58 +0000 (16:54 +0200)]
lvm/activate.py: clarify error message: fsid refers to osd_fsid

Users complained[1] the error message isn't clear, and they thought
it referred to the cluster fsid instead of the osd_fsid.
Made it clearer.

[1] rook/rook#4547

Fixes: https://tracker.ceph.com/issues/43442
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
(cherry picked from commit ff3ba9200a6b5856fbdf4372fe292a7b94dd2539)

5 years agoceph-volume: util: look for executable in $PATH 32860/head
Shyukri Shyukriev [Thu, 21 Nov 2019 14:04:02 +0000 (15:04 +0100)]
ceph-volume: util: look for executable in $PATH

Fixes: https://tracker.ceph.com/issues/36728
Fallback to predefined paths for backward compatibility.
Alter test involved for partial match in warning

Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit a8577085dc52b0f214d9568c29a9605d1a826a45)

5 years agoMerge pull request #30689 from smithfarm/wip-42120-nautilus
Yuri Weinstein [Sat, 8 Feb 2020 00:27:15 +0000 (16:27 -0800)]
Merge pull request #30689 from smithfarm/wip-42120-nautilus

nautilus: core: osd/OSDMap: health alert for non-power-of-two pg_num

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32857 from smithfarm/wip-43471-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:31:36 +0000 (14:31 -0800)]
Merge pull request #32857 from smithfarm/wip-43471-nautilus

nautilus: osd/PeeringState.cc: don't let num_objects become negative

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32848 from smithfarm/wip-43346-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:31:01 +0000 (14:31 -0800)]
Merge pull request #32848 from smithfarm/wip-43346-nautilus

nautilus: qa/suites/rados/thrash: force normal pg log length with cache tiering

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32847 from smithfarm/wip-43319-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:30:26 +0000 (14:30 -0800)]
Merge pull request #32847 from smithfarm/wip-43319-nautilus

nautilus: osd/PeeringState.cc: skip peer_purged when discovering all missing

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32843 from smithfarm/wip-43099-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:29:48 +0000 (14:29 -0800)]
Merge pull request #32843 from smithfarm/wip-43099-nautilus

nautilus: osd/OSD: enhance osd numa affinity compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32773 from dzafman/wip-43246-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:29:08 +0000 (14:29 -0800)]
Merge pull request #32773 from dzafman/wip-43246-nautilus

nautilus: osd: Use physical ratio for nearfull (doesn't include backfill resserve)

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge branch 'nautilus' into wip-42120-nautilus 30689/head
David Zafman [Fri, 7 Feb 2020 22:27:18 +0000 (14:27 -0800)]
Merge branch 'nautilus' into wip-42120-nautilus

5 years agoMerge pull request #32716 from smithfarm/wip-43650-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:11:40 +0000 (14:11 -0800)]
Merge pull request #32716 from smithfarm/wip-43650-nautilus

nautilus: osd: Diagnostic logging for upmap cleaning

Reviewed-by: David Zafman <dzafman@redhat.com>
5 years agoMerge pull request #32715 from smithfarm/wip-43620-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:09:27 +0000 (14:09 -0800)]
Merge pull request #32715 from smithfarm/wip-43620-nautilus

nautilus: kv: fix shutdown vs async compaction

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32259 from smithfarm/wip-43243-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:08:44 +0000 (14:08 -0800)]
Merge pull request #32259 from smithfarm/wip-43243-nautilus

nautilus: selinux: Allow ceph to read udev db

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32769 from liewegas/fix-42566-nautilus
Yuri Weinstein [Fri, 7 Feb 2020 22:03:28 +0000 (14:03 -0800)]
Merge pull request #32769 from liewegas/fix-42566-nautilus

nautilus: mgr/MgrClient: fix open condition

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoqa: test volumes plugin mount cleanup 33116/head
Patrick Donnelly [Tue, 3 Dec 2019 23:14:49 +0000 (15:14 -0800)]
qa: test volumes plugin mount cleanup

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 67143642db335fef2749559006776d4b4842f008)

5 years agopybind/mgr/volumes: use py3 items iterator
Patrick Donnelly [Tue, 3 Dec 2019 22:33:07 +0000 (14:33 -0800)]
pybind/mgr/volumes: use py3 items iterator

Otherwise this raises an exception.

Fixes: https://tracker.ceph.com/issues/43113
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 03f80804784052d7bf8a0802c319dccaa9b12389)

5 years agopybind/mgr/volumes: print errors in cleanup timer
Patrick Donnelly [Tue, 3 Dec 2019 22:32:24 +0000 (14:32 -0800)]
pybind/mgr/volumes: print errors in cleanup timer

The squelched error prevented us from knowing connection cleanup doesn't
work on py3.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b45c08b9977acbf8703259e78702b36ffba65162)

5 years agoqa: improve variable name
Patrick Donnelly [Wed, 4 Dec 2019 14:50:56 +0000 (06:50 -0800)]
qa: improve variable name

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoqa: test mgr cephfs mount blacklist
Patrick Donnelly [Tue, 3 Dec 2019 23:02:19 +0000 (15:02 -0800)]
qa: test mgr cephfs mount blacklist

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 98e3b7efe8bc92d88f161c892ce91a7fff67052c)
Note: removed mgr blacklist test which applies to Octopus.

5 years agoMerge pull request #32930 from rhcs-dashboard/wip-43845-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:33:33 +0000 (10:33 +0000)]
Merge pull request #32930 from rhcs-dashboard/wip-43845-nautilus

nautilus: mgr/dashboard: disable 'Add Capability' button in rgw user edit

Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #32827 from rhcs-dashboard/wip-43811-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:21:49 +0000 (10:21 +0000)]
Merge pull request #32827 from rhcs-dashboard/wip-43811-nautilus

nautilus: mgr/dashboard: check if user has config-opt permissions

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #31808 from bk201/wip-42956-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:20:36 +0000 (10:20 +0000)]
Merge pull request #31808 from bk201/wip-42956-nautilus

nautilus: mgr/dashboard: check embedded Grafana dashboard references

Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #31792 from rhcs-dashboard/wip-42936-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:19:33 +0000 (10:19 +0000)]
Merge pull request #31792 from rhcs-dashboard/wip-42936-nautilus

nautilus: mgr/dashboard: Dashboard can't handle self-signed cert on Grafana API

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #31190 from rhcs-dashboard/wip-42294-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:17:22 +0000 (10:17 +0000)]
Merge pull request #31190 from rhcs-dashboard/wip-42294-nautilus

nautilus: mgr/dashboard: add debug mode, and accept expected exception when SSL handshaking

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
5 years agoMerge pull request #32304 from ricardoasmarques/wip-43333-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:15:34 +0000 (10:15 +0000)]
Merge pull request #32304 from ricardoasmarques/wip-43333-nautilus

nautilus: mgr/dashboard: iSCSI targets not available if any gateway is down (and more...)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #32888 from shyukri/wip-42033-nautilus
Lenz Grimmer [Thu, 6 Feb 2020 10:13:27 +0000 (10:13 +0000)]
Merge pull request #32888 from shyukri/wip-42033-nautilus

nautilus: mgr/dashboard: Using wrong identifiers in RGW user/bucket datatables

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoqa: force creation of fs with EC default data pool 32600/head
Patrick Donnelly [Fri, 20 Dec 2019 20:57:31 +0000 (12:57 -0800)]
qa: force creation of fs with EC default data pool

In the future, we should add the EC data pool as a supplementary data
pool but that requires a mount to setup which is awkward in the code
here. When cephfs-shell is more widely available, this will be easier.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6e448f9aa0e71f1e820f4bd1b53e067268e4b67a)

5 years agoqa: add tests for adding EC data pools
Patrick Donnelly [Fri, 13 Dec 2019 00:43:14 +0000 (16:43 -0800)]
qa: add tests for adding EC data pools

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bf0cf8e08bfa49d2e84f079db2bd3b521ce5deec)

Conflicts:
qa/tasks/cephfs/filesystem.py

5 years agomon/MDSMonitor: warn when creating fs with default EC data pool
Patrick Donnelly [Thu, 7 Nov 2019 22:12:03 +0000 (14:12 -0800)]
mon/MDSMonitor: warn when creating fs with default EC data pool

Rationale can be found in [1]. Point is that EC pools incur a
significant performance penalty when dealing with small files and xattr
updates. This is because _every_ inode has a corresponding data pool
object with backtrace information stored in its xattr.

[1] doc/cephfs/createfs.rst

Fixes: https://tracker.ceph.com/issues/42450
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3e0aee5ed68b7b4018fb03d5160f3f19da0f6458)

5 years agoqa: add tests for CephFS admin commands
Patrick Donnelly [Wed, 30 Oct 2019 17:21:22 +0000 (10:21 -0700)]
qa: add tests for CephFS admin commands

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9dc07d80967287051eab3245e58d029157d3e1da)

5 years agomds: skip tell command scrub on multimds 32602/head
Patrick Donnelly [Mon, 20 Jan 2020 16:23:29 +0000 (08:23 -0800)]
mds: skip tell command scrub on multimds

The master version of this PR deals with the unified asok/tell
interface. Nautilus- are separated.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: throttle scrub start for multiple active MDS
Milind Changire [Fri, 10 Jan 2020 06:31:34 +0000 (12:01 +0530)]
mds: throttle scrub start for multiple active MDS

* add check to "scrub start" command handler to dishonor request if
  multiple MDS are "active" for a recursive scrub request
* add check to MDSRank::handle_mds_map() to see if scrubbing is in
  progress and max_mds is greater than 1, if so then scrubbing is
  aborted. This is supposed to take care of the condition when scrubbing
  has been started with max_mds == 1 and then bumped to a higher value

Fixes: https://tracker.ceph.com/issues/43483
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit f50b87bcb4737befcccf6310aaf522200361e731)

Conflicts:
src/mds/MDSRank.cc
src/mds/ScrubStack.h