]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoRevert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen" 25228/head
Yan, Zheng [Fri, 23 Nov 2018 03:29:52 +0000 (11:29 +0800)]
Revert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen"

6 years agoMerge pull request #24913 from smithfarm/wip-36428-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:49:16 +0000 (08:49 -0800)]
Merge pull request #24913 from smithfarm/wip-36428-mimic

mimic: tests: rbd: move OpenStack devstack test to rocky release

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24915 from smithfarm/wip-36529-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:48:51 +0000 (08:48 -0800)]
Merge pull request #24915 from smithfarm/wip-36529-mimic

mimic: rbd: TokenBucketThrottle: use reference to m_blockers.front() and then update it.

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24916 from smithfarm/wip-36555-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:48:29 +0000 (08:48 -0800)]
Merge pull request #24916 from smithfarm/wip-36555-mimic

mimic: rbd: [rbd-mirror] periodic mirror status timer might fail to be scheduled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24946 from jonsger/wip-36689-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:47:55 +0000 (08:47 -0800)]
Merge pull request #24946 from jonsger/wip-36689-mimic

mimic: tests: unittest_rbd_mirror: TestMockImageMap.AddInstancePingPongImageTest: Value of: it != peer_ack_ctxs->end()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24952 from jonsger/wip-36693-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:47:32 +0000 (08:47 -0800)]
Merge pull request #24952 from jonsger/wip-36693-mimic

mimic: rbd: [rbd-mirror] forced promotion after killing remote cluster results in stuck state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24988 from iridescent-rsy/mimic-36713
Yuri Weinstein [Thu, 22 Nov 2018 16:47:11 +0000 (08:47 -0800)]
Merge pull request #24988 from iridescent-rsy/mimic-36713

mimic: rbd: rbd-nbd: do not ceph_abort() after print the usages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25113 from swinds24/wip-36764-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:46:40 +0000 (08:46 -0800)]
Merge pull request #25113 from swinds24/wip-36764-mimic

mimic: mgr: hold lock while accessing the request list and submittin request

Reviewed-by: Boris Ranto <branto@redhat.com>
6 years agoMerge pull request #25115 from mcv21/mimic-36741
Yuri Weinstein [Thu, 22 Nov 2018 16:46:07 +0000 (08:46 -0800)]
Merge pull request #25115 from mcv21/mimic-36741

mimic: build/ops: debian: correct ceph-common relationship with older radosgw package

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25091 from jonsger/wip-36463-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:40:51 +0000 (08:40 -0800)]
Merge pull request #25091 from jonsger/wip-36463-mimic

mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #24987 from joscollin/wip-36690-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:40:10 +0000 (08:40 -0800)]
Merge pull request #24987 from joscollin/wip-36690-mimic

mimic: cephfs: client: request next osdmap for blacklisted client

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #24993 from jonsger/wip-36457-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:39:45 +0000 (08:39 -0800)]
Merge pull request #24993 from jonsger/wip-36457-mimic

mimic: cephfs: client: explicitly show blacklisted state via asok status command

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25036 from smithfarm/wip-36218-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:39:17 +0000 (08:39 -0800)]
Merge pull request #25036 from smithfarm/wip-36218-mimic

mimic: cephfs: some tool commands silently operate on only rank 0, even if multiple ranks exist

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25042 from smithfarm/wip-36461-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:38:48 +0000 (08:38 -0800)]
Merge pull request #25042 from smithfarm/wip-36461-mimic

mimic: mds: rctime not set on system inode (root) at startup

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25095 from jonsger/wip-37093-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:37:20 +0000 (08:37 -0800)]
Merge pull request #25095 from jonsger/wip-37093-mimic

mimic: mds: src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped) during max_mds thrashing

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge PR #25118 into mimic
Patrick Donnelly [Wed, 21 Nov 2018 18:11:16 +0000 (10:11 -0800)]
Merge PR #25118 into mimic

* refs/pull/25118/head:
mds: make timeout parameter optional for "cache drop"
test: add test for mds drop cache command
mds: command to trim mds cache and client caps
mds: implement journal flush as asynchronous context execution
mds: cleanup some asok commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25191 from badone/wip-mimic-ceph_ansible-specify-branch-stable-3.2
Brad Hubbard [Wed, 21 Nov 2018 07:05:43 +0000 (17:05 +1000)]
Merge pull request #25191 from badone/wip-mimic-ceph_ansible-specify-branch-stable-3.2

mimic: qa/ceph-ansible: Specify stable-3.2 branch

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #25193 from neha-ojha/wip-fix-thrash-old-clients-mimic
Neha Ojha [Wed, 21 Nov 2018 06:04:10 +0000 (22:04 -0800)]
Merge pull request #25193 from neha-ojha/wip-fix-thrash-old-clients-mimic

mimic: qa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoqa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel 25193/head
Neha Ojha [Tue, 20 Nov 2018 23:37:25 +0000 (23:37 +0000)]
qa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel

This fix goes directly into the mimic branch since the following packages
do not need to be installed on hammer and jewel.

- python34-rados
- python34-cephfs
- python3-rados
- python3-cephfs

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24899 from ivancich/rgw-renew-reshard-lock-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:12:33 +0000 (09:12 -0800)]
Merge pull request #24899 from ivancich/rgw-renew-reshard-lock-mimic

mimic: rgw: renew resharding locks to prevent expiration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24907 from smithfarm/wip-36223-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:12:07 +0000 (09:12 -0800)]
Merge pull request #24907 from smithfarm/wip-36223-mimic

mimic: rgw: default quota not set in radosgw for Openstack users

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24908 from smithfarm/wip-36415-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:11:39 +0000 (09:11 -0800)]
Merge pull request #24908 from smithfarm/wip-36415-mimic

mimic: librgw: crashes in multisite configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24910 from smithfarm/wip-36533-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:11:14 +0000 (09:11 -0800)]
Merge pull request #24910 from smithfarm/wip-36533-mimic

mimic: rgw: cls_user_remove_bucket does not write the modified cls_user_stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24911 from smithfarm/wip-36535-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:10:49 +0000 (09:10 -0800)]
Merge pull request #24911 from smithfarm/wip-36535-mimic

mimic: rgw: radosgw-admin user stats are incorrect when dynamic re-sharding is enabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24912 from smithfarm/wip-36539-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:10:15 +0000 (09:10 -0800)]
Merge pull request #24912 from smithfarm/wip-36539-mimic

mimic: use-after-free from RGWRadosGetOmapKeysCR::~RGWRadosGetOmapKeysCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25079 from jonsger/wip-36734-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:09:48 +0000 (09:09 -0800)]
Merge pull request #25079 from jonsger/wip-36734-mimic

mimic: rgw: beast frontend fails to parse ipv6 endpoints

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25085 from smithfarm/wip-37153-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:09:21 +0000 (09:09 -0800)]
Merge pull request #25085 from smithfarm/wip-37153-mimic

mimic: tests: ceph-admin-commands.sh workunit does not log what it's doing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25087 from jonsger/wip-36756-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:08:41 +0000 (09:08 -0800)]
Merge pull request #25087 from jonsger/wip-36756-mimic

mimic: rgw-admin: reshard add can add a non existant bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoqa/ceph-ansible: Specify stable-3.2 branch 25191/head
Brad Hubbard [Sun, 18 Nov 2018 23:37:02 +0000 (09:37 +1000)]
qa/ceph-ansible: Specify stable-3.2 branch

Ceph-ansible no longer supports mimic post stable-3.2.

https://tracker.ceph.com/issues/37332

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25133 from alfredodeza/mimic-c-v-dont-duplicated-rejected-reasons
Alfredo Deza [Fri, 16 Nov 2018 18:02:42 +0000 (13:02 -0500)]
Merge pull request #25133 from alfredodeza/mimic-c-v-dont-duplicated-rejected-reasons

mimic ceph-volume: rename Device property valid to available

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoc-v device test: rename valid to available 25133/head
Jan Fajerski [Fri, 16 Nov 2018 08:27:34 +0000 (09:27 +0100)]
c-v device test: rename valid to available

Update test to reflect 8a80990471108b0920d1d8aa1239733ae2b20e9c

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 047dc1c157e44642fa83873f02674b146e6d9028)

6 years agoinventory test: rename valid to available
Jan Fajerski [Fri, 16 Nov 2018 08:24:37 +0000 (09:24 +0100)]
inventory test: rename valid to available

Fixes test to reflect 8a80990471108b0920d1d8aa1239733ae2b20e9c

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit d895f97bb021c765f3c4d2d091b23bd0eef19f71)

6 years agoceph-volume: rename Device property valid to available
Jan Fajerski [Fri, 16 Nov 2018 08:22:06 +0000 (09:22 +0100)]
ceph-volume: rename Device property valid to available

This flag is used in the inventory reporting and available is deemed more
appropriate. Furthermore this fixes a bug where rejected_reasons
accumulated duplicate entries.

Fixes: http://tracker.ceph.com/issues/36701
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8a80990471108b0920d1d8aa1239733ae2b20e9c)

6 years agoMerge pull request #25130 from alfredodeza/mimic-c-v-inventory-doc
Alfredo Deza [Fri, 16 Nov 2018 15:22:24 +0000 (10:22 -0500)]
Merge pull request #25130 from alfredodeza/mimic-c-v-inventory-doc

mimic doc: add ceph-volume inventory sections

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agodoc: add ceph-volume inventory sections 25130/head
Jan Fajerski [Wed, 14 Nov 2018 13:09:37 +0000 (14:09 +0100)]
doc: add ceph-volume inventory sections

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit dfc0e1d9a69f6f9ffb1d3e0b0ec33a75827c36e0)

6 years agoMerge pull request #25116 from alfredodeza/mimic-rm26953
Andrew Schoen [Thu, 15 Nov 2018 17:23:54 +0000 (11:23 -0600)]
Merge pull request #25116 from alfredodeza/mimic-rm26953

mimic ceph-volume: allow to specify --cluster-fsid instead of reading from ceph.conf

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agomds: make timeout parameter optional for "cache drop" 25118/head
Venky Shankar [Thu, 11 Oct 2018 06:26:29 +0000 (02:26 -0400)]
mds: make timeout parameter optional for "cache drop"

Fixes: http://tracker.ceph.com/issues/36694
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 73459619747046d38df5f4ec6707db12dce77317)

 Conflicts:
        src/mds/MDSDaemon.cc

6 years agoceph-volume tests.lvm verify get_cluster_fsid can read from args or ceph.conf 25116/head
Alfredo Deza [Wed, 3 Oct 2018 19:21:03 +0000 (15:21 -0400)]
ceph-volume tests.lvm verify get_cluster_fsid can read from args or ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 02ea6dc0663cdaf764fdb1a1a5b31a736b854e6e)

6 years agoceph-volume lvm.prepare consume the value from --cluster-fsid if available
Alfredo Deza [Wed, 3 Oct 2018 17:32:05 +0000 (13:32 -0400)]
ceph-volume lvm.prepare consume the value from --cluster-fsid if available

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a92883e60f3f2206a11b43762b3afcb84b05f07e)

6 years agoceph-volume lvm.common add the --cluster-fsid flag
Alfredo Deza [Wed, 3 Oct 2018 17:31:43 +0000 (13:31 -0400)]
ceph-volume lvm.common add the --cluster-fsid flag

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 15546ae6991c244197537d81092875b12613e20b)

6 years agodebian: correct ceph-common relationship with older radosgw package 25115/head
Matthew Vernon [Thu, 8 Nov 2018 17:23:36 +0000 (17:23 +0000)]
debian: correct ceph-common relationship with older radosgw package

Fixes: https://tracker.ceph.com/issues/37273
9fd30b93f7281fad70b93512f0a25e3465f5b225 moved
/etc/bash_completion.d/radosgw-admin from radosgw to ceph-common. This
means that if you try and install a newer ceph-common over an older
radosgw, there's a conflict, and the install fails:

```
Unpacking ceph-common (12.2.8-1xenial) over (10.2.9-0ubuntu0.16.04.1) ...
dpkg: error processing archive ceph-common_12.2.8-1xenial_amd64.deb (--install):
 trying to overwrite '/etc/bash_completion.d/radosgw-admin', which is also in package radosgw 10.2.9-0ubuntu0.16.04.1
```

Per Debian policy (
https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-in-other-packages
) the correct way to handle a package taking over a file is for a
versioned Replaces and Breaks.

The change went into 12.0.3, so this commit adds Replaces and Breaks
against radosgw less than that version. It should be backported to
Luminous to avoid issues with upgrades from older versions (Jewel and
Kraken).

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit dd24ddcbf78104ec2370fc3e956984a57cde670b)

6 years agotest: add test for mds drop cache command
Venky Shankar [Wed, 22 Aug 2018 15:06:25 +0000 (11:06 -0400)]
test: add test for mds drop cache command

Fixes: https://tracker.ceph.com/issues/36282
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 43d1b8e174af333514d2b23d0a1be9ec3895fd25)

 Conflicts:
qa/tasks/cephfs/test_misc.py

6 years agomds: command to trim mds cache and client caps
Venky Shankar [Sun, 26 Aug 2018 13:48:56 +0000 (09:48 -0400)]
mds: command to trim mds cache and client caps

With this command, the MDS would request clients to release
caps followed by trimming its own cache and a journal flush.
The command accepts a timeout to wait for clients to respond
to session recall and flush messages.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit da70dde8aa23b0b24057089edaee4d6317f52bd4)

 Conflicts:
src/mds/MDSDaemon.cc
src/mds/MDSRank.cc
src/mds/Server.cc
src/mds/Server.h

6 years agomds: implement journal flush as asynchronous context execution
Venky Shankar [Fri, 14 Sep 2018 04:23:11 +0000 (00:23 -0400)]
mds: implement journal flush as asynchronous context execution

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 75f42340db0ee7263b0708675b8cdfda3bc78059)

 Conflicts:
src/mds/MDSRank.cc
src/mds/MDSRank.h

6 years agomds: cleanup some asok commands
Venky Shankar [Sun, 26 Aug 2018 12:55:05 +0000 (08:55 -0400)]
mds: cleanup some asok commands

Signed-off-by: Rishabh Dave <ridave@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4c03cfd99eb9144e92740345d6874d1f07831b00)

6 years agoceph-mgr: hold lock while accessing the request list and submitting request 25113/head
Jerry Lee [Mon, 12 Nov 2018 06:27:53 +0000 (14:27 +0800)]
ceph-mgr: hold lock while accessing the request list and submitting request

The request creation can fire up the notify event early and it can cause
a race condition where the actual request was not yet added to the
self.requests list which makes the submit_request() function waits
forever without accepting new requests.

https://marc.info/?l=ceph-devel&m=154104291714160&w=2

Fixes: https://tracker.ceph.com/issues/36764
Signed-off-by: Jerry Lee <leisurelysw24@gmail.com>
(cherry picked from commit b09aefc2d6af3de8cc3df30258dfdc639c919a9f)

Conflicts:
src/pybind/mgr/restful/module.py

6 years agoMerge pull request #25103 from alfredodeza/mimic-rm-27062
Alfredo Deza [Wed, 14 Nov 2018 23:16:45 +0000 (18:16 -0500)]
Merge pull request #25103 from alfredodeza/mimic-rm-27062

mimic ceph-volume: reject devices that have existing GPT headers

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25100 from alfredodeza/mimic-bz1644828
Andrew Schoen [Wed, 14 Nov 2018 20:17:35 +0000 (14:17 -0600)]
Merge pull request #25100 from alfredodeza/mimic-bz1644828

mimic ceph-volume: remove LVs when using zap --destroy

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: update tests after removing the LVPath arg validator 25103/head
Andrew Schoen [Wed, 14 Nov 2018 16:35:32 +0000 (10:35 -0600)]
ceph-volume: update tests after removing the LVPath arg validator

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit 0cb95ace88a731f7c30adf3f2286691fa3b2c49c)

6 years agoceph-volume: return lvs as vg/lv from ValidDevice if as_string is True
Andrew Schoen [Tue, 13 Nov 2018 19:02:39 +0000 (13:02 -0600)]
ceph-volume: return lvs as vg/lv from ValidDevice if as_string is True

All code in prepare and activate expect lvs to be in that format.
Eventually we can refactor all that to use the Device class as well.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit fdf7f3c575299b6be0ac40fb6eee5106cea010c9)

6 years agoceph-volume: adds lv_name property to Device class
Andrew Schoen [Tue, 13 Nov 2018 18:40:45 +0000 (12:40 -0600)]
ceph-volume: adds lv_name property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit cc0317efac5a3a25edcdb48af2a63a969c2ac43a)

6 years agoceph-volume: combine the LVPath arg validator with ValidDevice
Andrew Schoen [Tue, 13 Nov 2018 18:28:23 +0000 (12:28 -0600)]
ceph-volume: combine the LVPath arg validator with ValidDevice

This combines the two arg validators and makes both batch and
prepare/active both use ValidDevice.

This will also allow us to use lvs with a full path, like
/dev/vg/lv instead of always enforcing vg/lv.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit d6a5a7606c3a0c1f1624c5784abb02bb1a4e1620)

6 years agoceph-volume: adds has_gpt_headers property to Device class
Andrew Schoen [Tue, 13 Nov 2018 18:27:19 +0000 (12:27 -0600)]
ceph-volume: adds has_gpt_headers property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit 568a0ce5be0161541772ceacb9b81652f51659f5)

6 years agoceph-volume: add PTTYPE to blkid parser
Andrew Schoen [Tue, 13 Nov 2018 15:35:21 +0000 (09:35 -0600)]
ceph-volume: add PTTYPE to blkid parser

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit 3d86d2dda9fae87e143e5212d5f354a22f02909d)

6 years agoceph-volume tests update tests to handle new SystemExit exceptions in main 25100/head
Alfredo Deza [Tue, 13 Nov 2018 22:06:03 +0000 (17:06 -0500)]
ceph-volume tests update tests to handle new SystemExit exceptions in main

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 27d0a915d4a7f56aafc603a949b4e4c36772d236)

6 years agoceph-volume tests.api verify that Volume object can be removed
Alfredo Deza [Tue, 13 Nov 2018 22:05:31 +0000 (17:05 -0500)]
ceph-volume tests.api verify that Volume object can be removed

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9527cffbe335043f7c5a60162138dfe093ae51da)

6 years agoceph-volume api.lvm allow passing a Volume object to remove_lv
Alfredo Deza [Tue, 13 Nov 2018 21:56:02 +0000 (16:56 -0500)]
ceph-volume api.lvm allow passing a Volume object to remove_lv

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e7daa71604f4d08ace1125bb63dcfe909796d3ab)

6 years agoceph-volume tests.functional.lvm test full removal of LVs
Alfredo Deza [Tue, 13 Nov 2018 21:05:32 +0000 (16:05 -0500)]
ceph-volume tests.functional.lvm test full removal of LVs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4656bf0e333a50e0fca5fd0612cdda98ab5d70f9)

6 years agoceph-volume main raise sys.exit to prevent garbage in help
Alfredo Deza [Tue, 13 Nov 2018 13:32:16 +0000 (08:32 -0500)]
ceph-volume main raise sys.exit to prevent garbage in help

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f695d1716e9fc6252dd713eb8fb08c28b1284611)

6 years agoceph-volume lvm.zap remove vgs|lvs when --destroy is used
Alfredo Deza [Tue, 13 Nov 2018 13:30:23 +0000 (08:30 -0500)]
ceph-volume lvm.zap remove vgs|lvs when --destroy is used

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 677cbbb059185fe2808746b36bd3f988ff2af41c)

6 years agomds: flush dirty dirfrags that weren't logged when deactivating mds 25095/head
Yan, Zheng [Wed, 17 Oct 2018 09:22:47 +0000 (17:22 +0800)]
mds: flush dirty dirfrags that weren't logged when deactivating mds

CDir::log_mark_dirty() may mark dirfrag dirty, without journal event.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 68b6073af92595da92c0032e814a2bb3d3cba5cd)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agomds: use MDlog::trim_all() to trim log when deactivating mds
Yan, Zheng [Tue, 9 Oct 2018 10:30:55 +0000 (18:30 +0800)]
mds: use MDlog::trim_all() to trim log when deactivating mds

The problem of MDLog::trim(0) is that it expires current segment.
New log events (scatter nudge) may get added to current segment
when MDLog::trim(0) expires current segement.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 49c0f072f60deb5761e6b6a6518d88c59510ee9b)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agomds: don't cap log when there are replicated objects
Yan, Zheng [Tue, 9 Oct 2018 08:42:41 +0000 (16:42 +0800)]
mds: don't cap log when there are replicated objects

replicas may dirty scatter locks

Fixes: http://tracker.ceph.com/issues/36350
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 6c1d4c606a9f597196235aa5cd3fa6d98a683290)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
Conflicts:
src/mds/MDCache.cc: adapt to code around

6 years agoclient: fix fuse client can't read or write data due its caps is invalid 25091/head
YunfeiGuan [Wed, 26 Sep 2018 09:01:38 +0000 (17:01 +0800)]
client: fix fuse client can't read or write data due its caps is invalid

We can't do read or write if we don't have Fr or Fw.Similarly, if the
caps is invalid, we can't read or wirte either.

What's worse,the mds don't konw our cap gen is invalid and think client
inode's caps is right.Thus will lead client application hung all the time.

Fixes: http://tracker.ceph.com/issues/36189
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit 1d742aa51be07cebf1b0d19b6640684b263e6bfc)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agorgw: reshard add: fail correctly on a non existant bucket 25087/head
Abhishek Lekshmanan [Mon, 15 Oct 2018 20:00:33 +0000 (22:00 +0200)]
rgw: reshard add: fail correctly on a non existant bucket

Currently the method that validates the reshard params does not fail correctly
in case of a non existant bucket causing reshard add to print an error message
but add the non existant bucket to the queue anyway. This will later be a cause
of EBUSY errors as this entry on the logshard  can never be processed.

Fixes: http://tracker.ceph.com/issues/36449
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit c564b348a24edf2804a8f5f0d034f388be1c5c5f)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agotests: make ceph-admin-commands.sh log what it does 25085/head
Nathan Cutler [Tue, 13 Nov 2018 18:46:24 +0000 (19:46 +0100)]
tests: make ceph-admin-commands.sh log what it does

Fixes: http://tracker.ceph.com/issues/37089
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 0185abfeb448ea569439ec9e06f3625f4fa6cc86)

6 years agoMerge pull request #25066 from alfredodeza/mimic-rm36768
Andrew Schoen [Tue, 13 Nov 2018 16:11:59 +0000 (10:11 -0600)]
Merge pull request #25066 from alfredodeza/mimic-rm36768

mimic ceph-volume: patch Device when testing

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agorgw: beast frontend parses ipv6 addrs 25079/head
Casey Bodley [Thu, 1 Nov 2018 20:17:11 +0000 (16:17 -0400)]
rgw: beast frontend parses ipv6 addrs

Fixes: http://tracker.ceph.com/issues/36662
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a3b4124fd258887ff4cb3f65967f593658a41f4f)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agorgw: beast frontend uses default port 443 for ssl_endpoint
Casey Bodley [Wed, 31 Oct 2018 18:19:48 +0000 (14:19 -0400)]
rgw: beast frontend uses default port 443 for ssl_endpoint

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9836ee2643c38afcd0221488d607dffa359e9bd4)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agorgw: RemoteApplier::create_account() applies default quota config 24907/head
Casey Bodley [Wed, 19 Sep 2018 13:39:39 +0000 (09:39 -0400)]
rgw: RemoteApplier::create_account() applies default quota config

rgw users created to shadow users from external auth were missing the
logic to apply their default quota settings

Fixes: http://tracker.ceph.com/issues/24595
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit abb81b4fb5898d7cf39c86dc40d3b488717978c9)

6 years agorgw: add helper functions to apply configured default quotas
Casey Bodley [Wed, 19 Sep 2018 13:38:20 +0000 (09:38 -0400)]
rgw: add helper functions to apply configured default quotas

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ba83d0c0be3960e8f352c3058cb9a40020de02ae)

Conflicts:
    src/rgw/rgw_quota.cc
    src/rgw/rgw_quota.h
- mimic uses "md_config_t" (non-pointer) instead of "ConfigProxy"
(pointer)

6 years agoMerge pull request #24467 from vshankar/wip-26991
Yuri Weinstein [Mon, 12 Nov 2018 20:25:21 +0000 (12:25 -0800)]
Merge pull request #24467 from vshankar/wip-26991

mimic: mds: curate priority of perf counters sent to mgr

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoceph-volume tests patch Device() by splitting parametrized method 25066/head
Alfredo Deza [Mon, 12 Nov 2018 17:57:26 +0000 (12:57 -0500)]
ceph-volume tests patch Device() by splitting parametrized method

This was causing failures on systems where there is no LVM or where the
device names don't match. Patching is always recommended to avoid
conflicts with the system testing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 82d96f876484c9668e2fc4726057691ba2c50f06)

6 years agoceph-volume tests inventory should patch before using .get_devices()
Alfredo Deza [Mon, 12 Nov 2018 17:56:42 +0000 (12:56 -0500)]
ceph-volume tests inventory should patch before using .get_devices()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7ee7a59910c9837659419c235f669a53cc9f613c)

6 years agoMerge pull request #24931 from jonsger/mimic-backport-pr#24700
Yuri Weinstein [Mon, 12 Nov 2018 17:13:24 +0000 (09:13 -0800)]
Merge pull request #24931 from jonsger/mimic-backport-pr#24700

mimic: rgw: support server-side encryption when SSL is terminated in a proxy

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: remove redundant quota logic from admin/user api
Casey Bodley [Wed, 19 Sep 2018 13:35:03 +0000 (09:35 -0400)]
rgw: remove redundant quota logic from admin/user api

RGWOp_User_Create was applying the configured default quotas to the
RGWUserAdminOpState, but RGWUser::execute_add() will already apply those
defaults when the AdminOpState doesn't provide them

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3a252bbc59fd0a0fb8430e09948420c7092717a6)

6 years agoMerge pull request #25013 from alfredodeza/mimic-ceph-volume-inventory
Alfredo Deza [Mon, 12 Nov 2018 13:37:18 +0000 (08:37 -0500)]
Merge pull request #25013 from alfredodeza/mimic-ceph-volume-inventory

mimic ceph-volume: add inventory command

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoqa: add test for rctime on root inode 25042/head
Patrick Donnelly [Wed, 26 Sep 2018 18:24:35 +0000 (11:24 -0700)]
qa: add test for rctime on root inode

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ddbc3016a59a2ff639c6c5970ade79f6054692ee)

6 years agomds: set rctime on new system inode
Patrick Donnelly [Wed, 26 Sep 2018 17:08:45 +0000 (10:08 -0700)]
mds: set rctime on new system inode

Fixes: http://tracker.ceph.com/issues/36221
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a6fe15a29e53995e10e79e1338801cd53232ee38)

Conflicts:
src/mds/MDCache.cc
- master uses g_conf(), mimic g_conf
- master uses ceph_assert, mimic assert

6 years agotest: make rank argument mandatory when running journal_tool 25036/head
Venky Shankar [Tue, 4 Sep 2018 06:20:54 +0000 (02:20 -0400)]
test: make rank argument mandatory when running journal_tool

Also, fix a bunch of quirky journal_tool invocations that pass
"--rank" argument as the command argument rather than passing it
as function argument.

Fixes: https://tracker.ceph.com/issues/24780
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f65193d95534dd34b593c92e1848b181737b0561)

6 years agocephfs-journal-tool: make "--rank" argument mandatory
Venky Shankar [Mon, 23 Jul 2018 06:35:05 +0000 (02:35 -0400)]
cephfs-journal-tool: make "--rank" argument mandatory

... and do not silenty act on the default filesystem.
Force users to specify the filesystem name and rank.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5e4702e303bd70b4ff1de973375bef952c0d8b6a)

6 years agocephfs-journal-tool: pass local arg vector for Journal actions
Venky Shankar [Mon, 23 Jul 2018 06:30:47 +0000 (02:30 -0400)]
cephfs-journal-tool: pass local arg vector for Journal actions

Some helper functions clobber the passed in arg vector. This
causes operations on all but the first rank to fail as the
operations for the first rank tampers with the arg vector.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d5e482a6c2782993e6e9903eb6f11f5ebe260925)

6 years agocephfs-journal-tool: dump to per rank output file wherever necessary
Venky Shankar [Mon, 23 Jul 2018 06:28:07 +0000 (02:28 -0400)]
cephfs-journal-tool: dump to per rank output file wherever necessary

cephfs-journal-tool supports operations on all ranks. Operations
such as dump/export do not write to distinct filenames hence
overwriting the data dumped or exported for the previous rank.

With this change (and further commits), for operations on all
ranks, dump/export would write to distinct filenames suffixed
by the mds rank (.0, .1, etc..). For operations on a single rank
or if there exist a single rank, the passed in filename is used
as it is.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit cb19db75c7f23278d2598b58526d1bf8bf00b679)

6 years agoMerge pull request #25011 from alfredodeza/mimic-rm36470
Andrew Schoen [Fri, 9 Nov 2018 22:09:38 +0000 (16:09 -0600)]
Merge pull request #25011 from alfredodeza/mimic-rm36470

mimic ceph-volume enable  --no-systemd flag for simple sub-command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: add inventory command 25013/head
Jan Fajerski [Wed, 31 Oct 2018 13:59:05 +0000 (14:59 +0100)]
ceph-volume: add inventory command

The inventory command provides information about a nodes disk inventory.
Existing logical volumes on a disk or one of its partitions are scanned
and reported.
The output can be formatted as plain text or json.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 57adfc6bb8abd9fc84b1378d5f69a16994d19e11)

6 years agoMerge pull request #24980 from alfredodeza/mimic-bz1644265
Alfredo Deza [Fri, 9 Nov 2018 20:16:28 +0000 (15:16 -0500)]
Merge pull request #24980 from alfredodeza/mimic-bz1644265

mimic ceph-volume util.encryption robust blkid+lsblk detection of lockbox

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests ensure activate behavior with systemd disabling 25011/head
Alfredo Deza [Thu, 8 Nov 2018 19:14:17 +0000 (14:14 -0500)]
ceph-volume tests ensure activate behavior with systemd disabling

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 3e80118ccaccaa61e92b01f4f389d6056401d8a7)

6 years agoceph-volume simple.activate factor out systemd interactions for easier testing
Alfredo Deza [Thu, 8 Nov 2018 19:13:52 +0000 (14:13 -0500)]
ceph-volume simple.activate factor out systemd interactions for easier testing

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 18ddd962116b5359af770088535f502797ac79c4)

6 years agoceph-volume simple.trigger replace systemd with from_trigger keyword argument
Alfredo Deza [Thu, 8 Nov 2018 14:14:24 +0000 (09:14 -0500)]
ceph-volume simple.trigger replace systemd with from_trigger keyword argument

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 9526587403b76fa5397ae80b63328e3a59765c2e)

6 years agoceph-volume simple.activate allow a --no-systemd flag to skip systemctl actions
Alfredo Deza [Thu, 8 Nov 2018 14:13:31 +0000 (09:13 -0500)]
ceph-volume simple.activate allow a --no-systemd flag to skip systemctl actions

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 4d9ad0d3d4bedd5efa95653fbb86607f0b469174)

6 years agoclient: explicitly show blacklisted state via asok status command 24993/head
Zhi Zhang [Wed, 10 Oct 2018 03:15:36 +0000 (11:15 +0800)]
client: explicitly show blacklisted state via asok status command

It is easy for common users to understand why their clients not working
due to blacklisted state.

Fixes: http://tracker.ceph.com/issues/36352
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 0261506815e7e3f41656cc5fcf0461157e073179)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agorbd-nbd: do not ceph_abort() after print the usages. 24988/head
Shiyang Ruan [Tue, 30 Oct 2018 01:22:27 +0000 (09:22 +0800)]
rbd-nbd: do not ceph_abort() after print the usages.

Fixes: http://tracker.ceph.com/issues/36660
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
(cherry picked from commit a916342d36214f0de785d2ebf8852c47e2fa5b27)

6 years agoclient: request next osdmap for blacklisted client 24987/head
Zhi Zhang [Thu, 1 Nov 2018 06:52:04 +0000 (14:52 +0800)]
client: request next osdmap for blacklisted client

When client became blacklisted, it did't subscribe next osdmap. So when
it was removed from blacklist by expiration or commandline, it didn't
get updated osdmap which is the only way for the client to reset its
blacklisted flag.

Fixes: http://tracker.ceph.com/issues/36668
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 1baf91e7ef0419270d6ea5d7d68adb94a2403114)

6 years agoceph-volume util.encryption robust blkid+lsblk detection of lockbox 24980/head
Alfredo Deza [Fri, 2 Nov 2018 19:03:45 +0000 (15:03 -0400)]
ceph-volume util.encryption robust blkid+lsblk detection of lockbox

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fc3e6ea3d9200e0fe2a3aa2576fcd45db223c277)

6 years agoMerge pull request #24959 from alfredodeza/mimic-rm36672 25004/head
Alfredo Deza [Wed, 7 Nov 2018 11:47:51 +0000 (06:47 -0500)]
Merge pull request #24959 from alfredodeza/mimic-rm36672

mimic ceph-volume tests install ceph-ansible's requirements.txt dependencies

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests install ceph-ansible's requirements.txt dependencies 24959/head
Alfredo Deza [Thu, 1 Nov 2018 12:48:28 +0000 (08:48 -0400)]
ceph-volume tests install ceph-ansible's requirements.txt dependencies

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 52ddb69813c5849506db5f850a3f2532ff1124b5)

6 years agoMerge pull request #24957 from alfredodeza/mimic-rm36704
Andrew Schoen [Tue, 6 Nov 2018 21:24:20 +0000 (15:24 -0600)]
Merge pull request #24957 from alfredodeza/mimic-rm36704

mimic: ceph-volume tests.systemd update imports for systemd module

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.systemd update imports for systemd module 24957/head
Alfredo Deza [Tue, 6 Nov 2018 14:45:08 +0000 (09:45 -0500)]
ceph-volume tests.systemd update imports for systemd module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a63a6c86a03738bf50fa71a490951343a83afff6)

6 years agoceph-volume systemd import main so console_scripts work for executable
Alfredo Deza [Tue, 6 Nov 2018 14:44:39 +0000 (09:44 -0500)]
ceph-volume systemd import main so console_scripts work for executable

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 280e2baab27cc061d2d5e456f098cd3405f36873)

6 years agoMerge pull request #24826 from alfredodeza/mimic-rm36246
Andrew Schoen [Tue, 6 Nov 2018 21:06:30 +0000 (15:06 -0600)]
Merge pull request #24826 from alfredodeza/mimic-rm36246

mimic ceph-volume util.encryption don't push stderr to terminal

Reviewed-by: Andrew Schoen <aschoen@redhat.com>