]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoceph-volume: use terminal.MultiLogger in `lvm zap` 20438/head
Andrew Schoen [Mon, 12 Feb 2018 21:23:08 +0000 (15:23 -0600)]
ceph-volume: use terminal.MultiLogger in `lvm zap`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3f2e7074ee42ff8248afaed2aa9bfd919096fc38)

7 years agoceph-volume: use api helper methods for pvs and lvs in lvm zap
Andrew Schoen [Wed, 7 Feb 2018 15:06:45 +0000 (09:06 -0600)]
ceph-volume: use api helper methods for pvs and lvs in lvm zap

This is just a cleaner way of doing this instead of using PVolumes
and Volumes directly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit c8a6da4e1bb645ca5ee8cb8f5d861584fe70210d)

7 years agoceph-volume: fix documentation typos for lvm zap
Andrew Schoen [Wed, 7 Feb 2018 14:35:47 +0000 (08:35 -0600)]
ceph-volume: fix documentation typos for lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 5552cefc4ce6d6d819f43671cec83ca23b119665)

7 years agoceph-volume: when zapping unmount osd directories
Andrew Schoen [Thu, 1 Feb 2018 20:52:46 +0000 (14:52 -0600)]
ceph-volume: when zapping unmount osd directories

If you zap an lv, device or partition and it's currently mounted as a
ceph osd directory then it will be unmounted so the zap can complete.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit d4639ac116407f71ba6bff06a3202c1490d57dee)

7 years agoceph-volume: adds a util.system.unmount function
Andrew Schoen [Thu, 1 Feb 2018 20:29:30 +0000 (14:29 -0600)]
ceph-volume: adds a util.system.unmount function

This function will unmount the given path

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 6909baef542f3cd9b7d97113a6152280e72013f6)

7 years agoMerge pull request #20407 from cbodley/wip-qa-multisite-trim-luminous
Casey Bodley [Tue, 13 Feb 2018 21:43:14 +0000 (16:43 -0500)]
Merge pull request #20407 from cbodley/wip-qa-multisite-trim-luminous

luminous: qa/rgw: disable log trim in multisite suite

Reviewed-by: Yuri Weinstein <yuri.weinstein@gmail.com>
7 years agoqa/rgw: disable log trim in multisite suite 20407/head
Casey Bodley [Mon, 11 Dec 2017 21:14:05 +0000 (16:14 -0500)]
qa/rgw: disable log trim in multisite suite

the multisite tests run manual trim operations with radosgw-admin, which
can race with internal log trimming to produce tests failures

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f8909bb6a6320822da7a4c4dab242f84732ebf5d)

7 years agoMerge pull request #20368 from ceph/backport-20367
Andrew Schoen [Thu, 8 Feb 2018 17:56:12 +0000 (18:56 +0100)]
Merge pull request #20368 from ceph/backport-20367

luminous: ceph-volume: adds custom cluster name support to simple

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoMerge pull request #20357 from pdvian/wip-22938-luminous
Abhishek L [Thu, 8 Feb 2018 17:08:49 +0000 (18:08 +0100)]
Merge pull request #20357 from pdvian/wip-22938-luminous

luminous: rgw: unlink deleted bucket from bucket's owner

Reviewed-By: Orit Wasserman <owasserm@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
7 years agoceph-volume: use a custom cluster name in simple functional tests 20368/head
Andrew Schoen [Thu, 8 Feb 2018 13:09:57 +0000 (07:09 -0600)]
ceph-volume: use a custom cluster name in simple functional tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7f1dc6b3ab885253d28c223bda7e6e1232a84609)

7 years agoceph-volume: set conf.cluster name during simple activate
Andrew Schoen [Wed, 7 Feb 2018 20:42:24 +0000 (14:42 -0600)]
ceph-volume: set conf.cluster name during simple activate

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b1cee74fa2225fe10a024c7291bd23aadcc23441)

7 years agoMerge pull request #20338 from cbodley/wip-22930
Orit Wasserman [Thu, 8 Feb 2018 10:43:44 +0000 (12:43 +0200)]
Merge pull request #20338 from cbodley/wip-22930

luminous: rgw: fix for empty query string in beast frontend
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agorgw: unlink deleted bucket from bucket's owner 20357/head
Casey Bodley [Thu, 18 Jan 2018 19:53:35 +0000 (14:53 -0500)]
rgw: unlink deleted bucket from bucket's owner

if a bucket is deleted by an admin/system user instead of its
owner, the unlink would fail and the deleted bucket remained
visible to the original owner

Fixes: http://tracker.ceph.com/issues/22248
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4701e85a3ea72852a61a106724fa22497f3d5789)

7 years agoMerge pull request #20204 from tchaikov/wip-pr-18614-luminous
Yuri Weinstein [Wed, 7 Feb 2018 22:49:36 +0000 (14:49 -0800)]
Merge pull request #20204 from tchaikov/wip-pr-18614-luminous

luminous: osd,mgr: report pending creating pgs to mgr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #20249 from pdvian/wip-22864-luminous
Yuri Weinstein [Wed, 7 Feb 2018 22:45:33 +0000 (14:45 -0800)]
Merge pull request #20249 from pdvian/wip-22864-luminous

luminous: mds: fix scrub crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20256 from pdvian/wip-22860-luminous
Yuri Weinstein [Wed, 7 Feb 2018 22:45:07 +0000 (14:45 -0800)]
Merge pull request #20256 from pdvian/wip-22860-luminous

luminous: osdc: "FAILED assert(bh->last_write_tid > tid)" in powercycle-wip-yuri-master-1.19.18-distro-basic-smithi

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20299 from pdvian/wip-22859-luminous
Yuri Weinstein [Wed, 7 Feb 2018 22:44:39 +0000 (14:44 -0800)]
Merge pull request #20299 from pdvian/wip-22859-luminous

luminous: mds: set higher priority for some perf counters

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20300 from pdvian/wip-22867-luminous
Yuri Weinstein [Wed, 7 Feb 2018 22:44:03 +0000 (14:44 -0800)]
Merge pull request #20300 from pdvian/wip-22867-luminous

luminous: MDS : Avoid the assert failure when the inode for the cap_export from other…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #20339 from batrick/i22242
Yuri Weinstein [Wed, 7 Feb 2018 22:41:45 +0000 (14:41 -0800)]
Merge pull request #20339 from batrick/i22242

luminous: mds: limit size of subtree migration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #20340 from batrick/i22240
Yuri Weinstein [Wed, 7 Feb 2018 22:41:14 +0000 (14:41 -0800)]
Merge pull request #20340 from batrick/i22240

luminous: mds: prevent filelock from being stuck at XSYN state

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #20341 from batrick/i22089
Yuri Weinstein [Wed, 7 Feb 2018 22:40:33 +0000 (14:40 -0800)]
Merge pull request #20341 from batrick/i22089

luminous: mds: don't report repaired backtraces in damagetable, write back after repair, clean up scrub log

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #20337 from liewegas/wip-bluestore-aio-read
Yuri Weinstein [Wed, 7 Feb 2018 22:38:58 +0000 (14:38 -0800)]
Merge pull request #20337 from liewegas/wip-bluestore-aio-read

luminous: os/bluestore: propagate read EIO errors up the stack

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20350 from ceph/backport-20264
Andrew Schoen [Wed, 7 Feb 2018 18:02:31 +0000 (19:02 +0100)]
Merge pull request #20350 from ceph/backport-20264

luminous: ceph-volume dmcrypt support for simple

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume: do not test custom cluster names with simple 20350/head
Andrew Schoen [Tue, 6 Feb 2018 17:36:03 +0000 (11:36 -0600)]
ceph-volume: do not test custom cluster names with simple

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 63046e0d7bd05a42477dd1fbd4af11f81046b08d)

7 years agoceph-volume: sleep 2 minutes after reboot in simple tests
Andrew Schoen [Tue, 6 Feb 2018 16:46:44 +0000 (10:46 -0600)]
ceph-volume: sleep 2 minutes after reboot in simple tests

This will allow for some time for OSD services to start back up after
reboot. If not we see race conditions where tests fail because they
were run before the OSD finished startup.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 85b319a2c577c1fa86c6ccbbac61f015dd24ad21)

7 years agoceph-volume: adds the simple dmcrypt_plain and dmcrypt_luks tests
Andrew Schoen [Mon, 5 Feb 2018 20:18:28 +0000 (14:18 -0600)]
ceph-volume: adds the simple dmcrypt_plain and dmcrypt_luks tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3052010a80a1783b6ebd629a603638ab5806f8fd)

7 years agoceph-volume: fix ceph-volume simple scan help menu test
Andrew Schoen [Mon, 5 Feb 2018 19:53:13 +0000 (13:53 -0600)]
ceph-volume: fix ceph-volume simple scan help menu test

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2ee70bf2d1958c0655ddb5b7cc4d2ed3f4bc58b8)

7 years agoceph-volume tests.functional add simple xenial filestore dmcrypt plain support
Alfredo Deza [Fri, 2 Feb 2018 14:11:09 +0000 (09:11 -0500)]
ceph-volume tests.functional add simple xenial filestore dmcrypt plain support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 00b14177bdbe6d66424001b1c35ca8fa7b46977f)

7 years agoceph-volume tests.functional add simple xenial filestore dmcrypt luks support
Alfredo Deza [Fri, 2 Feb 2018 14:10:56 +0000 (09:10 -0500)]
ceph-volume tests.functional add simple xenial filestore dmcrypt luks support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0b2f869ccbb30c27bd19f2a05bb893957912142e)

7 years agoceph-volume tests.functional add simple xenial bluestore dmcrypt plain support
Alfredo Deza [Fri, 2 Feb 2018 14:10:36 +0000 (09:10 -0500)]
ceph-volume tests.functional add simple xenial bluestore dmcrypt plain support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0174f9e1646f4b51c712506f629b48c329f20ea1)

7 years agoceph-volume tests.functional add simple centos7 filestore dmcrypt plain support
Alfredo Deza [Fri, 2 Feb 2018 14:10:12 +0000 (09:10 -0500)]
ceph-volume tests.functional add simple centos7 filestore dmcrypt plain support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4c26162002709ccdd1b48c05126befdab7f269da)

7 years agoceph-volume tests.functional add simple centos7 filestore dmcrypt luks support
Alfredo Deza [Fri, 2 Feb 2018 14:09:57 +0000 (09:09 -0500)]
ceph-volume tests.functional add simple centos7 filestore dmcrypt luks support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a911d7c6a99e2ea235400614568d9cd362470cac)

7 years agoceph-volume tests.functional add simple centos7 bluestore dmcrypt plain support
Alfredo Deza [Fri, 2 Feb 2018 14:09:41 +0000 (09:09 -0500)]
ceph-volume tests.functional add simple centos7 bluestore dmcrypt plain support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8e0213ab053e80038bdae8d1b06d448fd397e3bc)

7 years agoceph-volume tests.functional add simple centos7 bluestore dmcrypt luks support
Alfredo Deza [Fri, 2 Feb 2018 14:09:21 +0000 (09:09 -0500)]
ceph-volume tests.functional add simple centos7 bluestore dmcrypt luks support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 76151c461bfe1e5ae4dc277c9d9357efb81e8c92)

7 years agoceph-volume tests.functional add simple bluestore dmcrypt luks support
Alfredo Deza [Fri, 2 Feb 2018 14:08:23 +0000 (09:08 -0500)]
ceph-volume tests.functional add simple bluestore dmcrypt luks support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d6e24022e8dfe845823fa0e87f3fba028c4b25b5)

7 years agodoc/ceph-volume lvm prepare fully supports encryption now
Alfredo Deza [Fri, 2 Feb 2018 13:36:45 +0000 (08:36 -0500)]
doc/ceph-volume lvm prepare fully supports encryption now

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0b2075759e8788b2c4ae46f94fdee953cc82b21e)

7 years agodoc/ceph-volume scan update for encryption support
Alfredo Deza [Fri, 2 Feb 2018 13:36:02 +0000 (08:36 -0500)]
doc/ceph-volume scan update for encryption support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e0d3bb82255a937cd937d33f966979fb541d1443)

7 years agodoc/ceph-volume remove notice that dmcrypt is not supported
Alfredo Deza [Fri, 2 Feb 2018 13:28:40 +0000 (08:28 -0500)]
doc/ceph-volume remove notice that dmcrypt is not supported

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a7d11ca27df1d22c8109cad5bf7be8fd23395c71)

7 years agodoc/man/ceph-volume add simple documentation
Alfredo Deza [Fri, 2 Feb 2018 13:26:58 +0000 (08:26 -0500)]
doc/man/ceph-volume add simple documentation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9524021aa15280d98f2b04d1fd66552733adf8a2)

7 years agoceph-volume simple.scan update help menu to indicate device support
Alfredo Deza [Fri, 2 Feb 2018 13:15:36 +0000 (08:15 -0500)]
ceph-volume simple.scan update help menu to indicate device support

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 09f35cb249674eda044b8fceea6f402ffac7731d)

7 years agoceph-volume tests for keyring parsing
Alfredo Deza [Fri, 2 Feb 2018 13:13:12 +0000 (08:13 -0500)]
ceph-volume tests for keyring parsing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b90c1a86843d8f0c8c7fb68fe059bdd72f3730fd)

7 years agoceph-volume tests for validate_devices
Alfredo Deza [Fri, 2 Feb 2018 13:12:51 +0000 (08:12 -0500)]
ceph-volume tests for validate_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bece7af0523b04d39c26011990cb1fe323aecdd8)

7 years agoceph-volume simple.activate b64decode keys for activation as well
Alfredo Deza [Fri, 2 Feb 2018 13:12:05 +0000 (08:12 -0500)]
ceph-volume simple.activate b64decode keys for activation as well

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 636ebc127797c8020cfb7aff09955b72beac8bdf)

7 years agoceph-volume tests add validation for lsblk parsers
Alfredo Deza [Thu, 1 Feb 2018 21:17:19 +0000 (16:17 -0500)]
ceph-volume tests add validation for lsblk parsers

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 03102e4417194e16deb8b738fcdaf87e699e6073)

7 years agoceph-volume tests validate parsing of cryptsetup
Alfredo Deza [Thu, 1 Feb 2018 20:51:41 +0000 (15:51 -0500)]
ceph-volume tests validate parsing of cryptsetup

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cce6a12f4a0d8d9a3d7dcb509b42ed2536d9257f)

7 years agoceph-volume tests add a stub for process.call
Alfredo Deza [Thu, 1 Feb 2018 20:51:12 +0000 (15:51 -0500)]
ceph-volume tests add a stub for process.call

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b80b61b90f8539b363c03c96e63f31c38c707fc8)

7 years agoceph-volume util.encryption add notes about extra b64decode call for ceph-disk
Alfredo Deza [Thu, 1 Feb 2018 20:35:11 +0000 (15:35 -0500)]
ceph-volume util.encryption add notes about extra b64decode call for ceph-disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c2367ef4a4db1e9c62e33dc211474f652d6bc6af)

7 years agoceph-volume util.encryption parse legacy encrypted with dirs too
Alfredo Deza [Thu, 1 Feb 2018 20:34:09 +0000 (15:34 -0500)]
ceph-volume util.encryption parse legacy encrypted with dirs too

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 32cb810f8ab7e4a710bfb2590a065dea41369fce)

7 years agoceph-volume terminal create a logger to get terminal+log messages in one call
Alfredo Deza [Thu, 1 Feb 2018 20:32:54 +0000 (15:32 -0500)]
ceph-volume terminal create a logger to get terminal+log messages in one call

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f6dd0ff15f0aeda9f660e923153f1d3dae68e583)

7 years agoceph-volume simple.activate support dmcrypted devices for both plain and luks
Alfredo Deza [Thu, 1 Feb 2018 20:32:16 +0000 (15:32 -0500)]
ceph-volume simple.activate support dmcrypted devices for both plain and luks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5ece73b533ee4b577c0ce61e3685a880e1de4851)

7 years agoceph-volume simple.scan parse the keyring out of the keyring file
Alfredo Deza [Thu, 1 Feb 2018 20:31:28 +0000 (15:31 -0500)]
ceph-volume simple.scan parse the keyring out of the keyring file

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a1124362816cbaaad7255b7ffc3698b68601ffe5)

Conflicts:
src/ceph-volume/ceph_volume/devices/simple/scan.py

7 years agoceph-volume simple.scan support dmcrypt OSDs when scanning
Alfredo Deza [Tue, 30 Jan 2018 21:23:06 +0000 (16:23 -0500)]
ceph-volume simple.scan support dmcrypt OSDs when scanning

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6c87d66d76ee1fc70f8dc6211d89dbd489c6ed3a)

7 years agoceph-volume util.encryption add a utilty to open plain encrypted devices
Alfredo Deza [Tue, 30 Jan 2018 14:35:53 +0000 (09:35 -0500)]
ceph-volume util.encryption add a utilty to open plain encrypted devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e5759e4fc9485e92a2faec277c8a920295751844)

7 years agoceph-volume util.system tmp mounts can now remove a dmcrypt mapper
Alfredo Deza [Tue, 30 Jan 2018 14:34:53 +0000 (09:34 -0500)]
ceph-volume util.system tmp mounts can now remove a dmcrypt mapper

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 60e8699650a491013d51239b9dc58160946d3bf9)

7 years agoceph-volume util.system do not 'translate' using realpath
Alfredo Deza [Tue, 30 Jan 2018 14:22:19 +0000 (09:22 -0500)]
ceph-volume util.system do not 'translate' using realpath

Using realpath always means that device mapper paths, like for dmcrypt
or LVM will get mangled and will return something like `/dev/dm-1` which
is not useful for anything in ceph-volume

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 469d01dc1aee8d6528944ef0acec58df868a9da7)

7 years agoceph-volume util.disk add a PART_ENTRY_TYPE detection utility
Alfredo Deza [Tue, 30 Jan 2018 14:19:06 +0000 (09:19 -0500)]
ceph-volume util.disk add a PART_ENTRY_TYPE detection utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cdb812438d6064089bac964cf38817d877afca38)

7 years agoceph-volume util.encryption add helpers for legacy devices
Alfredo Deza [Fri, 26 Jan 2018 15:44:14 +0000 (10:44 -0500)]
ceph-volume util.encryption add helpers for legacy devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d59b087e50e4e2076c59f6316cc89eb2a070aa52)

7 years agoceph-volume util.constants add ceph-disk partition labels
Alfredo Deza [Fri, 26 Jan 2018 15:25:31 +0000 (10:25 -0500)]
ceph-volume util.constants add ceph-disk partition labels

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 548a74bd239abe6b1b7a5a27fb2779800cfa7ac1)

7 years agoceph-volume util.disk support PKNAME and absolute paths in lsblk
Alfredo Deza [Thu, 25 Jan 2018 16:04:15 +0000 (11:04 -0500)]
ceph-volume util.disk support PKNAME and absolute paths in lsblk

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b90044c2ffff38d0dc36d598d237fcb066efe400)

7 years agoMerge pull request #20323 from ceph/backport-20203
Alfredo Deza [Wed, 7 Feb 2018 13:19:29 +0000 (08:19 -0500)]
Merge pull request #20323 from ceph/backport-20203

luminous: ceph-volume: fix usage of the --osd-id flag

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #20322 from ceph/backport-20059
Alfredo Deza [Wed, 7 Feb 2018 13:18:32 +0000 (08:18 -0500)]
Merge pull request #20322 from ceph/backport-20059

luminous: doc/ceph-volume OSD use the fsid file, not the osd_fsid

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #20162 from tchaikov/wip-pr-20130-luminous
Yuri Weinstein [Tue, 6 Feb 2018 22:21:16 +0000 (14:21 -0800)]
Merge pull request #20162 from tchaikov/wip-pr-20130-luminous

mgr: balancer: fixed mistype "AttributeError: 'Logger' object has no ...

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #20325 from jcsp/wip-22851-luminous
Yuri Weinstein [Tue, 6 Feb 2018 22:20:18 +0000 (14:20 -0800)]
Merge pull request #20325 from jcsp/wip-22851-luminous

luminous: pybind/mgr/dashboard: fix duplicated slash in html href

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #20326 from theanalyst/wip-22892-luminous
Yuri Weinstein [Tue, 6 Feb 2018 22:19:14 +0000 (14:19 -0800)]
Merge pull request #20326 from theanalyst/wip-22892-luminous

 luminous: _read_bdev_label unable to decode label at offset

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agomds: properly execute scrub finish context 20341/head
Yan, Zheng [Sun, 19 Nov 2017 12:25:22 +0000 (20:25 +0800)]
mds: properly execute scrub finish context

Bug was introduced by commit 7e52729699 (mds: flush after scrub repairs)

Fixes: http://tracker.ceph.com/issues/22058
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit ff71fbdcbb87ffc9baa02319ff0f336c173c5616)

7 years agomds: clean up clog damage message when repairing
John Spray [Wed, 25 Oct 2017 13:39:15 +0000 (09:39 -0400)]
mds: clean up clog damage message when repairing

If we've repaired then we don't need to be at WRN
and we don't need to advise the user to look up
the damage detail.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 3160ef2c0a66324621a95926bed0416bbd537353)

7 years agomds: flush after scrub repairs
John Spray [Wed, 25 Oct 2017 10:30:57 +0000 (06:30 -0400)]
mds: flush after scrub repairs

Otherwise, if we restart the MDS right after the scrub,
then scrub again, it will see the same inconsistency when
it looks at the on-disk state.

This involves adapting the use of ScrubHeader to be something
writeable during scrub to record whether repair happened.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7e52729699e87dd75022433153450fd4ba71b40e)

7 years agomds: don't report repaired backtraces in damagetable
John Spray [Wed, 25 Oct 2017 09:24:52 +0000 (05:24 -0400)]
mds: don't report repaired backtraces in damagetable

Fixes: http://tracker.ceph.com/issues/18743
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 444382c3370bafa21564bb1e6a5c80f3e4825f6a)

7 years agomds: cleaner scrub complete log msg
John Spray [Wed, 25 Oct 2017 09:24:03 +0000 (05:24 -0400)]
mds: cleaner scrub complete log msg

People often are not using a tag with their scrub,
let's not have the ugly "with tag ''" in that case.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5fd00a6e1b914880ab9f401faf591e73024f18ef)

7 years agomds: prevent filelock from being stuck at XSYN state 20340/head
Yan, Zheng [Tue, 7 Nov 2017 04:39:49 +0000 (12:39 +0800)]
mds: prevent filelock from being stuck at XSYN state

Lock:file_eval() does not properly handle the condition: filelock
is in XSYN state, both loner_cap and want_loner_cap are >= 0, and
loner_cap != want_loner_cap.

To set loner to the wanted one, mds need to revoke caps from the old
loner. Changing lock state to MIX can do the job.

Fixes: http://tracker.ceph.com/issues/22008
Signed-off-by: "Yan, Zheng" <zyan@redhat.com
(cherry picked from commit 67a428741fc82eb5f4120042197acf0cec34213a)

7 years agomds: implement LOCK_XSYN_LOCK and LOCK_XSYN_MIX
Yan, Zheng [Tue, 7 Nov 2017 02:50:14 +0000 (10:50 +0800)]
mds: implement LOCK_XSYN_LOCK and LOCK_XSYN_MIX

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c7261e1a6d95d1ed98a1d9066905aa75af4c2886)

7 years agomds: cleanup set/drop loner code
Yan, Zheng [Tue, 7 Nov 2017 02:37:44 +0000 (10:37 +0800)]
mds: cleanup set/drop loner code

integrate functionalities of try_drop_loner and try_set_loner into
CInode::choose_ideal_loner

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d4cbf3aeeada57a88b794d3d6d5fb1cd891e31cd)

7 years agoMerge pull request #19157 from pdvian/wip-22237-luminous
Yuri Weinstein [Tue, 6 Feb 2018 17:40:01 +0000 (09:40 -0800)]
Merge pull request #19157 from pdvian/wip-22237-luminous

luminous: mds: don't delay processing completed requests in replay queue

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #20327 from theanalyst/wip-22922-luminous
Yuri Weinstein [Tue, 6 Feb 2018 17:37:50 +0000 (09:37 -0800)]
Merge pull request #20327 from theanalyst/wip-22922-luminous

luminous: rgw: bucket resharding should not update bucket ACL or user stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agomds: limit size of MExportDir message 20339/head
Yan, Zheng [Fri, 3 Nov 2017 06:14:07 +0000 (14:14 +0800)]
mds: limit size of MExportDir message

Only export port of subtree if whole subtree tree is too large for
sinlge MExportDir message. The un-exported portion are added to
export queue.

Fixes: http://tracker.ceph.com/issues/21892
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit ca5c987860821a7784db1c6a74d1c661cdf427cc)

7 years agomds: optimize MDCache::try_subtree_merge
Yan, Zheng [Fri, 3 Nov 2017 08:13:32 +0000 (16:13 +0800)]
mds: optimize MDCache::try_subtree_merge

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 1f27ebbef95b39f7fa3c94d5dc9f191a58c9f9b4)

7 years agomds: optimize import/export state access
Yan, Zheng [Fri, 3 Nov 2017 04:03:44 +0000 (12:03 +0800)]
mds: optimize import/export state access

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit fb9277c14314aaa6a97ee23da9057ddc3c750408)

7 years agorgw: fix for empty query string in beast frontend 20338/head
Casey Bodley [Thu, 25 Jan 2018 17:09:20 +0000 (12:09 -0500)]
rgw: fix for empty query string in beast frontend

when the target does not contain a ?, the QUERY_STRING was being set to
the same value as REQUEST_URI. this QUERY_STRING is included in the
signature, and caused SignatureDoesNotMatch failures

Fixes: http://tracker.ceph.com/issues/22797
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ab9e79684ac7ae33b78522df6732b99271d10016)

7 years agoos/bluestore: propagate read-EIO for aio 20337/head
xie xingguo [Fri, 15 Sep 2017 14:28:40 +0000 (22:28 +0800)]
os/bluestore: propagate read-EIO for aio

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 301912603789d78e3560da84df1e337edd046e46)

7 years agoos/bluestore: report error and quit correctly when disk error happens.
Pan Liu [Wed, 6 Sep 2017 10:09:55 +0000 (18:09 +0800)]
os/bluestore: report error and quit correctly when disk error happens.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
(cherry picked from commit ed3175895fa92929089f7507e7d173f557d61e27)

7 years agoos/filestore: use __func__ instead of hardcode function name
Pan Liu [Wed, 6 Sep 2017 10:06:06 +0000 (18:06 +0800)]
os/filestore: use __func__ instead of hardcode function name

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
(cherry picked from commit fdb4048a92fad40beab1350a68b29681bd4b6131)

7 years agoos/bluestore: add "bluestore_read_eio" counter
xie xingguo [Fri, 15 Sep 2017 06:30:47 +0000 (14:30 +0800)]
os/bluestore: add "bluestore_read_eio" counter

To track down how many EIO errors have been propagated to
high level callers.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit d2d989fcc6a194b9374aeea64e45969b08990262)

7 years agoos/bluestore: propagate read-EIO to high level callers
xie xingguo [Fri, 15 Sep 2017 02:30:04 +0000 (10:30 +0800)]
os/bluestore: propagate read-EIO to high level callers

E.g., we can let auto-repair to properly handle this
instead of crashing the whole osd.

Observe this once on one of our test cluster:

/clove/vm/clove/ceph/rpmbuild/BUILD/ceph-12.2.0/src/os/bluestore/BlueStore.cc: 6604: FAILED assert(r == 0)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a51d9e979fbbe49917712889efb102806479fad2)

7 years agorgw: bucket resharding should not update bucket ACL or user stats 20327/head
Orit Wasserman [Sun, 21 Jan 2018 08:12:43 +0000 (10:12 +0200)]
rgw: bucket resharding should not update bucket ACL or user stats

bucket link op resets bucket ACL using rgw_link_bucket instead

Fixes: http://tracker.ceph.com/issues/22124
Fixes: http://tracker.ceph.com/issues/22742
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 1223baf76b45559f171c67c5e737f8e5f7d843e0)

7 years agoos/bluestore: change bdev parse error to ENOENT 20326/head
Sage Weil [Wed, 24 Jan 2018 03:07:08 +0000 (21:07 -0600)]
os/bluestore: change bdev parse error to ENOENT

If there is not a valid label, then the label is not found.  This is a
more reasonable error code than "Invalid argumnet".

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5cd98b0dfdc7758761ff789b12faca2266a7e128)

7 years agoos/bluestore: make bdev label parse error less noisy
Sage Weil [Wed, 24 Jan 2018 03:06:27 +0000 (21:06 -0600)]
os/bluestore: make bdev label parse error less noisy

This happens during the normal initialization of a new bluestore osd and it
is confusing for users.  Make it less noisy.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 14c498dd22d1e89d30a0896e89fe83e8adf6ac76)

7 years agoceph-volume: adds tests for zap --destroy and --osd-id 20323/head
Andrew Schoen [Wed, 31 Jan 2018 21:45:44 +0000 (15:45 -0600)]
ceph-volume: adds tests for zap --destroy and --osd-id

These are not added for dmcrypt right now because there is a bug related
to zapping encrypted disks that needs fixed first.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 8d38932c211c589b6171d9e17b100869c23890ac)

7 years agoceph-volume: no need to return osd_id from util.prepare.check_id
Andrew Schoen [Wed, 31 Jan 2018 15:04:09 +0000 (09:04 -0600)]
ceph-volume: no need to return osd_id from util.prepare.check_id

Now that osd_id is passed to util.prepare.create_id it doesn't
make any sense to return osd_id from check_id anymore as it's
not being used.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f88f2d3bc1f5dcae1e9b2be62dab20942374fc29)

7 years agoceph-volume: when reusing an osd ID you must give that id to 'osd new'
Andrew Schoen [Tue, 30 Jan 2018 21:00:04 +0000 (15:00 -0600)]
ceph-volume: when reusing an osd ID you must give that id to 'osd new'

If you do not then auth will not be created for the new OSD and the
daemon will not be able to start.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 43f699fcbc9f30875f70295e086a4e423008b266)

7 years agoceph-volume: prepare_* methods do not need to recreate fsid or osd_id
Andrew Schoen [Tue, 30 Jan 2018 17:39:05 +0000 (11:39 -0600)]
ceph-volume: prepare_* methods do not need to recreate fsid or osd_id

The prepare_filestore and prepare_bluestore methods do not need to
attempt to recreate osd_id or fsid because this happens in prepare
already.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7c809166aed82298b1fe1581f0e28174a535574f)

7 years agoceph_volume: modify util.prepare.check_id to handle stdout as a list
Andrew Schoen [Tue, 30 Jan 2018 17:32:53 +0000 (11:32 -0600)]
ceph_volume: modify util.prepare.check_id to handle stdout as a list

process.call returns stdout as a list, not a string

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a08fc639613b3d9bcb3a09875e535287dd58870a)

7 years agoceph-volume: tests for util.prepare.check_id
Andrew Schoen [Tue, 30 Jan 2018 15:22:58 +0000 (09:22 -0600)]
ceph-volume: tests for util.prepare.check_id

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0d27fff319f1c4705648fdaff18ee47441f26a24)

7 years agoceph-volume: return osd_id from util.prepare.check_id if it exists
Andrew Schoen [Tue, 30 Jan 2018 15:21:47 +0000 (09:21 -0600)]
ceph-volume: return osd_id from util.prepare.check_id if it exists

This also changes this so the osd_id is returned as a string so
an ID of 0 would evaluate to True.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 699aa65878ca18f2e2cd70d7444cad18b4a3fd98)

7 years agoceph-volume: check to see if an OSD id exists before reusing it
Andrew Schoen [Mon, 29 Jan 2018 16:48:09 +0000 (10:48 -0600)]
ceph-volume: check to see if an OSD id exists before reusing it

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 548eadeec6fb42a18c3e84dbbd8164862f8836e6)

7 years agoceph-volume: adds a prepare util for checking OSD ID existance
Andrew Schoen [Mon, 29 Jan 2018 16:43:04 +0000 (10:43 -0600)]
ceph-volume: adds a prepare util for checking OSD ID existance

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 4ac99de6d98b646bf160230584f6532775cdc9cb)

7 years agopybind/mgr/dashboard: fix duplicated slash in html href 20325/head
Shengjing Zhu [Thu, 1 Feb 2018 08:47:26 +0000 (16:47 +0800)]
pybind/mgr/dashboard: fix duplicated slash in html href

The url_prefix val in html templates is expected not containing '/'
in the end. Otherwise the href will be rendered as '//static/', which
causes browser treating it as http://static/.

Fixes: http://tracker.ceph.com/issues/22851
Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>
(cherry picked from commit 24fe1cae636bf00e3d5923c7e826992d42d92946)

7 years agodoc/ceph-volume OSD use the fsid file, not the osd_fsid 20322/head
Alfredo Deza [Mon, 22 Jan 2018 20:16:40 +0000 (15:16 -0500)]
doc/ceph-volume OSD use the fsid file, not the osd_fsid

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a5f26c622b0f11b7d8179da22d18f719b9febe0a)

7 years agoMerge pull request #20241 from ceph/backport-wip-rm22619
Andrew Schoen [Mon, 5 Feb 2018 15:08:27 +0000 (16:08 +0100)]
Merge pull request #20241 from ceph/backport-wip-rm22619

luminous ceph-volume: dmcrypt support for lvm

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoAvoid the assert failure when the inode for the cap_export from other MDS happened... 20300/head
Jianyu Li [Thu, 4 Jan 2018 16:54:30 +0000 (00:54 +0800)]
Avoid the assert failure when the inode for the cap_export from other MDS happened not in MDCache

Signed-off-by: Jianyu Li <joannyli@tencent.com>
(cherry picked from commit 82b9b838ae0bd17247f6296f9d7001331f50663c)

7 years agomds: set higher priority for some perf counters 20299/head
Shangzhong Zhu [Tue, 23 Jan 2018 12:35:36 +0000 (20:35 +0800)]
mds: set higher priority for some perf counters
Fixes: http://tracker.ceph.com/issues/22776
Signed-off-by: Shangzhong Zhu <zhu.shangzhong@zte.com.cn>
(cherry picked from commit 05c496d444b388ec505df4598abd3fc6309ed99f)