]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoqa: fix damage expectation setting 26047/head
Patrick Donnelly [Wed, 9 Jan 2019 00:26:14 +0000 (16:26 -0800)]
qa: fix damage expectation setting

The purge queue expectation was being ignored.

Fixes: http://tracker.ceph.com/issues/37837
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 360550b1ab143bb59fd38c77569f7f5ef762801a)

6 years agoqa: fix loop variable reference
Patrick Donnelly [Tue, 8 Jan 2019 23:51:53 +0000 (15:51 -0800)]
qa: fix loop variable reference

Otherwise the Mutation for Truncate is done on obj_id of the last iteration of the previous loop.

Fixes: http://tracker.ceph.com/issues/37836
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 909d0f1333a730c648c08cda047bb5f356a61986)

6 years agoMerge pull request #25554 from jan--f/25238-mimic
Alfredo Deza [Thu, 17 Jan 2019 17:05:55 +0000 (12:05 -0500)]
Merge pull request #25554 from jan--f/25238-mimic

mimic ceph-volume: introduce class hierachy for strategies

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume lvm.strategies remove unused import 25554/head
Alfredo Deza [Mon, 17 Dec 2018 18:34:58 +0000 (13:34 -0500)]
ceph-volume lvm.strategies remove unused import

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d127ae3e5901c6477ed44f4d50be834040c44b6f)

  Conflicts:
  src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py
          resolved by removing str_to_int import

6 years agoceph-volume: filestore strategy use strategy class hierarchy
Jan Fajerski [Mon, 26 Nov 2018 13:03:56 +0000 (14:03 +0100)]
ceph-volume: filestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 34df12ea6a8686f161b4bdfde92e557743f59cf4)

6 years agoceph-volume: bluestore strategy use strategy class hierarchy
Jan Fajerski [Mon, 26 Nov 2018 13:03:03 +0000 (14:03 +0100)]
ceph-volume: bluestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 302334ac6a4375125b6c1e68bf7160f8bc0597c2)

6 years agoceph-volume: add strategies.py to for shared code
Jan Fajerski [Mon, 26 Nov 2018 13:01:57 +0000 (14:01 +0100)]
ceph-volume: add strategies.py to for shared code

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b38987ee9652527e4845f2beb6406cbd398dac9b)

6 years agoMerge pull request #25923 from alfredodeza/mimic-ceph-volume-fix-json
Andrew Schoen [Fri, 11 Jan 2019 21:41:16 +0000 (15:41 -0600)]
Merge pull request #25923 from alfredodeza/mimic-ceph-volume-fix-json

mimic ceph-volume: fix JSON output in `inventory`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: fix JSON output in `inventory` 25923/head
Sebastian Wagner [Thu, 22 Nov 2018 17:01:50 +0000 (18:01 +0100)]
ceph-volume: fix JSON output in `inventory`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a3e6f569b4fa0419dff4690a72e9be6fe0a255c1)

6 years agoMerge pull request #25819 from batrick/mimic-pr24562
Nathan Cutler [Fri, 11 Jan 2019 12:18:35 +0000 (13:18 +0100)]
Merge pull request #25819 from batrick/mimic-pr24562

doc: removed warning for resolved errata

Reviewed-by: John Wilkins <jowilkin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25868 from dillaman/wip-37842-mimic
Jason Dillaman [Wed, 9 Jan 2019 22:17:20 +0000 (17:17 -0500)]
Merge pull request #25868 from dillaman/wip-37842-mimic

mimic: test/pybind/test_rbd: filter out unknown list_children2 keys

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agotest/pybind/test_rbd: filter out unknown list_children2 keys 25868/head
Mykola Golub [Tue, 8 Jan 2019 10:49:55 +0000 (12:49 +0200)]
test/pybind/test_rbd: filter out unknown list_children2 keys

Fixes: http://tracker.ceph.com/issues/37729
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 24cc60335d86ff625e0c5d98cab0c4449a24f296)

6 years agoMerge pull request #25837 from alfredodeza/mimic-rm37805
Andrew Schoen [Tue, 8 Jan 2019 20:18:29 +0000 (14:18 -0600)]
Merge pull request #25837 from alfredodeza/mimic-rm37805

mimic ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25775 from alfredodeza/mimic-rm37442
Andrew Schoen [Tue, 8 Jan 2019 20:17:24 +0000 (14:17 -0600)]
Merge pull request #25775 from alfredodeza/mimic-rm37442

mimic ceph-volume normalize comma to dot for string to int conversions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests/functional declare ceph-ansible roles instead of importing them 25837/head
Alfredo Deza [Mon, 7 Jan 2019 20:15:21 +0000 (15:15 -0500)]
ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad6b924e2bf3b06ec66eb5bc4fea065f5babc512)

6 years agoceph-volume tests.util ensure ints and strings with commas can be converted to ints 25775/head
Alfredo Deza [Wed, 2 Jan 2019 19:18:14 +0000 (14:18 -0500)]
ceph-volume tests.util ensure ints and strings with commas can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 91bc3a1479c550be7b48128d5755228a2007e4b0)

6 years agoceph-volume test verify strings with a comma can be converted to ints
Alfredo Deza [Fri, 21 Dec 2018 20:26:39 +0000 (15:26 -0500)]
ceph-volume test verify strings with a comma can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 818902e568adaefecc6da9afee58b3a804a0e4d0)

6 years agoceph-volume util normalize comma with dot for str-to-int conversion
Alfredo Deza [Fri, 21 Dec 2018 20:24:25 +0000 (15:24 -0500)]
ceph-volume util normalize comma with dot for str-to-int conversion

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit aa5323fbcfdc6c21177cb800884712b0252b84a0)

6 years agoMerge pull request #25777 from alfredodeza/mimic-rm37486
Andrew Schoen [Tue, 8 Jan 2019 15:39:24 +0000 (09:39 -0600)]
Merge pull request #25777 from alfredodeza/mimic-rm37486

mimic ceph-volume: set permissions right before prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.functional fix typo when stopping osd.0 in filestore 25777/head
Alfredo Deza [Mon, 17 Dec 2018 17:35:27 +0000 (12:35 -0500)]
ceph-volume tests.functional fix typo when stopping osd.0 in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fba43289a2ff8a94361cc016e7a4dfdc61f324e8)

6 years agoceph-volume tests.functional.lvm stop all osds before activating again to test permis...
Alfredo Deza [Thu, 13 Dec 2018 14:49:41 +0000 (09:49 -0500)]
ceph-volume tests.functional.lvm stop all osds before activating again to test permissions

Otherwise, ``ceph-volume lvm activate --all`` will skip OSDs that are
already running. So the test that verifies that permissions are
corrected will fail because ``activate`` will simply skip them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9be6b15d3316c8eab159bc5d374e95e23d082dde)

6 years agoceph-volume lvm.activate ensure filestore osd path is chowned always
Alfredo Deza [Thu, 13 Dec 2018 12:45:00 +0000 (07:45 -0500)]
ceph-volume lvm.activate ensure filestore osd path is chowned always

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d6c32b5db0627d95c2ffb9428e86271daa4a336)

6 years agoceph-volume: include the ceph-facts role when needed during testing
Andrew Schoen [Wed, 12 Dec 2018 20:10:59 +0000 (14:10 -0600)]
ceph-volume: include the ceph-facts role when needed during testing

The ceph-ansible stable-3.2 branch does not have the ceph-facts role
but master does. We need to make sure to import this role only when
it's supported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 569b4fb7b3894275a3b9f4ab603e32099301f9aa)

6 years agoceph-volume tests.functional set root:root in OSDs in filestore
Alfredo Deza [Mon, 10 Dec 2018 19:52:03 +0000 (14:52 -0500)]
ceph-volume tests.functional set root:root in OSDs in filestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 503396f685f6a93e0c63e97477677b7145f96c26)

6 years agoceph-volume tests.functional set root:root in OSDs in bluestore
Alfredo Deza [Mon, 10 Dec 2018 19:38:02 +0000 (14:38 -0500)]
ceph-volume tests.functional set root:root in OSDs in bluestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 98631fdeeb6dbe765d3eb80c5c1800946735f2a2)

6 years agoceph-volume lvm.activate always chown osd path before osd-prime-dir in bluestore
Alfredo Deza [Mon, 10 Dec 2018 18:27:20 +0000 (13:27 -0500)]
ceph-volume lvm.activate always chown osd path before osd-prime-dir in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0af31ec24533091aa4991b2343fe80da94777b4d)

6 years agoremoved warning for resolved errata 25819/head
Scoots Hamilton [Mon, 15 Oct 2018 15:18:43 +0000 (11:18 -0400)]
removed warning for resolved errata

Signed-off-by: Scoots Hamilton <scoots@redhat.com>
(cherry picked from commit 93528996189f2eaf42bae18db428f3ea18f28b14)

Conflicts:
doc/cephfs/cache-size-limits.rst

6 years agoMerge pull request #25749 from alfredodeza/mimic-cryptsetup-allow-discards
Andrew Schoen [Mon, 7 Jan 2019 19:56:38 +0000 (13:56 -0600)]
Merge pull request #25749 from alfredodeza/mimic-cryptsetup-allow-discards

mimic ceph-volume: enable device discards

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25551 from jan--f/25203-mimic
Alfredo Deza [Mon, 7 Jan 2019 12:36:05 +0000 (07:36 -0500)]
Merge pull request #25551 from jan--f/25203-mimic

mimic ceph-volume: fix Batch object in py3 environments

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PRs after 13.2.3 and before 13.2.4 back into mimic branch
David Galloway [Fri, 4 Jan 2019 18:40:42 +0000 (13:40 -0500)]
Merge PRs after 13.2.3 and before 13.2.4 back into mimic branch

6 years ago13.2.4 v13.2.4
Jenkins Build Slave User [Fri, 4 Jan 2019 15:40:42 +0000 (15:40 +0000)]
13.2.4

6 years agorgw: fix issues with 'enforce bounds' patch
Joao Eduardo Luis [Thu, 29 Nov 2018 01:05:31 +0000 (01:05 +0000)]
rgw: fix issues with 'enforce bounds' patch

The patch to enforce bounds on max-keys/max-uploads/max-parts had a few
issues that would prevent us from compiling it. Instead of changing the
code provided by the submitter, we're addressing them in a separate
commit to maintain the DCO.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 29bc434a6a81a2e5c5b8cfc4c8d5c82ca5bf538a)
mimic specific fixes:
As the largeish change from master g_conf() isn't in mimic yet, use the g_conf
global structure, also make rgw_op use the value from req_info ceph context as
we do for all the requests

6 years agorgw: enforce bounds on max-keys/max-uploads/max-parts
Robin H. Johnson [Fri, 21 Sep 2018 21:49:34 +0000 (14:49 -0700)]
rgw: enforce bounds on max-keys/max-uploads/max-parts

RGW S3 listing operations provided a way for authenticated users to
cause a denial of service against OMAPs holding bucket indices.

Bound the min & max values that a user could pass into the max-X
parameters, to keep the system safe. The default of 1000 is chosen to
match AWS S3 behavior.

Affected operations:
- ListBucket, via max-keys
- ListBucketVersions, via max-keys
- ListBucketMultiPartUploads, via max-uploads
- ListMultipartUploadParts, via max-parts

The Swift bucket listing codepath already enforced a limit, so is
unaffected by this issue.

Prior to this commit, the effective limit is the lower of
osd_max_omap_entries_per_request or osd_max_omap_bytes_per_request.

Backport: luminous, mimic
Fixes: http://tracker.ceph.com/issues/35994
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit d79f68a1e31f4bc917eec1b6bbc8e8446377dc6b)

 Conflicts:
src/common/options.cc:
  Conflicts due to options from master

6 years agomon/config-key: limit caps allowed to access the store
Joao Eduardo Luis [Wed, 17 Oct 2018 13:42:15 +0000 (14:42 +0100)]
mon/config-key: limit caps allowed to access the store

Henceforth, we'll require explicit `allow` caps for commands, or for the
config-key service. Blanket caps are no longer allowed for the
config-key service, except for 'allow *'.

(for luminous and mimic, we're also ensuring MonCap's parser is able to
understand forward slashes '/' when parsing prefixes)

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 5fff611041c5afeaf3c8eb09e4de0cc919d69237)

6 years agoMerge pull request #25291 from smithfarm/wip-37414-mimic
Yuri Weinstein [Thu, 3 Jan 2019 18:26:15 +0000 (10:26 -0800)]
Merge pull request #25291 from smithfarm/wip-37414-mimic

mimic: mgr/balancer: add crush_compat_metrics param to change optimization keys

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25771 from alfredodeza/mimic-ceph-volume-fix-join
Alfredo Deza [Thu, 3 Jan 2019 18:23:22 +0000 (13:23 -0500)]
Merge pull request #25771 from alfredodeza/mimic-ceph-volume-fix-join

mimic ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoceph-volume: Fix TypeError: join() takes exactly one argument (2 given) 25771/head
Sebastian Wagner [Mon, 10 Dec 2018 13:38:08 +0000 (14:38 +0100)]
ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 38f289b691ed1275bdf7342a4347434ed340c8fc)

6 years agoMerge pull request #25351 from alfredodeza/mimic-rm37449
Alfredo Deza [Thu, 3 Jan 2019 18:01:07 +0000 (13:01 -0500)]
Merge pull request #25351 from alfredodeza/mimic-rm37449

mimic ceph-volume  zap: improve zapping to remove all partitions and all LVs, encrypted or not

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24695 from vshankar/wip-35932
Yuri Weinstein [Thu, 3 Jan 2019 16:26:12 +0000 (08:26 -0800)]
Merge pull request #24695 from vshankar/wip-35932

mimic : client: retry remount on dcache invalidation failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25031 from smithfarm/wip-36576-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:25:35 +0000 (08:25 -0800)]
Merge pull request #25031 from smithfarm/wip-36576-mimic

mimic: cephfs: mgr/status: fix fs status subcommand did not show standby-replay MDS' perf info

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25198 from pdvian/wip-37285-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:24:02 +0000 (08:24 -0800)]
Merge pull request #25198 from pdvian/wip-37285-mimic

mimic: rgw: radosgw-admin: translate reshard status codes (trivial)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement
Yuri Weinstein [Thu, 3 Jan 2019 16:23:22 +0000 (08:23 -0800)]
Merge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement

mimic: qa/ceph-ansible: Move "branch" out of "vars" section

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #25218 from k0ste/mimic_backports
Yuri Weinstein [Thu, 3 Jan 2019 16:22:01 +0000 (08:22 -0800)]
Merge pull request #25218 from k0ste/mimic_backports

mimic: mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #25227 from pdvian/wip-37288-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:21:16 +0000 (08:21 -0800)]
Merge pull request #25227 from pdvian/wip-37288-mimic

mimic: tests: qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25242 from smithfarm/wip-37384-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:19:44 +0000 (08:19 -0800)]
Merge pull request #25242 from smithfarm/wip-37384-mimic

mimic: test: Start using GNU awk and fix archiving directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #25252 from pdvian/wip-37363-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:18:36 +0000 (08:18 -0800)]
Merge pull request #25252 from pdvian/wip-37363-mimic

mimic: librbd: fix missing unblock_writes if shrink is not allowed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25153 from jonsger/wip-24827-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:15:34 +0000 (08:15 -0800)]
Merge pull request #25153 from jonsger/wip-24827-mimic

mimic: build/ops: run-make-check.sh ccache tweaks

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25171 from pdvian/wip-36392-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:14:38 +0000 (08:14 -0800)]
Merge pull request #25171 from pdvian/wip-36392-mimic

mimic: rpm: Use hardened LDFLAGS

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25293 from smithfarm/wip-37419-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:13:35 +0000 (08:13 -0800)]
Merge pull request #25293 from smithfarm/wip-37419-mimic

mimic: mgr/balancer: add cmd to list all plans

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25295 from rzarzynski/wip-bug-36248-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:12:58 +0000 (08:12 -0800)]
Merge pull request #25295 from rzarzynski/wip-bug-36248-mimic

mimic: common: fix memory leaks in WeightedPriorityQueue.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25507 from smithfarm/wip-36729-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:11:42 +0000 (08:11 -0800)]
Merge pull request #25507 from smithfarm/wip-36729-mimic

mimic: osd: Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #25753 from alfredodeza/mimic-better-devid
Alfredo Deza [Thu, 3 Jan 2019 14:37:43 +0000 (09:37 -0500)]
Merge pull request #25753 from alfredodeza/mimic-better-devid

mimic common/blkdev, ceph-volume: improve get_device_id

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoceph-volume: Micron SSDs don't include vendor name in ID_SERIAL 25753/head
Sage Weil [Thu, 6 Dec 2018 16:02:35 +0000 (10:02 -0600)]
ceph-volume: Micron SSDs don't include vendor name in ID_SERIAL

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3e4a638c931a5198aca74dc4037d0b03d5e563ba)

6 years agoceph-volume: update get_device_id to match in-tree implementation
Sage Weil [Thu, 6 Dec 2018 15:43:36 +0000 (09:43 -0600)]
ceph-volume: update get_device_id to match in-tree implementation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c967dbf81f85602d2cd044ee5a91ba38cd76bdb9)

6 years agoceph-volume: enable device discards 25749/head
Jonas Jelten [Fri, 19 Oct 2018 13:15:03 +0000 (15:15 +0200)]
ceph-volume: enable device discards

When using SSDs as encrypted OSD device, discards do not pass the
encryption layer. This option activates discard requests.

Signed-off-by: Jonas Jelten <jj@stusta.net>
(cherry picked from commit 333330414730bf57c68efe6f1f13841b0a014dae)

6 years ago13.2.3 v13.2.3
Jenkins Build Slave User [Wed, 2 Jan 2019 18:00:57 +0000 (18:00 +0000)]
13.2.3

6 years agoceph-volume: add Batch smoketest 25551/head
Jan Fajerski [Mon, 26 Nov 2018 13:54:50 +0000 (14:54 +0100)]
ceph-volume: add Batch smoketest

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 936ee680f7633620c8a1127db9630a22326b9e2c)

6 years agoceph-volume: fix Batch object in py3 environments
Jan Fajerski [Fri, 30 Nov 2018 08:22:51 +0000 (09:22 +0100)]
ceph-volume: fix Batch object in py3 environments

Fixes: https://tracker.ceph.com/issues/37356
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 851c7693daa315b7972702bedf09dd66a37667d8)

6 years agoosd: combine recovery/scrub/snap sleep timer into one. 25507/head
Jianpeng Ma [Sat, 28 Apr 2018 02:46:55 +0000 (10:46 +0800)]
osd: combine recovery/scrub/snap sleep timer into one.

One SafeTime on thread. And those events if timer don't take long time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 610d5196c0132739e1ae133db0c7d896063470ad)

6 years agoMerge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes
Xie Xingguo [Thu, 13 Dec 2018 05:54:59 +0000 (13:54 +0800)]
Merge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes

mimic: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25285 from smithfarm/wip-37396-mimic
Yuri Weinstein [Thu, 13 Dec 2018 00:46:16 +0000 (16:46 -0800)]
Merge pull request #25285 from smithfarm/wip-37396-mimic

mimic: tests: /usr/bin/ld: cannot find -lradospp in rados mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>n
6 years agoosd: Add support for osd_delete_sleep to throttle pg deletion
David Zafman [Thu, 25 Oct 2018 05:00:57 +0000 (22:00 -0700)]
osd: Add support for osd_delete_sleep to throttle pg deletion

Fixes: http://tracker.ceph.com/issues/36474
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92f1f5859f142cb83cc1d1a367853ac91d6b01d4)

Conflicts:
    src/osd/OSD.cc
- use cct->_conf->get_val instead of cct->_conf.get_val

6 years agotest: Set any value for osd_delete_sleep to guarantee we are testing even SSD
David Zafman [Tue, 30 Oct 2018 18:42:05 +0000 (11:42 -0700)]
test: Set any value for osd_delete_sleep to guarantee we are testing even SSD

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f621a1190f45a780b71b916b31c1a01a20ece96)

6 years agoMerge pull request #25332 from pdvian/wip-36503-mimic
Yuri Weinstein [Wed, 12 Dec 2018 16:01:14 +0000 (08:01 -0800)]
Merge pull request #25332 from pdvian/wip-36503-mimic

mimic: tests: use timeout for fs asok operations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #25283 from jdurgin/wip-osd-memory-target-mimic
Josh Durgin [Mon, 10 Dec 2018 17:13:37 +0000 (12:13 -0500)]
Merge pull request #25283 from jdurgin/wip-osd-memory-target-mimic

mimic: core: os/bluestore: cache autotuning and memory limit

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25362 from noahdesu/mimic-docs
Noah Watkins [Fri, 7 Dec 2018 19:35:34 +0000 (11:35 -0800)]
Merge pull request #25362 from noahdesu/mimic-docs

mimic: docs: backport edit on github changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoosd/OSDMap: fix upmap mis-killing for erasure-coded PGs 25419/head
xie xingguo [Sat, 1 Dec 2018 09:42:01 +0000 (17:42 +0800)]
osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

The up-set of erasure-coded PGs may include CRUSH_ITEM_NONE,
which as a result causes mis-killing of valid upmap items.

Fixes: https://tracker.ceph.com/issues/37493
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: ningtao <ningtao@sangfor.com.cn>
(cherry picked from commit f043dcc6d8663ed960a354b2441fd3f8dd66c62d)

Conflicts:
        - *maybe_remove_pg_upmaps* input changed, in master
          we now have a passed in *nextmap* parameter

6 years agomon: auto clean up stale upmap items
xie xingguo [Wed, 21 Nov 2018 06:25:45 +0000 (14:25 +0800)]
mon: auto clean up stale upmap items

See https://www.spinics.net/lists/ceph-users/msg49518.html

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4181aa86906886423af80e478e45554606fc7cf6)

Conflicts:
        - *nextmap* has been renamed to *tmpmap*
        - add the *have_pg_upmaps* wrapper which is necessary for unittest

6 years agoosd/OSDMap: add pg-existence sanity check
xie xingguo [Wed, 21 Nov 2018 01:36:21 +0000 (09:36 +0800)]
osd/OSDMap: add pg-existence sanity check

The reason why __get_pg_pool_size(pg)__ or __get_pg_pool_crush_rule(pg)__ fails is
that the pg does not exist anymore. So it generally makes sense to check __pg_exists(pg)__
before moving further.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 79ef3bea7550559b7ea4e2b68dc407c7eb25defa)

Conflicts:
- *nextmap* has been renamed to *tmpmap*

6 years agoosd/OSDMap.cc: move sanity checks to the proper place
xie xingguo [Wed, 20 Jun 2018 01:27:46 +0000 (09:27 +0800)]
osd/OSDMap.cc: move sanity checks to the proper place

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3dcd4eaf867369437720a21daf3c344cc4b9844e)

6 years agoosd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone
xie xingguo [Wed, 20 Jun 2018 01:04:19 +0000 (09:04 +0800)]
osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone

If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.

The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:

```
src/osd/OSDMap.cc: 4078: FAILED assert(target > 0)
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8a11ed479d7ea95f76170e74123ca2c48e3a3963)

6 years agoMerge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id
Alfredo Deza [Tue, 4 Dec 2018 16:50:42 +0000 (11:50 -0500)]
Merge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id

mimic ceph-volume add device_id to inventory listing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agodoc/_templates/page.html: add link to report doc bugs 25362/head
Neha Ojha [Fri, 19 Oct 2018 20:26:53 +0000 (13:26 -0700)]
doc/_templates/page.html: add link to report doc bugs

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de30c4d3c5927bb58b49b44b391647e397e1fe4f)

6 years agoluminous: doc: show edit on github links and version warnings
Noah Watkins [Mon, 1 Oct 2018 23:54:19 +0000 (16:54 -0700)]
luminous: doc: show edit on github links and version warnings

backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.

this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
 Conflicts:
doc/conf.py

6 years agoceph-volume tests patch udevadm so it doesn't actually call udevadm in a test 25351/head
Alfredo Deza [Fri, 30 Nov 2018 17:31:02 +0000 (12:31 -0500)]
ceph-volume tests patch udevadm so it doesn't actually call udevadm in a test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f440fb96694e38f69e174c01f284ab9a2066bc8)

6 years agoceph-volume util.disk use device.path for error reporting
Alfredo Deza [Fri, 30 Nov 2018 13:27:01 +0000 (08:27 -0500)]
ceph-volume util.disk use device.path for error reporting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1670585ebed7c685b49a13c1066eedc1aa37d29b)

6 years agoceph-volume tests.functional add back sdc1 partitions with parted to redeploy after...
Alfredo Deza [Fri, 30 Nov 2018 11:44:56 +0000 (06:44 -0500)]
ceph-volume tests.functional add back sdc1 partitions with parted to redeploy after zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b0bc162b601c2dbc9827df0ab472ba2a1146076c)

6 years agoceph-volume tests.functional add back partitions with parted to redeploy after zapping
Alfredo Deza [Thu, 29 Nov 2018 23:07:09 +0000 (18:07 -0500)]
ceph-volume tests.functional add back partitions with parted to redeploy after zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 945e8f56ce21411f664a58147afb343f0671b591)

6 years agoceph-volume tests.functional recreate partitions previously removed with --destroy
Alfredo Deza [Thu, 29 Nov 2018 19:10:46 +0000 (14:10 -0500)]
ceph-volume tests.functional recreate partitions previously removed with --destroy

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e1d136ee3bcb2a7bbe0790b87e1d54a39b522e01)

6 years agoceph-volume tests.util ensure that both disk and device report is_device
Alfredo Deza [Wed, 28 Nov 2018 21:09:49 +0000 (16:09 -0500)]
ceph-volume tests.util ensure that both disk and device report is_device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit df8ba476d68a62f6e0449aa27c81f3effa1e0536)

6 years agoceph-volume tests.util ensure is_encrypted works for lv, partition, and mapper devices
Alfredo Deza [Wed, 28 Nov 2018 21:01:52 +0000 (16:01 -0500)]
ceph-volume tests.util ensure is_encrypted works for lv, partition, and mapper devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ff0166f9f445d229216113623f18244af188b57f)

6 years agoceph-volume tests.util ensure partitions are listed as top-level keys
Alfredo Deza [Wed, 28 Nov 2018 18:53:56 +0000 (13:53 -0500)]
ceph-volume tests.util ensure partitions are listed as top-level keys

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6dc01772245e557b73551111f7c48cc60aec0893)

6 years agoceph-volume tests add mapper test for dm paths
Alfredo Deza [Wed, 28 Nov 2018 18:47:01 +0000 (13:47 -0500)]
ceph-volume tests add mapper test for dm paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ebf7d7631be362e6a26af19225bfc07ec48138d)

6 years agoceph-volume tests update is_mapper unit test when zapping
Alfredo Deza [Wed, 28 Nov 2018 18:46:05 +0000 (13:46 -0500)]
ceph-volume tests update is_mapper unit test when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 830cd7d638b70e97723eb41f0d33f6a60e3ece32)

6 years agoceph-volume util.device check both dm and mapper paths for is_mapper property
Alfredo Deza [Wed, 28 Nov 2018 18:26:59 +0000 (13:26 -0500)]
ceph-volume util.device check both dm and mapper paths for is_mapper property

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ba83c1f37e6ba6fa695ff97a028479e88e2b762c)

6 years agoceph-volume lvm skip empty vg_name removals
Alfredo Deza [Wed, 28 Nov 2018 13:18:03 +0000 (08:18 -0500)]
ceph-volume lvm skip empty vg_name removals

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 14c74ede9a83975a3464c219cab0be3c06238ff9)

6 years agoceph-volume util.arg_validators optionally skip gpt header check
Alfredo Deza [Wed, 28 Nov 2018 12:56:34 +0000 (07:56 -0500)]
ceph-volume util.arg_validators optionally skip gpt header check

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 58be4d0e18c4b2f3560361345f91c5a1c7b6c5c4)

6 years agoceph-volume util.device if no vgs, set it to an empty list
Alfredo Deza [Wed, 28 Nov 2018 12:52:46 +0000 (07:52 -0500)]
ceph-volume util.device if no vgs, set it to an empty list

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 822ca670eef57e009e356fc58d6732cdeecdecdc)

6 years agoceph-volume util.device set vgs to Device object if available
Alfredo Deza [Wed, 28 Nov 2018 12:51:51 +0000 (07:51 -0500)]
ceph-volume util.device set vgs to Device object if available

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 91c01e88ecc8f94f78b30463f13a51affe405d31)

6 years agoceph-volume util update is_device when reported as device or disk
Alfredo Deza [Wed, 28 Nov 2018 12:50:50 +0000 (07:50 -0500)]
ceph-volume util update is_device when reported as device or disk

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f74635d16d7ceefb8e0de02ded85a6e7be53457f)

6 years agoceph-volume util add method to detect encryption on devices
Alfredo Deza [Wed, 28 Nov 2018 12:47:59 +0000 (07:47 -0500)]
ceph-volume util add method to detect encryption on devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1ee7435aef2f6475256e8482194d55e16b77eee8)

6 years agoceph-volume util.disk add a helper to remove partitions
Alfredo Deza [Tue, 27 Nov 2018 14:45:17 +0000 (09:45 -0500)]
ceph-volume util.disk add a helper to remove partitions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 870f9e5b48a5fe13e213be86ee8947ca7207c3bf)

6 years agoceph-volume util.disk include partitions as top level devices in get_devices
Alfredo Deza [Tue, 27 Nov 2018 14:31:58 +0000 (09:31 -0500)]
ceph-volume util.disk include partitions as top level devices in get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7f3c3598491e556ec1b93ea0a54f52144202fab6)

6 years agoceph-volume util.disk fix issue when capturing partition information
Alfredo Deza [Tue, 27 Nov 2018 14:26:33 +0000 (09:26 -0500)]
ceph-volume util.disk fix issue when capturing partition information

Before, the `if` condition meant that it would only work when the output
was '1', which is incorrect as that would only happen if a partition was
the first one, ignoring any other partition. The contents of that file
is the partition number, not a boolean to tell if it is a partition or
not.

It now includes the `holders` file contents which is needed for
dm-mapper work

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 28cb13a970909dc8efe508aee18f47e0fb3fdde3)

6 years agoceph-volume lvm.zap split zapping in methods per device type
Alfredo Deza [Tue, 27 Nov 2018 13:23:19 +0000 (08:23 -0500)]
ceph-volume lvm.zap split zapping in methods per device type

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0d2a9a40d2c301039d571101d1e92a82058a51e0)

6 years agoceph-volume: add test for disk.udevadm_property 25349/head
Jan Fajerski [Wed, 28 Nov 2018 09:21:36 +0000 (10:21 +0100)]
ceph-volume: add test for disk.udevadm_property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 58316e3f826f8bb91b375589aa3676b664ce3984)

6 years agoceph-volume: add device_id field to inventory output
Jan Fajerski [Wed, 21 Nov 2018 12:20:45 +0000 (13:20 +0100)]
ceph-volume: add device_id field to inventory output

This intends to mimic the C++ implementation in src/common/blkdev.cc.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 41da5e9a4334628db09e3166eaceff79b9b6c7e0)

6 years agoqa: use timeout for fs asok operations 25332/head
Patrick Donnelly [Sat, 6 Oct 2018 17:32:18 +0000 (10:32 -0700)]
qa: use timeout for fs asok operations

Otherwise a bug preventing an asok operation from completing will cause the
entire job to fail.

Fixes: http://tracker.ceph.com/issues/36335
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 296d89b0463ee3aad8dfb50b0e41a27e2c852ddb)

6 years agoMerge pull request #25294 from smithfarm/wip-37422-mimic
Nathan Cutler [Wed, 28 Nov 2018 21:13:02 +0000 (22:13 +0100)]
Merge pull request #25294 from smithfarm/wip-37422-mimic

mimic: build/ops: Can't compile Ceph on Fedora 29 as it doesn't recognize python*3*-tox as an install Tox

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25040 from smithfarm/wip-36749-mimic
Yuri Weinstein [Wed, 28 Nov 2018 20:27:07 +0000 (12:27 -0800)]
Merge pull request #25040 from smithfarm/wip-36749-mimic

mimic: mgr: [restful] deep_scrub is not a valid OSD command

Reviewed-by: Boris Ranto <branto@redhat.com>