]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoObjecter::calc_op_budget: Fix invalid access to extent union member 26066/head
Simon Ruggier [Sat, 15 Dec 2018 04:32:22 +0000 (23:32 -0500)]
Objecter::calc_op_budget: Fix invalid access to extent union member

Previously, the function was checking the extent length for any
operation with the CEPH_OSD_OP_MODE_RD and CEPH_OSD_OP_TYPE_DATA bits
set, but some operation types that have these bits set use a different
interpretation of the ceph_osd_op union.

This leads to a type safety problem where another field, such as the
object version, is interpreted as an extent length and added to the
budget. In addition to producing an incorrectly high budget for the
operation, this also triggers an assertion failure if the value is large
enough to overflow a signed 32-bit integer, as described in bug #9592
and bug #37932.

This change uses the ceph_osd_op_uses_extent function to prevent invalid
access to the union, instead of ceph_osd_op_type_data.

Fixes: http://tracker.ceph.com/issues/37932
Signed-off-by: Simon Ruggier <simon@platform.sh>
(cherry picked from commit 1cb91c2cdfa7556b23b9800c8d10033d0c815b4e)

6 years agoMerge pull request #26035 from noahdesu/mimic-pr26015
Noah Watkins [Mon, 21 Jan 2019 21:32:39 +0000 (13:32 -0800)]
Merge pull request #26035 from noahdesu/mimic-pr26015

mimic: cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26029 from alfredodeza/mimic-fix_raw_input
Andrew Schoen [Mon, 21 Jan 2019 21:28:57 +0000 (15:28 -0600)]
Merge pull request #26029 from alfredodeza/mimic-fix_raw_input

mimic ceph-volume: Adapt code to support Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #26013 from alfredodeza/mimic-bz1644847
Andrew Schoen [Mon, 21 Jan 2019 19:40:53 +0000 (13:40 -0600)]
Merge pull request #26013 from alfredodeza/mimic-bz1644847

mimic ceph-volume zap devices associated with an OSD ID and/or OSD FSID

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agocli: dump osd-fsid as part of osd find <id> 26035/head
Noah Watkins [Thu, 17 Jan 2019 19:16:44 +0000 (11:16 -0800)]
cli: dump osd-fsid as part of osd find <id>

Dumps the osd-fsid uuid as part of the `osd find <id>` command.
Currently this uuid is only available as part of `osd dump` but
ceph-ansible has a use case to interrogate a single osd without needing
the entire osdmap dump.

Fixes: http://tracker.ceph.com/issues/37966
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
(cherry picked from commit 4d01b2c96e0ea1122225c30da21bc39c40e15c0e)

6 years agoceph-volume: Adapt code to support Python3 26029/head
Volker Theile [Thu, 29 Nov 2018 12:48:30 +0000 (13:48 +0100)]
ceph-volume: Adapt code to support Python3

- raw_input() has been renamed to input() in Python3
- Changed signature of prompt_bool. Variables that are named like built-ins must be named like xxx_ and not _xxx

Fixes: https://tracker.ceph.com/issues/37470
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit fe25a0ea625e75c598f6d0749e7259eef167fa8e)

6 years agoceph-volume tests.functional.batch symlink test_zap to all batch scenarios 26013/head
Alfredo Deza [Fri, 7 Dec 2018 17:29:45 +0000 (12:29 -0500)]
ceph-volume tests.functional.batch symlink test_zap to all batch scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3c0058911fd0b3e273a7afb1a9f4f6072942d75)

6 years agoceph-volume tests.functional.batch add test_zap yaml to tox.ini
Alfredo Deza [Fri, 7 Dec 2018 17:28:37 +0000 (12:28 -0500)]
ceph-volume tests.functional.batch add test_zap yaml to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 25843b762953e53c08e6475d1f9dbfd5102a283f)

6 years agoceph-volume tests.functional.batch create a separate test_zap playbook
Alfredo Deza [Fri, 7 Dec 2018 17:08:44 +0000 (12:08 -0500)]
ceph-volume tests.functional.batch create a separate test_zap playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b8f0d483a6203de204fcc605ade536749a9f6c8)

6 years agoceph-volume lvm.zap update success message for OSD IDs
Alfredo Deza [Fri, 7 Dec 2018 12:53:43 +0000 (07:53 -0500)]
ceph-volume lvm.zap update success message for OSD IDs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 73bb17f79a5b67b28ceace9dc7dcc2dde38dad5e)

6 years agodoc/man/ceph-volume add zapping by osd-id examples
Alfredo Deza [Thu, 6 Dec 2018 20:30:38 +0000 (15:30 -0500)]
doc/man/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit acbf7c9b2ad2810621d7ed3987e814cd8c2a2f65)

6 years agodoc/ceph-volume add zapping by osd-id examples
Alfredo Deza [Thu, 6 Dec 2018 20:30:22 +0000 (15:30 -0500)]
doc/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f82304f4b5f5ecd865b6a208f894b59cec6386e2)

6 years agoceph-volume tests.lvm verify associated lvs by osd id+fsid behavior
Alfredo Deza [Thu, 6 Dec 2018 20:00:00 +0000 (15:00 -0500)]
ceph-volume tests.lvm verify associated lvs by osd id+fsid behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40a414464727375d5c558532aed3b70766ec5b78)

6 years agoceph-volume lvm.zap initial take on zapping by OSD ID, FSID
Alfredo Deza [Thu, 6 Dec 2018 15:54:11 +0000 (10:54 -0500)]
ceph-volume lvm.zap initial take on zapping by OSD ID, FSID

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 59f6cec6dd0687a2528f75a518bd0d4617c27fe0)

6 years agoMerge pull request #25554 from jan--f/25238-mimic
Alfredo Deza [Thu, 17 Jan 2019 17:05:55 +0000 (12:05 -0500)]
Merge pull request #25554 from jan--f/25238-mimic

mimic ceph-volume: introduce class hierachy for strategies

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume lvm.strategies remove unused import 25554/head
Alfredo Deza [Mon, 17 Dec 2018 18:34:58 +0000 (13:34 -0500)]
ceph-volume lvm.strategies remove unused import

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d127ae3e5901c6477ed44f4d50be834040c44b6f)

  Conflicts:
  src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py
          resolved by removing str_to_int import

6 years agoceph-volume: filestore strategy use strategy class hierarchy
Jan Fajerski [Mon, 26 Nov 2018 13:03:56 +0000 (14:03 +0100)]
ceph-volume: filestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 34df12ea6a8686f161b4bdfde92e557743f59cf4)

6 years agoceph-volume: bluestore strategy use strategy class hierarchy
Jan Fajerski [Mon, 26 Nov 2018 13:03:03 +0000 (14:03 +0100)]
ceph-volume: bluestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 302334ac6a4375125b6c1e68bf7160f8bc0597c2)

6 years agoceph-volume: add strategies.py to for shared code
Jan Fajerski [Mon, 26 Nov 2018 13:01:57 +0000 (14:01 +0100)]
ceph-volume: add strategies.py to for shared code

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b38987ee9652527e4845f2beb6406cbd398dac9b)

6 years agoMerge pull request #25923 from alfredodeza/mimic-ceph-volume-fix-json
Andrew Schoen [Fri, 11 Jan 2019 21:41:16 +0000 (15:41 -0600)]
Merge pull request #25923 from alfredodeza/mimic-ceph-volume-fix-json

mimic ceph-volume: fix JSON output in `inventory`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: fix JSON output in `inventory` 25923/head
Sebastian Wagner [Thu, 22 Nov 2018 17:01:50 +0000 (18:01 +0100)]
ceph-volume: fix JSON output in `inventory`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a3e6f569b4fa0419dff4690a72e9be6fe0a255c1)

6 years agoMerge pull request #25819 from batrick/mimic-pr24562
Nathan Cutler [Fri, 11 Jan 2019 12:18:35 +0000 (13:18 +0100)]
Merge pull request #25819 from batrick/mimic-pr24562

doc: removed warning for resolved errata

Reviewed-by: John Wilkins <jowilkin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25868 from dillaman/wip-37842-mimic
Jason Dillaman [Wed, 9 Jan 2019 22:17:20 +0000 (17:17 -0500)]
Merge pull request #25868 from dillaman/wip-37842-mimic

mimic: test/pybind/test_rbd: filter out unknown list_children2 keys

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agotest/pybind/test_rbd: filter out unknown list_children2 keys 25868/head
Mykola Golub [Tue, 8 Jan 2019 10:49:55 +0000 (12:49 +0200)]
test/pybind/test_rbd: filter out unknown list_children2 keys

Fixes: http://tracker.ceph.com/issues/37729
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 24cc60335d86ff625e0c5d98cab0c4449a24f296)

6 years agoMerge pull request #25837 from alfredodeza/mimic-rm37805
Andrew Schoen [Tue, 8 Jan 2019 20:18:29 +0000 (14:18 -0600)]
Merge pull request #25837 from alfredodeza/mimic-rm37805

mimic ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25775 from alfredodeza/mimic-rm37442
Andrew Schoen [Tue, 8 Jan 2019 20:17:24 +0000 (14:17 -0600)]
Merge pull request #25775 from alfredodeza/mimic-rm37442

mimic ceph-volume normalize comma to dot for string to int conversions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests/functional declare ceph-ansible roles instead of importing them 25837/head
Alfredo Deza [Mon, 7 Jan 2019 20:15:21 +0000 (15:15 -0500)]
ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad6b924e2bf3b06ec66eb5bc4fea065f5babc512)

6 years agoceph-volume tests.util ensure ints and strings with commas can be converted to ints 25775/head
Alfredo Deza [Wed, 2 Jan 2019 19:18:14 +0000 (14:18 -0500)]
ceph-volume tests.util ensure ints and strings with commas can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 91bc3a1479c550be7b48128d5755228a2007e4b0)

6 years agoceph-volume test verify strings with a comma can be converted to ints
Alfredo Deza [Fri, 21 Dec 2018 20:26:39 +0000 (15:26 -0500)]
ceph-volume test verify strings with a comma can be converted to ints

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 818902e568adaefecc6da9afee58b3a804a0e4d0)

6 years agoceph-volume util normalize comma with dot for str-to-int conversion
Alfredo Deza [Fri, 21 Dec 2018 20:24:25 +0000 (15:24 -0500)]
ceph-volume util normalize comma with dot for str-to-int conversion

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit aa5323fbcfdc6c21177cb800884712b0252b84a0)

6 years agoMerge pull request #25777 from alfredodeza/mimic-rm37486
Andrew Schoen [Tue, 8 Jan 2019 15:39:24 +0000 (09:39 -0600)]
Merge pull request #25777 from alfredodeza/mimic-rm37486

mimic ceph-volume: set permissions right before prime-osd-dir

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.functional fix typo when stopping osd.0 in filestore 25777/head
Alfredo Deza [Mon, 17 Dec 2018 17:35:27 +0000 (12:35 -0500)]
ceph-volume tests.functional fix typo when stopping osd.0 in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fba43289a2ff8a94361cc016e7a4dfdc61f324e8)

6 years agoceph-volume tests.functional.lvm stop all osds before activating again to test permis...
Alfredo Deza [Thu, 13 Dec 2018 14:49:41 +0000 (09:49 -0500)]
ceph-volume tests.functional.lvm stop all osds before activating again to test permissions

Otherwise, ``ceph-volume lvm activate --all`` will skip OSDs that are
already running. So the test that verifies that permissions are
corrected will fail because ``activate`` will simply skip them

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9be6b15d3316c8eab159bc5d374e95e23d082dde)

6 years agoceph-volume lvm.activate ensure filestore osd path is chowned always
Alfredo Deza [Thu, 13 Dec 2018 12:45:00 +0000 (07:45 -0500)]
ceph-volume lvm.activate ensure filestore osd path is chowned always

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d6c32b5db0627d95c2ffb9428e86271daa4a336)

6 years agoceph-volume: include the ceph-facts role when needed during testing
Andrew Schoen [Wed, 12 Dec 2018 20:10:59 +0000 (14:10 -0600)]
ceph-volume: include the ceph-facts role when needed during testing

The ceph-ansible stable-3.2 branch does not have the ceph-facts role
but master does. We need to make sure to import this role only when
it's supported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 569b4fb7b3894275a3b9f4ab603e32099301f9aa)

6 years agoceph-volume tests.functional set root:root in OSDs in filestore
Alfredo Deza [Mon, 10 Dec 2018 19:52:03 +0000 (14:52 -0500)]
ceph-volume tests.functional set root:root in OSDs in filestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 503396f685f6a93e0c63e97477677b7145f96c26)

6 years agoceph-volume tests.functional set root:root in OSDs in bluestore
Alfredo Deza [Mon, 10 Dec 2018 19:38:02 +0000 (14:38 -0500)]
ceph-volume tests.functional set root:root in OSDs in bluestore

To make sure that ceph-volume will be able to set the right permissions
when trying to activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 98631fdeeb6dbe765d3eb80c5c1800946735f2a2)

6 years agoceph-volume lvm.activate always chown osd path before osd-prime-dir in bluestore
Alfredo Deza [Mon, 10 Dec 2018 18:27:20 +0000 (13:27 -0500)]
ceph-volume lvm.activate always chown osd path before osd-prime-dir in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0af31ec24533091aa4991b2343fe80da94777b4d)

6 years agoremoved warning for resolved errata 25819/head
Scoots Hamilton [Mon, 15 Oct 2018 15:18:43 +0000 (11:18 -0400)]
removed warning for resolved errata

Signed-off-by: Scoots Hamilton <scoots@redhat.com>
(cherry picked from commit 93528996189f2eaf42bae18db428f3ea18f28b14)

Conflicts:
doc/cephfs/cache-size-limits.rst

6 years agoMerge pull request #25749 from alfredodeza/mimic-cryptsetup-allow-discards
Andrew Schoen [Mon, 7 Jan 2019 19:56:38 +0000 (13:56 -0600)]
Merge pull request #25749 from alfredodeza/mimic-cryptsetup-allow-discards

mimic ceph-volume: enable device discards

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25551 from jan--f/25203-mimic
Alfredo Deza [Mon, 7 Jan 2019 12:36:05 +0000 (07:36 -0500)]
Merge pull request #25551 from jan--f/25203-mimic

mimic ceph-volume: fix Batch object in py3 environments

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PRs after 13.2.3 and before 13.2.4 back into mimic branch
David Galloway [Fri, 4 Jan 2019 18:40:42 +0000 (13:40 -0500)]
Merge PRs after 13.2.3 and before 13.2.4 back into mimic branch

6 years ago13.2.4 v13.2.4
Jenkins Build Slave User [Fri, 4 Jan 2019 15:40:42 +0000 (15:40 +0000)]
13.2.4

6 years agorgw: fix issues with 'enforce bounds' patch
Joao Eduardo Luis [Thu, 29 Nov 2018 01:05:31 +0000 (01:05 +0000)]
rgw: fix issues with 'enforce bounds' patch

The patch to enforce bounds on max-keys/max-uploads/max-parts had a few
issues that would prevent us from compiling it. Instead of changing the
code provided by the submitter, we're addressing them in a separate
commit to maintain the DCO.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 29bc434a6a81a2e5c5b8cfc4c8d5c82ca5bf538a)
mimic specific fixes:
As the largeish change from master g_conf() isn't in mimic yet, use the g_conf
global structure, also make rgw_op use the value from req_info ceph context as
we do for all the requests

6 years agorgw: enforce bounds on max-keys/max-uploads/max-parts
Robin H. Johnson [Fri, 21 Sep 2018 21:49:34 +0000 (14:49 -0700)]
rgw: enforce bounds on max-keys/max-uploads/max-parts

RGW S3 listing operations provided a way for authenticated users to
cause a denial of service against OMAPs holding bucket indices.

Bound the min & max values that a user could pass into the max-X
parameters, to keep the system safe. The default of 1000 is chosen to
match AWS S3 behavior.

Affected operations:
- ListBucket, via max-keys
- ListBucketVersions, via max-keys
- ListBucketMultiPartUploads, via max-uploads
- ListMultipartUploadParts, via max-parts

The Swift bucket listing codepath already enforced a limit, so is
unaffected by this issue.

Prior to this commit, the effective limit is the lower of
osd_max_omap_entries_per_request or osd_max_omap_bytes_per_request.

Backport: luminous, mimic
Fixes: http://tracker.ceph.com/issues/35994
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit d79f68a1e31f4bc917eec1b6bbc8e8446377dc6b)

 Conflicts:
src/common/options.cc:
  Conflicts due to options from master

6 years agomon/config-key: limit caps allowed to access the store
Joao Eduardo Luis [Wed, 17 Oct 2018 13:42:15 +0000 (14:42 +0100)]
mon/config-key: limit caps allowed to access the store

Henceforth, we'll require explicit `allow` caps for commands, or for the
config-key service. Blanket caps are no longer allowed for the
config-key service, except for 'allow *'.

(for luminous and mimic, we're also ensuring MonCap's parser is able to
understand forward slashes '/' when parsing prefixes)

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 5fff611041c5afeaf3c8eb09e4de0cc919d69237)

6 years agoMerge pull request #25291 from smithfarm/wip-37414-mimic
Yuri Weinstein [Thu, 3 Jan 2019 18:26:15 +0000 (10:26 -0800)]
Merge pull request #25291 from smithfarm/wip-37414-mimic

mimic: mgr/balancer: add crush_compat_metrics param to change optimization keys

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25771 from alfredodeza/mimic-ceph-volume-fix-join
Alfredo Deza [Thu, 3 Jan 2019 18:23:22 +0000 (13:23 -0500)]
Merge pull request #25771 from alfredodeza/mimic-ceph-volume-fix-join

mimic ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoceph-volume: Fix TypeError: join() takes exactly one argument (2 given) 25771/head
Sebastian Wagner [Mon, 10 Dec 2018 13:38:08 +0000 (14:38 +0100)]
ceph-volume: Fix TypeError: join() takes exactly one argument (2 given)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 38f289b691ed1275bdf7342a4347434ed340c8fc)

6 years agoMerge pull request #25351 from alfredodeza/mimic-rm37449
Alfredo Deza [Thu, 3 Jan 2019 18:01:07 +0000 (13:01 -0500)]
Merge pull request #25351 from alfredodeza/mimic-rm37449

mimic ceph-volume  zap: improve zapping to remove all partitions and all LVs, encrypted or not

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24695 from vshankar/wip-35932
Yuri Weinstein [Thu, 3 Jan 2019 16:26:12 +0000 (08:26 -0800)]
Merge pull request #24695 from vshankar/wip-35932

mimic : client: retry remount on dcache invalidation failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25031 from smithfarm/wip-36576-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:25:35 +0000 (08:25 -0800)]
Merge pull request #25031 from smithfarm/wip-36576-mimic

mimic: cephfs: mgr/status: fix fs status subcommand did not show standby-replay MDS' perf info

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25198 from pdvian/wip-37285-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:24:02 +0000 (08:24 -0800)]
Merge pull request #25198 from pdvian/wip-37285-mimic

mimic: rgw: radosgw-admin: translate reshard status codes (trivial)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement
Yuri Weinstein [Thu, 3 Jan 2019 16:23:22 +0000 (08:23 -0800)]
Merge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement

mimic: qa/ceph-ansible: Move "branch" out of "vars" section

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #25218 from k0ste/mimic_backports
Yuri Weinstein [Thu, 3 Jan 2019 16:22:01 +0000 (08:22 -0800)]
Merge pull request #25218 from k0ste/mimic_backports

mimic: mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #25227 from pdvian/wip-37288-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:21:16 +0000 (08:21 -0800)]
Merge pull request #25227 from pdvian/wip-37288-mimic

mimic: tests: qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25242 from smithfarm/wip-37384-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:19:44 +0000 (08:19 -0800)]
Merge pull request #25242 from smithfarm/wip-37384-mimic

mimic: test: Start using GNU awk and fix archiving directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #25252 from pdvian/wip-37363-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:18:36 +0000 (08:18 -0800)]
Merge pull request #25252 from pdvian/wip-37363-mimic

mimic: librbd: fix missing unblock_writes if shrink is not allowed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25153 from jonsger/wip-24827-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:15:34 +0000 (08:15 -0800)]
Merge pull request #25153 from jonsger/wip-24827-mimic

mimic: build/ops: run-make-check.sh ccache tweaks

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25171 from pdvian/wip-36392-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:14:38 +0000 (08:14 -0800)]
Merge pull request #25171 from pdvian/wip-36392-mimic

mimic: rpm: Use hardened LDFLAGS

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25293 from smithfarm/wip-37419-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:13:35 +0000 (08:13 -0800)]
Merge pull request #25293 from smithfarm/wip-37419-mimic

mimic: mgr/balancer: add cmd to list all plans

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25295 from rzarzynski/wip-bug-36248-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:12:58 +0000 (08:12 -0800)]
Merge pull request #25295 from rzarzynski/wip-bug-36248-mimic

mimic: common: fix memory leaks in WeightedPriorityQueue.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25507 from smithfarm/wip-36729-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:11:42 +0000 (08:11 -0800)]
Merge pull request #25507 from smithfarm/wip-36729-mimic

mimic: osd: Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #25753 from alfredodeza/mimic-better-devid
Alfredo Deza [Thu, 3 Jan 2019 14:37:43 +0000 (09:37 -0500)]
Merge pull request #25753 from alfredodeza/mimic-better-devid

mimic common/blkdev, ceph-volume: improve get_device_id

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoceph-volume: Micron SSDs don't include vendor name in ID_SERIAL 25753/head
Sage Weil [Thu, 6 Dec 2018 16:02:35 +0000 (10:02 -0600)]
ceph-volume: Micron SSDs don't include vendor name in ID_SERIAL

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3e4a638c931a5198aca74dc4037d0b03d5e563ba)

6 years agoceph-volume: update get_device_id to match in-tree implementation
Sage Weil [Thu, 6 Dec 2018 15:43:36 +0000 (09:43 -0600)]
ceph-volume: update get_device_id to match in-tree implementation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c967dbf81f85602d2cd044ee5a91ba38cd76bdb9)

6 years agoceph-volume: enable device discards 25749/head
Jonas Jelten [Fri, 19 Oct 2018 13:15:03 +0000 (15:15 +0200)]
ceph-volume: enable device discards

When using SSDs as encrypted OSD device, discards do not pass the
encryption layer. This option activates discard requests.

Signed-off-by: Jonas Jelten <jj@stusta.net>
(cherry picked from commit 333330414730bf57c68efe6f1f13841b0a014dae)

6 years ago13.2.3 v13.2.3
Jenkins Build Slave User [Wed, 2 Jan 2019 18:00:57 +0000 (18:00 +0000)]
13.2.3

6 years agoceph-volume: add Batch smoketest 25551/head
Jan Fajerski [Mon, 26 Nov 2018 13:54:50 +0000 (14:54 +0100)]
ceph-volume: add Batch smoketest

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 936ee680f7633620c8a1127db9630a22326b9e2c)

6 years agoceph-volume: fix Batch object in py3 environments
Jan Fajerski [Fri, 30 Nov 2018 08:22:51 +0000 (09:22 +0100)]
ceph-volume: fix Batch object in py3 environments

Fixes: https://tracker.ceph.com/issues/37356
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 851c7693daa315b7972702bedf09dd66a37667d8)

6 years agoosd: combine recovery/scrub/snap sleep timer into one. 25507/head
Jianpeng Ma [Sat, 28 Apr 2018 02:46:55 +0000 (10:46 +0800)]
osd: combine recovery/scrub/snap sleep timer into one.

One SafeTime on thread. And those events if timer don't take long time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 610d5196c0132739e1ae133db0c7d896063470ad)

6 years agoMerge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes
Xie Xingguo [Thu, 13 Dec 2018 05:54:59 +0000 (13:54 +0800)]
Merge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes

mimic: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25285 from smithfarm/wip-37396-mimic
Yuri Weinstein [Thu, 13 Dec 2018 00:46:16 +0000 (16:46 -0800)]
Merge pull request #25285 from smithfarm/wip-37396-mimic

mimic: tests: /usr/bin/ld: cannot find -lradospp in rados mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>n
6 years agoosd: Add support for osd_delete_sleep to throttle pg deletion
David Zafman [Thu, 25 Oct 2018 05:00:57 +0000 (22:00 -0700)]
osd: Add support for osd_delete_sleep to throttle pg deletion

Fixes: http://tracker.ceph.com/issues/36474
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92f1f5859f142cb83cc1d1a367853ac91d6b01d4)

Conflicts:
    src/osd/OSD.cc
- use cct->_conf->get_val instead of cct->_conf.get_val

6 years agotest: Set any value for osd_delete_sleep to guarantee we are testing even SSD
David Zafman [Tue, 30 Oct 2018 18:42:05 +0000 (11:42 -0700)]
test: Set any value for osd_delete_sleep to guarantee we are testing even SSD

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f621a1190f45a780b71b916b31c1a01a20ece96)

6 years agoMerge pull request #25332 from pdvian/wip-36503-mimic
Yuri Weinstein [Wed, 12 Dec 2018 16:01:14 +0000 (08:01 -0800)]
Merge pull request #25332 from pdvian/wip-36503-mimic

mimic: tests: use timeout for fs asok operations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #25283 from jdurgin/wip-osd-memory-target-mimic
Josh Durgin [Mon, 10 Dec 2018 17:13:37 +0000 (12:13 -0500)]
Merge pull request #25283 from jdurgin/wip-osd-memory-target-mimic

mimic: core: os/bluestore: cache autotuning and memory limit

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25362 from noahdesu/mimic-docs
Noah Watkins [Fri, 7 Dec 2018 19:35:34 +0000 (11:35 -0800)]
Merge pull request #25362 from noahdesu/mimic-docs

mimic: docs: backport edit on github changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoosd/OSDMap: fix upmap mis-killing for erasure-coded PGs 25419/head
xie xingguo [Sat, 1 Dec 2018 09:42:01 +0000 (17:42 +0800)]
osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

The up-set of erasure-coded PGs may include CRUSH_ITEM_NONE,
which as a result causes mis-killing of valid upmap items.

Fixes: https://tracker.ceph.com/issues/37493
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: ningtao <ningtao@sangfor.com.cn>
(cherry picked from commit f043dcc6d8663ed960a354b2441fd3f8dd66c62d)

Conflicts:
        - *maybe_remove_pg_upmaps* input changed, in master
          we now have a passed in *nextmap* parameter

6 years agomon: auto clean up stale upmap items
xie xingguo [Wed, 21 Nov 2018 06:25:45 +0000 (14:25 +0800)]
mon: auto clean up stale upmap items

See https://www.spinics.net/lists/ceph-users/msg49518.html

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4181aa86906886423af80e478e45554606fc7cf6)

Conflicts:
        - *nextmap* has been renamed to *tmpmap*
        - add the *have_pg_upmaps* wrapper which is necessary for unittest

6 years agoosd/OSDMap: add pg-existence sanity check
xie xingguo [Wed, 21 Nov 2018 01:36:21 +0000 (09:36 +0800)]
osd/OSDMap: add pg-existence sanity check

The reason why __get_pg_pool_size(pg)__ or __get_pg_pool_crush_rule(pg)__ fails is
that the pg does not exist anymore. So it generally makes sense to check __pg_exists(pg)__
before moving further.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 79ef3bea7550559b7ea4e2b68dc407c7eb25defa)

Conflicts:
- *nextmap* has been renamed to *tmpmap*

6 years agoosd/OSDMap.cc: move sanity checks to the proper place
xie xingguo [Wed, 20 Jun 2018 01:27:46 +0000 (09:27 +0800)]
osd/OSDMap.cc: move sanity checks to the proper place

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3dcd4eaf867369437720a21daf3c344cc4b9844e)

6 years agoosd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone
xie xingguo [Wed, 20 Jun 2018 01:04:19 +0000 (09:04 +0800)]
osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone

If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.

The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:

```
src/osd/OSDMap.cc: 4078: FAILED assert(target > 0)
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8a11ed479d7ea95f76170e74123ca2c48e3a3963)

6 years agoMerge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id
Alfredo Deza [Tue, 4 Dec 2018 16:50:42 +0000 (11:50 -0500)]
Merge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id

mimic ceph-volume add device_id to inventory listing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agodoc/_templates/page.html: add link to report doc bugs 25362/head
Neha Ojha [Fri, 19 Oct 2018 20:26:53 +0000 (13:26 -0700)]
doc/_templates/page.html: add link to report doc bugs

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de30c4d3c5927bb58b49b44b391647e397e1fe4f)

6 years agoluminous: doc: show edit on github links and version warnings
Noah Watkins [Mon, 1 Oct 2018 23:54:19 +0000 (16:54 -0700)]
luminous: doc: show edit on github links and version warnings

backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.

this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
 Conflicts:
doc/conf.py

6 years agoceph-volume tests patch udevadm so it doesn't actually call udevadm in a test 25351/head
Alfredo Deza [Fri, 30 Nov 2018 17:31:02 +0000 (12:31 -0500)]
ceph-volume tests patch udevadm so it doesn't actually call udevadm in a test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f440fb96694e38f69e174c01f284ab9a2066bc8)

6 years agoceph-volume util.disk use device.path for error reporting
Alfredo Deza [Fri, 30 Nov 2018 13:27:01 +0000 (08:27 -0500)]
ceph-volume util.disk use device.path for error reporting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1670585ebed7c685b49a13c1066eedc1aa37d29b)

6 years agoceph-volume tests.functional add back sdc1 partitions with parted to redeploy after...
Alfredo Deza [Fri, 30 Nov 2018 11:44:56 +0000 (06:44 -0500)]
ceph-volume tests.functional add back sdc1 partitions with parted to redeploy after zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b0bc162b601c2dbc9827df0ab472ba2a1146076c)

6 years agoceph-volume tests.functional add back partitions with parted to redeploy after zapping
Alfredo Deza [Thu, 29 Nov 2018 23:07:09 +0000 (18:07 -0500)]
ceph-volume tests.functional add back partitions with parted to redeploy after zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 945e8f56ce21411f664a58147afb343f0671b591)

6 years agoceph-volume tests.functional recreate partitions previously removed with --destroy
Alfredo Deza [Thu, 29 Nov 2018 19:10:46 +0000 (14:10 -0500)]
ceph-volume tests.functional recreate partitions previously removed with --destroy

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e1d136ee3bcb2a7bbe0790b87e1d54a39b522e01)

6 years agoceph-volume tests.util ensure that both disk and device report is_device
Alfredo Deza [Wed, 28 Nov 2018 21:09:49 +0000 (16:09 -0500)]
ceph-volume tests.util ensure that both disk and device report is_device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit df8ba476d68a62f6e0449aa27c81f3effa1e0536)

6 years agoceph-volume tests.util ensure is_encrypted works for lv, partition, and mapper devices
Alfredo Deza [Wed, 28 Nov 2018 21:01:52 +0000 (16:01 -0500)]
ceph-volume tests.util ensure is_encrypted works for lv, partition, and mapper devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ff0166f9f445d229216113623f18244af188b57f)

6 years agoceph-volume tests.util ensure partitions are listed as top-level keys
Alfredo Deza [Wed, 28 Nov 2018 18:53:56 +0000 (13:53 -0500)]
ceph-volume tests.util ensure partitions are listed as top-level keys

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6dc01772245e557b73551111f7c48cc60aec0893)

6 years agoceph-volume tests add mapper test for dm paths
Alfredo Deza [Wed, 28 Nov 2018 18:47:01 +0000 (13:47 -0500)]
ceph-volume tests add mapper test for dm paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ebf7d7631be362e6a26af19225bfc07ec48138d)

6 years agoceph-volume tests update is_mapper unit test when zapping
Alfredo Deza [Wed, 28 Nov 2018 18:46:05 +0000 (13:46 -0500)]
ceph-volume tests update is_mapper unit test when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 830cd7d638b70e97723eb41f0d33f6a60e3ece32)

6 years agoceph-volume util.device check both dm and mapper paths for is_mapper property
Alfredo Deza [Wed, 28 Nov 2018 18:26:59 +0000 (13:26 -0500)]
ceph-volume util.device check both dm and mapper paths for is_mapper property

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ba83c1f37e6ba6fa695ff97a028479e88e2b762c)

6 years agoceph-volume lvm skip empty vg_name removals
Alfredo Deza [Wed, 28 Nov 2018 13:18:03 +0000 (08:18 -0500)]
ceph-volume lvm skip empty vg_name removals

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 14c74ede9a83975a3464c219cab0be3c06238ff9)

6 years agoceph-volume util.arg_validators optionally skip gpt header check
Alfredo Deza [Wed, 28 Nov 2018 12:56:34 +0000 (07:56 -0500)]
ceph-volume util.arg_validators optionally skip gpt header check

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 58be4d0e18c4b2f3560361345f91c5a1c7b6c5c4)

6 years agoceph-volume util.device if no vgs, set it to an empty list
Alfredo Deza [Wed, 28 Nov 2018 12:52:46 +0000 (07:52 -0500)]
ceph-volume util.device if no vgs, set it to an empty list

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 822ca670eef57e009e356fc58d6732cdeecdecdc)