]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoceph-volume: show correct rejected reason in inventory if device type is not acceptable 36452/head
Satoru Takeuchi [Fri, 22 May 2020 01:45:32 +0000 (01:45 +0000)]
ceph-volume: show correct rejected reason in inventory if device type is not acceptable

If device type is not acceptable in `c-v inventory`, its rejected reason
becomes "Insufficient space (<5GB)" by mistake. It's because sys_api is
empty due to skipping devices that are neither `disk` nor `device`. We
should report the target device is not acceptable in this case.

Fixes: https://tracker.ceph.com/issues/46102
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit 3e5d91d41f7275d4656019c1ca3fc80927d214c9)

4 years agoceph-volume: cleanup code
Satoru Takeuchi [Fri, 22 May 2020 01:07:17 +0000 (01:07 +0000)]
ceph-volume: cleanup code

Simplify the logic and fix a typo.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit 0169b72fff48134ef01802ade38c55281b9f4510)

4 years agoMerge pull request #36451 from jan--f/wip-45785-mimic
Jan Fajerski [Thu, 6 Aug 2020 16:39:25 +0000 (18:39 +0200)]
Merge pull request #36451 from jan--f/wip-45785-mimic

mimic: ceph-volume: add and delete lvm tags in a single lvchange call.

4 years agoceph-volume: fix Volume mock in lvm test case 36451/head
Jan Fajerski [Wed, 5 Aug 2020 11:57:03 +0000 (13:57 +0200)]
ceph-volume: fix Volume mock in lvm test case

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb432fe41d4ea8cb71aa592e0727d2da1978121f)

4 years agoceph-volume: add and delete lvm tags in a single lvchange call.
Jan Fajerski [Tue, 31 Mar 2020 14:07:45 +0000 (16:07 +0200)]
ceph-volume: add and delete lvm tags in a single lvchange call.

Otherwise we can end up in race-y situations when a concurrent c-v calls
sees only one tag but expects all tags to be present. Say if the
ceph.type tag is present, c-v expects ceph.osd_id to be present. By
setting/deleting tags in bulk, we use lvchange (and lvms internal
locking) as a sync mechanism.

Fixes: https://tracker.ceph.com/issues/44852
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 20ecc309371e53fda5d6a5b6cf6de6110dbe5497)

5 years agoMerge pull request #35489 from jdurgin/wip-radosbench-qa-mimic
Yuri Weinstein [Tue, 9 Jun 2020 22:22:38 +0000 (15:22 -0700)]
Merge pull request #35489 from jdurgin/wip-radosbench-qa-mimic

mimic: qa/tasks/radosbench: use long form of option for compatibility

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoqa/tasks/radosbench: use long form of option for compatibility 35489/head
Josh Durgin [Mon, 8 Jun 2020 18:29:43 +0000 (11:29 -0700)]
qa/tasks/radosbench: use long form of option for compatibility

Since the short version of --object-size changed from -o to -O, it
does not work with upgrade tests.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9673d37f1b72681f7583bf36537575d876da9712)

Conflicts:
qa/tasks/radosbench.py - trivial context difference

5 years agoMerge pull request #34737 from smithfarm/wip-44647-mimic
Yuri Weinstein [Fri, 5 Jun 2020 15:38:44 +0000 (08:38 -0700)]
Merge pull request #34737 from smithfarm/wip-44647-mimic

mimic: qa/workunits/rbd: wait for nbd map to close after unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #35164 from athanatos/sjust/wip-revert-wrong-pr-merge
Gregory Farnum [Thu, 21 May 2020 17:39:04 +0000 (10:39 -0700)]
Merge pull request #35164 from athanatos/sjust/wip-revert-wrong-pr-merge

Revert "doc: clarify WAL in bluestore-config-ref.rst"

5 years agoRevert "doc: clarify WAL in bluestore-config-ref.rst" 35164/head
Samuel Just [Thu, 21 May 2020 08:13:53 +0000 (01:13 -0700)]
Revert "doc: clarify WAL in bluestore-config-ref.rst"

This was merged to mimic by mistake.

This reverts commit 637ee39fafb3d0521bbd51681583504a2f6fd636.

Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoMerge pull request #35156 from CodingSpiderFox/patch-1
Samuel Just [Thu, 21 May 2020 08:10:34 +0000 (01:10 -0700)]
Merge pull request #35156 from CodingSpiderFox/patch-1

Clarify abbreviation that I was not aware of

Reviewed-by: Samuel Just <sjust@redhat.com>
5 years agodoc: clarify WAL in bluestore-config-ref.rst 35156/head 35165/head
CodingSpiderFox [Wed, 20 May 2020 18:18:03 +0000 (20:18 +0200)]
doc: clarify WAL in bluestore-config-ref.rst

Other people might not know this abbreviation too when diving into storage topics for the first time

Signed-off-by: CodingSpiderFox <codingspiderfox@gmail.com>
5 years agoMerge pull request #34277 from jan--f/wip-44687-mimic
Jan Fajerski [Tue, 5 May 2020 07:41:59 +0000 (09:41 +0200)]
Merge pull request #34277 from jan--f/wip-44687-mimic

mimic: ceph-volume: prepare: use *-slots arguments for implicit sizing

5 years agoqa/workunits/rbd: wait for nbd map to close after unmap 34737/head
Jason Dillaman [Wed, 11 Mar 2020 19:11:10 +0000 (15:11 -0400)]
qa/workunits/rbd: wait for nbd map to close after unmap

The unmap action only sends a signal to the kernel to notify the
rbd-nbd daemon to disconnect. Therefore, it's possible that an
unmap followed by an immediate re-map to the same device might
fail since the unmap is still in-progress.

Fixes: https://tracker.ceph.com/issues/44567
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 80a3f18cafb4add1624cc690bba436a1284dc634)

5 years agoMerge remote-tracking branch 'gh/mimic' into mimic
David Galloway [Thu, 23 Apr 2020 18:50:54 +0000 (14:50 -0400)]
Merge remote-tracking branch 'gh/mimic' into mimic

5 years agoMerge pull request #34713 from ceph/wip-yuriw-clients-upgrades-mimic
Yuri Weinstein [Thu, 23 Apr 2020 18:21:14 +0000 (11:21 -0700)]
Merge pull request #34713 from ceph/wip-yuriw-clients-upgrades-mimic

qa/tests: removed 2-workload/devstack-tempest-gate.yaml  tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoqa/tests: removed 2-workload/devstack-tempest-gate.yaml tests 34713/head
Yuri Weinstein [Thu, 23 Apr 2020 15:29:35 +0000 (08:29 -0700)]
qa/tests: removed 2-workload/devstack-tempest-gate.yaml  tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years ago13.2.10 v13.2.10
Jenkins Build Slave User [Thu, 23 Apr 2020 16:32:31 +0000 (16:32 +0000)]
13.2.10

5 years agoMerge pull request #34715 from ceph/mimic-rgw-post-obj-tagging
Abhishek L [Thu, 23 Apr 2020 16:16:42 +0000 (18:16 +0200)]
Merge pull request #34715 from ceph/mimic-rgw-post-obj-tagging

rgw: check for tagging element in POST Obj requests

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjami@redhat.com>
5 years agorgw: check for tagging element in POST Obj requests 34715/head
Abhishek Lekshmanan [Wed, 22 Apr 2020 09:24:34 +0000 (11:24 +0200)]
rgw: check for tagging element in POST Obj requests

Check for null element when reading the tagging field from POST obj XML

Fixes: https://tracker.ceph.com/issues/44967
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #34521 from smithfarm/wip-45057-mimic
Yuri Weinstein [Tue, 21 Apr 2020 23:20:02 +0000 (16:20 -0700)]
Merge pull request #34521 from smithfarm/wip-45057-mimic

mimic: qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoMerge pull request #34370 from smithfarm/wip-44083-mimic
Yuri Weinstein [Tue, 21 Apr 2020 23:18:53 +0000 (16:18 -0700)]
Merge pull request #34370 from smithfarm/wip-44083-mimic

mimic: qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years ago13.2.9 v13.2.9
Jenkins Build Slave User [Thu, 16 Apr 2020 15:49:40 +0000 (15:49 +0000)]
13.2.9

5 years agoqa/suites/rados/rest: Don't pass empty dict as data arg 34521/head
Brad Hubbard [Tue, 31 Mar 2020 02:49:26 +0000 (12:49 +1000)]
qa/suites/rados/rest: Don't pass empty dict as data arg

Passing an empty 'args' dict as a data argument when calling
requests.get somehow confuses the transaction, causing it to fail. Pass
'None' instead.

Fixes: https://tracker.ceph.com/issues/43720
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit e20e6fc3295da836fb5f098d46d6190ab28195d5)

5 years agoMerge pull request #34504 from ceph/wip-mimic-rgw-fixes
Abhishek L [Thu, 9 Apr 2020 18:29:18 +0000 (20:29 +0200)]
Merge pull request #34504 from ceph/wip-mimic-rgw-fixes

Reviewed-By: Yuri Weinstein <yuriw@redhat.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #34458 from badone/wip-44983-mimic
Yuri Weinstein [Thu, 9 Apr 2020 15:16:17 +0000 (08:16 -0700)]
Merge pull request #34458 from badone/wip-44983-mimic

mimic: selinux: Allow ceph-mgr access to httpd dir

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #34393 from tchaikov/mimic-cbt
Yuri Weinstein [Thu, 9 Apr 2020 15:14:25 +0000 (08:14 -0700)]
Merge pull request #34393 from tchaikov/mimic-cbt

mimic: qa/suites: use "mimic" branch for cbt based testing

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34479 from jan--f/wip-45004-mimic
Yuri Weinstein [Thu, 9 Apr 2020 15:04:39 +0000 (08:04 -0700)]
Merge pull request #34479 from jan--f/wip-45004-mimic

mimic: ceph-volume/batch: check lvs list before access

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #34476 from jan--f/wip-45000-mimic
Yuri Weinstein [Thu, 9 Apr 2020 15:04:02 +0000 (08:04 -0700)]
Merge pull request #34476 from jan--f/wip-45000-mimic

mimic: ceph-volume/batch: return success when all devices are filtered

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoceph-volume/batch: check lvs list before access 34479/head
Jan Fajerski [Wed, 8 Apr 2020 08:41:56 +0000 (10:41 +0200)]
ceph-volume/batch: check lvs list before access

Fixes: https://tracker.ceph.com/issues/44989
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c6dcf3161b2d69b88acdda473b85c9dd2c8433e3)

5 years agorgw: reject control characters in response-header actions 34504/head
Robin H. Johnson [Fri, 27 Mar 2020 19:48:13 +0000 (20:48 +0100)]
rgw: reject control characters in response-header actions

S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.

Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
5 years agorgw: EPERM to ERR_INVALID_REQUEST
Abhishek Lekshmanan [Fri, 27 Mar 2020 18:29:01 +0000 (19:29 +0100)]
rgw: EPERM to ERR_INVALID_REQUEST

As per Robin's comments and S3 spec

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agorgw: reject unauthenticated response-header actions
Matt Benjamin [Fri, 27 Mar 2020 17:13:48 +0000 (18:13 +0100)]
rgw: reject unauthenticated response-header actions

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)

5 years agoceph-volume/batch: return success when all devices are filtered 34476/head
Jan Fajerski [Wed, 8 Apr 2020 09:55:57 +0000 (11:55 +0200)]
ceph-volume/batch: return success when all devices are filtered

batch should only return an error if some (but not all) devices are
filtered. When only some devices are filtered the resulting osd layout
could look very different from what a user expects. If all devies are
filtered just return success.

Fixes: https://tracker.ceph.com/issues/44994
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb3eade3d00fdcdb34c4156f3c488bbb8747cd6f)

5 years agoMerge pull request #34444 from jan--f/wip-44973-mimic
Yuri Weinstein [Tue, 7 Apr 2020 15:06:58 +0000 (08:06 -0700)]
Merge pull request #34444 from jan--f/wip-44973-mimic

mimic: devices/simple/scan: Fix string in log statement

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agodevices/simple/scan: Fix string in log statement 34444/head
Jan Fajerski [Mon, 6 Apr 2020 08:51:27 +0000 (10:51 +0200)]
devices/simple/scan: Fix string in log statement

A multiline string enclosed by () gained a comma, making it into a
tuple, which has no format() method.

Fixes: https://tracker.ceph.com/issues/44949
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 292ae0305db3e48a87db45bcc7ca8cb837a08ead)

5 years agoselinux: Allow ceph-mgr access to httpd dir 34458/head
Brad Hubbard [Mon, 6 Apr 2020 23:35:06 +0000 (09:35 +1000)]
selinux: Allow ceph-mgr access to httpd dir

ceph-mgr loads modules which require read access and this causes a
denial on el7.

Fixes: https://tracker.ceph.com/issues/44216
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 35a7fc8249337c3c59f0c561632abf578f5d20fc)

5 years agoMerge pull request #34221 from ceph/wip-yuriw-clients-upgrades-mimic-octopus
Yuri Weinstein [Fri, 3 Apr 2020 16:16:04 +0000 (09:16 -0700)]
Merge pull request #34221 from ceph/wip-yuriw-clients-upgrades-mimic-octopus

qa/tests: initial check in for client-upgrade-mimic-octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #34379 from cbodley/wip-44898
Yuri Weinstein [Fri, 3 Apr 2020 14:58:26 +0000 (07:58 -0700)]
Merge pull request #34379 from cbodley/wip-44898

qa/swift: fix py3 backport regression

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
5 years agoqa/suites: use "mimic" branch for cbt based testing 34393/head
Kefu Chai [Fri, 3 Apr 2020 09:06:47 +0000 (17:06 +0800)]
qa/suites: use "mimic" branch for cbt based testing

cbt is using features like the f string introduced by python3.6, so we
need either stop testing on ubuntu 16.04, which came with python3.5, or
specify the a cbt branch which support python3.5. to address this issue
once and for all, it's simpler just pin the branch to "mimic" when
performing cbt based tests.

this change is not cherry-picked from master, as in master, we only
tests on distros which offers python3.6 and up.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoqa/tests: initial check in for client-upgrade-mimic-octopus 34221/head
Yuri Weinstein [Tue, 24 Mar 2020 19:56:06 +0000 (12:56 -0700)]
qa/tests: initial check in for client-upgrade-mimic-octopus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoqa/swift: fix py3 backport regression 34379/head
Casey Bodley [Thu, 2 Apr 2020 14:59:47 +0000 (10:59 -0400)]
qa/swift: fix py3 backport regression

this applies only to the mimic branch because it corrects a mimic
backport

Fixes: https://tracker.ceph.com/issues/44898
Signed-off-by: Casey Bodley <cbodley@redhat.com>
5 years agoqa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN 34370/head
Sage Weil [Sun, 9 Feb 2020 19:40:46 +0000 (13:40 -0600)]
qa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN

The skewed clock makes some mons miss elections.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 08b6a2bc008c006ff815dcb85bfb24b73072c7ab)

5 years agoqa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc
Sage Weil [Sun, 9 Feb 2020 16:55:03 +0000 (10:55 -0600)]
qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Fixes: https://tracker.ceph.com/issues/43889
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9f2a854b175f156d4ab7fba955aff515052c9d93)

5 years agoqa/suites/rados/multimon: skew clocks 2s (< paxos lease)
Sage Weil [Fri, 31 May 2019 14:50:44 +0000 (09:50 -0500)]
qa/suites/rados/multimon: skew clocks 2s (< paxos lease)

If the leader is the one with the accurate clock, it can still
form quorum, but if the leader has the skewed clock, all other mons appear
skewed from its perspective and no quorum is formed.  This leads to
intermittent failures, depending on the non-deterministic teuthology
deployment order and how the mon IPs sort.

Fix by reducing the skew.  This is enough skew to trigger a warning, but
not enough that it will break quorum.  This ensures that the parts of the
teuthology test that issue random mon commands won't fail (e.g., 'ceph osd
dump').

Fixes: http://tracker.ceph.com/issues/40112
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fd00136eb3b181ca34937fc7ea99427338ae6069)

5 years agoqa/suites/rados/multimon: create_rbd_pool: false
Sage Weil [Thu, 30 May 2019 20:43:48 +0000 (15:43 -0500)]
qa/suites/rados/multimon: create_rbd_pool: false

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d137b9345afd7e77f788e7142ff28a7f422c4f9)

5 years agoqa/suites/rados/multimon: no osds when skewing clock
Sage Weil [Thu, 30 May 2019 18:05:54 +0000 (13:05 -0500)]
qa/suites/rados/multimon: no osds when skewing clock

Sometimes the clock skew prevents the mon quorum from making progress and
processing the osd boot messages.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1991faaafe6e450896377bb69c04fbce84b1f989)

5 years agoMerge pull request #34193 from tchaikov/mimic-cbt-py3
Yuri Weinstein [Mon, 30 Mar 2020 16:37:01 +0000 (09:37 -0700)]
Merge pull request #34193 from tchaikov/mimic-cbt-py3

mimic: qa/tasks/cbt: install python3 deps

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoceph-volume: pass *-slots arguments to LV creation 34277/head
Jan Fajerski [Mon, 2 Mar 2020 14:16:39 +0000 (15:16 +0100)]
ceph-volume: pass *-slots arguments to LV creation

Fixes: https://tracker.ceph.com/issues/44494
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 61f0c3af0b0f36a03748733b16de1b11048342e6)

5 years agouse extent count for slots conversion instead of free count
Jan Fajerski [Wed, 4 Mar 2020 12:28:46 +0000 (13:28 +0100)]
use extent count for slots conversion instead of free count

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 4a81b4a2fbb4cafe45a8fe8cceeece636e51adb6)

5 years agoceph-volume: available_lvm: vg space takes precedence
Jan Fajerski [Wed, 4 Mar 2020 10:39:40 +0000 (11:39 +0100)]
ceph-volume: available_lvm: vg space takes precedence

This changes available_lvm to check for generic reasons only if no VGs
were found. A VG can contain a (mounted) lv, which triggers the
ro/locked test, despite the VG having space available.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c6d71eb0022935731bb9380098c975d9c5f27a86)

5 years agoqa/tasks/cbt: install python3 deps 34193/head
Sage Weil [Mon, 16 Dec 2019 16:17:26 +0000 (10:17 -0600)]
qa/tasks/cbt: install python3 deps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 299375852a4b3210cf5b95d3559d69080f14339f)

Conflict: qa/tasks/cbt.py: change python3-yaml to python36-PyYAML,
  as in EL7, python3-* packages are still named python36-*

5 years agoMerge pull request #33327 from smithfarm/wip-43257-mimic
Yuri Weinstein [Fri, 27 Mar 2020 21:45:32 +0000 (14:45 -0700)]
Merge pull request #33327 from smithfarm/wip-43257-mimic

mimic: common/config: update values when they are removed via mon

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31875 from smithfarm/wip-42258-mimic
Nathan Cutler [Fri, 27 Mar 2020 19:50:16 +0000 (20:50 +0100)]
Merge pull request #31875 from smithfarm/wip-42258-mimic

mimic: doc, qa: remove invalid option mon_pg_warn_max_per_osd

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #32911 from smithfarm/wip-43502-mimic
Yuri Weinstein [Fri, 27 Mar 2020 19:38:38 +0000 (12:38 -0700)]
Merge pull request #32911 from smithfarm/wip-43502-mimic

mimic: mount.ceph: give a hint message when no mds is up or cluster is laggy

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #33999 from shyukri/wip-42323-mimic
Yuri Weinstein [Thu, 26 Mar 2020 16:09:20 +0000 (09:09 -0700)]
Merge pull request #33999 from shyukri/wip-42323-mimic

mimic: qa: get rid of iterkeys for py3 compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33598 from trociny/wip-44262-mimic
Yuri Weinstein [Wed, 25 Mar 2020 17:58:06 +0000 (10:58 -0700)]
Merge pull request #33598 from trociny/wip-44262-mimic

mimic: rbd-mirror: improve detection of blacklisted state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #33966 from smithfarm/wip-44465-mimic
Yuri Weinstein [Wed, 25 Mar 2020 17:23:47 +0000 (10:23 -0700)]
Merge pull request #33966 from smithfarm/wip-44465-mimic

mimic: rgw: get barbican secret key request maybe return error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #33986 from toabctl/mimic-testtools
Yuri Weinstein [Wed, 18 Mar 2020 18:10:39 +0000 (11:10 -0700)]
Merge pull request #33986 from toabctl/mimic-testtools

mimic: ceph-disk,ceph-detect-init: Pin testtools test requirement for old py versions

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoqa: get rid of iterkeys for py3 compatibility 33999/head
Kyr Shatskyy [Fri, 11 Oct 2019 15:57:47 +0000 (17:57 +0200)]
qa: get rid of iterkeys for py3 compatibility

Fixes: https://tracker.ceph.com/issues/42287
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
(cherry picked from commit 5f95b532aa26249cecea4586166399a0bf10ad47)

Ensured all iterkeys -> keys under qa/tasks
Removed ceph_crash / install_packages methods by PR comments

5 years agoceph-detect-init & ceph-disk: Pin testtools test requirement 33986/head
Thomas Bechtold [Mon, 16 Mar 2020 09:12:55 +0000 (10:12 +0100)]
ceph-detect-init & ceph-disk: Pin testtools test requirement

When running make-check on python 3.4, the test requirements
installation fails with:

ERROR: testtools requires Python '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, \
  !=3.3.*, !=3.4.*' but the running Python is 3.4.6

This happens because there is a new testtols release on pypi from Mar
14, 2020. Pinning to an older version fixes this.

This is not needed for Master, Octopus and Nautilus  given that python
3.4 is already EOL[1] before Nautilus got released.

[1] https://www.python.org/downloads/release/python-3410/

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
5 years agoMerge pull request #33594 from neha-ojha/wip-44022-mimic
Yuri Weinstein [Sat, 14 Mar 2020 15:34:43 +0000 (08:34 -0700)]
Merge pull request #33594 from neha-ojha/wip-44022-mimic

mimic: osd/PeeringState.h: ignore MLogRec in Peering/GetInfo

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #33748 from cbodley/wip-44445
Yuri Weinstein [Sat, 14 Mar 2020 15:33:22 +0000 (08:33 -0700)]
Merge pull request #33748 from cbodley/wip-44445

mimic: rgw: making implicit_tenants backwards compatible

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoget barbican secret key request return error code 33966/head
Richard Bai(白学余) [Tue, 13 Aug 2019 13:16:49 +0000 (21:16 +0800)]
get barbican secret key request return error code

Signed-off-by: Richard Bai(白学余) <baixueyu@inspur.com>
(cherry picked from commit fbe2be57474df43996dd45bf04d1a1137a02c729)

Conflicts:
src/rgw/rgw_http_client.cc
- mimic uses Mutex::Locker instead of std::lock_guard

5 years agorgw: making implicit_tenants backwards compatible. 33748/head
Marcus Watts [Wed, 30 May 2018 20:37:31 +0000 (16:37 -0400)]
rgw: making implicit_tenants backwards compatible.

In jewel, "rgw keystone implicit tenants" only applied to swift. As of
luminous), this option applies to s3 also.
Sites that used this feature with jewel now have outstanding data that
depends on the old behavior.

The fix here is to expand "rgw keystone implicit tenants" so that it
can be set to any of "none", "all", "s3" or "swift" (also 0=false=none,
1=true=all).  When set to "s3" or "swift", the actual id lookup
is also partitioned.

Formerly "rgw keystone implicit tenants" was a legacy opt.
This change converts it to the new style of option,
including support for dynamically changing it.

Fixes: http://tracker.ceph.com/issues/24348
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit a28a38f6e91da3abe59c34fad0e059eeaf29a65f)
(cherry picked from commit d2bdea8e7bc8eae64e4b72f581e424a9a6871aed)

Conflicts:
  - forward port from luminous was easier than backport from master

5 years agoMerge pull request #31384 from less-is-morr/wip-42159-mimic
Yuri Weinstein [Fri, 6 Mar 2020 14:44:25 +0000 (06:44 -0800)]
Merge pull request #31384 from less-is-morr/wip-42159-mimic

mimic: cephfs: osdc/objecter: Fix last_sent in scientific format and add age to ops

Reviewed-by: Milind Changire <mchangir@redhat.com>
5 years agoMerge pull request #31379 from smithfarm/wip-42143-mimic
Yuri Weinstein [Fri, 6 Mar 2020 14:43:57 +0000 (06:43 -0800)]
Merge pull request #31379 from smithfarm/wip-42143-mimic

mimic: mds: split the dir if the op makes it oversized, because some ops maybe in flight

Reviewed-by: Milind Changire <mchangir@redhat.com>
5 years agoMerge pull request #31284 from ukernel/mimic-42156
Yuri Weinstein [Fri, 6 Mar 2020 14:43:27 +0000 (06:43 -0800)]
Merge pull request #31284 from ukernel/mimic-42156

mimic: mds: fix infinite loop in Locker::file_update_finish

Reviewed-by: Milind Changire <mchangir@redhat.com>
5 years agoMerge pull request #31283 from ukernel/mimic-41114
Yuri Weinstein [Fri, 6 Mar 2020 14:42:56 +0000 (06:42 -0800)]
Merge pull request #31283 from ukernel/mimic-41114

mimic: client: more precise CEPH_CLIENT_CAPS_PENDING_CAPSNAP

Reviewed-by: Milind Changire <mchangir@redhat.com>
5 years agoMerge pull request #31282 from ukernel/mimic-38643
Yuri Weinstein [Fri, 6 Mar 2020 14:42:26 +0000 (06:42 -0800)]
Merge pull request #31282 from ukernel/mimic-38643

mimic: mds: avoid calling clientreplay_done() prematurely

Reviewed-by: Milind Changire <mchangir@redhat.com>
5 years agoMerge pull request #31281 from ukernel/mimic-37906
Yuri Weinstein [Fri, 6 Mar 2020 14:41:46 +0000 (06:41 -0800)]
Merge pull request #31281 from ukernel/mimic-37906

mimic: tools/cephfs: make 'cephfs-data-scan scan_links' reconstruct snaptable

Reviewed-by: Milind Changire <mchangir@redhat.com>
5 years agoMerge pull request #31381 from smithfarm/wip-42148-mimic
Yuri Weinstein [Thu, 5 Mar 2020 19:51:38 +0000 (11:51 -0800)]
Merge pull request #31381 from smithfarm/wip-42148-mimic

mimic: mds: mds returns -5(EIO) error when the deleted file does not exist

Reviewed-by: huanwen ren <ren.huanwen@zte.com.cn>
5 years agoMerge pull request #32757 from batrick/i43347-mimic
Yuri Weinstein [Thu, 5 Mar 2020 19:50:18 +0000 (11:50 -0800)]
Merge pull request #32757 from batrick/i43347-mimic

mimic: mds: fix assert(omap_num_objs <= MAX_OBJECTS) of OpenFileTable

Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #31380 from smithfarm/wip-42146-mimic
Yuri Weinstein [Thu, 5 Mar 2020 19:49:41 +0000 (11:49 -0800)]
Merge pull request #31380 from smithfarm/wip-42146-mimic

mimic: client: remove Inode.dir_contacts field and handle bad whence value to llseek gracefully

Reviewed-by: Varsha Rao <varao@redhat.com>
5 years agoMerge pull request #30110 from smithfarm/wip-40494-mimic
Yuri Weinstein [Thu, 5 Mar 2020 19:48:15 +0000 (11:48 -0800)]
Merge pull request #30110 from smithfarm/wip-40494-mimic

mimic: cephfs: test_volume_client: declare only one default for python version

Reviewed-by: Venky Shankar <vshankar@redhat.com>
5 years agoMerge pull request #33626 from jan--f/wip-38472-notracker-mimic
Jan Fajerski [Fri, 28 Feb 2020 17:40:06 +0000 (18:40 +0100)]
Merge pull request #33626 from jan--f/wip-38472-notracker-mimic

mimic: ceph-volume: don't create osd['block.db'] by default

5 years agoceph-volume: don't create osd['block.db'] by default 33626/head
Jan Fajerski [Mon, 25 Feb 2019 11:55:42 +0000 (12:55 +0100)]
ceph-volume: don't create osd['block.db'] by default

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f648ba1d9a1ef12fa8dc35101ef485726c002987)

5 years agoMerge pull request #33622 from jan--f/wip-32828-notracker-mimic
Jan Fajerski [Fri, 28 Feb 2020 17:00:46 +0000 (18:00 +0100)]
Merge pull request #33622 from jan--f/wip-32828-notracker-mimic

mimic: ceph-volume: add db and wal support to raw mode

5 years agoMerge pull request #33623 from jan--f/wip-43872-mimic-encore
Jan Fajerski [Fri, 28 Feb 2020 17:00:36 +0000 (18:00 +0100)]
Merge pull request #33623 from jan--f/wip-43872-mimic-encore

mimic: ceph-volume: batch bluestore fix create_lvs call

5 years agoceph-volume: batch bluestore fix create_lvs call 33623/head
Jan Fajerski [Tue, 28 Jan 2020 08:25:39 +0000 (09:25 +0100)]
ceph-volume: batch bluestore fix create_lvs call

Fixes: https://tracker.ceph.com/issues/43844
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit df18497bc9eaf1922e5c885e8cc124e439c59364)

5 years agoceph-volume: add db and wal support to raw mode 33622/head
Sébastien Han [Fri, 24 Jan 2020 15:29:54 +0000 (16:29 +0100)]
ceph-volume: add db and wal support to raw mode

Using the raw mode, the OSD preparation can now be called with extra
arguments to specify a block device for either rocksdb db or wal.

The code expects the device to be available and no validation will be
performed. Also, the entire block device will be consumed.

The goal is run this on Kubernetes with Rook and OSD are running on PVC.
Users will request PVC from a certain size, the request will be
acknowledged and a block will be created, then attached to the machine.
Later on consumed by Rook's preparation job.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit c8f23c7182e2681f2bc756776a890995aca1efd2)

5 years agoceph-volume: fix raw list
Sébastien Han [Fri, 24 Jan 2020 16:25:42 +0000 (17:25 +0100)]
ceph-volume: fix raw list

There is no json anymore, just plain output.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit e99eb8cc95c5f50ec13170f202226c3581e3f5e2)

5 years agoceph-volume: remove unnecessary comment
Sébastien Han [Fri, 24 Jan 2020 15:28:17 +0000 (16:28 +0100)]
ceph-volume: remove unnecessary comment

This comment is not relevant since we will never bring systemd support
to the raw mode.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit af3f5fded95e4dcc0dfa2a0b2eec5fb2c4252b6c)

5 years agoceph-volume: remove osd-id and osd-fsid flag
Sébastien Han [Fri, 24 Jan 2020 15:27:10 +0000 (16:27 +0100)]
ceph-volume: remove osd-id and osd-fsid flag

These are not used since we only rely on the device path so removing
them to avid confusion.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit 525957be98a0a6ff742f6bae4450f291591f516f)

5 years agoceph-volume: remove "create" call from raw mode
Sébastien Han [Fri, 24 Jan 2020 15:23:10 +0000 (16:23 +0100)]
ceph-volume: remove "create" call from raw mode

This mode is intented to work when the cluster is running on Kubernetes
and OSDs are running on PVC.
So we always have a "prepare" pod and an "activate" pod.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit c7a2ecc59394a9eb323a19ff233a2e3dfe04a018)

5 years agoMerge pull request #33615 from jan--f/wip-44154-mimic
Jan Fajerski [Fri, 28 Feb 2020 15:18:38 +0000 (16:18 +0100)]
Merge pull request #33615 from jan--f/wip-44154-mimic

mimic: ceph-volume: don't remove vg twice when zapping filestore

5 years agoMerge pull request #33617 from jan--f/wip-44326-mimic
Jan Fajerski [Fri, 28 Feb 2020 15:17:47 +0000 (16:17 +0100)]
Merge pull request #33617 from jan--f/wip-44326-mimic

mimic: ceph-volume: lvm: get_device_vgs() filter by provided prefix

5 years agoMerge pull request #33618 from jan--f/wip-44332-mimic
Jan Fajerski [Fri, 28 Feb 2020 15:17:36 +0000 (16:17 +0100)]
Merge pull request #33618 from jan--f/wip-44332-mimic

mimic: ceph-volume: strip _dmcrypt suffix in simple scan json output

5 years agoceph-volume: strip _dmcrypt suffix in simple scan json output 33618/head
Jan Fajerski [Wed, 5 Feb 2020 05:01:16 +0000 (06:01 +0100)]
ceph-volume: strip _dmcrypt suffix in simple scan json output

LUKS encrypted OSDs name their block* files with a _dmcrypt suffix.
activate fails on json files like this. Stripping this suffix in scan
fixes this.

Fixes: https://tracker.ceph.com/issues/43966
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2ddf76d118d77659c590ea076d34ce9a8e351a86)

5 years agoceph-volume: don't create Volume/VolumeGroup objects with empty name 33617/head
Jan Fajerski [Tue, 25 Feb 2020 15:18:57 +0000 (16:18 +0100)]
ceph-volume: don't create Volume/VolumeGroup objects with empty name

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 235b19eb66b9ba9beab99508f2fdf7b6afe26f31)

5 years agoceph-volume: get_device_lvs filter by provided prefix
Jan Fajerski [Tue, 25 Feb 2020 15:18:24 +0000 (16:18 +0100)]
ceph-volume: get_device_lvs filter by provided prefix

Fixes: https://tracker.ceph.com/issues/44124
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ac43d37f581510d1854e76afb634a142ad9a0fc0)

5 years agoceph-volume: lvm.get_device_vgs: don't create VolumeGroup with empty name
Jan Fajerski [Tue, 25 Feb 2020 15:27:08 +0000 (16:27 +0100)]
ceph-volume: lvm.get_device_vgs: don't create VolumeGroup with empty name

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 3c4c8c5e4a242985389c8d1c85b45cac89b2dc89)

5 years agoceph-volume: lvm: get_device_vgs() filter by provided prefix
Yehuda Sadeh [Sat, 22 Feb 2020 01:44:11 +0000 (17:44 -0800)]
ceph-volume: lvm: get_device_vgs() filter by provided prefix

Fixes: https://tracker.ceph.com/issues/44246
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit b48caf4873c11690d36a88760504f98b46101995)

5 years agoceph-volume: don't remove vg twice when zapping filestore 33615/head
Jan Fajerski [Fri, 14 Feb 2020 13:10:36 +0000 (14:10 +0100)]
ceph-volume: don't remove vg twice when zapping filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: https://tracker.ceph.com/issues/44149
(cherry picked from commit bccdf6eafaf851d5092bb99d61edd44cd36d9dd2)

5 years agoMerge pull request #33610 from jan--f/wip-44134-mimic
Jan Fajerski [Fri, 28 Feb 2020 13:59:11 +0000 (14:59 +0100)]
Merge pull request #33610 from jan--f/wip-44134-mimic

mimic: ceph-volume: avoid calling zap_lv with a LV-less VG

5 years agoMerge pull request #33611 from jan--f/wip-44151-mimic
Jan Fajerski [Fri, 28 Feb 2020 13:58:43 +0000 (14:58 +0100)]
Merge pull request #33611 from jan--f/wip-44151-mimic

mimic: ceph-volume: pass journal_size as Size not string

5 years agoMerge pull request #33609 from jan--f/wip-44111-mimic
Jan Fajerski [Fri, 28 Feb 2020 13:58:25 +0000 (14:58 +0100)]
Merge pull request #33609 from jan--f/wip-44111-mimic

mimic: ceph-volume: use get_device_vgs in has_common_vg

5 years agoMerge pull request #33606 from jan--f/wip-44036-mimic
Jan Fajerski [Fri, 28 Feb 2020 13:54:33 +0000 (14:54 +0100)]
Merge pull request #33606 from jan--f/wip-44036-mimic

mimic: ceph-volume: finer grained availability notion in inventory

5 years agoMerge pull request #33607 from jan--f/wip-44048-mimic
Jan Fajerski [Fri, 28 Feb 2020 13:50:01 +0000 (14:50 +0100)]
Merge pull request #33607 from jan--f/wip-44048-mimic

mimic: ceph-volume: skip osd creation when already done

5 years agoMerge pull request #33608 from jan--f/wip-44110-mimic
Jan Fajerski [Fri, 28 Feb 2020 13:47:10 +0000 (14:47 +0100)]
Merge pull request #33608 from jan--f/wip-44110-mimic

mimic: ceph-volume: fix is_ceph_device for lvm batch