]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoqa/suites/rados/cephadm: collect all cephadm tests together here 33062/head
Sage Weil [Fri, 7 Feb 2020 14:43:38 +0000 (08:43 -0600)]
qa/suites/rados/cephadm: collect all cephadm tests together here

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/workunits/cephadm/test_repos: add test for the repo commands
Sage Weil [Thu, 6 Feb 2020 17:02:31 +0000 (11:02 -0600)]
qa/workunits/cephadm/test_repos: add test for the repo commands

This isn't a great test, but it is something.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agocephadm: add '{add,rm}-repo', with initial centos/rhel/fedora/ubuntu support
Sage Weil [Tue, 4 Feb 2020 14:28:05 +0000 (08:28 -0600)]
cephadm: add '{add,rm}-repo', with initial centos/rhel/fedora/ubuntu support

Other distros to follow.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33111 into master
Sage Weil [Fri, 7 Feb 2020 17:10:29 +0000 (11:10 -0600)]
Merge PR #33111 into master

* refs/pull/33111/head:
qa/suites/rados/cephadm[-smoke]: test podman on ubuntu 18.04

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge pull request #33128 from pponnuvel/fix_incorrect_link_documentation
Kefu Chai [Fri, 7 Feb 2020 16:42:50 +0000 (00:42 +0800)]
Merge pull request #33128 from pponnuvel/fix_incorrect_link_documentation

coding-style: update a link and fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocoding-style: update a link and fix typos. 33128/head
Ponnuvel Palaniyappan [Fri, 7 Feb 2020 15:55:08 +0000 (15:55 +0000)]
coding-style: update a link and fix typos.

Signed-off-by: Ponnuvel Palaniyappan <pponnuvel@gmail.com>
5 years agoMerge PR #33042 into master
Sage Weil [Fri, 7 Feb 2020 14:47:52 +0000 (08:47 -0600)]
Merge PR #33042 into master

* refs/pull/33042/head:
cephadm: bootstrap: warn on fqdn hostname

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoqa/suites/rados/cephadm[-smoke]: test podman on ubuntu 18.04 33111/head
Sage Weil [Thu, 6 Feb 2020 14:54:06 +0000 (08:54 -0600)]
qa/suites/rados/cephadm[-smoke]: test podman on ubuntu 18.04

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32634 from jan--f/c-v-inventory-fixes
Jan Fajerski [Fri, 7 Feb 2020 13:46:25 +0000 (14:46 +0100)]
Merge pull request #32634 from jan--f/c-v-inventory-fixes

ceph-volume: finer grained availability notion in inventory.

5 years agoMerge pull request #31978 from jan--f/c-v-batch-no-db-dev-drop
Jan Fajerski [Fri, 7 Feb 2020 13:41:27 +0000 (14:41 +0100)]
Merge pull request #31978 from jan--f/c-v-batch-no-db-dev-drop

ceph-volume/batch: fail on filtered devices when non-interactive

5 years agoMerge PR #33075 into master
Sage Weil [Fri, 7 Feb 2020 13:16:14 +0000 (07:16 -0600)]
Merge PR #33075 into master

* refs/pull/33075/head:
examples/librados: fix bufferlist::copy() in hello_world.cc.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoceph-volume: add available property in target specific flavors 32634/head
Jan Fajerski [Mon, 6 Jan 2020 17:02:57 +0000 (18:02 +0100)]
ceph-volume: add available property in target specific flavors

This adds two properties available_[lvm,raw] to device (and thus inventory).
The goal is to have different notions of availability based on the
intended use case. For example finding LVM structures make a drive
unavailable for the raw mode, but might be available for the lvm mode.

Fixes: https://tracker.ceph.com/issues/43400
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #33112 from jan--f/c-v-lvm-list-regression-31700
Jan Fajerski [Fri, 7 Feb 2020 10:30:01 +0000 (11:30 +0100)]
Merge pull request #33112 from jan--f/c-v-lvm-list-regression-31700

ceph-volume: fix regression and improve output in lvm list

5 years agoMerge pull request #32985 from sebastian-philipp/mgr-progress-mypy
Kefu Chai [Fri, 7 Feb 2020 07:49:37 +0000 (15:49 +0800)]
Merge pull request #32985 from sebastian-philipp/mgr-progress-mypy

mgr/progress: Add integration to pybind/mgr/tox.ini

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #32432 into master
Patrick Donnelly [Fri, 7 Feb 2020 03:29:20 +0000 (19:29 -0800)]
Merge PR #32432 into master

* refs/pull/32432/head:
mds: Reorganize structure members in snap header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #33114 into master
Sage Weil [Thu, 6 Feb 2020 22:12:39 +0000 (16:12 -0600)]
Merge PR #33114 into master

* refs/pull/33114/head:
cephadm:Fix name argument parsing during image check for non-ceph components

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agocephadm:Fix name argument parsing during image check for non-ceph components 33114/head
Daniel-Pivonka [Thu, 6 Feb 2020 20:47:26 +0000 (15:47 -0500)]
cephadm:Fix name argument parsing during image check for non-ceph components

bug in parsing introduced in 97def7c
args.name may exist but will be none if flag is not used
check the value in addition to checking if it exists

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
5 years agoMerge PR #33109 into master
Sage Weil [Thu, 6 Feb 2020 20:42:46 +0000 (14:42 -0600)]
Merge PR #33109 into master

* refs/pull/33109/head:
cephadm: fix inspect-image

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #33089 into master
Sage Weil [Thu, 6 Feb 2020 20:42:34 +0000 (14:42 -0600)]
Merge PR #33089 into master

* refs/pull/33089/head:
cephadm: re-introduce the `podman logs` command

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33110 into master
Sage Weil [Thu, 6 Feb 2020 19:13:48 +0000 (13:13 -0600)]
Merge PR #33110 into master

* refs/pull/33110/head:
qa/distros: rhel and centos: whitelist cephadm logrotate selinux denial

Reviewed-by: Boris Ranto <branto@redhat.com>
5 years agoceph-volume: fix various lvm list issues 33112/head
Jan Fajerski [Thu, 6 Feb 2020 15:49:12 +0000 (16:49 +0100)]
ceph-volume: fix various lvm list issues

A single report on a non-lvm device now works.
Format was cleaned up, report lvm journal,wal, db only once.

Fixes: https://tracker.ceph.com/issues/44009
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
5 years agoceph-volume: add get_device_lvs to easily retrieve all lvs per device
Jan Fajerski [Thu, 6 Feb 2020 15:47:08 +0000 (16:47 +0100)]
ceph-volume: add get_device_lvs to easily retrieve all lvs per device

Also drop the sep argument from get_lvs and siblings, unused.
Introduce LV_CMD_OPTIONS to unify options to lvs.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
5 years agocephadm: re-introduce the `podman logs` command 33089/head
Michael Fritch [Wed, 5 Feb 2020 16:02:02 +0000 (09:02 -0700)]
cephadm: re-introduce the `podman logs` command

Fixes: https://tracker.ceph.com/issues/43973
Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #33093 into master
Sage Weil [Thu, 6 Feb 2020 15:22:21 +0000 (09:22 -0600)]
Merge PR #33093 into master

* refs/pull/33093/head:
build-integration-branch: don't fail on existing branch

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agocephadm: fix inspect-image 33109/head
Sage Weil [Thu, 6 Feb 2020 15:18:13 +0000 (09:18 -0600)]
cephadm: fix inspect-image

This was broken by d8debba782cd4f40ed13db7f1af8ef43503ccec5
because the 'images' json output works with podman but not with
docker.  (Also, the inspect command is more explicit and cleaner.)

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/distros: rhel and centos: whitelist cephadm logrotate selinux denial 33110/head
Sage Weil [Thu, 6 Feb 2020 14:22:40 +0000 (08:22 -0600)]
qa/distros: rhel and centos: whitelist cephadm logrotate selinux denial

This is fixed in RHEL 8.1.1 (and by extension centos/rhel 8.2+).

No fix for el 7 yet

Partially-fixes: https://tracker.ceph.com/issues/43703
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33071 into master
Sage Weil [Thu, 6 Feb 2020 12:33:36 +0000 (06:33 -0600)]
Merge PR #33071 into master

* refs/pull/33071/head:
mgr/cephadm: remove item from cache when removing

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #33059 from tspmelo/wip-node-10-18-1
Lenz Grimmer [Thu, 6 Feb 2020 11:34:56 +0000 (11:34 +0000)]
Merge pull request #33059 from tspmelo/wip-node-10-18-1

make-dist: Bump Node.js to v10.18.1

Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #32546 from votdev/issue_43089_passwd_cmplx_config
Tatjana Dehler [Thu, 6 Feb 2020 08:44:48 +0000 (09:44 +0100)]
Merge pull request #32546 from votdev/issue_43089_passwd_cmplx_config

mgr/dashboard: Make password policy check configurable

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #32881 from tchaikov/wip-43657 31568/head
Kefu Chai [Thu, 6 Feb 2020 08:05:31 +0000 (16:05 +0800)]
Merge pull request #32881 from tchaikov/wip-43657

mgr/orchestrator: use deepcopy for copying exceptions

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoqa/tasks/cephadm: test "orchestrator host ls" 32881/head
Kefu Chai [Fri, 31 Jan 2020 06:01:52 +0000 (14:01 +0800)]
qa/tasks/cephadm: test "orchestrator host ls"

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoqa/tasks: drop test_cephadm_orchestrator.py
Kefu Chai [Fri, 31 Jan 2020 05:52:49 +0000 (13:52 +0800)]
qa/tasks: drop test_cephadm_orchestrator.py

this test will end with a failure like

```
2020-01-30T18:15:15.870 INFO:tasks.ceph.mgr.x.smithi042.stderr:Warning: Permanently added 'smithi042.front.sepia.ceph.com,172.21.15.42' (ECDSA) to the list of known hosts.
2020-01-30T18:15:15.925 INFO:tasks.ceph.mgr.x.smithi042.stderr:Permission denied, please try again.
2020-01-30T18:15:15.932 INFO:tasks.ceph.mgr.x.smithi042.stderr:Permission denied, please try again.
2020-01-30T18:15:15.939 INFO:tasks.ceph.mgr.x.smithi042.stderr:root@smithi042.front.sepia.ceph.com: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
```

because mgr is not able to establish an ssh connection to that host with "root".
please note, the teuthology worker is acting using the "ubuntu" account on the
test node, and by default, "root" does not have its pubkey. and actually
`qa/tasks/cephadm.py` does push the pubkey to all the managed hosts before
testing cephadm.

since `qa/tasks/cephadm.py` is a better test for cephadm, let's just
drop this one.

as suites/rados/cephadm already covers cephadm

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agomgr/orchestrator: use deepcopy for copying exceptions
Kefu Chai [Wed, 29 Jan 2020 15:27:47 +0000 (23:27 +0800)]
mgr/orchestrator: use deepcopy for copying exceptions

since rexec module has been removed in python3, we cannot use it
anymore.

Fixes: https://tracker.ceph.com/issues/43657
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agobuild-integration-branch: don't fail on existing branch 33093/head
Sage Weil [Wed, 5 Feb 2020 22:56:42 +0000 (16:56 -0600)]
build-integration-branch: don't fail on existing branch

This behavior is too annoying, and you can always get back to something
clobbered with 'git reflog'.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #32946 into master
Sage Weil [Wed, 5 Feb 2020 22:47:21 +0000 (16:47 -0600)]
Merge PR #32946 into master

* refs/pull/32946/head:
qa/suites/rados: improve valgrind leak check
common/ceph_context: add an asok command to deliberately leak memory

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #33055 into master
Sage Weil [Wed, 5 Feb 2020 22:09:12 +0000 (16:09 -0600)]
Merge PR #33055 into master

* refs/pull/33055/head:
qa/tasks/mgr/test_orchestrator_cli: support multiple DriveGroups

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #33056 into master
Sage Weil [Wed, 5 Feb 2020 22:08:40 +0000 (16:08 -0600)]
Merge PR #33056 into master

* refs/pull/33056/head:
common: fix clang compile errors from cython_modules

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #33058 into master
Sage Weil [Wed, 5 Feb 2020 22:07:43 +0000 (16:07 -0600)]
Merge PR #33058 into master

* refs/pull/33058/head:
mgr/cephadm: enforce that a host is a valid DNS name
mgr/cephadm: verify host's hostname matches our host name
cephadm: check-host: add optional --expect-hostname

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge PR #33069 into master
Sage Weil [Wed, 5 Feb 2020 22:07:28 +0000 (16:07 -0600)]
Merge PR #33069 into master

* refs/pull/33069/head:
cephadm: use appropriate default image for non-ceph components

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #33039 into master
Sage Weil [Wed, 5 Feb 2020 18:55:53 +0000 (12:55 -0600)]
Merge PR #33039 into master

* refs/pull/33039/head:
osd/OSD: prevent down osds from immediately rejoining the culster
osd/OSD: trim osd_markdown_log in tick() thread

Reviewed-by: yanjun <yan.jun8@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: enforce that a host is a valid DNS name 33058/head
Sage Weil [Wed, 5 Feb 2020 17:19:59 +0000 (11:19 -0600)]
mgr/cephadm: enforce that a host is a valid DNS name

This combines the hostname restrictions

 * 1-63 chars
 * a-z, A-Z, 0-9, -

and the DNS name restrictions

 * .-delimited
 * no empty components (or leading or trailing .)
 * 250 chars total max

Note that this allows bare IPv4 addresses (which are indistinguishable from
a valid DNS name, AFAICS), but disallows bare IPv6 addresses.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: remove item from cache when removing 33071/head
Sage Weil [Tue, 4 Feb 2020 19:06:49 +0000 (13:06 -0600)]
mgr/cephadm: remove item from cache when removing

This makes the daemon disappear immediately from 'service ls', and also
avoids a temporary health warning about a stray service.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/dashboard: Fix fsevents and node-gyp error 33059/head
Tiago Melo [Wed, 5 Feb 2020 16:54:48 +0000 (15:54 -0100)]
mgr/dashboard: Fix fsevents and node-gyp error

Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agomake-dist: Bump Node.js to v10.18.1
Tiago Melo [Tue, 4 Feb 2020 13:54:22 +0000 (12:54 -0100)]
make-dist: Bump Node.js to v10.18.1

This will fix an error caused by the usage of the latest version of Angular CLI
and Node.js v10.16.0.

Fixes: https://tracker.ceph.com/issues/43961
Signed-off-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #33085 from rzarzynski/wip-client-bl_iter_advance
Kefu Chai [Wed, 5 Feb 2020 16:26:12 +0000 (00:26 +0800)]
Merge pull request #33085 from rzarzynski/wip-client-bl_iter_advance

client: fix FTBFS due to bl::iterator::advance().

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoclient: fix FTBFS due to bl::iterator::advance(). 33085/head
Radoslaw Zarzynski [Wed, 5 Feb 2020 16:08:34 +0000 (17:08 +0100)]
client: fix FTBFS due to bl::iterator::advance().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge pull request #33026 from liewegas/wip-el81
Kefu Chai [Wed, 5 Feb 2020 15:44:15 +0000 (23:44 +0800)]
Merge pull request #33026 from liewegas/wip-el81

qa/distros: add rhel/centos 8.1

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #32996 from cbodley/wip-rgw-put-multipart-stripe
Casey Bodley [Wed, 5 Feb 2020 13:42:50 +0000 (08:42 -0500)]
Merge pull request #32996 from cbodley/wip-rgw-put-multipart-stripe

rgw: MultipartObjectProcessor supports stripe size > chunk size

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
5 years agoMerge pull request #32811 from multi-arch/master
Casey Bodley [Wed, 5 Feb 2020 13:40:08 +0000 (08:40 -0500)]
Merge pull request #32811 from multi-arch/master

test/rgw: fix test_rgw_reshard_wait with -DHAVE_BOOST_CONTEXT=OFF

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge PR #33051 into master
Sage Weil [Wed, 5 Feb 2020 13:36:01 +0000 (07:36 -0600)]
Merge PR #33051 into master

* refs/pull/33051/head:
mgr/telemetry: check get_metadata return val

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33018 from mgfritch/cephadm-docker-disabled
Kefu Chai [Wed, 5 Feb 2020 12:46:16 +0000 (20:46 +0800)]
Merge pull request #33018 from mgfritch/cephadm-docker-disabled

qa/workunits/cephadm/test_cephadm.sh: skip docker when service is disabled

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32982 from krig/cephadm-fixes
Kefu Chai [Wed, 5 Feb 2020 12:44:58 +0000 (20:44 +0800)]
Merge pull request #32982 from krig/cephadm-fixes

cephadm: Read ceph version from io.ceph.version label if set

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33025 from neha-ojha/wip-no-mgr
Kefu Chai [Wed, 5 Feb 2020 12:36:12 +0000 (20:36 +0800)]
Merge pull request #33025 from neha-ojha/wip-no-mgr

mon/MgrMonitor.cc: warn about missing mgr in a cluster with osds

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33077 from guits/guits-cv_fix_listing
Jan Fajerski [Wed, 5 Feb 2020 12:29:43 +0000 (13:29 +0100)]
Merge pull request #33077 from guits/guits-cv_fix_listing

ceph-volume: fix lvm list

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #33004 from matthewoliver/argparge_matchcnt_kwargs
Kefu Chai [Wed, 5 Feb 2020 12:29:35 +0000 (20:29 +0800)]
Merge pull request #33004 from matthewoliver/argparge_matchcnt_kwargs

ceph_argparse: increment matchcnt on kwargs

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33074 from guits/guits-cv_quick_fix
Jan Fajerski [Wed, 5 Feb 2020 12:29:03 +0000 (13:29 +0100)]
Merge pull request #33074 from guits/guits-cv_quick_fix

ceph-volume: fix has_bluestore_label() function

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #33015 from rouming/double-unlock-p1-fix
Kefu Chai [Wed, 5 Feb 2020 12:18:56 +0000 (20:18 +0800)]
Merge pull request #33015 from rouming/double-unlock-p1-fix

msg/async: open() should be called with connection locked

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
5 years agoMerge pull request #33029 from yaarith/wip-telemetry-show-device
Kefu Chai [Wed, 5 Feb 2020 12:11:39 +0000 (20:11 +0800)]
Merge pull request #33029 from yaarith/wip-telemetry-show-device

mgr/telemetry: anonymizing smartctl report itself

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33003 from tchaikov/wip-buffer-list-advance
Kefu Chai [Wed, 5 Feb 2020 12:10:42 +0000 (20:10 +0800)]
Merge pull request #33003 from tchaikov/wip-buffer-list-advance

include/buffer: add operator+=() for list::iterator

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge pull request #33034 from rzarzynski/wip-include-ceph_atomic_in_mempool
Kefu Chai [Wed, 5 Feb 2020 12:00:46 +0000 (20:00 +0800)]
Merge pull request #33034 from rzarzynski/wip-include-ceph_atomic_in_mempool

include: switch mempool.h to ceph::atomic.

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoceph-volume: remove stderr in has_bluestore_label() 33074/head
Guillaume Abrioux [Wed, 5 Feb 2020 01:15:17 +0000 (02:15 +0100)]
ceph-volume: remove stderr in has_bluestore_label()

We don't want to generate this log when a call to
`has_bluestore_label()` fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
5 years agoceph-volume: fix has_bluestore_label() function
Guillaume Abrioux [Tue, 4 Feb 2020 21:02:26 +0000 (22:02 +0100)]
ceph-volume: fix has_bluestore_label() function

When using vg/lv, this function throws an error like following:

```
 stderr: unable to read label for test_group/data-lv2: (2) No such file or directory
 stderr: 2020-02-04T21:03:32.153+0000 7fe091af4200 -1 bluestore(test_group/data-lv2) _read_bdev_label failed to open test_group/data-lv2: (2) No such file or directory
```

using `self.abspath` fixes this error.

Fixes: https://tracker.ceph.com/issues/43970
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
5 years agoceph-volume: fix lvm list 33077/head
Guillaume Abrioux [Wed, 5 Feb 2020 01:29:14 +0000 (02:29 +0100)]
ceph-volume: fix lvm list

17957d9beb42a04b8f180ccb7ba07d43179a41d3 introduced a regression in `lvm
list`.

When passing a vg/lv path for generating a single report, it fails
because the filter used in the `lvs` command isn't right. It uses the lv
name instead of the vg name because `os.path.basename(device)` is used
while it should be `os.path.dirname(device)`

Fixes: https://tracker.ceph.com/issues/43969
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
5 years agoMerge pull request #33076 from liu-chunmei/fix_crimson_pg_coll
Kefu Chai [Wed, 5 Feb 2020 04:50:25 +0000 (12:50 +0800)]
Merge pull request #33076 from liu-chunmei/fix_crimson_pg_coll

crimson: fix crimson pg coll usage error

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33057 from wjwithagen/wjw-fix-buffer.copy
Kefu Chai [Wed, 5 Feb 2020 03:15:37 +0000 (11:15 +0800)]
Merge pull request #33057 from wjwithagen/wjw-fix-buffer.copy

rbd-ggate: fix fallout from bufferlist.copy() change

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson: fix crimson pg coll usage error 33076/head
Chunmei Liu [Wed, 5 Feb 2020 00:42:21 +0000 (16:42 -0800)]
crimson: fix crimson pg coll usage error

coll in pg isn't initialized, should use coll_ref->get_cid() insead.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
5 years agocephadm: use appropriate default image for non-ceph components 33069/head
Sage Weil [Tue, 4 Feb 2020 16:27:58 +0000 (10:27 -0600)]
cephadm: use appropriate default image for non-ceph components

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33066 from gregsfortytwo/wip-specfile
Gregory Farnum [Tue, 4 Feb 2020 22:08:14 +0000 (14:08 -0800)]
Merge pull request #33066 from gregsfortytwo/wip-specfile

rpm: fix up a specfile syntax error

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoexamples/librados: fix bufferlist::copy() in hello_world.cc. 33075/head
Radoslaw Zarzynski [Tue, 4 Feb 2020 21:31:07 +0000 (22:31 +0100)]
examples/librados: fix bufferlist::copy() in hello_world.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge pull request #30033 from theanalyst/rgw/public-buckets
Abhishek L [Tue, 4 Feb 2020 18:01:05 +0000 (19:01 +0100)]
Merge pull request #30033 from theanalyst/rgw/public-buckets

rgw: add PublicAccessBlock set of APIs on buckets

Reviewed-By: Casey Bodley <cbodley@redhat.com>
5 years agorpm: fix up a specfile syntax error 33066/head
Greg Farnum [Tue, 4 Feb 2020 17:10:17 +0000 (09:10 -0800)]
rpm: fix up a specfile syntax error

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
5 years agoMerge pull request #33049 from yuvalif/wip-yuval-fix-43768
Casey Bodley [Tue, 4 Feb 2020 16:44:46 +0000 (11:44 -0500)]
Merge pull request #33049 from yuvalif/wip-yuval-fix-43768

qa/rgw/pubsub: fix tests to sync from master

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agomgr/cephadm: verify host's hostname matches our host name
Sage Weil [Tue, 4 Feb 2020 13:23:13 +0000 (07:23 -0600)]
mgr/cephadm: verify host's hostname matches our host name

This ensures that we have a consistent view between the hosts that cephadm
is using and the hostname being reported as metadata by the ceph daemons.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agocephadm: check-host: add optional --expect-hostname
Sage Weil [Tue, 4 Feb 2020 13:17:30 +0000 (07:17 -0600)]
cephadm: check-host: add optional --expect-hostname

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agodoc: add Pending Release Notes entry on public access config 30033/head
Abhishek Lekshmanan [Tue, 4 Feb 2020 16:28:46 +0000 (17:28 +0100)]
doc: add Pending Release Notes entry on public access config

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #32119 from joke-lee/rgw-sts-crash-duration-invalid
Abhishek L [Tue, 4 Feb 2020 16:24:12 +0000 (17:24 +0100)]
Merge pull request #32119 from joke-lee/rgw-sts-crash-duration-invalid

rgw: fix rgw crash when duration is invalid in sts request

5 years agoMerge pull request #31987 from linuxbox2/rgw-putacls-no-body
Abhishek L [Tue, 4 Feb 2020 16:22:35 +0000 (17:22 +0100)]
Merge pull request #31987 from linuxbox2/rgw-putacls-no-body

rgw: s3: don't require a body in S3 put-object-acl

5 years agoMerge pull request #30684 from theanalyst/rgw/qa/rgw-admin-user-stats
Abhishek L [Tue, 4 Feb 2020 16:21:25 +0000 (17:21 +0100)]
Merge pull request #30684 from theanalyst/rgw/qa/rgw-admin-user-stats

qa: radosgw_admin: validate a simple user stats output

Reviewed-By: Casey Bodley <cbodley@redhat.com>
5 years agoMerge PR #33048 into master
Sage Weil [Tue, 4 Feb 2020 13:08:22 +0000 (07:08 -0600)]
Merge PR #33048 into master

* refs/pull/33048/head:
cephadm: fix error handling in `command_check_host()`
cephadm: increase default retry_max value

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #32995 into master
Sage Weil [Tue, 4 Feb 2020 13:07:01 +0000 (07:07 -0600)]
Merge PR #32995 into master

* refs/pull/32995/head:
cephadm: add group 'disk' to privileged container
cephadm: adopt: disable ceph-volume unit on host
cephadm: refactor privileged arg handling
cephadm: chown block symlink targets
cephadm-adoption-corpus: add stud-mon.tgz
cephadm: adopt: rename leveldb *.ldb -> *.sst
cephadm: adopt: by default, pull first
cephadm: adopt: chown data content

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agorbd-ggate: fix fallout from bufferlist.copy() change 33057/head
Willem Jan Withagen [Tue, 4 Feb 2020 11:19:43 +0000 (12:19 +0100)]
rbd-ggate: fix fallout from bufferlist.copy() change

fixes: #3281
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agocommon: fix clang compile errors from cython_modules 33056/head
Mark Kogan [Tue, 4 Feb 2020 10:30:55 +0000 (12:30 +0200)]
common: fix clang compile errors from cython_modules

Signed-off-by: Mark Kogan <mkogan@redhat.com>
5 years agoqa/tasks/mgr/test_orchestrator_cli: support multiple DriveGroups 33055/head
Kiefer Chang [Tue, 4 Feb 2020 06:27:17 +0000 (14:27 +0800)]
qa/tasks/mgr/test_orchestrator_cli: support multiple DriveGroups

create_osds interface in Orchestrator supports multiple named DriveGroups
since https://github.com/ceph/ceph/pull/32972. Adapt the changes in
the test.

Fixes: https://tracker.ceph.com/issues/43945
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
5 years agoMerge PR #33020 into master
Sage Weil [Tue, 4 Feb 2020 03:28:40 +0000 (21:28 -0600)]
Merge PR #33020 into master

* refs/pull/33020/head:
osdc/Objecter: inline pool full check
osdc/Objecter: remove duplicated pause check code
osdc/Objecter: only pause if respects_full()
osdc/Objecter: move respects_full() to op_target_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #32831 into master
Sage Weil [Tue, 4 Feb 2020 03:28:19 +0000 (21:28 -0600)]
Merge PR #32831 into master

* refs/pull/32831/head:
common, include: drop the copy{_in} from bufferlist entirely.
os/bluestore: switch copy_in() users to bufferlist::iterator.
osdc: switch users of bufferlist::copy{_in} to iterators.
osd: switch users of bufferlist::copy{_in} to iterators.
rgw: switch copy{_in} users to bufferlist::iterator.
ec: switch users of bufferlist::copy{_in} to iterators.
cls/queue: switch users of bufferlist::copy{_in} to iterators.
client: switch users of bufferlist::copy{_in} to iterators.
*: switch trivial users of bufferlist::copy{_in} to iterators.
test/bl: switch copy{_in} users to bufferlist::iterator.
common, include: kill the bl::last_p member.
common: encode for std::list<T> doesn't use bl::copy_in() anymore.

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoceph_argparse: increment matchcnt on kwargs 33004/head
Matthew Oliver [Tue, 4 Feb 2020 02:29:48 +0000 (13:29 +1100)]
ceph_argparse: increment matchcnt on kwargs

Currently when you pass a param in on the ceph cli as a kwarg
(--<param_name>) the matchcnt isn't incremented in the validate method
which is used to choose the right command signature.

This means currently things fail like:

  ceph orchestrator rgw rm --realm_name=default --zone_name=default
  no valid command found; 1 closest matches:
  orchestrator rgw rm <realm_name> <zone_name>
  Error EINVAL: invalid command

The '--realm_name' and '--zone_name' isn't counted to the matchcnt, so
'orchestrator rgw rm' isn't picked as the valid command.

This patch simply corrects this by incrementing matchcnt on the kwarg
validate path before calling shortcircuiting the loop.

Fixes: https://tracker.ceph.com/issues/43803
Signed-off-by: Matthew Oliver <moliver@suse.com>
5 years agomgr/telemetry: check get_metadata return val 33051/head
Yaarit Hatuka [Mon, 3 Feb 2020 19:19:39 +0000 (14:19 -0500)]
mgr/telemetry: check get_metadata return val

get_metada() returns 'None' when requesting a missing service, hence
trying to access its content fails. Added a check for osd and mgr
get_metadata() calls.

Fixes: https://tracker.ceph.com/issues/43642
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
5 years agoMerge pull request #32531 from zdover23/wip-doc-landing-page-update
Josh Durgin [Mon, 3 Feb 2020 23:50:25 +0000 (15:50 -0800)]
Merge pull request #32531 from zdover23/wip-doc-landing-page-update

doc: Added the crisp getting started guide to index.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agocephadm: add group 'disk' to privileged container 32995/head
Sage Weil [Mon, 3 Feb 2020 22:49:20 +0000 (16:49 -0600)]
cephadm: add group 'disk' to privileged container

This lets the osd read block devs that are group rw disk even after they
drop root privs.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #32977 into master
Sage Weil [Mon, 3 Feb 2020 22:28:04 +0000 (16:28 -0600)]
Merge PR #32977 into master

* refs/pull/32977/head:
qa/workunits/cephadm/test_cephadm.sh: add missing monitoring tests
cephadm: simplify Monitoring.components structure
cephadm: add proper tox type for monitoring components

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
5 years agoMerge PR #33012 into master
Sage Weil [Mon, 3 Feb 2020 22:27:51 +0000 (16:27 -0600)]
Merge PR #33012 into master

* refs/pull/33012/head:
mgr/cephadm: prefix daemon ids with hostname
cephadm: bootstrap: name mgr with $hostname.$random

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agocephadm: adopt: disable ceph-volume unit on host
Sage Weil [Fri, 31 Jan 2020 16:56:18 +0000 (10:56 -0600)]
cephadm: adopt: disable ceph-volume unit on host

This might be a simple or lvm unit.  Disable it so that the host doesn't
try to start this OSD after a reboot.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agocephadm: refactor privileged arg handling
Sage Weil [Fri, 31 Jan 2020 15:19:13 +0000 (09:19 -0600)]
cephadm: refactor privileged arg handling

Pass a bool if we want a privileged container instead of explicitly
passing --privileged.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agocephadm: fix error handling in `command_check_host()` 33048/head
Guillaume Abrioux [Mon, 3 Feb 2020 17:34:11 +0000 (18:34 +0100)]
cephadm: fix error handling in `command_check_host()`

`find_program()` raises `ValueError` when the executable hasn't been
found. It means we need to catch `ValueError` exception in
`command_check_host()` and raise `Error` instead of `RuntimeError` since
only `Error` is caught at the end.

Typical failure:

```
INFO:cephadm:/usr/bin/ceph:stderr Error ENOENT: New host mon1 failed check: ['INFO:cephadm:podman|docker (/bin/podman) is present', 'INFO:cephadm:systemctl is present', 'Traceback (most recent call last):', '  File "<stdin>", line 2820, in <module>', '  File "<stdin>", line 2434, in command_check_host', '  File "<stdin>", line 796, in find_program', 'ValueError: lvcreate not found']
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
5 years agorgw: move public access conf to perm_state_base
Abhishek Lekshmanan [Wed, 29 Jan 2020 18:18:24 +0000 (19:18 +0100)]
rgw: move public access conf to perm_state_base

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agotest: rgw_iam_policy update tests with new Get Actions
Abhishek Lekshmanan [Tue, 10 Dec 2019 16:56:45 +0000 (17:56 +0100)]
test: rgw_iam_policy update tests with new Get Actions

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agorgw: iam_policy: add all the actions to actpairs map
Abhishek Lekshmanan [Tue, 10 Dec 2019 16:56:22 +0000 (17:56 +0100)]
rgw: iam_policy: add all the actions to actpairs map

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agorgw: public access: drop unused function
Abhishek Lekshmanan [Tue, 3 Dec 2019 17:58:29 +0000 (18:58 +0100)]
rgw: public access: drop unused function

Also cleanup the comment to mention why we've deviated from the spec here

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agorgw: rename ACL & policy IsPublic to is_public
Abhishek Lekshmanan [Wed, 16 Oct 2019 11:38:10 +0000 (13:38 +0200)]
rgw: rename ACL & policy IsPublic to is_public

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agorgw: PublicAccessConfiguration -> PublicAccessBlockConfiguration
Abhishek Lekshmanan [Wed, 16 Oct 2019 11:22:09 +0000 (13:22 +0200)]
rgw: PublicAccessConfiguration -> PublicAccessBlockConfiguration

also drop iam namespace

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>