]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agominor fix: run tests in python venv 34352/head
Shilpa Jagannath [Thu, 30 Apr 2020 07:07:33 +0000 (12:37 +0530)]
minor fix: run tests in python venv

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agoqa/rgw: some basic reshard tests
Shilpa Jagannath [Fri, 24 Apr 2020 06:27:35 +0000 (11:57 +0530)]
qa/rgw: some basic reshard tests

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agominor fix: pass bucket_info as const ref in bi_list()
Shilpa Jagannath [Fri, 24 Apr 2020 06:25:12 +0000 (11:55 +0530)]
minor fix: pass bucket_info as const ref in bi_list()

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agorgw: parameterized bucket_index_layout_generation in BucketShard::init()
Shilpa Jagannath [Thu, 2 Apr 2020 18:17:08 +0000 (23:47 +0530)]
rgw: parameterized bucket_index_layout_generation in BucketShard::init()
for better flexibility.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agorgw: more multisite dynamic resharding pieces.
Shilpa Jagannath [Wed, 1 Apr 2020 17:28:46 +0000 (22:58 +0530)]
rgw: more multisite dynamic resharding pieces.

rados object names of bi shard to take a unique id called
generation number.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agoMerge pull request #34752 from tchaikov/fix-test-perf-counters-mgr-get
Kefu Chai [Mon, 27 Apr 2020 04:00:40 +0000 (12:00 +0800)]
Merge pull request #34752 from tchaikov/fix-test-perf-counters-mgr-get

mgr/dashboard: fix API tests to be py3 compatible

Reviewed-by: Xiubo Li <xiubli@redhat.com>
5 years agoMerge pull request #34292 from cbodley/wip-bufferlist-comparison
Kefu Chai [Mon, 27 Apr 2020 03:38:16 +0000 (11:38 +0800)]
Merge pull request #34292 from cbodley/wip-bufferlist-comparison

bufferlist: use iterators for comparison operators

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34090 from trociny/wip-ceph-erasure-code-tool
Kefu Chai [Mon, 27 Apr 2020 03:37:00 +0000 (11:37 +0800)]
Merge pull request #34090 from trociny/wip-ceph-erasure-code-tool

tools/erasure-code: new tool to encode/decode files

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34683 from ifed01/wip-ifed-fix-freelist-label
Kefu Chai [Mon, 27 Apr 2020 03:35:37 +0000 (11:35 +0800)]
Merge pull request #34683 from ifed01/wip-ifed-fix-freelist-label

os/bluestore: fix freelist's meta usage when bdev label is unsupported

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
5 years agoMerge pull request #34588 from ifed01/wip-ifed-lower-spinner-mas
Kefu Chai [Mon, 27 Apr 2020 03:34:14 +0000 (11:34 +0800)]
Merge pull request #34588 from ifed01/wip-ifed-lower-spinner-mas

os/bluestore:  set bluestore_min_alloc_size to 4K for spinners

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34006 from aclamk/wip-rocksdb-sharding
Kefu Chai [Mon, 27 Apr 2020 03:31:53 +0000 (11:31 +0800)]
Merge pull request #34006 from aclamk/wip-rocksdb-sharding

kv/RocksDBStore,os/blustore: shard rocksdb database using column families

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #33790 from bstillwell/add_conf_option_zstd_compression_level
Kefu Chai [Mon, 27 Apr 2020 03:28:05 +0000 (11:28 +0800)]
Merge pull request #33790 from bstillwell/add_conf_option_zstd_compression_level

compressor: Add a config option to specify Zstd compression level

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34635 from tchaikov/wip-mgr-py3
Kefu Chai [Mon, 27 Apr 2020 03:27:19 +0000 (11:27 +0800)]
Merge pull request #34635 from tchaikov/wip-mgr-py3

mgr: drop python2 support

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34732 from lxbsz/fs_fix_new
Kefu Chai [Sat, 25 Apr 2020 17:46:30 +0000 (01:46 +0800)]
Merge pull request #34732 from lxbsz/fs_fix_new

qa: misc fixes to run for cephfs qa test

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31912 from zjcmszh/cmpext
Kefu Chai [Sat, 25 Apr 2020 14:29:57 +0000 (22:29 +0800)]
Merge pull request #31912 from zjcmszh/cmpext

pybind/rados: add Ioctx::cmpext()

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
5 years agotest/pybind/test_rados.py: test Ioctx::cmpext() 31912/head
zhangjiao [Thu, 28 Nov 2019 05:21:57 +0000 (13:21 +0800)]
test/pybind/test_rados.py: test Ioctx::cmpext()
Signed-off-by: Zhang Jiao <zhangjiao@cmss.chinamobile.com>
5 years agopybind/rados: add Ioctx:cmpext()
zhangjiao [Thu, 28 Nov 2019 05:15:37 +0000 (13:15 +0800)]
pybind/rados: add Ioctx:cmpext()
Signed-off-by: Zhang Jiao <zhangjiao@cmss.chinamobile.com>
5 years agoMerge pull request #34578 from lxbsz/volume_client_py3
Kefu Chai [Sat, 25 Apr 2020 10:58:43 +0000 (18:58 +0800)]
Merge pull request #34578 from lxbsz/volume_client_py3

qa/volume_client: be python3 compatible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoqa/tasks/cephfs: switch to StringIO to fix byte-like errors 34732/head
Xiubo Li [Thu, 23 Apr 2020 11:21:46 +0000 (07:21 -0400)]
qa/tasks/cephfs: switch to StringIO to fix byte-like errors

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/cephfs: switch to mount_wait() to avoid possible umounted assert
Xiubo Li [Thu, 23 Apr 2020 07:21:09 +0000 (03:21 -0400)]
qa/cephfs: switch to mount_wait() to avoid possible umounted assert

For the fuse mount, it sometime will need to wait to a moment to let
the client get ready.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/tasks/cephfs/fuse_mount: fix possible chmod 1777 error
Xiubo Li [Fri, 24 Apr 2020 02:24:01 +0000 (22:24 -0400)]
qa/tasks/cephfs/fuse_mount: fix possible chmod 1777 error

INFO:teuthology.orchestra.run.smithi13a2:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2)
INFO:teuthology.orchestra.run.smithi132.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.2': Permission denied
DEBUG:teuthology.orchestra.run:got remote process result: 1

Here just wait and rety for 10 times.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/tasks/vstart_runner: remove the stdin str type check
Xiubo Li [Thu, 23 Apr 2020 11:39:31 +0000 (07:39 -0400)]
qa/tasks/vstart_runner: remove the stdin str type check

For the "ceph-dencoder" it will use the byte type data.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/cephfs: fix AssertionError: CommandFailedError not raised under py3
Xiubo Li [Tue, 21 Apr 2020 02:22:55 +0000 (22:22 -0400)]
qa/cephfs: fix AssertionError: CommandFailedError not raised under py3

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoMerge pull request #34697 from dillaman/wip-44596
Mykola Golub [Sat, 25 Apr 2020 07:29:01 +0000 (10:29 +0300)]
Merge pull request #34697 from dillaman/wip-44596

rbd-mirror: skip snapshot image-sync for newly created/cloned images

Reviewed-by: Mykola Golub <mgolub@suse.com>
5 years agoMerge pull request #34751 from tchaikov/wip-doc-fix
Kefu Chai [Sat, 25 Apr 2020 02:58:40 +0000 (10:58 +0800)]
Merge pull request #34751 from tchaikov/wip-doc-fix

doc/releases/mimic.rst: add title underline

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoMerge pull request #34741 from tchaikov/wip-teuthology-cronjobs-mimic
Kefu Chai [Sat, 25 Apr 2020 02:23:08 +0000 (10:23 +0800)]
Merge pull request #34741 from tchaikov/wip-teuthology-cronjobs-mimic

qa/tests: pass "-t 2" to teuthology-suite when testing mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agomgr/dashboard: fix API tests 34752/head
Alfonso Martínez [Fri, 24 Apr 2020 08:53:24 +0000 (10:53 +0200)]
mgr/dashboard: fix API tests

* test_perf_counters_mgr_get
* test_selftest_cluster_log

Fixes: https://tracker.ceph.com/issues/45170
Fixes: https://tracker.ceph.com/issues/45246
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
5 years agoqa/tasks/vstart_runner.py: use StringIO for capturing text
Kefu Chai [Sat, 25 Apr 2020 01:47:00 +0000 (09:47 +0800)]
qa/tasks/vstart_runner.py: use StringIO for capturing text

to be consistent with 8bfe977854686149c74de3ebaed7b571d43404dd

Fixes: https://tracker.ceph.com/issues/45246
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agodoc/releases/general.rst: add mising link targets 34751/head
Kefu Chai [Sat, 25 Apr 2020 01:37:59 +0000 (09:37 +0800)]
doc/releases/general.rst: add mising link targets

they are referenced by 60c825486916d0b0b43248a0966d2ecb90f352df

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agodoc/releases: silence sphinx warnings
Kefu Chai [Sat, 25 Apr 2020 01:24:37 +0000 (09:24 +0800)]
doc/releases: silence sphinx warnings

* mimic.rst: add title underline

to address sphinx warning like:
```
WARNING: Title underline too short.
```

* octopus.rst: use consistent underline

to address following sphinx warning:
```
doc/releases/octopus.rst:49: WARNING: Title level inconsistent:
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34698 from matthewoliver/cephadm_check_pool_exists
Sebastian Wagner [Fri, 24 Apr 2020 16:39:10 +0000 (18:39 +0200)]
Merge pull request #34698 from matthewoliver/cephadm_check_pool_exists

cephadm: Check for pool existence for iscsi And NFS

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #34676 from sebastian-philipp/cephadm-inventory-class
Sebastian Wagner [Fri, 24 Apr 2020 16:26:55 +0000 (18:26 +0200)]
Merge pull request #34676 from sebastian-philipp/cephadm-inventory-class

mgr/cephadm: add Inventory class

5 years agoMerge pull request #34695 from mgfritch/cephadm-iscsi-container-image
Sebastian Wagner [Fri, 24 Apr 2020 16:26:22 +0000 (18:26 +0200)]
Merge pull request #34695 from mgfritch/cephadm-iscsi-container-image

mgr/cephadm: allow config for an iscsi `container_image`

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #34718 from mgfritch/cephadm-mgr-is-available
Sebastian Wagner [Fri, 24 Apr 2020 16:25:11 +0000 (18:25 +0200)]
Merge pull request #34718 from mgfritch/cephadm-mgr-is-available

cephadm: catch Exception during is_mgr_available

Reviewed-by: Joshua Schmid <jschmid@suse.de>
5 years agoMerge pull request #34619 from sebastian-philipp/qa-adopt-specific-commit
Sebastian Wagner [Fri, 24 Apr 2020 16:01:18 +0000 (18:01 +0200)]
Merge pull request #34619 from sebastian-philipp/qa-adopt-specific-commit

qa/cephadm: make test_adoption checkout a specific commit

5 years agoMerge pull request #34693 from sebastian-philipp/cephadm-warn-master-improvement
Sebastian Wagner [Fri, 24 Apr 2020 16:00:42 +0000 (18:00 +0200)]
Merge pull request #34693 from sebastian-philipp/cephadm-warn-master-improvement

cephadm: improve warn message

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34675 from sebastian-philipp/cephadm-service-rm-feedback
Sebastian Wagner [Fri, 24 Apr 2020 16:00:18 +0000 (18:00 +0200)]
Merge pull request #34675 from sebastian-philipp/cephadm-service-rm-feedback

mgr/cephadm: remove_service: Feedback, if spec not found

Reviewed-by: Joshua Schmid <jschmid@suse.de>
5 years agoqa/tests: pass "-t 2" to teuthology-suite when testing mimic 34741/head
Kefu Chai [Fri, 24 Apr 2020 15:08:25 +0000 (23:08 +0800)]
qa/tests: pass "-t 2" to teuthology-suite when testing mimic

it's an oversight in 21dacc823be184718584c5fd3b647e87b3eeba42

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34142 from rhcs-dashboard/wip-44667-fix-reload-issues
Lenz Grimmer [Fri, 24 Apr 2020 13:58:17 +0000 (15:58 +0200)]
Merge pull request #34142 from rhcs-dashboard/wip-44667-fix-reload-issues

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #34730 from theanalyst/docs/release-updates-04-20
Abhishek L [Fri, 24 Apr 2020 12:57:28 +0000 (14:57 +0200)]
Merge pull request #34730 from theanalyst/docs/release-updates-04-20

docs: release updates for nautilus,mimic,octopus

Reviewed-By: Nathan Cutler <ncutler@suse.com>
5 years agoos/bluestore: Enable sharding by default 34006/head
Adam Kupczyk [Fri, 24 Apr 2020 12:36:05 +0000 (14:36 +0200)]
os/bluestore: Enable sharding by default

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
5 years agoMerge pull request #34673 from sebastian-philipp/cephadm-man
Sebastian Wagner [Fri, 24 Apr 2020 12:06:57 +0000 (14:06 +0200)]
Merge pull request #34673 from sebastian-philipp/cephadm-man

doc/man: add cephadm

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
5 years agoMerge pull request #34725 from lxbsz/_type_equality_funcs
Kefu Chai [Fri, 24 Apr 2020 10:59:31 +0000 (18:59 +0800)]
Merge pull request #34725 from lxbsz/_type_equality_funcs

qa/tasks/cephfs/test_scrub: fix self.assertEqual no attribute '_type_equality_funcs'

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agodoc: update release.yml with octopus, nautilus & mimic 34730/head
Abhishek Lekshmanan [Fri, 24 Apr 2020 10:35:21 +0000 (12:35 +0200)]
doc: update release.yml with octopus, nautilus & mimic

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agodoc: releases: add release notes for v14.2.9
Abhishek Lekshmanan [Fri, 24 Apr 2020 10:34:59 +0000 (12:34 +0200)]
doc: releases: add release notes for v14.2.9

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #34726 from lxbsz/dict_key_error
Kefu Chai [Fri, 24 Apr 2020 10:33:33 +0000 (18:33 +0800)]
Merge pull request #34726 from lxbsz/dict_key_error

qa/tasks/cephfs/cephfs_test_case: switch 'dict_keys' object to list

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agodoc: release notes for v13.2.10 mimic
Abhishek Lekshmanan [Fri, 24 Apr 2020 10:28:44 +0000 (12:28 +0200)]
doc: release notes for v13.2.10 mimic

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agodoc: releases: release notes for v13.2.9 mimic
Abhishek Lekshmanan [Thu, 16 Apr 2020 14:40:21 +0000 (16:40 +0200)]
doc: releases: release notes for v13.2.9 mimic

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agodoc: releases: v15.2.1 release notes
Abhishek Lekshmanan [Tue, 7 Apr 2020 09:44:14 +0000 (11:44 +0200)]
doc: releases: v15.2.1 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoqa/tasks/cephfs/test_volumes: to make sure the size is int type 34726/head
Xiubo Li [Fri, 24 Apr 2020 01:08:23 +0000 (21:08 -0400)]
qa/tasks/cephfs/test_volumes: to make sure the size is int type

Fix "Error EINVAL: invalid size option '4915200.0'".

In pytho2 int/2 will get a int type result, but in python3 it maybe
a float type.

Fixes: https://tracker.ceph.com/issues/45247
Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/tasks/cephfs/mount: to make sure that the count/seek are int type
Xiubo Li [Thu, 23 Apr 2020 21:31:24 +0000 (17:31 -0400)]
qa/tasks/cephfs/mount: to make sure that the count/seek are int type

To fix "stderr:dd: invalid number: '5.0'".

Fixes: https://tracker.ceph.com/issues/45247
Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/tasks/cephfs/test_scrub: fix self.assertEqual no attribute '_type_equality_funcs' 34725/head
Xiubo Li [Thu, 23 Apr 2020 05:08:04 +0000 (01:08 -0400)]
qa/tasks/cephfs/test_scrub: fix self.assertEqual no attribute '_type_equality_funcs'

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/tasks/cephfs/cephfs_test_case: switch 'dict_keys' object to list
Xiubo Li [Thu, 23 Apr 2020 13:23:31 +0000 (09:23 -0400)]
qa/tasks/cephfs/cephfs_test_case: switch 'dict_keys' object to list

Fix the "TypeError: 'dict_keys' object does not support indexing" under
py3.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoMerge pull request #33769 from lxbsz/qa_vstart_runner_os
Kefu Chai [Fri, 24 Apr 2020 07:41:54 +0000 (15:41 +0800)]
Merge pull request #33769 from lxbsz/qa_vstart_runner_os

qa/vstart_runner.py: add os and arch properties support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoqa/tasks/vstart_runner: make remote.sh return str as default 33769/head
Xiubo Li [Thu, 23 Apr 2020 23:14:36 +0000 (19:14 -0400)]
qa/tasks/vstart_runner: make remote.sh return str as default

To address the issue in https://github.com/ceph/teuthology/pull/1459.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/vstart_runner.py: add os and arch properties support
Xiubo Li [Fri, 6 Mar 2020 01:10:47 +0000 (20:10 -0500)]
qa/vstart_runner.py: add os and arch properties support

Fixes: https://tracker.ceph.com/issues/44456
Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoMerge pull request #34685 from lxbsz/vstart_runner
Kefu Chai [Fri, 24 Apr 2020 01:09:49 +0000 (09:09 +0800)]
Merge pull request #34685 from lxbsz/vstart_runner

qa/tasks/cephfs/mount: switch to StringIO to fix TypeErrors

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoqa/tasks/cephfs/mount: switch to StringIO to fix TypeErrors 34685/head
Xiubo Li [Wed, 22 Apr 2020 12:34:05 +0000 (08:34 -0400)]
qa/tasks/cephfs/mount: switch to StringIO to fix TypeErrors

TypeError: cannot use a string pattern on a bytes-like object

Fixes: https://tracker.ceph.com/issues/45175
Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agoqa/vstart_runner: remove the debug info
Xiubo Li [Wed, 22 Apr 2020 12:03:27 +0000 (08:03 -0400)]
qa/vstart_runner: remove the debug info

Fixes: https://tracker.ceph.com/issues/45175
Signed-off-by: Xiubo Li <xiubli@redhat.com>
5 years agocephadm: Check for pool existance for iscsi And NFS 34698/head
Matthew Oliver [Thu, 23 Apr 2020 01:07:19 +0000 (11:07 +1000)]
cephadm: Check for pool existance for iscsi And NFS

Currently both iscsi and NFS require pools to be specified when they are
deployed. However, we don't actaully check these pools exist. Leading to
broken containers.

This patch uses the rados client that is part of the mgrmodule to check
that the specified pool exists. As we need to check in 2 different
daemons, a helper method:

   def _check_pool_exists(self, pool, service_name):

was added to `cephadm/module.py`.

Fixes: https://tracker.ceph.com/issues/45161
Signed-off-by: Matthew Oliver <moliver@suse.com>
5 years agocephadm: catch Exception during is_mgr_available 34718/head
Michael Fritch [Thu, 23 Apr 2020 21:09:03 +0000 (15:09 -0600)]
cephadm: catch Exception during is_mgr_available

Traceback (most recent call last):
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 4578, in <module>
    r = args.func()
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 1121, in _default_image
    return func()
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2489, in command_bootstrap
    is_available('mgr', is_mgr_available)
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 895, in is_available
    if func():
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2486, in is_mgr_available
    out = cli(['status', '-f', 'json-pretty'], timeout=timeout)
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2403, in cli
    return CephContainer(
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2140, in run
    out, _, _ = call_throws(
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 838, in call_throws
    raise RuntimeError('Failed command: %s' % ' '.join(command))

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agorbd-mirror: skip snapshot image sync if mirror snapshot is marked clean 34697/head
Jason Dillaman [Wed, 22 Apr 2020 17:51:58 +0000 (13:51 -0400)]
rbd-mirror: skip snapshot image sync if mirror snapshot is marked clean

This is currently only utilized for the case where a newly created image
has mirroring enabled at time of creation, but it could be expanded in the
future if we track writes.

Fixes: https://tracker.ceph.com/issues/44596
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: EnableRequest now accepts a boolean to indicate a clean image
Jason Dillaman [Wed, 22 Apr 2020 15:57:45 +0000 (11:57 -0400)]
librbd: EnableRequest now accepts a boolean to indicate a clean image

If the image is clean, it's treated as if it was newly created and
therefore clean since snapshot id 0. The CreateRequest and
CloneRequest state machines pass true for this bool if mirroring
is being enabled during creation.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: pass clean_since_snap_id to CreatePrimaryRequest
Jason Dillaman [Wed, 22 Apr 2020 15:42:17 +0000 (11:42 -0400)]
librbd: pass clean_since_snap_id to CreatePrimaryRequest

This will be stored in the primary MirrorSnapshotNamespace for use by
the rbd-mirror snapshot replayer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agocls/rbd: added clean_since_snap_id to MirrorSnapshotNamespace
Jason Dillaman [Tue, 21 Apr 2020 15:07:30 +0000 (11:07 -0400)]
cls/rbd: added clean_since_snap_id to MirrorSnapshotNamespace

It is only valid for primary snapshots so we can re-use the space
previously used by the primary_snap_id without the need to bump the
version id.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agorbd: add '--mirror-image-mode' optional to image creation actions
Jason Dillaman [Tue, 21 Apr 2020 00:37:56 +0000 (20:37 -0400)]
rbd: add '--mirror-image-mode' optional to image creation actions

This allows the user to enable mirroring while creating, cloning,
copying, importing, or migrating an image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: save requested mirror image mode during migration prepare
Jason Dillaman [Mon, 20 Apr 2020 23:34:57 +0000 (19:34 -0400)]
librbd: save requested mirror image mode during migration prepare

Once the migration is committed, the requested mode will be applied.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: permit mirroring to be enabled upon image cloning
Jason Dillaman [Mon, 20 Apr 2020 22:50:43 +0000 (18:50 -0400)]
librbd: permit mirroring to be enabled upon image cloning

The librbd clone API's ImageOptions parameter now supports passing
the mirror image mode.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: add non-primary global image id to mirror enable
Jason Dillaman [Mon, 20 Apr 2020 22:33:32 +0000 (18:33 -0400)]
librbd: add non-primary global image id to mirror enable

The constructor that took an ImageCtx did not support passing the
non-primary global image id and instead defaulted to an empty string.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: permit mirroring to be enabled upon image creation
Jason Dillaman [Mon, 20 Apr 2020 21:04:02 +0000 (17:04 -0400)]
librbd: permit mirroring to be enabled upon image creation

The librbd create API's ImageOptions parameter now supports passing
the mirror image mode.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: mirror enable state machine might need to open image
Jason Dillaman [Mon, 20 Apr 2020 22:16:40 +0000 (18:16 -0400)]
librbd: mirror enable state machine might need to open image

If attempting to create a snapshot-based mirroring primary snapshot,
the image needs to first be opened. If we weren't supplied an image,
open the image, create the snapshot, and close the image again.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #34570 from cbodley/wip-bucket-layout-json
Casey Bodley [Thu, 23 Apr 2020 15:16:55 +0000 (11:16 -0400)]
Merge pull request #34570 from cbodley/wip-bucket-layout-json

rgw: bucket layout fixes

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agoMerge pull request #34645 from cbodley/wip-rgw-reshard-target-layout
Casey Bodley [Thu, 23 Apr 2020 15:16:44 +0000 (11:16 -0400)]
Merge pull request #34645 from cbodley/wip-rgw-reshard-target-layout

rgw: use target bucket layout for reshard

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
5 years agoMerge pull request #33937 from changchengx/unsigned_compare
Jason Dillaman [Thu, 23 Apr 2020 14:48:33 +0000 (10:48 -0400)]
Merge pull request #33937 from changchengx/unsigned_compare

librbd: fix unsigned value compared with zero

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #34694 from mikechristie/ceph-iscsi-selinux-fixes
Boris Ranto [Thu, 23 Apr 2020 11:52:36 +0000 (13:52 +0200)]
Merge pull request #34694 from mikechristie/ceph-iscsi-selinux-fixes

ceph-iscsi:  selinux fixes

Reviewed-by: Boris Ranto <branto@redhat.com>
5 years agodoc/man: add cephadm 34673/head
Sebastian Wagner [Wed, 22 Apr 2020 07:58:55 +0000 (09:58 +0200)]
doc/man: add cephadm

Fixes: https://tracker.ceph.com/issues/44971
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/cephadm: unify apply tests 34675/head
Sebastian Wagner [Fri, 17 Apr 2020 13:15:34 +0000 (15:15 +0200)]
mgr/cephadm: unify apply tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/cephadm: unified add_daemon tests
Sebastian Wagner [Fri, 17 Apr 2020 12:44:52 +0000 (14:44 +0200)]
mgr/cephadm: unified add_daemon tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/cephadm: add cleanup to test_cephadm
Sebastian Wagner [Fri, 17 Apr 2020 10:55:25 +0000 (12:55 +0200)]
mgr/cephadm: add cleanup to test_cephadm

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agomgr/cephadm: remove_service: Feedback, if spec not found
Sebastian Wagner [Fri, 17 Apr 2020 10:47:46 +0000 (12:47 +0200)]
mgr/cephadm: remove_service: Feedback, if spec not found

`ceph orch rm my-unknown-service` still returned
`Removed service my-unknown-service`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #34678 from sebastian-philipp/doc-cephadm-install-fix-placement
Sebastian Wagner [Thu, 23 Apr 2020 10:20:30 +0000 (12:20 +0200)]
Merge pull request #34678 from sebastian-philipp/doc-cephadm-install-fix-placement

doc/cephadm: install.rst: fix placement spec.

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34674 from sebastian-philipp/cephadm-ceph-volume-lock-if-fsid
Sebastian Wagner [Thu, 23 Apr 2020 10:09:22 +0000 (12:09 +0200)]
Merge pull request #34674 from sebastian-philipp/cephadm-ceph-volume-lock-if-fsid

cephadm: Aquire lock, if fsid != None

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34620 from mgfritch/cephadm-timeout-test-fixup
Sebastian Wagner [Thu, 23 Apr 2020 10:03:20 +0000 (12:03 +0200)]
Merge pull request #34620 from mgfritch/cephadm-timeout-test-fixup

qa/workunits/cephadm/test_cephadm.sh: extend timeout test

5 years agoMerge pull request #34600 from dsavineau/issue_45120
Sebastian Wagner [Thu, 23 Apr 2020 10:02:31 +0000 (12:02 +0200)]
Merge pull request #34600 from dsavineau/issue_45120

cephadm: fix adopt prometheus command

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34688 from sebastian-philipp/qa-test-cephadm-octopus-version
Sebastian Wagner [Thu, 23 Apr 2020 10:01:48 +0000 (12:01 +0200)]
Merge pull request #34688 from sebastian-philipp/qa-test-cephadm-octopus-version

qa/workunits/cephadm/test_cephadm: Also test octopus

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34691 from sebastian-philipp/cephadm-ssk-keyring-check_call
Sebastian Wagner [Thu, 23 Apr 2020 10:01:20 +0000 (12:01 +0200)]
Merge pull request #34691 from sebastian-philipp/cephadm-ssk-keyring-check_call

mgr/cephadm: use check_call  for `ssh-keygen`

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #33065 from ifed01/wip-ifed-spurious-readerr-alert
Igor Fedotov [Thu, 23 Apr 2020 08:32:25 +0000 (11:32 +0300)]
Merge pull request #33065 from ifed01/wip-ifed-spurious-readerr-alert

os/bluestore: introduce health alert on spurious read errors.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge pull request #34666 from rhcs-dashboard/44405-test-totp-increase-time
Laura Paduano [Thu, 23 Apr 2020 07:47:58 +0000 (09:47 +0200)]
Merge pull request #34666 from rhcs-dashboard/44405-test-totp-increase-time

mgr/dashboard: fix tasks.mgr.dashboard.test_rgw.RgwBucketTest.test_all

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34483 from dabukalam/master
Kefu Chai [Thu, 23 Apr 2020 07:33:06 +0000 (15:33 +0800)]
Merge pull request #34483 from dabukalam/master

doc/rados: Remove obsolete placement group note from EC pool creation

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33916 from aclamk/fix-bs-avl-memaccounting
Kefu Chai [Thu, 23 Apr 2020 07:22:19 +0000 (15:22 +0800)]
Merge pull request #33916 from aclamk/fix-bs-avl-memaccounting

os/bluestore: Adds shutdown in destructor of AvlAllocator

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34559 from tchaikov/wip-19076-refactor
Kefu Chai [Thu, 23 Apr 2020 02:34:26 +0000 (10:34 +0800)]
Merge pull request #34559 from tchaikov/wip-19076-refactor

mon/OSDMonitor: refactor loops for better readability

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoMerge pull request #34328 from liewegas/cpatch
Josh Durgin [Thu, 23 Apr 2020 01:34:36 +0000 (18:34 -0700)]
Merge pull request #34328 from liewegas/cpatch

script/cpatch: initial version

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoMerge pull request #34430 from yison/rwl-image-writeback-cache-seq4
Jason Dillaman [Wed, 22 Apr 2020 23:28:17 +0000 (19:28 -0400)]
Merge pull request #34430 from yison/rwl-image-writeback-cache-seq4

rbd/cache: Replicated Write Log core codes part 4

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agomgr/cephadm: allow config for an iscsi `container_image` 34695/head
Michael Fritch [Wed, 22 Apr 2020 19:28:43 +0000 (13:28 -0600)]
mgr/cephadm: allow config for an iscsi `container_image`

ceph config set client.iscsi container_image <container_image>

Fixes: https://tracker.ceph.com/issues/45162
Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agoselinux: Fix ceph-iscsi etc access 34694/head
Mike Christie [Thu, 9 Jan 2020 00:37:15 +0000 (18:37 -0600)]
selinux: Fix ceph-iscsi etc access

This fixes the selinux errors like this for /etc/target

-----------------------------------
Additional Information:
Source Context                system_u:system_r:ceph_t:s0
Target Context                system_u:object_r:targetd_etc_rw_t:s0
Target Objects                target [ dir ]
Source                        rbd-target-api
Source Path                   rbd-target-api
Port                          <Unknown>
Host                          ans8
Source RPM Packages
Target RPM Packages
Policy RPM                    selinux-policy-3.14.3-20.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     ans8
Platform                      Linux ans8 4.18.0-147.el8.x86_64 #1 SMP
Thu Sep 26
                              15:52:44 UTC 2019 x86_64 x86_64
Alert Count                   1
First Seen                    2020-01-08 18:39:48 EST
Last Seen                     2020-01-08 18:39:48 EST
Local ID                      9a13ee18-eaf2-4f2a-872f-2809ee4928f6

Raw Audit Messages
type=AVC msg=audit(1578526788.148:69): avc:  denied  { search } for
pid=995 comm="rbd-target-api" name="target" dev="sda1" ino=52198
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:targetd_etc_rw_t:s0 tclass=dir permissive=1

Hash: rbd-target-api,ceph_t,targetd_etc_rw_t,dir,search

which are a result of the rtslib library the ceph-iscsi daemons use
accessing /etc/target to read/write a file which stores meta data the
target uses.

Signed-off-by: Mike Christie <mchristi@redhat.com>
5 years agoselinux: Fix ceph-iscsi configfs access
Mike Christie [Thu, 9 Jan 2020 00:03:40 +0000 (18:03 -0600)]
selinux: Fix ceph-iscsi configfs access

This fixes the the following selinux error when using ceph-iscsi's
rbd-target-api daemon (rbd-target-gw has the same issue). They are
a result of the a python library, rtslib, which the daemons use.

Additional Information:
Source Context                system_u:system_r:ceph_t:s0
Target Context                system_u:object_r:configfs_t:s0
Target Objects
/sys/kernel/config/target/iscsi/iqn.2003-01.com.re
                              dhat:ceph-iscsi/tpgt_1/attrib/authentication
[
                              file ]
Source                        rbd-target-api
Source Path                   /usr/libexec/platform-python3.6
Port                          <Unknown>
Host                          ans8
Source RPM Packages           platform-python-3.6.8-15.1.el8.x86_64
Target RPM Packages
Policy RPM                    selinux-policy-3.14.3-20.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     ans8
Platform                      Linux ans8 4.18.0-147.el8.x86_64 #1 SMP
Thu Sep 26
                              15:52:44 UTC 2019 x86_64 x86_64
Alert Count                   1
First Seen                    2020-01-08 18:39:47 EST
Last Seen                     2020-01-08 18:39:47 EST
Local ID                      6f8c3415-7a50-4dc8-b3d2-2621e1d00ca3

Raw Audit Messages
type=AVC msg=audit(1578526787.577:68): avc:  denied  { ioctl } for
pid=995 comm="rbd-target-api"
path="/sys/kernel/config/target/iscsi/iqn.2003-01.com.redhat:ceph-iscsi/tpgt_1/attrib/authentication"
dev="configfs" ino=25703 ioctlcmd=0x5401
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:configfs_t:s0 tclass=file permissive=1

type=SYSCALL msg=audit(1578526787.577:68): arch=x86_64 syscall=ioctl
success=no exit=ENOTTY a0=34 a1=5401 a2=7ffd4f8f1f60 a3=3052cd2d95839b96
items=0 ppid=1 pid=995 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0
egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=rbd-target-api
exe=/usr/libexec/platform-python3.6 subj=system_u:system_r:ceph_t:s0
key=(null)

Hash: rbd-target-api,ceph_t,configfs_t,file,ioctl

Signed-off-by: Mike Christie <mchristi@redhat.com>
5 years agocephadm: improve warn message 34693/head
Sebastian Wagner [Wed, 22 Apr 2020 16:26:13 +0000 (18:26 +0200)]
cephadm: improve warn message

make it more fancy!

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-authored-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34669 from yuvalif/remove_queue_dependency
Casey Bodley [Wed, 22 Apr 2020 14:35:48 +0000 (10:35 -0400)]
Merge pull request #34669 from yuvalif/remove_queue_dependency

cls/2pc_queue: remove the dependency of cls_2pc_queue with cls_queue

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34664 from yuvalif/clarify_topic_creation_format
Casey Bodley [Wed, 22 Apr 2020 14:35:28 +0000 (10:35 -0400)]
Merge pull request #34664 from yuvalif/clarify_topic_creation_format

rgw/doc: clarify topic creation format

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agomgr/cephadm: use check_call for `ssh-keygen` 34691/head
Sebastian Wagner [Wed, 22 Apr 2020 14:10:27 +0000 (16:10 +0200)]
mgr/cephadm: use check_call  for `ssh-keygen`

Mainly to provide any reasonable error

Fixes: https://tracker.ceph.com/issues/44832
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>