]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoMerge pull request #30391 from pdvian/wip-41805-mimic
Yuri Weinstein [Thu, 3 Oct 2019 23:29:28 +0000 (16:29 -0700)]
Merge pull request #30391 from pdvian/wip-41805-mimic

mimic: mgr: do not reset reported if a new metric is not collected

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #29253 from smithfarm/wip-40464-mimic
Yuri Weinstein [Thu, 3 Oct 2019 23:27:05 +0000 (16:27 -0700)]
Merge pull request #29253 from smithfarm/wip-40464-mimic

mimic: core: osd beacon sometimes has empty pg list

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #29245 from smithfarm/wip-38351-mimic
Yuri Weinstein [Thu, 3 Oct 2019 23:25:52 +0000 (16:25 -0700)]
Merge pull request #29245 from smithfarm/wip-38351-mimic

mimic: tests: Limit loops waiting for force-backfill/force-recovery to happen

Reviewed-by: David Zafman <dzafman@redhat.com>
5 years agoMerge pull request #29250 from smithfarm/wip-38856-mimic
Yuri Weinstein [Thu, 3 Oct 2019 23:23:55 +0000 (16:23 -0700)]
Merge pull request #29250 from smithfarm/wip-38856-mimic

mimic: core: should set EPOLLET flag on del_event()

5 years agoMerge pull request #29251 from smithfarm/wip-40179-mimic
Yuri Weinstein [Thu, 3 Oct 2019 23:23:23 +0000 (16:23 -0700)]
Merge pull request #29251 from smithfarm/wip-40179-mimic

mimic: tests: qa/standalone/scrub/osd-scrub-snaps.sh sometimes fails

Reviewed-by: David Zafman <dzafman@redhat.com>
5 years agoMerge pull request #29765 from smithfarm/wip-40130-mimic
Nathan Cutler [Wed, 2 Oct 2019 13:13:13 +0000 (15:13 +0200)]
Merge pull request #29765 from smithfarm/wip-40130-mimic

mimic: doc: Document behaviour of fsync-after-close

Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #30687 from smithfarm/wip-42104-mimic
Nathan Cutler [Wed, 2 Oct 2019 12:33:56 +0000 (14:33 +0200)]
Merge pull request #30687 from smithfarm/wip-42104-mimic

mimic: build/ops: admin/build-doc: use python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoadmin/build-doc: use python3 (follow-on fix) 30687/head
Nathan Cutler [Wed, 2 Oct 2019 10:05:47 +0000 (12:05 +0200)]
admin/build-doc: use python3 (follow-on fix)

Unfortunately, a5127318cf2feb12b5fd61aeed35b48bc31b4e34 (first attempt to
cherry-pick ace8cb15a600daf5261c0ab349b43010a238a1d9) was incomplete.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(partial manual cherry-pick of ace8cb15a600daf5261c0ab349b43010a238a1d9)

5 years agodoc: fix urls in posix.rst 29765/head
Jos Collin [Thu, 25 Jul 2019 04:33:12 +0000 (10:03 +0530)]
doc: fix urls in posix.rst

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit c5c5f031f77f778af73f8659b01695b88f158807)

5 years agodoc: cephfs: add section on fsync error reporting to posix.rst
Jeff Layton [Wed, 29 May 2019 18:35:11 +0000 (14:35 -0400)]
doc: cephfs: add section on fsync error reporting to posix.rst

Fixes: http://tracker.ceph.com/issues/24641
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit b52d85c2a71950db222e3649d5265e5891368ba1)

5 years agoMerge pull request #30663 from smithfarm/wip-fix-docs-build-check-mimic
Nathan Cutler [Tue, 1 Oct 2019 15:54:30 +0000 (17:54 +0200)]
Merge pull request #30663 from smithfarm/wip-fix-docs-build-check-mimic

mimic: build/ops: admin/build-doc: use python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30131 from smithfarm/wip-40736-mimic
Nathan Cutler [Tue, 1 Oct 2019 13:15:30 +0000 (15:15 +0200)]
Merge pull request #30131 from smithfarm/wip-40736-mimic

mimic: doc/rgw: document use of 'realm pull' instead of 'period pull'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #30099 from smithfarm/wip-40673-mimic
Nathan Cutler [Tue, 1 Oct 2019 13:14:27 +0000 (15:14 +0200)]
Merge pull request #30099 from smithfarm/wip-40673-mimic

mimic: docs: fix rgw_ldap_dnattr username token

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoadmin/build-doc: use python3 30663/head
Kefu Chai [Wed, 7 Aug 2019 09:46:13 +0000 (17:46 +0800)]
admin/build-doc: use python3

to address https://github.com/sphinx-doc/sphinx/issues/3620, we need to
use sphinx with its fix at
https://github.com/sphinx-doc/sphinx/commit/e049f86b2de1cfdf8a74c88dc9593d047c85d5cb
in other words, we need to use sphinx v2.0.0 and up. but sphinx 2.0
requires python >= 3.5, so we have to use python3 for building the
documents.

in this change:

* doc-requirements.txt: install python3 packages on debian derivatives
* build-doc: install python3.6 packages from EPEL7, and use python3
  venv for using sphinx2
* doc-requirements.txt: bump up all python packages to latest
  stable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ace8cb15a600daf5261c0ab349b43010a238a1d9)

Conflicts:
admin/doc-requirements.txt

5 years agoMerge pull request #30070 from smithfarm/wip-40139-mimic
Nathan Cutler [Tue, 1 Oct 2019 12:36:34 +0000 (14:36 +0200)]
Merge pull request #30070 from smithfarm/wip-40139-mimic

mimic: doc: describe metadata_heap cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #29039 from dcasier/wip-40100-mimic
Alfredo Deza [Tue, 1 Oct 2019 11:42:49 +0000 (07:42 -0400)]
Merge pull request #29039 from dcasier/wip-40100-mimic

mimic: ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agoMerge pull request #29257 from smithfarm/wip-39602-mimic
Nathan Cutler [Tue, 1 Oct 2019 11:27:50 +0000 (13:27 +0200)]
Merge pull request #29257 from smithfarm/wip-39602-mimic

mimic: doc: rgw: CreateBucketConfiguration for s3 PUT Bucket request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #30555 from alfredodeza/mimic-guits-41392
Alfredo Deza [Mon, 30 Sep 2019 13:24:01 +0000 (09:24 -0400)]
Merge pull request #30555 from alfredodeza/mimic-guits-41392

mimic ceph-volume: do not fail when trying to remove crypt mapper

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #30096 from jan--f/wip-41612-mimic
Alfredo Deza [Fri, 27 Sep 2019 14:45:02 +0000 (10:45 -0400)]
Merge pull request #30096 from jan--f/wip-41612-mimic

mimic: ceph-volume: minor optimizations related to class Volumes's use

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agoceph-volume: do not fail when trying to remove crypt mapper 30555/head
Guillaume Abrioux [Fri, 20 Sep 2019 12:57:58 +0000 (14:57 +0200)]
ceph-volume: do not fail when trying to remove crypt mapper

In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.

Closes: https://tracker.ceph.com/issues/41392
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 80d3e4ca054b4d1334c2fb99fede46ab54ca4305)

5 years agoMerge pull request #30303 from alfredodeza/mimic-ceph-volume-zap-fix
Alfredo Deza [Mon, 23 Sep 2019 18:48:00 +0000 (14:48 -0400)]
Merge pull request #30303 from alfredodeza/mimic-ceph-volume-zap-fix

mimic ceph-volume lvm.zap fix cleanup for db partitions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
5 years agoMerge pull request #30334 from jan--f/wip-disjoint-device-lists-mimic
Alfredo Deza [Thu, 19 Sep 2019 20:49:59 +0000 (16:49 -0400)]
Merge pull request #30334 from jan--f/wip-disjoint-device-lists-mimic

mimic: ceph-volume: ensure device lists are disjoint

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agoMerge pull request #30333 from jan--f/wip-41762-mimic
Alfredo Deza [Thu, 19 Sep 2019 16:36:43 +0000 (12:36 -0400)]
Merge pull request #30333 from jan--f/wip-41762-mimic

mimic: ceph-volume: implement __format__ in Size to format sizes in py3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agomgr: do not reset reported if a new metric is not collected 30391/head
Ilsoo Byun [Tue, 10 Sep 2019 09:13:02 +0000 (18:13 +0900)]
mgr: do not reset reported if a new metric is not collected

Fixes: https://tracker.ceph.com/issues/41741
Signed-off-by: Ilsoo Byun <ilsoobyun@linecorp.com>
(cherry picked from commit 02cc60f6935a5005aa461da183c6c4332503be83)

5 years agoMerge pull request #30295 from alfredodeza/mimic-rm41378-2
Alfredo Deza [Thu, 12 Sep 2019 10:54:59 +0000 (06:54 -0400)]
Merge pull request #30295 from alfredodeza/mimic-rm41378-2

mimic ceph-volume: pre-install python-apt and its variants before test runs

5 years agoMerge pull request #30301 from alfredodeza/mimic-rm41660
Alfredo Deza [Thu, 12 Sep 2019 10:54:22 +0000 (06:54 -0400)]
Merge pull request #30301 from alfredodeza/mimic-rm41660

mimic ceph-volume: fix stderr failure to decode/encode when redirected

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #30306 from alfredodeza/mimic-rm40664
Jan Fajerski [Thu, 12 Sep 2019 10:05:39 +0000 (12:05 +0200)]
Merge pull request #30306 from alfredodeza/mimic-rm40664

mimic ceph-volume  zap always skips block.db, leaves them around

5 years agoceph-volume: don't keep device lists as sets 30334/head
Jan Fajerski [Thu, 15 Aug 2019 10:20:00 +0000 (12:20 +0200)]
ceph-volume: don't keep device lists as sets

This was introduced by #27754. The explicit device lists were cast to
sets but other parts of the code where not updated accordingly. To avoid
touching all code places, only cast to sets for disjoint test and keep
lists otherwise.

Fixes: https://tracker.ceph.com/issues/41292
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0534cf188a671096d5ddb9d48cdae3dccc6c0b18)

5 years agoceph-volume: batch ensure device lists are disjoint
Jan Fajerski [Thu, 11 Apr 2019 14:58:32 +0000 (16:58 +0200)]
ceph-volume: batch ensure device lists are disjoint

Fixes: https://tracker.ceph.com/issues/41018
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8fcdb4d628f3a55e4783f0a3df7ac7c0a36eaa43)

5 years agoceph-volume: implement __format__ in Size to format sizes in py3 30333/head
Jan Fajerski [Wed, 13 Feb 2019 12:48:55 +0000 (13:48 +0100)]
ceph-volume: implement __format__ in Size to format sizes in py3

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7a741f201a078dfd9bce34e66ed16bcb37d99f7f)

5 years agoceph-volume tests: verify that wipefs tries several times 30306/head
Alfredo Deza [Thu, 11 Jul 2019 15:56:07 +0000 (11:56 -0400)]
ceph-volume tests: verify that wipefs tries several times

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 845e62cb4ba2636325304db2c2d65a185ace25cf)

5 years agoceph-volume lvm.zap: retry wipefs several times to prevent race condition failures
Alfredo Deza [Thu, 11 Jul 2019 13:34:10 +0000 (09:34 -0400)]
ceph-volume lvm.zap: retry wipefs several times to prevent race condition failures

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b7b8b106fa485d3b8345b583fbc7f74e30cbc03e)

5 years agoceph-volume lvm.zap fix cleanup for db partitions 30303/head
Dominik Csapak [Tue, 28 May 2019 14:29:21 +0000 (16:29 +0200)]
ceph-volume lvm.zap fix cleanup for db partitions

this uses the correct type 'db' for db type partitions, else
a block.db parition does not get cleaned up by ceph-volume zap

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
(cherry picked from commit 140af8a7eee8d230ac9d94ba100320a20c137a12)

5 years agoceph-volume tests create a test file for checking unicode output 30301/head
Alfredo Deza [Mon, 9 Sep 2019 22:55:13 +0000 (18:55 -0400)]
ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 79196354aca3b3d30c7de1e2ac8b5a3c06ee8330)

5 years agoceph-volume tests create a shell test for functional unicode
Alfredo Deza [Mon, 9 Sep 2019 22:54:51 +0000 (18:54 -0400)]
ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 595e492783283359b613d199a75f3e7495bfb9ed)

5 years agoceph-volume tests verify new logging fallback and encodings in terminal
Alfredo Deza [Mon, 9 Sep 2019 19:20:13 +0000 (15:20 -0400)]
ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ee18ebc078b2d380b88f3726737184f34345e2f3)

5 years agoceph-volume terminal remove unicode stream handler
Alfredo Deza [Mon, 9 Sep 2019 17:45:08 +0000 (13:45 -0400)]
ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe66e01f6a4b19bb15695537713a629a951b28ce)

5 years agoceph-volume: instantiate the new terminal logger in main()
Alfredo Deza [Mon, 9 Sep 2019 17:44:36 +0000 (13:44 -0400)]
ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfb8422e262dcce49c50610a3977bfe1edc2a435)

5 years agoceph-volume create a logger for the terminal
Alfredo Deza [Mon, 9 Sep 2019 17:44:17 +0000 (13:44 -0400)]
ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 95b16b516c0736839fe50f12ecbeb7cd62c67335)

5 years agoceph-volume create a new tox.ini for shell-based tests
Alfredo Deza [Mon, 9 Sep 2019 17:40:06 +0000 (13:40 -0400)]
ceph-volume create a new tox.ini for shell-based tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4612ab3ae62a3e8e1cc9e51489feab0c938296d3)

5 years agoceph-volume tests pre-instrall python-apt to prevent auto-install failing later 30295/head
Alfredo Deza [Tue, 3 Sep 2019 14:00:14 +0000 (10:00 -0400)]
ceph-volume tests pre-instrall python-apt to prevent auto-install failing later

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40cac7b9510f7904c2f9dfb1294d809e04bdcf7e)

5 years agoMerge pull request #30188 from smithfarm/wip-41332-mimic
Nathan Cutler [Mon, 9 Sep 2019 22:36:41 +0000 (00:36 +0200)]
Merge pull request #30188 from smithfarm/wip-41332-mimic

mimic: build/ops: rpm: always build ceph-test package

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30227 from smithfarm/wip-41530-mimic
Nathan Cutler [Sun, 8 Sep 2019 09:30:27 +0000 (11:30 +0200)]
Merge pull request #30227 from smithfarm/wip-41530-mimic

mimic: doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30244 from smithfarm/wip-41701-mimic
Nathan Cutler [Sun, 8 Sep 2019 09:16:59 +0000 (11:16 +0200)]
Merge pull request #30244 from smithfarm/wip-41701-mimic

mimic: build/ops: install-deps.sh: install `python*-devel` for python*rpm-macros

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoinstall-deps.sh: install `python*-devel` for python*rpm-macros 30244/head
Kefu Chai [Thu, 5 Sep 2019 15:17:06 +0000 (23:17 +0800)]
install-deps.sh: install `python*-devel` for python*rpm-macros

in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.

but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.

it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".

as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.

once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0b2be2b6710695867197ee84e411bdf0a843a06a)

Conflicts:
install-deps.sh
- VERSION_ID in mimic, MAJOR_VERSION in master

5 years agodoc: default values for mon_health_to_clog_* were flipped 30227/head
James McClune [Sun, 25 Aug 2019 03:24:31 +0000 (23:24 -0400)]
doc: default values for mon_health_to_clog_* were flipped

On a freshly installed nautilus cluster (i.e. 14.2.2), the default values are:

mon_health_to_clog_interval = 3600
mon_health_to_clog_tick_interval = 60.000000

Fixes: https://tracker.ceph.com/issues/41403
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
(cherry picked from commit 820ada1e58c3713b7de8a4ce669185d26860b84b)

5 years agorpm: always build ceph-test package 30188/head
Nathan Cutler [Thu, 15 Aug 2019 11:28:51 +0000 (13:28 +0200)]
rpm: always build ceph-test package

Fixes: https://tracker.ceph.com/issues/41296
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b9c744301051aa637f948be074be7b53c26cb504)

5 years agoMerge pull request #30128 from smithfarm/wip-41645-mimic
Nathan Cutler [Thu, 5 Sep 2019 14:08:34 +0000 (16:08 +0200)]
Merge pull request #30128 from smithfarm/wip-41645-mimic

mimic: build/ops: install-deps.sh: Remove CR repo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agodoc/rgw: document use of 'realm pull' instead of 'period pull' 30131/head
Casey Bodley [Thu, 9 May 2019 19:57:36 +0000 (15:57 -0400)]
doc/rgw: document use of 'realm pull' instead of 'period pull'

'radosgw-admin period pull' fetches a period configuration, but does not
update the realm's current_period to use it. the 'realm pull' command
does both, and the difference is especially important in the failover
case

Fixes: http://tracker.ceph.com/issues/39655
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f0059614b8e7055e4f1d6dfdefe362e2e81815ef)

5 years agoinstall-deps.sh: Remove CR repo 30128/head
Brad Hubbard [Thu, 22 Nov 2018 00:07:22 +0000 (10:07 +1000)]
install-deps.sh: Remove CR repo

Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.

Fixes: https://tracker.ceph.com/issues/37335
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit f39d9b7d1f4f2b63233802211a9ed0e641446ca1)

5 years agodocs: fix rgw ldap username token 30099/head
Thomas Kriechbaumer [Sun, 5 May 2019 15:40:40 +0000 (17:40 +0200)]
docs: fix rgw ldap username token

Signed-off-by: Thomas Kriechbaumer <thomas@kriechbaumer.name>
(cherry picked from commit 009f404983558ac084aa6772954ac918db19d890)

5 years agoceph-volume: test number of times LVs list was created 30096/head
Rishabh Dave [Mon, 19 Aug 2019 13:33:21 +0000 (19:03 +0530)]
ceph-volume: test number of times LVs list was created

Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 20d6264538dab444c0ef13f40c6c21ec3d2057f8)

5 years agoceph-volume: reuse list of LVs
Rishabh Dave [Wed, 14 Aug 2019 15:57:38 +0000 (21:27 +0530)]
ceph-volume: reuse list of LVs

Avoid creating of list of LVs in different methods of the same module
and resue them.

Fixes: https://tracker.ceph.com/issues/37490
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit ab3a58e78124a984d162b2f63992d01c5e59e2a7)

5 years agodoc: describe metadata_heap cleanup 30070/head
Dan van der Ster [Tue, 12 Mar 2019 15:42:25 +0000 (16:42 +0100)]
doc: describe metadata_heap cleanup

Fixes: http://tracker.ceph.com/issues/18174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit e76604224c0e74bbb3350743910d263c6591fd26)

6 years agoMerge pull request #29900 from alfredodeza/mimic-rm41378
Andrew Schoen [Mon, 26 Aug 2019 17:21:33 +0000 (12:21 -0500)]
Merge pull request #29900 from alfredodeza/mimic-rm41378

mimic ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt 29900/head
Alfredo Deza [Wed, 21 Aug 2019 18:15:32 +0000 (14:15 -0400)]
ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89231c9a60a46ba0691f787efbfe5ab35f05c679)

6 years agoMerge pull request #29770 from alfredodeza/mimic-bz-1738379
Alfredo Deza [Wed, 21 Aug 2019 12:11:26 +0000 (08:11 -0400)]
Merge pull request #29770 from alfredodeza/mimic-bz-1738379

mimic ceph-volume: use the OSD identifier when reporting success

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoceph-volume devices.lvm zap use the identifier to report success 29770/head
Alfredo Deza [Tue, 20 Aug 2019 12:04:56 +0000 (08:04 -0400)]
ceph-volume devices.lvm zap use the identifier to report success

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d0d5750c1ba508152c1d5111f2003a039a27145f)

6 years agoceph-volume tests check success message when zapping
Alfredo Deza [Tue, 20 Aug 2019 12:04:26 +0000 (08:04 -0400)]
ceph-volume tests check success message when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88c7429a60fdab6222d14132bea93b9bf773c720)

6 years agoMerge pull request #29661 from jan--f/wip-41246-mimic
Alfredo Deza [Tue, 20 Aug 2019 11:38:45 +0000 (07:38 -0400)]
Merge pull request #29661 from jan--f/wip-41246-mimic

mimic: simple functional tests drop test for lvm zap

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #29688 from jan--f/wip-41298-mimic
Alfredo Deza [Tue, 20 Aug 2019 11:37:25 +0000 (07:37 -0400)]
Merge pull request #29688 from jan--f/wip-41298-mimic

mimic: batch functional idempotency test fails since message is now on stderr

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #29618 from pdvian/wip-40943-mimic
Neha Ojha [Fri, 16 Aug 2019 21:07:51 +0000 (14:07 -0700)]
Merge pull request #29618 from pdvian/wip-40943-mimic

mimic: mon/OSDMonitor.cc: better error message about min_size

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29699 from pdvian/wip-41086-mimic
Neha Ojha [Fri, 16 Aug 2019 16:31:51 +0000 (09:31 -0700)]
Merge pull request #29699 from pdvian/wip-41086-mimic

mimic: Change default for bluestore_fsck_on_mount_deep as false

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agocommon/options.cc: common/options.cc: change default value of bluestore_fsck_on_umoun... 29699/head
Neha Ojha [Wed, 31 Jul 2019 20:08:24 +0000 (13:08 -0700)]
common/options.cc: common/options.cc: change default value of bluestore_fsck_on_umount_deep to false

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 8c86c3a5b4bdca258016019f63359f52aa8cd84b)

6 years agocommon/options.cc: change default value of bluestore_fsck_on_mount_deep to false
Neha Ojha [Wed, 31 Jul 2019 00:05:18 +0000 (17:05 -0700)]
common/options.cc: change default value of bluestore_fsck_on_mount_deep to false

Fixes: https://tracker.ceph.com/issues/41017
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7c03f088f1004d4e26b6bdea8fa58f8455f1d6a2)

6 years agoMerge pull request #29289 from smithfarm/wip-40497-mimic
Nathan Cutler [Thu, 15 Aug 2019 15:23:28 +0000 (17:23 +0200)]
Merge pull request #29289 from smithfarm/wip-40497-mimic

mimic: doc: Object Gateway multisite document read-only argument error

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoceph-volume: fix batch functional tests, idempotent test must check stderr 29688/head
Jan Fajerski [Thu, 15 Aug 2019 11:28:42 +0000 (13:28 +0200)]
ceph-volume: fix batch functional tests, idempotent test must check stderr

Fixes: https://tracker.ceph.com/issues/41295
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 88807110f3014e7a2a39c37ae5c64bb3ab107d80)

6 years agoceph-volume: don't try to test lvm zap on simple tests 29661/head
Jan Fajerski [Wed, 14 Aug 2019 12:36:58 +0000 (14:36 +0200)]
ceph-volume: don't try to test lvm zap on simple tests

Fixes: https://tracker.ceph.com/issues/41245
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 081109859a06f1b2ff6652dce07da4ddf6fb258e)

6 years agoMerge pull request #29631 from jan--f/backport-mimic-26826
Jan Fajerski [Wed, 14 Aug 2019 12:29:51 +0000 (14:29 +0200)]
Merge pull request #29631 from jan--f/backport-mimic-26826

ceph-volume: more mimic backports

6 years agoMerge pull request #29602 from jan--f/wip-41201-mimic
Jan Fajerski [Wed, 14 Aug 2019 12:28:33 +0000 (14:28 +0200)]
Merge pull request #29602 from jan--f/wip-41201-mimic

mimic: ceph-volume prints log messages to stdout

6 years agoceph-volume: make systemctl.get_running_osds resilient to garbage output 29631/head
Andrew Schoen [Fri, 8 Mar 2019 15:53:08 +0000 (09:53 -0600)]
ceph-volume: make systemctl.get_running_osds resilient to garbage output

Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f3707f1ad43513c2a6df17e9dce756533c3b4a1d)

6 years agoceph-volume: remove obsoleted simple unit test
Andrew Schoen [Thu, 7 Mar 2019 18:38:02 +0000 (12:38 -0600)]
ceph-volume: remove obsoleted simple unit test

This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b59dcd01c2dd386ca1b63eb66ed243afffb80211)

6 years agoceph-volume: tox tests should ignore the tests/functional directory
Andrew Schoen [Thu, 7 Mar 2019 18:37:00 +0000 (12:37 -0600)]
ceph-volume: tox tests should ignore the tests/functional directory

Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1881f386d8207ee8c10f0863c4a1abf18cf3c5d9)

6 years agoceph-volume: use `simple scan` and `simple activate --all` in tests
Andrew Schoen [Thu, 7 Mar 2019 17:57:03 +0000 (11:57 -0600)]
ceph-volume: use `simple scan` and `simple activate --all` in tests

In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 87c33f149a83bb1175bfb473b226e8c46e4e2a6e)

6 years agoceph-volume: update docs and help menu for `simple scan`
Andrew Schoen [Thu, 7 Mar 2019 17:10:19 +0000 (11:10 -0600)]
ceph-volume: update docs and help menu for `simple scan`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f7943ef6007fb05d83602bc22cbacf3bb67d8625)

6 years agoceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids
Andrew Schoen [Thu, 7 Mar 2019 16:01:40 +0000 (10:01 -0600)]
ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7cb098b6899542c875d911d28771a4c450d0bb3a)

6 years agoceph-volume: update `simple scan` to scan all running OSDs
Andrew Schoen [Wed, 6 Mar 2019 22:52:33 +0000 (16:52 -0600)]
ceph-volume: update `simple scan` to scan all running OSDs

If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 28e454ec30f867ae1229c66d263bcd608654553f)

6 years agoceph-volume: add systemd.systemctl.get_running_osd_ids
Andrew Schoen [Wed, 6 Mar 2019 22:51:06 +0000 (16:51 -0600)]
ceph-volume: add systemd.systemctl.get_running_osd_ids

This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 15d20e29a2f2d74caf05dfec559e30941c7a46ab)

6 years agoceph-volume tests.functional.simple add zap by osd id to xenial
Alfredo Deza [Thu, 6 Dec 2018 20:18:06 +0000 (15:18 -0500)]
ceph-volume tests.functional.simple add zap by osd id to xenial

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f53f67ca3d25deb82c20d31df418a19948ba6f64)

6 years agoceph-volume tests.functional.simple add zap by osd id to centos7
Alfredo Deza [Thu, 6 Dec 2018 20:17:55 +0000 (15:17 -0500)]
ceph-volume tests.functional.simple add zap by osd id to centos7

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8955ec14566c48a08bb4826256b422aa29a2c16e)

Conflicts:
src/ceph-volume/ceph_volume/tests/functional/simple/centos7/filestore/activate/test.yml
        remove with_items key

6 years agomon/OSDMonitor.cc: better error message about min_size 29618/head
Neha Ojha [Tue, 23 Jul 2019 00:50:31 +0000 (17:50 -0700)]
mon/OSDMonitor.cc: better error message about min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ce25ac0a9196f90acef82915f122c5a06277993a)

6 years agoceph-volume: never log to stdout, use stderr instead 29602/head
Jan Fajerski [Thu, 8 Aug 2019 08:42:37 +0000 (10:42 +0200)]
ceph-volume: never log to stdout, use stderr instead

We should never print log messages to stdout, as this should be reserved
for output of ceph-volume.

Fixes: https://tracker.ceph.com/issues/41158
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b8d6dcbe9f803c96c0af68da54f1262e9b6a9e77)

6 years agoceph-volume: terminal: encode unicode when writing to stdout
Kefu Chai [Sun, 24 Mar 2019 09:11:31 +0000 (17:11 +0800)]
ceph-volume: terminal: encode unicode when writing to stdout

python determins the encoding of stdout and stderr based on the LC_CTYPE
and PYTHONIOENCODING env variable, by default, python3's sys.stdout uses
'utf-8' as its encoding, so it will be able to write unicode string even
the stdout is not attached to a tty device. but when it comes to
python2, it will default to ascii if neither of these variabls is set.
so, if we are writing unicode using `_Write` in an environment where
LC_CTYPE and/or PYTHONIOENCODING are using non UTF-8 encoding, it chokes
by raising `UnicodeEncodeError` exception.

in this change, we add a wrapper around `_Write._writer` so it is able
to write unicode string in such a non-unicode-friendly environment.

for more info related the encoding of stdout and stderr, see
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONIOENCODING .

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 77912c0c71874a23785d3bdd109ad61e4f4c0e28)

6 years agoMerge pull request #29507 from jan--f/wip-41138-mimic
Jan Fajerski [Wed, 7 Aug 2019 08:18:45 +0000 (10:18 +0200)]
Merge pull request #29507 from jan--f/wip-41138-mimic

mimic: ceph-volume prints errors to stdout with --format json

6 years agoceph-volume: print most logging messages to stderr 29507/head
Jan Fajerski [Mon, 4 Mar 2019 10:28:13 +0000 (11:28 +0100)]
ceph-volume: print most logging messages to stderr

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: http://tracker.ceph.com/issues/38548
(cherry picked from commit 1f8f1e95dee88b4bab2d6931e720ed83584d2887)

 Conflicts:
src/pybind/mgr/test_orchestrator/module.py
        removed

6 years agoMerge pull request #29417 from jan--f/wip-41022-mimic
Jan Fajerski [Wed, 7 Aug 2019 06:44:00 +0000 (08:44 +0200)]
Merge pull request #29417 from jan--f/wip-41022-mimic

mimic: simple: when 'type' file is not present activate fails

6 years agoMerge pull request #29255 from smithfarm/wip-39274-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:19:37 +0000 (14:19 -0700)]
Merge pull request #29255 from smithfarm/wip-39274-mimic

mimic: rgw: S3 policy evaluated incorrectly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29256 from smithfarm/wip-39415-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:19:15 +0000 (14:19 -0700)]
Merge pull request #29256 from smithfarm/wip-39415-mimic

mimic: rgw: multisite: period pusher gets 403 Forbidden against other zonegroups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29262 from smithfarm/wip-39748-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:18:55 +0000 (14:18 -0700)]
Merge pull request #29262 from smithfarm/wip-39748-mimic

mimic: rgw: Add support for --bypass-gc flag of radosgw-admin bucket rm command in RGW Multi-site

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29267 from smithfarm/wip-40128-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:18:30 +0000 (14:18 -0700)]
Merge pull request #29267 from smithfarm/wip-40128-mimic

mimic: rgw: Swift interface: server side copy fails if object name contains '?'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29268 from smithfarm/wip-40136-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:18:07 +0000 (14:18 -0700)]
Merge pull request #29268 from smithfarm/wip-40136-mimic

mimic: rgw: the Multi-Object Delete operation of S3 API wrongly handles the Code response element

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29270 from smithfarm/wip-40141-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:17:35 +0000 (14:17 -0700)]
Merge pull request #29270 from smithfarm/wip-40141-mimic

mimic: rgw: multisite: 'radosgw-admin bucket sync status' should call syncs_from(source.name) instead of id

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29273 from smithfarm/wip-40144-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:16:18 +0000 (14:16 -0700)]
Merge pull request #29273 from smithfarm/wip-40144-mimic

mimic: rgw: Multisite sync corruption for large multipart obj

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29277 from smithfarm/wip-40225-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:15:35 +0000 (14:15 -0700)]
Merge pull request #29277 from smithfarm/wip-40225-mimic

mimic: rgw_file: include tenant when hashing bucket names

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agoMerge pull request #29279 from smithfarm/wip-40351-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:15:06 +0000 (14:15 -0700)]
Merge pull request #29279 from smithfarm/wip-40351-mimic

mimic: rgw/OutputDataSocket: append_output(buffer::list&) says it will (but does not) discard output at data_max_backlog

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29284 from smithfarm/wip-40354-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:14:44 +0000 (14:14 -0700)]
Merge pull request #29284 from smithfarm/wip-40354-mimic

mimic: rgw: multisite: RGWListBucketIndexesCR for data full sync needs pagination

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29285 from smithfarm/wip-40357-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:14:21 +0000 (14:14 -0700)]
Merge pull request #29285 from smithfarm/wip-40357-mimic

mimic: rgw: provide admin-friendly reshard status output

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29288 from smithfarm/wip-40360-mimic
Yuri Weinstein [Tue, 6 Aug 2019 21:13:54 +0000 (14:13 -0700)]
Merge pull request #29288 from smithfarm/wip-40360-mimic

mimic: rgw: set null version object issues

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agomimic: ceph-volume broken assertion errors after pytest changes 29417/head
Jan Fajerski [Tue, 6 Aug 2019 13:34:12 +0000 (15:34 +0200)]
mimic: ceph-volume broken assertion errors after pytest changes

This fixes a backported test that was not fixed on a backport. Original
fix: https://github.com/ceph/ceph/pull/28948. Backport:
https://github.com/ceph/ceph/pull/29243.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #29463 from jan--f/wip-41056-mimic
Alfredo Deza [Mon, 5 Aug 2019 20:17:53 +0000 (16:17 -0400)]
Merge pull request #29463 from jan--f/wip-41056-mimic

mimic: ceph-volume does not recognize wal/db partitions created by ceph-disk

Reviewed-by: Alfredo Deza <adeza@redhat.com>