]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoceph-volume tests ensure activate behavior with systemd disabling 25011/head
Alfredo Deza [Thu, 8 Nov 2018 19:14:17 +0000 (14:14 -0500)]
ceph-volume tests ensure activate behavior with systemd disabling

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 3e80118ccaccaa61e92b01f4f389d6056401d8a7)

6 years agoceph-volume simple.activate factor out systemd interactions for easier testing
Alfredo Deza [Thu, 8 Nov 2018 19:13:52 +0000 (14:13 -0500)]
ceph-volume simple.activate factor out systemd interactions for easier testing

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 18ddd962116b5359af770088535f502797ac79c4)

6 years agoceph-volume simple.trigger replace systemd with from_trigger keyword argument
Alfredo Deza [Thu, 8 Nov 2018 14:14:24 +0000 (09:14 -0500)]
ceph-volume simple.trigger replace systemd with from_trigger keyword argument

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 9526587403b76fa5397ae80b63328e3a59765c2e)

6 years agoceph-volume simple.activate allow a --no-systemd flag to skip systemctl actions
Alfredo Deza [Thu, 8 Nov 2018 14:13:31 +0000 (09:13 -0500)]
ceph-volume simple.activate allow a --no-systemd flag to skip systemctl actions

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
(cherry picked from commit 4d9ad0d3d4bedd5efa95653fbb86607f0b469174)

6 years agoMerge pull request #24959 from alfredodeza/mimic-rm36672 25004/head
Alfredo Deza [Wed, 7 Nov 2018 11:47:51 +0000 (06:47 -0500)]
Merge pull request #24959 from alfredodeza/mimic-rm36672

mimic ceph-volume tests install ceph-ansible's requirements.txt dependencies

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests install ceph-ansible's requirements.txt dependencies 24959/head
Alfredo Deza [Thu, 1 Nov 2018 12:48:28 +0000 (08:48 -0400)]
ceph-volume tests install ceph-ansible's requirements.txt dependencies

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 52ddb69813c5849506db5f850a3f2532ff1124b5)

6 years agoMerge pull request #24957 from alfredodeza/mimic-rm36704
Andrew Schoen [Tue, 6 Nov 2018 21:24:20 +0000 (15:24 -0600)]
Merge pull request #24957 from alfredodeza/mimic-rm36704

mimic: ceph-volume tests.systemd update imports for systemd module

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.systemd update imports for systemd module 24957/head
Alfredo Deza [Tue, 6 Nov 2018 14:45:08 +0000 (09:45 -0500)]
ceph-volume tests.systemd update imports for systemd module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a63a6c86a03738bf50fa71a490951343a83afff6)

6 years agoceph-volume systemd import main so console_scripts work for executable
Alfredo Deza [Tue, 6 Nov 2018 14:44:39 +0000 (09:44 -0500)]
ceph-volume systemd import main so console_scripts work for executable

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 280e2baab27cc061d2d5e456f098cd3405f36873)

6 years agoMerge pull request #24826 from alfredodeza/mimic-rm36246
Andrew Schoen [Tue, 6 Nov 2018 21:06:30 +0000 (15:06 -0600)]
Merge pull request #24826 from alfredodeza/mimic-rm36246

mimic ceph-volume util.encryption don't push stderr to terminal

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24852 from ceph/mimic-rm36648
Andrew Schoen [Tue, 6 Nov 2018 20:33:44 +0000 (14:33 -0600)]
Merge pull request #24852 from ceph/mimic-rm36648

mimic ceph-volume systemd import main so console_scripts work for executable

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume systemd import main so console_scripts work for executable 24852/head
Alfredo Deza [Tue, 30 Oct 2018 19:50:28 +0000 (15:50 -0400)]
ceph-volume systemd import main so console_scripts work for executable

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0bcda52566279404b88184b2ce7b2200d581b0b7)

6 years agoMerge pull request #24838 from alfredodeza/mimic-no-scripts
Andrew Schoen [Tue, 6 Nov 2018 19:51:01 +0000 (13:51 -0600)]
Merge pull request #24838 from alfredodeza/mimic-no-scripts

mimic ceph-volume: use console_scripts

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24873 from trociny/wip-36643-mimic
Yuri Weinstein [Sun, 4 Nov 2018 14:31:25 +0000 (06:31 -0800)]
Merge pull request #24873 from trociny/wip-36643-mimic

mimic: osdc: reduce ObjectCacher's memory fragments

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24860 from jonsger/mimic-backport-pr#23242
Yuri Weinstein [Fri, 2 Nov 2018 14:51:43 +0000 (07:51 -0700)]
Merge pull request #24860 from jonsger/mimic-backport-pr#23242

mimic: rgw: set default objecter_inflight_ops = 24576

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24811 from dillaman/wip-36569-mimic
Yuri Weinstein [Thu, 1 Nov 2018 19:40:23 +0000 (12:40 -0700)]
Merge pull request #24811 from dillaman/wip-36569-mimic

mimic: qa/tasks/qemu: use unique clone directory to avoid race with workunit

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #24443 from smithfarm/wip-26850-mimic
Yuri Weinstein [Thu, 1 Nov 2018 16:59:04 +0000 (09:59 -0700)]
Merge pull request #24443 from smithfarm/wip-26850-mimic

mimic: ceph_volume_client: py3 compatible

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoosdc: reduce ObjectCacher's memory fragments 24873/head
Yan, Zheng [Thu, 27 Sep 2018 05:10:55 +0000 (13:10 +0800)]
osdc: reduce ObjectCacher's memory fragments

Fixes: http://tracker.ceph.com/issues/36192
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 5d4f82117ed20f7a40259c3f5471e7a96a4c456f)

Conflicts:
src/osdc/ObjectCacher.cc: trivial

6 years agorgw: set default objecter_inflight_ops = 24576 24860/head
Matt Benjamin [Wed, 25 Jul 2018 20:06:33 +0000 (16:06 -0400)]
rgw: set default objecter_inflight_ops = 24576

Fixes: http://tracker.ceph.com/issues/25109
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 102c5bd)
Signed-off-by: Jonathan Brielmaier <jbrielmaier@suse.de>
6 years agoMerge pull request #24745 from smithfarm/wip-36592-mimic
Yuri Weinstein [Tue, 30 Oct 2018 19:46:11 +0000 (12:46 -0700)]
Merge pull request #24745 from smithfarm/wip-36592-mimic

mimic: segv in BlueStore::OldExtent::create

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #24488 from tspmelo/wip-24907-mimic
Yuri Weinstein [Tue, 30 Oct 2018 19:45:17 +0000 (12:45 -0700)]
Merge pull request #24488 from tspmelo/wip-24907-mimic

mimic: mgr/dashboard: Add support for URI encode

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoceph-volume: use console_scripts 24838/head
Mehdi Abaakouk [Fri, 26 Oct 2018 09:59:55 +0000 (11:59 +0200)]
ceph-volume: use console_scripts

Using "#!/usr/bin/env python" is not recommended as it's not portable.

setuptools provides an console_scripts entry_point that generates
scripts that always have the good sheban whatever the target operating
system and python version/distribution.

http://tracker.ceph.com/issues/36601

Signed-off-by: Mehdi Abaakouk <sileht@sileht.net>
(cherry picked from commit 6a2ed6c285b74f463f223d3501def8ef4e6283b1)

6 years agoMerge pull request #24435 from smithfarm/wip-32092-mimic
Yuri Weinstein [Tue, 30 Oct 2018 17:07:49 +0000 (10:07 -0700)]
Merge pull request #24435 from smithfarm/wip-32092-mimic

mimic: mds: migrate strays part by part when shutdown mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #24485 from ukernel/mimic-36346
Yuri Weinstein [Tue, 30 Oct 2018 16:49:19 +0000 (09:49 -0700)]
Merge pull request #24485 from ukernel/mimic-36346

mimic: Revert "mimic: cephfs-journal-tool: enable purge_queue journal's event commands"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24572 from pdvian/wip-36280-mimic
Yuri Weinstein [Tue, 30 Oct 2018 16:48:40 +0000 (09:48 -0700)]
Merge pull request #24572 from pdvian/wip-36280-mimic

mimic: qa: tolerate failed rank while waiting for state

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24583 from smithfarm/wip-24933-mimic
Yuri Weinstein [Tue, 30 Oct 2018 16:48:08 +0000 (09:48 -0700)]
Merge pull request #24583 from smithfarm/wip-24933-mimic

mimic: cephfs-journal-tool: wrong layout info used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #24661 from vshankar/wip-35975
Yuri Weinstein [Tue, 30 Oct 2018 16:47:34 +0000 (09:47 -0700)]
Merge pull request #24661 from vshankar/wip-35975

mimic: mds: evict cap revoke non-responding clients

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24684 from smithfarm/wip-36501-mimic
Yuri Weinstein [Tue, 30 Oct 2018 16:46:31 +0000 (09:46 -0700)]
Merge pull request #24684 from smithfarm/wip-36501-mimic

mimic: qa: increase rm timeout for workunit cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoceph-volume util.encryption don't push stderr to terminal when cryptsetup fails to... 24826/head
Alfredo Deza [Wed, 3 Oct 2018 12:31:20 +0000 (08:31 -0400)]
ceph-volume util.encryption don't push stderr to terminal when cryptsetup fails to find device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40f919850fddd14659243c47860224981439906a)

6 years agoqa/tasks/qemu: use unique clone directory to avoid race with workunit 24811/head
Jason Dillaman [Mon, 22 Oct 2018 14:44:40 +0000 (10:44 -0400)]
qa/tasks/qemu: use unique clone directory to avoid race with workunit

If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5d56014c61b107dcb5d05c2221c2e844324f304c)

6 years agoMerge pull request #24708 from smithfarm/wip-26839-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:21:12 +0000 (16:21 -0400)]
Merge pull request #24708 from smithfarm/wip-26839-mimic

mimic: librados application's symbol could conflict with the libceph-common

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24710 from smithfarm/wip-35977-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:20:22 +0000 (16:20 -0400)]
Merge pull request #24710 from smithfarm/wip-35977-mimic

mimic: multisite: incremental data sync makes unnecessary call to RGWReadRemoteDataLogShardInfoCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24734 from b-ranto/wip-mimic-http
Yuri Weinstein [Mon, 29 Oct 2018 20:19:06 +0000 (16:19 -0400)]
Merge pull request #24734 from b-ranto/wip-mimic-http

mimic: mgr/dashboard: Add http support to dashboard

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by:  Volker Theile <vtheile@suse.com>

6 years agoMerge pull request #23916 from liewegas/wip-pod-hostname-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:17:17 +0000 (16:17 -0400)]
Merge pull request #23916 from liewegas/wip-pod-hostname-mimic

mimic: common: get real hostname from container/pod environment

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24260 from tchaikov/mimic-35969
Yuri Weinstein [Mon, 29 Oct 2018 20:16:09 +0000 (16:16 -0400)]
Merge pull request #24260 from tchaikov/mimic-35969

mimic: rpm: use updated gperftools

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #24431 from smithfarm/wip-35936-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:14:51 +0000 (16:14 -0400)]
Merge pull request #24431 from smithfarm/wip-35936-mimic

mimic: OSDMapMapping does not handle active.size() > pool size

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #24478 from tspmelo/wip-27851-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:14:13 +0000 (16:14 -0400)]
Merge pull request #24478 from tspmelo/wip-27851-mimic

mimic: mgr/dashboard: Remove fieldsets when using CdTable

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agoMerge pull request #24622 from dillaman/wip-36430-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:12:47 +0000 (16:12 -0400)]
Merge pull request #24622 from dillaman/wip-36430-mimic

mimic: qa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
6 years agoMerge pull request #24551 from pdvian/wip-36276-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:11:38 +0000 (16:11 -0400)]
Merge pull request #24551 from pdvian/wip-36276-mimic

mimic: test/rbd_mirror: race in WaitingOnLeaderReleaseLeader

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24647 from liewegas/wip-pr-23273-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:09:47 +0000 (16:09 -0400)]
Merge pull request #24647 from liewegas/wip-pr-23273-mimic

mimic: os/bluestore: handle spurious read errors

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #24711 from smithfarm/wip-36238-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:07:53 +0000 (16:07 -0400)]
Merge pull request #24711 from smithfarm/wip-36238-mimic

mimic: [test] periodic seg faults within unittest_librbd

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24782 from alfredodeza/mimic-evelu-ceph-volume-choose_disk
Andrew Schoen [Mon, 29 Oct 2018 15:44:30 +0000 (10:44 -0500)]
Merge pull request #24782 from alfredodeza/mimic-evelu-ceph-volume-choose_disk

mimic Additional work on ceph-volume to add some choose_disk capabilities

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agodoc: v13.2.3 release notes for wrong mds backport 24485/head
Yan, Zheng [Mon, 29 Oct 2018 02:11:35 +0000 (10:11 +0800)]
doc: v13.2.3 release notes for wrong mds backport

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
6 years agoceph_volume: Checking device validity at init time 24782/head
Erwan Velu [Wed, 10 Oct 2018 18:26:01 +0000 (20:26 +0200)]
ceph_volume: Checking device validity at init time

When initializing the Device structure, it have to run is_valid() to
ensure the data structures (_is_valid & rejected_reasons) to be
populated accordingly to the device state.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit ef0ceef7c717ef615dc214299cb4a6b32c1edbe6)

6 years agoceph_volume: Rejecting locked devices
Erwan Velu [Tue, 9 Oct 2018 20:51:39 +0000 (22:51 +0200)]
ceph_volume: Rejecting locked devices

If we cannot open a block device in O_RDWR in exclusive mode, it means
someone is actually using it like a raw database or similar.

In that case, the device should be considered as unusable as OSDs will
not be in a position to use it.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit d5de9583ee6c4d9ccaab2b1146606a0ee316cc27)

6 years agoceph_volume: Reporting nr_requests
Erwan Velu [Tue, 9 Oct 2018 20:28:19 +0000 (22:28 +0200)]
ceph_volume: Reporting nr_requests

We are already reporting the rotational & scheduler of a disk device.
Reporting the nr_requests could be useful to get how many concurrent IOs
the device supports/reports.

That could help detecting badly detected/configured devices.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit e0ea3d475a85844afdcdf92cf638b8128c8e2d9e)

6 years agoceph_volume: Reporting firmware revision
Erwan Velu [Tue, 9 Oct 2018 20:26:28 +0000 (22:26 +0200)]
ceph_volume: Reporting firmware revision

We are already reporting model & vendor of a given disk, let's also
report the revision of the firmware. That is useful to filter-out some
known broken revisions.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 59720794079bef0ba7f8bc2d9dfc8aaddd210005)

6 years agoceph_volume: Rejecting Read-only devices
Erwan Velu [Tue, 9 Oct 2018 20:02:31 +0000 (22:02 +0200)]
ceph_volume: Rejecting Read-only devices

If a devices is said to be read-only, there is no chance we can actually
use it. So let's report it as unusable.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit f1a94350064263f27f67d2c715c5ca5922481a01)

6 years agoceph_volume: Adding Device.is_valid()
Erwan Velu [Tue, 9 Oct 2018 19:04:02 +0000 (21:04 +0200)]
ceph_volume: Adding Device.is_valid()

A block device can be filtered-out/ignored because it have features that
doesn't match Ceph's expectations.

As of today, the current code was rejected removable devices but it was
pretty hidden from the user, and implicit in the get_devices() function.

This patch is creating a new is_valid() function to perform all the
rejection tests and returns if this device can be used in the Ceph
context or not.

If is_valid() is returning False, the 'rejected_reasons' list reports all
the reasons why that devices got rejected.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit d8fdf0b7532cdffcae56511c377679e51841caec)

6 years agoMerge pull request #24449 from alfredodeza/mimic-rm24795
Andrew Schoen [Fri, 26 Oct 2018 20:00:08 +0000 (15:00 -0500)]
Merge pull request #24449 from alfredodeza/mimic-rm24795

mimic ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24760 from ceph/backport-mimic-24587
Alfredo Deza [Fri, 26 Oct 2018 13:46:57 +0000 (09:46 -0400)]
Merge pull request #24760 from ceph/backport-mimic-24587

mimic: ceph-volume: adds a --prepare flag to `lvm batch`

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume lvm.prepare update help to indicate partitions are needed, not devices 24449/head
Alfredo Deza [Wed, 3 Oct 2018 12:11:58 +0000 (08:11 -0400)]
ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d31dd95b95445825f5e6669dd4ecb3118b09fdcf)

6 years agoceph-volume: update man page for batch --prepare 24760/head
Andrew Schoen [Thu, 25 Oct 2018 12:51:41 +0000 (07:51 -0500)]
ceph-volume: update man page for batch --prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit faa3aed4ad4df2dbdd853235e364d6e3b9b176b6)

6 years agoceph-volume: docs for --prepare flag of lvm batch
Andrew Schoen [Mon, 15 Oct 2018 15:38:43 +0000 (10:38 -0500)]
ceph-volume: docs for --prepare flag of lvm batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b4825e1bd7b75a0a53ee5678230b262286293d75)

6 years agoceph-volume: when --prepare is added to batch the OSDs are only prepared
Andrew Schoen [Mon, 15 Oct 2018 15:35:12 +0000 (10:35 -0500)]
ceph-volume: when --prepare is added to batch the OSDs are only prepared

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit abd238dc1f33252a43adc8b86efb3d3b4a076964)

6 years agoceph-volume: adds a --prepare flag to ceph-volume lvm batch
Andrew Schoen [Mon, 15 Oct 2018 14:33:30 +0000 (09:33 -0500)]
ceph-volume: adds a --prepare flag to ceph-volume lvm batch

This flag will only prepare the OSDs, not activate them. This is useful
in our containerized ceph solution.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 4368ca7d48f4ce35f67854e8b0ca2be9babc5371)

6 years agoMerge pull request #24753 from alfredodeza/mimic-rm36386
Andrew Schoen [Thu, 25 Oct 2018 16:10:47 +0000 (11:10 -0500)]
Merge pull request #24753 from alfredodeza/mimic-rm36386

mimic ceph-volume remove version reporting from help menu

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume remove version reporting from help menu 24753/head
Alfredo Deza [Wed, 10 Oct 2018 19:35:31 +0000 (15:35 -0400)]
ceph-volume remove version reporting from help menu

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 20db7bf585f6433c6705396264005a4227cbf2e3)

6 years agoMerge pull request #24740 from alfredodeza/mimic-rm36492
Alfredo Deza [Thu, 25 Oct 2018 12:04:34 +0000 (08:04 -0400)]
Merge pull request #24740 from alfredodeza/mimic-rm36492

mimic ceph-volume: do not send (lvm) stderr/stdout to the terminal, use the logfile

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.api update monkeypatching to use **kw 24740/head
Alfredo Deza [Wed, 24 Oct 2018 14:55:06 +0000 (10:55 -0400)]
ceph-volume tests.api update monkeypatching to use **kw

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d6bcda75f846bc7c44c7b16b3687a65fed7cafc)

6 years agoceph-volume tests allow **kw when monkeypatching
Alfredo Deza [Wed, 24 Oct 2018 14:54:48 +0000 (10:54 -0400)]
ceph-volume tests allow **kw when monkeypatching

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 04e51c956eadcbd38f39ec3dd4b78009d2cd4a70)

6 years agoceph-volume api.lvm: do not spit out errors on the terminal from LVM
Alfredo Deza [Wed, 24 Oct 2018 14:54:18 +0000 (10:54 -0400)]
ceph-volume api.lvm: do not spit out errors on the terminal from LVM

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 581eed36ea0e7b5c5e41f8b83350403bcf73a31a)

6 years agoMerge pull request #24588 from ceph/backport-mimic-24404
Alfredo Deza [Wed, 24 Oct 2018 21:22:50 +0000 (17:22 -0400)]
Merge pull request #24588 from ceph/backport-mimic-24404

mimic: ceph-volume: make `lvm batch` idempotent

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoos/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put 24745/head
Sage Weil [Mon, 22 Oct 2018 19:38:48 +0000 (14:38 -0500)]
os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put

A                             B
SharedBlobSet::lookup()
  takes lock
  nref is not 0
                              SharedBlob::put()
                                --nref
returns SharedBlobRef,
  ++nref
                                takes cache lock
                                SharedBlobSet::remove
                                  takes lock
                                  removes
                                deletes SharedBlob

-> A ends up with a ref to deleted SharedBlob

Fix by verifying that nref is still zero in SharedBlobSet::remove(),
while we are holding the SharedBlobSet::lock.  The lock ensures that we
have increased the ref for the lookup before entering remove, so we can
verify that nref is still zero before removing it.  If not, we have
raced, and put() bails out and does nothing.

Fixes: http://tracker.ceph.com/issues/36526
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 020bd7b5f38a82d9eef5e25e6f4a4dd12b066915)

Conflicts:
src/os/bluestore/BlueStore.h

6 years agolibrbd: object copy state machine might dereference a deleted object 24711/head
Jason Dillaman [Wed, 26 Sep 2018 18:39:25 +0000 (14:39 -0400)]
librbd: object copy state machine might dereference a deleted object

Fixes: http://tracker.ceph.com/issues/36220
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1a9c3be2ceffcb981910de429a7da0fbec43e643)

6 years agoMerge pull request #24441 from joscollin/wip-36313-mimic
Nathan Cutler [Tue, 23 Oct 2018 06:26:55 +0000 (08:26 +0200)]
Merge pull request #24441 from joscollin/wip-36313-mimic

mimic: doc: fix broken fstab url in cephfs/fuse

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agorpm: Use updated gperftools-libs at runtime 24260/head
Brad Hubbard [Tue, 16 Oct 2018 01:57:05 +0000 (11:57 +1000)]
rpm: Use updated gperftools-libs at runtime

Due to ABI breakage in libtcmalloc.so.4 we need to specify the minimum
version to be used at runtime to be greater than or equal to the version
used at build time.

Fixes: http://tracker.ceph.com/issues/36508
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 24636267f957ff9518396ac1b8212f2402286d82)

6 years agorpm: use updated gperftools
Kefu Chai [Mon, 17 Sep 2018 06:09:17 +0000 (14:09 +0800)]
rpm: use updated gperftools

make sure we only build with the higher version of gperftools on
distros where both 2.4 and 2.6.1 are packaged. see
https://git.centos.org/summary/rpms!gperftools.git . at the time of
writing, gperftools 2.6.1 is packaged for CentOS/RHEL 7, if gperftools
(>= 2.4) is required by Ceph, and user already has this version
installed, when new Ceph packages are installed, the updated gperftools
2.6.1 version won't be installed as a dependency. when launching
Ceph compiled with tcmalloc enabled, we will have

symbol lookup error: ceph-osd: undefined symbol: _ZdaPvm

so, by bumping up the required version of gperftools, the updated
gperftools will be installed.

see https://software.opensuse.org/package/gperftools, openSUSE/SLE offer
2.5. so they are safe at this moment.

Fixes: http://tracker.ceph.com/issues/35969
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e69e50a859acebe0bf854c12a71bfe745e0c1cf6)

6 years agorgw: remove unused query for remote datalog info 24710/head
Casey Bodley [Wed, 15 Aug 2018 20:09:43 +0000 (16:09 -0400)]
rgw: remove unused query for remote datalog info

Fixes: http://tracker.ceph.com/issues/26952
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a543df729a6ca415648678e2ae2ed46eceaba086)

Conflicts:
src/rgw/rgw_data_sync.cc
- trivial whitespace

6 years agorgw: remove unused datalog trim detection logic
Casey Bodley [Wed, 15 Aug 2018 20:07:32 +0000 (16:07 -0400)]
rgw: remove unused datalog trim detection logic

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0d47743e0f6b9f51848056885831910d0ea5b282)

6 years agorgw: incremental data sync uses truncated flag to detect end of listing
Casey Bodley [Wed, 15 Aug 2018 20:04:37 +0000 (16:04 -0400)]
rgw: incremental data sync uses truncated flag to detect end of listing

we call wait() after incremental sync if we've reached the end of the
datalog listing. the existing logic compares our local marker with the
remote's high marker, with some extra code to handle the case where the
remote log was trimmed

all of this can be simplified by using the 'truncated' flag returned
with the RGWReadRemoteDataLogShardCR used to list the remote datalog

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fb2b618658d8ff9c311838f59e396ab24c21e9e8)

6 years agotest: use death_test_style="threadsafe" for Mutex.NormalAsserts 24708/head
Kefu Chai [Mon, 30 Jul 2018 13:29:03 +0000 (21:29 +0800)]
test: use death_test_style="threadsafe" for Mutex.NormalAsserts

use threadsafe for running the Mutex assert test to run the test from
beginning to avoid the hang. this test overrides the symbol of
ceph::__ceph_assert_fail() with a local one offered by the application
links against libceph-common. but we intentionally forbid this behavior:
we do not allow libceph-common to reference the symbols exposed from
application with the same name. see http://tracker.ceph.com/issues/25154

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1739d6b77796026d0e3e7cf49aa90e1728e622cc)

6 years agocmake: always prefer local symbols
Kefu Chai [Mon, 30 Jul 2018 11:55:51 +0000 (19:55 +0800)]
cmake: always prefer local symbols

Fixes: http://tracker.ceph.com/issues/25154
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 38e344659980dded622322d42a7742c0e50107f8)

6 years agoMerge pull request #24017 from dzafman/wip-35909
Yuri Weinstein [Mon, 22 Oct 2018 15:40:24 +0000 (11:40 -0400)]
Merge pull request #24017 from dzafman/wip-35909

mimic: test: Use a grep pattern that works across releases

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24037 from smithfarm/wip-21057-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:39:41 +0000 (11:39 -0400)]
Merge pull request #24037 from smithfarm/wip-21057-mimic

mimic: debian/control: require fuse for ceph-fuse

Reviewed-by: James Page <james.page@ubuntu.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24480 from pdvian/wip-36146-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:37:09 +0000 (11:37 -0400)]
Merge pull request #24480 from pdvian/wip-36146-mimic

mimic: fsck: cid is improperly matched to oid

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24581 from smithfarm/wip-36437-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:55 +0000 (11:34 -0400)]
Merge pull request #24581 from smithfarm/wip-36437-mimic

mimic: ec: src/common/interval_map.h: 161: FAILED assert(len > 0)

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24600 from pdvian/wip-36296-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:02 +0000 (11:34 -0400)]
Merge pull request #24600 from pdvian/wip-36296-mimic

mimic : osdc/Objecter: possible race condition with connection reset

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24601 from pdvian/wip-36298-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:32:42 +0000 (11:32 -0400)]
Merge pull request #24601 from pdvian/wip-36298-mimic

mimic: osd: add creating to pg_string_state

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoqa/tasks/workunit: use suite branch/SHA1 when cloning workunits 24622/head
Jason Dillaman [Fri, 12 Oct 2018 14:02:35 +0000 (10:02 -0400)]
qa/tasks/workunit: use suite branch/SHA1 when cloning workunits

Right now it's using the Ceph branch/SHA1 but it's using the suite
Git URL.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1e3dc02604cf7c0d3991dfd9fe2596ea34e80bad)

6 years agoqa/tasks: qemu task now uses a relative path in suite repo for test
Jason Dillaman [Thu, 11 Oct 2018 20:21:35 +0000 (16:21 -0400)]
qa/tasks: qemu task now uses a relative path in suite repo for test

This makes it easier to re-run tests against a suite branch without
requiring a full ceph-ci build and repo.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c1f950236b2f1ba3b588fc638ebf8cb785c36e08)

6 years agoqa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh
Jason Dillaman [Thu, 11 Oct 2018 19:17:25 +0000 (15:17 -0400)]
qa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh

Fixes: http://tracker.ceph.com/issues/36409
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ddb7e5eb80ebb78beff6dfb25042f0016a474491)

6 years agoqa/tasks/cram: tasks now must live in the repository
Ilya Dryomov [Thu, 6 Sep 2018 14:07:08 +0000 (16:07 +0200)]
qa/tasks/cram: tasks now must live in the repository

Commit 0d8887652d53 ("qa/tasks/cram: use suite_repo repository for all
cram jobs") removed hardcoded git.ceph.com links, but as it turned out
it is still used for nightlies.  There is no good way to accommodate
the different URL schemes, so let's get rid of URLs altogether.

Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 592f566b4e270d3833ba896dabb193d5241dd5de)

6 years agoqa/tasks/workunit: factor out overrides and refspec logic
Ilya Dryomov [Thu, 6 Sep 2018 14:53:25 +0000 (16:53 +0200)]
qa/tasks/workunit: factor out overrides and refspec logic

Allow for reuse in the cram task.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit e1c89b51c80407fadbfee82d5d396cfcbd791aae)

6 years agoqa/tasks/cram: use suite_repo repository for all cram jobs
Ilya Dryomov [Mon, 3 Sep 2018 15:40:08 +0000 (17:40 +0200)]
qa/tasks/cram: use suite_repo repository for all cram jobs

Currently git.ceph.com is hardcoded for all cram jobs.  Testing
modifications is a pain: one needs to push to either ceph/ceph.git or
ceph/ceph-ci.git (depending on where the ceph branch is at, triggering
unnecessary builds in the latter case) and wait for the mirror to sync.
Runs scheduled against branches in developer's forks fail.

Move away from git.ceph.com to allow mixing branches and repositories,
similar to workunits.

Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0d8887652d5312f7059ab2bdb52c948aa01680b0)

6 years agoqa: increase timeout for cleanup 24684/head
Patrick Donnelly [Tue, 9 Oct 2018 20:31:26 +0000 (13:31 -0700)]
qa: increase timeout for cleanup

Apparently 15m is not long enough for some workunits like fsstress.

Fixes: http://tracker.ceph.com/issues/36365
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit dc6e7eadf8ee21ec1ecc5b3fcf49967966ad96e0)

6 years agoMerge pull request #24379 from smithfarm/wip-36136-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:51:33 +0000 (16:51 -0400)]
Merge pull request #24379 from smithfarm/wip-36136-mimic

mimic: mds: rctime may go back

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24384 from smithfarm/wip-32104-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:54 +0000 (16:50 -0400)]
Merge pull request #24384 from smithfarm/wip-32104-mimic

mimic: mds: allows client to create .. and . dirents

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #24385 from smithfarm/wip-36134-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:20 +0000 (16:50 -0400)]
Merge pull request #24385 from smithfarm/wip-36134-mimic

mimic: client: update ctime when modifying file content

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24408 from smithfarm/wip-36153-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:49:25 +0000 (16:49 -0400)]
Merge pull request #24408 from smithfarm/wip-36153-mimic

mimic: qa: fsstress workunit does not execute in parallel on same host without clobbering files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24411 from smithfarm/wip-36195-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:52 +0000 (16:48 -0400)]
Merge pull request #24411 from smithfarm/wip-36195-mimic

mimic: mds: internal op missing events time 'throttled', 'all_read', 'dispatched'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24438 from pdvian/wip-36156-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:28 +0000 (16:48 -0400)]
Merge pull request #24438 from pdvian/wip-36156-mimic

mimic: qa: extend timeout for SessionMap flush

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24439 from pdvian/wip-36197-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:52 +0000 (16:47 -0400)]
Merge pull request #24439 from pdvian/wip-36197-mimic

mimic: ceph-fuse: add to selinux profile

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24463 from pdvian/wip-36199-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:09 +0000 (16:47 -0400)]
Merge pull request #24463 from pdvian/wip-36199-mimic

mimic:mds: fix mds damaged due to unexpected journal length

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24464 from pdvian/wip-36205-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:46:30 +0000 (16:46 -0400)]
Merge pull request #24464 from pdvian/wip-36205-mimic

mimic: libcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24470 from smithfarm/wip-36203-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:45:53 +0000 (16:45 -0400)]
Merge pull request #24470 from smithfarm/wip-36203-mimic

mimic: ceph fs add_data_pool applies pool application metadata incorrectly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #23087 from pdvian/wip-24928-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:44:39 +0000 (16:44 -0400)]
Merge pull request #23087 from pdvian/wip-24928-mimic

mimic: qa: test_recovery_pool tries asok on wrong node

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23506 from ukernel/mimic-26858
Yuri Weinstein [Fri, 19 Oct 2018 20:44:14 +0000 (16:44 -0400)]
Merge pull request #23506 from ukernel/mimic-26858

mimic: mds: reset heartbeat map at potential time-consuming places

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23952 from ukernel/mimic-mds-export-size
Yuri Weinstein [Fri, 19 Oct 2018 20:43:47 +0000 (16:43 -0400)]
Merge pull request #23952 from ukernel/mimic-mds-export-size

mimic: mds: optimize the way how max export size is enforced

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24161 from pdvian/wip-35719-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:24 +0000 (16:43 -0400)]
Merge pull request #24161 from pdvian/wip-35719-mimic

mimic: mds: print is_laggy message once

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>