]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agolvm zap: refuse to zap mapper devices 22965/head
Andrew Schoen [Thu, 26 Jul 2018 19:14:00 +0000 (14:14 -0500)]
lvm zap: refuse to zap mapper devices

Fixes: http://tracker.ceph.com/issues/24504
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 8e602d52657d50888a83963148334deb1f8d4e1c)

7 years agoadds the util.disk.is_mapper_device method
Andrew Schoen [Thu, 28 Jun 2018 17:14:12 +0000 (12:14 -0500)]
adds the util.disk.is_mapper_device method

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 852a466fdfa29c91f266ec122dcd5b01b8d1fca5)

7 years agoMerge pull request #23149 from alfredodeza/mimic-rm24952
Andrew Schoen [Thu, 26 Jul 2018 20:46:54 +0000 (20:46 +0000)]
Merge pull request #23149 from alfredodeza/mimic-rm24952

mimic: ceph-volume lvm.listing only include devices if they exist

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoMerge pull request #23101 from ceph/backport-mimic-23093
Alfredo Deza [Thu, 26 Jul 2018 18:01:00 +0000 (14:01 -0400)]
Merge pull request #23101 from ceph/backport-mimic-23093

mimic: ceph-volume: an OSD ID must exist and be destroyed before reuse

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume lvm.listing only include devices if they exist 23149/head
Alfredo Deza [Wed, 18 Jul 2018 20:50:25 +0000 (16:50 -0400)]
ceph-volume lvm.listing only include devices if they exist

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 37565f81634f47c389926d300ed5f92b223bddea)

7 years agoceph-volume: do not pin the testinfra version for the simple tests
Andrew Schoen [Thu, 26 Jul 2018 17:41:15 +0000 (12:41 -0500)]
ceph-volume: do not pin the testinfra version for the simple tests

For testinfra to work with a 2.6.x version of ansible it must use the
latest version.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 70b86012b56fd4b66fc0c7ce88a6cfbe895a0702)

7 years agoMerge pull request #23269 from ceph/backport-mimic-23263
Alfredo Deza [Thu, 26 Jul 2018 17:49:48 +0000 (13:49 -0400)]
Merge pull request #23269 from ceph/backport-mimic-23263

mimic: ceph-volume: update version of ansible to 2.6.x for simple tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: update version of ansible to 2.6.x for simple tests 23269/head
Andrew Schoen [Thu, 26 Jul 2018 15:09:17 +0000 (10:09 -0500)]
ceph-volume: update version of ansible to 2.6.x for simple tests

ceph-ansible now requires a 2.5.x or 2.6.x version of ansible if you're
using the master branch. This updates our functional tests for the
simple subcommand to use a 2.6.x version of ansible.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 654578928dd840801967c5409b5b19ad6cf90317)

7 years agoMerge pull request #23262 from liewegas/wip-mimic-p2p
Sage Weil [Thu, 26 Jul 2018 16:31:44 +0000 (11:31 -0500)]
Merge pull request #23262 from liewegas/wip-mimic-p2p

mimic: qa/suites/upgrade/mimic-p2p: allow target version to apply

7 years agoqa/suites/upgrade/mimic-p2p: allow target version to apply 23262/head
Sage Weil [Thu, 26 Jul 2018 13:07:54 +0000 (08:07 -0500)]
qa/suites/upgrade/mimic-p2p: allow target version to apply

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge PR #23246 into mimic
Sage Weil [Thu, 26 Jul 2018 13:47:28 +0000 (08:47 -0500)]
Merge PR #23246 into mimic

* refs/pull/23246/head:
rpm,deb: fix chown to use -h
rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agogithubmap: update contributors
Sage Weil [Thu, 26 Jul 2018 13:47:28 +0000 (08:47 -0500)]
githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge PR #23250 into mimic
Sage Weil [Thu, 26 Jul 2018 13:47:17 +0000 (08:47 -0500)]
Merge PR #23250 into mimic

* refs/pull/23250/head:
osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agoosd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic 23250/head
Sage Weil [Wed, 25 Jul 2018 22:12:43 +0000 (17:12 -0500)]
osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

This was delayed and did not appear in 13.2.0.

Fixes: 1fdc85f224d52655d74eff9a29b6029b974661b3
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 37781061022a92e88dff91ff18fd0fc14b5436de)

7 years agorpm,deb: fix chown to use -h 23246/head
Sage Weil [Fri, 13 Jul 2018 22:10:38 +0000 (17:10 -0500)]
rpm,deb: fix chown to use -h

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 92207c76b91c26c0558688a358ade7c0d94d1b59)

7 years agorpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade
Sage Weil [Fri, 13 Jul 2018 14:56:48 +0000 (09:56 -0500)]
rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade

This works around https://tracker.ceph.com/issues/24903

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 58cde3fd793c037beca6645cc30b2ae1c30e2af1)

7 years agoMerge pull request #23244 from alfredodeza/mimic-ansible-26
Andrew Schoen [Wed, 25 Jul 2018 20:58:16 +0000 (20:58 +0000)]
Merge pull request #23244 from alfredodeza/mimic-ansible-26

mimic ceph-volume tests/functional use Ansible 2.6

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests/functional use testinfra latest for ansible 2.6 23244/head
Alfredo Deza [Mon, 23 Jul 2018 18:16:41 +0000 (14:16 -0400)]
ceph-volume tests/functional use testinfra latest for ansible 2.6

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4f48ceab975539c9ce989aade7a1f3ae7d5b51d5)

7 years agoceph-volume tests/functional use Ansible 2.6
Alfredo Deza [Mon, 23 Jul 2018 14:18:34 +0000 (10:18 -0400)]
ceph-volume tests/functional use Ansible 2.6

The ceph-ansible project now refuses to use 2.4 even though it works
with that version

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9570864dd614d2442278e2ba3501788fefd753e3)

7 years agoMerge PR #23233 into mimic
Sage Weil [Wed, 25 Jul 2018 16:31:38 +0000 (11:31 -0500)]
Merge PR #23233 into mimic

* refs/pull/23233/head:
mon: don't require CEPHX_V2 from mons until nautilus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agomon: don't require CEPHX_V2 from mons until nautilus 23233/head
Sage Weil [Wed, 25 Jul 2018 16:24:27 +0000 (11:24 -0500)]
mon: don't require CEPHX_V2 from mons until nautilus

The mimic 13.2.0 didn't have it.

Fixes: 3dc80e5f9b6ebf1bc1cecbd95b288005216bdbec
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7ba0c14ce75f6936b59a3c61ea34a6f0fb909280)

# Conflicts:
# src/mon/Monitor.cc

- dropped the nautilus cases in the original commit

7 years agoMerge pull request #23209 from smithfarm/wip-25081-mimic
Nathan Cutler [Wed, 25 Jul 2018 07:24:14 +0000 (09:24 +0200)]
Merge pull request #23209 from smithfarm/wip-25081-mimic

mimic: qa: krbd_exclusive_option.sh: bump lock_timeout to 60 seconds

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #23136 from smithfarm/wip-25010-mimic
Yuri Weinstein [Tue, 24 Jul 2018 19:47:39 +0000 (12:47 -0700)]
Merge pull request #23136 from smithfarm/wip-25010-mimic

mimic: tests: osd sends op_reply out of order

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoqa: krbd_exclusive_option.sh: bump lock_timeout to 60 seconds 23209/head
Ilya Dryomov [Wed, 20 Jun 2018 16:22:36 +0000 (18:22 +0200)]
qa: krbd_exclusive_option.sh: bump lock_timeout to 60 seconds

Avoid sporadic failures in combination with msgr-failures/many.yaml,
where assert_locked() might take over 10 seconds.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 69619b69663e026aa4e9c7c21c1fc879782c26e9)

7 years agoMerge pull request #23178 from smithfarm/wip-25065-mimic
Nathan Cutler [Tue, 24 Jul 2018 15:48:41 +0000 (17:48 +0200)]
Merge pull request #23178 from smithfarm/wip-25065-mimic

mimic: build/ops: install-deps.sh fails on newest openSUSE Leap

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #23176 from smithfarm/wip-25062-mimic
Nathan Cutler [Tue, 24 Jul 2018 15:47:51 +0000 (17:47 +0200)]
Merge pull request #23176 from smithfarm/wip-25062-mimic

mimic: doc: ceph-bluestore-tool manpage not getting rendered correctly

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
7 years agoMerge pull request #23154 from smithfarm/wip-25034-mimic
Yuri Weinstein [Tue, 24 Jul 2018 15:15:29 +0000 (08:15 -0700)]
Merge pull request #23154 from smithfarm/wip-25034-mimic

mimic: tests: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) in powercycle

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #23156 from yuriw/wip-yuriw-25005-mimic
Yuri Weinstein [Tue, 24 Jul 2018 15:14:33 +0000 (08:14 -0700)]
Merge pull request #23156 from yuriw/wip-yuriw-25005-mimic

mimic: qa/tests -  added overrides stanza to allow runs on ovh on rhel OS

7 years agoMerge PR #23184 into mimic
Sage Weil [Tue, 24 Jul 2018 14:05:24 +0000 (09:05 -0500)]
Merge PR #23184 into mimic

* refs/pull/23184/head:
common: add osd_distrust_data_digest option
osd: apply osd_skip_data_digest to read operations

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #23164 from smithfarm/wip-25056-mimic
Sage Weil [Tue, 24 Jul 2018 02:43:30 +0000 (21:43 -0500)]
Merge pull request #23164 from smithfarm/wip-25056-mimic

mimic: tests: upgrade/luminous-x: whitelist REQUEST_SLOW for rados_mon_thrash

7 years agocommon: add osd_distrust_data_digest option 23184/head
Sage Weil [Mon, 23 Jul 2018 15:11:53 +0000 (10:11 -0500)]
common: add osd_distrust_data_digest option

If we have reason to distrust the stored full-object oi data digests,
this option will ignore them (and opportunistically clear them).  It
basically affects the same behaviors that osd_skip_data_digest except
the previous option only takes effect if bluestore is in use, while this
option is unconditional.

This serves a workaround for clusters that suffered the 12.2.6 bug that
produced bad full-object digests and *also* had PGs' primaries move from
bluestore to filestore nodes.  In those situations, this option can be
set to true.

After all OSDs are upgrade and all PGs come back with a clean deep
scrub this option can be disabled again.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b8476ebc5b3c5b5d801bf875bbd6dce5b113325c)

# Conflicts:
# src/osd/PrimaryLogPG.cc

- osd->osd_skip_data_digest, not g_conf->osd_skip_data_digest

7 years agoosd: apply osd_skip_data_digest to read operations
Sage Weil [Mon, 23 Jul 2018 15:07:55 +0000 (10:07 -0500)]
osd: apply osd_skip_data_digest to read operations

If we would skip updating data digests, apply that same decision to read
operations as well.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 49207a6cb388446361aafacfaa863d66dd452ded)

# Conflicts:
# src/osd/PrimaryLogPG.cc

- one of the do_read() cases moved
- osd->osd_skip_data_digest (observed), not g_conf->...

7 years agoMerge pull request #23122 from yuriw/wip-yuriw-24926-mimic
Nathan Cutler [Mon, 23 Jul 2018 14:10:51 +0000 (16:10 +0200)]
Merge pull request #23122 from yuriw/wip-yuriw-24926-mimic

mimic - qa/tests: Set ansible-version: 2.4

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoAdd new OpenSUSE Leap id for install-deps.sh 23178/head
Kyr Shatskyy [Mon, 2 Jul 2018 13:23:58 +0000 (15:23 +0200)]
Add new OpenSUSE Leap id for install-deps.sh

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
(cherry picked from commit bd0448c0b5a09a9314fddf1757502529801366e3)

7 years agodoc: use :command: for subcommands in ceph-bluestore-tool manpage 23176/head
Nathan Cutler [Wed, 18 Jul 2018 12:01:42 +0000 (14:01 +0200)]
doc: use :command: for subcommands in ceph-bluestore-tool manpage

Older versions of Sphinx, such as the one in CentOS 7, do not render "..
option::" lines correctly if the option contains a hyphen but does not start
with a hyphen. And ceph-bluestore-tool appears to be the only Ceph manpage
affected by this bug.

Fixes: http://tracker.ceph.com/issues/24800
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7c9229e7876dfc1bf6e415ed7cce924c3b50925d)

7 years agotests: upgrade/luminous-x: whitelist REQUEST_SLOW for rados_mon_thrash 23164/head
Nathan Cutler [Sat, 21 Jul 2018 09:22:35 +0000 (11:22 +0200)]
tests: upgrade/luminous-x: whitelist REQUEST_SLOW for rados_mon_thrash

Fixes: http://tracker.ceph.com/issues/25051
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 2555524f1fcf230f5468e1a5fd20f9fb741e03aa)

7 years agoqa/tests - added overrides stanza to allow runs on ovh on rhel OS 23156/head
Yuri Weinstein [Sat, 21 Jul 2018 02:57:05 +0000 (19:57 -0700)]
qa/tests -  added overrides stanza to allow runs on ovh on rhel OS

Fixes: https://tracker.ceph.com/issues/25005
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoqa/suites/powercycle: whitelist MDS_SLOW_REQUEST 23154/head
Neha Ojha [Fri, 20 Jul 2018 19:50:02 +0000 (12:50 -0700)]
qa/suites/powercycle: whitelist MDS_SLOW_REQUEST

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ac87fd946c8806f6f84d6c6ea124b4731b7b8294)

7 years agoqa/tests: Set ansible-version: 2.4 23122/head
Yuri Weinstein [Wed, 18 Jul 2018 23:47:42 +0000 (16:47 -0700)]
qa/tests: Set ansible-version: 2.4

Fixes http://tracker.ceph.com/issues/24926
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #23148 from alfredodeza/mimic-rm24961
Andrew Schoen [Fri, 20 Jul 2018 19:24:33 +0000 (19:24 +0000)]
Merge pull request #23148 from alfredodeza/mimic-rm24961

ceph-volume tests/functional run lvm list after OSD provisioning

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests/functional run lvm list after OSD provisioning 23148/head
Alfredo Deza [Wed, 18 Jul 2018 16:39:01 +0000 (12:39 -0400)]
ceph-volume tests/functional run lvm list after OSD provisioning

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e01fb206d67f26724f3b03e4b22f5ba6eea22207)

7 years agoMerge pull request #23127 from ceph/backport-mimic-23117
Alfredo Deza [Fri, 20 Jul 2018 18:14:30 +0000 (14:14 -0400)]
Merge pull request #23127 from ceph/backport-mimic-23117

mimic: ceph-volume: unmount lvs correctly before zapping

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoqa/suites: set osd_pg_log_dups_tracked in cfuse_workunit_suites_fsync.yaml 23136/head
Neha Ojha [Tue, 1 May 2018 23:20:14 +0000 (16:20 -0700)]
qa/suites: set osd_pg_log_dups_tracked in cfuse_workunit_suites_fsync.yaml

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 94a2d810826d710c3c71403d6a78249cc66a39e7)

7 years agoceph-volume: unmount lvs correctly before zapping 23127/head
Andrew Schoen [Wed, 18 Jul 2018 16:08:15 +0000 (11:08 -0500)]
ceph-volume: unmount lvs correctly before zapping

If the lv found does not include the meatadata to construct an osd path,
then we should try to unmount it's path instead.

Fixes: http://tracker.ceph.com/issues/24796
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 5b175a5bb68264b80c00cb4f04026a4d19ca20ad)

7 years agoMerge pull request #22845 from smithfarm/wip-24736-mimic
Yuri Weinstein [Wed, 18 Jul 2018 15:18:52 +0000 (08:18 -0700)]
Merge pull request #22845 from smithfarm/wip-24736-mimic

mimic: add unit test for cls bi list command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #21523 from smithfarm/wip-18163
Yuri Weinstein [Wed, 18 Jul 2018 14:35:16 +0000 (07:35 -0700)]
Merge pull request #21523 from smithfarm/wip-18163

mimic: ceph-detect-init: stop using platform.linux_distribution

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22684 from pdvian/wip-24472-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:32:14 +0000 (07:32 -0700)]
Merge pull request #22684 from pdvian/wip-24472-mimic

mimic: spdk: fix ceph-osd crash when activate SPDK

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22948 from tchaikov/mimic-pr-22871
Yuri Weinstein [Wed, 18 Jul 2018 14:31:34 +0000 (07:31 -0700)]
Merge pull request #22948 from tchaikov/mimic-pr-22871

mimic: deb/rpm: add python-six as build-time and run-time dependency

Reviewed-by:  Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22988 from pdvian/wip-24825-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:28:01 +0000 (07:28 -0700)]
Merge pull request #22988 from pdvian/wip-24825-mimic

mimic: tests: make test_ceph_argparse.py pass on py3-only systems

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #23017 from ceph/wip-cd-fix-supp-distro
Yuri Weinstein [Wed, 18 Jul 2018 14:26:50 +0000 (07:26 -0700)]
Merge pull request #23017 from ceph/wip-cd-fix-supp-distro

mimic: qa/tests: fix supported distro lists for ceph-deploy

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #23030 from smithfarm/wip-24904-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:25:47 +0000 (07:25 -0700)]
Merge pull request #23030 from smithfarm/wip-24904-mimic

mimic: qa: client socket inaccessible without sudo

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #23049 from zeichenanonym/mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:25:14 +0000 (07:25 -0700)]
Merge pull request #23049 from zeichenanonym/mimic

mimic: include: fix build with older clang (OSX target)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
7 years agoMerge pull request #23084 from dzafman/wip-disable-skip-dd-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:23:55 +0000 (07:23 -0700)]
Merge pull request #23084 from dzafman/wip-disable-skip-dd-mimic

mimic: osd: Change osd_skip_data_digest default to false and make it LEVEL_DEV

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #23094 from theanalyst/wip-mimic-24958
Yuri Weinstein [Wed, 18 Jul 2018 14:23:19 +0000 (07:23 -0700)]
Merge pull request #23094 from theanalyst/wip-mimic-24958

mimic: osd: fix deep scrub with osd_skip_data_digest=true (default) and blue…

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #23096 from cbodley/wip-mimic-qa-rgw-no-ec-cache
Yuri Weinstein [Wed, 18 Jul 2018 14:22:09 +0000 (07:22 -0700)]
Merge pull request #23096 from cbodley/wip-mimic-qa-rgw-no-ec-cache

mimic: qa/rgw: disable testing on ec-cache pools

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #23024 from smithfarm/wip-24865-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:20:17 +0000 (07:20 -0700)]
Merge pull request #23024 from smithfarm/wip-24865-mimic

mimic: common: Abort in OSDMap::decode() during qa/standalone/erasure-code/test-erasure-eio.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #23065 from pdvian/wip-24887-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:17:28 +0000 (07:17 -0700)]
Merge pull request #23065 from pdvian/wip-24887-mimic

mimic: os/bluestore: fix races on SharedBlob::coll in ~SharedBlob.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
7 years agoMerge pull request #23070 from smithfarm/wip-24942-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:16:41 +0000 (07:16 -0700)]
Merge pull request #23070 from smithfarm/wip-24942-mimic

mimic: build/ops: ceph.git has two different versions of dpdk in the source tree

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22812 from smithfarm/wip-24705-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:13:01 +0000 (13:13 -0700)]
Merge pull request #22812 from smithfarm/wip-24705-mimic

mimic: cephfs: allow prohibiting user snapshots in CephFS

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22835 from batrick/i24758
Yuri Weinstein [Tue, 17 Jul 2018 20:11:17 +0000 (13:11 -0700)]
Merge pull request #22835 from batrick/i24758

mimic: test gets ENOSPC from bluestore block device

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22869 from pdvian/wip-24583-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:10:36 +0000 (13:10 -0700)]
Merge pull request #22869 from pdvian/wip-24583-mimic

mimic: osdc: Fix the wrong BufferHead offset

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22956 from joscollin/wip-24829-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:09:27 +0000 (13:09 -0700)]
Merge pull request #22956 from joscollin/wip-24829-mimic

mimic: qa: fix ffsb cd argument

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22981 from joscollin/wip-24861-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:08:57 +0000 (13:08 -0700)]
Merge pull request #22981 from joscollin/wip-24861-mimic

mimic: cephfs-journal-tool: Fix purging when importing an zero-length journal.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoceph-volume: an osd ID must exist and be destroyed before reuse 23101/head
Andrew Schoen [Mon, 16 Jul 2018 21:06:03 +0000 (16:06 -0500)]
ceph-volume: an osd ID must exist and be destroyed before reuse

Fixes: https://tracker.ceph.com/issues/24044
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 02a13f2de0783ff3c680a6042388a4692250edfb)

7 years agoceph-volume util/prepare.py
Ron Allred [Wed, 23 May 2018 14:34:41 +0000 (08:34 -0600)]
ceph-volume util/prepare.py

Replaces PR#21887
Rename function check_id to osd_id_exists in util/prepare.py
Fix boolean analysis of osd_id_exists when --osd-id is provided on command line

Signed-off-by: Ron Allred <rallred@itrefined.com>
(cherry picked from commit 4641ebb075adc987e83d7440f9756104fc04f83a)

7 years agoMerge pull request #22933 from smithfarm/wip-24813-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:37:16 +0000 (09:37 -0700)]
Merge pull request #22933 from smithfarm/wip-24813-mimic

mimic: rgw: REST admin metadata API paging failure bucket & bucket.instance: InvalidArgument

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22466 from pdvian/wip-24313-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:36:44 +0000 (09:36 -0700)]
Merge pull request #22466 from pdvian/wip-24313-mimic

mimic: test/rgw: fix for bucket checkpoints

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22523 from smithfarm/wip-24253-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:36:10 +0000 (09:36 -0700)]
Merge pull request #22523 from smithfarm/wip-24253-mimic

mimic: rgw: Admin OPS Api overwrites email when user is modified

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #22590 from pdvian/wip-24352-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:35:33 +0000 (09:35 -0700)]
Merge pull request #22590 from pdvian/wip-24352-mimic

mimic: rgw: add configurable AWS-compat invalid range get behavior

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
7 years agoMerge pull request #22643 from pdvian/wip-24385-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:34:15 +0000 (09:34 -0700)]
Merge pull request #22643 from pdvian/wip-24385-mimic

mimic: objects in cache never refresh after rgw_cache_expiry_interval

Reviewed-by: Adam Emerson <aemerson@redhat.com>
7 years agoMerge pull request #22815 from smithfarm/wip-24700-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:33:30 +0000 (09:33 -0700)]
Merge pull request #22815 from smithfarm/wip-24700-mimic

mimic: rgw-multisite: endless loop in RGWBucketShardIncrementalSyncCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22818 from smithfarm/wip-24701-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:33:07 +0000 (09:33 -0700)]
Merge pull request #22818 from smithfarm/wip-24701-mimic

mimic: rgw: index complete miss zones_trace set

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22821 from smithfarm/wip-24702-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:32:39 +0000 (09:32 -0700)]
Merge pull request #22821 from smithfarm/wip-24702-mimic

mimic: rgw: meta and data notify thread miss stop cr manager

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22880 from smithfarm/wip-24783-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:32:08 +0000 (09:32 -0700)]
Merge pull request #22880 from smithfarm/wip-24783-mimic

mimic: rgw: set cr state if aio_read err return in RGWCloneMetaLogCoroutine::state_send_rest_request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22926 from smithfarm/wip-24619-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:29:06 +0000 (09:29 -0700)]
Merge pull request #22926 from smithfarm/wip-24619-mimic

mimic: multisite: RGWSyncTraceNode released twice and crashed in reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22927 from smithfarm/wip-24631-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:26:15 +0000 (09:26 -0700)]
Merge pull request #22927 from smithfarm/wip-24631-mimic

mimic: cls_bucket_list fails causes cascading osd crashes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22929 from smithfarm/wip-24633-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:25:49 +0000 (09:25 -0700)]
Merge pull request #22929 from smithfarm/wip-24633-mimic

mimic: rgw performance regression for luminous 12.2.4

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22935 from smithfarm/wip-24809-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:25:05 +0000 (09:25 -0700)]
Merge pull request #22935 from smithfarm/wip-24809-mimic

mimic: rgw: Invalid Access-Control-Request-Request may bypass validate_cors_rule_method

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #22941 from joscollin/wip-24807-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:24:32 +0000 (09:24 -0700)]
Merge pull request #22941 from joscollin/wip-24807-mimic

mimic: rgw: fix gc may cause a large number of read traffic

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #23001 from pdvian/wip-24831-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:00:57 +0000 (09:00 -0700)]
Merge pull request #23001 from pdvian/wip-24831-mimic

mimic: rgw: "radosgw-admin objects expire" always returns ok even if the pro…

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #23021 from pdvian/wip-24834-mimic
Yuri Weinstein [Tue, 17 Jul 2018 15:59:20 +0000 (08:59 -0700)]
Merge pull request #23021 from pdvian/wip-24834-mimic

mimic: rgw: radogw-admin reshard status command should print text for reshar…

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoceph-detect-init: stop using platform.linux_distribution 21523/head
Nathan Cutler [Thu, 19 Apr 2018 11:21:31 +0000 (13:21 +0200)]
ceph-detect-init: stop using platform.linux_distribution

This commit introduces a new function, _extract_from_os_release, which is
used (only on Linux) to extract the ID and VERSION_ID fields from the
/etc/os-release file. This is sufficient to positively determine the OS
and version on these systems, and enable ceph-detect-init to correctly
return systemd on them.

The commit includes unit tests asserting that _extract_from_os_release really
extracts the strings that we expect it to, using the contents of
/etc/os-release from a number of different operating systems targeted by Ceph.

This commit is not cherry-picked because ceph-detect-init has been dropped in
master.

Fixes: http://tracker.ceph.com/issues/18163
Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoqa/rgw: disable testing on ec-cache pools 23096/head
Casey Bodley [Mon, 21 May 2018 17:28:57 +0000 (13:28 -0400)]
qa/rgw: disable testing on ec-cache pools

Fixes: http://tracker.ceph.com/issues/23965
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7cb50b495bb878483f9b9f77b137b6b831c96ccc)

7 years agoosd: fix deep scrub with osd_skip_data_digest=true (default) and bluestore 23094/head
Sage Weil [Sat, 14 Jul 2018 17:39:15 +0000 (12:39 -0500)]
osd: fix deep scrub with osd_skip_data_digest=true (default) and bluestore

The osd_skip_data_digest is about skipping the storing and checking of a
full-object data digest.  Here it is also disabling crc checks during
deep-scrub, which prevents us from detecting crc differences across
replicas.  Fix!

Fixes: http://tracker.ceph.com/issues/24922
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 101e88aa70ac56100a631a2da503b8b0ecb4f625)

Conflicts:
src/osd/ECBackend.cc
  drop the non existant skip_data_digest in L
  drop the non existant skip_data_digest in L

7 years agocommon: Fix description for osd_skip_data_digest 23084/head
David Zafman [Mon, 16 Jul 2018 21:32:54 +0000 (14:32 -0700)]
common: Fix description for osd_skip_data_digest

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3b7f502c85593a1c69b49cb6022c7a57e37d1254)

7 years agocommon/options: disable osd_skip_data_digest
Sage Weil [Sat, 14 Jul 2018 20:03:43 +0000 (15:03 -0500)]
common/options: disable osd_skip_data_digest

This is flawed in that it stops doing digests based on the primary's
backend store, not the replicas.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c7d8add701f156eb8bd8c7dc8ae9903bd782e77f)

7 years agocommon/options: make osd_skip_data_digest a DEV option
Sage Weil [Sat, 14 Jul 2018 18:26:31 +0000 (13:26 -0500)]
common/options: make osd_skip_data_digest a DEV option

And document/warn to never turn it off after it's been on.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c854ed9b33a5a293c8ace15718e26f74237e3332)

7 years agoMerge pull request #22222 from pdvian/wip-24203-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:23:03 +0000 (08:23 -0700)]
Merge pull request #22222 from pdvian/wip-24203-mimic

mimic: rbd: Prevent the use of internal feature bits from outside cls/rbd

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22662 from pdvian/wip-24388-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:22:31 +0000 (08:22 -0700)]
Merge pull request #22662 from pdvian/wip-24388-mimic

mimic: librbd: utilize the journal disabled policy when removing images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22667 from pdvian/wip-24390-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:21:56 +0000 (08:21 -0700)]
Merge pull request #22667 from pdvian/wip-24390-mimic

mimic: rbd: rbd-mirror daemon failed to stop on active/passive test case

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22754 from pdvian/wip-24496-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:21:26 +0000 (08:21 -0700)]
Merge pull request #22754 from pdvian/wip-24496-mimic

mimic: librbd: force 'invalid object map' flag on-disk update

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22843 from smithfarm/wip-24734-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:20:55 +0000 (08:20 -0700)]
Merge pull request #22843 from smithfarm/wip-24734-mimic

mimic: build/ops: order rbdmap.service before remote-fs-pre.target

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #22891 from smithfarm/wip-24476-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:19:59 +0000 (08:19 -0700)]
Merge pull request #22891 from smithfarm/wip-24476-mimic

mimic: rbd: rbd trash purge --threshold should support data pool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22892 from smithfarm/wip-24519-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:19:29 +0000 (08:19 -0700)]
Merge pull request #22892 from smithfarm/wip-24519-mimic

mimic: rbd: rbd-mirror simple image map policy doesn't always level-load instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22894 from smithfarm/wip-24596-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:18:58 +0000 (08:18 -0700)]
Merge pull request #22894 from smithfarm/wip-24596-mimic

mimic: rbd: yet another case when deep copying a clone may result in invalid object map

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22943 from pdvian/wip-24698-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:17:52 +0000 (08:17 -0700)]
Merge pull request #22943 from pdvian/wip-24698-mimic

mimic: librbd: fix crash when opening nonexistent snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agodpdk: drop dpdk submodule 23070/head
Kefu Chai [Mon, 7 May 2018 14:31:51 +0000 (22:31 +0800)]
dpdk: drop dpdk submodule

we are using spdk/dpdk for async msgr's dpdk backend since 01a9f178. so
no need to keep dpdk submodule anymore.

Fixes: http://tracker.ceph.com/issues/24032
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb8087dfac31b8490fefdfca28d389b7b9901ef8)

7 years agoos/bluestore: fix races on SharedBlob::coll in ~SharedBlob. 23065/head
Radoslaw Zarzynski [Tue, 10 Jul 2018 23:14:54 +0000 (01:14 +0200)]
os/bluestore: fix races on SharedBlob::coll in ~SharedBlob.

Example scenario:
  ```
  A: BlueStore::Collection::split_cache(src, dest)
  A: std::lock(src->cache->lock, dest->cache->lock)

  B: SharedBlob::~SharedBlob
  B: waits coll->cache->lock // coll == src

  A: sb->coll := dest
  A: unlocks both src and dest's cache mutexes

  C: // any locked operation on dest's cache
  C: acquires dest->cache->lock
  C: begins the op

  B: // with the src's cache mutex acquired
  B: BufferSpace::_clear(coll->cache) // coll == dest
  B: // oops, B operates on cache already locked by C
  ```

Fixes: http://tracker.ceph.com/issues/24859
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit d1b16749c7d32eead6e326c492ef9edf0d03a2b8)

7 years agoinclude: fix build with older clang 23049/head
Christopher Blum [Sat, 14 Jul 2018 15:01:59 +0000 (17:01 +0200)]
include: fix build with older clang

to fix the FTBFS on clang shipped with xcode 9.1 (based on Clang 4.0).
This fixes ceph client library builds for the OSX target
initally targetted with PR #23034.

refs: #23034

Signed-off-by: Christopher Blum <zeichenanonym@web.de>
7 years agoMerge pull request #23034 from tchaikov/mimic-clang-4
Kefu Chai [Sat, 14 Jul 2018 12:50:49 +0000 (20:50 +0800)]
Merge pull request #23034 from tchaikov/mimic-clang-4

include: fix build with older clang

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>