]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoceph-volume tests validate the new utils for prompts
Alfredo Deza [Mon, 16 Jul 2018 13:23:14 +0000 (09:23 -0400)]
ceph-volume tests validate the new utils for prompts

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2d988337a96fe6cc8fffb37e4641c86cd47e1139)

7 years agoceph-volume tests verify str_to_int utility
Alfredo Deza [Fri, 13 Jul 2018 12:55:46 +0000 (08:55 -0400)]
ceph-volume tests verify str_to_int utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 56b1bf633d3d16cf335043450df95f191e2907a4)

7 years agoceph-volume util create prompt helpers
Alfredo Deza [Mon, 16 Jul 2018 13:22:50 +0000 (09:22 -0400)]
ceph-volume util create prompt helpers

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0283fcc38b13c2d090b1d25be732e6f5f43b834c)

7 years agoceph-volume util move the str_to_int utility out of lvm
Alfredo Deza [Fri, 13 Jul 2018 12:54:33 +0000 (08:54 -0400)]
ceph-volume util move the str_to_int utility out of lvm

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit cb5b30ab43f37adc45fb6114452b37722c65501e)

7 years agoceph-volume tests run to log_descriptors, verifying utf-8
Alfredo Deza [Wed, 25 Jul 2018 12:40:53 +0000 (08:40 -0400)]
ceph-volume tests run to log_descriptors, verifying utf-8

Signed-off-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume process decode to utf-8 when logging descriptors
Alfredo Deza [Wed, 25 Jul 2018 12:39:16 +0000 (08:39 -0400)]
ceph-volume process decode to utf-8 when logging descriptors

Signed-off-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume tests.process ensure that stdin works on py3 and py2.7
Alfredo Deza [Fri, 20 Jul 2018 13:49:59 +0000 (09:49 -0400)]
ceph-volume tests.process ensure that stdin works on py3 and py2.7

Signed-off-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume process encode stdin with utf-8 to prevent bytes TypeError
Alfredo Deza [Fri, 20 Jul 2018 13:48:38 +0000 (09:48 -0400)]
ceph-volume process encode stdin with utf-8 to prevent bytes TypeError

Signed-off-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #23274 from yuriw/wip-yuriw-25123-luminous
Nathan Cutler [Fri, 27 Jul 2018 04:18:54 +0000 (06:18 +0200)]
Merge pull request #23274 from yuriw/wip-yuriw-25123-luminous

qa/upgrade/luminous-p2p: cleaned up "#branch: luminous" lines

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #23179 from smithfarm/wip-25066-luminous
Nathan Cutler [Fri, 27 Jul 2018 03:46:53 +0000 (05:46 +0200)]
Merge pull request #23179 from smithfarm/wip-25066-luminous

luminous: build/ops: install-deps.sh fails on newest openSUSE Leap

Reviewed-by: Tim Serong <tserong@suse.com>
7 years agoqa/tests: cleaned up "#branch: luminous" lines 23274/head
Yuri Weinstein [Thu, 26 Jul 2018 22:13:38 +0000 (15:13 -0700)]
qa/tests: cleaned up "#branch: luminous" lines

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #23265 from yuriw/wip-yuriw-25123-luminous
Yuri Weinstein [Thu, 26 Jul 2018 22:12:26 +0000 (15:12 -0700)]
Merge pull request #23265 from yuriw/wip-yuriw-25123-luminous

qa/tests: added 12.2.7 to the mix

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #23150 from alfredodeza/luminous-rm24952
Andrew Schoen [Thu, 26 Jul 2018 20:48:09 +0000 (20:48 +0000)]
Merge pull request #23150 from alfredodeza/luminous-rm24952

luminous: ceph-volume lvm.listing only include devices if they exist

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoqa/tests: added 12.2.7 to the mix 23265/head
Yuri Weinstein [Thu, 26 Jul 2018 15:57:09 +0000 (08:57 -0700)]
qa/tests: added 12.2.7 to the mix
also added 'overall HEALTH_' to the whitelist

Fixes: https://tracker.ceph.com/issues/25123
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #23177 from smithfarm/wip-25063-luminous
Nathan Cutler [Thu, 26 Jul 2018 20:05:37 +0000 (22:05 +0200)]
Merge pull request #23177 from smithfarm/wip-25063-luminous

luminous: doc: ceph-bluestore-tool manpage not getting rendered correctly

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
7 years agoceph-volume lvm.listing only include devices if they exist 23150/head
Alfredo Deza [Wed, 18 Jul 2018 20:50:25 +0000 (16:50 -0400)]
ceph-volume lvm.listing only include devices if they exist

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 37565f81634f47c389926d300ed5f92b223bddea)

7 years agoceph-volume: do not pin the testinfra version for the simple tests
Andrew Schoen [Thu, 26 Jul 2018 17:41:15 +0000 (12:41 -0500)]
ceph-volume: do not pin the testinfra version for the simple tests

For testinfra to work with a 2.6.x version of ansible it must use the
latest version.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 70b86012b56fd4b66fc0c7ce88a6cfbe895a0702)

7 years agoMerge pull request #23264 from ceph/backport-luminous-23263
Alfredo Deza [Thu, 26 Jul 2018 17:44:32 +0000 (13:44 -0400)]
Merge pull request #23264 from ceph/backport-luminous-23263

luminous: ceph-volume: update version of ansible to 2.6.x for simple tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: update version of ansible to 2.6.x for simple tests 23264/head
Andrew Schoen [Thu, 26 Jul 2018 15:09:17 +0000 (10:09 -0500)]
ceph-volume: update version of ansible to 2.6.x for simple tests

ceph-ansible now requires a 2.5.x or 2.6.x version of ansible if you're
using the master branch. This updates our functional tests for the
simple subcommand to use a 2.6.x version of ansible.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 654578928dd840801967c5409b5b19ad6cf90317)

7 years agoMerge pull request #23245 from alfredodeza/luminous-ansible-26
Andrew Schoen [Wed, 25 Jul 2018 21:03:02 +0000 (21:03 +0000)]
Merge pull request #23245 from alfredodeza/luminous-ansible-26

luminous ceph-volume tests/functional use Ansible 2.6

Reviewed-by: Andrew Schoen <aschoen@redhat.com
7 years agoceph-volume tests/functional use testinfra latest for ansible 2.6 23245/head
Alfredo Deza [Mon, 23 Jul 2018 18:16:41 +0000 (14:16 -0400)]
ceph-volume tests/functional use testinfra latest for ansible 2.6

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4f48ceab975539c9ce989aade7a1f3ae7d5b51d5)

7 years agoceph-volume tests/functional use Ansible 2.6
Alfredo Deza [Mon, 23 Jul 2018 14:18:34 +0000 (10:18 -0400)]
ceph-volume tests/functional use Ansible 2.6

The ceph-ansible project now refuses to use 2.4 even though it works
with that version

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9570864dd614d2442278e2ba3501788fefd753e3)

7 years agoMerge pull request #23235 from rhcs-dashboard/wip-25102-luminous
Ernesto Puerta [Wed, 25 Jul 2018 18:33:21 +0000 (20:33 +0200)]
Merge pull request #23235 from rhcs-dashboard/wip-25102-luminous

pybind/mgr/mgr_module: make rados handle available to all modules

7 years agopybind/mgr/mgr_module: make rados handle available to all modules 23235/head
Sage Weil [Tue, 16 Jan 2018 21:02:14 +0000 (15:02 -0600)]
pybind/mgr/mgr_module: make rados handle available to all modules

Fixes: http://tracker.ceph.com/issues/25102
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 366905fb5779ecb110ec4216afe7cde14fbbb2d8)
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
7 years agoMerge pull request #23227 from liewegas/wip-25057-luminous
Sage Weil [Wed, 25 Jul 2018 12:07:58 +0000 (07:07 -0500)]
Merge pull request #23227 from liewegas/wip-25057-luminous

luminous: osd/OSDMap: CRUSH_TUNABLES5 added in jewel, not kraken

7 years agoosd/OSDMap: CRUSH_TUNABLES5 added in jewel, not kraken 23227/head
Sage Weil [Wed, 25 Jul 2018 01:30:54 +0000 (20:30 -0500)]
osd/OSDMap: CRUSH_TUNABLES5 added in jewel, not kraken

In 0ceb5c0f3694725c7043ea99b077816da2ca44a7 I screwed up the version that
CRUSH_TUANBLES5 (chooseleaf_stable) was added: it comes from
043a7378ec4e828f2568ea3906d9076848161ccd, v10.0.3, which leads up to
jewel.

This, in turn, means that when jewel tunables are set we do not actually
encode the jewel tunables.  At least, luminous doesn't; jewel uses a
different code path to decode how to encode the OSDMap.

This, in turn, leads to crc errors:

- luminous mons running
- require_osd_release is jewel
- jewel tunables are set.  mon in-memory crushmap gets chooseleaf_stable=1
- osds are luminous, but don't see the new field in the crushmap
- upgrade finishes, and require_osd_Release becomes luminous
- the full osdmap encodes the chooseleaf_stable field, finally, and it
  is 1
- the osds try to encode the same thing, but their crushmap has 0.
-> crc mismatch

This is a regression from 0ceb5c0f3694725c7043ea99b077816da2ca44a7,
backported to luminous in 686b054f48c50859bd5e8bad93007bf9d7d7899c in
12.2.6.

Fixes: 686b054f48c50859bd5e8bad93007bf9d7d7899c
Fixes: http://tracker.ceph.com/issues/25057
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7e9d24d0602abefa4cca245f5f86dcd4b412b97a)

7 years agoMerge pull request #23219 from liewegas/wip-slow-requests-upgrade-luminous
Sage Weil [Tue, 24 Jul 2018 22:55:55 +0000 (17:55 -0500)]
Merge pull request #23219 from liewegas/wip-slow-requests-upgrade-luminous

luminous: qa/suites/upgrade/jewel-x: whitelist 'slow requests'

7 years agoqa/suites/upgrade/jewel-x: whitelist 'slow requests' 23219/head
Sage Weil [Tue, 24 Jul 2018 22:54:27 +0000 (17:54 -0500)]
qa/suites/upgrade/jewel-x: whitelist 'slow requests'

This message is generated by jewel but not luminous.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoAdd new OpenSUSE Leap id for install-deps.sh 23179/head
Kyr Shatskyy [Mon, 2 Jul 2018 13:23:58 +0000 (15:23 +0200)]
Add new OpenSUSE Leap id for install-deps.sh

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
(cherry picked from commit bd0448c0b5a09a9314fddf1757502529801366e3)

Conflicts:
install-deps.sh (trivial resolution)

7 years agodoc: use :command: for subcommands in ceph-bluestore-tool manpage 23177/head
Nathan Cutler [Wed, 18 Jul 2018 12:01:42 +0000 (14:01 +0200)]
doc: use :command: for subcommands in ceph-bluestore-tool manpage

Older versions of Sphinx, such as the one in CentOS 7, do not render "..
option::" lines correctly if the option contains a hyphen but does not start
with a hyphen. And ceph-bluestore-tool appears to be the only Ceph manpage
affected by this bug.

Fixes: http://tracker.ceph.com/issues/24800
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7c9229e7876dfc1bf6e415ed7cce924c3b50925d)

7 years agoMerge pull request #23147 from alfredodeza/luminous-rm24961
Andrew Schoen [Fri, 20 Jul 2018 19:25:20 +0000 (19:25 +0000)]
Merge pull request #23147 from alfredodeza/luminous-rm24961

ceph-volume tests/functional run lvm list after OSD provisioning

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests/functional run lvm list after OSD provisioning 23147/head
Alfredo Deza [Wed, 18 Jul 2018 16:39:01 +0000 (12:39 -0400)]
ceph-volume tests/functional run lvm list after OSD provisioning

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e01fb206d67f26724f3b03e4b22f5ba6eea22207)

7 years agoMerge pull request #23102 from ceph/backport-luminous-23093
Alfredo Deza [Fri, 20 Jul 2018 18:24:48 +0000 (14:24 -0400)]
Merge pull request #23102 from ceph/backport-luminous-23093

luminous: ceph-volume: an OSD ID must exist and be destroyed before reuse

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #23128 from ceph/backport-luminous-23117
Alfredo Deza [Fri, 20 Jul 2018 16:38:45 +0000 (12:38 -0400)]
Merge pull request #23128 from ceph/backport-luminous-23117

luminous: ceph-volume: unmount lvs correctly before zapping

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: unmount lvs correctly before zapping 23128/head
Andrew Schoen [Wed, 18 Jul 2018 16:08:15 +0000 (11:08 -0500)]
ceph-volume: unmount lvs correctly before zapping

If the lv found does not include the meatadata to construct an osd path,
then we should try to unmount it's path instead.

Fixes: http://tracker.ceph.com/issues/24796
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 5b175a5bb68264b80c00cb4f04026a4d19ca20ad)

7 years agoceph-volume: an osd ID must exist and be destroyed before reuse 23102/head
Andrew Schoen [Mon, 16 Jul 2018 21:06:03 +0000 (16:06 -0500)]
ceph-volume: an osd ID must exist and be destroyed before reuse

Fixes: https://tracker.ceph.com/issues/24044
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 02a13f2de0783ff3c680a6042388a4692250edfb)

7 years agoceph-volume util/prepare.py
Ron Allred [Wed, 23 May 2018 14:34:41 +0000 (08:34 -0600)]
ceph-volume util/prepare.py

Replaces PR#21887
Rename function check_id to osd_id_exists in util/prepare.py
Fix boolean analysis of osd_id_exists when --osd-id is provided on command line

Signed-off-by: Ron Allred <rallred@itrefined.com>
(cherry picked from commit 4641ebb075adc987e83d7440f9756104fc04f83a)

7 years ago12.2.7 v12.2.7
Jenkins Build Slave User [Mon, 16 Jul 2018 16:00:31 +0000 (16:00 +0000)]
12.2.7

7 years agoMerge PR #23055 into luminous
Sage Weil [Mon, 16 Jul 2018 15:50:19 +0000 (10:50 -0500)]
Merge PR #23055 into luminous

* refs/pull/23055/head:
common/options: disable osd_skip_data_digest
common/options: make osd_skip_data_digest a DEV option
common: add osd_distrust_data_digest option
osd: apply osd_skip_data_digest to read operations
osd: do not skip_data_digests during scrub
osd/PrimaryLogPG: use legacy style for osd_skip_data_digest

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agocommon/options: disable osd_skip_data_digest 23055/head
Sage Weil [Sat, 14 Jul 2018 20:03:43 +0000 (15:03 -0500)]
common/options: disable osd_skip_data_digest

This is flawed in that it stops doing digests based on the primary's
backend store, not the replicas.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocommon/options: make osd_skip_data_digest a DEV option
Sage Weil [Sat, 14 Jul 2018 18:26:31 +0000 (13:26 -0500)]
common/options: make osd_skip_data_digest a DEV option

And document/warn to never turn it off after it's been on.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocommon: add osd_distrust_data_digest option
Sage Weil [Sat, 14 Jul 2018 13:59:15 +0000 (08:59 -0500)]
common: add osd_distrust_data_digest option

If we have reason to distrust the stored full-object oi data digests,
this option will ignore them (and opportunistically clear them).  It
basically affects the same behaviors that osd_skip_data_digest except
the previous option only takes effect if bluestore is in use, while this
option is unconditional.

This serves a workaround for clusters that suffered the 12.2.6 bug that
produced bad full-object digests and *also* had PGs' primaries move from
bluestore to filestore nodes.  In those situations, this option can be
set to true.

After all OSDs are upgrade and all PGs come back with a clean deep
scrub this option can be disabled again.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd: apply osd_skip_data_digest to read operations
Sage Weil [Sat, 14 Jul 2018 18:12:54 +0000 (13:12 -0500)]
osd: apply osd_skip_data_digest to read operations

If we would skip updating data digests, apply that same decision to read
operations as well.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd: do not skip_data_digests during scrub
Sage Weil [Sat, 14 Jul 2018 16:45:12 +0000 (11:45 -0500)]
osd: do not skip_data_digests during scrub

The goal was to skip storing per-object digests.  Scrub still needs to
calculate these digests in order to do a deep scrub, however!

This is not a cherry-pick of master because the code has changed a bit
and because the master fix is not merged yet.

Fixes: http://tracker.ceph.com/issues/24922
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd/PrimaryLogPG: use legacy style for osd_skip_data_digest
Sage Weil [Sat, 14 Jul 2018 13:51:00 +0000 (08:51 -0500)]
osd/PrimaryLogPG: use legacy style for osd_skip_data_digest

This is more efficient.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #23039 from liewegas/wip-24903-workaround-luminous
Sage Weil [Fri, 13 Jul 2018 22:12:52 +0000 (17:12 -0500)]
Merge pull request #23039 from liewegas/wip-24903-workaround-luminous

rpm,deb: fix chown to use -h

7 years agorpm,deb: fix chown to use -h 23039/head
Sage Weil [Fri, 13 Jul 2018 22:10:38 +0000 (17:10 -0500)]
rpm,deb: fix chown to use -h

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 92207c76b91c26c0558688a358ade7c0d94d1b59)

7 years agoMerge PR #23028 into luminous
Sage Weil [Fri, 13 Jul 2018 19:44:09 +0000 (14:44 -0500)]
Merge PR #23028 into luminous

* refs/pull/23028/head:
osd/PrimaryLogPG: clear data digest on WRITEFULL if skip_data_digest

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge PR #23035 into luminous
Sage Weil [Fri, 13 Jul 2018 19:44:01 +0000 (14:44 -0500)]
Merge PR #23035 into luminous

* refs/pull/23035/head:
rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade

7 years agorpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade 23035/head
Sage Weil [Fri, 13 Jul 2018 14:56:48 +0000 (09:56 -0500)]
rpm,deb: chown bluestore block* links to ceph.ceph on install/upgrade

This works around https://tracker.ceph.com/issues/24903

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 58cde3fd793c037beca6645cc30b2ae1c30e2af1)

7 years agoMerge pull request #22369 from linuxbox2/wip-luminous-rgw-cache-time-added
Matt Benjamin [Fri, 13 Jul 2018 15:53:36 +0000 (11:53 -0400)]
Merge pull request #22369 from linuxbox2/wip-luminous-rgw-cache-time-added

Wip luminous rgw cache time added

7 years agoMerge PR #22962 into luminous
Sage Weil [Fri, 13 Jul 2018 12:11:54 +0000 (07:11 -0500)]
Merge PR #22962 into luminous

* refs/pull/22962/head:
osd/PrimaryLogPG: rebuild attrs from clients

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge PR #22976 into luminous
Sage Weil [Fri, 13 Jul 2018 12:11:48 +0000 (07:11 -0500)]
Merge PR #22976 into luminous

* refs/pull/22976/head:
qa/standalone/osd/ec-error-rollforward: reproduce bug 24597
qa/standalone/osd/repro_long_log.sh: fix test
osd/PG: do not blindly roll forward to log.head

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge PR #23007 into luminous
Sage Weil [Fri, 13 Jul 2018 12:11:43 +0000 (07:11 -0500)]
Merge PR #23007 into luminous

* refs/pull/23007/head:
qa/suites/rados/singleton/all/osd-recovery*: whitelist REQUEST_SLOW

7 years agoosd/PrimaryLogPG: clear data digest on WRITEFULL if skip_data_digest 23028/head
Sage Weil [Thu, 26 Apr 2018 14:15:50 +0000 (09:15 -0500)]
osd/PrimaryLogPG: clear data digest on WRITEFULL if skip_data_digest

If we detect we are on bluestore and should skip the data digest, we
need to clear any old digest on WRITEFULL.  This aligns us with the
WRITE behavior, which is also either an update or a clear (but never
neither!).

Fixes: https://tracker.ceph.com/issues/23871
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fe5038c7f9577327f82913b4565712c53903ee48)

7 years agoMerge PR #22872 into luminous
Patrick Donnelly [Thu, 12 Jul 2018 19:31:29 +0000 (12:31 -0700)]
Merge PR #22872 into luminous

* refs/pull/22872/head:
doc: outline steps to take CephFS fs down

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge PR #22963 into luminous
Patrick Donnelly [Thu, 12 Jul 2018 19:28:24 +0000 (12:28 -0700)]
Merge PR #22963 into luminous

* refs/pull/22963/head:
mon/AuthMonitor: improve error message

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
7 years agoqa/suites/rados/singleton/all/osd-recovery*: whitelist REQUEST_SLOW 23007/head
Sage Weil [Thu, 12 Jul 2018 13:16:03 +0000 (08:16 -0500)]
qa/suites/rados/singleton/all/osd-recovery*: whitelist REQUEST_SLOW

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/standalone/osd/ec-error-rollforward: reproduce bug 24597 22976/head
Sage Weil [Wed, 11 Jul 2018 21:14:38 +0000 (16:14 -0500)]
qa/standalone/osd/ec-error-rollforward: reproduce bug 24597

This reproduces http://tracker.ceph.com/issues/24597

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4108ebc0abae83777ebc69bda610167f39e22109)

7 years agoqa/standalone/osd/repro_long_log.sh: fix test
Sage Weil [Wed, 11 Jul 2018 12:10:28 +0000 (07:10 -0500)]
qa/standalone/osd/repro_long_log.sh: fix test

The log trimming case wasn't quite right.  Before HEAD^ we were
rolling forward too aggressively and miscalculating the can_rollforward_to,
which affected the trim_to calculation.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f9fdd98e2dde3979ec88a3f761d76f7b297415c)

7 years agoosd/PG: do not blindly roll forward to log.head
Sage Weil [Wed, 11 Jul 2018 01:22:49 +0000 (20:22 -0500)]
osd/PG: do not blindly roll forward to log.head

If we are told we can roll forward by the primary, we should only roll
forward as far as the primary says we can.

This probably came out of the similar case in append_log(), but notably
that roll_forward() only happens if !transaction_applied (i.e., backfill
target), and that condition is not checked here.

Fixes: http://tracker.ceph.com/issues/24597
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 858ff647ff7bd1f6277823b5185ead5ffebc05b0)

7 years agomon/AuthMonitor: improve error message 22963/head
Douglas Fuller [Mon, 20 Nov 2017 18:12:08 +0000 (13:12 -0500)]
mon/AuthMonitor: improve error message

Improve the error message for 'ceph fs authorize' where the
given entity exists but has different auth caps.

Fixes: http://tracker.ceph.com/issues/21765
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 447b3d4852acd2db656c973cc224fb77d3fff590)

7 years agoosd/PrimaryLogPG: rebuild attrs from clients 22962/head
Sage Weil [Wed, 4 Jul 2018 19:19:04 +0000 (14:19 -0500)]
osd/PrimaryLogPG: rebuild attrs from clients

Ensure that buffers coming in via client ops are rebuilt before populating
the PGTransaction.  This ensures that we don't pin the raw buffers for
the entire incoming message in memory.

In the past we've addressed this issue at the ObjectStore layer, but we
did not consider the attr_cache in ObjectContext.  Rebuilding the buffers
at this point will sanitize any incoming attribute before it reaches
either PGBackend or ObjectContext object_cache or the ObjectStore
implementation.

Fixes: http://tracker.ceph.com/issues/24768
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a0f8d5e6fa8276f06a9b7a7b18ea6f15fa8cb86)

7 years ago12.2.6 v12.2.6
Jenkins Build Slave User [Mon, 9 Jul 2018 16:18:47 +0000 (16:18 +0000)]
12.2.6

7 years agoMerge pull request #22911 from dillaman/wip-24740-luminous
Nathan Cutler [Mon, 9 Jul 2018 14:01:38 +0000 (16:01 +0200)]
Merge pull request #22911 from dillaman/wip-24740-luminous

luminous: qa/workunits/rbd: adapt import_export test to handle multiple units

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge remote-tracking branch 'private/wip-mon-snap-caps-luminous' into luminous
Sage Weil [Mon, 9 Jul 2018 13:36:02 +0000 (08:36 -0500)]
Merge remote-tracking branch 'private/wip-mon-snap-caps-luminous' into luminous

7 years agoMerge remote-tracking branch 'private/wip-cx-luminous' into luminous
Sage Weil [Mon, 9 Jul 2018 13:12:04 +0000 (08:12 -0500)]
Merge remote-tracking branch 'private/wip-cx-luminous' into luminous

7 years agoqa/workunits/rbd: adapt import_export test to handle multiple units 22911/head
Jason Dillaman [Fri, 6 Jul 2018 17:12:53 +0000 (13:12 -0400)]
qa/workunits/rbd: adapt import_export test to handle multiple units

Fixes: http://tracker.ceph.com/issues/24740
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22855 from jcsp/wip-luminous-cephfs-doc
John Spray [Thu, 5 Jul 2018 13:40:51 +0000 (14:40 +0100)]
Merge pull request #22855 from jcsp/wip-luminous-cephfs-doc

[luminous] doc/cephfs: remove lingering "experimental" note about multimds

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agodoc: outline steps to take CephFS fs down 22872/head
Patrick Donnelly [Fri, 6 Apr 2018 03:06:29 +0000 (20:06 -0700)]
doc: outline steps to take CephFS fs down

Fixes: https://tracker.ceph.com/issues/23427
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9aa13bf0b0bf20b4fcf4949af60ebfff1e8fea0a)

7 years agodoc/cephfs: remove lingering "experimental" note about multimds 22855/head
John Spray [Wed, 4 Jul 2018 16:04:09 +0000 (17:04 +0100)]
doc/cephfs: remove lingering "experimental" note about multimds

Signed-off-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #22831 from ceph/backport-luminous-22828
Alfredo Deza [Tue, 3 Jul 2018 22:45:13 +0000 (18:45 -0400)]
Merge pull request #22831 from ceph/backport-luminous-22828

luminous: ceph-volume: provide a nice errror message when missing ceph.conf

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: always ignore a missing ceph conf in main.py 22831/head
Andrew Schoen [Tue, 3 Jul 2018 11:45:24 +0000 (06:45 -0500)]
ceph-volume: always ignore a missing ceph conf in main.py

Now that we have a nice error message when a ceph.conf is missing
and we try to use values from it, maintaining a list of commands that
don't need ceph.conf isn't as helpful. We had actually missed 'simple
trigger' when we first implemented this causing all our luminous tests
for simple to fail when we backported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0487a9ac60c5ba289b7dd7c730b014a298c5998f)

7 years agoceph-volume: show a nice error message when ceph.conf is not loaded
Andrew Schoen [Mon, 2 Jul 2018 20:58:44 +0000 (15:58 -0500)]
ceph-volume: show a nice error message when ceph.conf is not loaded

If the ceph configuration file is not loaded correctly and then values
from it are used then an undescript error message is shown, e.g.

AttributeError: 'property' object has no attribute 'get'

With this change that same error condition shows the message:

RuntimeError: No valid ceph configuration file was loaded.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit e42ecb831914b07c7c8559d4556a8aabd2accd0c)

7 years agoMerge pull request #22746 from ceph/backport-luminous-22724
Alfredo Deza [Thu, 28 Jun 2018 14:47:14 +0000 (10:47 -0400)]
Merge pull request #22746 from ceph/backport-luminous-22724

luminous: ceph-volume: error on commands that need ceph.conf to operate

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: error on commands that need ceph.conf to operate 22746/head
Andrew Schoen [Tue, 26 Jun 2018 19:19:43 +0000 (14:19 -0500)]
ceph-volume: error on commands that need ceph.conf to operate

We had been ignoring the failure to load ceph.conf for all subcommands
but most of them require that a ceph.conf be present. This changes that
so only commands that do not need ceph.conf ignore the failure to find
it.

Fixes: http://tracker.ceph.com/issues/23941
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit de8b301ac20ae07f59a19bc1b3a083e9c010667d)

7 years agoqa/workunits/rados: test pool op permissions
Jason Dillaman [Thu, 21 Jun 2018 02:20:14 +0000 (22:20 -0400)]
qa/workunits/rados: test pool op permissions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 97e3f0aa013dad49de1a55f8025327de7e801822)
(cherry picked from commit b0d6e82a8c95380183af099a50a638f308b67090)

Conflicts:
qa/workunits/rados/test_pool_access.sh: use CEPH_KEYRING env variable

7 years agoqa/workunits/rbd: test self-managed snapshot create/remove permissions
Jason Dillaman [Tue, 5 Jun 2018 19:40:44 +0000 (15:40 -0400)]
qa/workunits/rbd: test self-managed snapshot create/remove permissions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b12dd0bf419ae834abb31c712830fa9c4b5cda9c)
(cherry picked from commit 37926b092dd1434e8fdca4620e86c0168aa26b24)

Conflicts:
qa/workunits/rbd/permissions.sh: use CEPH_KEYRING env variable

7 years agopybind/rados: new methods for manipulating self-managed snapshots
Jason Dillaman [Tue, 5 Jun 2018 18:48:17 +0000 (14:48 -0400)]
pybind/rados: new methods for manipulating self-managed snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 21538304d78df9f3b9f54a5b3c9b6b62fc3e6b48)
(cherry picked from commit 5bd343db3903d8aca45a40436a6dc240bdc1a0f3)

7 years agomon/OSDMonitor: enforce caps for all remaining pool ops
Jason Dillaman [Thu, 21 Jun 2018 01:30:47 +0000 (21:30 -0400)]
mon/OSDMonitor: enforce caps for all remaining pool ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 61c06200fe59996bba2bb65fc402207bc10fd459)
(cherry picked from commit fb4526690ccd29f1a03d3cd7e5484b9e2191b04b)

7 years agomon/OSDMonitor: enforce caps when creating/deleting unmanaged snapshots
Jason Dillaman [Tue, 5 Jun 2018 17:24:48 +0000 (13:24 -0400)]
mon/OSDMonitor: enforce caps when creating/deleting unmanaged snapshots

The entity will require write access to the OSD service or permission
for the synthetic "osd pool op unmanaged-snap" command.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4972e054b32c8200600f27564d50d443e683153e)

Conflicts:
src/mon/OSDMonitor.cc: use 'bufferlist::begin' instead of 'bufferlist::cbegin'
(cherry picked from commit 3dfb23d036c068ec5166151ed63628738d68908a)

Conflicts:
src/mon/CMakeLists.txt: added 'OSDCap.cc'

7 years agoMerge pull request #22716 from liewegas/wip-24661-luminous
Alfredo Deza [Tue, 26 Jun 2018 15:02:07 +0000 (11:02 -0400)]
Merge pull request #22716 from liewegas/wip-24661-luminous

luminous: os/bluestore: don't store/use path_block.{db,wal} from meta

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.de>
7 years agoceph-volume util make chown links affect links as well as paths 22716/head
Alfredo Deza [Fri, 8 Jun 2018 12:54:07 +0000 (08:54 -0400)]
ceph-volume util make chown links affect links as well as paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7f79932f2c522da222a27c15182434914ca9a738)

7 years agoceph-volume lvm.activate chown bluestore symlinks
Alfredo Deza [Fri, 8 Jun 2018 12:57:54 +0000 (08:57 -0400)]
ceph-volume lvm.activate chown bluestore symlinks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2a0911d04cc1eeb19ea2e1d6069095f9e2af26fe)

7 years agoos/bluestore: make bad symlink target error messages more informative
Sage Weil [Fri, 8 Jun 2018 13:34:53 +0000 (08:34 -0500)]
os/bluestore: make bad symlink target error messages more informative

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit aea215103caf669c3e289cc07f978511d61f68b0)

7 years agoos/bluestore: don't store/use path_block.{db,wal} from meta
Sage Weil [Tue, 26 Jun 2018 11:39:17 +0000 (06:39 -0500)]
os/bluestore: don't store/use path_block.{db,wal} from meta

This reverts most of 8d07fa5ad0772b4d979dc10695d230074d1f1e69.

ceph-volume goes to great lengths to ensure that the symlinks in the
osd dir are accurate.  Having these values here is an opportunity to
get them out of sync.  And that can happen very easily if the initial
mkfs was performed using a /dev/sdX device name (which is unstable
across reboots).  Even after ceph-volume corrects the symlink, bluestore
will continue to use the stale device path.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef510e43f0ee14b49e99beed9ae8feda6db3429a)

# Conflicts:
# src/os/bluestore/BlueStore.cc

- add_block_device has an extra arg in newer code

7 years agoMerge pull request #21368 from smithfarm/wip-23675-luminous
Yuri Weinstein [Fri, 22 Jun 2018 16:25:45 +0000 (09:25 -0700)]
Merge pull request #21368 from smithfarm/wip-23675-luminous

luminous: qa/workunits/mon/test_mon_config_key.py fails on master

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22115 from pdvian/wip-24048-luminous
Yuri Weinstein [Fri, 22 Jun 2018 16:24:59 +0000 (09:24 -0700)]
Merge pull request #22115 from pdvian/wip-24048-luminous

luminous: crush, osd: handle multiple parents properly when applying pg upmaps

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #22215 from tchaikov/luminous-24025
Yuri Weinstein [Fri, 22 Jun 2018 16:24:18 +0000 (09:24 -0700)]
Merge pull request #22215 from tchaikov/luminous-24025

luminous: cmake/modules/BuildRocksDB.cmake: enable compressions for rocksdb

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22269 from xiaoxichen/backport_22243
Yuri Weinstein [Fri, 22 Jun 2018 16:23:50 +0000 (09:23 -0700)]
Merge pull request #22269 from xiaoxichen/backport_22243

luminous: mon/OSDMonitor: Respect paxos_propose_interval

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22296 from tchaikov/luminous-24321
Yuri Weinstein [Fri, 22 Jun 2018 16:22:55 +0000 (09:22 -0700)]
Merge pull request #22296 from tchaikov/luminous-24321

luminous: qa: wait longer for osd to flush pg stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22366 from b-ranto/wip-branto-mgr-luminous
Yuri Weinstein [Fri, 22 Jun 2018 16:22:16 +0000 (09:22 -0700)]
Merge pull request #22366 from b-ranto/wip-branto-mgr-luminous

luminous: Sync up ceph-mgr prometheus module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
7 years agoMerge pull request #22650 from liewegas/wip-24319-luminous
Yuri Weinstein [Fri, 22 Jun 2018 00:30:03 +0000 (17:30 -0700)]
Merge pull request #22650 from liewegas/wip-24319-luminous

luminous: os/bluestore: fix SharedBlobSet refcounting race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22567 from pdvian/wip-24334-luminous
Yuri Weinstein [Fri, 22 Jun 2018 00:23:44 +0000 (17:23 -0700)]
Merge pull request #22567 from pdvian/wip-24334-luminous

luminous: ARMv8 feature detection broken, leading to illegal instruction crashes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22568 from pdvian/wip-24351-luminous
Yuri Weinstein [Fri, 22 Jun 2018 00:23:10 +0000 (17:23 -0700)]
Merge pull request #22568 from pdvian/wip-24351-luminous

luminous: slow mon ops from osd_failure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22381 from pdvian/wip-24258-luminous
Yuri Weinstein [Fri, 22 Jun 2018 00:21:11 +0000 (17:21 -0700)]
Merge pull request #22381 from pdvian/wip-24258-luminous

luminous: crush: fix device_class_clone for unpopulated/empty weight-sets

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22391 from pdvian/wip-24290-luminous
Yuri Weinstein [Fri, 22 Jun 2018 00:20:37 +0000 (17:20 -0700)]
Merge pull request #22391 from pdvian/wip-24290-luminous

luminous: common: fix typo in rados bench write JSON output

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22592 from pdvian/wip-24356-luminous
Yuri Weinstein [Fri, 22 Jun 2018 00:18:27 +0000 (17:18 -0700)]
Merge pull request #22592 from pdvian/wip-24356-luminous

luminous: osd: increase default hard pg limit

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22474 from yuriw/wip-yuriw-p2p-luminous
Josh Durgin [Thu, 21 Jun 2018 23:56:16 +0000 (16:56 -0700)]
Merge pull request #22474 from yuriw/wip-yuriw-p2p-luminous

qa/tests - added tag 12.2.5. to the mix and removed -x path as unneeded

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoqa/tests - added tag 12.2.5. to the mix and removed -x path as unneeded 22474/head
Yuri Weinstein [Fri, 8 Jun 2018 14:53:37 +0000 (07:53 -0700)]
qa/tests - added tag 12.2.5. to the mix and removed -x path as unneeded

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>