]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agocommon: Fix description for osd_skip_data_digest 23084/head
David Zafman [Mon, 16 Jul 2018 21:32:54 +0000 (14:32 -0700)]
common: Fix description for osd_skip_data_digest

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3b7f502c85593a1c69b49cb6022c7a57e37d1254)

7 years agocommon/options: disable osd_skip_data_digest
Sage Weil [Sat, 14 Jul 2018 20:03:43 +0000 (15:03 -0500)]
common/options: disable osd_skip_data_digest

This is flawed in that it stops doing digests based on the primary's
backend store, not the replicas.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c7d8add701f156eb8bd8c7dc8ae9903bd782e77f)

7 years agocommon/options: make osd_skip_data_digest a DEV option
Sage Weil [Sat, 14 Jul 2018 18:26:31 +0000 (13:26 -0500)]
common/options: make osd_skip_data_digest a DEV option

And document/warn to never turn it off after it's been on.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c854ed9b33a5a293c8ace15718e26f74237e3332)

7 years agoMerge pull request #22222 from pdvian/wip-24203-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:23:03 +0000 (08:23 -0700)]
Merge pull request #22222 from pdvian/wip-24203-mimic

mimic: rbd: Prevent the use of internal feature bits from outside cls/rbd

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22662 from pdvian/wip-24388-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:22:31 +0000 (08:22 -0700)]
Merge pull request #22662 from pdvian/wip-24388-mimic

mimic: librbd: utilize the journal disabled policy when removing images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22667 from pdvian/wip-24390-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:21:56 +0000 (08:21 -0700)]
Merge pull request #22667 from pdvian/wip-24390-mimic

mimic: rbd: rbd-mirror daemon failed to stop on active/passive test case

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22754 from pdvian/wip-24496-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:21:26 +0000 (08:21 -0700)]
Merge pull request #22754 from pdvian/wip-24496-mimic

mimic: librbd: force 'invalid object map' flag on-disk update

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22843 from smithfarm/wip-24734-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:20:55 +0000 (08:20 -0700)]
Merge pull request #22843 from smithfarm/wip-24734-mimic

mimic: build/ops: order rbdmap.service before remote-fs-pre.target

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #22891 from smithfarm/wip-24476-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:19:59 +0000 (08:19 -0700)]
Merge pull request #22891 from smithfarm/wip-24476-mimic

mimic: rbd: rbd trash purge --threshold should support data pool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22892 from smithfarm/wip-24519-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:19:29 +0000 (08:19 -0700)]
Merge pull request #22892 from smithfarm/wip-24519-mimic

mimic: rbd: rbd-mirror simple image map policy doesn't always level-load instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22894 from smithfarm/wip-24596-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:18:58 +0000 (08:18 -0700)]
Merge pull request #22894 from smithfarm/wip-24596-mimic

mimic: rbd: yet another case when deep copying a clone may result in invalid object map

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22943 from pdvian/wip-24698-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:17:52 +0000 (08:17 -0700)]
Merge pull request #22943 from pdvian/wip-24698-mimic

mimic: librbd: fix crash when opening nonexistent snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #23034 from tchaikov/mimic-clang-4
Kefu Chai [Sat, 14 Jul 2018 12:50:49 +0000 (20:50 +0800)]
Merge pull request #23034 from tchaikov/mimic-clang-4

include: fix build with older clang

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
7 years agoinclude: fix build with older clang 23034/head
Kefu Chai [Fri, 13 Jul 2018 16:40:36 +0000 (00:40 +0800)]
include: fix build with older clang

to fix the FTBFS on clang shipped with xcode 9.1 (based on Clang 4.0).

Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22997 from liewegas/wip-24597-mimic
Yuri Weinstein [Fri, 13 Jul 2018 15:18:12 +0000 (08:18 -0700)]
Merge pull request #22997 from liewegas/wip-24597-mimic

mimic: osd: do not blindly roll forward to log.head

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22842 from pdvian/wip-24547-mimic
Yuri Weinstein [Thu, 12 Jul 2018 16:37:03 +0000 (09:37 -0700)]
Merge pull request #22842 from pdvian/wip-24547-mimic

mimic: rgw: have a configurable authentication order

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22888 from smithfarm/wip-24617-mimic
Yuri Weinstein [Thu, 12 Jul 2018 16:36:24 +0000 (09:36 -0700)]
Merge pull request #22888 from smithfarm/wip-24617-mimic

mimic: ValueError: too many values to unpack due to lack of subdir

Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #22986 from joscollin/wip-24843-mimic
Yuri Weinstein [Thu, 12 Jul 2018 16:35:53 +0000 (09:35 -0700)]
Merge pull request #22986 from joscollin/wip-24843-mimic

mimic: rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22291 from pdvian/wip-24204-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:06:41 +0000 (17:06 -0700)]
Merge pull request #22291 from pdvian/wip-24204-mimic

mimic: ceph_test_rados_api_misc: fix LibRadosMiscPool.PoolCreationRace

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22412 from tchaikov/mimic-cmake-boost-1.67
Yuri Weinstein [Thu, 12 Jul 2018 00:06:20 +0000 (17:06 -0700)]
Merge pull request #22412 from tchaikov/mimic-cmake-boost-1.67

mimic: cmake: bump up the required boost version to 1.67

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22492 from tchaikov/mimic-24321
Yuri Weinstein [Thu, 12 Jul 2018 00:05:33 +0000 (17:05 -0700)]
Merge pull request #22492 from tchaikov/mimic-24321

mimic: qa: wait longer for osd to flush pg stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22777 from croit/mimic-22977
Yuri Weinstein [Thu, 12 Jul 2018 00:04:58 +0000 (17:04 -0700)]
Merge pull request #22777 from croit/mimic-22977

mimic:  common/hobject: improved hash calculation for hobject_t etc

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22805 from smithfarm/wip-24708-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:04:30 +0000 (17:04 -0700)]
Merge pull request #22805 from smithfarm/wip-24708-mimic

mimic: ceph osd safe-to-destroy crashes the mgr

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #22813 from smithfarm/wip-24747-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:03:47 +0000 (17:03 -0700)]
Merge pull request #22813 from smithfarm/wip-24747-mimic

mimic: change default filestore_merge_threshold to -10

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22887 from smithfarm/wip-24473-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:02:45 +0000 (17:02 -0700)]
Merge pull request #22887 from smithfarm/wip-24473-mimic

mimic: cosbench stuck at booting cosbench driver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22889 from smithfarm/wip-24618-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:02:07 +0000 (17:02 -0700)]
Merge pull request #22889 from smithfarm/wip-24618-mimic

mimic: osd: choose_acting loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22910 from ifed01/wip-ifed-fix-23540-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:00:33 +0000 (17:00 -0700)]
Merge pull request #22910 from ifed01/wip-ifed-fix-23540-mimic

mimic: os/bluestore: fix incomplete faulty range marking when doing compression

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22960 from pdvian/wip-24805-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:00:02 +0000 (17:00 -0700)]
Merge pull request #22960 from pdvian/wip-24805-mimic

mimic: osd/PrimaryLogPG: rebuild attrs from clients

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22550 from smithfarm/wip-24296-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:25:45 +0000 (16:25 -0700)]
Merge pull request #22550 from smithfarm/wip-24296-mimic

mimic: repeated eviction of idle client until some IO happens

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #22810 from smithfarm/wip-24703-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:24:54 +0000 (16:24 -0700)]
Merge pull request #22810 from smithfarm/wip-24703-mimic

mimic: PurgeQueue sometimes ignores Journaler errors

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22816 from pdvian/wip-24537-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:24:32 +0000 (16:24 -0700)]
Merge pull request #22816 from pdvian/wip-24537-mimic

mimic: common/DecayCounter: set last_decay to current time when decoding dec…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22884 from smithfarm/wip-24704-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:24:06 +0000 (16:24 -0700)]
Merge pull request #22884 from smithfarm/wip-24704-mimic

mimic: mds: low wrlock efficiency due to dirfrags traversal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoqa/standalone/osd/ec-error-rollforward: reproduce bug 24597 22997/head
Sage Weil [Wed, 11 Jul 2018 21:14:38 +0000 (16:14 -0500)]
qa/standalone/osd/ec-error-rollforward: reproduce bug 24597

This reproduces http://tracker.ceph.com/issues/24597

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4108ebc0abae83777ebc69bda610167f39e22109)

7 years agoqa/standalone/osd/repro_long_log.sh: fix test
Sage Weil [Wed, 11 Jul 2018 12:10:28 +0000 (07:10 -0500)]
qa/standalone/osd/repro_long_log.sh: fix test

The log trimming case wasn't quite right.  Before HEAD^ we were
rolling forward too aggressively and miscalculating the can_rollforward_to,
which affected the trim_to calculation.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f9fdd98e2dde3979ec88a3f761d76f7b297415c)

7 years agoosd/PG: do not blindly roll forward to log.head
Sage Weil [Wed, 11 Jul 2018 01:22:49 +0000 (20:22 -0500)]
osd/PG: do not blindly roll forward to log.head

If we are told we can roll forward by the primary, we should only roll
forward as far as the primary says we can.

This probably came out of the similar case in append_log(), but notably
that roll_forward() only happens if !transaction_applied (i.e., backfill
target), and that condition is not checked here.

Fixes: http://tracker.ceph.com/issues/24597
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 858ff647ff7bd1f6277823b5185ead5ffebc05b0)

7 years agoMerge pull request #22886 from smithfarm/wip-24707-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:19:14 +0000 (13:19 -0700)]
Merge pull request #22886 from smithfarm/wip-24707-mimic

mimic: mgr/dashboard: Float numbers incorrectly formatted

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22265 from liewegas/wip-24308-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:17:08 +0000 (13:17 -0700)]
Merge pull request #22265 from liewegas/wip-24308-mimic

mimic: mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22707 from pdvian/wip-24475-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:16:23 +0000 (13:16 -0700)]
Merge pull request #22707 from pdvian/wip-24475-mimic

mimic: librados: fix buffer overflow for aio_exec python binding

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22826 from smithfarm/wip-24738-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:15:29 +0000 (13:15 -0700)]
Merge pull request #22826 from smithfarm/wip-24738-mimic

mimic: Bring back diff -y for non-FreeBSD

Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #22851 from smithfarm/wip-24766-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:14:37 +0000 (13:14 -0700)]
Merge pull request #22851 from smithfarm/wip-24766-mimic

mimic: build/ops: Mimic build fails with -DWITH_RADOSGW=0

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Bassam Tabbara <bassam.tabbara@quantum.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
7 years agoMerge pull request #22859 from smithfarm/wip-24769-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:11:49 +0000 (13:11 -0700)]
Merge pull request #22859 from smithfarm/wip-24769-mimic

mimic: set correctly shard for existed Collection

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22861 from smithfarm/wip-24771-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:11:17 +0000 (13:11 -0700)]
Merge pull request #22861 from smithfarm/wip-24771-mimic

mimic: osd: may get empty info at recovery

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22917 from smithfarm/wip-24803-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:09:29 +0000 (13:09 -0700)]
Merge pull request #22917 from smithfarm/wip-24803-mimic

mimic: Python bindings use iteritems method which is not Python 3 compatible

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agorgw: require --yes-i-really-mean-it to run radosgw-admin orphans find 22986/head
Matt Benjamin [Wed, 16 May 2018 17:04:55 +0000 (13:04 -0400)]
rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Incorrect use of orphans find can lead to data loss. Warn users to be
extra cautious.

Fixes: http://tracker.ceph.com/issues/24146
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 3ff47c7f3eb5964464c8cd49144546ce532ed7f7)

7 years agolibrbd: fix gmock warnings introduced after 9e5b87c 22894/head
Mykola Golub [Tue, 19 Jun 2018 13:22:15 +0000 (16:22 +0300)]
librbd: fix gmock warnings introduced after 9e5b87c

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit f434c5c70bc3f51849d1439dcd8448022cb5e056)

7 years agolibrbd: deep_copy: update end_size only if zero interval caused truncate
Mykola Golub [Tue, 19 Jun 2018 12:05:27 +0000 (15:05 +0300)]
librbd: deep_copy: update end_size only if zero interval caused truncate

The problem shown up when stripping was used, and assembling a
nonexistent destination object from source objects, and one of these
objects existed and snap diff returned a zero interval. The non-zero
end_size in that case triggered invalid object map update.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d1122573b8c6a69857004d042cc75e9581deebda)

7 years agolibrbd: deep_copy: don't hide parent if zero overlap for snapshot
Mykola Golub [Sat, 16 Jun 2018 11:19:53 +0000 (14:19 +0300)]
librbd: deep_copy: don't hide parent if zero overlap for snapshot

Fixes: http://tracker.ceph.com/issues/24545
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d3d730ce0f9f7f287e4b60cface66e9d9fb6882e)

7 years agolibrbd: remove unused header
Mykola Golub [Sat, 9 Dec 2017 15:10:10 +0000 (17:10 +0200)]
librbd: remove unused header

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 84ed0c5726e6a0707189f7adec0dcfff0fe73c57)

7 years agoMerge pull request #22549 from smithfarm/wip-24391-mimic
Yuri Weinstein [Tue, 10 Jul 2018 16:01:44 +0000 (09:01 -0700)]
Merge pull request #22549 from smithfarm/wip-24391-mimic

mimic: [rbd-mirror] entries_behind_master will not be zero after mirror over

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22768 from pdvian/wip-24499-mimic
Yuri Weinstein [Tue, 10 Jul 2018 16:00:53 +0000 (09:00 -0700)]
Merge pull request #22768 from pdvian/wip-24499-mimic

mimic: librbd: deep_copy: resize head object map if needed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22770 from jcsp/wip-24689
Yuri Weinstein [Tue, 10 Jul 2018 15:59:35 +0000 (08:59 -0700)]
Merge pull request #22770 from jcsp/wip-24689

mimic: mon: exception for dashboard in config-key warning

Reviewed-by: Ricardo Dias <rdias@suse.com>
7 years agoMerge pull request #22268 from xiaoxichen/backport_22243_mimic
Kefu Chai [Tue, 10 Jul 2018 12:41:50 +0000 (20:41 +0800)]
Merge pull request #22268 from xiaoxichen/backport_22243_mimic

mimic: mon/OSDMonitor: Respect paxos_propose_interval

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoosd/PrimaryLogPG: rebuild attrs from clients 22960/head
Sage Weil [Wed, 4 Jul 2018 19:19:04 +0000 (14:19 -0500)]
osd/PrimaryLogPG: rebuild attrs from clients

Ensure that buffers coming in via client ops are rebuilt before populating
the PGTransaction.  This ensures that we don't pin the raw buffers for
the entire incoming message in memory.

In the past we've addressed this issue at the ObjectStore layer, but we
did not consider the attr_cache in ObjectContext.  Rebuilding the buffers
at this point will sanitize any incoming attribute before it reaches
either PGBackend or ObjectContext object_cache or the ObjectStore
implementation.

Fixes: http://tracker.ceph.com/issues/24768
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a0f8d5e6fa8276f06a9b7a7b18ea6f15fa8cb86)

7 years agoMerge pull request #22700 from yuriw/wip-yuriw-22532-backport
Yuri Weinstein [Mon, 9 Jul 2018 20:31:20 +0000 (13:31 -0700)]
Merge pull request #22700 from yuriw/wip-yuriw-22532-backport

mimic: qa/suites: Add supported-random-distro$ links.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Warren Usui <wusui@redhat.com>

7 years agoMerge pull request #22791 from pdvian/wip-24534-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:28:24 +0000 (13:28 -0700)]
Merge pull request #22791 from pdvian/wip-24534-mimic

mimic: client: fix bug #24491 _ll_drop_pins may access invalid iterator

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22819 from pdvian/wip-24539-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:27:59 +0000 (13:27 -0700)]
Merge pull request #22819 from pdvian/wip-24539-mimic

mimic: client:  update inode fields according to issued caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22841 from pdvian/wip-24541-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:27:26 +0000 (13:27 -0700)]
Merge pull request #22841 from pdvian/wip-24541-mimic

mimic: mds: add magic to header of open file table

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22882 from smithfarm/wip-24310-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:25:59 +0000 (13:25 -0700)]
Merge pull request #22882 from smithfarm/wip-24310-mimic

mimic: pjd: cd: too many arguments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22900 from pdvian/wip-24585-mimic
Alfredo Deza [Mon, 9 Jul 2018 19:36:42 +0000 (15:36 -0400)]
Merge pull request #22900 from pdvian/wip-24585-mimic

mimic: ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #22849 from smithfarm/wip-24764-mimic
Lenz Grimmer [Mon, 9 Jul 2018 18:52:50 +0000 (20:52 +0200)]
Merge pull request #22849 from smithfarm/wip-24764-mimic

mimic: mgr/dashboard: Missing breadcrumb on monitor performance counters page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
7 years agoMerge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:29:13 +0000 (08:29 -0500)]
Merge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic

7 years agoMerge remote-tracking branch 'private/wip-cx-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:09:38 +0000 (08:09 -0500)]
Merge remote-tracking branch 'private/wip-cx-mimic' into mimic

7 years agolibrbd: fix crash when opening nonexistent snapshot 22943/head
Mykola Golub [Fri, 22 Jun 2018 17:50:51 +0000 (20:50 +0300)]
librbd: fix crash when opening nonexistent snapshot

Fixes: http://tracker.ceph.com/issues/24637
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 70936c3bb5c281eed1c0bb5e26337efdedfa1e06)

7 years agoMerge pull request #22297 from tchaikov/mimic-24322
Yuri Weinstein [Fri, 6 Jul 2018 21:50:12 +0000 (14:50 -0700)]
Merge pull request #22297 from tchaikov/mimic-24322

mimic: mon/OSDMonitor: no_reply on MOSDFailure messages

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22333 from pdvian/wip-24246-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:49:43 +0000 (14:49 -0700)]
Merge pull request #22333 from pdvian/wip-24246-mimic

mimic: mon/MgrMonitor: change 'unresponsive' message to info level

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22361 from tchaikov/mimic-24361
Yuri Weinstein [Fri, 6 Jul 2018 21:48:58 +0000 (14:48 -0700)]
Merge pull request #22361 from tchaikov/mimic-24361

mimic: mon: enable level_compaction_dynamic_level_bytes for rocksdb

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22382 from pdvian/wip-24261-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:48:27 +0000 (14:48 -0700)]
Merge pull request #22382 from pdvian/wip-24261-mimic

mimic: bluestore: flush_commit is racy

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22394 from tchaikov/mimic-24406
Yuri Weinstein [Fri, 6 Jul 2018 21:47:30 +0000 (14:47 -0700)]
Merge pull request #22394 from tchaikov/mimic-24406

mimic: osd: retry to read object attrs at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22406 from pdvian/wip-24292-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:44:44 +0000 (14:44 -0700)]
Merge pull request #22406 from pdvian/wip-24292-mimic

mimic: common: fix typo in rados bench write JSON output

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22525 from tchaikov/mimic-pr-22336
Yuri Weinstein [Fri, 6 Jul 2018 21:43:42 +0000 (14:43 -0700)]
Merge pull request #22525 from tchaikov/mimic-pr-22336

mimic: os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
7 years agoMerge pull request #22552 from smithfarm/wip-24468-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:42:40 +0000 (14:42 -0700)]
Merge pull request #22552 from smithfarm/wip-24468-mimic

mimic: tell ... config rm <foo> not idempotent

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22559 from pdvian/wip-24332-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:42:08 +0000 (14:42 -0700)]
Merge pull request #22559 from pdvian/wip-24332-mimic

mimic: PG: add custom_reaction Backfilled and release reservations after bac…

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #22651 from liewegas/wip-24423-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:40:33 +0000 (14:40 -0700)]
Merge pull request #22651 from liewegas/wip-24423-mimic

mimic: osd: fix getting osd maps on initial osd startup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22722 from liewegas/wip-hobject-hash-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:39:55 +0000 (14:39 -0700)]
Merge pull request #22722 from liewegas/wip-hobject-hash-mimic

mimic: common/hobject: improved hash calculation

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
7 years agoMerge pull request #22730 from pdvian/wip-24494-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:38:59 +0000 (14:38 -0700)]
Merge pull request #22730 from pdvian/wip-24494-mimic

mimic: osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agomgr: replace iteritems with six.iteritems (mgr_module.py) 22917/head
Nathan Cutler [Thu, 5 Jul 2018 15:34:51 +0000 (17:34 +0200)]
mgr: replace iteritems with six.iteritems (mgr_module.py)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 093562f6de706cb810fef5b0a297fb2422c65c78)

7 years agomgr: replace iteritems with six.iteritems (telegraf)
Nathan Cutler [Thu, 5 Jul 2018 15:24:43 +0000 (17:24 +0200)]
mgr: replace iteritems with six.iteritems (telegraf)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit ab001e134238f609823d0aad25f5e758107d9369)

Conflicts:
src/pybind/mgr/telegraf/module.py (trivial)

7 years agomgr: replace iteritems with six.iteritems (status)
Nathan Cutler [Thu, 5 Jul 2018 15:23:24 +0000 (17:23 +0200)]
mgr: replace iteritems with six.iteritems (status)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 5866377ec39a3f0954afc8b0a3a8a544f188f307)

7 years agomgr: replace iteritems with six.iteritems (restful)
Nathan Cutler [Thu, 5 Jul 2018 15:22:16 +0000 (17:22 +0200)]
mgr: replace iteritems with six.iteritems (restful)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit c02c6601755efbe5fb06021d314d5b2d8aa6c79e)

7 years agomgr: replace iteritems with six.iteritems (influx)
Nathan Cutler [Thu, 5 Jul 2018 15:20:56 +0000 (17:20 +0200)]
mgr: replace iteritems with six.iteritems (influx)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit a62ab8ae8c91b91bb6c5367a6af197e5d85fe10a)

7 years agomgr: replace iteritems with six.iteritems (balancer)
Nathan Cutler [Thu, 5 Jul 2018 15:17:36 +0000 (17:17 +0200)]
mgr: replace iteritems with six.iteritems (balancer)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 962f0c416c90c2b319d0a9c403fe235086446080)

7 years agomgr/dashboard: Fix formatter service unit test 22886/head
Tiago Melo [Wed, 30 May 2018 12:20:05 +0000 (13:20 +0100)]
mgr/dashboard: Fix formatter service unit test

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 5058ff960733f19937be1be4c401e89bf744138b)

7 years agoos/bluestore: fix incomplete faulty range marking when doing compression 22910/head
Igor Fedotov [Thu, 5 Jul 2018 11:27:12 +0000 (14:27 +0300)]
os/bluestore: fix incomplete faulty range marking when doing compression
GC.

Under some scenarios GC might process an extent range where some inner extents are left untouched by GC (as there is no need for that). Hence GC doesn't invaliate these inner extents with fault_range call. If untouched extents are mapped to unloaded shards it results in subsequent assertion on o->extent_map.dirty_range() call.
The solution is to invalidate the whole extent range when doing GC.

Fixes: https://tracker.ceph.com/issues/23540
Fixes: https://tracker.ceph.com/issues/24799
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0f04d4484c8663767bdb60f743a8835897013b5a)

7 years agomon: exception for dashboard in config-key warning 22770/head
John Spray [Fri, 29 Jun 2018 10:36:39 +0000 (11:36 +0100)]
mon: exception for dashboard in config-key warning

This warning went in with the expectation that nobody
would be using config-key commands for modules any more,
but the dashboard does use these in order to get the
"-i" functionality on the CLI for loading certs/keys.

In Nautilus they can switch to using "-i" on real module
commands, but for Mimic let's silence the warning for
the dashboard module.

Fixes: https://tracker.ceph.com/issues/24689
Signed-off-by: John Spray <john.spray@redhat.com>
7 years agoceph-volume lvm.common use destroy-new, doesn't need admin keyring 22900/head
Alfredo Deza [Fri, 8 Jun 2018 19:36:17 +0000 (15:36 -0400)]
ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 57fb124ce74fe4fb091091be6b4fa63e36d567a2)

7 years agorbd-mirror: fix state transition table for disassociation 22892/head
Venky Shankar [Mon, 4 Jun 2018 09:34:58 +0000 (05:34 -0400)]
rbd-mirror: fix state transition table for disassociation

The final state transition when disassociating (removing) images
does not purge the image state map for a given image. This can
also result in uneven balance of images across instances as the
policy implementation relies on this structure to figure out
total number of images tracked.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 895d69f226afd0dfa33909ff9766abf780531787)

7 years agotest: add rbd-mirror image map rebalancer unit test
Venky Shankar [Sat, 2 Jun 2018 14:55:22 +0000 (10:55 -0400)]
test: add rbd-mirror image map rebalancer unit test

Signed-off-by: Venky Shankar <vshankar@redhat.com>
Fixes: http://tracker.ceph.com/issues/24161
(cherry picked from commit d491615f084806bc1de408098813d65853ca97af)

7 years agorbd-mirror: schedule rebalancer to level-load instances
Venky Shankar [Tue, 29 May 2018 05:45:40 +0000 (01:45 -0400)]
rbd-mirror: schedule rebalancer to level-load instances

Policy implementation takes care of evenly balancing images
across rbd mirror instances. This is done when images are
added to the map and/or instances are added or removed with
the exception of image removal -- removing images does not
reshuffle other (mapped) images which can result in some of
the instances under loaded (in worst case, if one removes
images which all map to a particular instance, that instance
would remain idle until more images are added or a shuffle is
triggered).

We could possibly trigger map shuffle when images are removed,
but that would change the interface between Policy and ImageMap
class (in the form of changes to Policy::remove_images()). Also,
policy (and its implementations) would have to do more work when
the above class method is invoked.

Therefore, an interval based rebalancer is added to ImageMap for
periodic rebalancing of images only if the following conditions
are met:

    - policy has been idle for a configured time duration
    - no scheduled or in-transit operations

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit d96cee13d7427bca617e473c522ad7cf00274566)

7 years agorbd-mirror: fix state comparison in Policy::is_state_scheduled()
Venky Shankar [Mon, 28 May 2018 04:53:30 +0000 (00:53 -0400)]
rbd-mirror: fix state comparison in Policy::is_state_scheduled()

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bd83d93453b8fb7289594f8ad388f5c83bab84b9)

7 years agorbd-mirror: remove unneeded instance_ids parameter
Venky Shankar [Fri, 25 May 2018 10:48:10 +0000 (06:48 -0400)]
rbd-mirror: remove unneeded instance_ids parameter

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 65abcd480100516a7186ab6ffbf22907c5a3156c)

7 years agorbd trash: add data pool support 22891/head
Mahati Chamarthy [Wed, 4 Apr 2018 15:24:23 +0000 (20:54 +0530)]
rbd trash: add data pool support

checks implicitly for datapool utilization of all images and removes
respective pool's trashed images until threshold value is hit.

Fixes: http://tracker.ceph.com/issues/22872
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
(cherry picked from commit 00d5c18b18fcbde6d7fee1673b1b084d492d87c1)

7 years agoosd/PG: restrict async_recovery_targets to up osds 22889/head
Neha Ojha [Wed, 20 Jun 2018 17:20:58 +0000 (13:20 -0400)]
osd/PG: restrict async_recovery_targets to up osds

When an osd that is part of the acting set and not the up set, gets chosen
as an async_recovery_target, it gets removed from the acting set. Since this
osd is no longer in the up or acting set, it is classified as a stray in
the next peering cycle. This results in choose_acting() looping between two
proposed acting sets.

To avoid this, we will only choose up osds as async_recovery_targets.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7f1b6adec31e71ef4b3ec9b3c5cba8b1a4b7746f)

7 years agoPG: do not choose stray osds as async_recovery_targets
Neha Ojha [Wed, 30 May 2018 18:33:41 +0000 (11:33 -0700)]
PG: do not choose stray osds as async_recovery_targets

Without this change, we might accept stray osds as async_recovery_targets,
and need to ensure that they get a chance to become part of the acting set
after recovery is over.

However, when choose_acting() is called in the Recovered state, we set
restrict_to_up_acting=true, which does not allow them to get back to the
acting set.

Therefore, similar to backfill, do not allow stray osds to become
async_recovery_targets.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit d96732d2448173f70f6514610d1c3780a7c8fab1)

7 years agoqa/suites/rados/objectstore: move yaml files under a subdir 22888/head
Neha Ojha [Wed, 23 May 2018 22:27:41 +0000 (15:27 -0700)]
qa/suites/rados/objectstore: move yaml files under a subdir

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7cfc95b367d8b80d7e42b92f3c0f18147f6aae9c)

7 years agoqa: whitelist SMALLER_PGP_NUM
Neha Ojha [Wed, 23 May 2018 22:15:54 +0000 (15:15 -0700)]
qa: whitelist SMALLER_PGP_NUM

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit dca3275f0c8f3ce96a73a4143c5658d4a3d73b59)

7 years agoqa/tasks/cbt.py: changes to run on bionic 22887/head
Neha Ojha [Tue, 5 Jun 2018 00:23:21 +0000 (17:23 -0700)]
qa/tasks/cbt.py: changes to run on bionic

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit bf62e1fadc6a49a511e3e569cc5c6fa1a0cf78f0)

7 years agomgr/dashboard: Format small numbers correctly
Stephan Müller [Mon, 14 May 2018 13:11:27 +0000 (15:11 +0200)]
mgr/dashboard: Format small numbers correctly

The issue was triggered by numbers that a lower than 1.
Doing a logarithm with a number lower than 1 leads to
negative value that is not handled anywhere in The formatter service as
a result the final value will be quirky.
The negative number will also be used as index in the units array, where
it will return "undefined".

Fixes #24081

Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit b8ecac3603c01764d83e2b8862f3b6f847321e85)

7 years agomds: avoid traversing all dirfrags when trying to get wrlocks 22884/head
Xuehan Xu [Fri, 8 Jun 2018 14:29:30 +0000 (22:29 +0800)]
mds: avoid traversing all dirfrags when trying to get wrlocks

Fixes: http://tracker.ceph.com/issues/24467
Signed-off-by: Xuehan Xu <xuxuehan@360.cn>
(cherry picked from commit da02fbcb0afd1b72d6c030f06613d8b18028a13c)

7 years agoqa/workunits/suites/pjd.sh: use correct dir name 22882/head
Neha Ojha [Fri, 25 May 2018 00:03:01 +0000 (17:03 -0700)]
qa/workunits/suites/pjd.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 8fbab63436467aeb72b43f9db8c6b182e48d993c)

7 years agoMerge pull request #22854 from jcsp/wip-mimic-cephfs-doc
John Spray [Thu, 5 Jul 2018 13:41:06 +0000 (14:41 +0100)]
Merge pull request #22854 from jcsp/wip-mimic-cephfs-doc

[mimic] doc/cephfs: remove lingering "experimental" note about multimds

Reviewed-by: Nathan Cutler <ncutler@suse.com>