]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoceph-volume: extracts batch.filter_devices from Batch._get_strategy 24588/head
Andrew Schoen [Wed, 10 Oct 2018 19:28:29 +0000 (15:28 -0400)]
ceph-volume: extracts batch.filter_devices from Batch._get_strategy

This allows us to easily provide tests for that method.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit df7ef5383bb476ee020b898cd2e8fbce044fc07e)

6 years agoceph-volume: failing to get block db size from conf logs an exception
Andrew Schoen [Wed, 10 Oct 2018 18:05:25 +0000 (14:05 -0400)]
ceph-volume: failing to get block db size from conf logs an exception

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 172d4af9b97d8d64ce35dc95efb72d9f190bc170)

6 years agoceph-volume: when all devices are filtered exit gracefully
Andrew Schoen [Tue, 9 Oct 2018 18:05:54 +0000 (14:05 -0400)]
ceph-volume: when all devices are filtered exit gracefully

Even if all devices are filtered we want to return a 0 exit code and
make sure the json reporting still works.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit aa4fcd602f8b1a1b17bc12e59334b19508f97d6d)

6 years agoceph-volume: filter devices used by journals/block.db
Andrew Schoen [Mon, 8 Oct 2018 13:57:07 +0000 (09:57 -0400)]
ceph-volume: filter devices used by journals/block.db

If after filterering of data/block devices there are only
one device left it can not be used if it is an SSD and
has been used previously as a journal or block.db

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit fc9a10e54813a402ecd1d823ea5a33f85e8eb963)

6 years agoceph-volume: add rotational property to Device class
Andrew Schoen [Mon, 8 Oct 2018 13:39:05 +0000 (09:39 -0400)]
ceph-volume: add rotational property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7f6bfaaf0cf30a56481d6d67f2fa2a7e785cc344)

6 years agoceph-volume: add info about filtered devices to batch pretty reports
Andrew Schoen [Fri, 5 Oct 2018 21:18:48 +0000 (16:18 -0500)]
ceph-volume: add info about filtered devices to batch pretty reports

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 62426db799c54436c50b5751c8e72b3dbd710a2a)

6 years agoceph-volume: remove the used_by_ceph key in the json output
Andrew Schoen [Fri, 5 Oct 2018 15:45:35 +0000 (10:45 -0500)]
ceph-volume: remove the used_by_ceph key in the json output

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3b6d82afe7ee83283b25cefda00c7a59cf5144af)

6 years agoceph-volume: fix idempotency checks for lvm batch tests
Andrew Schoen [Fri, 5 Oct 2018 15:39:54 +0000 (10:39 -0500)]
ceph-volume: fix idempotency checks for lvm batch tests

The mixed type tests will change strategy after the idempotency test so
we need to handle that in test playbook.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1143482061da50910f4f883aafdbe0a0e4269f39)

6 years agoceph-volume: fix bluestore strategy json reporting and type
Andrew Schoen [Fri, 5 Oct 2018 15:38:11 +0000 (10:38 -0500)]
ceph-volume: fix bluestore strategy json reporting and type

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b3cf90604d31709996538e612b1767d6fd4da8b5)

6 years agoceph-volume: ignore failure to load ceph configuration for block.db size
Andrew Schoen [Thu, 4 Oct 2018 17:47:48 +0000 (12:47 -0500)]
ceph-volume: ignore failure to load ceph configuration for block.db size

If we fail to load a ceph configureation file when trying to get the
block.db size then just use defaults instead of throwing an error.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a7ee36ca92b6592b6b3e218252c6a4c30416591b)

6 years agoceph-volume: fix strategy comparison in 'lvm batch'
Andrew Schoen [Thu, 4 Oct 2018 16:54:06 +0000 (11:54 -0500)]
ceph-volume: fix strategy comparison in 'lvm batch'

This also fixes some small json reporting issues with the
filestore MixedType strategy

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1dd15025bfd52af9c31cd281f92d743b9ca0eeb8)

6 years agoceph-volume: raise a non zero exit code if strategy changes with batch
Andrew Schoen [Wed, 3 Oct 2018 20:01:08 +0000 (15:01 -0500)]
ceph-volume: raise a non zero exit code if strategy changes with batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9dfc00f8e10e10fbdc52c44259596c6a96a90edd)

6 years agoceph-volume: add functional tests to ensure lvm batch is idempotent
Andrew Schoen [Wed, 3 Oct 2018 17:13:27 +0000 (12:13 -0500)]
ceph-volume: add functional tests to ensure lvm batch is idempotent

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9752d03cc16664cddd9ac7741284a2fb5b31f0e7)

6 years agoceph-volume: add tests for util.device.Device.used_by_ceph
Andrew Schoen [Wed, 3 Oct 2018 15:19:45 +0000 (10:19 -0500)]
ceph-volume: add tests for util.device.Device.used_by_ceph

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a28e6531e47d60d9eb7f62b67f122578daa2a683)

6 years agoceph-volume: update tests to account for filtered_devices in batch
Andrew Schoen [Tue, 2 Oct 2018 20:23:39 +0000 (15:23 -0500)]
ceph-volume: update tests to account for filtered_devices in batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 36396229d3b8d20862565b35d431ff22fa92cd1c)

6 years agoceph-volume: consider block and data devices used_by_ceph
Andrew Schoen [Tue, 2 Oct 2018 20:08:10 +0000 (15:08 -0500)]
ceph-volume: consider block and data devices used_by_ceph

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 67512530116c16d072c91867a259aeb429d32ff6)

6 years agoceph-volume: add filtered_devices and used_by_ceph to all batch reports
Andrew Schoen [Tue, 2 Oct 2018 14:48:27 +0000 (09:48 -0500)]
ceph-volume: add filtered_devices and used_by_ceph to all batch reports

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0718d2e2663c8e807dbb5171a58759c76ed92c08)

6 years agoceph-volume: pick strategy for batch with only the unused devices
Andrew Schoen [Thu, 27 Sep 2018 20:22:17 +0000 (15:22 -0500)]
ceph-volume: pick strategy for batch with only the unused devices

This will pick a strategy, filter out any devices already been used by
ceph and then pick a strategy again. If the strategy has changed the
call should error, if the strategy is the same proceed. If there are no
unused devices then the command is a noop.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 4529f2d6053b0b07583a3d7501f4e05e08cad385)

6 years agoceph-volume: adds a 'changed' key to lvm batch --report
Andrew Schoen [Thu, 27 Sep 2018 13:55:20 +0000 (08:55 -0500)]
ceph-volume: adds a 'changed' key to lvm batch --report

This will indicate if the command would result in any OSDs being created
or not. Other tooling can use that key for idempotency checks.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 10f1d577d4c4e66c77046fc3b274d6653af99586)

6 years agoceph-volume: adds used_by_ceph to filestore singletype batch report
Andrew Schoen [Wed, 26 Sep 2018 21:07:30 +0000 (16:07 -0500)]
ceph-volume: adds used_by_ceph to filestore singletype batch report

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 48d10c9ccf13ba7f42750b37d26aeb28f6b1c606)

6 years agoceph-volume: adds a used_by_ceph property to the Device class
Andrew Schoen [Wed, 26 Sep 2018 21:01:30 +0000 (16:01 -0500)]
ceph-volume: adds a used_by_ceph property to the Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9d49a3708e34bccd183d5c32d022c36d8b118b42)

6 years agoceph-volume: adds a lvs property to the Device class
Andrew Schoen [Wed, 26 Sep 2018 20:53:26 +0000 (15:53 -0500)]
ceph-volume: adds a lvs property to the Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3dae3247adc1b96ca688ec81cec03180d5943823)

6 years agoceph-volume: add vg_name to the Device class
Andrew Schoen [Wed, 26 Sep 2018 19:29:41 +0000 (14:29 -0500)]
ceph-volume: add vg_name to the Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 619810c0ef48c1f19db8821a33f9c48cc11a0161)

6 years agoMerge pull request #24176 from pdvian/wip-35836-mimic 24567/head
Neha Ojha [Wed, 10 Oct 2018 02:05:46 +0000 (19:05 -0700)]
Merge pull request #24176 from pdvian/wip-35836-mimic

mimic: mon: mgr options not parse propertly

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #24281 from pdvian/wip-35930-mimic
Neha Ojha [Wed, 10 Oct 2018 02:00:09 +0000 (19:00 -0700)]
Merge pull request #24281 from pdvian/wip-35930-mimic

mimic: mon/OSDMonitor: invalidate max_failed_since on cancel_report

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24113 from smithfarm/wip-35717-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:15:48 +0000 (09:15 -0700)]
Merge pull request #24113 from smithfarm/wip-35717-mimic

mimic: msg: challenging authorizer messages appear at debug_ms=0

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24241 from smithfarm/wip-36121-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:14:49 +0000 (09:14 -0700)]
Merge pull request #24241 from smithfarm/wip-36121-mimic

mimic: kernel_untar_build.sh: bison: command not found

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24298 from pdvian/wip-35959-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:14:22 +0000 (09:14 -0700)]
Merge pull request #24298 from pdvian/wip-35959-mimic

mimic: osd/ECBackend: don't get result code of subchunk-read overwritten

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24309 from neha-ojha/wip-36239-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:13:49 +0000 (09:13 -0700)]
Merge pull request #24309 from neha-ojha/wip-36239-mimic

mimic: osd/PrimaryLogPG: fix potential pg-log overtrimming

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24344 from pdvian/wip-35963-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:13:11 +0000 (09:13 -0700)]
Merge pull request #24344 from pdvian/wip-35963-mimic

mimic: osd/PG: avoid choose_acting picking want with > pool size items

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24345 from pdvian/wip-35964-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:12:48 +0000 (09:12 -0700)]
Merge pull request #24345 from pdvian/wip-35964-mimic

mimic: RADOS: probably missing clone location for async_recovery_targets

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24233 from liewegas/wip-35998-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:41:00 +0000 (14:41 -0700)]
Merge pull request #24233 from liewegas/wip-35998-mimic

mimic: mgr/DaemonServer: fix Session leak

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24339 from ifed01/wip-ifed-fix-repairer-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:19:25 +0000 (14:19 -0700)]
Merge pull request #24339 from ifed01/wip-ifed-fix-repairer-mimic

mimic: os/bluestore: fix bloom filter num entry miscalculation in repairer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24352 from ifed01/wip-ifed-bluestore-tool-fixes-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:18:49 +0000 (14:18 -0700)]
Merge pull request #24352 from ifed01/wip-ifed-bluestore-tool-fixes-mimic

mimic: ceph-bluestore-tool: fix set label functionality for specific keys

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #24386 from pdvian/wip-36127-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:18:12 +0000 (14:18 -0700)]
Merge pull request #24386 from pdvian/wip-36127-mimic

mimic: msg/async: clean up local buffers on dispatch

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #24420 from karunjosy/wip-36308-mimic
Nathan Cutler [Thu, 4 Oct 2018 09:39:52 +0000 (11:39 +0200)]
Merge pull request #24420 from karunjosy/wip-36308-mimic

mimic: doc: Typo error on cephfs/fuse/

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agodoc: Fix typo error on cephfs/fuse/ 24420/head
Karun Josy [Tue, 2 Oct 2018 11:53:25 +0000 (17:23 +0530)]
doc: Fix typo error on cephfs/fuse/

Fixes: http://tracker.ceph.com/issues/36180
Signed-off-by: Karun Josy <kjosy@redhat.com>
(cherry picked from commit d312cdbd36d9c0e76409feac1f3e289b247dc2dc)

6 years agoMerge pull request #24002 from pdvian/wip-35705-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:21:28 +0000 (08:21 -0700)]
Merge pull request #24002 from pdvian/wip-35705-mimic

mimic:rbd: fix error import when the input is a pipe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24122 from pdvian/wip-35712-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:21:04 +0000 (08:21 -0700)]
Merge pull request #24122 from pdvian/wip-35712-mimic

mimic: librbd: journaling unable request can not be sent to remote lock owner

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24137 from pdvian/wip-35714-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:20:30 +0000 (08:20 -0700)]
Merge pull request #24137 from pdvian/wip-35714-mimic

mimic: librbd: ensure exclusive lock acquired when removing sync point snaps…

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24240 from smithfarm/wip-35077-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:19:59 +0000 (08:19 -0700)]
Merge pull request #24240 from smithfarm/wip-35077-mimic

mimic: qa: install dependencies for rbd_workunit_kernel_untar_build

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
6 years agoMerge pull request #24244 from dillaman/wip-36117-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:19:22 +0000 (08:19 -0700)]
Merge pull request #24244 from dillaman/wip-36117-mimic

mimic: test/librbd: not valid to have different parents between image snapshots

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24321 from smithfarm/wip-36120-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:17:43 +0000 (08:17 -0700)]
Merge pull request #24321 from smithfarm/wip-36120-mimic

mimic: [rbd-mirror] failed assertion when updating mirror status

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24356 from dillaman/wip-35957-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:16:36 +0000 (08:16 -0700)]
Merge pull request #24356 from dillaman/wip-35957-mimic

mimic: librbd: fixed assert when flattening clone with zero overlap

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24381 from alfredodeza/mimic-rm36247
Andrew Schoen [Wed, 3 Oct 2018 15:00:22 +0000 (10:00 -0500)]
Merge pull request #24381 from alfredodeza/mimic-rm36247

mimic ceph-volume: skip processing devices that don't exist when scanning system disks

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agomsg/async: clean up local buffers on dispatch 24386/head
Greg Farnum [Fri, 14 Sep 2018 17:58:49 +0000 (10:58 -0700)]
msg/async: clean up local buffers on dispatch

The AsyncConnection keeps local (member variable) bufferlists of incoming
messages before they're placed into the Message's front/data/middle buffers.
Previously these were reset only when a new Message is being received, which
means in steady state we store a full Message for every Connection even if
it's inactive!

Instead we obviously want to drop our local references to Message state
once it's been dispatched, so that it can go away.

Fixes: http://tracker.ceph.com/issues/35987
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 47ed036753223c44c7bf66c64d4a4adfe7267c0a)

6 years agoceph-volume util.disk when there are no devices mapped, skip to the next one 24381/head
Alfredo Deza [Tue, 2 Oct 2018 15:18:44 +0000 (11:18 -0400)]
ceph-volume util.disk when there are no devices mapped, skip to the next one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9b0f472abadde26fce2a603fca5c466ebb770d4a)

6 years agoceph-volume tests.util verify devices that don't exist don't break get_devices
Alfredo Deza [Tue, 2 Oct 2018 15:18:22 +0000 (11:18 -0400)]
ceph-volume tests.util verify devices that don't exist don't break get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0176c1965e6ed532b38dfee03af0e9e3a85149d6)

6 years agoMerge pull request #24357 from alfredodeza/mimic-rm36249
Andrew Schoen [Tue, 2 Oct 2018 11:31:50 +0000 (06:31 -0500)]
Merge pull request #24357 from alfredodeza/mimic-rm36249

mimic ceph-volume:  activate option --auto-detect-objectstore respects --no-systemd

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume lvm.activate pass no_systemd args when detecting objectstore 24357/head
Alfredo Deza [Mon, 1 Oct 2018 15:54:19 +0000 (11:54 -0400)]
ceph-volume lvm.activate pass no_systemd args when detecting objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 29de6820a9a7108c8de41036add9332f9b910359)

6 years agoceph-volume tests.lvm verify activate works with --auto-detect-objectstore
Alfredo Deza [Mon, 1 Oct 2018 15:53:37 +0000 (11:53 -0400)]
ceph-volume tests.lvm verify activate works with --auto-detect-objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 60dbeb6184797ccffeaeea2e85ef2d7fb4a1c54c)

6 years agolibrbd: fixed assert when flattening clone with zero overlap 24356/head
Jason Dillaman [Wed, 12 Sep 2018 01:52:05 +0000 (21:52 -0400)]
librbd: fixed assert when flattening clone with zero overlap

Fixes: http://tracker.ceph.com/issues/35702
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8e9035443322f8cbdb8c7aff5da0d1c7599a2592)

Conflicts:
src/librbd/Operations.cc: trivial resolution
src/test/librbd/test_librbd.cc: trivial resolution

6 years agoMerge pull request #24334 from alfredodeza/mimic-rm36098
Andrew Schoen [Mon, 1 Oct 2018 14:54:23 +0000 (09:54 -0500)]
Merge pull request #24334 from alfredodeza/mimic-rm36098

mimic ceph-volume:  lsblk can fail to find PARTLABEL, must fallback to blkid

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agotools/ceph-bluestore-tool: update size label on bluefs expand 24351/head 24352/head
Igor Fedotov [Thu, 24 May 2018 16:05:48 +0000 (19:05 +0300)]
tools/ceph-bluestore-tool: update size label on bluefs expand

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 71c3b58da4e7ced3422bce2b1da0e3fa9331530b)

6 years agotools/ceph-bluestore-tool: add support for proper setting size, btime,
Igor Fedotov [Thu, 24 May 2018 13:32:01 +0000 (16:32 +0300)]
tools/ceph-bluestore-tool: add support for proper setting size, btime,
description, osd_uuid fields

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ffac450da5d6e09cf14b8363b35f21819b48f38b)

6 years agotool/ceph-bluestore-tool: avoid mon/config access when calling global_init
Igor Fedotov [Fri, 18 May 2018 14:47:29 +0000 (17:47 +0300)]
tool/ceph-bluestore-tool: avoid mon/config access when calling global_init
This caused a permanent stuck on tool's startup.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fa65e7bf65bea514b66f56053d5cbf2c16045f4c)

6 years agoosd/PrimaryLogPG: is_degraded_on_async_recovery_target - restrict to async_recovery_t... 24345/head
xie xingguo [Mon, 3 Sep 2018 04:05:14 +0000 (12:05 +0800)]
osd/PrimaryLogPG: is_degraded_on_async_recovery_target - restrict to async_recovery_targets only

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 6923a4371b6757c7a2a4dfb6320e7190ce61c645)

6 years agoosd/PrimaryLogPG: update missing_loc more carefully
xie xingguo [Tue, 14 Aug 2018 07:30:37 +0000 (15:30 +0800)]
osd/PrimaryLogPG: update missing_loc more carefully

There could be clones and hence we should update missing_loc
based on log entry itself.

Also I think the **have** field of missing_loc is not quite accurate here
but should be fine.

Fixes: http://tracker.ceph.com/issues/35546
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit b09dedf2e9f37ae117a909796727adea71e32422)

6 years agoosd/PG: avoid choose_acting picking want with > pool size items 24344/head
Sage Weil [Tue, 11 Sep 2018 16:12:22 +0000 (11:12 -0500)]
osd/PG: avoid choose_acting picking want with > pool size items

If the pool size recently changed, we might see an up that is larger than
the pool size.  Or, we might already have selected a primary that is not
part of up, and then add all up osds and end up with a want that is too
big.

Fixes: http://tracker.ceph.com/issues/35924
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c3e2990c39952a1fc4593076c5b2f0dea6cdcdd3)

6 years agoceph-volume simple.scan better detection for ceph-disk data devices 24334/head
Alfredo Deza [Thu, 27 Sep 2018 20:17:29 +0000 (16:17 -0400)]
ceph-volume simple.scan better detection for ceph-disk data devices

Uses the new ``CephDiskDevice`` to look into PARTLABEL from both lsblk
(the default) falling back to blkid, which in some cases has the right
value when lsblk fails.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c502e5f64aa4f0d832c14680e1731e58d8a3770b)

6 years agoceph-volume tests.util verify CephDiskDevice behavior
Alfredo Deza [Thu, 27 Sep 2018 20:15:26 +0000 (16:15 -0400)]
ceph-volume tests.util verify CephDiskDevice behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f072eb3f940a6714089b37a6fbacd549420a454)

6 years agoceph-volume tests.conftest add blkid patching to device_info fixture
Alfredo Deza [Thu, 27 Sep 2018 20:15:00 +0000 (16:15 -0400)]
ceph-volume tests.conftest add blkid patching to device_info fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5bfc8c26e639dba955f83f53893ce311e6e047f1)

6 years agoceph-volume util.device add a ceph-disk device object to detect types
Alfredo Deza [Thu, 27 Sep 2018 20:14:28 +0000 (16:14 -0400)]
ceph-volume util.device add a ceph-disk device object to detect types

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 200e6bfd934b34aa8175330aef32c51505ec3392)

6 years agoceph-volume tests.util verify new blkid API calls
Alfredo Deza [Tue, 25 Sep 2018 19:27:47 +0000 (15:27 -0400)]
ceph-volume tests.util verify new blkid API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ead3c7e7d19635a741a9da6fc330ec12a68b3d77)

6 years agoceph-volume util.disk expand blkid's API calls
Alfredo Deza [Tue, 25 Sep 2018 19:25:13 +0000 (15:25 -0400)]
ceph-volume util.disk expand blkid's API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dbde6a5b611fdd4bcd942df5dbab94ab5076fdff)

6 years agoMerge pull request #24337 from alfredodeza/mimic-rm36251
Andrew Schoen [Fri, 28 Sep 2018 19:19:40 +0000 (14:19 -0500)]
Merge pull request #24337 from alfredodeza/mimic-rm36251

mimic ceph-volume add new ceph-handlers role from ceph-ansible

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.functional add new ceph-handlers role from ceph-ansible 24337/head
Alfredo Deza [Fri, 28 Sep 2018 16:40:09 +0000 (12:40 -0400)]
ceph-volume tests.functional add new ceph-handlers role from ceph-ansible

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 956dc5e65c6e35e27ab6524d4fb9fdd05d616cf6)

6 years agoos/bluestore: fix bloom filter num entry miscalculation in the repairer 24339/head
Igor Fedotov [Thu, 13 Sep 2018 13:22:39 +0000 (16:22 +0300)]
os/bluestore: fix bloom filter num entry miscalculation in the repairer

This could  cause an assertion due to an access to uninitialized bloom
filter. This happened  when detecting errors involving physical extents close to the end of a large partition.
E.g. it was observed in http://tracker.ceph.com/issues/25001 while trying to repair an OSD.

Fixes: https://tracker.ceph.com/issues/35971
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 50919813a15d9c29c182062f88885890c55eb76e)

6 years agoMerge pull request #24306 from alfredodeza/mimic-batch-report-fix
Andrew Schoen [Fri, 28 Sep 2018 14:52:33 +0000 (09:52 -0500)]
Merge pull request #24306 from alfredodeza/mimic-batch-report-fix

mimic ceph-volume: fix journal and filestore data size in `lvm batch --report`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agorbd-mirror: instantiate the status formatter before changing state 24321/head
Jason Dillaman [Wed, 19 Sep 2018 18:24:31 +0000 (14:24 -0400)]
rbd-mirror: instantiate the status formatter before changing state

This will avoid a possible race between pre-queued status updates
firing between the time the state has been changed and the formatter
has been instantiated.

Fixes: http://tracker.ceph.com/issues/36084
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit bbdc545ec1602f928c89c85d61c975abdf67be5b)

6 years agoosd/PrimaryLogPG: fix potential pg-log overtrimming 24309/head
xie xingguo [Mon, 30 Jul 2018 10:56:56 +0000 (18:56 +0800)]
osd/PrimaryLogPG: fix potential pg-log overtrimming

In https://github.com/ceph/ceph/pull/21580 I set a trap to catch some wired
and random segmentfaults and in a recent QA run I was able to observe it was
successfully triggered by one of the test case, see:

```
http://qa-proxy.ceph.com/teuthology/xxg-2018-07-30_05:25:06-rados-wip-hb-peers-distro-basic-smithi/2837916/teuthology.log
```

The root cause is that there might be holes on log versions, thus the
approx_size() method should (almost) always overestimate the actual number of log entries.
As a result, we might be at the risk of overtrimming log entries.

https://github.com/ceph/ceph/pull/18338 reveals a probably easier way
to fix the above problem but unfortunately it also can cause big performance regression
and hence comes this pr..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3654d56985c67d15506fa37b56ef5b0c04e01a65)

Conflicts:
src/osd/PrimaryLogPG.cc: trivial resolution

6 years agoceph-volume: only run ceph-common once during tests 24306/head
Andrew Schoen [Tue, 25 Sep 2018 17:09:15 +0000 (12:09 -0500)]
ceph-volume: only run ceph-common once during tests

Using import_role we can have just one play and avoid rerunning
ceph-common over and over again in daemon specific plays.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a81386123e66e2d61391371cec4485b3148ff5a7)

6 years agoceph-volume: fail fast during tests
Andrew Schoen [Mon, 24 Sep 2018 21:41:37 +0000 (16:41 -0500)]
ceph-volume: fail fast during tests

This also rsyncs the ceph-volume code to the testing vms before
a ceph.conf is generated because ceph-volume is needed now for this
to figure out the number of osds when you're using 'lvm batch'.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit eed244473d11a153cd7a37e0e9bfaf804a20db89)

6 years agoceph-volume: when using batch --report return journal/data size as ints
Andrew Schoen [Mon, 24 Sep 2018 21:37:35 +0000 (16:37 -0500)]
ceph-volume: when using batch --report return journal/data size as ints

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a5e8bc97e70195cee54c4b66d10684d6cd3f5e38)

6 years agoMerge pull request #24279 from alfredodeza/mimic-rm27210
Andrew Schoen [Thu, 27 Sep 2018 16:10:29 +0000 (11:10 -0500)]
Merge pull request #24279 from alfredodeza/mimic-rm27210

mimic ceph-volume custom cluster names fail on filestore trigger

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24210 from pdvian/wip-35855-mimic
Nathan Cutler [Thu, 27 Sep 2018 14:30:21 +0000 (16:30 +0200)]
Merge pull request #24210 from pdvian/wip-35855-mimic

mimic : doc: remove deprecated 'scrubq' from ceph(8)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoosd/ECBackend: don't get result code of subchunk-read overwritten 24298/head
songweibin [Sat, 1 Sep 2018 08:42:55 +0000 (16:42 +0800)]
osd/ECBackend: don't get result code of subchunk-read overwritten

Fixes: http://tracker.ceph.com/issues/21769
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 66bfaaca20e1a4e61e523aa260d60ce368f66df2)

Conflicts:
src/osd/ECBackend.cc : Resolved in handle_sub_read

6 years agoMerge pull request #24254 from yuriw/wip-yuriw-p2p-mimic
Yuri Weinstein [Wed, 26 Sep 2018 15:38:13 +0000 (08:38 -0700)]
Merge pull request #24254 from yuriw/wip-yuriw-p2p-mimic

qa/tests: added v13.2.2 to the mix

6 years agoMerge pull request #24258 from pdvian/wip-35925-mimic
Lenz Grimmer [Wed, 26 Sep 2018 07:19:36 +0000 (09:19 +0200)]
Merge pull request #24258 from pdvian/wip-35925-mimic

mimic: mgr/dashboard: Progress bar does not stop in TableKeyValueComponent

Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomon/OSDMonitor: invalidate max_failed_since on cancel_report 24281/head
xie xingguo [Thu, 6 Sep 2018 02:44:26 +0000 (10:44 +0800)]
mon/OSDMonitor: invalidate max_failed_since on cancel_report

max_failed_since might reference the very failure-report which is to be
cancelled. We can simply invalidate it here to make **get_failed_since()**
recalculate if necessary.

Fixes: http://tracker.ceph.com/issues/35860
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 91a2d408a9349db08e916e2145040df8b1f17e14)

6 years agoceph-volume tests/functional update dmcrypt test playbooks to use --cluster 24279/head
Alfredo Deza [Mon, 24 Sep 2018 18:44:31 +0000 (14:44 -0400)]
ceph-volume tests/functional update dmcrypt test playbooks to use --cluster

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 96e29adf72804cb4fca62c04416b8fe2d23587bb)

6 years agoceph-volume tests/functional/batch update tests to use 'test' cluster name
Alfredo Deza [Mon, 24 Sep 2018 16:52:50 +0000 (12:52 -0400)]
ceph-volume tests/functional/batch update tests to use 'test' cluster name

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5f72d438da94323c0b04f261b1a60b13eaf03966)

6 years agoceph-volume tests/functional/lvm update tests to use 'test' cluster name
Alfredo Deza [Mon, 24 Sep 2018 16:52:33 +0000 (12:52 -0400)]
ceph-volume tests/functional/lvm update tests to use 'test' cluster name

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7b0d2dc66968f81192f152fa166fbc2cb4dcb438)

6 years agoceph-volume tests/functional update playbooks to use --cluster
Alfredo Deza [Mon, 24 Sep 2018 16:51:47 +0000 (12:51 -0400)]
ceph-volume tests/functional update playbooks to use --cluster

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ba10b6820fd7f352ed6ba78246a6976c54783cbd)

6 years agoceph-volume main move out the load_ceph_conf_path to the configuration module
Alfredo Deza [Mon, 24 Sep 2018 16:27:04 +0000 (12:27 -0400)]
ceph-volume main move out the load_ceph_conf_path to the configuration module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 532908099eb63bd805db8aecae1ef12301076410)

6 years agoceph-volume tests.devices patch configuration.load to avoid raising non-existing...
Alfredo Deza [Mon, 24 Sep 2018 14:34:55 +0000 (10:34 -0400)]
ceph-volume tests.devices patch configuration.load to avoid raising non-existing ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe80e5399e9dd1f6b9f94404d732a76f961d9eb4)

6 years agoceph-volume tests reset the cluster name after every unit test
Alfredo Deza [Mon, 24 Sep 2018 14:29:12 +0000 (10:29 -0400)]
ceph-volume tests reset the cluster name after every unit test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 877ee1a27e0651355812e27fbc6ac0a87ea56a68)

6 years agoceph-volume tests.functional add custom cluster name to centos7-filestore-create
Alfredo Deza [Wed, 19 Sep 2018 17:23:19 +0000 (13:23 -0400)]
ceph-volume tests.functional add custom cluster name to centos7-filestore-create

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e61848dc97b638c6b8e60f0896d325f436142864)

6 years agoceph-volume lvm.activate load the config always by reading the cluster name from...
Alfredo Deza [Wed, 19 Sep 2018 16:04:41 +0000 (12:04 -0400)]
ceph-volume lvm.activate load the config always by reading the cluster name from the LV

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a88cbfc73a258b51b20e060e269234962444ce63)

6 years agoceph-volume configuration be able to set conf.ceph
Alfredo Deza [Wed, 19 Sep 2018 16:04:17 +0000 (12:04 -0400)]
ceph-volume configuration be able to set conf.ceph

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4f8226f359d94a86a39492ed653d9a9e5d8dd8f9)

6 years agomgr/dashboard: Progress bar does not stop in TableKeyValueComponent 24258/head
Volker Theile [Mon, 10 Sep 2018 15:27:58 +0000 (17:27 +0200)]
mgr/dashboard: Progress bar does not stop in TableKeyValueComponent

Fixes: https://tracker.ceph.com/issues/35907
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7243d9b67dc8c15283e30f889eb4d3fd00a5701a)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-key-value/table-key-value.component.ts :
: Resolved in TableKeyValueComponent

6 years agoqa/tests: added v13.2.2 to the mix 24254/head
Yuri Weinstein [Mon, 24 Sep 2018 20:28:38 +0000 (13:28 -0700)]
qa/tests: added v13.2.2 to the mix

Fixes: http://tracker.ceph.com/issues/36169
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years ago13.2.2 v13.2.2
Jenkins Build Slave User [Mon, 24 Sep 2018 17:22:30 +0000 (17:22 +0000)]
13.2.2

6 years agotest/librbd: not valid to have different parents between image snapshots 24244/head
Jason Dillaman [Tue, 4 Sep 2018 12:31:07 +0000 (08:31 -0400)]
test/librbd: not valid to have different parents between image snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3b8f4f136b52d591dc39e6c4f8bd59c1e63c158a)

Fixes: http://tracker.ceph.com/issues/36117
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoqa: install build dependencies for cfuse_workunit_kernel_untar_build.yaml 24241/head
Neha Ojha [Tue, 18 Sep 2018 22:40:50 +0000 (15:40 -0700)]
qa: install build dependencies for cfuse_workunit_kernel_untar_build.yaml

Fixes: https://tracker.ceph.com/issues/36076
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 38ef3da8d27e24576193cbf3f9238f2c5b586c09)

6 years agoqa: rbd_workunit_kernel_untar_build: install build dependencies 24240/head
Ilya Dryomov [Fri, 31 Aug 2018 11:30:30 +0000 (13:30 +0200)]
qa: rbd_workunit_kernel_untar_build: install build dependencies

Commit f0fe0936e64d ("qa: use recent kernel to kernel build testing")
bumped the kernel to 4.17.

Fixes: http://tracker.ceph.com/issues/35074
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a4df8c3562602acd996940cd4d9ae5384a085302)

6 years agomgr/DaemonServer: fix Session leak 24233/head
Sage Weil [Sat, 22 Sep 2018 19:27:56 +0000 (14:27 -0500)]
mgr/DaemonServer: fix Session leak

This bug does not exist in master.

Fixes: https://tracker.ceph.com/issues/35998
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #24208 from alfredodeza/mimic-rm36088
Andrew Schoen [Fri, 21 Sep 2018 14:01:42 +0000 (09:01 -0500)]
Merge pull request #24208 from alfredodeza/mimic-rm36088

mimic ceph-volume  batch: allow journal+block.db sizing on the CLI

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24205 from alfredodeza/mimic-rm24794
Andrew Schoen [Fri, 21 Sep 2018 14:00:33 +0000 (09:00 -0500)]
Merge pull request #24205 from alfredodeza/mimic-rm24794

mimic ceph-volume: earlier detection for --journal and --filestore flag requirements

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agodoc: remove deprecated 'scrubq' from ceph(8) 24210/head
Ruben Kerkhof [Thu, 6 Sep 2018 12:17:23 +0000 (14:17 +0200)]
doc: remove deprecated 'scrubq' from ceph(8)

The option was removed in a30cbe9adea34782be8699c010d4281d245feb6b

Fixes: http://tracker.ceph.com/issues/35813
Signed-off-by: Ruben Kerkhof <ruben@rubenkerkhof.com>
(cherry picked from commit d05cd1a3e612f52eab9bde42c184022205f65384)