]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #24622 from dillaman/wip-36430-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:12:47 +0000 (16:12 -0400)]
Merge pull request #24622 from dillaman/wip-36430-mimic

mimic: qa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
6 years agoMerge pull request #24551 from pdvian/wip-36276-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:11:38 +0000 (16:11 -0400)]
Merge pull request #24551 from pdvian/wip-36276-mimic

mimic: test/rbd_mirror: race in WaitingOnLeaderReleaseLeader

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24647 from liewegas/wip-pr-23273-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:09:47 +0000 (16:09 -0400)]
Merge pull request #24647 from liewegas/wip-pr-23273-mimic

mimic: os/bluestore: handle spurious read errors

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #24711 from smithfarm/wip-36238-mimic
Yuri Weinstein [Mon, 29 Oct 2018 20:07:53 +0000 (16:07 -0400)]
Merge pull request #24711 from smithfarm/wip-36238-mimic

mimic: [test] periodic seg faults within unittest_librbd

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24782 from alfredodeza/mimic-evelu-ceph-volume-choose_disk
Andrew Schoen [Mon, 29 Oct 2018 15:44:30 +0000 (10:44 -0500)]
Merge pull request #24782 from alfredodeza/mimic-evelu-ceph-volume-choose_disk

mimic Additional work on ceph-volume to add some choose_disk capabilities

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph_volume: Checking device validity at init time 24782/head
Erwan Velu [Wed, 10 Oct 2018 18:26:01 +0000 (20:26 +0200)]
ceph_volume: Checking device validity at init time

When initializing the Device structure, it have to run is_valid() to
ensure the data structures (_is_valid & rejected_reasons) to be
populated accordingly to the device state.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit ef0ceef7c717ef615dc214299cb4a6b32c1edbe6)

6 years agoceph_volume: Rejecting locked devices
Erwan Velu [Tue, 9 Oct 2018 20:51:39 +0000 (22:51 +0200)]
ceph_volume: Rejecting locked devices

If we cannot open a block device in O_RDWR in exclusive mode, it means
someone is actually using it like a raw database or similar.

In that case, the device should be considered as unusable as OSDs will
not be in a position to use it.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit d5de9583ee6c4d9ccaab2b1146606a0ee316cc27)

6 years agoceph_volume: Reporting nr_requests
Erwan Velu [Tue, 9 Oct 2018 20:28:19 +0000 (22:28 +0200)]
ceph_volume: Reporting nr_requests

We are already reporting the rotational & scheduler of a disk device.
Reporting the nr_requests could be useful to get how many concurrent IOs
the device supports/reports.

That could help detecting badly detected/configured devices.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit e0ea3d475a85844afdcdf92cf638b8128c8e2d9e)

6 years agoceph_volume: Reporting firmware revision
Erwan Velu [Tue, 9 Oct 2018 20:26:28 +0000 (22:26 +0200)]
ceph_volume: Reporting firmware revision

We are already reporting model & vendor of a given disk, let's also
report the revision of the firmware. That is useful to filter-out some
known broken revisions.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 59720794079bef0ba7f8bc2d9dfc8aaddd210005)

6 years agoceph_volume: Rejecting Read-only devices
Erwan Velu [Tue, 9 Oct 2018 20:02:31 +0000 (22:02 +0200)]
ceph_volume: Rejecting Read-only devices

If a devices is said to be read-only, there is no chance we can actually
use it. So let's report it as unusable.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit f1a94350064263f27f67d2c715c5ca5922481a01)

6 years agoceph_volume: Adding Device.is_valid()
Erwan Velu [Tue, 9 Oct 2018 19:04:02 +0000 (21:04 +0200)]
ceph_volume: Adding Device.is_valid()

A block device can be filtered-out/ignored because it have features that
doesn't match Ceph's expectations.

As of today, the current code was rejected removable devices but it was
pretty hidden from the user, and implicit in the get_devices() function.

This patch is creating a new is_valid() function to perform all the
rejection tests and returns if this device can be used in the Ceph
context or not.

If is_valid() is returning False, the 'rejected_reasons' list reports all
the reasons why that devices got rejected.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit d8fdf0b7532cdffcae56511c377679e51841caec)

6 years agoMerge pull request #24449 from alfredodeza/mimic-rm24795
Andrew Schoen [Fri, 26 Oct 2018 20:00:08 +0000 (15:00 -0500)]
Merge pull request #24449 from alfredodeza/mimic-rm24795

mimic ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24760 from ceph/backport-mimic-24587
Alfredo Deza [Fri, 26 Oct 2018 13:46:57 +0000 (09:46 -0400)]
Merge pull request #24760 from ceph/backport-mimic-24587

mimic: ceph-volume: adds a --prepare flag to `lvm batch`

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume lvm.prepare update help to indicate partitions are needed, not devices 24449/head
Alfredo Deza [Wed, 3 Oct 2018 12:11:58 +0000 (08:11 -0400)]
ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d31dd95b95445825f5e6669dd4ecb3118b09fdcf)

6 years agoceph-volume: update man page for batch --prepare 24760/head
Andrew Schoen [Thu, 25 Oct 2018 12:51:41 +0000 (07:51 -0500)]
ceph-volume: update man page for batch --prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit faa3aed4ad4df2dbdd853235e364d6e3b9b176b6)

6 years agoceph-volume: docs for --prepare flag of lvm batch
Andrew Schoen [Mon, 15 Oct 2018 15:38:43 +0000 (10:38 -0500)]
ceph-volume: docs for --prepare flag of lvm batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b4825e1bd7b75a0a53ee5678230b262286293d75)

6 years agoceph-volume: when --prepare is added to batch the OSDs are only prepared
Andrew Schoen [Mon, 15 Oct 2018 15:35:12 +0000 (10:35 -0500)]
ceph-volume: when --prepare is added to batch the OSDs are only prepared

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit abd238dc1f33252a43adc8b86efb3d3b4a076964)

6 years agoceph-volume: adds a --prepare flag to ceph-volume lvm batch
Andrew Schoen [Mon, 15 Oct 2018 14:33:30 +0000 (09:33 -0500)]
ceph-volume: adds a --prepare flag to ceph-volume lvm batch

This flag will only prepare the OSDs, not activate them. This is useful
in our containerized ceph solution.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 4368ca7d48f4ce35f67854e8b0ca2be9babc5371)

6 years agoMerge pull request #24753 from alfredodeza/mimic-rm36386
Andrew Schoen [Thu, 25 Oct 2018 16:10:47 +0000 (11:10 -0500)]
Merge pull request #24753 from alfredodeza/mimic-rm36386

mimic ceph-volume remove version reporting from help menu

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume remove version reporting from help menu 24753/head
Alfredo Deza [Wed, 10 Oct 2018 19:35:31 +0000 (15:35 -0400)]
ceph-volume remove version reporting from help menu

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 20db7bf585f6433c6705396264005a4227cbf2e3)

6 years agoMerge pull request #24740 from alfredodeza/mimic-rm36492
Alfredo Deza [Thu, 25 Oct 2018 12:04:34 +0000 (08:04 -0400)]
Merge pull request #24740 from alfredodeza/mimic-rm36492

mimic ceph-volume: do not send (lvm) stderr/stdout to the terminal, use the logfile

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.api update monkeypatching to use **kw 24740/head
Alfredo Deza [Wed, 24 Oct 2018 14:55:06 +0000 (10:55 -0400)]
ceph-volume tests.api update monkeypatching to use **kw

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d6bcda75f846bc7c44c7b16b3687a65fed7cafc)

6 years agoceph-volume tests allow **kw when monkeypatching
Alfredo Deza [Wed, 24 Oct 2018 14:54:48 +0000 (10:54 -0400)]
ceph-volume tests allow **kw when monkeypatching

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 04e51c956eadcbd38f39ec3dd4b78009d2cd4a70)

6 years agoceph-volume api.lvm: do not spit out errors on the terminal from LVM
Alfredo Deza [Wed, 24 Oct 2018 14:54:18 +0000 (10:54 -0400)]
ceph-volume api.lvm: do not spit out errors on the terminal from LVM

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 581eed36ea0e7b5c5e41f8b83350403bcf73a31a)

6 years agoMerge pull request #24588 from ceph/backport-mimic-24404
Alfredo Deza [Wed, 24 Oct 2018 21:22:50 +0000 (17:22 -0400)]
Merge pull request #24588 from ceph/backport-mimic-24404

mimic: ceph-volume: make `lvm batch` idempotent

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agolibrbd: object copy state machine might dereference a deleted object 24711/head
Jason Dillaman [Wed, 26 Sep 2018 18:39:25 +0000 (14:39 -0400)]
librbd: object copy state machine might dereference a deleted object

Fixes: http://tracker.ceph.com/issues/36220
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1a9c3be2ceffcb981910de429a7da0fbec43e643)

6 years agoMerge pull request #24441 from joscollin/wip-36313-mimic
Nathan Cutler [Tue, 23 Oct 2018 06:26:55 +0000 (08:26 +0200)]
Merge pull request #24441 from joscollin/wip-36313-mimic

mimic: doc: fix broken fstab url in cephfs/fuse

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #24017 from dzafman/wip-35909
Yuri Weinstein [Mon, 22 Oct 2018 15:40:24 +0000 (11:40 -0400)]
Merge pull request #24017 from dzafman/wip-35909

mimic: test: Use a grep pattern that works across releases

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24037 from smithfarm/wip-21057-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:39:41 +0000 (11:39 -0400)]
Merge pull request #24037 from smithfarm/wip-21057-mimic

mimic: debian/control: require fuse for ceph-fuse

Reviewed-by: James Page <james.page@ubuntu.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24480 from pdvian/wip-36146-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:37:09 +0000 (11:37 -0400)]
Merge pull request #24480 from pdvian/wip-36146-mimic

mimic: fsck: cid is improperly matched to oid

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24581 from smithfarm/wip-36437-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:55 +0000 (11:34 -0400)]
Merge pull request #24581 from smithfarm/wip-36437-mimic

mimic: ec: src/common/interval_map.h: 161: FAILED assert(len > 0)

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24600 from pdvian/wip-36296-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:02 +0000 (11:34 -0400)]
Merge pull request #24600 from pdvian/wip-36296-mimic

mimic : osdc/Objecter: possible race condition with connection reset

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24601 from pdvian/wip-36298-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:32:42 +0000 (11:32 -0400)]
Merge pull request #24601 from pdvian/wip-36298-mimic

mimic: osd: add creating to pg_string_state

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoqa/tasks/workunit: use suite branch/SHA1 when cloning workunits 24622/head
Jason Dillaman [Fri, 12 Oct 2018 14:02:35 +0000 (10:02 -0400)]
qa/tasks/workunit: use suite branch/SHA1 when cloning workunits

Right now it's using the Ceph branch/SHA1 but it's using the suite
Git URL.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1e3dc02604cf7c0d3991dfd9fe2596ea34e80bad)

6 years agoqa/tasks: qemu task now uses a relative path in suite repo for test
Jason Dillaman [Thu, 11 Oct 2018 20:21:35 +0000 (16:21 -0400)]
qa/tasks: qemu task now uses a relative path in suite repo for test

This makes it easier to re-run tests against a suite branch without
requiring a full ceph-ci build and repo.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c1f950236b2f1ba3b588fc638ebf8cb785c36e08)

6 years agoqa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh
Jason Dillaman [Thu, 11 Oct 2018 19:17:25 +0000 (15:17 -0400)]
qa/workunits: replace 'realpath' with 'readlink -f' in fsstress.sh

Fixes: http://tracker.ceph.com/issues/36409
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ddb7e5eb80ebb78beff6dfb25042f0016a474491)

6 years agoqa/tasks/cram: tasks now must live in the repository
Ilya Dryomov [Thu, 6 Sep 2018 14:07:08 +0000 (16:07 +0200)]
qa/tasks/cram: tasks now must live in the repository

Commit 0d8887652d53 ("qa/tasks/cram: use suite_repo repository for all
cram jobs") removed hardcoded git.ceph.com links, but as it turned out
it is still used for nightlies.  There is no good way to accommodate
the different URL schemes, so let's get rid of URLs altogether.

Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 592f566b4e270d3833ba896dabb193d5241dd5de)

6 years agoqa/tasks/workunit: factor out overrides and refspec logic
Ilya Dryomov [Thu, 6 Sep 2018 14:53:25 +0000 (16:53 +0200)]
qa/tasks/workunit: factor out overrides and refspec logic

Allow for reuse in the cram task.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit e1c89b51c80407fadbfee82d5d396cfcbd791aae)

6 years agoqa/tasks/cram: use suite_repo repository for all cram jobs
Ilya Dryomov [Mon, 3 Sep 2018 15:40:08 +0000 (17:40 +0200)]
qa/tasks/cram: use suite_repo repository for all cram jobs

Currently git.ceph.com is hardcoded for all cram jobs.  Testing
modifications is a pain: one needs to push to either ceph/ceph.git or
ceph/ceph-ci.git (depending on where the ceph branch is at, triggering
unnecessary builds in the latter case) and wait for the mirror to sync.
Runs scheduled against branches in developer's forks fail.

Move away from git.ceph.com to allow mixing branches and repositories,
similar to workunits.

Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0d8887652d5312f7059ab2bdb52c948aa01680b0)

6 years agoMerge pull request #24379 from smithfarm/wip-36136-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:51:33 +0000 (16:51 -0400)]
Merge pull request #24379 from smithfarm/wip-36136-mimic

mimic: mds: rctime may go back

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24384 from smithfarm/wip-32104-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:54 +0000 (16:50 -0400)]
Merge pull request #24384 from smithfarm/wip-32104-mimic

mimic: mds: allows client to create .. and . dirents

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #24385 from smithfarm/wip-36134-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:20 +0000 (16:50 -0400)]
Merge pull request #24385 from smithfarm/wip-36134-mimic

mimic: client: update ctime when modifying file content

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24408 from smithfarm/wip-36153-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:49:25 +0000 (16:49 -0400)]
Merge pull request #24408 from smithfarm/wip-36153-mimic

mimic: qa: fsstress workunit does not execute in parallel on same host without clobbering files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24411 from smithfarm/wip-36195-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:52 +0000 (16:48 -0400)]
Merge pull request #24411 from smithfarm/wip-36195-mimic

mimic: mds: internal op missing events time 'throttled', 'all_read', 'dispatched'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24438 from pdvian/wip-36156-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:28 +0000 (16:48 -0400)]
Merge pull request #24438 from pdvian/wip-36156-mimic

mimic: qa: extend timeout for SessionMap flush

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24439 from pdvian/wip-36197-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:52 +0000 (16:47 -0400)]
Merge pull request #24439 from pdvian/wip-36197-mimic

mimic: ceph-fuse: add to selinux profile

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24463 from pdvian/wip-36199-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:09 +0000 (16:47 -0400)]
Merge pull request #24463 from pdvian/wip-36199-mimic

mimic:mds: fix mds damaged due to unexpected journal length

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24464 from pdvian/wip-36205-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:46:30 +0000 (16:46 -0400)]
Merge pull request #24464 from pdvian/wip-36205-mimic

mimic: libcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24470 from smithfarm/wip-36203-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:45:53 +0000 (16:45 -0400)]
Merge pull request #24470 from smithfarm/wip-36203-mimic

mimic: ceph fs add_data_pool applies pool application metadata incorrectly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #23087 from pdvian/wip-24928-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:44:39 +0000 (16:44 -0400)]
Merge pull request #23087 from pdvian/wip-24928-mimic

mimic: qa: test_recovery_pool tries asok on wrong node

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23506 from ukernel/mimic-26858
Yuri Weinstein [Fri, 19 Oct 2018 20:44:14 +0000 (16:44 -0400)]
Merge pull request #23506 from ukernel/mimic-26858

mimic: mds: reset heartbeat map at potential time-consuming places

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23952 from ukernel/mimic-mds-export-size
Yuri Weinstein [Fri, 19 Oct 2018 20:43:47 +0000 (16:43 -0400)]
Merge pull request #23952 from ukernel/mimic-mds-export-size

mimic: mds: optimize the way how max export size is enforced

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24161 from pdvian/wip-35719-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:24 +0000 (16:43 -0400)]
Merge pull request #24161 from pdvian/wip-35719-mimic

mimic: mds: print is_laggy message once

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24187 from pdvian/wip-35841-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:02 +0000 (16:43 -0400)]
Merge pull request #24187 from pdvian/wip-35841-mimic

mimic: client: fix use-after-free in Client::link()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24269 from smithfarm/wip-36102-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:42:29 +0000 (16:42 -0400)]
Merge pull request #24269 from smithfarm/wip-36102-mimic

mimic: qa: remove knfs site from future releases

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24272 from batrick/i35858
Yuri Weinstein [Fri, 19 Oct 2018 20:41:34 +0000 (16:41 -0400)]
Merge pull request #24272 from batrick/i35858

mimic: mon: test if gid exists in pending for prepare_beacon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24295 from pdvian/wip-35934-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:40:56 +0000 (16:40 -0400)]
Merge pull request #24295 from pdvian/wip-35934-mimic

mimic: client:two ceph-fuse client, one can not list out files created by an…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24377 from smithfarm/wip-35940-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:40:23 +0000 (16:40 -0400)]
Merge pull request #24377 from smithfarm/wip-35940-mimic

mimic: client: statfs inode count odd

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoos/bluestore: handle spurious read errors 24647/head
Paul Emmerich [Thu, 26 Jul 2018 19:24:38 +0000 (21:24 +0200)]
os/bluestore: handle spurious read errors

Some kernels (4.9+) sometime fail to return data when reading
from a block device under memory pressure. This patch retries
the read if the checksum verification fails, tests show that
the first retried read succeeds in ~99.5% of the cases, so
3 attempts are made by default before giving up on the data.

Works-around: http://tracker.ceph.com/issues/22464
Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit cffcbc73aaaa874829d5fc9091af3042b887f9a7)

- conflict due to adjacent tests in store_test
- g_conf, not g_conf()

6 years agoMerge pull request #24319 from smithfarm/wip-35979-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:11:38 +0000 (16:11 -0400)]
Merge pull request #24319 from smithfarm/wip-35979-mimic

mimic: multisite: data sync error repo processing does not back off on empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23197 from pdvian/wip-25024-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:09:22 +0000 (16:09 -0400)]
Merge pull request #23197 from pdvian/wip-25024-mimic

mimic: qa/suites/rgw/verify/tasks/cls_rgw: test cls_rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #24071 from pdvian/wip-35708-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:06:48 +0000 (16:06 -0400)]
Merge pull request #24071 from pdvian/wip-35708-mimic

mimic: rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24101 from pdvian/wip-35710-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:05:48 +0000 (16:05 -0400)]
Merge pull request #24101 from pdvian/wip-35710-mimic

mimic: rgw: fix deadlock on RGWIndexCompletionManager::stop

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24189 from joke-lee/mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:05:17 +0000 (16:05 -0400)]
Merge pull request #24189 from joke-lee/mimic

mimic:rgw: return x-amz-version-id: null when delete obj in versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24235 from pdvian/wip-35857-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:04:52 +0000 (16:04 -0400)]
Merge pull request #24235 from pdvian/wip-35857-mimic

mimic: multisite: segfault on shutdown/realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24363 from pdvian/wip-36125-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:04:10 +0000 (16:04 -0400)]
Merge pull request #24363 from pdvian/wip-36125-mimic

mimic:rgw: fix chunked-encoding for chunks >1MiB

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24417 from pdvian/wip-36138-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:01:17 +0000 (16:01 -0400)]
Merge pull request #24417 from pdvian/wip-36138-mimic

mimic: rgw: async sync_object and remove_object does not access coroutine me…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24571 from joke-lee/mimic-list-RGWPostObj-when-enable-versioning
Yuri Weinstein [Wed, 17 Oct 2018 20:00:41 +0000 (16:00 -0400)]
Merge pull request #24571 from joke-lee/mimic-list-RGWPostObj-when-enable-versioning

mimic:rgw: list bucket can not show the object uploaded by RGWPostObj when enable bucket versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23026 from smithfarm/wip-24889-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:58:16 +0000 (15:58 -0400)]
Merge pull request #23026 from smithfarm/wip-24889-mimic

mimic: osd: crash in OpTracker::unregister_inflight_op via OSD::get_health_metrics

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #24116 from dzafman/wip-35992
Yuri Weinstein [Wed, 17 Oct 2018 19:57:22 +0000 (15:57 -0400)]
Merge pull request #24116 from dzafman/wip-35992

mimic: tools: ceph-objectstore-tool: Allow target level as first positional …

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #24401 from dillaman/wip-36144-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:56:50 +0000 (15:56 -0400)]
Merge pull request #24401 from dillaman/wip-36144-mimic

mimic: librbd: blacklisted client might not notice it lost the lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24573 from pdvian/wip-36292-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:56:08 +0000 (15:56 -0400)]
Merge pull request #24573 from pdvian/wip-36292-mimic

mimic: osd: Remove old bft= which has been superceded by backfill=

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agotest/objectstore: set pool for fsck test 24480/head
Kefu Chai [Fri, 21 Sep 2018 11:56:08 +0000 (19:56 +0800)]
test/objectstore: set pool for fsck test

since 0bd2546eaca72ed0122a9c2648df4bef05b0d5d2, we check the pool id
of object when performing fsck to ensure we are looking at the right
collection, but the test is still using the pool id set by the
constructor of hobject_t. so all objects we created in that test belong
to the POOL_META. while the collection is created with the pool id of
`555`. hence the test fails.

Fixes: http://tracker.ceph.com/issues/36099
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 8a814b2eb6f6d16233304e07b2adc5951f5fb312)

6 years agoosd: add creating to pg_string_state 24601/head
Dan van der Ster [Tue, 25 Sep 2018 08:39:37 +0000 (10:39 +0200)]
osd: add creating to pg_string_state

Fixes: http://tracker.ceph.com/issues/36174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit d38f6a11701ec788e4d384aa5b0ae65b8e57da64)

Conflicts:
src/osd/osd_types.cc : Resolved in pg_string_state

6 years agoosdc/Objecter: possible race condition with connection reset 24600/head
Jason Dillaman [Tue, 25 Sep 2018 18:18:00 +0000 (14:18 -0400)]
osdc/Objecter: possible race condition with connection reset

If the connection quickly fails before the private session reference
can be associated with the connection, the connection will remain
closed and any OSD ops against the session will remain stuck.

Fixes: http://tracker.ceph.com/issues/36183
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74ca33cb49d2c258324447b1ca366ed4e604202a)

6 years agoMerge pull request #24388 from pdvian/wip-36129-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:52:18 +0000 (16:52 -0400)]
Merge pull request #24388 from pdvian/wip-36129-mimic

mimic: rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24400 from smithfarm/wip-36201-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:53 +0000 (16:51 -0400)]
Merge pull request #24400 from smithfarm/wip-36201-mimic

mimic: multisite: intermittent test_bucket_index_log_trim failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24413 from dillaman/wip-36225-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:11 +0000 (16:51 -0400)]
Merge pull request #24413 from dillaman/wip-36225-mimic

mimic: librbd: object map improperly flagged as invalidated

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24414 from smithfarm/wip-36208-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:38 +0000 (16:50 -0400)]
Merge pull request #24414 from smithfarm/wip-36208-mimic

mimic: multisite: invalid read in RGWCloneMetaLogCoroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24418 from pdvian/wip-36140-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:13 +0000 (16:50 -0400)]
Merge pull request #24418 from pdvian/wip-36140-mimic

mimic: rgw: raise debug level on redundant data sync error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24436 from croit/mimic-36041
Yuri Weinstein [Mon, 15 Oct 2018 20:49:40 +0000 (16:49 -0400)]
Merge pull request #24436 from croit/mimic-36041

mimic: rgw/beast: drop privileges after binding ports

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24481 from pdvian/wip-36150-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:49:10 +0000 (16:49 -0400)]
Merge pull request #24481 from pdvian/wip-36150-mimic

mimic: crush/CrushWrapper: fix crush tree json dumper

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24516 from pdvian/wip-36211-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:38 +0000 (16:48 -0400)]
Merge pull request #24516 from pdvian/wip-36211-mimic

mimic: rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24518 from pdvian/wip-36213-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:13 +0000 (16:48 -0400)]
Merge pull request #24518 from pdvian/wip-36213-mimic

mimic:rgw: fix leak of curl handle on shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24526 from theanalyst/wip-mimic-36381
Yuri Weinstein [Mon, 15 Oct 2018 20:47:55 +0000 (16:47 -0400)]
Merge pull request #24526 from theanalyst/wip-mimic-36381

mimic: rgw: resharding produces invalid values of bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24536 from pdvian/wip-36216-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:47:13 +0000 (16:47 -0400)]
Merge pull request #24536 from pdvian/wip-36216-mimic

mimic: multisite: data full sync does not limit concurrent bucket sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24537 from pdvian/wip-36228-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:46:36 +0000 (16:46 -0400)]
Merge pull request #24537 from pdvian/wip-36228-mimic

mimic: qa: add test that builds example librados programs

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoceph-volume: extracts batch.filter_devices from Batch._get_strategy 24588/head
Andrew Schoen [Wed, 10 Oct 2018 19:28:29 +0000 (15:28 -0400)]
ceph-volume: extracts batch.filter_devices from Batch._get_strategy

This allows us to easily provide tests for that method.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit df7ef5383bb476ee020b898cd2e8fbce044fc07e)

6 years agoceph-volume: failing to get block db size from conf logs an exception
Andrew Schoen [Wed, 10 Oct 2018 18:05:25 +0000 (14:05 -0400)]
ceph-volume: failing to get block db size from conf logs an exception

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 172d4af9b97d8d64ce35dc95efb72d9f190bc170)

6 years agoceph-volume: when all devices are filtered exit gracefully
Andrew Schoen [Tue, 9 Oct 2018 18:05:54 +0000 (14:05 -0400)]
ceph-volume: when all devices are filtered exit gracefully

Even if all devices are filtered we want to return a 0 exit code and
make sure the json reporting still works.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit aa4fcd602f8b1a1b17bc12e59334b19508f97d6d)

6 years agoceph-volume: filter devices used by journals/block.db
Andrew Schoen [Mon, 8 Oct 2018 13:57:07 +0000 (09:57 -0400)]
ceph-volume: filter devices used by journals/block.db

If after filterering of data/block devices there are only
one device left it can not be used if it is an SSD and
has been used previously as a journal or block.db

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit fc9a10e54813a402ecd1d823ea5a33f85e8eb963)

6 years agoceph-volume: add rotational property to Device class
Andrew Schoen [Mon, 8 Oct 2018 13:39:05 +0000 (09:39 -0400)]
ceph-volume: add rotational property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7f6bfaaf0cf30a56481d6d67f2fa2a7e785cc344)

6 years agoceph-volume: add info about filtered devices to batch pretty reports
Andrew Schoen [Fri, 5 Oct 2018 21:18:48 +0000 (16:18 -0500)]
ceph-volume: add info about filtered devices to batch pretty reports

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 62426db799c54436c50b5751c8e72b3dbd710a2a)

6 years agoceph-volume: remove the used_by_ceph key in the json output
Andrew Schoen [Fri, 5 Oct 2018 15:45:35 +0000 (10:45 -0500)]
ceph-volume: remove the used_by_ceph key in the json output

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3b6d82afe7ee83283b25cefda00c7a59cf5144af)

6 years agoceph-volume: fix idempotency checks for lvm batch tests
Andrew Schoen [Fri, 5 Oct 2018 15:39:54 +0000 (10:39 -0500)]
ceph-volume: fix idempotency checks for lvm batch tests

The mixed type tests will change strategy after the idempotency test so
we need to handle that in test playbook.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1143482061da50910f4f883aafdbe0a0e4269f39)

6 years agoceph-volume: fix bluestore strategy json reporting and type
Andrew Schoen [Fri, 5 Oct 2018 15:38:11 +0000 (10:38 -0500)]
ceph-volume: fix bluestore strategy json reporting and type

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b3cf90604d31709996538e612b1767d6fd4da8b5)

6 years agoceph-volume: ignore failure to load ceph configuration for block.db size
Andrew Schoen [Thu, 4 Oct 2018 17:47:48 +0000 (12:47 -0500)]
ceph-volume: ignore failure to load ceph configuration for block.db size

If we fail to load a ceph configureation file when trying to get the
block.db size then just use defaults instead of throwing an error.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a7ee36ca92b6592b6b3e218252c6a4c30416591b)

6 years agoceph-volume: fix strategy comparison in 'lvm batch'
Andrew Schoen [Thu, 4 Oct 2018 16:54:06 +0000 (11:54 -0500)]
ceph-volume: fix strategy comparison in 'lvm batch'

This also fixes some small json reporting issues with the
filestore MixedType strategy

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1dd15025bfd52af9c31cd281f92d743b9ca0eeb8)

6 years agoceph-volume: raise a non zero exit code if strategy changes with batch
Andrew Schoen [Wed, 3 Oct 2018 20:01:08 +0000 (15:01 -0500)]
ceph-volume: raise a non zero exit code if strategy changes with batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9dfc00f8e10e10fbdc52c44259596c6a96a90edd)

6 years agoceph-volume: add functional tests to ensure lvm batch is idempotent
Andrew Schoen [Wed, 3 Oct 2018 17:13:27 +0000 (12:13 -0500)]
ceph-volume: add functional tests to ensure lvm batch is idempotent

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9752d03cc16664cddd9ac7741284a2fb5b31f0e7)