]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoceph_volume: Rejecting locked devices
Erwan Velu [Tue, 9 Oct 2018 20:51:39 +0000 (22:51 +0200)]
ceph_volume: Rejecting locked devices

If we cannot open a block device in O_RDWR in exclusive mode, it means
someone is actually using it like a raw database or similar.

In that case, the device should be considered as unusable as OSDs will
not be in a position to use it.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit d5de9583ee6c4d9ccaab2b1146606a0ee316cc27)

7 years agoceph_volume: Reporting nr_requests
Erwan Velu [Tue, 9 Oct 2018 20:28:19 +0000 (22:28 +0200)]
ceph_volume: Reporting nr_requests

We are already reporting the rotational & scheduler of a disk device.
Reporting the nr_requests could be useful to get how many concurrent IOs
the device supports/reports.

That could help detecting badly detected/configured devices.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit e0ea3d475a85844afdcdf92cf638b8128c8e2d9e)

7 years agoceph_volume: Reporting firmware revision
Erwan Velu [Tue, 9 Oct 2018 20:26:28 +0000 (22:26 +0200)]
ceph_volume: Reporting firmware revision

We are already reporting model & vendor of a given disk, let's also
report the revision of the firmware. That is useful to filter-out some
known broken revisions.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 59720794079bef0ba7f8bc2d9dfc8aaddd210005)

7 years agoceph_volume: Rejecting Read-only devices
Erwan Velu [Tue, 9 Oct 2018 20:02:31 +0000 (22:02 +0200)]
ceph_volume: Rejecting Read-only devices

If a devices is said to be read-only, there is no chance we can actually
use it. So let's report it as unusable.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit f1a94350064263f27f67d2c715c5ca5922481a01)

7 years agoceph_volume: Adding Device.is_valid()
Erwan Velu [Tue, 9 Oct 2018 19:04:02 +0000 (21:04 +0200)]
ceph_volume: Adding Device.is_valid()

A block device can be filtered-out/ignored because it have features that
doesn't match Ceph's expectations.

As of today, the current code was rejected removable devices but it was
pretty hidden from the user, and implicit in the get_devices() function.

This patch is creating a new is_valid() function to perform all the
rejection tests and returns if this device can be used in the Ceph
context or not.

If is_valid() is returning False, the 'rejected_reasons' list reports all
the reasons why that devices got rejected.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit d8fdf0b7532cdffcae56511c377679e51841caec)

7 years agoMerge pull request #24449 from alfredodeza/mimic-rm24795
Andrew Schoen [Fri, 26 Oct 2018 20:00:08 +0000 (15:00 -0500)]
Merge pull request #24449 from alfredodeza/mimic-rm24795

mimic ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoMerge pull request #24760 from ceph/backport-mimic-24587
Alfredo Deza [Fri, 26 Oct 2018 13:46:57 +0000 (09:46 -0400)]
Merge pull request #24760 from ceph/backport-mimic-24587

mimic: ceph-volume: adds a --prepare flag to `lvm batch`

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume lvm.prepare update help to indicate partitions are needed, not devices 24449/head
Alfredo Deza [Wed, 3 Oct 2018 12:11:58 +0000 (08:11 -0400)]
ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d31dd95b95445825f5e6669dd4ecb3118b09fdcf)

7 years agoceph-volume: update man page for batch --prepare 24760/head
Andrew Schoen [Thu, 25 Oct 2018 12:51:41 +0000 (07:51 -0500)]
ceph-volume: update man page for batch --prepare

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit faa3aed4ad4df2dbdd853235e364d6e3b9b176b6)

7 years agoceph-volume: docs for --prepare flag of lvm batch
Andrew Schoen [Mon, 15 Oct 2018 15:38:43 +0000 (10:38 -0500)]
ceph-volume: docs for --prepare flag of lvm batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b4825e1bd7b75a0a53ee5678230b262286293d75)

7 years agoceph-volume: when --prepare is added to batch the OSDs are only prepared
Andrew Schoen [Mon, 15 Oct 2018 15:35:12 +0000 (10:35 -0500)]
ceph-volume: when --prepare is added to batch the OSDs are only prepared

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit abd238dc1f33252a43adc8b86efb3d3b4a076964)

7 years agoceph-volume: adds a --prepare flag to ceph-volume lvm batch
Andrew Schoen [Mon, 15 Oct 2018 14:33:30 +0000 (09:33 -0500)]
ceph-volume: adds a --prepare flag to ceph-volume lvm batch

This flag will only prepare the OSDs, not activate them. This is useful
in our containerized ceph solution.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 4368ca7d48f4ce35f67854e8b0ca2be9babc5371)

7 years agoMerge pull request #24753 from alfredodeza/mimic-rm36386
Andrew Schoen [Thu, 25 Oct 2018 16:10:47 +0000 (11:10 -0500)]
Merge pull request #24753 from alfredodeza/mimic-rm36386

mimic ceph-volume remove version reporting from help menu

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume remove version reporting from help menu 24753/head
Alfredo Deza [Wed, 10 Oct 2018 19:35:31 +0000 (15:35 -0400)]
ceph-volume remove version reporting from help menu

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 20db7bf585f6433c6705396264005a4227cbf2e3)

7 years agoMerge pull request #24740 from alfredodeza/mimic-rm36492
Alfredo Deza [Thu, 25 Oct 2018 12:04:34 +0000 (08:04 -0400)]
Merge pull request #24740 from alfredodeza/mimic-rm36492

mimic ceph-volume: do not send (lvm) stderr/stdout to the terminal, use the logfile

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests.api update monkeypatching to use **kw 24740/head
Alfredo Deza [Wed, 24 Oct 2018 14:55:06 +0000 (10:55 -0400)]
ceph-volume tests.api update monkeypatching to use **kw

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d6bcda75f846bc7c44c7b16b3687a65fed7cafc)

7 years agoceph-volume tests allow **kw when monkeypatching
Alfredo Deza [Wed, 24 Oct 2018 14:54:48 +0000 (10:54 -0400)]
ceph-volume tests allow **kw when monkeypatching

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 04e51c956eadcbd38f39ec3dd4b78009d2cd4a70)

7 years agoceph-volume api.lvm: do not spit out errors on the terminal from LVM
Alfredo Deza [Wed, 24 Oct 2018 14:54:18 +0000 (10:54 -0400)]
ceph-volume api.lvm: do not spit out errors on the terminal from LVM

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 581eed36ea0e7b5c5e41f8b83350403bcf73a31a)

7 years agoMerge pull request #24588 from ceph/backport-mimic-24404
Alfredo Deza [Wed, 24 Oct 2018 21:22:50 +0000 (17:22 -0400)]
Merge pull request #24588 from ceph/backport-mimic-24404

mimic: ceph-volume: make `lvm batch` idempotent

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #24441 from joscollin/wip-36313-mimic
Nathan Cutler [Tue, 23 Oct 2018 06:26:55 +0000 (08:26 +0200)]
Merge pull request #24441 from joscollin/wip-36313-mimic

mimic: doc: fix broken fstab url in cephfs/fuse

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #24017 from dzafman/wip-35909
Yuri Weinstein [Mon, 22 Oct 2018 15:40:24 +0000 (11:40 -0400)]
Merge pull request #24017 from dzafman/wip-35909

mimic: test: Use a grep pattern that works across releases

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #24037 from smithfarm/wip-21057-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:39:41 +0000 (11:39 -0400)]
Merge pull request #24037 from smithfarm/wip-21057-mimic

mimic: debian/control: require fuse for ceph-fuse

Reviewed-by: James Page <james.page@ubuntu.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #24480 from pdvian/wip-36146-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:37:09 +0000 (11:37 -0400)]
Merge pull request #24480 from pdvian/wip-36146-mimic

mimic: fsck: cid is improperly matched to oid

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #24581 from smithfarm/wip-36437-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:55 +0000 (11:34 -0400)]
Merge pull request #24581 from smithfarm/wip-36437-mimic

mimic: ec: src/common/interval_map.h: 161: FAILED assert(len > 0)

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #24600 from pdvian/wip-36296-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:02 +0000 (11:34 -0400)]
Merge pull request #24600 from pdvian/wip-36296-mimic

mimic : osdc/Objecter: possible race condition with connection reset

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #24601 from pdvian/wip-36298-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:32:42 +0000 (11:32 -0400)]
Merge pull request #24601 from pdvian/wip-36298-mimic

mimic: osd: add creating to pg_string_state

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #24379 from smithfarm/wip-36136-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:51:33 +0000 (16:51 -0400)]
Merge pull request #24379 from smithfarm/wip-36136-mimic

mimic: mds: rctime may go back

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24384 from smithfarm/wip-32104-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:54 +0000 (16:50 -0400)]
Merge pull request #24384 from smithfarm/wip-32104-mimic

mimic: mds: allows client to create .. and . dirents

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

7 years agoMerge pull request #24385 from smithfarm/wip-36134-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:20 +0000 (16:50 -0400)]
Merge pull request #24385 from smithfarm/wip-36134-mimic

mimic: client: update ctime when modifying file content

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24408 from smithfarm/wip-36153-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:49:25 +0000 (16:49 -0400)]
Merge pull request #24408 from smithfarm/wip-36153-mimic

mimic: qa: fsstress workunit does not execute in parallel on same host without clobbering files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24411 from smithfarm/wip-36195-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:52 +0000 (16:48 -0400)]
Merge pull request #24411 from smithfarm/wip-36195-mimic

mimic: mds: internal op missing events time 'throttled', 'all_read', 'dispatched'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24438 from pdvian/wip-36156-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:28 +0000 (16:48 -0400)]
Merge pull request #24438 from pdvian/wip-36156-mimic

mimic: qa: extend timeout for SessionMap flush

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24439 from pdvian/wip-36197-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:52 +0000 (16:47 -0400)]
Merge pull request #24439 from pdvian/wip-36197-mimic

mimic: ceph-fuse: add to selinux profile

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24463 from pdvian/wip-36199-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:09 +0000 (16:47 -0400)]
Merge pull request #24463 from pdvian/wip-36199-mimic

mimic:mds: fix mds damaged due to unexpected journal length

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24464 from pdvian/wip-36205-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:46:30 +0000 (16:46 -0400)]
Merge pull request #24464 from pdvian/wip-36205-mimic

mimic: libcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24470 from smithfarm/wip-36203-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:45:53 +0000 (16:45 -0400)]
Merge pull request #24470 from smithfarm/wip-36203-mimic

mimic: ceph fs add_data_pool applies pool application metadata incorrectly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #23087 from pdvian/wip-24928-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:44:39 +0000 (16:44 -0400)]
Merge pull request #23087 from pdvian/wip-24928-mimic

mimic: qa: test_recovery_pool tries asok on wrong node

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #23506 from ukernel/mimic-26858
Yuri Weinstein [Fri, 19 Oct 2018 20:44:14 +0000 (16:44 -0400)]
Merge pull request #23506 from ukernel/mimic-26858

mimic: mds: reset heartbeat map at potential time-consuming places

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #23952 from ukernel/mimic-mds-export-size
Yuri Weinstein [Fri, 19 Oct 2018 20:43:47 +0000 (16:43 -0400)]
Merge pull request #23952 from ukernel/mimic-mds-export-size

mimic: mds: optimize the way how max export size is enforced

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24161 from pdvian/wip-35719-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:24 +0000 (16:43 -0400)]
Merge pull request #24161 from pdvian/wip-35719-mimic

mimic: mds: print is_laggy message once

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24187 from pdvian/wip-35841-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:02 +0000 (16:43 -0400)]
Merge pull request #24187 from pdvian/wip-35841-mimic

mimic: client: fix use-after-free in Client::link()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24269 from smithfarm/wip-36102-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:42:29 +0000 (16:42 -0400)]
Merge pull request #24269 from smithfarm/wip-36102-mimic

mimic: qa: remove knfs site from future releases

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24272 from batrick/i35858
Yuri Weinstein [Fri, 19 Oct 2018 20:41:34 +0000 (16:41 -0400)]
Merge pull request #24272 from batrick/i35858

mimic: mon: test if gid exists in pending for prepare_beacon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24295 from pdvian/wip-35934-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:40:56 +0000 (16:40 -0400)]
Merge pull request #24295 from pdvian/wip-35934-mimic

mimic: client:two ceph-fuse client, one can not list out files created by an…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24377 from smithfarm/wip-35940-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:40:23 +0000 (16:40 -0400)]
Merge pull request #24377 from smithfarm/wip-35940-mimic

mimic: client: statfs inode count odd

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #24319 from smithfarm/wip-35979-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:11:38 +0000 (16:11 -0400)]
Merge pull request #24319 from smithfarm/wip-35979-mimic

mimic: multisite: data sync error repo processing does not back off on empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #23197 from pdvian/wip-25024-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:09:22 +0000 (16:09 -0400)]
Merge pull request #23197 from pdvian/wip-25024-mimic

mimic: qa/suites/rgw/verify/tasks/cls_rgw: test cls_rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #24071 from pdvian/wip-35708-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:06:48 +0000 (16:06 -0400)]
Merge pull request #24071 from pdvian/wip-35708-mimic

mimic: rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24101 from pdvian/wip-35710-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:05:48 +0000 (16:05 -0400)]
Merge pull request #24101 from pdvian/wip-35710-mimic

mimic: rgw: fix deadlock on RGWIndexCompletionManager::stop

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24189 from joke-lee/mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:05:17 +0000 (16:05 -0400)]
Merge pull request #24189 from joke-lee/mimic

mimic:rgw: return x-amz-version-id: null when delete obj in versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24235 from pdvian/wip-35857-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:04:52 +0000 (16:04 -0400)]
Merge pull request #24235 from pdvian/wip-35857-mimic

mimic: multisite: segfault on shutdown/realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24363 from pdvian/wip-36125-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:04:10 +0000 (16:04 -0400)]
Merge pull request #24363 from pdvian/wip-36125-mimic

mimic:rgw: fix chunked-encoding for chunks >1MiB

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24417 from pdvian/wip-36138-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:01:17 +0000 (16:01 -0400)]
Merge pull request #24417 from pdvian/wip-36138-mimic

mimic: rgw: async sync_object and remove_object does not access coroutine me…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24571 from joke-lee/mimic-list-RGWPostObj-when-enable-versioning
Yuri Weinstein [Wed, 17 Oct 2018 20:00:41 +0000 (16:00 -0400)]
Merge pull request #24571 from joke-lee/mimic-list-RGWPostObj-when-enable-versioning

mimic:rgw: list bucket can not show the object uploaded by RGWPostObj when enable bucket versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #23026 from smithfarm/wip-24889-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:58:16 +0000 (15:58 -0400)]
Merge pull request #23026 from smithfarm/wip-24889-mimic

mimic: osd: crash in OpTracker::unregister_inflight_op via OSD::get_health_metrics

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
7 years agoMerge pull request #24116 from dzafman/wip-35992
Yuri Weinstein [Wed, 17 Oct 2018 19:57:22 +0000 (15:57 -0400)]
Merge pull request #24116 from dzafman/wip-35992

mimic: tools: ceph-objectstore-tool: Allow target level as first positional …

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #24401 from dillaman/wip-36144-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:56:50 +0000 (15:56 -0400)]
Merge pull request #24401 from dillaman/wip-36144-mimic

mimic: librbd: blacklisted client might not notice it lost the lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #24573 from pdvian/wip-36292-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:56:08 +0000 (15:56 -0400)]
Merge pull request #24573 from pdvian/wip-36292-mimic

mimic: osd: Remove old bft= which has been superceded by backfill=

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agotest/objectstore: set pool for fsck test 24480/head
Kefu Chai [Fri, 21 Sep 2018 11:56:08 +0000 (19:56 +0800)]
test/objectstore: set pool for fsck test

since 0bd2546eaca72ed0122a9c2648df4bef05b0d5d2, we check the pool id
of object when performing fsck to ensure we are looking at the right
collection, but the test is still using the pool id set by the
constructor of hobject_t. so all objects we created in that test belong
to the POOL_META. while the collection is created with the pool id of
`555`. hence the test fails.

Fixes: http://tracker.ceph.com/issues/36099
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 8a814b2eb6f6d16233304e07b2adc5951f5fb312)

7 years agoosd: add creating to pg_string_state 24601/head
Dan van der Ster [Tue, 25 Sep 2018 08:39:37 +0000 (10:39 +0200)]
osd: add creating to pg_string_state

Fixes: http://tracker.ceph.com/issues/36174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit d38f6a11701ec788e4d384aa5b0ae65b8e57da64)

Conflicts:
src/osd/osd_types.cc : Resolved in pg_string_state

7 years agoosdc/Objecter: possible race condition with connection reset 24600/head
Jason Dillaman [Tue, 25 Sep 2018 18:18:00 +0000 (14:18 -0400)]
osdc/Objecter: possible race condition with connection reset

If the connection quickly fails before the private session reference
can be associated with the connection, the connection will remain
closed and any OSD ops against the session will remain stuck.

Fixes: http://tracker.ceph.com/issues/36183
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74ca33cb49d2c258324447b1ca366ed4e604202a)

7 years agoMerge pull request #24388 from pdvian/wip-36129-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:52:18 +0000 (16:52 -0400)]
Merge pull request #24388 from pdvian/wip-36129-mimic

mimic: rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24400 from smithfarm/wip-36201-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:53 +0000 (16:51 -0400)]
Merge pull request #24400 from smithfarm/wip-36201-mimic

mimic: multisite: intermittent test_bucket_index_log_trim failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24413 from dillaman/wip-36225-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:11 +0000 (16:51 -0400)]
Merge pull request #24413 from dillaman/wip-36225-mimic

mimic: librbd: object map improperly flagged as invalidated

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #24414 from smithfarm/wip-36208-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:38 +0000 (16:50 -0400)]
Merge pull request #24414 from smithfarm/wip-36208-mimic

mimic: multisite: invalid read in RGWCloneMetaLogCoroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24418 from pdvian/wip-36140-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:13 +0000 (16:50 -0400)]
Merge pull request #24418 from pdvian/wip-36140-mimic

mimic: rgw: raise debug level on redundant data sync error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24436 from croit/mimic-36041
Yuri Weinstein [Mon, 15 Oct 2018 20:49:40 +0000 (16:49 -0400)]
Merge pull request #24436 from croit/mimic-36041

mimic: rgw/beast: drop privileges after binding ports

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24481 from pdvian/wip-36150-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:49:10 +0000 (16:49 -0400)]
Merge pull request #24481 from pdvian/wip-36150-mimic

mimic: crush/CrushWrapper: fix crush tree json dumper

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #24516 from pdvian/wip-36211-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:38 +0000 (16:48 -0400)]
Merge pull request #24516 from pdvian/wip-36211-mimic

mimic: rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24518 from pdvian/wip-36213-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:13 +0000 (16:48 -0400)]
Merge pull request #24518 from pdvian/wip-36213-mimic

mimic:rgw: fix leak of curl handle on shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24526 from theanalyst/wip-mimic-36381
Yuri Weinstein [Mon, 15 Oct 2018 20:47:55 +0000 (16:47 -0400)]
Merge pull request #24526 from theanalyst/wip-mimic-36381

mimic: rgw: resharding produces invalid values of bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24536 from pdvian/wip-36216-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:47:13 +0000 (16:47 -0400)]
Merge pull request #24536 from pdvian/wip-36216-mimic

mimic: multisite: data full sync does not limit concurrent bucket sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #24537 from pdvian/wip-36228-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:46:36 +0000 (16:46 -0400)]
Merge pull request #24537 from pdvian/wip-36228-mimic

mimic: qa: add test that builds example librados programs

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoceph-volume: extracts batch.filter_devices from Batch._get_strategy 24588/head
Andrew Schoen [Wed, 10 Oct 2018 19:28:29 +0000 (15:28 -0400)]
ceph-volume: extracts batch.filter_devices from Batch._get_strategy

This allows us to easily provide tests for that method.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit df7ef5383bb476ee020b898cd2e8fbce044fc07e)

7 years agoceph-volume: failing to get block db size from conf logs an exception
Andrew Schoen [Wed, 10 Oct 2018 18:05:25 +0000 (14:05 -0400)]
ceph-volume: failing to get block db size from conf logs an exception

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 172d4af9b97d8d64ce35dc95efb72d9f190bc170)

7 years agoceph-volume: when all devices are filtered exit gracefully
Andrew Schoen [Tue, 9 Oct 2018 18:05:54 +0000 (14:05 -0400)]
ceph-volume: when all devices are filtered exit gracefully

Even if all devices are filtered we want to return a 0 exit code and
make sure the json reporting still works.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit aa4fcd602f8b1a1b17bc12e59334b19508f97d6d)

7 years agoceph-volume: filter devices used by journals/block.db
Andrew Schoen [Mon, 8 Oct 2018 13:57:07 +0000 (09:57 -0400)]
ceph-volume: filter devices used by journals/block.db

If after filterering of data/block devices there are only
one device left it can not be used if it is an SSD and
has been used previously as a journal or block.db

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit fc9a10e54813a402ecd1d823ea5a33f85e8eb963)

7 years agoceph-volume: add rotational property to Device class
Andrew Schoen [Mon, 8 Oct 2018 13:39:05 +0000 (09:39 -0400)]
ceph-volume: add rotational property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7f6bfaaf0cf30a56481d6d67f2fa2a7e785cc344)

7 years agoceph-volume: add info about filtered devices to batch pretty reports
Andrew Schoen [Fri, 5 Oct 2018 21:18:48 +0000 (16:18 -0500)]
ceph-volume: add info about filtered devices to batch pretty reports

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 62426db799c54436c50b5751c8e72b3dbd710a2a)

7 years agoceph-volume: remove the used_by_ceph key in the json output
Andrew Schoen [Fri, 5 Oct 2018 15:45:35 +0000 (10:45 -0500)]
ceph-volume: remove the used_by_ceph key in the json output

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3b6d82afe7ee83283b25cefda00c7a59cf5144af)

7 years agoceph-volume: fix idempotency checks for lvm batch tests
Andrew Schoen [Fri, 5 Oct 2018 15:39:54 +0000 (10:39 -0500)]
ceph-volume: fix idempotency checks for lvm batch tests

The mixed type tests will change strategy after the idempotency test so
we need to handle that in test playbook.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1143482061da50910f4f883aafdbe0a0e4269f39)

7 years agoceph-volume: fix bluestore strategy json reporting and type
Andrew Schoen [Fri, 5 Oct 2018 15:38:11 +0000 (10:38 -0500)]
ceph-volume: fix bluestore strategy json reporting and type

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b3cf90604d31709996538e612b1767d6fd4da8b5)

7 years agoceph-volume: ignore failure to load ceph configuration for block.db size
Andrew Schoen [Thu, 4 Oct 2018 17:47:48 +0000 (12:47 -0500)]
ceph-volume: ignore failure to load ceph configuration for block.db size

If we fail to load a ceph configureation file when trying to get the
block.db size then just use defaults instead of throwing an error.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a7ee36ca92b6592b6b3e218252c6a4c30416591b)

7 years agoceph-volume: fix strategy comparison in 'lvm batch'
Andrew Schoen [Thu, 4 Oct 2018 16:54:06 +0000 (11:54 -0500)]
ceph-volume: fix strategy comparison in 'lvm batch'

This also fixes some small json reporting issues with the
filestore MixedType strategy

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1dd15025bfd52af9c31cd281f92d743b9ca0eeb8)

7 years agoceph-volume: raise a non zero exit code if strategy changes with batch
Andrew Schoen [Wed, 3 Oct 2018 20:01:08 +0000 (15:01 -0500)]
ceph-volume: raise a non zero exit code if strategy changes with batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9dfc00f8e10e10fbdc52c44259596c6a96a90edd)

7 years agoceph-volume: add functional tests to ensure lvm batch is idempotent
Andrew Schoen [Wed, 3 Oct 2018 17:13:27 +0000 (12:13 -0500)]
ceph-volume: add functional tests to ensure lvm batch is idempotent

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9752d03cc16664cddd9ac7741284a2fb5b31f0e7)

7 years agoceph-volume: add tests for util.device.Device.used_by_ceph
Andrew Schoen [Wed, 3 Oct 2018 15:19:45 +0000 (10:19 -0500)]
ceph-volume: add tests for util.device.Device.used_by_ceph

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a28e6531e47d60d9eb7f62b67f122578daa2a683)

7 years agoceph-volume: update tests to account for filtered_devices in batch
Andrew Schoen [Tue, 2 Oct 2018 20:23:39 +0000 (15:23 -0500)]
ceph-volume: update tests to account for filtered_devices in batch

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 36396229d3b8d20862565b35d431ff22fa92cd1c)

7 years agoceph-volume: consider block and data devices used_by_ceph
Andrew Schoen [Tue, 2 Oct 2018 20:08:10 +0000 (15:08 -0500)]
ceph-volume: consider block and data devices used_by_ceph

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 67512530116c16d072c91867a259aeb429d32ff6)

7 years agoceph-volume: add filtered_devices and used_by_ceph to all batch reports
Andrew Schoen [Tue, 2 Oct 2018 14:48:27 +0000 (09:48 -0500)]
ceph-volume: add filtered_devices and used_by_ceph to all batch reports

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0718d2e2663c8e807dbb5171a58759c76ed92c08)

7 years agoceph-volume: pick strategy for batch with only the unused devices
Andrew Schoen [Thu, 27 Sep 2018 20:22:17 +0000 (15:22 -0500)]
ceph-volume: pick strategy for batch with only the unused devices

This will pick a strategy, filter out any devices already been used by
ceph and then pick a strategy again. If the strategy has changed the
call should error, if the strategy is the same proceed. If there are no
unused devices then the command is a noop.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 4529f2d6053b0b07583a3d7501f4e05e08cad385)

7 years agoceph-volume: adds a 'changed' key to lvm batch --report
Andrew Schoen [Thu, 27 Sep 2018 13:55:20 +0000 (08:55 -0500)]
ceph-volume: adds a 'changed' key to lvm batch --report

This will indicate if the command would result in any OSDs being created
or not. Other tooling can use that key for idempotency checks.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 10f1d577d4c4e66c77046fc3b274d6653af99586)

7 years agoceph-volume: adds used_by_ceph to filestore singletype batch report
Andrew Schoen [Wed, 26 Sep 2018 21:07:30 +0000 (16:07 -0500)]
ceph-volume: adds used_by_ceph to filestore singletype batch report

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 48d10c9ccf13ba7f42750b37d26aeb28f6b1c606)

7 years agoceph-volume: adds a used_by_ceph property to the Device class
Andrew Schoen [Wed, 26 Sep 2018 21:01:30 +0000 (16:01 -0500)]
ceph-volume: adds a used_by_ceph property to the Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9d49a3708e34bccd183d5c32d022c36d8b118b42)

7 years agoceph-volume: adds a lvs property to the Device class
Andrew Schoen [Wed, 26 Sep 2018 20:53:26 +0000 (15:53 -0500)]
ceph-volume: adds a lvs property to the Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3dae3247adc1b96ca688ec81cec03180d5943823)

7 years agoceph-volume: add vg_name to the Device class
Andrew Schoen [Wed, 26 Sep 2018 19:29:41 +0000 (14:29 -0500)]
ceph-volume: add vg_name to the Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 619810c0ef48c1f19db8821a33f9c48cc11a0161)

7 years agoosd/PrimaryLogPG.cc: reassign size only when object size > truncate_size 24581/head
Neha Ojha [Tue, 9 Oct 2018 22:57:15 +0000 (15:57 -0700)]
osd/PrimaryLogPG.cc: reassign size only when object size > truncate_size

Before setting size equal to op.extent.truncate_size, we need to check
if the size of the object is greater than the truncate_size. We do not
need to set size to op.extent.truncate_size, in the case where the size of
the object is less than op.extent.truncate_size.

Without this change, we were always setting size =
op.extent.truncate_size, when (seq < op.extent.truncate_seq) and
(op.extent.offset + op.extent.length > op.extent.truncate_size), were both
true. This ended up in:

1. overestimating the size of the object
2. not considering the correct size of the object, for
   the later checks, which calculate op.extent.length for the read ops
3. causing crashes when trying to read more data than what was present

Fixes: http://tracker.ceph.com/issues/21931
Fixes: http://tracker.ceph.com/issues/22330
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 76c57810ee2346c392834206331aacb0faaa5b54)

7 years agoosd: print offset and length to track trimtrunc
Neha Ojha [Wed, 26 Sep 2018 23:31:44 +0000 (16:31 -0700)]
osd: print offset and length to track trimtrunc

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 07f7bd69425ed2c6b2a97bbf0034232a56fed046)

Conflicts:
src/osd/ECBackend.cc
- trivial: mimic uses assert() instead of ceph_assert()

7 years agorgw: list bucket can not show the object uploaded by RGWPostObj 24571/head
yuliyang [Sun, 30 Sep 2018 09:26:58 +0000 (17:26 +0800)]
rgw: list bucket can not show the object uploaded by RGWPostObj
when enable bucket versioning

fix https://tracker.ceph.com/issues/36265

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
(cherry picked from commit 38407f69da898eacf77027d04e2b9544f310ebff)

7 years agoosd: Remove old bft= which has been superceded by backfill= 24573/head
David Zafman [Mon, 24 Sep 2018 21:21:59 +0000 (14:21 -0700)]
osd: Remove old bft= which has been superceded by backfill=

Fixes: http://tracker.ceph.com/issues/36170
Caused by: 1b4f86cb0

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 6847d3fd6f7a6cd671cb7b4fa14240f0e6df3c09)