]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agocmake: update BuildSPDK for spdk-18.05 22547/head
Kefu Chai [Thu, 14 Jun 2018 01:32:08 +0000 (09:32 +0800)]
cmake: update BuildSPDK for spdk-18.05

in spdk v18.05, libuuid is linked by libspdk_util.a, in which,
it is used by lib/util/uuid.c. and libspdk_vol.a uses the wrapper
function exposed by libspdk_util.a, so update the CMakefile script to
reflect the change.

Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agospdk: update to latest spdk-18.05 branch
Kefu Chai [Wed, 13 Jun 2018 13:35:24 +0000 (21:35 +0800)]
spdk: update to latest spdk-18.05 branch

also bump dpdk to to spdk-18.05 branch. this should fix the build
failure on Fedora 28

Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22500 from thinkercui/bugfix
Josh Durgin [Wed, 13 Jun 2018 20:57:57 +0000 (16:57 -0400)]
Merge pull request #22500 from thinkercui/bugfix

osd: fix memory leak in EC fast and error read

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #21969 from zhongyimao/interlock_fast-diff/object-map
Jason Dillaman [Wed, 13 Jun 2018 20:22:04 +0000 (16:22 -0400)]
Merge pull request #21969 from zhongyimao/interlock_fast-diff/object-map

rbd: interlock object-map/fast-diff features together

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22304 from vshankar/wip-scheduled-rebalancer
Jason Dillaman [Wed, 13 Jun 2018 20:21:12 +0000 (16:21 -0400)]
Merge pull request #22304 from vshankar/wip-scheduled-rebalancer

rbd-mirror: schedule rebalancer to level-load instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agotest: add rbd-mirror image map rebalancer unit test 22304/head
Venky Shankar [Sat, 2 Jun 2018 14:55:22 +0000 (10:55 -0400)]
test: add rbd-mirror image map rebalancer unit test

Signed-off-by: Venky Shankar <vshankar@redhat.com>
Fixes: http://tracker.ceph.com/issues/24161
7 years agorbd-mirror: schedule rebalancer to level-load instances
Venky Shankar [Tue, 29 May 2018 05:45:40 +0000 (01:45 -0400)]
rbd-mirror: schedule rebalancer to level-load instances

Policy implementation takes care of evenly balancing images
across rbd mirror instances. This is done when images are
added to the map and/or instances are added or removed with
the exception of image removal -- removing images does not
reshuffle other (mapped) images which can result in some of
the instances under loaded (in worst case, if one removes
images which all map to a particular instance, that instance
would remain idle until more images are added or a shuffle is
triggered).

We could possibly trigger map shuffle when images are removed,
but that would change the interface between Policy and ImageMap
class (in the form of changes to Policy::remove_images()). Also,
policy (and its implementations) would have to do more work when
the above class method is invoked.

Therefore, an interval based rebalancer is added to ImageMap for
periodic rebalancing of images only if the following conditions
are met:

    - policy has been idle for a configured time duration
    - no scheduled or in-transit operations

Signed-off-by: Venky Shankar <vshankar@redhat.com>
7 years agorbd-mirror: fix state transition table for disassociation
Venky Shankar [Mon, 4 Jun 2018 09:34:58 +0000 (05:34 -0400)]
rbd-mirror: fix state transition table for disassociation

The final state transition when disassociating (removing) images
does not purge the image state map for a given image. This can
also result in uneven balance of images across instances as the
policy implementation relies on this structure to figure out
total number of images tracked.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
7 years agorbd-mirror: fix state comparison in Policy::is_state_scheduled()
Venky Shankar [Mon, 28 May 2018 04:53:30 +0000 (00:53 -0400)]
rbd-mirror: fix state comparison in Policy::is_state_scheduled()

Signed-off-by: Venky Shankar <vshankar@redhat.com>
7 years agorbd-mirror: remove unneeded instance_ids parameter
Venky Shankar [Fri, 25 May 2018 10:48:10 +0000 (06:48 -0400)]
rbd-mirror: remove unneeded instance_ids parameter

Signed-off-by: Venky Shankar <vshankar@redhat.com>
7 years agoMerge pull request #22284 from tspmelo/wip-unit-test-api
Lenz Grimmer [Wed, 13 Jun 2018 10:20:18 +0000 (12:20 +0200)]
Merge pull request #22284 from tspmelo/wip-unit-test-api

mgr/dashboard: Add unit test for frontend api services

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge pull request #22422 from Devp00l/wip-stringify-array-objects
Lenz Grimmer [Wed, 13 Jun 2018 08:55:15 +0000 (10:55 +0200)]
Merge pull request #22422 from Devp00l/wip-stringify-array-objects

mgr/dashboard: Stringify object[] in KV-table

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge pull request #22531 from yuriw/wip-yuriw-crontab
Yuri Weinstein [Tue, 12 Jun 2018 19:58:43 +0000 (12:58 -0700)]
Merge pull request #22531 from yuriw/wip-yuriw-crontab

qa/tests: removed all jewel runs and reduced runs on ovh

7 years agoMerge PR #22527 into master
Sage Weil [Tue, 12 Jun 2018 18:21:31 +0000 (13:21 -0500)]
Merge PR #22527 into master

* refs/pull/22527/head:
test/encoding/types.h: Adjusting src/messages entries
test/encoding/types.h: Adjusting src/cls entries
test/encoding/types.h: Adjusting src/rgw entries
test/encoding/types.h: Adjusting src/rbd_replay entries
test/encoding/types.h: Adjusting src/mds entries
test/encoding/types.h: Adjusting src/mon entries
test/encoding/types.h: Adjusting src/os entries
test/encoding/types.h: Adjusting src/osd entries
test/encoding/types.h: Adjusting src/common entries
test/encoding/types.h: Adjusting src/test/encoding entries
test/encoding/types.h: Adjusting src/include/ entries

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoqa/tests: removed all jewel runs and reduced runs on ovh 22531/head
Yuri Weinstein [Tue, 12 Jun 2018 16:49:09 +0000 (09:49 -0700)]
qa/tests: removed all jewel runs and reduced runs on ovh

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/messages entries 22527/head
Erwan Velu [Mon, 11 Jun 2018 14:24:35 +0000 (16:24 +0200)]
test/encoding/types.h: Adjusting src/messages entries

This patch is about:
- Adding the missing headers from src/messages
- Adding the missing types from the existing headers
- Adding blank lines to keep includes and types in a visual block.
  That is much easier to read and keep the same visual as the rest of the file.

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/cls entries
Erwan Velu [Mon, 11 Jun 2018 14:22:05 +0000 (16:22 +0200)]
test/encoding/types.h: Adjusting src/cls entries

This patch is about:
- Adding the missing headers from src/cls
- Adding the missing types from the existing headers
- Ordering types to get them in the file's order (that's easier to update)

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/rgw entries
Erwan Velu [Mon, 11 Jun 2018 14:20:01 +0000 (16:20 +0200)]
test/encoding/types.h: Adjusting src/rgw entries

This patch is about:
- Adding the missing headers from src/rgw
- Adding the missing types from the existing headers
- Ordering types to get them in the file's order (that's easier to update)

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/rbd_replay entries
Erwan Velu [Mon, 11 Jun 2018 14:17:08 +0000 (16:17 +0200)]
test/encoding/types.h: Adjusting src/rbd_replay entries

This patch is about removing a simple typo.
The semi-colong isn't necessary here.

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/mds entries
Erwan Velu [Mon, 11 Jun 2018 14:09:53 +0000 (16:09 +0200)]
test/encoding/types.h: Adjusting src/mds entries

This patch is about:
- Adding the missing headers from src/mds
- Adding the missing types from the existing headers
- Adding blank lines to keep includes and types in a visual block.
  That is much easier to read and keep the same visual as the rest of the file.

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/mon entries
Erwan Velu [Mon, 11 Jun 2018 14:07:03 +0000 (16:07 +0200)]
test/encoding/types.h: Adjusting src/mon entries

This patch is about:
- Adding the missing types from the existing headers
- Ordering types to get them in the file's order (that's easier to update)

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/os entries
Erwan Velu [Mon, 11 Jun 2018 14:04:08 +0000 (16:04 +0200)]
test/encoding/types.h: Adjusting src/os entries

This patch is about:
- Adding the missing headers from src/os
- Adding the missing types from the existing headers
- Ordering types to get them in the file's order (that's easier to update)

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/osd entries
Erwan Velu [Mon, 11 Jun 2018 14:01:12 +0000 (16:01 +0200)]
test/encoding/types.h: Adjusting src/osd entries

This patch is about:
- Adding the missing headers from src/osd
- Adding the missing types from the existing headers
- Ordering types to get them in the file's order (that's easier to update)
- Moving crush entries avoid let all the osd entries in a collocated place

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agomgr/dashboard: Add unit test for frontend api services 22284/head
Tiago Melo [Sun, 10 Jun 2018 00:42:35 +0000 (01:42 +0100)]
mgr/dashboard: Add unit test for frontend api services

Signed-off-by: Tiago Melo <tmelo@suse.com>
7 years agomgr/dashboard: Prettier KV-table 22422/head
Stephan Müller [Tue, 5 Jun 2018 14:15:36 +0000 (16:15 +0200)]
mgr/dashboard: Prettier KV-table

Signed-off-by: Stephan Müller <smueller@suse.com>
7 years agomgr/dashboard: Stringify object[] in KV-table
Stephan Müller [Tue, 5 Jun 2018 14:11:12 +0000 (16:11 +0200)]
mgr/dashboard: Stringify object[] in KV-table

The problem was that object[] weren't handled in the key value table
before. Now they will be stringified to prevent an output like
'[Object object]'.

Signed-off-by: Stephan Müller <smueller@suse.com>
7 years agoMerge pull request #22082 from tspmelo/wip-angular-6
Lenz Grimmer [Tue, 12 Jun 2018 13:32:58 +0000 (15:32 +0200)]
Merge pull request #22082 from tspmelo/wip-angular-6

mgr/dashboard: Update Angular to version 6

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge PR #21825 into master
Sage Weil [Tue, 12 Jun 2018 13:31:16 +0000 (08:31 -0500)]
Merge PR #21825 into master

* refs/pull/21825/head:
os/bluestore: rename new bitmap allocator class to BitmapAllocator.
os/bluestore: perform allocations aligned with min_length  in new bitmap allocator
test/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
os/bluestore: respect min_length as allocation granularity for new
os/bluestore: cosmetic new allocator internal method rename.
os/bluestore: properly respect min_length when analysing partially free
os/bluestore: cosmetic cleanup in new bitmap allocator.
os/bluestore: more verbose logging in new bitmap allocator
os/bluestore: default to bitmap allocator for bluestore/bluefs
os/bluestore: remove original bitmap allocator
os/bluestore: align BitMap allocator's init_rm_free/init_add_free parameters with min_alloc_size
os/bluestore: fix improper access to a BitmapFastAllocator::last_pos
test/allocator: move bluestore allocator's benchmarks to a standalone UT
os/bluestore: add new bitmap allocator
test/allocator: get rid off bitmap allocator specifics and introduce new
test/fio: dump mempool on job completion

Reviewed-by: Varada Kari <varadaraja.kari@flipkart.com>
7 years agoMerge pull request #22504 from tspmelo/wip-table-filtered-numbers
Lenz Grimmer [Tue, 12 Jun 2018 13:30:17 +0000 (15:30 +0200)]
Merge pull request #22504 from tspmelo/wip-table-filtered-numbers

mgr/dashboard: Add filtered rows number in table footer

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge PR #22423 into master
Sage Weil [Tue, 12 Jun 2018 13:27:17 +0000 (08:27 -0500)]
Merge PR #22423 into master

* refs/pull/22423/head:
common/blkdev: fix non-linux build for get_device_id
common/blkdev: do not include vendor in fallback mode
mgr: allow open-ended life expectancy
mgr: use prettified durations for life expectancy
common/ceph_time: very lame timespan_str() helper
mgr: "predicted failure" -> "life expectancy", and as a range
mgr: detailed output for 'ceph device ...'
mgr/DaemonState: skip blank device_ids
osd: skip blank device ids
mgr: track and report device names too
mgr/ActivePyModule: expose 'devices' to python modules
mgr: move dump/print to DeviceState
mgr: implement 'device {set,rm}-predicted-failure' commands
mgr: load persistent device metadata on mgr stat
include/utime: add parse() method
mgr: tolerate racing config-key change during load_store()
mgr: implement 'device {ls,ls-by-daemon,ls-by-host,info}' commands
mgr/DaemonState: populate DeviceState structures
mgr: parse out devids from daemon metadata
common/blkdev: add modelines
osd: include device_ids in metadata
common/blkdev: add get_device_id() helper.

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
7 years agogithubmap: update contributors
Sage Weil [Tue, 12 Jun 2018 13:27:16 +0000 (08:27 -0500)]
githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22522 from tchaikov/wip-githubmap
Sage Weil [Tue, 12 Jun 2018 13:06:10 +0000 (08:06 -0500)]
Merge pull request #22522 from tchaikov/wip-githubmap

githubmap: update contributors

7 years agotest/encoding/types.h: Adjusting src/common entries
Erwan Velu [Mon, 11 Jun 2018 13:52:03 +0000 (15:52 +0200)]
test/encoding/types.h: Adjusting src/common entries

This patch is about:
- Moving some common/ entries to keep them in a collocated place, and in the file order : that's easier to update this way.

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/test/encoding entries
Erwan Velu [Mon, 11 Jun 2018 13:46:21 +0000 (15:46 +0200)]
test/encoding/types.h: Adjusting src/test/encoding entries

This patch is about :
- Moving encoding headers in a visual block (easier to update)

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agotest/encoding/types.h: Adjusting src/include/ entries
Erwan Velu [Mon, 11 Jun 2018 13:42:29 +0000 (15:42 +0200)]
test/encoding/types.h: Adjusting src/include/ entries

This patch is about
- Moving fs_types.h with the other src/include files, this is easier to update.

Signed-off-by: Erwan Velu <erwan@redhat.com>
7 years agoos/bluestore: rename new bitmap allocator class to BitmapAllocator. 21825/head
Igor Fedotov [Tue, 12 Jun 2018 11:50:30 +0000 (14:50 +0300)]
os/bluestore: rename new bitmap allocator class to BitmapAllocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: perform allocations aligned with min_length in new bitmap allocator
Igor Fedotov [Fri, 1 Jun 2018 18:59:15 +0000 (21:59 +0300)]
os/bluestore: perform allocations aligned with min_length  in new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agotest/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
Igor Fedotov [Fri, 1 Jun 2018 13:35:04 +0000 (16:35 +0300)]
test/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
ASSERT_EQ

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: respect min_length as allocation granularity for new
Igor Fedotov [Thu, 31 May 2018 13:53:31 +0000 (16:53 +0300)]
os/bluestore: respect min_length as allocation granularity for new
bitmap allocator.
It was used a real minimum threshold before this fix which allowed e.g.
allocated extent length to be equal to min_length + 1.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: cosmetic new allocator internal method rename.
Igor Fedotov [Thu, 31 May 2018 11:28:28 +0000 (14:28 +0300)]
os/bluestore: cosmetic new allocator internal method rename.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: properly respect min_length when analysing partially free
Igor Fedotov [Thu, 31 May 2018 10:53:37 +0000 (13:53 +0300)]
os/bluestore: properly respect min_length when analysing partially free
slots in bitmap allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: cosmetic cleanup in new bitmap allocator.
Igor Fedotov [Thu, 31 May 2018 10:34:21 +0000 (13:34 +0300)]
os/bluestore: cosmetic cleanup in new bitmap allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: more verbose logging in new bitmap allocator
Igor Fedotov [Wed, 30 May 2018 12:24:05 +0000 (15:24 +0300)]
os/bluestore: more verbose logging in new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: default to bitmap allocator for bluestore/bluefs
Igor Fedotov [Fri, 18 May 2018 19:14:59 +0000 (22:14 +0300)]
os/bluestore: default to bitmap allocator for bluestore/bluefs

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: remove original bitmap allocator
Igor Fedotov [Mon, 14 May 2018 13:09:22 +0000 (16:09 +0300)]
os/bluestore: remove original bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: align BitMap allocator's init_rm_free/init_add_free parameters with...
Igor Fedotov [Fri, 11 May 2018 21:26:49 +0000 (00:26 +0300)]
os/bluestore: align BitMap allocator's init_rm_free/init_add_free parameters with min_alloc_size

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: fix improper access to a BitmapFastAllocator::last_pos
Igor Fedotov [Thu, 10 May 2018 15:54:36 +0000 (18:54 +0300)]
os/bluestore: fix improper access to a BitmapFastAllocator::last_pos
from multiple threads.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agotest/allocator: move bluestore allocator's benchmarks to a standalone UT
Igor Fedotov [Fri, 4 May 2018 17:03:37 +0000 (20:03 +0300)]
test/allocator: move bluestore allocator's benchmarks to a standalone UT

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agoos/bluestore: add new bitmap allocator
Igor Fedotov [Thu, 3 May 2018 17:22:15 +0000 (20:22 +0300)]
os/bluestore: add new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agotest/allocator: get rid off bitmap allocator specifics and introduce new
Igor Fedotov [Thu, 3 May 2018 16:25:01 +0000 (19:25 +0300)]
test/allocator: get rid off bitmap allocator specifics and introduce new
performance test cases

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agotest/fio: dump mempool on job completion
Igor Fedotov [Thu, 3 May 2018 16:16:12 +0000 (19:16 +0300)]
test/fio: dump mempool on job completion

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
7 years agomgr/dashboard: Add filtered rows number in table footer 22504/head
Tiago Melo [Mon, 11 Jun 2018 11:26:41 +0000 (12:26 +0100)]
mgr/dashboard: Add filtered rows number in table footer

Signed-off-by: Tiago Melo <tmelo@suse.com>
7 years agogithubmap: update contributors 22522/head
Kefu Chai [Tue, 12 Jun 2018 04:50:19 +0000 (12:50 +0800)]
githubmap: update contributors

Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22497 from liewegas/wip-24486
Kefu Chai [Tue, 12 Jun 2018 03:05:12 +0000 (11:05 +0800)]
Merge pull request #22497 from liewegas/wip-24486

osd/Session: fix invalid iterator dereference in Session::have_backoff()

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agorbd: interlock object-map/fast-diff features together 21969/head
Mao Zhongyi [Wed, 16 May 2018 05:27:35 +0000 (13:27 +0800)]
rbd: interlock object-map/fast-diff features together

Since we only support Jewel and later releases, which both support
object-map and fast-diff, enabling/disabling object-map should always
enable/disable fast-diff.

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>
7 years agoMerge PR #22501 into master
Sage Weil [Mon, 11 Jun 2018 21:14:13 +0000 (16:14 -0500)]
Merge PR #22501 into master

* refs/pull/22501/head:
messages/MOSDPGLog: encode epoch for query_epoch for pre-nautilus OSDs

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge PR #22499 into master
Sage Weil [Mon, 11 Jun 2018 21:13:38 +0000 (16:13 -0500)]
Merge PR #22499 into master

* refs/pull/22499/head:
os/bluestore: fix SharedBlobSet deregister race

Reviewed-by: git push gh masterVarada Kari <varadaraja.kari@flipkart.com>
7 years agogithubmap: update contributors
Sage Weil [Mon, 11 Jun 2018 21:13:37 +0000 (16:13 -0500)]
githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22496 from liewegas/wip-localpool-config
Sage Weil [Mon, 11 Jun 2018 21:08:18 +0000 (16:08 -0500)]
Merge pull request #22496 from liewegas/wip-localpool-config

qa/workunits/mgr/test_localpool.sh: use new config syntax

7 years agocommon/blkdev: fix non-linux build for get_device_id 22423/head
Sage Weil [Mon, 11 Jun 2018 14:36:12 +0000 (09:36 -0500)]
common/blkdev: fix non-linux build for get_device_id

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22467 from zhongyimao/update-the-vstart-options
Kefu Chai [Mon, 11 Jun 2018 13:01:45 +0000 (21:01 +0800)]
Merge pull request #22467 from zhongyimao/update-the-vstart-options

doc: document vstart options

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agocommon/blkdev: do not include vendor in fallback mode
Sage Weil [Mon, 11 Jun 2018 13:00:33 +0000 (08:00 -0500)]
common/blkdev: do not include vendor in fallback mode

The 'vendor' file usually contains a string like "ATA", not an actual
vendor.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: allow open-ended life expectancy
Sage Weil [Fri, 8 Jun 2018 03:04:09 +0000 (22:04 -0500)]
mgr: allow open-ended life expectancy

If you leave off the second part of the life expectancy interval, it means
"more than".  e.g., Setting only the first time to today + one month would
mean "more than one month".

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: use prettified durations for life expectancy
Sage Weil [Fri, 8 Jun 2018 02:48:19 +0000 (21:48 -0500)]
mgr: use prettified durations for life expectancy

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocommon/ceph_time: very lame timespan_str() helper
Sage Weil [Fri, 8 Jun 2018 02:46:52 +0000 (21:46 -0500)]
common/ceph_time: very lame timespan_str() helper

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: "predicted failure" -> "life expectancy", and as a range
Sage Weil [Fri, 8 Jun 2018 02:22:47 +0000 (21:22 -0500)]
mgr: "predicted failure" -> "life expectancy", and as a range

1- Change terminology from "predicted failure" to "life expectancy", which
has a more intuitive meaning.

2- Change from an expected time stamp to a range.  Any expectancy has an
inherent uncertainty, so a single time is not meaningful.  Instead, express
as a range, e.g. "4-6 weeks from now", where we record the min and max
date as well as when the prediction was made.

This is still a bit awkward, but I'm not sure what is better.  It will
always be a bit awkward to express uncertainty since in precise terms it
is probably a 90% confidence interval or something.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: detailed output for 'ceph device ...'
Sage Weil [Wed, 6 Jun 2018 20:23:01 +0000 (15:23 -0500)]
mgr: detailed output for 'ceph device ...'

For JSON ls* commands, dump the full device metadata.

For plain commands, dump tables with useful information.

This is nicer for humans and jq-users, less friendly for those who want
to write a simple bash loop.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr/DaemonState: skip blank device_ids
Sage Weil [Wed, 6 Jun 2018 13:46:51 +0000 (08:46 -0500)]
mgr/DaemonState: skip blank device_ids

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd: skip blank device ids
Sage Weil [Wed, 6 Jun 2018 13:46:10 +0000 (08:46 -0500)]
osd: skip blank device ids

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: track and report device names too
Sage Weil [Wed, 6 Jun 2018 13:44:42 +0000 (08:44 -0500)]
mgr: track and report device names too

If you're looking at a specific device by its unique id, it's also nice
to know what device name (sdb etc) it is and on which host.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr/ActivePyModule: expose 'devices' to python modules
Sage Weil [Tue, 5 Jun 2018 19:21:46 +0000 (14:21 -0500)]
mgr/ActivePyModule: expose 'devices' to python modules

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: move dump/print to DeviceState
Sage Weil [Tue, 5 Jun 2018 19:21:31 +0000 (14:21 -0500)]
mgr: move dump/print to DeviceState

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: implement 'device {set,rm}-predicted-failure' commands
Sage Weil [Tue, 5 Jun 2018 19:03:42 +0000 (14:03 -0500)]
mgr: implement 'device {set,rm}-predicted-failure' commands

Record predicted device failure time.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: load persistent device metadata on mgr stat
Sage Weil [Tue, 5 Jun 2018 18:57:09 +0000 (13:57 -0500)]
mgr: load persistent device metadata on mgr stat

Prune DeviceState when there is no persistent metadata and no daemon
references.  Load persistent metadata from config-key.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoinclude/utime: add parse() method
Sage Weil [Tue, 5 Jun 2018 18:57:17 +0000 (13:57 -0500)]
include/utime: add parse() method

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22009 from markhpc/wip-bs-cache-autotune
Sage Weil [Mon, 11 Jun 2018 12:27:10 +0000 (07:27 -0500)]
Merge pull request #22009 from markhpc/wip-bs-cache-autotune

common/PriorityCache: First Step toward priority based caching

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agomstop.sh: Fix print information 22467/head
Mao Zhongyi [Thu, 7 Jun 2018 07:21:23 +0000 (15:21 +0800)]
mstop.sh: Fix print information

The print information of mstop.sh is as following:

entity=osd pid=16347 name=osd.1
kill  16347...kill  16347...16462
pid=16462
entity=osd pid=16462 name=osd.2
kill  16462...kill  16462...[root@maozy build]#

It's not clear enough, so fix it.

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>
7 years agodoc: vstart update a few options
Mao Zhongyi [Thu, 7 Jun 2018 07:05:28 +0000 (15:05 +0800)]
doc: vstart update a few options

Add missing several options in vstart and make
these options arranged in alphabetical order.

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>
7 years agoMerge pull request #22493 from wjwithagen/wjw-include-types-buffer
Kefu Chai [Mon, 11 Jun 2018 10:40:28 +0000 (18:40 +0800)]
Merge pull request #22493 from wjwithagen/wjw-include-types-buffer

common: include include/types.h early, otherwise Clang will error.

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22494 from wjwithagen/wjw-fix-oath2
Kefu Chai [Mon, 11 Jun 2018 10:38:30 +0000 (18:38 +0800)]
Merge pull request #22494 from wjwithagen/wjw-fix-oath2

cmake: oath lives in liboath

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agomgr/dashboard: Remove unused code 22082/head
Tiago Melo [Tue, 5 Jun 2018 19:23:20 +0000 (20:23 +0100)]
mgr/dashboard: Remove unused code

Signed-off-by: Tiago Melo <tmelo@suse.com>
7 years agomgr/dashboard: Update services to use the new 'provideIn' configuration
Tiago Melo [Tue, 5 Jun 2018 15:43:55 +0000 (16:43 +0100)]
mgr/dashboard: Update services to use the new 'provideIn' configuration

Angular 6 introduced a new way to define where a service should be provided,
which helps during the build of the project.

Signed-off-by: Tiago Melo <tmelo@suse.com>
7 years agomgr/dashboard: Update Angular to version 6
Tiago Melo [Tue, 5 Jun 2018 15:42:55 +0000 (16:42 +0100)]
mgr/dashboard: Update Angular to version 6

Signed-off-by: Tiago Melo <tmelo@suse.com>
7 years agoMerge pull request #22298 from p-na/settings-unset
Lenz Grimmer [Mon, 11 Jun 2018 09:28:47 +0000 (11:28 +0200)]
Merge pull request #22298 from p-na/settings-unset

mgr/dashboard: Reset settings to their default values

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge pull request #21783 from Devp00l/wip-static-testbed
Lenz Grimmer [Mon, 11 Jun 2018 09:01:11 +0000 (11:01 +0200)]
Merge pull request #21783 from Devp00l/wip-static-testbed

mgr/dashboard: Resolve TestBed performance issue

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
7 years agodoc: drop the unsupported option -r from vstart
Mao Zhongyi [Thu, 7 Jun 2018 06:21:05 +0000 (14:21 +0800)]
doc: drop the unsupported option -r from vstart

vstart no longer supports -r option after commit 8dcf8d6e,
but it is not cleaned from mstart, so drop it.

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>
7 years agomessages/MOSDPGLog: encode epoch for query_epoch for pre-nautilus OSDs 22501/head
Sage Weil [Mon, 11 Jun 2018 02:41:54 +0000 (21:41 -0500)]
messages/MOSDPGLog: encode epoch for query_epoch for pre-nautilus OSDs

This fix is analogous to d5a3ae6008e4399b336f3a72d8dee328f97a0002.  As of
84a1984b7cb70c74eaf517ffc424228fa954ed37 we set query_epoch to a lower
value reliably, but we need to encode the latest epoch for pre-nautilus
OSDs because they do not set last_peering_epoch reliably and havin a lower
query_epoch will cause them to discard this message.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd: fix memory leak at EC read 22500/head
cuixf [Mon, 11 Jun 2018 02:06:42 +0000 (22:06 -0400)]
osd: fix memory leak at EC read

Clean the in_progess and shard_to_read_map when the read op is complete,
to prevent memory leak and primary osd crash.

Signed-off-by: xiaofei cui <cuixiaofei@sangfor.com>
7 years agoos/bluestore: fix SharedBlobSet deregister race 22499/head
Sage Weil [Sun, 10 Jun 2018 20:57:31 +0000 (15:57 -0500)]
os/bluestore: fix SharedBlobSet deregister race

In commit 8c8944b2c45ca9dc5b8fd4db1590e1d24206c0b3 we fixed one narrow race
and introduced a new (probably less narrow) one:

A: put shared blob foo, nref = 0, start removing self from set
B: open_shared_blob -> lookup gets nullptr (nref==null), creates a new shared blob
  B: takes lock, sets sb_map[sbid] = newblob
A: gets lock, erases sb_map[sbid]
B: open_shared_blob -> lookup gets null, creates another new shared blob

Fix by only removing the sb_map entry for the nref=0 sb if it still points
to us.  If it doesn't, that means some new blob has shown up in its place.

Fixes: http://tracker.ceph.com/issues/24319
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22460 from liewegas/wip-upgrade-distro
Sage Weil [Sun, 10 Jun 2018 18:42:23 +0000 (13:42 -0500)]
Merge pull request #22460 from liewegas/wip-upgrade-distro

qa/suites/rados/upgrade: remove stray link

7 years agoosd/Session: fix invalid iterator dereference in Sessoin::have_backoff() 22497/head
Sage Weil [Sun, 10 Jun 2018 18:37:56 +0000 (13:37 -0500)]
osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

If p is i->second.end(), we do want to back up a position, but we
shouldn't dereference p for p->first.

Fixes: http://tracker.ceph.com/issues/24486
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/workunits/mgr/test_localpool.sh: use new config syntax 22496/head
Sage Weil [Sun, 10 Jun 2018 18:08:45 +0000 (13:08 -0500)]
qa/workunits/mgr/test_localpool.sh: use new config syntax

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocmake: cmake: oath lives in liboath 22494/head
Willem Jan Withagen [Sat, 9 Jun 2018 11:43:27 +0000 (13:43 +0200)]
cmake: cmake: oath lives in liboath

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
7 years agocommon: use include/types.h early on, otherwise Clang will error. 22493/head
Willem Jan Withagen [Sun, 10 Jun 2018 13:25:46 +0000 (15:25 +0200)]
common: use include/types.h early on, otherwise Clang will error.

This is due that FreeBSD does not have loff_t as type.
And with the wrong order of includes the standard linux typedef
is included and used.

In file included from /home/wjw/wip/src/common/buffer.cc:21:
/home/wjw/wip/src/include/buffer_raw.h:92:41: error: unknown type name 'loff_t'; did you mean 'off_t'?
    virtual int zero_copy_to_fd(int fd, loff_t *offset) {
                                        ^~~~~~
                                        off_t
/usr/include/sys/uio.h:49:17: note: 'off_t' declared here
typedef __off_t off_t;
                ^
1 error generated.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
7 years agoMerge pull request #22487 from tchaikov/wip-24452
Kefu Chai [Sun, 10 Jun 2018 12:51:54 +0000 (20:51 +0800)]
Merge pull request #22487 from tchaikov/wip-24452

osd: always set query_epoch explicitly for MOSDPGLog

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22483 from tchaikov/wip-osd-cleanup
Kefu Chai [Sun, 10 Jun 2018 02:05:53 +0000 (10:05 +0800)]
Merge pull request #22483 from tchaikov/wip-osd-cleanup

osd: do not include Messenger.h if not necessary

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22488 from ai-traders/fix-gcc-install
Kefu Chai [Sun, 10 Jun 2018 01:59:00 +0000 (09:59 +0800)]
Merge pull request #22488 from ai-traders/fix-gcc-install

install-deps.sh: fix installing gcc on ubuntu when no old compiler

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agomgr: tolerate racing config-key change during load_store()
Sage Weil [Tue, 5 Jun 2018 17:55:52 +0000 (12:55 -0500)]
mgr: tolerate racing config-key change during load_store()

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: implement 'device {ls,ls-by-daemon,ls-by-host,info}' commands
Sage Weil [Tue, 5 Jun 2018 13:47:45 +0000 (08:47 -0500)]
mgr: implement 'device {ls,ls-by-daemon,ls-by-host,info}' commands

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr/DaemonState: populate DeviceState structures
Sage Weil [Tue, 5 Jun 2018 13:47:27 +0000 (08:47 -0500)]
mgr/DaemonState: populate DeviceState structures

Track device <-> daemon (and server) mappings.

Signed-off-by: Sage Weil <sage@redhat.com>