]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #25746 from tchaikov/wip-pg-autoscale-mode
Kefu Chai [Thu, 3 Jan 2019 06:10:15 +0000 (14:10 +0800)]
Merge pull request #25746 from tchaikov/wip-pg-autoscale-mode

osd/OSDMap: set pg_autoscale_mode with setting from conf

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25588 from tchaikov/wip-auth-silence-gcc-warning
Kefu Chai [Thu, 3 Jan 2019 06:09:20 +0000 (14:09 +0800)]
Merge pull request #25588 from tchaikov/wip-auth-silence-gcc-warning

cls/rbd: init local var with known value

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
6 years agocls/rbd: init local var with known value 25588/head
Kefu Chai [Thu, 20 Dec 2018 03:24:28 +0000 (11:24 +0800)]
cls/rbd: init local var with known value

DirectoryState does not have an "invalid" enum so far, since it's
defined using `enum class`, init a value of this type with a known value
could be a better choice even it is always initialized before being read.

this silences the GCC warning of:

src/cls/rbd/cls_rbd.cc:3147:3: warning: ‘on_disk_directory_state’ may be
used uninitialized in this function [-Wmaybe-uninitialized]
   if (directory_state != on_disk_directory_state) {
   ^~

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25687 from tchaikov/wip-pg-stats
Kefu Chai [Thu, 3 Jan 2019 01:41:46 +0000 (09:41 +0800)]
Merge pull request #25687 from tchaikov/wip-pg-stats

osd/PrimaryLogPG: do not count failed read in delta_stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25705 from joscollin/wip-doc-common
Gregory Farnum [Wed, 2 Jan 2019 22:21:28 +0000 (14:21 -0800)]
Merge pull request #25705 from joscollin/wip-doc-common

doc: Fix Create a Cluster url in Running Multiple Clusters

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #25450 from ivancich/wip-rgw-category-cleanup
J. Eric Ivancich [Wed, 2 Jan 2019 22:00:28 +0000 (17:00 -0500)]
Merge pull request #25450 from ivancich/wip-rgw-category-cleanup

rgw: clean-up -- use enum class for stats category

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
6 years agorgw: clean-up -- remove unnessary "struct"s 25450/head
J. Eric Ivancich [Thu, 13 Dec 2018 20:55:20 +0000 (15:55 -0500)]
rgw: clean-up -- remove unnessary "struct"s

In C++ "struct" is unnecessary when referencing a defined
struct. Clean up all uses in src/cls/rgw source files.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agorgw: clean-up use enum class for stats category
J. Eric Ivancich [Wed, 5 Dec 2018 15:18:22 +0000 (10:18 -0500)]
rgw: clean-up use enum class for stats category

The stats entries for rgw buckets has a category, which used a
combination of uint8_t and enum RGWObjClass. Clean this up by
converting RGWObjClass to an enum class and using that
throughout. This provides type safety and better code clarity. Also,
add some source code documentation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #24676 from TheJJ/cryptsetup-allow-discards
Alfredo Deza [Wed, 2 Jan 2019 18:23:21 +0000 (13:23 -0500)]
Merge pull request #24676 from TheJJ/cryptsetup-allow-discards

ceph-volume: enable device discards

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoosd/OSDMap: set pg_autoscale_mode with setting from conf 25746/head
Kefu Chai [Wed, 2 Jan 2019 16:20:13 +0000 (00:20 +0800)]
osd/OSDMap: set pg_autoscale_mode with setting from conf

* update build_simple_optioned() to set pg_autoscale_mode with the setting
  read from conf, otherwise it will be a random value in heap.
* update cli test accordingly, otherwise we will have

/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/osdmaptool/create-racks.t:
failed
---
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/osdmaptool/create-racks.t
+++
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/osdmaptool/create-racks.t.err
@@ -796,7 +796,7 @@
   nearfull_ratio 0
   min_compat_client jewel

-  pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash
rjenkins pg_num 15296 pgp_num 15296 last_change 0 flags hashpspool
stripe_width 0 application rbd
+  pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash
rjenkins pg_num 15296 pgp_num 15296 autoscale_mode ??? last_change 0
flags hashpspool stripe_width 0 application rbd

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25441 from theanalyst/decode-refcount-types
Kefu Chai [Wed, 2 Jan 2019 16:45:50 +0000 (00:45 +0800)]
Merge pull request #25441 from theanalyst/decode-refcount-types

cls: refcount: add obj_refcount to ceph-dencoder

Reviewed-By: Casey Bodley <cbodley@redhat.com>
6 years agoceph-volume: enable device discards 24676/head
Jonas Jelten [Fri, 19 Oct 2018 13:15:03 +0000 (15:15 +0200)]
ceph-volume: enable device discards

When using SSDs as encrypted OSD device, discards do not pass the
encryption layer. This option activates discard requests.

Signed-off-by: Jonas Jelten <jj@stusta.net>
6 years agoMerge pull request #25541 from theanalyst/wip-es-log-fixes
Abhishek L [Wed, 2 Jan 2019 12:56:37 +0000 (13:56 +0100)]
Merge pull request #25541 from theanalyst/wip-es-log-fixes

rgw: sync module: avoid printing attrs of objects in log

Reviewed-By: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25731 from liewegas/wip-37775
Kefu Chai [Wed, 2 Jan 2019 10:41:01 +0000 (18:41 +0800)]
Merge pull request #25731 from liewegas/wip-37775

osd: reliably send pg_created messages to the mon

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25732 from liewegas/wip-37776
Kefu Chai [Tue, 1 Jan 2019 16:04:39 +0000 (00:04 +0800)]
Merge pull request #25732 from liewegas/wip-37776

qa/workunits/rados/test_health_warnings: prevent out osds

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25726 from tchaikov/wip-37751
Kefu Chai [Tue, 1 Jan 2019 15:49:27 +0000 (23:49 +0800)]
Merge pull request #25726 from tchaikov/wip-37751

osd: unlock osd_lock when tweaking osd settings

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25733 from tchaikov/wip-37719-workaround
Kefu Chai [Tue, 1 Jan 2019 15:44:23 +0000 (23:44 +0800)]
Merge pull request #25733 from tchaikov/wip-37719-workaround

librbd: workaround an ICE of GCC

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agocrimson: workaround an ICE of GCC 25733/head
Kefu Chai [Tue, 1 Jan 2019 08:38:15 +0000 (16:38 +0800)]
crimson: workaround an ICE of GCC

this change works around the FTBFS on arm64:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/crimson/common/config_proxy.h:74:13:
internal compiler error: in tsubst_decomp_names, at cp/pt.c:16537
             for (auto& [obs, keys] : rev_obs) {
             ^~~
Please submit a full bug report,
with preprocessed source if appropriate.

it seems that this issue is a dup of
https://bugzilla.redhat.com/show_bug.cgi?id=1639019 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibrbd: workaround an ICE of GCC
Kefu Chai [Tue, 1 Jan 2019 07:49:05 +0000 (15:49 +0800)]
librbd: workaround an ICE of GCC

GCC is somehow annoyed at seeing the combination of decltype and
initializer_list in this place. i tried to remove the `if` clause, and
only left the `else` block, GCC was happy with that change. i also tried
to pass an empty `{}` to `decltype(reply.lockers)`, and GCC was also
happy with that. so i guess there are multiple factors taking effect in
this problem. probably any of them could be the last straw that breaks
GCC.

but we cannot have a minimal reproducer for this issue here without more
efforts. and `reply.lockers` is empty after `reply` is constructed, so
it would be simpler if we just add the locker info to it instead of
assigning a newly constructed `map` to it.

Fixes: http://tracker.ceph.com/issues/37719
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25640 from joscollin/wip-mon-parse
Kefu Chai [Tue, 1 Jan 2019 06:39:59 +0000 (14:39 +0800)]
Merge pull request #25640 from joscollin/wip-mon-parse

mon,osd: avoid str copy in parse

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25534 from trociny/wip-36244
Kefu Chai [Tue, 1 Jan 2019 06:12:23 +0000 (14:12 +0800)]
Merge pull request #25534 from trociny/wip-36244

mgr: fix crash due to multiple sessions from daemons with same name

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25597 into master
Sage Weil [Tue, 1 Jan 2019 04:31:18 +0000 (22:31 -0600)]
Merge PR #25597 into master

* refs/pull/25597/head:
mgr/hello: define some module options
pybind/mgr/mgr_module: normalize defaults to str when type is missing
mgr/telegraf: specify option types
mgr/telemetry: specify option types
mgr/zabbix: specify option types
mgr/localpool: document options and specify in native types
mgr/devicehealth: document options and specify in native types
mgr/balancer: document and specify options
common/options: make runtime vs not runtime explicit, not type-dependent
mon/MgrMonitor: mark module options with FLAG_MGR
mon/MgrMonitor: make find_module_option handle localized option names
pybind/mgr/mgr_module: set values as string
mgr: return options as appropriate python type
mgr/PythonCompat: python 3 cludges
pybind/mgr/mgr_module: make use of defined default value
mgr/PyModule: populate ModuleOptions and expose to mon
mon/ConfigMonitor: unify module options with built-in options
mon/MgrMonitor: populate options
mon/MgrMap: define ModuleOptions and include in map
common/options: expand type helpers and make them static
common/options: pin enums to values

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge PR #25652 into master
Sage Weil [Tue, 1 Jan 2019 04:30:40 +0000 (22:30 -0600)]
Merge PR #25652 into master

* refs/pull/25652/head:
osd/OSDMap: disallow new upmaps on pgs that are pending merge
osd/PG: align past_intervals and last_epoch_clean for fabricated merge target

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #25730 into master
Sage Weil [Tue, 1 Jan 2019 04:30:03 +0000 (22:30 -0600)]
Merge PR #25730 into master

* refs/pull/25730/head:
Revert "os/objectstore: add new op OP_CREATE for create a new object."

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #25670 into master
Sage Weil [Tue, 1 Jan 2019 04:28:47 +0000 (22:28 -0600)]
Merge PR #25670 into master

* refs/pull/25670/head:
os/bluestore: KernelDevice::read() does the EIO mapping now.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #25408 into master
Sage Weil [Mon, 31 Dec 2018 21:24:26 +0000 (15:24 -0600)]
Merge PR #25408 into master

* refs/pull/25408/head:
mgr/hello: Make use of `HandleCommandResult`.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoqa/workunits/rados/test_health_warnings: prevent out osds 25732/head
Sage Weil [Mon, 31 Dec 2018 17:18:05 +0000 (11:18 -0600)]
qa/workunits/rados/test_health_warnings: prevent out osds

Fixes: http://tracker.ceph.com/issues/37776
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd: reliably send pg_created messages to the mon 25731/head
Sage Weil [Mon, 31 Dec 2018 17:05:03 +0000 (11:05 -0600)]
osd: reliably send pg_created messages to the mon

The OSD has to reliably deliver a pg_created message to the mon in order
for the mon to clear the pool's CREATING flag.  Previously, a mon
connection reset would drop the message.

Restructure this to:

- queue a message any time a PG peers and the pool as the CREATING flag
- track pending messages in OSDService
- resend on mon connect
- prune messages for pools that no longer have the CREATING flag

This new strategy can result in resends of these messages to the mon in
cases where the mon already knows the PG was created.  However, pool
creation is rare, and these extra messages are cheap.  And we can avoid
this overhead if we like by limiting the number of PGs that the mon can
create explicitly if we choose (by lowering mon_osd_max_initial_pgs).

Fixes: http://tracker.ceph.com/issues/37775
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25437 from Aran85/fix-objecter-reset
Kefu Chai [Mon, 31 Dec 2018 16:56:52 +0000 (00:56 +0800)]
Merge pull request #25437 from Aran85/fix-objecter-reset

objecter: avoid race when reset down osd's session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #21138 from zouaiguo/fix_mgr_restful_osd_error
Kefu Chai [Mon, 31 Dec 2018 16:52:38 +0000 (00:52 +0800)]
Merge pull request #21138 from zouaiguo/fix_mgr_restful_osd_error

mgr/restful: fix py got exception when get osd info

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoRevert "os/objectstore: add new op OP_CREATE for create a new object." 25730/head
Kefu Chai [Mon, 31 Dec 2018 15:04:20 +0000 (23:04 +0800)]
Revert "os/objectstore: add new op OP_CREATE for create a new object."

This reverts commit 22f73ba62eb1fe337622ea2aefdcd829f8def851.

Fixes: http://tracker.ceph.com/issues/37774
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #22385 from majianpeng/touch-omit-kvdb
Kefu Chai [Mon, 31 Dec 2018 14:35:09 +0000 (22:35 +0800)]
Merge pull request #22385 from majianpeng/touch-omit-kvdb

os/objectstore: add a new op OP_CREATE.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25653 from rhcs-dashboard/37717-update-health-test
Kefu Chai [Mon, 31 Dec 2018 14:30:13 +0000 (22:30 +0800)]
Merge pull request #25653 from rhcs-dashboard/37717-update-health-test

mgr/dashboard: updated API test

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: unlock osd_lock when tweaking osd settings 25726/head
Kefu Chai [Sun, 30 Dec 2018 13:57:04 +0000 (21:57 +0800)]
osd: unlock osd_lock when tweaking osd settings

unlock osd_lock when serving "debug kick_recovery_wq" command

we need to unlock osd_lock temporarily when updating the osd settings,
otherwise we will run into assert failure. because
OSD::handle_conf_change() acquires the osd_lock which is not a recursive
lock.

Fixes: http://tracker.ceph.com/issues/37751
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: use unlock_guard for unlock osd temporarily
Kefu Chai [Sun, 30 Dec 2018 13:46:55 +0000 (21:46 +0800)]
osd: use unlock_guard for unlock osd temporarily

when OSD::do_command() gets called, osd_lock is acquired. but when
serving some of these commands, we need to call methods which also
acquire the osd_lock by themselves. for instance,
OSD::handle_conf_change() gets called by cct->_conf.apply_changes().
to allow them to do so, we unlock osd_lock before calling those methods,
and re-lock it after done with them.

unlock_guard is introduced to unlock and re-lock the lock in a RAII style.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25723 from falcon78921/wip-doc-37770
Kefu Chai [Sun, 30 Dec 2018 09:40:48 +0000 (17:40 +0800)]
Merge pull request #25723 from falcon78921/wip-doc-37770

doc: fix the formatting of HTTP Frontends documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agodoc: cleanup HTTP Frontends documentation 25723/head
James McClune [Sat, 29 Dec 2018 02:32:24 +0000 (21:32 -0500)]
doc: cleanup HTTP Frontends documentation

Added literal block to /etc/ceph/ceph.conf example.

Fixes: https://tracker.ceph.com/issues/37770
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
6 years agomon: avoid str copy in parse 25640/head
Jos Collin [Thu, 20 Dec 2018 15:20:33 +0000 (20:50 +0530)]
mon: avoid str copy in parse

Avoids the input string copy.
It doesn't make any difference when the input string is copied.

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agodoc: Fixed double spaces 25705/head
Jos Collin [Wed, 26 Dec 2018 05:53:36 +0000 (11:23 +0530)]
doc: Fixed double spaces

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agodoc: Fixed Create a Cluster url
Jos Collin [Wed, 26 Dec 2018 05:51:59 +0000 (11:21 +0530)]
doc: Fixed Create a Cluster url

Fixes: http://tracker.ceph.com/issues/37764
Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agoMerge pull request #25702 from wjwithagen/wjw-fix-bluestore-warning
Xie Xingguo [Wed, 26 Dec 2018 00:52:57 +0000 (08:52 +0800)]
Merge pull request #25702 from wjwithagen/wjw-fix-bluestore-warning

bluestore: silence Clang warning on possible uninitialize usuage

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge PR #25591 into master
Patrick Donnelly [Tue, 25 Dec 2018 18:21:04 +0000 (10:21 -0800)]
Merge PR #25591 into master

* refs/pull/25591/head:
mds: clean up log messages for standby-replay

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agobluestore: silence Clang warning on possible uninitialize usuage 25702/head
Willem Jan Withagen [Tue, 25 Dec 2018 14:52:41 +0000 (15:52 +0100)]
bluestore: silence Clang warning on possible uninitialize usuage

```
/home/wjw/ceph.head/ceph/src/os/bluestore/BlueStore.cc:5935:15: note: initialize the variable 'reserved' to silence this warning
  int reserved;
              ^
               = 0
1 warning generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agomgr/dashboard: updated health controller & API test 25653/head
alfonsomthd [Mon, 24 Dec 2018 13:45:34 +0000 (14:45 +0100)]
mgr/dashboard: updated health controller & API test

Updated health controller & test to reflect changes introduced
in 'df' payload.

Return 'total_used_raw_bytes' instead of 'total_used_bytes'
to match CLI 'bin/rados df' used/avail summary in
Landing Page (frontend component).

Do not return 'stats_by_class' to save bandwidth as they are
not needed (right now) in the dashboard.

Fixes: https://tracker.ceph.com/issues/37717
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoMerge pull request #25688 from wjwithagen/wjw-fix-kerberos
Kefu Chai [Mon, 24 Dec 2018 11:38:18 +0000 (19:38 +0800)]
Merge pull request #25688 from wjwithagen/wjw-fix-kerberos

install-deps.sh: Add Kerberos requirement for FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agobuild: Add Kerberos requirement for FreeBSD 25688/head
Willem Jan Withagen [Sun, 23 Dec 2018 20:24:45 +0000 (21:24 +0100)]
build: Add Kerberos requirement for FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #25675 from dzafman/wip-36525
Kefu Chai [Sun, 23 Dec 2018 16:16:00 +0000 (00:16 +0800)]
Merge pull request #25675 from dzafman/wip-36525

test: Disable duplicate request command test during scrub testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25582 from trociny/wip-osd-perf-query-wrs
Kefu Chai [Sun, 23 Dec 2018 16:12:47 +0000 (00:12 +0800)]
Merge pull request #25582 from trociny/wip-osd-perf-query-wrs

osd: Weighted Random Sampling for dynamic perf stats

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoosd/PrimaryLogPG: do not count failed read in delta_stats 25687/head
Kefu Chai [Sun, 23 Dec 2018 14:37:21 +0000 (22:37 +0800)]
osd/PrimaryLogPG: do not count failed read in delta_stats

ctx->delta_stats is accumulated to PG.info.stats.stats in
ReplicatedBackend::submit_transaction() before queueing the transaction.

and `stats` is used for stats reporting. so i think we should not take
the failed ops into account.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25584 from thinkercui/bugfix
Kefu Chai [Sun, 23 Dec 2018 14:30:58 +0000 (22:30 +0800)]
Merge pull request #25584 from thinkercui/bugfix

osd/PrimaryLogPG: fix the extent length error of the sync read

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #25626 from xiexingguo/wip-no-upmap-for-merge
Kefu Chai [Sun, 23 Dec 2018 14:28:49 +0000 (22:28 +0800)]
Merge pull request #25626 from xiexingguo/wip-no-upmap-for-merge

mgr/balancer: skip auto-balancing for pools with pending pg-merge

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25645 from sebastian-philipp/orchestrator-ansible-minor-cleanup
Kefu Chai [Sun, 23 Dec 2018 14:28:12 +0000 (22:28 +0800)]
Merge pull request #25645 from sebastian-philipp/orchestrator-ansible-minor-cleanup

mgr/ansible: Python 3 fix

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25619 into master
Patrick Donnelly [Sun, 23 Dec 2018 04:16:13 +0000 (20:16 -0800)]
Merge PR #25619 into master

* refs/pull/25619/head:
mds: use frag_vec_t for efficient frag storage
mds: do not copy session map accidentally
mds: use vector for subtree access
include/frag: use more efficient small vectors
StackStringStream: add strv method short-cut
include: add small_vector encode/decode/<<
mds: scope using directives

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge PR #25618 into master
Sage Weil [Sat, 22 Dec 2018 22:04:55 +0000 (16:04 -0600)]
Merge PR #25618 into master

* refs/pull/25618/head:
mgr/dashboard: Updated API pool tests

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agoMerge PR #25648 into master
Patrick Donnelly [Sat, 22 Dec 2018 20:34:57 +0000 (12:34 -0800)]
Merge PR #25648 into master

* refs/pull/25648/head:
client: add latency statistics for a read/fsync operation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #24797 into master
Patrick Donnelly [Sat, 22 Dec 2018 20:33:19 +0000 (12:33 -0800)]
Merge PR #24797 into master

* refs/pull/24797/head:
mds: handle state change race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #23126 into master
Patrick Donnelly [Sat, 22 Dec 2018 20:27:57 +0000 (12:27 -0800)]
Merge PR #23126 into master

* refs/pull/23126/head:
wherever: guard handle_conf_change() from concurrent execution
config: drop config_proxy::lock when invoking config observer

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25636 from trociny/wip-prometheus-rbd-ns
Jason Dillaman [Sat, 22 Dec 2018 15:25:32 +0000 (10:25 -0500)]
Merge pull request #25636 from trociny/wip-prometheus-rbd-ns

pybind/mgr/prometheus: rbd stats namespace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25666 from trociny/wip-instance-watcher-assert
Jason Dillaman [Sat, 22 Dec 2018 15:25:17 +0000 (10:25 -0500)]
Merge pull request #25666 from trociny/wip-instance-watcher-assert

rbd_mirror: assert no requests on destroying InstanceWatcher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25667 from trociny/wip-37672
Jason Dillaman [Sat, 22 Dec 2018 15:24:22 +0000 (10:24 -0500)]
Merge pull request #25667 from trociny/wip-37672

test/librados_test_stub: handle object doesn't exist gracefully

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agotest: Disable duplicate request command test during scrub testing 25675/head
David Zafman [Fri, 21 Dec 2018 21:25:10 +0000 (13:25 -0800)]
test: Disable duplicate request command test during scrub testing

Scrub testing requires an orderly control of scrubbing.  Most but not
all the time, the duplicate scrub request is ignored because the first
request hasn't finished.  Teuthology enables this environment variable
in the workunit handling.

Fixes: https://tracker.ceph.com/issues/36525
Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #25671 from tchaikov/wip-seastar-pthread
Kefu Chai [Sat, 22 Dec 2018 01:51:52 +0000 (09:51 +0800)]
Merge pull request #25671 from tchaikov/wip-seastar-pthread

seastar: pickup the change to link pthread

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #25608 from tchaikov/bluestore_fbsd
Kefu Chai [Sat, 22 Dec 2018 01:39:28 +0000 (09:39 +0800)]
Merge pull request #25608 from tchaikov/bluestore_fbsd

os/bluestore: support for FreeBSD

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #25536 into master
Patrick Donnelly [Sat, 22 Dec 2018 01:16:11 +0000 (17:16 -0800)]
Merge PR #25536 into master

* refs/pull/25536/head:
mds: don't wait for snapclient sync when handling reconnect
mds: extend reconnect period when mds is busy
mds: renew client session when receiving any message
mds: use mono_clock for reconnect timeout

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #25658 into master
Patrick Donnelly [Sat, 22 Dec 2018 01:13:05 +0000 (17:13 -0800)]
Merge PR #25658 into master

* refs/pull/25658/head:
MDSMonitor: allow beacons from stopping MDS that was laggy

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agodo_freebsd.sh: enable WITH_BLUESTORE 25608/head
Kefu Chai [Tue, 18 Dec 2018 14:42:42 +0000 (22:42 +0800)]
do_freebsd.sh: enable WITH_BLUESTORE

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: s/event/events/
Kefu Chai [Tue, 18 Dec 2018 11:46:02 +0000 (19:46 +0800)]
os/bluestore: s/event/events/

because `events` is an array, so name it with a plural name.

Signed-off-by: Alan Somers <asomers@gmail.com>
6 years agoos/bluestore: support for FreeBSD
Alan Somers [Mon, 25 Sep 2017 20:53:14 +0000 (14:53 -0600)]
os/bluestore: support for FreeBSD

Signed-off-by: Alan Somers <asomers@gmail.com>
6 years agoos: rename aio.h to avoid conflict
Kefu Chai [Tue, 18 Dec 2018 11:31:15 +0000 (19:31 +0800)]
os: rename aio.h to avoid conflict

FreeBSD's libc offers <aio.h>.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25620 into master
Sage Weil [Fri, 21 Dec 2018 16:12:42 +0000 (10:12 -0600)]
Merge PR #25620 into master

* refs/pull/25620/head:
os/bluestore: more robust handling for lack of per-pool stats cases.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoseastar: pickup the change to link pthread 25671/head
Kefu Chai [Fri, 21 Dec 2018 16:06:00 +0000 (00:06 +0800)]
seastar: pickup the change to link pthread

otherwise seastar applications might fail to link on ubuntu xenial due
to unresolvable symbols like thread_setname_np()

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25632 into master
Sage Weil [Fri, 21 Dec 2018 15:49:37 +0000 (09:49 -0600)]
Merge PR #25632 into master

* refs/pull/25632/head:
test: add a json tester
ceph_json: parser sets data on all types when parsing

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #22800 from wjwithagen/wjw-ctest-timeout
Kefu Chai [Fri, 21 Dec 2018 15:40:27 +0000 (23:40 +0800)]
Merge pull request #22800 from wjwithagen/wjw-ctest-timeout

cmake: allow setting of the CTest timeout during building.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: KernelDevice::read() does the EIO mapping now. 25670/head
Radoslaw Zarzynski [Fri, 21 Dec 2018 11:22:20 +0000 (12:22 +0100)]
os/bluestore: KernelDevice::read() does the EIO mapping now.

Fixes: http://tracker.ceph.com/issues/36455
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomgr/hello: define some module options 25597/head
Sage Weil [Fri, 21 Dec 2018 14:55:34 +0000 (08:55 -0600)]
mgr/hello: define some module options

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agotest/librados_test_stub: handle object doesn't exist gracefully 25667/head
Mykola Golub [Fri, 21 Dec 2018 11:30:09 +0000 (13:30 +0200)]
test/librados_test_stub: handle object doesn't exist gracefully

Fixes: https://tracker.ceph.com/issues/37672
Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agotest/rbd_mirror: fix typo in InstanceWatcher mock test
Mykola Golub [Fri, 21 Dec 2018 11:23:00 +0000 (13:23 +0200)]
test/rbd_mirror: fix typo in InstanceWatcher mock test

Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agorbd_mirror: assert no requests on destroying InstanceWatcher 25666/head
Mykola Golub [Fri, 21 Dec 2018 11:18:49 +0000 (13:18 +0200)]
rbd_mirror: assert no requests on destroying InstanceWatcher

Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agomgr/hello: Make use of `HandleCommandResult`. 25408/head
Sebastian Wagner [Wed, 5 Dec 2018 11:41:13 +0000 (12:41 +0100)]
mgr/hello: Make use of `HandleCommandResult`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #25661 from joscollin/wip-doc-changing-monitors
Kefu Chai [Fri, 21 Dec 2018 06:46:48 +0000 (14:46 +0800)]
Merge pull request #25661 from joscollin/wip-doc-changing-monitors

doc: fix typo in add-or-rm-mons

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25639 from oliveiradan/wip-krb_pr25566_cleanup_oliveiradan
Kefu Chai [Fri, 21 Dec 2018 06:36:51 +0000 (14:36 +0800)]
Merge pull request #25639 from oliveiradan/wip-krb_pr25566_cleanup_oliveiradan

auth/krb: Fix Kerberos build warnings

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25537 from jecluis/dev/wip-mgr-module-error
Kefu Chai [Fri, 21 Dec 2018 06:09:40 +0000 (14:09 +0800)]
Merge pull request #25537 from jecluis/dev/wip-mgr-module-error

mgr: make module error message more descriptive

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #25595 from rzarzynski/wip-ec-drop_bl_crosstalk_dependency
Kefu Chai [Fri, 21 Dec 2018 06:06:08 +0000 (14:06 +0800)]
Merge pull request #25595 from rzarzynski/wip-ec-drop_bl_crosstalk_dependency

ec: lrc doesn't depend on crosstalks between bufferlists anymore.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25503 from ceph/rbd-mirror-ceph-base
Kefu Chai [Fri, 21 Dec 2018 06:04:34 +0000 (14:04 +0800)]
Merge pull request #25503 from ceph/rbd-mirror-ceph-base

spec: requires ceph base instead of common

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc: fix typo in add-or-rm-mons 25661/head
Jos Collin [Fri, 21 Dec 2018 05:17:04 +0000 (10:47 +0530)]
doc: fix typo in add-or-rm-mons

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agopybind/mgr/mgr_module: normalize defaults to str when type is missing
Sage Weil [Wed, 19 Dec 2018 22:59:09 +0000 (16:59 -0600)]
pybind/mgr/mgr_module: normalize defaults to str when type is missing

If the module doesn't specify a type for the option, it is probably
an older module.  Make sure that the default value we provide is normalized
to a string, just in case the module declares something like

  {
     'name': 'my_option',
     'default': True,
  }

where the default is *not* a string, but an user-set value coming from
the config infrastructure would be a string.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/telegraf: specify option types
Sage Weil [Wed, 19 Dec 2018 22:52:33 +0000 (16:52 -0600)]
mgr/telegraf: specify option types

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/telemetry: specify option types
Sage Weil [Wed, 19 Dec 2018 22:52:23 +0000 (16:52 -0600)]
mgr/telemetry: specify option types

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/zabbix: specify option types
Sage Weil [Tue, 18 Dec 2018 20:25:09 +0000 (14:25 -0600)]
mgr/zabbix: specify option types

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/localpool: document options and specify in native types
Sage Weil [Tue, 18 Dec 2018 20:22:31 +0000 (14:22 -0600)]
mgr/localpool: document options and specify in native types

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/devicehealth: document options and specify in native types
Sage Weil [Tue, 18 Dec 2018 20:21:26 +0000 (14:21 -0600)]
mgr/devicehealth: document options and specify in native types

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/balancer: document and specify options
Sage Weil [Mon, 17 Dec 2018 20:47:06 +0000 (14:47 -0600)]
mgr/balancer: document and specify options

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/options: make runtime vs not runtime explicit, not type-dependent
Sage Weil [Wed, 19 Dec 2018 00:06:27 +0000 (18:06 -0600)]
common/options: make runtime vs not runtime explicit, not type-dependent

For built-in types, we have a kludge that non-string types all appear to
be runtime-updateable.  There's no such distinction for module options.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/MgrMonitor: mark module options with FLAG_MGR
Sage Weil [Wed, 19 Dec 2018 00:05:36 +0000 (18:05 -0600)]
mon/MgrMonitor: mark module options with FLAG_MGR

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/MgrMonitor: make find_module_option handle localized option names
Sage Weil [Wed, 19 Dec 2018 00:05:02 +0000 (18:05 -0600)]
mon/MgrMonitor: make find_module_option handle localized option names

If we get "mgr/$module/$instance/$option", normalize it to
"mgr/$module/$option" before looking it up.  That's because a
'mgr/dashboard/x/server_port' in our database is really the
'mgr/dashboard/server_port' option.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agopybind/mgr/mgr_module: set values as string
Sage Weil [Tue, 18 Dec 2018 20:19:06 +0000 (14:19 -0600)]
pybind/mgr/mgr_module: set values as string

The C++ code expects a string, and stores everything as a string, so
just pass it in that way.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr: return options as appropriate python type
Sage Weil [Tue, 18 Dec 2018 20:11:30 +0000 (14:11 -0600)]
mgr: return options as appropriate python type

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/PythonCompat: python 3 cludges
Sage Weil [Thu, 20 Dec 2018 13:07:51 +0000 (07:07 -0600)]
mgr/PythonCompat: python 3 cludges

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agopybind/mgr/mgr_module: make use of defined default value
Sage Weil [Mon, 17 Dec 2018 21:06:49 +0000 (15:06 -0600)]
pybind/mgr/mgr_module: make use of defined default value

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/OSDMap: disallow new upmaps on pgs that are pending merge 25652/head
Sage Weil [Thu, 20 Dec 2018 16:03:10 +0000 (10:03 -0600)]
osd/OSDMap: disallow new upmaps on pgs that are pending merge

It's critical that we keep the source and target PGs mapped to
the same place.  Prevent new pg_upmaps on merge source or target
PGs.

Signed-off-by: Sage Weil <sage@redhat.com>