]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agodoc/releases/releases.yaml: set target EOL to Jun 1 for L and M 28864/head
Sage Weil [Wed, 3 Jul 2019 15:55:03 +0000 (10:55 -0500)]
doc/releases/releases.yaml: set target EOL to Jun 1 for L and M

Don't list an actual Luminous EOL yet.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/releases: update schedule to describe a 12-month cycle
Sage Weil [Wed, 3 Jul 2019 15:54:39 +0000 (10:54 -0500)]
doc/releases: update schedule to describe a 12-month cycle

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: bucket modification time to local time (#27669)
Lenz Grimmer [Tue, 30 Apr 2019 13:42:25 +0000 (15:42 +0200)]
mgr/dashboard: bucket modification time to local time (#27669)

mgr/dashboard: bucket modification time to local time

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge PR #27875 into master
Sage Weil [Tue, 30 Apr 2019 13:28:21 +0000 (08:28 -0500)]
Merge PR #27875 into master

* refs/pull/27875/head:
mgr/DaemonServer: Fix pgp_num throttling min
mgr/DaemonServer: allow pgp_num increases to be fast if pool is empty(ish)
mgr: fix pgp_num vs merge pacer

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26185 from jmolmo/hosts_management_ls
Sebastian Wagner [Tue, 30 Apr 2019 11:36:43 +0000 (13:36 +0200)]
Merge pull request #26185 from jmolmo/hosts_management_ls

mgr/ansible: Host ls implementation

6 years agoMerge pull request #27883 from majianpeng/crimson-compiler-error
Kefu Chai [Tue, 30 Apr 2019 11:36:24 +0000 (19:36 +0800)]
Merge pull request #27883 from majianpeng/crimson-compiler-error

test/crimson: fix a compiler error.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27607 from sebastian-philipp/test-orchestrator-id-race
Sebastian Wagner [Tue, 30 Apr 2019 11:29:37 +0000 (13:29 +0200)]
Merge pull request #27607 from sebastian-philipp/test-orchestrator-id-race

mgr/test_orchestrator: Fix 'TestWriteCompletion' object has no attribute 'id'

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
6 years agoMerge pull request #27614 from smithfarm/wip-39323
Nathan Cutler [Tue, 30 Apr 2019 08:13:00 +0000 (10:13 +0200)]
Merge pull request #27614 from smithfarm/wip-39323

tests: mgr/tox: make run-tox.sh scripts more robust

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #27613 from Devp00l/wip-issue-23858
Lenz Grimmer [Tue, 30 Apr 2019 07:43:01 +0000 (09:43 +0200)]
Merge pull request #27613 from Devp00l/wip-issue-23858

mgr/dashboard: RBD snapshot name suggestion with local time suffix

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agotest/crimson: fix a compiler error. 27883/head
Jianpeng Ma [Tue, 30 Apr 2019 07:29:08 +0000 (15:29 +0800)]
test/crimson: fix a compiler error.

When with -DWITH_SEASTAR=ON, met the following bug:
/home/ceph/src/test/crimson/test_alien_echo.cc:164:13: error: ‘client’ was not declared in this scope
             client.msgr->set_require_authorizer(false);
             ^~~~~~
/home/ceph/src/test/crimson/test_alien_echo.cc:164:13: note: suggested alternative: ‘client_t’
             client.msgr->set_require_authorizer(false);
             ^~~~~~
             client_t
/home/ceph/src/test/crimson/test_alien_echo.cc: In instantiation of ‘seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)>::<lambda(auto:48&)> [with auto:48 = seastar_pingpong::Client; auto:47 = ceph::net::Messenger*]’:
/usr/include/c++/7/type_traits:2428:26:   required by substitution of ‘template<class _Fn, class ... _Args> static std::__result_of_success<decltype (declval<_Fn>()((declval<_Args>)()...)), std::__invoke_other> std::__result_of_other_impl::_S_test(int) [with _Fn = seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)> [with auto:47 = ceph::net::Messenger*]::<lambda(auto:48&)>; _Args = {seastar_pingpong::Client&}]’
/usr/include/c++/7/type_traits:2439:55:   required from ‘struct std::__result_of_impl<false, false, seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)> [with auto:47 = ceph::net::Messenger*]::<lambda(auto:48&)>, seastar_pingpong::Client&>’
/usr/include/c++/7/type_traits:2444:12:   required from ‘struct std::__invoke_result<seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)> [with auto:47 = ceph::net::Messenger*]::<lambda(auto:48&)>, seastar_pingpong::Client&>’
/usr/include/c++/7/type_traits:2457:12:   required from ‘class std::result_of<seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)> [with auto:47 = ceph::net::Messenger*]::<lambda(auto:48&)>(seastar_pingpong::Client&)>’
/home//ceph/src/seastar/include/seastar/core/do_with.hh:93:17:   required from ‘auto seastar::do_with(T&&, F&&) [with T = seastar_pingpong::Client; F = seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)> [with auto:47 = ceph::net::Messenger*]::<lambda(auto:48&)>]’
/home/ceph/src/test/crimson/test_alien_echo.cc:184:32:   required from ‘seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)> [with auto:47 = ceph::net::Messenger*]’
/usr/include/c++/7/type_traits:2428:26:   required by substitution of ‘template<class _Fn, class ... _Args> static std::__result_of_success<decltype (declval<_Fn>()((declval<_Args>)()...)), std::__invoke_other> std::__result_of_other_impl::_S_test(int) [with _Fn = seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)>; _Args = {ceph::net::Messenger*&&}]’
/usr/include/c++/7/type_traits:2439:55:   required from ‘struct std::__result_of_impl<false, false, seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)>, ceph::net::Messenger*&&>’
/usr/include/c++/7/type_traits:2444:12:   required from ‘struct std::__invoke_result<seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)>, ceph::net::Messenger*&&>’
/usr/include/c++/7/type_traits:2457:12:   required from ‘class std::result_of<seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)>(ceph::net::Messenger*&&)>’
/home//ceph/src/seastar/include/seastar/core/future.hh:980:30:   required by substitution of ‘template<class Func, class Result>  requires  CanApply<Func, ceph::net::Messenger*> Result seastar::future<ceph::net::Messenger*>::then<Func, Result>(Func&&) [with Func = seastar_echo(entity_addr_t, echo_role, unsigned int)::<lambda(auto:47)>; Result = <missing>]’
/home//ceph/src/test/crimson/test_alien_echo.cc:207:8:   required from here
/home/ceph/src/test/crimson/test_alien_echo.cc:190:13: error: base operand of ‘->’ has non-pointer type ‘ceph::net::Messenger’
             client.msgr->set_require_authorizer(false);
             ^~~~~~

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agoMerge pull request #27880 from majianpeng/fio_ceph_messenger
Kefu Chai [Tue, 30 Apr 2019 06:49:05 +0000 (14:49 +0800)]
Merge pull request #27880 from majianpeng/fio_ceph_messenger

test/fio: fix a compiler error.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agotest/fio: fix a compiler error. 27880/head
Jianpeng Ma [Tue, 30 Apr 2019 05:48:47 +0000 (13:48 +0800)]
test/fio: fix a compiler error.

Met the following bug when -DWITH_FIO=ON:
/home/src/test/fio/fio_ceph_messenger.cc:217:5: error: ‘require_authorizer’ was not declared in this scope
     require_authorizer = false;
This bug introduce by f10660e84f85116b9 which omit this.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agomgr/DaemonServer: Fix pgp_num throttling min 27875/head
Sage Weil [Mon, 29 Apr 2019 21:27:58 +0000 (16:27 -0500)]
mgr/DaemonServer: Fix pgp_num throttling min

This was meant to limit the size of any single change to 1/2 of the
max_misplaced (see the comment above!).

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/DaemonServer: allow pgp_num increases to be fast if pool is empty(ish)
Sage Weil [Mon, 29 Apr 2019 21:17:40 +0000 (16:17 -0500)]
mgr/DaemonServer: allow pgp_num increases to be fast if pool is empty(ish)

If we just created a new pool and it's basically empty, there is no reason
to throttle the pgp_num changes.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr: fix pgp_num vs merge pacer
Sage Weil [Mon, 29 Apr 2019 21:17:01 +0000 (16:17 -0500)]
mgr: fix pgp_num vs merge pacer

This condition should only kick in for pgp_num reductions, or else we'll
end up jump to the end instead of slowing down merges.  (For splits, we
don't need to pace like this at all.)

Fixes 76503a1438fa1f166d2c230c73ca8d7b67e6468d

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27867 from tchaikov/wip-ceph-object-corpus
Gregory Farnum [Mon, 29 Apr 2019 20:45:16 +0000 (13:45 -0700)]
Merge pull request #27867 from tchaikov/wip-ceph-object-corpus

ceph-object-corpus: pick up 15.0.0-539-g191ab33faf

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #27730 from cbodley/wip-rgw-boost-170
Casey Bodley [Mon, 29 Apr 2019 17:06:47 +0000 (13:06 -0400)]
Merge pull request #27730 from cbodley/wip-rgw-boost-170

rgw: fix compile errors with boost 1.70

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
6 years agoMerge pull request #27861 from tchaikov/wip-utime-cleanup
Kefu Chai [Mon, 29 Apr 2019 16:17:08 +0000 (00:17 +0800)]
Merge pull request #27861 from tchaikov/wip-utime-cleanup

include/utime: do not cast sec to time_t

Reviewed-by: Samuel Just <sjust@redhat.com>
6 years agoceph-object-corpus: pick up 15.0.0-539-g191ab33faf 27867/head
Kefu Chai [Mon, 29 Apr 2019 16:13:41 +0000 (00:13 +0800)]
ceph-object-corpus: pick up 15.0.0-539-g191ab33faf

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27784 from Devp00l/wip-39483
Lenz Grimmer [Mon, 29 Apr 2019 16:05:26 +0000 (18:05 +0200)]
Merge pull request #27784 from Devp00l/wip-39483

mgr/dashboard: Make all columns sortable

6 years agoMerge PR #27590 into master
Sage Weil [Mon, 29 Apr 2019 16:04:31 +0000 (11:04 -0500)]
Merge PR #27590 into master

* refs/pull/27590/head:
messages: #include necessary header

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27804 into master
Sage Weil [Mon, 29 Apr 2019 16:04:15 +0000 (11:04 -0500)]
Merge PR #27804 into master

* refs/pull/27804/head:
common, osd: remove erroneous 'section' in dump functions
dencoder: include some missed types
common: make utime_t dencoder-compatible
common: make uuid_d dencoder-compatible
osd: make watch_item_t dencoder-compatible

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27832 into master
Sage Weil [Mon, 29 Apr 2019 16:03:53 +0000 (11:03 -0500)]
Merge PR #27832 into master

* refs/pull/27832/head:
qa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning
osd: reenable authorizer on hb messenger for nautilus+
mon/MonClient: do not crash on empty auth request buffer
msg: set_require_authorizer on messenger, not dispatcher
mon/Monitor: allow probe if MMonProbe::mon_release == 0
mon: fix min_mon_release print

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27788 from cyx1231st/wip-seastar-msgr-batch-send
Kefu Chai [Mon, 29 Apr 2019 15:38:16 +0000 (23:38 +0800)]
Merge pull request #27788 from cyx1231st/wip-seastar-msgr-batch-send

crimson/net: improve batching in the write path

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomgr/dashboard: RBD snapshot name suggestion with local time suffix 27613/head
Stephan Müller [Tue, 16 Apr 2019 15:22:41 +0000 (17:22 +0200)]
mgr/dashboard: RBD snapshot name suggestion with local time suffix

The new suffix is still a time that follows the ISO standard as it can
be converted into any other time zone through the time zone prefix of
itself.

Fixes: https://tracker.ceph.com/issues/23858
Signed-off-by: Stephan Müller <smueller@suse.com>
6 years agoMerge pull request #27776 from JunHe77/install-deps
Kefu Chai [Mon, 29 Apr 2019 14:34:10 +0000 (22:34 +0800)]
Merge pull request #27776 from JunHe77/install-deps

install-deps.sh: add option to skip prebuilt boost-* pkgs installation

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/dashboard: Make all columns sortable 27784/head
Stephan Müller [Thu, 25 Apr 2019 12:34:00 +0000 (14:34 +0200)]
mgr/dashboard: Make all columns sortable

Fixes: https://tracker.ceph.com/issues/39483
Signed-off-by: Stephan Müller <smueller@suse.com>
6 years agoinclude/utime: do not cast sec to time_t 27861/head
Kefu Chai [Mon, 29 Apr 2019 12:48:30 +0000 (20:48 +0800)]
include/utime: do not cast sec to time_t

strictly speaking, time_t is an opaque time, and in this context, we
need a uint32_t, so it is not necessary and wrong. let's remove it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/dashboard: auth: fix user lastUpdate timestamp generation (#27604)
Lenz Grimmer [Mon, 29 Apr 2019 12:23:17 +0000 (14:23 +0200)]
mgr/dashboard: auth: fix user lastUpdate timestamp generation (#27604)

mgr/dashboard: auth: fix user lastUpdate timestamp generation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomonitoring: Prometheus default alerts (#27596)
Lenz Grimmer [Mon, 29 Apr 2019 11:36:05 +0000 (13:36 +0200)]
monitoring: Prometheus default alerts (#27596)

monitoring: Prometheus default alerts

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #27837 from cyx1231st/wip-seastar-msgr-fix-read
Kefu Chai [Mon, 29 Apr 2019 10:51:09 +0000 (18:51 +0800)]
Merge pull request #27837 from cyx1231st/wip-seastar-msgr-fix-read

crimson/net: misc fixes in v1 read path

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/net: misc fixes in v1 read path 27837/head
Yingxin Cheng [Mon, 29 Apr 2019 09:03:00 +0000 (17:03 +0800)]
crimson/net: misc fixes in v1 read path

* abort when decoding/signature-check failed;
* change log-level to warn;
* free the decoded message when failed;
* add unlikely to read path;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
6 years agocrimson/net: increase out buffer to 65536 27788/head
Yingxin Cheng [Thu, 11 Apr 2019 14:23:10 +0000 (22:23 +0800)]
crimson/net: increase out buffer to 65536

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
6 years agocrimson/net: group write-related general implementations
Yingxin Cheng [Thu, 25 Apr 2019 22:10:32 +0000 (06:10 +0800)]
crimson/net: group write-related general implementations

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
6 years agocrimson/net: use optional keepalive_ack to notify writes
Yingxin Cheng [Thu, 25 Apr 2019 21:50:19 +0000 (05:50 +0800)]
crimson/net: use optional keepalive_ack to notify writes

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
6 years agocrimson/net: gather message buffers when send
Yingxin Cheng [Thu, 25 Apr 2019 19:50:00 +0000 (03:50 +0800)]
crimson/net: gather message buffers when send

Gather buffers from pending messages/keepalive and send them together.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
6 years agoMerge pull request #27809 from tchaikov/wip-crimson-auth
Kefu Chai [Mon, 29 Apr 2019 01:10:16 +0000 (09:10 +0800)]
Merge pull request #27809 from tchaikov/wip-crimson-auth

crimson,auth: fix FTBFS of crimson-osd and fix v1/v2 auth

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #27831 from tchaikov/wip-crimson-silence-warning
Kefu Chai [Mon, 29 Apr 2019 01:09:15 +0000 (09:09 +0800)]
Merge pull request #27831 from tchaikov/wip-crimson-silence-warning

crimson/net: prefer <fmt/chrono.h> over <fmt/time.h>

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
6 years agoqa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning 27832/head
Sage Weil [Mon, 29 Apr 2019 00:20:57 +0000 (19:20 -0500)]
qa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27756 from dillaman/wip-39025
Mykola Golub [Sun, 28 Apr 2019 17:03:02 +0000 (20:03 +0300)]
Merge pull request #27756 from dillaman/wip-39025

librbd: initial consolidation of internal locks

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agocrimson/osd: silence -Wunused-variable warning 27809/head
Kefu Chai [Sun, 28 Apr 2019 15:31:31 +0000 (23:31 +0800)]
crimson/osd: silence -Wunused-variable warning

silence warning like

/home/kchai/ceph/src/crimson/osd/heartbeat.cc:314:24: warning: unused
variable ‘peer’ [-Wunused-variable]
       auto& [peer, info] = item;
                        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon: log when a monitor is fully connected
Kefu Chai [Fri, 26 Apr 2019 08:35:54 +0000 (16:35 +0800)]
crimson/mon: log when a monitor is fully connected

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon: hold rotating_keyring using a pointer
Kefu Chai [Fri, 26 Apr 2019 08:09:01 +0000 (16:09 +0800)]
crimson/mon: hold rotating_keyring using a pointer

we could have kept rotating_keyring as a plain member variable, but
`AuthClientHandler` keeps a weak reference to it, we are not able to
update it to point it to the new rotating_keyring in the newly created
active_con created from the connected pending connection -- the move
ctor of ceph::mon::Connection does create a new RotatingKeyring. so this
has two consequences:

- the raw pointer held by AuthClientHandler is not valid anymore, after
  the pending connection is destroyed when it is promoted to active_con.
  so we are writing to freed memory when renewing the rotating keyring.
- we won't have access to the updated keyring.

if we use a std::vector<unique_ptr<Connection>> for pending_conns, this
change won't be needed. we could make this change in a future change
though.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoauth,mon,crimson: pass KeyStore by const reference
Kefu Chai [Fri, 26 Apr 2019 08:01:52 +0000 (16:01 +0800)]
auth,mon,crimson: pass KeyStore by const reference

AuthAuthorizeHandler::verify_authorizer() neither changes the keystore,
nor expects a nullptr. so we should pass the keystore by const reference
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/crimson: do not link against ceph-common
Kefu Chai [Fri, 26 Apr 2019 04:03:59 +0000 (12:03 +0800)]
test/crimson: do not link against ceph-common

crimson-common and ceph-common have subtle difference which is
conditionalized by WITH_SEASTAR macro. and the different implementations
share the same name, so we should not link crimson executables against ceph-common .

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoauth: do not use cct unless i have to
Kefu Chai [Fri, 26 Apr 2019 03:39:12 +0000 (11:39 +0800)]
auth: do not use cct unless i have to

crimson messenger's tests does not use ConfigProxy at the time of writing,
and ldout() cannot tell if a log message should be written to the
logfile or not without querying config subsystem, so, to avoid involving
config system in crimson messenger's tests, the log line is disabled if
WITH_SEASTAR, we will kill this workaround once we ditch crimson/net/Config.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon: renew rotating secret if necessary
Kefu Chai [Fri, 26 Apr 2019 02:04:03 +0000 (10:04 +0800)]
crimson/mon: renew rotating secret if necessary

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoauth/Auth: pass utime_t by const reference
Kefu Chai [Fri, 26 Apr 2019 02:03:12 +0000 (10:03 +0800)]
auth/Auth: pass utime_t by const reference

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoauth/cephx: do not require a cct for accessing config
Kefu Chai [Fri, 26 Apr 2019 01:55:59 +0000 (09:55 +0800)]
auth/cephx: do not require a cct for accessing config

crimson/msgr does not depend on ConfigProxy at the time of writing. we
want to decouple it from config subsystem in hope to have less cross
subsystem dependencies and speed up the compiling speed.

but, to introduce cephx to msgr, we need to enable msgr to read from
ConfigProxy at runtime. so, to avoid keeping a CephContext instance
simply for accessing the ConfigProxy reference in it, it would be more
efficient to define a free function `conf()` which hides the difference
between crimson and classic user of auth/cephx.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/net: learn my address from peer
Kefu Chai [Thu, 25 Apr 2019 14:10:16 +0000 (22:10 +0800)]
crimson/net: learn my address from peer

OSD always relies on its server to tell its address, without this
feature, it will advertise a blank ip address to monitor in MOSDBoot.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon: pass cct to verify_authorizer()
Kefu Chai [Thu, 25 Apr 2019 14:04:50 +0000 (22:04 +0800)]
crimson/mon: pass cct to verify_authorizer()

CephxAuthorizeHandler::verify_authorizer() uses it for querying config
and for a RNG, so we have to pass it a CephContext which has `_conf` and
`random()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: pass `Connection*` to Dispatch::ms_dispatch()
Kefu Chai [Thu, 25 Apr 2019 12:31:17 +0000 (20:31 +0800)]
crimson/osd: pass `Connection*` to Dispatch::ms_dispatch()

it's a regression introduced by c69bf0c002f1b62283b3eb5df62fdcb65963b4ae

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/net: sign message and check it if authorizer is available
Kefu Chai [Thu, 25 Apr 2019 12:23:03 +0000 (20:23 +0800)]
crimson/net: sign message and check it if authorizer is available

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon: set entity_name before trying to connect to mon
Kefu Chai [Thu, 25 Apr 2019 12:19:03 +0000 (20:19 +0800)]
crimson/mon: set entity_name before trying to connect to mon

otherwise Monitor.cc or AuthMonitor.cc won't be able to find it in the
keyring.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mgr: connect mgr with v2 protocol
Kefu Chai [Thu, 25 Apr 2019 12:18:21 +0000 (20:18 +0800)]
crimson/mgr: connect mgr with v2 protocol

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: remove ms_get_authorizer() and friends
Kefu Chai [Thu, 25 Apr 2019 05:48:04 +0000 (13:48 +0800)]
crimson: remove ms_get_authorizer() and friends

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/crimson: disable authorizer for tests
Kefu Chai [Sun, 28 Apr 2019 14:46:45 +0000 (22:46 +0800)]
test/crimson: disable authorizer for tests

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: sync msgr's require_authorizer with osdmap
Kefu Chai [Sun, 28 Apr 2019 14:44:40 +0000 (22:44 +0800)]
crimson/osd: sync msgr's require_authorizer with osdmap

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon: tolerate lack of authorizer for some messengers
Kefu Chai [Thu, 25 Apr 2019 05:36:59 +0000 (13:36 +0800)]
crimson/mon: tolerate lack of authorizer for some messengers

also, we should not crash on empty auth request buffer.

see also 64eddc4d and b99daab1

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/net: migrate protocol v1 to auth-client/server
Yingxin Cheng [Fri, 19 Apr 2019 01:44:25 +0000 (09:44 +0800)]
crimson/net: migrate protocol v1 to auth-client/server

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: reenable authorizer on hb messenger for nautilus+
Sage Weil [Sun, 28 Apr 2019 15:37:08 +0000 (10:37 -0500)]
osd: reenable authorizer on hb messenger for nautilus+

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocrimson: add Messenger::{set,get}_require_authorizer()
Kefu Chai [Thu, 18 Apr 2019 14:48:01 +0000 (22:48 +0800)]
crimson: add Messenger::{set,get}_require_authorizer()

This flag is used for compatibility with pre-nautilus OSDs, which do not
send authorizers on the OSD heartbeat connections.  However, because the
AuthServer is implemented by MonClient, which is shared across all
OSD messengers, we can't set this to false for the OSD without disabling
all auth.  Instead, make it a Messenger property, and set it only on the
heartbeat server messengers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon/MonClient: do not crash on empty auth request buffer
Sage Weil [Sun, 28 Apr 2019 13:44:27 +0000 (08:44 -0500)]
mon/MonClient: do not crash on empty auth request buffer

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg: set_require_authorizer on messenger, not dispatcher
Sage Weil [Sun, 28 Apr 2019 13:43:28 +0000 (08:43 -0500)]
msg: set_require_authorizer on messenger, not dispatcher

This flag is used for compatibility with pre-nautilus OSDs, which do not
send authorizers on the OSD heartbeat connections.  However, because the
AuthServer is implemented by MonClient, which is shared across all
OSD messengers, we can't set this to false for the OSD without disabling
all auth.  Instead, make it a Messenger property, and set it only on the
heartbeat server messengers.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agolibrbd: removed 'ImageCtx::parent_lock' 27756/head
Jason Dillaman [Tue, 23 Apr 2019 16:54:30 +0000 (12:54 -0400)]
librbd: removed 'ImageCtx::parent_lock'

The ImageCtx parent-related data structures are now protected
by the image_lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: removed 'ImageCtx::object_map_lock'
Jason Dillaman [Tue, 23 Apr 2019 14:22:30 +0000 (10:22 -0400)]
librbd: removed 'ImageCtx::object_map_lock'

The 'image_lock' now protects the 'object_map' pointer and
the contents of the object-map itself are protected by a
private lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: protect object map with a private member lock
Jason Dillaman [Tue, 23 Apr 2019 13:48:52 +0000 (09:48 -0400)]
librbd: protect object map with a private member lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: removed 'ImageCtx::md_lock'
Jason Dillaman [Fri, 19 Apr 2019 18:46:02 +0000 (14:46 -0400)]
librbd: removed 'ImageCtx::md_lock'

This lock used to protect the IO pathway to prevent writes but
that is now handled by the io::ImageRequestWQ. Additional
historical uses have been temporarily moved to the
'ImageCtx::image_lock'

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: renamed 'ImageCtx::snap_lock' to 'image_lock'
Jason Dillaman [Fri, 19 Apr 2019 20:52:23 +0000 (16:52 -0400)]
librbd: renamed 'ImageCtx::snap_lock' to 'image_lock'

This is the first step in consolidating multiple locks under a single
reader/writer lock for simplicity.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27766 from majianpeng/msg-noneed-set-connection
Kefu Chai [Sun, 28 Apr 2019 11:20:49 +0000 (19:20 +0800)]
Merge pull request #27766 from majianpeng/msg-noneed-set-connection

msg/async: no-need set connection for Message.

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #27767 from tchaikov/wip-buf-denc-const
Kefu Chai [Sun, 28 Apr 2019 11:18:38 +0000 (19:18 +0800)]
Merge pull request #27767 from tchaikov/wip-buf-denc-const

buffer, denc: more constness

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27671 from yuvalif/fix_arn_issue
Yuval Lifshitz [Sun, 28 Apr 2019 08:43:02 +0000 (11:43 +0300)]
Merge pull request #27671 from yuvalif/fix_arn_issue

rgw/pubsub: fix topic arn. tenant support to multisite tests

6 years agocrimson/net: prefer <fmt/chrono.h> over <fmt/time.h> 27831/head
Kefu Chai [Sun, 28 Apr 2019 08:02:06 +0000 (16:02 +0800)]
crimson/net: prefer <fmt/chrono.h> over <fmt/time.h>

in latest libfmt, <fmt/time.h> is deprecated.

to silence warnings like

/home/kchai/ceph/src/fmt/include/fmt/time.h:13:2: warning: #warning
fmt/time.h is deprecated, use fmt/chrono.h instead [-Wcpp]
 #warning fmt/time.h is deprecated, use fmt/chrono.h instead
  ^~~~~~~
In file included from
/home/kchai/ceph/src/seastar/include/seastar/core/reactor.hh:72:0,
                 from
/home/kchai/ceph/src/seastar/include/seastar/core/sharded.hh:24,
                 from /home/kchai/ceph/src/crimson/net/Fwd.h:18,
                 from /home/kchai/ceph/src/crimson/net/Protocol.h:9,
                 from /home/kchai/ceph/src/crimson/net/ProtocolV2.h:6,
                 from /home/kchai/ceph/src/crimson/net/ProtocolV2.cc:4:

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon/Monitor: allow probe if MMonProbe::mon_release == 0
Sage Weil [Sat, 27 Apr 2019 16:03:30 +0000 (11:03 -0500)]
mon/Monitor: allow probe if MMonProbe::mon_release == 0

Mimic doesn't set this, but we still need to allow them to probe us.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27757 from dillaman/wip-39021-2
Mykola Golub [Sat, 27 Apr 2019 15:58:53 +0000 (18:58 +0300)]
Merge pull request #27757 from dillaman/wip-39021-2

librbd: copyup read stats were incorrectly tied to child

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agomon: fix min_mon_release print
Sage Weil [Sat, 27 Apr 2019 15:55:57 +0000 (10:55 -0500)]
mon: fix min_mon_release print

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #18694 into master
Sage Weil [Sat, 27 Apr 2019 15:41:18 +0000 (10:41 -0500)]
Merge PR #18694 into master

* refs/pull/18694/head:
Merge branch 'master' into sk-add_current_num_in_quota_info
mon/OSDMonitor.cc: Add current numbers of objects and bytes

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #22411 into master
Sage Weil [Sat, 27 Apr 2019 15:41:01 +0000 (10:41 -0500)]
Merge PR #22411 into master

* refs/pull/22411/head:
common/WorkQueue: narrow ThreadPool::_lock in func worker.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge PR #27821 into master
Sage Weil [Sat, 27 Apr 2019 15:40:44 +0000 (10:40 -0500)]
Merge PR #27821 into master

* refs/pull/27821/head:
common/options: allow osd_pool_default_pg_autoscale_mode to update a runtime

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27823 into master
Sage Weil [Sat, 27 Apr 2019 15:40:29 +0000 (10:40 -0500)]
Merge PR #27823 into master

* refs/pull/27823/head:
msg/async/ProtocolV1: generate authorizer for mon to mon connections
mon/Monitor: require authorizer from peer monitors

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27798 from neha-ojha/wip-39441
Neha Ojha [Fri, 26 Apr 2019 23:00:53 +0000 (16:00 -0700)]
Merge pull request #27798 from neha-ojha/wip-39441

osd/PG: do not use approx_missing_objects pre-nautilus

Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #27209 from dzafman/wip-38840
David Zafman [Fri, 26 Apr 2019 22:23:37 +0000 (15:23 -0700)]
Merge pull request #27209 from dzafman/wip-38840

osd: Don't evict after a flush if intersecting scrub range

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27769 from dzafman/wip-39333
David Zafman [Fri, 26 Apr 2019 18:55:04 +0000 (11:55 -0700)]
Merge pull request #27769 from dzafman/wip-39333

osd-backfill-space.sh test failed in TEST_backfill_multi_partial()

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27732 into master
Sage Weil [Fri, 26 Apr 2019 18:24:01 +0000 (13:24 -0500)]
Merge PR #27732 into master

* refs/pull/27732/head:
common/assert: include ceph_abort_msg(arg) arg in log output

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge PR #27729 into master
Sage Weil [Fri, 26 Apr 2019 18:23:36 +0000 (13:23 -0500)]
Merge PR #27729 into master

* refs/pull/27729/head:
osd: take heartbeat_lock when calling heartbeat()

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #26176 into master
Sage Weil [Fri, 26 Apr 2019 18:23:21 +0000 (13:23 -0500)]
Merge PR #26176 into master

* refs/pull/26176/head:
os/bluestore: do not check osd_max_object_size in _open_path()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27786 into master
Sage Weil [Fri, 26 Apr 2019 18:23:04 +0000 (13:23 -0500)]
Merge PR #27786 into master

* refs/pull/27786/head:
qa/suites/upgrade/*-x-singleton: enable bluestore debugging settings

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agomsg/async/ProtocolV1: generate authorizer for mon to mon connections 27823/head
Sage Weil [Fri, 26 Apr 2019 18:07:31 +0000 (13:07 -0500)]
msg/async/ProtocolV1: generate authorizer for mon to mon connections

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/Monitor: require authorizer from peer monitors
Sage Weil [Fri, 26 Apr 2019 18:07:06 +0000 (13:07 -0500)]
mon/Monitor: require authorizer from peer monitors

Mon to mon links use authorizers.  It's only non-mons connecting to mons
that use MAuth messages.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon, osd: remove erroneous 'section' in dump functions 27804/head
Greg Farnum [Fri, 26 Apr 2019 18:03:53 +0000 (11:03 -0700)]
common, osd: remove erroneous 'section' in dump functions

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
6 years agotest: osd-backfill-spsace.sh doesn't matter which PG wins the race 27769/head
David Zafman [Thu, 25 Apr 2019 03:39:02 +0000 (20:39 -0700)]
test: osd-backfill-spsace.sh doesn't matter which PG wins the race

Fixes: http://tracker.ceph.com/issues/39333
Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agocommon/options: allow osd_pool_default_pg_autoscale_mode to update a runtime 27821/head
Sage Weil [Fri, 26 Apr 2019 16:26:21 +0000 (11:26 -0500)]
common/options: allow osd_pool_default_pg_autoscale_mode to update a runtime

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27789 into master
Sage Weil [Fri, 26 Apr 2019 15:40:47 +0000 (10:40 -0500)]
Merge PR #27789 into master

* refs/pull/27789/head:
os/bluestore: correctly measure deferred writes into new blobs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge PR #27655 into master
Sage Weil [Fri, 26 Apr 2019 15:40:31 +0000 (10:40 -0500)]
Merge PR #27655 into master

* refs/pull/27655/head:
common/options: flag misc ms_* options STARTUP
common/options: flag misc options STARTUP
common/options: mark cluster log options with FLAG_RUNTIME, use get_val
common/options: mark a bunch of options with FLAG_STARTUP

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27724 from idryomov/wip-rbd-copyup-zeros
Jason Dillaman [Fri, 26 Apr 2019 12:11:58 +0000 (08:11 -0400)]
Merge pull request #27724 from idryomov/wip-rbd-copyup-zeros

librbd: snapshot object maps can go inconsistent during copyup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27755 from trociny/wip-rbd-io-scheduler-leak
Jason Dillaman [Fri, 26 Apr 2019 12:11:17 +0000 (08:11 -0400)]
Merge pull request #27755 from trociny/wip-rbd-io-scheduler-leak

librbd: leak on canceling simple io scheduler timer task

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agomonitoring: add a few prometheus alerts 27596/head
Jan Fajerski [Mon, 15 Apr 2019 13:35:09 +0000 (15:35 +0200)]
monitoring: add a few prometheus alerts

Alerts are from
https://github.com/SUSE/DeepSea/blob/SES5/srv/salt/ceph/monitoring/prometheus/files/ses_default_alerts.yml
but updated for the mgr module and node_exporter >= 0.15.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
6 years agodencoder: include some missed types
Greg Farnum [Fri, 26 Apr 2019 04:25:03 +0000 (21:25 -0700)]
dencoder: include some missed types

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
6 years agocommon: make utime_t dencoder-compatible
Greg Farnum [Fri, 26 Apr 2019 05:57:38 +0000 (22:57 -0700)]
common: make utime_t dencoder-compatible

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
6 years agocommon: make uuid_d dencoder-compatible
Greg Farnum [Fri, 26 Apr 2019 05:25:31 +0000 (22:25 -0700)]
common: make uuid_d dencoder-compatible

Signed-off-by: Greg Farnum <gfarnum@redhat.com>