]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agodoc/mgr/dashboard: note dashboard as a separate package 26815/head
Kefu Chai [Thu, 7 Mar 2019 14:31:17 +0000 (22:31 +0800)]
doc/mgr/dashboard: note dashboard as a separate package

Fixes: https://tracker.ceph.com/issues/38584
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodoc/releases/nautilus: dashboard package notes
Kefu Chai [Thu, 7 Mar 2019 12:08:44 +0000 (20:08 +0800)]
doc/releases/nautilus: dashboard package notes

Fixes: https://tracker.ceph.com/issues/38584
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26716 from rjfd/wip-dashboard-ganesha-docs
Ricardo Dias [Thu, 7 Mar 2019 12:10:37 +0000 (12:10 +0000)]
Merge pull request #26716 from rjfd/wip-dashboard-ganesha-docs

mgr/dashboard: nfs-ganesha: controller API documentation

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoMerge PR #26775 into master
Ilya Dryomov [Thu, 7 Mar 2019 11:19:36 +0000 (12:19 +0100)]
Merge PR #26775 into master

* refs/pull/26775/head:
qa/suites/krbd/wac: bluestore snippet is placed incorrectly

Reviewed-by: Mike Christie <mchristi@redhat.com>
6 years agoMerge pull request #26778 from xiexingguo/wip-fix-wrong-lpr
Xie Xingguo [Thu, 7 Mar 2019 08:40:59 +0000 (16:40 +0800)]
Merge pull request #26778 from xiexingguo/wip-fix-wrong-lpr

osd/PrimaryLogPG: fix last_peering_reset checking on manifest flushing

Reviewed-by: Myoungwon Oh <ommw@sk.com>
6 years agoMerge pull request #26633 from jtlayton/wip-nfs-scale
Sebastian Wagner [Wed, 6 Mar 2019 18:08:48 +0000 (19:08 +0100)]
Merge pull request #26633 from jtlayton/wip-nfs-scale

mgr/orchestrator: Allow the orchestrator to scale the NFS server count

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge PR #26748 into master
Sage Weil [Wed, 6 Mar 2019 15:22:42 +0000 (09:22 -0600)]
Merge PR #26748 into master

* refs/pull/26748/head:
ceph_test_rados_[a]io_pp: add test verifying read-into-bufferlist behavior

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agomgr/orchestrator: allow scaling the NFS server count up and down 26633/head
Jeff Layton [Mon, 25 Feb 2019 14:21:08 +0000 (09:21 -0500)]
mgr/orchestrator: allow scaling the NFS server count up and down

Add a new 'ceph orchestrator nfs update' command that will take the
NFS clustername and a new count as arguments. That will get translated
to a StatelessServiceSpec and passed to update_stateless_service.

Also, add the necessary stubs to the test_orchestrator and the CLI
QA test.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agomgr/rook: allow scaling nfs count
Jeff Layton [Mon, 25 Feb 2019 14:27:02 +0000 (09:27 -0500)]
mgr/rook: allow scaling nfs count

Allow rook to handle scaling the NFS server count up and down in an NFS
cluster. We just manifest these changes as change to the
spec.server.active field in the CRD.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agoorchestrator: drop id_ argument from update_stateless_service
Jeff Layton [Thu, 28 Feb 2019 17:33:04 +0000 (12:33 -0500)]
orchestrator: drop id_ argument from update_stateless_service

...we have a "name" field in StatelessServiceSpec.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agomgr/orchestrator: just keep a single count value in StatelessServiceSpec
Jeff Layton [Tue, 26 Feb 2019 19:53:45 +0000 (14:53 -0500)]
mgr/orchestrator: just keep a single count value in StatelessServiceSpec

We currently have a min_size/max_size values in here, but we don't have
any orchestrators that can take advantage of two values. Let's just keep
a simple count for now, until we do.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #26388 from tspmelo/wip-iscsi-no-auth-ui
Ricardo Marques [Wed, 6 Mar 2019 12:11:05 +0000 (12:11 +0000)]
Merge pull request #26388 from tspmelo/wip-iscsi-no-auth-ui

mgr/dashboard: Add UI for disabling ACL authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #26382 from ricardoasmarques/no-auth
Volker Theile [Wed, 6 Mar 2019 11:45:33 +0000 (12:45 +0100)]
Merge pull request #26382 from ricardoasmarques/no-auth

mgr/dashboard: iSCSI - Add support for disabling ACL authentication (backend)

Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #26783 from cyx1231st/wip-fix-async-echo
Kefu Chai [Wed, 6 Mar 2019 10:15:09 +0000 (18:15 +0800)]
Merge pull request #26783 from cyx1231st/wip-fix-async-echo

test/crimson: add dummy_auth to test_async_echo

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agotest/crimson: add dummy_auth to test_async_echo 26783/head
Yingxin Cheng [Wed, 6 Mar 2019 09:29:04 +0000 (17:29 +0800)]
test/crimson: add dummy_auth to test_async_echo

fix echo between async-msgrs

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
6 years agoosd/PrimaryLogPG: fix last_peering_reset checking on manifest flushing 26778/head
xie xingguo [Wed, 6 Mar 2019 06:11:16 +0000 (14:11 +0800)]
osd/PrimaryLogPG: fix last_peering_reset checking on manifest flushing

```handle_manifest_flush``` is obviously using the wrong
**last_peering_reset** to check whether a new peering procedure
has been re-initialized by then.

Fix by using a different alias of the local copy of the
pg-wide **last_peering_reset** variable, which is less confusing and
error-prone.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #26749 into master
Sage Weil [Wed, 6 Mar 2019 03:12:19 +0000 (21:12 -0600)]
Merge PR #26749 into master

* refs/pull/26749/head:
common/util: report CONTAINER_IMAGE in daemon metadata

Reviewed-by: Sébastien Han <seb@redhat.com>
6 years agoMerge pull request #26774 from liu-chunmei/ceph_seastar_osd_preboot
Kefu Chai [Wed, 6 Mar 2019 02:42:55 +0000 (10:42 +0800)]
Merge pull request #26774 from liu-chunmei/ceph_seastar_osd_preboot

crimson/osd: correct the order of parameters passed to OSD::_preboot()

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: correct the order of parameters passed to OSD::_preboot() 26774/head
chunmei Liu [Wed, 6 Mar 2019 01:34:28 +0000 (17:34 -0800)]
crimson/osd: correct the order of parameters passed to OSD::_preboot()

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
6 years agoqa/suites/krbd/wac: bluestore snippet is placed incorrectly 26775/head
Ilya Dryomov [Tue, 5 Mar 2019 22:07:27 +0000 (23:07 +0100)]
qa/suites/krbd/wac: bluestore snippet is placed incorrectly

Instead of generating three tests, each with bluestore-bitmap.yaml, it
generates four tests: one consisting of just bluestore-bitmap.yaml and
the other three without any trace of bluestore.  This was introduced in
commit 711df71790fa ("qa: objectstore snippets for krbd").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agoMerge PR #26740 into master
Sage Weil [Tue, 5 Mar 2019 20:05:54 +0000 (14:05 -0600)]
Merge PR #26740 into master

* refs/pull/26740/head:
rpm: advance spec file copyright date

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
6 years agoMerge PR #26767 into master
Sage Weil [Tue, 5 Mar 2019 17:08:03 +0000 (11:08 -0600)]
Merge PR #26767 into master

* refs/pull/26767/head:
Revert "msg/async/Protocol*: send keep alive if existing wins"

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26719 into master
Sage Weil [Tue, 5 Mar 2019 15:12:43 +0000 (09:12 -0600)]
Merge PR #26719 into master

* refs/pull/26719/head:
rpm: re-add pyOpenSSL as ceph-mgr runtime dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25540 into master
Sage Weil [Tue, 5 Mar 2019 15:12:26 +0000 (09:12 -0600)]
Merge PR #25540 into master

* refs/pull/25540/head:
mgr/ServiceMap: show service's id on summary

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26567 into master
Sage Weil [Tue, 5 Mar 2019 15:11:53 +0000 (09:11 -0600)]
Merge PR #26567 into master

* refs/pull/26567/head:
os/bluestore: fix bluefs inability to reclaim space

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge PR #26752 into master
Sage Weil [Tue, 5 Mar 2019 15:10:33 +0000 (09:10 -0600)]
Merge PR #26752 into master

* refs/pull/26752/head:
msg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26742 into master
Sage Weil [Tue, 5 Mar 2019 15:09:29 +0000 (09:09 -0600)]
Merge PR #26742 into master

* refs/pull/26742/head:
osd/PG: do not touch this->cct after PG is destroyed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #26460 into master
Patrick Donnelly [Tue, 5 Mar 2019 14:25:19 +0000 (06:25 -0800)]
Merge PR #26460 into master

* refs/pull/26460/head:
client: parameter "cap" is not used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
Xie Xingguo [Tue, 5 Mar 2019 12:16:18 +0000 (20:16 +0800)]
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions

mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoRevert "msg/async/Protocol*: send keep alive if existing wins" 26767/head
xie xingguo [Tue, 5 Mar 2019 11:46:09 +0000 (19:46 +0800)]
Revert "msg/async/Protocol*: send keep alive if existing wins"

This reverts commit 61b9432ef9a3847eceb96f8d5a854567c49bbf61.

If we are in the middle of replacing, we can not queue any further
write events into the old center because we may end up replacing
existing connection's center with a new one, and hence executing
the newly queued write events in the old thread.

See **transfer_existing** for a detailed description.

Also the patch does not make a lot of sense for the original issue
it tried to resolve, because **send_keepalive** is a pure noop if the
underlying connection is not ready, which is obviously true for the
case demonstrated in http://tracker.ceph.com/issues/38493..

Fixes: http://tracker.ceph.com/issues/38569
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoclient: parameter "cap" is not used 26460/head
huanwen ren [Fri, 15 Feb 2019 18:58:01 +0000 (02:58 +0800)]
client: parameter "cap" is not used

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
6 years agoMerge pull request #26715 from sebastian-philipp/script-pypy
Sebastian Wagner [Tue, 5 Mar 2019 09:18:18 +0000 (10:18 +0100)]
Merge pull request #26715 from sebastian-philipp/script-pypy

src/script: add run_mypy to run static type checking on Python code

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomon/OSDMonitor: add boundary check for pool recovery_priority 26729/head
xie xingguo [Sat, 2 Mar 2019 01:43:58 +0000 (09:43 +0800)]
mon/OSDMonitor: add boundary check for pool recovery_priority

See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #26444 into master
Sage Weil [Mon, 4 Mar 2019 23:26:12 +0000 (17:26 -0600)]
Merge PR #26444 into master

* refs/pull/26444/head:
ceph_features: deprecate MON_SINGLE_PAXOS
osdmap: map versions to releases
mon: retire some `osd set/unset` flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #26754 into master
Sage Weil [Mon, 4 Mar 2019 23:15:41 +0000 (17:15 -0600)]
Merge PR #26754 into master

* refs/pull/26754/head:
osdc: remove lderr() hexdump from handle_osd_op_reply

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26744 into master
Sage Weil [Mon, 4 Mar 2019 23:13:30 +0000 (17:13 -0600)]
Merge PR #26744 into master

* refs/pull/26744/head:
mds: fix encoded version of mds_info_t

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26694 into master
Sage Weil [Mon, 4 Mar 2019 22:25:18 +0000 (16:25 -0600)]
Merge PR #26694 into master

* refs/pull/26694/head:
rpm: drop use of $FIRST_ARG

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
6 years agoMerge pull request #26753 from ceph/ceph-volume-osd-ports
Alfredo Deza [Mon, 4 Mar 2019 21:58:36 +0000 (16:58 -0500)]
Merge pull request #26753 from ceph/ceph-volume-osd-ports

ceph-volume: set number of osd ports in the tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoosdc: remove lderr() hexdump from handle_osd_op_reply 26754/head
Casey Bodley [Mon, 4 Mar 2019 21:29:51 +0000 (16:29 -0500)]
osdc: remove lderr() hexdump from handle_osd_op_reply

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #26631 into master
Sage Weil [Mon, 4 Mar 2019 21:00:09 +0000 (15:00 -0600)]
Merge PR #26631 into master

* refs/pull/26631/head:
msg/async, v2: fix unused variable warning in ::discard_out_queue().

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoceph-volume: set number of osd ports in the tests 26753/head
Andrew Schoen [Mon, 4 Mar 2019 20:43:37 +0000 (14:43 -0600)]
ceph-volume: set number of osd ports in the tests

The number of ports the OSDs listen on depend on the version of ceph
being used, so we need to test for that number accordingly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph_test_rados_[a]io_pp: add test verifying read-into-bufferlist behavior 26748/head
Sage Weil [Mon, 4 Mar 2019 18:52:42 +0000 (12:52 -0600)]
ceph_test_rados_[a]io_pp: add test verifying read-into-bufferlist behavior

libradosstriper relies on this, and other callers may too.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode. 26752/head
Radoslaw Zarzynski [Mon, 4 Mar 2019 15:21:47 +0000 (16:21 +0100)]
msg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #26745 into master
Sage Weil [Mon, 4 Mar 2019 19:45:23 +0000 (13:45 -0600)]
Merge PR #26745 into master

* refs/pull/26745/head:
ceph_release: we're in rc phase

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agocommon/util: report CONTAINER_IMAGE in daemon metadata 26749/head
Sage Weil [Mon, 4 Mar 2019 19:04:22 +0000 (13:04 -0600)]
common/util: report CONTAINER_IMAGE in daemon metadata

Rook will set this to the registry + image string.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26741 from cbodley/wip-test-rgw-reshard-wait2
Casey Bodley [Mon, 4 Mar 2019 17:30:48 +0000 (12:30 -0500)]
Merge pull request #26741 from cbodley/wip-test-rgw-reshard-wait2

test/rgw: fix race in test_rgw_reshard_wait

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoceph_release: we're in rc phase 26745/head
Sage Weil [Mon, 4 Mar 2019 17:17:20 +0000 (11:17 -0600)]
ceph_release: we're in rc phase

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomds: fix encoded version of mds_info_t 26744/head
Patrick Donnelly [Mon, 4 Mar 2019 17:06:16 +0000 (09:06 -0800)]
mds: fix encoded version of mds_info_t

Introduced-by: 1f5892f82046ebaac3b66dbd8e686825ebb1775c
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #26635 from noahdesu/cherrypy-ipv6
Sebastian Wagner [Mon, 4 Mar 2019 16:51:34 +0000 (17:51 +0100)]
Merge pull request #26635 from noahdesu/cherrypy-ipv6

mgr/dashboard: fix for using '::' on hosts without ipv6

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #26739 from alfredodeza/wip-rm38568
Andrew Schoen [Mon, 4 Mar 2019 16:29:33 +0000 (10:29 -0600)]
Merge pull request #26739 from alfredodeza/wip-rm38568

ceph-volume: replace testinfra command with py.test

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge PR #26734 into master
Patrick Donnelly [Mon, 4 Mar 2019 15:39:08 +0000 (07:39 -0800)]
Merge PR #26734 into master

* refs/pull/26734/head:
doc: correct vstart.sh file location

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoosd/PG: do not touch this->cct after PG is destroyed 26742/head
Sage Weil [Mon, 4 Mar 2019 15:13:06 +0000 (09:13 -0600)]
osd/PG: do not touch this->cct after PG is destroyed

- thread A drops last second-to-last ref
- thread B drops last ref
- thread B deletes pg
- thread A does

  lgeneric_subdout(cct, refs, 1) << "PG::put " << this << " "
  ..

touching this->cct, a use-after-free that valgrind notices with

  <kind>InvalidRead</kind>
  <what>Invalid read of size 8</what>
  <stack>
    <frame>
      <ip>0x74ED34</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>PG::put(char const*)</fn>
      <dir>/usr/src/debug/ceph-14.1.0-283-g569f086/src/osd</dir>
      <file>PG.cc</file>
      <line>179</line>
    </frame>
    <frame>
      <ip>0x7C1E76</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>ContainerContext&lt;boost::intrusive_ptr&lt;PG&gt; &gt;::~ContainerContext()</fn>
      <dir>/usr/src/debug/ceph-14.1.0-283-g569f086/src/osd</dir>
      <file>PG.h</file>
      <line>566</line>
    </frame>
    <frame>
      <ip>0xC95675</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Finisher::finisher_thread_entry()</fn>
      <dir>/usr/src/debug/ceph-14.1.0-283-g569f086/src/common</dir>
      <file>Finisher.cc</file>
      <line>67</line>
    </frame>
    <frame>
      <ip>0xD656E24</ip>
      <obj>/usr/lib64/libpthread-2.17.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0xE5B4BAC</ip>
      <obj>/usr/lib64/libc-2.17.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
  <auxwhat>Address 0x470b29e8 is 152 bytes inside a block of size 11,632 free'd</auxwhat>
  <stack>
  ...

Fixes: http://tracker.ceph.com/issues/38484
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26682 from yuvalif/add_rgw_xml_unit_tests
Casey Bodley [Mon, 4 Mar 2019 15:04:17 +0000 (10:04 -0500)]
Merge pull request #26682 from yuvalif/add_rgw_xml_unit_tests

Add rgw xml unit tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26680 from yuvalif/add_kill_9_to_mstop
Casey Bodley [Mon, 4 Mar 2019 15:03:34 +0000 (10:03 -0500)]
Merge pull request #26680 from yuvalif/add_kill_9_to_mstop

mstop.sh allow kill -9 after failing to kill procs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agorpm: advance spec file copyright date 26740/head
Nathan Cutler [Mon, 4 Mar 2019 14:50:24 +0000 (15:50 +0100)]
rpm: advance spec file copyright date

Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge PR #26704 into master
Sage Weil [Mon, 4 Mar 2019 14:44:56 +0000 (08:44 -0600)]
Merge PR #26704 into master

* refs/pull/26704/head:
msg/async, v2: drop alloc_aligned_buffer().
msg/async, v2: introduce frame late abort facility.
Revert "msg/async, v2: move ceph_msg_header2 to last frame segment."
msg, msg/async, v2: introduce late message abort facility.
msg/async, v2: failure of msg decode doesn't block throtlles.
msg/async, v2: move ceph_msg_header2 to last frame segment.
fixup: use frame epilogue for crc32 integrity checking.
msg/async, v2: epilogue size is variable in secure mode.
msg/async, v2: drop support for the buggy rx_buffers mechanism.
Revert "msg/async, v2: add flags field to frame's epilogue."
msg/async, v2: add flags field to frame's epilogue.
msg/async, v2: drop onwire_segment_t as epilogue had derogated it.
msg, msg/async, v2: drop crc fields from ceph_msg_header2.
msg/async, v2: use frame epilogue for crc32 integrity checking.
msg/async, v2: clean the ProtocolV2::{front,middle,data} up.
msg/async, v2: clean the ProtocolV2::epilogue up.
msg/async, v2: move crypto processing to segment reader.
msg/async, v2: handle epilogue separately from payload/data.
msg/async, v2: dissect decryption from SignedEncryptedFrame.
msg/async, v2: unify WAIT frames with other payload frames.
msg/async, v2: implement epilogue handling in secure mode.
msg/async, v2: message frames are pre-dispatched now.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agotest/rgw: fix race in test_rgw_reshard_wait 26741/head
Casey Bodley [Mon, 4 Mar 2019 14:44:24 +0000 (09:44 -0500)]
test/rgw: fix race in test_rgw_reshard_wait

start the timer before scheduling the wait

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #26696 into master
Sage Weil [Mon, 4 Mar 2019 14:32:18 +0000 (08:32 -0600)]
Merge PR #26696 into master

* refs/pull/26696/head:
osdc/Objecter: preserve read-into-existing-buffer behavior
osdc/Objecter: don't use post_rx_buffer or revoke_rx_buffer at all
msg/Connection: disable {post,revoke}_rx_buffer
msg/async/ProtocolV1: disable rx_buffers behavior
msg/simple/Pipe: disable rx_buffer code

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #26727 into master
Sage Weil [Mon, 4 Mar 2019 14:13:34 +0000 (08:13 -0600)]
Merge PR #26727 into master

* refs/pull/26727/head:
qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap
osd: add 'send_beacon' tell/asok command(s)

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoceph-volume tests.simple replace testinfra command for py.test 26739/head
Alfredo Deza [Mon, 4 Mar 2019 14:05:07 +0000 (09:05 -0500)]
ceph-volume tests.simple replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume tests.lvm replace testinfra command for py.test
Alfredo Deza [Mon, 4 Mar 2019 14:04:51 +0000 (09:04 -0500)]
ceph-volume tests.lvm replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume tests.batch replace testinfra command for py.test
Alfredo Deza [Mon, 4 Mar 2019 14:04:36 +0000 (09:04 -0500)]
ceph-volume tests.batch replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agosrc/script: add run_mypy to run static type checking on Python code 26715/head
Sebastian Wagner [Fri, 1 Mar 2019 10:59:39 +0000 (11:59 +0100)]
src/script: add run_mypy to run static type checking on Python code

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agodoc: correct vstart.sh file location 26734/head
Changcheng Liu [Mon, 4 Mar 2019 08:55:31 +0000 (16:55 +0800)]
doc: correct vstart.sh file location

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agoMerge pull request #26683 from dillaman/wip-38500
Mykola Golub [Mon, 4 Mar 2019 08:43:35 +0000 (10:43 +0200)]
Merge pull request #26683 from dillaman/wip-38500

librbd: race condition possible when validating RBD pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agomsg/async, v2: drop alloc_aligned_buffer(). 26704/head
Radoslaw Zarzynski [Mon, 4 Mar 2019 00:51:57 +0000 (01:51 +0100)]
msg/async, v2: drop alloc_aligned_buffer().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomsg/async, v2: introduce frame late abort facility.
Radoslaw Zarzynski [Mon, 4 Mar 2019 00:43:43 +0000 (01:43 +0100)]
msg/async, v2: introduce frame late abort facility.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #26698 into master
Sage Weil [Sun, 3 Mar 2019 17:39:52 +0000 (11:39 -0600)]
Merge PR #26698 into master

* refs/pull/26698/head:
common/str_map: better trim() impl using boost
common/str_map: fix trim() on empty string

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosdc/Objecter: preserve read-into-existing-buffer behavior 26696/head
Sage Weil [Fri, 1 Mar 2019 20:12:42 +0000 (14:12 -0600)]
osdc/Objecter: preserve read-into-existing-buffer behavior

The libradosstriper::RadosStriperImpl::aio_read populates the target
outbl with a static buffer and relies on us reading into it.  This was
actually not reliable in the past (it could fail if the rx_buffers
optimization failed due to a retransmit or something else) but nevertheless
libradosstriper requires it to work *at all*.

Resolve this by modifying Objecter to copy the result into any provided
buffer at the lowest layer.  This should capture any other such user who
needed this behavior.

On the other hand, it will break any user who inadvertantly reads into a
non-empty bufferlist.  Given that any such user would already previously
have seen bad behavior due to the rx_buffers optimization, we expect
there to be 0 such instances.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap 26727/head
Sage Weil [Fri, 1 Mar 2019 21:48:24 +0000 (15:48 -0600)]
qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap

This test introduces a map gap.  What *should* happen is that when there is
such a gap, we cannot import.  Previously, the test didn't reliably produce
a map gap at all, and didn't check that import failed--it verified that it
passed.

Fix the test so that it reliably produces a gap *and* reports
min_last_epoch_clean to the mon so we can trim.  Then verify we fail to
import, but can with --force.  But remove the pg again, because if we
force an import with a map gap the osd will refuse to start.

Fixes: http://tracker.ceph.com/issues/38525
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd: add 'send_beacon' tell/asok command(s)
Sage Weil [Fri, 1 Mar 2019 21:47:17 +0000 (15:47 -0600)]
osd: add 'send_beacon' tell/asok command(s)

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25538 from mkogan1/wip-rgw-reshard-status
J. Eric Ivancich [Sun, 3 Mar 2019 13:58:20 +0000 (08:58 -0500)]
Merge pull request #25538 from mkogan1/wip-rgw-reshard-status

rgw: change the "rgw admin status" 'num_shards' output to signed int

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agorgw: add xml attributes test, override xml_handle_data 26682/head
Yuval Lifshitz [Sun, 3 Mar 2019 10:02:37 +0000 (12:02 +0200)]
rgw: add xml attributes test, override xml_handle_data

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
6 years agoMerge pull request #26668 from xiexingguo/wip-38493
Xie Xingguo [Sun, 3 Mar 2019 06:43:15 +0000 (14:43 +0800)]
Merge pull request #26668 from xiexingguo/wip-38493

msg/async/Protocol*: send keep alive if existing wins

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26723 from liewegas/wip-38537
Kefu Chai [Sun, 3 Mar 2019 02:09:14 +0000 (10:09 +0800)]
Merge pull request #26723 from liewegas/wip-38537

mgr/BaseMgrModule: drop GIL for ceph_send_command

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoRevert "msg/async, v2: move ceph_msg_header2 to last frame segment."
Radoslaw Zarzynski [Sat, 2 Mar 2019 20:11:42 +0000 (21:11 +0100)]
Revert "msg/async, v2: move ceph_msg_header2 to last frame segment."

This reverts commit 69dbc92ba8af844be0e3d3fc1b01287082753e56.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #26726 into master
Patrick Donnelly [Sat, 2 Mar 2019 19:24:22 +0000 (11:24 -0800)]
Merge PR #26726 into master

* refs/pull/26726/head:
mds: Fix use of auto prototype, it is a non std extension

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomgr/dashboard: nfs-ganesha: controller API documentation 26716/head
Ricardo Dias [Thu, 28 Feb 2019 15:21:49 +0000 (15:21 +0000)]
mgr/dashboard: nfs-ganesha: controller API documentation

Signed-off-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge PR #26720 into master
Sage Weil [Sat, 2 Mar 2019 14:31:54 +0000 (08:31 -0600)]
Merge PR #26720 into master

* refs/pull/26720/head:
ceph-osd: infer objectstore type of 'type' file is missing

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agomsg, msg/async, v2: introduce late message abort facility.
Radoslaw Zarzynski [Sat, 2 Mar 2019 13:13:48 +0000 (14:13 +0100)]
msg, msg/async, v2: introduce late message abort facility.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomsg/async, v2: failure of msg decode doesn't block throtlles.
Radoslaw Zarzynski [Sat, 2 Mar 2019 13:07:17 +0000 (14:07 +0100)]
msg/async, v2: failure of msg decode doesn't block throtlles.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoceph_features: deprecate MON_SINGLE_PAXOS 26444/head
Joao Eduardo Luis [Thu, 14 Feb 2019 15:34:07 +0000 (15:34 +0000)]
ceph_features: deprecate MON_SINGLE_PAXOS

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
6 years agoosdmap: map versions to releases
Joao Eduardo Luis [Thu, 14 Feb 2019 14:22:05 +0000 (14:22 +0000)]
osdmap: map versions to releases

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
6 years agomon: retire some `osd set/unset` flags
Joao Eduardo Luis [Thu, 14 Feb 2019 13:55:30 +0000 (13:55 +0000)]
mon: retire some `osd set/unset` flags

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
6 years agoMerge pull request #26705 from dzafman/wip-23999
Xie Xingguo [Sat, 2 Mar 2019 01:43:33 +0000 (09:43 +0800)]
Merge pull request #26705 from dzafman/wip-23999

Improve docs osd_recovery_priority, osd_recovery_op_priority and related

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #26666 from badone/wip-kvstore_tool-ice
Brad Hubbard [Fri, 1 Mar 2019 23:13:58 +0000 (09:13 +1000)]
Merge pull request #26666 from badone/wip-kvstore_tool-ice

tools/ceph_kvstore_tool: Move summary output to print_summary

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agocommon, doc: Improve docs osd_recovery_priority, osd_recovery_op_priority and related 26705/head
David Zafman [Wed, 6 Feb 2019 01:05:18 +0000 (17:05 -0800)]
common, doc: Improve docs osd_recovery_priority, osd_recovery_op_priority and related

Add option desciptions for osd_recovery_priority and osd_recovery_op_priority

Fixes: https://tracker.ceph.com/issues/23999
Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoceph-osd: infer objectstore type of 'type' file is missing 26720/head
Sage Weil [Fri, 1 Mar 2019 14:57:36 +0000 (08:57 -0600)]
ceph-osd: infer objectstore type of 'type' file is missing

The current value of osd_objectstore should have no bearing--we should
rely on the state in front of us.

- If there is a directory called current/, assume filestore.
- If there is a symlink called block, assume bluestore.
- If we see none of those, fail.

Fixes: http://tracker.ceph.com/issues/38517
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: fix for using '::' on hosts without ipv6 26635/head
Noah Watkins [Fri, 22 Feb 2019 22:13:47 +0000 (14:13 -0800)]
mgr/dashboard: fix for using '::' on hosts without ipv6

CherryPy throws an uncaught exception using '::' when the host does not
have an ipv6 loopback address. This is currently the case in kubernetes.
This patch disables the routine in cherrypy that performs the validation
that requested server ports are actually bound.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
6 years agomsg/async, v2: move ceph_msg_header2 to last frame segment.
Radoslaw Zarzynski [Fri, 1 Mar 2019 21:47:22 +0000 (22:47 +0100)]
msg/async, v2: move ceph_msg_header2 to last frame segment.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #26692 into master
Sage Weil [Fri, 1 Mar 2019 20:36:35 +0000 (14:36 -0600)]
Merge PR #26692 into master

* refs/pull/26692/head:
test/test_ipaddr: added test for pick_address when ipv4 and ipv6 are enabled
common/pick_address: improves log message when no IP is found
pick_address: error out in dual stack mode if both addr types cannot be provided

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomds: Fix use of auto prototype, it is a non std extension 26726/head
Willem Jan Withagen [Fri, 1 Mar 2019 20:36:26 +0000 (21:36 +0100)]
mds: Fix use of auto prototype, it is a non std extension

Clang complains:
In file included from /home/jenkins/workspace/ceph-master/src/msg/Message.cc:128:
In file included from /home/jenkins/workspace/ceph-master/src/messages/MFSMap.h:20:
/home/jenkins/workspace/ceph-master/src/mds/FSMap.h:330:49: error: 'auto' not allowed in function prototype
  void modify_filesystem(fs_cluster_id_t fscid, auto&& fn)
                                                ^~~~
/home/jenkins/workspace/ceph-master/src/mds/FSMap.h:341:37: error: 'auto' not allowed in function prototype
  void modify_daemon(mds_gid_t who, auto&& fn)
                                    ^~~~

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoosdc/Objecter: don't use post_rx_buffer or revoke_rx_buffer at all
Sage Weil [Fri, 1 Mar 2019 20:09:16 +0000 (14:09 -0600)]
osdc/Objecter: don't use post_rx_buffer or revoke_rx_buffer at all

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agofixup: use frame epilogue for crc32 integrity checking.
Radoslaw Zarzynski [Fri, 1 Mar 2019 20:08:08 +0000 (21:08 +0100)]
fixup: use frame epilogue for crc32 integrity checking.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomsg/async, v2: epilogue size is variable in secure mode.
Radoslaw Zarzynski [Fri, 1 Mar 2019 19:18:11 +0000 (20:18 +0100)]
msg/async, v2: epilogue size is variable in secure mode.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomsg/async, v2: drop support for the buggy rx_buffers mechanism.
Radoslaw Zarzynski [Fri, 1 Mar 2019 17:33:25 +0000 (18:33 +0100)]
msg/async, v2: drop support for the buggy rx_buffers mechanism.

See:
  * https://github.com/ceph/ceph/pull/26696,
  * http://tracker.ceph.com/issues/22480.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomgr/BaseMgrModule: drop GIL for ceph_send_command 26723/head
Sage Weil [Fri, 1 Mar 2019 17:00:55 +0000 (11:00 -0600)]
mgr/BaseMgrModule: drop GIL for ceph_send_command

Otherwise, we can easily deadlock, since other bits of the code hold the
objecter lock and then take the GIL.

Fixes: http://tracker.ceph.com/issues/38537
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoRevert "msg/async, v2: add flags field to frame's epilogue."
Radoslaw Zarzynski [Fri, 1 Mar 2019 16:37:47 +0000 (17:37 +0100)]
Revert "msg/async, v2: add flags field to frame's epilogue."

This reverts commit e18d65ae9e7113831480f5c3efad015e83bc1200.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #26688 from yangdongsheng/librbd_fix_all_throttled
Jason Dillaman [Fri, 1 Mar 2019 15:43:36 +0000 (10:43 -0500)]
Merge pull request #26688 from yangdongsheng/librbd_fix_all_throttled

librbd: fix were_all_throttled() to avoid incorrect ret-value

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge PR #26659 into master
Patrick Donnelly [Fri, 1 Mar 2019 14:53:20 +0000 (06:53 -0800)]
Merge PR #26659 into master

* refs/pull/26659/head:
qa: remove libcephfs-java tests

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>