]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agorgw, Policy should be url_decode when assume_role 28704/head
yuliyang [Sat, 22 Jun 2019 06:58:31 +0000 (14:58 +0800)]
rgw, Policy should be url_decode when assume_role

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
6 years agoMerge pull request #28559 from dillaman/wip-40368
Mykola Golub [Sat, 15 Jun 2019 08:34:34 +0000 (11:34 +0300)]
Merge pull request #28559 from dillaman/wip-40368

librbd: avoid dereferencing an empty container during deep-copy

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #28487 from xiexingguo/wip-objecter-pg-mapping-cache
Xie Xingguo [Sat, 15 Jun 2019 05:31:02 +0000 (13:31 +0800)]
Merge pull request #28487 from xiexingguo/wip-objecter-pg-mapping-cache

osdc/Objecter: pg-mapping cache

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #28082 into master
Patrick Donnelly [Fri, 14 Jun 2019 21:51:07 +0000 (14:51 -0700)]
Merge PR #28082 into master

* refs/pull/28082/head:
mgr / volumes: improve error handling
py / cephfs: invoke base class Error::__init__() from OSError::__init__()
mgr / volumes: carve out subvolume operations as a separate class
mgr / volumes: introduce subvolume specification class
mgr / volumes: carve of volume as a separate class

6 years agoMerge PR #28198 into master
Patrick Donnelly [Fri, 14 Jun 2019 18:44:17 +0000 (11:44 -0700)]
Merge PR #28198 into master

* refs/pull/28198/head:
qa/tasks: upgrade command arguments checks in vstart_runner.py

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agolibrbd: avoid dereferencing an empty container during deep-copy 28559/head
Jason Dillaman [Fri, 14 Jun 2019 17:34:24 +0000 (13:34 -0400)]
librbd: avoid dereferencing an empty container during deep-copy

While updating the image copy progress, a 'while' loop will pop the
last item from the queue and then deference the empty queue.

Fixes: http://tracker.ceph.com/issues/40368
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27773 from istudko/patch-1
Kefu Chai [Fri, 14 Jun 2019 17:20:51 +0000 (01:20 +0800)]
Merge pull request #27773 from istudko/patch-1

doc/releases/nautilus: Correct a systemctl command in an upgrade guide

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27201 from bstillwell/bootstrap-keyring-caps-doc-fix
Kefu Chai [Fri, 14 Jun 2019 17:19:26 +0000 (01:19 +0800)]
Merge pull request #27201 from bstillwell/bootstrap-keyring-caps-doc-fix

doc: Add missing mgr cap for the bootstrap keyring

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28526 from Osa1989/patch-1
Kefu Chai [Fri, 14 Jun 2019 17:07:24 +0000 (01:07 +0800)]
Merge pull request #28526 from Osa1989/patch-1

doc/releases/nautilus.rst: fix command to check min_compat_client

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28522 from neha-ojha/wip-add-client-endpoints
Neha Ojha [Fri, 14 Jun 2019 16:34:57 +0000 (09:34 -0700)]
Merge pull request #28522 from neha-ojha/wip-add-client-endpoints

 qa/tasks/cbt.py: add support for client_endpoints

Reviewed-by: Mark Nelson <mnelson@redhat.com>
6 years agoMerge pull request #28346 from smithfarm/wip-39447-alt
Mykola Golub [Fri, 14 Jun 2019 15:19:31 +0000 (18:19 +0300)]
Merge pull request #28346 from smithfarm/wip-39447-alt

qa/rbd: add cram-based snap diff test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #28532 from cbodley/wip-40304
Yuri Weinstein [Fri, 14 Jun 2019 15:16:32 +0000 (08:16 -0700)]
Merge pull request #28532 from cbodley/wip-40304

qa/rgw: skip swift tests on rhel 7.6+

6 years agodoc/releases/nautilus.rst: fix command to check min_compat_client 28526/head
Osa1989 [Thu, 13 Jun 2019 08:31:02 +0000 (10:31 +0200)]
doc/releases/nautilus.rst: fix command to check min_compat_client

Command should be "ceph osd dump | grep min_compat_client"

Signed-off-by: Osama Elswah <osama.ibrahim.89@gmail.com>
6 years agoMerge pull request #27951 from ifed01/wip-ifed-fix-slowop-log
Igor Fedotov [Fri, 14 Jun 2019 10:43:49 +0000 (13:43 +0300)]
Merge pull request #27951 from ifed01/wip-ifed-fix-slowop-log

os/bluestore: fix origin reference in logging slow ops.

6 years agoMerge pull request #28490 from xiexingguo/wip-40193
Kefu Chai [Fri, 14 Jun 2019 10:29:38 +0000 (18:29 +0800)]
Merge pull request #28490 from xiexingguo/wip-40193

mon/OSDMonitor: report pg[pgp]_num_target instead of pg[pgp]_num

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #28322 from liewegas/wip-preextend
Kefu Chai [Fri, 14 Jun 2019 10:27:44 +0000 (18:27 +0800)]
Merge pull request #28322 from liewegas/wip-preextend

os/bluestore: bluefs_preextend_wal_files=true

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #28260 from s0nea/wip-dashboard-pool-delete-flag
Laura Paduano [Fri, 14 Jun 2019 10:02:02 +0000 (12:02 +0200)]
Merge pull request #28260 from s0nea/wip-dashboard-pool-delete-flag

mgr/dashboard: consider 'mon_allow_pool_delete' flag

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #28534 from tchaikov/wip-crimson-mkkey
Kefu Chai [Fri, 14 Jun 2019 09:34:42 +0000 (17:34 +0800)]
Merge pull request #28534 from tchaikov/wip-crimson-mkkey

crimson/osd: add --mkkey support

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #28464 from majianpeng/fio-ceph-messenger-multi-client
Kefu Chai [Fri, 14 Jun 2019 09:14:43 +0000 (17:14 +0800)]
Merge pull request #28464 from majianpeng/fio-ceph-messenger-multi-client

test/fio/fio_ceph_messenger: make exec multi client on the same host.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28404 from xiexingguo/wip-osd-noup-stall
Kefu Chai [Fri, 14 Jun 2019 08:37:33 +0000 (16:37 +0800)]
Merge pull request #28404 from xiexingguo/wip-osd-noup-stall

osd/OSD: keep synchronizing with mon if stuck at booting

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agocrimson/osd: ignore exception when stopping osd 28534/head
Kefu Chai [Fri, 14 Jun 2019 06:57:55 +0000 (14:57 +0800)]
crimson/osd: ignore exception when stopping osd

OSD::stop() is a top-level method in the sense that it will be called
directly when engine stops. so it should not throw any exceptions.
instead, we will print out the error and ignore the exception if any.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: add --mkkey support
Kefu Chai [Thu, 13 Jun 2019 14:48:48 +0000 (22:48 +0800)]
crimson/osd: add --mkkey support

see qa/tasks/ceph.py for how it is used, and why we need it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/os: use buffer::write_file() helper
Kefu Chai [Thu, 13 Jun 2019 14:44:16 +0000 (22:44 +0800)]
crimson/os: use buffer::write_file() helper

instead of using the blocking calls

so we can write to file without blocking the reactor

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: add buffer_io helpers
Kefu Chai [Fri, 14 Jun 2019 03:49:47 +0000 (11:49 +0800)]
crimson: add buffer_io helpers

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28507 from tchaikov/wip-cmake-dpdk-boost
Kefu Chai [Fri, 14 Jun 2019 07:43:41 +0000 (15:43 +0800)]
Merge pull request #28507 from tchaikov/wip-cmake-dpdk-boost

cmake: update SPDK to build with GCC-9

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoqa/rbd: add cram-based snap diff test 28346/head
Nathan Cutler [Fri, 31 May 2019 10:57:15 +0000 (12:57 +0200)]
qa/rbd: add cram-based snap diff test

Fixes: http://tracker.ceph.com/issues/39447
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoauth/KeyRing: remove KeyRing::create_empty()
Kefu Chai [Thu, 13 Jun 2019 14:38:02 +0000 (22:38 +0800)]
auth/KeyRing: remove KeyRing::create_empty()

it never fails unless out of memory. and it would be simpler to just
allocate it on stack.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosdc/Objecter: pg-mapping cache 28487/head
xie xingguo [Thu, 6 Jun 2019 08:12:27 +0000 (16:12 +0800)]
osdc/Objecter: pg-mapping cache

The CRUSH-based addressing is some kind of CPU intensive task
and hence should be avoid whenever possible.

The patch introduces a per objecter pg-mapping cache, which as
a result can saved us 10%+ CPU (fio, 4k randow read, 24k IOPS):

Was:
$ top -Hp 415864
top - 14:45:39 up 6 days,  5:45,  3 users,  load average: 9.67, 8.66, 8.45
Threads:  23 total,   3 running,  20 sleeping,   0 stopped,   0 zombie
%Cpu(s): 21.6 us, 14.5 sy,  0.0 ni, 59.6 id,  1.4 wa,  0.0 hi,  3.0 si,  0.0 st
KiB Mem : 19616576+total, 12773412+free, 50964508 used, 17467128 buff/cache
KiB Swap:  4194300 total,  4194300 free,        0 used. 14352892+avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
 415942 root      20   0 1957800 492820 123880 R 80.0  0.3   0:13.75 tp_librbd

Now:
$ top -Hp 475779
top - 10:22:05 up  1:18,  4 users,  load average: 2.65, 1.44, 1.60
Threads:  23 total,   2 running,  21 sleeping,   0 stopped,   0 zombie
%Cpu(s):  6.1 us,  2.2 sy,  0.0 ni, 91.4 id,  0.0 wa,  0.0 hi,  0.2 si,  0.0 st
KiB Mem : 19616576+total, 13919555+free, 27280820 used, 29689392 buff/cache
KiB Swap:  4194300 total,  4194300 free,        0 used. 16798102+avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
 476231 root      20   0 1957808 491712 123836 S 71.1  0.3   0:41.47 tp_librbd

And below is a more detailed report from the perf tool:

Was:
$ perf report -g graph
  Children      Self  Command    Shared Object        Symbol
+   29.33%     0.03%  tp_librbd  librados.so.2.0.0    [.] Objecter::op_submit
+   28.83%     0.09%  tp_librbd  librados.so.2.0.0    [.] Objecter::_op_submit_with_budget
+   27.12%     0.17%  tp_librbd  librados.so.2.0.0    [.] Objecter::_op_submit
+   16.47%     0.26%  tp_librbd  librados.so.2.0.0    [.] Objecter::_calc_target
+   15.04%     0.10%  tp_librbd  libceph-common.so.0  [.] OSDMap::_pg_to_up_acting_osds
+   13.52%     0.16%  tp_librbd  libceph-common.so.0  [.] OSDMap::_pg_to_raw_osds

Now:
$ perf report -g graph
  Children      Self  Command    Shared Object        Symbol
+   17.84%     0.04%  tp_librbd  librados.so.2.0.0    [.] Objecter::op_submit
+   17.34%     0.06%  tp_librbd  librados.so.2.0.0    [.] Objecter::_op_submit_with_budget
+   15.80%     0.17%  tp_librbd  librados.so.2.0.0    [.] Objecter::_op_submit
+    6.11%     2.02%  tp_librbd  librados.so.2.0.0    [.] Objecter::_calc_target

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #28535 from tchaikov/wip-crimson-packaging
Kefu Chai [Fri, 14 Jun 2019 05:27:45 +0000 (13:27 +0800)]
Merge pull request #28535 from tchaikov/wip-crimson-packaging

debian/: use ceph-osd for packaging crimson-osd

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomgr / volumes: improve error handling 28082/head
Venky Shankar [Sun, 19 May 2019 17:34:57 +0000 (13:34 -0400)]
mgr / volumes: improve error handling

This cleans up lots of return-fu statements and make the source
look much more pythonic. Functions should raise an instance of
VolumeException() class wherever necessary (error handling).

Fixes: http://tracker.ceph.com/issues/39969
Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agopy / cephfs: invoke base class Error::__init__() from OSError::__init__()
Venky Shankar [Sun, 19 May 2019 17:20:51 +0000 (13:20 -0400)]
py / cephfs: invoke base class Error::__init__() from OSError::__init__()

This will allow access to `args` when catching `Error`.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agomgr / volumes: carve out subvolume operations as a separate class
Venky Shankar [Fri, 17 May 2019 05:40:37 +0000 (01:40 -0400)]
mgr / volumes: carve out subvolume operations as a separate class

Move (and refactor) subvolume specific operations from main module
source to SubVolume class in a new subvolume source. Also, provide
hooks in VolumeClient to forward subvolume specific opertaions to
this class.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agomgr / volumes: introduce subvolume specification class
Venky Shankar [Thu, 16 May 2019 05:54:12 +0000 (01:54 -0400)]
mgr / volumes: introduce subvolume specification class

Specifications class that represents a unique subvolume
identified by (subvolume-id, group-id) tuple. Provide
heleper functions for fetching  various subvolume specific
attributes such as subvolume path, group path etc.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agomgr / volumes: carve of volume as a separate class
Venky Shankar [Mon, 13 May 2019 12:15:35 +0000 (17:45 +0530)]
mgr / volumes: carve of volume as a separate class

Move volume operations (create/delete) from main module source
to VolumeClient class in new volume source.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge PR #28456 into master
Patrick Donnelly [Thu, 13 Jun 2019 19:22:05 +0000 (12:22 -0700)]
Merge PR #28456 into master

* refs/pull/28456/head:
mds: ignore sessionmap version mismatch if mds_wipe_sessions is set
mds: fix corner case of replaying open sessions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28521 from alimaredia/wip-qa-rgw-ssl
Casey Bodley [Thu, 13 Jun 2019 19:00:21 +0000 (15:00 -0400)]
Merge pull request #28521 from alimaredia/wip-qa-rgw-ssl

qa: use curl in wait_for_radosgw() in util/rgw.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28415 from linuxbox2/wip-full-odsock
Matt Benjamin [Thu, 13 Jun 2019 17:57:25 +0000 (13:57 -0400)]
Merge pull request #28415 from linuxbox2/wip-full-odsock

rgw/OutputDataSocket: actually discard data on full buffer

6 years agoqa: use curl in wait_for_radosgw() in util/rgw.py 28521/head
Ali Maredia [Wed, 12 Jun 2019 20:12:47 +0000 (16:12 -0400)]
qa: use curl in wait_for_radosgw() in util/rgw.py

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agoMerge pull request #25496 from mkogan1/wip-rgw-dynamic-resharding-docs
Casey Bodley [Thu, 13 Jun 2019 16:01:16 +0000 (12:01 -0400)]
Merge pull request #25496 from mkogan1/wip-rgw-dynamic-resharding-docs

rgw: update the "radosgw-admin reshard status"

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoqa/rgw: skip swift tests on rhel 7.6+ 28532/head
Casey Bodley [Thu, 13 Jun 2019 13:45:29 +0000 (09:45 -0400)]
qa/rgw: skip swift tests on rhel 7.6+

swift bootstrap fails because libev-devel is not available

Fixes: http://tracker.ceph.com/issues/40304
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agodebian/: use ceph-osd for packaging crimson-osd 28535/head
Kefu Chai [Thu, 13 Jun 2019 08:28:29 +0000 (16:28 +0800)]
debian/: use ceph-osd for packaging crimson-osd

* debian/: remove ceph-crimson-osd package.
* debian/control: set `CEPH_OSD_BASENAME` env variable, which
  will be consumed by `ceph-osd.install`. alternatively, we could
  rename crimson-osd to ceph-osd in `override_dh_auto_install`,
  but let's go with this way at this moment, unless `mv` in
  `override_dh_auto_install` is proved to be better.
* ceph-osd.install: replace ceph-osd with crimson-osd if
  `CEPH_EXTRA_CMAKE_ARGS` has `WITH_SEASTAR` in it. this only
  happens when we are packaging the "crimson" flavor packages from
  jenkins.
* ceph-osd.install: `chmod +x` this file, as we need to use
  `/usr/bin/dh-exec` as the interpreter of it to perform variable
  substitution and install.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: silence -Waddress-of-packed-member warnings from SPDK 28507/head
Kefu Chai [Wed, 12 Jun 2019 13:27:33 +0000 (21:27 +0800)]
cmake: silence -Waddress-of-packed-member warnings from SPDK

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: silence GCC-9 warnings from rocksdb
Kefu Chai [Wed, 12 Jun 2019 08:50:14 +0000 (16:50 +0800)]
cmake: silence GCC-9 warnings from rocksdb

GCC-9 complains:

rocksdb/db/version_edit.h:86:8: warning: implicitly-declared ‘constexpr
rocksdb::FileDescriptor::FileDescriptor(const rocksdb::FileDescriptor&)’
is deprecated [-Wdeprecated-copy]
   86 | struct FileMetaData {
      |        ^~~~~~~~~~~~
/var/ssd/ceph/src/rocksdb/db/version_edit.h:55:19: note: because
‘rocksdb::FileDescriptor’ has user-provided ‘rocksdb::FileDescriptor&
rocksdb::FileDescriptor::operator=(const rocksdb::FileDescriptor&)’
   55 |   FileDescriptor& operator=(const FileDescriptor& fd) {
      |                   ^~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: set ccache for rocksdb
Kefu Chai [Wed, 12 Jun 2019 10:54:26 +0000 (18:54 +0800)]
cmake: set ccache for rocksdb

should use launcher to set ccache

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: update built libs of SPDK and DPDK
Kefu Chai [Wed, 12 Jun 2019 11:18:53 +0000 (19:18 +0800)]
cmake: update built libs of SPDK and DPDK

we need to link against these new libs for fulfilling the dependencies
required by new SPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: disable isal and vhost support of SPDK
Kefu Chai [Wed, 12 Jun 2019 08:32:55 +0000 (16:32 +0800)]
cmake: disable isal and vhost support of SPDK

* isal needs nasm, which is a new dependency. so let's disable it
  at this moment.
* vhost requires DPDK compiled with vhost enabled. but it's disabled
  when we compile DPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agospdk: pick up latest spdk
Kefu Chai [Wed, 12 Jun 2019 08:04:41 +0000 (16:04 +0800)]
spdk: pick up latest spdk

to silence the warnings reported by GCC-9

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: provide admin friendly reshard status output 25496/head
Mark Kogan [Sun, 12 May 2019 13:45:30 +0000 (09:45 -0400)]
rgw: provide admin friendly reshard status output

Signed-off-by: Mark Kogan <mkogan@redhat.com>
6 years agoMerge pull request #28446 from tspmelo/wip-prepare-backport
Nathan Cutler [Thu, 13 Jun 2019 09:01:30 +0000 (11:01 +0200)]
Merge pull request #28446 from tspmelo/wip-prepare-backport

script/ceph-backport.sh: Add prepare function

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28515 from hvhaugwitz/mondb-recovery-script
Kefu Chai [Thu, 13 Jun 2019 00:36:16 +0000 (08:36 +0800)]
Merge pull request #28515 from hvhaugwitz/mondb-recovery-script

doc: update mondb recovery script

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/tasks/cbt.py: add support for client_endpoints 28522/head
Neha Ojha [Wed, 12 Jun 2019 00:33:53 +0000 (00:33 +0000)]
qa/tasks/cbt.py: add support for client_endpoints

Note that this will only work for librbdfio for now.

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #28473 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:30:45 +0000 (14:30 -0700)]
Merge PR #28473 into master

* refs/pull/28473/head:
cephfs-shell: Catch OSError exceptions in lcd

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #28475 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:29:46 +0000 (14:29 -0700)]
Merge PR #28475 into master

* refs/pull/28475/head:
cephfs-shell: Fix 'lls' command errors

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #28407 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:27:53 +0000 (14:27 -0700)]
Merge PR #28407 into master

* refs/pull/28407/head:
mgr/volumes: fix error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27856 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:27:07 +0000 (14:27 -0700)]
Merge PR #27856 into master

* refs/pull/27856/head:
test: remove fs/test-volume.sh workunit
test: port fs/volume related tests to python

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #28196 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:16:31 +0000 (14:16 -0700)]
Merge PR #28196 into master

* refs/pull/28196/head:
ceph_volume_client: decode d_name before using it

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
6 years agoMerge PR #28221 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:15:30 +0000 (14:15 -0700)]
Merge PR #28221 into master

* refs/pull/28221/head:
qa/tasks/cephfs/test_volume_client: print py2 or py3 which the test case runs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
6 years agoMerge PR #26737 into master
Patrick Donnelly [Wed, 12 Jun 2019 21:07:22 +0000 (14:07 -0700)]
Merge PR #26737 into master

* refs/pull/26737/head:
qa/cephfs: update tests for stale session handling
mds: change how mds revoke stale caps
mds: don't mark unresponsive sessions holding no caps stale

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28518 from yuriw/wip-yuriw-crontab-master
Yuri Weinstein [Wed, 12 Jun 2019 20:49:41 +0000 (13:49 -0700)]
Merge pull request #28518 from yuriw/wip-yuriw-crontab-master

qa/tests: removed upgrade/client-upgrade-hammer becasue ubuntu 14.04 …

6 years agoMerge pull request #28466 from hoerup/update-releases
Neha Ojha [Wed, 12 Jun 2019 19:45:33 +0000 (12:45 -0700)]
Merge pull request #28466 from hoerup/update-releases

Doc: update release schedule

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Noah Watkins <nwatkins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoDoc: update release schedule 28466/head
Torben Hørup [Mon, 10 Jun 2019 07:19:16 +0000 (09:19 +0200)]
Doc: update release schedule

Added nautilus + the latest releases from mimic and luminous
in order to keep dumpling column, ceph_releases is modified to allow 13 columns

Signed-off-by: Torben Hørup <th@synack.fo>
6 years agoMerge pull request #28470 from yuvalif/wip-yuval-sync-type-pubsub
Yuval Lifshitz [Wed, 12 Jun 2019 17:01:31 +0000 (20:01 +0300)]
Merge pull request #28470 from yuvalif/wip-yuval-sync-type-pubsub

rgw/pubsub: use incremental sync for pubsub module by default

6 years agotest: remove fs/test-volume.sh workunit 27856/head
Venky Shankar [Tue, 4 Jun 2019 12:56:03 +0000 (08:56 -0400)]
test: remove fs/test-volume.sh workunit

.. since its ported to python now.

Fixes: http://tracker.ceph.com/issues/39949
Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agotest: port fs/volume related tests to python
Venky Shankar [Tue, 4 Jun 2019 10:09:56 +0000 (06:09 -0400)]
test: port fs/volume related tests to python

... and add subvolume related tests.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
6 years agoqa/tests: removed upgrade/client-upgrade-hammer becasue ubuntu 14.04 is not supported 28518/head
Yuri Weinstein [Wed, 12 Jun 2019 16:13:15 +0000 (09:13 -0700)]
qa/tests: removed upgrade/client-upgrade-hammer becasue ubuntu 14.04 is not supported

Also removed ubuntu 14.04 filters

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agodoc: update mondb recovery script 28515/head
Hannes von Haugwitz [Wed, 12 Jun 2019 13:18:17 +0000 (15:18 +0200)]
doc: update mondb recovery script

- be specific about stopped OSDs
- add missing '--no-mon-config' option
- fix indent of here script delimiting identifier
- use $host variable in for loop

Signed-off-by: Hannes von Haugwitz <hannes@vonhaugwitz.com>
6 years agoMerge pull request #28508 from smithfarm/wip-40295
Kefu Chai [Wed, 12 Jun 2019 15:05:23 +0000 (23:05 +0800)]
Merge pull request #28508 from smithfarm/wip-40295

cmake: set empty-string RPATH for ceph-osd

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #22446 into master
Sage Weil [Wed, 12 Jun 2019 14:11:49 +0000 (09:11 -0500)]
Merge PR #22446 into master

* refs/pull/22446/head:
relicense LGPL-2.1 code as LGPL-2.1 or LGPL-3.0
COPYING: add reference to a few Apache-2.0 licensed source files
debian/copyright: sync with COPYING
COPYING: LGPL2.1 -> LGPL-2.1
COPYING: extra bits from debian/copyright

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoscript/ceph-backport.sh: Add prepare function 28446/head
Tiago Melo [Tue, 4 Jun 2019 11:28:26 +0000 (12:28 +0100)]
script/ceph-backport.sh: Add prepare function

Add a new flag, --prepare, that when used will fetch all necessary info and
cherrypick all original commits automatically.
It will also prevent the main code to be executed.

After running with --prepare, and if there were no conflicts, you can simply
remove the flag from the command and run the script again.
This time it will run the main code and create the PR and update the backport
issue.

p.e.:
ceph-backport.sh 19206 jewel --prepare
ceph-backport.sh 19206 jewel

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #28484 from tchaikov/wip-test_librados_build
Kefu Chai [Wed, 12 Jun 2019 11:47:46 +0000 (19:47 +0800)]
Merge pull request #28484 from tchaikov/wip-test_librados_build

qa/workunits/rados/test_librados_build.sh: install build deps

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28486 from tchaikov/wip-remove-digest-from-buffer.h
Kefu Chai [Wed, 12 Jun 2019 11:29:59 +0000 (19:29 +0800)]
Merge pull request #28486 from tchaikov/wip-remove-digest-from-buffer.h

buffer,crypto,tools: extract digest methods out of bufferlist

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Myoungwon Oh <omwmw@sk.com>
6 years agoMerge pull request #28494 from ricardoasmarques/update-iscsi-dependencies-versions
Jason Dillaman [Wed, 12 Jun 2019 11:21:30 +0000 (07:21 -0400)]
Merge pull request #28494 from ricardoasmarques/update-iscsi-dependencies-versions

doc: Update 'python-rtslib' and 'tcmu-runner' min versions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agocmake: set empty-string RPATH for ceph-osd 28508/head
Nathan Cutler [Fri, 15 Mar 2019 12:50:51 +0000 (13:50 +0100)]
cmake: set empty-string RPATH for ceph-osd

Fixes: http://tracker.ceph.com/issues/40295
Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/workunits/rados/test_envlibrados_for_rocksdb: support SUSE distros 28484/head
Kefu Chai [Wed, 12 Jun 2019 09:38:26 +0000 (17:38 +0800)]
qa/workunits/rados/test_envlibrados_for_rocksdb: support SUSE distros

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/workunits/rados/test_librados_build.sh: install build deps
Kefu Chai [Tue, 11 Jun 2019 05:16:48 +0000 (13:16 +0800)]
qa/workunits/rados/test_librados_build.sh: install build deps

to build helloworld of librados, we need to install the build
dependencies.

Fixes: https://tracker.ceph.com/issues/40155
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/workunits/rados/test_envlibrados_for_rocksdb: use helper script
Kefu Chai [Wed, 12 Jun 2019 04:32:58 +0000 (12:32 +0800)]
qa/workunits/rados/test_envlibrados_for_rocksdb: use helper script

instead of having its own copy of `install()`, try to leverage the
helper in ceph-helpers-root.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/workunits/ceph-helpers-root: use /etc/os-release instead
Kefu Chai [Wed, 12 Jun 2019 04:29:45 +0000 (12:29 +0800)]
qa/workunits/ceph-helpers-root: use /etc/os-release instead

as lsb_release does not work on SUSE distros

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28465 from tchaikov/wip-xio-cleanup
Kefu Chai [Wed, 12 Jun 2019 09:16:03 +0000 (17:16 +0800)]
Merge pull request #28465 from tchaikov/wip-xio-cleanup

include/config-h.in.cmake: remove HAVE_XIO

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agocmake: move boost patch to cmake/modules
Kefu Chai [Wed, 12 Jun 2019 07:49:11 +0000 (15:49 +0800)]
cmake: move boost patch to cmake/modules

as it's used by cmake only

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake/modules/BuildDPDK.cmake: update to adapt upstream changes
Kefu Chai [Wed, 12 Jun 2019 07:21:45 +0000 (15:21 +0800)]
cmake/modules/BuildDPDK.cmake: update to adapt upstream changes

* update execenv to remove "app", the targets ending with "app" changed
  to the ones without "app" now.
* pass CC by make variables not env variables, otherwise the default
  `cc` is used

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28495 from tchaikov/wip-dashboard-portal_ip_addresses
Kefu Chai [Wed, 12 Jun 2019 05:53:38 +0000 (13:53 +0800)]
Merge pull request #28495 from tchaikov/wip-dashboard-portal_ip_addresses

mgr/dashboard: take portal_ip_addresses as a list

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agomgr/dashboard: s/portal_ip_address/portal_ip_addresses/ 28495/head
Kefu Chai [Tue, 11 Jun 2019 16:28:53 +0000 (00:28 +0800)]
mgr/dashboard: s/portal_ip_address/portal_ip_addresses/

and take portal_ip_addresses as a list

it's a follow-up fix of #28084

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28290 from b-ranto/wip-selinux-rhel8
Kefu Chai [Wed, 12 Jun 2019 02:17:16 +0000 (10:17 +0800)]
Merge pull request #28290 from b-ranto/wip-selinux-rhel8

selinux: Update the policy for RHEL8

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/suites/perf-basic/workloads: add client_endpoint_rbd_4K_rand_write.yaml
Neha Ojha [Wed, 12 Jun 2019 00:16:59 +0000 (00:16 +0000)]
qa/suites/perf-basic/workloads: add client_endpoint_rbd_4K_rand_write.yaml

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27507 from pritha-srivastava/wip-rgw-cross-tenant-doc
Casey Bodley [Tue, 11 Jun 2019 17:16:28 +0000 (13:16 -0400)]
Merge pull request #27507 from pritha-srivastava/wip-rgw-cross-tenant-doc

rgw: Improving doc for Cross Project(Tenant) access with Openstack Ke…

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agodoc: Update 'python-rtslib' and 'tcmu-runner' min versions 28494/head
Ricardo Marques [Tue, 11 Jun 2019 15:53:41 +0000 (16:53 +0100)]
doc: Update 'python-rtslib' and 'tcmu-runner' min versions

Signed-off-by: Ricardo Marques <rimarques@suse.com>
6 years agorgw: update the "radosgw-admin reshard status"
Mark Kogan [Tue, 11 Dec 2018 17:43:32 +0000 (12:43 -0500)]
rgw: update the "radosgw-admin reshard status"
command documentation with expected output examples

Fixes: http://tracker.ceph.com/issues/37615
Signed-off-by: Mark Kogan <mkogan@redhat.com>
6 years agomon/OSDMonitor: report pg[pgp]_num_target instead of pg[pgp]_num 28490/head
xie xingguo [Mon, 10 Jun 2019 07:58:37 +0000 (15:58 +0800)]
mon/OSDMonitor: report pg[pgp]_num_target instead of pg[pgp]_num

As by calling "ceph osd pool set <pool-name> pg_num", we set the
pg_num_target instead. Hence we should report pg_num_target as well
when invoked by the "ceph osd pool get <pool-name> pg_num" call.

I doubt an ideal fix would be exposing both the pg_num(*actual*) and
pg_num_target concepts to user, but for now I think we can stop here
and get some more feedback first.

Fixes: http://tracker.ceph.com/issues/40193
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #28084 from ricardoasmarques/improve-portal-ips-calculation
Kefu Chai [Tue, 11 Jun 2019 12:17:32 +0000 (20:17 +0800)]
Merge pull request #28084 from ricardoasmarques/improve-portal-ips-calculation

mgr/dashboard: Optimize portal IPs calculation

Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #28454 from tchaikov/wip-gcc-9-on-bionic
Kefu Chai [Tue, 11 Jun 2019 12:14:51 +0000 (20:14 +0800)]
Merge pull request #28454 from tchaikov/wip-gcc-9-on-bionic

install-deps.sh: use GCC-9 on bionic

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #28246 from kungf/mgr_default_addr
Kefu Chai [Tue, 11 Jun 2019 11:48:00 +0000 (19:48 +0800)]
Merge pull request #28246 from kungf/mgr_default_addr

mgr: use ipv4 default when ipv6 was disabled

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28481 from iotcg/common
Kefu Chai [Tue, 11 Jun 2019 11:35:19 +0000 (19:35 +0800)]
Merge pull request #28481 from iotcg/common

include: buffer_raw.h: Copyright time fix

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agobuffer,crypto,osd,tools: extract digest methods out of bufferlist 28486/head
Kefu Chai [Tue, 11 Jun 2019 07:47:37 +0000 (15:47 +0800)]
buffer,crypto,osd,tools: extract digest methods out of bufferlist

they were consumers of ceph::list::buffers(), and there is no need to
make them member functions of ceph::list, so move them into ceph::crypto
namespace. and update the unit test and ceph_dedup_tool.cc accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/tasks/cephfs/test_volume_client: print py2 or py3 which the test case runs 28221/head
Lianne [Tue, 11 Jun 2019 10:27:06 +0000 (18:27 +0800)]
qa/tasks/cephfs/test_volume_client: print py2 or py3 which the test case runs

Fixes: http://tracker.ceph.com/issues/40184
Signed-off-by: Lianne <liyan.wang@xtaotech.com>
6 years agocommon/ceph_crypto: add digest_size to digest classes
Kefu Chai [Tue, 11 Jun 2019 06:42:14 +0000 (14:42 +0800)]
common/ceph_crypto: add digest_size to digest classes

prepare for extracting the digest methods from ceph::buffer::list, so we
can use them in a templated implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/fio/fio_ceph_messenger: make exec multi client on the same host. 28464/head
Jianpeng Ma [Tue, 11 Jun 2019 07:21:39 +0000 (15:21 +0800)]
test/fio/fio_ceph_messenger: make exec multi client on the same host.

When create Messenger, for client we should give different nonce to
make multi clients on the same host can work. Otherwise, server only
accept one connection.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agoMerge pull request #28443 from dzafman/wip-40198
Kefu Chai [Tue, 11 Jun 2019 06:03:23 +0000 (14:03 +0800)]
Merge pull request #28443 from dzafman/wip-40198

osd: Don't randomize deep scrubs when noscrub set

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #28431 from cbodley/wip-40188
Kefu Chai [Tue, 11 Jun 2019 06:02:28 +0000 (14:02 +0800)]
Merge pull request #28431 from cbodley/wip-40188

common: OutputDataSocket retakes mutex on error path

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoMerge pull request #28427 from tchaikov/wip-40174
Kefu Chai [Tue, 11 Jun 2019 05:59:38 +0000 (13:59 +0800)]
Merge pull request #28427 from tchaikov/wip-40174

mgr/influx: try to call close()

Reviewed-by: Wido den Hollander <wido@42on.com>
6 years agoMerge pull request #28254 from myoungwon/wip-hashfuncs-dedup
Kefu Chai [Tue, 11 Jun 2019 05:57:36 +0000 (13:57 +0800)]
Merge pull request #28254 from myoungwon/wip-hashfuncs-dedup

common,osd: add hash algorithms for dedup fingerprint

Reviewed-by: Sage Weil <sage@redhat.com>