]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agocmake: extract rgw_*_scheduler.cc into rgw_schedulers 28252/head
Kefu Chai [Tue, 4 Jun 2019 12:59:24 +0000 (20:59 +0800)]
cmake: extract rgw_*_scheduler.cc into rgw_schedulers

i was running into following issue:

../../../lib/libradosgw_a.a(rgw_loadgen_process.cc.o): In function
`RGWLoadGenProcess::run()':
/home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_loadgen_process.cc:102:
undefined reference to `signal_shutdown()'
collect2: error: ld returned 1 exit status

signal_shutdown() is only used by rgw_main.cc to notify the main thread,
but we don't need this facility in unit test exercising the schedulers,
so, in this change, rgw schedulers are extracted into librgw_schedulers,
so this library can be consumed by the unit test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: link rbd_reply against rbd_replay_types
Kefu Chai [Mon, 3 Jun 2019 17:06:29 +0000 (01:06 +0800)]
cmake: link rbd_reply against rbd_replay_types

should make it `PUBLIC` because, for instance,

src/rbd_replay/actions.hpp uses
`rbd_replay::action::IoActionBase::dump()` and
`rbd_replay::action::ActionBase::dump()`

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: link libosd against libos
Kefu Chai [Mon, 3 Jun 2019 16:13:42 +0000 (00:13 +0800)]
cmake: link libosd against libos

apparently, libosd uses libos, but it does not use leveldb directly,
so s/leveldb/os/.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: link rgw tests against ${rgw_libs}
Kefu Chai [Mon, 3 Jun 2019 08:36:12 +0000 (16:36 +0800)]
cmake: link rgw tests against ${rgw_libs}

if `WITH_RADOSGW_AMQP_ENDPOINT` is enabled, we should always link rgw
applications against `RabbitMQ::RabbitMQ` or `amqp_mock`, so, to
simplify the linkage dependencies, we should just use `${rgw_libs}`
instead of using `rgw_a` directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: link libkv against libheap_profiler
Kefu Chai [Mon, 3 Jun 2019 08:34:46 +0000 (16:34 +0800)]
cmake: link libkv against libheap_profiler

PriorityCache.cc uses `ceph_heap_*` functions, and libkv contains
`PriorityCache.cc`, so it should link against libheap_profiler.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: link libkv against common_prioritycache_obj
Kefu Chai [Mon, 3 Jun 2019 07:02:51 +0000 (15:02 +0800)]
cmake: link libkv against common_prioritycache_obj

as both `kv/rocksdb_cache/BinnedLRUCache.cc` and
`kv/rocksdb_cache/ShardedCache.cc` uses `PriorityCache` which is in turn
offered by common_prioritycache_obj, we need to link libkv against the
latter.

libos uses `PriorityCache` as well, but since libos links against
libkv, so it's safe.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: remove kv_objs target
Kefu Chai [Mon, 3 Jun 2019 05:27:57 +0000 (13:27 +0800)]
cmake: remove kv_objs target

libkv is better than kv_objs, as it is able to bring more
properties with with it, for instance, the public include
directories, the public/private libraries it is linked against.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: use BUILD_BYPRODUCTS to fix build w/ ninja
Kefu Chai [Mon, 27 May 2019 11:53:42 +0000 (19:53 +0800)]
cmake: use BUILD_BYPRODUCTS to fix build w/ ninja

ninja scan the build dependencies differently than GNU make, so we need
to specify the product of `ExternalProject_Add()`. with this change,
we can build `vstart-base` without building the ExternalProject
separately.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: include googletest instead of its subdirectory
Kefu Chai [Mon, 27 May 2019 07:09:25 +0000 (15:09 +0800)]
cmake: include googletest instead of its subdirectory

so its own settings can be populated properly, among other things,
`GOOGLETEST_VERSION` is set in the CMakeLists.txt in its root directory.
without it, cmake complains when interpreting

project(gtest VERSION ${GOOGLETEST_VERSION} LANGUAGES CXX C)

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: do not disable Seastar_TESTING explicitly
Kefu Chai [Mon, 27 May 2019 07:08:43 +0000 (15:08 +0800)]
cmake: do not disable Seastar_TESTING explicitly

this option is disabled by default if seastar is included as a
subdirectory.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28155 from alimaredia/wip-remove-mon-valgrind-check-in-rgw-suite
Matt Benjamin [Fri, 24 May 2019 16:04:51 +0000 (12:04 -0400)]
Merge pull request #28155 from alimaredia/wip-remove-mon-valgrind-check-in-rgw-suite

qa: remove mon valgrind check in rgw verfiy suite

6 years agomgr/dashboard: Bug Fix Error on Mgr Dashboard NFS (#28075)
Lenz Grimmer [Thu, 23 May 2019 08:47:55 +0000 (10:47 +0200)]
mgr/dashboard: Bug Fix Error on Mgr Dashboard NFS (#28075)

mgr/dashboard: Bug Fix Error on Mgr Dashboard NFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #28169 from bk201/wip-39971
Lenz Grimmer [Wed, 22 May 2019 16:23:34 +0000 (18:23 +0200)]
Merge pull request #28169 from bk201/wip-39971

mgr/dashboard: fix some performance data are not displayed

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
6 years agoMerge pull request #28181 from smithfarm/wip-39981
Nathan Cutler [Wed, 22 May 2019 16:02:45 +0000 (18:02 +0200)]
Merge pull request #28181 from smithfarm/wip-39981

do_cmake.sh: use bash

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #28093 from dillaman/wip-39703
Mykola Golub [Tue, 21 May 2019 18:52:59 +0000 (21:52 +0300)]
Merge pull request #28093 from dillaman/wip-39703

librbd: use custom allocator for aligned boost::lockfree::queue

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #28147 from bk201/wip-39933
Lenz Grimmer [Tue, 21 May 2019 12:00:54 +0000 (14:00 +0200)]
Merge pull request #28147 from bk201/wip-39933

mgr/dashboard: fix backend error when updating RBD interlocked features

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agomgr/dashboard: Fix tcmu-runner perf counters page (#28154)
Lenz Grimmer [Tue, 21 May 2019 09:28:02 +0000 (11:28 +0200)]
mgr/dashboard: Fix tcmu-runner perf counters page (#28154)

mgr/dashboard: Fix tcmu-runner perf counters page

Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #26518 from linuxbox2/wip-rgw-lc-prefix-2
Matt Benjamin [Tue, 21 May 2019 07:38:56 +0000 (03:38 -0400)]
Merge pull request #26518 from linuxbox2/wip-rgw-lc-prefix-2

rgw: lifecycle: alternate solution to prefix_map conflict

6 years agorgw: lifecycle: fix object iteration for >1 same-prefix rules 26518/head
Matt Benjamin [Thu, 28 Feb 2019 04:08:54 +0000 (23:08 -0500)]
rgw: lifecycle: fix object iteration for >1 same-prefix rules

Rules of like-prefix must visit every matching object.  A next
logical step is to combine tag checking for rules of like-prefix.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agorgw: lifecycle: fix RGWLifecycleConfiguration::valid()
Matt Benjamin [Mon, 25 Feb 2019 20:36:07 +0000 (15:36 -0500)]
rgw: lifecycle: fix RGWLifecycleConfiguration::valid()

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agorgw: lifecycle: alternate solution to prefix_map conflict
Matt Benjamin [Tue, 19 Feb 2019 18:29:03 +0000 (13:29 -0500)]
rgw: lifecycle: alternate solution to prefix_map conflict

A rule containing a tag filter but no constraining prefix is
legal, as is a sequence of >1 such rules.

As currently designed, enforcement requires rules to be
installed in prefix_map as well as rule_map, though only the
latter is encoded.

Fixes: http://tracker.ceph.com/issues/37879
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agodo_cmake.sh: use bash 28181/head
Nathan Cutler [Mon, 20 May 2019 15:01:40 +0000 (17:01 +0200)]
do_cmake.sh: use bash

Fixes: http://tracker.ceph.com/issues/39981
Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #27996 from jan--f/package-prometheus-alerts-suse
Nathan Cutler [Mon, 20 May 2019 10:25:44 +0000 (12:25 +0200)]
Merge pull request #27996 from jan--f/package-prometheus-alerts-suse

ceph.spec.in: package prometheus default alerts for SUSE

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agomgr/dashboard: fix some performance data are not displayed 28169/head
Kiefer Chang [Mon, 20 May 2019 06:18:32 +0000 (14:18 +0800)]
mgr/dashboard: fix some performance data are not displayed

Several Grafana dashboards were updated and their uids are changed.
Update corresponding uids in angular templates. Also fix that when a user
clicks performance details of an OSD, wrong OSD metrics are displayed.

Fixes: http://tracker.ceph.com/issues/39971
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
6 years agoMerge pull request #28094 from tchaikov/wip-install-deps-gcc-8
Kefu Chai [Sun, 19 May 2019 21:44:01 +0000 (23:44 +0200)]
Merge pull request #28094 from tchaikov/wip-install-deps-gcc-8

install-deps.sh: use gcc-8 on xenial and trusty

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #28079 from tchaikov/wip-crimson-rados-cleanup
Kefu Chai [Sun, 19 May 2019 21:41:58 +0000 (23:41 +0200)]
Merge pull request #28079 from tchaikov/wip-crimson-rados-cleanup

crimson: support pgnls and delete op

Reviewed-off-by: Samuel Just <sjust@redhat.com>
6 years agoqa: remove mon valgrind check in rgw verfiy suite 28155/head
Ali Maredia [Thu, 16 May 2019 16:03:20 +0000 (12:03 -0400)]
qa: remove mon valgrind check in rgw verfiy suite

memory leaks in the monitor are causing a significant
percentage of jobs run in the rgw verify suite to
fail even though the jobs succeeded before hand.

See: http://tracker.ceph.com/issues/38827 for
root cause

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agoMerge pull request #27653 from tspmelo/wip-nfs-pseudo
Ricardo Marques [Fri, 17 May 2019 14:15:10 +0000 (15:15 +0100)]
Merge pull request #27653 from tspmelo/wip-nfs-pseudo

mgr/dashboard: Improve NFS Pseudo pattern message

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #28023 from rjfd/wip-fix-ceph-mgr-systemd
Nathan Cutler [Fri, 17 May 2019 13:57:28 +0000 (15:57 +0200)]
Merge pull request #28023 from rjfd/wip-fix-ceph-mgr-systemd

systemd: ceph-mgr: set MemoryDenyWriteExecute to false

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #28044 from dillaman/wip-librbd-io-perf
Mykola Golub [Fri, 17 May 2019 06:03:13 +0000 (09:03 +0300)]
Merge pull request #28044 from dillaman/wip-librbd-io-perf

librbd: tweaks to increase IOPS and reduce CPU usage

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #28067 from yuriw/wip-yuriw-crontab-master
Yuri Weinstein [Thu, 16 May 2019 20:14:53 +0000 (13:14 -0700)]
Merge pull request #28067 from yuriw/wip-yuriw-crontab-master

qa/tests: added new client-upgrade-*-nautilus suites for jewel, luminous, mimic

6 years agomgr/dashboard: Fix tcmu-runner perf counters page 28154/head
Ricardo Marques [Thu, 16 May 2019 15:48:39 +0000 (16:48 +0100)]
mgr/dashboard: Fix tcmu-runner perf counters page

Fixes: https://tracker.ceph.com/issues/39954
Signed-off-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27681 from tspmelo/wip-charts
Ricardo Marques [Thu, 16 May 2019 15:55:18 +0000 (16:55 +0100)]
Merge pull request #27681 from tspmelo/wip-charts

mgr/dashboard: Unify the look of dashboard charts

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
6 years agoMerge pull request #28113 from hjwsm1989/fix-krb-compile-error
Kefu Chai [Thu, 16 May 2019 15:32:32 +0000 (23:32 +0800)]
Merge pull request #28113 from hjwsm1989/fix-krb-compile-error

auth/krb: fix Kerberos compile error

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/dashboard: fix backend error when updating RBD interlocked features 28147/head
Kiefer Chang [Thu, 16 May 2019 07:53:35 +0000 (15:53 +0800)]
mgr/dashboard: fix backend error when updating RBD interlocked features

Now object-map and fast-diff features are interlocked. Allow
enabling/disabling these two features together without raising an already
enabled/disabled error.

Fixes: http://tracker.ceph.com/issues/39933
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
6 years agomgr/dashboard: Bug Fix Error on Mgr Dashboard NFS 28075/head
Nur Faizin [Mon, 13 May 2019 03:42:25 +0000 (10:42 +0700)]
mgr/dashboard: Bug Fix Error on Mgr Dashboard NFS

This change fixes error on NFS Ganesha management dashboard that throws an error when request daemon list.

Fixes: https://tracker.ceph.com/issues/39709
Signed-off-by: Nur Faizin <nur.faizin91@gmail.com>
6 years agoauth/krb: fix Kerberos compile error 28113/head
huangjun [Wed, 15 May 2019 15:59:08 +0000 (23:59 +0800)]
auth/krb: fix Kerberos compile error

Fixes: http://tracker.ceph.com/issues/39948
Signed-off-by: huangjun <huangjun@xsky.com>
6 years agoMerge PR #28080 into master
Patrick Donnelly [Wed, 15 May 2019 22:48:15 +0000 (15:48 -0700)]
Merge PR #28080 into master

* refs/pull/28080/head:
cephfs-shell: Remove whitespace after '('
cephfs-shell: Remove unused 'flag' variable
cephfs-shell: Fix flake8 invalid escape sequence warning
cephfs-shell: Remove unnecessary empty lines
cephfs-shell: Fix flake8 bare 'except' warnings

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27822 into master
Patrick Donnelly [Wed, 15 May 2019 22:46:39 +0000 (15:46 -0700)]
Merge PR #27822 into master

* refs/pull/27822/head:
cephfs-shell: Fix mkdir relative path error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoqa/tests: added new client-upgrade-*-nautilus suites for jewel, luminous, mimic 28067/head
Yuri Weinstein [Fri, 10 May 2019 16:28:56 +0000 (09:28 -0700)]
qa/tests: added new client-upgrade-*-nautilus suites for jewel, luminous, mimic

NOTE:

!!!! three suites below MUST use --suite-branch jewel, luminous, mimic (see https://tracker.ceph.com/issues/24021)
ref: https://github.com/ceph/ceph/pull/27983; https://github.com/ceph/ceph/pull/27934; https://github.com/ceph/ceph/pull/28027
--filter "ubuntu_16.04,centos_7.6,rhel_7.6" - test ONLY supported distro BEFORE mimic
--filter "ubuntu_16.04,ubuntu_18.04,centos_7.6,rhel_7.6" - test ONLY supported distro AFTER mimic

Merge after https://github.com/ceph/ceph/pull/28027

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agolibrbd: use custom allocator for aligned boost::lockfree::queue 28093/head
Jason Dillaman [Tue, 14 May 2019 13:34:04 +0000 (09:34 -0400)]
librbd: use custom allocator for aligned boost::lockfree::queue

If tcmalloc is in-use as the allocator and its version is less than 2.6.2,
it might be missing support for 'aligned_alloc'. This can result in the
glibc version of 'aligned_alloc' being used to allocate memory that is
then freed by tcmalloc -- resulting in a crash.

Fixes: http://tracker.ceph.com/issues/39703
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agomgr/dashboard: Unify the look of dashboard charts 27681/head
Tiago Melo [Thu, 18 Apr 2019 15:09:07 +0000 (15:09 +0000)]
mgr/dashboard: Unify the look of dashboard charts

Fixes: http://tracker.ceph.com/issues/39384
Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #28092 from tchaikov/wip-dashboard-package
Kefu Chai [Wed, 15 May 2019 10:58:31 +0000 (18:58 +0800)]
Merge pull request #28092 from tchaikov/wip-dashboard-package

mgr/dashboard: Update tar to v4.4.8

Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #28043 from jan--f/update-grafana-dashboards
Laura Paduano [Wed, 15 May 2019 08:09:18 +0000 (10:09 +0200)]
Merge pull request #28043 from jan--f/update-grafana-dashboards

Update grafana dashboards

Reviewed-by: kmurarka@redhat.com
Reviewed-by: epuertat@redhat.com
6 years agoMerge PR #27377 into master
Patrick Donnelly [Tue, 14 May 2019 20:14:21 +0000 (13:14 -0700)]
Merge PR #27377 into master

* refs/pull/27377/head:
qa: fs Ignore getfattr errors for ceph.dir.pin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27594 into master
Patrick Donnelly [Tue, 14 May 2019 20:07:49 +0000 (13:07 -0700)]
Merge PR #27594 into master

* refs/pull/27594/head:
mgr/volumes: allow creation/deletion of FS subvolume group snapshots
mgr/volumes: allow creation/deletion of FS subvolume groups
mgr/volumes: allow creation/deletion of snapshots of FS subvolumes
mgr/volumes: allow fetching path of FS subvolumes
mgr/volumes: use the fs_subvolume module
mgr/volumes: add fs_subvolume module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27483 into master
Patrick Donnelly [Tue, 14 May 2019 19:52:54 +0000 (12:52 -0700)]
Merge PR #27483 into master

* refs/pull/27483/head:
cephfs-shell: add quota management

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27753 into master
Patrick Donnelly [Tue, 14 May 2019 19:51:45 +0000 (12:51 -0700)]
Merge PR #27753 into master

* refs/pull/27753/head:
cephfs-shell: Add stat command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27872 into master
Patrick Donnelly [Tue, 14 May 2019 19:50:28 +0000 (12:50 -0700)]
Merge PR #27872 into master

* refs/pull/27872/head:
cephfs-shell: teuthology tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoinstall-deps.sh: use gcc-8 on xenial and trusty 28094/head
Kefu Chai [Tue, 14 May 2019 14:37:00 +0000 (22:37 +0800)]
install-deps.sh: use gcc-8 on xenial and trusty

to be consistent with #27183 and https://github.com/ceph/ceph-build/pull/1286

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/volumes: allow creation/deletion of FS subvolume group snapshots 27594/head
Ramana Raja [Tue, 30 Apr 2019 10:01:27 +0000 (15:31 +0530)]
mgr/volumes: allow creation/deletion of FS subvolume group snapshots

... using `ceph fs subvolumegroup snapshot create/rm` commands.

Signed-off-by: Ramana Raja <rraja@redhat.com>
6 years agomgr/volumes: allow creation/deletion of FS subvolume groups
Ramana Raja [Mon, 29 Apr 2019 12:31:49 +0000 (18:01 +0530)]
mgr/volumes: allow creation/deletion of FS subvolume groups

... using `ceph fs subvolumegroup create/rm` commands.
FS subvolume groups are parent directories of FS subvolumes. They
can be directly mapped to OpenStack Manila share groups.

Signed-off-by: Ramana Raja <rraja@redhat.com>
6 years agomgr/volumes: allow creation/deletion of snapshots of FS subvolumes
Ramana Raja [Fri, 19 Apr 2019 16:56:38 +0000 (22:26 +0530)]
mgr/volumes: allow creation/deletion of snapshots of FS subvolumes

... using `ceph fs subvolume snapshot create/rm` commands.

Signed-off-by: Ramana Raja <rraja@redhat.com>
6 years agomgr/volumes: allow fetching path of FS subvolumes
Ramana Raja [Tue, 23 Apr 2019 13:03:58 +0000 (18:33 +0530)]
mgr/volumes: allow fetching path of FS subvolumes

... using `ceph fs subvolume getpath` command.

Signed-off-by: Ramana Raja <rraja@redhat.com>
6 years agomgr/volumes: use the fs_subvolume module
Ramana Raja [Sun, 14 Apr 2019 13:38:50 +0000 (19:08 +0530)]
mgr/volumes: use the fs_subvolume module

... instead of src/pybind/ceph_volume_client.py to manage CephFS
subvolumes using `ceph fs subvolume` set of commands.

Signed-off-by: Ramana Raja <rraja@redhat.com>
6 years agomgr/volumes: add fs_subvolume module
Ramana Raja [Sat, 13 Apr 2019 16:06:05 +0000 (21:36 +0530)]
mgr/volumes: add fs_subvolume module

fs_subvolume module provisions and manages CephFS subvolumes, which are
CephFS subdirectories with a desired layout and quota. Its code is
heavily borrowed from, src/pybind/ceph_volume_client.py

Fixes: http://tracker.ceph.com/issues/39610
Signed-off-by: Ramana Raja <rraja@redhat.com>
6 years agomonitoring: update Grafana dashboards 28043/head
Jan Fajerski [Wed, 6 Mar 2019 08:25:24 +0000 (09:25 +0100)]
monitoring: update Grafana dashboards

Fix various panels that used outdated metric names, cluncky or
unnecessary label_replace calls. Also unify the style of many panels.

Fixes: http://tracker.ceph.com/issues/39652
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
6 years agomgr/dashboard: Update tar to v4.4.8 28092/head
Kefu Chai [Tue, 14 May 2019 10:30:01 +0000 (18:30 +0800)]
mgr/dashboard: Update tar to v4.4.8

This update fixes an Arbitrary File Overwrite issue (CVE-2018-20834).
see https://nvd.nist.gov/vuln/detail/CVE-2018-20834

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28085 from rzarzynski/wip-crimson-no-atomics-in-refcntobj
Kefu Chai [Tue, 14 May 2019 01:15:28 +0000 (09:15 +0800)]
Merge pull request #28085 from rzarzynski/wip-crimson-no-atomics-in-refcntobj

crimson, common: RefCountedObj doesn't use atomics in Seastar builds.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agolibrbd: avoid bl manipulation on single object read/writes 28044/head
Jason Dillaman [Thu, 9 May 2019 03:31:01 +0000 (23:31 -0400)]
librbd: avoid bl manipulation on single object read/writes

If a read or write only spans a single object and buffer extent,
there isn't any need to build a new bufferlist from a full sub-bl.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: switch to lightweight striper for hot IO path
Jason Dillaman [Thu, 9 May 2019 03:03:31 +0000 (23:03 -0400)]
librbd: switch to lightweight striper for hot IO path

Optimize non-complex IO requests using the new lightweight object
striper that avoids heap allocations for small requests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: drop data oid parameter from IO path
Jason Dillaman [Tue, 7 May 2019 20:49:54 +0000 (16:49 -0400)]
librbd: drop data oid parameter from IO path

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: helper method to generate data object OIDs on-demand
Jason Dillaman [Tue, 7 May 2019 16:49:02 +0000 (12:49 -0400)]
librbd: helper method to generate data object OIDs on-demand

The new generator avoids multiple string length and copy operations
that are currently required in the Striper.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoosdc/Striper: implemented a lightweight version of StripedReadResult
Jason Dillaman [Thu, 9 May 2019 02:06:47 +0000 (22:06 -0400)]
osdc/Striper: implemented a lightweight version of StripedReadResult

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoosdc/Striper: implemented a lightweight version of file_to_extents
Jason Dillaman [Wed, 8 May 2019 21:34:02 +0000 (17:34 -0400)]
osdc/Striper: implemented a lightweight version of file_to_extents

The previous versions of file_to_extents resulted in numerous heap
allocations for maps, vectors, and strings -- in addition to using
a CPU inefficient oid name formater.

librbd doesn't require the use of object_t and it would be better to
avoid any heap allocations on the IO path for the common case.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agocommon/Finisher: avoid memory re-allocations for finisher queue
Jason Dillaman [Wed, 8 May 2019 18:09:17 +0000 (14:09 -0400)]
common/Finisher: avoid memory re-allocations for finisher queue

Since there is only a single thread associated with the Finisher,
swap between two queues to avoid the need to re-allocate a new
vector for each iteration through the loop. Also replace the
condition broadcast with a signal since there is only a single
thread to wake up.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27999 from trociny/wip-rbd-sparse_copyup
Jason Dillaman [Mon, 13 May 2019 16:15:51 +0000 (12:15 -0400)]
Merge pull request #27999 from trociny/wip-rbd-sparse_copyup

librbd: clone copy-on-write operations should preserve sparseness

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agocrimson/osd: add crimson_debug_pg_always_active option 28079/head
Kefu Chai [Thu, 9 May 2019 10:51:31 +0000 (18:51 +0800)]
crimson/osd: add crimson_debug_pg_always_active option

before crimson is able to peer, we should leave this option ON.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: add preliminary delete support
Kefu Chai [Mon, 13 May 2019 11:45:05 +0000 (19:45 +0800)]
crimson/osd: add preliminary delete support

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: add pgnls support
Kefu Chai [Mon, 13 May 2019 11:08:42 +0000 (19:08 +0800)]
crimson/osd: add pgnls support

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28071 from tchaikov/wip-crimson-perf-regres
Kefu Chai [Mon, 13 May 2019 16:00:00 +0000 (00:00 +0800)]
Merge pull request #28071 from tchaikov/wip-crimson-perf-regres

crimson,osd: performance fixes

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson, common: RefCountedObj doesn't use atomics in SeaStar builds. 28085/head
Radoslaw Zarzynski [Mon, 13 May 2019 15:57:12 +0000 (17:57 +0200)]
crimson, common: RefCountedObj doesn't use atomics in SeaStar builds.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomgr/dashboard: Improve NFS Pseudo pattern message 27653/head
Tiago Melo [Wed, 17 Apr 2019 16:46:47 +0000 (16:46 +0000)]
mgr/dashboard: Improve NFS Pseudo pattern message

Fixes: http://tracker.ceph.com/issues/39327
Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #27270 from theanalyst/beast-fe-ipv6
Casey Bodley [Mon, 13 May 2019 14:02:21 +0000 (10:02 -0400)]
Merge pull request #27270 from theanalyst/beast-fe-ipv6

rgw: normalize v6 endpoint behaviour for the beast frontend

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26281 from smithfarm/wip-git-describe-long
Nathan Cutler [Mon, 13 May 2019 13:10:20 +0000 (15:10 +0200)]
Merge pull request #26281 from smithfarm/wip-git-describe-long

make-dist: set version number only once

Reviewed-by: Tim Serong <tserong@suse.com>
6 years agoMerge pull request #28045 from cbodley/wip-39635
Casey Bodley [Mon, 13 May 2019 13:09:17 +0000 (09:09 -0400)]
Merge pull request #28045 from cbodley/wip-39635

cls/user: cls_user_set_buckets_info overwrites creation_time

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #27980 from cbodley/wip-24991
Casey Bodley [Mon, 13 May 2019 13:07:04 +0000 (09:07 -0400)]
Merge pull request #27980 from cbodley/wip-24991

rgw: delete_obj_index() takes mtime for bilog

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #28031 from ivancich/wip-rgw-admin-unordered
Casey Bodley [Mon, 13 May 2019 13:05:29 +0000 (09:05 -0400)]
Merge pull request #28031 from ivancich/wip-rgw-admin-unordered

rgw: allow radosgw-admin to list bucket w --allow-unordered

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #27676 from mkogan1/wip-rgw-39380
Casey Bodley [Mon, 13 May 2019 12:53:58 +0000 (08:53 -0400)]
Merge pull request #27676 from mkogan1/wip-rgw-39380

rgw: fix default_placement containing "/" when storage_class is standard

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27984 from cbodley/wip-test-rgw-multisite-sh
Casey Bodley [Mon, 13 May 2019 12:51:47 +0000 (08:51 -0400)]
Merge pull request #27984 from cbodley/wip-test-rgw-multisite-sh

test/rgw: fix test-rgw-multisite.sh script for creating multisite clusters

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
6 years agoMerge pull request #27807 from cyb70289/utf8-encode
Kefu Chai [Mon, 13 May 2019 12:42:24 +0000 (20:42 +0800)]
Merge pull request #27807 from cyb70289/utf8-encode

common: optimize encode_utf8

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocephfs-shell: Remove whitespace after '(' 28080/head
Varsha Rao [Mon, 13 May 2019 12:15:35 +0000 (17:45 +0530)]
cephfs-shell: Remove whitespace after '('

This patch fixes flake8 whitespace error.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
6 years agocephfs-shell: Remove unused 'flag' variable
Varsha Rao [Mon, 13 May 2019 12:11:06 +0000 (17:41 +0530)]
cephfs-shell: Remove unused 'flag' variable

This patch fixes flake8 unused local variable error.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
6 years agocephfs-shell: Fix flake8 invalid escape sequence warning
Varsha Rao [Mon, 13 May 2019 12:00:01 +0000 (17:30 +0530)]
cephfs-shell: Fix flake8 invalid escape sequence warning

Backslash-character pair is not a valid escape sequence, since Python 3.6
version. Prefixing character 'r' to the escape sequence, fixes the warning.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
6 years agocephfs-shell: Remove unnecessary empty lines
Varsha Rao [Mon, 13 May 2019 11:52:07 +0000 (17:22 +0530)]
cephfs-shell: Remove unnecessary empty lines

This patch fixes flake8 too many blank lines error.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
6 years agocephfs-shell: Fix flake8 bare 'except' warnings
Varsha Rao [Mon, 13 May 2019 11:43:39 +0000 (17:13 +0530)]
cephfs-shell: Fix flake8 bare 'except' warnings

Instead of bare 'except', catch just libcephfs exceptions.

Fixes: https://tracker.ceph.com/issues/39717
Signed-off-by: Varsha Rao <varao@redhat.com>
6 years agoMerge pull request #28051 from dillaman/wip-39521
Mykola Golub [Mon, 13 May 2019 12:25:09 +0000 (15:25 +0300)]
Merge pull request #28051 from dillaman/wip-39521

librbd: fix issues with object-map/fast-diff feature interlock

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #28063 from dillaman/wip-39656
Mykola Golub [Mon, 13 May 2019 12:24:39 +0000 (15:24 +0300)]
Merge pull request #28063 from dillaman/wip-39656

qa/workunits/rbd: use https protocol for devstack git operations

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agocmake: define BOOST_SP_DISABLE_THREADS for crimson 28071/head
Kefu Chai [Sun, 12 May 2019 14:28:57 +0000 (22:28 +0800)]
cmake: define BOOST_SP_DISABLE_THREADS for crimson

because we are single-threaded

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/osd_internal_types: bring moving semantics to ObjectState.
Radoslaw Zarzynski [Sat, 11 May 2019 01:41:52 +0000 (21:41 -0400)]
osd/osd_internal_types: bring moving semantics to ObjectState.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: simplify write handling in PG::do_osd_ops.
Radoslaw Zarzynski [Fri, 10 May 2019 17:51:41 +0000 (13:51 -0400)]
crimson/osd: simplify write handling in PG::do_osd_ops.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agolibrbd: clone copy-on-write operations should preserve sparseness 27999/head
Mykola Golub [Tue, 7 May 2019 05:41:27 +0000 (06:41 +0100)]
librbd: clone copy-on-write operations should preserve sparseness

Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #27870 from theanalyst/rgw-objexp-fixes-cli
J. Eric Ivancich [Sat, 11 May 2019 13:10:54 +0000 (09:10 -0400)]
Merge pull request #27870 from theanalyst/rgw-objexp-fixes-cli

rgw: object expirer fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #27926 from tchaikov/wip-38627
Kefu Chai [Sat, 11 May 2019 01:37:39 +0000 (09:37 +0800)]
Merge pull request #27926 from tchaikov/wip-38627

pybind/mgr: convert str to int using "int()"

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
6 years agoMerge pull request #27914 from dzafman/wip-39304
David Zafman [Sat, 11 May 2019 00:41:31 +0000 (17:41 -0700)]
Merge pull request #27914 from dzafman/wip-39304

osd: Include dups in copy_after() and copy_up_to()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27961 from dzafman/wip-39582
David Zafman [Fri, 10 May 2019 22:46:16 +0000 (15:46 -0700)]
Merge pull request #27961 from dzafman/wip-39582

osd: Output Base64 encoding of CRC header if binary data present

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: Fix merge_log_dups() not remove dup corresponding to log.tail 27914/head
David Zafman [Thu, 9 May 2019 17:38:36 +0000 (10:38 -0700)]
osd: Fix merge_log_dups() not remove dup corresponding to log.tail

The log tail is not inclusive of the log, so not an overlap
Fix unittest_pglog to verify proper operation

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agotest: unittest_pglog: Additional test cases for trim, copy_up_to and copy_after
David Zafman [Tue, 7 May 2019 04:19:59 +0000 (21:19 -0700)]
test: unittest_pglog: Additional test cases for trim, copy_up_to and copy_after

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agotest: unittest_pglog: Remove irrelevant config values
David Zafman [Tue, 7 May 2019 02:57:55 +0000 (19:57 -0700)]
test: unittest_pglog: Remove irrelevant config values

osd_min_pg_log_entries and osd_max_pg_log_entries are not involved
in the log handling code being tested.

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agotest: Adding standalone test of log copy handling
David Zafman [Fri, 3 May 2019 20:33:39 +0000 (13:33 -0700)]
test: Adding standalone test of log copy handling

Signed-off-by: David Zafman <dzafman@redhat.com>