]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agodebian/control: Build-Depends on g++ 30410/head
Kefu Chai [Mon, 16 Sep 2019 15:44:10 +0000 (23:44 +0800)]
debian/control: Build-Depends on g++

as a C++ project, we should depends on g++. actually, GCC-7 is able to
support C++17. but we use C++17 features extensively, and have been
running into GCC bugs every now and then. so would be ideal if we can
use newer GCC if possible.

we've dropped the support of ubuntu/xenial. while on ubuntu/bionic, only
GCC-7.4 is available. so let's require g++ (>= 7) then.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
6 years agoMerge pull request #30368 from liewegas/wip-snapmapper-cleanup
Kefu Chai [Mon, 16 Sep 2019 11:41:05 +0000 (19:41 +0800)]
Merge pull request #30368 from liewegas/wip-snapmapper-cleanup

osd/SnapMapper: remove pre-octopus snapmapper keys after conversion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #30356 from majianpeng/msg-reduece-duplicated-code
Kefu Chai [Mon, 16 Sep 2019 11:39:45 +0000 (19:39 +0800)]
Merge pull request #30356 from majianpeng/msg-reduece-duplicated-code

msg: add func is_blackhole to reduce duplicated code.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #30350 from liewegas/wip-bluestore-poolstats-configs
Kefu Chai [Mon, 16 Sep 2019 11:37:27 +0000 (19:37 +0800)]
Merge pull request #30350 from liewegas/wip-bluestore-poolstats-configs

os/bluestore: simplify per-pool-stat config options

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #30398 from tspmelo/wip-backport-head
Nathan Cutler [Mon, 16 Sep 2019 10:53:12 +0000 (12:53 +0200)]
Merge pull request #30398 from tspmelo/wip-backport-head

script/ceph-backport.sh: Fix verification of git repository

6 years agoscript/ceph-backport.sh: Fix verification of git repository 30398/head
Tiago Melo [Mon, 16 Sep 2019 10:19:35 +0000 (10:19 +0000)]
script/ceph-backport.sh: Fix verification of git repository

In git 2.16 running "git show-ref HEAD" returns an error and
prevents the script to procced.

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29549 from smanjara/wip-multisite-metadata
J. Eric Ivancich [Sun, 15 Sep 2019 16:54:24 +0000 (12:54 -0400)]
Merge pull request #29549 from smanjara/wip-multisite-metadata

rgw: Allow admin APIs that write metadata to be executed first on the mast…

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #30174 from mkogan1/wip-rgw-40700
J. Eric Ivancich [Sun, 15 Sep 2019 16:48:28 +0000 (12:48 -0400)]
Merge pull request #30174 from mkogan1/wip-rgw-40700

rgw: fix memory growth while deleting objects with

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #30379 from smithfarm/wip-41821
Nathan Cutler [Sun, 15 Sep 2019 07:32:12 +0000 (09:32 +0200)]
Merge pull request #30379 from smithfarm/wip-41821

tools/setup-virtualenv.sh: do not default to python2.7

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
6 years agoMerge pull request #30362 from uweigand/fix-unaligned-check
Kefu Chai [Sat, 14 Sep 2019 01:46:41 +0000 (09:46 +0800)]
Merge pull request #30362 from uweigand/fix-unaligned-check

cmake: Fix unaligned check on big-endian systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #30374 from tchaikov/wip-cmake-brotli
Kefu Chai [Sat, 14 Sep 2019 01:43:04 +0000 (09:43 +0800)]
Merge pull request #30374 from tchaikov/wip-cmake-brotli

cmake: s/bortli_libs/brotli_libs/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #30366 from liu-chunmei/ceph_seastar_MonClient
Kefu Chai [Sat, 14 Sep 2019 01:05:48 +0000 (09:05 +0800)]
Merge pull request #30366 from liu-chunmei/ceph_seastar_MonClient

crimson/mon: use shared_future for waiting MauthReply

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #30380 from rzarzynski/wip-crimson-cyanread-returns-bl
Kefu Chai [Sat, 14 Sep 2019 01:04:30 +0000 (09:04 +0800)]
Merge pull request #30380 from rzarzynski/wip-crimson-cyanread-returns-bl

crimson/os: Object::read() returns bufferlist instead of never used errcode

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #30339 from rzarzynski/wip-crimson-cls-2
Kefu Chai [Sat, 14 Sep 2019 01:01:10 +0000 (09:01 +0800)]
Merge pull request #30339 from rzarzynski/wip-crimson-cls-2

crimson: serve basic RBD traffic coming from fio

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/mon:use shared_future for waiting MauthReply 30366/head
chunmei Liu [Thu, 12 Sep 2019 21:01:37 +0000 (14:01 -0700)]
crimson/mon:use shared_future for waiting MauthReply

when monitor-side close the connection, msgr call MonClient
ms_handle_reset cause reply.get_future be called twice then
assert happen in promise.get_future.
promise<T...>::get_future() noexcept {
    assert(!this->_future && this->_state && !this->_task);
    return future<T...>(this);
}
use shared_promise instead of promise to solve it.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
6 years agoMerge pull request #30363 from idryomov/wip-krbd-huge-osdmap
Ilya Dryomov [Fri, 13 Sep 2019 20:35:57 +0000 (22:35 +0200)]
Merge pull request #30363 from idryomov/wip-krbd-huge-osdmap

qa: add script to test how libceph handles huge osdmaps

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agocrimson/osd: add dummy support for CEPH_OSD_OP_WATCH. 30339/head
Radoslaw Zarzynski [Wed, 28 Aug 2019 14:40:30 +0000 (16:40 +0200)]
crimson/osd: add dummy support for CEPH_OSD_OP_WATCH.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_get_client_features().
Radoslaw Zarzynski [Thu, 29 Aug 2019 14:32:44 +0000 (16:32 +0200)]
crimson/osd: implement cls_get_client_features().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: switch get_orig_source_inst() to get_message() in OpsExecuter.
Radoslaw Zarzynski [Thu, 29 Aug 2019 14:32:16 +0000 (16:32 +0200)]
crimson/osd: switch get_orig_source_inst() to get_message() in OpsExecuter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_map_set_val().
Radoslaw Zarzynski [Wed, 28 Aug 2019 15:10:08 +0000 (17:10 +0200)]
crimson/osd: implement cls_cxx_map_set_val().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_get_request_origin().
Radoslaw Zarzynski [Wed, 28 Aug 2019 14:39:59 +0000 (16:39 +0200)]
crimson/osd: implement cls_get_request_origin().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPGETVALS.
Radoslaw Zarzynski [Wed, 28 Aug 2019 11:41:50 +0000 (13:41 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPGETVALS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #28560 into master
Patrick Donnelly [Fri, 13 Sep 2019 19:38:47 +0000 (12:38 -0700)]
Merge PR #28560 into master

* refs/pull/28560/head:
cephfs-shell: handle du's arguments elsewhere outside do_du()
cephfs-shell: reuse code
cephfs-shell: rewrite call to perror in do_du
pybind/cephfs: define variable for hexcode used in stat()
test_cephfs_shell: test cephfs-shell command at invocation
cephfs-shell: refactor do_du()
cephfs-shell: option -r is not for reverse
cephfs-shell: extend to_bytes()
test_cephfs_shell: test du with no args
test_cephfs_shell: test du with multiple paths in args
test_cephfs_shell: test behaviour of "du -r"
test_cephfs_shell: test du's output for softlinks
qa/cephfs: add convenience method lstat()
qa/cephfs: add option to make stat() unfollow symlinks
test_cephfs_shell: test du's output for hardlinks
test_cephfs_shell: test du's output for directories
test_cephfs_shell: test du's output for regular files
test_cephfs_shell: add a method to get command output
test_cephfs_shell: allow cmd as list too
test_cephfs_shell: rename and rewrite _cephfs_shell()
test_cephfs_shell: copy humanize() from cephfs-shell
cephfs-shell: print disk usage for non-directory files too
pybind/cephfs: add method that stats symlinks without following
cephfs-shell: Fix 'du' command error

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29907 into master
Patrick Donnelly [Fri, 13 Sep 2019 19:04:44 +0000 (12:04 -0700)]
Merge PR #29907 into master

* refs/pull/29907/head:
doc: add a doc for vstart_runner.py

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #30185 into master
Patrick Donnelly [Fri, 13 Sep 2019 19:02:20 +0000 (12:02 -0700)]
Merge PR #30185 into master

* refs/pull/30185/head:
mgr/volumes: drop unnecessary size

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agocrimson/os: fix ENODATA handling in CyanStore::get_attrs().
Radoslaw Zarzynski [Tue, 27 Aug 2019 20:47:37 +0000 (22:47 +0200)]
crimson/os: fix ENODATA handling in CyanStore::get_attrs().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPGETKEYS.
Radoslaw Zarzynski [Tue, 27 Aug 2019 19:33:30 +0000 (21:33 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPGETKEYS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: PGBackend::write() alters object size now.
Radoslaw Zarzynski [Tue, 27 Aug 2019 19:32:16 +0000 (21:32 +0200)]
crimson/osd: PGBackend::write() alters object size now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_write2().
Radoslaw Zarzynski [Tue, 27 Aug 2019 19:31:07 +0000 (21:31 +0200)]
crimson/osd: implement cls_cxx_write2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPSETVALS.
Radoslaw Zarzynski [Tue, 27 Aug 2019 15:23:37 +0000 (17:23 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPSETVALS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPGETVALSBYKEYS.
Radoslaw Zarzynski [Tue, 27 Aug 2019 14:43:14 +0000 (16:43 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPGETVALSBYKEYS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #29037 from zhangsw/rgw-reshard-numshards
Casey Bodley [Fri, 13 Sep 2019 18:04:13 +0000 (14:04 -0400)]
Merge pull request #29037 from zhangsw/rgw-reshard-numshards

rgw: shard number must be non-negative when resharding the bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agocrimson/osd: add support for CEPH_OSD_OP_CREATE.
Radoslaw Zarzynski [Tue, 27 Aug 2019 14:01:49 +0000 (16:01 +0200)]
crimson/osd: add support for CEPH_OSD_OP_CREATE.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: ignore CEPH_OSD_OP_TMAPUP like ceph-osd does for EC pools.
Radoslaw Zarzynski [Mon, 26 Aug 2019 18:29:47 +0000 (20:29 +0200)]
crimson/osd: ignore CEPH_OSD_OP_TMAPUP like ceph-osd does for EC pools.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoqa: add script to test how libceph handles huge osdmaps 30363/head
Ilya Dryomov [Wed, 11 Sep 2019 11:15:59 +0000 (13:15 +0200)]
qa: add script to test how libceph handles huge osdmaps

That code will also handle moderately-sized osdmaps when the memory is
fragmented.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agoMerge pull request #29623 from majianpeng/bluestore-combine-deferred_finisher-and...
Kefu Chai [Fri, 13 Sep 2019 16:35:44 +0000 (00:35 +0800)]
Merge pull request #29623 from majianpeng/bluestore-combine-deferred_finisher-and-finisher

os/bluestore: Merge deferred_finisher and finisher.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #30342 from ifed01/wip-ifed-fix-blues-alerts
Kefu Chai [Fri, 13 Sep 2019 16:34:20 +0000 (00:34 +0800)]
Merge pull request #30342 from ifed01/wip-ifed-fix-blues-alerts

mon/pgmap: fix bluestore alerts output.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agocrimson/os: Object::read() returns bufferlist instead of never used errcode. 30380/head
Radoslaw Zarzynski [Mon, 9 Sep 2019 11:38:11 +0000 (13:38 +0200)]
crimson/os: Object::read() returns bufferlist instead of never used errcode.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #30372 from tchaikov/wip-crimson-single-pg-peering
Kefu Chai [Fri, 13 Sep 2019 15:54:22 +0000 (23:54 +0800)]
Merge pull request #30372 from tchaikov/wip-crimson-single-pg-peering

crimson/osd: use single-pg peering ops

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #24794 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:36:32 +0000 (08:36 -0700)]
Merge PR #24794 into master

* refs/pull/24794/head:
mds: move BatchOp to separate translation unit
mds: use auto ref during iteration
mds: answering all pending getattr/lookups targeting the same inode in one go

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #28702 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:35:04 +0000 (08:35 -0700)]
Merge PR #28702 into master

* refs/pull/28702/head:
qa: update json format from session listing
mds: recall caps from quiescent sessions
mds: use Session::dump method uniformly
mds: use auto to deduce iterator type
mds: simplify method definition
mds: remove useless debug message
mds: use const get_request_count
mds: use session_info_t socket inst for addr
mds: refactor session lookup
mds: add explicit trim flag
mds: alphabetize tracked config keys
common: provide method to get half-life
common: correct variable name

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge PR #29005 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:33:48 +0000 (08:33 -0700)]
Merge PR #29005 into master

* refs/pull/29005/head:
client: nfs-ganesha with cephfs client, removing dir reports not empty

Reviewed-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge PR #29594 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:32:10 +0000 (08:32 -0700)]
Merge PR #29594 into master

* refs/pull/29594/head:
mds: reject sessionless messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29752 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:30:13 +0000 (08:30 -0700)]
Merge PR #29752 into master

* refs/pull/29752/head:
mds: make MDSIOContextBase delete itself when shutting down

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #30361 into master
Sage Weil [Fri, 13 Sep 2019 14:59:01 +0000 (09:59 -0500)]
Merge PR #30361 into master

* refs/pull/30361/head:
qa/tasks/mon_thrash: sync force requires some force flags

Reviewed-by: João Eduardo Luís <joao@suse.de>
6 years agoMerge pull request #30357 from zhangsw/rgw-cleanup-bilog-status
Casey Bodley [Fri, 13 Sep 2019 14:43:14 +0000 (10:43 -0400)]
Merge pull request #30357 from zhangsw/rgw-cleanup-bilog-status

rgw: add missing bilog status to help info.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agotools/setup-virtualenv.sh: do not default to python2.7 30379/head
Nathan Cutler [Fri, 13 Sep 2019 14:12:52 +0000 (16:12 +0200)]
tools/setup-virtualenv.sh: do not default to python2.7

This commit changes the script's behavior when run without a --python option.
Before, we were running virtualenv with "--python python2.7" in that case.

That is not a reasonable default in September 2019, given that Python 2 EOL is
coming on January 1, 2020. Also, it breaks setup-virtualenv.sh in py3-only
environments.

With this patch, the script will not pass any "--python" option to virtualenv
(i.e., it will let virtualenv do whatever it thinks is best) in this case.

Fixes: https://tracker.ceph.com/issues/41821
Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #29081 from wintamute/patch-1
Alfredo Deza [Fri, 13 Sep 2019 14:11:55 +0000 (10:11 -0400)]
Merge pull request #29081 from wintamute/patch-1

Documentation: Add missing ceph-volume lvm batch argument to ceph-volume.rst

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PR #30352 into master
Sage Weil [Fri, 13 Sep 2019 13:22:42 +0000 (08:22 -0500)]
Merge PR #30352 into master

* refs/pull/30352/head:
mon: disable min pg per osd warning

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #30284 from Damenly/missing_cmath
Kefu Chai [Fri, 13 Sep 2019 13:06:12 +0000 (21:06 +0800)]
Merge pull request #30284 from Damenly/missing_cmath

tests: add missing header cmath to test/mon/test_mon_memory_target.cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agodoc: Add missing ceph-volume lvm batch argument to ceph-volume.rst 29081/head
Andreas Krebs [Wed, 17 Jul 2019 11:09:34 +0000 (13:09 +0200)]
doc: Add missing ceph-volume lvm batch argument to ceph-volume.rst

The optional argument '--osds-per-device' is missing. Since this is quite helpful when creating NVMe based OSDs, it should be documented.

Signed-off-by: Andreas Krebs <wintamute@gmail.com>
6 years agomgr/dashboard: Increase the global value of jasmine timeout (#30359)
Lenz Grimmer [Fri, 13 Sep 2019 12:18:09 +0000 (12:18 +0000)]
mgr/dashboard: Increase the global value of jasmine timeout (#30359)

mgr/dashboard: Increase the global value of jasmine timeout

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agocmake: s/bortli_libs/brotli_libs/ 30374/head
Kefu Chai [Fri, 13 Sep 2019 09:30:22 +0000 (17:30 +0800)]
cmake: s/bortli_libs/brotli_libs/

it's merely a typo.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29761 from wjwithagen/wjw-fix-venv-python-version
Kefu Chai [Fri, 13 Sep 2019 08:59:27 +0000 (16:59 +0800)]
Merge pull request #29761 from wjwithagen/wjw-fix-venv-python-version

test/venv: do not hardwire to py2.7 for tox tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/dashboard: Fix calculation of PG Status percentage (#30343)
Lenz Grimmer [Fri, 13 Sep 2019 08:45:20 +0000 (08:45 +0000)]
mgr/dashboard: Fix calculation of PG Status percentage (#30343)

mgr/dashboard: Fix calculation of PG Status percentage

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agocrimson/osd: require_osd_release >= octopus to boot 30372/head
Kefu Chai [Fri, 13 Sep 2019 07:38:50 +0000 (15:38 +0800)]
crimson/osd: require_osd_release >= octopus to boot

we assume that crimson will GA after octopus+2, so no need to be
compatible with pre-octopus releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: use single-pg peering ops
Kefu Chai [Fri, 13 Sep 2019 07:18:58 +0000 (15:18 +0800)]
crimson/osd: use single-pg peering ops

classic OSD started to use single-pg peering ops since ce05c172, and it
switched over to these ops since octopus. and it's assumed that
crimson-osd won't be GA until octopus+2 release, so it's not needed to
support pre-octopus releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: pass octopus to PeeringCtx's constructor
Kefu Chai [Fri, 13 Sep 2019 05:14:02 +0000 (13:14 +0800)]
crimson/osd: pass octopus to PeeringCtx's constructor

to accommodate the change of 77ecc327

classic OSD started to use single-pg peering ops since ce05c172, and it
switched over to these ops since octopus. and it's assumed that
crimson-osd won't be GA until octopus+2 release, so it's not needed to
support pre-octopus releases. hence "octopus" is passed to PeeringCtx's
constructor.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #30285 from IlsooByun/slow_op_report
Kefu Chai [Fri, 13 Sep 2019 07:53:30 +0000 (15:53 +0800)]
Merge pull request #30285 from IlsooByun/slow_op_report

mgr: do not reset reported if a new metric is not collected

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocephfs-shell: handle du's arguments elsewhere outside do_du() 28560/head
Rishabh Dave [Fri, 23 Aug 2019 12:50:04 +0000 (18:20 +0530)]
cephfs-shell: handle du's arguments elsewhere outside do_du()

Create new action class for do_du()'s args.paths and push the code that
handles arguments for do_du() in there. The intention is to keep only
the code that does some disk usage stuff in do_du().

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agocephfs-shell: reuse code
Rishabh Dave [Tue, 20 Aug 2019 12:18:07 +0000 (17:48 +0530)]
cephfs-shell: reuse code

Make path_to_bytes.__call__() call to_bytes() and get rid of the code
within path_to_bytes.__call__().

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agocephfs-shell: rewrite call to perror in do_du
Rishabh Dave [Mon, 19 Aug 2019 05:28:09 +0000 (10:58 +0530)]
cephfs-shell: rewrite call to perror in do_du

Adapt to new signature of cmd2.Cmd.perror.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agopybind/cephfs: define variable for hexcode used in stat()
Rishabh Dave [Wed, 7 Aug 2019 05:05:57 +0000 (10:35 +0530)]
pybind/cephfs: define variable for hexcode used in stat()

Instead of using the same hexcode at multiple locations, define it as a
variable and use the variable instead.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test cephfs-shell command at invocation
Rishabh Dave [Thu, 1 Aug 2019 20:22:22 +0000 (01:52 +0530)]
test_cephfs_shell: test cephfs-shell command at invocation

Verify that "cephfs-shell -c ceph.conf <cephfs-shell-cmd>" works as
expected.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agocephfs-shell: refactor do_du()
Rishabh Dave [Fri, 16 Aug 2019 17:37:32 +0000 (23:07 +0530)]
cephfs-shell: refactor do_du()

Improve code's readability and keep all paths as bytes.
Specifically, define a function within do do_du() to simplify the code
that handles recursion, rename args.dirs to args.paths, get rid of
dir_passed and use args.paths directly instead,  change the default
value of args.dirs to a list with one member that contains current
working directory, rewrite the docstring, and few more similar changes.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agocephfs-shell: option -r is not for reverse
Rishabh Dave [Fri, 16 Aug 2019 17:53:42 +0000 (23:23 +0530)]
cephfs-shell: option -r is not for reverse

Option "-r" for du command stands for recursive and not reverse.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agocephfs-shell: extend to_bytes()
Rishabh Dave [Thu, 15 Aug 2019 06:22:12 +0000 (11:52 +0530)]
cephfs-shell: extend to_bytes()

Make to_bytes() handle lists, None as well as other simple types as
well. In case of None, to_bytes() returns None instead of returning
b'None'.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test du with no args
Rishabh Dave [Tue, 20 Aug 2019 22:01:55 +0000 (03:31 +0530)]
test_cephfs_shell: test du with no args

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test du with multiple paths in args
Rishabh Dave [Tue, 20 Aug 2019 20:27:58 +0000 (01:57 +0530)]
test_cephfs_shell: test du with multiple paths in args

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test behaviour of "du -r"
Rishabh Dave [Tue, 20 Aug 2019 19:51:39 +0000 (01:21 +0530)]
test_cephfs_shell: test behaviour of "du -r"

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test du's output for softlinks
Rishabh Dave [Tue, 18 Jun 2019 13:45:02 +0000 (19:15 +0530)]
test_cephfs_shell: test du's output for softlinks

Test that CephFS shell command du prints output for softlinks as
expected

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agoqa/cephfs: add convenience method lstat()
Rishabh Dave [Mon, 26 Aug 2019 05:28:32 +0000 (10:58 +0530)]
qa/cephfs: add convenience method lstat()

It's more convenient lstat(path) than stat(path, follow_symlink=False).

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agoqa/cephfs: add option to make stat() unfollow symlinks
Rishabh Dave [Sat, 27 Jul 2019 09:32:37 +0000 (15:02 +0530)]
qa/cephfs: add option to make stat() unfollow symlinks

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test du's output for hardlinks
Rishabh Dave [Tue, 18 Jun 2019 13:24:19 +0000 (18:54 +0530)]
test_cephfs_shell: test du's output for hardlinks

Test that CephFS shell command du prints output for directories as
expected.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test du's output for directories
Rishabh Dave [Tue, 18 Jun 2019 10:49:53 +0000 (16:19 +0530)]
test_cephfs_shell: test du's output for directories

Test that CephFS shell command du prints output for directories as
expected.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: test du's output for regular files
Rishabh Dave [Tue, 18 Jun 2019 10:45:11 +0000 (16:15 +0530)]
test_cephfs_shell: test du's output for regular files

Test that CephFS shell command du prints an output for regular files
correctly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: add a method to get command output
Rishabh Dave [Wed, 17 Jul 2019 10:56:29 +0000 (16:26 +0530)]
test_cephfs_shell: add a method to get command output

Let run_cephfs_shell_cmd() return the object instead of output in string
format so that stderr, return value, etc. of the command executed
remain available. And add a new method that returns only output of the
CephFS shell command executed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: allow cmd as list too
Rishabh Dave [Wed, 24 Jul 2019 07:02:07 +0000 (12:32 +0530)]
test_cephfs_shell: allow cmd as list too

Convert cmd to string if it is passed as list to
run_cephfs_shell_cmd(). Allowing cmd to be list would be nice since
elsewhere, in tests, command to be executed must be a list of arguments
rather than string of arguments separated by spaces.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: rename and rewrite _cephfs_shell()
Rishabh Dave [Wed, 19 Jun 2019 08:09:19 +0000 (13:39 +0530)]
test_cephfs_shell: rename and rewrite _cephfs_shell()

Rename _cephfs_shell() to run_cephfs_shell_cmd() and add a default
client for run_cephfs_shell_cmd() since most of tests in this
testsuite will need only one client.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agotest_cephfs_shell: copy humanize() from cephfs-shell
Rishabh Dave [Tue, 18 Jun 2019 10:39:04 +0000 (16:09 +0530)]
test_cephfs_shell: copy humanize() from cephfs-shell

Copy the method named humanize from cephfs-shell so that output can be
compared. Unfortunately, importing this method isn't possible since the
dash in "cephfs-shell" leads to an syntax error.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agoqa: update json format from session listing 28702/head
Patrick Donnelly [Fri, 13 Sep 2019 00:42:17 +0000 (17:42 -0700)]
qa: update json format from session listing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agocephfs-shell: print disk usage for non-directory files too
Rishabh Dave [Mon, 17 Jun 2019 12:37:09 +0000 (18:07 +0530)]
cephfs-shell: print disk usage for non-directory files too

Currently when du crashes when it comes across a non-directory files.
Instead of doing that, print disk usage for regular files.

Fixes: http://tracker.ceph.com/issues/40371
Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agopybind/cephfs: add method that stats symlinks without following
Rishabh Dave [Sat, 13 Jul 2019 05:59:17 +0000 (11:29 +0530)]
pybind/cephfs: add method that stats symlinks without following

Add a new Python binding equivalent to lstat so that information about
the symlink itself can be also obtained, along with other type of files.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
6 years agoMerge pull request #30112 from liewegas/wip-autoscaler-on
Kefu Chai [Fri, 13 Sep 2019 04:15:05 +0000 (12:15 +0800)]
Merge pull request #30112 from liewegas/wip-autoscaler-on

common: default pg_autoscale_mode=on for new pools

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomds: recall caps from quiescent sessions
Patrick Donnelly [Mon, 26 Aug 2019 21:39:30 +0000 (14:39 -0700)]
mds: recall caps from quiescent sessions

This introduces two new config options [1,2] that dictate when a session
is considered quiescent by the MDS. (Options are documented fully in
options.cc.) When a session is quiescent, the MDS will preemptively
recall caps to reduce the outstanding capabilities which optimizes for
reducing work during failover.

[1] mds_session_cache_liveness_magnitude
[2] mds_session_cache_liveness_decay_rate

Fixes: https://tracker.ceph.com/issues/22446
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: use Session::dump method uniformly
Patrick Donnelly [Fri, 2 Aug 2019 20:34:56 +0000 (13:34 -0700)]
mds: use Session::dump method uniformly

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: use auto to deduce iterator type
Patrick Donnelly [Fri, 2 Aug 2019 20:32:55 +0000 (13:32 -0700)]
mds: use auto to deduce iterator type

This prevents accidental type mismatches.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: simplify method definition
Patrick Donnelly [Fri, 2 Aug 2019 20:32:15 +0000 (13:32 -0700)]
mds: simplify method definition

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: remove useless debug message
Patrick Donnelly [Fri, 2 Aug 2019 20:31:41 +0000 (13:31 -0700)]
mds: remove useless debug message

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: use const get_request_count
Patrick Donnelly [Fri, 2 Aug 2019 20:29:38 +0000 (13:29 -0700)]
mds: use const get_request_count

So it can be used from a const dump method. Make the elist mutable
instead.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: use session_info_t socket inst for addr
Patrick Donnelly [Fri, 2 Aug 2019 20:28:35 +0000 (13:28 -0700)]
mds: use session_info_t socket inst for addr

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: refactor session lookup
Patrick Donnelly [Fri, 2 Aug 2019 20:25:49 +0000 (13:25 -0700)]
mds: refactor session lookup

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: add explicit trim flag
Patrick Donnelly [Fri, 2 Aug 2019 20:22:51 +0000 (13:22 -0700)]
mds: add explicit trim flag

So that the intent of the caller is clear to trim due to the cache being
too full.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: alphabetize tracked config keys
Patrick Donnelly [Fri, 2 Aug 2019 20:17:25 +0000 (13:17 -0700)]
mds: alphabetize tracked config keys

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agocommon: provide method to get half-life
Patrick Donnelly [Fri, 2 Aug 2019 20:10:52 +0000 (13:10 -0700)]
common: provide method to get half-life

Print the half-life in Formatter output as it is human understandable
and the value used when configuring the counters.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agocommon: correct variable name
Patrick Donnelly [Fri, 2 Aug 2019 20:10:21 +0000 (13:10 -0700)]
common: correct variable name

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomsg: add func is_blackhole to reduce duplicated code. 30356/head
Jianpeng Ma [Fri, 13 Sep 2019 00:00:13 +0000 (08:00 +0800)]
msg: add func is_blackhole to reduce duplicated code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agoMerge PR #29574 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:24:14 +0000 (16:24 -0700)]
Merge PR #29574 into master

* refs/pull/29574/head:
mds: Reorganize class members in FSMapUser header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29883 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:23:26 +0000 (16:23 -0700)]
Merge PR #29883 into master

* refs/pull/29883/head:
mds: Reorganize class members in InoTable header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #30090 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:20:49 +0000 (16:20 -0700)]
Merge PR #30090 into master

* refs/pull/30090/head:
mds: Reorganize class members in Anchor header

Reviewed-by: Jos Collin <jcollin@redhat.com>