]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoscript/run_tox.sh: Don't overwrite the Ceph's bulid dir 29925/head
Sebastian Wagner [Tue, 27 Aug 2019 08:36:43 +0000 (10:36 +0200)]
script/run_tox.sh: Don't overwrite the Ceph's bulid dir

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #29549 from smanjara/wip-multisite-metadata
J. Eric Ivancich [Sun, 15 Sep 2019 16:54:24 +0000 (12:54 -0400)]
Merge pull request #29549 from smanjara/wip-multisite-metadata

rgw: Allow admin APIs that write metadata to be executed first on the mast…

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #30174 from mkogan1/wip-rgw-40700
J. Eric Ivancich [Sun, 15 Sep 2019 16:48:28 +0000 (12:48 -0400)]
Merge pull request #30174 from mkogan1/wip-rgw-40700

rgw: fix memory growth while deleting objects with

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #30379 from smithfarm/wip-41821
Nathan Cutler [Sun, 15 Sep 2019 07:32:12 +0000 (09:32 +0200)]
Merge pull request #30379 from smithfarm/wip-41821

tools/setup-virtualenv.sh: do not default to python2.7

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
5 years agoMerge pull request #30362 from uweigand/fix-unaligned-check
Kefu Chai [Sat, 14 Sep 2019 01:46:41 +0000 (09:46 +0800)]
Merge pull request #30362 from uweigand/fix-unaligned-check

cmake: Fix unaligned check on big-endian systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30374 from tchaikov/wip-cmake-brotli
Kefu Chai [Sat, 14 Sep 2019 01:43:04 +0000 (09:43 +0800)]
Merge pull request #30374 from tchaikov/wip-cmake-brotli

cmake: s/bortli_libs/brotli_libs/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #30366 from liu-chunmei/ceph_seastar_MonClient
Kefu Chai [Sat, 14 Sep 2019 01:05:48 +0000 (09:05 +0800)]
Merge pull request #30366 from liu-chunmei/ceph_seastar_MonClient

crimson/mon: use shared_future for waiting MauthReply

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30380 from rzarzynski/wip-crimson-cyanread-returns-bl
Kefu Chai [Sat, 14 Sep 2019 01:04:30 +0000 (09:04 +0800)]
Merge pull request #30380 from rzarzynski/wip-crimson-cyanread-returns-bl

crimson/os: Object::read() returns bufferlist instead of never used errcode

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30339 from rzarzynski/wip-crimson-cls-2
Kefu Chai [Sat, 14 Sep 2019 01:01:10 +0000 (09:01 +0800)]
Merge pull request #30339 from rzarzynski/wip-crimson-cls-2

crimson: serve basic RBD traffic coming from fio

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/mon:use shared_future for waiting MauthReply 30366/head
chunmei Liu [Thu, 12 Sep 2019 21:01:37 +0000 (14:01 -0700)]
crimson/mon:use shared_future for waiting MauthReply

when monitor-side close the connection, msgr call MonClient
ms_handle_reset cause reply.get_future be called twice then
assert happen in promise.get_future.
promise<T...>::get_future() noexcept {
    assert(!this->_future && this->_state && !this->_task);
    return future<T...>(this);
}
use shared_promise instead of promise to solve it.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
5 years agoMerge pull request #30363 from idryomov/wip-krbd-huge-osdmap
Ilya Dryomov [Fri, 13 Sep 2019 20:35:57 +0000 (22:35 +0200)]
Merge pull request #30363 from idryomov/wip-krbd-huge-osdmap

qa: add script to test how libceph handles huge osdmaps

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agocrimson/osd: add dummy support for CEPH_OSD_OP_WATCH. 30339/head
Radoslaw Zarzynski [Wed, 28 Aug 2019 14:40:30 +0000 (16:40 +0200)]
crimson/osd: add dummy support for CEPH_OSD_OP_WATCH.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: implement cls_get_client_features().
Radoslaw Zarzynski [Thu, 29 Aug 2019 14:32:44 +0000 (16:32 +0200)]
crimson/osd: implement cls_get_client_features().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: switch get_orig_source_inst() to get_message() in OpsExecuter.
Radoslaw Zarzynski [Thu, 29 Aug 2019 14:32:16 +0000 (16:32 +0200)]
crimson/osd: switch get_orig_source_inst() to get_message() in OpsExecuter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: implement cls_cxx_map_set_val().
Radoslaw Zarzynski [Wed, 28 Aug 2019 15:10:08 +0000 (17:10 +0200)]
crimson/osd: implement cls_cxx_map_set_val().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: implement cls_get_request_origin().
Radoslaw Zarzynski [Wed, 28 Aug 2019 14:39:59 +0000 (16:39 +0200)]
crimson/osd: implement cls_get_request_origin().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPGETVALS.
Radoslaw Zarzynski [Wed, 28 Aug 2019 11:41:50 +0000 (13:41 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPGETVALS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge PR #28560 into master
Patrick Donnelly [Fri, 13 Sep 2019 19:38:47 +0000 (12:38 -0700)]
Merge PR #28560 into master

* refs/pull/28560/head:
cephfs-shell: handle du's arguments elsewhere outside do_du()
cephfs-shell: reuse code
cephfs-shell: rewrite call to perror in do_du
pybind/cephfs: define variable for hexcode used in stat()
test_cephfs_shell: test cephfs-shell command at invocation
cephfs-shell: refactor do_du()
cephfs-shell: option -r is not for reverse
cephfs-shell: extend to_bytes()
test_cephfs_shell: test du with no args
test_cephfs_shell: test du with multiple paths in args
test_cephfs_shell: test behaviour of "du -r"
test_cephfs_shell: test du's output for softlinks
qa/cephfs: add convenience method lstat()
qa/cephfs: add option to make stat() unfollow symlinks
test_cephfs_shell: test du's output for hardlinks
test_cephfs_shell: test du's output for directories
test_cephfs_shell: test du's output for regular files
test_cephfs_shell: add a method to get command output
test_cephfs_shell: allow cmd as list too
test_cephfs_shell: rename and rewrite _cephfs_shell()
test_cephfs_shell: copy humanize() from cephfs-shell
cephfs-shell: print disk usage for non-directory files too
pybind/cephfs: add method that stats symlinks without following
cephfs-shell: Fix 'du' command error

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #29907 into master
Patrick Donnelly [Fri, 13 Sep 2019 19:04:44 +0000 (12:04 -0700)]
Merge PR #29907 into master

* refs/pull/29907/head:
doc: add a doc for vstart_runner.py

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #30185 into master
Patrick Donnelly [Fri, 13 Sep 2019 19:02:20 +0000 (12:02 -0700)]
Merge PR #30185 into master

* refs/pull/30185/head:
mgr/volumes: drop unnecessary size

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agocrimson/os: fix ENODATA handling in CyanStore::get_attrs().
Radoslaw Zarzynski [Tue, 27 Aug 2019 20:47:37 +0000 (22:47 +0200)]
crimson/os: fix ENODATA handling in CyanStore::get_attrs().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPGETKEYS.
Radoslaw Zarzynski [Tue, 27 Aug 2019 19:33:30 +0000 (21:33 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPGETKEYS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: PGBackend::write() alters object size now.
Radoslaw Zarzynski [Tue, 27 Aug 2019 19:32:16 +0000 (21:32 +0200)]
crimson/osd: PGBackend::write() alters object size now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: implement cls_cxx_write2().
Radoslaw Zarzynski [Tue, 27 Aug 2019 19:31:07 +0000 (21:31 +0200)]
crimson/osd: implement cls_cxx_write2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPSETVALS.
Radoslaw Zarzynski [Tue, 27 Aug 2019 15:23:37 +0000 (17:23 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPSETVALS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: add support for CEPH_OSD_OP_OMAPGETVALSBYKEYS.
Radoslaw Zarzynski [Tue, 27 Aug 2019 14:43:14 +0000 (16:43 +0200)]
crimson/osd: add support for CEPH_OSD_OP_OMAPGETVALSBYKEYS.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge pull request #29037 from zhangsw/rgw-reshard-numshards
Casey Bodley [Fri, 13 Sep 2019 18:04:13 +0000 (14:04 -0400)]
Merge pull request #29037 from zhangsw/rgw-reshard-numshards

rgw: shard number must be non-negative when resharding the bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agocrimson/osd: add support for CEPH_OSD_OP_CREATE.
Radoslaw Zarzynski [Tue, 27 Aug 2019 14:01:49 +0000 (16:01 +0200)]
crimson/osd: add support for CEPH_OSD_OP_CREATE.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agocrimson/osd: ignore CEPH_OSD_OP_TMAPUP like ceph-osd does for EC pools.
Radoslaw Zarzynski [Mon, 26 Aug 2019 18:29:47 +0000 (20:29 +0200)]
crimson/osd: ignore CEPH_OSD_OP_TMAPUP like ceph-osd does for EC pools.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoqa: add script to test how libceph handles huge osdmaps 30363/head
Ilya Dryomov [Wed, 11 Sep 2019 11:15:59 +0000 (13:15 +0200)]
qa: add script to test how libceph handles huge osdmaps

That code will also handle moderately-sized osdmaps when the memory is
fragmented.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
5 years agoMerge pull request #29623 from majianpeng/bluestore-combine-deferred_finisher-and...
Kefu Chai [Fri, 13 Sep 2019 16:35:44 +0000 (00:35 +0800)]
Merge pull request #29623 from majianpeng/bluestore-combine-deferred_finisher-and-finisher

os/bluestore: Merge deferred_finisher and finisher.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge pull request #30342 from ifed01/wip-ifed-fix-blues-alerts
Kefu Chai [Fri, 13 Sep 2019 16:34:20 +0000 (00:34 +0800)]
Merge pull request #30342 from ifed01/wip-ifed-fix-blues-alerts

mon/pgmap: fix bluestore alerts output.

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agocrimson/os: Object::read() returns bufferlist instead of never used errcode. 30380/head
Radoslaw Zarzynski [Mon, 9 Sep 2019 11:38:11 +0000 (13:38 +0200)]
crimson/os: Object::read() returns bufferlist instead of never used errcode.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge pull request #30372 from tchaikov/wip-crimson-single-pg-peering
Kefu Chai [Fri, 13 Sep 2019 15:54:22 +0000 (23:54 +0800)]
Merge pull request #30372 from tchaikov/wip-crimson-single-pg-peering

crimson/osd: use single-pg peering ops

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 years agoMerge PR #24794 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:36:32 +0000 (08:36 -0700)]
Merge PR #24794 into master

* refs/pull/24794/head:
mds: move BatchOp to separate translation unit
mds: use auto ref during iteration
mds: answering all pending getattr/lookups targeting the same inode in one go

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #28702 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:35:04 +0000 (08:35 -0700)]
Merge PR #28702 into master

* refs/pull/28702/head:
qa: update json format from session listing
mds: recall caps from quiescent sessions
mds: use Session::dump method uniformly
mds: use auto to deduce iterator type
mds: simplify method definition
mds: remove useless debug message
mds: use const get_request_count
mds: use session_info_t socket inst for addr
mds: refactor session lookup
mds: add explicit trim flag
mds: alphabetize tracked config keys
common: provide method to get half-life
common: correct variable name

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
5 years agoMerge PR #29005 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:33:48 +0000 (08:33 -0700)]
Merge PR #29005 into master

* refs/pull/29005/head:
client: nfs-ganesha with cephfs client, removing dir reports not empty

Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge PR #29594 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:32:10 +0000 (08:32 -0700)]
Merge PR #29594 into master

* refs/pull/29594/head:
mds: reject sessionless messages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #29752 into master
Patrick Donnelly [Fri, 13 Sep 2019 15:30:13 +0000 (08:30 -0700)]
Merge PR #29752 into master

* refs/pull/29752/head:
mds: make MDSIOContextBase delete itself when shutting down

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #30361 into master
Sage Weil [Fri, 13 Sep 2019 14:59:01 +0000 (09:59 -0500)]
Merge PR #30361 into master

* refs/pull/30361/head:
qa/tasks/mon_thrash: sync force requires some force flags

Reviewed-by: João Eduardo Luís <joao@suse.de>
5 years agoMerge pull request #30357 from zhangsw/rgw-cleanup-bilog-status
Casey Bodley [Fri, 13 Sep 2019 14:43:14 +0000 (10:43 -0400)]
Merge pull request #30357 from zhangsw/rgw-cleanup-bilog-status

rgw: add missing bilog status to help info.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agotools/setup-virtualenv.sh: do not default to python2.7 30379/head
Nathan Cutler [Fri, 13 Sep 2019 14:12:52 +0000 (16:12 +0200)]
tools/setup-virtualenv.sh: do not default to python2.7

This commit changes the script's behavior when run without a --python option.
Before, we were running virtualenv with "--python python2.7" in that case.

That is not a reasonable default in September 2019, given that Python 2 EOL is
coming on January 1, 2020. Also, it breaks setup-virtualenv.sh in py3-only
environments.

With this patch, the script will not pass any "--python" option to virtualenv
(i.e., it will let virtualenv do whatever it thinks is best) in this case.

Fixes: https://tracker.ceph.com/issues/41821
Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #29081 from wintamute/patch-1
Alfredo Deza [Fri, 13 Sep 2019 14:11:55 +0000 (10:11 -0400)]
Merge pull request #29081 from wintamute/patch-1

Documentation: Add missing ceph-volume lvm batch argument to ceph-volume.rst

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agoMerge PR #30352 into master
Sage Weil [Fri, 13 Sep 2019 13:22:42 +0000 (08:22 -0500)]
Merge PR #30352 into master

* refs/pull/30352/head:
mon: disable min pg per osd warning

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #30284 from Damenly/missing_cmath
Kefu Chai [Fri, 13 Sep 2019 13:06:12 +0000 (21:06 +0800)]
Merge pull request #30284 from Damenly/missing_cmath

tests: add missing header cmath to test/mon/test_mon_memory_target.cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agodoc: Add missing ceph-volume lvm batch argument to ceph-volume.rst 29081/head
Andreas Krebs [Wed, 17 Jul 2019 11:09:34 +0000 (13:09 +0200)]
doc: Add missing ceph-volume lvm batch argument to ceph-volume.rst

The optional argument '--osds-per-device' is missing. Since this is quite helpful when creating NVMe based OSDs, it should be documented.

Signed-off-by: Andreas Krebs <wintamute@gmail.com>
5 years agomgr/dashboard: Increase the global value of jasmine timeout (#30359)
Lenz Grimmer [Fri, 13 Sep 2019 12:18:09 +0000 (12:18 +0000)]
mgr/dashboard: Increase the global value of jasmine timeout (#30359)

mgr/dashboard: Increase the global value of jasmine timeout

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agocmake: s/bortli_libs/brotli_libs/ 30374/head
Kefu Chai [Fri, 13 Sep 2019 09:30:22 +0000 (17:30 +0800)]
cmake: s/bortli_libs/brotli_libs/

it's merely a typo.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #29761 from wjwithagen/wjw-fix-venv-python-version
Kefu Chai [Fri, 13 Sep 2019 08:59:27 +0000 (16:59 +0800)]
Merge pull request #29761 from wjwithagen/wjw-fix-venv-python-version

test/venv: do not hardwire to py2.7 for tox tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agomgr/dashboard: Fix calculation of PG Status percentage (#30343)
Lenz Grimmer [Fri, 13 Sep 2019 08:45:20 +0000 (08:45 +0000)]
mgr/dashboard: Fix calculation of PG Status percentage (#30343)

mgr/dashboard: Fix calculation of PG Status percentage

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agocrimson/osd: require_osd_release >= octopus to boot 30372/head
Kefu Chai [Fri, 13 Sep 2019 07:38:50 +0000 (15:38 +0800)]
crimson/osd: require_osd_release >= octopus to boot

we assume that crimson will GA after octopus+2, so no need to be
compatible with pre-octopus releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/osd: use single-pg peering ops
Kefu Chai [Fri, 13 Sep 2019 07:18:58 +0000 (15:18 +0800)]
crimson/osd: use single-pg peering ops

classic OSD started to use single-pg peering ops since ce05c172, and it
switched over to these ops since octopus. and it's assumed that
crimson-osd won't be GA until octopus+2 release, so it's not needed to
support pre-octopus releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/osd: pass octopus to PeeringCtx's constructor
Kefu Chai [Fri, 13 Sep 2019 05:14:02 +0000 (13:14 +0800)]
crimson/osd: pass octopus to PeeringCtx's constructor

to accommodate the change of 77ecc327

classic OSD started to use single-pg peering ops since ce05c172, and it
switched over to these ops since octopus. and it's assumed that
crimson-osd won't be GA until octopus+2 release, so it's not needed to
support pre-octopus releases. hence "octopus" is passed to PeeringCtx's
constructor.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30285 from IlsooByun/slow_op_report
Kefu Chai [Fri, 13 Sep 2019 07:53:30 +0000 (15:53 +0800)]
Merge pull request #30285 from IlsooByun/slow_op_report

mgr: do not reset reported if a new metric is not collected

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocephfs-shell: handle du's arguments elsewhere outside do_du() 28560/head
Rishabh Dave [Fri, 23 Aug 2019 12:50:04 +0000 (18:20 +0530)]
cephfs-shell: handle du's arguments elsewhere outside do_du()

Create new action class for do_du()'s args.paths and push the code that
handles arguments for do_du() in there. The intention is to keep only
the code that does some disk usage stuff in do_du().

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agocephfs-shell: reuse code
Rishabh Dave [Tue, 20 Aug 2019 12:18:07 +0000 (17:48 +0530)]
cephfs-shell: reuse code

Make path_to_bytes.__call__() call to_bytes() and get rid of the code
within path_to_bytes.__call__().

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agocephfs-shell: rewrite call to perror in do_du
Rishabh Dave [Mon, 19 Aug 2019 05:28:09 +0000 (10:58 +0530)]
cephfs-shell: rewrite call to perror in do_du

Adapt to new signature of cmd2.Cmd.perror.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agopybind/cephfs: define variable for hexcode used in stat()
Rishabh Dave [Wed, 7 Aug 2019 05:05:57 +0000 (10:35 +0530)]
pybind/cephfs: define variable for hexcode used in stat()

Instead of using the same hexcode at multiple locations, define it as a
variable and use the variable instead.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test cephfs-shell command at invocation
Rishabh Dave [Thu, 1 Aug 2019 20:22:22 +0000 (01:52 +0530)]
test_cephfs_shell: test cephfs-shell command at invocation

Verify that "cephfs-shell -c ceph.conf <cephfs-shell-cmd>" works as
expected.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agocephfs-shell: refactor do_du()
Rishabh Dave [Fri, 16 Aug 2019 17:37:32 +0000 (23:07 +0530)]
cephfs-shell: refactor do_du()

Improve code's readability and keep all paths as bytes.
Specifically, define a function within do do_du() to simplify the code
that handles recursion, rename args.dirs to args.paths, get rid of
dir_passed and use args.paths directly instead,  change the default
value of args.dirs to a list with one member that contains current
working directory, rewrite the docstring, and few more similar changes.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agocephfs-shell: option -r is not for reverse
Rishabh Dave [Fri, 16 Aug 2019 17:53:42 +0000 (23:23 +0530)]
cephfs-shell: option -r is not for reverse

Option "-r" for du command stands for recursive and not reverse.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agocephfs-shell: extend to_bytes()
Rishabh Dave [Thu, 15 Aug 2019 06:22:12 +0000 (11:52 +0530)]
cephfs-shell: extend to_bytes()

Make to_bytes() handle lists, None as well as other simple types as
well. In case of None, to_bytes() returns None instead of returning
b'None'.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test du with no args
Rishabh Dave [Tue, 20 Aug 2019 22:01:55 +0000 (03:31 +0530)]
test_cephfs_shell: test du with no args

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test du with multiple paths in args
Rishabh Dave [Tue, 20 Aug 2019 20:27:58 +0000 (01:57 +0530)]
test_cephfs_shell: test du with multiple paths in args

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test behaviour of "du -r"
Rishabh Dave [Tue, 20 Aug 2019 19:51:39 +0000 (01:21 +0530)]
test_cephfs_shell: test behaviour of "du -r"

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test du's output for softlinks
Rishabh Dave [Tue, 18 Jun 2019 13:45:02 +0000 (19:15 +0530)]
test_cephfs_shell: test du's output for softlinks

Test that CephFS shell command du prints output for softlinks as
expected

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agoqa/cephfs: add convenience method lstat()
Rishabh Dave [Mon, 26 Aug 2019 05:28:32 +0000 (10:58 +0530)]
qa/cephfs: add convenience method lstat()

It's more convenient lstat(path) than stat(path, follow_symlink=False).

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agoqa/cephfs: add option to make stat() unfollow symlinks
Rishabh Dave [Sat, 27 Jul 2019 09:32:37 +0000 (15:02 +0530)]
qa/cephfs: add option to make stat() unfollow symlinks

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test du's output for hardlinks
Rishabh Dave [Tue, 18 Jun 2019 13:24:19 +0000 (18:54 +0530)]
test_cephfs_shell: test du's output for hardlinks

Test that CephFS shell command du prints output for directories as
expected.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test du's output for directories
Rishabh Dave [Tue, 18 Jun 2019 10:49:53 +0000 (16:19 +0530)]
test_cephfs_shell: test du's output for directories

Test that CephFS shell command du prints output for directories as
expected.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: test du's output for regular files
Rishabh Dave [Tue, 18 Jun 2019 10:45:11 +0000 (16:15 +0530)]
test_cephfs_shell: test du's output for regular files

Test that CephFS shell command du prints an output for regular files
correctly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: add a method to get command output
Rishabh Dave [Wed, 17 Jul 2019 10:56:29 +0000 (16:26 +0530)]
test_cephfs_shell: add a method to get command output

Let run_cephfs_shell_cmd() return the object instead of output in string
format so that stderr, return value, etc. of the command executed
remain available. And add a new method that returns only output of the
CephFS shell command executed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: allow cmd as list too
Rishabh Dave [Wed, 24 Jul 2019 07:02:07 +0000 (12:32 +0530)]
test_cephfs_shell: allow cmd as list too

Convert cmd to string if it is passed as list to
run_cephfs_shell_cmd(). Allowing cmd to be list would be nice since
elsewhere, in tests, command to be executed must be a list of arguments
rather than string of arguments separated by spaces.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: rename and rewrite _cephfs_shell()
Rishabh Dave [Wed, 19 Jun 2019 08:09:19 +0000 (13:39 +0530)]
test_cephfs_shell: rename and rewrite _cephfs_shell()

Rename _cephfs_shell() to run_cephfs_shell_cmd() and add a default
client for run_cephfs_shell_cmd() since most of tests in this
testsuite will need only one client.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agotest_cephfs_shell: copy humanize() from cephfs-shell
Rishabh Dave [Tue, 18 Jun 2019 10:39:04 +0000 (16:09 +0530)]
test_cephfs_shell: copy humanize() from cephfs-shell

Copy the method named humanize from cephfs-shell so that output can be
compared. Unfortunately, importing this method isn't possible since the
dash in "cephfs-shell" leads to an syntax error.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agoqa: update json format from session listing 28702/head
Patrick Donnelly [Fri, 13 Sep 2019 00:42:17 +0000 (17:42 -0700)]
qa: update json format from session listing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agocephfs-shell: print disk usage for non-directory files too
Rishabh Dave [Mon, 17 Jun 2019 12:37:09 +0000 (18:07 +0530)]
cephfs-shell: print disk usage for non-directory files too

Currently when du crashes when it comes across a non-directory files.
Instead of doing that, print disk usage for regular files.

Fixes: http://tracker.ceph.com/issues/40371
Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agopybind/cephfs: add method that stats symlinks without following
Rishabh Dave [Sat, 13 Jul 2019 05:59:17 +0000 (11:29 +0530)]
pybind/cephfs: add method that stats symlinks without following

Add a new Python binding equivalent to lstat so that information about
the symlink itself can be also obtained, along with other type of files.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
5 years agoMerge pull request #30112 from liewegas/wip-autoscaler-on
Kefu Chai [Fri, 13 Sep 2019 04:15:05 +0000 (12:15 +0800)]
Merge pull request #30112 from liewegas/wip-autoscaler-on

common: default pg_autoscale_mode=on for new pools

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agomds: recall caps from quiescent sessions
Patrick Donnelly [Mon, 26 Aug 2019 21:39:30 +0000 (14:39 -0700)]
mds: recall caps from quiescent sessions

This introduces two new config options [1,2] that dictate when a session
is considered quiescent by the MDS. (Options are documented fully in
options.cc.) When a session is quiescent, the MDS will preemptively
recall caps to reduce the outstanding capabilities which optimizes for
reducing work during failover.

[1] mds_session_cache_liveness_magnitude
[2] mds_session_cache_liveness_decay_rate

Fixes: https://tracker.ceph.com/issues/22446
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: use Session::dump method uniformly
Patrick Donnelly [Fri, 2 Aug 2019 20:34:56 +0000 (13:34 -0700)]
mds: use Session::dump method uniformly

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: use auto to deduce iterator type
Patrick Donnelly [Fri, 2 Aug 2019 20:32:55 +0000 (13:32 -0700)]
mds: use auto to deduce iterator type

This prevents accidental type mismatches.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: simplify method definition
Patrick Donnelly [Fri, 2 Aug 2019 20:32:15 +0000 (13:32 -0700)]
mds: simplify method definition

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: remove useless debug message
Patrick Donnelly [Fri, 2 Aug 2019 20:31:41 +0000 (13:31 -0700)]
mds: remove useless debug message

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: use const get_request_count
Patrick Donnelly [Fri, 2 Aug 2019 20:29:38 +0000 (13:29 -0700)]
mds: use const get_request_count

So it can be used from a const dump method. Make the elist mutable
instead.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: use session_info_t socket inst for addr
Patrick Donnelly [Fri, 2 Aug 2019 20:28:35 +0000 (13:28 -0700)]
mds: use session_info_t socket inst for addr

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: refactor session lookup
Patrick Donnelly [Fri, 2 Aug 2019 20:25:49 +0000 (13:25 -0700)]
mds: refactor session lookup

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: add explicit trim flag
Patrick Donnelly [Fri, 2 Aug 2019 20:22:51 +0000 (13:22 -0700)]
mds: add explicit trim flag

So that the intent of the caller is clear to trim due to the cache being
too full.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agomds: alphabetize tracked config keys
Patrick Donnelly [Fri, 2 Aug 2019 20:17:25 +0000 (13:17 -0700)]
mds: alphabetize tracked config keys

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agocommon: provide method to get half-life
Patrick Donnelly [Fri, 2 Aug 2019 20:10:52 +0000 (13:10 -0700)]
common: provide method to get half-life

Print the half-life in Formatter output as it is human understandable
and the value used when configuring the counters.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agocommon: correct variable name
Patrick Donnelly [Fri, 2 Aug 2019 20:10:21 +0000 (13:10 -0700)]
common: correct variable name

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #29574 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:24:14 +0000 (16:24 -0700)]
Merge PR #29574 into master

* refs/pull/29574/head:
mds: Reorganize class members in FSMapUser header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #29883 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:23:26 +0000 (16:23 -0700)]
Merge PR #29883 into master

* refs/pull/29883/head:
mds: Reorganize class members in InoTable header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #30090 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:20:49 +0000 (16:20 -0700)]
Merge PR #30090 into master

* refs/pull/30090/head:
mds: Reorganize class members in Anchor header

Reviewed-by: Jos Collin <jcollin@redhat.com>
5 years agoMerge PR #30143 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:20:02 +0000 (16:20 -0700)]
Merge PR #30143 into master

* refs/pull/30143/head:
mds: Reorganize class members in LocalLock header

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #30205 into master
Patrick Donnelly [Thu, 12 Sep 2019 23:19:11 +0000 (16:19 -0700)]
Merge PR #30205 into master

* refs/pull/30205/head:
mds: Reorganize class members in LogEvent header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
5 years agoMerge PR #30191 into master
Sage Weil [Thu, 12 Sep 2019 16:45:47 +0000 (11:45 -0500)]
Merge PR #30191 into master

* refs/pull/30191/head:
cls/hello: disable write returning data clobbering demo
messages/MOSDOpReply: assert no write result payloads
osd/PrimaryLogPG: assert write result == 0
osd/PrimaryLogPG: pass TMAPGET errors up
osd/PrimaryLogPG: set result=0 for successful writes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #29792 from sebastian-philipp/orchestrator-api-replace
Sebastian Wagner [Thu, 12 Sep 2019 15:46:27 +0000 (17:46 +0200)]
Merge pull request #29792 from sebastian-philipp/orchestrator-api-replace

mgr/orchestrator: Document OSD replacement

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
5 years agocommon: default pg_autoscale_mode=on for new pools 30112/head
Sage Weil [Tue, 3 Sep 2019 15:54:47 +0000 (10:54 -0500)]
common: default pg_autoscale_mode=on for new pools

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agocls/hello: disable write returning data clobbering demo 30191/head
Sage Weil [Tue, 10 Sep 2019 12:08:56 +0000 (07:08 -0500)]
cls/hello: disable write returning data clobbering demo

Signed-off-by: Sage Weil <sage@redhat.com>