]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agorgw: rgw_log_entry always initialize total_time
Kefu Chai [Sun, 25 Aug 2019 06:54:30 +0000 (14:54 +0800)]
rgw: rgw_log_entry always initialize total_time

zero-initialize rgw_log_entry::total_time

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/ServiceMap: use plain gid
Kefu Chai [Sun, 25 Aug 2019 06:53:20 +0000 (14:53 +0800)]
mgr/ServiceMap: use plain gid

* partially revert 5c25a018. which is not backward compatible.
* change `ServiceMap::get_daemon()` so it returns a
  `pair<Daemon*,bool>`.

git is an `optional<uint64_t>`, so we cannot dump it without checking.

Fixes: https://tracker.ceph.com/issues/41424
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: init pg_pool_t::pg_autoscale_mode in ctor
Kefu Chai [Sun, 25 Aug 2019 04:08:39 +0000 (12:08 +0800)]
osd: init pg_pool_t::pg_autoscale_mode in ctor

so its value is determined

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: use scoped enum for pg_pool_t::pg_autoscale_mode
Kefu Chai [Sun, 25 Aug 2019 04:06:51 +0000 (12:06 +0800)]
osd: use scoped enum for pg_pool_t::pg_autoscale_mode

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: use in-class initialization in pg_pool_t
Kefu Chai [Sun, 25 Aug 2019 03:49:50 +0000 (11:49 +0800)]
osd: use in-class initialization in pg_pool_t

less repeat this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: implement copy ctor and assignment for bluestore_blob_use_tracker_t
Kefu Chai [Sun, 25 Aug 2019 03:48:42 +0000 (11:48 +0800)]
os/bluestore: implement copy ctor and assignment for bluestore_blob_use_tracker_t

to address the `check-generated.sh` test failure.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: RGWObjManifest::obj_iterator::seek always update locations
Kefu Chai [Sat, 24 Aug 2019 15:41:57 +0000 (23:41 +0800)]
rgw: RGWObjManifest::obj_iterator::seek always update locations

we should update explicit loc and location even if
`ofs >= manifest->obj_size`. there is chance that we are updating an end
iterator whose ofs is equal to obj_size. before being updated, the end
iterator points to an implicit location, while after being updated, the
manifest could be using an explicit location, so we should update the
end iterator as well.

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: RGWObjManifest::obj_iterator: init member variables in-class
Kefu Chai [Sun, 25 Aug 2019 01:07:48 +0000 (09:07 +0800)]
rgw: RGWObjManifest::obj_iterator: init member variables in-class

* remove the init() member function which is solely used by the
  constructors.
* use constructor delegation to minimize the repeating

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: update_iterator() in RGWObjManifest::set_explicit()
Kefu Chai [Sat, 24 Aug 2019 06:12:06 +0000 (14:12 +0800)]
rgw: update_iterator() in RGWObjManifest::set_explicit()

we should update the end iterator when updating RGWObjManifest with its
parts

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29913 from tchaikov/wip-debian/control
Kefu Chai [Tue, 27 Aug 2019 02:28:00 +0000 (10:28 +0800)]
Merge pull request #29913 from tchaikov/wip-debian/control

debian/control: fix Build-Depends

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agodebian/control: fix Build-Depends 29913/head
Kefu Chai [Tue, 27 Aug 2019 02:05:15 +0000 (10:05 +0800)]
debian/control: fix Build-Depends

it's a regression introduced by 5d6d770e

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #29867 into master
Patrick Donnelly [Mon, 26 Aug 2019 21:09:39 +0000 (14:09 -0700)]
Merge PR #29867 into master

* refs/pull/29867/head:
doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29822 from vumrao/wip-vumrao-41399
Neha Ojha [Mon, 26 Aug 2019 19:57:58 +0000 (12:57 -0700)]
Merge pull request #29822 from vumrao/wip-vumrao-41399

os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoos/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1 29822/head
Vikhyat Umrao [Thu, 22 Aug 2019 16:45:40 +0000 (09:45 -0700)]
os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
6 years agoMerge pull request #29903 from tchaikov/wip-pip-download
Kefu Chai [Mon, 26 Aug 2019 16:44:28 +0000 (00:44 +0800)]
Merge pull request #29903 from tchaikov/wip-pip-download

install-deps.sh: download wheel using 'pip wheel'

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoinstall-deps.sh: download wheel using 'pip wheel' 29903/head
Kefu Chai [Mon, 26 Aug 2019 03:36:50 +0000 (11:36 +0800)]
install-deps.sh: download wheel using 'pip wheel'

otherwise we will fail to install the build dependencies of
`lazy-object-proxy` from the wheelhouse. as `lazy-object-proxy` does not
add `setuptools_scm` in its `setup.py`, instead it lists
`setuptools_scm` in `setup.cfg` and `pyproject.toml` as a `build-system`
requires. but unfortunately, `pip download` only downloads the
install/run-time dependencies at this moment. and `lazy-object-proxy`
does not offer binary package for at least python2.7.

ideally, `pip download` should collects its dependencies like

Collecting setuptools_scm>=3.3.1 (from lazy-object-proxy->astroid<3,>=2.2.0->pylint->-r requirements-lint.txt (line 1))

so we need to use `pip wheel` do download build-time dependencies

see also https://github.com/pypa/pip/issues/6222

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29854 from tchaikov/wip-sloppy-cond
Kefu Chai [Mon, 26 Aug 2019 14:38:50 +0000 (22:38 +0800)]
Merge pull request #29854 from tchaikov/wip-sloppy-cond

common/condition_variable_debug: do not assert() if sloppy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #29866 from joscollin/wip-fix-stupid
Kefu Chai [Mon, 26 Aug 2019 14:36:41 +0000 (22:36 +0800)]
Merge pull request #29866 from joscollin/wip-fix-stupid

os/bluestore: silence StupidAllocator reorder warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29890 from rhcs-dashboard/wip-signed-master
Alfredo Deza [Mon, 26 Aug 2019 12:20:57 +0000 (08:20 -0400)]
Merge pull request #29890 from rhcs-dashboard/wip-signed-master

github: display phrase for signed-off check

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #29831 from tchaikov/wip-python-int
Kefu Chai [Mon, 26 Aug 2019 12:06:21 +0000 (20:06 +0800)]
Merge pull request #29831 from tchaikov/wip-python-int

mgr/BaseMgrModule: use PyInt_Check() to compatible with py2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agomgr/dashboard: Edit Pool Placement Group E2E Test (#29779)
Lenz Grimmer [Mon, 26 Aug 2019 11:36:53 +0000 (11:36 +0000)]
mgr/dashboard: Edit Pool Placement Group E2E Test (#29779)

mgr/dashboard: Edit Pool Placement Group E2E Test

Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29707 from rafaelquint/dashdoc
Lenz Grimmer [Mon, 26 Aug 2019 11:33:41 +0000 (11:33 +0000)]
Merge pull request #29707 from rafaelquint/dashdoc

mgr/dashboard: Extend "Writing End-to-End Tests" section (describe vs it)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agogithub: display phrase for signed-off check 29890/head
Ernesto Puerta [Mon, 26 Aug 2019 09:16:25 +0000 (11:16 +0200)]
github: display phrase for signed-off check

https://github.com/ceph/ceph-build/pull/1379 will introduce a
trigger-phrase for pr-commit check (DCO/signed-off).

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #29840 from tchaikov/wip-install-deps.sh-without-python-saml
Kefu Chai [Mon, 26 Aug 2019 08:40:02 +0000 (16:40 +0800)]
Merge pull request #29840 from tchaikov/wip-install-deps.sh-without-python-saml

install-deps.sh,deb,rpm: move python-saml deps into debian/control an…

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #29808 from dillaman/wip-41356
Mykola Golub [Sun, 25 Aug 2019 17:23:51 +0000 (20:23 +0300)]
Merge pull request #29808 from dillaman/wip-41356

rbd-mirror: ignore errors relating to parsing the cluster config file

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agodoc: default values for mon_health_to_clog_* were flipped 29867/head
James McClune [Sun, 25 Aug 2019 03:24:31 +0000 (23:24 -0400)]
doc: default values for mon_health_to_clog_* were flipped

On a freshly installed nautilus cluster (i.e. 14.2.2), the default values are:

mon_health_to_clog_interval = 3600
mon_health_to_clog_tick_interval = 60.000000

Fixes: https://tracker.ceph.com/issues/41403
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
6 years agoMerge pull request #29651 from rzarzynski/wip-crimson-cls
Kefu Chai [Sun, 25 Aug 2019 07:25:41 +0000 (15:25 +0800)]
Merge pull request #29651 from rzarzynski/wip-crimson-cls

crimson, osd: add support for Ceph Classes, part 1

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29595 from bk201/wip-41151
Kefu Chai [Sun, 25 Aug 2019 07:13:33 +0000 (15:13 +0800)]
Merge pull request #29595 from bk201/wip-41151

mgr/test_orchestrator: Allow initializing dummy data

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #29787 from NancySu05/fix-osd
Kefu Chai [Sun, 25 Aug 2019 07:11:51 +0000 (15:11 +0800)]
Merge pull request #29787 from NancySu05/fix-osd

osd: add log information to record the cause of do_osd_ops failure

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: silence StupidAllocator reorder warning 29866/head
Jos Collin [Sun, 25 Aug 2019 03:04:59 +0000 (08:34 +0530)]
os/bluestore: silence StupidAllocator reorder warning

warning: ‘StupidAllocator::last_alloc’ will be initialized after [-Wreorder]

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agoMerge PR #29542 into master
Patrick Donnelly [Sun, 25 Aug 2019 04:35:14 +0000 (21:35 -0700)]
Merge PR #29542 into master

* refs/pull/29542/head:
mds: trim cache on regular schedule

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #29760 into master
Patrick Donnelly [Sun, 25 Aug 2019 04:31:58 +0000 (21:31 -0700)]
Merge PR #29760 into master

* refs/pull/29760/head:
mgr/volumes: cleanup FS subvolume or subvolume group path
mgr/volumes: give useful error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29843 into master
Patrick Donnelly [Sun, 25 Aug 2019 04:31:27 +0000 (21:31 -0700)]
Merge PR #29843 into master

* refs/pull/29843/head:
qa/tasks: drop object inherit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agomds: trim cache on regular schedule 29542/head
Patrick Donnelly [Wed, 7 Aug 2019 17:35:02 +0000 (10:35 -0700)]
mds: trim cache on regular schedule

Do this outside the standard tick interval as it needs to be driven more
frequently to keep up with client workloads that generate a lot of
capabilities.

Fixes: https://tracker.ceph.com/issues/41141
Fixes: https://tracker.ceph.com/issues/41140
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29754 from xiexingguo/wip-inc-recovery-3
Xie Xingguo [Sat, 24 Aug 2019 05:51:21 +0000 (13:51 +0800)]
Merge pull request #29754 from xiexingguo/wip-inc-recovery-3

osd: misc inc-recovery compat fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #28344 from iotcg/rdma
Kefu Chai [Sat, 24 Aug 2019 03:37:03 +0000 (11:37 +0800)]
Merge pull request #28344 from iotcg/rdma

check rdma configuration and fix some logic problem

Reviewed-by: Roman Penyaev <rpenyaev@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: dissect pg listing implementation from OpsExecuter. 29651/head
Radoslaw Zarzynski [Tue, 13 Aug 2019 12:59:51 +0000 (14:59 +0200)]
crimson/osd: dissect pg listing implementation from OpsExecuter.

Additionally improve const-correctness for PGBackend.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: do_pgnls_common isn't the member of OpsExecuter anymore.
Radoslaw Zarzynski [Mon, 12 Aug 2019 18:30:17 +0000 (20:30 +0200)]
crimson/osd: do_pgnls_common isn't the member of OpsExecuter anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/os, crimson/osd: make xattr retrieval const.
Radoslaw Zarzynski [Mon, 12 Aug 2019 18:28:37 +0000 (20:28 +0200)]
crimson/os, crimson/osd: make xattr retrieval const.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/os, crimson/osd: make list_objects() const.
Radoslaw Zarzynski [Mon, 12 Aug 2019 18:11:20 +0000 (20:11 +0200)]
crimson/os, crimson/osd: make list_objects() const.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: adopt PGLSPlainFilter.
Radoslaw Zarzynski [Mon, 12 Aug 2019 16:15:53 +0000 (18:15 +0200)]
crimson/osd: adopt PGLSPlainFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoosd: share PGLSPlainFilter with crimson.
Radoslaw Zarzynski [Mon, 12 Aug 2019 14:41:19 +0000 (16:41 +0200)]
osd: share PGLSPlainFilter with crimson.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoobjclass, osd: dissect PGLSFilter into the osd_types.h header.
Radoslaw Zarzynski [Mon, 12 Aug 2019 13:32:38 +0000 (15:32 +0200)]
objclass, osd: dissect PGLSFilter into the osd_types.h header.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement PGLSFilter.
Radoslaw Zarzynski [Wed, 7 Aug 2019 17:59:27 +0000 (19:59 +0200)]
crimson/osd: implement PGLSFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: OpsExecuter differentiates read and modifying operations.
Radoslaw Zarzynski [Tue, 6 Aug 2019 11:56:51 +0000 (13:56 +0200)]
crimson/osd: OpsExecuter differentiates read and modifying operations.

This is necessary to pass ClsHello.BadMethods.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement CEPH_OSD_OP_GETXATTR and cls_cxx_getxattr().
Radoslaw Zarzynski [Mon, 5 Aug 2019 15:24:24 +0000 (17:24 +0200)]
crimson/osd: implement CEPH_OSD_OP_GETXATTR and cls_cxx_getxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: differentiate ENOENT during do_osd_call().
Radoslaw Zarzynski [Mon, 5 Aug 2019 11:58:06 +0000 (13:58 +0200)]
crimson/osd: differentiate ENOENT during do_osd_call().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_read2().
Radoslaw Zarzynski [Mon, 5 Aug 2019 11:53:13 +0000 (13:53 +0200)]
crimson/osd: implement cls_cxx_read2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_setxattr().
Radoslaw Zarzynski [Mon, 5 Aug 2019 06:48:55 +0000 (08:48 +0200)]
crimson/osd: implement cls_cxx_setxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement CEPH_OSD_OP_SETXATTR.
Radoslaw Zarzynski [Mon, 5 Aug 2019 06:48:19 +0000 (08:48 +0200)]
crimson/osd: implement CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/os: port get_max_attr_name_length() from MemStore.
Radoslaw Zarzynski [Mon, 5 Aug 2019 06:47:08 +0000 (08:47 +0200)]
crimson/os: port get_max_attr_name_length() from MemStore.

This method is an enabler for CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoobjclass, osd: drop unused variable.
Radoslaw Zarzynski [Mon, 5 Aug 2019 05:35:01 +0000 (07:35 +0200)]
objclass, osd: drop unused variable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_write_full().
Radoslaw Zarzynski [Sat, 3 Aug 2019 07:14:00 +0000 (09:14 +0200)]
crimson/osd: implement cls_cxx_write_full().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: introduce OpsExecuter to uniform calling CEPH_OSD_OPS.
Radoslaw Zarzynski [Fri, 2 Aug 2019 12:27:52 +0000 (14:27 +0200)]
crimson/osd: introduce OpsExecuter to uniform calling CEPH_OSD_OPS.

OSD has two entry points for executing CEPH_OSD_OP_*:
  1. the MOSDOp message handler,
  2. the Object Class API (cls_* and cls_cxx_* functions).

We definitely want to address these two users without code
duplication. However, exposing the entire PG to Obj Class
would break encapsulation. Moreover, there is difference
in life times between PG and sequence-of-operations-from-
MOSDOp.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_stat().
Radoslaw Zarzynski [Fri, 2 Aug 2019 11:46:20 +0000 (13:46 +0200)]
crimson/osd: implement cls_cxx_stat().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement CEPH_OSD_OP_STAT.
Radoslaw Zarzynski [Wed, 31 Jul 2019 18:08:59 +0000 (20:08 +0200)]
crimson/osd: implement CEPH_OSD_OP_STAT.

This commit is enabler of cls_cxx_stat(), and thus the RecordHello
and ClsHello unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement CEPH_OSD_OP_CALL.
Radoslaw Zarzynski [Wed, 31 Jul 2019 17:52:45 +0000 (19:52 +0200)]
crimson/osd: implement CEPH_OSD_OP_CALL.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: rename crimson_error and introduce namespace.
Radoslaw Zarzynski [Mon, 19 Aug 2019 13:34:51 +0000 (15:34 +0200)]
crimson/osd: rename crimson_error and introduce namespace.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: exceptions derive from std::system_error now.
Radoslaw Zarzynski [Wed, 31 Jul 2019 17:38:30 +0000 (19:38 +0200)]
crimson/osd: exceptions derive from std::system_error now.

This change is be useful especially for CEPH_OSD_OP_CALL
which will be brought by further commits. The issue here
is that the Ceph Classes will be based on existing iface
handling errors with usual, int-based ret codes. These
codes needs to be glued with the existing handling  in
pg::do_osd_ops().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson: put extra debugs into do_osd_op().
Radoslaw Zarzynski [Tue, 30 Jul 2019 13:39:04 +0000 (15:39 +0200)]
crimson: put extra debugs into do_osd_op().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoosd: switch ClassHandler::get_method() to std::string.
Radoslaw Zarzynski [Tue, 30 Jul 2019 13:37:54 +0000 (15:37 +0200)]
osd: switch ClassHandler::get_method() to std::string.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: mock OSD-specific parts of the objclass API.
Radoslaw Zarzynski [Thu, 25 Jul 2019 17:06:50 +0000 (19:06 +0200)]
crimson/osd: mock OSD-specific parts of the objclass API.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoobjclass, crimson: initialize the ClassHandler.
Radoslaw Zarzynski [Thu, 25 Jul 2019 11:51:44 +0000 (13:51 +0200)]
objclass, crimson: initialize the ClassHandler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoobjclass, osd: dissect the OSD-specific parts from class_api.cc.
Radoslaw Zarzynski [Fri, 26 Jul 2019 17:36:35 +0000 (19:36 +0200)]
objclass, osd: dissect the OSD-specific parts from class_api.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoosd: improve formatting and refactor ClassHandler.h a bit.
Radoslaw Zarzynski [Thu, 25 Jul 2019 09:28:26 +0000 (11:28 +0200)]
osd: improve formatting and refactor ClassHandler.h a bit.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge PR #28855 into master
Patrick Donnelly [Fri, 23 Aug 2019 23:16:16 +0000 (16:16 -0700)]
Merge PR #28855 into master

* refs/pull/28855/head:
doc: document scrub summary in ceph status output
test: extend scrub control test to validate mds task status
mds: send scrub state changes to cluster log.
mds: periodically sent mds scrub status to ceph manager
mgr, mon: allow normal ceph services to register with manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29167 into master
Patrick Donnelly [Fri, 23 Aug 2019 23:10:40 +0000 (16:10 -0700)]
Merge PR #29167 into master

* refs/pull/29167/head:
client: return -eio when sync file which unsafe reqs has been dropped

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29572 into master
Patrick Donnelly [Fri, 23 Aug 2019 23:06:51 +0000 (16:06 -0700)]
Merge PR #29572 into master

* refs/pull/29572/head:
mds: Reorganize class members in FSMap header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoinstall-deps.sh,deb,rpm: move python-saml deps into debian/control and ceph.spec.in 29840/head
Kefu Chai [Fri, 23 Aug 2019 08:42:22 +0000 (16:42 +0800)]
install-deps.sh,deb,rpm: move python-saml deps into debian/control and ceph.spec.in

these dependencies are only used for building python-saml which is in
turn used for the SAML support. this feature is tested using
`test_sso.py` while performing dashboard tests. we do not package or
ship python-saml along with other Ceph packages. so let's move these
dependencies to the "make check" sections in ceph.spec.in and
debian/control for simplifying install-deps.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29796 from trociny/wip-journal-player-handle_cache_rebalanced2
Jason Dillaman [Fri, 23 Aug 2019 17:40:15 +0000 (13:40 -0400)]
Merge pull request #29796 from trociny/wip-journal-player-handle_cache_rebalanced2

journal: fix race between player shut down and cache rebalance

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #29775 from trociny/wip-41229
Jason Dillaman [Fri, 23 Aug 2019 17:39:37 +0000 (13:39 -0400)]
Merge pull request #29775 from trociny/wip-41229

librbd: always try to acquire exclusive lock when removing image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #29459 from zy751713126/config_set
Jason Dillaman [Fri, 23 Aug 2019 17:39:07 +0000 (13:39 -0400)]
Merge pull request #29459 from zy751713126/config_set

pybind/rbd: add config_set/get/remove api in rbd.pyx

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge PR #29715 into master
Patrick Donnelly [Fri, 23 Aug 2019 17:09:17 +0000 (10:09 -0700)]
Merge PR #29715 into master

* refs/pull/29715/head:
qa: fix broken ceph.restart marking of OSDs down
qa: add debugging failed osd-release setting

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #29821 into master
Patrick Donnelly [Fri, 23 Aug 2019 17:00:52 +0000 (10:00 -0700)]
Merge PR #29821 into master

* refs/pull/29821/head:
qa: stop DaemonWatchdog for each cluster in daemon roles

Reviewed-by: Jos Collin <jcollin@redhat.com>
6 years agocommon/condition_variable_debug: do not assert() if sloppy 29854/head
Kefu Chai [Fri, 23 Aug 2019 16:28:45 +0000 (00:28 +0800)]
common/condition_variable_debug: do not assert() if sloppy

replicate the behavior of `Cond::SloppySignal()` to avoid crash like

 ceph version Development (no_version) octopus (dev)
  1: <ceph::__ceph_assert_fail(char const*, char const, int, char const)+0x1e0> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  2: <ceph::__ceph_assert_fail(ceph::assert_data const&)+0x2f> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  3: <ceph::condition_variable_debug::notify_all(bool)+0x59> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  4: <FileJournal::check_for_full(unsigned long, long, long)+0x4e2> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  5: <FileJournal::prepare_single_write(FileJournal::write_item&, ceph::buffer::v14_2_0::list&, long&, unsigned long&, long&)+0x71> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  6: <FileJournal::prepare_multi_write(ceph::buffer::v14_2_0::list&, unsigned long&, ceph::buffer::v14_2_0::list&)+0x19d> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  7: <FileJournal::write_thread_entry(void)+0x561> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  8: <FileJournal::Writer::entry(void)+0x19> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  9: <Thread::entry_wrapper(void)+0x84> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd
  10: <Thread::_entry_func(void*)+0x15> at /home/jenkins/workspace/ceph-master36/build/bin/ceph-osd

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #29575 into master
Sage Weil [Fri, 23 Aug 2019 16:26:28 +0000 (11:26 -0500)]
Merge PR #29575 into master

* refs/pull/29575/head:
objclass, osd: improve const-correctness of PGLSFilter.
common: add bl::contents_equal() override for void* + size_t.
osd: refactor manufacturing of PGLSFilter.
osd: don't carry PGLSFilter between multiple ops in MOSDOp.

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #28727 into master
Sage Weil [Fri, 23 Aug 2019 16:25:28 +0000 (11:25 -0500)]
Merge PR #28727 into master

* refs/pull/28727/head:
test/crimson: resolve name collision
test: switch to ldout; let users specify mon debug level
test: add new ElectionLogic unit test framework
elector: const-ify a bunch of functions
elector: swap order of parameters in ElectionLogic::receive_propose
elector: Update Elector and ElectionLogic function documentation
elector: persist the epoch in bump_epoch()
elector: make some more ElectionLogic members private
elector: fix privacy and restore dout in Elector
elector: don't clear peer_info in bump_epoch()
elector: split ElectionLogic into its own compilation unit
elector: move all the elector callouts into the Elector
elector: make ElectionLogic private to Elector; undo most public shenanigans
elector: create declare_standlone_victory in Elector/Logic for Monitor
elector: make ElectionLogic::declare_victory private
elector: route _bump_epoch through the interface-to-be
elector: rename handle_propose_logic -> receive_propose
elector: hoist handle_victory into ElectionLogic
elector: hoist handle_ack into ElectionLogic
elector: hoist victory into ElectionLogic
elector: hoist expire into ElectionLogic
elector: hoist start into ElectionLogic
elector: hoist participating into ElectionLogic
elector: hoist init into ElectionLogic
elector: hoist defer into ElectionLogic
elector: split handle_propose in two and hoist into ElectionLogic
elector: hoist bump_epoch into ElectionLogic
elector: store accessors for ElectionLogic
elector: hoist Elector data bits out into a new ElectionLogic class
mon: Rearrange Paxos::dispatch to be a little cleaner

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #15183 into master
Sage Weil [Fri, 23 Aug 2019 15:46:33 +0000 (10:46 -0500)]
Merge PR #15183 into master

* refs/pull/15183/head:
kv/rocksdb: support rmrange unconditionally
cls/rgw: rgw_bi_log_trim() uses cls_cxx_map_remove_range()
cls/log: cls_log_trim() uses cls_cxx_map_remove_range()
test/cls: add cls_log.trim_by_marker test
test/cls: test_cls_log doesn't allocate ObjectOperations
test/cls: test_cls_log uses fixture for temporary pool
test/cls: add cls_rgw.bi_log_trim test
cls/rgw: expose cls_rgw_bilog_list/trim() for single shard
test/cls: test_cls_rgw uses cls_rgw_obj_key
test/cls: test_cls_rgw doesn't allocate ObjectOperations
test/cls: test_cls_rgw uses fixture for temporary pool
objclass: add cls_cxx_map_remove_range()
librados: add rados_write_op_omap_rm_range2()
osdc: add Objecter omap_rm_range()
osd: add CEPH_OSD_OP_OMAPRMKEYRANGE to do_osd_ops()
osd: add omap_rmkeyrange() to PGTransaction
os: add bufferlist overload for omap_rmkeyrange()
tracing: add do_osd_op_pre_omaprmkeyrange
rados: add CEPH_OSD_OP_OMAPRMKEYRANGE

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/BaseMgrModule: use PyInt_Check() to compatible with py2 29831/head
Kefu Chai [Fri, 23 Aug 2019 02:36:44 +0000 (10:36 +0800)]
mgr/BaseMgrModule: use PyInt_Check() to compatible with py2

in python2, we have both `PyLongObject` and `PyIntObject` for presenting
`long` and `int` types. and in python3, `PyIntObject` is dropped, `int`
is represented using `PyLongObject`. and the related functions like
`PyInt_Check()` and `PyInt_AsLong()` are removed along with
`PyIntObject`.

so we should use different set of functions for converting an `int` python
object to native type in C++. since we don't use `long` for "count" when
calling `MgrModule.set_health_checks()`, in this change, we just define
a macro to unify the difference between python2 and python3.

the same applies to "max_count" parameter of `MgrModule.add_osd_perf_query()`
call.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29778 from cbodley/wip-41212
Casey Bodley [Fri, 23 Aug 2019 14:24:41 +0000 (10:24 -0400)]
Merge pull request #29778 from cbodley/wip-41212

vstart: move [client.rgw] config into [client]

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
6 years agomgr/dashboard: User Management E2E tests (#29641)
Lenz Grimmer [Fri, 23 Aug 2019 14:00:16 +0000 (14:00 +0000)]
mgr/dashboard: User Management E2E tests (#29641)

mgr/dashboard: User Management E2E tests

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agoqa/tasks: drop object inherit 29843/head
Jos Collin [Fri, 23 Aug 2019 09:59:27 +0000 (15:29 +0530)]
qa/tasks: drop object inherit

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agomgr/dashboard: run-backend-api-tests.sh CI improvements (#29504)
Lenz Grimmer [Fri, 23 Aug 2019 09:11:39 +0000 (09:11 +0000)]
mgr/dashboard: run-backend-api-tests.sh CI improvements (#29504)

mgr/dashboard: run-backend-api-tests.sh CI improvements

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #29590 from Aran85/fix_proc_replica_log
Kefu Chai [Fri, 23 Aug 2019 06:59:02 +0000 (14:59 +0800)]
Merge pull request #29590 from Aran85/fix_proc_replica_log

osd: merge replica log on primary need according to replica log's crt

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agomsg/async/Stack: rename variable to improve readability 28344/head
Changcheng Liu [Wed, 31 Jul 2019 01:33:51 +0000 (09:33 +0800)]
msg/async/Stack: rename variable to improve readability

1. rename var i to be worker_id when creating Worker
"i" is assigned to be Worker::id, it means worker's id

2. rename EventCenter::idx to EventCenter::center_id
"idx" is EventCenter's index in global_centers obj.
rename it to be center_id.

3. rename EventCenter::init API's parameter n to be nevent
"n" is actually assigned to EventCenter::nevent. rename it
to be "nevent".

4. rename EventCenter::init API's paramter t to be type
"t" is corresponding to Epoll Driver's implementation's type.

5. rename EpollDriver::size to be EpollDriver::nevent
"size" is actually epoll events number, rename it to be "nevent"

6. use event_id as index name to get event instead of "j"

7. rename "nw" to be "nowait"

8. Processor::start unify variable name with Processor::accept & Processor::stop
==> auto &l to be auto &listen_socket

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: remove stack from RDMAWorker
Changcheng Liu [Wed, 7 Aug 2019 07:13:38 +0000 (15:13 +0800)]
msg/async/rdma: remove stack from RDMAWorker

There's no need to cache stack since RDMAWorker already has
Inifiniband obj ib & RDMADispatcher obj dispatcher.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: use shared_ptr to manage RDMADispatcher obj
Changcheng Liu [Wed, 7 Aug 2019 07:08:15 +0000 (15:08 +0800)]
msg/async/rdma: use shared_ptr to manage RDMADispatcher obj

1. Don't use bare pointer to manage RDMADispatcher obj.

2. access RDMADispatcher obj directly instead of accessing it
from RDMAStack. This could avoid caching RDMAStack obj in
RDMAWorker & RDMADispatcher.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: remove stack from RDMADispatcher
Changcheng Liu [Wed, 7 Aug 2019 06:27:05 +0000 (14:27 +0800)]
msg/async/rdma: remove stack from RDMADispatcher

There's no need to cache stack since RDMADispatcher already has
Inifiniband obj ib.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: use shared_ptr to manage Infiniband obj
Changcheng Liu [Wed, 7 Aug 2019 06:19:11 +0000 (14:19 +0800)]
msg/async/rdma: use shared_ptr to manage Infiniband obj

1. Don't use bare pointer to manage Infiniband obj.

2. access Infiniband obj directly instead of accessing it from
RDMAStack. This could avoid caching RDMAStack obj in RDMAWorker
& RDMADispatcher.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: implement function to prefetch buffers
Changcheng Liu [Fri, 2 Aug 2019 03:23:08 +0000 (11:23 +0800)]
msg/async/rdma: implement function to prefetch buffers

The original RDMAConnectedSocketImpl::read read date from buffers and
prefertch data into buffers for next round of reading. It makes the
logical a little complex and the code isn't smooth to be read.
In this patch:
1) RDMAConnectedSocketImpl::buffer_prefetch private API is added to
prefetch data into buffers at the head of read_buffers.
2) reduce one time of calling notify() to reduce context switches.
It's really not needed to notify upper layer to read data since current
read operation hasn't finished yet.
3) Simplify RDMAConnectedSocketImpl::read implementation.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: remove redundant code
Changcheng Liu [Wed, 19 Jun 2019 07:53:08 +0000 (15:53 +0800)]
msg/async/rdma: remove redundant code

1. Below three bits are meaningless in pollfd::events field:
   POLLERR, POLLHUP, or POLLNVAL.
2. QueuePair::pd is initialized in the initialize list.
   There's no need to assign same value to it.
3. Remove the never used function Chunk::set_bound
4. Remove the never used function Chunk::set_offset
5. Remove the never used function QueuePair::is_error
6. Remove SimplePolicyMessenger used vars
7. remove socket_fd() interface since it's never used.
   All data write/read is based on ConnectedSocketImpl::fd.
   So, there's no need to expose socket_fd since it's never used.
8. Remove RDMAServerSocketImpl::get_fd which is not used.
   BTW, RDMAServerSocketImpl::fd has the same function as get_fd.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: show port state with string
Changcheng Liu [Mon, 19 Aug 2019 02:48:52 +0000 (10:48 +0800)]
msg/async/rdma: show port state with string

Show the port state with string is more easy to be read through
value.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agomsg/async/rdma: convert port_id from type uint8_t to int for output
Changcheng Liu [Wed, 14 Aug 2019 06:18:33 +0000 (14:18 +0800)]
msg/async/rdma: convert port_id from type uint8_t to int for output

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
6 years agoMerge pull request #29747 from liewegas/wip-39546
Kefu Chai [Fri, 23 Aug 2019 05:28:52 +0000 (13:28 +0800)]
Merge pull request #29747 from liewegas/wip-39546

osd/PeeringState: do not complain about past_intervals constrained by oldest epoch

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29624 from NancySu05/osdmonitor_markmedown
Kefu Chai [Fri, 23 Aug 2019 05:23:46 +0000 (13:23 +0800)]
Merge pull request #29624 from NancySu05/osdmonitor_markmedown

mon:C_AckMarkedDown has not handled the Callback Arguments

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29738 from ifed01/wip-ifed-alloc-cleanup
Kefu Chai [Fri, 23 Aug 2019 05:22:52 +0000 (13:22 +0800)]
Merge pull request #29738 from ifed01/wip-ifed-alloc-cleanup

os/bluestore: minor improvements/cleanup around allocator

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
6 years agoMerge pull request #29614 from votdev/issue_41205
Kefu Chai [Fri, 23 Aug 2019 05:20:52 +0000 (13:20 +0800)]
Merge pull request #29614 from votdev/issue_41205

mgr/dashboard: Access control database does not restore disabled users correctly

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
6 years agoMerge pull request #29146 from badone/wip-tracker-40835-OSDCap.PoolClassRNS-abort
Kefu Chai [Fri, 23 Aug 2019 05:16:22 +0000 (13:16 +0800)]
Merge pull request #29146 from badone/wip-tracker-40835-OSDCap.PoolClassRNS-abort

osd/OSDCap: Check for empty namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25697 from Aran85/fix-onode-trim
Kefu Chai [Fri, 23 Aug 2019 05:15:27 +0000 (13:15 +0800)]
Merge pull request #25697 from Aran85/fix-onode-trim

os/bluestore: more aggressive deferred submit when onode trim skipping

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #28488 from liuchang0812/show-pool-id-in-pool-ls-cmd
Kefu Chai [Fri, 23 Aug 2019 05:13:32 +0000 (13:13 +0800)]
Merge pull request #28488 from liuchang0812/show-pool-id-in-pool-ls-cmd

mon: show pool id in pool ls command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>