]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agomgr/dashboard: migrate E2E daemons to async/await
Patrick Seidensal [Mon, 19 Aug 2019 08:23:39 +0000 (10:23 +0200)]
mgr/dashboard: migrate E2E daemons to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: migrate E2E rgw-users to async/await
Patrick Seidensal [Mon, 19 Aug 2019 08:21:50 +0000 (10:21 +0200)]
mgr/dashboard: migrate E2E rgw-users to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: migrate E2E buckets to async/await
Patrick Seidensal [Fri, 16 Aug 2019 12:39:54 +0000 (14:39 +0200)]
mgr/dashboard: migrate E2E buckets to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: migrate E2E Helper to async/await
Patrick Seidensal [Fri, 16 Aug 2019 12:39:35 +0000 (14:39 +0200)]
mgr/dashboard: migrate E2E Helper to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: run feature toggle service outside of Angular zone
Patrick Seidensal [Fri, 16 Aug 2019 12:35:32 +0000 (14:35 +0200)]
mgr/dashboard: run feature toggle service outside of Angular zone

to prevent Proctator from hanging indefinitely when testing pages where
the feature toggles are retrieved.

Fixes: mgr/dashboard: Switch to async/await for E2E tests
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: replace E2E test `moveClick` with `clickCheckbox`
Patrick Seidensal [Wed, 14 Aug 2019 09:59:45 +0000 (11:59 +0200)]
mgr/dashboard: replace E2E test `moveClick` with `clickCheckbox`

Ceph Dashboard replaces the original checkbox with a fancy replacement
using the pseudo selector "::before" inside the <label> tag. When the
fancy replacement checkbox is clicked, it actually triggers a click on
the label of the checkbox. `clickCheckbox` does the same.

Removed `moveClick` as it was used to perform an action that the user
would not be able to perform, namely clicking on the checkbox instead of
the checkboxes' label. It was also used for cases where it was not
necessary to be used for, hence the replacement `clickCheckbox` has been
introduced. It checks if the argument passed to it is of type `checkbox`
or `label`. If it is of type `label`, the label will be clicked. If it
is of type `checkbox`, the corresponding `label` is determined and
clicked. This replaces the old implementation with a solution that has
an easy to understand purpose. It also prevents every false application.

This method is designed to be used on all checkboxes of Ceph Dashboard.

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: migrate E2E pools to async/await
Patrick Seidensal [Fri, 16 Aug 2019 12:38:54 +0000 (14:38 +0200)]
mgr/dashboard: migrate E2E pools to async/await

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: prepare PageHelper to migrate pools
Patrick Seidensal [Wed, 21 Aug 2019 12:26:02 +0000 (14:26 +0200)]
mgr/dashboard: prepare PageHelper to migrate pools

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agomgr/dashboard: disable selenium promise manager
Patrick Seidensal [Wed, 21 Aug 2019 12:21:42 +0000 (14:21 +0200)]
mgr/dashboard: disable selenium promise manager

and migrate `protractor.conf.js` to use async/await.

Fixes: https://tracker.ceph.com/issues/40693
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
6 years agoMerge pull request #29889 from smithfarm/wip-clay-typo-fwdport
Nathan Cutler [Wed, 28 Aug 2019 07:32:15 +0000 (09:32 +0200)]
Merge pull request #29889 from smithfarm/wip-clay-typo-fwdport

doc/rados: Correcting some typos in the clay code documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29858 from dillaman/wip-41409
Mykola Golub [Wed, 28 Aug 2019 05:19:25 +0000 (08:19 +0300)]
Merge pull request #29858 from dillaman/wip-41409

test/librbd: set nbd timeout due to newer kernels defaulting it on

Reviewed-by: Mike Christie <mchristi@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #29932 from smithfarm/wip-doc-layout-fwdport
Gregory Farnum [Wed, 28 Aug 2019 03:18:44 +0000 (20:18 -0700)]
Merge pull request #29932 from smithfarm/wip-doc-layout-fwdport

doc: fix typo in doc/radosgw/layout.rst

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge PR #28819 into master
Patrick Donnelly [Tue, 27 Aug 2019 22:57:44 +0000 (15:57 -0700)]
Merge PR #28819 into master

* refs/pull/28819/head:
doc/man/8/monmaptool: document the `--addv` argument.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29470 from liu-chunmei/ceph_seastar_cyan_store
Kefu Chai [Tue, 27 Aug 2019 16:47:58 +0000 (00:47 +0800)]
Merge pull request #29470 from liu-chunmei/ceph_seastar_cyan_store

crimson: futurized CyanStore's member functions and Collection

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/os/cyan_store: remove stale comment 29470/head
Kefu Chai [Sat, 24 Aug 2019 03:09:04 +0000 (11:09 +0800)]
crimson/os/cyan_store: remove stale comment

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: add FuturizedCollection
chunmei Liu [Thu, 22 Aug 2019 07:47:40 +0000 (00:47 -0700)]
crimson: add FuturizedCollection

we will need to talk to BlueStore, and currently `ceph::os::Collection`
only works with `CyanStore`, so a wrapper around `ceph::os::Collection`
is added. we can subclass it to adapt to CyanStore and to BlueStore.

the underlying concrete `FuturizedStore` classes are responsible to
cast `FuturizedCollection` to their own `Collection` type if they
want to access the interface not exposed by `FuturizedCollection`.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
6 years agocrimson/os: s/try_emplace()/insert()/
Kefu Chai [Sat, 24 Aug 2019 02:57:32 +0000 (10:57 +0800)]
crimson/os: s/try_emplace()/insert()/

avoid specifying the type of `map<>::value_type`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: always use Collection::get_cid()
Kefu Chai [Sat, 24 Aug 2019 01:55:22 +0000 (09:55 +0800)]
crimson: always use Collection::get_cid()

prepare for the futurized Collection, which will be an abstract
interface allowing us to talk to BlueStore with one of its concrete
subclass.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoos/ObjectStore: fix typo in comments
Kefu Chai [Thu, 22 Aug 2019 11:27:39 +0000 (19:27 +0800)]
os/ObjectStore: fix typo in comments

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: lift continuation in PGMeta::load()
Kefu Chai [Thu, 22 Aug 2019 08:47:19 +0000 (16:47 +0800)]
crimson/osd: lift continuation in PGMeta::load()

after making `open_collection()` return a future, we don't need to chain
the body of `PGMeta::load()` after `omap_get_values()`. instead we can
lift it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: add futurized interfaces to CyanStore
chunmei Liu [Thu, 22 Aug 2019 06:50:02 +0000 (23:50 -0700)]
crimson: add futurized interfaces to CyanStore

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
6 years agoMerge pull request #29853 from dillaman/wip-41411
Mykola Golub [Tue, 27 Aug 2019 13:27:20 +0000 (16:27 +0300)]
Merge pull request #29853 from dillaman/wip-41411

test/cli-integration/rbd: fixed missing image and snap ids

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agodoc: modify the wrong word "defails" to "details". 29932/head
ypdai [Tue, 12 Feb 2019 09:50:07 +0000 (17:50 +0800)]
doc: modify the wrong word "defails" to "details".

Signed-off-by: ypdai <self19900924@gmail.com>
(cherry picked from commit 8cefe3de7835ce136826faf595122cc210bf90af)

6 years agoMerge pull request #29877 from yuvalif/pubsub_regression_issue2
Yuval Lifshitz [Tue, 27 Aug 2019 13:18:33 +0000 (16:18 +0300)]
Merge pull request #29877 from yuvalif/pubsub_regression_issue2

rgw/pubsub: service reordering issue

6 years agoMerge pull request #29797 from smithfarm/wip-backport-resolve-issue
Nathan Cutler [Tue, 27 Aug 2019 13:14:01 +0000 (15:14 +0200)]
Merge pull request #29797 from smithfarm/wip-backport-resolve-issue

script: add backport-resolve-issue

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29904 from smithfarm/wip-bci-resolve-parent
Nathan Cutler [Tue, 27 Aug 2019 13:07:51 +0000 (15:07 +0200)]
Merge pull request #29904 from smithfarm/wip-bci-resolve-parent

script/backport-create-issue: add --resolve-parent feature

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29912 from tchaikov/wip-wheels-only-for-make-check
Kefu Chai [Tue, 27 Aug 2019 12:49:01 +0000 (20:49 +0800)]
Merge pull request #29912 from tchaikov/wip-wheels-only-for-make-check

install-deps.sh: only prepare wheels for "make check"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoscript: add backport-resolve-issue 29797/head
Nathan Cutler [Wed, 21 Aug 2019 14:49:26 +0000 (16:49 +0200)]
script: add backport-resolve-issue

This script processes GitHub backport PRs, checking for proper cross-linking
with a Redmine Backport tracker issue and, if a PR is merged and properly
cross-linked, it can optionally resolve the tracker issue and correctly
populate the "Target version" field.

The script takes a single positional argument, which is optional. If the
argument is an integer, it is assumed to be a GitHub backport PR ID (e.g. "28549").
In this mode ("single PR mode") the script processes a single GitHub backport
PR and terminates.

If the argument is not an integer, or is missing, it is assumed to be a
commit (SHA1 or tag) to start from. If no positional argument is given, it
defaults to the tag "BRI", which might have been added by the last run of the
script. This mode is called "scan merge commits mode".

In both modes, the script scans a local git repo, which is assumed to be
in the current working directory. In single PR mode, the script will work
only if the PR's merge commit is present in the current branch of the local
git repo. In scan merge commits mode, the script starts from the given SHA1
or tag, taking each merge commit in turn and attempting to obtain the GitHub
PR number for each.

For each GitHub PR, the script interactively displays all relevant information
(NOTE: this includes displaying the GitHub PR and Redmine backport issue in
web browser tabs!) and prompts the user for her preferred disposition.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #29790 from hjwsm1989/fix-unittest-osdmap
Kefu Chai [Tue, 27 Aug 2019 11:42:21 +0000 (19:42 +0800)]
Merge pull request #29790 from hjwsm1989/fix-unittest-osdmap

tests/osd: fix typo in unittest_osdmap

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agorgw/pubsub: service reordering issue 29877/head
Yuval Lifshitz [Sun, 25 Aug 2019 14:12:53 +0000 (17:12 +0300)]
rgw/pubsub: service reordering issue

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
6 years agoinstall-deps.sh: only prepare wheels for "make check" 29912/head
Kefu Chai [Tue, 27 Aug 2019 01:22:34 +0000 (09:22 +0800)]
install-deps.sh: only prepare wheels for "make check"

* move `for_make_check` to the beginning of script, as FreeBSD will also
  use this variable
* extract `preload_wheels_for_tox()` function out to improve readability
* call `preload_wheels_for_tox()` only if `for_make_check` is true

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodoc/man/8/monmaptool: document the `--addv` argument. 28819/head
Luca Castoro [Thu, 27 Jun 2019 13:58:06 +0000 (15:58 +0200)]
doc/man/8/monmaptool: document the `--addv` argument.

Fixes: http://tracker.ceph.com/issues/40568
Signed-off-by: Luca Castoro <luca.castoro@brightcomputing.com>
6 years agoMerge pull request #29090 from liewegas/wip-40792
Kefu Chai [Tue, 27 Aug 2019 09:34:13 +0000 (17:34 +0800)]
Merge pull request #29090 from liewegas/wip-40792

mon/MonClient: ENXIO when sending command to down mon

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #28177 from jecluis/wip-noreply-failures
Kefu Chai [Tue, 27 Aug 2019 09:32:54 +0000 (17:32 +0800)]
Merge pull request #28177 from jecluis/wip-noreply-failures

mon: ensure prepare_failure() marks no_reply on op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoscript/backport-create-issue: add --resolve-parent feature 29904/head
Nathan Cutler [Mon, 26 Aug 2019 13:05:09 +0000 (15:05 +0200)]
script/backport-create-issue: add --resolve-parent feature

When --resolve-parent is provided on the command line, the script
will check the status of backport issues for those parent issues
all of whose backport issues already exist. If all the backport
issues are in status "Resolved", the parent issue's status is set
to "Resolved" as well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #29862 from tchaikov/wip-41416
Kefu Chai [Tue, 27 Aug 2019 09:16:41 +0000 (17:16 +0800)]
Merge pull request #29862 from tchaikov/wip-41416

rgw,bluestore: fixes to address failures from check-generated.sh

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #29893 from xiexingguo/wip-inc-recovery-5
Xie Xingguo [Tue, 27 Aug 2019 05:16:20 +0000 (13:16 +0800)]
Merge pull request #29893 from xiexingguo/wip-inc-recovery-5

osd/osd_types: inc-recovery - add special handler for lost_revert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agotest/encoding/check-generated.sh: show diff if cmp fails 29862/head
Kefu Chai [Sat, 24 Aug 2019 06:12:41 +0000 (14:12 +0800)]
test/encoding/check-generated.sh: show diff if cmp fails

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/encoding/check-generated.sh: run loop in the same shell
Kefu Chai [Sun, 25 Aug 2019 04:09:24 +0000 (12:09 +0800)]
test/encoding/check-generated.sh: run loop in the same shell

set parent scope variables in the same shell.

foo | while read ....

in the above statement, `while read ...` is executed in a subshell. so
it cannot change the bash variables in its parent shell.

in this change, the output of `foo` is redirected to the stdin of `while
read` statement. so we can capture the test failures.

before this change, the test always succeed, even if there are failures.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/DecayCounter: do not decay when testing denc
Kefu Chai [Sun, 25 Aug 2019 13:22:16 +0000 (21:22 +0800)]
common/DecayCounter: do not decay when testing denc

check-generated.sh tries to verify the invariance of a type after
encoding and decoding. so we should keep the value of `DecayCounter` the
same when encoding/decoding/dumping.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: rgw_log_entry always initialize total_time
Kefu Chai [Sun, 25 Aug 2019 06:54:30 +0000 (14:54 +0800)]
rgw: rgw_log_entry always initialize total_time

zero-initialize rgw_log_entry::total_time

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/ServiceMap: use plain gid
Kefu Chai [Sun, 25 Aug 2019 06:53:20 +0000 (14:53 +0800)]
mgr/ServiceMap: use plain gid

* partially revert 5c25a018. which is not backward compatible.
* change `ServiceMap::get_daemon()` so it returns a
  `pair<Daemon*,bool>`.

git is an `optional<uint64_t>`, so we cannot dump it without checking.

Fixes: https://tracker.ceph.com/issues/41424
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: init pg_pool_t::pg_autoscale_mode in ctor
Kefu Chai [Sun, 25 Aug 2019 04:08:39 +0000 (12:08 +0800)]
osd: init pg_pool_t::pg_autoscale_mode in ctor

so its value is determined

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: use scoped enum for pg_pool_t::pg_autoscale_mode
Kefu Chai [Sun, 25 Aug 2019 04:06:51 +0000 (12:06 +0800)]
osd: use scoped enum for pg_pool_t::pg_autoscale_mode

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: use in-class initialization in pg_pool_t
Kefu Chai [Sun, 25 Aug 2019 03:49:50 +0000 (11:49 +0800)]
osd: use in-class initialization in pg_pool_t

less repeat this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: implement copy ctor and assignment for bluestore_blob_use_tracker_t
Kefu Chai [Sun, 25 Aug 2019 03:48:42 +0000 (11:48 +0800)]
os/bluestore: implement copy ctor and assignment for bluestore_blob_use_tracker_t

to address the `check-generated.sh` test failure.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: RGWObjManifest::obj_iterator::seek always update locations
Kefu Chai [Sat, 24 Aug 2019 15:41:57 +0000 (23:41 +0800)]
rgw: RGWObjManifest::obj_iterator::seek always update locations

we should update explicit loc and location even if
`ofs >= manifest->obj_size`. there is chance that we are updating an end
iterator whose ofs is equal to obj_size. before being updated, the end
iterator points to an implicit location, while after being updated, the
manifest could be using an explicit location, so we should update the
end iterator as well.

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: RGWObjManifest::obj_iterator: init member variables in-class
Kefu Chai [Sun, 25 Aug 2019 01:07:48 +0000 (09:07 +0800)]
rgw: RGWObjManifest::obj_iterator: init member variables in-class

* remove the init() member function which is solely used by the
  constructors.
* use constructor delegation to minimize the repeating

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: update_iterator() in RGWObjManifest::set_explicit()
Kefu Chai [Sat, 24 Aug 2019 06:12:06 +0000 (14:12 +0800)]
rgw: update_iterator() in RGWObjManifest::set_explicit()

we should update the end iterator when updating RGWObjManifest with its
parts

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29913 from tchaikov/wip-debian/control
Kefu Chai [Tue, 27 Aug 2019 02:28:00 +0000 (10:28 +0800)]
Merge pull request #29913 from tchaikov/wip-debian/control

debian/control: fix Build-Depends

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agodebian/control: fix Build-Depends 29913/head
Kefu Chai [Tue, 27 Aug 2019 02:05:15 +0000 (10:05 +0800)]
debian/control: fix Build-Depends

it's a regression introduced by 5d6d770e

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #29867 into master
Patrick Donnelly [Mon, 26 Aug 2019 21:09:39 +0000 (14:09 -0700)]
Merge PR #29867 into master

* refs/pull/29867/head:
doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29822 from vumrao/wip-vumrao-41399
Neha Ojha [Mon, 26 Aug 2019 19:57:58 +0000 (12:57 -0700)]
Merge pull request #29822 from vumrao/wip-vumrao-41399

os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoos/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1 29822/head
Vikhyat Umrao [Thu, 22 Aug 2019 16:45:40 +0000 (09:45 -0700)]
os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
6 years agoMerge pull request #29903 from tchaikov/wip-pip-download
Kefu Chai [Mon, 26 Aug 2019 16:44:28 +0000 (00:44 +0800)]
Merge pull request #29903 from tchaikov/wip-pip-download

install-deps.sh: download wheel using 'pip wheel'

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoinstall-deps.sh: download wheel using 'pip wheel' 29903/head
Kefu Chai [Mon, 26 Aug 2019 03:36:50 +0000 (11:36 +0800)]
install-deps.sh: download wheel using 'pip wheel'

otherwise we will fail to install the build dependencies of
`lazy-object-proxy` from the wheelhouse. as `lazy-object-proxy` does not
add `setuptools_scm` in its `setup.py`, instead it lists
`setuptools_scm` in `setup.cfg` and `pyproject.toml` as a `build-system`
requires. but unfortunately, `pip download` only downloads the
install/run-time dependencies at this moment. and `lazy-object-proxy`
does not offer binary package for at least python2.7.

ideally, `pip download` should collects its dependencies like

Collecting setuptools_scm>=3.3.1 (from lazy-object-proxy->astroid<3,>=2.2.0->pylint->-r requirements-lint.txt (line 1))

so we need to use `pip wheel` do download build-time dependencies

see also https://github.com/pypa/pip/issues/6222

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29854 from tchaikov/wip-sloppy-cond
Kefu Chai [Mon, 26 Aug 2019 14:38:50 +0000 (22:38 +0800)]
Merge pull request #29854 from tchaikov/wip-sloppy-cond

common/condition_variable_debug: do not assert() if sloppy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #29866 from joscollin/wip-fix-stupid
Kefu Chai [Mon, 26 Aug 2019 14:36:41 +0000 (22:36 +0800)]
Merge pull request #29866 from joscollin/wip-fix-stupid

os/bluestore: silence StupidAllocator reorder warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29890 from rhcs-dashboard/wip-signed-master
Alfredo Deza [Mon, 26 Aug 2019 12:20:57 +0000 (08:20 -0400)]
Merge pull request #29890 from rhcs-dashboard/wip-signed-master

github: display phrase for signed-off check

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #29831 from tchaikov/wip-python-int
Kefu Chai [Mon, 26 Aug 2019 12:06:21 +0000 (20:06 +0800)]
Merge pull request #29831 from tchaikov/wip-python-int

mgr/BaseMgrModule: use PyInt_Check() to compatible with py2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agomgr/dashboard: Edit Pool Placement Group E2E Test (#29779)
Lenz Grimmer [Mon, 26 Aug 2019 11:36:53 +0000 (11:36 +0000)]
mgr/dashboard: Edit Pool Placement Group E2E Test (#29779)

mgr/dashboard: Edit Pool Placement Group E2E Test

Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #29707 from rafaelquint/dashdoc
Lenz Grimmer [Mon, 26 Aug 2019 11:33:41 +0000 (11:33 +0000)]
Merge pull request #29707 from rafaelquint/dashdoc

mgr/dashboard: Extend "Writing End-to-End Tests" section (describe vs it)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
6 years agogithub: display phrase for signed-off check 29890/head
Ernesto Puerta [Mon, 26 Aug 2019 09:16:25 +0000 (11:16 +0200)]
github: display phrase for signed-off check

https://github.com/ceph/ceph-build/pull/1379 will introduce a
trigger-phrase for pr-commit check (DCO/signed-off).

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agodoc/rados: Correcting some typos in the clay code documentation 29889/head
Myna V [Fri, 16 Aug 2019 06:28:44 +0000 (11:58 +0530)]
doc/rados: Correcting some typos in the clay code documentation
Signed-off-by: Myna <mynaramana@gmail.com>
(cherry picked from commit a20ba26721e1528f8a528e179c8decfb0d5f1374)

Note: This documentation fix was merged to nautilus via
https://github.com/ceph/ceph/pull/29191 without being merged to master, first.
This commit forward-ports the fix to master.

6 years agoMerge pull request #29840 from tchaikov/wip-install-deps.sh-without-python-saml
Kefu Chai [Mon, 26 Aug 2019 08:40:02 +0000 (16:40 +0800)]
Merge pull request #29840 from tchaikov/wip-install-deps.sh-without-python-saml

install-deps.sh,deb,rpm: move python-saml deps into debian/control an…

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoosd/osd_types: add_next_event - add special handler for lost_revert 29893/head
xie xingguo [Mon, 26 Aug 2019 02:28:34 +0000 (10:28 +0800)]
osd/osd_types: add_next_event - add special handler for lost_revert

For unfound objects, we might append LOST_REVERT log entries,
which shall allow these objects to be reverted to the newest
available version later.
However, we are currently lack of support to rewind the clean_regions
portion too when marking unfound objects as lost with inc-recovery mode
enabled. Hence we must mark these unfound-revert objects as fully dirty
to make sure they can be correctly recovered.

E.g.,:
- primary is pulling object A from replica 1
- object A is corrupted on replica 1
- object A is now unfound
- mark object A as lost, replica 1 will persist a wrong
  missing item for object A..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoosd/osd_types: drop 'new_object' from constructor
xie xingguo [Sat, 24 Aug 2019 01:51:20 +0000 (09:51 +0800)]
osd/osd_types: drop 'new_object' from constructor

There is no consumer.
Actually, I think this field is only meaningful to be used
to indicate whether we should initiate an inc-recovery or not.
If not, then we shall fall back to triggering a full-recovery
instead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoosd/osd_types: drop 'new_object' from missing.add
xie xingguo [Sat, 24 Aug 2019 05:31:20 +0000 (13:31 +0800)]
osd/osd_types: drop 'new_object' from missing.add

because below here we know we'll always mark object as fully dirty.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoosd/osd_types: always call mark_fully_dirty for missing.add
xie xingguo [Sat, 24 Aug 2019 02:56:57 +0000 (10:56 +0800)]
osd/osd_types: always call mark_fully_dirty for missing.add

In general we shall build missing set (and hence clean_regions)
based on pg log. However, currently there are still 5 cases we might call
missing.add to add a new pg_missing_item into the missing set
explicitly (or replace an existing pg_missing_item entirely):

1. we explicitly build missing set on startup, in which case
we know we are trying to be compatiable with pre-kraken versions,
so it should be ok for us to disable inc-recovery.

2. we are currently processing authoritative log, and there are
some divergent objects detected. For simplicity (and correctness),
we should disable inc-recovery entirly for these objects.

3. we are re-building missing set, e.g., due to the global
CEPH_OSDMAP_RECOVERY_DELETES policy changing.
In this case we know we are at the end of upgrading from a
pervious version that is lack of CEPH_OSDMAP_RECOVERY_DELETES support.
Hence it should be the recommended option to disable inc-recovery
simultaneously since these objects should be lack of inc-recovery support
too.

4. we are adding or re-adding missing object into primary's missing_loc.
It doesn't matter whether we have a correct clean_regions there
since we never actually refer to that field from missing_loc
when we actually start to perform object recovery later.

5. we are auto-repairing a corrupted object and hence the need of
adding it to the corresponding missing set first, e.g, by leveraging
the existing recovery procedure. In this case, we always disable
inc-recovery to make sure this object can be fully (and correctly)
recovered later.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #29808 from dillaman/wip-41356
Mykola Golub [Sun, 25 Aug 2019 17:23:51 +0000 (20:23 +0300)]
Merge pull request #29808 from dillaman/wip-41356

rbd-mirror: ignore errors relating to parsing the cluster config file

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agodoc: default values for mon_health_to_clog_* were flipped 29867/head
James McClune [Sun, 25 Aug 2019 03:24:31 +0000 (23:24 -0400)]
doc: default values for mon_health_to_clog_* were flipped

On a freshly installed nautilus cluster (i.e. 14.2.2), the default values are:

mon_health_to_clog_interval = 3600
mon_health_to_clog_tick_interval = 60.000000

Fixes: https://tracker.ceph.com/issues/41403
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
6 years agoMerge pull request #29651 from rzarzynski/wip-crimson-cls
Kefu Chai [Sun, 25 Aug 2019 07:25:41 +0000 (15:25 +0800)]
Merge pull request #29651 from rzarzynski/wip-crimson-cls

crimson, osd: add support for Ceph Classes, part 1

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29595 from bk201/wip-41151
Kefu Chai [Sun, 25 Aug 2019 07:13:33 +0000 (15:13 +0800)]
Merge pull request #29595 from bk201/wip-41151

mgr/test_orchestrator: Allow initializing dummy data

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge pull request #29787 from NancySu05/fix-osd
Kefu Chai [Sun, 25 Aug 2019 07:11:51 +0000 (15:11 +0800)]
Merge pull request #29787 from NancySu05/fix-osd

osd: add log information to record the cause of do_osd_ops failure

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoos/bluestore: silence StupidAllocator reorder warning 29866/head
Jos Collin [Sun, 25 Aug 2019 03:04:59 +0000 (08:34 +0530)]
os/bluestore: silence StupidAllocator reorder warning

warning: ‘StupidAllocator::last_alloc’ will be initialized after [-Wreorder]

Signed-off-by: Jos Collin <jcollin@redhat.com>
6 years agoMerge PR #29542 into master
Patrick Donnelly [Sun, 25 Aug 2019 04:35:14 +0000 (21:35 -0700)]
Merge PR #29542 into master

* refs/pull/29542/head:
mds: trim cache on regular schedule

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #29760 into master
Patrick Donnelly [Sun, 25 Aug 2019 04:31:58 +0000 (21:31 -0700)]
Merge PR #29760 into master

* refs/pull/29760/head:
mgr/volumes: cleanup FS subvolume or subvolume group path
mgr/volumes: give useful error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #29843 into master
Patrick Donnelly [Sun, 25 Aug 2019 04:31:27 +0000 (21:31 -0700)]
Merge PR #29843 into master

* refs/pull/29843/head:
qa/tasks: drop object inherit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agotests/osd: fix typo in unittest_osdmap 29790/head
huangjun [Wed, 21 Aug 2019 11:29:53 +0000 (19:29 +0800)]
tests/osd: fix typo in unittest_osdmap

Signed-off-by: huangjun <huangjun@xsky.com>
6 years agomds: trim cache on regular schedule 29542/head
Patrick Donnelly [Wed, 7 Aug 2019 17:35:02 +0000 (10:35 -0700)]
mds: trim cache on regular schedule

Do this outside the standard tick interval as it needs to be driven more
frequently to keep up with client workloads that generate a lot of
capabilities.

Fixes: https://tracker.ceph.com/issues/41141
Fixes: https://tracker.ceph.com/issues/41140
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #29754 from xiexingguo/wip-inc-recovery-3
Xie Xingguo [Sat, 24 Aug 2019 05:51:21 +0000 (13:51 +0800)]
Merge pull request #29754 from xiexingguo/wip-inc-recovery-3

osd: misc inc-recovery compat fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #28344 from iotcg/rdma
Kefu Chai [Sat, 24 Aug 2019 03:37:03 +0000 (11:37 +0800)]
Merge pull request #28344 from iotcg/rdma

check rdma configuration and fix some logic problem

Reviewed-by: Roman Penyaev <rpenyaev@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: dissect pg listing implementation from OpsExecuter. 29651/head
Radoslaw Zarzynski [Tue, 13 Aug 2019 12:59:51 +0000 (14:59 +0200)]
crimson/osd: dissect pg listing implementation from OpsExecuter.

Additionally improve const-correctness for PGBackend.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: do_pgnls_common isn't the member of OpsExecuter anymore.
Radoslaw Zarzynski [Mon, 12 Aug 2019 18:30:17 +0000 (20:30 +0200)]
crimson/osd: do_pgnls_common isn't the member of OpsExecuter anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/os, crimson/osd: make xattr retrieval const.
Radoslaw Zarzynski [Mon, 12 Aug 2019 18:28:37 +0000 (20:28 +0200)]
crimson/os, crimson/osd: make xattr retrieval const.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/os, crimson/osd: make list_objects() const.
Radoslaw Zarzynski [Mon, 12 Aug 2019 18:11:20 +0000 (20:11 +0200)]
crimson/os, crimson/osd: make list_objects() const.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: adopt PGLSPlainFilter.
Radoslaw Zarzynski [Mon, 12 Aug 2019 16:15:53 +0000 (18:15 +0200)]
crimson/osd: adopt PGLSPlainFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoosd: share PGLSPlainFilter with crimson.
Radoslaw Zarzynski [Mon, 12 Aug 2019 14:41:19 +0000 (16:41 +0200)]
osd: share PGLSPlainFilter with crimson.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoobjclass, osd: dissect PGLSFilter into the osd_types.h header.
Radoslaw Zarzynski [Mon, 12 Aug 2019 13:32:38 +0000 (15:32 +0200)]
objclass, osd: dissect PGLSFilter into the osd_types.h header.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement PGLSFilter.
Radoslaw Zarzynski [Wed, 7 Aug 2019 17:59:27 +0000 (19:59 +0200)]
crimson/osd: implement PGLSFilter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: OpsExecuter differentiates read and modifying operations.
Radoslaw Zarzynski [Tue, 6 Aug 2019 11:56:51 +0000 (13:56 +0200)]
crimson/osd: OpsExecuter differentiates read and modifying operations.

This is necessary to pass ClsHello.BadMethods.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement CEPH_OSD_OP_GETXATTR and cls_cxx_getxattr().
Radoslaw Zarzynski [Mon, 5 Aug 2019 15:24:24 +0000 (17:24 +0200)]
crimson/osd: implement CEPH_OSD_OP_GETXATTR and cls_cxx_getxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: differentiate ENOENT during do_osd_call().
Radoslaw Zarzynski [Mon, 5 Aug 2019 11:58:06 +0000 (13:58 +0200)]
crimson/osd: differentiate ENOENT during do_osd_call().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_read2().
Radoslaw Zarzynski [Mon, 5 Aug 2019 11:53:13 +0000 (13:53 +0200)]
crimson/osd: implement cls_cxx_read2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_setxattr().
Radoslaw Zarzynski [Mon, 5 Aug 2019 06:48:55 +0000 (08:48 +0200)]
crimson/osd: implement cls_cxx_setxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement CEPH_OSD_OP_SETXATTR.
Radoslaw Zarzynski [Mon, 5 Aug 2019 06:48:19 +0000 (08:48 +0200)]
crimson/osd: implement CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/os: port get_max_attr_name_length() from MemStore.
Radoslaw Zarzynski [Mon, 5 Aug 2019 06:47:08 +0000 (08:47 +0200)]
crimson/os: port get_max_attr_name_length() from MemStore.

This method is an enabler for CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoobjclass, osd: drop unused variable.
Radoslaw Zarzynski [Mon, 5 Aug 2019 05:35:01 +0000 (07:35 +0200)]
objclass, osd: drop unused variable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: implement cls_cxx_write_full().
Radoslaw Zarzynski [Sat, 3 Aug 2019 07:14:00 +0000 (09:14 +0200)]
crimson/osd: implement cls_cxx_write_full().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrimson/osd: introduce OpsExecuter to uniform calling CEPH_OSD_OPS.
Radoslaw Zarzynski [Fri, 2 Aug 2019 12:27:52 +0000 (14:27 +0200)]
crimson/osd: introduce OpsExecuter to uniform calling CEPH_OSD_OPS.

OSD has two entry points for executing CEPH_OSD_OP_*:
  1. the MOSDOp message handler,
  2. the Object Class API (cls_* and cls_cxx_* functions).

We definitely want to address these two users without code
duplication. However, exposing the entire PG to Obj Class
would break encapsulation. Moreover, there is difference
in life times between PG and sequence-of-operations-from-
MOSDOp.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>