]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoscript/run_tox.sh: add --help option 29457/head
Kefu Chai [Mon, 5 Aug 2019 16:45:43 +0000 (00:45 +0800)]
script/run_tox.sh: add --help option

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: pass tox envs to run_tox.sh explicitly
Kefu Chai [Mon, 5 Aug 2019 15:37:52 +0000 (23:37 +0800)]
cmake: pass tox envs to run_tox.sh explicitly

* add two optional keyword parameters to `add_tox_test()`
* use the default `TOX_PATH` parameter instead passing it explicitly,
  as it is always `${CMAKE_CURRENT_SOURCE_DIR}`
* drop the code to guess the tox envs in `run_tox.sh`, and always pass
  them explicitly in CMake script using the `--tox-envs` argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: move tests to where they are implemented
Kefu Chai [Sun, 4 Aug 2019 06:52:50 +0000 (14:52 +0800)]
cmake: move tests to where they are implemented

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: do not add gtest-parallel_ext if already added
Kefu Chai [Mon, 5 Aug 2019 01:41:28 +0000 (09:41 +0800)]
cmake: do not add gtest-parallel_ext if already added

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: use script/run_tox.sh for running tox for dashboard
Kefu Chai [Fri, 2 Aug 2019 05:26:36 +0000 (13:26 +0800)]
cmake: use script/run_tox.sh for running tox for dashboard

* adapt script/run_tox.sh to use different envs for dashboard's tox
  test.
* use script/run_tox.sh for running dashboard's tox test
* remove pybind/mgr/dashboard/run-tox.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake/modules/AddCephTest.cmake: make ${name}-venv a target
Kefu Chai [Sun, 4 Aug 2019 06:31:03 +0000 (14:31 +0800)]
cmake/modules/AddCephTest.cmake: make ${name}-venv a target

so we can prepare the venv for tox with `make tests`. and it's optional,
as run_tox.sh will check for it and setup a venv if it's not around.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agosrc: remove unused run-tox.sh
Kefu Chai [Fri, 2 Aug 2019 04:40:02 +0000 (12:40 +0800)]
src: remove unused run-tox.sh

they are not used anymore after b9c9f74f

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29403 from rhcs-dashboard/wip-41013-master
Kefu Chai [Mon, 5 Aug 2019 10:07:39 +0000 (18:07 +0800)]
Merge pull request #29403 from rhcs-dashboard/wip-41013-master

install-deps.sh: remove failing error catching

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29476 from tchaikov/wip-test/journal-close-object
Kefu Chai [Mon, 5 Aug 2019 09:37:33 +0000 (17:37 +0800)]
Merge pull request #29476 from tchaikov/wip-test/journal-close-object

test/journal: always close object

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agotest/journal: always close object 29476/head
Kefu Chai [Sun, 4 Aug 2019 14:48:31 +0000 (22:48 +0800)]
test/journal: always close object

otherwise when we destruct `TestObjectRecorder::Handler::lock` or
`journal::JournalRecorder::m_object_locks`, it/they would be still
being waited by some condition variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29474 from wjwithagen/wjw-fix-rbd_ggate_Server.cc
Kefu Chai [Sun, 4 Aug 2019 14:42:19 +0000 (22:42 +0800)]
Merge pull request #29474 from wjwithagen/wjw-fix-rbd_ggate_Server.cc

rbd-ggate: fix compile errors from ceph::mutex update

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agorbd-ggate: fix compile errors from ceph::mutex update 29474/head
Willem Jan Withagen [Sat, 3 Aug 2019 20:48:03 +0000 (22:48 +0200)]
rbd-ggate: fix compile errors from ceph::mutex update

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #29472 from wjwithagen/wjw-fix-src/osdc/ObjectCacher.cc
Kefu Chai [Sat, 3 Aug 2019 16:58:13 +0000 (00:58 +0800)]
Merge pull request #29472 from wjwithagen/wjw-fix-src/osdc/ObjectCacher.cc

osdc: Fix a missing ':' for the correct namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosdc: Fix a missing ':' for the correct namespace 29472/head
Willem Jan Withagen [Sat, 3 Aug 2019 15:04:46 +0000 (17:04 +0200)]
osdc: Fix a missing ':' for the correct namespace

Clang complains:
/home/jenkins/workspace/ceph-master/src/osdc/ObjectCacher.cc:1848:22: error: no viable constructor or deduction guide for deduction of template arguments of 'unique_lock'
    std::unique_lock l{lock, std:adopt_lock};

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #29322 from rzarzynski/wip-objclass-cleanup
Kefu Chai [Sat, 3 Aug 2019 12:25:39 +0000 (20:25 +0800)]
Merge pull request #29322 from rzarzynski/wip-objclass-cleanup

objclass, osd: clean up the cls-host interface. Turn ClassHandler into singleton

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29113 from tchaikov/wip-ceph-mutex
Kefu Chai [Sat, 3 Aug 2019 12:16:24 +0000 (20:16 +0800)]
Merge pull request #29113 from tchaikov/wip-ceph-mutex

src/: s/Mutex/ceph::mutex/

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agocommon/Mutex: remove Mutex 29113/head
Kefu Chai [Mon, 22 Jul 2019 07:07:06 +0000 (15:07 +0800)]
common/Mutex: remove Mutex

as it's replaced by `ceph::mutex`

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/Cond.h: remove Cond
Kefu Chai [Mon, 22 Jul 2019 06:49:49 +0000 (14:49 +0800)]
common/Cond.h: remove Cond

as it's replaced by ceph::condition_variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/PG: support is_locked() in non-debug mode
Kefu Chai [Sat, 20 Jul 2019 11:34:02 +0000 (19:34 +0800)]
osd/PG: support is_locked() in non-debug mode

`PG` print details info in the prefix of logging messages if the PG is
being locked by current thread. but `ceph::mutex` is an alias of
`std::mutex` in non-debug mode, so neither `mutex::is_locked_by_me()` nor
`mutex::is_locked()` is supported in non-debug mode. to continue supporting
this feature, `PG::locked_by` is added to memorize the thread id of the owner
of the lock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/ceph_context.cc: include necessary header
Kefu Chai [Fri, 19 Jul 2019 16:21:04 +0000 (00:21 +0800)]
common/ceph_context.cc: include necessary header

for using functions like `lockdep_unregister_ceph_context()`

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/testmsgr: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:38:02 +0000 (21:38 +0800)]
test/testmsgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/librados: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:34:40 +0000 (21:34 +0800)]
test/librados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/compressor: do not include unused header
Kefu Chai [Thu, 18 Jul 2019 13:33:23 +0000 (21:33 +0800)]
test/compressor: do not include unused header

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/common/Throttle: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:33:14 +0000 (21:33 +0800)]
test/common/Throttle: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/obj_bencher: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:28:43 +0000 (21:28 +0800)]
common/obj_bencher: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoauth: remove unused forwarde declaration
Kefu Chai [Thu, 18 Jul 2019 13:28:21 +0000 (21:28 +0800)]
auth: remove unused forwarde declaration

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/librados_test_stub: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 13:18:40 +0000 (21:18 +0800)]
test/librados_test_stub: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibcephfs: remove unused #include
Kefu Chai [Wed, 17 Jul 2019 13:17:36 +0000 (21:17 +0800)]
libcephfs: remove unused #include

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocls_kvs: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 13:17:07 +0000 (21:17 +0800)]
cls_kvs: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrush: remove unused #include
Kefu Chai [Wed, 17 Jul 2019 13:16:20 +0000 (21:16 +0800)]
crush: remove unused #include

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agofusetrace: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 09:02:31 +0000 (17:02 +0800)]
fusetrace: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd_nbd: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:31:42 +0000 (16:31 +0800)]
tools/rbd_nbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd_ggate: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:31:07 +0000 (16:31 +0800)]
tools/rbd_ggate: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:30:56 +0000 (16:30 +0800)]
tools/rbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/ceph_dedup_tool: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:30:02 +0000 (16:30 +0800)]
tools/ceph_dedup_tool: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/xattr_bench: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:29:50 +0000 (16:29 +0800)]
test/xattr_bench: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/test_snap_mapper: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:29:36 +0000 (16:29 +0800)]
test/test_snap_mapper: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/osd: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:28:43 +0000 (16:28 +0800)]
test/osd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/omap_bench: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:28:33 +0000 (16:28 +0800)]
test/omap_bench: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/msgr: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:28:11 +0000 (16:28 +0800)]
test/msgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/admin_socket: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:26:36 +0000 (16:26 +0800)]
test/admin_socket: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agokv: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:25:09 +0000 (16:25 +0800)]
kv: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rados: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:45:15 +0000 (12:45 +0800)]
tools/rados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/immutable_object_cache: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:45:01 +0000 (12:45 +0800)]
tools/immutable_object_cache: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/cephfs: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:44:47 +0000 (12:44 +0800)]
tools/cephfs: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/os/filestore: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:44:27 +0000 (12:44 +0800)]
test/os/filestore: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd_mirror: s/Mutex/ceph::Formatter/
Kefu Chai [Fri, 19 Jul 2019 17:13:38 +0000 (01:13 +0800)]
tools/rbd_mirror: s/Mutex/ceph::Formatter/

* s/Formatter/ceph::Formatter/ in tools/rbd_mirror/ImageSyncThrottler,
  as we cannot rely on `using ceph` or `using ceph::Formatter` in
  some other included header files.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/perf_local: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:43:32 +0000 (12:43 +0800)]
test/perf_local: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/osdc: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:43:18 +0000 (12:43 +0800)]
test/osdc: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/objectstore: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:43:04 +0000 (12:43 +0800)]
test/objectstore: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/mon: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:42:36 +0000 (12:42 +0800)]
test/mon: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/librbd: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:42:23 +0000 (12:42 +0800)]
test/librbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/journal: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:41:22 +0000 (12:41 +0800)]
test/journal: s/Mutex/ceph::mutex/

as we cannot create a `smart_pointer<ceph::mutex>` due to limitation of
`ceph::make_mutex()`, we need to refactor `TestObjectRecorder` to remove
its `TearDown()` method, as when `TearDown()` is called, all local
variables have been destroyed, including the `ceph::mutex` instances. so
we need to introduce a helper of `ObjectRecorderFlusher`, to flush the
objects before `mutex` instances are destroyed. to simplify the
interfaces, it flushes in its dtor. so its lifecycle should be shorter
than those of mutexes. that's why, mutexes are created before `flusher`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:40:52 +0000 (12:40 +0800)]
rgw: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibrbd: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:38:40 +0000 (12:38 +0800)]
librbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agojournal: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:35:23 +0000 (12:35 +0800)]
journal: s/Mutex/ceph::mutex/

* FutureImpl::m_lock is an exception. as, before this change, the lock
  was initialized like `m_lock("FutureImpl::m_lock", false, false)`, see
  the declaration of
  `Mutex(const std::string &n, bool r = false, bool ld=true, bool bt=false)`
  so `m_lock` is actually not using the extra features offered by
  `Mutex` like runtime lockdeps check. and `mutex_debugging_base` does
  not allow us to disable lockdeps individually. but it does use the `is_locked()`
  method. so instead of using `ceph::mutex` directly, a cutomized
  `ceph::mutex` is added for `CEPH_DEBUG_MUTEX` build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/mutex_debug: add lockdep parameter to mutex's ctor
Kefu Chai [Thu, 25 Jul 2019 16:32:57 +0000 (00:32 +0800)]
common/mutex_debug: add lockdep parameter to mutex's ctor

so we can disable lockdep when constructing a ceph::mutex. otherwise
it's difficult to use the locks from standard library and to use
`ceph_assert(ceph_mutex_is_locked(lock))` at the same time.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/mutex_debug.h: pass "no_lockdep" to try_lock()
Kefu Chai [Sat, 20 Jul 2019 16:41:18 +0000 (00:41 +0800)]
common/mutex_debug.h: pass "no_lockdep" to try_lock()

there is chance that we want to disable lockdep, so we shold pass
`no_lockdep` down to `try_lock()` in `lock()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoinclude/Context: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:34:19 +0000 (12:34 +0800)]
include/Context: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoerasure-code: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:33:41 +0000 (12:33 +0800)]
erasure-code: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/ceph_mutex: add more ceph_mutex_is_*() helpers
Kefu Chai [Sun, 7 Jul 2019 08:17:35 +0000 (16:17 +0800)]
common/ceph_mutex: add more ceph_mutex_is_*() helpers

we have caller sites like

ceph_assert(!client->client_lock.is_locked_by_me())

in `src/client/Client.cc` and `src/librados/RadosClient.cc`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/ceph_mutex: improve ceph::shared_mutex
Kefu Chai [Sun, 7 Jul 2019 04:32:53 +0000 (12:32 +0800)]
common/ceph_mutex: improve ceph::shared_mutex

add helpers for supporting `ceph_mutex_is_locked()` and
`ceph_mutex_is_wlocked()` macros

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/Timer: add necessary #include
Kefu Chai [Sun, 7 Jul 2019 03:24:42 +0000 (11:24 +0800)]
common/Timer: add necessary #include

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/RWLock: add RWLock::{lock,unlock}_shared()
Kefu Chai [Sun, 7 Jul 2019 03:21:12 +0000 (11:21 +0800)]
common/RWLock: add RWLock::{lock,unlock}_shared()

so RWLock can be used with std::shared_lock,

rgw is using RWLock'prioritize_write feature, which is not offered by
std::shared_mutex, but we want to unify the way we use mutex, so add
these helper functions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:20:13 +0000 (11:20 +0800)]
common: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/HeartbeatMap.cc: include necessary header
Kefu Chai [Sun, 7 Jul 2019 03:19:42 +0000 (11:19 +0800)]
common/HeartbeatMap.cc: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/Cond.h: include necessary header
Kefu Chai [Sun, 7 Jul 2019 03:19:26 +0000 (11:19 +0800)]
common/Cond.h: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocls/rgw: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:18:41 +0000 (11:18 +0800)]
cls/rgw: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomds: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:18:14 +0000 (11:18 +0800)]
mds: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosdc: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:16:20 +0000 (11:16 +0800)]
osdc: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:16:03 +0000 (11:16 +0800)]
osd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoos: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:15:41 +0000 (11:15 +0800)]
os: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomsg: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:15:22 +0000 (11:15 +0800)]
msg: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:15:03 +0000 (11:15 +0800)]
mon: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:14:49 +0000 (11:14 +0800)]
mgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibrados: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:14:32 +0000 (11:14 +0800)]
librados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoinclude/utime: add utime_t<>(duration)
Kefu Chai [Sun, 7 Jul 2019 03:13:46 +0000 (11:13 +0800)]
include/utime: add utime_t<>(duration)

to ease the conversion from chrono durations to utime_t

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoinclude/C_Lock: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:12:28 +0000 (11:12 +0800)]
include/C_Lock: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/condition_variable_debug: fix wait_until()
Kefu Chai [Sun, 7 Jul 2019 03:09:36 +0000 (11:09 +0800)]
common/condition_variable_debug: fix wait_until()

to_timespec() is not a member function of time_point, it is a member
funcion of real_clock / coarse_real_clock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/timer: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:07:38 +0000 (11:07 +0800)]
common/timer: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoclient: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:06:13 +0000 (11:06 +0800)]
client: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29291 from tchaikov/wip-cmake-cleanups
Kefu Chai [Sat, 3 Aug 2019 00:46:17 +0000 (08:46 +0800)]
Merge pull request #29291 from tchaikov/wip-cmake-cleanups

cmake: require CMake v3.10.2

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoinstall-deps.sh: remove failing error catching 29403/head
Ernesto Puerta [Tue, 30 Jul 2019 16:45:44 +0000 (18:45 +0200)]
install-deps.sh: remove failing error catching

Fixes: https://tracker.ceph.com/issues/41013
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #29408 from neha-ojha/wip-bs-fsck-mnt
Neha Ojha [Fri, 2 Aug 2019 18:51:22 +0000 (11:51 -0700)]
Merge pull request #29408 from neha-ojha/wip-bs-fsck-mnt

common/options.cc: change default value of bluestore_fsck_on_mount_deep to false

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #28637 from linuxbox2/wip-rgw-blorderflat
Matt Benjamin [Fri, 2 Aug 2019 17:42:22 +0000 (13:42 -0400)]
Merge pull request #28637 from linuxbox2/wip-rgw-blorderflat

rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered

6 years agorgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered 28637/head
Matt Benjamin [Tue, 18 Jun 2019 20:12:55 +0000 (16:12 -0400)]
rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered
(RGWRados and CLS).

Probably faster, allocating less.  Definitely not slower.

Examples from single-OSD vstart.sh cluster, Ceph built at -O2

BEFORE

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real 4m48.991s
user 3m45.260s
sys 0m7.174s

(2nd run)

radosgw
Samples: 81K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 3189324729
Overhead  Shared Object         Symbol
   7.06%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   6.85%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   6.15%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   4.12%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   4.11%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.49%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   3.33%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   3.04%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.46%  radosgw               [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare
   2.45%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   2.36%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.07%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.94%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.93%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   1.85%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.79%  radosgw               [.] rgw_bucket_dir::decode
   1.76%  libceph-common.so.0   [.] operator<<
   1.42%  radosgw               [.] ceph::decode
   1.35%  libstdc++.so.6.0.25   [.] std::_Rb_tree_insert_and_rebalance
   1.33%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::ReleaseToSpans
   1.31%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.24%  [kernel]              [k] copy_user_enhanced_fast_string
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 23K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 5059851086
Overhead  Shared Object         Symbol
   8.57%  libc-2.27.so          [.] vfprintf
   4.90%  ceph-osd              [.] ceph::logging::Log::_flush
   4.66%  libc-2.27.so          [.] _IO_default_xsputn
   3.49%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.93%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.70%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.11%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   1.91%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   1.75%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.64%  libc-2.27.so          [.] _itoa_word
   1.58%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.33%  [kernel]              [k] stackleak_erase
   1.27%  [kernel]              [k] copy_user_enhanced_fast_string
   1.22%  libc-2.27.so          [.] __strlen_avx2
   1.15%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   1.12%  libstdc++.so.6.0.25   [.] std::ostream::_M_insert<unsigned long>
   1.10%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   1.04%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.01%  libc-2.27.so          [.] __tz_convert
   0.94%  [kernel]              [k] entry_SYSCALL_64
   0.94%  ceph-osd              [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   0.93%  ceph-osd              [.] ceph::logging::Log::submit_entry
For a higher level overview, try: perf top --sort comm,dso

AFTER

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real 4m51.488s
user 3m36.785s
sys 0m5.689s

(1st run)

radosgw

Samples: 52K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 4426952205
Overhead  Shared Object         Symbol
   6.11%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   5.83%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   3.89%  radosgw               [.] rgw_bucket_dir::decode
   3.73%  radosgw               [.] rgw_bucket_dir_entry::rgw_bucket_dir_entry
   3.68%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   3.37%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.14%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   2.55%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.22%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   2.02%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.79%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.77%  [kernel]              [k] n_tty_write
   1.74%  libc-2.27.so          [.] vfprintf
   1.71%  libtcmalloc.so.4.5.1  [.] operator new[]
   1.65%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.58%  radosgw               [.] ceph::decode
   1.38%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.22%  [kernel]              [k] stackleak_erase
   1.12%  libceph-common.so.0   [.] operator<<
   1.09%  libc-2.27.so          [.] _IO_default_xsputn
   1.06%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.04%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 134K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 26819176020
Overhead  Shared Object         Symbol
   8.82%  libc-2.27.so          [.] vfprintf
   6.88%  ceph-osd              [.] ceph::logging::Log::_flush
   4.80%  libc-2.27.so          [.] _IO_default_xsputn
   4.15%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   2.69%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.54%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.40%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.13%  libcls_rgw.so.1.0.0   [.] boost::container::dtl::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
   1.61%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.54%  libc-2.27.so          [.] _itoa_word
   1.49%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.41%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.41%  ceph-osd              [.] ceph::logging::Log::submit_entry
   1.23%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.19%  libc-2.27.so          [.] __tz_convert
   1.08%  libc-2.27.so          [.] __strlen_avx2
   1.08%  [kernel]              [k] stackleak_erase
   1.06%  libcls_rgw.so.1.0.0   [.] rgw_bucket_list
   1.02%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   0.99%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   0.96%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  [kernel]              [k] entry_SYSCALL_64
For a higher level overview, try: perf top --sort comm,dso

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agocmake: require CMake v3.10.2 29291/head
Kefu Chai [Wed, 24 Jul 2019 16:57:27 +0000 (00:57 +0800)]
cmake: require CMake v3.10.2

since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25962 from greenx/master
Casey Bodley [Fri, 2 Aug 2019 14:04:07 +0000 (10:04 -0400)]
Merge pull request #25962 from greenx/master

rgw: swift: bugfix: https://tracker.ceph.com/issues/37765

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoinstall-deps.sh: install cmake 3.10.2 and up on xenial
Kefu Chai [Fri, 2 Aug 2019 12:23:32 +0000 (20:23 +0800)]
install-deps.sh: install cmake 3.10.2 and up on xenial

to enable us to build on xenial, install newer cmake.
cmake 3.10.2 is the version offered by bionic.

with this change, we can safely require cmake 3.10.2 in our cmake
script. as EPEL7 offers cmake 3.13

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29401 from jan--f/c-v-inventory-parttype
Jan Fajerski [Fri, 2 Aug 2019 09:17:17 +0000 (11:17 +0200)]
Merge pull request #29401 from jan--f/c-v-inventory-parttype

ceph-volume: detect ceph-disk osd if PARTLABEL is missing

6 years agoMerge pull request #29046 from p-na/wip-pna-disable-user
Ricardo Dias [Fri, 2 Aug 2019 09:05:21 +0000 (10:05 +0100)]
Merge pull request #29046 from p-na/wip-pna-disable-user

mgr/dashboard: Provide user enable/disable capability

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #29088 from votdev/issue_24662
Ricardo Dias [Fri, 2 Aug 2019 09:02:38 +0000 (10:02 +0100)]
Merge pull request #29088 from votdev/issue_24662

mgr/dashboard: Allow disabling redirection on standby Dashboards

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge PR #29379 into master
Sage Weil [Fri, 2 Aug 2019 06:19:57 +0000 (01:19 -0500)]
Merge PR #29379 into master

* refs/pull/29379/head:
os/bluestore: s/align_down/p2align/

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #28881 into master
Sage Weil [Fri, 2 Aug 2019 06:17:48 +0000 (01:17 -0500)]
Merge PR #28881 into master

* refs/pull/28881/head:
os/Transaction: dump alloc hint flags in op

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29349 from smithfarm/wip-40975
Kefu Chai [Fri, 2 Aug 2019 03:52:07 +0000 (11:52 +0800)]
Merge pull request #29349 from smithfarm/wip-40975

rpm: put librgw lttng SOs in the librgw-devel package

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29397 from wjwithagen/wjw-fix-malloc.h
Kefu Chai [Fri, 2 Aug 2019 03:50:29 +0000 (11:50 +0800)]
Merge pull request #29397 from wjwithagen/wjw-fix-malloc.h

common: use of malloc.h is deprecated

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29396 from wjwithagen/wjw-fix-boost-1.70
Kefu Chai [Fri, 2 Aug 2019 03:49:34 +0000 (11:49 +0800)]
Merge pull request #29396 from wjwithagen/wjw-fix-boost-1.70

cmake: update FindBoost.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29181 from liewegas/wip-rgw-meta-pool-priority
J. Eric Ivancich [Fri, 2 Aug 2019 00:46:39 +0000 (20:46 -0400)]
Merge pull request #29181 from liewegas/wip-rgw-meta-pool-priority

rgw: add rgw_rados_pool_recovery_priority (default 5)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28587 from zhangsw/rgw-bugfix-versioning
J. Eric Ivancich [Thu, 1 Aug 2019 23:33:40 +0000 (19:33 -0400)]
Merge pull request #28587 from zhangsw/rgw-bugfix-versioning

rgw: fix a bug that lifecycle expiraton generates delete marker continuously

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28789 from taodd/rgw-fix-remove-bucket
J. Eric Ivancich [Thu, 1 Aug 2019 23:30:43 +0000 (19:30 -0400)]
Merge pull request #28789 from taodd/rgw-fix-remove-bucket

rgw: fix drain handles error when deleting bucket with bypass-gc option

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>