]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoinstall-deps.sh: use chacra for cmake repo 29475/head
Kefu Chai [Sun, 4 Aug 2019 03:41:26 +0000 (11:41 +0800)]
install-deps.sh: use chacra for cmake repo

two reasons:

- do not rely on kitware repo so we have better control of the
  cmake3: the `debian` directory is tracked by
  https://github.com/tchaikov/ceph-cmake
- chacra repo also offers aarch64 build of cmake3

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29472 from wjwithagen/wjw-fix-src/osdc/ObjectCacher.cc
Kefu Chai [Sat, 3 Aug 2019 16:58:13 +0000 (00:58 +0800)]
Merge pull request #29472 from wjwithagen/wjw-fix-src/osdc/ObjectCacher.cc

osdc: Fix a missing ':' for the correct namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosdc: Fix a missing ':' for the correct namespace 29472/head
Willem Jan Withagen [Sat, 3 Aug 2019 15:04:46 +0000 (17:04 +0200)]
osdc: Fix a missing ':' for the correct namespace

Clang complains:
/home/jenkins/workspace/ceph-master/src/osdc/ObjectCacher.cc:1848:22: error: no viable constructor or deduction guide for deduction of template arguments of 'unique_lock'
    std::unique_lock l{lock, std:adopt_lock};

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #29322 from rzarzynski/wip-objclass-cleanup
Kefu Chai [Sat, 3 Aug 2019 12:25:39 +0000 (20:25 +0800)]
Merge pull request #29322 from rzarzynski/wip-objclass-cleanup

objclass, osd: clean up the cls-host interface. Turn ClassHandler into singleton

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29113 from tchaikov/wip-ceph-mutex
Kefu Chai [Sat, 3 Aug 2019 12:16:24 +0000 (20:16 +0800)]
Merge pull request #29113 from tchaikov/wip-ceph-mutex

src/: s/Mutex/ceph::mutex/

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agocommon/Mutex: remove Mutex 29113/head
Kefu Chai [Mon, 22 Jul 2019 07:07:06 +0000 (15:07 +0800)]
common/Mutex: remove Mutex

as it's replaced by `ceph::mutex`

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/Cond.h: remove Cond
Kefu Chai [Mon, 22 Jul 2019 06:49:49 +0000 (14:49 +0800)]
common/Cond.h: remove Cond

as it's replaced by ceph::condition_variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/PG: support is_locked() in non-debug mode
Kefu Chai [Sat, 20 Jul 2019 11:34:02 +0000 (19:34 +0800)]
osd/PG: support is_locked() in non-debug mode

`PG` print details info in the prefix of logging messages if the PG is
being locked by current thread. but `ceph::mutex` is an alias of
`std::mutex` in non-debug mode, so neither `mutex::is_locked_by_me()` nor
`mutex::is_locked()` is supported in non-debug mode. to continue supporting
this feature, `PG::locked_by` is added to memorize the thread id of the owner
of the lock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/ceph_context.cc: include necessary header
Kefu Chai [Fri, 19 Jul 2019 16:21:04 +0000 (00:21 +0800)]
common/ceph_context.cc: include necessary header

for using functions like `lockdep_unregister_ceph_context()`

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/testmsgr: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:38:02 +0000 (21:38 +0800)]
test/testmsgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/librados: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:34:40 +0000 (21:34 +0800)]
test/librados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/compressor: do not include unused header
Kefu Chai [Thu, 18 Jul 2019 13:33:23 +0000 (21:33 +0800)]
test/compressor: do not include unused header

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/common/Throttle: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:33:14 +0000 (21:33 +0800)]
test/common/Throttle: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/obj_bencher: s/Mutex/ceph::mutex/
Kefu Chai [Thu, 18 Jul 2019 13:28:43 +0000 (21:28 +0800)]
common/obj_bencher: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoauth: remove unused forwarde declaration
Kefu Chai [Thu, 18 Jul 2019 13:28:21 +0000 (21:28 +0800)]
auth: remove unused forwarde declaration

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/librados_test_stub: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 13:18:40 +0000 (21:18 +0800)]
test/librados_test_stub: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibcephfs: remove unused #include
Kefu Chai [Wed, 17 Jul 2019 13:17:36 +0000 (21:17 +0800)]
libcephfs: remove unused #include

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocls_kvs: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 13:17:07 +0000 (21:17 +0800)]
cls_kvs: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrush: remove unused #include
Kefu Chai [Wed, 17 Jul 2019 13:16:20 +0000 (21:16 +0800)]
crush: remove unused #include

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agofusetrace: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 09:02:31 +0000 (17:02 +0800)]
fusetrace: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd_nbd: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:31:42 +0000 (16:31 +0800)]
tools/rbd_nbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd_ggate: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:31:07 +0000 (16:31 +0800)]
tools/rbd_ggate: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:30:56 +0000 (16:30 +0800)]
tools/rbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/ceph_dedup_tool: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:30:02 +0000 (16:30 +0800)]
tools/ceph_dedup_tool: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/xattr_bench: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:29:50 +0000 (16:29 +0800)]
test/xattr_bench: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/test_snap_mapper: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:29:36 +0000 (16:29 +0800)]
test/test_snap_mapper: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/osd: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:28:43 +0000 (16:28 +0800)]
test/osd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/omap_bench: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:28:33 +0000 (16:28 +0800)]
test/omap_bench: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/msgr: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:28:11 +0000 (16:28 +0800)]
test/msgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/admin_socket: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:26:36 +0000 (16:26 +0800)]
test/admin_socket: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agokv: s/Mutex/ceph::mutex/
Kefu Chai [Wed, 17 Jul 2019 08:25:09 +0000 (16:25 +0800)]
kv: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rados: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:45:15 +0000 (12:45 +0800)]
tools/rados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/immutable_object_cache: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:45:01 +0000 (12:45 +0800)]
tools/immutable_object_cache: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/cephfs: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:44:47 +0000 (12:44 +0800)]
tools/cephfs: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/os/filestore: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:44:27 +0000 (12:44 +0800)]
test/os/filestore: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/rbd_mirror: s/Mutex/ceph::Formatter/
Kefu Chai [Fri, 19 Jul 2019 17:13:38 +0000 (01:13 +0800)]
tools/rbd_mirror: s/Mutex/ceph::Formatter/

* s/Formatter/ceph::Formatter/ in tools/rbd_mirror/ImageSyncThrottler,
  as we cannot rely on `using ceph` or `using ceph::Formatter` in
  some other included header files.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/perf_local: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:43:32 +0000 (12:43 +0800)]
test/perf_local: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/osdc: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:43:18 +0000 (12:43 +0800)]
test/osdc: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/objectstore: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:43:04 +0000 (12:43 +0800)]
test/objectstore: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/mon: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:42:36 +0000 (12:42 +0800)]
test/mon: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/librbd: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:42:23 +0000 (12:42 +0800)]
test/librbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/journal: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:41:22 +0000 (12:41 +0800)]
test/journal: s/Mutex/ceph::mutex/

as we cannot create a `smart_pointer<ceph::mutex>` due to limitation of
`ceph::make_mutex()`, we need to refactor `TestObjectRecorder` to remove
its `TearDown()` method, as when `TearDown()` is called, all local
variables have been destroyed, including the `ceph::mutex` instances. so
we need to introduce a helper of `ObjectRecorderFlusher`, to flush the
objects before `mutex` instances are destroyed. to simplify the
interfaces, it flushes in its dtor. so its lifecycle should be shorter
than those of mutexes. that's why, mutexes are created before `flusher`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorgw: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:40:52 +0000 (12:40 +0800)]
rgw: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibrbd: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:38:40 +0000 (12:38 +0800)]
librbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agojournal: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:35:23 +0000 (12:35 +0800)]
journal: s/Mutex/ceph::mutex/

* FutureImpl::m_lock is an exception. as, before this change, the lock
  was initialized like `m_lock("FutureImpl::m_lock", false, false)`, see
  the declaration of
  `Mutex(const std::string &n, bool r = false, bool ld=true, bool bt=false)`
  so `m_lock` is actually not using the extra features offered by
  `Mutex` like runtime lockdeps check. and `mutex_debugging_base` does
  not allow us to disable lockdeps individually. but it does use the `is_locked()`
  method. so instead of using `ceph::mutex` directly, a cutomized
  `ceph::mutex` is added for `CEPH_DEBUG_MUTEX` build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/mutex_debug: add lockdep parameter to mutex's ctor
Kefu Chai [Thu, 25 Jul 2019 16:32:57 +0000 (00:32 +0800)]
common/mutex_debug: add lockdep parameter to mutex's ctor

so we can disable lockdep when constructing a ceph::mutex. otherwise
it's difficult to use the locks from standard library and to use
`ceph_assert(ceph_mutex_is_locked(lock))` at the same time.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/mutex_debug.h: pass "no_lockdep" to try_lock()
Kefu Chai [Sat, 20 Jul 2019 16:41:18 +0000 (00:41 +0800)]
common/mutex_debug.h: pass "no_lockdep" to try_lock()

there is chance that we want to disable lockdep, so we shold pass
`no_lockdep` down to `try_lock()` in `lock()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoinclude/Context: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:34:19 +0000 (12:34 +0800)]
include/Context: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoerasure-code: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 04:33:41 +0000 (12:33 +0800)]
erasure-code: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/ceph_mutex: add more ceph_mutex_is_*() helpers
Kefu Chai [Sun, 7 Jul 2019 08:17:35 +0000 (16:17 +0800)]
common/ceph_mutex: add more ceph_mutex_is_*() helpers

we have caller sites like

ceph_assert(!client->client_lock.is_locked_by_me())

in `src/client/Client.cc` and `src/librados/RadosClient.cc`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/ceph_mutex: improve ceph::shared_mutex
Kefu Chai [Sun, 7 Jul 2019 04:32:53 +0000 (12:32 +0800)]
common/ceph_mutex: improve ceph::shared_mutex

add helpers for supporting `ceph_mutex_is_locked()` and
`ceph_mutex_is_wlocked()` macros

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/Timer: add necessary #include
Kefu Chai [Sun, 7 Jul 2019 03:24:42 +0000 (11:24 +0800)]
common/Timer: add necessary #include

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/RWLock: add RWLock::{lock,unlock}_shared()
Kefu Chai [Sun, 7 Jul 2019 03:21:12 +0000 (11:21 +0800)]
common/RWLock: add RWLock::{lock,unlock}_shared()

so RWLock can be used with std::shared_lock,

rgw is using RWLock'prioritize_write feature, which is not offered by
std::shared_mutex, but we want to unify the way we use mutex, so add
these helper functions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:20:13 +0000 (11:20 +0800)]
common: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/HeartbeatMap.cc: include necessary header
Kefu Chai [Sun, 7 Jul 2019 03:19:42 +0000 (11:19 +0800)]
common/HeartbeatMap.cc: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/Cond.h: include necessary header
Kefu Chai [Sun, 7 Jul 2019 03:19:26 +0000 (11:19 +0800)]
common/Cond.h: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocls/rgw: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:18:41 +0000 (11:18 +0800)]
cls/rgw: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomds: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:18:14 +0000 (11:18 +0800)]
mds: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosdc: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:16:20 +0000 (11:16 +0800)]
osdc: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:16:03 +0000 (11:16 +0800)]
osd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoos: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:15:41 +0000 (11:15 +0800)]
os: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomsg: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:15:22 +0000 (11:15 +0800)]
msg: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:15:03 +0000 (11:15 +0800)]
mon: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:14:49 +0000 (11:14 +0800)]
mgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agolibrados: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:14:32 +0000 (11:14 +0800)]
librados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoinclude/utime: add utime_t<>(duration)
Kefu Chai [Sun, 7 Jul 2019 03:13:46 +0000 (11:13 +0800)]
include/utime: add utime_t<>(duration)

to ease the conversion from chrono durations to utime_t

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoinclude/C_Lock: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:12:28 +0000 (11:12 +0800)]
include/C_Lock: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/condition_variable_debug: fix wait_until()
Kefu Chai [Sun, 7 Jul 2019 03:09:36 +0000 (11:09 +0800)]
common/condition_variable_debug: fix wait_until()

to_timespec() is not a member function of time_point, it is a member
funcion of real_clock / coarse_real_clock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocommon/timer: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:07:38 +0000 (11:07 +0800)]
common/timer: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoclient: s/Mutex/ceph::mutex/
Kefu Chai [Sun, 7 Jul 2019 03:06:13 +0000 (11:06 +0800)]
client: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29291 from tchaikov/wip-cmake-cleanups
Kefu Chai [Sat, 3 Aug 2019 00:46:17 +0000 (08:46 +0800)]
Merge pull request #29291 from tchaikov/wip-cmake-cleanups

cmake: require CMake v3.10.2

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #29408 from neha-ojha/wip-bs-fsck-mnt
Neha Ojha [Fri, 2 Aug 2019 18:51:22 +0000 (11:51 -0700)]
Merge pull request #29408 from neha-ojha/wip-bs-fsck-mnt

common/options.cc: change default value of bluestore_fsck_on_mount_deep to false

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #28637 from linuxbox2/wip-rgw-blorderflat
Matt Benjamin [Fri, 2 Aug 2019 17:42:22 +0000 (13:42 -0400)]
Merge pull request #28637 from linuxbox2/wip-rgw-blorderflat

rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered

6 years agorgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered 28637/head
Matt Benjamin [Tue, 18 Jun 2019 20:12:55 +0000 (16:12 -0400)]
rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered
(RGWRados and CLS).

Probably faster, allocating less.  Definitely not slower.

Examples from single-OSD vstart.sh cluster, Ceph built at -O2

BEFORE

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real 4m48.991s
user 3m45.260s
sys 0m7.174s

(2nd run)

radosgw
Samples: 81K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 3189324729
Overhead  Shared Object         Symbol
   7.06%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   6.85%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   6.15%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   4.12%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   4.11%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.49%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   3.33%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   3.04%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.46%  radosgw               [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare
   2.45%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   2.36%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.07%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.94%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.93%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   1.85%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.79%  radosgw               [.] rgw_bucket_dir::decode
   1.76%  libceph-common.so.0   [.] operator<<
   1.42%  radosgw               [.] ceph::decode
   1.35%  libstdc++.so.6.0.25   [.] std::_Rb_tree_insert_and_rebalance
   1.33%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::ReleaseToSpans
   1.31%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.24%  [kernel]              [k] copy_user_enhanced_fast_string
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 23K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 5059851086
Overhead  Shared Object         Symbol
   8.57%  libc-2.27.so          [.] vfprintf
   4.90%  ceph-osd              [.] ceph::logging::Log::_flush
   4.66%  libc-2.27.so          [.] _IO_default_xsputn
   3.49%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.93%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.70%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.11%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   1.91%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   1.75%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.64%  libc-2.27.so          [.] _itoa_word
   1.58%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.33%  [kernel]              [k] stackleak_erase
   1.27%  [kernel]              [k] copy_user_enhanced_fast_string
   1.22%  libc-2.27.so          [.] __strlen_avx2
   1.15%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   1.12%  libstdc++.so.6.0.25   [.] std::ostream::_M_insert<unsigned long>
   1.10%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   1.04%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.01%  libc-2.27.so          [.] __tz_convert
   0.94%  [kernel]              [k] entry_SYSCALL_64
   0.94%  ceph-osd              [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   0.93%  ceph-osd              [.] ceph::logging::Log::submit_entry
For a higher level overview, try: perf top --sort comm,dso

AFTER

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real 4m51.488s
user 3m36.785s
sys 0m5.689s

(1st run)

radosgw

Samples: 52K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 4426952205
Overhead  Shared Object         Symbol
   6.11%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   5.83%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   3.89%  radosgw               [.] rgw_bucket_dir::decode
   3.73%  radosgw               [.] rgw_bucket_dir_entry::rgw_bucket_dir_entry
   3.68%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   3.37%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.14%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   2.55%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.22%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   2.02%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.79%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.77%  [kernel]              [k] n_tty_write
   1.74%  libc-2.27.so          [.] vfprintf
   1.71%  libtcmalloc.so.4.5.1  [.] operator new[]
   1.65%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.58%  radosgw               [.] ceph::decode
   1.38%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.22%  [kernel]              [k] stackleak_erase
   1.12%  libceph-common.so.0   [.] operator<<
   1.09%  libc-2.27.so          [.] _IO_default_xsputn
   1.06%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.04%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 134K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 26819176020
Overhead  Shared Object         Symbol
   8.82%  libc-2.27.so          [.] vfprintf
   6.88%  ceph-osd              [.] ceph::logging::Log::_flush
   4.80%  libc-2.27.so          [.] _IO_default_xsputn
   4.15%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   2.69%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.54%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.40%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.13%  libcls_rgw.so.1.0.0   [.] boost::container::dtl::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
   1.61%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.54%  libc-2.27.so          [.] _itoa_word
   1.49%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.41%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.41%  ceph-osd              [.] ceph::logging::Log::submit_entry
   1.23%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.19%  libc-2.27.so          [.] __tz_convert
   1.08%  libc-2.27.so          [.] __strlen_avx2
   1.08%  [kernel]              [k] stackleak_erase
   1.06%  libcls_rgw.so.1.0.0   [.] rgw_bucket_list
   1.02%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   0.99%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   0.96%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  [kernel]              [k] entry_SYSCALL_64
For a higher level overview, try: perf top --sort comm,dso

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agocmake: require CMake v3.10.2 29291/head
Kefu Chai [Wed, 24 Jul 2019 16:57:27 +0000 (00:57 +0800)]
cmake: require CMake v3.10.2

since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25962 from greenx/master
Casey Bodley [Fri, 2 Aug 2019 14:04:07 +0000 (10:04 -0400)]
Merge pull request #25962 from greenx/master

rgw: swift: bugfix: https://tracker.ceph.com/issues/37765

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoinstall-deps.sh: install cmake 3.10.2 and up on xenial
Kefu Chai [Fri, 2 Aug 2019 12:23:32 +0000 (20:23 +0800)]
install-deps.sh: install cmake 3.10.2 and up on xenial

to enable us to build on xenial, install newer cmake.
cmake 3.10.2 is the version offered by bionic.

with this change, we can safely require cmake 3.10.2 in our cmake
script. as EPEL7 offers cmake 3.13

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29401 from jan--f/c-v-inventory-parttype
Jan Fajerski [Fri, 2 Aug 2019 09:17:17 +0000 (11:17 +0200)]
Merge pull request #29401 from jan--f/c-v-inventory-parttype

ceph-volume: detect ceph-disk osd if PARTLABEL is missing

6 years agoMerge pull request #29046 from p-na/wip-pna-disable-user
Ricardo Dias [Fri, 2 Aug 2019 09:05:21 +0000 (10:05 +0100)]
Merge pull request #29046 from p-na/wip-pna-disable-user

mgr/dashboard: Provide user enable/disable capability

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #29088 from votdev/issue_24662
Ricardo Dias [Fri, 2 Aug 2019 09:02:38 +0000 (10:02 +0100)]
Merge pull request #29088 from votdev/issue_24662

mgr/dashboard: Allow disabling redirection on standby Dashboards

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge PR #29379 into master
Sage Weil [Fri, 2 Aug 2019 06:19:57 +0000 (01:19 -0500)]
Merge PR #29379 into master

* refs/pull/29379/head:
os/bluestore: s/align_down/p2align/

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #28881 into master
Sage Weil [Fri, 2 Aug 2019 06:17:48 +0000 (01:17 -0500)]
Merge PR #28881 into master

* refs/pull/28881/head:
os/Transaction: dump alloc hint flags in op

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29349 from smithfarm/wip-40975
Kefu Chai [Fri, 2 Aug 2019 03:52:07 +0000 (11:52 +0800)]
Merge pull request #29349 from smithfarm/wip-40975

rpm: put librgw lttng SOs in the librgw-devel package

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29397 from wjwithagen/wjw-fix-malloc.h
Kefu Chai [Fri, 2 Aug 2019 03:50:29 +0000 (11:50 +0800)]
Merge pull request #29397 from wjwithagen/wjw-fix-malloc.h

common: use of malloc.h is deprecated

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29396 from wjwithagen/wjw-fix-boost-1.70
Kefu Chai [Fri, 2 Aug 2019 03:49:34 +0000 (11:49 +0800)]
Merge pull request #29396 from wjwithagen/wjw-fix-boost-1.70

cmake: update FindBoost.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29181 from liewegas/wip-rgw-meta-pool-priority
J. Eric Ivancich [Fri, 2 Aug 2019 00:46:39 +0000 (20:46 -0400)]
Merge pull request #29181 from liewegas/wip-rgw-meta-pool-priority

rgw: add rgw_rados_pool_recovery_priority (default 5)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28587 from zhangsw/rgw-bugfix-versioning
J. Eric Ivancich [Thu, 1 Aug 2019 23:33:40 +0000 (19:33 -0400)]
Merge pull request #28587 from zhangsw/rgw-bugfix-versioning

rgw: fix a bug that lifecycle expiraton generates delete marker continuously

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28789 from taodd/rgw-fix-remove-bucket
J. Eric Ivancich [Thu, 1 Aug 2019 23:30:43 +0000 (19:30 -0400)]
Merge pull request #28789 from taodd/rgw-fix-remove-bucket

rgw: fix drain handles error when deleting bucket with bypass-gc option

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #29178 from ivancich/wip-small-rgw-efficiency
J. Eric Ivancich [Thu, 1 Aug 2019 22:36:30 +0000 (18:36 -0400)]
Merge pull request #29178 from ivancich/wip-small-rgw-efficiency

rgw: add a small efficiency

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29179 from ivancich/wip-bucket-list-excessive-max-entries
J. Eric Ivancich [Thu, 1 Aug 2019 22:34:35 +0000 (18:34 -0400)]
Merge pull request #29179 from ivancich/wip-bucket-list-excessive-max-entries

rgw: mitigate bucket list with max-entries excessively high

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #29240 from mdw-at-linuxbox/wip-master-rgw-swift-metadata-versioning
J. Eric Ivancich [Thu, 1 Aug 2019 22:28:19 +0000 (18:28 -0400)]
Merge pull request #29240 from mdw-at-linuxbox/wip-master-rgw-swift-metadata-versioning

rgw: Fix bucket versioning vs. swift metadata bug.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
6 years agoMerge pull request #28331 from linuxbox2/wip-rgw-uinfo-by-acc
Matt Benjamin [Thu, 1 Aug 2019 20:07:34 +0000 (16:07 -0400)]
Merge pull request #28331 from linuxbox2/wip-rgw-uinfo-by-acc

rgw: permit rgw-admin to populate user info by access-key

6 years agoMerge pull request #29413 from tchaikov/wip-remove-test/python
Kefu Chai [Thu, 1 Aug 2019 16:38:28 +0000 (00:38 +0800)]
Merge pull request #29413 from tchaikov/wip-remove-test/python

test/python: remove stale tests

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #29445 from ricardoasmarques/fix-doc-rbd-namespace
Jason Dillaman [Thu, 1 Aug 2019 16:03:25 +0000 (12:03 -0400)]
Merge pull request #29445 from ricardoasmarques/fix-doc-rbd-namespace

doc: Fix rbd namespace documentation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc: Fix rbd namespace documentation 29445/head
Ricardo Marques [Thu, 1 Aug 2019 10:27:22 +0000 (11:27 +0100)]
doc: Fix rbd namespace documentation

Pool and namespace should be separated by a slash.

Fixes: https://tracker.ceph.com/issues/41046
Signed-off-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #29446 from tchaikov/wip-cmake-tox-test
Kefu Chai [Thu, 1 Aug 2019 14:04:30 +0000 (22:04 +0800)]
Merge pull request #29446 from tchaikov/wip-cmake-tox-test

cmake: add add_tox_test()

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #29423 from rhcs-dashboard/wip-41023-master
Kefu Chai [Thu, 1 Aug 2019 13:43:15 +0000 (21:43 +0800)]
Merge pull request #29423 from rhcs-dashboard/wip-41023-master

doc: Show Jenkins commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #29297 from iotcg/starlingx
Kefu Chai [Thu, 1 Aug 2019 13:41:06 +0000 (21:41 +0800)]
Merge pull request #29297 from iotcg/starlingx

ceph-objectstore-tool: set log date format

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoceph-volume: fall back to PARTTYPE if PARTLABEL is empty 29401/head
Jan Fajerski [Tue, 30 Jul 2019 15:32:06 +0000 (17:32 +0200)]
ceph-volume: fall back to PARTTYPE if PARTLABEL is empty

In some cases ceph-disk does not populate PARTLABEL for wal and db
partitions. This commit adds the assumption that the empty string is a
valid label and falls back to identifying those disks by PARTTYPE.

Fixes: https://tracker.ceph.com/issues/40917
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
6 years agoceph-volume: adjust tests for empty PARTLABEL fields
Jan Fajerski [Tue, 30 Jul 2019 15:29:09 +0000 (17:29 +0200)]
ceph-volume: adjust tests for empty PARTLABEL fields

It seems like ceph-disk sometimes creates OSDs where the wal and db
partitions have no PARTLABEL set. This commit alters tests so that the
empty string is considered a valid label.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>