crimson/osd/heartbeat: adapt to new MOSDPing fields

Signed-off-by: Sage Weil <sage@redhat.com>

crimson/osdmap_service: add get_mnow(), get_up_epoch()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PeeringState: take HeartbeatStamps refs for current interval

Signed-off-by: Sage Weil <sage@redhat.com>

osd: track clock delta between peer OSDs

We need to keep track of the monotonic clock deltas between peer OSDs in
order to be able to exchange timestamps across messages.  We need an
upper and lower bound on this delta, depending on the context where it is
used.

We can use the existing ping messages to get this by assuming that a ping
message is sent instantly to get a bound, and to share our delta in a
follow-up reply to share the other bound.  The ping sender will get both
bounds with a single ping + ping_reply exchange.  The ping receiver will
get the delta value from the next round's ping.

Include up_from in the ping messages to ensure we don't mix up different
instances of the same OSD.

Signed-off-by: Sage Weil <sage@inktank.com>

osd: add get_mnow() interface to OSDService, PG, PeeringState

Single helper to provide a monotonic time offset from startup_time.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: record startup_time

Signed-off-by: Sage Weil <sage@redhat.com>

osd: some minor refactoring/cleanup in handle_osd_ping

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29337 into master

* refs/pull/29337/head:
mon: enable telemetry module by default
mgr/telemetry: force re-opt-in if the report contents change
mgr/telemetry: less noise in the log
mgr/telemetry: wake up serve on config change
mgr/telemetry: track telemetry report revisions

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29457 from tchaikov/wip-consolidate-run-tox

cmake: one run_tox.sh to rule them all

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

script/run_tox.sh: add --help option

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #29492 into master

* refs/pull/29492/head:
doc: Fix for new ceph-devel mailing list

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

cmake: pass tox envs to run_tox.sh explicitly

* add two optional keyword parameters to `add_tox_test()`
* use the default `TOX_PATH` parameter instead passing it explicitly,
  as it is always `${CMAKE_CURRENT_SOURCE_DIR}`
* drop the code to guess the tox envs in `run_tox.sh`, and always pass
  them explicitly in CMake script using the `--tox-envs` argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29471 from tchaikov/wip-googletest

googletest: pick up change to suppress CMP0048 warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: Fix for new ceph-devel mailing list

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #29458 from mikulely/rgw-drop-flush

rgw: drop dead flush_read_list declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27675 from jan--f/c-v-console-logging-to-stderr

ceph-volume: print most logging messages to stderr

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

cmake: move tests to where they are implemented

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: do not add gtest-parallel_ext if already added

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use script/run_tox.sh for running tox for dashboard

* adapt script/run_tox.sh to use different envs for dashboard's tox
  test.
* use script/run_tox.sh for running dashboard's tox test
* remove pybind/mgr/dashboard/run-tox.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake/modules/AddCephTest.cmake: make ${name}-venv a target

so we can prepare the venv for tox with `make tests`. and it's optional,
as run_tox.sh will check for it and setup a venv if it's not around.

Signed-off-by: Kefu Chai <kchai@redhat.com>

src: remove unused run-tox.sh

they are not used anymore after b9c9f74f

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29393 from ifed01/wip-ifed-bluestore-header-cleanup

os/bluestore: remove unused _tune_cache_size() method declaration

Reviewed-by: Mark Nelson <mnelson@redhat.com>

mgr/dashboard: update npm packages (#28343)

mgr/dashboard: update npm packages

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29403 from rhcs-dashboard/wip-41013-master

install-deps.sh: remove failing error catching

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29476 from tchaikov/wip-test/journal-close-object

test/journal: always close object

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/journal: always close object

otherwise when we destruct `TestObjectRecorder::Handler::lock` or
`journal::JournalRecorder::m_object_locks`, it/they would be still
being waited by some condition variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29474 from wjwithagen/wjw-fix-rbd_ggate_Server.cc

rbd-ggate: fix compile errors from ceph::mutex update

Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd-ggate: fix compile errors from ceph::mutex update

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

googletest: pick up change to suppress CMP0048 warning

newer cmake complains about googletest should have set CMP0048

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29472 from wjwithagen/wjw-fix-src/osdc/ObjectCacher.cc

osdc: Fix a missing ':' for the correct namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>

osdc: Fix a missing ':' for the correct namespace

Clang complains:
/home/jenkins/workspace/ceph-master/src/osdc/ObjectCacher.cc:1848:22: error: no viable constructor or deduction guide for deduction of template arguments of 'unique_lock'
    std::unique_lock l{lock, std:adopt_lock};

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #29322 from rzarzynski/wip-objclass-cleanup

objclass, osd: clean up the cls-host interface. Turn ClassHandler into singleton

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29113 from tchaikov/wip-ceph-mutex

src/: s/Mutex/ceph::mutex/

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

common/Mutex: remove Mutex

as it's replaced by `ceph::mutex`

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/Cond.h: remove Cond

as it's replaced by ceph::condition_variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PG: support is_locked() in non-debug mode

`PG` print details info in the prefix of logging messages if the PG is
being locked by current thread. but `ceph::mutex` is an alias of
`std::mutex` in non-debug mode, so neither `mutex::is_locked_by_me()` nor
`mutex::is_locked()` is supported in non-debug mode. to continue supporting
this feature, `PG::locked_by` is added to memorize the thread id of the owner
of the lock.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_context.cc: include necessary header

for using functions like `lockdep_unregister_ceph_context()`

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/testmsgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/librados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/compressor: do not include unused header

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/common/Throttle: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/obj_bencher: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth: remove unused forwarde declaration

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/librados_test_stub: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

libcephfs: remove unused #include

Signed-off-by: Kefu Chai <kchai@redhat.com>

cls_kvs: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

crush: remove unused #include

Signed-off-by: Kefu Chai <kchai@redhat.com>

fusetrace: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rbd_nbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rbd_ggate: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/ceph_dedup_tool: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/xattr_bench: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/test_snap_mapper: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/omap_bench: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/msgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/admin_socket: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

kv: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/immutable_object_cache: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/cephfs: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/os/filestore: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rbd_mirror: s/Mutex/ceph::Formatter/

* s/Formatter/ceph::Formatter/ in tools/rbd_mirror/ImageSyncThrottler,
  as we cannot rely on `using ceph` or `using ceph::Formatter` in
  some other included header files.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/perf_local: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osdc: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/objectstore: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/mon: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/librbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/journal: s/Mutex/ceph::mutex/

as we cannot create a `smart_pointer<ceph::mutex>` due to limitation of
`ceph::make_mutex()`, we need to refactor `TestObjectRecorder` to remove
its `TearDown()` method, as when `TearDown()` is called, all local
variables have been destroyed, including the `ceph::mutex` instances. so
we need to introduce a helper of `ObjectRecorderFlusher`, to flush the
objects before `mutex` instances are destroyed. to simplify the
interfaces, it flushes in its dtor. so its lifecycle should be shorter
than those of mutexes. that's why, mutexes are created before `flusher`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

journal: s/Mutex/ceph::mutex/

* FutureImpl::m_lock is an exception. as, before this change, the lock
  was initialized like `m_lock("FutureImpl::m_lock", false, false)`, see
  the declaration of
  `Mutex(const std::string &n, bool r = false, bool ld=true, bool bt=false)`
  so `m_lock` is actually not using the extra features offered by
  `Mutex` like runtime lockdeps check. and `mutex_debugging_base` does
  not allow us to disable lockdeps individually. but it does use the `is_locked()`
  method. so instead of using `ceph::mutex` directly, a cutomized
  `ceph::mutex` is added for `CEPH_DEBUG_MUTEX` build.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/mutex_debug: add lockdep parameter to mutex's ctor

so we can disable lockdep when constructing a ceph::mutex. otherwise
it's difficult to use the locks from standard library and to use
`ceph_assert(ceph_mutex_is_locked(lock))` at the same time.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/mutex_debug.h: pass "no_lockdep" to try_lock()

there is chance that we want to disable lockdep, so we shold pass
`no_lockdep` down to `try_lock()` in `lock()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/Context: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

erasure-code: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_mutex: add more ceph_mutex_is_*() helpers

we have caller sites like

ceph_assert(!client->client_lock.is_locked_by_me())

in `src/client/Client.cc` and `src/librados/RadosClient.cc`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_mutex: improve ceph::shared_mutex

add helpers for supporting `ceph_mutex_is_locked()` and
`ceph_mutex_is_wlocked()` macros

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/Timer: add necessary #include

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/RWLock: add RWLock::{lock,unlock}_shared()

so RWLock can be used with std::shared_lock,

rgw is using RWLock'prioritize_write feature, which is not offered by
std::shared_mutex, but we want to unify the way we use mutex, so add
these helper functions.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/HeartbeatMap.cc: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/Cond.h: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>

cls/rgw: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdc: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

os: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/utime: add utime_t<>(duration)

to ease the conversion from chrono durations to utime_t

Signed-off-by: Kefu Chai <kchai@redhat.com>

include/C_Lock: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/condition_variable_debug: fix wait_until()

to_timespec() is not a member function of time_point, it is a member
funcion of real_clock / coarse_real_clock.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/timer: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

client: s/Mutex/ceph::mutex/

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29291 from tchaikov/wip-cmake-cleanups

cmake: require CMake v3.10.2

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

install-deps.sh: remove failing error catching

Fixes: https://tracker.ceph.com/issues/41013
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>