This is a milestone in the CephFS shell project. QA testing is still in the
works and will be merged separately. Installation code via CMake will also be
written in another PR.
* refs/pull/23158/head:
doc: add cephfs-shell to cephfs doc index
cephfs-shell: add new CephFS shell
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
The CephFS shell is an alternative client to access CephFS without mounting in
the usual way. The shell provides commands to interactively access the file
system, like put, ls, chmod, etc.
test: use death_test_style="threadsafe" for Mutex.NormalAsserts
use threadsafe for running the Mutex assert test to run the test from
beginning to avoid the hang. this test overrides the symbol of
ceph::__ceph_assert_fail() with a local one offered by the application
links against libceph-common. but we intentionally forbid this behavior:
we do not allow libceph-common to reference the symbols exposed from
application with the same name. see http://tracker.ceph.com/issues/25154
Kefu Chai [Thu, 2 Aug 2018 11:04:30 +0000 (19:04 +0800)]
cmake: require 4.0.0 <= libfmt-dev < 5.0.0
FMT_VERSION is now living in fmt/core.h in libfmt >= 5.0, so we cannot
parse fmt/format.h for libfmt's version anymore. also we don't need
Findfmt.cmake now. because libfmt-dev 4.0 and up is required by seastar,
also it's libfmt-dev 3.x's cmake module which fails to offer fmt::fmt
target. in other words, it's safe to drop libfmt 3.x support. but 5.0.0
is not compatible with seastar, we will have failures like:
In file included from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/aligned_buffer.hh:25:0,
from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/reactor.hh:26,
from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/alien.hh:35,
from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/src/core/alien.cc:23:
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/print.hh:
In function 'seastar::sstring seastar::format(const char*, A&& ...)':
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/print.hh:135:10:
error: 'MemoryWriter' is not a member of 'fmt'
fmt::MemoryWriter out;
^~~~~~~~~~~~
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/print.hh:136:5:
error: 'out' was not declared in this scope
out.write(fmt, std::forward<A>(a)...);
^~~
so, we should build libfmt even if we have libfmt-dev 5.x
Kefu Chai [Thu, 2 Aug 2018 05:59:31 +0000 (13:59 +0800)]
cmake: extract sanitizer detection out
also, we don't need to pass '-lasan' or '-ltsan' to linker. it's
suggested to use the "-fsanitize=${sanitizer}".
please note, this module is compatible with the one used in seastar on
purpose, as seastar is included in ceph using add_subdirectory(), and it
in turn add its own cmake modules directory using list(APPEND ...), so
cmake/modules/FindSantitizers.cmake is prefered over
src/seastar/cmake/FindSantitizers.cmake.
Kefu Chai [Thu, 2 Aug 2018 05:45:51 +0000 (13:45 +0800)]
cmake: fix StdFilesystem detection
before this change, we fails to detect std::filesystem with clang++,
because the cmake project created by try_compile() only expands
following options passed from its caller:
- COMPILE_DEFINITIONS
- INCLUDE_DIRECTORIES
- LINK_DIRECTORIES
- LINK_LIBRARIES
which do not include CMAKE_CXX_FLAGS, so either we need to (ab)use
COMPILE_DEFINITIONS for passing -std=c++17, or we can change the
CMAKE_CXX_FLAGS in the parent env, as it turns out the created cmake
project does inherit this flag from current project. in this change,
we use the COMPILE_DEFINITIONS approach: simpler this way. and we
can drop it once cmake 3.8 is required.
Jianpeng Ma [Thu, 2 Aug 2018 08:37:39 +0000 (16:37 +0800)]
cmake: fix a cmake error when with -DALLOCATOR=jemalloc.
When exec: ./do_cmake.sh -DALLOCATOR=jemalloc. Met the following
messages:
>> CMake Error at cmake/modules/FindJeMalloc.cmake:28 (endforeach):
endforeach An ENDFOREACH command was found outside of a proper FOREACH
ENDFOREACH structure. Or its arguments did not match the opening FOREACH
command.
>> Call Stack (most recent call first):
CMakeLists.txt:326 (find_package)
Yan, Zheng [Thu, 2 Aug 2018 02:05:12 +0000 (10:05 +0800)]
mds: update MDSRank::cluster_degraded before handling mds failure
Migrator checks MDSRank::is_cluster_degraded() to decide if it needs to
send message to other mds. MDSRank::cluster_degraded should get updated
before calling Migrator::handle_mds_failure_or_stop().
Introduced by commit 7de9da4a "mds: handle discontinuous mdsmap"
* refs/pull/23195/head:
mds: use vector for context lists
common: make C_ContextsBase container agnostic
common: make finish_contexts container agnostic
mds: add vector def for MDS contexts
mds: use compact map to manage waiting list
Sage Weil [Wed, 1 Aug 2018 01:16:30 +0000 (20:16 -0500)]
Merge PR #23223 into master
* refs/pull/23223/head:
osd/PG: kill dead functions and related options
iosd/osd_type: kill unused input ec_pool for iterate_mayberw_back_to
common: kill dead options
osd/PG: do not initialize up/acting twice
osd/PG: clear missing_loc properly if last location is gone
Sage Weil [Tue, 31 Jul 2018 22:23:48 +0000 (17:23 -0500)]
Merge PR #22692 into master
* refs/pull/22692/head:
doc/mgr/devicehealth: document devicehealth module
doc/rados/operations/health-checks: document DEVICE_HEALTH* messages
mgr/devicehealth: fix style for returns
mgr/devicehealth: use constants for health warnings
mgr/devicehealth: deal with as many daemons as we can until limit
mgr/devicehealth: warn if too many daemons are expected to fail soon
mgr/devicehealth: set primary-affinity 0 for failing devices
msg/devicehealth: fix config options
mgr/devicehealth: only fetch osdmap once from check_health
mgr/devicehealth: revise health messages
mgr/devicehealth: add 'device check-health' command and run periodically
mgr/devicehealth: fix new options
mgr/devicehealth: add helpers to life_expectancy_response()
mgr/devicehealth: simplify setting defaults
common/blkdev remove debug statements
Yaarit Hatuka [Mon, 25 Jun 2018 13:19:22 +0000 (08:19 -0500)]
mgr/devicehealth: add helpers to life_expectancy_response()
- if mark_out_threshold is met we write to log.warn instead of raising a
health warning.
- check that OSD is 'in' before calling mark_out().
- raise a health warning in case OSD is marked 'out' but still has PGs
attached to it.
- cast thresholds default values to string.
- add SCSI multipath support to health warning message.
- change health warning message.
src/osd/PG.cc: remove redundant call to trim_log()
This change is motived by the failure tracked in
https://tracker.ceph.com/issues/25198. The failure highlights a case, when a
call to trim_log() after the PG has recovered, races with the previous op,
on a replica OSD. Since the previous operation has not completed, the
last_complete value for that OSD is not valid, when we try to trim the
log. It is also worth noting that the race is due to MOSDPGTrim going through
the strict queue as a peering message vs regular ops going through the
non-strict queue.
During the investigation of this bug, we noticed that, with
https://tracker.ceph.com/issues/23979, we allow pg log trimming to
happen on the primary and replicas, whenever we cross the upper bound of
the pg log. This also ensures that pg log trimming happens while processing
any new op.
Therefore, the function trim_log(), which earlier served the purpose of
trimming logs on the primary and replicas, just before the PG went into
the Recovered state, is no more required. This acted like a last line of
defense to trim logs, when we did not need the logs any more. But, this call
seems redundant now, because, we are limiting the pg log length at all times.
Sage Weil [Mon, 30 Jul 2018 19:18:07 +0000 (14:18 -0500)]
pybind/rados/rados: do not pass prval from stack
The prval is a pointer to an int to write the final completion code of
the rados op. This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).
We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).
Fixes: http://tracker.ceph.com/issues/25175 Signed-off-by: Sage Weil <sage@redhat.com>
projects / ceph.git / log