Kefu Chai [Tue, 14 Jan 2020 11:28:35 +0000 (19:28 +0800)]
include/cpp-btree: adapt from abseil::btree
adapt based on btree implementation from Abseil Common Libraries.
with following changes:
* remove abseil dependencies. as abseil libraries do support C++11,
they are written in C++11 compatible way. and use its homebrew
alternatives for the facilities offered by C++14 and C++17. but
Ceph is in C++17, so replace those facilities using the ones
from the C++ standard library
* change `key_compare_to_adapter` to adapt to arithmetic types, and
those who have `compare()` defined.
* move SlotPolicy into *_params types, simpler this way.
* continue using *_fields types. the layout facilities is nice. but
we don't need this flexibility at this moment.
* continue using `empty_base_handle` for EBCO, CompressedTuple is
nice, but it involves more dependencies which we don't want at
this moment.
* node handle support is dropped. it's part of C++17 standard for
node-based containers. we don't use it at this moment. but it's
relatively simple if we want to add it back.
Sage Weil [Tue, 14 Jan 2020 21:59:45 +0000 (15:59 -0600)]
Merge PR #32095 into master
* refs/pull/32095/head:
ceph-volume: raw: activate: drop --all, --osd-id and --osd-fsid args
raw: fix activate args
raw: --osd-fsid throughout
ceph-volume: add raw mode
ceph-volume: show devices with bluestore labels and unavailable
Casey Bodley [Tue, 14 Jan 2020 14:42:52 +0000 (09:42 -0500)]
qa/rgw: remove test against hadoop v2.8.5
the hadoop branch rel/release-2.8.5 fails to build with:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:37 min
[INFO] Finished at: 2020-01-14T13:09:02Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) on project hadoop-aws: An Ant BuildException has occured: Unable to create javax script engine for javascript
Sage Weil [Fri, 6 Dec 2019 20:08:12 +0000 (14:08 -0600)]
ceph-volume: add raw mode
list, prepare, activate, create (prepare+activate) basic bluestore OSDs
on a single device.
No support (or plans to ever support) dmcrypt.
I think support for db/wal could be added later without much trouble.
For now, we require --no-systemd and --bluestore flags because we haven't
implemented support for anything else but suspect we will in the future,
and would like to ensure users are being explicit about what they're
asking for.
Rodrigo Severo [Fri, 10 Jan 2020 14:38:19 +0000 (11:38 -0300)]
doc/install/upgrading-ceph: using systemd commands in Ubuntu instructions
Before this change, we were using the "restart" command which is a symlink to
initctl(8). and it's offered by upstart. since ubuntu/vivid, ubuntu moved from
upstart to systemd.
Please see https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039465.html
Casey Bodley [Mon, 13 Jan 2020 19:44:47 +0000 (14:44 -0500)]
Merge pull request #32534 from cbodley/wip-43512
rgw multisite: enforce spawn window for incremental data sync
Reviewed-by: Daniel Gryniewicz <dang@redhat.com> Reviewed-by: Eric J. Ivancich <ivancich@redhat.com> Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Kefu Chai [Sun, 12 Jan 2020 09:59:11 +0000 (17:59 +0800)]
ceph.spec.in: package .pyc files on el7
* package .pyc files on el7
per
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_byte_compiling
it's required to package .pyc files
* use python3 to bytecompile .py files on el7
per
https://fedoraproject.org/wiki/Changes/No_more_automagic_Python_bytecompilation#Status_quo,
`brp-python-bytecompile` script bytecompile .py files automatically, the
.py files under interpreter-specific directories, such as `/usr/lib/python3.6/`,
are compiled using the appropriate compiler. but the mgr modules are
located under `/usr/share/ceph/mgr/`. they are compiled using
`${__python}`, which is still python2 in RHEL/CentOS 7. since we've
dropped the support of python2 in octopus. we should either stop
bytecompiling the .py source files or start using python3 to compile
them. otherwise python2 will fail to compile source files using python3
specific syntax like:
Jason Dillaman [Sun, 12 Jan 2020 15:05:04 +0000 (10:05 -0500)]
rbd-mirror: skip closing local image if it was already closed
If the journal replayer finishes relaying (error or promotion), it will
close the local image. However, the image replayer state machine will also
shut down the journal replayer (again) which might result in attempting
to close the local image again.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Kefu Chai [Sat, 11 Jan 2020 14:41:59 +0000 (22:41 +0800)]
common/strtol.cc: refactor strict_si_cast() to silence warnings
to silence warnings like
../src/common/strtol.cc:286:7: warning: implicit conversion from 'promoted_t' (aka 'long long') to 'double' changes value from 9223372036854775807 to 9223372036854775808
[-Wimplicit-int-float-conversion]
static_cast<promoted_t>(std::numeric_limits<T>::max()) / pow (10, m)) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~
../src/common/strtol.cc:294:15: note: in instantiation of function template specialization 'strict_si_cast<long>' requested here
template long strict_si_cast<long>(std::string_view str, std::string *err);
^
Kefu Chai [Sat, 11 Jan 2020 14:30:49 +0000 (22:30 +0800)]
src/: mark final class as final
to silence warnings from clang-10 like:
```
../src/os/bluestore/ceph_aio.h:120:18: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
~aio_queue_t() final {
^
../src/os/bluestore/ceph_aio.h:108:8: note: mark 'aio_queue_t' as 'final' to silence this warning
struct aio_queue_t : public io_queue_t {
^
1 warning generated.
```
Kefu Chai [Sat, 11 Jan 2020 08:37:17 +0000 (16:37 +0800)]
crimson: avoid creating temporary with protected dtor
to address the compiler error from clang 9.0.1:
../src/crimson/os/cyanstore/cyan_collection.cc:16:34: error: temporary of type 'boost::intrusive_ref_counter<Object, boost::thread_unsafe_counter>' has protected destructor
return new crimson::os::Object{};
^
/opt/ceph/include/boost/smart_ptr/intrusive_ref_counter.hpp:157:30: note: declared protected here
BOOST_DEFAULTED_FUNCTION(~intrusive_ref_counter(), {})
^
Kefu Chai [Sat, 11 Jan 2020 08:26:54 +0000 (16:26 +0800)]
crimson: move() explicitly to avoid copy
to silence following warnings from clang 9.0.1:
src/crimson/osd/heartbeat.cc:143:14: warning: local variable 'extras' will be copied despite being returned by name [-Wreturn-std-move]
return extras;
^~~~~~
../src/crimson/osd/heartbeat.cc:143:14: note: call 'std::move' explicitly to avoid copying
return extras;
^~~~~~
std::move(extras)
Kefu Chai [Sat, 11 Jan 2020 08:07:42 +0000 (16:07 +0800)]
crimson: do not capture unused variables
this silences warnings from clang:
src/crimson/osd/heartbeat.cc:109:14: warning: lambda capture 'this' is
not used [-Wunused-lambda-capture]
.then([this, &info=peer_info->second] (auto xcon_front, auto
xcon_back) {
^~~~~
Kefu Chai [Sat, 11 Jan 2020 10:29:29 +0000 (18:29 +0800)]
cmake: do not include global_context.cc multiple times
otherwise ODR is violated:
==449025==ERROR: AddressSanitizer: odr-violation (0x000000f03700):
[1] size=8 'g_ceph_context' ../src/global/global_context.cc:24:14
[2] size=8 'g_ceph_context' ../src/global/global_context.cc:24:14
These globals were registered at these points:
[1]:
#0 0x4779bd in __asan_register_globals (/var/ssd/ceph/clang-build/bin/ceph-conf+0x4779bd)
#1 0x56e9cb in asan.module_ctor (/var/ssd/ceph/clang-build/bin/ceph-conf+0x56e9cb)
[2]:
#0 0x4779bd in __asan_register_globals (/var/ssd/ceph/clang-build/bin/ceph-conf+0x4779bd)
#1 0x7fe5fed12aeb in asan.module_ctor (/var/ssd/ceph/clang-build/lib/libceph-common.so.2+0x2f34aeb)
==449025==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
Ronen Friedman [Sat, 11 Jan 2020 07:42:49 +0000 (09:42 +0200)]
crimson: fix aarch64 ctest failure by removing some lambda attributes
There seems to be no "universally accepted" way to declare a lambda as
[[always_inline]]. "Universally accepted" here meaning: accepted with
no error or warning by gcc8, gcc9 and clang.
See for example: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60503
Patrick Donnelly [Sat, 11 Jan 2020 00:16:00 +0000 (16:16 -0800)]
Merge PR #32213 into master
* refs/pull/32213/head:
cephfs-shell: don't catch libcephfs.Error unnecessarily
cephfs-shell: make every command set a return value on failure
pybind/cephfs: move LibCephFSStateError closer to base class
pybind/cephfs: add method to get error code in Error and OSError
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Milind Changire [Fri, 10 Jan 2020 06:31:34 +0000 (12:01 +0530)]
mds: throttle scrub start for multiple active MDS
* add check to "scrub start" command handler to dishonor request if
multiple MDS are "active" for a recursive scrub request
* add check to MDSRank::handle_mds_map() to see if scrubbing is in
progress and max_mds is greater than 1, if so then scrubbing is
aborted. This is supposed to take care of the condition when scrubbing
has been started with max_mds == 1 and then bumped to a higher value
Kefu Chai [Fri, 10 Jan 2020 05:28:10 +0000 (13:28 +0800)]
qa/tasks/mgr: set mgr module option with --force
if mgr is not active, monitor will refuse to set any option consumed by
mgr modules.
the reason the tests pass somtimes is that, we have a racing here:
1. stop all mgr daemons
2. MgrMonitor gets updated and updates its mgr_module_options
accordingly.
3. in TestDashboard.setUp(), we reset the port number for dashboard
using "ceph config set mgr mgr/dashboard/y/ssl_server_port 7789"
4. restart all mgr daemons
but the 2nd step and 3rd step could race with each other, if the 2nd
step happens after 3rd step, the test passes. otherwise it fails.
in this change, "--force" is passed to the "ceph config set" command,
so ConfigMonitor can bypass the sanity test for the option, and just
set this option.
Zhi Zhang [Fri, 10 Jan 2020 02:48:03 +0000 (10:48 +0800)]
client: disallow changing fuse_default_permissions option at runtime
If fuse_default_permissions is false when initializing fuse, then
ceph-fuse will use its own permission check. If changig this option
to true at runtime, there will be no permission check at both fuse
side and ceph-fuse side.
Jason Dillaman [Wed, 8 Jan 2020 14:21:30 +0000 (09:21 -0500)]
rbd-mirror: bootstrap and related state machines now uses StateBuilder
This removes all the journal-specific variables from the image replayer
and bootstrap state machines. Instead, these details are hidden behind
the abstract StateBuilder class and its derived journal class.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
projects / ceph.git / log