Stephan Müller [Tue, 14 Jan 2020 13:38:03 +0000 (14:38 +0100)]
mgr/dashbaord: Fix E2E pools page failure
The problem was that pg autoscaling is set to "on" by default. Therefore
the pg_num field is hidden, which caused the error. Now the test sets
the pg autoscaling to "off" revealing the pg_num field to make the tests
pass.
Fixes: https://tracker.ceph.com/issues/43594 Fixes: https://tracker.ceph.com/issues/43102 Signed-off-by: Stephan Müller <smueller@suse.com>
Patrick Donnelly [Wed, 15 Jan 2020 04:32:44 +0000 (20:32 -0800)]
Merge PR #31763 into master
* refs/pull/31763/head:
test: auto-upgrade subvolume test
mgr/volumes: tie everything together to implement versioned subvolumes
mgr/volumes: provide subvolume create/remove/open APIs
mgr/volumes: implement subvolume based on subvolume template
mgr/volumes: implement subvolume group based on group template
mgr/volumes: implement trash as a subvolume group
mgr/volumes: snapshot util module
mgr/volumes: template for implementing groups and subvolumes
mgr/volumes: implement filesystem volume module
mgr/volumes: lock module to serialize volume operations
mgr/volumes: introduce volume specification module
mgr/volumes: add fs_util helper module
Reviewed-by: Ramana Raja <rraja@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Tue, 14 Jan 2020 21:59:45 +0000 (15:59 -0600)]
Merge PR #32095 into master
* refs/pull/32095/head:
ceph-volume: raw: activate: drop --all, --osd-id and --osd-fsid args
raw: fix activate args
raw: --osd-fsid throughout
ceph-volume: add raw mode
ceph-volume: show devices with bluestore labels and unavailable
Casey Bodley [Tue, 14 Jan 2020 14:42:52 +0000 (09:42 -0500)]
qa/rgw: remove test against hadoop v2.8.5
the hadoop branch rel/release-2.8.5 fails to build with:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:37 min
[INFO] Finished at: 2020-01-14T13:09:02Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) on project hadoop-aws: An Ant BuildException has occured: Unable to create javax script engine for javascript
Kefu Chai [Tue, 14 Jan 2020 08:47:41 +0000 (16:47 +0800)]
common/util: use ifstream to read from /proc files
refactor `collect_sys_info()` to use ifstream, getline and boost string
helpers to read and parse the /proc/meminfo and /proc/cpuinfo. for
better readability, and it's less error-prone.
Sage Weil [Mon, 13 Jan 2020 13:11:51 +0000 (07:11 -0600)]
osd/PG: clean up fastinfo key when last_update does not increase
If the last_update field goes backwards, we write a full info instead of
the fastinfo key. However, a stale fastinfo key may still be present, and
it may have a last_update that looks valid to a restarting OSD.
Fixes: https://tracker.ceph.com/issues/43580 Reported-by: song <jinglesong0814@163.com> Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 6 Dec 2019 20:08:12 +0000 (14:08 -0600)]
ceph-volume: add raw mode
list, prepare, activate, create (prepare+activate) basic bluestore OSDs
on a single device.
No support (or plans to ever support) dmcrypt.
I think support for db/wal could be added later without much trouble.
For now, we require --no-systemd and --bluestore flags because we haven't
implemented support for anything else but suspect we will in the future,
and would like to ensure users are being explicit about what they're
asking for.
Michael Fritch [Mon, 13 Jan 2020 21:54:32 +0000 (14:54 -0700)]
qa/workunits/cephadm/test_cephadm: add SUDO to `vgchange`
2020-01-13T10:48:14.235 INFO:tasks.workunit.client.0.smithi177.stderr:+ vgchange -an test_cephadm
2020-01-13T10:48:14.239 INFO:tasks.workunit.client.0.smithi177.stderr: WARNING: Running as a non-root user. Functionality may be unavailable.
2020-01-13T10:48:14.239 INFO:tasks.workunit.client.0.smithi177.stderr: /run/lvm/lvmetad.socket: access failed: Permission denied
2020-01-13T10:48:14.240 INFO:tasks.workunit.client.0.smithi177.stderr: WARNING: Failed to connect to lvmetad. Falling back to device scanning.
2020-01-13T10:48:14.260 INFO:tasks.workunit.client.0.smithi177.stderr: /dev/mapper/control: open failed: Permission denied
2020-01-13T10:48:14.261 INFO:tasks.workunit.client.0.smithi177.stderr: Failure to communicate with kernel device-mapper driver.
2020-01-13T10:48:14.261 INFO:tasks.workunit.client.0.smithi177.stderr: Incompatible libdevmapper 1.02.145 (2017-11-03) and kernel driver (unknown version).
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr: /run/lock/lvm/V_test_cephadm:aux: open failed: Permission denied
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr: Can't get lock for test_cephadm
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr: Cannot process volume group test_cephadm
Rodrigo Severo [Fri, 10 Jan 2020 14:38:19 +0000 (11:38 -0300)]
doc/install/upgrading-ceph: using systemd commands in Ubuntu instructions
Before this change, we were using the "restart" command which is a symlink to
initctl(8). and it's offered by upstart. since ubuntu/vivid, ubuntu moved from
upstart to systemd.
Please see https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039465.html
Casey Bodley [Mon, 13 Jan 2020 19:44:47 +0000 (14:44 -0500)]
Merge pull request #32534 from cbodley/wip-43512
rgw multisite: enforce spawn window for incremental data sync
Reviewed-by: Daniel Gryniewicz <dang@redhat.com> Reviewed-by: Eric J. Ivancich <ivancich@redhat.com> Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Kefu Chai [Sun, 12 Jan 2020 09:59:11 +0000 (17:59 +0800)]
ceph.spec.in: package .pyc files on el7
* package .pyc files on el7
per
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_byte_compiling
it's required to package .pyc files
* use python3 to bytecompile .py files on el7
per
https://fedoraproject.org/wiki/Changes/No_more_automagic_Python_bytecompilation#Status_quo,
`brp-python-bytecompile` script bytecompile .py files automatically, the
.py files under interpreter-specific directories, such as `/usr/lib/python3.6/`,
are compiled using the appropriate compiler. but the mgr modules are
located under `/usr/share/ceph/mgr/`. they are compiled using
`${__python}`, which is still python2 in RHEL/CentOS 7. since we've
dropped the support of python2 in octopus. we should either stop
bytecompiling the .py source files or start using python3 to compile
them. otherwise python2 will fail to compile source files using python3
specific syntax like:
Jason Dillaman [Sun, 12 Jan 2020 15:05:04 +0000 (10:05 -0500)]
rbd-mirror: skip closing local image if it was already closed
If the journal replayer finishes relaying (error or promotion), it will
close the local image. However, the image replayer state machine will also
shut down the journal replayer (again) which might result in attempting
to close the local image again.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Kefu Chai [Sat, 11 Jan 2020 14:41:59 +0000 (22:41 +0800)]
common/strtol.cc: refactor strict_si_cast() to silence warnings
to silence warnings like
../src/common/strtol.cc:286:7: warning: implicit conversion from 'promoted_t' (aka 'long long') to 'double' changes value from 9223372036854775807 to 9223372036854775808
[-Wimplicit-int-float-conversion]
static_cast<promoted_t>(std::numeric_limits<T>::max()) / pow (10, m)) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~
../src/common/strtol.cc:294:15: note: in instantiation of function template specialization 'strict_si_cast<long>' requested here
template long strict_si_cast<long>(std::string_view str, std::string *err);
^
Kefu Chai [Sat, 11 Jan 2020 14:30:49 +0000 (22:30 +0800)]
src/: mark final class as final
to silence warnings from clang-10 like:
```
../src/os/bluestore/ceph_aio.h:120:18: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
~aio_queue_t() final {
^
../src/os/bluestore/ceph_aio.h:108:8: note: mark 'aio_queue_t' as 'final' to silence this warning
struct aio_queue_t : public io_queue_t {
^
1 warning generated.
```
Kefu Chai [Sat, 11 Jan 2020 08:37:17 +0000 (16:37 +0800)]
crimson: avoid creating temporary with protected dtor
to address the compiler error from clang 9.0.1:
../src/crimson/os/cyanstore/cyan_collection.cc:16:34: error: temporary of type 'boost::intrusive_ref_counter<Object, boost::thread_unsafe_counter>' has protected destructor
return new crimson::os::Object{};
^
/opt/ceph/include/boost/smart_ptr/intrusive_ref_counter.hpp:157:30: note: declared protected here
BOOST_DEFAULTED_FUNCTION(~intrusive_ref_counter(), {})
^
Kefu Chai [Sat, 11 Jan 2020 08:26:54 +0000 (16:26 +0800)]
crimson: move() explicitly to avoid copy
to silence following warnings from clang 9.0.1:
src/crimson/osd/heartbeat.cc:143:14: warning: local variable 'extras' will be copied despite being returned by name [-Wreturn-std-move]
return extras;
^~~~~~
../src/crimson/osd/heartbeat.cc:143:14: note: call 'std::move' explicitly to avoid copying
return extras;
^~~~~~
std::move(extras)
Kefu Chai [Sat, 11 Jan 2020 08:07:42 +0000 (16:07 +0800)]
crimson: do not capture unused variables
this silences warnings from clang:
src/crimson/osd/heartbeat.cc:109:14: warning: lambda capture 'this' is
not used [-Wunused-lambda-capture]
.then([this, &info=peer_info->second] (auto xcon_front, auto
xcon_back) {
^~~~~
Kefu Chai [Sat, 11 Jan 2020 10:29:29 +0000 (18:29 +0800)]
cmake: do not include global_context.cc multiple times
otherwise ODR is violated:
==449025==ERROR: AddressSanitizer: odr-violation (0x000000f03700):
[1] size=8 'g_ceph_context' ../src/global/global_context.cc:24:14
[2] size=8 'g_ceph_context' ../src/global/global_context.cc:24:14
These globals were registered at these points:
[1]:
#0 0x4779bd in __asan_register_globals (/var/ssd/ceph/clang-build/bin/ceph-conf+0x4779bd)
#1 0x56e9cb in asan.module_ctor (/var/ssd/ceph/clang-build/bin/ceph-conf+0x56e9cb)
[2]:
#0 0x4779bd in __asan_register_globals (/var/ssd/ceph/clang-build/bin/ceph-conf+0x4779bd)
#1 0x7fe5fed12aeb in asan.module_ctor (/var/ssd/ceph/clang-build/lib/libceph-common.so.2+0x2f34aeb)
==449025==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
Ronen Friedman [Sat, 11 Jan 2020 07:42:49 +0000 (09:42 +0200)]
crimson: fix aarch64 ctest failure by removing some lambda attributes
There seems to be no "universally accepted" way to declare a lambda as
[[always_inline]]. "Universally accepted" here meaning: accepted with
no error or warning by gcc8, gcc9 and clang.
See for example: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60503
projects / ceph.git / log