Kefu Chai [Fri, 18 Feb 2022 15:23:54 +0000 (23:23 +0800)]
crimson: specialize fmt::formatter<>() for crimson types
otherwise we'd have FTBFS like
/usr/include/fmt/core.h:1727:3: error: static_assert failed due to requirement 'formattable' "Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api$
static_assert(
^
/usr/include/fmt/core.h:1853:23: note: in instantiation of function template specialization 'fmt::detail::make_arg<true, fmt::basic_format_context<fmt::appender, char>, fmt::detail::type::custom_type, crimson::os:$
data_{detail::make_arg<
please note, delta_op_t is lifted out of the templated outer class
to avoid the headache of specialization of template of template in
another namespace.
Zac Dover [Sat, 13 Aug 2022 23:14:56 +0000 (09:14 +1000)]
doc/rados/operations: add prompts to operating.rst
This commit adds ".. prompt:: bash $"-style prompts to operating.rst.
This brings this file up to the standard established in 2020 when
Kefu added support for the ".. prompt::" directive.
This commit is a part of an initiative to modernize the presentation
of all BASH commands in the RADOS documentation.
The progress of this project can be tracked here:
https://tracker.ceph.com/issues/57108
Kefu Chai [Wed, 10 Aug 2022 14:35:14 +0000 (22:35 +0800)]
ceph.spec.in: %enable_devtoolset11 only if the macro is defined
there is chance that we are using `yum-builddep` to prepare the
build dependencies. in that case, gcc-toolset-11-build is not
installed. it's like a chicken-egg dilemma, but the point is
`yum-builddep` is able to pull in the gcc-toolset-11-build. once
gcc-toolset-11-build is installed, we will have the %enable_devtoolset11
rpm macro.
Kefu Chai [Mon, 8 Aug 2022 12:40:52 +0000 (20:40 +0800)]
ceph.spec.in: add libatomic to BuildRequires on fedora
otherwise we'd have failures like
/opt/compiler-explorer/gcc-trunk-20220808/bin/../lib/gcc/x86_64-linux-gnu/13.0.0/../../../../x86_64-linux-gnu/bin/ld:
/tmp/ccVlMbVh.o: in function `std::atomic<tagged_ptr>::store(tagged_ptr,
std::memory_order)':
/opt/compiler-explorer/gcc-trunk-20220808/include/c++/13.0.0/atomic:273:
undefined reference to `__atomic_store_16'
when generating the building system using CMake on fedora 36.
Kefu Chai [Thu, 4 Aug 2022 03:57:05 +0000 (11:57 +0800)]
ceph.spec.in: BuildRequires gcc-toolset-11-libatomic-devel for aarch64
to address following failure when generating the building system
using CMake:
```
-- Performing Test HAVE_LIBATOMIC
-- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/c++ requires libatomic,
but it is not found
```
Kefu Chai [Thu, 4 Aug 2022 01:00:29 +0000 (09:00 +0800)]
ceph.spec.in: call %enable_devtoolset11 on centos8
before this change %enable_devtoolset11 is called only when building
with crimson on centos8.
after this change %enable_devtoolset11 is called when building on
centos8. because we've started using gcc-toolset-11 for building
rpm packages on centos8 after the C++20 migration. so, to build
with gcc-11, we need to enable it.
also, because gcc-toolset-11 is used, we have to disable
annotated_build.
gcc-toolset-11-annobin is already installed, but ceph.spec.in adds
"-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" which needs the gcc
plugin too
resolves this failure during the cmake configure step:
-- Check for working CXX compiler: /opt/rh/gcc-toolset-11/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/gcc-toolset-11/root/usr/bin/c++ - broken
CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:59 (message):
The C++ compiler
Kefu Chai [Mon, 27 Jun 2022 13:18:57 +0000 (21:18 +0800)]
ceph.spec.in: use %enable_devtoolset11 to enable GTS-11
%enable_devtoolset11 redefines %___build_pre by appending
`source scl_source enable gcc-toolset-11` to it. `___build_pre` should
be able to populate this setting to both %build and %install. and hence
address the FTBFS where we need to use the tool chain from GTS-11.
This can be surprising but we actually compile things during
the `install` stage of `rpm-build`. The example is the pybind's
`setup.py` which builds `rados_dummy.c`.
ceph.spec.in: use gcc-toolset-10 for building crimson
This commit bumps up the toolset version but only to build crimson.
That is, the classical OSD stays unaffected.
The reason behind the upgrade is the following FTBFS:
```
[ 32%] Building CXX object src/seastar/CMakeFiles/seastar.dir/src/core/reactor.cc.o
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc: In constructor ‘seastar::reactor::reactor(std::shared_ptr<seastar::smp>, seastar::alien::instance&, unsigned int, seastar::reactor_backend_selector, seastar::reactor_config)’:
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc:926:90: error: use of deleted function ‘seastar::condition_variable::condition_variable()’
926 | , _thread_pool(std::make_unique<thread_pool>(this, seastar::format("syscall-{}", id))) {
| ^
In file included from /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/include/seastar/core/reactor.hh:74,
from /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc:32:
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/include/seastar/core/condition-variable.hh:157:5: note: ‘seastar::condition_variable::condition_variable() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘’
157 | condition_variable() noexcept = default;
```
Kefu Chai [Sat, 6 Aug 2022 00:24:12 +0000 (08:24 +0800)]
mgr/dashboard: bump up teuthology
to include the fix of e7c5d67e10fe29da22180f9e09b8973ae166c8fc,
see https://github.com/ceph/teuthology/pull/1746.
to address the test failure on ubuntu jammy. where we have python3.10
osd: Handle oncommits and wait for future work items from mClock queue
When a worker thread with the smallest thread index waits for future work
items from the mClock queue, oncommit callbacks are called. But after the
callback, the thread has to continue waiting instead of returning back to
the ShardedThreadPool::shardedthreadpool_worker() loop. Returning results
in the threads with the smallest index across all shards to busy loop
causing very high CPU utilization.
The fix involves reacquiring the shard_lock and waiting on sdata_cond
until notified or until time period lapses. After this, the smallest
thread index repopulates the oncommit queue from the context_queue
if there were any additions.
Kefu Chai [Fri, 5 Aug 2022 08:40:41 +0000 (16:40 +0800)]
cmake: remove spaces in macro used for compiling cython code
we are facing following FTBFS on jammy + GCC-11.2 + Cython 0.29 +
CMake 3.22:
creating /home/jenkins-build/build/workspace/ceph-api/build/lib/cython_modules/temp.linux-x86_64-3.10/home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs
compile options: '-I/usr/include/python3.10 -I/usr/include/python3.10 -c'
extra options: '-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -iquote/home/jenkins-build/build/workspace/ceph-api/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2'
cc: /home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs/cephfs.c
cc: warning: ##: linker input file unused because linking not done
cc: error: ##: linker input file not found: No such file or directory
cc: warning: 0: linker input file unused because linking not done
cc: error: 0: linker input file not found: No such file or directory
it seems cython is not able to escape the space in the "extra options"
anymore, so the "##" and "0" are considered as object files passed to
compiler in addition to cephfs.c.
in this change the spaces are removed to help cython to make the right
decision.
mgr/dashboard: cluster > hosts: host list tables doesn't show all services deployed
Fixes: https://tracker.ceph.com/issues/53210 Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Service instances was displaying only the ceph services, but with these changes it'll display instances
of cephadm services as well.
We don't backport PRs merged into doc/releases. Therefore, when one browses to an older Ceph release version on docs.ceph.com (e.g., https://docs.ceph.com/en/pacific/), the information is out of date at best.
The doc/releases page is only accurate if browsing https://docs.ceph.com/en/latest/, for example.
So this post_checkout command will make sure we've checked out doc/releases from main before building and publishing.
Luis Domingues [Tue, 19 Jul 2022 09:04:34 +0000 (11:04 +0200)]
mgr/cephadm: add parsing for config on osd specs
Cephadm, while parsing spec files, can parse ceph configuration
for almost all the services, except for OSDs, where it fails
with a nasty "unexpected keyword argument config".
mgr/volumes: Fix subvolume creation in FIPS enabled system.
The md5 checksum is used in the construction of legacy
subvolume config filename. It's not used for security reason.
Hence marking the 'usedforsecurity' flag to false to
make it FIPs compliant.
The usage of md5 was always in there. The commit 373a04cf734
made it to get exercised in 'open_subvol' which is pre-requisite
for all the subvolume operations and hence subvolume
creation has failed.
When the class `Device` is instantiated with a path instead of a
block device, it fails like following.
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/ceph_volume/util/device.py", line 130, in __init__
self._parse()
File "/usr/lib/python3.6/site-packages/ceph_volume/util/device.py", line 233, in _parse
self.ceph_device = disk.has_bluestore_label(self.path)
File "/usr/lib/python3.6/site-packages/ceph_volume/util/disk.py", line 906, in has_bluestore_label
with open(device_path, "rb") as fd:
IsADirectoryError: [Errno 21] Is a directory: '/var/lib/ceph/osd/ceph-0/'
```
passing a path instead of a block device is valid, `simple scan` needs it.
ceph-volume unit tests shouldn't actually create contents on the
filesystem from where it runs (even though they are written in a tmp
dir), let's use pyfakefs.
ceph-volume shouldn't report devices `/dev/sdy` and `/dev/sdz`, they will never be
available in such a scenario. Considering this, in a host with a bunch of mpath devices
it will pollute the inventory output.
Zack Cerza [Tue, 21 Jun 2022 17:28:30 +0000 (11:28 -0600)]
ceph-volume: Rename env var; add warning
So that we can have a nice big warning that fires once per invocation, I
think using a callable class with a class attribute seems like a decent
approach. A closure could work too.
Zack Cerza [Tue, 17 May 2022 17:29:02 +0000 (11:29 -0600)]
ceph-volume: Optionally consume loop devices
A similar proposal was rejected in #24765; I understand the logic
behind the rejection, but this will allow us to run Ceph clusters on
machines that lack disk resources for testing purposes. We just need to
make it impossible to accidentally enable, and make it clear it is
unsupported.
projects / ceph.git / log