cmake: link librados applications against ceph-common
to address link failures like:
[100%] Linking CXX executable ../../../bin/unittest_global_doublefree
/opt/rh/gcc-toolset-12/root/usr/bin/ld: /opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/libstdc++_nonshared.a(sstream-inst80.o): undefined reference to symbol '_ZTVNSt7__cxx1119basic_ostringstreamIwSt11char_traitsIwESaIwEEE@@GLIBCXX_3.4.21'
/opt/rh/gcc-toolset-12/root/usr/bin/ld: /usr/lib64/libstdc++.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
this happens when using gcc-toolset to build the tree.
because neither librados.so nor libcephfs exposes libstdc++ symbols
to executable linking against it. while CMake uses "c++" to link
C++ executables. the "c++" executable comes from GTS links the C++
executables agaist
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/libstdc++.so,
which in turn is a ld script:
```
$ cat /opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/libstdc++.so
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-x86-64)
INPUT ( /usr/lib64/libstdc++.so.6 -lstdc++_nonshared )
```
but the thing is, stdc++_nonshared references some symbols
provided by libstdc++.so.6, and it is listed before it. that's
why "ld" is not able to resolve the referenced symbols used by
the executable, despite that they are provided by libstdc++ in
this case.
in this change, ceph-common is added to the linkage of executables
linked against librados and/or libcephfs, even the executables
in question does not reference ceph-common symbols. unlike librados,
libcephfs and librgw, ceph-common is an internal library, which does
not hide *any* symbols from its consumer, it is also able to provide
symbols from C++ standard library linked by it. so, in our case,
we can link the C++ executables against ceph-common for accessing
the C++ standard library. the reason why we don't link aginst libstdc++
explictly is that, we should leave this to the C++ compiler instead of
referencing a specific C++ standard library explictly by its name.
what if user wants to link against libc++ instead of libstdc++?
another fix could be to remove '-Wl,--as-needed' linker options
from the command line linking the librados applications, so the linker
does not ignore the symbols from libstdc++ when resolving the ones
referenced by stdc++_nonshared, but that would be complicated.
please note, linking against ceph-common does not change the linkage
of
* Ceph executables compiled using non-gcc-toolset toolchain, because we
always pass '-Wl,--as-needed' to "c++" when linking executables,
so "ld" should be able to drop ceph-common even we instruct it
to link against ceph-common. so it would be a no-op in this case.
* 3rd party librados executables compiled using non-gcc-toolset toolchain,
but linked against librados compiled using gcc-toolset toolchain.
because they still link against the /usr/lib64/libstdc++.so.6, when
these executables are compiled and linked. and librados is always
able to access libceph-common. so librados is safe.
Zac Dover [Wed, 29 Jun 2022 12:57:13 +0000 (22:57 +1000)]
doc/index.rst: add link to Dev Guide basic workfl.
This PR adds a link to the "Basic Workflow" section of the
Developer Guide on the landing page of docs.ceph.com.
This PR is meant to improve the documentation for developers
new to Ceph and to guide them to instructions that will allow
them to become full-fledged contributors to the Ceph project
as quickly as possible.
The "Basic Workflow" page of the Developer Guide contains
information that answers almost all of the questions that I had
about contributing to the Ceph project when I was new to it,
and I am finally acting on my long-held conviction that the
"Basic Workflow" page of the Developer Guide should have a more
prominent position in the documentation suite than it has had.
Yingxin Cheng [Fri, 24 Jun 2022 03:04:50 +0000 (11:04 +0800)]
crimson/os/seastore/segment_cleaner: increase avaliable ratio limit
Journal trimming may consume unexpected number of segments when the
available ratio limit is reached with user transactions blocked, causing
ceph_abort(). So increase the limit as a simple workaround.
Yingxin Cheng [Fri, 24 Jun 2022 05:25:51 +0000 (13:25 +0800)]
crimson/os/seastore: improve GC policies with modify-time
* record_header_t to store the average modify time for dirty extents.
* Drop tracking rewrite-time.
* Drop the last-modify field in extent_info_t.
* Maintain modify-time during rewriting.
* Introduce 3 GC policies: greedy, benefit, and cost-benefit.
Yingxin Cheng [Fri, 27 May 2022 09:13:06 +0000 (17:13 +0800)]
crimson/os/seastore: implement generational GC
Place extents into the dedicated RecordSubmitter by their data-category
and reclaimed-count. Segments of different data-category or
reclaimed-count should have different locality in the access patterns,
which is the foundation to form a desired bimodal distribution of
segment utilizations, so that GC can be more efficient.
Alex Handy [Tue, 28 Jun 2022 17:27:23 +0000 (10:27 -0700)]
doc/cephadm: osd: minor fixes in config examples
- Corrected drive counts (8 to 10) and rotational flag use (not consistent with HDD) in the "Multiple OSD specs for a single host" example
- Adjusted vendor names in "Multiple hosts with the same disk layout" for consistency of examples
Kefu Chai [Mon, 27 Jun 2022 13:18:57 +0000 (21:18 +0800)]
ceph.spec.in: use %enable_devtoolset11 to enable GTS-11
%enable_devtoolset11 redefines %___build_pre by appending
`source scl_source enable gcc-toolset-11` to it. `___build_pre` should
be able to populate this setting to both %build and %install. and hence
address the FTBFS where we need to use the tool chain from GTS-11.
This can be surprising but we actually compile things during
the `install` stage of `rpm-build`. The example is the pybind's
`setup.py` which builds `rados_dummy.c`.
Yin Congmin [Sat, 25 Jun 2022 09:43:52 +0000 (17:43 +0800)]
cmake: rename a series of pmem libraries to pmdk
At first, libpmem was the only library. Later, pmem related libraries
such as libpmemobj and libpmem2 were gradually added. These libraries
were also integrated into one named pmdk. So rename to pmdk.
Kefu Chai [Sat, 25 Jun 2022 14:27:02 +0000 (22:27 +0800)]
cmake: use CMAKE_<LANG>_COMPILER_LAUNCHER for configuring ccache
ccache only works for c and c++, so instead of using the universal
`RULE_LAUNCH_COMPILE` use `CMAKE_<LANG>_COMPILER_LAUNCHER` instead,
so ccache is only configured for c and c++ compilation. this is a better
solution for integrating ccache into our building system.
Yin Congmin [Fri, 13 May 2022 12:44:53 +0000 (20:44 +0800)]
install-deps: install pmdk libraries
Install libpmem and libpmemobj under focal ubuntu. the version of apt
list can meet the current requirements. libpmemobj require >=1.8.
Libpmem has no version requirements.
Yin Congmin [Sat, 25 Jun 2022 09:04:44 +0000 (17:04 +0800)]
cmake: lower the required version of libpmem to 1.8
The upgrade of pmemobj in https://github.com/ceph/ceph/pull/40493
is to introduce new API. The minimum version requirement is 1.8.
Therefore, the requirements for find_package can be lowered.
David Galloway [Fri, 24 Jun 2022 16:27:43 +0000 (12:27 -0400)]
.github: Add labels while PR is open
I think https://github.com/tibdex/backport will only create backport PRs if our doc/releases PRs are labelled *and then* closed. This action currently labels after the PR is closed.
Signed-off-by: David Galloway <dgallowa@redhat.com>
Ronen Friedman [Mon, 20 Jun 2022 12:47:57 +0000 (12:47 +0000)]
scrub/osd: disable blocked-scrub warnings during some tests
As some Teuthology tests seem to block objects for long minutes,
we must not issue the "scrub is blocked for too long" warning
(that warning causes the tests to fail).
A new configuration parameter now controls the grace period before
the warning is issued. Some tests were modified to set this
configuration parameter to a large value.
we do not require to maintain our own patched version. Also, we were
using git fetch as a temporary change, now that tracing is always on, it
should be right time to adapt to using opentelemetry-cpp as a submodule.
This removes compile time fetch for opentelemetry-cpp instead use
official opentelemetry-cpp lib as a submodule.
Kefu Chai [Tue, 21 Jun 2022 15:28:23 +0000 (23:28 +0800)]
install-deps.sh: do not install libpmem from chacra
this change reverts 17d2bc3707bb0078e2fa1b4eef31b39804e45135, before
we recreate a chacra repo hosting libpmem packages, we are not able
to query the repo from shaman or pull the dependencies from chacra.
in future, we should be able to get the libpmem dependencies from
offical ubuntu package repo and fedora, CentOS Stream and RHEL repos.
Zac Dover [Tue, 21 Jun 2022 14:09:05 +0000 (00:09 +1000)]
doc/dev: add context note to dev guide config
This PR adds a note directing first-time cloners of
their Ceph git forks to make sure to cd into the ceph/
directory before trying to run the "git config" commands.
Prashant D [Tue, 21 Jun 2022 06:53:41 +0000 (02:53 -0400)]
pybind/mgr/autoscaler: Donot show NEW PG_NUM value if autoscaler is not on
When noautscale is set, autoscale-status shows NEW PG_NUM
value if pool pg_num is more than 96. If autoscaler is in
off or warn mode for the pool then donot adjust the final
pg count for the pool.
Fixes: https://tracker.ceph.com/issues/56136 Signed-off-by: Prashant D <pdhange@redhat.com>
Ilya Dryomov [Sun, 19 Jun 2022 10:12:01 +0000 (12:12 +0200)]
mgr/rbd_support: always rescan image mirror snapshots on refresh
Establishing a watch on rbd_mirroring object and skipping rescanning
image mirror snapshots on periodic refresh unless rbd_mirroring object
gets notified in the interim is flawed. rbd_mirroring object is
notified when mirroring is enabled or disabled on some image (including
when the image is removed), but it is not notified when images are
promoted or demoted. However, load_pool_images() discards images that
are not primary at the time of the scan. If the image is promoted
later, no snapshots are created even if the schedule is in place. This
happens regardless of whether the schedule is added before or after the
promotion.
This effectively reverts commit 69259c8d3722 ("mgr/rbd_support: make
mirror_snapshot_schedule rescan only updated pools"). An alternative
fix could be to stop discarding non-primary images (i.e. drop
if not info['primary']:
continue
check added in commit d39eb283c5ce ("mgr/rbd_support: mirror snapshot
schedule should skip non-primary images")), but that would clutter the
queue and therefore "rbd mirror snapshot schedule status" output with
bogus entries. Performing a rescan roughly every 60 seconds should be
manageable: currently it amounts to a single mirror_image_status_list
request, followed by mirror_image_get, get_snapcontext and snapshot_get
requests for each snapshot-based mirroring enabled image and concluded
by a single dir_list request. Among these, per-image get_snapcontext
and snapshot_get requests are necessary for determining primaryness.
projects / ceph.git / log