Without loading the ansible.cfg file from ceph-ansible project, we don't
have the pipelining enabled which can result in significant performance
improvement.
This removes the ANSIBLE_ACTION_PLUGINS, ANSIBLE_RETRY_FILES_ENABLED and
ANSIBLE_SSH_RETRIES environment variables as it is already included in the
ansible.cfg file.
ceph-volume/tests: update ansible ssh_args env var
The ansible ssh_args parameter is usually defined in the ansible.cfg file.
Currently this variable is overrided in tox to manage the vagrant ssh file
but we lost all default values.
updates:
* custom target jaeger_base
* target jaeger-base which encapsulates all jaeger libs
* include external libraries path needed linking libraries and including
opentracing and jaegertracing headers files
cmake: change debian DESTDIR for installing jaeger deps
debian uses debian/tmp as destination dir for installing build files,
but since we are using common path($build_dir/external) available both
for rpm and debian based dependency installation, it becomes far more
complicated to maintain include/link path for these external projects.
elaborating on it:
path we are configuring for both rpm and debian installing including,
and linking of external librarires:
/build/ceph-17.0.0-5779-g928f9e55/obj-x86_64-linux-gnu/external/
debian appends DESTDIR to this path, and hence our predefined target
artificats cannot find correct path for external libs, I tried adding
ENV${DESTDIR} so that it could include correct external lib install
path, but it still cannot find them:
failed to link in case of:
- install(DIRECTORY $ENV{DESTDIR}${CMAKE_BINARY_DIR}/external/include/jaegertracing
- $ENV{DESTDIR}${CMAKE_BINARY_DIR}/external/include/opentracing
- DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
- include_directories(SYSTEM ${CMAKE_INSTALL_INCLUDEDIR}/jaegertracing)
- include_directories(SYSTEM ${CMAKE_INSTALL_INCLUDEDIR}/opentracing)
cmake: move ExternalProjectHelper & IncludeJaeger to BuildJaeger
set_library_properties_for_external_project assists with setting right target
properties for all jaeger dependencies.
IncludeJaeger would take care of linking and creating these targets
having them spread out when they are highly coupled seems not optimal.
debian, install-deps.sh: use Build-Profiles to optionally build with jaeger
we will now be using BuildProfileSpec based optional pkg building,
removing comment from .install file is no longer needed If
pkg.ceph.jaeger is enabled debian/control, it shall work to install
jaeger and it's dependencies.
ceph.spec, cmake, debian: use thrift 0.13+ from distro pkg
the change to build and ship libthift was added when we didn't have 0.13.0
version shipped via distro pkgs, now that centos 8 and F34 supports req.
version, we do not need to build and ship it with jaeger library.
* since focal and centos both have yaml-cpp 0.6 available, which dropped
having boost as it's dependency, moving to 0.6 seems a good upgrade.
* cmake: delete Buildyaml, since distro suppilies v0.6 this is not needed
This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)
doc/man/8/ceph-volume: specify "program" for subcommands
as per
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html
> Like py:currentmodule, this directive produces no output. Instead, it
> serves to notify Sphinx that all following option directives document
> options for the program called name.
> ...
> The program name may contain spaces (in case you want to document
> subcommands like svn add and svn commit separately).
tasks/vstart_runner.py:691: error: Definition of "_run_python" in base class "LocalCephFSMount" is incompatible with definition in base class "CephFSMount"
tasks/vstart_runner.py:705: error: Definition of "_run_python" in base class "LocalCephFSMount" is incompatible with definition in base class "CephFSMount"
Sebastian Wagner [Tue, 20 Jul 2021 14:09:57 +0000 (16:09 +0200)]
cephadm: haproxy 2.4 defaults to a different container user.
Another alternative would be to investigage a different setup
leverageing `--sysctl net.ipv4.ip_unprivileged_port_start=0`,
but that would be a larger PR.
Fixes: https://tracker.ceph.com/issues/51355 Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
without specifying -Og, -O0 is used. as per
https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
> -O0
> Reduce compilation time and make debugging produce the expected
> results. This is the default.
and
> -Og
> Optimize debugging experience. -Og should be the optimization level of
> choice for the standard edit-compile-debug cycle, offering a reasonable
> level of optimization while maintaining fast compilation and a good
> debugging experience. It is a better choice than -O0 for producing
> debuggable code because some compiler passes that collect debug
> information are disabled at -O0.
and Debug is the default built type if .git directory is found under
the root source directory, so by adding "-Og -g" to CMAKE_C_FLAGS_DEBUG,
developers can have better debugging experience when testing Ceph
built from a git repo. but the downside is that it might take longer
to build the tree.
this change should also enable us to link crimson on aarch64.
the same applies to Clang,
> -O0 Means “no optimization”: this level compiles the fastest and generates the most debuggable code.
> -O1 Somewhere between -O0 and -O2.
> -O2 Moderate level of optimization which enables most optimizations.
> -Og Like -O1. In future versions, this option might disable different optimizations in order to improve debuggability.
see
https://clang.llvm.org/docs/CommandGuide/clang.html#cmdoption-o0
see also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101448
crimson/os/seastore: measure transactional efforts that are discarded or committed
The efforts of a transaction include the number and bytes of its read,
mutate, retire and fresh extents, and the bytes of delta generated.
This helps to understand the following aspects:
* The ratio of discarded efforts vs committed efforts;
* The average efforts of a transaction;
* The distribution of read/mutate/delta/retire/fresh efforts;
* The memory overhead and potential disk overhead of a transaction;
* How early a transaction invalidation happens;
* The average extent length;
It is possible to extend the effort metrics to be labeled by extent
types, in case we want to distinguish and profile the efforts at the
sub-component level.
crimson/os/seastore: assert the committing delta is not empty
It makes no sense to commit an empty delta. It is mostly an issue that
user forget to generate delta during mutation, or there are futile
copy-on-write operations.
per https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
> -Og
> Optimize debugging experience. -Og should be the optimization level of
> choice for the standard edit-compile-debug cycle, offering a reasonable
> level of optimization while maintaining fast compilation and a good
> debugging experience. It is a better choice than -O0 for producing
> debuggable code because some compiler passes that collect debug
> information are disabled at -O0.
so instead of using -O0, for better developer experience, -Og is
recommended.
qa/standalone: fixing the timings when waiting for deep-scrub to start
initiate_and_fetch_state() initiates a scrub, then polls the published
PG state looking for 'scrubbing'. Calling flush_pg_stats() as part of
the polling process might cause the scrub and the following recovery to
be missed altogether.
Note: this polling mechanism is definitely not robust. Will be
redesigned in the future.
mon/MgrMonitor: drop nautilus modules from always_on_modules
the list of always_on_modules for the older versions is only for the
upgrade use case, where the monitor is still the old release, so
we have to keep an entry for the old release in the always_on_modules
map.
but since we don't upgrade from a release older than 2 releases, there
is no need to keep an entry for nautilus.
test/rgw: fix use of poll() with timers in unittest_rgw_dmclock_scheduler
the AsyncScheduler uses an asio timer to dispatch work to its executor
with an optional delay. when no delay is requested, it waits on the
timer with an expiration time in the past (crimson::dmclock::TimeZero)
tests are failing here because poll() is returning without executing the
handlers of those expired timers
asio implements these timers with timerfd and epoll. debugging with
strace, i see that these timers armed with timerfd_settime() are not
always immediately ready according to epoll_wait():
This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)
``` Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
Laura Flores [Mon, 19 Jul 2021 17:17:58 +0000 (17:17 +0000)]
doc/dev/perf_counters: update docs to include more context about perf counter usage
The Perf Counters docs, although informative, are lacking for users and developers who are wondering what they can do with their perf counter data. I wrote an extra paragraph here that outlines some ways in which the counters can be used, including diagnosing problems in a cluster and identifying workload patterns.
cmake: use add_compile_options() to pass compile options
add_definitions() is for adding "-D" define flags, while "-include" does
not define flags, so let's be explicit and use add_compile_options()
instead.
this change partially reverts 5fb90bb79671b8e22f507728e4b4b948a591f98f.
back then, we were using ubuntu bionic for building windows packages.
but since we've switched to ubuntu focal, where cmake 3.16 is available,
there is no reason to stick with add_definitions() anymore.
projects / ceph.git / log