Lucian Petrut [Mon, 8 May 2023 12:47:42 +0000 (12:47 +0000)]
common: use signedspan for monotonic ceph clocks
The monotonic clocks are commonly used for measuring time deltas,
which can be negative.
ceph::mono_clock and ceph::coarse_mono_clock currently use
unsigned duration types [1]. The difference operators are overloaded
in order to ensure that the result is signed [2][3].
However, we still have issues when unsigned timespans are compared.
For example, std::condition::wait_for can hang indefinitely due
to underflows [4][5]. It ends up using our unsigned type for a
negative timespan, which is then compared to
std::chrono::duration<Rep,Period>::zero.
In order to avoid such problems, we'll simply use a signed type
for monotonic clock durations.
With signed timespans, we can no longer assume that time_point::zero()
is equal to time_point::min(), so we're updating it accodingly.
Lucian Petrut [Thu, 4 May 2023 13:16:56 +0000 (13:16 +0000)]
test: add timer loop tests
We've been experiencing timer hangs with mingw-llvm.
std::condition_variable::wait_for was returning a few microseconds
before the requested time and then hanging when called with a
small interval (e.g. microseconds).
This was affecting the OSD periodic tick, which would hang after
a while (20m up to 2h).
The issue can be reproduced with a timer loop and a small interval
(e.g. 40us), in which case the timer is likely to hang after about
10s.
We're adding some tests, while the actual mingw-llvm issue will
be mitigated in a separate commit.
winpthreads is a library that emulates the pthreads API using
Windows primitives. It's also used by the mingw/gcc libstdc++
for std::thread.
The issue is that winpthreads isn't well maintained. There
have been numerous bugs that haven't been addressed in years.
Specifically, we've been hitting deadlocks because of the
winpthreads rw lock implementation.
This change will allow building Ceph for Windows using mingw/llvm,
which uses libc++ and doesn't rely on winpthreads.
We're checking a permission denied exception message that's
runtime specific:
/mnt/data/workspace/ceph_mingw_clang/src/test/dokan/dokan.cc:252: Failure
Expected equality of these values:
e.what()
Which is: "filesystem error: in remove: Permission denied
[\"Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5\"]"
exception_msg.c_str()
Which is: "filesystem error: cannot remove: No such device
[Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5]"
In order to support libc++, we'll drop the exception message
assertion and rely on the exception type.
common: avoid using pthread native handle if not available
Especially when targeting Windows, llvm may not necessarily
use pthreads for std::thread. In this case, we must not use the
"native" thread handle with the pthreads API.
We'll update the ceph_pthread_getname and ceph_pthread_setname
wrappers, adding a new one: ceph_pthread_kill.
clang errors out because of a few type mismatches that gcc
ignored through "-fpermissive".
We'll need to cast a few void pointers to the appropriate type.
There's also a function that doesn't have an explicit return type,
which was omitted by mistake.
Some symbols from the crc32, arch and fmt libs
are re-exported by libceph-common:
FAILED: bin/unittest_time.exe
ld.lld: error: fmt::v9::format_error::~format_error() was replaced
llvm throws errors because of the duplicate symbols.
One workaround is to use objects instead of static libs
for the libs. For libfmt we'll use the header-only version.
Lucian Petrut [Mon, 21 Aug 2023 06:27:24 +0000 (06:27 +0000)]
cmake: honor WITH_FMT_HEADER_ONLY with vendored fmt lib
We're using the vendored fmt lib when there is no system library
available. However, there is an inconsistency: the
WITH_FMT_HEADER_ONLY setting is ignored by the vendored library.
In order to address this, we'll use the fmt-header-only alias
if WITH_FMT_HEADER_ONLY is set.
We're fixing a few rbd-wnbd issues that are currently ignored
by mingw-gcc but not by llvm:
* checking if an uint is smaller than 0
* qualified method names must be used when passing the address
* duplicate symbol "shutdown_lock"
* add missing const cast when passing WNBD interface
We're currently setting FMT_USE_TZSET=0 when building libfmt
in order to avoid the _tzset function, which is unavailable
under Mingw:
https://github.com/ceph/ceph/commit/aa5769ecf1d80fc9824280d2e90fd4c61a0e7769
The issue is that it still gets used by fmt/chrono.h, which is
why we'll move this definition to the top level cmake file.
Note that the Windows build is currently failing as a result of
a recent change: https://github.com/ceph/ceph/pull/52590/files
In file included from ceph/src/common/ceph_time.h:22,
from ceph/src/include/encoding.h:31,
from ceph/src/include/uuid.h:9,
from ceph/src/include/types.h:21,
from ceph/src/crush/CrushWrapper.h:14,
from ceph/src/crush/CrushCompiler.h:7,
from ceph/src/crush/CrushCompiler.cc:4:
ceph/src/fmt/include/fmt/chrono.h: In lambda function:
ceph/src/fmt/include/fmt/chrono.h:953:5: error: ‘_tzset’ was
not declared in this scope; did you mean ‘tzset’?
953 | _tzset();
| ^~~~~~
| tzset
avanthakkar [Mon, 26 Jun 2023 07:11:24 +0000 (12:41 +0530)]
mgr/dashboard: empty grafana panels for performance of daemons
Fixes: https://tracker.ceph.com/issues/61792 Signed-off-by: avanthakkar <avanjohn@gmail.com>
Removing the `ceph-` prefix from ceph_daemon label to adopt it with the label
format used by queries in grafana dashboards. Also changing the
`instance_id` label for rgw to match the values coming from
exporter and prometheus module
Ville Ojamo [Thu, 27 Jul 2023 07:56:58 +0000 (14:56 +0700)]
doc/radosgw: Add missing space to date option spec in admin.rst
The start time and end time CLI option specification is missing a space between the date and the optional time value. Also expand the text to talk about "optional time" after the date.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Adam King [Thu, 27 Jul 2023 13:58:51 +0000 (09:58 -0400)]
Merge pull request #52084 from rhcs-dashboard/fix-exporter-addrs
exporter: ceph-exporter scrapes failing on multi-homed server
Reviewed-by: Adam King <adking@redhat.com> Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com> Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Add instructions directing the reader to install the "python3-routes"
package. This package is required in order to launch the dashboard after
the installation procedure has completed, but is not yet included in the
install-deps.sh script.
mengxiangrui [Sat, 21 Aug 2021 07:20:00 +0000 (15:20 +0800)]
rgw: fix the Content-Length in response header is inconsistent with response body size when rgw returns default html error page in static website
The default html error page as response body should be built completely include three ending html symbols(/ul, /body and /html) before rgw computes Content-Length in response header. The Content-Length in response header will be consistent with response body size. Client can get complete page.
Jos Collin [Mon, 24 Jul 2023 08:46:52 +0000 (14:16 +0530)]
qa: fix cephfs-mirror unwinding and 'fs volume create/rm' order
* Fixes the 'fs volume create' happens before the cephfs-mirror daemon start.
* Fixes the 'fs volume rm' happen only after the cephfs-mirror daemon unwinding.
- This prevents the issue of mirror-daemon not returning from a libcephfs call, as
the volumes were deleted during the cephfs_mirror_thrash ing.
Fixes: https://tracker.ceph.com/issues/61182 Signed-off-by: Jos Collin <jcollin@redhat.com>
* refs/pull/48038/head:
client test: Add fsync to ll_preadv_pwritev test
libcephfs: Option to write + fsync via ceph_ll_nonblocking_readv_writev
Client: Hook nonblocking fsync into the write path of ll_preadv_pwritev
Client: Add non-blocking fsync
Client/Inode: wait_for_caps fixups
Client: change several waitfor_* to use Context list
test: Add nonblocking I/O client test
libcephfs: Add nonblocking readv/writev I/O interface
Client: Add ll_preadv_pwritev to expose non-blocking I/O to libcephfs
Client: Add non-blocking helper classes
Client: Break some code into new methods in prep for non-blocking I/O
Buffers: Add function to buffer.h to copy bufferlist to an iovec
ObjectCacher: Prepare file_write path for non-blocking I/O
Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Frank S. Filz [Tue, 6 Sep 2022 18:44:43 +0000 (11:44 -0700)]
Client/Inode: wait_for_caps fixups
The non-blocking flush requires us to be able to re-add to
wait_for_caps but if we simply add to the list, we get stuck in an
infinite loop. Add a wait_for_caps_pending list to add to, and then
when done signalling, we move the wait_for_caps_pending items onto the
wait_for_caps list.
Also in handle_cap_flush_ack(), we need to complete the caps flushing
before signalling since with non-blocking flush, we will be actually
examining the caps from the completion rather than signalling a
condition variable in the completion.
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
Frank S. Filz [Wed, 29 Jun 2022 22:39:12 +0000 (15:39 -0700)]
Client: change several waitfor_* to use Context list
Change waitfor_caps, waitfor_safe and waitfor_commit to Context list.
To make a non-blocking version of fsync (to be used for non-blocking write
and commit), we need to be able to signal an arbitrary Context on completion
of either of these lists.
add_nonblocking_onfinish_to_context_list Adds such a Context to the list.
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
Frank S. Filz [Wed, 4 May 2022 20:35:44 +0000 (13:35 -0700)]
ObjectCacher: Prepare file_write path for non-blocking I/O
For non-blocking I/O, we will want to be able to override
block_writes_upfront so rename the member cfg_block_writes_upfront and add
an option to pass block_writes_upfront as a parameter along with a member
access method so caller can pass cfg_block_writes_upfront.
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
projects / ceph.git / log