The `ceph-mgr` package lost its dependency on `python3-bcrypt` when
the dependencies got moved from d/control to a dh_python3 compatible
requires file. Add it again as the bcrypt module is still used there.
Otherwise one gets errors when, e.g., calling `ceph -s` after a fresh
installation:
> 13 mgr modules have failed dependencies
> Module 'balancer' has failed dependency: No module named 'bcrypt'
> Module 'crash' has failed dependency: No module named 'bcrypt'
> Module 'devicehealth' has failed dependency: No module named 'bcrypt'
> Module 'iostat' has failed dependency: No module named 'bcrypt'
> Module 'nfs' has failed dependency: No module named 'bcrypt'
> Module 'orchestrator' has failed dependency: No module named 'bcrypt'
> Module 'pg_autoscaler' has failed dependency: No module named 'bcrypt'
> Module 'progress' has failed dependency: No module named 'bcrypt'
> Module 'rbd_support' has failed dependency: No module named 'bcrypt'
> Module 'restful' has failed dependency: No module named 'bcrypt'
> Module 'status' has failed dependency: No module named 'bcrypt'
> Module 'telemetry' has failed dependency: No module named 'bcrypt'
> Module 'volumes' has failed dependency: No module named 'bcrypt'
Fixes: https://tracker.ceph.com/issues/63637 Fixes: ef19547e83e ("debian: add .requires for specifying python3 deps") Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Ali Masarwa [Mon, 4 Sep 2023 12:25:44 +0000 (15:25 +0300)]
Merge pull request #52964 from AliMasarweh/wip-alimasa-persistant-q-enhance
RGW: added a per topic configuration to control the notification persistency
Adding the configuration of persistency per topic that will override the global settings
qa: add "failover / failback loop" test for rbd-mirror
For snapshot-based mirroring, check that demote (or other mirror
snapshots) don't pile up. Nothing in particular to assert on for
journal-based mirroring but the test is still useful.
Ilya Dryomov [Sat, 26 Aug 2023 11:04:52 +0000 (13:04 +0200)]
librbd: make CreatePrimaryRequest remove any unlinked mirror snapshots
After commit ac552c9b4d65 ("librbd: localize snap_remove op for mirror
snapshots"), rbd-mirror daemon no longer removes mirror snapshots when
it's done syncing them -- instead it only unlinks from them. However,
CreatePrimaryRequest state machine was not adjusted to compensate and
hence two cases were missed:
- primary demotion snapshot (rbd-mirror daemon unlinks from primary
demotion snapshots just like it does from regular primary snapshots);
this comes up when an image is demoted but then promoted on the same
cluster
- non-primary demotion snapshot (unlike regular non-primary snapshots,
non-primary demotion snapshots store peer uuids and rbd-mirror daemon
does unlinking just like in the case of primary snapshots); this
comes up when an image is demoted and promoted on the other cluster
Related is the case of orphan snapshots. Since they are dummy to begin
with, CreatePrimaryRequest would now clean up the orphan snapshot after
the creation of the force promote snapshot.
Fixes: https://tracker.ceph.com/issues/61707 Co-authored-by: Christopher Hoffman <choffman@redhat.com> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Ilya Dryomov [Tue, 22 Aug 2023 15:27:50 +0000 (17:27 +0200)]
librbd: don't attempt to remove image state on orphan snapshots
Despite being mirror snapshots, orphan snapshots don't have image
state: see CreateNonPrimaryRequest::write_image_state() for a similar
is_orphan() check. Attempting to remove image state generates bogus
"failed to read image state object" and "failed to remove image state"
errors.
Adam Kupczyk [Fri, 11 Aug 2023 14:24:57 +0000 (14:24 +0000)]
tools/variable_load: Add generator of variable workload
The tool is dedicated to create highly variable workloads.
The intended audience is
1) scraper testing - tool to sniff on OSD ops
2) CoDel testing - bandwidth/latency optimization algorithm in BS
Initial commit.
Boost stacktrace defines a few UUIDs that were recently added
to mingw as well [1], causing compilation errors [2]:
In file included from libs/stacktrace/build/../src/windbg.cpp:9:
./boost/stacktrace/detail/frame_msvc.ipp:31:5: error: redefinition of
'__mingw_uuidof_s<IDebugClient>'
__CRT_UUID_DECL(IDebugClient,0x27fe5639,...
We'll apply a fix that hasn't merged upsteam yet [3].
Lucian Petrut [Wed, 23 Aug 2023 14:21:12 +0000 (14:21 +0000)]
cmake: fix mingw-llvm shared linking
mingw-llvm can't handle the '--exclude-libs' linker flag, so
we'll have to skip it.
At the same time, cmake can't locate the boost shared libs as the
import libs are not generated when using mingw-llvm due to boost's
clang-linux.jam file. For now, we'll patch the cmake files, using
the dlls as import libs (which is allowed by mingw).
While at it, we'll avoid linking the static AND dynamic boost libs,
speeding the build.
Lucian Petrut [Mon, 8 May 2023 12:47:42 +0000 (12:47 +0000)]
common: use signedspan for monotonic ceph clocks
The monotonic clocks are commonly used for measuring time deltas,
which can be negative.
ceph::mono_clock and ceph::coarse_mono_clock currently use
unsigned duration types [1]. The difference operators are overloaded
in order to ensure that the result is signed [2][3].
However, we still have issues when unsigned timespans are compared.
For example, std::condition::wait_for can hang indefinitely due
to underflows [4][5]. It ends up using our unsigned type for a
negative timespan, which is then compared to
std::chrono::duration<Rep,Period>::zero.
In order to avoid such problems, we'll simply use a signed type
for monotonic clock durations.
With signed timespans, we can no longer assume that time_point::zero()
is equal to time_point::min(), so we're updating it accodingly.
Lucian Petrut [Thu, 4 May 2023 13:16:56 +0000 (13:16 +0000)]
test: add timer loop tests
We've been experiencing timer hangs with mingw-llvm.
std::condition_variable::wait_for was returning a few microseconds
before the requested time and then hanging when called with a
small interval (e.g. microseconds).
This was affecting the OSD periodic tick, which would hang after
a while (20m up to 2h).
The issue can be reproduced with a timer loop and a small interval
(e.g. 40us), in which case the timer is likely to hang after about
10s.
We're adding some tests, while the actual mingw-llvm issue will
be mitigated in a separate commit.
winpthreads is a library that emulates the pthreads API using
Windows primitives. It's also used by the mingw/gcc libstdc++
for std::thread.
The issue is that winpthreads isn't well maintained. There
have been numerous bugs that haven't been addressed in years.
Specifically, we've been hitting deadlocks because of the
winpthreads rw lock implementation.
This change will allow building Ceph for Windows using mingw/llvm,
which uses libc++ and doesn't rely on winpthreads.
We're checking a permission denied exception message that's
runtime specific:
/mnt/data/workspace/ceph_mingw_clang/src/test/dokan/dokan.cc:252: Failure
Expected equality of these values:
e.what()
Which is: "filesystem error: in remove: Permission denied
[\"Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5\"]"
exception_msg.c_str()
Which is: "filesystem error: cannot remove: No such device
[Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5]"
In order to support libc++, we'll drop the exception message
assertion and rely on the exception type.
common: avoid using pthread native handle if not available
Especially when targeting Windows, llvm may not necessarily
use pthreads for std::thread. In this case, we must not use the
"native" thread handle with the pthreads API.
We'll update the ceph_pthread_getname and ceph_pthread_setname
wrappers, adding a new one: ceph_pthread_kill.
clang errors out because of a few type mismatches that gcc
ignored through "-fpermissive".
We'll need to cast a few void pointers to the appropriate type.
There's also a function that doesn't have an explicit return type,
which was omitted by mistake.
Some symbols from the crc32, arch and fmt libs
are re-exported by libceph-common:
FAILED: bin/unittest_time.exe
ld.lld: error: fmt::v9::format_error::~format_error() was replaced
llvm throws errors because of the duplicate symbols.
One workaround is to use objects instead of static libs
for the libs. For libfmt we'll use the header-only version.
Milind Changire [Tue, 29 Aug 2023 14:43:50 +0000 (20:13 +0530)]
Merge PR #52686 into main
* refs/pull/52686/head:
PendingReleaseNotes: note about mandatory fs argument
doc/cephfs: add note about mandatory --fs argument to snap-schedule
qa: add test for mandatory fs argument to snap-schedule commands
mgr/snap-schedule: tweaks to keep mypy happy
mgr/snap_schedule: validate fs before execution
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
Nizamudeen A [Tue, 29 Aug 2023 14:16:07 +0000 (19:46 +0530)]
mgr/dashboard: fix cephfs create form validator
dashboard was allowing to create a filesystem with / in its name but the
cli throws out error in doing so. And the created volume in dashboard
just gets ended up as a stale volume
Fixes: https://tracker.ceph.com/issues/62628 Signed-off-by: Nizamudeen A <nia@redhat.com>
Ilya Dryomov [Wed, 7 Jun 2023 19:34:07 +0000 (21:34 +0200)]
doc: drop mention of rbd_mirror_journal_max_fetch_bytes option
It was removed in commit 1ef12ea0d29f ("rbd-mirror: remove
rbd_mirror_journal_max_fetch_bytes option") in 2019. Commit 32375cb789d7 ("doc: misc clarity and capitalization") added a "tip"
mentioning it in 2020 in an attempt to capture advice [1].
Lucian Petrut [Thu, 24 Aug 2023 11:41:01 +0000 (11:41 +0000)]
rbd-wnbd: wait for disks to become available
After a WNBD mapping is created, the driver informs Storport that
the bus changed, expecting it to rescan the bus and expose the disk.
This is an asynchronous process and it usually takes a matter of
milliseconds, up to a few seconds under significant load.
The fsx librbd test ocasionally fails to open the disk as it isn't
ready yet. Unlike the Python rbd-wnbd test, it doesn't perform
any polling.
For convenience, WNBD now includes a helper function called
WnbdPollDiskNumber. We're going to use it to wait for the new
disk attachments to become available.
Ramana Raja [Mon, 14 Aug 2023 15:27:13 +0000 (11:27 -0400)]
librbd/ImageWatcher: kick-start ExclusiveLock state machine
... that is stalled on waiting for lock and let it detect client
blocklisting. This would propagate the blocklist error to the caller
requesting an operation needing an exclusive lock.
When a caller requests a librbd operation that requires an exclusive
lock, the librbd client checks whether the exclusive lock is
held by another client. If the lock is held by another client, librbd
stalls its ExclusiveLock state machine and through its ImageWatcher
notifies the lock owner that it wants the exclusive lock. After
receiving the response from the lock owner, the ImageWatcher schedules
another lock request. Meanwhile if the client gets blocklisted, the
ImageWatcher fails to schedule another lock request and returns. The
ExclusiveLock state machine remains stalled and the blocklist error is
not propagated to the caller. Instead, when scheduling another lock
request, make the ImageWatcher call the ExclusiveLock state machine's
peer notification handler if the client is blocklisted. This allows
the Exclusive lock state machine to detect that the client has been
blocklisted in its send_acquire_lock() member function and propagate
the blocklist error to the caller.
Fixes: https://tracker.ceph.com/issues/61607 Signed-off-by: Ramana Raja <rraja@redhat.com>
Zac Dover [Wed, 23 Aug 2023 15:15:54 +0000 (01:15 +1000)]
doc/start: edit os-recommendations.rst
Improve the grammar in one sentence of the "Platforms" section of
doc/start/os-recommendations.rst. Improving that grammar involved
splitting the sentence into two sentences, but that's life. Update:
Anthony substantially rewrote this, so credit for this should rightly
go to him.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
projects / ceph.git / log