crimson/os/seastore/onode_manager: populate value recorders of onodes to
be erased
Otherwise, the following modification sequence with the same transaction
might lead to onode extents' crc inconsistency during journal replay:
1. modify the last mapping in an onode extent;
2. erase the last mapping in that onode extent.
During journal replay, if the first modification is not recorded in the
delta, the onode extent's content would be inconsistent with that before
the system reboot
Nizamudeen A [Wed, 30 Aug 2023 05:20:30 +0000 (10:50 +0530)]
mgr/dashboard: remove green tick on old password field
a green tick is showing to the field where we enter the old password in
login password change form. It starts showing green tick as soon as we
start typing on it. Removing that because its misleads the user.
Fixes: https://tracker.ceph.com/issues/62644 Signed-off-by: Nizamudeen A <nia@redhat.com>
Ali Masarwa [Mon, 4 Sep 2023 12:25:44 +0000 (15:25 +0300)]
Merge pull request #52964 from AliMasarweh/wip-alimasa-persistant-q-enhance
RGW: added a per topic configuration to control the notification persistency
Adding the configuration of persistency per topic that will override the global settings
qa: add "failover / failback loop" test for rbd-mirror
For snapshot-based mirroring, check that demote (or other mirror
snapshots) don't pile up. Nothing in particular to assert on for
journal-based mirroring but the test is still useful.
Ilya Dryomov [Sat, 26 Aug 2023 11:04:52 +0000 (13:04 +0200)]
librbd: make CreatePrimaryRequest remove any unlinked mirror snapshots
After commit ac552c9b4d65 ("librbd: localize snap_remove op for mirror
snapshots"), rbd-mirror daemon no longer removes mirror snapshots when
it's done syncing them -- instead it only unlinks from them. However,
CreatePrimaryRequest state machine was not adjusted to compensate and
hence two cases were missed:
- primary demotion snapshot (rbd-mirror daemon unlinks from primary
demotion snapshots just like it does from regular primary snapshots);
this comes up when an image is demoted but then promoted on the same
cluster
- non-primary demotion snapshot (unlike regular non-primary snapshots,
non-primary demotion snapshots store peer uuids and rbd-mirror daemon
does unlinking just like in the case of primary snapshots); this
comes up when an image is demoted and promoted on the other cluster
Related is the case of orphan snapshots. Since they are dummy to begin
with, CreatePrimaryRequest would now clean up the orphan snapshot after
the creation of the force promote snapshot.
Fixes: https://tracker.ceph.com/issues/61707 Co-authored-by: Christopher Hoffman <choffman@redhat.com> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Ilya Dryomov [Tue, 22 Aug 2023 15:27:50 +0000 (17:27 +0200)]
librbd: don't attempt to remove image state on orphan snapshots
Despite being mirror snapshots, orphan snapshots don't have image
state: see CreateNonPrimaryRequest::write_image_state() for a similar
is_orphan() check. Attempting to remove image state generates bogus
"failed to read image state object" and "failed to remove image state"
errors.
Adam Kupczyk [Fri, 11 Aug 2023 14:24:57 +0000 (14:24 +0000)]
tools/variable_load: Add generator of variable workload
The tool is dedicated to create highly variable workloads.
The intended audience is
1) scraper testing - tool to sniff on OSD ops
2) CoDel testing - bandwidth/latency optimization algorithm in BS
Initial commit.
John Mulligan [Thu, 17 Aug 2023 19:11:57 +0000 (15:11 -0400)]
cephadm: black format net_utils.py
This contains one small manual change reversing the black formatting.
According to the black docs [1], flake8's E203 is not pep8 compliant but
since flake8 is currently mandatory and black formatting is not we'll
leave it in a form that makes flake8 happy and attempt to resolve the
disagreement some time in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Pair-programmed-with: Adam King <adking@redhat.com> Co-authored-by: Adam King <adking@redhat.com> Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 16 Aug 2023 20:44:55 +0000 (16:44 -0400)]
cephadm: black format file_utils.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Pair-programmed-with: Adam King <adking@redhat.com> Co-authored-by: Adam King <adking@redhat.com>
Pair-programmed-with: Adam King <adking@redhat.com> Co-authored-by: Adam King <adking@redhat.com>
John Mulligan [Wed, 16 Aug 2023 19:23:42 +0000 (15:23 -0400)]
cephadm: move command call functions to call_wrappers.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Pair-programmed-with: Adam King <adking@redhat.com> Co-authored-by: Adam King <adking@redhat.com> Signed-off-by: John Mulligan <jmulligan@redhat.com>
Boost stacktrace defines a few UUIDs that were recently added
to mingw as well [1], causing compilation errors [2]:
In file included from libs/stacktrace/build/../src/windbg.cpp:9:
./boost/stacktrace/detail/frame_msvc.ipp:31:5: error: redefinition of
'__mingw_uuidof_s<IDebugClient>'
__CRT_UUID_DECL(IDebugClient,0x27fe5639,...
We'll apply a fix that hasn't merged upsteam yet [3].
John Mulligan [Thu, 10 Aug 2023 13:46:27 +0000 (09:46 -0400)]
cephadm: prepare for using black code formatting for cephadmlib
Currently this is not checked/required but we'd like to use a consistent
formatting tool for the new files in cephadmlib. This let's us do so
incrementally.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Pair-programmed-with: Adam King <adking@redhat.com> Co-authored-by: Adam King <adking@redhat.com>
Lucian Petrut [Wed, 23 Aug 2023 14:21:12 +0000 (14:21 +0000)]
cmake: fix mingw-llvm shared linking
mingw-llvm can't handle the '--exclude-libs' linker flag, so
we'll have to skip it.
At the same time, cmake can't locate the boost shared libs as the
import libs are not generated when using mingw-llvm due to boost's
clang-linux.jam file. For now, we'll patch the cmake files, using
the dlls as import libs (which is allowed by mingw).
While at it, we'll avoid linking the static AND dynamic boost libs,
speeding the build.
Lucian Petrut [Mon, 8 May 2023 12:47:42 +0000 (12:47 +0000)]
common: use signedspan for monotonic ceph clocks
The monotonic clocks are commonly used for measuring time deltas,
which can be negative.
ceph::mono_clock and ceph::coarse_mono_clock currently use
unsigned duration types [1]. The difference operators are overloaded
in order to ensure that the result is signed [2][3].
However, we still have issues when unsigned timespans are compared.
For example, std::condition::wait_for can hang indefinitely due
to underflows [4][5]. It ends up using our unsigned type for a
negative timespan, which is then compared to
std::chrono::duration<Rep,Period>::zero.
In order to avoid such problems, we'll simply use a signed type
for monotonic clock durations.
With signed timespans, we can no longer assume that time_point::zero()
is equal to time_point::min(), so we're updating it accodingly.
Lucian Petrut [Thu, 4 May 2023 13:16:56 +0000 (13:16 +0000)]
test: add timer loop tests
We've been experiencing timer hangs with mingw-llvm.
std::condition_variable::wait_for was returning a few microseconds
before the requested time and then hanging when called with a
small interval (e.g. microseconds).
This was affecting the OSD periodic tick, which would hang after
a while (20m up to 2h).
The issue can be reproduced with a timer loop and a small interval
(e.g. 40us), in which case the timer is likely to hang after about
10s.
We're adding some tests, while the actual mingw-llvm issue will
be mitigated in a separate commit.
winpthreads is a library that emulates the pthreads API using
Windows primitives. It's also used by the mingw/gcc libstdc++
for std::thread.
The issue is that winpthreads isn't well maintained. There
have been numerous bugs that haven't been addressed in years.
Specifically, we've been hitting deadlocks because of the
winpthreads rw lock implementation.
This change will allow building Ceph for Windows using mingw/llvm,
which uses libc++ and doesn't rely on winpthreads.
We're checking a permission denied exception message that's
runtime specific:
/mnt/data/workspace/ceph_mingw_clang/src/test/dokan/dokan.cc:252: Failure
Expected equality of these values:
e.what()
Which is: "filesystem error: in remove: Permission denied
[\"Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5\"]"
exception_msg.c_str()
Which is: "filesystem error: cannot remove: No such device
[Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5]"
In order to support libc++, we'll drop the exception message
assertion and rely on the exception type.
common: avoid using pthread native handle if not available
Especially when targeting Windows, llvm may not necessarily
use pthreads for std::thread. In this case, we must not use the
"native" thread handle with the pthreads API.
We'll update the ceph_pthread_getname and ceph_pthread_setname
wrappers, adding a new one: ceph_pthread_kill.
clang errors out because of a few type mismatches that gcc
ignored through "-fpermissive".
We'll need to cast a few void pointers to the appropriate type.
There's also a function that doesn't have an explicit return type,
which was omitted by mistake.
Some symbols from the crc32, arch and fmt libs
are re-exported by libceph-common:
FAILED: bin/unittest_time.exe
ld.lld: error: fmt::v9::format_error::~format_error() was replaced
llvm throws errors because of the duplicate symbols.
One workaround is to use objects instead of static libs
for the libs. For libfmt we'll use the header-only version.
Milind Changire [Tue, 29 Aug 2023 14:43:50 +0000 (20:13 +0530)]
Merge PR #52686 into main
* refs/pull/52686/head:
PendingReleaseNotes: note about mandatory fs argument
doc/cephfs: add note about mandatory --fs argument to snap-schedule
qa: add test for mandatory fs argument to snap-schedule commands
mgr/snap-schedule: tweaks to keep mypy happy
mgr/snap_schedule: validate fs before execution
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
Nizamudeen A [Tue, 29 Aug 2023 14:16:07 +0000 (19:46 +0530)]
mgr/dashboard: fix cephfs create form validator
dashboard was allowing to create a filesystem with / in its name but the
cli throws out error in doing so. And the created volume in dashboard
just gets ended up as a stale volume
Fixes: https://tracker.ceph.com/issues/62628 Signed-off-by: Nizamudeen A <nia@redhat.com>
Ilya Dryomov [Wed, 7 Jun 2023 19:34:07 +0000 (21:34 +0200)]
doc: drop mention of rbd_mirror_journal_max_fetch_bytes option
It was removed in commit 1ef12ea0d29f ("rbd-mirror: remove
rbd_mirror_journal_max_fetch_bytes option") in 2019. Commit 32375cb789d7 ("doc: misc clarity and capitalization") added a "tip"
mentioning it in 2020 in an attempt to capture advice [1].
Lucian Petrut [Thu, 24 Aug 2023 11:41:01 +0000 (11:41 +0000)]
rbd-wnbd: wait for disks to become available
After a WNBD mapping is created, the driver informs Storport that
the bus changed, expecting it to rescan the bus and expose the disk.
This is an asynchronous process and it usually takes a matter of
milliseconds, up to a few seconds under significant load.
The fsx librbd test ocasionally fails to open the disk as it isn't
ready yet. Unlike the Python rbd-wnbd test, it doesn't perform
any polling.
For convenience, WNBD now includes a helper function called
WnbdPollDiskNumber. We're going to use it to wait for the new
disk attachments to become available.
projects / ceph.git / log