Or Ozeri [Thu, 19 Nov 2020 06:26:51 +0000 (08:26 +0200)]
librbd: fix crypto prepare_copyup to use new remap_extents api
This commit changes the crypto prepare_copyup implementation to correctly
translate object extents to image extents, via the remap_extents api.
Additionally, we fix relevant unit tests that call this api to correctly expect these calls.
Ken Dreyer [Wed, 18 Nov 2020 16:18:56 +0000 (09:18 -0700)]
rpm: cephadm package is noarch
The cephadm package contains an architecture-independent Python script,
empty directories, and an empty authorized_keys file. There are no
architecture-dependent files here, so we can use a single noarch RPM
across all host architectures.
Zac Dover [Wed, 18 Nov 2020 08:56:14 +0000 (18:56 +1000)]
doc/dev: update unit test declaration
This commit updates the text in the "How
Unit Tests Are Declared" section of
tests-unit-tests.rst. This commit breaks
long sentences into shorter sentences and
breaks a sentence into a bulleted list that
might reduce the reader's cognitive load.
Lucian Petrut [Fri, 13 Nov 2020 12:18:32 +0000 (12:18 +0000)]
compat,msg: improve Windows socket checks
win_socketpair can fail with EADDRINUSE under load, which will
lead to an unhandled exception/crash as per this commit [1].
This change adds a retry, also ensuring that the right error code
gets propagated (the one returned by WSAGetLastError() instead of
the generic SOCKET_ERROR).
While at it, we're fixing the "win_socketpair" indentation and
addressing the SOCKET to int casts.
This change will allow mapping rbd images on Windows, leveraging the
WNBD[1] Virtual Storport Miniport driver [2].
The behavior and CLI is similar to the Linux rbd-nbd, with a few
notable differences:
* device paths cannot be requested. The disk number and path will
be picked by Windows. If a device path is provided by the user
when mapping an image, it will be used as an identifier, which
can also be used when unmapping the image.
* the "show" command was added, which describes a specific mapping.
This can be used for retrieving the disk path.
* the "service" command was added, allowing rbd-wnbd to run as a
Windows service. All mappings are currently perisistent, being
recreated when the service stops, unless explicitly unmapped.
The service disconnects the mappings when being stopped.
* the "list" command also includes a "status" column.
The purpose of the "service" mode is to ensure that mappings survive
reboots and that the Windows service start order can be adjusted so
that rbd images can be mapped before starting services that may depend
on it, such as VMMS.
The mapped images can either be consumed by the host directly or exposed
to Hyper-V VMs.
While at it, we'll skip building rbd-mirror as it's quite unlikely that
this daemon is going to be used on Windows for now.
Lucian Petrut [Tue, 23 Jun 2020 09:00:54 +0000 (09:00 +0000)]
build: disable stack protection on Windows
Passing "-fstack-protector-strong" doesn't seem to work with Mingw,
complaining about undefied "__stack_chk_fail". For this reason,
we'll disable it for now.
Lucian Petrut [Wed, 6 May 2020 10:59:39 +0000 (10:59 +0000)]
rbd: fix import image path parsing on Windows
When importing an image, the rbd command uses only the file name
and expects "/" to be used as a separator. On Windows, it will
use the entire path as image name since the path separator is not
the same.
This change updates it so that the "\\" path separator can be
properly handled as well.
Kefu Chai [Wed, 18 Nov 2020 10:03:55 +0000 (18:03 +0800)]
crimson/os: do not capture unused variables
silences warnings reported by clang iike:
btree_lba_manager.cc:439:50: warning: lambda capture 't' is not used [-Wunused-lambda-capture]
lba_node->get_node_meta().depth).safe_then([=, &t](LBANodeRef c) {
~~~^
1 warning generated.
Kefu Chai [Wed, 18 Nov 2020 09:52:13 +0000 (17:52 +0800)]
cmake: set advice.detachedHead=false when cloning liburing
silence "detached HEAD" warning like:
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
Kefu Chai [Wed, 11 Nov 2020 10:36:22 +0000 (18:36 +0800)]
crimson/osd: add PG::with_clone_obc()
this method replaces `PG::get_or_load_clone_obc()`. so we can
with `seastar::with_lock()` to ensure that `lock.unlock()` is always
called when accessing clone obc.
Kefu Chai [Tue, 10 Nov 2020 13:03:02 +0000 (21:03 +0800)]
crimson/osd: add PG::with_head_obc()
this method replicates `PG::get_or_load_head_obc()`. but uses a different
way to ensure that the "lock" on obc is always released even if the
called func throws. it always guard the called func with a
`with_lock()`, so `lock.unlock()` is always called. the plan is to
replace `PG::get_or_load_head_obc()` with `PG::with_head_obc()` in
the following changes piecemeal.
Zac Dover [Tue, 17 Nov 2020 13:43:03 +0000 (23:43 +1000)]
doc/dev: update unit test 1 of 7
This commit updates the language at the beginning
of the Unit Test section of the developer guide.
The language in this edit is more grammatical than
it used to be, and should now reduce the reader's
cognitive load.
Zac Dover [Tue, 17 Nov 2020 14:02:32 +0000 (00:02 +1000)]
doc/dev: update unit tests 2 of 7
This commit updates the "What does 'make
check' mean" section of the "Unit Tests" chapter
of the Developer Guide. It makes the wording a
bit less editorial and attempts to reduce the
reader's cognitive load.
Patrick Donnelly [Tue, 17 Nov 2020 15:33:20 +0000 (07:33 -0800)]
Merge PR #35749 into master
* refs/pull/35749/head:
Update Release notes for multimds scrub
qa/cephfs: log-ignorelist scrub errors
qa/cephfs: Add more tests for multimds scrub
qa/cephfs: add tests for multimds scrub
qa/cephfs: update existing scrub test cases
mds: don't skip validating disk state of symlink
mds: abort/pause/resume scrubs in multiple mds
mds: track scrub status in multiple mds
mds: remove on_finish from {CInode,CDir}::scrub_info_t
Continuation: don't delete self while there are in-processing stages
mds: auth pin CInode when validating its disk state
mds: rdlock file/nest lock when accumulating stats of subtree dirfrags
mds: multiple mds scrub support
include/frag: add encode/decode functions for fragset_t
mds: remove object can't be scrubbed immediately from scrub stack
mds: prevent dirfrag scrub/fragment from running at the same time
mds: change scrub traverse from post-order to breadth-first search
mds: make both CInode and CDir as entities of scrub
mds: remove ScrubStack::scrubstack
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kamoltat [Mon, 16 Nov 2020 08:52:12 +0000 (08:52 +0000)]
qa/mgr/test_progress: fix bug 48217
Fixes a failing test case regarding osd coming back
after being marked out. The old test case wasn't accounting
for a specific event, therefore this resulted in the failure.
The fix basically accounts for a specific event of osd being
marked in/out.
Or Ozeri [Mon, 2 Nov 2020 14:41:53 +0000 (16:41 +0200)]
librbd: add crypto image dispatch layer
This commit adds a crypto image dispatch layer, in addition to the existing crypto object dispatch layer.
The single purpose of this layer is to translate logical image offset to "on-disk" offsets,
given a crypto header which occupies the beginning of the image data.
This calculation is done via a new remap_extents api,
which is used in to wrap existing Striper translations between object and image extents.
projects / ceph.git / log