Kefu Chai [Wed, 31 Mar 2021 11:00:59 +0000 (19:00 +0800)]
mgr/dashboard: encode non-ascii string before passing it to exec_cmd()
because on Python3, tempfile.TemporaryFile() is opened in binary mode by
default, we need to encode non-ascii string before write to it.
otherwise, we have following failure:
def test_unicode_password(self):
self.test_create_user()
password = '\u7ae0\u9c7c\u4e0d\u662f\u5bc6\u7801'
with tempfile.TemporaryFile(mode='w+') as pwd_file:
> pwd_file.write(password)
E UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-5: ordinal not in range(256)
Kefu Chai [Tue, 30 Mar 2021 02:38:43 +0000 (10:38 +0800)]
debian/control: install python3-* packages for "make check"
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
debian/control: this change is not cherry-picked from master,
the corresponding commit in master is 50162091461e42939375475f70ecfd0817f2551c, but that commit also includes
the changes to update the runtime dependencies to python3. but we only
need to update the dependencies for running "make check". so instead
of cherry-picking from master, a separated change is made here.
Conflicts:
debian/control: we still need python 2.7 at runtime
so ignore that change, but we need to use tox instead of
python-tox for running "make check" on focal, so we
need the tox change in this commit.
Dan van der Ster [Tue, 23 Mar 2021 10:28:37 +0000 (11:28 +0100)]
test_ipaddr: check that we correctly skip loopback
We should skip devices named 'lo' or of the form 'lo:0' regardless
of their IP address.
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch> Related-to: https://tracker.ceph.com/issues/49938
(cherry picked from commit 780125d1ed93cd7b17172752b3e76186a524103b)
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch> Fixes: https://tracker.ceph.com/issues/49938
(cherry picked from commit 6147c0917157efd2d35610e759685656a4989abb)
Kefu Chai [Thu, 25 Mar 2021 09:08:48 +0000 (17:08 +0800)]
run-make-check.sh: let ctest generate XML output
to enable XUnit plugin of jenkins to consume the ctest output and
publish it in the dashboard, we need to
* let ctest generate XML output instead of plain text output
* do not fail the test if any test case fails. this allows the publisher
to do its job by checking the XML output.
* prevent ctest from compressing the output. see
https://issues.jenkins.io/browse/JENKINS-21737
Kefu Chai [Tue, 23 Mar 2021 08:06:45 +0000 (16:06 +0800)]
pybind/ceph_daemon: do not fail if prettytable is not available
ubuntu focal does not package python-prettytable. but we need to run
"make check" on focal. it turns out the prettytable is not a must have
for running "make check", so just skip it if it is not around.
this change is not cherry-picked from master, as we have dropped python2
support in master, and python3-prettytable is packged fro python3 on
ubuntu focal. also nautilus is the latest release which has python2
support.
Conflicts:
cmake/modules/CephChecks.cmake
src/test/fio/CMakeLists.txt: check gettid() in /CMakeLists.txt
instead, as nautilus does not have cmake/modules/CephChecks.cmake by
then.
Kefu Chai [Sat, 20 Mar 2021 05:00:01 +0000 (13:00 +0800)]
install-deps.sh: remove existing ceph-libboost of different version
we install different versions of precompiled ceph-libboost packages
for different branches when building and testing them on ubuntu test
nodes. for instance,
- nautilus, octopus: v1.72
- pacific: v1.73
they share the same set of test nodes. and these ceph-libboost packages
conflict with each other, because they install files to the same places.
in order to avoid the confliction, we should uninstall existing packages
before installing a different version of ceph-libboost packages.
ceph-libboost${version}-dev is a package providing the shared headers of
boost library, so, in this change we check if it is installed before
returning or removing the existing packages.
Ilya Dryomov [Wed, 17 Mar 2021 10:00:33 +0000 (11:00 +0100)]
qa: krbd_blkroset.t: update for separate hw and user read-only flags
Since kernel 5.12, hardware read-only state and user read-only
policy (BLKROGET/SET ioctls) are tracked separately in the block
layer. As the purpose of our ->set_read_only() method was exactly
that, it was removed.
As a side effect, BLKROSET no longer returns EROFS on an attempt
to make a read-only mapping read-write with "blockdev --setrw".
The policy gets updated, but the device remains read-only as before
because the hardware (== mapping) state is controlled by the driver.
Kefu Chai [Sat, 6 Mar 2021 16:32:42 +0000 (00:32 +0800)]
.github: correct the regex in mileston workflow
also use pull_request_target event so the action is run in the
context of the base of the pull request. this helps us to overcome
the "Resource not accessible by integration" issue where the action
is run in the context of the pull request.
BuildBoost.cmake (used when we're building the submodule) doesn't
provide parity with FindBoost.cmake (used with system Boost).
Specifically, it doesn't set the _FOUND variables for the various
components, making it hard to depend on finding those features.
Set Boost_<component>_FOUND for all the components we're building in
BuildBoost.cmake to make using these variables possible.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
(cherry picked from commit 0f4cb207bb4a9905619894286edd41a89379a747) Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
(cherry picked from commit 4ca4201b7fe3e0ca172548204b4b888a0908d162) Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Conflicts:
src/cls/CMakeLists.txt
src/test/rgw/CMakeLists.txt
- Add spawn headers to includes to fix the two build errors below.
No linking is needed since the files don't use 'spawn::' at all.
In file included from /git/ceph/src/rgw/rgw_common.h:31:0,
from /git/ceph/src/cls/otp/cls_otp_client.cc:25:
/git/ceph/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
#include <spawn/spawn.hpp>
^~~~~~~~~~~~~~~~~
compilation terminated.
src/cls/CMakeFiles/cls_otp_client.dir/build.make:62: recipe for target 'src/cls/CMakeFiles/cls_otp_client.dir/otp/cls_otp_client.cc.o' failed
In file included from /git/ceph/src/rgw/rgw_dmclock_scheduler.h:21:0,
from /git/ceph/src/rgw/rgw_dmclock_sync_scheduler.h:18,
from /git/ceph/src/test/rgw/test_rgw_dmclock_scheduler.cc:17:
/git/ceph/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
#include <spawn/spawn.hpp>
^~~~~~~~~~~~~~~~~
compilation terminated.
src/test/rgw/CMakeFiles/unittest_rgw_dmclock_scheduler.dir/build.make:62: recipe for target 'src/test/rgw/CMakeFiles/unittest_rgw_dmclock_scheduler.dir/test_rgw_dmclock_scheduler.cc.o' failed
Casey Bodley [Wed, 6 Nov 2019 20:57:01 +0000 (15:57 -0500)]
rgw: use new spawn() implementation
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 769841a08c3e79985d9634f06c9ff4d62647dcda) Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Conflicts:
src/rgw/CMakeLists.txt
- Remove changes for 'rgw_schedulers' cmake target, not in Nautilus.
- Link 'radosgw_a' against 'spawn'; transitivity from 'rgw_schedulers'
(which is public) is lost, and 'rgw_a'/'rgw_libs' (which is private
to 'radosgw_a') isn't enough to build 'rgw_main.cc' ( error below.)
src/rgw/rgw_aio.cc
- This file doesn't exist in Nautilus; similar changes are done in
other files.
src/rgw/rgw_aio_throttle.h
- No changes required; the base for the changes (e.g., class, variables)
are not in Nautilus.
src/rgw/rgw_asio_frontend.cc
- Less changes required, similarly; commit dd4350b not in Nautilus.
Build error:
In file included from /git/ceph/src/rgw/rgw_common.h:31:0,
from /git/ceph/src/rgw/rgw_main.cc:15:
/git/ceph/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
#include <spawn/spawn.hpp>
^~~~~~~~~~~~~~~~~
compilation terminated.
src/rgw/CMakeFiles/radosgw.dir/build.make:62: recipe for target 'src/rgw/CMakeFiles/radosgw.dir/rgw_main.cc.o' failed
Igor Fedotov [Fri, 26 Feb 2021 14:16:11 +0000 (17:16 +0300)]
os/bluestore: go beyond pinned onodes while trimming the cache.
One might face lack of cache trimming when there is a bunch of pinned entries on the top of Onode's cache LRU list. If these pinned entries stay in the state for a long time cache might start using too much memory causing OSD to go out of osd-memory-target limit. Pinned state tend to happen to osdmap onodes.
The proposed patch preserves last trim position in the LRU list (if it pointed to a pinned entry) and proceeds trimming from that position if it wasn't invalidated. LRU nature of the list enables to do that safely since no new entries appear above the previously present entry while it's not touched.
Fixes: https://tracker.ceph.com/issues/48729 Signed-off-by: Igor Fedotov <ifedotov@suse.com>
Adam Kupczyk [Sat, 30 Jan 2021 11:57:05 +0000 (12:57 +0100)]
os/bluestore: Add option to check BlueFS reads
Add option "bluefs_check_for_zeros" to check if there are any zero-filled page.
If so, reread data. It is known that sometimes BlueStore gets such pages.
See "bluestore_retry_disk_reads".
- docstring added to describe the link to mgr/prometheus conflicted with the
const fmt definition for the message. resolved by adding doc under the const
definition.
Paul Cuzner [Thu, 8 Oct 2020 03:30:56 +0000 (16:30 +1300)]
mgr/prometheus: Add healthcheck metric for SLOW_OPS
SLOW_OPS is triggered by op tracker, and generates a health
alert but healthchecks do not create metrics for prometheus to
use as alert triggers. This change adds SLOW_OPS metric, and
provides a simple means to extend to other relevant health
checks in the future
If the extract of the value from the health check message fails
we log an error and remove the metric from the metric set. In
addition the metric description has changed to better reflect
the scenarios where SLOW_OPS can be triggered.
projects / ceph.git / log