luo rixin [Thu, 20 Feb 2020 12:07:39 +0000 (20:07 +0800)]
test/TestOSDScrub: fix mktime() error
The var tm tm isn't initialized, when the tm.tm_isdst is a
positive value, mktime(&tm) return -1 result in test failed
in ubuntu 19.10 for aarch64 GLIBC2.30.
Nizamudeen A [Tue, 23 Mar 2021 07:10:46 +0000 (12:40 +0530)]
mgr/dashboard: Fix for alert notification message being undefined
Prometheus alert notification message in the dashboard always comes up
as undefined. Its because we were showing the alert.summary instead of
alert.description for displaying the message. I couldn't find the
summary field in the ceph_default_alerts.yml file. So removed all the
Summary fields from the dashboard code.
Kefu Chai [Thu, 19 Dec 2019 03:36:59 +0000 (11:36 +0800)]
test/pybind: s/nosetests/python3/
different distros package python3-nose in different ways by adding
different postfix to "/usr/bin/nosetests" to differentiate it from
its python2 counterpart.
* on bionic, python3-nose offers "nosetests3"
* on el8, python3-nose offers "nosetests-3" and "nosetests-3.6"
Kefu Chai [Wed, 31 Mar 2021 11:00:59 +0000 (19:00 +0800)]
mgr/dashboard: encode non-ascii string before passing it to exec_cmd()
because on Python3, tempfile.TemporaryFile() is opened in binary mode by
default, we need to encode non-ascii string before write to it.
otherwise, we have following failure:
def test_unicode_password(self):
self.test_create_user()
password = '\u7ae0\u9c7c\u4e0d\u662f\u5bc6\u7801'
with tempfile.TemporaryFile(mode='w+') as pwd_file:
> pwd_file.write(password)
E UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-5: ordinal not in range(256)
Kefu Chai [Wed, 31 Mar 2021 04:15:17 +0000 (12:15 +0800)]
cmake: allow use libzstd in system
since we are moving the test nodes from bionic to focal, we are able to
use the prebuilt libzstd libraries when running "make check". to speed
up the build and test, in this change:
* add FindZstd.cmake which allows us to use the libzstd in system
* extract BuildZstd.cmake for better readability
* add an option named "WITH_SYSTEM_ZSTD", which defaults to "OFF",
so user can enable it on demand.
Since the v1.4.0 release there have been a few improvements to Zstandard
including improved compression ratios, faster compression, and faster
decompression.
Kefu Chai [Tue, 30 Mar 2021 02:38:43 +0000 (10:38 +0800)]
debian/control: install python3-* packages for "make check"
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
debian/control: this change is not cherry-picked from master,
the corresponding commit in master is 50162091461e42939375475f70ecfd0817f2551c, but that commit also includes
the changes to update the runtime dependencies to python3. but we only
need to update the dependencies for running "make check". so instead
of cherry-picking from master, a separated change is made here.
Conflicts:
debian/control: we still need python 2.7 at runtime
so ignore that change, but we need to use tox instead of
python-tox for running "make check" on focal, so we
need the tox change in this commit.
Dan van der Ster [Tue, 23 Mar 2021 10:28:37 +0000 (11:28 +0100)]
test_ipaddr: check that we correctly skip loopback
We should skip devices named 'lo' or of the form 'lo:0' regardless
of their IP address.
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch> Related-to: https://tracker.ceph.com/issues/49938
(cherry picked from commit 780125d1ed93cd7b17172752b3e76186a524103b)
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch> Fixes: https://tracker.ceph.com/issues/49938
(cherry picked from commit 6147c0917157efd2d35610e759685656a4989abb)
Kefu Chai [Thu, 25 Mar 2021 09:08:48 +0000 (17:08 +0800)]
run-make-check.sh: let ctest generate XML output
to enable XUnit plugin of jenkins to consume the ctest output and
publish it in the dashboard, we need to
* let ctest generate XML output instead of plain text output
* do not fail the test if any test case fails. this allows the publisher
to do its job by checking the XML output.
* prevent ctest from compressing the output. see
https://issues.jenkins.io/browse/JENKINS-21737
Kefu Chai [Tue, 23 Mar 2021 08:06:45 +0000 (16:06 +0800)]
pybind/ceph_daemon: do not fail if prettytable is not available
ubuntu focal does not package python-prettytable. but we need to run
"make check" on focal. it turns out the prettytable is not a must have
for running "make check", so just skip it if it is not around.
this change is not cherry-picked from master, as we have dropped python2
support in master, and python3-prettytable is packged fro python3 on
ubuntu focal. also nautilus is the latest release which has python2
support.
Conflicts:
cmake/modules/CephChecks.cmake
src/test/fio/CMakeLists.txt: check gettid() in /CMakeLists.txt
instead, as nautilus does not have cmake/modules/CephChecks.cmake by
then.
The osd_fast_shutdown option may cause the cluster log to receive
too many entries of 'osd.X reported immediately failed by osd.Y',
depending on cluster scale.
This might be an issue for LMA stacks/tools that check ceph logs
for failed lines, and then require additional logic to filter on
an intended OSD (fast) shutdown; might not be an option/possible,
and require an admin to analyze.
So, add osd_fast_shutdown_notify_mon option for OSD to also tell
the monitor it is shutting down (done in slow/non-fast shutdown)
under osd_fast_shutdown.
This introduces minimal delay (the ack from the mon is required
to prevent the messages), and addresses the cluster log issue.
Note: the osd_mon_shutdown_timeout option can be used to control
the maximum amount of time waiting for the monitor ack to arrive.
Fixes: http://tracker.ceph.com/issues/46978 Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
(cherry picked from commit c75734729764868c5c501722fc8de08dac9ebd4a)
Kefu Chai [Sat, 20 Mar 2021 05:00:01 +0000 (13:00 +0800)]
install-deps.sh: remove existing ceph-libboost of different version
we install different versions of precompiled ceph-libboost packages
for different branches when building and testing them on ubuntu test
nodes. for instance,
- nautilus, octopus: v1.72
- pacific: v1.73
they share the same set of test nodes. and these ceph-libboost packages
conflict with each other, because they install files to the same places.
in order to avoid the confliction, we should uninstall existing packages
before installing a different version of ceph-libboost packages.
ceph-libboost${version}-dev is a package providing the shared headers of
boost library, so, in this change we check if it is installed before
returning or removing the existing packages.
Ilya Dryomov [Wed, 17 Mar 2021 10:00:33 +0000 (11:00 +0100)]
qa: krbd_blkroset.t: update for separate hw and user read-only flags
Since kernel 5.12, hardware read-only state and user read-only
policy (BLKROGET/SET ioctls) are tracked separately in the block
layer. As the purpose of our ->set_read_only() method was exactly
that, it was removed.
As a side effect, BLKROSET no longer returns EROFS on an attempt
to make a read-only mapping read-write with "blockdev --setrw".
The policy gets updated, but the device remains read-only as before
because the hardware (== mapping) state is controlled by the driver.
Neha Ojha [Tue, 9 Mar 2021 00:48:58 +0000 (00:48 +0000)]
pybind/mgr/balancer/module.py: assign weight-sets to all buckets before balancing
Add an additional check to make sure that the choose_args section has the same
number of buckets as the crushmap. If not, ensure that
get_compat_weight_set_weights assigns weight-sets to all buckets.
Without this change, if we end up with an orig_ws, which has fewer buckets
than the crushmap, the mgr will crash due a KeyError in do_crush_compat().
- document the guideline for locking when working with python GIL
- add primitives to extract the patterns for acquiring/releasing
GIL. so they can be reused.
Conflicts:
src/mgr/ActivePyModules.cc
src/mgr/DaemonState.cc
- convert std::pair to DaemonKey::type-name in a few places
- Removed "mds_metadata" which doesn't exist in latest Nautilus
instead of using `std::pair<>` for presenting it. there are three
advantages:
1. better names for readability: `type` and `name` are better names
than `first` and `second`
2. so we can specliaze its behavior by its type: define
`operator<<` once, and use it everywhere. no need to worry
about ADL to pickup the generic operator of `operator<< (..., pair<>)`
anymore. so we can safely use `<< key`, and no need to
use `<< key.first << '.' << key.second`. kill the printed form of
`osd,1` once and for all, all of them are now "osd.1".
3. consolidate the print/parse in a single place
Kefu Chai [Sat, 6 Mar 2021 16:32:42 +0000 (00:32 +0800)]
.github: correct the regex in mileston workflow
also use pull_request_target event so the action is run in the
context of the base of the pull request. this helps us to overcome
the "Resource not accessible by integration" issue where the action
is run in the context of the pull request.
Yaarit Hatuka [Wed, 17 Feb 2021 22:26:59 +0000 (22:26 +0000)]
mgr/telemetry: check if 'ident' channel is active
When compiling the telemetry reports we check for active channels. The
'ident' channel check was missing, hence even if users turned this
channel on, its content could not be appended to the report.
BuildBoost.cmake (used when we're building the submodule) doesn't
provide parity with FindBoost.cmake (used with system Boost).
Specifically, it doesn't set the _FOUND variables for the various
components, making it hard to depend on finding those features.
Set Boost_<component>_FOUND for all the components we're building in
BuildBoost.cmake to make using these variables possible.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
(cherry picked from commit 0f4cb207bb4a9905619894286edd41a89379a747) Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
(cherry picked from commit 4ca4201b7fe3e0ca172548204b4b888a0908d162) Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Conflicts:
src/cls/CMakeLists.txt
src/test/rgw/CMakeLists.txt
- Add spawn headers to includes to fix the two build errors below.
No linking is needed since the files don't use 'spawn::' at all.
In file included from /git/ceph/src/rgw/rgw_common.h:31:0,
from /git/ceph/src/cls/otp/cls_otp_client.cc:25:
/git/ceph/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
#include <spawn/spawn.hpp>
^~~~~~~~~~~~~~~~~
compilation terminated.
src/cls/CMakeFiles/cls_otp_client.dir/build.make:62: recipe for target 'src/cls/CMakeFiles/cls_otp_client.dir/otp/cls_otp_client.cc.o' failed
In file included from /git/ceph/src/rgw/rgw_dmclock_scheduler.h:21:0,
from /git/ceph/src/rgw/rgw_dmclock_sync_scheduler.h:18,
from /git/ceph/src/test/rgw/test_rgw_dmclock_scheduler.cc:17:
/git/ceph/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
#include <spawn/spawn.hpp>
^~~~~~~~~~~~~~~~~
compilation terminated.
src/test/rgw/CMakeFiles/unittest_rgw_dmclock_scheduler.dir/build.make:62: recipe for target 'src/test/rgw/CMakeFiles/unittest_rgw_dmclock_scheduler.dir/test_rgw_dmclock_scheduler.cc.o' failed
projects / ceph.git / log