Kefu Chai [Wed, 7 Aug 2019 09:46:13 +0000 (17:46 +0800)]
admin/build-doc: use python3
to address https://github.com/sphinx-doc/sphinx/issues/3620, we need to
use sphinx with its fix at
https://github.com/sphinx-doc/sphinx/commit/e049f86b2de1cfdf8a74c88dc9593d047c85d5cb
in other words, we need to use sphinx v2.0.0 and up. but sphinx 2.0
requires python >= 3.5, so we have to use python3 for building the
documents.
in this change:
* doc-requirements.txt: install python3 packages on debian derivatives
* build-doc: install python3.6 packages from EPEL7, and use python3
venv for using sphinx2
* doc-requirements.txt: bump up all python packages to latest
stable.
ceph-volume: do not fail when trying to remove crypt mapper
In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.
Patrick Donnelly [Mon, 23 Sep 2019 12:15:19 +0000 (05:15 -0700)]
Merge PR #29926 into nautilus
* refs/pull/29926/head:
mgr/volumes: return string type to ceph-manager
qa/tasks: test for prevent negative subvolume size
mgr/volumes: prevent negative subvolume size
mgr/volumes: drop unnecessary size
mgr/volumes: cleanup FS subvolume or subvolume group path
mgr/volumes: give useful error message
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
The 'volumes' ceph-mgr module returns a byte type instead of a string
type as the stdout result of commands, `fs subvolume getpath` and
`fs subvolumegroup getpath` to the ceph-mgr. This causes the commands
to hang and the ceph-mgr to utilize 100% CPU. Fix the 'volumes'
ceph-mgr module to return string type to ceph-mgr as the stdout
result of the commands.
Alfonso Martínez [Thu, 22 Aug 2019 13:33:02 +0000 (15:33 +0200)]
mgr/dashboard: run-backend-api-tests.sh CI improvements
As there is now a jenkins job to run this script
(see https://github.com/ceph/ceph-build/pull/1351),
this refactoring adapt the script to be run in a jenkins job as well as locally.
otherwise we have
```
Running setup.py egg_info for package apache-libcloud
libcloud/utils/py3.py:76: UserWarning: Missing backports.ssl_match_hostname package
warnings.warn("Missing backports.ssl_match_hostname package")
Traceback (most recent call last):
...
File "build/bdist.linux-x86_64/egg/setuptools_scm/version.py", line 66, in _warn_if_setuptools_outdated
...
setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)
...
Command python setup.py egg_info failed with error code 1 in /tmp/tmp.i95Fg82Ea4/venv/build/apache-libcloud
```
and more important thing is, setuptools >= 12 allows us to install the
python modules in venv, without uninstalling existing modules if they
are also installed in by system packages.
This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing
install-deps.sh: install `python*-devel` for python*rpm-macros
in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.
but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.
it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".
as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.
once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7
This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing
cmake: enforce C++17 instead of relying on cmake-compile-features
this change is a follow-up of #30114
in FindBoost.cmake, we try to deduce the C++ flags by checking the C++
features offered by different C++ standard like C++98, C++11 and C++14
against the ones required by different boost component. then choose the
C++ dialect based on the `CXX_STANDARD` and `COMPILE_FEATURES`.
in the case of nautilus, since cmake 3.5.1 does not offer
CXX_STANDARD=17 support by then, we have to add `-std=c++17` to
`CMAKE_CXX_FLAGS` manually. but unfortunately, the
`cmake-compile-features` machinary does not check `CMAKE_CXX_FLAGS` for
the default C++ standard required by the project. so it falls back to
the minimal C++ standard required by the target.
since Boost::fiber and Boost::context share the same C++ feature set
dependencies which can be fulfilled by C++11, and by default,
`CXX_EXTENSIONS` is enabled, `--std=gnu++11` is used as we are using gcc
in our building hosts. so we have CXX_FLAGS like:
-std=c++17 ... -std=gnu++11
when compiling targets linked against Boost::context. and it seems the
last one takes effect. that's why some targets failed to compile if
the makefiles were generated with cmake 3.5.1.
in this change, `INTERFACE_COMPILE_FEATURES <features>` is replaced with
`CXX_STANDARD 17`. this renders this setting a no-op, as CMake 3.5.1
does not translate this setting to any CXX_FLAGS. but this does not
hurt, as we use C++17 project-wide, and C++17 is able to offer all
compile features required by Boost::context.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: this change is not cherry-picked from master, because in
master, we use cmake 3.10 and up which offers the support of
`CXX_STANDARD=17`, hence cmake is able to bump up the C++ standard
to C++17 instead of falling back to C++11.
Conflicts: dmclock
this change is not cherry-picked from master, because master is using
a different dmclock version. so in this change, only the minimum fix
is included.
so fmt won't fallback to C++11 if `CMAKE_CXX_STANDARD` is not set
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change is not cherry-picked from master, because
- the seastar submodule in nautilus is not in sync with that in
master
- master is not using fmt submodule included by seastar anymore
- master is using cmake 3.10, so there is no need to address
this issue in master on fmt front.
Matt Benjamin [Thu, 8 Aug 2019 20:21:24 +0000 (16:21 -0400)]
rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 0d3338c2d2271bbf6f4c7774ad175f7ee42d07f6)
If we allocate a new extent that is contiguous with the last extent,
just extend it. This avoids having long vectors of continguous extents
when a single large extent would suffice--especially with log files.
Sage Weil [Fri, 9 Aug 2019 17:25:59 +0000 (12:25 -0500)]
os/bluestore: do not set osd_memory_target default from cgroup limit
On the aarch64 box I'm testing, this gives us a value of 7378697629483768832, which is not what we want.
I think we are better off relying on this limit being explicitly set via
environment variables (POD_* by kuberentes/rook) or via the command line.
This partially reverts 5c6b533697814af8acfd9e731a2599b2294687ef, but not
all of it, since we wan to keep the option itself, as it is now used by
common/config.cc when dealing with the POD_MEMORY_LIMIT env var.
projects / ceph.git / log