install-deps.sh: install `python*-devel` for python*rpm-macros
in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.
but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.
it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".
as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.
once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7
cmake: enforce C++17 instead of relying on cmake-compile-features
this change is a follow-up of #30114
in FindBoost.cmake, we try to deduce the C++ flags by checking the C++
features offered by different C++ standard like C++98, C++11 and C++14
against the ones required by different boost component. then choose the
C++ dialect based on the `CXX_STANDARD` and `COMPILE_FEATURES`.
in the case of nautilus, since cmake 3.5.1 does not offer
CXX_STANDARD=17 support by then, we have to add `-std=c++17` to
`CMAKE_CXX_FLAGS` manually. but unfortunately, the
`cmake-compile-features` machinary does not check `CMAKE_CXX_FLAGS` for
the default C++ standard required by the project. so it falls back to
the minimal C++ standard required by the target.
since Boost::fiber and Boost::context share the same C++ feature set
dependencies which can be fulfilled by C++11, and by default,
`CXX_EXTENSIONS` is enabled, `--std=gnu++11` is used as we are using gcc
in our building hosts. so we have CXX_FLAGS like:
-std=c++17 ... -std=gnu++11
when compiling targets linked against Boost::context. and it seems the
last one takes effect. that's why some targets failed to compile if
the makefiles were generated with cmake 3.5.1.
in this change, `INTERFACE_COMPILE_FEATURES <features>` is replaced with
`CXX_STANDARD 17`. this renders this setting a no-op, as CMake 3.5.1
does not translate this setting to any CXX_FLAGS. but this does not
hurt, as we use C++17 project-wide, and C++17 is able to offer all
compile features required by Boost::context.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: this change is not cherry-picked from master, because in
master, we use cmake 3.10 and up which offers the support of
`CXX_STANDARD=17`, hence cmake is able to bump up the C++ standard
to C++17 instead of falling back to C++11.
Conflicts: dmclock
this change is not cherry-picked from master, because master is using
a different dmclock version. so in this change, only the minimum fix
is included.
so fmt won't fallback to C++11 if `CMAKE_CXX_STANDARD` is not set
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change is not cherry-picked from master, because
- the seastar submodule in nautilus is not in sync with that in
master
- master is not using fmt submodule included by seastar anymore
- master is using cmake 3.10, so there is no need to address
this issue in master on fmt front.
Matt Benjamin [Thu, 8 Aug 2019 20:21:24 +0000 (16:21 -0400)]
rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 0d3338c2d2271bbf6f4c7774ad175f7ee42d07f6)
If we allocate a new extent that is contiguous with the last extent,
just extend it. This avoids having long vectors of continguous extents
when a single large extent would suffice--especially with log files.
Sage Weil [Fri, 9 Aug 2019 17:25:59 +0000 (12:25 -0500)]
os/bluestore: do not set osd_memory_target default from cgroup limit
On the aarch64 box I'm testing, this gives us a value of 7378697629483768832, which is not what we want.
I think we are better off relying on this limit being explicitly set via
environment variables (POD_* by kuberentes/rook) or via the command line.
This partially reverts 5c6b533697814af8acfd9e731a2599b2294687ef, but not
all of it, since we wan to keep the option itself, as it is now used by
common/config.cc when dealing with the POD_MEMORY_LIMIT env var.
Jason Dillaman [Thu, 13 Jun 2019 19:22:54 +0000 (15:22 -0400)]
librbd: tweaks to improve throughput for journaled IO
Increase the per-object append flush bytes to 1MiB to mimic the 16MiB
in-memory writeback cache. Finally, default enable the new journal
writethrough until flush option.
Performance results comparing no journal, pre-change, and post-change:
no journal original modified
4K 40310.84 7223.73 27427.27
32K 21530.37 3256.39 10284.05
256K 3159.67 564.73 1627.16
4M 223.03 48.03 91.91
Fixes: http://tracker.ceph.com/issues/40072 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5f3b043896762fa97eb35648f8639004c45f5978)
Mykola Golub [Thu, 13 Jun 2019 11:05:17 +0000 (12:05 +0100)]
journal: wait for in flight advance sets on stopping recorder
Before object overflow detection optimization (83461c42b) the
overflow was detected when trying to send data, so when an object
set was advanced we always had some data and flush always
completed later. But now we detect overflow on client side, it
may happen when buffer is empty and flush may complete
when "advance object set" is still in flight.
Mykola Golub [Fri, 24 May 2019 06:07:51 +0000 (07:07 +0100)]
journal: optimize object overflow detection
Previously to detect overflow we were sending journal append
requests until -EOVERFLOW is returned by osd. This means that we
had at least one waste (rejected) request per object set (though
there may be more if the number of in-flight appends is not
limited).
We can easily predict when the osd will start to return
-EOVERFLOW and avoid such additional requests.
Jason Dillaman [Tue, 16 Jul 2019 17:54:16 +0000 (13:54 -0400)]
mon/mgr: add 'rbd_support' to list of always-on mgr modules
The 'rbd_support' mgr module provides many background functions that
are required by the 'rbd' CLI performance queries, as well as future
background long-running maintenance tasks.
Fixes: http://tracker.ceph.com/issues/40790 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7231aab4b844729c1c35da37515dce58026c26a2)
Jason Dillaman [Fri, 2 Aug 2019 15:57:29 +0000 (11:57 -0400)]
pybind/mgr/rbd_support: use image ids to detect duplicate tasks
This helps to to avoid the case where new tasks were not being scheduled
when an image name was re-used after having a task created under the
same name.
Fixes: https://tracker.ceph.com/issues/41032 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7adb17f83106b2cba87cf343c23fd6d2d26ae0b3)
Jason Dillaman [Thu, 25 Jul 2019 16:36:19 +0000 (12:36 -0400)]
pybind/mgr: handle duplicate rbd task commands
The 'ceph' CLI will duplicate commands within teuthology to test
the MONs idempotency. This shouldn't be required for the MGR module,
but we can keep a fixed set of completed tests to handle this
possible command replay.
This callback can be used to track progress and also to attempt to cancel
the operation while it's in-progress by returning a negative error code
from the callback.
librbd: the first post-migration snapshot isn't always dirty
Currently, the first post-migration snapshot is always marked EXISTS
(i.e. dirty). This is wrong, because the data can be inherited from
a pre-migration snapshot, handled by deep copy.
Mark all post-migration snapshots EXISTS_CLEAN in this case.
librbd: don't update snapshot object maps if copyup data is all zeros
If the data read from the parent is all zeros, deep copyup isn't
performed. However snapshot object maps are updated unconditionally,
causing inconsistencies where nonexistent objects are marked
OBJECT_EXISTS or OBJECT_EXISTS_CLEAN.
Jan Fajerski [Thu, 15 Aug 2019 10:20:00 +0000 (12:20 +0200)]
ceph-volume: don't keep device lists as sets
This was introduced by #27754. The explicit device lists were cast to
sets but other parts of the code where not updated accordingly. To avoid
touching all code places, only cast to sets for disjoint test and keep
lists otherwise.
Fixes: https://tracker.ceph.com/issues/41292 Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0534cf188a671096d5ddb9d48cdae3dccc6c0b18)
projects / ceph.git / log