Jason Dillaman [Wed, 19 Jun 2019 15:09:54 +0000 (11:09 -0400)]
rbd: use the ordered throttle for the export action
This allows exports to STDOUT to use multiple concurrent operations
and also fixes a potential race condition with concurrent callbacks
and file seeking.
Fixes: http://tracker.ceph.com/issues/40435 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 751bf6957020ad02b3a29b005b9792b9c6547f96)
Conflicts:
src/tools/rbd/action/Export.cc
- mimic has g_conf->get_val and int64_t "rbd_concurrent_management_ops"
Jeff Layton [Thu, 26 Sep 2019 11:50:28 +0000 (07:50 -0400)]
ceph.spec.in: fix Cython package dependency for Fedora
Fedora distros do not have python3?-Cython packages, but they do have
python3-Cython ones. Fix the BuildRequires so that we only use the
python3_version_nodots based version string for RHEL.
Fixes: https://tracker.ceph.com/issues/42032 Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 40c5a18f81c67c96f6a35e592ac3141b87c19701)
Kefu Chai [Fri, 30 Aug 2019 11:49:28 +0000 (19:49 +0800)]
ceph.spec.in: s/pkgversion/version_nodots/
`python3_pkgversion` is now defined as 3, while we don't have packages
like python3-Cython yet in EPEL7. but we do have `python36-Cython`. so
let's use `python3_version_nodots` instead.
tl;dr: this change addresses the failures of "make check" runs on arm64
builders when they try to build `mgr-dashboard-test-venv` target.
long story: without this change, we will fail to pull in
setuptools >= 36, and as a result pip will fail to import
`setuptools.build_meta` in `pip/_vendor/pep517/_in_process.py`. and will
a `BackendUnavailable` exception thrown by `_call_hook()` in
`pip/_vendor/pep517/wrappers.py`. since the issue addressed by 30ce5e55
has been addressed since setuptools >= 36.0.1, we should be safe to
upgrade to the latest setuptools now.
Kefu Chai [Wed, 19 Dec 2018 08:37:55 +0000 (16:37 +0800)]
install-deps.sh: skip unavailable repos
as centos-sclo-rh-source leads us to 404 at this moment. and we are not
using the source repo for building ceph. so we can just skip any
unavailable repo.
Changcheng Liu [Fri, 26 Oct 2018 06:32:09 +0000 (14:32 +0800)]
install-deps.sh: correct gcc version info as major.minor.patch
On ubuntu 18.04, "gcc -dumpversion" output "7", it result in
that below check will return false:
dpkg --compare-versions 7 ge 7.0
Then, this script will install other gcc.
Actully, the full gcc version "gcc -dumpfullversion" output is
"7.3.0", than below check will turn true:
dpkg --compare-version 7.3.0 ge 7.0
So, there's no need to install other gcc.
In case of hitting error on ubuntu16.04, use below parameter:
gcc -dumpfullversion -dumpversion
Kefu Chai [Sat, 10 Nov 2018 21:33:43 +0000 (13:33 -0800)]
install-deps: install setuptools before upgrading virtualenv
this should address the failures when running install-deps.sh, like
Downloading/unpacking virtualenv
Running setup.py egg_info for package virtualenv
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
warnings.warn(msg)
error in virtualenv setup command: 'extras_require' must be a
dictionary whose values are strings or lists of strings containing valid
project/version requirement specifiers.
Complete output from command python setup.py egg_info:
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
warnings.warn(msg)
error in virtualenv setup command: 'extras_require' must be a dictionary
whose values are strings or lists of strings containing valid
project/version requirement specifiers.
this only happens on very old virtualenv shipped with RHEL7.4
per apt.conf(5), and
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=550564,
"Acquire::Translation" is not documented and will be deprecated. it
would be better to use "Acquire::Languages" solely. and it's consistent
with https://github.com/ceph/ceph-build/pull/1137 .
Nathan Cutler [Wed, 1 Aug 2018 10:33:58 +0000 (12:33 +0200)]
build/ops: unify command substitution in install-deps.sh
The $() form is preferable to `` because folks (like me) might be using
` as a keyboard shortcut to GNU Screen, causing havoc to ensue whenever
copy-pasting the ` character.
Kefu Chai [Wed, 7 Aug 2019 09:46:13 +0000 (17:46 +0800)]
admin/build-doc: use python3
to address https://github.com/sphinx-doc/sphinx/issues/3620, we need to
use sphinx with its fix at
https://github.com/sphinx-doc/sphinx/commit/e049f86b2de1cfdf8a74c88dc9593d047c85d5cb
in other words, we need to use sphinx v2.0.0 and up. but sphinx 2.0
requires python >= 3.5, so we have to use python3 for building the
documents.
in this change:
* doc-requirements.txt: install python3 packages on debian derivatives
* build-doc: install python3.6 packages from EPEL7, and use python3
venv for using sphinx2
* doc-requirements.txt: bump up all python packages to latest
stable.
Matt Benjamin [Thu, 8 Aug 2019 20:21:24 +0000 (16:21 -0400)]
rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 0d3338c2d2271bbf6f4c7774ad175f7ee42d07f6)
Matt Benjamin [Tue, 18 Jun 2019 11:39:09 +0000 (07:39 -0400)]
rgw_file: advance_mtime() should consider namespace expiration
Predictably, slow NFS operations like READDIR will overlap mtime
advance, so don't advance faster than the nfs namespace
expiration timer.
Fixes: http://tracker.ceph.com/issues/40415 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit fc14eeb6dbf7deaf11a70a3d3b48ce41998d4e39)
Jason Dillaman [Thu, 18 Apr 2019 17:17:56 +0000 (13:17 -0400)]
librbd: async open/close should free ImageCtx before issuing callback
The destructor for ImageCtx attempts to access librados::IoCtx objects
that might have been destroyed immediately after the async open (failure)
and close actions completed.
Fixes: http://tracker.ceph.com/issues/39031 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit be419a1d6fd1e176ab4fdebfaa17726bb63d6ac8)
Matt Benjamin [Wed, 11 Sep 2019 17:22:26 +0000 (13:22 -0400)]
rgw: ldap auth: S3 auth failure should return InvalidAccessKeyId
LDAP was out of sync with the other auth modules, and the default
error code is not correct for S3.
Fixes: https://tracker.ceph.com/issues/41760 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 226227cf2d825b3414b7522a9f738180142f3956)
On Debian-based distros, hexdump(1) comes from FreeBSD in bsdmainutils
package (other distros ship a Linux version from util-linux). It turns
out that '-s skip' for anything but a regular file has been broken for
ages in FreeBSD -- instead of calling fseek(), it consumed the input
byte by byte with getchar(). This was fixed in commit 7cd5ae492557
"hexdump(1): Speed up -s flag on devices" last year, but it hasn't made
it to the Debian world yet.
bsdmainutils_9.0.6ubuntu3 in xenial had a custom patch for that, but
it's no longer there in bsdmainutils_11.1.2ubuntu1 in bionic. hexdump
-s <large offset> is now a fancy infinite loop!
ceph-volume: do not fail when trying to remove crypt mapper
In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.
Casey Bodley [Tue, 20 Aug 2019 17:18:45 +0000 (13:18 -0400)]
rgw: increase beast parse buffer size to 64k
the flat_buffer starts very small and rarely grew above 512 bytes when
reading headers. this buffer's capacity() is what determines the size of
reads when when recv_body() calls boost::beast::http::async_read_some().
raising this buffer size to a static 64k greatly reduces the number of
socket reads
Reported-by: Marcus Watts <mwatts@redhat.com> Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f13c6914eba9ad05c7bcff6eda81bb858f0f2349)
projects / ceph.git / log