Sort through and batch bucket instances so that multiple calls to reading
current bucket info and locking can be avoided. For the most trivial case when
the bucket is already deleted we exit early with all the stale instances. When
the bucket reshard is in progress we only process the stale entries with status
done, if the bucket is available for locking then we lock down and mark the
other instances as well.
rgw: implement listing stale instances from a reshard
Dynamic resharding used to leave behind stale bucket instances; walk through the
metadata pool and identify these instances by comparing the reshard status. If
the reshard status is done, these instances are ok to be cleared. For reshard
status of none we compare against the bucket entry point to ensure that we don't
match the current entry point.
Sage Weil [Fri, 16 Nov 2018 13:11:15 +0000 (07:11 -0600)]
Merge PR #25052 into master
* refs/pull/25052/head:
mgr: lock pg_map too for osd_pool_stats and notify_osdmap
mgr/DaemonServer: use with_osdmap_and_pgmap instead of nested calls
mgr/ClusterState: add with_osdmap_and_pgmap
hsiang41 [Wed, 7 Nov 2018 14:05:35 +0000 (22:05 +0800)]
mgr: Separate diskprediction cloud plugin from the diskprediction plugin
Separate diskprediction local cloud from the diskprediction plugin.
Devicehealth invoke device prediction function related on the global
configuration "device_failure_prediction_mode".
Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
in 37f73607 and 3ec41030, we detect invalid "k". and we should always
detect invalid combinations of parameters in
ErasureCodeInterface::init(), and return -EINVAL instead of "autotune" them
as put in the commit message of c5bafae6, if the profile is invalid for
given erasure plugin. so ErasureCodeBench::encode() and
ErasureCodeBench::decode() will bail out with the error message returned by
ErasureCodeInterface::init() if the parameters are not valid.
also, i checked all ErasureCodeInterface::init() implementations, none
of them is overwriting the settings without returning a non-zero return
code. so we are not overriding or autotuning the settings specified by
user.
Kefu Chai [Thu, 15 Nov 2018 05:56:19 +0000 (13:56 +0800)]
tools/ceph_kvstore_tool: do not open rocksdb when repairing it
before this change, the `need_open_db` parameter is passed to the
constructor of BlueStore as `min_alloc_size`. and rocksdb will fail to
repair because Repairer::Run() also tries to acquire the db lock, and it
will fail to do so if the lock file is already acquired by
BlueStore::_mount().
Kefu Chai [Thu, 15 Nov 2018 01:47:27 +0000 (09:47 +0800)]
qa: use FOUND_VAR to be backward compatible with cmake 2.8.12
before this change, we assume that the variable set if rados::radospp is
found will be radospp_FOUND, but this is a feature cmake 3, see
https://cmake.org/cmake/help/v3.3/module/FindPackageHandleStandardArgs.html
while the cmake shipped by centos is cmake 2.8.12, where the variable
name will be <UPPERCASED_NAME>_FOUND, see
https://cmake.org/cmake/help/v2.8.12/cmake.html#module:FindPackageHandleStandardArgs
in the test of test_envlibrados_for_rocksdb.sh, we are using cmake not
the cmake3 offered by EPEL7, so RADOSPP_FOUND will be set instead. that's why
executable env_librados_test will fail to link against rados::radospp.
as rados::radospp won't be defined if radospp_FOUND is not defined/set.
after this change, the 2nd mode of FIND_PACKAGE_HANDLE_STANDARD_ARGS()
is used instead to ensure that radospp_FOUND is defined even if cmake
2.8.12 is used.
also, the message() commands for debugging purpose are removed.
Kefu Chai [Thu, 15 Nov 2018 03:34:29 +0000 (11:34 +0800)]
qa/suites: add librados2 to "extra_packages" for upgrade tests
we use the playbook of "testnodes.yml" defined by ceph-cm-ansible for
initializing test nodes, and the role of "testnode" is used by
testnodes.yml. "testnode" requires "qemu-system-x86" or "qemu-kvm"
package to be installed. the qemu in turn depends on librbd1 and
librados2.
before librados3 was introduced, this worked perfectly. because in ceph
repo, qa/packages/packages.yaml defines the default set of packages the
"install" tasks should install. and in that yaml file, librados2 was
listed. so the package management system will overwrite the librados2
installed by ansible playbook with the version specified by the
"install" task, as apt/yum thinks this is what user requires explicitly,
so it's fine to install a different version of librados2.
after librados3 was introduced, librados2 was removed from
qa/packages/packages.yaml. because, by default, we need to install
librados3 instead of librados2 for ready a nautilus cluster. but the
problem is, the packge list also applies to "install" tasks installing
releases before nautilus, where we still need to replace the librados2
installed by ansible.
so, to address this issue, "librados2" is added to "extra_packages" of
the "install" tasks of tests installing old releases to install
librados2 explicitly instead of as a dependency of other ceph packages
like librbd1.
Kefu Chai [Tue, 13 Nov 2018 08:45:10 +0000 (16:45 +0800)]
os/tests: silence -Wsign-compare warning
silence warning like
In file included from
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/objectstore/store_test.cc:25:0:
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/googletest/googletest/include/gtest/gtest.h:
In instantiation of 'testing::AssertionResult
testing::internal::CmpHelperEQ(const char*, const char*, const T1&,
const T2&) [with T1 = int; T2 = long unsigned int]':
Patrick Donnelly [Tue, 13 Nov 2018 21:03:02 +0000 (13:03 -0800)]
Merge PR #24490 into master
* refs/pull/24490/head:
mds: flush dirty dirfrags that weren't logged when deactivating mds
mds: use MDlog::trim_all() to trim log when deactivating mds
mds: don't cap log when there are replicated objects
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Alfredo Deza [Mon, 12 Nov 2018 17:57:26 +0000 (12:57 -0500)]
ceph-volume tests patch Device() by splitting parametrized method
This was causing failures on systems where there is no LVM or where the
device names don't match. Patching is always recommended to avoid
conflicts with the system testing
alfonsomthd [Mon, 12 Nov 2018 10:51:02 +0000 (11:51 +0100)]
mgr/dashboard: add pytest plugin: faulthandler
- Plugin for pytest that automatically enables the faulthandler module during tests.
- Very useful when you run tests and face segmentation faults.
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Jerry Lee [Mon, 12 Nov 2018 06:27:53 +0000 (14:27 +0800)]
ceph-mgr: hold lock while accessing the request list and submitting request
The request creation can fire up the notify event early and it can cause
a race condition where the actual request was not yet added to the
self.requests list which makes the submit_request() function waits
forever without accepting new requests.
Kefu Chai [Sat, 10 Nov 2018 21:33:43 +0000 (13:33 -0800)]
install-deps: install setuptools before upgrading virtualenv
this should address the failures when running install-deps.sh, like
Downloading/unpacking virtualenv
Running setup.py egg_info for package virtualenv
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
warnings.warn(msg)
error in virtualenv setup command: 'extras_require' must be a
dictionary whose values are strings or lists of strings containing valid
project/version requirement specifiers.
Complete output from command python setup.py egg_info:
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
warnings.warn(msg)
error in virtualenv setup command: 'extras_require' must be a dictionary
whose values are strings or lists of strings containing valid
project/version requirement specifiers.
this only happens on very old virtualenv shipped with RHEL7.4
projects / ceph-ci.git / log