Zengran Zhang [Sun, 17 Mar 2019 02:05:11 +0000 (10:05 +0800)]
OSD: rollforward may need to mark pglog dirty
if we rollforward at the end of PG::activate(), we may advance the *crt*,
but we did not mart the log dirty, this means we will not update the crt
within the transaction of rollforward, so it is inconsistent.
Kefu Chai [Thu, 10 Oct 2019 01:54:50 +0000 (09:54 +0800)]
ceph.spec.in: use python_provide macro
our python3 bindings are now named `python3-<modname>` after python3 is
now maintained by RHEL/CentOS instead EPEL. to help the users using
`python36-<modname>`, we should "Provide" `python36-<modname>`.
the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel. and we do install
`python36-devel` in install-deps.sh, and install `python3-devel` in
ceph-*build/build/setup_rpm
see also
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides
Kefu Chai [Thu, 30 May 2019 15:44:37 +0000 (23:44 +0800)]
qa/standalone/ceph-helpers: resurrect all OSD before waiting for health
address the regression introduced by e62cfceb
in e62cfceb, we wanted to test the newly introduced TOO_FEW_OSDS
warning, so we increased the number of OSD to the size of pool, so if
the number of OSD is less than pool size, monitor will send a warning
message.
but we need to bring all OSDs back if we are expecting a healthy
cluster. in this change, all OSDs are resurrect before
`wait_for_health_ok`.
Conflicts
src/mon/PGMap.cc
- cct->_conf->get_val in luminous
src/common/options.cc
- no mon_max_snap_prune_per_epoch in luminous
- in luminous, osd_pool_default_size is INT_T (int64_t). In master, it is UINT_T
(uint64_t). So I changed the type to int64_t.
osd/OSDMap: do not trust partially simplified pg_upmap_item
If we simplified a partially no-op pg_upmap_item, we shall still
continue to verify that the remaining part is valid.
The bug is introduced by 02e5499b350bcd7d9eac98b2072052a9a4a1f535,
before which we always validate the correctness of a pg_upmap_item
before trying to cancel or simplify it.
mon/OSDMonitor: trim no-longer-exist failure reporters
remove the report from failure_info in OSDMonitor::check_failure(), if
the reporter does not exist in the osdmap any longer. otherwise, we will
run into assert() failure when trying to lookup the osd in osdmap using
`osdmap.get_xinfo()`.
tools/rbd-ggate: close log before running postfork
Otherwise the assert in Log:open() will trigger since
all data is kept during the fork and also the `is_started` flag
Fixes: https://tracker.ceph.com/issues/41592 Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit 21bdd4c6ecd24b2a7dbf8691705f228ac11481ad)
Conflicts:
src/tools/rbd_ggate/main.cc : Resolved in do_map
Jason Dillaman [Wed, 19 Jun 2019 15:09:54 +0000 (11:09 -0400)]
rbd: use the ordered throttle for the export action
This allows exports to STDOUT to use multiple concurrent operations
and also fixes a potential race condition with concurrent callbacks
and file seeking.
Fixes: http://tracker.ceph.com/issues/40435 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 751bf6957020ad02b3a29b005b9792b9c6547f96)
Conflicts:
src/tools/rbd/action/Export.cc
- mimic has g_conf->get_val and int64_t "rbd_concurrent_management_ops"
Kefu Chai [Fri, 30 Aug 2019 11:49:28 +0000 (19:49 +0800)]
ceph.spec.in: s/pkgversion/version_nodots/
`python3_pkgversion` is now defined as 3, while we don't have packages
like python3-Cython yet in EPEL7. but we do have `python36-Cython`. so
let's use `python3_version_nodots` instead.
tl;dr: this change addresses the failures of "make check" runs on arm64
builders when they try to build `mgr-dashboard-test-venv` target.
long story: without this change, we will fail to pull in
setuptools >= 36, and as a result pip will fail to import
`setuptools.build_meta` in `pip/_vendor/pep517/_in_process.py`. and will
a `BackendUnavailable` exception thrown by `_call_hook()` in
`pip/_vendor/pep517/wrappers.py`. since the issue addressed by 30ce5e55
has been addressed since setuptools >= 36.0.1, we should be safe to
upgrade to the latest setuptools now.
Kefu Chai [Wed, 19 Dec 2018 08:37:55 +0000 (16:37 +0800)]
install-deps.sh: skip unavailable repos
as centos-sclo-rh-source leads us to 404 at this moment. and we are not
using the source repo for building ceph. so we can just skip any
unavailable repo.
Kefu Chai [Sat, 10 Nov 2018 21:33:43 +0000 (13:33 -0800)]
install-deps: install setuptools before upgrading virtualenv
this should address the failures when running install-deps.sh, like
Downloading/unpacking virtualenv
Running setup.py egg_info for package virtualenv
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
warnings.warn(msg)
error in virtualenv setup command: 'extras_require' must be a
dictionary whose values are strings or lists of strings containing valid
project/version requirement specifiers.
Complete output from command python setup.py egg_info:
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
warnings.warn(msg)
error in virtualenv setup command: 'extras_require' must be a dictionary
whose values are strings or lists of strings containing valid
project/version requirement specifiers.
this only happens on very old virtualenv shipped with RHEL7.4
Nathan Cutler [Wed, 1 Aug 2018 10:33:58 +0000 (12:33 +0200)]
build/ops: unify command substitution in install-deps.sh
The $() form is preferable to `` because folks (like me) might be using
` as a keyboard shortcut to GNU Screen, causing havoc to ensue whenever
copy-pasting the ` character.
install-deps.sh: install `python*-devel` for python*rpm-macros
in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.
but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.
it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".
as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.
once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7
Brad Hubbard [Thu, 22 Nov 2018 00:07:22 +0000 (10:07 +1000)]
install-deps.sh: Remove CR repo
Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.
Nathan Cutler [Wed, 1 Aug 2018 10:52:45 +0000 (12:52 +0200)]
build/ops: refrain from installing/using lsb_release in install-deps.sh
Unfortunately the mapping between release number and codename (which is only
relevant for Debian and Ubuntu btw) is not available from /etc/os-release.
In that one respect, lsb_release was "better".
However, when I weigh the advantages of obtaining that mapping from an external
tool, with the (substantial) risk that the external dependency might cause
trouble on one or more supported distros (to say nothing of the non- or
semi-/pseudo-supported ones), against the work involved in maintaining a
hard-coded mapping (negligible), the needle on my scale immediately swings
toward eliminating the dependency.
Also, I see this commit as part of the longer-term effort to completely expunge
lsb_release from our codebase. See git log --grep lsb_release.
For another example of an external distro-detection tool (albeit one that was
included in Python 2) gone awry, see http://tracker.ceph.com/issues/18163.
Kefu Chai [Thu, 28 Dec 2017 02:18:11 +0000 (10:18 +0800)]
install-deps.sh: check if have access to stdout
$- is "hB" if launched via command line, and it is "himBH" only when
we are actually *in* an interactive shell. so checking "test -t 1" is
what we want.
Kefu Chai [Fri, 8 Dec 2017 08:34:59 +0000 (16:34 +0800)]
install-deps.sh: use DTS on centos if GCC is too old
please note, run-make-check.sh sources install-deps.sh here to import
the $PATH and other environmental variables, which could be changed by
the the DTS "enable" script.
Nathan Cutler [Wed, 2 Oct 2019 12:19:04 +0000 (14:19 +0200)]
qa/rados/upgrade/jewel-x-singleton: do not run on Bionic
The test was failing because it did not explicitly set "os_type" and
"os_version", and teuthology defaulted to Ubuntu Bionic.
Jewel is not supported on Ubuntu Bionic, so we should not try to run tests on
that Ubuntu version.
NOTE: This commit is not cherry-picked from master because the fix is
luminous-specific (there is no "rados/upgrade/jewel-x-singleton" test after
luminous).
qa/rados/upgrade/jewel-x-singleton: set RBD_FEATURES
This commit cannot be cherry-picked from master because luminous is the only
non-EOL stable branch that still includes the
qa/rados/upgrade/jewel-x-singleton suite.
projects / ceph.git / log