Kefu Chai [Fri, 5 Aug 2022 08:40:41 +0000 (16:40 +0800)]
cmake: remove spaces in macro used for compiling cython code
we are facing following FTBFS on jammy + GCC-11.2 + Cython 0.29 +
CMake 3.22:
creating /home/jenkins-build/build/workspace/ceph-api/build/lib/cython_modules/temp.linux-x86_64-3.10/home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs
compile options: '-I/usr/include/python3.10 -I/usr/include/python3.10 -c'
extra options: '-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -iquote/home/jenkins-build/build/workspace/ceph-api/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2'
cc: /home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs/cephfs.c
cc: warning: ##: linker input file unused because linking not done
cc: error: ##: linker input file not found: No such file or directory
cc: warning: 0: linker input file unused because linking not done
cc: error: 0: linker input file not found: No such file or directory
it seems cython is not able to escape the space in the "extra options"
anymore, so the "##" and "0" are considered as object files passed to
compiler in addition to cephfs.c.
in this change the spaces are removed to help cython to make the right
decision.
rgw: maintain object instance within RGWRadosObject::get_obj_state method
It seems that the object instance was being stripped from the rgw_obj
inadvertently wihtin the RGWRadosObject::get_obj_state method. This
change fixes some S3 behavior related to concurrency and ECANCELED errors.
We don't backport PRs merged into doc/releases. Therefore, when one browses to an older Ceph release version on docs.ceph.com (e.g., https://docs.ceph.com/en/pacific/), the information is out of date at best.
The doc/releases page is only accurate if browsing https://docs.ceph.com/en/latest/, for example.
So this post_checkout command will make sure we've checked out doc/releases from main before building and publishing.
when option --gpg-url is specified, the name used for the gpg filename is missing and throws an exception
this adds the string "manual" to the gpg key : /etc/apt/trusted.gpg.d/ceph.manual.gpg
Adam King [Tue, 26 Jul 2022 13:55:05 +0000 (09:55 -0400)]
mgr/cephadm: clear error message when resuming upgrade
the message field in the output of "ceph orch upgrade status"
will first take the value of the error field of the UpgradeState,
and if only if it blank/None, display an info string we periodically
update throughout the upgrade with useful info such as that
we're upgrading a daemon of a particular type or pulling an image
on a certain host. When an upgrade fails, we set the error field
of the UpgradeState, pause the upgrade and raise a health warning.
Sometimes, the user is able to resolve the issue and simply resume
the upgrade. The issue here is, in that case, the error field of
the UpgradeState is still set, so instead of seeing the useful info
messages, it will continue to display an error message that may
no longer be relevant. By emptying the error field of the UpgradeState
when upgrades are resumed, we return to normal behavior of
displaying the info string, and will only show another error message
if another error actually occurs.
Marcus Watts [Thu, 7 Jul 2022 07:33:31 +0000 (03:33 -0400)]
rgw: better tenant id from the uri on anonymous access
When anonymous tries access public bucket, it gets 404,
because rgw doesn't check tenant correctly.
A previous fix for this broke legacy implicit tenants,
because it didn't check for anonymous access. This version
restricts its behavior to the anonymous user.
Fixes: https://tracker.ceph.com/issues/48001 https://tracker.ceph.com/issues/48382
Original fix by
Author: Rafał Wądołowski <rafal@rafal.net.pl> Signed-off-by: Rafał Wądołowski <rwadolowski@cloudferro.com>
This fix Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit b5880caa505e2df7ff035537c19a8be3a3a8bb8f)
test/encoding: verify that e.what() starts with expected str
boost changes the way how it prints boost::system::system_error in
boost 1.79 -- it appends the stringified error_category at end of
exception::what(), and our buffer::malformed_input is a subclass
of boost::system::system_error.
so we cannot just compare the return value of what() with the
expected string, to be more future proof, let's check if i
starts with the expected string instead.
Deferred writes can sometimes update regions that are no longer mapped to any object.
This cannot happen when BlueStore is running, as blobs are being held,
and allocations are not released until deferred op is executed.
However in case of restart allocations that deferred is targetting are already freed.
Deferred replay is done on BlueStore bootup, before any new object can be allocated,
so no collision with object is possible.
But BlueFS can allocate space from block with deferred ops still pending.
Fixes: https://tracker.ceph.com/issues/54547 Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 2afb951a490aac9ca4d01614867396cbd06b793d)
Conflicts: src/os/bluestore/BlueStore.cc
Modified non-critical BlueFS call foreach_block_extents to get_block_extents.
Kefu Chai [Mon, 28 Feb 2022 13:46:39 +0000 (21:46 +0800)]
include/buffer: include <memory>
to address following FTBFS:
/usr/bin/ccache /usr/bin/clang++-13 -DBOOST_ALL_NO_LIB -DBOOST_ASIO_DISABLE_CONCEPTS -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_T$
In file included from /var/ssd/ceph/src/crimson/os/seastore/seastore_types.cc:4:
In file included from /var/ssd/ceph/src/crimson/os/seastore/seastore_types.h:14:
In file included from /var/ssd/ceph/src/include/denc.h:47:
/var/ssd/ceph/src/include/buffer.h:98:37: error: no template named 'unique_ptr' in namespace 'std'; did you mean 'boost::movelib::unique_ptr'?
struct unique_leakable_ptr : public std::unique_ptr<T, ceph::nop_delete<T>> {
^~~~~~~~~~~~~~~
boost::movelib::unique_ptr
/opt/ceph/include/boost/move/unique_ptr.hpp:354:7: note: 'boost::movelib::unique_ptr' declared here
class unique_ptr
^
libcephsqlite: ceph-mgr crashes when compiled with gcc12
regex in libcephsqlite, when compiled with GCC12 treats '-' as a range
operator resulting in the following error.
"Invalid start of '[x-x]' range in regular expression"
Kotresh HR [Fri, 4 Feb 2022 09:58:39 +0000 (15:28 +0530)]
qa: validate subvolume discover on upgrade
Validate subvolume discover on upgrade from
legacy subvolume to v1. The handcrafted
`.meta' file on legacy subvolume root should
not be used for any subvolume apis like getpath,
authorize.
Kotresh HR [Fri, 4 Feb 2022 09:25:03 +0000 (14:55 +0530)]
mgr/volumes: Fix subvolume discover during upgrade
Fixes the subvolume discover to use the correct
metadata file after an upgrade from legacy subvolume
to v1. The fix makes sure, it doesn't use the
handcrafted metadata file placed in the subvolume
root of legacy subvolume.
Co-authored-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch> Co-authored-by: Dan van der Ster <daniel.vanderster@cern.ch> Co-authored-by: Ramana Raja <rraja@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 7eba9cab6cfb9a13a84062177d7a0fa228311e13)
(cherry picked from commit f8c04135150a7fb3c43607b43a8214e0d57547bc)
Adam C. Emerson [Tue, 8 Feb 2022 18:47:49 +0000 (13:47 -0500)]
rgw: Fix data race in ChangeStatus
Fixes: https://tracker.ceph.com/issues/54208 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 27f5ba9e5f649d8767c8ab44d56404e0186f6fc1) Fixes: https://tracker.ceph.com/issues/54491 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Fri, 8 Jul 2022 18:58:16 +0000 (14:58 -0400)]
rgw: Guard against malformed bucket URLs
Misplaced colons can result in radosgw thinking is has a bucket URL
but with no bucket name, leading to a crash later on.
Fixes: https://tracker.ceph.com/issues/55765 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 3ee9a3b41a289a926fed8b8927ca2a93b4f120a6) Fixes: https://tracker.ceph.com/issues/56586 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Laura Flores [Mon, 4 Oct 2021 04:41:10 +0000 (04:41 +0000)]
os/bluestore: update priorities and nicks of bluestore perf counters
These perf counters do not show up in telemetry unless they are set to a "useful" priority or higher. Fetching these counters in telemetry may help to diagnose problems with RocksDB / BlueFS prefetching / insufficient cache sizes.
Note: This backport (and the original PR) exposed an extra 34 perf counters/OSD to Prometheus. Given Pacific is a stable release and not to add that much extra load we are adapting this backport and only exposing the 2 required perf-coun>
librbd: bail from schedule_request_lock() if already lock owner
Race condition may be hit if there are multiple pending locks for the
same image and pending callbacks. Abort exclusive lock process if
already exclusive lock owner.
Fixes: https://tracker.ceph.com/issues/56549 Signed-off-by: Christopher Hoffman <choffman@redhat.com>
(cherry picked from commit 3527d2c764626c09c5ede80ae844551fd8845756)
Ilya Dryomov [Fri, 20 May 2022 12:05:03 +0000 (14:05 +0200)]
qa/suites/rbd: disable workunit timeout for dynamic_features_no_cache
The I/O workload in this test is xfstests (qa/run_xfstests_qemu.sh)
which isn't subjected to any timeout other than global max_job_time
limit in any other subsuite (e.g. qemu/workloads/qemu_xfstests.yaml).
But here, there is a parallel "op" workload defined as a workunit.
The workunit task has a default timeout of 3 hours which is effectively
imposed on the entire job. In the "rbd cache = false" configuration,
it's sometimes exceeded.
Tatjana Dehler [Thu, 7 Jul 2022 15:21:14 +0000 (17:21 +0200)]
mgr/dashboard: prevent alert redirect
Prevent Alertmanager alerts from being redirected to the active mgr
dashboard instance. There are two reasons for it:
1. It doesn't bring any additional benefit. The Alertmanager config
includes all available mgr instances - active and passive ones. In
case of an alert, it will be sent to all of them. It ensures that
the active mgr dashboard will receive the alert in any case.
2. The redirect URL includes the mgr IP and NOT the FQDN. This leads
to issues in environments where an SSL certificate is configured and
matches the FQDNs, only.
Fixes: https://tracker.ceph.com/issues/56401 Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 965005e0789e566ccadce7a326b0e197ab8d7f5f)
rbd: don't default empty pool name unless namespace is specified
Commit 96f05a7956b3 ("rbd: delay determination of default pool name")
broke "rbd perf image iostat" and "rbd perf image iotop" GLOBAL_POOL_KEY
support (the ability to blend all rbd pools together into a single
view).
It doesn't really thrash anything, just repeatedly restarts the
workload on top of a dirty cache file. rbd_pwl_cache_recovery is
more on point and gets covered by existing CODEOWNERS.
Yin Congmin [Fri, 7 Jan 2022 07:03:44 +0000 (15:03 +0800)]
qa/tasks: add thrash test for persistent write log cache
add thrash test for persistent write log cache. run rbd bench
on persistent write log cache, thrashes rbd bench, test the
recovery function of persistent write log cache.
Python 3.10 doesn't include the _Py_fopen() function. Boost
1.75.0 includes a patch which switches to using fopen() for
python versions >= 3.1, but Pacific is using boost 1.73.0,
which still uses _Py_fopen(). This commit adds the boost
1.75.0 patch to `make-dist`, so it's applied to our copy of
the boost source which is then used when building RPM packages.
Fixes: https://tracker.ceph.com/issues/56466
please note, this change is not cherry-picked from the
"main" branch. as we use boost 1.75 already in that branch,
but to minimize the risk of switching boost from 1.73 to
1.75 in a LTS branch like pacific, we just add a fix to
address this particular issue in boost 1.73.
Signed-off-by: Tim Serong <tserong@suse.com> Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Igor Fedotov [Mon, 11 Jul 2022 14:59:00 +0000 (17:59 +0300)]
pacific: rgw: do not permit locked object version removal
This commit targets Pacific explicitly and hence "violates" regular
backporting policy due to the fact that Quincy onward releases aren't
impacted by the bug. Post-Pacific releases have got a major update
through https://github.com/ceph/ceph/pull/39399 which in particular fixes the
case.
For reasons we would prefer not to backport this update completely hence
just the partial minor patch here.
Some more details on the bug can be found in the relevant ticket's comments as
well.
Additional note about the implementation: originally the patch relied on using
RGWObject::get_obj function but this implementation causes rgw process
access vialotaion when running some of test cases in s3-tests suite.
The root cause is apparently an uninitialized bucket pointer at RGWObject under
some circumstances. Hence that get_obj() implementation is copy-pasted to
RGWRadosObject::get_obj_state() and altered a bit to use local bucket reference.
Fixes: https://tracker.ceph.com/issues/55766 Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
This check was added in commit ecd3778a6f9a ("rbd-mirror: ensure that
the last non-primary snapshot cannot be pruned") as an additional
safeguard against pruning an incomplete non-primary snapshot in case
there is no predecessor mirror snapshot. However it still fires if the
predecessor is there but happens to be a primary demotion snapshot.
A bogus "incomplete local non-primary snapshot" error is reported and
the replayer gets stuck.
Remove completed_non_primary_snapshots_exist tracking as the presence
of the predecessor in the incomplete non-primary snapshot pruning arm
is already ensured by "m_local_snap_id_start > 0" condition.
Incomplete non-primary snapshot handling is bifurcated depending
on whether any data objects have been copied. If no data objects
have been copied, an incomplete non-primary snapshot is assumed to
be malformed and gets pruned; the sync is restarted from scratch.
Image contexts are reopen even though we pass the context as an
argument. This commit changes that so you can forget about reopening
a rbd image context again.
projects / ceph.git / log