Greg Farnum [Thu, 11 Nov 2021 20:20:11 +0000 (20:20 +0000)]
mon: MonMap: do not increase mon_info_t's compatv in stretch mode, for real
This was supposed to be fixed a year ago in commit 2e3643647bfbe955b54c62c8aaf114744dedb86e, but it set compat_v to 4 instead of all
the way back to 1 as it should have.
Our testing for stretch mode in these areas is just not very thorough -- the
kernel only supports compat_v 1 and apparently nobody's noticed the issue
since then? :/
As the prior commit says, you can't set locations without being gated on a
server feature bit, so simply cancelling this enforcement is completely safe.
Casey Bodley [Tue, 9 Nov 2021 02:24:52 +0000 (21:24 -0500)]
cls/rgw: index cancelation still cleans up remove_objs
when multipart uploads complete their final bucket index transaction,
they pass the list of part objects in 'remove_objs' for bulk removal -
the part objects, along with their bucket stats, get replaced by the
head object
but if CompleteMultipart races with another upload, the head object
write will fail with ECANCELED and the bucket index transaction gets
canceled with CLS_RGW_OP_CANCEL. these canceled uploads still need to
clean up their 'remove_objs', but cancelation was returning too early.
as a result, these bucket index entries get orphaned and leave the
bucket stats inconsistent
this commit reworks rgw_bucket_complete_op() so that CLS_RGW_OP_CANCEL
is handled the same way as OP_ADD and OP_DEL, so always runs the loop to
clean up 'remove_objs'
Mark Kogan [Mon, 15 Nov 2021 15:50:49 +0000 (15:50 +0000)]
rgwi/beast: stream timer with duration 0 disables timeout
fixes all S3 operations failing with:
`2021-11-15T15:46:05.992+0000 7ffee17fa700 20 failed to read header: Bad file descriptor`
when `--rgw_frontends="beast port=8000 request_timeout_ms=0"`
The calls to remove a bucket had parameters to specify a prefix and
delimiter, which does not make sense. This was precipitated due to some
existing Swift protocol logic, but buckets are removed irrespective of
prefix and delimiter. So the functions and calls are adjusted to
remove those parameters. Additionally, those same parameters were
removed for aborting incomplete multipart uploads.
Additionally a bug is fixed in which during bucket removal, multipart
uploads were only removed if the prefix was non-empty.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Greg Farnum [Fri, 12 Nov 2021 23:05:02 +0000 (23:05 +0000)]
mon: MonMap: display disallowed_leaders whenever they're set
In c59a6f89465e3933631afa2ba92e8c1ae1c31c06, I erroneously changed
the CLI display output so it would only dump disallowed_leaders in
stretch mode. But they can also be set in connectivity or disallow
election modes and we want users to be able to see them then as well.
J. Eric Ivancich [Thu, 11 Nov 2021 22:20:55 +0000 (17:20 -0500)]
rgw: make some logging easier to read
While __PRETTY_FUNCTION__ includes more information, it can clutter
the logs. So this reverts some uses of __PRETTY_FUNCTION__ back to
__func__.
I'm thinking that a strategy going forward is for the function entry
logging to use __PRETTY_FUNCTION__ to disambiguate overloaded
functions, but all others in the function simply to use __func__.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
J. Eric Ivancich [Thu, 11 Nov 2021 16:10:17 +0000 (11:10 -0500)]
rgw: add ability to easily display ListParams
During debugging it can be useful to see all the contents of
rgw::sal::Bucket::ListParams. This allows the structure to be dumped
to an output stream in human-readable format.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
J. Eric Ivancich [Thu, 11 Nov 2021 22:20:24 +0000 (17:20 -0500)]
rgw: fix `bi put` not using right bucket index shard
When `radosgw-admin bi put` adds an entry for an incomplete multipart
upload, the bucket index shard is not calculated correctly. It should
be based on the name of the ultimate object. However the calculation
was including the added organizational suffixes as well. This corrects
that.
NOTE: When entries are not put in the correct index shard, unordered
listing becomes unreliable, perhaps causing entries to be skipped or
infinite loops to form.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Casey Bodley [Thu, 11 Nov 2021 17:01:06 +0000 (12:01 -0500)]
rgw/beast: reference count Connections for timeout_handler
resolves a use-after-free in the timeout_handler, where a timeout fires
and schedules the timeout_handler for execution, but the coroutine exits
and destroys the socket before asio executes the timeout_handler
timeout_handler now holds a reference on the Connection to extend its
lifetime
now that the Connection is allocated on the heap, we can include the
parse_buffer in this memory instead of allocating it separately
Casey Bodley [Sat, 30 Oct 2021 23:47:02 +0000 (19:47 -0400)]
rgw/beast: replace beast::tcp_stream with manual timeouts
remove the beast::tcp_stream wrapper from the socket, and track timeouts
manually with a timeout_timer. this timer uses ceph's coarse_mono_clock
which is cheaper to sample than std::chrono::steady_clock
Casey Bodley [Mon, 1 Nov 2021 17:14:16 +0000 (13:14 -0400)]
spawn: use explicit strand executor
the default spawn::yield_context uses the polymorphic boost::asio::executor
to support any executor type
rgw's beast frontend always uses the same executor type for these
coroutines, so we can use that type directly to avoid the overhead of
type erasure and virtual function calls
Yin Congmin [Fri, 12 Nov 2021 08:54:31 +0000 (16:54 +0800)]
qa/suites/rbd/persistent-writeback-cache: add test case
Add the test case which size is 8GB, So that some problems that occur
only in test scenarios above 4GB may be found in this test. For example,
the variables of 32-bit may be unexpected value when it operates with
a 64 bit value.
Sage Weil [Thu, 11 Nov 2021 15:31:22 +0000 (10:31 -0500)]
Merge PR #43046 into master
* refs/pull/43046/head:
mgr/rook: get running pods, auth rm, better error checking for orch nfs
qa/tasks/rook: add apply nfs to rook qa task
mgr/rook: prevent creation of NFS clusters not in .nfs rados pool
mgr/rook, mgr/nfs: update rook orchestrator to create and use .nfs pool
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com> Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
Roland Sommer [Fri, 8 Oct 2021 06:40:26 +0000 (08:40 +0200)]
mgr/prometheus: Make standby discoverable
Enable config settings to modify standby's behaviour on the index page
This makes the standby discoverable by reverse proxy or loadbalancer
setups. Testing for the empty response of the '/metrics' endpoint would
trigger metric collection on the active manager instance.
The newly added configuration options settings standby_behaviour and
standby_error_status_code are documented and flagged as runtime, as
modifying both settings has an immediate effect (no restart required).
Co-authored-by: Ernesto Puerta <37327689+epuertat@users.noreply.github.com> Signed-off-by: Roland Sommer <rol@ndsommer.de> Fixes: https://tracker.ceph.com/issues/53229
Patrick Donnelly [Wed, 10 Nov 2021 18:58:48 +0000 (13:58 -0500)]
Merge PR #42520 into master
* refs/pull/42520/head:
test: add cephfs-mirror HA active/active workunit and test yamls
test: add cephfs_mirror thrasher
tasks/cephfs_mirror: optionally run in foreground
mgr/mirroring: throttle directory reassigment to mirror daemons
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Joseph Sawaya [Tue, 14 Sep 2021 18:54:41 +0000 (14:54 -0400)]
mgr/rook: get running pods, auth rm, better error checking for orch nfs
This commit updates orch ls to show the age and the number of running nfs
pods, removes auth entities when removing an nfs service and implements
better error checking when creating nfs daemons.
Joseph Sawaya [Fri, 30 Jul 2021 16:07:31 +0000 (12:07 -0400)]
mgr/rook, mgr/nfs: update rook orchestrator to create and use .nfs pool
This commit moves the functionality for creating the .nfs pool from the
nfs module to the rook module and makes the rook module use the .nfs
pool when creating an NFS daemon.
projects / ceph.git / log