Venky Shankar [Fri, 25 Aug 2023 13:16:19 +0000 (18:46 +0530)]
Merge PR #52944 into main
* refs/pull/52944/head:
PendingReleaseNotes: add a note for `mds_session_metadata_threshold` mds config
test: add test to verify that a buggy client is blocklisted
mds: add perf counter to track number of sessions evicted due to metadata threshold being exceeded
mds: blocklist clients with "bloated" session metadata
Reviewed-by: Robin H. Johnson <robbat2@orbis-terrarum.net> Reviewed-by: Dhairya Parmar <dparmar@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Aashish Sharma [Wed, 23 Aug 2023 09:59:44 +0000 (15:29 +0530)]
mgr/dashboard: Create realm sets to default
In Multisite page, When we create a realm the realm sets to default even if some other realm is already default and default checkbox in unchecked as well while creating.
osd/SnapMapper:
Maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim() to prevent searching depleted prefixes.
We got 8 distinct hash prefixes used for searching objects owned by a given PG.
On each call to SnapMapper::get_next_objects_to_trim() we start from the first prefix even after all objects mapped to it were depleted.
This means that we will be searching for 1 non-existing prefix after the first prefix was depleted, 2 after the first two prefixes were depleted... and so on until we will search 7 non-existing prefixes after the first 7 prefixes were depleted.
This is a performance improvement PR only!
It maintains the existing behavior and does not try to fix/change any of the TRIM logic.
I added an extra step after the last object is trimmed doing a full scan of the DB and only if no object was found it will return ENOENT.
This should make the new code no-worse than existing code which returns ENOENT after a full scan found no object.
It should not impact performance in real life snaps as it should only happen once per-snap.
added snap-mapper tests to rados-test-suite
disabled osd_debug_trim_objects when running (SnapMapperTest, prefix_itr) to prevent asserts(as this code does illegal inserts into DELETED snaps)
Code beautifing
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
Adam King [Fri, 18 Aug 2023 16:32:38 +0000 (12:32 -0400)]
cephadm: have attempt_bind raise up OSErrors
Before it was always converting the OSError to
our self-defined "Error" class. This causes an issue
with the port_in_use function that has special handling
for OSError when the errno is EADDRNOTAVAIL or
EAFNOSUPPORT. Since the error being raised was no
longer an OSError it wasn't being caught and checked
properly in port_in_use.
This has the additional property of being necessary
to check port availability for haproxy on its VIP. If
we fail deployment when EADDRNOTAVAIL is raised, it becomes
difficult to deploy the ingress service. If we deploy
haproxy first it fails because the VIP isn't available
yet (since keepalive isn't up) and it fails saying the port
it wants to bind to is unavailable (specifically EADDRNOTAVAIL).
If we try to deploy keepalive first it fails because it
needs to know the location of the haproxy daemons in
order to build its config file. This has worked in the past
by just having the haproxy fail to bind at first and then
fix itself once the keepalive daemon is deployed. That
no longer works if the haproxy daemon fails to deploy
because cephadm is reporting the port it needs is
unavailable. Since EADDRNOTAVAIL when deploying
haproxy likely means the VIP is not up rather than
something else is taking up the port it needs, fixing
the handling of this allows ingress deployment to
work while also allowing multiple haproxy daemons
on the same host to use the same frontend port
bound to different VIPs.
Adam King [Fri, 11 Aug 2023 18:00:31 +0000 (14:00 -0400)]
mgr/cephadm: only check haproxy frontend_port conflicts on VIP
If we know what IP the frontend_port will be binding
to, we can pass that down through the port_ips mapping
so cephadm will only check if that port on that specific
VIP if in use. This allows multiple haproxy daemons
to be bound to the same port on different VIPs on the
same host.
Note that you still must use a different monitor port
for the two different ingress services as that port
is bound to on the actual IP of the host. Only the
frontend port can be the same for haproxies on the
same host as long as the VIP is different.
Fixes: https://tracker.ceph.com/issues/57614 Signed-off-by: Adam King <adking@redhat.com>
Adam King [Fri, 11 Aug 2023 17:35:10 +0000 (13:35 -0400)]
cephadm: support for specifying IP a port will be bound to on
This is mostly for checking for port conflicts.
Currently, we just check if the port is bound to
on any IP on the host. This mechanism should allow
certain daemon types to specify a port -> IP mapping
that will be passed to the cephadm binary. That
mapping will then be used by cephadm to only
check for the port being bound to on that specific
IP rather than any IP on the host. The end result
is we could have daemons bound to the same
port on different IPs on the same node.
It's expected that daemon types will set this
up as part of their prepare_create or generate_config
functions where they may have more info about the
specific IPs and ports they need.
Direct leak of 8 byte(s) in 1 object(s) allocated from:
#0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
#1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)
SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```
Zac Dover [Tue, 22 Aug 2023 05:30:21 +0000 (15:30 +1000)]
doc/start: refactor ABC test chart
Refactor the ABC test chart so that the information about which tests
have been run is presented in the center of the chart instead of, as it
was before, in a superscript.
Venky Shankar [Fri, 11 Aug 2023 08:36:52 +0000 (04:36 -0400)]
mds: blocklist clients with "bloated" session metadata
Buggy clients (or maybe a MDS bug) causes a huge buildup of
`completed_requests` metadata in its session information.
This could cause the MDS to go read-only when its flushing
session metadata to the journal since the bloated metadata
causes the ODSOp payload to exceed the maximum write size.
Blocklist such clients so as to allow the MDS to continue
servicing requests.
Venky Shankar [Thu, 17 Aug 2023 13:01:42 +0000 (18:31 +0530)]
Revert "qa/cephfs: fix ior build failure"
PR #52416 is incomplete and got merged on the basis of a linked
test run which passed, however, subsequent update to the PR
caused the test to start failing again.
Casey Bodley [Tue, 8 Aug 2023 14:00:22 +0000 (10:00 -0400)]
script: fix credits.sh showing 0 lines changed for mapped emails
script was showing 0 for several contributors because the commit Author:
did not match their mapped address:
```
Number of lines added and removed, by authors
...
306 0 Radoslaw Zarzynski <rzarzynski@redhat.com>
307 0 Mike Perez <miperez@redhat.com>
308 0 Michael J. Kidd <linuxkidd@redhat.com>
309 0 Lukas Mayer <lmayer@wind.gmbh>
310 0 Luis Henriques <lhenriques@suse.com>
311 0 Kyle McGough <kmcgough@digitalocean.com>
312 0 João Eduardo Luís <joao@suse.de>
313 0 JinyongHa <jy200.ha@samsung.com>
314 0 Ilya Dryomov <idryomov@redhat.com>
```
in the case of Ilya and Radoslaw:
```
$ git log --no-merges --pretty='%ae' v17.2.6..v18.2.0 | sed -e "s/'/ /g" | sort -u | grep -e idryomov -e rzarzyns
idryomov@gmail.com
rzarzyns@redhat.com
```
this output gets fed into `git log --numstat --author="$mail"` to fetch
the stats, but that command maps idryomov@gmail.com -> idryomov@redhat.com
and rzarzyns@redhat.com -> rzarzynski@redhat.com so no commits matched.
disabling the mapping for this command with --no-mailmap allows it to
fetch their stats correctly:
```
14 11809 Radoslaw Zarzynski <rzarzynski@redhat.com>
16 10051 Ilya Dryomov <idryomov@redhat.com>
```
projects / ceph.git / log