Adam King [Fri, 18 Aug 2023 16:32:38 +0000 (12:32 -0400)]
cephadm: have attempt_bind raise up OSErrors
Before it was always converting the OSError to
our self-defined "Error" class. This causes an issue
with the port_in_use function that has special handling
for OSError when the errno is EADDRNOTAVAIL or
EAFNOSUPPORT. Since the error being raised was no
longer an OSError it wasn't being caught and checked
properly in port_in_use.
This has the additional property of being necessary
to check port availability for haproxy on its VIP. If
we fail deployment when EADDRNOTAVAIL is raised, it becomes
difficult to deploy the ingress service. If we deploy
haproxy first it fails because the VIP isn't available
yet (since keepalive isn't up) and it fails saying the port
it wants to bind to is unavailable (specifically EADDRNOTAVAIL).
If we try to deploy keepalive first it fails because it
needs to know the location of the haproxy daemons in
order to build its config file. This has worked in the past
by just having the haproxy fail to bind at first and then
fix itself once the keepalive daemon is deployed. That
no longer works if the haproxy daemon fails to deploy
because cephadm is reporting the port it needs is
unavailable. Since EADDRNOTAVAIL when deploying
haproxy likely means the VIP is not up rather than
something else is taking up the port it needs, fixing
the handling of this allows ingress deployment to
work while also allowing multiple haproxy daemons
on the same host to use the same frontend port
bound to different VIPs.
Adam King [Fri, 11 Aug 2023 18:00:31 +0000 (14:00 -0400)]
mgr/cephadm: only check haproxy frontend_port conflicts on VIP
If we know what IP the frontend_port will be binding
to, we can pass that down through the port_ips mapping
so cephadm will only check if that port on that specific
VIP if in use. This allows multiple haproxy daemons
to be bound to the same port on different VIPs on the
same host.
Note that you still must use a different monitor port
for the two different ingress services as that port
is bound to on the actual IP of the host. Only the
frontend port can be the same for haproxies on the
same host as long as the VIP is different.
Fixes: https://tracker.ceph.com/issues/57614 Signed-off-by: Adam King <adking@redhat.com>
Adam King [Fri, 11 Aug 2023 17:35:10 +0000 (13:35 -0400)]
cephadm: support for specifying IP a port will be bound to on
This is mostly for checking for port conflicts.
Currently, we just check if the port is bound to
on any IP on the host. This mechanism should allow
certain daemon types to specify a port -> IP mapping
that will be passed to the cephadm binary. That
mapping will then be used by cephadm to only
check for the port being bound to on that specific
IP rather than any IP on the host. The end result
is we could have daemons bound to the same
port on different IPs on the same node.
It's expected that daemon types will set this
up as part of their prepare_create or generate_config
functions where they may have more info about the
specific IPs and ports they need.
Venky Shankar [Thu, 17 Aug 2023 13:01:42 +0000 (18:31 +0530)]
Revert "qa/cephfs: fix ior build failure"
PR #52416 is incomplete and got merged on the basis of a linked
test run which passed, however, subsequent update to the PR
caused the test to start failing again.
Casey Bodley [Tue, 8 Aug 2023 14:00:22 +0000 (10:00 -0400)]
script: fix credits.sh showing 0 lines changed for mapped emails
script was showing 0 for several contributors because the commit Author:
did not match their mapped address:
```
Number of lines added and removed, by authors
...
306 0 Radoslaw Zarzynski <rzarzynski@redhat.com>
307 0 Mike Perez <miperez@redhat.com>
308 0 Michael J. Kidd <linuxkidd@redhat.com>
309 0 Lukas Mayer <lmayer@wind.gmbh>
310 0 Luis Henriques <lhenriques@suse.com>
311 0 Kyle McGough <kmcgough@digitalocean.com>
312 0 João Eduardo Luís <joao@suse.de>
313 0 JinyongHa <jy200.ha@samsung.com>
314 0 Ilya Dryomov <idryomov@redhat.com>
```
in the case of Ilya and Radoslaw:
```
$ git log --no-merges --pretty='%ae' v17.2.6..v18.2.0 | sed -e "s/'/ /g" | sort -u | grep -e idryomov -e rzarzyns
idryomov@gmail.com
rzarzyns@redhat.com
```
this output gets fed into `git log --numstat --author="$mail"` to fetch
the stats, but that command maps idryomov@gmail.com -> idryomov@redhat.com
and rzarzyns@redhat.com -> rzarzynski@redhat.com so no commits matched.
disabling the mapping for this command with --no-mailmap allows it to
fetch their stats correctly:
```
14 11809 Radoslaw Zarzynski <rzarzynski@redhat.com>
16 10051 Ilya Dryomov <idryomov@redhat.com>
```
Tobias Urdin [Mon, 7 Aug 2023 20:34:43 +0000 (20:34 +0000)]
rgw/auth: handle HTTP OPTIONS with v4 auth
This adds code to properly verify the signature
for HTTP OPTIONS calls that is preflight CORS
requests passing the expected method in the
access-control-request-method header.
tengjie5 [Tue, 8 Aug 2023 02:13:58 +0000 (10:13 +0800)]
doc: modify and add descriptions for cephfs-shell commands. Fixes: https://tracker.ceph.com/issues/62074 Signed-off-by: Teng Jie <tengjie5@asiainfo.com>
crimson/os/seastore/journal/cbj: introduce magic value to identify that written records are correct
The cbj would replay invalid records if the same write pattern (mkfs -> write -> mkfs -> write) repeats.
On the second mkfs(), current cbj wrongly recognizes the old records are valid beceause they have
correct seq. number at appropriate offset even though the records are expired.
To solve this, this commit introduces random value to identify the journal during mkfs()
Lucian Petrut [Fri, 4 Nov 2022 10:30:43 +0000 (12:30 +0200)]
test/librbd/fsx: add rbd-wnbd tests
rbd-wnbd can be used to map rbd images as Windows block devices.
This change will allow exercising rbd-wnbd and the wnbd driver
as opposed to using librbd directly.
One thing to note here is that read and write operations must be
512B aligned. Also, wnbd does not currently support "compare
and write".
For this reason, we'll stick to the basic operations for the time
being.
Note that we're renaming krbd_read/krbd_write to generic_pread
and generic_pwrite, reusing those functions for wnbd operations.
We're also removing the duplicated ggate read/write operations,
using the generic functions instead.
Adam King [Fri, 2 Jun 2023 00:06:35 +0000 (20:06 -0400)]
cephadm: add tcmu-runner to logrotate config
This process could be used to set up the tcmu-runner
to log to a file much like other ceph daemons
- create /etc/tcmu directory
- create /etc/tcmu/tcmu.conf directory with default options
- change dir to /var/log
- change log level to 4
- add -v /etc/tcmu:/etc/tcmu to tcmu-runner container podman line in unit.run
In order to support this (mostly for debugging) we should
add tcmu-runner to the logrotate config
Fixes: https://tracker.ceph.com/issues/61571 Signed-off-by: Adam King <adking@redhat.com>
Adam King [Fri, 7 Jul 2023 15:03:56 +0000 (11:03 -0400)]
qa/cephadm: add test for ca signed keys
Test that bootstraps with a CA signed key using
the use_ca_signed_key cephadm override. Then follows
up by doing a check-host on each host which verifies
the cephadm mgr module can reach and authenticate with
the nodes using the new key setup.
This probably should really be a workunit, but
I didn't want to create a full new section for
this test and I needed a section that didn't
already run the cephadm task for every test. I could
see this being moved into some sort of
"test_special_deployment_scenarios" section in the future
Adam King [Fri, 7 Jul 2023 14:36:39 +0000 (10:36 -0400)]
qa/cephadm: add ca signed key to cephadm task
To allow bootstrapping a cluster using a CA signed
key instead of the standard pubkey authentication.
Will allow explicit testing of this as we add support
for it
Adam King [Sat, 3 Jun 2023 18:39:05 +0000 (14:39 -0400)]
doc/cephadm: document how to pass self made SSH key pairs to bootstrap
This didn't seem to exist in the install section of
the cephadm docs. Wanted to add it in before adding
documentation for bootstrapping with CA signed keys.
galsalomon66 [Sun, 25 Jun 2023 22:10:38 +0000 (01:10 +0300)]
- s3select fixes, related to Trino/TPCDS benchmark
- s3select submodule
- add log-message for better tracing the flow timeline
- fix valgrind issue
- new s3select version. fix crash upon wrong initialization on JSON flow
- s3select module. a fix for a constant number of columns(128).
- s3select module. QE fixes
Zac Dover [Tue, 15 Aug 2023 03:28:00 +0000 (13:28 +1000)]
doc: improve submodule update command - README.md
Improve the portion of README.md that directs the reader to update the
submodules. Previously, the text assumed that the reader knew that they
should cd into the ceph directory. Now the text is explicit about the
necessity of this action.
John Mulligan [Mon, 14 Aug 2023 18:10:39 +0000 (14:10 -0400)]
doc: do not recommend pulling cephadm from git
First off, these instructions are wrong because the file has been
renamed to cephadm.py, and will not succeed as written. Amusingly, due
to curl's default behavior (and my lack of access to a time machine) the
initial command will seem to have succeeded until the user discovers
that HTML delivered as part of the body of a 404 response is not an
executable script.
Furthermore, in preparation for Reef we updated the documentation in the
cephadm subsection to use the results of a build, rather than source
files, because reef now contains a "compiled" cephadm. Currently, the
contents of the compilation is essentially the same-old cephadm script
but that will no longer be the case in the very near future (see 7c6a2cad96405616719b85c740aabd98558c93ac). We missed this general
section of the Ceph docs that repeats the instructions to copy a raw
python file out of the Ceph tree on GitHub. Update the section to
contain a simplified version of the instruction found in
doc/cephadm/install.rst.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Nizamudeen A [Mon, 14 Aug 2023 17:53:11 +0000 (23:23 +0530)]
.github: fix add to dashboard failing
```
Error: Request failed due to following response errors:
- Could not resolve to a ProjectV2 with the number 2.
```
On further investigation, it looks like it could be a permission issue.
I followed the recommendation in their documentation to fix it: https://github.com/actions/add-to-project#creating-a-pat-and-adding-it-to-your-repository
projects / ceph.git / log