]> git.apps.os.sepia.ceph.com Git - teuthology.git/log
teuthology.git
12 days agoMerge pull request #2082 from ceph/arm64-runners
Zack Cerza [Tue, 9 Sep 2025 19:27:34 +0000 (13:27 -0600)]
Merge pull request #2082 from ceph/arm64-runners

workflows/dev_container: Use native arm64 runners

12 days agosetup.cfg: Restrict gevent to <25 2082/head
Zack Cerza [Tue, 9 Sep 2025 19:03:02 +0000 (13:03 -0600)]
setup.cfg: Restrict gevent to <25

This should work around the hangs we have been seeing in CI.

Signed-off-by: Zack Cerza <zack@cerza.org>
12 days agoworkflows/dev_container: Use native arm64 runners
Zack Cerza [Tue, 9 Sep 2025 18:17:19 +0000 (12:17 -0600)]
workflows/dev_container: Use native arm64 runners

Signed-off-by: Zack Cerza <zack@cerza.org>
3 weeks agoMerge pull request #2077 from kshtsk/wip-downburst-packages
Zack Cerza [Wed, 27 Aug 2025 21:01:22 +0000 (15:01 -0600)]
Merge pull request #2077 from kshtsk/wip-downburst-packages

Introduce multiple machine type instead of single vps for downburst

3 weeks agoprovision/downburst: support multiple config per machine_type 2077/head
Kyr Shatskyy [Sun, 24 Aug 2025 12:00:58 +0000 (14:00 +0200)]
provision/downburst: support multiple config per machine_type

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
3 weeks agodocs/laptop/targets.sql: add semicolon to commit
Kyr Shatskyy [Mon, 25 Aug 2025 13:50:49 +0000 (15:50 +0200)]
docs/laptop/targets.sql: add semicolon to commit

Apparently semicolon needed after transaction,
otherwise it gets stuck in a transaction queue
and locks the table

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
3 weeks agoMerge pull request #2075 from kshtsk/wip-teuthology-git-url
Zack Cerza [Tue, 26 Aug 2025 16:42:18 +0000 (10:42 -0600)]
Merge pull request #2075 from kshtsk/wip-teuthology-git-url

teuthology/config: add teuthology_git_url option

3 weeks agoMerge pull request #2072 from ceph/update-gevent
Zack Cerza [Tue, 26 Aug 2025 16:29:56 +0000 (10:29 -0600)]
Merge pull request #2072 from ceph/update-gevent

requirements.txt: Update gevent

3 weeks agoMerge pull request #2076 from kshtsk/wip-rocky-alma-9.6
kyr [Mon, 25 Aug 2025 21:48:34 +0000 (23:48 +0200)]
Merge pull request #2076 from kshtsk/wip-rocky-alma-9.6

orchestra/opsys: release shift to 9.6 for rocky and alma linux

3 weeks agoorchestra/opsys: release shift to 9.6 for rocky and alma linux 2076/head
Kyr Shatskyy [Thu, 21 Aug 2025 21:57:54 +0000 (23:57 +0200)]
orchestra/opsys: release shift to 9.6 for rocky and alma linux

The release 9.5 is gone and all files removed on RockyLinux
and AlmaLinux sites, so just update defaults to the latest.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
4 weeks agoprovision/downburst: install bind-utils for host command on centos
Kyr Shatskyy [Sat, 5 Apr 2025 18:39:16 +0000 (20:39 +0200)]
provision/downburst: install bind-utils for host command on centos

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
4 weeks agoprovision/downburst: install pip for centos
Kyr Shatskyy [Sat, 5 Apr 2025 16:34:13 +0000 (18:34 +0200)]
provision/downburst: install pip for centos

By some reason, ansible is not install pip3 for centos:

  Unable to find any of pip3 to use.  pip needs to be installed.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
4 weeks agoteuthology/config: add teuthology_git_url option 2075/head
Kyr Shatskyy [Thu, 21 Aug 2025 08:45:17 +0000 (10:45 +0200)]
teuthology/config: add teuthology_git_url option

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
4 weeks agoMerge pull request #2074 from kshtsk/wip-fix-revive-openstack
kyr [Mon, 18 Aug 2025 11:36:47 +0000 (13:36 +0200)]
Merge pull request #2074 from kshtsk/wip-fix-revive-openstack

setup.cfg: fix teuthology-openstack dependency

5 weeks agosetup.cfg: fix teuthology-openstack dependency 2074/head
Kyr Shatskyy [Fri, 15 Aug 2025 14:33:59 +0000 (16:33 +0200)]
setup.cfg: fix teuthology-openstack dependency

The PR #2050 "Revive teuthology-openstack" introduced new dependency
after refactoring partly of teuthology.openstack instance classes
to use api instead of command line. However the openstacksdk
added to wrong section options.extras_require.openstack, that
intended to be used for provisioning openstack nodes via libcloud,
which is optional and is not installable by default.

I.e. `pip install -e ".[openstack]"` must be used in order to get it.

However the refactoring forced unconditional openstack requirement.

That's why this dependency should go to options.install_requires in order
to make `pip install -e .` work again.

Fixes: 252ab4f6a44c0d64b1e4cfb2b86e9bf58b5d9ffb
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
5 weeks agoMerge pull request #2073 from kshtsk/wip-fedora-4x
kyr [Thu, 14 Aug 2025 22:21:25 +0000 (00:21 +0200)]
Merge pull request #2073 from kshtsk/wip-fedora-4x

Fedora 4x

5 weeks agorequirements.txt: Update gevent update-gevent 2072/head
Zack Cerza [Thu, 14 Aug 2025 19:30:30 +0000 (13:30 -0600)]
requirements.txt: Update gevent

Signed-off-by: Zack Cerza <zack@cerza.org>
5 weeks agobootstrap: Fedora does not have python3.12 anymore 2073/head
Kyr Shatskyy [Thu, 14 Aug 2025 18:56:02 +0000 (20:56 +0200)]
bootstrap: Fedora does not have python3.12 anymore

Latest Fedora, 41 and 42, has python3.13 installed
by default, just use python3 to allow use default
version.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
5 weeks agobootstrap: add -q for rpm --whatprovides
Kyr Shatskyy [Thu, 14 Aug 2025 18:52:20 +0000 (20:52 +0200)]
bootstrap: add -q for rpm --whatprovides

The rpm -q option lost during refactoring which is obviously
needed when requesting the package name.

Fixes: c290c6851a7c10aaa4a5e18e1c14d750c338e339
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
8 weeks agoMerge pull request #2068 from batrick/jouranlctl-all
Zack Cerza [Fri, 25 Jul 2025 22:53:51 +0000 (16:53 -0600)]
Merge pull request #2068 from batrick/jouranlctl-all

tasks/internal/syslog: gather all journald entries

8 weeks agoMerge pull request #2070 from deepssin/sec_rule
Zack Cerza [Fri, 25 Jul 2025 22:01:41 +0000 (16:01 -0600)]
Merge pull request #2070 from deepssin/sec_rule

[Openstack]allow all TCP ingress to prevent node communication failures

8 weeks agoMerge pull request #2069 from deepssin/openstack_issues
Zack Cerza [Fri, 25 Jul 2025 22:00:44 +0000 (16:00 -0600)]
Merge pull request #2069 from deepssin/openstack_issues

Add centos9 stream user data file & add centos9 support to openstack

8 weeks agoMerge pull request #2067 from kshtsk/wip-remote-resolve-ip
Zack Cerza [Fri, 25 Jul 2025 21:52:38 +0000 (15:52 -0600)]
Merge pull request #2067 from kshtsk/wip-remote-resolve-ip

orchestra/remote: add resolve_ip method

8 weeks ago[Openstack]allow all TCP ingress to prevent node communication failures 2070/head
deepssin [Thu, 24 Jul 2025 10:28:40 +0000 (06:28 -0400)]
[Openstack]allow all TCP ingress to prevent node communication failures

Signed-off-by: deepssin <deepssin@redhat.com>
8 weeks agoAdd centos9 stream user data file & add centos9 support to openstack 2069/head
deepssin [Thu, 24 Jul 2025 08:02:24 +0000 (04:02 -0400)]
Add centos9 stream user data file & add centos9 support to openstack

Signed-off-by: deepssin <deepssin@redhat.com>
8 weeks agocontainers/teuthology-dev/Dockerfile: fix running bootstrap 2067/head
Kyr Shatskyy [Wed, 23 Jul 2025 14:03:35 +0000 (16:03 +0200)]
containers/teuthology-dev/Dockerfile: fix running bootstrap

The `git config --unset` returns 5 if there is missing variable supposed
to be unset. Just always return true for this command so the bootstrap
can proceed.

Fixes: 7cc6ba4838150acd17649381313093185075ecb0
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
8 weeks agobootstrap: fix syntax error
Kyr Shatskyy [Wed, 23 Jul 2025 13:23:48 +0000 (15:23 +0200)]
bootstrap: fix syntax error

This patch addresses the error in the logs, like this:

  Using python: python3.12
  /usr/bin/lsb_release
  Deleting existing virtual environment
  Creating new venv at ./virtualenv
  ./bootstrap: line 131: [: missing `]'
  ./bootstrap: line 131: 12: command not found

Fixes: 0b5a5bc629180cf5318b77a8ecb4c131abbd057d
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
8 weeks agoworkflows/dependencies: apt update
Kyr Shatskyy [Fri, 18 Jul 2025 15:29:10 +0000 (17:29 +0200)]
workflows/dependencies: apt update

Update before bootstrap

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
8 weeks agoorchestra/remote: add resolve_ip method
Kyr Shatskyy [Wed, 19 Mar 2025 13:49:51 +0000 (14:49 +0100)]
orchestra/remote: add resolve_ip method

Add utility method to resolve a hostname from within remote host.
This is useful to resolve ip address of the remote host itself,
because getting ip address from transport object of ssh is not
suitable because it may have only bastion host and port, which
is not relevant for the purpose of configuring a cluster.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
2 months agotask/internal/syslog: parallelize and debugify gzip of syslogs jouranlctl-all 2068/head
Patrick Donnelly [Mon, 21 Jul 2025 15:49:32 +0000 (11:49 -0400)]
task/internal/syslog: parallelize and debugify gzip of syslogs

To match what we do for the ceph task.

Looks like:

    2025-07-21T17:04:25.354 INFO:teuthology.task.internal.syslog:Compressing syslogs...
    2025-07-21T17:04:25.354 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose --
    2025-07-21T17:04:25.357 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose --
    2025-07-21T17:04:25.359 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose --
    2025-07-21T17:04:25.380 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
    2025-07-21T17:04:25.380 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
    2025-07-21T17:04:25.380 INFO:teuthology.orchestra.run.smithi117.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:   gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log
    2025-07-21T17:04:25.381 INFO:teuthology.orchestra.run.smithi117.stderr:  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
    2025-07-21T17:04:25.381 INFO:teuthology.orchestra.run.smithi117.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:     0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
    2025-07-21T17:04:25.385 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
    2025-07-21T17:04:25.385 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
    2025-07-21T17:04:25.385 INFO:teuthology.orchestra.run.smithi142.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:   gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log
    2025-07-21T17:04:25.385 INFO:teuthology.orchestra.run.smithi142.stderr:  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
    2025-07-21T17:04:25.385 INFO:teuthology.orchestra.run.smithi142.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:     0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
    2025-07-21T17:04:25.386 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
    2025-07-21T17:04:25.387 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
    2025-07-21T17:04:25.387 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log
    2025-07-21T17:04:25.387 INFO:teuthology.orchestra.run.smithi186.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:     0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
    2025-07-21T17:04:25.387 INFO:teuthology.orchestra.run.smithi186.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:     0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
    2025-07-21T17:04:25.423 INFO:teuthology.orchestra.run.smithi142.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log:      88.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz
    2025-07-21T17:04:25.426 INFO:teuthology.orchestra.run.smithi186.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log:      88.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz
    2025-07-21T17:04:25.502 INFO:teuthology.orchestra.run.smithi117.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log:      90.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 months agotask/internal/syslog: gzip journal along with everything else
Patrick Donnelly [Mon, 21 Jul 2025 13:42:36 +0000 (09:42 -0400)]
task/internal/syslog: gzip journal along with everything else

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 months agotasks/internal/syslog: gather all journald entries
Patrick Donnelly [Mon, 21 Jul 2025 13:39:03 +0000 (09:39 -0400)]
tasks/internal/syslog: gather all journald entries

Soemtimes a machine crashes during tests and log entries around that time are
not persisted because we're only looking at this boot.

(This choice seem odd since the lifetime of these operating system images is
the same as the teuthology run.)

Now looks like:

    2025-07-21T15:01:22.249 INFO:teuthology.task.internal.syslog:Gathering journactl...
    2025-07-21T15:01:22.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log
    2025-07-21T15:01:22.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log
    2025-07-21T15:01:22.256 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log

From: /teuthology/pdonnell-2025-07-21_14:01:58-fs-main-distro-default-smithi/8399938/teuthology.log

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 months agoMerge pull request #2066 from deepssin/issue-72055
Zack Cerza [Wed, 16 Jul 2025 16:19:55 +0000 (10:19 -0600)]
Merge pull request #2066 from deepssin/issue-72055

Add openstack dependencies in docs/requirements

2 months agoAdd openstack dependencies in docs/requirements 2066/head
deepssin [Wed, 16 Jul 2025 10:49:17 +0000 (06:49 -0400)]
Add openstack dependencies in docs/requirements

Signed-off-by: deepssin <deepssin@redhat.com>
2 months agoMerge pull request #2065 from ceph/job-flavor
Zack Cerza [Mon, 14 Jul 2025 22:59:29 +0000 (16:59 -0600)]
Merge pull request #2065 from ceph/job-flavor

Add flavor to job yaml (at Job creation)

2 months agoMerge pull request #2064 from ceph/run-dev-container-ci
Zack Cerza [Mon, 14 Jul 2025 22:57:32 +0000 (16:57 -0600)]
Merge pull request #2064 from ceph/run-dev-container-ci

ci: run dev_container build for PRs

2 months agoci: run dev_container build for PRs 2064/head
Vallari Agrawal [Thu, 10 Jul 2025 06:30:08 +0000 (12:00 +0530)]
ci: run dev_container build for PRs

Build the image for PRs but do not push.
For pushes to main branch, push to registery.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2 months agoAdd flavor to job yaml (at Job creation) 2065/head
Vallari Agrawal [Fri, 11 Jul 2025 14:16:22 +0000 (19:46 +0530)]
Add flavor to job yaml (at Job creation)

Currently, when creating jobs, paddles
does not get Job.flavor information.
This leads to "flavor" getting set only
after the job is completed.

This commit adds "flavor" to job yaml config
so Job.flavor is set when creating jobs.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2 months agoMerge pull request #2063 from deepssin/issue-72055
Vallari Agrawal [Fri, 11 Jul 2025 03:32:49 +0000 (09:02 +0530)]
Merge pull request #2063 from deepssin/issue-72055

Add jq and curl to teuthology-dev Dockerfile

2 months agoAdd jq and curl to teuthology-dev Dockerfile 2063/head
deepssin [Thu, 10 Jul 2025 05:58:46 +0000 (01:58 -0400)]
Add jq and curl to teuthology-dev Dockerfile

Signed-off-by: deepssin <deepssin@redhat.com>
2 months agoMerge pull request #2062 from VallariAg/fix-container-image
Zack Cerza [Wed, 9 Jul 2025 17:58:46 +0000 (11:58 -0600)]
Merge pull request #2062 from VallariAg/fix-container-image

containers: set image as ceph.containers.image

2 months agocontainers: set image as ceph.containers.image 2062/head
Vallari Agrawal [Wed, 9 Jul 2025 17:15:24 +0000 (22:45 +0530)]
containers: set image as ceph.containers.image

instead of ceph.image

As cephadm task overrides from here: https://github.com/ceph/ceph/blob/ef15ecb8821fd79e24b046d2812696bd290c8489/qa/tasks/cephadm.py#L2301

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2 months agoMerge pull request #2061 from ceph/wip-rocky10
David Galloway [Wed, 2 Jul 2025 18:15:44 +0000 (14:15 -0400)]
Merge pull request #2061 from ceph/wip-rocky10

opsys.py: Add Rocky 10

2 months agoMerge pull request #2060 from ceph/wip-base-config-fix
Vallari Agrawal [Wed, 2 Jul 2025 16:43:41 +0000 (22:13 +0530)]
Merge pull request #2060 from ceph/wip-base-config-fix

Fix wrong os-type issue

2 months agoMerge pull request #2050 from deepssin/enable-teuthology-openstack
Zack Cerza [Wed, 2 Jul 2025 16:36:05 +0000 (10:36 -0600)]
Merge pull request #2050 from deepssin/enable-teuthology-openstack

Revive teuthology-openstack

2 months agotests: add unit test test_update_key and test_sha1 wip-base-config-fix 2060/head
Vallari Agrawal [Tue, 1 Jul 2025 15:12:53 +0000 (20:42 +0530)]
tests: add unit test test_update_key and test_sha1

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2 months agosuite: Add reproducer for issue 71911
Zack Cerza [Mon, 30 Jun 2025 21:02:11 +0000 (15:02 -0600)]
suite: Add reproducer for issue 71911

See: https://tracker.ceph.com/issues/71911

Signed-off-by: Zack Cerza <zack@cerza.org>
(cherry picked from commit be1d157d5a27e5eaaa791f674ee34e25d718ff61)

2 months agosuite/run.py: update sha1 from basic_config to parsed_yaml
Vallari Agrawal [Tue, 1 Jul 2025 14:10:01 +0000 (19:40 +0530)]
suite/run.py: update sha1 from basic_config to parsed_yaml

This is because during backtracking to find
build when using --newest, the new build/suite
sha is updated on basic_config. And job's parsed_yaml
is never updated with new sha.

Fixes: https://tracker.ceph.com/issues/71612
Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2 months agoopsys.py: Add Rocky 10 2061/head
David Galloway [Tue, 1 Jul 2025 14:50:51 +0000 (10:50 -0400)]
opsys.py: Add Rocky 10

Signed-off-by: David Galloway <david.galloway@ibm.com>
2 months agoRevert "teuthology/schedule.py: update parsed_yaml with base_config"
Vallari Agrawal [Tue, 1 Jul 2025 12:26:10 +0000 (17:56 +0530)]
Revert "teuthology/schedule.py: update parsed_yaml with base_config"

This reverts commit 6c356472c37fc4334749d744738ff1ed4ebb031c.

This commit caused a regression: https://tracker.ceph.com/issues/71911

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2 months agoAdd jq and curl in docker-compose Dockerfile 2050/head
deepssin [Thu, 26 Jun 2025 11:12:41 +0000 (07:12 -0400)]
Add jq and curl in docker-compose Dockerfile

Signed-off-by: deepssin <deepssin@redhat.com>
3 months agoMerge pull request #2055 from kamoltat/wip-ksirivad-fix-71612
Zack Cerza [Wed, 18 Jun 2025 21:33:56 +0000 (15:33 -0600)]
Merge pull request #2055 from kamoltat/wip-ksirivad-fix-71612

suite/run.py: suite_hash should use backtracked sha1

3 months agoRevive teuthology-openstack
deepssin [Mon, 16 Jun 2025 13:55:21 +0000 (09:55 -0400)]
Revive teuthology-openstack

Signed-off-by: deepssin <deepssin@redhat.com>
3 months agosuite/test/test_run.py: Add tests for SHA1 handling with --newest 2055/head
Kamoltat Sirivadhna [Thu, 12 Jun 2025 21:03:59 +0000 (17:03 -0400)]
suite/test/test_run.py: Add tests for SHA1 handling with --newest

Added two test cases to verify SHA1 handling when using --newest backtracking:

1. test_newest_success_same_branch_same_repo: Tests when ceph_branch and
   suite_branch are the same. Verifies that both ceph_hash and suite_hash
   are updated to the backtracked working SHA1.

2. test_newest_success_diff_branch_diff_repo: Tests when ceph_branch and
   suite_branch differ. Verifies that only ceph_hash is updated to the
   working SHA1, while suite_hash remains as the original suite_sha1.

Both tests verify the complete flow through collect_jobs() and
schedule_suite(), ensuring the YAML files generated for each job
contain the correct SHA1 references.

Fixes: https://tracker.ceph.com/issues/71668
Signed-off-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
3 months agosuite/run.py: suite_hash should use backtracked sha1
Kamoltat Sirivadhna [Thu, 12 Jun 2025 19:29:33 +0000 (15:29 -0400)]
suite/run.py: suite_hash should use backtracked sha1

In schedule_suite() we only try to find backtracked sha1
for ceph sha1 when --newest is provided and packages are
not found in shaman. However, what about suite_sha1?
Currently, we do not use the backtracked sha1 for suite_sha1.
This commit will make sure that suite_sha1 is using
the backtracked sha1 when ceph_branch and suite_branch are the same and
ceph_repo and suite_repo are the same. This ensures that suite_sha1
only use the backtracked sha1 when the user clearly expects
shaman to not have the latest sha1 and want to use the --newest sha1
for both the ceph code base and suite qa code base.

Fixes: https://tracker.ceph.com/issues/71668
Signed-off-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
3 months agoMerge pull request #2054 from ceph/wip-base_config-sha-fix
Zack Cerza [Thu, 12 Jun 2025 20:02:24 +0000 (14:02 -0600)]
Merge pull request #2054 from ceph/wip-base_config-sha-fix

teuthology/schedule.py: update parsed_yaml with base_config

3 months agoteuthology/schedule.py: update parsed_yaml with base_config 2054/head
Vallari Agrawal [Thu, 12 Jun 2025 14:13:57 +0000 (19:43 +0530)]
teuthology/schedule.py: update parsed_yaml with base_config

In `schedule_suite` method,
`self.base_config` gets updated many times while backtracking
when using "--newest" flag. These changes were never updated
to `parsed_yaml` (job yaml) in `configs`.
As `configs` is intialised before backtracking.

Before, we used to write base_config to a tmp
file and pass that to teuthology-schedule
which used to take care of updates to base_config.
But this logic was removed in https://github.com/ceph/teuthology/pull/2008/files
so the updates to base_config then never make
it to job yaml anymore.

Fixes: https://tracker.ceph.com/issues/71612
Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
4 months agoMerge pull request #1931 from NitzanMordhai/wip-nitzan-scrapy-decode-gzip-backtrace
kyr [Sun, 18 May 2025 09:22:35 +0000 (11:22 +0200)]
Merge pull request #1931 from NitzanMordhai/wip-nitzan-scrapy-decode-gzip-backtrace

scrape: searching backtrace with gzip

4 months agoscrape: searching backtrace with gzip 1931/head
nmordech@redhat.com [Sun, 31 Mar 2024 11:49:03 +0000 (11:49 +0000)]
scrape: searching backtrace with gzip

Scrape script that trying to find backtrace in gzip log files
can hit TypeError: a bytes-like object is required, not 'str' error
and fail to collect results. the gzip file need to be decoded.

Fixes: https://tracker.ceph.com/issues/64402
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
4 months agoMerge pull request #2047 from ceph/drop2004
Zack Cerza [Mon, 12 May 2025 22:28:54 +0000 (16:28 -0600)]
Merge pull request #2047 from ceph/drop2004

ci: Drop Ubuntu 20.04

4 months agoci: Drop Ubuntu 20.04 2047/head
Zack Cerza [Mon, 12 May 2025 21:57:30 +0000 (15:57 -0600)]
ci: Drop Ubuntu 20.04

GitHub doesn't provide the image anymore.

Signed-off-by: Zack Cerza <zack@cerza.org>
4 months agoMerge pull request #2045 from kshtsk/wip-packages-duplication
kyr [Wed, 7 May 2025 16:00:35 +0000 (18:00 +0200)]
Merge pull request #2045 from kshtsk/wip-packages-duplication

Fix packages duplication when extra system packages requested

5 months agotask/install/rpm: drop code duplication for extra_system_packages 2045/head
Kyr Shatskyy [Wed, 16 Apr 2025 14:41:54 +0000 (16:41 +0200)]
task/install/rpm: drop code duplication for extra_system_packages

After introducing install_ceph_packages option there is added
new extra_system_packages list append instruction which made
adding the packages twice if the option is set to False.
So we just drop the first attempt to update install package list.

Fixes: 0177823
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
5 months agotask/install: do not duplicate package names
Kyr Shatskyy [Wed, 16 Apr 2025 14:02:12 +0000 (16:02 +0200)]
task/install: do not duplicate package names

Get rid of package names duplication when extra_system_packages provided
for rpm.

For example, when it is requested 'bzip2' and 'perl-Test-Harness' to be
installed as extra system packages, there can be observed 5 times
mentioning of the packages, see the log excerpt:

  2025-04-09T12:30:08.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bzip2, perl-Test-Harness, bzip2, perl-Test-Harness, bzip2, perl-Test-Harness, bzip2, perl-Test-Harness, bzip2, perl-Test-Harness on remote rpm x86_64

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
5 months agoMerge pull request #2042 from kshtsk/wip-downburst-timesync
Zack Cerza [Wed, 9 Apr 2025 15:23:24 +0000 (09:23 -0600)]
Merge pull request #2042 from kshtsk/wip-downburst-timesync

When downburst used, command cloud-init to install ntp or chrony for time sync

5 months agoprovision/downburst: drop redhat-lsb-core requirement for centos 2042/head
Kyr Shatskyy [Sat, 5 Apr 2025 18:25:14 +0000 (20:25 +0200)]
provision/downburst: drop redhat-lsb-core requirement for centos

This package cannot be found, and in fact it's not required,
because better use /etc/os-release

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
5 months agoprovision/downburst: add ntp or chrony to cloud-init packages
Kyr Shatskyy [Thu, 3 Apr 2025 10:35:52 +0000 (12:35 +0200)]
provision/downburst: add ntp or chrony to cloud-init packages

Since downburst may use default cloud images which mostly
miss ntp by default, make sure we preinstall ntp or chrony
required for ceph cluster to sync the date time, otherwise
osd self scrub check will fail.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
5 months agoMerge pull request #2039 from ceph/dependabot/pip/jinja2-3.1.6
kyr [Thu, 3 Apr 2025 10:50:04 +0000 (12:50 +0200)]
Merge pull request #2039 from ceph/dependabot/pip/jinja2-3.1.6

build(deps): bump jinja2 from 3.1.5 to 3.1.6

5 months agoMerge pull request #2035 from VallariAg/ci-integration-logs-archive
Zack Cerza [Wed, 2 Apr 2025 19:55:55 +0000 (13:55 -0600)]
Merge pull request #2035 from VallariAg/ci-integration-logs-archive

ci: save integration logs artifacts

5 months agoci: save intergation logs artifacts 2035/head
Vallari Agrawal [Fri, 7 Mar 2025 06:39:32 +0000 (12:09 +0530)]
ci: save intergation logs artifacts

Mount volume for archive_dir and then
upload it's logfiles as GA artifacts,
so we can use download-artifacts action
to download logs.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
6 months agobuild(deps): bump jinja2 from 3.1.5 to 3.1.6 2039/head
dependabot[bot] [Fri, 21 Mar 2025 18:20:29 +0000 (18:20 +0000)]
build(deps): bump jinja2 from 3.1.5 to 3.1.6

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
6 months agoMerge pull request #2038 from ceph/actions-pinning
David Galloway [Fri, 21 Mar 2025 18:19:25 +0000 (14:19 -0400)]
Merge pull request #2038 from ceph/actions-pinning

workflows/dev_container: Pin docker actions

6 months agoworkflows/dev_container: Pin docker actions 2038/head
Zack Cerza [Fri, 21 Mar 2025 18:02:06 +0000 (12:02 -0600)]
workflows/dev_container: Pin docker actions

These are the most recent releases for these actions.

Signed-off-by: Zack Cerza <zack@cerza.org>
6 months agoMerge pull request #2026 from badone/main
kyr [Fri, 21 Mar 2025 10:52:03 +0000 (11:52 +0100)]
Merge pull request #2026 from badone/main

task/internal/__init__.py:  Revisit core program extraction regex

6 months agotask/internal/__init__.py: Revisit core program extraction regex 2026/head
Brad Hubbard [Tue, 18 Feb 2025 03:56:18 +0000 (13:56 +1000)]
task/internal/__init__.py:  Revisit core program extraction regex

Fixes: https://tracker.ceph.com/issues/61586
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 months agoMerge pull request #1773 from batrick/1772-revert
Ilya Dryomov [Fri, 7 Mar 2025 17:12:37 +0000 (18:12 +0100)]
Merge pull request #1773 from batrick/1772-revert

Revert "Merge PR #1772 into master"

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
6 months agoMerge pull request #2034 from VallariAg/fix-locked-grace-period
Zack Cerza [Thu, 6 Mar 2025 16:25:52 +0000 (09:25 -0700)]
Merge pull request #2034 from VallariAg/fix-locked-grace-period

lock/query.py: add grace_time param to node_active_job()

6 months agolock/query.py: add grace_time param to node_active_job() 2034/head
Vallari Agrawal [Thu, 6 Mar 2025 13:38:29 +0000 (19:08 +0530)]
lock/query.py: add grace_time param to node_active_job()

This is because we only want this grace period
when looking for stale jobs. Not when jobs are
being unlocked by the supervisor by unlock_one_safe().

Fixes: https://github.com/ceph/teuthology/pull/2033#issuecomment-2703733380
Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
6 months agoMerge pull request #2033 from ceph/node-cleanup-grace
Zack Cerza [Wed, 5 Mar 2025 16:50:39 +0000 (09:50 -0700)]
Merge pull request #2033 from ceph/node-cleanup-grace

node-cleanup: Grace period for inactive jobs

6 months agonode-cleanup: Grace period for inactive jobs 2033/head
Zack Cerza [Tue, 4 Mar 2025 23:37:53 +0000 (16:37 -0700)]
node-cleanup: Grace period for inactive jobs

Once a job is marked finished, the supervisor may still be waiting to unlock its
nodes. Give jobs five minutes to clean up nodes before we consider them "stale".

Signed-off-by: Zack Cerza <zack@cerza.org>
6 months agonode-cleanup: Use plain log format
Zack Cerza [Tue, 4 Mar 2025 23:09:30 +0000 (16:09 -0700)]
node-cleanup: Use plain log format

Signed-off-by: Zack Cerza <zack@cerza.org>
6 months agonode-cleanup: Provide link to job
Zack Cerza [Tue, 4 Mar 2025 21:43:59 +0000 (14:43 -0700)]
node-cleanup: Provide link to job

Instead of just the raw description.

Signed-off-by: Zack Cerza <zack@cerza.org>
6 months agoRevert "Merge PR #1772 into master" 1773/head
Patrick Donnelly [Thu, 23 Jun 2022 14:18:47 +0000 (10:18 -0400)]
Revert "Merge PR #1772 into master"

This reverts commit 082ae7ef4302fa54665ed0a2535e8e254118dcfd, reversing
changes made to 31a356e0c737f1781079045bb8dc366982be8713.

The original ea170935d4b1c78dc6fd6beae6b3fda65b296f57 removed a method that had
been moved to ceph.git but that broke upgrade tests from releases up to Octopus
where get_valgrind_args had not been backported. So a revert was done until all
upgrade paths have the method.

Now that Quincy is EOL, we can revert the revert.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 months agoMerge pull request #2030 from jmundack/add_codeowners
Zack Cerza [Thu, 27 Feb 2025 21:09:29 +0000 (14:09 -0700)]
Merge pull request #2030 from jmundack/add_codeowners

Add CODEOWNERS file

6 months agoAdd CODEOWNERS file 2030/head
Joseph Mundackal [Thu, 27 Feb 2025 00:15:10 +0000 (19:15 -0500)]
Add CODEOWNERS file

6 months agoMerge pull request #2020 from ceph/dependabot/pip/virtualenv-20.26.6
kyr [Thu, 27 Feb 2025 00:13:31 +0000 (01:13 +0100)]
Merge pull request #2020 from ceph/dependabot/pip/virtualenv-20.26.6

build(deps): bump virtualenv from 20.25.0 to 20.26.6

6 months agoMerge pull request #2028 from kshtsk/wip-docs-laptop-dispatcher
Zack Cerza [Wed, 26 Feb 2025 21:12:56 +0000 (14:12 -0700)]
Merge pull request #2028 from kshtsk/wip-docs-laptop-dispatcher

docs/laptop: replace workers with dispatcher

6 months agoMerge pull request #2027 from kshtsk/wip-almarocky
Zack Cerza [Wed, 26 Feb 2025 21:09:59 +0000 (14:09 -0700)]
Merge pull request #2027 from kshtsk/wip-almarocky

opsys: add AlmaLinux and RockyLinux and Tumbleweed to distro codename map

6 months agoorchestra/opsys: add tumbleweed version to the distro version map 2027/head
Kyrylo Shatskyy [Wed, 26 Feb 2025 18:42:05 +0000 (19:42 +0100)]
orchestra/opsys: add tumbleweed version to the distro version map

openSUSE Tumbleweed is a rolling release, so it has fixed version number
and changable version id is represented by a date of snapshot,
example of vm image:

openSUSE-Tumbleweed-Minimal-VM.x86_64-1.0.0-Cloud-Snapshot20250224.qcow2

In comparison, openSUSE Leap has regular release, incremental version, for example:
15.0, 15.1, ..., 15.6, etc., and recently introduced 16.0 alpha.

Because there is no sense to stick to the date, it is changing almost daily,
it is suggested to fix Tumbleweed version to 1.0 to distinguish from Leap.
As a side effect, once Teuthology locks a node with imaged Tumbleweed
it updates os version correctly to date of release in paddles, and
pulpito display it interface correspondingly as the date.

Signed-off-by: Kyrylo Shatskyy <kyrylo.shatskyy@clyso.com>
6 months agoorchestra/opsys: add 'alma' and 'rocky' to default os map
Kyr Shatskyy [Wed, 19 Feb 2025 22:50:39 +0000 (23:50 +0100)]
orchestra/opsys: add 'alma' and 'rocky' to default os map

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
6 months agoorchestra/opsys: add almalinux versions
Kyr Shatskyy [Tue, 11 Feb 2025 14:47:56 +0000 (15:47 +0100)]
orchestra/opsys: add almalinux versions

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
6 months agoorchestra/opsys: add 'rocky' to DISTRO_CODENAME_MAP
Kyr Shatskyy [Wed, 5 Feb 2025 17:28:32 +0000 (18:28 +0100)]
orchestra/opsys: add 'rocky' to DISTRO_CODENAME_MAP

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@clyso.com>
6 months agodocs/laptop: replace workers with dispatcher 2028/head
Kyr Shatskyy [Tue, 25 Feb 2025 15:59:16 +0000 (16:59 +0100)]
docs/laptop: replace workers with dispatcher

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
7 months agoMerge pull request #2025 from ceph/nuke-unlock
Zack Cerza [Thu, 13 Feb 2025 21:57:30 +0000 (14:57 -0700)]
Merge pull request #2025 from ceph/nuke-unlock

supervisor: Unlock nodes after reimage failure

7 months agonode_active_job: Improve http error handling 2025/head
Zack Cerza [Mon, 3 Feb 2025 21:17:32 +0000 (14:17 -0700)]
node_active_job: Improve http error handling

Signed-off-by: Zack Cerza <zack1@ibm.com>
7 months agonode_cleanup: Log owner when query fails
Zack Cerza [Mon, 3 Feb 2025 21:12:10 +0000 (14:12 -0700)]
node_cleanup: Log owner when query fails

Signed-off-by: Zack Cerza <zack1@ibm.com>
7 months agosupervisor: Unlock nodes after reimage failure
Zack Cerza [Mon, 3 Feb 2025 21:06:57 +0000 (14:06 -0700)]
supervisor: Unlock nodes after reimage failure

Signed-off-by: Zack Cerza <zack1@ibm.com>
7 months agonode_cleanup: Log a summary of what was done
Zack Cerza [Mon, 3 Feb 2025 21:04:11 +0000 (14:04 -0700)]
node_cleanup: Log a summary of what was done

Signed-off-by: Zack Cerza <zack1@ibm.com>
7 months agoMerge pull request #2024 from kshtsk/wip-64452
Zack Cerza [Thu, 30 Jan 2025 22:28:35 +0000 (15:28 -0700)]
Merge pull request #2024 from kshtsk/wip-64452

7 months agoMerge pull request #1984 from kshtsk/wip-laptop-dispatcher
Zack Cerza [Thu, 30 Jan 2025 22:27:13 +0000 (15:27 -0700)]
Merge pull request #1984 from kshtsk/wip-laptop-dispatcher