systemd test

Revert "testnode: try centos9 again"

This reverts commit fc94a550e4c426faae9bba4631d476221f242630.

testnode: try centos9 again

testnode dockerfile sed format

testnode container changes

testnode call systemd explicitly

test_locking: Tolerate CentOS Stream's versioning

It's not a problem functionally, so explicitly allow it in the unit
test.

Signed-off-by: Zack Cerza <zack@redhat.com>

teuth_suite none

containers/testnode: no_cgroups_split

testnode install lsb_release

crun

drop catatonit

testnode: try centos 8.stream

testnode kill init

testnode drop fuse, add catatonit

teuth.sh drop filter

containers/teuthology-dev: Parametrize suite

containers/testnode: osd_method

containers/testnode: Install pkgs for ceph tests

containers/testnode: Write /scratch_devs

containers/testnode: Manage a few items in /run/

bc of systemd

containers/testnode: Use systemd

containers/testnode: Support nesting

containers/testnode: Install podman

To get a recent version we have to switch OSes.

containers/teuthology: Pin base image to focal

containers/teuthology: Shorten watchdog_interval

TEMP? teuth-dev dockerfile rm ansible

TEMP cephadm image

containers: Use secrets for SSH keys, not env vars

FIXME docs/docker-compose: Move to new containers/ dir

the dir is not yet empty

dispatcher: Log the path to the supervisor logfile

Since we're no longer logging that output from the dispatcher, make it
easy to find the supervisor log.

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Keep job output out of stdout/err

If we want the dispatcher to run under systemd, let's keep its output a
bit more terse. The logfile will still be available.

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Exit gracefully on SIGTERM

Well, relatively soon after receiving it.

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Log why we're stopping

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1814 from NitzanMordhai/wip-nitzan-scrapy-deadjobs-nonetype-match

scrapy: comparing dead job skip typeNone

Merge pull request #1793 from cybozu/fix-docker-compose-postgres

Specify PostgreSQL v14 to avoid permission errors.

scrapy: comparing dead job skip typeNone
SequenceMatcher can't get NoneType when comparing 2 dead jobs,
adding check and compare for those jobs.

Fixes: https://tracker.ceph.com/issues/58264
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>

Merge pull request #1811 from ceph/wip-58491

misc.get_system_type: Use Remote.os

misc.get_system_type: Use Remote.os

... instead of lsb_release, as newer distros don't ship it.

Fixes: #58491
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1810 from ceph/selinux-denial-fix

task/selinux: Fix regressed grepping of audit logs

task/selinux: Fix regressed grepping of audit logs

Fixes: https://tracker.ceph.com/issues/58610
Introduced: a106217
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1805 from ceph/el-py39

bootstrap: Correct python3 package names for EL8/9

Merge pull request #1807 from kshtsk/wip-update-leap

orchestra/opsys: add newer sle and leap versions

orchestra/opsys: add newer sle and leap versions

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

bootstrap: Correct python3 package names for EL8/9

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1806 from ceph/fix-pytest

tests: Fix warnings, deprecations, and broken tests

Merge pull request #1802 from ceph/faster-newest

suite: Speed up 'newest' feature

tox.ini: Update to reflect passenv change

See the second bullet point here:
https://tox.wiki/en/4.0.3/faq.html#tox-4-changed-ini-rules

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Fix bugs revealed by mock's recent change

Once the asserts were corrected, these tests started failing.
Fortunately, they didn't seem to be masking any actual bugs.

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Fix broken Mock assert statements

This breaks (or, rather, reveals the broken state of) several unit
tests. A subsequent commit will fix them.

See https://github.com/testing-cabal/mock/commit/21787a948cf1186f3bd38462d5b912784fa81218

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Make sure to patch repo_utils ops

These tests were hitting the network.

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Rename setup() and teardown() methods

These were nose methods, and pytest is dropping support for them.

Signed-off-by: Zack Cerza <zack@redhat.com>

Fix 'invalid escape sequence' deprecation warnings

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1799 from dmick/wip-bootstrap

bootstrap: rpm systems: use --whatprovides to test for package presence

Merge pull request #1795 from dmick/wip-log-fog-exceptions

provision/fog.py: log exceptions while looping for ssh

bootstrap: rpm systems: use --whatprovides to test for package presence

the package name isn't necessarily the name of the dependency

Signed-off-by: Dan Mick <dmick@redhat.com>

provision/fog.py: log exceptions while looping for ssh

Make sure the user knows of potential configuration problems, etc.

Fixes: https://tracker.ceph.com/issues/58015
Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1797 from ceph/unique-dispatcher

dispatcher: Refuse to start extra dispatchers

suite: Speed up 'newest' feature

For a 'newest' value of N, we were calling find_git_parent N times.
find_git_parent was invoking githelper's 'refresh' endpoint each time,
which in my testing took 20-30 seconds. So, for a sha1 that needed 20
backtracks, I was seeing teuthology-suite take eight minutes to
complete. With this change, it took 30 seconds for the same
teuthology-suite invocation.

We were also not including the trailing slash on the 'refresh' endpoint,
resulting in a 308 redirect each time - doubling the request count.
That's fixed as well.

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Refuse to start extra dispatchers

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1798 from ceph/wip-py38

Drop Python 3.6; Allow choosing Python version at runtime

Update packaging

"pyproject-api 1.2.1 has requirement packaging>=21.3, but you have
packaging 21.0."

Signed-off-by: Zack Cerza <zack@redhat.com>

Update tomli

This fixes a CI breakage caused by inconsistent dependencies,
specifically: "pyproject-api 1.2.1 has requirement tomli>=2.0.1;
python_version < "3.11", but you have tomli 1.2.1."

Signed-off-by: Zack Cerza <zack@redhat.com>

Update pip-tools

The previous version was incompatible with newer pip versions!

Signed-off-by: Zack Cerza <zack@redhat.com>

Default to python 3.8

Signed-off-by: Zack Cerza <zack@redhat.com>

docker-compose: Pin postgres to v14

Starting with v15, paddles is broken out of the box. We'll need to fix
that, of course, but not here.

Signed-off-by: Zack Cerza <zack@redhat.com>

Drop Python 3.6 as a supported version

It was EOLed a year ago.

Signed-off-by: Zack Cerza <zack@redhat.com>

.github: Test with more Python versions

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Rebuild wrong-python venvs

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Use --use-pep517 when installing reqs

See https://github.com/pypa/pip/issues/8559

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Support alternate python versions

And check for their usability before proceeding

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1794 from ceph/githelper

config: Update githelper URL

config: Update githelper URL

Signed-off-by: Zack Cerza <zack@redhat.com>

Specify PostgreSQL v14 to avoid permission errors.

PostgreSQL v15  has a notable change that revokes the CREATE permission
from users except a database owner. So we can not execute paddles using
it. As a workaround, I specify PostgreSQL version.

ref: https://www.postgresql.org/about/news/postgresql-15-released-2526/

Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>

Merge pull request #1790 from ceph/no-devtmpfs-scratch-devs