setup.cfg: Set flake8 max line length to 100

It's what black uses.

Signed-off-by: Zack Cerza <zack@redhat.com>

teuthology-exporter: Report job status

Signed-off-by: Zack Cerza <zack@redhat.com>

Add teuthology-exporter, a Prometheus exporter

Signed-off-by: Zack Cerza <zack@redhat.com>

Add active_machine_types config item

I'm adding this so that the exporter can know which machines types to
care about.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1818 from ceph/fix-nop

task.ansible: Fix an IndexError w/ 0 remotes

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1815 from kshtsk/wip-sle15sp3

orchestra/opsys: add sle 15 sp5

Merge pull request #1816 from ceph/paddles-fault-tolerance

supervisor: Ignore some reporting failures

Specifically, the pings coming from the watchdog loop

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Do not time out when locking machines

Signed-off-by: Zack Cerza <zack@redhat.com>

safe_while: Retry forever when tries == -1

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1812 from ceph/orch-ssh-key

orchestra/opsys: add sle 15 sp5

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1814 from NitzanMordhai/wip-nitzan-scrapy-deadjobs-nonetype-match

scrapy: comparing dead job skip typeNone

Merge pull request #1793 from cybozu/fix-docker-compose-postgres

Specify PostgreSQL v14 to avoid permission errors.

scrapy: comparing dead job skip typeNone
SequenceMatcher can't get NoneType when comparing 2 dead jobs,
adding check and compare for those jobs.

Fixes: https://tracker.ceph.com/issues/58264
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>

orchestra.connection: Look for 'ssh_key' in config

On systems where SSH is configured using ~/.ssh/config.d/, we don't have
a great way of locating and processing the full configuration, so we
might not find the correct key to use. Allow the user to specify one key
to use for all test nodes if they wish.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1811 from ceph/wip-58491

misc.get_system_type: Use Remote.os

misc.get_system_type: Use Remote.os

... instead of lsb_release, as newer distros don't ship it.

Fixes: #58491
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1810 from ceph/selinux-denial-fix

task/selinux: Fix regressed grepping of audit logs

task/selinux: Fix regressed grepping of audit logs

Fixes: https://tracker.ceph.com/issues/58610
Introduced: a106217
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1805 from ceph/el-py39

bootstrap: Correct python3 package names for EL8/9

Merge pull request #1807 from kshtsk/wip-update-leap

orchestra/opsys: add newer sle and leap versions

orchestra/opsys: add newer sle and leap versions

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

bootstrap: Correct python3 package names for EL8/9

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1806 from ceph/fix-pytest

tests: Fix warnings, deprecations, and broken tests

Merge pull request #1802 from ceph/faster-newest

suite: Speed up 'newest' feature

tox.ini: Update to reflect passenv change

See the second bullet point here:
https://tox.wiki/en/4.0.3/faq.html#tox-4-changed-ini-rules

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Fix bugs revealed by mock's recent change

Once the asserts were corrected, these tests started failing.
Fortunately, they didn't seem to be masking any actual bugs.

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Fix broken Mock assert statements

This breaks (or, rather, reveals the broken state of) several unit
tests. A subsequent commit will fix them.

See https://github.com/testing-cabal/mock/commit/21787a948cf1186f3bd38462d5b912784fa81218

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Make sure to patch repo_utils ops

These tests were hitting the network.

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Rename setup() and teardown() methods

These were nose methods, and pytest is dropping support for them.

Signed-off-by: Zack Cerza <zack@redhat.com>

Fix 'invalid escape sequence' deprecation warnings

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1799 from dmick/wip-bootstrap

bootstrap: rpm systems: use --whatprovides to test for package presence

Merge pull request #1795 from dmick/wip-log-fog-exceptions

provision/fog.py: log exceptions while looping for ssh

bootstrap: rpm systems: use --whatprovides to test for package presence

the package name isn't necessarily the name of the dependency

Signed-off-by: Dan Mick <dmick@redhat.com>

provision/fog.py: log exceptions while looping for ssh

Make sure the user knows of potential configuration problems, etc.

Fixes: https://tracker.ceph.com/issues/58015
Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1797 from ceph/unique-dispatcher

dispatcher: Refuse to start extra dispatchers

suite: Speed up 'newest' feature

For a 'newest' value of N, we were calling find_git_parent N times.
find_git_parent was invoking githelper's 'refresh' endpoint each time,
which in my testing took 20-30 seconds. So, for a sha1 that needed 20
backtracks, I was seeing teuthology-suite take eight minutes to
complete. With this change, it took 30 seconds for the same
teuthology-suite invocation.

We were also not including the trailing slash on the 'refresh' endpoint,
resulting in a 308 redirect each time - doubling the request count.
That's fixed as well.

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Refuse to start extra dispatchers

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1798 from ceph/wip-py38

Drop Python 3.6; Allow choosing Python version at runtime

Update packaging

"pyproject-api 1.2.1 has requirement packaging>=21.3, but you have
packaging 21.0."

Signed-off-by: Zack Cerza <zack@redhat.com>

Update tomli

This fixes a CI breakage caused by inconsistent dependencies,
specifically: "pyproject-api 1.2.1 has requirement tomli>=2.0.1;
python_version < "3.11", but you have tomli 1.2.1."

Signed-off-by: Zack Cerza <zack@redhat.com>

Update pip-tools

The previous version was incompatible with newer pip versions!

Signed-off-by: Zack Cerza <zack@redhat.com>

Default to python 3.8

Signed-off-by: Zack Cerza <zack@redhat.com>

docker-compose: Pin postgres to v14

Starting with v15, paddles is broken out of the box. We'll need to fix
that, of course, but not here.

Signed-off-by: Zack Cerza <zack@redhat.com>

Drop Python 3.6 as a supported version

It was EOLed a year ago.

Signed-off-by: Zack Cerza <zack@redhat.com>

.github: Test with more Python versions

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Rebuild wrong-python venvs

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Use --use-pep517 when installing reqs

See https://github.com/pypa/pip/issues/8559

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Support alternate python versions

And check for their usability before proceeding

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1794 from ceph/githelper

config: Update githelper URL

config: Update githelper URL

Signed-off-by: Zack Cerza <zack@redhat.com>

Specify PostgreSQL v14 to avoid permission errors.

PostgreSQL v15  has a notable change that revokes the CREATE permission
from users except a database owner. So we can not execute paddles using
it. As a workaround, I specify PostgreSQL version.

ref: https://www.postgresql.org/about/news/postgresql-15-released-2526/

Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>

Merge pull request #1790 from ceph/no-devtmpfs-scratch-devs