task/kernel.py: sort installed kernels by version

rpm -q --last sorts by timestamp-of-install, which does not
necessarily correlate with "latest version".  sort -rV does.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1804 from ceph/dependabot/pip/wheel-0.38.1

build(deps): bump wheel from 0.36.2 to 0.38.1

build(deps): bump wheel from 0.36.2 to 0.38.1

Bumps [wheel](https://github.com/pypa/wheel) from 0.36.2 to 0.38.1.
- [Release notes](https://github.com/pypa/wheel/releases)
- [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst)
- [Commits](https://github.com/pypa/wheel/compare/0.36.2...0.38.1)

---
updated-dependencies:
- dependency-name: wheel
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Merge pull request #1831 from kamoltat/wip-ksirivad-rerun-readme

teuthology-suite: --seed & --subset now also stored in teuthology.log, config.yaml and orig.config.yaml
Reviewed-by: Zack Cerza <zcerza@redhat.com>

Merge pull request #1834 from ceph/disp_ls_remote

worker.prep_job: Skip job if ls_remote fails

worker.prep_job: Skip job if ls_remote fails

This is preferable to letting the dispatcher die.

Signed-off-by: Zack Cerza <zack@redhat.com>

suite/run.py: Added seed and subset to base_config

In addition to being stored in results.log
`--seed` and `--subset` are now also stored in:

`teuthlogy.log`, `config.yaml` and `orig.config.yaml`.

Fixes: https://tracker.ceph.com/issues/59300
Signed-off-by: Kamoltat <ksirivad@redhat.com>

teuthology-suite: Log errors & warnings if results.log is missing during --rerun

Notify the user if `results.log` is missing when
they issue a rerun.

Also, edited teuthology-suite doc
to inform the user about how `--rerun` by
default parse `--seed`, `--subset` and
`--no-nested-subset` from `results.log`
by default

Fixes: https://tracker.ceph.com/issues/59300
Signed-off-by: Kamoltat <ksirivad@redhat.com>

Merge pull request #1832 from ceph/deps

Merge pull request #1827 from ceph/tox-no-osp

Drop argparse as a requirement

It's part of the standard library.

Signed-off-by: Zack Cerza <zack@redhat.com>

requirements.txt: Add prometheus_client

This should have been added a couple PRs back.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1833 from ceph/gha-ubuntu-version

bootstrap: apt-get update before installing

Signed-off-by: Zack Cerza <zack@redhat.com>

.github: Explicitly define test matrix

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1824 from batrick/parallel-gzip

teuthology: do not compress tarballs when pulling dir

Where we use this, it's for pulling log files that are already
compressed. Do not waste time double compressing!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

teuthology/misc: give verbose gzip output

For future analysis.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

teuthology/misc: use medium compression

To speed things up.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

teuthology/misc: parallelize gzip

Our machines have lots of cores, use them!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #1826 from ceph/job-time

tox: Don't run openstack by default

It's quite time-consuming, and we're not sure if it's in use at all.

Signed-off-by: Zack Cerza <zack@redhat.com>

sueprvisor: Do not instrument certain job times

This should only really include first/last-in-suite jobs.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1819 from ceph/timing

task.internal: Fix import of get_builder_project

This is another problematic circular import.

Signed-off-by: Zack Cerza <zack@redhat.com>

task.kernel: Make task() a context manager

This will cause it to be instrumented for timing via the bits in
run_tasks.

Signed-off-by: Zack Cerza <zack@redhat.com>

exporter: Make JobResults a singleton

This should avoid "Duplicated timeseries in CollectorRegistry" errors.

Signed-off-by: Zack Cerza <zack@redhat.com>

docs: Add new document for teuthology-exporter

Signed-off-by: Zack Cerza <zack@redhat.com>

supervisor: Add instrumentation for reimaging time

Signed-off-by: Zack Cerza <zack@redhat.com>

repo_utils: Add instrumentation for bootstrap time

And fix a couple of linter errors.

Signed-off-by: Zack Cerza <zack@redhat.com>

run_tasks: Add instrumentation for task time

Signed-off-by: Zack Cerza <zack@redhat.com>

supervisor: Add instrumentation for job time

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Add instrumentation for locking time

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1825 from ceph/cl-import

nuke: Fix import of task.internal.check_lock

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1823 from ceph/fix-kill

task.cephmetrics: Fix broken import

Signed-off-by: Zack Cerza <zack@redhat.com>

Fix more circular imports

Signed-off-by: Zack Cerza <zack@redhat.com>

Test all module imports

Signed-off-by: Zack Cerza <zack@redhat.com>

Restore some import statements

Some of the circular import fixes were overkill.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1822 from ceph/fix-kill

kill.kill_job: Tweak job status reporting

kill.kill_job: Tweak job status reporting

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 45117a8781ce6438f22140c00d05435c7781648b)

Merge pull request #1817 from ceph/exporter

Fix a circular import

This was exposed during development of the exporter.

Signed-off-by: Zack Cerza <zack@redhat.com>

setup.cfg: Set flake8 max line length to 100

It's what black uses.

Signed-off-by: Zack Cerza <zack@redhat.com>

teuthology-exporter: Report job status

Signed-off-by: Zack Cerza <zack@redhat.com>

Add teuthology-exporter, a Prometheus exporter

Signed-off-by: Zack Cerza <zack@redhat.com>

Add active_machine_types config item

I'm adding this so that the exporter can know which machines types to
care about.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1818 from ceph/fix-nop

task.ansible: Fix an IndexError w/ 0 remotes

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1815 from kshtsk/wip-sle15sp3

orchestra/opsys: add sle 15 sp5

Merge pull request #1816 from ceph/paddles-fault-tolerance

supervisor: Ignore some reporting failures

Specifically, the pings coming from the watchdog loop

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Do not time out when locking machines

Signed-off-by: Zack Cerza <zack@redhat.com>

safe_while: Retry forever when tries == -1

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1812 from ceph/orch-ssh-key

orchestra/opsys: add sle 15 sp5

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1814 from NitzanMordhai/wip-nitzan-scrapy-deadjobs-nonetype-match

scrapy: comparing dead job skip typeNone

Merge pull request #1793 from cybozu/fix-docker-compose-postgres

Specify PostgreSQL v14 to avoid permission errors.

scrapy: comparing dead job skip typeNone
SequenceMatcher can't get NoneType when comparing 2 dead jobs,
adding check and compare for those jobs.

Fixes: https://tracker.ceph.com/issues/58264
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>

orchestra.connection: Look for 'ssh_key' in config

On systems where SSH is configured using ~/.ssh/config.d/, we don't have
a great way of locating and processing the full configuration, so we
might not find the correct key to use. Allow the user to specify one key
to use for all test nodes if they wish.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1811 from ceph/wip-58491

misc.get_system_type: Use Remote.os

misc.get_system_type: Use Remote.os

... instead of lsb_release, as newer distros don't ship it.

Fixes: #58491
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1810 from ceph/selinux-denial-fix

task/selinux: Fix regressed grepping of audit logs

task/selinux: Fix regressed grepping of audit logs

Fixes: https://tracker.ceph.com/issues/58610
Introduced: a106217
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1805 from ceph/el-py39

bootstrap: Correct python3 package names for EL8/9

Merge pull request #1807 from kshtsk/wip-update-leap

orchestra/opsys: add newer sle and leap versions

orchestra/opsys: add newer sle and leap versions

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

bootstrap: Correct python3 package names for EL8/9

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1806 from ceph/fix-pytest

tests: Fix warnings, deprecations, and broken tests

Merge pull request #1802 from ceph/faster-newest

suite: Speed up 'newest' feature

tox.ini: Update to reflect passenv change

See the second bullet point here:
https://tox.wiki/en/4.0.3/faq.html#tox-4-changed-ini-rules

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Fix bugs revealed by mock's recent change

Once the asserts were corrected, these tests started failing.
Fortunately, they didn't seem to be masking any actual bugs.

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Fix broken Mock assert statements

This breaks (or, rather, reveals the broken state of) several unit
tests. A subsequent commit will fix them.

See https://github.com/testing-cabal/mock/commit/21787a948cf1186f3bd38462d5b912784fa81218

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Make sure to patch repo_utils ops

These tests were hitting the network.

Signed-off-by: Zack Cerza <zack@redhat.com>

tests: Rename setup() and teardown() methods

These were nose methods, and pytest is dropping support for them.

Signed-off-by: Zack Cerza <zack@redhat.com>

Fix 'invalid escape sequence' deprecation warnings

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1799 from dmick/wip-bootstrap

bootstrap: rpm systems: use --whatprovides to test for package presence

Merge pull request #1795 from dmick/wip-log-fog-exceptions

provision/fog.py: log exceptions while looping for ssh

bootstrap: rpm systems: use --whatprovides to test for package presence

the package name isn't necessarily the name of the dependency

Signed-off-by: Dan Mick <dmick@redhat.com>

provision/fog.py: log exceptions while looping for ssh

Make sure the user knows of potential configuration problems, etc.

Fixes: https://tracker.ceph.com/issues/58015
Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1797 from ceph/unique-dispatcher

dispatcher: Refuse to start extra dispatchers

suite: Speed up 'newest' feature

For a 'newest' value of N, we were calling find_git_parent N times.
find_git_parent was invoking githelper's 'refresh' endpoint each time,
which in my testing took 20-30 seconds. So, for a sha1 that needed 20
backtracks, I was seeing teuthology-suite take eight minutes to
complete. With this change, it took 30 seconds for the same
teuthology-suite invocation.

We were also not including the trailing slash on the 'refresh' endpoint,
resulting in a 308 redirect each time - doubling the request count.
That's fixed as well.

Signed-off-by: Zack Cerza <zack@redhat.com>

dispatcher: Refuse to start extra dispatchers

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1798 from ceph/wip-py38

Drop Python 3.6; Allow choosing Python version at runtime

Update packaging

"pyproject-api 1.2.1 has requirement packaging>=21.3, but you have
packaging 21.0."

Signed-off-by: Zack Cerza <zack@redhat.com>

Update tomli

This fixes a CI breakage caused by inconsistent dependencies,
specifically: "pyproject-api 1.2.1 has requirement tomli>=2.0.1;
python_version < "3.11", but you have tomli 1.2.1."

Signed-off-by: Zack Cerza <zack@redhat.com>

Update pip-tools

The previous version was incompatible with newer pip versions!

Signed-off-by: Zack Cerza <zack@redhat.com>

Default to python 3.8

Signed-off-by: Zack Cerza <zack@redhat.com>

docker-compose: Pin postgres to v14

Starting with v15, paddles is broken out of the box. We'll need to fix
that, of course, but not here.

Signed-off-by: Zack Cerza <zack@redhat.com>

Drop Python 3.6 as a supported version

It was EOLed a year ago.

Signed-off-by: Zack Cerza <zack@redhat.com>

.github: Test with more Python versions

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Rebuild wrong-python venvs

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Use --use-pep517 when installing reqs

See https://github.com/pypa/pip/issues/8559

Signed-off-by: Zack Cerza <zack@redhat.com>

bootstrap: Support alternate python versions

And check for their usability before proceeding

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1794 from ceph/githelper

config: Update githelper URL

config: Update githelper URL

Signed-off-by: Zack Cerza <zack@redhat.com>

Specify PostgreSQL v14 to avoid permission errors.

PostgreSQL v15  has a notable change that revokes the CREATE permission
from users except a database owner. So we can not execute paddles using
it. As a workaround, I specify PostgreSQL version.

ref: https://www.postgresql.org/about/news/postgresql-15-released-2526/

Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>

Merge pull request #1790 from ceph/no-devtmpfs-scratch-devs