Kyr Shatskyy [Wed, 8 Jun 2022 11:28:15 +0000 (13:28 +0200)]
suite: fix type error when description is none
teuthology-watch fails when run is complete and jobs' description
gets None value
Example:
2022-06-07 16:41:22,538.538 INFO:teuthology.suite:waiting for the run runner-2022-06-07_16:41:04-suse:tier0-ses7p-none-default-ecp to complete
2022-06-07 16:41:22,539.539 DEBUG:teuthology.suite:the list of unfinished jobs will be displayed every 5.0 minutes
2022-06-07 16:46:22,599.599 DEBUG:teuthology.suite:wait for jobs ['654']
2022-06-07 16:51:22,633.633 DEBUG:teuthology.suite:wait for jobs ['654']
2022-06-07 16:51:22,686.686 INFO:teuthology.suite:wait is done
Traceback (most recent call last):
File "/home/runner/src/teuthology_master/virtualenv/bin/teuthology-wait", line 33, in <module>
sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-wait')())
File "/home/runner/src/teuthology_master/scripts/wait.py", line 30, in main
return teuthology.suite.wait(name, config.max_job_time, None)
File "/home/runner/src/teuthology_master/teuthology/suite/__init__.py", line 234, in wait
log.info(job['status'] + " " + url + " " + job['description'])
TypeError: must be str, not NoneType
Most runs don't use --no-nested-subset and for those that used --subset,
the if conditions would correctly pickup "seed" (when it mattered).
However, when --subset was not specified in the original run, the "seed"
was not correctly picked up. Therefore, the result of inserting the "if
no_nested_subset is None:" before the "elif seed is None:" caused it to
never read the seed for most folks teuthology runs.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 24 May 2022 16:14:04 +0000 (12:14 -0400)]
Merge PR #1704 into master
* refs/pull/1704/head:
teuthology/suite/test: test nested subsets
teuthology: add option to disable nested subsets
teuthology/suite: create nested matrix subsets
teuthology/suite: patch builtin open method
teuthology/test: use correct exception type
teuthology/suite/test: make sure patchers are cleaned up on exception
teuthology/suite/test: clarify variable name
Patrick Donnelly [Fri, 14 Jan 2022 20:25:14 +0000 (15:25 -0500)]
teuthology/suite: create nested matrix subsets
The general idea is to allow the `%` convolution operator to also subset
the resulting matrix. This is done by specifying a number of divisions
for the subset in the `%` file. Such as:
dir/%:
8
This commit maps a matrix index range of `[0, Subset.size())` to the
matrix it is taking a subset of, `[0, Matrix.size())`. To get full
coverage, a random number is used to specify "which" subset to use.
Contrast with the `--subset` argument to `teuthology-suite` which lets
you specify which subset.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
If config.teuthology_path is set, we don't fetch teuthology at runtime.
This can lead to confusing situations if --teuthology-branch is passed
to teuthology-suite, as the job config will claim a branch is used when
it is not. We can avoid this problem by refusing to schedule if there
is a mismatch.
With this change, we can avoid rerunning bootstrap on build if
requirements.txt hasn't changed. Build times after non-requirements
changes go from 3-5min to ~2s.
The docker-compose dev setup was the inspiration for this feature; it
allows us to split the installation of dependencies and the project
itself into two phases. This in turn allows us to avoid rerunning
bootstrap if requirements.txt didn't change.
* move the dependencies from options.install_requires to
options.extras_require if those dependencies are only used by tests.
as per
https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/#id5
> install_requires is a setuptools setup.py keyword that should be
> used to specify what a project minimally needs to run correctly.
* remove the duplicated dependencies from options.extras_require
if they are also listed in options.install_requires.
* update requirements.txt using
"pip-compile --extra=test pyproject.toml", so requirements.txt
can include the dependencies required by tests.
Zack Cerza [Thu, 31 Mar 2022 19:32:14 +0000 (15:32 -0400)]
Modernized versioning
Using setuptools_scm; we can automatically generate version numbers
instead of using this odd combination of hardcoding and running git
commands. This method is faster as well.
While doing packaging work, I noticed that teuthology began claiming it
couldn't find the 'tests' task. After some slightly painful debugging I
realized the issue was that the task was trying to import pytest, which
wasn't installed. The ModuleNotFoundError that was being raised was
being confused with the exception that would be raised if the task
couldn't be found at all. With this change, we see the root cause.
Instead of scheduling the teuthology:no-ceph suite and then exiting
when it finishes, schedule nothing and don't exit automatically. This
allows running whatever - and however many - tests are desired.
The docker-compose dev setup was the inspiration for this feature; it
allows us to split the installation of dependencies and the project
itself into two phases. This in turn allows us to avoid rerunning
bootstrap if requirements.txt didn't change.
Zack Cerza [Thu, 31 Mar 2022 19:32:14 +0000 (15:32 -0400)]
Modernized versioning
Using setuptools_scm; we can automatically generate version numbers
instead of using this odd combination of hardcoding and running git
commands. This method is faster as well.
This effectively hardcoded the path to the virtualenv we'll use, which
aside from being bad generally, also prevented us from supporting
absolute-pathed virtualenvs.
While doing packaging work, I noticed that teuthology began claiming it
couldn't find the 'tests' task. After some slightly painful debugging I
realized the issue was that the task was trying to import pytest, which
wasn't installed. The ModuleNotFoundError that was being raised was
being confused with the exception that would be raised if the task
couldn't be found at all. With this change, we see the root cause.
projects / teuthology.git / log