ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
task/ssh_keys: use remote.write_file instead misc.create_file
Because misc.create_file with data arguments creates file,
changes permissions, and later appends the data using
misc.append_lines_to_file, this logic makes the algorythm
above to fail if the file created without write permissions.
2020-09-05T10:00:32.003 INFO:teuthology.task.ssh_keys:pushing keys to smithi086.front.sepia.ceph.com for ubuntu
2020-09-05T10:00:32.003 INFO:teuthology.orchestra.run.smithi086:> rm -f -- /home/ubuntu/.ssh/id_rsa
2020-09-05T10:00:32.046 INFO:teuthology.orchestra.run.smithi086:> touch /home/ubuntu/.ssh/id_rsa && chmod 500 -- /home/ubuntu/.ssh/id_rsa
2020-09-05T10:00:32.095 INFO:teuthology.orchestra.run.smithi086:> set -ex
2020-09-05T10:00:32.096 INFO:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/.ssh/id_rsa conv=notrunc oflag=append
2020-09-05T10:00:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:+ dd of=/home/ubuntu/.ssh/id_rsa conv=notrunc oflag=append
2020-09-05T10:00:32.142 INFO:teuthology.orchestra.run.smithi086.stderr:dd: failed to open '/home/ubuntu/.ssh/id_rsa': Permission denied
2020-09-05T10:00:32.142 DEBUG:teuthology.orchestra.run:got remote process result: 1
However we have more advanced remote.write_file function now,
which does not have such issues and moreover creates file
with the data provided in a single hop without trying to
download the file locally.
The yaml.safe_load reads the fail_log opened file
and shifts the offset to the end of stream.
However in case of error we need to shift offset
to the begin of the file stream, so we can read
data again.
Deprecate misc.write_file() and misc.sudo_write_file()
in favor of the orchestra.remote package methods.
The code of the misc's methods is now calling the
remote's ones.
Patrick Donnelly [Wed, 26 Aug 2020 14:29:27 +0000 (07:29 -0700)]
task/install: skip package removal by default
Every teuthology job cleans up its package install after completion.
This was necessary cleanup when we didn't reimage boxes before the use
of FOG as we wanted a "clean" slate for the next job to acquire the
machine. Now this is just unnecessary work which takes up valuable
machine time. For one job I looked at, this takes about 2 minutes.
We should still test that there are no unexpected issues with removing
the packages but this can be delegated to a small subset of smoke tests.
That will be posted in another PR to ceph.git.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Kyr Shatskyy [Tue, 25 Aug 2020 07:40:03 +0000 (09:40 +0200)]
kill: find targets for killing a job
In some circumstances teuthology-kill does not unlock the nodes
if it is used to kill just a single job from a run, for example,
while a job in the middle of locking many targets the job_info
does not include targets yet, correspondingly if someone kills
the job at that point, the targets remain in locked state.
This commit address the issue.
Varsha Rao [Wed, 19 Aug 2020 11:32:58 +0000 (17:02 +0530)]
teuthology-suite: Check the priority of jobs to be run
Check if the passed testing priority is according to the range mentioned in
developer guide[1]. This patch also adds '--force-priority' flag which
allows to skip the priority check.
sunilkumarn417 [Tue, 18 Aug 2020 08:03:04 +0000 (13:33 +0530)]
Fix py3 comparison issue(None with integers) in pid method and
Fix split issue which creates empty string value due to newline character at
end of command response.
please note, some packages not related to python-openstackclient are
also updated. the reason is that, because before this change, all
packages are pin'ed to a certain version, but after this change
only the direct dependencies are pin'ed using requirements.txt,
so, if any of the direct dependencies does not pin its own dependencies
to a certain version, pip-compile will pick the latest available version
from pypi to fulfill the requirement, and generated requirements.txt
accordingly.
requirements.in: add requirements.in back for tracking pinned direct requirements
and update `update-requirements.sh` to use requirements.in for building
requirements.txt
for some install_requires versus requirements.txt, see
https://packaging.python.org/discussions/install-requires-vs-requirements/.
in an ideal world, we should have not put those non-essential dependencies in
``setup.py``.
teuthology: don't use git to define version for released teuthology
After installing teuthology using pip there can be found error messages
in the log when running any command, for example 'teuthology --version':
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Can't get version from git rev-parse Command '['git', 'rev-parse', '--short', 'HEAD']' returned non-zero exit status 128.
1.0.0
Kyr Shatskyy [Sun, 10 May 2020 14:45:09 +0000 (16:45 +0200)]
suite: allow override architecture
By default for generation job configuration and filtering
them out there is 'arch' field is used determined automatically
via request to paddles database for give machine_type.
This makes it possible to override the 'arch' value with
teuthology-suite '--arch' parameter. This is only useful
at the moment just when a user don't one to make any request
to paddles.
Originally the arch is used to filter out the suites
which are not supposed to be queued on the given nodes.
In future we probably need to have tests with heterogeneous
configuration which will use multiple architectures.
run_tasks.py: Python3 don't have exception.message anymore.
This was probably working fine due to six library which was removed
recently: https://github.com/ceph/teuthology/commit/9f99b9298265be583c350c71de47109bac4bf6f1
Error seen:
```
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/worker/src/github.com_ceph_teuthology_master/teuthology/contextutil.py", line 33, in nested
yield vars
File "/home/worker/src/github.com_ceph_teuthology_master/teuthology/task/install/__init__.py", line 612, in task
yield
File "/home/worker/src/github.com_ceph_teuthology_master/teuthology/run_tasks.py", line 151, in run_tasks
if e.message == 'too many values to unpack':
AttributeError: 'ValueError' object has no attribute 'message'
```
Nathan Cutler [Tue, 30 Jun 2020 12:14:45 +0000 (14:14 +0200)]
kernel: add kernel version matching code for opensuse
The current code for checking if the running kernel version matches the
most recent kernel version in the repos does not work on opensuse when
"-k distro" is given.
This commit adds an opensuse-specific codepath with a version match
check that works in the opensuse testing environment.
Vasu Kulkarni [Thu, 24 Jan 2019 21:47:58 +0000 (13:47 -0800)]
scripts: add cli tools for reimaging nodes without locking
Add teuthology-reimage cli tool to be able to provision nodes using
Fog or Pelagos without locking and unlocking.
This is useful, for example, when someone just locks the node for
development or debugging purposes and do not want to release while
resetting the image, because it can happen that there are no free
nodes available.
Kyr Shatskyy [Tue, 12 May 2020 16:27:49 +0000 (18:27 +0200)]
schedule: do not report status for first and last in suite jobs
Addresses the issue when teuthology run gets stuck with
first_in_suite or laste_in_suite jobs in queued state.
Attention: This change requires the next steps,
which are not mutually exclusive:
1) server teuthology worker restart, otherwise old
worker's code will try to remove reported job
from paddles and exit with unexpected exception.
2) user's teuthology runner environment should be
updated to recent code, because new workers will
not cleanup FIS and LIS jobs and they will remain
in paddles, correspondingly the run will get stuck.
projects / teuthology.git / log