7b27e1db7: openstack: support /etc/network/intefaces injection 2358562cf: ensure VMs always have /etc/hosts set up 4378a505d: always allow unsigned deb packages 50b2db521: openstack: encode instance name with the full IP 6e828a33b: openstack: add 8.8.8.8 as a last resort resolver
Robin H. Johnson [Thu, 10 Dec 2015 19:55:53 +0000 (11:55 -0800)]
buildpackages: catch instance creation failure.
Split the sleep from the server creation, so we catch 'server create'
failures (eg due to quota):
> Quota exceeded for cores: Requested 16, but already used 10 of 20 cores
> (HTTP 403) (Request-ID: req-6467934e-db50-4479-995c-4d44dedf553a)
Robin H. Johnson [Thu, 10 Dec 2015 19:55:49 +0000 (11:55 -0800)]
buildpackages: Be robust about addresses
OpenStack could tell us the VM has multiple networks, and offers no
guarantee about the order of addresses either (the old code failed if
the v4 IP was first).
For now, take the first listed network, and the first listed IPv4
address therein. Comments contain more detailed examples of possible
output from openstack tool.
Also remove the need for using jq to parse the output.
Loic Dachary [Sun, 6 Dec 2015 12:08:20 +0000 (13:08 +0100)]
workunit: fetch from --ceph-git-url
The commit from which workunits are fetched must be retrieved
from --ceph-git-url via teuth_config.get_ceph_git_url() instead of
assuming it is available via git://git.ceph.com/ceph.git.
Using git://git.ceph.com/ceph.git is convenient because it supports git
archive. In the general case, some git servers such as github do not
support git archive and a full git clone must be done instead.
Loic Dachary [Sun, 6 Dec 2015 11:30:13 +0000 (12:30 +0100)]
workunit: allow tag to override sha1
The sha1 for the workunit task is always set by the suite.py task. The
tag must be checked before the sha1 othewise it cannot be used to
override the sha1.
Loic Dachary [Mon, 30 Nov 2015 15:55:38 +0000 (16:55 +0100)]
buildpackages: create the image if it does not exist
Since buildpackages runs before target provisioning, it is possible that
the desired image does not yet exist on a newly provisionned tenant (or
region).
Loic Dachary [Sat, 21 Nov 2015 11:12:49 +0000 (12:12 +0100)]
openstack: rbd/{thrash,qemu}: allocate three disks, always
The thrasher and qemu need disk attached to run against xfs or btrfs,
otherwise it will use the local file system which is always ext4. It
will succeed but this is a kind of false positive since no xfs or btrfs
related tests were run.
Loic Dachary [Sat, 14 Nov 2015 12:26:00 +0000 (13:26 +0100)]
buildpackages: refactor to not require remote
Most of the flavor, sha1, tag etc. selection logic as implemented in the
packaging module of teuthology relies on remote hosts. This is complex
to tests and inconvenient because hosts must be provisionned even before
trying to figure out which packages need to be installed.
Using remote hosts is necessary when bare metal targets are used because
teuthology must adapt to the operating system already installed. The
selection logic in the context of dynamically provisionned targets is
simpler because it is defined by the job being run.
The buildpackages is refactored to use only the job configuration to
figure out which packages must be built. It makes it specific to targets
that are dynamically provisionned. It would have to be modified to query
the remote host in the case of bare metal targets.
Loic Dachary [Tue, 27 Oct 2015 15:06:46 +0000 (00:06 +0900)]
buildpackages: fetch tags from the official Ceph repository
A clone of Ceph is not automagically updated with the tags from the
official Ceph repository. For a pull request based on master, git
describe will use whatever tags existed at the time the clone was made,
unless the author pull them from the official Ceph repository and later
git push --tags them.
The output of git describe is used to name the packages and if the
official tags are not present, the packages will be incorrectly
named. For instance instead of 9.0.3-34 the packages could be named
0.87-8433 because the v0.87 tag is the most recent tag in the
repository. That confuses the install task that will fail with:
'ceph version 0.87 was not installed, found 9.0.3.'
Link the distro directory to the directory containing all supported
distros. Add the x86_64 arch contraint required by the isa plugin to an
isolated file that is combined with all jobs.
Loic Dachary [Mon, 12 Oct 2015 16:46:38 +0000 (18:46 +0200)]
buildpackages: get ceph submodules
Prior to v0.80.9, autogen.sh did not get submodules. Copy/paste the
submodule initialization from newer autogen.sh in common.sh so that
v0.80.8 and below can be rebuilt from sources. It does not hurt to
update the submodules twice.
Loic Dachary [Fri, 9 Oct 2015 08:05:58 +0000 (10:05 +0200)]
buildpackages: walk the whole config tree to find sha1 to build
It is not enough to look for the first install task. In upgrade tests,
the install.upgrade task requires more packages to be built. In more
complicated tests using sequential and parallel tasks, the actual
install or install.upgrade task may be deeper in the config tree.
The install config may have contradicting tag/branch and sha1. When
suite.py prepares the jobs, it always overrides the sha1 with whatever
default is provided on the command line with --distro and what is found
in the gitbuilder. If it turns out that the tag or the branch in the
install config task is about another sha1, it will override anyway.
Instead of obtaining the tag, branch and sha1 directly from the
packaging.GitbuilderProject object, compute them from the returned
uri_reference data member. The uri_reference is used by the install task
to fetch packages in the gitbuilders and this is what buildpackages
needs to build.
Loic Dachary [Tue, 6 Oct 2015 13:47:34 +0000 (15:47 +0200)]
buildpackages: prefer GitbuilderProject.os_{type,version} over config.
The config['os_type'] and config['os_version'] are not always set for a given
job (for instance, in the rbd suite). When a suite runs, it relies on
default values, depending on the target Operating System and internal,
hard coded values associating ubuntu to 14.04 etc.
Instead of using config['os_{type,version}'] use the GitbuilderProject
equivalent which is set with the appropriate defaults.
tasks/ceph_fuse.py: virtual machines need flexible mount timeout
1) add a wait time before the mount attempt to let the cluster get set up.
By default this should be skipped, but for VMs and known-slow systems we
can give them 60 seconds.
2) Make the timeout configurable, with a 30-second default, but override it
for VM tests.
When using the teuthology-openstack command, buildpackages is implicit
and does not need to be included when running the test suite. Move the
stanza to the debug directory as a reminder when debuging.
Conflicts:
suites/rbd/qemu/workloads/qemu_bonnie.yaml
suites/rbd/qemu/workloads/qemu_fsstress.yaml
suites/rbd/qemu/workloads/qemu_iozone.yaml.disabled
suites/rbd/qemu/workloads/qemu_xfstests.yaml
suites/rbd/singleton/all/formatted-output.yaml
the conflicts are because https is used in master instead of http,
the rest is otherwise identical.
The config paramter of download_ceph_deploy does not have a ceph-deploy
item, therefore the ceph-deploy-branch parameter is always assumed to be
master.
ceph-deploy: always zap disk before creating an osd
The existing logic is to ceph-deploy osd create --zap-disk which will
zap the data device before preparing it. However it will not zap the
journal device (see http://tracker.ceph.com/issues/13291).
If ceph-deploy osd create fails, a fall back will zap both the data
device and the journal and try prepare again. This could work if
the device preparation and activation was synchronous and catch all
errors that could be caused by an unclean journal device. However,
the activation is asynchronous and it is entirely possible for a device
to be prepared successfully and fail to activate in the background.
The data and journal device are always zapped before calling ceph-deploy
osd create. The logic is simpler and the overhead is low.
buildpackages: user-data must be per os-type/os-version
CentOS 6.5 needs to install a package and reboot to grow the root file
system. Instead of assuming a common user-data.txt file can fit all
Operating Systems, make one user data per os-type/os-version combination.
The process run by flock must not inherit the file descriptor because
this will cause the lock to be held forever should the command survive
the call to flock. This is for instance the case for the ssh-agent.
buildpackages: get Ceph git url via get_ceph_git_url()
Instead of relying on git_base_url, use the get_ceph_git_url() to obtain
the URL of the Ceph repository to use with git clone. This allows the
user to override it via the git_ceph_url configuration option and the
--git-ceph-url command line option to teuthology-openstack.
projects / ceph.git / log