Merge pull request #927 from dachary/wip-16972-quiet

openstack: do not display openstack client warnings

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #928 from ceph/revert-925-wip-ovh-json

Revert "misc.sh(): Don't capture stderr by default"

Reviewed-by: Dan Mick <dmick@redhat.com>

Revert "misc.sh(): Don't capture stderr by default"

openstack: do not display openstack client warnings

Because they are concatenated with the STDOUT by the sh function despite
being output on STDERR and that breaks json parsing.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #926 from ceph/wip-beanstalkd-socketerror

worker: Don't die on a beanstalkc SocketError

Reviewed-by: Dan Mick <dmick@redhat.com>

Don't die on a beanstalkc SocketError

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #925 from ceph/wip-ovh-json

misc.sh(): Don't capture stderr by default

Reviewed-by: Dan Mick <dmick@redhat.com>

misc.sh(): Don't capture stderr by default

If not capturing it, log it.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #924 from smithfarm/wip-lock-centos

teuthology-lock: add centos 7.2 to downburst default table

Merge pull request #923 from ceph/wip-14794

valgrind: wildcard boost supression

Merge pull request #916 from smithfarm/wip-python-split

packages.yaml: reflect python-ceph package split

teuthology-lock: drop openSUSE 12.2 from downburst default table

AFAIK Downburst does not provide any openSUSE images.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

teuthology-lock: add centos 7.2 to downburst default table

When downburst is unavailable, teuthology-lock reverts to default_table and
fails because the only supported CentOS version, 7.2., is missing.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

valgrind: wildcard boost supression

...so that it will match mangled names.

Fixes: http://tracker.ceph.com/issues/14794
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #921 from ceph/wip-ansible-dead

CephLab: playbook failures result in status 'dead'

CephLab: playbook failures result in status 'dead'

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #914 from ceph/wip-orchestra-logging

orchestra.run logging

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #920 from ceph/wip-worker-sentinel

Fix an obvious bug in sentinel()

Reviewed-by: Dan Mick <dmick@redhat.com>

Fix an obvious bug in sentinel()

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #917 from ceph/wip-dv-broken

Fix regression when using teuthology-suite with --kernel

Reviewed-by: Dan Mick <dmick@redhat.com>

choose_kernel(): fix call to get_gitbuilder_hash

http://tracker.ceph.com/issues/16838
Fixes: 16838
Signed-off-by: Zack Cerza <zack@redhat.com>

Remove defaults for get_gitbuilder_hash args

http://tracker.ceph.com/issues/16838
Fixes: 16838
Signed-off-by: Zack Cerza <zack@redhat.com>

Add reproducer for teuthology-suite regression

http://tracker.ceph.com/issues/16838

Signed-off-by: Zack Cerza <zack@redhat.com>

Update tests to reflect internal changes

Signed-off-by: Zack Cerza <zack@redhat.com>

Instead of StreamHandler, rework copy_file_to()

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #913 from dmick/wip-os-version

teuthology-suite: add --D/--distroversion

Drop use of xreadlines()

Signed-off-by: Zack Cerza <zack@redhat.com>

GitbuilderProject: fix/improve warning about multiple refs

warn() tries to warn you when you overspecify a commit (by
giving more than one of ref, tag, branch, or sha1).

1) fix it to actually work (it was always warning)
2) Improve the warning to indicate not only which is taking
precedence but what was supplied for the others.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

suite: Allow distroversion to contain either codename or version

Signed-off-by: Dan Mick <dan.mick@redhat.com>

suite/tests: add distroversion

Signed-off-by: Dan Mick <dan.mick@redhat.com>

suite: Add distroversion

Allow -D/--distroversion to select version of distro to schedule.
Arg can be either numeric version or codename.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Add OS class method 'version_codename' to infer one from the other

Signed-off-by: Dan Mick <dan.mick@redhat.com>

packages.yaml: reflect python-ceph package split

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #915 from dachary/wip-14799-valgrind

valgrind: add suppression for tcmalloc in libboost_thread-mt.so.1.53.0

valgrind: add suppression for tcmalloc in libboost_thread-mt.so.1.53.0

Use regular expression to catch:

tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long, unsigned long)

Fixes: http://tracker.ceph.com/issues/14799
Signed-off-by: Loic Dachary <loic@dachary.org>

Move some stream management into RemoteProcess

For better organization and less copypasta.

Signed-off-by: Zack Cerza <zack@redhat.com>

Log stderr and stdout as long as they aren't pipes

Signed-off-by: Zack Cerza <zack@redhat.com>

Rewrite orchestra.run tests using mock

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #912 from smithfarm/wip-drop-ceph-devel

packages.yaml: drop ceph-devel from rpm section

packages.yaml: drop ceph-devel from rpm section

Upgrade tests on https://github.com/ceph/ceph/9744 (which drops the ceph-devel
package after a long period of deprecation) are currently failing
because teuthology tries to install ceph-devel, even though it is not needed
for any tests.

See also https://github.com/ceph/ceph-qa-suite/pull/1068

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #905 from lande1234/wip-lan-testing

fix wrong words

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #910 from ceph/wip-worker-startup

Stop workers on demand

Dan Mick <dmick@redhat.com>

Merge pull request #911 from ceph/wip-queue-status

teuthology-queue: add --status

teuthology-queue: add --status

Prints queue status, e.g.:

    $ teuthology-queue -m smithi --status
    {'paused': False, 'count': 2900, 'name': 'smithi'}

Signed-off-by: Zack Cerza <zack@redhat.com>

Stop workers on demand

Generalize need_restart()

So that we can also use it to tell the workers to stop

Signed-off-by: Zack Cerza <zack@redhat.com>

Revert "task/packages: install libcephfs1-dev"

This reverts commit 48be9e89df559dce13731fcdf59153181ffd2646.

Fixes: http://tracker.ceph.com/issues/16703
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #909 from ceph/wip-worker-startup

Fix slow worker startup

Reviewed-by: Dan Mick <dmick@redhat.com>

Fix slow worker startup

We can selectively skip fetching and/or bootstrapping if either have
been completed successfully in the last 60s

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #908 from ceph/wip-sched-fail-ceph-ver

Fix regression in schedule failure emails

Reviewed-by: Dan Mick <dmick@redhat.com>

Fix regression in schedule failure emails

Emails weren't getting sent when builds were broken

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #904 from dmick/wip-version

teuthology: log version number

Merge pull request #907 from ceph/wip-16640

task/packages: install libcephfs1-dev

task/packages: install libcephfs1-dev

The way we currently build the cephfs java
bindings depends on the unversioned .so file
which is in the -dev package, not the
main one.

Fixes: http://tracker.ceph.com/issues/16640
Signed-off-by: John Spray <john.spray@redhat.com>

teuthology: log version number

Round 2, this time with testing

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #902 from dmick/wip-schedule-suite-test

Add tests for schedule_suite(); fix some schedule_fail arguments

Merge pull request #906 from dmick/wip-email

Resume paying attention to --email

Resume paying attention to --email

Change in 61350db caused email arg to be lost.  Fix, add test

Signed-off-by: Dan Mick <dan.mick@redhat.com>

fix wrong words

Signed-off-by: lande1234 <lan.de3@zte.com.cn>

Merge pull request #903 from ceph/revert-901-wip-version

Revert "Add teuthology git version query/logging"

Revert "Add teuthology git version query/logging"

Add tests for suite.run.schedule_suite

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #901 from dmick/wip-version

Add teuthology git version query/logging

Add teuthology git version query/logging

Tested exception handling by renaming /usr/bin/git

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Fix arguments to schedule_fail() again

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #890 from ceph/wip-fixlistindex-wusui

Fix resolve_role_list in daemon.py

Merge pull request #897 from ceph/wip-provision-subpackage

Make teuthology.provision a subpackage

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #864 from jcsp/wip-nocolor

task/ansible: disable color output

Merge pull request #898 from ceph/wip-gbp-dmick

Add 'tag' and 'ref' to GitbuilderProject's init, warn if more than one set, and tests

Fix resolve_role_list in daemon.py

resolve_role_list did not handle cases where roles specified contained wild-cards or
were not specified.  ceph.role.a would end up being resolved as ceph.rolea.

Fixes: tracker issue 16053
Signed-off-by: Warren Usui <wusui@redhat.com>

Merge pull request #894 from ceph/wip-newest

Complete --newest feature, add some cleanups

Use 'conf' instead of 'fn' for the processed arg YamlConfig

Signed-off-by: Dan Mick <dan.mick@redhat.com>

suite.main: use processed args consistently

Signed-off-by: Dan Mick <dan.mick@redhat.com>

suite.run.collect_jobs(): deepcopy base_config

deep_merge, despite the name, doesn't do fully deep merging;
it recurses through the whole object, but does not deepcopy
component objects at the lowest level.  This meant that
self.base_config could be corrupted by the second deep_merge
(if base_config contained keys that were augmented by
parsed_yaml, which was often the case, for things like overrides).

Signed-off-by: Dan Mick <dan.mick@redhat.com>

suite: move config tmpfile management to Run

base_config changes over the course of the run, and
so cannot be written to a tmpfile until it's completely
finalized.  Move the create/write/rm out of main() and into
schedule_suite().

Signed-off-by: Dan Mick <dan.mick@redhat.com>

GitbuilderProject tests: also test for the right log msg

Signed-off-by: Dan Mick <dan.mick@redhat.com>

GitbuilderProject: if multiple ref-selectors present, warn

...about which one is actually being used

Signed-off-by: Dan Mick <dan.mick@redhat.com>

GitbuilderProject: allow setting tag or ref in config dict

Precedence: ref, tag, branch, sha1.  (this is debatable.)

Also change log.debug() default-to-master to log.warning()

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #899 from ceph/wip-ansible-2

Move to ansible 2

Reviewed-by: Dan Mick <dmick@redhat.com>

Use ansible >= 2.0

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #895 from ceph/wip-no-pyc

Stop generating .pyc files

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #896 from ceph/wip-test-reqs

Add test requirements

Reviewed-by: Dan Mick <dmick@redhat.com>

Add test requirements

Signed-off-by: Zack Cerza <zack@redhat.com>

Remove leftover .pyc files

Signed-off-by: Zack Cerza <zack@redhat.com>

Don't write pyc files

The .pyc files that python generates can be problematic if we move
modules around; teuthology processes end up picking up the old .pyc
files instead of finding the new .py files

Signed-off-by: Zack Cerza <zack@redhat.com>

Move openstack-specific code to its own module

Signed-off-by: Zack Cerza <zack@redhat.com>

Move downburst-specific code to its own module

Signed-off-by: Zack Cerza <zack@redhat.com>

Make teuthology.provision a subpackage

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #893 from ceph/wip-gbp

GitbuilderProject: respect branch in config

Reviewed-by: Dan Mick <dmick@redhat.com>

GitbuilderProject: respect branch in config

Signed-off-by: Zack Cerza <zack@redhat.com>

GitbuilderProject test: respect branch in config

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #892 from ceph/wip-10481-new

Install -release RPMs directly

Reviewed-by: Dan Mick <dmick@redhat.com>

Install -release RPMs directly

This avoids having to dance around with wget and rm -f.
Fixes: 10481
http://tracker.ceph.com/issues/10481

Signed-off-by: Zack Cerza <zack@redhat.com>

valgrind: make threadlocalptr whitelist more generic

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #888 from dmick/master

suite.py: must pass at least once through the backtrack loop

suite.py: must pass at least once through the backtrack loop

Changing the default of newest to '0' meant that the combined
loop which handles both backtrack and non-backtrack conditions
was never executing, leading to undefined variables.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #887 from dmick/master

suite.py: defaulting newest to 10 means it's always enabled

suite.py: defaulting newest to 10 means it's always enabled

Even if --newest isn't supplied, docopts will create it and
set it to the default.  Change the default to 0.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #886 from dmick/master

suite: --newest fixes: failure to rebuild base_config on sha1 backtrack, spurious log msg

suite/run.py: rebuild base config on sha1 backtrack

Signed-off-by: Dan Mick <dan.mick@redhat.com>