Merge pull request #825 from ceph/wip-kill-doc

Mention that -kill must be run on the master node

Mention that -kill must be run on the master node

There isn't currently a remote kill-a-process feature.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #824 from ceph/wip-ver-update

rh version update

Install ceph-fuse for 2.0 by default for tests to work

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Package name split, ceph is now ceph-mon, ceph-osd

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add ceph-selinux to pkg list for 1.3.2 builds

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #797 from dachary/wip-openstack-override-clone

openstack: allow override of openstack:clone:

Merge pull request #822 from dachary/wip-14997-openstack-stale-keypairs

openstack: protect against stale keypairs

openstack: protect against stale keypairs

http://tracker.ceph.com/issues/14997 Fixes: #14997

Signed-off-by: Loic Dachary <loic@dachary.org>

Update rh_version supported in downstream

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #821 from ceph/wip-yum-priorities

install: Fix non-idempotent check_obsoletes tweak

Reviewed-by: Dan Mick<dmick@redhat.com>

Merge pull request #813 from smithfarm/wip-fix-siteconfig

docs/siteconfig.rst: adapt comments to reality

docs/siteconfig.rst: consistency

Change "count: 0" to "count: 3" for consistency with the comment.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

install: Fix non-idempotent check_obsoletes tweak

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #820 from ceph/wip-nrpe

selinux: Whitelist nrpe

selinux: Whitelist nagios' nrpe

Signed-off-by: Zack Cerza <zack@redhat.com>

selinux: Fix linter complaints

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph.conf: do not warn on straw calc version = 0

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #818 from dachary/wip-14799-valgrind

valgrind: add suppression for tcmalloc in libboost_thread-mt.so.1.53.0

valgrind: add suppression for tcmalloc in libboost_thread-mt.so.1.53.0

http://tracker.ceph.com/issues/14799 Fixes: #14799

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #817 from ceph/wip-valgrind-supp

valgrind: ignore boost thread-local warning

Reviewed-by: Dan Mick <dmick@redhat.com>

valgrind: ignore boost thread-local warning

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #814 from ceph/wip-15014

install task: Make package lists configurable

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #816 from ceph/wip-big-vps

lock: Pass canonicalized hostname to create_if_vm

Reviewed-by: Dan Mick <dmick@redhat.com>

lock: Pass canonicalized hostname to create_if_vm

Signed-off-by: Zack Cerza <zack@redhat.com>

install: Remove old "lib deps" cruft

Signed-off-by: Zack Cerza <zack@redhat.com>

nuke: Use install_task.get_package_list()

(Properly named task.install.get_package_list())

Signed-off-by: Zack Cerza <zack@redhat.com>

install: Add get_package_list()

Move package lists out of the module and into packages.yaml. Enable
overriding package lists via the job config.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #815 from ceph/wip-14855

misc.get_wwn_id_map: not finding wwns is not an error

misc.get_wwn_id_map: not finding wwns is not an error

Fixes: #14855
Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #790 from ceph/wip-fixdenials

Add whitelist option and fix for cephtest denials

Add whitelist option and fix for cephtest denials

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #812 from ceph/wip-gevent

Unpin gevent and avoid patching subprocess

Reviewed-by: Dan Mick <dmick@redhat.com>

Unpin gevent and avoid patching subprocess

Recently, gevent updated to 1.1.0, breaking a few things like misc.sh().
PR #810 pinned gevent to 1.0.2, but I wanted to properly investigate the
issue to not be stuck on an old version. Some git bisects later, I
discovered that gevent began patching the subprocess module by default
in:

https://github.com/gevent/gevent/commit/fc73e9afa34d811ede5d62531025d2e3876b8c01

Unpinning gevent and also telling it not to patch subprocess fixes the
issue.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #811 from ceph/wip-prime

mon osd prime pg temp = true

Reviewed-by: Samuel Just <sjust@redhat.com>

ceph.conf: mon osd prime pg temp = true

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #809 from smithfarm/wip_doc_fake_fs

doc: update documentation of make_fake_fstools()

Merge pull request #808 from smithfarm/wip-doc-teuos

README.rst: mention teuthology-openstack in Provided Utilities

Merge pull request #810 from dachary/wip-14990-gevent

pin gevent 1.0.2

pin gevent 1.0.2

http://tracker.ceph.com/issues/14990 Fixes: #14990

Signed-off-by: Loic Dachary <loic@dachary.org>

doc: update documentation of make_fake_fstools()

When the documentation of make_fake_fstools() was written, the function
evidently only returned two values. Now it returns four values. Additional
return values might be added later.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

README.rst: mention teuthology-openstack in Provided Utilities

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #807 from ceph/wip-repo-set-remote

repo_utils: Call git set remote-url

repo_utils: Call git set remote-url

This will cause git repos on-disk to be updated to point at the correct
remote based on config.ceph_git_base_url

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #804 from dachary/wip-openstack-vault

openstack: create default ~/.vault_pass.txt

Merge pull request #806 from dachary/wip-openstack-sitepackage

openstack: do not use sitepackages for openstack-integration tests

Merge pull request #805 from dachary/wip-openstack-test-selection

openstack: test selection for openstack-integration

openstack: do not use sitepackages for openstack-integration tests

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: test selection for openstack-integration

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: create default ~/.vault_pass.txt

It is now required by teuthology.

http://tracker.ceph.com/issues/14914 Fixes: #14914

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: allow override of openstack:clone:

When running from a wip teuthology branch, it is inconvenient to tweak
~/.teuthology.yaml. Add --teuthology-git-url and --teuthology-branch to
take precendence and override the defaults.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #794 from dachary/wip-openstack-wait

openstack: add --wait to block until the suite finishes

Merge pull request #802 from dachary/wip-openstack-neutron

openstack: openstacklient >= 2.1.0 does not depend on neutronclient

Merge pull request #801 from dachary/wip-openstack-centos-7-2

openstack: package CentOS 7.2 data file

Merge pull request #800 from dachary/wip-openstack-cleanup

openstack: only destroy target/ceph- VMs on restart

Merge pull request #799 from dachary/wip-openstack-timeout

openstack: default to 9h job timeout

Merge pull request #796 from dachary/wip-archive-upload

suite/openstack: add --archive-upload-url

Merge pull request #795 from dachary/wip-openstack-suite-exit-code

suite/openstack: implement the exit code

openstack: openstacklient >= 2.1.0 does not depend on neutronclient

Explicitly require it python-neutronclient.

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: package CentOS 7.2 data file

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: only destroy target/ceph- VMs on restart

Obsolete resources will be taken care of by the teuthology-nuke
--stale-openstack cron job. We just want to avoid running into obsolete
targets.

The ceph-* targets which are created by the buildpackages task must also
be disposed of.

The timestamps dir is wiped out, except for packages-repository, if
it exists.

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: default to 9h job timeout

Because a few job take more than 6h. Ideally this could be overriden on
a case by case basis.

Signed-off-by: Loic Dachary <loic@dachary.org>

suite/openstack: add --archive-upload-url

When uploading job results with --upload, it may be non-trivial to
figure out the public facing URL where the job will be visible. When
used via teuthology-openstack, it defaults to

    http://teuthology-logs.public.ceph.com

so that using --wait --upload --teardown displays the URLs of the
uploaded job results that are accessible even after the cluster is
gone.

Signed-off-by: Loic Dachary <loic@dachary.org>

suite/openstack: implement the exit code

Make it so the exit code of the command reflects the success or failure
of the suite. This is useful when running with --wait.

Signed-off-by: Loic Dachary <loic@dachary.org>

openstack: add --wait to block until the suite finishes

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #791 from ceph/wip-ceph-ansible

ceph_ansible: provide devices, monitor_interface, public_network

Reviewed-by: Dan Mick <dmick@redhat.com>

Set network-related hostvars if necessary

Specifically, monitor_interface and public_network

Signed-off-by: Zack Cerza <zack@redhat.com>

Add Remote.{ip_address, interface, cidr}

Signed-off-by: Zack Cerza <zack@redhat.com>

Use misc.get_scratch_devices() when necessary

That is, if osd_auto_discovery is disabled.

Signed-off-by: Zack Cerza <zack@redhat.com>

Refactor hosts file generation

This is in preparation to enable us to write host-specific vars to the
file.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #793 from ceph/wip-suite-error

Add a warning when gitbuilder 404s

Reviewed-by: Dan Mick <dmick@redhat.com>

Add a warning when gitbuilder 404s

When determining the ceph hash, we usually ask gitbuilder. When that
fails, the error is misleading but it would take a bit of refactoring to
truly fix it. Let's at least put a useful warning before the confusing error
message.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #792 from ceph/wip-os-version-str

get_distro_version: Always return a string

Reviewed-by: Dan Mick <dmick@redhat.com>

get_distro_version: Always return a string

Signed-off-by: Zack Cerza <zack@redhat.com>

Unpin requests version

setuptools 20.2 won't parse this version requirement. 2.9.1 appears to
work just fine anyway.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #789 from ceph/wip-ceph-ansible

Add ceph_ansible task

Reviewed-by: Dan Mick <mick@redhat.com>

Add ceph_ansible task

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Patch more aggressively in unit tests

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Don't modify plays

We were applying hosts=all to each play when using a generated playbook.
I can't remember why I thought that was a good idea, and it's tripping
up CephAnsible.

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Run ansible-playbook in repo dir

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Place generated playbooks in repo dir

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: write hosts file in _write_hosts_file()

This is to make implementing the CephAnsible task cleaner.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #788 from ceph/wip-worker-debug

worker: When running a job, log the full command

Reviewed-by: Dan Mick <dmick@redhat.com>

worker: When running a job, log the full command

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #785 from ceph/wip-timing-fix

Fix bug when running archiveless jobs

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #784 from ceph/wip-no-debug

task/install: only install debug packages if debuginfo: true

Fix bug when running archiveless jobs

We were trying to tell the timer to write to a nonexistent file

Signed-off-by: Zack Cerza <zack@redhat.com>

task/install: only install debug packages if debuginfo: true

Note that our kludge to handle extras does not include debuginfo packages,
which means we won't get those for ceph-deploy runs for now.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #782 from ceph/wip-timer

Task timing

Reviewed-by: Dan Mick <dmick@redhat.com>

Set config.archive_path if it's not set already

This will only affect non-scheduled jobs

Signed-off-by: Zack Cerza <zack@redhat.com>

Collect timing data for task execution

Just before we execute a task's enter or exit method, create a new timer
mark. If the archive is enabled, each timer.mark() call writes (or
re-writes) all the timing data to timing.yaml inside the archive.

Signed-off-by: Zack Cerza <zack@redhat.com>

Add teuthology.timer

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #773 from dachary/wip-subset-fix

fix a --subset bug that misses facets in some cases

matrix: reimpliment Sum

See the docstring for details on the new implementation.  The
old one didn't really satisfy the minscanlen properties at the
tail of the sequence.

Signed-off-by: Samuel Just <sjust@redhat.com>

matrix: fix Product index offset and minscanlen value

Incrementing the index on each iteration is problematic because you
may not see a particular element for 2*minscanlen indices.
Decrementing it ensures we'll see any particular submat element every
max(minscanlen) + 1 indicies.

Signed-off-by: Samuel Just <sjust@redhat.com>

suite: add randomized tests verifying subset coverage

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

matrix: add __str__

Signed-off-by: Samuel Just <sjust@redhat.com>

suite: ensure that convolutions are cycled out to satisfy mincyclicity

Signed-off-by: Samuel Just <sjust@redhat.com>

suite: fix rounding bug in _build_matrix

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #781 from ceph/wip-tidy

task/internal: list leftover files during tidy stage

Reviewed-by: Dan Mick <dmick@redhat.com>