enable lsof when unmounting nfs clients

Because the lsof interface is strange, use the verbose flag so we get
some output even if there is no use, and ignore errors because it will
return 1 if there's no filesystem use.

Signed-off-by: Greg Farnum <greg@inktank.com>

Improve os-release parsing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite teuthology-updatekeys

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Skip hostname test

Due to the workaround put in place recently

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Re-implement Distribution as OS

It now uses /etc/os-release instead of lsb_release, like the cool kids
are doing.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #328 from ceph/wip-8700-wusui

Use docopts to parse daemon-helper

Merge pull request #333 from ceph/wip-9495-wusui

Run chef before operations that need it.

Merge pull request #331 from ceph/wip-transport-none

orchestra: fix ConnectionLostError path in wait()

Add a workaround for Remote.hostname

Since 'hostname --fqdn' isn't working reliably

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Run chef before operations that need it.

Chef, if it is needed, should be run before kernel tasks
and before all other user defined tasks.  It should not be
run by default.

Fixes: 9495
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix daemon-helper argument parsing.

Fixes: 8700
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #332 from ceph/wip-yaml-path

teuthology-suite: fix handling of extra YAML files

teuthology-suite: fix handling of extra YAML files

Previously these had to be absolute paths, or relative
paths had to be present in the pwd and in ceph-qa-suite.

With this change you can now pass a relative path
like 'debug/mds_client.yaml' to reference a debug
fragment from ceph-qa-suite.

Signed-off-by: John Spray <john.spray@redhat.com>

orchestra: fix ConnectionLostError path in wait()

Sometimes it would generate AttributeError instead
when it tried to call is_active() on None.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #326 from ceph/wip-9318-wusui

Avoid running chef automatically.

Merge pull request #297 from ceph/wip-8712-wusui

Check os-type and os-version for vps

Make push_inventory() a regular function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Avoid running chef automatically

teuthology --lock formerly automatically ran chef on startup of
virtual machines. This covered up a bug where ceph-deploy would
not install properly. These changes remove the automatic running
of chef, and fixed some related issues when setting up keys on
virtual machines.

Fixes 9318:
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Use teuthology.task.internal.push_inventory

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add push_inventory() subtask

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add lock.update_inventory()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.inventory_info

A property that returns a dict of attributes in the format that the
paddles lock server likes.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.host_key

Turns out we can easily grab the ssh key from paramiko once a connection
is established.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Set Remote.hostname by asking the host itself

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.arch

Returns the result of 'uname -p'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add caching lsb_release parser

In far too many places do we remotely exec lsb_release in some form,
then parse the output. In shifting to a more stable interface between
teuthology and its tasks, this seemed important to me.

Remote objects now have a 'distro' property that, when first accessed,
calls 'lsb_release -a' remotely and parses the results.

I've included tests and documentation.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Check os-type and os-version for vps

Fixes: 8712
Signed-off-by: Warren Usui warren.usui@inktank.com

Merge pull request #327 from dachary/wip-exec-documentation

documentation: updates to the exec tasks

documentation: updates to the exec tasks

* Quoting is important
* Behavior on error

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

In Remote.run(), attempt to reconnect if necessary

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move more exceptions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

valgrind: suppress tcmalloc crap for centos 6.5

Fixes: #9336
Signed-off-by: Sage Weil <sage@redhat.com>

Remove unnecessary filter in do_update_keys()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't explode if get_status fails

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #325 from ceph/wip-valgrind

valgrind: use --time-stamp=yes

valgrind: adjust suppressions for wheezy

Signed-off-by: Sage Weil <sage@redhat.com>

valgrind: use --time-stamp=yes

This doesn't help on the precise version, but maybe it will on
future versions... because if so, it would be really helpful
it matching valgrind errors to daemon logs!

In any case, though, I verified it is at worse harmless.

Signed-off-by: Sage Weil <sage@redhat.com>

Update downburst docs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #324 from ceph/wip-hadoop-nuke

nuke: kill hadoop processes

nuke: kill hadoop processes

Signed-off-by: John Spray <john.spray@redhat.com>

Drop --allow-hosts None

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use setuptools, not distribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #323 from ceph/pypi

Prepare for PyPI

Reviewed-by: Sage Weil <sage@redhat.com>

Add install_requires and tests_require

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add MANIFEST.in

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add LICENSE

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #308 from ceph/wip-6146-wusui

Add ability to update with ceph-deploy

Merge pull request #314 from ceph/wip-8923

Speed up teuthology-kill.

Add more info in comments.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #319 from ceph/wip-8231

remote: Add ability to push a file to a remote machine

Merge pull request #299 from ceph/wip-7810

Wip 7810

orchestra: more docstring

Signed-off-by: John Spray <john.spray@redhat.com>

orchestra: fine tune docstrings.

Signed-off-by: John Spray <john.spray@redhat.com>

Don't explode on ConnectionError in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

remote: Add ability to push a file to a remote machine

Add put_file() and _sftp_put_file() without sudo support

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #317 from dachary/wip-nuke-on-error

convert nuke-on-error to boolean

Merge pull request #321 from ceph/wip-install

install libcephfs-java

Only pass lists to ssh_keyscan()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

More hostname normalization

This time, for ssh_keyscan()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

For clones, look for error regardless of exit code

This fixes a bug on older git versions, where it would happily cloen a
repo and give you the wrong branch if the one you requested did not
exist.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Normalize input hostnames in various functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task/install: install libcephfs-java (deb)

Accidentally removed in 0c75c6b1f7654b5a9b45d73c6b6aa47a523e1841

Signed-off-by: Sage Weil <sage@redhat.com>

task/install: move package list to top of file

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #320 from ceph/wip-sort-brief

lock.py: sort --brief output by status['name']

lock.py: sort --brief output by status['name']

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Fix a linter error

Wasn't much of a bug, but still.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix module reference

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Hostname handling for updatekeys

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Exit with code already returned by lock.main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Less dangerous hostname mangling

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix --unlock

Note to self: clean up user/ctx.owner confusion

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Return old-style hostnames from lock_many()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't "import misc as teuthology"

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #318 from ceph/wip-defaults

change defaults for mira,plana,burnupi

More hostname handling fixes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Canonicalize hostnames"

This reverts commit 29e8deefaeef21d1e7f6c40d6fca2605324dec69.

Remove the 'user@' prefix before checking status

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow passing user=None to remove the 'user@'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Canonicalize hostnames in yet another place

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

suite: scheule in mira,burnupi,plana by deafult

(was plana)

Signed-off-by: Sage Weil <sage@redhat.com>

teuthology-queue: list multi queue by default

Signed-off-by: Sage Weil <sage@redhat.com>

Canonicalize hostnames

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new vm_host attribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't dump all vm_host details when listing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Reflect updated field names

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update provision module to use new lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.shortname instead of regex

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unlock_many() and use it when possible

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update internal task to use new lockserver

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make far less requests when simply locking

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Optimize some list_locks() calls

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Properly query for multiple machine types

...in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use consistent URI for lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove old lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Change default lock server to paddles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use /nodes/ as the lock server endpoint

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update to use new ssh-keyscan API

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite ssh public key updating

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>