Make push_inventory() a regular function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use teuthology.task.internal.push_inventory

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add push_inventory() subtask

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add lock.update_inventory()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.inventory_info

A property that returns a dict of attributes in the format that the
paddles lock server likes.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.host_key

Turns out we can easily grab the ssh key from paramiko once a connection
is established.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Set Remote.hostname by asking the host itself

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.arch

Returns the result of 'uname -p'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add caching lsb_release parser

In far too many places do we remotely exec lsb_release in some form,
then parse the output. In shifting to a more stable interface between
teuthology and its tasks, this seemed important to me.

Remote objects now have a 'distro' property that, when first accessed,
calls 'lsb_release -a' remotely and parses the results.

I've included tests and documentation.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #327 from dachary/wip-exec-documentation

documentation: updates to the exec tasks

documentation: updates to the exec tasks

* Quoting is important
* Behavior on error

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

In Remote.run(), attempt to reconnect if necessary

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move more exceptions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

valgrind: suppress tcmalloc crap for centos 6.5

Fixes: #9336
Signed-off-by: Sage Weil <sage@redhat.com>

Remove unnecessary filter in do_update_keys()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't explode if get_status fails

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #325 from ceph/wip-valgrind

valgrind: use --time-stamp=yes

valgrind: adjust suppressions for wheezy

Signed-off-by: Sage Weil <sage@redhat.com>

valgrind: use --time-stamp=yes

This doesn't help on the precise version, but maybe it will on
future versions... because if so, it would be really helpful
it matching valgrind errors to daemon logs!

In any case, though, I verified it is at worse harmless.

Signed-off-by: Sage Weil <sage@redhat.com>

Update downburst docs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #324 from ceph/wip-hadoop-nuke

nuke: kill hadoop processes

nuke: kill hadoop processes

Signed-off-by: John Spray <john.spray@redhat.com>

Drop --allow-hosts None

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use setuptools, not distribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #323 from ceph/pypi

Prepare for PyPI

Reviewed-by: Sage Weil <sage@redhat.com>

Add install_requires and tests_require

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add MANIFEST.in

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add LICENSE

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #308 from ceph/wip-6146-wusui

Add ability to update with ceph-deploy

Merge pull request #314 from ceph/wip-8923

Speed up teuthology-kill.

Add more info in comments.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #319 from ceph/wip-8231

remote: Add ability to push a file to a remote machine

Merge pull request #299 from ceph/wip-7810

Wip 7810

orchestra: more docstring

Signed-off-by: John Spray <john.spray@redhat.com>

orchestra: fine tune docstrings.

Signed-off-by: John Spray <john.spray@redhat.com>

Don't explode on ConnectionError in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

remote: Add ability to push a file to a remote machine

Add put_file() and _sftp_put_file() without sudo support

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #317 from dachary/wip-nuke-on-error

convert nuke-on-error to boolean

Merge pull request #321 from ceph/wip-install

install libcephfs-java

Only pass lists to ssh_keyscan()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

More hostname normalization

This time, for ssh_keyscan()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

For clones, look for error regardless of exit code

This fixes a bug on older git versions, where it would happily cloen a
repo and give you the wrong branch if the one you requested did not
exist.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Normalize input hostnames in various functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task/install: install libcephfs-java (deb)

Accidentally removed in 0c75c6b1f7654b5a9b45d73c6b6aa47a523e1841

Signed-off-by: Sage Weil <sage@redhat.com>

task/install: move package list to top of file

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #320 from ceph/wip-sort-brief

lock.py: sort --brief output by status['name']

lock.py: sort --brief output by status['name']

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Fix a linter error

Wasn't much of a bug, but still.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix module reference

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Hostname handling for updatekeys

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Exit with code already returned by lock.main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Less dangerous hostname mangling

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix --unlock

Note to self: clean up user/ctx.owner confusion

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Return old-style hostnames from lock_many()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't "import misc as teuthology"

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #318 from ceph/wip-defaults

change defaults for mira,plana,burnupi

More hostname handling fixes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Canonicalize hostnames"

This reverts commit 29e8deefaeef21d1e7f6c40d6fca2605324dec69.

Remove the 'user@' prefix before checking status

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow passing user=None to remove the 'user@'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Canonicalize hostnames in yet another place

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

suite: scheule in mira,burnupi,plana by deafult

(was plana)

Signed-off-by: Sage Weil <sage@redhat.com>

teuthology-queue: list multi queue by default

Signed-off-by: Sage Weil <sage@redhat.com>

Canonicalize hostnames

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new vm_host attribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't dump all vm_host details when listing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Reflect updated field names

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update provision module to use new lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.shortname instead of regex

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unlock_many() and use it when possible

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update internal task to use new lockserver

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make far less requests when simply locking

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Optimize some list_locks() calls

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Properly query for multiple machine types

...in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use consistent URI for lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove old lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Change default lock server to paddles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use /nodes/ as the lock server endpoint

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update to use new ssh-keyscan API

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite ssh public key updating

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new ssh_keyscan() function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

No need to be passing ctx to update_lock()

... or update_keys() or scan_for_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add new ssh_keyscan() method

This is just a cleaner implementation that will replace the existing
one(s).

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Re-order functions in module

This is for readability.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

keyscan_check(): don't require ctx to be passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update lock_many(), update_lock(), and main()

Also remove lockstatus.send_request()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix PEP-8 issues

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactoring

Create a new teuthology.provision module and move virtual machine
functions there. Move hostname manipulation functions to
teuthology.misc.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update do_summary()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make (de)canonicalize_hostname() work without user

Add tests proving they still work *with* usernames as well.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unit tests for (de)canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update/rewrite several functions to use paddles

Functions updated: lock.lock_one(), lock.unlock_one(),
lock.destroy_if_vm(), lockstatus.get_status()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

convert nuke-on-error to boolean

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

ceph.conf: mon reweight min pgs per osd = 4

So teh cephtool reweight-by-pg test will pass with mini clusters.

Signed-off-by: Sage Weil <sage@redhat.com>

daemon: add DaemonGroup.resolve_role_list

Used for tasks that want to take a flexible list
of roles from their user config, including a
null (everything) default and * wildcards.

Signed-off-by: John Spray <john.spray@redhat.com>

nuke: clear test firewall rules

Signed-off-by: John Spray <john.spray@redhat.com>

Add --suite-path

Slightly less ugly than setting PYTHONPATH

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>