Update downburst docs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #324 from ceph/wip-hadoop-nuke

nuke: kill hadoop processes

nuke: kill hadoop processes

Signed-off-by: John Spray <john.spray@redhat.com>

Drop --allow-hosts None

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use setuptools, not distribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #323 from ceph/pypi

Prepare for PyPI

Reviewed-by: Sage Weil <sage@redhat.com>

Add install_requires and tests_require

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add MANIFEST.in

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add LICENSE

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #308 from ceph/wip-6146-wusui

Add ability to update with ceph-deploy

Merge pull request #314 from ceph/wip-8923

Speed up teuthology-kill.

Add more info in comments.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #319 from ceph/wip-8231

remote: Add ability to push a file to a remote machine

Merge pull request #299 from ceph/wip-7810

Wip 7810

orchestra: more docstring

Signed-off-by: John Spray <john.spray@redhat.com>

orchestra: fine tune docstrings.

Signed-off-by: John Spray <john.spray@redhat.com>

Don't explode on ConnectionError in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

remote: Add ability to push a file to a remote machine

Add put_file() and _sftp_put_file() without sudo support

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #317 from dachary/wip-nuke-on-error

convert nuke-on-error to boolean

Merge pull request #321 from ceph/wip-install

install libcephfs-java

Only pass lists to ssh_keyscan()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

More hostname normalization

This time, for ssh_keyscan()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

For clones, look for error regardless of exit code

This fixes a bug on older git versions, where it would happily cloen a
repo and give you the wrong branch if the one you requested did not
exist.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Normalize input hostnames in various functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task/install: install libcephfs-java (deb)

Accidentally removed in 0c75c6b1f7654b5a9b45d73c6b6aa47a523e1841

Signed-off-by: Sage Weil <sage@redhat.com>

task/install: move package list to top of file

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #320 from ceph/wip-sort-brief

lock.py: sort --brief output by status['name']

lock.py: sort --brief output by status['name']

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Fix a linter error

Wasn't much of a bug, but still.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix module reference

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Hostname handling for updatekeys

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Exit with code already returned by lock.main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Less dangerous hostname mangling

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix --unlock

Note to self: clean up user/ctx.owner confusion

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Return old-style hostnames from lock_many()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't "import misc as teuthology"

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #318 from ceph/wip-defaults

change defaults for mira,plana,burnupi

More hostname handling fixes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Canonicalize hostnames"

This reverts commit 29e8deefaeef21d1e7f6c40d6fca2605324dec69.

Remove the 'user@' prefix before checking status

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow passing user=None to remove the 'user@'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Canonicalize hostnames in yet another place

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

suite: scheule in mira,burnupi,plana by deafult

(was plana)

Signed-off-by: Sage Weil <sage@redhat.com>

teuthology-queue: list multi queue by default

Signed-off-by: Sage Weil <sage@redhat.com>

Canonicalize hostnames

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new vm_host attribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't dump all vm_host details when listing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Reflect updated field names

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update provision module to use new lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.shortname instead of regex

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unlock_many() and use it when possible

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update internal task to use new lockserver

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make far less requests when simply locking

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Optimize some list_locks() calls

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Properly query for multiple machine types

...in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use consistent URI for lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove old lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Change default lock server to paddles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use /nodes/ as the lock server endpoint

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update to use new ssh-keyscan API

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite ssh public key updating

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new ssh_keyscan() function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

No need to be passing ctx to update_lock()

... or update_keys() or scan_for_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add new ssh_keyscan() method

This is just a cleaner implementation that will replace the existing
one(s).

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Re-order functions in module

This is for readability.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

keyscan_check(): don't require ctx to be passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update lock_many(), update_lock(), and main()

Also remove lockstatus.send_request()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix PEP-8 issues

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactoring

Create a new teuthology.provision module and move virtual machine
functions there. Move hostname manipulation functions to
teuthology.misc.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update do_summary()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make (de)canonicalize_hostname() work without user

Add tests proving they still work *with* usernames as well.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unit tests for (de)canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update/rewrite several functions to use paddles

Functions updated: lock.lock_one(), lock.unlock_one(),
lock.destroy_if_vm(), lockstatus.get_status()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

convert nuke-on-error to boolean

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

ceph.conf: mon reweight min pgs per osd = 4

So teh cephtool reweight-by-pg test will pass with mini clusters.

Signed-off-by: Sage Weil <sage@redhat.com>

daemon: add DaemonGroup.resolve_role_list

Used for tasks that want to take a flexible list
of roles from their user config, including a
null (everything) default and * wildcards.

Signed-off-by: John Spray <john.spray@redhat.com>

nuke: clear test firewall rules

Signed-off-by: John Spray <john.spray@redhat.com>

Add --suite-path

Slightly less ugly than setting PYTHONPATH

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #316 from ceph/wip-9183-wusui

Fix return from schedule_suite.

Fix return from schedule_suite.

Schedule_suite returns the number of jobs scheduled (accounting for
filters).  Information displayed also indicates how many jobs were
run and how many were filtered out.

Fixes: 9183
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #309 from ceph/wip-8954-wusui

Add filter and filter-out options to suites.

valgrind: make trusty tcmalloc supporession more general

To capture this:

  <kind>SyscallParam</kind>
  <what>Syscall param msync(start) points to unaddressable byte(s)</what>
  <stack>
    <frame>
      <ip>0x5081B3D</ip>
      <obj>/lib/x86_64-linux-gnu/libpthread-2.19.so</obj>
      <dir>/build/buildd/eglibc-2.19/nptl/../sysdeps/unix</dir>
      <file>syscall-template.S</file>
      <line>81</line>
    </frame>
    <frame>
      <ip>0x76BA123</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BCEF6</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BE151</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BE4E8</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BAA30</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
      <fn>_ULx86_64_step</fn>
    </frame>
    <frame>
      <ip>0x5C42052</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>GetStackTrace(void**, int, int)</fn>
    </frame>
    <frame>
      <ip>0x5C349F4</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>tcmalloc::PageHeap::GrowHeap(unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x5C34D02</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>tcmalloc::PageHeap::New(unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x5C25D0C</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
    </frame>
    <frame>
      <ip>0x5C46378</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>operator new[](unsigned long)</fn>
    </frame>

Signed-off-by: Sage Weil <sage@redhat.com>

Use suite_path, if set, for one-off jobs as well

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add filter and filter-off suite options.

Filter only runs suite jobs that contain the text passed as part of
their description or in the names of the composite set of yaml files.
Filter-off only runs suite jobs that do not contain the text passed
as either their description or in the names of the composite set of
yaml files.
Fixes: 8954
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #313 from ceph/wip-rhel

install: use rhel builder for rhel

Merge pull request #315 from ceph/wip-8877

Skip nuke and just destroy VPS if nuke is --unlock

Skip nuke and just destroy VPS if nuke is --unlock

If the machine is being unlocked after than we definitely want to
just destroy it. This way nukes will always suceed with --unlock
even if the guest is down or in a borked state.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Speed up teuthology-kill.

Read less information when its not needed for killing runs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Print progress of reaidng job info when killing.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

install: use rhel builder for rhel

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #312 from ceph/wip-tls-leak

valgrind: ignore TLS leak in glibc

Add ability to upgrade with ceph-deploy

Fixes: 6146
Signed-off-by: Warren Usui <warren.usui@inktank.com>

valgrind: suppress eglibc leak of TLS

<error>
  <unique>0xf9</unique>
  <tid>1</tid>
  <kind>Leak_PossiblyLost</kind>
  <xwhat>
    <text>320 bytes in 1 blocks are possibly lost in loss record 248 of 289</text>
    <leakedbytes>320</leakedbytes>
    <leakedblocks>1</leakedblocks>
  </xwhat>
  <stack>
    <frame>
      <ip>0x4C29DB4</ip>
      <obj>/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>calloc</fn>
    </frame>
    <frame>
      <ip>0x4012074</ip>
      <obj>/lib/x86_64-linux-gnu/ld-2.15.so</obj>
      <fn>_dl_allocate_tls</fn>
      <dir>/build/buildd/eglibc-2.15/elf</dir>
      <file>dl-tls.c</file>
      <line>297</line>
    </frame>
    <frame>
      <ip>0x5289ABC</ip>
      <obj>/lib/x86_64-linux-gnu/libpthread-2.15.so</obj>
      <fn>pthread_create@@GLIBC_2.2.5</fn>
      <dir>/build/buildd/eglibc-2.15/nptl</dir>
      <file>allocatestack.c</file>
      <line>571</line>
    </frame>
    <frame>
      <ip>0xA6A89A</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Thread::try_create(unsigned long)</fn>
      <dir>/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-0.83-444-ge74ad5f/src/common</dir>
      <file>Thread.cc</file>
      <line>112</line>
    </frame>
    <frame>
      <ip>0xA6A94E</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Thread::create(unsigned long)</fn>
      <dir>/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-0.83-444-ge74ad5f/src/common</dir>
      <file>Thread.cc</file>
      <line>122</line>
    </frame>

...

ubuntu@teuthology:/a/teuthology-2014-08-13_23:04:01-fs-next-testing-basic-multi/423897

Signed-off-by: Sage Weil <sage@redhat.com>

Perform some sanity checks on --name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make --name required

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't fetch the suite if we can already find it

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Attach FileHandlers to the RootLogger object

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>