Exit with code already returned by lock.main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Less dangerous hostname mangling

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix --unlock

Note to self: clean up user/ctx.owner confusion

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Return old-style hostnames from lock_many()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't "import misc as teuthology"

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #318 from ceph/wip-defaults

change defaults for mira,plana,burnupi

More hostname handling fixes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Canonicalize hostnames"

This reverts commit 29e8deefaeef21d1e7f6c40d6fca2605324dec69.

Remove the 'user@' prefix before checking status

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow passing user=None to remove the 'user@'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Canonicalize hostnames in yet another place

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

suite: scheule in mira,burnupi,plana by deafult

(was plana)

Signed-off-by: Sage Weil <sage@redhat.com>

teuthology-queue: list multi queue by default

Signed-off-by: Sage Weil <sage@redhat.com>

Canonicalize hostnames

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new vm_host attribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't dump all vm_host details when listing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Reflect updated field names

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update provision module to use new lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.shortname instead of regex

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unlock_many() and use it when possible

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update internal task to use new lockserver

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make far less requests when simply locking

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Optimize some list_locks() calls

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Properly query for multiple machine types

...in list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use consistent URI for lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove old lock server

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Change default lock server to paddles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use /nodes/ as the lock server endpoint

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update to use new ssh-keyscan API

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite ssh public key updating

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new ssh_keyscan() function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

No need to be passing ctx to update_lock()

... or update_keys() or scan_for_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add new ssh_keyscan() method

This is just a cleaner implementation that will replace the existing
one(s).

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Re-order functions in module

This is for readability.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

keyscan_check(): don't require ctx to be passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update lock_many(), update_lock(), and main()

Also remove lockstatus.send_request()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix PEP-8 issues

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactoring

Create a new teuthology.provision module and move virtual machine
functions there. Move hostname manipulation functions to
teuthology.misc.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update do_summary()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rewrite list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make (de)canonicalize_hostname() work without user

Add tests proving they still work *with* usernames as well.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unit tests for (de)canonicalize_hostname()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update/rewrite several functions to use paddles

Functions updated: lock.lock_one(), lock.unlock_one(),
lock.destroy_if_vm(), lockstatus.get_status()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

ceph.conf: mon reweight min pgs per osd = 4

So teh cephtool reweight-by-pg test will pass with mini clusters.

Signed-off-by: Sage Weil <sage@redhat.com>

Add --suite-path

Slightly less ugly than setting PYTHONPATH

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #316 from ceph/wip-9183-wusui

Fix return from schedule_suite.

Fix return from schedule_suite.

Schedule_suite returns the number of jobs scheduled (accounting for
filters).  Information displayed also indicates how many jobs were
run and how many were filtered out.

Fixes: 9183
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #309 from ceph/wip-8954-wusui

Add filter and filter-out options to suites.

valgrind: make trusty tcmalloc supporession more general

To capture this:

  <kind>SyscallParam</kind>
  <what>Syscall param msync(start) points to unaddressable byte(s)</what>
  <stack>
    <frame>
      <ip>0x5081B3D</ip>
      <obj>/lib/x86_64-linux-gnu/libpthread-2.19.so</obj>
      <dir>/build/buildd/eglibc-2.19/nptl/../sysdeps/unix</dir>
      <file>syscall-template.S</file>
      <line>81</line>
    </frame>
    <frame>
      <ip>0x76BA123</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BCEF6</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BE151</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BE4E8</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
    </frame>
    <frame>
      <ip>0x76BAA30</ip>
      <obj>/usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1</obj>
      <fn>_ULx86_64_step</fn>
    </frame>
    <frame>
      <ip>0x5C42052</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>GetStackTrace(void**, int, int)</fn>
    </frame>
    <frame>
      <ip>0x5C349F4</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>tcmalloc::PageHeap::GrowHeap(unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x5C34D02</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>tcmalloc::PageHeap::New(unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x5C25D0C</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
    </frame>
    <frame>
      <ip>0x5C46378</ip>
      <obj>/usr/lib/libtcmalloc.so.4.1.2</obj>
      <fn>operator new[](unsigned long)</fn>
    </frame>

Signed-off-by: Sage Weil <sage@redhat.com>

Use suite_path, if set, for one-off jobs as well

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add filter and filter-off suite options.

Filter only runs suite jobs that contain the text passed as part of
their description or in the names of the composite set of yaml files.
Filter-off only runs suite jobs that do not contain the text passed
as either their description or in the names of the composite set of
yaml files.
Fixes: 8954
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #313 from ceph/wip-rhel

install: use rhel builder for rhel

Merge pull request #315 from ceph/wip-8877

Skip nuke and just destroy VPS if nuke is --unlock

Skip nuke and just destroy VPS if nuke is --unlock

If the machine is being unlocked after than we definitely want to
just destroy it. This way nukes will always suceed with --unlock
even if the guest is down or in a borked state.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

install: use rhel builder for rhel

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #312 from ceph/wip-tls-leak

valgrind: ignore TLS leak in glibc

valgrind: suppress eglibc leak of TLS

<error>
  <unique>0xf9</unique>
  <tid>1</tid>
  <kind>Leak_PossiblyLost</kind>
  <xwhat>
    <text>320 bytes in 1 blocks are possibly lost in loss record 248 of 289</text>
    <leakedbytes>320</leakedbytes>
    <leakedblocks>1</leakedblocks>
  </xwhat>
  <stack>
    <frame>
      <ip>0x4C29DB4</ip>
      <obj>/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>calloc</fn>
    </frame>
    <frame>
      <ip>0x4012074</ip>
      <obj>/lib/x86_64-linux-gnu/ld-2.15.so</obj>
      <fn>_dl_allocate_tls</fn>
      <dir>/build/buildd/eglibc-2.15/elf</dir>
      <file>dl-tls.c</file>
      <line>297</line>
    </frame>
    <frame>
      <ip>0x5289ABC</ip>
      <obj>/lib/x86_64-linux-gnu/libpthread-2.15.so</obj>
      <fn>pthread_create@@GLIBC_2.2.5</fn>
      <dir>/build/buildd/eglibc-2.15/nptl</dir>
      <file>allocatestack.c</file>
      <line>571</line>
    </frame>
    <frame>
      <ip>0xA6A89A</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Thread::try_create(unsigned long)</fn>
      <dir>/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-0.83-444-ge74ad5f/src/common</dir>
      <file>Thread.cc</file>
      <line>112</line>
    </frame>
    <frame>
      <ip>0xA6A94E</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Thread::create(unsigned long)</fn>
      <dir>/srv/autobuild-ceph/gitbuilder.git/build/out~/ceph-0.83-444-ge74ad5f/src/common</dir>
      <file>Thread.cc</file>
      <line>122</line>
    </frame>

...

ubuntu@teuthology:/a/teuthology-2014-08-13_23:04:01-fs-next-testing-basic-multi/423897

Signed-off-by: Sage Weil <sage@redhat.com>

Perform some sanity checks on --name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make --name required

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't fetch the suite if we can already find it

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Attach FileHandlers to the RootLogger object

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't remove lockfiles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add retries to fetch_teuthology()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add BootstrapError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

If bootstrap fails, remove the virtualenv

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix another variable name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add message to exception log

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add retries to fetch_qa_suite()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use GitError instead of RuntimeError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add GitError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix some variable naming

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #310 from ceph/wip-valgrind

misc: pass premable through when no valgrind

misc: pass premable through when no valgrind

Signed-off-by: Sage Weil <sage@redhat.com>

Be more informative if owner info is missing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix format string

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't redefine 'remote'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't import from inside functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Attempt fetches on init, before running jobs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Also rmtree on RuntimeError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Disable pyc removal for now

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Bump all log levels to info

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Default to using the master branch of teuthology

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

For non-scheduled jobs, check out the suite repo.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

After resetting a repo, remove all pyc files

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make teuthology-{suite,worker} share repo code

Repo-fetching code, that is.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Reimplement the file locker as a context manager

Also give it a 'noop' flag to enable its callers to optionally not use
locking.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move exception into a new exceptions module

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move functions into repo_utils

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix NameError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Correctly find both internal and external tasks

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove most ceph-specific tasks. They are in ceph-qa-suite now.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #306 from ceph/wip-9024

ignore errors if 'ps aux' fails

ignore errors if 'ps aux' fails

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

Merge pull request #305 from ceph/wip-8640

implement `--version` and improve `setup.py`

create a --version flag for teuthology

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

pull the version and description in setup.py

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

add an initial version to teuthology

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

Tweak usage statement

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>