Split prepare_and_schedule() out of main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use the return value of schedule_suite()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Organize arguments into logical sections

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Tweak usage formatting

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't reread the entire yaml file...

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

'and' is a thing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Take machine_type instead of worker

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add functions for querying gitbuilder repos

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Silence connection pool logging

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove 'template' feature

Neither Sage nor I could find anything that used it

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Only allow scheduling one suite per call.

Also remove all traces of the extra and confusing term 'collection'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Port to docopt

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactor collection scheduling out of main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

If archive_base is None, use config's

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #272 from ceph/release

Create a python package for teuthology

Update setup.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow killing jobs by passing a 'jobspec'.

See teuthology-kill --help

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a couple tests for teuthology.schedule

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make conf_file optional

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add .ropeproject to .gitignore

I use a vim plugin that creates lots of directories with that name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add [--] to usage statement

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Automatically watch 'multi' tube if tube_name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #270 from ceph/wip-schedule-suite

Refactor teuthology.schedule...

Refactor teuthology.schedule...

... to separate the assembling of the job config from the scheduling of
the job. Also port its argument parsing to docopt.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remote 'delete' and 'show' features

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #268 from ceph/wip-5400

Support installing ceph kernels from gitbuilder on rpm machines.

Better variable names.

As suggested by review.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Disable "fix" for corrupt repo detection

It's causing more problems than it's solving at this point

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use correct cwd

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #269 from ceph/wip-radosfix

rados: Add hit_set_list missing from possible op weights

Update to reflect relocated get_jobs()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

rados: Add hit_set_list missing from possible op weights

For the future help keep in sync with test/osd/TestRados.cc by listing
in order of op_types in that code.

Signed-off-by: David Zafman <david.zafman@inktank.com>

Move teuthology-ls's implementation to ls.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix corrupt repo detection

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Added requested comment.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused variables and functions.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update --help output

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Support installing ceph kernels from gitbuilder on rpm machines.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #267 from ceph/wip-runxfstests

wip-runxfstests

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Add a timeout for unmounting

Ten minutes seems like it might be longer than necessary, but I doubt
it's too short.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #265 from ceph/wip-orchestra

Refactor teuthology.orchestra.run

Remove some duplicate code.

Well, not exactly duplicate, but functionally compatible. No sense in
maintaining two "walk the filesystem to assemble job information"
utilities.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Try a different approach to zombie prevention

The previous method I had committed had the unfortunate side-effect of
preventing teuthology job processes from giving their returncode to the
worker process. This method shouldn't do that.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

radosgw-admin: adjust bucket link interface

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rbd: change test_script to run_xfstests_krbd.sh

Invoke run_xfstests.sh through a wrapper, which supplies an exclusion
list, in order to skip tests we don't want to run.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

rbd: recognize 'randomize' parameter

randomize (default: false) randomizes xfstests test order.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

Merge pull request #266 from ceph/wip-8489

Give daemons a five-minute timeout for stopping

Reviewed-by: Sage Weil <sage@inktank.com>

Give daemons a five-minute timeout for stopping

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add RemoteProcess.__repr__()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update users of the teuthology.orchestra.run APIs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactor teuthology.orchestra.run

RemoteProcess behaves more like subprocess.Popen, with some important
differences.

A summary of the API changes:
* RemoteProcess.exitstatus is either an int or None; it is never a callable
  nor a gevent.AsyncResult.
* New method: RemoteProcess.execute()
* New method: RemoteProcess.poll()
* New method: RemoteProcess.wait()
* New attribute: RemoteProcess.returncode - alias to exitstatus
* New property: RemoteProcess.finished - added because returncode can be None
  if the connection was interrupted
* run.execute() is removed.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Avoid piles of zombies

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add comment explaining 'preexec_fn=os.setpgrp'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Check for broken repos before updating

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

More verbose logging for daemon restarts

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Bump log level to info

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't block on teuthology-results processes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Correct logging when running teuthology-results

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #263 from qakapil/patch-two

updated the get_system_type function

updated the get_system_type function

updated the get_system_type function
to add the rpm condition for SUSE.

Signed-off-by: ksharma@suse.com

Merge pull request #259 from ceph/wip-journal-tool

mds_journal_migration task

tasks/mds_journal_migration: create.

This is for testing the changes to CephFS
journal format, especially the migration
of journals from old to new format.

This is a teuthology task rather than being
a workunit script, because it involves
manipulating services and config during the
procedure.

Signed-off-by: John Spray <john.spray@inktank.com>

task/ceph: separate ceph.conf into function

This is so that other tasks can modify ceph.conf
as needed.

Signed-off-by: John Spray <john.spray@inktank.com>

tasks: Use '_' instead of '-' in names

Python modules are not allowed to contain hyphens
in the name.  Using hyphens only works here because
we're using low level __import__.

Change run_tasks to replace '-' with '_' when reading
configs, and rename the task modules to have valid
python module names.

Aside from general hygiene, the motivation to do this
is to allow one task to build on code in another
task by importing it.

Signed-off-by: John Spray <john.spray@inktank.com>

Don't crash when an invalid branch is passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Set results_server by default

So that queued jobs get pushed to paddles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Process queued jobs synchronously

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #260 from ceph/wip-8422

Add exception hook to teuthology-worker

Add exception hook to teuthology-worker

Workers processes are dying occasionally, and this should cause the
exceptions to be logged.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #258 from ceph/wip-fix-format-samba

Fixed formatting and added a better variable for time out

Fixed broken quotations

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed formatting and added a better variable for time out
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

task/: add populate_rbd_pool for quickly spinning up a pool with rbd stuff

Signed-off-by: Samuel Just <sam.just@inktank.com>

ceph_manager: reset timeout if we are making progress in wait_for_recovery

Signed-off-by: Samuel Just <sam.just@inktank.com>

Avoid calling yaml.safe_load() twice on each job

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Split progress indicator out to separate functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

increased sleep to 100 as 90 seemed insufficient
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added more logging
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added sleep log info line
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #257 from ceph/wip-samba-fix

increased time wait to 90 sec so tests does not fail

increased time wait to 90 sec so tests does not fail

Add Pulpito links to teuthology emails

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Downgrade paramiko to work around an SFTP bug

See http://tracker.ceph.com/issues/8327

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #256 from ceph/wip-6542-wusui

Add missng docstrings to repair_test.py

Add missng docstrings to repair_test.py

Fixes: 6542
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix unit tests under Jenkins

os.getlogin() was throwing:
  OSError: [Errno 25] Inappropriate ioctl for device

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use VersionNotFoundError packages are missing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use config.archive_base if one isn't passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add retries to orchestra.connection.connect()

This is an attempt to fix: http://tracker.ceph.com/issues/8314

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix typo

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Better logging

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use try_mark_run_dead()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add try_mark_run_dead()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move list of exceptions to catch

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move "no results server" warning

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix name parsing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #255 from ceph/wip-6921-wusui

Allow .teuthology.yaml to set downburst path