Add [--] to usage statement

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Automatically watch 'multi' tube if tube_name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #270 from ceph/wip-schedule-suite

Refactor teuthology.schedule...

Refactor teuthology.schedule...

... to separate the assembling of the job config from the scheduling of
the job. Also port its argument parsing to docopt.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remote 'delete' and 'show' features

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #268 from ceph/wip-5400

Support installing ceph kernels from gitbuilder on rpm machines.

Better variable names.

As suggested by review.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Disable "fix" for corrupt repo detection

It's causing more problems than it's solving at this point

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use correct cwd

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #269 from ceph/wip-radosfix

rados: Add hit_set_list missing from possible op weights

Update to reflect relocated get_jobs()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

rados: Add hit_set_list missing from possible op weights

For the future help keep in sync with test/osd/TestRados.cc by listing
in order of op_types in that code.

Signed-off-by: David Zafman <david.zafman@inktank.com>

Move teuthology-ls's implementation to ls.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix corrupt repo detection

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Added requested comment.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused variables and functions.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update --help output

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Support installing ceph kernels from gitbuilder on rpm machines.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #267 from ceph/wip-runxfstests

wip-runxfstests

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Add a timeout for unmounting

Ten minutes seems like it might be longer than necessary, but I doubt
it's too short.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #265 from ceph/wip-orchestra

Refactor teuthology.orchestra.run

Remove some duplicate code.

Well, not exactly duplicate, but functionally compatible. No sense in
maintaining two "walk the filesystem to assemble job information"
utilities.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Try a different approach to zombie prevention

The previous method I had committed had the unfortunate side-effect of
preventing teuthology job processes from giving their returncode to the
worker process. This method shouldn't do that.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

radosgw-admin: adjust bucket link interface

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rbd: change test_script to run_xfstests_krbd.sh

Invoke run_xfstests.sh through a wrapper, which supplies an exclusion
list, in order to skip tests we don't want to run.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

rbd: recognize 'randomize' parameter

randomize (default: false) randomizes xfstests test order.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

Merge pull request #266 from ceph/wip-8489

Give daemons a five-minute timeout for stopping

Reviewed-by: Sage Weil <sage@inktank.com>

Give daemons a five-minute timeout for stopping

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add RemoteProcess.__repr__()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update users of the teuthology.orchestra.run APIs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactor teuthology.orchestra.run

RemoteProcess behaves more like subprocess.Popen, with some important
differences.

A summary of the API changes:
* RemoteProcess.exitstatus is either an int or None; it is never a callable
  nor a gevent.AsyncResult.
* New method: RemoteProcess.execute()
* New method: RemoteProcess.poll()
* New method: RemoteProcess.wait()
* New attribute: RemoteProcess.returncode - alias to exitstatus
* New property: RemoteProcess.finished - added because returncode can be None
  if the connection was interrupted
* run.execute() is removed.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Avoid piles of zombies

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add comment explaining 'preexec_fn=os.setpgrp'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Check for broken repos before updating

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

More verbose logging for daemon restarts

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Bump log level to info

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't block on teuthology-results processes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Correct logging when running teuthology-results

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #263 from qakapil/patch-two

updated the get_system_type function

updated the get_system_type function

updated the get_system_type function
to add the rpm condition for SUSE.

Signed-off-by: ksharma@suse.com

Merge pull request #259 from ceph/wip-journal-tool

mds_journal_migration task

tasks/mds_journal_migration: create.

This is for testing the changes to CephFS
journal format, especially the migration
of journals from old to new format.

This is a teuthology task rather than being
a workunit script, because it involves
manipulating services and config during the
procedure.

Signed-off-by: John Spray <john.spray@inktank.com>

task/ceph: separate ceph.conf into function

This is so that other tasks can modify ceph.conf
as needed.

Signed-off-by: John Spray <john.spray@inktank.com>

tasks: Use '_' instead of '-' in names

Python modules are not allowed to contain hyphens
in the name.  Using hyphens only works here because
we're using low level __import__.

Change run_tasks to replace '-' with '_' when reading
configs, and rename the task modules to have valid
python module names.

Aside from general hygiene, the motivation to do this
is to allow one task to build on code in another
task by importing it.

Signed-off-by: John Spray <john.spray@inktank.com>

Don't crash when an invalid branch is passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Set results_server by default

So that queued jobs get pushed to paddles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Process queued jobs synchronously

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #260 from ceph/wip-8422

Add exception hook to teuthology-worker

Add exception hook to teuthology-worker

Workers processes are dying occasionally, and this should cause the
exceptions to be logged.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #258 from ceph/wip-fix-format-samba

Fixed formatting and added a better variable for time out

Fixed broken quotations

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed formatting and added a better variable for time out
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

task/: add populate_rbd_pool for quickly spinning up a pool with rbd stuff

Signed-off-by: Samuel Just <sam.just@inktank.com>

ceph_manager: reset timeout if we are making progress in wait_for_recovery

Signed-off-by: Samuel Just <sam.just@inktank.com>

Avoid calling yaml.safe_load() twice on each job

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Split progress indicator out to separate functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

increased sleep to 100 as 90 seemed insufficient
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added more logging
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added sleep log info line
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #257 from ceph/wip-samba-fix

increased time wait to 90 sec so tests does not fail

increased time wait to 90 sec so tests does not fail

Add Pulpito links to teuthology emails

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Downgrade paramiko to work around an SFTP bug

See http://tracker.ceph.com/issues/8327

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #256 from ceph/wip-6542-wusui

Add missng docstrings to repair_test.py

Add missng docstrings to repair_test.py

Fixes: 6542
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix unit tests under Jenkins

os.getlogin() was throwing:
  OSError: [Errno 25] Inappropriate ioctl for device

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use VersionNotFoundError packages are missing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use config.archive_base if one isn't passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add retries to orchestra.connection.connect()

This is an attempt to fix: http://tracker.ceph.com/issues/8314

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix typo

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Better logging

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use try_mark_run_dead()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add try_mark_run_dead()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move list of exceptions to catch

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move "no results server" warning

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix name parsing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #255 from ceph/wip-6921-wusui

Allow .teuthology.yaml to set downburst path

Don't pass a custom logger anymore

We already use the hostname in command execution calls

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use 'stderr' and 'stdout' as logger names

We were using just 'err' and 'out', which isn't very intuitive.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Catch any Unicode errors that manage to sneak in

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Express hostnames as child logger names

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.user

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.user attribute

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.shortname in logs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make Remote.shortname actually short

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remote.hostname doesn't have to be a property

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix PEP-8 issues

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Pass hostname to execute()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use 'true' instead of 'echo online'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.ensure_online()

If the connection is alive, do nothing. If not, reconnect. Allow any
exceptions to bubble up to the caller. This is intended to fix unhelpful
'Bad File Descriptor' errors we were seeing when VMs go down.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Remote.hostname in logs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Consolidate log file setup into shared function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use Unicode format strings

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Revert "Show hostname instead of IP in errors""

This reverts commit 10fee0e368750cf4cd953db5700df59c7f611119.

Conflicts:
teuthology/orchestra/run.py

Allow .teuthology.yaml to set downburst path

If .teuthology.yaml defines downburst, _get_downburst_exec()
now returns that value as the path to the downburst executable.

Also cleaned up code in create_if_vm.  ctx.downburst_conf was
never defined, so the code that handled the AttributeError
was always being executed.

Fixes: 6921
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #254 from ceph/wip-7707-wusui

Use master as default for debian upgrade.

Force log lines to be interpreted as UTF-8

Any invalid UTF-8 byte will be replaced with a Unicode replacement
character: U+FFFD or '�'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use binary flag for paramiko ChannelFiles

This works around http://tracker.ceph.com/issues/8313

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>