Add more info in comments.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Speed up teuthology-kill.

Read less information when its not needed for killing runs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Print progress of reaidng job info when killing.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Perform some sanity checks on --name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make --name required

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't fetch the suite if we can already find it

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Attach FileHandlers to the RootLogger object

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't remove lockfiles

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add retries to fetch_teuthology()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add BootstrapError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

If bootstrap fails, remove the virtualenv

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix another variable name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add message to exception log

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add retries to fetch_qa_suite()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use GitError instead of RuntimeError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add GitError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix some variable naming

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #310 from ceph/wip-valgrind

misc: pass premable through when no valgrind

misc: pass premable through when no valgrind

Signed-off-by: Sage Weil <sage@redhat.com>

Be more informative if owner info is missing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix format string

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't redefine 'remote'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't import from inside functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Attempt fetches on init, before running jobs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Also rmtree on RuntimeError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Disable pyc removal for now

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Bump all log levels to info

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Default to using the master branch of teuthology

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

For non-scheduled jobs, check out the suite repo.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

After resetting a repo, remove all pyc files

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make teuthology-{suite,worker} share repo code

Repo-fetching code, that is.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Reimplement the file locker as a context manager

Also give it a 'noop' flag to enable its callers to optionally not use
locking.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move exception into a new exceptions module

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move functions into repo_utils

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix NameError

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Correctly find both internal and external tasks

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove most ceph-specific tasks. They are in ceph-qa-suite now.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #306 from ceph/wip-9024

ignore errors if 'ps aux' fails

ignore errors if 'ps aux' fails

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

Merge pull request #305 from ceph/wip-8640

implement `--version` and improve `setup.py`

create a --version flag for teuthology

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

pull the version and description in setup.py

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

add an initial version to teuthology

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

Tweak usage statement

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #304 from ceph/wip-fix-build-matrix

suite: fix build_matrix when non-yaml present in % dir

suite: fix build_matrix when non-yaml present in % dir

First, if we get a non-yaml file, return an empty list.

Second, if we are in a % mode, exclude an empty raw result from the
combination.  Otherwise, we get no resulting jobs.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #303 from ceph/wip-8930

Tasks are failing since using "data" pool no longer part of default install

Reviewed-by: Sage Weil <sage@redhat.com>

Tasks are failing since using "data" pool no longer part of default install

Create a pool specifically for each task

Fixes: 8930
Signed-off-by: David Zafman <david.zafman@inktank.com>

Expand '~' in ssh key paths

... since paramiko doesn't bother to do this

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #301 from ceph/wip-nuke-vpm

nuke: allow nuking vpm hosts

ceph_manager: increase osd revival timeout from 75 -> 150 seconds

Saw this fail on a firefly run just bc of valgrind slowness.  Make the
timeout higher!

Signed-off-by: Sage Weil <sage@redhat.com>

valgrind: ignore ec plugin factory leaks

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #300 from ceph/wip-mpi-vers

task/mpi: Explicit check for version mismatch

Merge pull request #302 from ceph/wip-8850

tell us if you are really not running any more

add another call to use "ps aux"

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

tell us if you are really not running any more

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

nuke: allow nuking vpm hosts

Skip the console check, though.

Signed-off-by: Sage Weil <sage@redhat.com>

task/mpi: Explicit check for version mismatch

Instead of proceeding to have mpiexec fail or hang,
do an up-front check that the MPI version is the same
on all of the nodes in the test.

Signed-off-by: John Spray <john.spray@redhat.com>

Fix package_version_for_hash() call

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task/mds_journal_migration: fix cleanup

Previously this was remounting in the background
which meant that if a workunit was executed immediately
afterwards it could put files in the mount dir before
the FS was mounted, causing the mount to fail.

Signed-off-by: John Spray <john.spray@redhat.com>

task/workunit: add sudo to dir delete

This failure to delete used to be hidden in
a catch-all exception handler that was
removed in 396dc073.

This was causing failures in kcephfs suite
like:
Command failed on plana96 with status 1: 'rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0'

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #296 from ceph/wip-standby-mds

Wip standby mds

task/osd_recovery: use rbd instead of metadata pool

Signed-off-by: Sage Weil <sage@redhat.com>

task/osd_backfill: use 'rbd' instead of 'data' pool

Signed-off-by: Sage Weil <sage@redhat.com>

task/workunit: general tidy

* PEP8-ize whitespace
* specialize a catch-all except:
* don't alias 'misc' to 'teuthology'

Signed-off-by: John Spray <john.spray@redhat.com>

task/workunit: clean up dir deletion

This was always trying to remove the mountpoint,
and then swallowing the exception and printing
it to the log.

Instead:
 * Ensure it's only trying to delete mnt if it created it
 * Don't swallow those exceptions: if deleting the dirs
   fails then something has gone wrong and it should bubble
   up.

Signed-off-by: John Spray <john.spray@redhat.com>

task/cephfs: generalise Filesystem for multi-MDS

This enables tasks like mds_journal_migration to be
run in an environment with standby-replay MDSs present.

Signed-off-by: John Spray <john.spray@redhat.com>

Remove oddly-placed, broken import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Rename CephState to DaemonGroup and move to orchestra

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move DaemonState to teuthology.orchestra

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fail sooner if an invalid config_yaml is passed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove kcon_most

It was unused.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #293 from ceph/wip-ship-utilities

move ship_utilities to install task (from ceph test)

ceph_deploy: use new ship_utilities location

Signed-off-by: Sage Weil <sage@redhat.com>

task/mds_journal_migration: use existing clients

Since refactoring ceph_fuse and kclient tasks
to store Mount objects on ctx, the mds_journal_migration
task can use those instead of explicitly instantiating its own.

Signed-off-by: John Spray <john.spray@redhat.com>

move ship_utilities to install task (from ceph test)

No need to install ceph just to get daemon-helper, adjust-ulimits, etc..
These utilities are used by lots of tasks, not just ceph.

Signed-off-by: Sage Weil <sage@redhat.com>

Fix get_hash() arg ordering

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #287 from ceph/wip-mds-sessions

Wip mds sessions

Merge pull request #288 from ceph/wip-upgrade-fix

task/ceph: Make cephfs_setup cuttlefish-compatible

Merge pull request #291 from ceph/wip-rgw-pools

rgw: set data extra pool correctly

Backport: next, firefly
Reviewed-by: Sage Weil <sage@redhat.com>

rgw: set data extra pool correctly

When configuring multiregion zone, set the data extra pool

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Add and use new fetch() method

The old fetch_branch() wasn't behaving properly with the ceph.com git
mirror. This method works with github.com and ceph.com. Add a couple
unit tests, and leave the old fetch_branch() in place for now, even
though nothing uses it.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #290 from ceph/wip-deep-scrub

task/ceph.py: deep-scrub instead of scrub

Fix a bug where results_email was being ignored

Referring to the setting in ~/.teuthology.yaml

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task/ceph.py: deep-scrub instead of scrub

Signed-off-by: Samuel Just <sam.just@inktank.com>

README: teuthology-suite

Signed-off-by: Sage Weil <sage@redhat.com>

task/kclient: remove redundant double loop

Signed-off-by: John Spray <john.spray@redhat.com>

task/cephfs: use dedent for embedded python

Signed-off-by: John Spray <john.spray@redhat.com>

task: fix admin_socket for non-int ids

This was working for OSDs and for my config
where MDS happened to have numeric ID, but
in general service IDs are strings, not ints.

Signed-off-by: John Spray <john.spray@redhat.com>

task: add mds_client_recovery

This task exercises the CephFS session recovery
behavior.

Signed-off-by: John Spray <john.spray@redhat.com>

task/mds_journal_migration: only mount my client

Pass explicit client ID list to ceph_fuse context so
that it doesn't try to mount all clients in the config,
in case there are other clients.

Also allow configuration to specify which client
this task should use.

Signed-off-by: John Spray <john.spray@redhat.com>

task: refactor kclient into KernelMount

Additionally make both kclient and ceph_fuse
tasks yield a collection of CephFSMount
objects so that subsequent tasks can
retrieve them.

Signed-off-by: John Spray <john.spray@redhat.com>

task: create FuseMount class from ceph_fuse

This is for use by other tasks that need filesystem
mounts under their own control.

Signed-off-by: John Spray <john.spray@redhat.com>

Fix the incomplete substitute_placeholders() fix

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a debug statement to reset_repo()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Be smarter about choosing branches

Part of this is also about failing sooner, and not allowing invalid
configurations to enter the queue. This commit also fixes an obscure bug
in substitute_placeholders(). Finally, it adds unit tests for all bugs
fixed.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task: move Filesystem class to shared location

...in preparation for tests other than mds_journal_migration
to use it.

Signed-off-by: John Spray <john.spray@redhat.com>

Fall back to master for suite_branch

The reason for this is that last-in-suite jobs don't have suite_branch
or branch set. We just want to use master for those.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a flag to dump the entire job body.

... with a warning.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add missing placeholder

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>