Add a couple unit tests for teuthology.parallel

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #280 from dachary/wip-erasure-code-profile

add rgw.create_ec_pool default profile

add rgw.create_ec_pool default profile

the former arguments had defaults, the caller do not set them

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #275 from ceph/wip-replace-update-sh

Replace update.sh functionality

Rename suite_base to suite_dir

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add --suite-branch and --suite-base

The former lets you specify a specific ceph-qa-suite branch to use for
testing. The latter lets you specify an as-is directory to use, for
example if you want ro run tests that you don't want to commit yet.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use a test-specific name/email for the git calls

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add more unit tests

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #277 from dachary/wip-erasure-code-profile

erasure code profile: fix bugs found while testing manually

Merge pull request #278 from ceph/wip-nullfs

Support the new fs syntax.

Reviewed-by: Greg Farnum <greg@inktank.com>

erasure-code: fix typo in create_pool argument name

missing _name at the end

Signed-off-by: Loic Dachary <loic@dachary.org>

erasure-code: do not prepend 'ceph' to the erasure-code-profile command

When used in the radosgw context, the 'ceph' string must be present but
when used in ceph_manager, the 'ceph' string is already included.

Signed-off-by: Loic Dachary <loic@dachary.org>

erasure-code: str() to avoid + conversion errors

if the value from the yaml file is an int, it must be converted into a
string bevore being concatenated with '='

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #271 from dachary/wip-erasure-code-profile

erasure-code: separate profile from pool creation

Fix linter errors

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a basic validation of the branch value

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Run unit tests offline

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

erasure-code: separate profile from pool creation

Instead of being hard coded, the erasure code profile is read from yaml
as:

    erasure_code_profile:
      name: teuthologyprofile
      k: 2
      m: 1
      ruleset-failure-domain: osd

Which translates into

    ceph osd erasure-code-profile set teuthologyprofile \
        k=2 m=1 ruleset-failure-domain=osd

The semantic of k/m etc. depends on the erasure code plugin, but it is
common to use k as the dividing factor for each object and m as the
number of coding chunks.

Using a different set of parameters or a different plugin can
be done without modifying the teuthology code:

    erasure_code_profile:
      name: LRCprofile
      plugin: LRC
      mapping: __DD__DD
      layers: [[ "_cDD_cDD", "" ], [ "cDDD____", "" ], [ "____cDDD", ""]]

For backward compatibility, the default erasure code profile is set to

      k: 2
      m: 1
      ruleset-failure-domain: osd

which matches the previously hardcoded default of the corresponding
functions.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #233 from ceph/wip-background

run something in the background

background_exec: run something in the background

This is a contextmanager task that will run some command in the background
for the duration of any subsequent tasks, and kill it in the cleanup
phase.

Signed-off-by: Sage Weil <sage@inktank.com>

daemon-helper: optional kill an entire process group

Signed-off-by: Sage Weil <sage@inktank.com>

Split up repo helper into separate functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

task: update 'newfs' users to 'fs new'

Signed-off-by: John Spray <john.spray@redhat.com>

task/ceph: Invoke newfs for CephFS as needed

New ceph versions will no longer create the
CephFS pools and will not enable the filesystem
by default.  This change adds an explicit step
to the cluster initialization to create the
filesystem if it doesn't already exist.

Signed-off-by: John Spray <john.spray@inktank.com>

Use corrected tube_name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Return corrected tube_name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use teuthology.beanstalk

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Document checkout_repo()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Drop a "fix" for a bug that probably never existed

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a note about teuthology scheduling

Specifically, that we always schedule using the master branch.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Improve missing branch detection and logging

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add unit tests for repo_utils

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use 'ceph-qa-suite.git' for the repo name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Also handle teuthology repo checkouts

Our shell script did this also.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Handle ceph-qa-suite checkouts

Previously we had an internally-maintained shell script that did this.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Generalize error message

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix linter errors

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move repo checkout code to new module repo_utils

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use a more informative variable name for dest_path

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #274 from dachary/wip-facet

document how to specify a facet in teuthology-suite

document how to specify a facet in teuthology-suite

Signed-off-by: Loic Dachary <loic@dachary.org>

Only delete jobs that match the pattern

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Swap order of job_id and run_name when deleting

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Skip the kernel stanza if the branch passed is '-'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add more docstrings

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

No need to explicitly return None here

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update unit tests to reflect renamed function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a few more docstrings and a debug statement

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix a couple linter errors

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove schedule_suite.sh

Its entire functionality is now provided by teuthology-schedule.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix bug where the base yaml wasn't being merged

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

With dry_run, quote any individual args

So that individual commands can be copied and pasted
in order to execute them individually.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add lots of unit tests for teuthology.suite

More to come...

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Raise ScheduleFailError even if not sending email

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Do everything that schedule_suite.sh does

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add default results_email

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't schedule an email job if email isn't set

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Split prepare_and_schedule() out of main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use the return value of schedule_suite()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Organize arguments into logical sections

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Tweak usage formatting

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't reread the entire yaml file...

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused functions

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

'and' is a thing

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Take machine_type instead of worker

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add functions for querying gitbuilder repos

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Silence connection pool logging

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove 'template' feature

Neither Sage nor I could find anything that used it

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Only allow scheduling one suite per call.

Also remove all traces of the extra and confusing term 'collection'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Port to docopt

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Refactor collection scheduling out of main()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

If archive_base is None, use config's

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #272 from ceph/release

Create a python package for teuthology

Update setup.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow killing jobs by passing a 'jobspec'.

See teuthology-kill --help

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a couple tests for teuthology.schedule

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make conf_file optional

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add .ropeproject to .gitignore

I use a vim plugin that creates lots of directories with that name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add [--] to usage statement

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Automatically watch 'multi' tube if tube_name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #270 from ceph/wip-schedule-suite

Refactor teuthology.schedule...

Refactor teuthology.schedule...

... to separate the assembling of the job config from the scheduling of
the job. Also port its argument parsing to docopt.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remote 'delete' and 'show' features

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #268 from ceph/wip-5400

Support installing ceph kernels from gitbuilder on rpm machines.

Better variable names.

As suggested by review.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Disable "fix" for corrupt repo detection

It's causing more problems than it's solving at this point

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use correct cwd

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #269 from ceph/wip-radosfix

rados: Add hit_set_list missing from possible op weights

Update to reflect relocated get_jobs()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

rados: Add hit_set_list missing from possible op weights

For the future help keep in sync with test/osd/TestRados.cc by listing
in order of op_types in that code.

Signed-off-by: David Zafman <david.zafman@inktank.com>

Move teuthology-ls's implementation to ls.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix corrupt repo detection

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Added requested comment.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused variables and functions.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update --help output

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Support installing ceph kernels from gitbuilder on rpm machines.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #267 from ceph/wip-runxfstests

wip-runxfstests

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Add a timeout for unmounting

Ten minutes seems like it might be longer than necessary, but I doubt
it's too short.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>