install.upgrade: fix overrides of sha1|tag|branch

If the upgrade task config has a branch: (or tag or sha1), do
not apply the sha1|branch|tag overrides keys.  This fixes the
breakage from 280f783c2e8dda0df6afb4de0b115aad1614fbdc which
made

overrides:
  install:
    ceph:
      sha1: f88866fe9c218e03b3f89ff21e2b7b655aa74294
tasks:
...
- install.upgrade:
    all:
      branch: dumpling

...use the sha1 from the overrides instead of the explicitly
specified branch.  The intention was to only use the overrides
when the version was not specified (whether it was sha1 or
branch or tag).

At some point we should probably make the same change for
install function in install.py, but let's fix this first to
get the upgrade tests working.

Signed-off-by: Sage Weil <sage@inktank.com>

ceph_manager: remove 6116 workaround

Signed-off-by: Samuel Just <sam.just@inktank.com>

Merge branch 'next'

radosgw-agent: add metadata-only option to task config

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

radosgw-agent: move positional argument last

Other args are treated as positional after the first one.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

internal: fix log for python 2.6

{} was introduced to format strings in 2.7

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

rgw: fix whitespace

Replace tabs with spaces and remove trailing spaces.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

rgw: remove unused variables

Region name isn't needed for the new radosgw-agent
command line args.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Reflect radosgw-agent option changes in teuthology

Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>

Merge pull request #144 from ceph/wip-doc5939-wusui

Added docstrings, and improved some of the comments on several tasks.

Merge pull request #145 from ceph/wip-5939

Document install task

Fix some PEP-8 issues

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix param docstring format to be more Sphinx-y

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add docstrings to every function. And a few other comments.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use worker httpd instead of prefork (like ubuntu) on rpm distros.

Ubuntu's default apache uses worker instead of prefork like rpm
based distro's. If rpm use httpd.worker instead of httpd so that
the -X behavior will not be blocked by a single request.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Use worker httpd instead of prefork (like ubuntu) on rpm distros.

Ubuntu's default apache uses worker instead of prefork like rpm
based distro's. If rpm use httpd.worker instead of httpd so that
the -X behavior will not be blocked by a single request.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #143 from ceph/wip-6576

fix rbd fsx on 32-bit

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

rbd_fsx: do not exceed 250GB for fsx image

This breaks on 32-bit architectures because fsx can't allocate the
in-memory buffer or hit some other limit.  500MG also seems to run out of
memory (tho it at least does not segfault).

Fixes: #6576
Signed-off-by: Sage Weil <sage@inktank.com>

ceph_manager: workaround for 6116

This is an annoying race, we really should delay going
clean until the backfill peer has acknoledged the clean
info, but we currently don't.  In order to prevent this
bug from messing up the nightlies, we'll delay killing
the peer for 20s to make it likely that the backfill
peer has gotten the clean info.

Workaround: #6116
Signed-off-by: Samuel Just <sam.just@inktank.com>

rgw: revert to -X for apache2

-DNO_DETACH doesn't seem to work.

Signed-off-by: Sage Weil <sage@inktank.com>

valgrind: fix libleveldb suppression on dynamically linked leveldb

(The function names don't show up here)

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #142 from ceph/wip-6342

More robust machine_type and pid detection

More robust machine_type and pid detection

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #139 from ceph/wip-6342

Add teuthology-kill

Merge pull request #141 from ceph/wip-nuke

fix nuke

nuke: remove old log arg to nuke_one call

Signed-off-by: Sage Weil <sage@inktank.com>

Rewrite portions of teuthology.kill

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

replacing -X option with -DNO_DETACH to ensure the
apache process is detached at the end of run.

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #138 from ceph/wip-refactor-scripts

Cleanup

Remove needless arg from list_locks()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make nuke use its own logger

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add teuthology-kill

Currently implements killing suites-runs.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #137 from ceph/wip-upgrade

install: allow use of overrides for upgrade version

Merge pull request #136 from ceph/wip-nuke

nuke: fix import

nuke: fix import

I think broken by f28a7ebc2c3662a8a9b8eebf09c47b6c710e4d26

Signed-off-by: Sage Weil <sage@inktank.com>

install: allow use of overrides for upgrade version

Signed-off-by: Sage Weil <sage@inktank.com>

Fix broken logic in running_jobs_for_run()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix typo

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #134 from ceph/wip-6546

Fix race condition between tests starting and teuthology-results being run

Make check for running jobs part of the loop

Use a new, tested implementation to check for running jobs.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

ResultsSerializer.running_jobs_for_run() and test

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #133 from ceph/wip-store-results

Fixes for reporting and verbosity

Added docstrings, and improved some of the comments on several tasks.

Put machine_type in config.yaml

Reason being, if --worker is passed to teuthology-schedule, we are
already placing the job in the tube for that machine type. If the job
config doesn't specify a machine_type, it will later be assigned to
'plana' by teuthology-run without actually modifying the config. This
can result in jobs running on hardware other than what was requested.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make verbosity propagate correctly to modules

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Debug output for all http requests

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix broken lambda

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Be more verbose when waiting for free machines

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Give reporting a verbose flag

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Lack of a job_id should be a warning

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #132 from ceph/wip-refactor-scripts

Fix imports; fix keyscan_check()

Exempt teuthology-updatekeys from test_invalid

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Ignore the exit code of ssh-keyscan

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix a circular import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move imports to top-level

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #131 from ceph/wip-store-results

Reporting tweakage

Move part of report_all_runs() into report_runs()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Set a default timeout of 20s for HTTP connections

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move monkey patching to __init__.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't attempt to report if there's no job_id

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add basic tests for scripts

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Properly express conflicting options

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

PEP-8

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

PEP-8

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-coverage's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

PEP-8

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-results' arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

PEP-8

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-report's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-updatekeys' arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move schedule() to new schedule module

Move teuthology-schedule's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

PEP-8

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-lock's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

PEP-8

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-worker's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-ls's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

More PEP-8 cleanup

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-suite's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology-nuke's arg parsing to scripts/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add scripts/ module

PEP-8 cleanup

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

ignoring -X option in apache as it was causing s3test failures on the rpms.

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #126 from ceph/wip-store-results

Reflect paddles API change: jobs in jobs/

Reflect paddles API change: jobs in jobs/

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #125 from ceph/wip-store-results

Add more tests for ReportSerializer

Add more tests for ReportSerializer

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #124 from ceph/wip-store-results

Various fixes related to reporting

Tweak logic of try_push_job_info()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make most of lock_machines() PEP-8 clean.

Also, use some better variable names.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make teuthology-schedule also print the job's name

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Store the job_id as a str, not an int.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move teuthology.report import; add large warning

Because of the monkey patching, if we get the import order wrong bad
things happen.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #123 from ceph/wip-store-results

Fix issue with monkey patching

Add (and use) try_push_job_info() for easy pushin'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move new imports to inside main()

The monkey patch stuff was causing problems again.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

schedule_suite.sh: include default machine type in job name

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #121 from ceph/wip-store-results

Submit teuthology results to a paddles instance

Add test for test_all_runs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>