swift: added "--cluster" to rgw-admin command for multisite support

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #470 from ceph/wip-remote

Add timeouts to Remote connection functions

Merge pull request #466 from ceph/wip-11426

Log stderr in get_latest_image_version_deb()

Merge pull request #462 from ceph/wip-ssh-keys

When modifying authorized_keys, store a backup

Merge pull request #460 from zhouyuan/mkdir_p

Make parent directories as needed

swift: set full access to subusers creation

Default subuser permissions are 'none'.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Remove most ceph-specific tasks. They are in ceph-qa-suite now.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Lines formerly of the form '(remote,) = ctx.cluster.only(role).remotes.keys()'"

This reverts commit d693b3f8950ffd1f2492a4db0f8234fee31f00f0.

Lines formerly of the form '(remote,) = ctx.cluster.only(role).remotes.keys()'
and '(remote,) = ctx.cluster.only(role).remotes.iterkeys()' would fail with
ValueError and no message if there were less than 0 or more than 1 key.
Now a new function, get_single_remote_value() is called which prints out
more understandable messages.

Fixes: 7510
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #186 from ceph/wip-7369

Fix #7369: "sed expression must be raw string"

Merge pull request #188 from ceph/wip-calamari-onefile

Add Calamari test tasks, test script

Merge pull request #192 from ceph/wip-6537-wusui

Readjust the indentation of mon_clock_skew_check.py and mon_thrash.py.

Merge pull request #194 from ceph/wip-6534-wusui

Add docstrings to internal.py

Merge pull request #193 from ceph/wip-6538-wusui

Add doc strings to Swift tests

Merge pull request #187 from ceph/wip-better-debug

Debug output improvements

Add doc strings to Swift tests

Fixes: 6538
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Add docstrings to s3 related tasks.

Fixes: 6536
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix namespace collision

Merge pull request #106 from ceph/wip-mirror

Remove lots of ceph.com hardcoding; default to upstream sources

Don't hardcode the git://ceph.com/git/ mirror

Default to https://github.com/ceph/ but add a ~/teuthology.yaml option

Merge pull request #78 from ceph/wip-6247

Move helper scripts to /usr/local/bin to clean up logs.

Helper scripts live in /usr/local/bin now!

s3tests: extend for multi-region tests

Added code to the s3tests task to extract
multi-region info so that that data
can be added to the S3TEST_CONF file
used to run S3 tests.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #41 from ceph/wip-3791

Various usability and documentation fixes

Merge pull request #40 from ceph/wip-teutharm-wusui

Wip teutharm wusui

Fix some instances where print is being used instead of log

s3/swift tests: call radosgw-admin as the right client

This allows the right region and zone info to be read from ceph.conf

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

s3tests: clone correct branch

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'master' of github.com:ceph/teuthology

Merge remote-tracking branch 'origin/wip-sandon-vm'

Conflicts:
teuthology/lock.py
teuthology/misc.py
teuthology/task/install.py

Merge branch 'wip-centos-rgw'

s3tests: fix client configurations that aren't dictionaries

They're always used as dictionaries later on.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #15 from ceph/wip-ulimits

Reviewed-by: Warren Usui <warren.usui@inktank.com>

enable-coredump -> adjust-ulimits

and set max_files to be big, too!

Merge branch 'wip-teuth4768a-wusui'

Conflicts:
teuthology/task/install.py

Merge branch 'next'

s3tests: add force-branch with higher precdence than 'branch'

This way we can force a branch despite something in overrides.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote branch 'origin/next'

fix some errors found by pyflakes

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

s3tests: revert useless portion of 1c50db6a4630d07e72144dafd985c397f8a42dc5

Perhaps it was attempting to debug something, but it shouldn't have been committed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

rgw tests: remove users after each test

These should all be cleanup up at some point. They're
almost all the same code.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

rgw tests: clean up immediately after the test

There's no need for an explicit cleanup function, so move it back
to where it came from (except in s3roundtrip, which did not have it).

Instead, since these use a nested contextmanager, pass through
and yield to the top-level run_tasks after the nested
contextmanager has finished (and thus run all the cleanup steps
in the subtasks for this test).

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

swift, s3readwrite: add missing yield

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

s3tests, s3readwrite, swift: cleanup explicitly

Cleaning up test dir explicitly after run, so that
consecutive runs don't fail.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge remote-tracking branch 'origin/wip-3634'

Merge branch 'unstable'

Conflicts:
teuthology/task/workunit.py

Install ceph debs and use installed debs

The ceph task installs ceph using the debian
packages now, and all invocations of binaries installed
in {tmpdir}/binary/usr/local/bin/ are replace with
the use of the binaries installed in standard locations
by the debs.

Author:    Sander Pool <sander.pool@inktank.com>
Signed-off-by: Sam Lang <sam.lang@inktank.com>

Replace /tmp/cephtest/ with configurable path

Teuthology uses /tmp/cephtest/ as the scratch test directory for
a run.  This patch replaces /tmp/cephtest/ everywhere with a
per-run directory: {basedir}/{rundir} where {basedir} is a directory
configured in .teuthology.yaml (/tmp/cephtest if not specified),
and {rundir} is the name of the run, as given in --name.  If no name
is specified, {user}-{timestamp} is used.

To get the old behavior (/tmp/cephtest), set test_path: /tmp/cephtest
in .teuthology.yaml.

This change was modivated by #3782, which requires a test dir that
survives across reboots, but also resolves #3767.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

task/swift: change upstream repository url

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'wip-mon-thrasher'

s3tests: fix typo

rgw-logsocket: a task to verify opslog socket works

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

s3tests: run against arbitrary branch/sha1 of s3-tests.git

pull s3-tests.git using git, not http

ceph.newdream.net -> ceph.com

Merge branch 'master' of github.com:ceph/teuthology

github.com/NewDreamNetwork -> github.com/ceph

Add necessary imports for s3 tasks, and keep them alphabetical.

rgw: access key uses url safe chars

Signed-off-by: Yehuda Sadeh <yehuda.sadeh@dreamhost.com>

use local mirrors for (most) github urls

A cronjob on ceph.newdream.net updates these every 15 minutes.  Sigh.

Rename "testrados" and "testswift" tasks to not begin with "test".

Anything "test*" looks like a unit test, and shouldn't be used for
actual code.

testswift: fix config

rgw: add swift task

still not completely working (for some reason it skips all the tests)

s3-tests: use radosgw-admin instead of radosgw_admin

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

s3tests: Clone repository from github.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Move orchestra to teuthology.orchestra so there's just one top-level package.

Callers of task s3tests.create_users don't need to provide dummy "fixtures" dict.

allow s3tests.create_users defaults be overridden

Make targets a dictionary mapping hosts to ssh host keys.

Skip s3-tests marked fails_on_rgw, they will fail anyway.

The shell exits after the command, hence there is no need for pushd/popd.

Add s3tests task.