use itertools for seconds sum

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

tests for the new while helper

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

add a helper for while loops

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

schedule_suite: debug filestore = 20 too

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #205 from kdreyer-inktank/jenkins

jenkins: fail fast during job execution

Merge pull request #206 from ceph/wip-osd-debug

schedule_suite: debug osd = 20 for the time being

schedule_suite: debug osd = 20 for the time being

This will make it easier to chase the first pass of firefly bugs.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #203 from ceph/wip-devstack

Add devstack task

jenkins: fail fast during job execution

Jenkins will dynamically write the commands in its database into a
script and run it with "-x" and "-e" in order to display each step and
fail after the first step fails.

Now that our commands are stored in Git, ensure that we're doing the
same thing.

Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>

Merge pull request #204 from ceph/wip-fix7397-wusui

Do not spawn a parallel task if dictionary entry does not exist.

Add unit test for task.devstack.parse_os_table()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add debug output for reconnect() failures

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add explanations to asserts

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Guard against ctx.config being unset or None

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Do not spawn a parallel task if dictionary entry does not exist.

Fixes: 7397
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Remove unused import

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't run the exercise subtask by default

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a retry to the rbd volume verification

This was running into an auth problem intermittently that Josh Durgin
said is likely not a real problem. Still, try the original call first
and fall back to a different one if it fails.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix incorrect exception name in docstring

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move external docs URLs

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Wait 30s for Keystone to start

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Change usages of StringIO.read() to getvalue()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Source openrc before creating volume

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add a new subtask: smoke()

Currently it creates a volume, then verifies that the volume exists in
Ceph - as opposed to some other place. This is intended to verify that
we've configured DevStack at least somewhat correctly.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Forget about saving the exercise.sh log, for now

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Allow using different devstack branches

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add devstack.exercise subtask

The subtask runs devstack/exercise.sh

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make reconnect() use Remote.reconnect()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move reboot() to teuthology.misc

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add documentation to reboot()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add note about memory requirements

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Simplify create_if_vm's downburst config logic

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move restart_apache() to after the reboot

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make rejoin-stack.sh start screen in detached mode

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Actually call Remote.reconnect()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Tweaks to is_online and reconnect()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add logging calls to each helper function

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Set apache ServerName, reboot, restart devstack

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.reconnect()

Also rename Remote._connect() to Remote.connect() and change its
semantics to match Remote.reconnect()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use new init parameters for Remote

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Make Remote.is_online more accurate

To really know if the connection is active, we need to read something
from the remote host.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Remote instances can now establish SSH connections

Previously, they required an active ssh connection object to be passed
to them. That behavior is still functional.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.is_online property

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add Remote.hostname property

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

get_file() returns a str, not a StringIO...

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use ConfigParser.set() instead of update()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Work around a bug in ConfigParser

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Call seek(0) on StringIO objects before reading

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix typoed virsh call

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Don't pass owner to copy_key()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use get_file() and sudo_write_file()

... instead of copy_file()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Split out key generation

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add update_devstack_config_files()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add set_libvirt_secret() and split out other steps

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Split out devstack-ceph configuration

This is starting to get long, so create configure_devstack_and_ceph()

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Copy ceph auth keys over to devstack node

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add optional 'owner' arg to sudo_write_file()

This gets passed directly to chown as 'sudo chown $owner $file'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Fix linter errors.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add skeleton devstack task

Currently only installs devstack on one node, and creates some pools on
the ceph cluster.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add copy_file() to scp files between remotes

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge pull request #201 from ceph/wip-6531-wusui

Fix formatting and add docstrings to ceph_manager.py

Add back in print in ceph_manager that the last checkin accidentally removed

Fix formatting and add docstrings to ceph_manager.py

Fixes: 6531
Signed-off-by: Warren Usui <warren.usui@inktank.com>

jenkins: add debugging output to shell script

When we migrated the Jenkins commands into Git, the exact commands are
no longer visible unless we run with "-x".

Add "set -x" to the script to enable debugging.

Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>

Merge pull request #195 from ceph/wip-7003-wusui

Add docstrings to the orchestra code.  Also fix minor formatting issues

Merge pull request #186 from ceph/wip-7369

Fix #7369: "sed expression must be raw string"

Fix syntax error in first docstring added.

Merge pull request #190 from kdreyer-inktank/jenkins

add jenkins-pull-requests-build script

Merge pull request #199 from ceph/wip-6540-wusui

Add docstrings.  Fix formatting issues for rgw.py

Merge pull request #196 from ceph/wip-6541-wusui

Fix docstrings in radosgw-admin-rest.py

Merge pull request #197 from ceph/wip-6535-wusui

Fix docstrings for task/blktrace.py, peering_speed_test.py, proc_thrashe...

Merge pull request #198 from ceph/wip-6539-wusui

Fix docstrings and minor formatting in task/ssh_keys.py

Merge pull request #200 from ceph/wip-6532-wusui

Fixed task/args.py docstrings.

Revert "Show hostname instead of IP in errors"

This reverts commit eafeece243607eef8209eaa8c08eecc7d1098eb1.

I was having problems with my newly-created devstack task hanging
inexplicably. It turns out this commit was responsible.

add jenkins-pull-requests-build script

Move our Jenkins pull requests build script out of Jenkins' database and
into Git so that we can have greater visibility on future changes.

Going forward, we will modify the teuthology-pull-requests Jenkins task
to only run this script.

Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>

radosbench: simplify pool creation and add ec_pool support

Signed-off-by: Samuel Just <sam.just@inktank.com>

task/: add ec_pool and append to rados.py

Signed-off-by: Samuel Just <sam.just@inktank.com>

Revert "Capture stderr and include in CommandFailedError"

This reverts commit d0a434923100dd16548e5f76eaffa5462c4b91d4.

filestore_idempotent: filestore -> objectstore path change

Signed-off-by: Sage Weil <sage@inktank.com>

thrashosds: allow primary-affinity thrashing to be disabled

Signed-off-by: Sage Weil <sage@inktank.com>

thrashosds: randomly adjust primary_affinity

Signed-off-by: Sage Weil <sage@inktank.com>

ceph.conf: mon osd allow primary affinity = true

Signed-off-by: Sage Weil <sage@inktank.com>

Fixed task/args.py docstrings.

Add docstrings.  Fix formatting issues for rgw.py
Fix s3tests calls for rgw-logsocket.py.

Fixes: 6540
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix docstrings and minor formatting in task/ssh_keys.py

Fixes: 6539
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix docstrings for task/blktrace.py, peering_speed_test.py, proc_thrasher.py and
recovery_bench.py.

Fixes: 6535
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Fix docstrings in radosgw-admin-rest.py

Fixes: 6541
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #188 from ceph/wip-calamari-onefile

Add Calamari test tasks, test script

Add docstrings to the orchestra code.  Also fix minor formatting issues
(mostly column start locations).

Fixes: 7003
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Review: switch to selecting remotes by roles in the 'normal' way

This buys us, also, the ability to select remotes by rule, i.e.
"any osd or mon gets calamari-agent installed"

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Review: refactor packaging routines to common file

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Merge pull request #192 from ceph/wip-6537-wusui

Readjust the indentation of mon_clock_skew_check.py and mon_thrash.py.

Merge pull request #194 from ceph/wip-6534-wusui

Add docstrings to internal.py

Merge pull request #193 from ceph/wip-6538-wusui

Add doc strings to Swift tests

Merge pull request #191 from ceph/wip-7002-wusui

Added some docstrings.

Merge pull request #187 from ceph/wip-better-debug

Debug output improvements

Add docstrings to internal.py

Fixes: 6534
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Add doc strings to Swift tests

Fixes: 6538
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Readjust the indentation of mon_clock_skew_check.py and mon_thrash.py.
Added docstrings.

Fixes: 6537
Signed-off-by: Warren Usui <warren.usui@inktank.com>

Review: Rename calamari_testdir/ to calamari/