Added version v0.80.6

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/singleton-nomsgr/all/objectstore: clean up

ceph_test_objectstore scribbles in $CWD.  Keep it inside $TESTDIR and
clean up when we are done.

Fixes: #8395
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 5be30f0d9bba46684ecf37310a6d09ae4f0ac528)

suites/fs: update to latest mdtest

They appear to have (accidentally?) fixed whatever
was crashing.

Fixes: #9177
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit bda325b33b036ac110bbc65f62019aa76d05ae3b)

ceph_manager: reset timeout if we are making progress in wait_for_recovery

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 8d9d724b7f2f6c2984b0ee58a5a91f79820719f2)

thrashers/mapgap.yaml: ignore osd_map_cache_size warning

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 77670aeadf9f2f43db5858ad096016353ce578bb)

rados/singleton/all/thrash_cache_writeback_forward_none: thrash cache pool drain

Enable/disable cache pool against a running workload.

Backport: firefly
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f8282df84d5ce760653ff1d3a9d8ea4e1391fba0)

set boto timeout, too, for s3tests

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9e5397237857d9015ac8c9e908f0cbeb1777576d)

Conflicts:

machine_types/vps.yaml

tasks/s3tests: push boto config with idle_timeout setting

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9d466aa110e293cb88c1f61cc7391d59b64b553b)

thrashosds: increase osd revive timeout (75s -> 150s)

This is needed when running valgrind.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 12a391ea011a772dbcc720b825f7b8dd8ecbf702)

use 'mon create-initial' always

But don't error if it fails, as this would mean that the monitors
are just taking longer to form quorum. Go and try the next block which will
wait up to 15 minutes for a successful gatherkeys to happen (that only works
if monitors have formed quorum).

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
(cherry picked from commit 70a1f18adfdf673b983cc7f5f2c78abb899001c7)

ignore errors on informational service status

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
(cherry picked from commit 5b946e1a6df8b7f80001f5176435eb36b12711e3)

use the right syntax for RHEL/CentOS distros to check for ceph status

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
(cherry picked from commit 4b15d0118ec000fe1e947131a62ef9990b23e9f4)

rgw: httpd instead of httpd.worker

httpd exists on rhel 6.5 too ...

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f7b32bcc318c4a700628d87826080a043f89a22c)

rgw: need alll of mod unixd, version, authz

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6392758f1bf503c3bbb2b3ad2f2ee9e6f79d6841)

tasks/rgw: include mod_authz

As per http://www.webhostingtalk.com/showthread.php?t=1173594

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 27b7eceeae1e48d0ad8dd820c3b84442bfffa406)

tasks/rgw: get mpm_event frmo mods-available, not mods-enabled

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2aae91929f720c54f406b7612631d82b27c98fe6)

Add extra conf for Apache 2.4

Inside a conditional to affect only 2.4, set User, Group, and the
module config to load mpm_event.  This is normally done with the
default configuration files, but since this abbreviated conf bypasses
those, we must set them here.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 9de5bd1d2302907c59594eb4df8b4d954d5e8e94)

Merge pull request #97 from dachary/firefly

rgw.yaml: keep the client.0 key even if empty (firefly)

rgw.yaml: keep the client.0 key even if empty

Because rgw.py iterates over it to run the rgw server. If it is removed
the rgw servers are not run and all fails.

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 5cfcb76860302386375ca0cf1c7a960ef6659a75)

Merge pull request #93 from ceph/wip-9126-firefly

rgw: s/idle_timeout/default_idle_timeout/

rgw: s/idle_timeout/default_idle_timeout/

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from da00662191e9ec173be50b551253f4460f369f9da)

rgw: add default_idle_timeout to allow override

Globally overriding the rgw idle_timeout is not possible because it it
needs to be done on a per client.0, client.1, etc. basis. Add the
default_idle_timeout key to the rgw config : it defaults to the
previously hardcoded default (30) and can be changed via the override.

The existing tasks that were previously overriding the idle_timeout on a
per client basis are changed to use the default_idle_timeout instead for
consistency and to allow a global override.

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 54a7298cdd8d40aaf79615a1f580b66f83e48419)

Conflicts:
machine_types/vps.yaml
suites/smoke/verify/tasks/rgw_s3tests.yaml

ceph_manager: longer osd restart timeout

We bumped this on master and dumpling already

Signed-off-by: Sage Weil <sage@redhat.com>

Use Remote.user

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use teuthology's DaemonGroup

It has timeout support.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Use newer orchestra API

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Update module references

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Merge branch 'teuth_firefly' into tasks_firefly

Import teuthology tasks (firefly branch)

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Move write_secret_file() into task_util/kclient.py

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add wait_until_fuse_mounted()

This was removed in a refactor in master, but it would take lots of work
to backport the entire refactor.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

ignore errors if 'ps aux' fails

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
(cherry picked from commit f9aa9c1e647dc3376a33d0f7147501bffcddf838)

add another call to use "ps aux"

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
(cherry picked from commit 4686115c4830939ea6c9d7a27fd89d7cec9d4d5d)

tell us if you are really not running any more

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
(cherry picked from commit ec342d814676d08eb833f478ac7ffdb21cc99c06)

Merge pull request #74 from ceph/wip_fix_fuse_upgrade-firefly

Yet anothe indentation fix

Yet anothe indentation fix

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #73 from ceph/wip_fix_fuse_upgrade-firefly

Fixed typos

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Fixed typos

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #72 from ceph/wip_fix_fuse_upgrade-firefly

Fixed to run blogbench.sh on client:0

Fixed to run blogbench.sh on client:0

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

valgrind: ignore ec plugin factory leaks

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7c3fc404d6a84c6601400e364bd98ea89df13922)

use notcmalloc builder for valgrind leak checking

This reverts 453d72b294af6d3db1ae7a212ba9b43209d69f26.  It turns out that
tcmalloc defeats the leak checking in valgrind.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 279f39d4c18004ed7b005b430d3862b8b2ec2f73)

Merge pull request #71 from ceph/wip_fix_fuse_upgrade-firefly

Fixed miss-config for ceph-fuse

Fixed ceph-fuse: [client.0] syntax

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed miss-config for ceph-fuse

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

ceph_manager: increase osd revival timeout from 75 -> 150 seconds

Saw this fail on a firefly run just bc of valgrind slowness.  Make the
timeout higher!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f84458bd93b74873687f0099850ef5cb1e3de1b5)

Merge pull request #70 from ceph/wip_add_v0.80.5

Added new ceph version

Merge pull request #69 from ceph/wip_add_new_client1

Added client.1 in order to separate rgw tests to avoid failures on creat...

Reviewed-by: Sage Weil <sage@redhat.com>

Added new ceph version

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added client.1 in order to separate rgw tests to avoid failures on creating a directory on top of exiting directory

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

ceph_deploy: use new ship_utilities location

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 828bb2a6ea934b8788570ca3f9871a637dbd3c70)

move ship_utilities to install task (from ceph test)

No need to install ceph just to get daemon-helper, adjust-ulimits, etc..
These utilities are used by lots of tasks, not just ceph.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2dad906f4ac4658381f3984a8fe99f696f710fac)

rados/singleton-nomsgr/all/alloc-hint: fix config syntax

Put in [osd] section.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2b4f7402007e6a81dce10f3b3b25f8c410070195)

Merge branch 'firefly' of https://github.com/ceph/ceph-qa-suite into firefly

Added distros

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/verify: increase api tests timeout

This has been timing out for weeks to months (hard to say since before that
we were consistently hitting an OSD crash on shutdown).  The runs I've looked
at are making progress and seem fine ... just slow because of valgrind and
because of thrashing.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 175057f3bceb50623f6bfeded3fe0b9257edea7d)

Merge pull request #65 from ceph/wip_fix_missed_ceph_task

Fixed missing ceph task

Fixed missing ceph task

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/singleton-nomsgr/all/alloc-hint: enable xfs extsize for test

This is off by default; enable it for this test.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 02a6c917eca6237118163ecd49ed7ca56bda275b)

Merge pull request #63 from ceph/wip-rgw-firefly

only test rgw on new firefly releases to avoid old bug

upgrade/firefly: only test rgw on newer firefly releases

This avoids #8311 by not testing rgw when the blank pool bug was present.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: set data extra pool correctly

When configuring multiregion zone, set the data extra pool

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit 63fb271b7ecd9c1ae16d6fff3a1630df2dd09e08)

Merge pull request #58 from ceph/wip_add_v0.80.3

Added v0.80.3 to the mix

Removed whitelist lines since it's not needed for version v0.80.3

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added v0.80.3 to the mix

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge branch 'wip_stable_upgrade2' into firefly

upgrade/firefly: don't use overrides

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: ignore mon scrub errors from early firefly versions

See #8815

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: do intermediate s3tests on original test set

The later s3-tests fail on early firefly point releases.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: upgrade to firefly, not dumpling

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: ignore crush tunables on upgrade

..then adjust them

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: fix final steps

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: upgrade mds after osd

Signed-off-by: Sage Weil <sage@redhat.com>

suites/upgrade/firefly: test rgw at the end

...and do all final tests for all runs

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/firefly: consolidate into a single collection

Signed-off-by: Sage Weil <sage@inktank.com>

Added upgrade tests - stable

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added requested comment.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Remove unused variables and functions.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Support installing ceph kernels from gitbuilder on rpm machines.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

tasks: Use '_' instead of '-' in names

Python modules are not allowed to contain hyphens
in the name.  Using hyphens only works here because
we're using low level __import__.

Change run_tasks to replace '-' with '_' when reading
configs, and rename the task modules to have valid
python module names.

Aside from general hygiene, the motivation to do this
is to allow one task to build on code in another
task by importing it.

Signed-off-by: John Spray <john.spray@inktank.com>
(cherry picked from commit 79dfe2cbefbbf14b69ae10c44a89a4fe45fedbe5)

Merge pull request #258 from ceph/wip-fix-format-samba

Fixed formatting and added a better variable for time out
Conflicts:

teuthology/task/samba.py

Merge pull request #251 from ceph/wip-8295

rgw: fix indentation for cache_pools

rgw: fix indentation for cache_pools

Fixes: #8295
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #250 from ceph/wip-fix-thrasher

ceph_manager: reset osd weights to 1 when waiting for clean

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

ceph_manager: reset osd weights to 1 when waiting for clean

If we leave the weights adjusted, we can get PGs stuck in a remapped state
because we are probabilistically rejecting placement.  Avoid this by
setting on in osds to 1.0 before waiting for clean.

Signed-off-by: Sage Weil <sage@inktank.com>

Restrict paramiko to old versions for now

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Revert "Handle raw data I/O."

This reverts commit 257e1459fa064701d789f0ad54384bb80b45e6d9.

Revert "Clean up remote.py and misc.py changes."

This reverts commit 74eff43ee1a2b73159277370cfa9d194e42bf49c.

ceph_manager: fix float stringification

Signed-off-by: Sage Weil <sage@inktank.com>

ceph_manager: fix typo

From ce7fa1839f4b3e3db675b2d68a2bb57849f58c1e. Tested this time.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #248 from ceph/wip-thrash-osd-weights

ceph_manager: randomly reweight in osds

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

ceph_manager: randomly reweight in osds

Signed-off-by: Sage Weil <sage@inktank.com>

Bump paramiko to 1.12.0

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

rgw: option to create a cache pool

64mb for now!

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #247 from ceph/requests-sessions

Use a requests.Session object for retries instead of safe_while

Add branch name to job config

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add suite name to job config

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add /build and /*.yaml to gitignore

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Calculate a timeout to use based on queue size

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>

Add --runs, to print only run names

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>