erasure-code: enable ec-rados-default.yaml

The ec-rados-default.yaml started with:

  workload:
    sequential:

which is only suitable for suites/upgrade/giant-x/parallel/2-workload/sequential_run/ec-rados-default.yaml

because suites/upgrade/giant-x/parallel/1-giant-install/giant.yaml has

   - parallel:
      - workload
      - upgrade-sequence

The same file was included in contexts where the parallel task was not
used and the workload did not run:

  ./suites/upgrade/firefly-x/stress-split-erasure-code/5-workload/ec-rados-default.yaml
  ./suites/upgrade/giant-x/stress-split-erasure-code/5-workload/ec-rados-default.yaml
  ./suites/upgrade/giant-x/stress-split-erasure-code-x86_64/5-workload/ec-rados-default.yaml

The ec-rados-default.yaml is modified to be a task instead of a
sequential task in a parallel tasks. The ec-rados-sequential.yaml is
added and is linked in
suites/upgrade/giant-x/parallel/2-workload/sequential_run instead of ec-rados-default.yaml.

http://tracker.ceph.com/issues/11221 Fixes: #11221

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #375 from dachary/wip-rgw-regional-summary

ensure summary is looked for the user we need (part 2)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ensure summary is looked for the user we need (part 2)

Move the get_user_summary(out, user) logic to util.rgw so that it can be
shared between radosgw_admin_rest.py and radosgw_admin.py and modify
them accordingly.

http://tracker.ceph.com/issues/11180 Fixes: #11180

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #373 from ceph/wip-10861

fs: fix up dd testing again

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Travis Rhoden <trhoden@redhat.com>

Merge pull request #371 from ceph/wip-iso-style

Add support for ISOs to tasks/calamari_setup

fs: fix up dd testing again

We need to change into the directory during the same shell session as we actually
run the dd...

Fixes: #10861
Backport: hammer, giant, firefly, dumpling

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

calamari_setup: oops: ice-tools is still where it was on github.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #372 from ceph/wip-11180

ensure summary is looked for the user we need

ensure summary is looked for the user we need

Signed-off-by: Alfredo Deza <adeza@redhat.com>

calamari_setup: add -y to yum localinstall icesetup

also fix a few flake8 things

Signed-off-by: Ubuntu <ubuntu@vpm047.front.sepia.ceph.com>

calamari_setup: handle iso, preserving support for tar.gz

Signed-off-by: Dan Mick <dan.mick@redhat.com>

calamari_setup: small comment/logmsg fixes, correct git user path

Signed-off-by: Dan Mick <dan.mick@redhat.com>

calamari_setup: make iceball fetch/creators return path to iceball

Also, widen the concept of "iceball" to include both iso and tar.gz

Signed-off-by: Dan Mick <dan.mick@redhat.com>

calamari_setup: Refactor/simplify in preparation for iso support

Signed-off-by: Gregory Meno <gmeno@redhat.com>

Merge pull request #364 from ceph/wip-11043

Implemented "exclude_packages" feature #11043
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Implemented "exclude_packages" feature #11043

Fixed per Josh's comment

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Implemented "exclude_packages" feature #11043

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #357 from ceph/wip-10949

tasks/cephfs: clean up core on deliberate crash

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: clean up core on deliberate crash

To avoid internal.coredump task synthesizing a failure
during teardown from the core we left behind.

Fixes: #10949
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #362 from ceph/wip-11020

Control the nfs service correctly on rpm-based systems

Control the nfs service correctly on rpm-based systems

This fixes issue #11020

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #360 from ceph/wip-11013

Fixes #11013, use time.sleep instead of manager.sleep which isn't there.

Fixes #11013, use time.sleep instead of manager.sleep which isn't there.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #356 from ceph/wip-10948

tasks/cephfs: don't run iptables in parallel

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #358 from ceph/wip-10712

tasks: fix intermittent failure in TestFlush

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #334 from ceph/wip-fuse-umount

tasks/cephfs: fix fuse force umount (again)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks: fix intermittent failure in TestFlush

This was an overly strict success condition: the
flush operation doesn't promise to leave you an empty
journal, it promises that anything in the journal
before the flush will be flushed.

Fixes: #10712
Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: don't run iptables in parallel

Where multiple MDSs were on the same node, trying
to concurrently update their firewall state was
causing an exception because the iptables command
errors out if another instance is already running.

Fixes: #10948
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #354 from ceph/wip-multi-versions

Changed suite name from multi-versions to client-upgrade

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Changed suite name from multi-versions to client-upgrade

Moved it under upgrades
Replaced rbd workunit

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

tasks/cephfs: fix/improve fuse force umount (again)

teuthology helpfully escapes things for us so
the \; didn't need the backslash.  The logic
was still falling over in some cases too.

Additionally, make the FUSE /sys/ abort operation
more surgical by working out the connection name
of our own mount during mount().

Signed-off-by: John Spray <john.spray@redhat.com>

msgr: move async msgr tests to singleton-nomsgr

Introduced: e92d2e4ca0013fa379fdf5830d4fe28abf65c33b
Fixes: 10918
Signed-off-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

exclude the virtualenv from being linted in tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b944b06dc2479d77d6efac4f13932cb102ddfe1c)
(cherry picked from commit 6de0a684d5b4460390e610806201e3eaf05feb6e)

fix path for tox.ini to work on ceph-qa-suite only

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4eb1787280f9f128cb9352ecbe8957a4be94fff9)

Merge pull request #347 from ceph/wip-10954

impelement a tox job to catch bad linter errors

remove uneeded variable assignment for call

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove redefined StringIO import

Signed-off-by: Alfredo Deza <adeza@redhat.com>

stdin is no longer a kwarg

Signed-off-by: Alfredo Deza <adeza@redhat.com>

add the log object to ceph_manager

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove unused import from repair test

Signed-off-by: Alfredo Deza <adeza@redhat.com>

add missing mon object to osd_failsafe

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove unused imports from populate_rbd

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove unused imports from peering_speed

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove unused variable assignment

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove unused import in calamari_setup

Signed-off-by: Alfredo Deza <adeza@redhat.com>

fix undefined var in divergent_priors

Signed-off-by: Alfredo Deza <adeza@redhat.com>

fix lint errors on ceph_fuse

Signed-off-by: Alfredo Deza <adeza@redhat.com>

initial tox.ini to run flake8

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ignore tox hidden dir

Signed-off-by: Alfredo Deza <adeza@redhat.com>

rbd: fix copy-on-read setting

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

rbd: add merge_diff test

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #346 from ceph/wip-librbd-default-config

suites/rbd: add object map tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

suites/rbd: add object map tests

Enable the object map feature for CLI tests and librbd API tests.
Moved the CLI tests to a new group to exercise old format, basic
new format, and new format w/ object map.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph: ugh fix syntax

Signed-off-by: Sage Weil <sage@redhat.com>

ceph: fix ps axuf lsof line

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #344 from tchaikov/log-exec-from-thrasher

Thrasher: log backtrace of thrown exception

Reviewed-by: Samuel Just <sjust@redhat.com>

Thrasher: log backtrace of thrown exception

* add a wrapper to log uncaught exception to self.logger, greenlet also
  prints the backtrace and exception to stderr, but teuthology.log does
  not capture stderr. so we need to catch them by ourselves to reveal
  more info to root-cause this issue.
* log uncaught exception thrown by Thrasher.do_thrash() to self.log.

See: #10630
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #343 from ceph/wip-10680

Use an NFS mount instead of virtio-9p in the qemu task

Use an NFS mount instead of virtio-9p in the qemu task

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #342 from ceph/wip-psauxf

ceph: ps axf too before lsof

ceph: ps axf too before lsof

Specifically, I want to know *who* is running the ceph-osd that is
holding the files open.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #341 from ceph/wip-path

workunit: include /usr/sbin in the PATH for all commands

Merge pull request #340 from ceph/hadoop-workunits

Hadoop workunits

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

workunit: include /usr/sbin in the PATH for all commands

Fixes: #10600
Backport: hammer, giant, firefly, dumpling

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #274 from ceph/wip-promote

rados/thrash: add readproxy facet

Merge pull request #313 from armaseg/master

teuthology: some suites still use http://ceph.newdream.net

hadoop: fixup for new replication test

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge remote-tracking branch 'gh/hammer'

Fixed typoes

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit e2794173026c196a30ee042e6e77c4a278267dc2)

Fixed typo causing "too many values to unpack" error

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit fdd175514738fd4f6cf3dea6d56ffb2eda78d64b)

Fixes #10704

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit a9252436e81ab7d21110ffbe2f6b469f08e47d82)

Fixed typoes

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit e2794173026c196a30ee042e6e77c4a278267dc2)

Fixed typo causing "too many values to unpack" error

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit fdd175514738fd4f6cf3dea6d56ffb2eda78d64b)

Fixes #10704

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit a9252436e81ab7d21110ffbe2f6b469f08e47d82)

Merge pull request #337 from ceph/wip-umount

lsof on umount failure; catpure logs for aggressively

ceph: lsof if umount fails

Signed-off-by: Sage Weil <sage@redhat.com>

ceph: archive logs at very end

This ensures that we still gather the logs even if the other nested tasks
throw an exception in the finally block.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #339 from ceph/wip-10652-2

Fixes #10652

Fixes #10652

Removed firefly-giant dir as it's not needed
Fixed branch assignment in dumpling-x

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixes #10652
Removed firefly-giant dir as it's not needed
Fixed branch assignment in dumpling-x

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit a91994f09201fdf497798d2c8051b27189b5e486)

hadoop: add terasort task

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge branch 'hammer' of https://github.com/ceph/ceph-qa-suite into hammer

Fixes #10652

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixes http://tracker.ceph.com/issues/10652

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #333 from ceph/teuthology-update

Mirror the supported distros used in nightlies in the teuthology suite

Mirror the supported distros used in nightlies in the teuthology suite

I decided not to symlink because I'm using a file name convention to
easily filter out distros that need a vps.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Fixes #10869, added {role} to the dir name.

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
Fixed typo

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Thrasher: Fix log message

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #330 from ceph/teuthology-cleanup

Make jobs in the teuthology suite that install ceph run the 'tests' task

Run the 'tests' task for jobs in the teuthology suite that install ceph

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Ignore *.pyc files and __pycache__ folders

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #329 from ceph/wip-10828

normalize acl responses from rgw

rename test to tests, follows best practices for tests dirs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

create a new get_acl helper to ensure proper xml is compared

Signed-off-by: Alfredo Deza <adeza@redhat.com>

tests for the new acl helper in radosgw_admin

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #328 from ceph/wip-10736

Created a new suite to test teuthology

Created a new suite to test teuthology

This suite locks machines of varying distros and runs the 'tests' tasks
against them. This will test both vps locking / provisioning and
baremetal locking.

Also, we test the chef and install tasks with ubuntu and rhel.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #327 from ceph/wip-10702

tasks/cephfs: fix fuse force unmount

tasks/cephfs: fix fuse force unmount

This was broken in the case of multiple
mounts, and in the case of stuck mounts.

Fixes: #10702
Signed-off-by: John Spray <john.spray@redhat.com>

Corrected "branch" to be outside of the 'clients:' section

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>