Drop chef from the teuthology suite

Chef has been replaced by ansible, which will be added to jobs at
schedule time.

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit afe148820a96e503fec65b902cdbec33fb407273)

Drop chef from tgt suite

Chef has been replaced by ansible, which will be added to jobs at
schedule time.

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 5b57b81e90393d42494c7b02fb263f1601421f16)

Merge pull request #494 from ceph/wip-hammer-2

Changed s3tests ref to firefly

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Changed s3tests ref to firefly
Per IRC chat by Yehuda, Josh and Dan:

"the purpose of firefly-original in general was for upgrades from prior point releases of firefly, where newer tests on the firefly branch would fail
it shouldn't be used if it's just a firefly->hammer upgrade though, the plain old firefly branch should be fine for that"

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Fixes #11570
Removed test_cls_rbd.sh per Josh's suggestion
Removed 'branch: hammer' from 3-upgrade-sequence/upgrade-*

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 52fa9a32c8dd5cf527838cd92ca8eec312ab3d41)

Merge pull request #488 from ceph/hammer-11756

tasks/cephfs: mount fusectl before listing fuse connections

Reviewed-by: John Spray <john.spray@redhat.com>

tasks/cephfs: mount fusectl before listing fuse connections

Fixes: #11756
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit f6bba29fe9c54f6d4909833eab855450879ee192)

Merge pull request #482 from ceph/wip-fuse-umount-hammer

tasks/cephfs: fix/improve fuse force umount (again)

Removed rhel* configurations with a goal that they will be covered in the Octo lab
and won't be used by vps runs in Sepia lab

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit adfa1ee393c0ea0a36c0dfb8d7fda1a82f4b29ea)

tasks/cephfs: fix/improve fuse force umount (again)

teuthology helpfully escapes things for us so
the \; didn't need the backslash.  The logic
was still falling over in some cases too.

Additionally, make the FUSE /sys/ abort operation
more surgical by working out the connection name
of our own mount during mount().

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit fc4e90930b2c6ba0d395ae7d6edb8f8daba3fd64)

Merge pull request #469 from ceph/wip_remove_test_cls_rbd_2

Wip remove test_cls_rbd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Removed test_cls_rbd.sh as it conflicts with giant osds

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Initial checking for #12023
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 029fdfd741a280bf5f2712fcd706f8d80a199782)

rbd: use formatted output test from the hammer branch

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Added script to schedule 1/14th part of rados

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit a7064a7b3d8a865dead3800a4d97bd70da6dec33)

Merge pull request #439 from ceph/wip-11594

Make sure that ulimits are adjusted for ceph-objectstore-tool

Make sure that ulimits are adjusted for ceph-objectstore-tool

Fixes: #11204
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e6ce90fdb16dcda6902ecf7857d96a86d6e3fbe5)

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh re: #11547
Removed per Josh's suggestion fs as redundant and renamed rados to bacis

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit dcfd6a3b099fd0e14fbfd47a8678a411387e01da)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #432 from dachary/wip-11429-hammer

suites/rados: add test for 11429 (hammer)

Reviewed-by: Samuel Just <sjust@redhat.com>

Initial check in for upgrades/hammer stable upgrade suite

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 8c823e7813d23f71bb8a7586c968c3e035f1d4b2)

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

squash: use sleep instead of utility.sleep

(cherry picked from commit cc107525bca02ca44fabe8513203680e0d1e422b)

squash: use != not is not in radosbench

(cherry picked from commit 91bc0e91864903f6635879432e6dc7a0b36c76c2)

squash: remove utility, moved to teuthology.git

(cherry picked from commit 73577f1077dcfb059d7a96d7502a147a943cd261)

squash: ceph_manager: add utility_task doc string

(cherry picked from commit 2a60852a1d0b0b7980004cf262d4af8ea4011667)

suites/rados: add test for 11429

This patch also adds some convenience facilities for making
some of the ceph_manager methods into tasks usable from a
yaml file.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 015ed70f8a46a44e35433d1f701e37f68af31cf9)

Added hammer-client-x to the mix
Renamed dirs to better reflect what is being tested
Removed old dirs
Added logging
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 3c8730d035a90f221c05526983c40f0a11da1ba6)

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #430 from ceph/wip-11495-hammer

#11495/hammer: backport ceph_deploy task fix

Merge pull request #429 from dachary/wip-10712-hammer

tasks: fix intermittent failure in TestFlush

Reviewed-by: John Spray <john.spray@redhat.com>

tasks/ceph_deploy: remove some dead code

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 99951f8150f1487c3bad47c2b199a137226055cb)

tasks/ceph_deploy: fix for multiple mons

Now that service IDs are modified during run, we have
to avoid repeatedly evaluating first_mon for where
to run ceph_deploy, as the answer will change.

Fixes: #11495
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit dbe2ad24212b9ff9a183d931402a49b544307e00)

tasks: fix intermittent failure in TestFlush

This was an overly strict success condition: the
flush operation doesn't promise to leave you an empty
journal, it promises that anything in the journal
before the flush will be flushed.

Fixes: #10712
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d20d9238cd59bca002d682cbcdc444e15e856de1)

knfs: remove the MDS debugging

It doesn't fit on the typica boxes with debugging, and we probably don't need
it for any NFS issues which do arise...

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 47fff708f936b7380446a4452a9f69f4d1c93c05)

Removed redundant tests issue #11520
Reduced nodes to be locked to two

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 8ab46b42c5ae387141917bb8eff35960e9c45699)

Merge pull request #424 from ceph/wip_ceph_deploy

Removed unneeded workloads

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Removed unneeded workloads

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #400 from ceph/wip_ceph_deploy

 Wip ceph deploy

Merge pull request #418 from ceph/hammer-10624

[hammer backport] task/samba: ignore return code of fuser/losf

task/samba: ignore return code of fuser/losf

if mount point is not used by anyone, both fuser and losf return 1

Fixes: #10624
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit c049387b7b0832e78a4a1370691798f726985799)

Merge pull request #397 from ceph/wip-11316-hammer

hammer: fix ceph-deploy task with CephFS (#11316)

tasks/ceph_deploy: configure CephFS

This test apparently had not been touched since
"fs new" was added.  In addition to calling
Filesystem.create:
 * modify the get_nodes_using_role
   function to modify ctx.cluster.remotes so that the
   service IDs match what ceph-deploy will set
 * log exceptions during ceph_deploy setup, as otherwise
   they can get lost if another exception occurs during
   teardown (so that it's all easier to debug).
 * default to passing --dev=master during install, so
   that we don't error out horribly when run without
   an explicit branch set (e.g. when run outside
   scheduled suite)

Fixes: #11316
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7c2e6cdc6cd3d02969d3494d120b12daaa13efa8)

tasks/cephfs: cope with missing ctx.daemons attr

...so that we may be used with the ceph_deploy task
that doesn't set that up.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f36011051cabb8e7b06194c778e640e98d328e8a)

tasks/cephfs: tweak use of mon for admin commands

... s/mon_remote/admin_remote/ and allow caller to pass
in which remote they want to use for that.  Enables use
with ceph_deploy task which does not give admin keys
to mons.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8b61310ce3eedeb8ac60446933cd5db548de6434)

tasks/ceph: refactor legacy FS configuration check

Move up into Filesystem so that this can be used from
the ceph_deploy task as well.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit ea7c39222ae084c90b5a8510ce5af35a2f75a6e3)

suites/fs: update log whitelist

These ones had been passing because of #11177,
now that that is fixed they require an update.

Fixes: #11300
Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit fa2a86196c3ed39c1726be689e42ed5d4aae197a)

Merge pull request #410 from ceph/wip-hammer-git

git.ceph.com (hammer)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

git.ceph.com (hammer)

Signed-off-by: Sage Weil <sage@redhat.com>

Reduced ammount of nodes from 4 to 3
Added logging
Changed replica "osd pool default size: 2"

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

smoke: white marked down on python test

Backport: hammer
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8cc4a5b0ed21b8ddef7e304d02e903d960b8bcfb)

Removed per #11070 and resolves #11043

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 8a0de9a8ba18b8761f1612e4cdc5a306d320e30d)

Reduced ammount of nodes from 4 to 3
Changed default 'replica' to 2
Reduced osd to 2 per node
Added logging

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

msgr: move async msgr tests to singleton-nomsgr

Introduced: e92d2e4ca0013fa379fdf5830d4fe28abf65c33b
Fixes: 10918
Signed-off-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e9d713b75182292c7b14c862bdeb92279ebd6b32)

Merge pull request #391 from ceph/wip-11306

Fixed #11306 Whitelist WRN "failed to encode map"

Reviewed-by: Samuel Just <sjust@redhat.com>

Fixed #11306 Whitelist WRN "failed to encode map"

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/thrash*: make scrubs happen a lot

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 4d8bc2127f753fb5b935af6c3607e39b077e83d2)

(scrub|rados)_test: tolerate best guess digest errors as well

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 9175f88943d2ea8382e999b7b0bee0b0352f8aef)

rados_python: whitelist wrongly marked me down

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 90393886fcf7f0a72829605ad4e52bfb7dcdf14b)

fs: fix up dd testing again

We need to change into the directory during the same shell session as we actually
run the dd...

Fixes: #10861
Backport: hammer, giant, firefly, dumpling

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 8aeacaa59f32b15aa95a529d1040cb73cbd7efcc)

tasks/watch_notify_same_primary: wait for watch before notify

Make sure watch is done registering and ready before sending the
notifies.

Fixes: #10634
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit bafe87a8e51cdedad84ac6ea8ecd6504a7d9dd2e)

ensure summary is looked for the user we need (part 2)

Move the get_user_summary(out, user) logic to util.rgw so that it can be
shared between radosgw_admin_rest.py and radosgw_admin.py and modify
them accordingly.

http://tracker.ceph.com/issues/11180 Fixes: #11180

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 97e6d808f086834f2350084d5de36be285ad2bde)

Merge pull request #377 from ceph/wip-11166

Fixes #11166, whitelisted 'Missing health data for MDS'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Fixes #11166, whitelisted 'Missing health data for MDS'

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #374 from ceph/wip-11139

ceph_manager: Check for exit status 11 from ceph-objectstore-tool import

ceph_manager: Check for exit status 11 from ceph-objectstore-tool import

Fixes: #11139
Signed-off-by: David Zafman <dzafman@redhat.com>

ensure summary is looked for the user we need

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f7ea0a424a53bf68cad4cd4c7c78a07401dd6851)

Implemented "exclude_packages" feature #11043

Fixed per Josh's comment

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 4250e22dde45c97ad24793f13fa41e95f16d03a4)

Merge pull request #363 from ceph/hammer-backports-hadoop

hadoop: fixup for new replication test

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

hadoop: fixup for new replication test

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
(cherry picked from commit 6a348ae1870eb4e22a35f4ea86d897fb6d960b88)

Control the nfs service correctly on rpm-based systems

This fixes issue #11020

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 2e4d884fdaabbeca67b519537a80166d3cc99fa3)

Fixes #11013, use time.sleep instead of manager.sleep which isn't there.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 811c02e3922c4ae29c09bb5f7b48840d0eced951)

Use an NFS mount instead of virtio-9p in the qemu task

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 93f2bea3530a523c7b2ac0575033697cea893f2d)

workunit: include /usr/sbin in the PATH for all commands

Fixes: #10600
Backport: hammer, giant, firefly, dumpling

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 0e53f5f38b0cb06a6f4989e622a279c8ba9104eb)

rename test to tests, follows best practices for tests dirs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7b2062602f7d3de206f4660c3e2847f86ac41e24)

create a new get_acl helper to ensure proper xml is compared

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9b6ff638735d03c1f3304198fad51ca701a2c8bd)

tests for the new acl helper in radosgw_admin

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3801ac9357f78ac05fa850d8944651f26c556d68)

Changed suite name from multi-versions to client-upgrade

Moved it under upgrades
Replaced rbd workunit

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit bcf7eabc5f0bcd0c6665bc047c076925fbc9bdf5)

Merge pull request #349 from ceph/wip-10954-hammer

lint fixes for hammer

remove unused import from repair_test

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ignore tox hidden dir

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4068f73ade68e36c1e7653733eacf2b1288cd753)

remove unused variable assignment

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9df27f7a1040dae69ef65122d13d1640cb1b6a66)

remove unused import from fuse

Signed-off-by: Alfredo Deza <adeza@redhat.com>

remove uneeded variable assignment for call

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0a65e90db2e40b04c7cbb2347f4c759e283a2c2e)

remove unused import in calamari_setup

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d22cd6917bf77a57c323684b8c6e3687238df324)

remove unused imports from populate_rbd

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 959c4914629bd237d10cf5789906eba42b56a984)

remove unused imports from peering_speed

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6d45352782deb5606c8dcad39c75a94e748e292c)

remove redefined StringIO import

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3a18cb29a855a4b507f79a349821cb246a12c8c6)

stdin is no longer a kwarg

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 49a61dc2d2f66bd8dde34ba67dbd98d4c5148c9e)

add the log object to ceph_manager

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f7c1ca4a1e23b03ecf86d5e67d563f95b3a82f99)

add missing mon object to osd_failsafe

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1673be45c0e3aeb1eadad7ecd5b87570e9bac5ca)

fix undefined var in divergent_priors

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 26c5a82af9cf4efd3601b6a32aa5444f61fa9f81)

exclude the virtualenv from being linted in tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>

fix path for tox.ini to work on ceph-qa-suite only

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4eb1787280f9f128cb9352ecbe8957a4be94fff9)

initial tox.ini to run flake8

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7108b699fda247ce12cbeaea9dd0a32843775c27)
(cherry picked from commit e1c85f330b834394d099653d3c618d41cd438cff)

rbd: fix copy-on-read setting

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit aa6240c6f4cb8f258072fbfb94edb027bff6a3f4)

rbd: add merge_diff test

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit e7bf3d05a63f7cf99b3ff46fb9ac96e73ffb5c40)

suites/rbd: add object map tests

Enable the object map feature for CLI tests and librbd API tests.
Moved the CLI tests to a new group to exercise old format, basic
new format, and new format w/ object map.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b277946848fc2b5f454a7c6ffd8fa2a359369102)

ceph: ugh fix syntax

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1922c61bbf1165e23650e74767ac4d5f63becdbc)

Fixes #10869, added {role} to the dir name.

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
Fixed typo

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 9942440ddf9f6847e76de3b49f1ad0d98c62a3d0)

ceph: fix ps axuf lsof line

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 18307be0ca97f7221d6202884f8a68a4b26c1dbf)

ceph: ps axf too before lsof

Specifically, I want to know *who* is running the ceph-osd that is
holding the files open.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a68281e1476e6af38237e1d1031dd7bd0980ef9f)

ceph: lsof if umount fails

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 077e917af1394e5999293031afcc476b2bdce1e4)