Drop chef from the teuthology suite

Chef has been replaced by ansible, which will be added to jobs at
schedule time.

Signed-off-by: Zack Cerza <zack@redhat.com>

Drop chef from tgt suite

Chef has been replaced by ansible, which will be added to jobs at
schedule time.

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 5b57b81e90393d42494c7b02fb263f1601421f16)

Removed rhel* configurations with a goal that they will be covered in the Octo lab
and won't be used by vps runs in Sepia lab

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit adfa1ee393c0ea0a36c0dfb8d7fda1a82f4b29ea)

Merge pull request #478 from ceph/wip-12172-backport-next

cephfs/test_auto_repair.py: flush journal after umount

cephfs/test_auto_repair.py: flush journal after umount

When client capabilities get released, MDS may update corresponding
inodes' client writable range and mark those inodes dirty. The auto
repair test expects MDS to trim inodes from its cache, but MDS can't
trim dirty inodes. So we should flush journal after umount.

Fixes: #12172
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 835ad8b8f3038290846c612b66f3bffcc5755338)

upgrade-x: use compatible rados api tests

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

ceph: fix up log rotation stopper

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 451c5ca79de14a8462a76dc8287be1f4fd72fa1b)

Merge remote-tracking branch 'gh/next'

Merge pull request #455 from ceph/wip-11937

thrasher: Can't test ceph-objectstore-tool if nodes turned off (power…

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #452 from ceph/wip-logrotate

Logrotate

Reviewed-by: John Spray <john.spray@redhat.com>

ceph: update log rotation for review comments

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

thrasher: Can't test ceph-objectstore-tool if nodes turned off (powercycle)

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #440 from ceph/wip-fs-clientnum

Reduce fs test node usage

Reviewed-by: John Spray <john.spray@redhat.com>

add log-rotate configs to the userspace suite

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

ceph: support arbitrarily-named daemons in logrotate

And make it more configurable in terms of sizes.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

ceph: enable mds log rotation

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites/multimds: use 2-node clusters for pure userspace subsuites

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites/fs: use a two-node layout for most userspace tests

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites/fs: switch standby-replay tests to use a smaller cluster

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #449 from ceph/wip-fsync

tasks/cephfs: add TestClientRecovery.test_fsync

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #451 from athanatos/wip-11677

Wip 11677

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #450 from ceph/wip-fix-blacklist

tasks/cephfs: fix typo in blacklist clearing

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: add TestClientRecovery.test_fsync

To test that metadata written recently is
preserved across a client+server crash when
barriered with a directory fsync.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix typo in blacklist clearing

Broken in aa0ffb3

Signed-off-by: John Spray <john.spray@redhat.com>

rados: add write_excl and append_excl

Signed-off-by: Samuel Just <sjust@redhat.com>

rados/thrash: add test for radosgw with snaps

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #443 from ceph/wip-11301

tasks/cephfs: reset osd blacklist between tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #447 from ceph/wip-11779

tasks/cephfs: fix timing in test_full

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #444 from ceph/wip-11756

tasks/cephfs: mount fusectl before listing fuse connections

Reviewed-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix timing in test_full

The interval between writes was too short because
it was not taking account of the way OSDMap full
flags are set on tick rather than immediately.

Fixes: #11779
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #438 from ceph/wip-11405

client-upgrade: test librbd backwards compatibility

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tasks/cephfs: mount fusectl before listing fuse connections

Fixes: #11756
Signed-off-by: Yan, Zheng <zyan@redhat.com>

tasks/cephfs: reset osd blacklist between tests

...to avoid OSDMap modifications happening in the
background due to blacklist expiry.

Fixes: #11301
Signed-off-by: John Spray <john.spray@redhat.com>

Added script to schedule 1/14th part of rados

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit a7064a7b3d8a865dead3800a4d97bd70da6dec33)

Added script to schedule 1/14th part of rados

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #442 from ceph/wip-client-cache-size

cephfs/test_client_limits.py: invalidate kernel dcache according to client cache size

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #425 from ceph/wip-flock-optional

cephfs/mount: check FUSE version before invoking flock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #398 from ceph/wip-quota-full

Test for CephFS full handling with pool quotas (#11276)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #402 from ceph/wip-11343

Reviewed-by: Samuel Just <sjust@redhat.com>

cephfs/test_client_limits.py: invalidate kernel dcache according to client_cache_size

Signed-off-by: Yan, Zheng <zyan@redhat.com>

cephfs/test_client_recovery: check FUSE version before invoking flock

flock only works properly on FUSE versions >=2.9, which is newer
than eg Ubuntu Precise. So check the version on our client mounts and
only test flock if it's at least that new.

Fixes: #9995
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites: update log whitelist for TestQuotaFull

A quota ENOSPC looks different in the log than
as capacity ENOSPC

Signed-off-by: John Spray <john.spray@redhat.com>

suites: tweak config in mds-full task

The task runs a lot faster with a higher
OSD report frequency.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: cover pool quota in test_full

Run the same procedure as TestClusterFull, but
instead of limiting OSD memstore size, use pool
quota on the data pool.

Signed-off-by: John Spray <john.spray@redhat.com>

samba: disable MDS debugging to reduce log sizes

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh re: #11547
Removed per Josh's suggestion fs as redundant and renamed rados to bacis

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit dcfd6a3b099fd0e14fbfd47a8678a411387e01da)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #436 from ceph/wip_11547_2

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh re: #11547
Removed per Josh's suggestion fs as redundant and renamed rados to bacis

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #431 from ceph/wip-more-log-whitelist

suites/fs: add another log whitelist to journal-repair

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #426 from ceph/wip-11372

add new valgrind tests to rbd suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #433 from ceph/wip_11400

Initial check in for upgrades/hammer stable upgrade suite

Initial check in for upgrades/hammer stable upgrade suite
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

reg11184: Add regression test for 11184

Create divergent priors and a split and then move a pg using
ceph-objectstore-tool export/import
Add yaml file to run the reg11184 task

Fixes: #11343
Signed-off-by: David Zafman <dzafman@redhat.com>

divergent_priors2: Do divergent priors and ceph-objectstore-tool export/import

Based on tasks/divergent_priors.py but also do simple export/remove/import on
same osd.
Add yaml file to run the divergent_priors2 task

Signed-off-by: David Zafman <dzafman@redhat.com>

divergent_priors: Fix divergent_priors task

Flake8 fixes
Use new set_recovery_delay admin socket command
Fix bad value set for filestore_blackhole
Make sure log trims and only require 100 objects
Use kick_recovery_wq to properly set osd_recovery_delay_start to 0
Write and remove divergent and verify removal was undone
Fix to make compatible with wip-10809-11135-10290
Make sure to set_recovery_delay in a non-racey way (while osd running but down)
Leave divergent "in" so its PGs aren't treated as strays
Add yaml file to run the divergent_priors task

Signed-off-by: David Zafman <dzafman@redhat.com>

client-upgrade: added RBD notification tests

Verify correct behavior between two librbd versions when
proxying notify messages.

Fixes: #11405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

client-upgrade: added librbd API test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

suites/fs: add another log whitelist to journal-repair

This is the table unreadable error for mds1_inotable that
we get when trying to start an MDS whose objects are gone.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'gh/next'

Conflicts:
suites/fs/recovery/tasks/client-limits.yaml
suites/fs/recovery/tasks/mds-full.yaml
tasks/ceph_deploy.py

smoke: white marked down on python test

Backport: hammer
Signed-off-by: Sage Weil <sage@redhat.com>

Added hammer-client-x to the mix
Renamed dirs to better reflect what is being tested
Removed old dirs
Added logging
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 3c8730d035a90f221c05526983c40f0a11da1ba6)

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #428 from athanatos/wip-11429

suites/rados: add test for 11429

Reviewed-by: Zack Cerza <zack@redhat.com>

knfs: remove the MDS debugging

It doesn't fit on the typica boxes with debugging, and we probably don't need
it for any NFS issues which do arise...

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

squash: use sleep instead of utility.sleep

squash: use != not is not in radosbench

squash: remove utility, moved to teuthology.git

squash: ceph_manager: add utility_task doc string

suites/rados: add test for 11429

This patch also adds some convenience facilities for making
some of the ceph_manager methods into tasks usable from a
yaml file.

Signed-off-by: Samuel Just <sjust@redhat.com>

Removed redundant tests issue #11520
Reduced nodes to be locked to two

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 8ab46b42c5ae387141917bb8eff35960e9c45699)

Merge pull request #427 from ceph/wip_11520

Removed redundant tests issue #11520
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Removed redundant tests issue #11520
Reduced nodes to be locked to two

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

All changes from https://github.com/ceph/ceph-qa-suite/commit/3ce379a689440ce084b2df79db59dd76e1b2db6c

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Removed unneeded workloads
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 18a79fb8252d9ceee7162e8f774ab6bf806484c5)

Conflicts:

suites/ceph-deploy/fs/tasks/cfuse_workunit_suites_dbench.yaml
suites/ceph-deploy/fs/tasks/cfuse_workunit_suites_fsstress.yaml
suites/ceph-deploy/rados/tasks/rados_python.yaml
suites/ceph-deploy/rados/tasks/rados_workunit_loadgen_big.yaml
suites/ceph-deploy/rbd/tasks/rbd_api_tests_old_format.yaml
suites/ceph-deploy/rbd/tasks/rbd_cli_tests.yaml
suites/ceph-deploy/rbd/tasks/rbd_cls_test.yaml

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Removed unneeded workloads

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 18a79fb8252d9ceee7162e8f774ab6bf806484c5)

Fixing commits from https://github.com/ceph/ceph-qa-suite/pull/400/commits

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Reduced ammount of nodes from 4 to 3
Changed default 'replica' to 2
Reduced osd to 2 per node
Added logging

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit 3ce379a689440ce084b2df79db59dd76e1b2db6c)

Merge pull request #423 from ceph/wip-11495

Fix ceph_deploy task with multiple mons

Merge pull request #421 from ceph/wip-client-osd-caps

tasks/cephfs: add test_pool_perm

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

tasks/ceph_deploy: remove some dead code

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/ceph_deploy: fix for multiple mons

Now that service IDs are modified during run, we have
to avoid repeatedly evaluating first_mon for where
to run ceph_deploy, as the answer will change.

Fixes: #11495
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #422 from ceph/wip-log-whitelist

suites: log whitelist for journal repair test

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

suites/rbd: add new valgrind memtest tests

Exercise the C-API, Python-API, and FSX under valgrind.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tasks: add support for running fsx under valgrind

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

suites: log whitelist for journal repair test

"bad backtrace on dir ino" is emitted because
journaltool's recover-dentries doesn't write
backtraces.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: add test_pool_perm

This tests that client checks pool permission before
copy data to its cache.

Signed-off-by: Yan, Zheng <zyan@redhat.com>

rbd.xfstests: remove bad assert

Misguided input validation. A better error comes
from the run_xfstests() method later anyway.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Initial checkin for hammer-x suite
Used "ceph osd crush tunables hammer"

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>
(cherry picked from commit f825d32d9deab22f6113b96a6b36734fe1bdb289)

Merge pull request #405 from ceph/wip-10781

Initial checkin for hammer-x suite

Merge pull request #414 from ceph/wip_11413

Added hammer-client-x to the mix

xfstests: fix typo

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

xfstests: define defaults in root task

The early non-defaults caused failures due to xfstests_url: None not
being overridden by run_xfstests(). Move the defaults to xfstests() and
don pass xfstests_branch past that point.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

task/samba: ignore return code of fuser/losf

if mount point is not used by anyone, both fuser and losf return 1

Fixes: #10624
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit c049387b7b0832e78a4a1370691798f726985799)

task/samba: use SIGTERM to stop samba server

man samba(8) contains sentences:

To shut down a user's smbd process it is recommended that SIGKILL (-9)
NOT be used, except as a last resort, as this may leave the shared
memory area in an inconsistent state. The safe way to terminate an smbd
is to send it a SIGTERM (-15) signal and wait for it to die on its own.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 86bd6bc3771694cd9ee5cd071255f3cf4c40c812)

Merge pull request #417 from ceph/wip-10657

cephfs/test_client_limits.py: test for client oldest tid warning

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #416 from ceph/wip-11316-next

Backport to next: fix ceph-deploy task with CephFS (#11316)

Merge pull request #415 from ceph/wip-10624

task/samba: ignore return code of fuser/losf

Reviewed-by: Sage Weil <sage@redhat.com>

cephfs/test_client_limits.py: test for client oldest tid warning

Signed-off-by: Yan, Zheng <zyan@redhat.com>

tasks/ceph_deploy: configure CephFS

This test apparently had not been touched since
"fs new" was added.  In addition to calling
Filesystem.create:
 * modify the get_nodes_using_role
   function to modify ctx.cluster.remotes so that the
   service IDs match what ceph-deploy will set
 * log exceptions during ceph_deploy setup, as otherwise
   they can get lost if another exception occurs during
   teardown (so that it's all easier to debug).
 * default to passing --dev=master during install, so
   that we don't error out horribly when run without
   an explicit branch set (e.g. when run outside
   scheduled suite)

Fixes: #11316
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7c2e6cdc6cd3d02969d3494d120b12daaa13efa8)

tasks/cephfs: cope with missing ctx.daemons attr

...so that we may be used with the ceph_deploy task
that doesn't set that up.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f36011051cabb8e7b06194c778e640e98d328e8a)

tasks/cephfs: tweak use of mon for admin commands

... s/mon_remote/admin_remote/ and allow caller to pass
in which remote they want to use for that.  Enables use
with ceph_deploy task which does not give admin keys
to mons.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8b61310ce3eedeb8ac60446933cd5db548de6434)