Merge pull request #465 from ceph/wip-11570

hammer suite split - older and newer

Merge pull request #384 from ceph/wip-dirfrag-enable

suites/fs: enable directory fragmentation

suites/fs: enable directory fragmentation

Setting this in tests globally as a precursor
to enabling it by default in the shipping code.
Set a low mds bal split size in order to make
us hit the fragmentation code more often in
than we usually would in a single MDS environment.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #470 from ceph/wip-4161

Add cephfs TestBacktrace

Merge pull request #466 from ceph/wip-damaged-fixes

Tests for wip-damaged-fixes

Merge pull request #485 from athanatos/wip-12231

3-size-2-min-size: keep 4 in during thrashing

Reviewed-by: David Zafman <dzafman@redhat.com>

3-size-2-min-size: keep 4 in during thrashing

Workaround for 12231.

Fixes: #12231
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #476 from t-miyamae/wip-10886

erasure code: added shec's initial ceph-qa-suite

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #481 from dmick/master

calamari_nosetests: verify that calamari_branch exists

Merge pull request #483 from ceph/wip-excl

fix hammer-x upgrades with excl stuff in ceph_test_rados

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yuri Weinstein <yuriw@redhat.com>

Merge pull request #484 from ceph/wip_remove_rhel

Removed rhel* configurations with a goal that they will be covered in…

Removed rhel* configurations with a goal that they will be covered in the Octo lab
and won't be used by vps runs in Sepia lab

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

upgrade/hammer-x: drop symlinks to ec jobs and disable *_excl

Signed-off-by: Sage Weil <sage@redhat.com>

suites/upgrade/hammer-x: do not pass _excl weights to ceph_test_rados

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/rados: make {write,append}_excl conditional

For upgrade tests we don't want to pass these args to (old)
ceph_test_rados.

Signed-off-by: Sage Weil <sage@redhat.com>

calamari_nosetests: verify that calamari_branch exists

Fixes: #12206
Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #479 from ceph/wip-rgw-task

rgw: do not enable both tcp and uds for fastcgi

rgw: do not enable both tcp and uds for fastcgi

This fixes a bug where we were setting up mod_fastcgi to use both tcp
and uds.

Merge pull request #477 from ceph/wip-12172-backport-next

cephfs/test_auto_repair.py: flush journal after umount

cephfs/test_auto_repair.py: flush journal after umount

When client capabilities get released, MDS may update corresponding
inodes' client writable range and mark those inodes dirty. The auto
repair test expects MDS to trim inodes from its cache, but MDS can't
trim dirty inodes. So we should flush journal after umount.

Fixes: #12172
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 835ad8b8f3038290846c612b66f3bffcc5755338)

erasure code: added shec's initial ceph-qa-suite

Fixes: #10886(http://tracker.ceph.com/issues/10886)
Signed-off-by: Shotaro Kawaguchi <kawaguchi.s@jp.fujitsu.com>

Merge branch 'master' of https://github.com/ceph/ceph-qa-suite into wip-11570
Added time-outs to s3tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #474 from ceph/wip-wusui-12185

tasks/calamari_setup.py cal_svr.run arg list is missing args keyword.

Reviewed-by: Dan Mick <dmick@redhat.com>

tasks/calamari_setup.py cal_svr.run arg list is missing args keyword.

Fixes #12185

Signed-off-by: Warren Usui <warren.usui@inktank.com>

Merge pull request #473 from ceph/wip-12172

Wip 12172

Reviewed-by: John Spray <john.spray@redhat.com>

cephfs/test_auto_repair.py: flush journal after umount

When client capabilities get released, MDS may update corresponding
inodes' client writable range and mark those inodes dirty. The auto
repair test expects MDS to trim inodes from its cache, but MDS can't
trim dirty inodes. So we should flush journal after umount.

Fixes: #12172
Signed-off-by: Yan, Zheng <zyan@redhat.com>

suites: yaml for test_damage

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: add test_backtrace

This is for verifying the new layout-writing behaviour.  While
we're at it, test that the pre-existing backtrace behaviours
are really happening (updating old_pools)

Signed-off-by: John Spray <john.spray@redhat.com>

Fixes #11570
Removed test_cls_rbd.sh per Josh's suggestion
Removed 'branch: hammer' from 3-upgrade-sequence/upgrade-*

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #366 from ceph/wip-10551

Support the usage of mod_proxy_fcgi for rgw tests

Adds a set of rgw tests to the teuthology suite

I added these so I could source control a good yaml fragment that I
created to test rgw.  Adding the suite allows me to test the addition of
mod_proxy_fcgi across multiple distros.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Adds mod_proxy_fcgi support and makes it the default setup for rgw

This also adds a 'use_fastcgi' override config option for the rgw task.

If specified we'll configure things to use mod_fastcgi instead of
mod_proxy_fcgi.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

suites: hook in cephfs/test_damage

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: add TestDamage

Tests MDS response to badness in the metadata tool

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: allow stdin for Filesystem.rados

Useful for puts when injecting data to test
corruption handling.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: add CephFSMount.stat

Signed-off-by: John Spray <john.spray@redhat.com>

calamari_setup.py: fix tarball installs

May be obsolete soon, but in tarballs, ice_setup is still a script

Fixes: #12046
Signed-off-by: Dan Mick <dan.mick@redhat.com>

calamari_setup.py: tear down mounted ISO

Fixes: #11457
Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #462 from ceph/wip-12023

Add point releases (tags) to hammer release  #12023

Initial checking for #12023
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge branch 'next'

upgrade-x: use compatible rados api tests

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #454 from ceph/wip-cdtests

Single node ceph-deploy tests to exercise commonly used cli

Reviewed-by: Travis Rhoden <trhoden@redhat.com>

ceph: fix up log rotation stopper

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 451c5ca79de14a8462a76dc8287be1f4fd72fa1b)

Single Node Ceph-deploy tests to exercise commonly used CLI and test init startup
   - Add 1node tests to Smoke and include ceph-deploy test
   - Review Comments , add rgw cli test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #460 from ceph/wip-logrotate-fix

ceph: fix up log rotation stopper

Reviewed-by: John Spray <john.spray@redhat.com>

ceph: fix up log rotation stopper

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #457 from ceph/wip-11913

tasks/cephfs: fix race in test_full

Reviewed by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: fix race in test_full

Sometimes mount A would get a cap revoke when mount
B did its last IO, resulting in mount A's OSD epoch
getting updated too.

Fix by making sure mount B is the last one to have
done IO before we do the barrier, so that when
it does IO again after the barrier, mount A can't
be holding any caps that B would need.

Fixes: #11913
Signed-off-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'gh/next'

Merge pull request #455 from ceph/wip-11937

thrasher: Can't test ceph-objectstore-tool if nodes turned off (power…

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #452 from ceph/wip-logrotate

Logrotate

Reviewed-by: John Spray <john.spray@redhat.com>

ceph: update log rotation for review comments

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

thrasher: Can't test ceph-objectstore-tool if nodes turned off (powercycle)

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #440 from ceph/wip-fs-clientnum

Reduce fs test node usage

Reviewed-by: John Spray <john.spray@redhat.com>

add log-rotate configs to the userspace suite

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

ceph: support arbitrarily-named daemons in logrotate

And make it more configurable in terms of sizes.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

ceph: enable mds log rotation

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites/multimds: use 2-node clusters for pure userspace subsuites

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites/fs: use a two-node layout for most userspace tests

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites/fs: switch standby-replay tests to use a smaller cluster

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #449 from ceph/wip-fsync

tasks/cephfs: add TestClientRecovery.test_fsync

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #451 from athanatos/wip-11677

Wip 11677

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #450 from ceph/wip-fix-blacklist

tasks/cephfs: fix typo in blacklist clearing

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: add TestClientRecovery.test_fsync

To test that metadata written recently is
preserved across a client+server crash when
barriered with a directory fsync.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix typo in blacklist clearing

Broken in aa0ffb3

Signed-off-by: John Spray <john.spray@redhat.com>

rados: add write_excl and append_excl

Signed-off-by: Samuel Just <sjust@redhat.com>

rados/thrash: add test for radosgw with snaps

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #443 from ceph/wip-11301

tasks/cephfs: reset osd blacklist between tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #447 from ceph/wip-11779

tasks/cephfs: fix timing in test_full

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #444 from ceph/wip-11756

tasks/cephfs: mount fusectl before listing fuse connections

Reviewed-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix timing in test_full

The interval between writes was too short because
it was not taking account of the way OSDMap full
flags are set on tick rather than immediately.

Fixes: #11779
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #438 from ceph/wip-11405

client-upgrade: test librbd backwards compatibility

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tasks/cephfs: mount fusectl before listing fuse connections

Fixes: #11756
Signed-off-by: Yan, Zheng <zyan@redhat.com>

tasks/cephfs: reset osd blacklist between tests

...to avoid OSDMap modifications happening in the
background due to blacklist expiry.

Fixes: #11301
Signed-off-by: John Spray <john.spray@redhat.com>

Added script to schedule 1/14th part of rados

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit a7064a7b3d8a865dead3800a4d97bd70da6dec33)

Added script to schedule 1/14th part of rados

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #442 from ceph/wip-client-cache-size

cephfs/test_client_limits.py: invalidate kernel dcache according to client cache size

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #425 from ceph/wip-flock-optional

cephfs/mount: check FUSE version before invoking flock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #398 from ceph/wip-quota-full

Test for CephFS full handling with pool quotas (#11276)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #402 from ceph/wip-11343

Reviewed-by: Samuel Just <sjust@redhat.com>

cephfs/test_client_limits.py: invalidate kernel dcache according to client_cache_size

Signed-off-by: Yan, Zheng <zyan@redhat.com>

cephfs/test_client_recovery: check FUSE version before invoking flock

flock only works properly on FUSE versions >=2.9, which is newer
than eg Ubuntu Precise. So check the version on our client mounts and
only test flock if it's at least that new.

Fixes: #9995
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

suites: update log whitelist for TestQuotaFull

A quota ENOSPC looks different in the log than
as capacity ENOSPC

Signed-off-by: John Spray <john.spray@redhat.com>

suites: tweak config in mds-full task

The task runs a lot faster with a higher
OSD report frequency.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: cover pool quota in test_full

Run the same procedure as TestClusterFull, but
instead of limiting OSD memstore size, use pool
quota on the data pool.

Signed-off-by: John Spray <john.spray@redhat.com>

samba: disable MDS debugging to reduce log sizes

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh re: #11547
Removed per Josh's suggestion fs as redundant and renamed rados to bacis

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit dcfd6a3b099fd0e14fbfd47a8678a411387e01da)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #436 from ceph/wip_11547_2

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Replaced work units with ceph-deploy/ceph-deploy_hello_world.sh re: #11547
Removed per Josh's suggestion fs as redundant and renamed rados to bacis

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #431 from ceph/wip-more-log-whitelist

suites/fs: add another log whitelist to journal-repair

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #426 from ceph/wip-11372

add new valgrind tests to rbd suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #433 from ceph/wip_11400

Initial check in for upgrades/hammer stable upgrade suite

Initial check in for upgrades/hammer stable upgrade suite
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

reg11184: Add regression test for 11184

Create divergent priors and a split and then move a pg using
ceph-objectstore-tool export/import
Add yaml file to run the reg11184 task

Fixes: #11343
Signed-off-by: David Zafman <dzafman@redhat.com>

divergent_priors2: Do divergent priors and ceph-objectstore-tool export/import

Based on tasks/divergent_priors.py but also do simple export/remove/import on
same osd.
Add yaml file to run the divergent_priors2 task

Signed-off-by: David Zafman <dzafman@redhat.com>

divergent_priors: Fix divergent_priors task

Flake8 fixes
Use new set_recovery_delay admin socket command
Fix bad value set for filestore_blackhole
Make sure log trims and only require 100 objects
Use kick_recovery_wq to properly set osd_recovery_delay_start to 0
Write and remove divergent and verify removal was undone
Fix to make compatible with wip-10809-11135-10290
Make sure to set_recovery_delay in a non-racey way (while osd running but down)
Leave divergent "in" so its PGs aren't treated as strays
Add yaml file to run the divergent_priors task

Signed-off-by: David Zafman <dzafman@redhat.com>

client-upgrade: added RBD notification tests

Verify correct behavior between two librbd versions when
proxying notify messages.

Fixes: #11405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

client-upgrade: added librbd API test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

suites/fs: add another log whitelist to journal-repair

This is the table unreadable error for mds1_inotable that
we get when trying to start an MDS whose objects are gone.

Signed-off-by: John Spray <john.spray@redhat.com>