buildpackages: allow for more parallel builds

Builds are in parallel if the OS type / version and the SHA1 are
different. Having just one build at a time for a given SHA1 slows thing
down when there are more than a few supported OS.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: enable autoindex in the packages repository

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: do not requires the long tag version

So that v10.1.0 is used instead of 10.1.0-0-g35ac827 when possible.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: also pull tags from the user ceph fork

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: create release packages

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #875 from ceph/wip-15124

tasks/cephfs: support old mdsmap command during setup

Merge pull request #836 from ceph/wip-fix_cdtest

fix ceph-deploy test

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #861 from ceph/wip-vstart-fixes

tasks/cephfs: vstart & fuse improvements

suites: fix assumed filesystem pool name

Tests shouldn't really assume the names of
cephfs pools, but because it's hardcoded in
the yaml here I'll just update it.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: support old mdsmap command during setup

While Filesystem at large requires the new commands, for
use from the `ceph` task we must support old style commands,
as the ceph task is used to instantiate old clusters during
upgrade testing.

Fixes: #15124, #15049, #15106
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #877 from dachary/wip-workunit

workunit: git-reset --hard does not accept a branch

Reviewed by: Nathan Cutler <ncutler@suse.com>

workunit: git-reset --hard does not accept a branch

Uses git clone --branch refspec because it agrees to use either a tag or
a branch.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #871 from ceph/wip-rbd-client-upgrade

upgrade/client-upgrade: only use supported features for RBD tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

upgrade/client-upgrade: only use supported features for RBD tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #870 from ceph/wip-rbd-mirror

rbd: add some missing workunits

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Fix review comments

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #868 from ceph/wip-fio-ver

Update fio version to 2.7

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: add some missing workunits

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #863 from ceph/wip-15049

rados/singleton-nomsgr/all/11429: no mds

Merge pull request #824 from ceph/wip-multi-filesystems

Test updates for multi-filesystem support

Update fio version to 2.7

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

tasks/cephfs: enable multi fs flag

Cluster won't let you create more than without
setting this flag explicitly.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: debug in wait_for_daemons

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: switch off extra daemons in TestStrays

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: further thrasher fixes

Move the thrasher-specific methods out of CephManager
into MDSThrasher and plumb them into MDSCluster.

Signed-off-by: John Spray <john.spray@redhat.com

tasks/cephfs: simplify recreate()

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: update test_journal_repair for multi fs

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: add TestStandbyReplay

In addition to the multi-fs tests, more localised
standby-replay cases.

Signed-off-by: John Spray <john.spray@redhat.com>

suites: add fs/multifs

Separate because it wants a larger MDS count.  Points
to the new tests in test_failover.py.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix datascan for multiple filesystems

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: move the journaltool smoke test into python land

This is one of those things that had a convoluted .yaml file
because it predated the ability to write explicit procedural
python tests.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: rename Filesystem.reset to recreate

Name was ambiguous wrt "fs reset"

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: fix mds_thrash for multi filesystems

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: update test_runner for mds_cluster

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: generalise cephfs classes for multi-fs

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: add cephfs TestMultiFilesystems

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #865 from ceph/wip-dbg-packages-1

packages: update for all the new debug packages we have in jewel

packages: update for all the new debug packages we have in jewel

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #859 from ceph/wip-15014

Add packages.yaml

Removed branch: firefly from 4-after.yaml

Fixes #15012

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 872aed9e72d0bac1fc21152544eef3c42abbd0bb)

rados/singleton-nomsgr/all/11429: no mds

The newer cephfs logic wants to use the 'status' command on the
daemon, but that doesn't exist on firefly.  Luckly we don't need
it for this test.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #862 from ceph/wip-odsync

rados: Enable odsync write for pggrow thrasher

rados: Enable odsync write for pggrow thrasher

Signed-off-by: Samuel Just <sjust@redhat.com>

tasks/cephfs: vstart & fuse improvements

A couple of corner cases that would get tripped up sometimes
in a vstart_runner.py environment.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'origin/greg-fs-testing-3-7-damage'

Merge pull request #731 from ceph/wip-scrub-repair

Wip scrub repair

Merge pull request #860 from ceph/wip-krbd-fio

Fixes for krbd test cases

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Reflect that ceph-dbg is dropped

Signed-off-by: Zack Cerza <zack@redhat.com>

Add packages.yaml

Signed-off-by: Zack Cerza <zack@redhat.com>

Fix recent failures due to HEALTH_WARN check issues

There were recent failures due to HEALTH_WARN check unrelated
to script, this fix basically bypasses this issue by using
osd pool default size: 1 in ceph.conf and now
checks for HEALTH_OK instead of HEALTH_WARN
It also adds the meta information to tasks which describes test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

rados/thrash/workloads/radosbench: shorten 300s->200s

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'origin/wip-damage-table' into greg-fs-testing

https://github.com/ceph/ceph-qa-suite/pull/782

Conflicts:
tasks/cephfs/test_scrub_checks.py

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge remote-tracking branch 'origin/wip-scrub-repair' into greg-fs-testing

https://github.com/ceph/ceph-qa-suite/pull/731

krbd: do not test unsupported striping feature

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tasks/rbd_fio: create sane image name based on feature set

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #858 from ceph/wip-pg-dump

tasks/ceph_manager: dump pgs if other peering timeouts expire

tasks/ceph_manager: dump pgs if other peering timeouts expire

We were doing this for one of the recovery timeouts but not all of them.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #857 from smithfarm/wip-move-plus-to-clusters

suites/smoke/1node: move + to clusters/ subdirectory

suites/smoke/1node: move + to clusters/ subdirectory

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #854 from SUSE/wip-opensuse-distros

distros: add yaml fragments for openSUSE 13.2 and 42.1

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tasks: add TestDamage.test_damaged_dentry

A more detailed test for the case of damaged dentries,
because the handling involves the system passing through
a state where the containing dir has invalid stats,
and has the complete flag set even though the
dentry in question is missing.

Signed-off-by: John Spray <john.spray@redhat.com>

suites: update log whitelist for TestDamage

For branch wip-damage-table.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: update TestDamage

...to account for the behavioural changes with
the addition of DamageTable.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: move wait_for_health up into CephFSTestCase

...so that TestDamage can use it too.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: optionally stat() in background

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: avoid sleep in test_client_cache_size

Instead of waiting 30s and then checking, use
wait_until_true to complete as soon as the
condition is met.  Test runs faster this way.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: remove extraneous ;s in test_client_limits

These are harmless but they stand out in some editors.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: add test case for scrub repair

Signed-off-by: Yan, Zheng <zyan@redhat.com>

distros: add yaml fragments for openSUSE 13.2 and 42.1

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge remote-tracking branch 'gh/wip-file-layout'

Merge remote-tracking branch 'gh/jewel'

Merge pull request #852 from ceph/wip-ceph-ansible

ceph-ansible: Add openstack hints for volumes

ceph-ansible: Add openstack hints for volumes

3x20GB per instance

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #851 from ceph/wip-ceph-ansible

ceph-ansible: Install ceph-test

Merge pull request #848 from ceph/wip-ceph-ansible

ceph-ansible suite improvements

Install ceph-test

Signed-off-by: Zack Cerza <zack@redhat.com>

Disable rbd_cli_tests for now

They don't seem to ever fail.

Signed-off-by: Zack Cerza <zack@redhat.com>

Add ceph-deploy_hello_world.sh

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #843 from ceph/wip-list-inconsistent-obj

tasks/scrub_test: match the NOSNAP with "head"

Reviewed-by: Samuel Just <sjust@redhat.com>

suites/fs/verify: debug mon too

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/cephfs/filesystem: ceph_file_layout -> file_layout_t

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #844 from ceph/wip-rbd-cli-features

rbd/cli: add older default features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #846 from dachary/wip-14931-cram

cram: pin to version 0.6 because 0.7 handles \r differently

Reviewed-by: Xiaoxi Chen <xiaoxchen@ebay.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cram: pin to version 0.6 because 0.7 handles \r differently

http://tracker.ceph.com/issues/14931 Fixes: #14931

Signed-off-by: Loic Dachary <loic@dachary.org>

tasks/scrub_test: match the NOSNAP with "head"

* rados cli prints "head" at seeing CEPH_NOSNAP, so update the test
  accordingly
* print inconsistent objs before testing them

Fixes: #14924
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd/cli: add older default features

Explicitly set features and default format so they don't become out of
date. Remove none.yaml, since default.yaml is an explicit version of
it now.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #842 from ceph/wip-rbd-features

rbd: updated tests to use new rbd default feature set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #841 from ceph/wip-krbd-default-features

krbd: override rbd defaults features to 1

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #821 from ceph/wip-fsx-journal

rbd: add fsx journal replay test case

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Make the keyring world-readable

Signed-off-by: Zack Cerza <zack@redhat.com>

Rename mon.0 to mon.a

Signed-off-by: Zack Cerza <zack@redhat.com>

Run 'ceph health' after deployment

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #832 from athanatos/wip-lost

Wip lost

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #839 from ceph/wip-ceph-ansible

Add ceph-ansible suite

rbd: updated tests to use new rbd default feature set

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

krbd: override rbd defaults features to 1

Jewel will update the default feature set to features
unsupported by krbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Fix old and new pep8 issues

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add ceph-ansible suite

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #837 from ceph/wip-valgrind

suites: debuginfo: true for valgrind

Merge pull request #833 from ceph/wip-async

Revert "basic/msgr: remove async and random for now"