Merge pull request #902 from smithfarm/wip-14984

rbd: add singleton to assert no rbdmap regression

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #907 from dachary/wip-15236-infernalis-x

upgrade/infernalis-x: allocate OpenStack volumes for each OSD

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #908 from dachary/wip-archs

archs: add files for each known architecture

Merge pull request #869 from ceph/wip-15093-jewel

Added blogbench.yaml to parallel before and after -x upgrade steps

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Added blogbench.yaml to parallel before and after -x upgrade steps

Fixes #15093

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #827 from ceph/jewel-14805

Disable client-side permission checking in Hadoop.

It's not set up to pass appropriate UIDs/GIDs through, so everything breaks.

buildpackages: do not try tcmalloc on arm64

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: desambiguate nic on server create

When there are more than one network, it is required to select the one
to be attached to the nic of the instance.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge remote-tracking branch 'gh/jewel'

dummy: reduce run time, run user.yaml playbook

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #898 from ceph/wip-15206

test_msgr: improve debug ms level

buildpackages: build will work on a range of flavors

When selecting a flavor to build, we prefer the largest we can find and
fallback to smaller ones, down to the minimum acceptable flavor, if
there is no other available.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: do not hardcode x86_64 in build scripts

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: use architecture suffixed images

Signed-off-by: Loic Dachary <loic@dachary.org>

archs: add files for each known architecture

So that they can be prepend to teuthology-suite to run a job on a
designated architecture.

Signed-off-by: Loic Dachary <loic@dachary.org>

suites: run rados bench for max 180 seconds

If we want a longer runtime than that, run it several times in
sequence.

This limits the amount of disk space we'll consume.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2b3ec8de8d08d9a2b694f898f47d8ca27d0aff1f)

Merge pull request #910 from ceph/wip-oldupgrades

suites/upgrades: clean up old upgrade suites

Merge pull request #909 from ceph/wip-radosbench

suites: run rados bench for max 180 seconds

suites/upgrade: remove stable upgrade suites

These are maintained in their respective branches (e.g.,
the latest suites/upgrade/hammer is in the hammer branch).

Signed-off-by: Sage Weil <sage@redhat.com>

suites/upgrade: remove obsolete *-x suites

These versions can no longer upgrade to jewel.

Signed-off-by: Sage Weil <sage@redhat.com>

suites: run rados bench for max 180 seconds

If we want a longer runtime than that, run it several times in
sequence.

This limits the amount of disk space we'll consume.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #905 from ceph/wip-15065

tasks/cephfs: extend TestMultiFilesystems

fs: remove extra ceph-fuse invocation from TestMisc yaml fragment

If we ever re-enable the kernel client on this subsuite (unlikely) we'll
need to move it, but for now that yaml frag belongs in the mount/ dir.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: test for request caps during lookup/open

Signed-off-by: Yan, Zheng <zyan@redhat.com>

upgrade/infernalis-x: allocate OpenStack volumes for each OSD

The intention is to run with xfs which requires a separate disk for each
OSD. In addition the tests will exhaust the available root disk space
otherwise.

http://tracker.ceph.com/issues/15236 Fixes: #15236

Signed-off-by: Loic Dachary <loic@dachary.org>

suites/upgrade/hammer-x: do upgrade before staritng working

Otherwise crushtool test will fail while creating pools.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #874 from ceph/jewel-15008

Jewel 15008

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #853 from ceph/wip-cephfs-test-python

suites: invoke libcephfs python tests

tasks/cephfs: extend TestMultiFilesystems

...with cases for standby_for_rank and standby_for_fscid.

Fixes: #15065
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #904 from dachary/wip-rbd-upgrade-use-jewel

upgrade: use infernalis instead of hammer for rbd tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

upgrade: use infernalis instead of hammer for rbd tests

There are only few differences at the moment but it may be confusing
later on if a bug is fixed in the infernalis workunits later on.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #903 from ceph/wip-15204

tasks: fix ceph_deploy call to Filesystem()

Merge remote-tracking branch 'gh/jewel'

Merge pull request #895 from ceph/wip-rbd-mirror-test

suites/rbd: small cleanup and add ceph_test_rbd_mirror workloads

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

upgrade/infernalis-x/point-to-point: do upgrade before test workload

Test creates pools, that can fail while crushtool is being
overwritten.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/hammer-x/f-h-x-offline: fix mon statup

OSDs are down, so we can't wait for healthy.

Signed-off-by: Sage Weil <sage@redhat.com>

tasks: fix ceph_deploy call to Filesystem()

Fixes: #15204
Signed-off-by: John Spray <john.spray@redhat.com>

suites/rbd: add ceph_test_rbd_mirror workloads

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

rbd: add singleton to assert no rbdmap regression

http://tracker.ceph.com/issues/14984 References: #14984

Signed-off-by: Nathan Cutler <ncutler@suse.com>

upgrade/infernalis-x/point-to-point: run rgw + s3tests in sequential

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/infernalis-x/parallel: do upgrade before workload

This avoids upgrades while pools are being created (which may run
crushtool and fail).

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/hammer-x/f-h-x-offline: stop mons during upgrade

Mons also can't go firefly -> hammer and form a quorum; stop them
first.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/infernalis-x/point-to-point: upgrade before workload

Upgrading the mons can make pool creation fail.  Don't do it in
parallel with the rados api test workload (which does a lot of
that).

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/infernalis-x/point-to-point: run steps in series

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/infernalis-x/point-to-point: remove stray upgrade steps

This looks like an accidental cut and paste.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/infernalis-x/parallel: do workload in series

Same fix as b483793f6774db5c12ec348da3804f9adb44b810.  This makes
the cls test run later so that it doesn't race with the package
upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #899 from dachary/wip-buildpackages

buildpackages: provision a small build machine for tests

test_msgr: improve debug ms level

buildpackages: provision a small build machine for tests

Signed-off-by: Loic Dachary <loic@dachary.org>

suites/rbd/maintenance: use symlinks for cluster config

This is the same 3-node cluster as most tests use, and qemu with
xfstests already has an openstack config in the qemu sub-suite.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #893 from ceph/wip-rbd-maint-ops

suite/rbd: test dynamic features and rebuild object map

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

upgrade/hammer-x/parallel: run workloads in sequence, not parallel

Signed-off-by: Sage Weil <sage@redhat.com>

suite/rbd: test dynamic features and rebuild object map

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #888 from ceph/wip-15171

suites/rados/thrash: vary min_{read,write}_recency_for_promote

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #892 from ceph/wip-rbd-qemu-journaling

suites/rbd: add qemu tests with journaling

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

suites/rbd: add qemu tests with journaling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Changed run time for radosbench to 200

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 0ad1e4006b1f1fd0a06bfb13e428e16312c99173)

Merge pull request #890 from ceph/wip-fix-powercycle

Changed run time for radosbench to 200

Changed run time for radosbench to 200

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge remote-tracking branch 'gh/jewel'

upgrade/hammer-x: fix ec portion of workload

This should go in the workload sequence.

*also*, it'll be the first item, which pushes the many pool
creations in the api tests off until after packages install,
which avoids a problem with ceph-mon running crushtool while it
is being extracted.

Signed-off-by: Sage Weil <sage@redhat.com>

suites/rados/thrash: vary min_{read,write}_recency_for_promote

Hopefully this will shake out #15171

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #887 from dmick/master

tasks/ceph.py: Remove *.pid at end of run

tasks/ceph.py: Remove *.pid at end of run

http://tracker.ceph.com/issues/15162
Fixes: #15162
Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #885 from ceph/wip-15139

rados/singleton-nomsgr: run singleton upgrade tests on ubuntu

Merge pull request #886 from ceph/wip-15095

upgrade/infernalis-x: do not run *Whiteout tests against post-infernalis cluster

Reviewed-by: Josh Durgin <jdurgin@redhat.com

upgrade/infernalis-x: do not to tier *Whiteout tests against post-infernalis cluster

Fixes: #15095
Signed-off-by: Sage Weil <sage@redhat.com>

rados/singleton-nomsgr: run singleton upgrade tests on ubuntu

Fixes: #15139
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #882 from dachary/wip-buildpackages

buildpackages: allow for packages to be re-used for releases

buildpackages: ensure clone to a ceph directory

In case the repository from which the clone is not named ceph.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: for source inclusion in debian

We do not build debian versions (i.e. -2 -3 etc). Even if the debian
detection logic infers that the previous debian version already had the
source package, it is incorrect.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: allow for more parallel builds

Builds are in parallel if the OS type / version and the SHA1 are
different. Having just one build at a time for a given SHA1 slows thing
down when there are more than a few supported OS.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: enable autoindex in the packages repository

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: do not requires the long tag version

So that v10.1.0 is used instead of 10.1.0-0-g35ac827 when possible.

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: also pull tags from the user ceph fork

Signed-off-by: Loic Dachary <loic@dachary.org>

buildpackages: create release packages

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #880 from ceph/wip-rados-vs-mds

suites: remove unused mdss from RADOS tests

Merge pull request #881 from dachary/wip-workunit

workunit: sha1/branch/tag reset method

Reviewed-by: Nathan Cutler <ncutler@suse.com>

workunit: git checkout sha1/branch/tag

git clone --branch refspec only accepts branches and tags but not a sha1
git reset --hard refspec only accepts tags and sha1 but not branches

Use git checkout because it interprets branches, tags and sha1 as
expected.

$ git checkout jewel
Branch jewel set up to track remote branch jewel from origin.
Switched to a new branch 'jewel'
$ git checkout v0.94.6
HEAD is now at e832001... 0.94.6
$ git checkout b2503b0e15c0b13f480f0835060479717b9cf935
HEAD is now at 95cefea... 0.94.3

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #875 from ceph/wip-15124

tasks/cephfs: support old mdsmap command during setup

Merge pull request #836 from ceph/wip-fix_cdtest

fix ceph-deploy test

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #861 from ceph/wip-vstart-fixes

tasks/cephfs: vstart & fuse improvements

Merge pull request #867 from ceph/jewel-15050

tasks/cephfs: cover fancy layout in test_strays

suites: fix assumed filesystem pool name

Tests shouldn't really assume the names of
cephfs pools, but because it's hardcoded in
the yaml here I'll just update it.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: support old mdsmap command during setup

While Filesystem at large requires the new commands, for
use from the `ceph` task we must support old style commands,
as the ceph task is used to instantiate old clusters during
upgrade testing.

Fixes: #15124, #15049, #15106
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #877 from dachary/wip-workunit

workunit: git-reset --hard does not accept a branch

Reviewed by: Nathan Cutler <ncutler@suse.com>

workunit: git-reset --hard does not accept a branch

Uses git clone --branch refspec because it agrees to use either a tag or
a branch.

Signed-off-by: Loic Dachary <loic@dachary.org>

suites: remove unused mdss from RADOS tests

These daemons were apparently unused in tests that
created an MDS daemon but never use a fuse task
to access it.

Signed-off-by: John Spray <john.spray@redhat.com>

cephfs/test_client_limits.py: trim cache when mounting into subdir

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #871 from ceph/wip-rbd-client-upgrade

upgrade/client-upgrade: only use supported features for RBD tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

upgrade/client-upgrade: only use supported features for RBD tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #870 from ceph/wip-rbd-mirror

rbd: add some missing workunits

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Fix review comments

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #868 from ceph/wip-fio-ver

Update fio version to 2.7

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: add some missing workunits

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #863 from ceph/wip-15049

rados/singleton-nomsgr/all/11429: no mds

Merge pull request #824 from ceph/wip-multi-filesystems

Test updates for multi-filesystem support

Update fio version to 2.7

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge branch 'master' into jewel