rados: fix require_kraken_osds flag

Signed-off-by: Sage Weil <sage@redhat.com>

rados/singleton-nomsgr: drop 16113 and 13234

A stop at jewel is required, and these are pre-jewel bugs.

Signed-off-by: Sage Weil <sage@redhat.com>

rados: infernalis -> kraken is not supported; stop at jewel

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #1206 from ceph/wip-thrash-min-size-2

rados: move readwrite test out of thrash matrix

rados: move readwrite test out of thrash matrix

The thrash matrix includes min_size = 2 and size = 2.  This
is fine and good, except that the inevitable slow rados ops
that generates make readwrite.yaml issue timeouts and
fail.  Since the workload isn't meant to tolerate slow
requests, move it into the basic/ group.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #1204 from ceph/wip-jd-testing

suites/upgrade: additional point-to-point Jewel upgrade workaround

Merge pull request #1205 from ceph/wip-kraken

upgrade: set kraken osd flag on upgrade

Reviewed-by: Kefu Chai <kchai@redhat.com>

upgrade/jewel-x: set kraken flag when osds finish

Signed-off-by: Sage Weil <sage@redhat.com>

rados/singleton-nomsgr: set require_*_osds in upgrade tests

Signed-off-by: Sage Weil <sage@redhat.com>

suites/upgrade: additional point-to-point Jewel upgrade workaround

Upgrade happens in parallel with cls_rbd test run which can result in
a false-positive test failure.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

releases/kraken.yaml: set require_kraken_osds

Signed-off-by: Sage Weil <sage@redhat.com>

releases/jewel: set require_jewel_osds

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #1187 from ceph/wip-getuid

fs: enable ceph-fuse permission checking for all pjd suites

Merge pull request #1196 from ceph/wip-dump-stuck

tasks/dump_stuck: fix failure

/home/teuthworker/archive/sage-dump-stuck/457624

Merge pull request #1189 from ceph/wip-zafman-fixes

Fixes for rados suite test caused failures and improvements

Reviewed-by: Samuel Just <sjust@redhat.com>

tasks/dump_suck: mark down osds one at a time

This forces them to be unclean, *then* stale.  This ensures
that after they are both down they are both *always* unclean,
whereas previously it would be possible for them to be only
stale and not unclean.

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/dump_stuck: fix unclean count

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/dump_stuck: more verbose

Signed-off-by: Sage Weil <sage@redhat.com>

multibench: Fixed so I can use this standalone

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: do_pg_scrub() don't resubmit a request for 2 minutes

Add 6 minute fatal timeout
Warn when repair is being resubmitted because it causes races

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #1194 from ceph/wip-17416

suites/rbd: increase timeout for rbd-mirror stress test

suites/rbd: increase timeout for rbd-mirror stress test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

fs: enable ceph-fuse permission checking for all pjd suites

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #1193 from ceph/wip-16016

suites: update log whitelist for scrub msg

Merge pull request #1184 from ceph/rgw-lifecycle-testing

rgw - Add config for testing lifecycle expiration

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

suites: update log whitelist for scrub msg

Fixes: http://tracker.ceph.com/issues/16016
Signed-off-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'upstream/pull/1176/head'

* upstream/pull/1176/head:
  tasks/cephfs: add default paths to vstart_runner

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

tasks/cephfs: add default paths to vstart_runner

So that for folks with sources in typical locations
(or typical on my workstation at least!) invoking
vstart_runner is less of a mouthful.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1188 from ceph/revert-1128-wip-16805

Revert "ceph_manager: lower mon_reweight_min_bytes_per_osd to 10"

Reviewed-by: Samuel Just <sjust@redhat.com>

Revert "ceph_manager: lower mon_reweight_min_bytes_per_osd to 10"

Merge pull request #1185 from jcsp/wip-dirfrags-2

tasks/ceph: enable dirfrags in cephfs

Merge pull request #1186 from ceph/wip-ubuntu-java

suites/fs: pin java test to ubuntu

tasks/ceph: enable dirfrags in cephfs

Otherwise places we set mds_bal_frag have
no effect.

Signed-off-by: John Spray <john.spray@redhat.com>

suites/fs: remove duplicated frag_enable.yaml

...so that all these point back to basic/dirfrag

Signed-off-by: John Spray <john.spray@redhat.com>

rgw - Add config for testing lifecycle expiration

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #1177 from ceph/wip-librbd-disable-helgrind

suites/rbd: remove helgrind test cases

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1180 from ceph/wip-bluestore-code

misc fixes to test bluestore, but not actual tests in the suite (yet)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: add bdev_inject_crash_probability support

Only do the failure injection 50% of the time; otherwise, just
kill as usual.

Signed-off-by: Sage Weil <sage@redhat.com>
# Conflicts:
# tasks/ceph_manager.py

inject bdev failures when killing osd on bluestore

Signed-off-by: Sage Weil <sage@redhat.com>

objectstore/bluestore: bluefs env mirror

Catch bugs in bluefs or rocksdb env layer.

Signed-off-by: Sage Weil <sage@redhat.com>

objectstore/bluestore.yaml: debug rocksdb

Signed-off-by: Sage Weil <sage@redhat.com>

objectstore: add filestore and bluestore stubs

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/ceph_manager: respect bdev_inject_crash in kill_osd()

Note that this will only work with bluestore osds.

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/thrashosds: fix powercycle, bdev_inject_crash

The CephManager.kill_osd expects these in its config, not
Thrasher's.

Signed-off-by: Sage Weil <sage@redhat.com>

rados/basic/tasks/repair_test: enable bluestore error injection

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #1179 from ceph/wip-16688-master

Removed 11429 config

Reviewed-by: Samuel Just <sjust@redhat.com>

Removed 11429 config

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #1178 from ceph/wip-smoke-krbd-iozone

smoke: mask out unsupported image features for krbd iozone test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

smoke: mask out unsupported image features for krbd iozone test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1127 from ryneli/master

suites/rados: add test case for rockdb's envlibrados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

suites/rbd: remove helgrind test cases

Helgrind cannot properly handle Ceph's use of std::mutex --
resulting in numerous false positives and potential assertion
failures.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1169 from ceph/wip-kefu-17179

tasks: add rebuild_mondb

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #1136 from ceph/wip-16129

 rgw: add sleep to let the sync agent init

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #978 from ceph/wip-ec-fast-read

test fast ec read

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1171 from dachary/wip-16506-ceph-coverage

upgrade: ceph-test is needed for ceph-coverage

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

upgrade: ceph-test is needed for ceph-coverage

Do not exclude the ceph-test package otherwise the ceph-coverage
executable is not installed.

Fixes: http://tracker.ceph.com/issues/16506
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1118 from ceph/wip-scrub

tasks/scrub_test: use fuse path (do not assume filestore!)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1172 from trociny/wip-16717

suites/upgrade: override rbd_default_features config back to pre-Jewel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

suites/upgrade: override rbd_default_features config back to pre-Jewel

Fixes: http://tracker.ceph.com/issues/16717
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

suites: fix samba xfs symlink

I broke this in bafbeebf

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1124 from ceph/wip-offline-split

ceph_manager: test offline split via ceph-objectstore-tool

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1170 from ceph/wip-16921

rbd: move nbd test workload to separate client host from OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd: move nbd test workload to separate client host from OSDs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tasks/scrub_test: use fuse path (do not assume filestore!)

Signed-off-by: Sage Weil <sage@redhat.com>

tasks: add rebuild_mondb

* tasks/rebuild_mondb.py: this task
  1. removes all store.db on all monitors
  2. rebuild the store.db for the first mon
  3. start the first mon
  4. run mkfs on other mon
  5. and revive them
* suites/rados/singleton/all/rebuild-mon-db.yaml
  1. run rados/test.sh
  2. run rebuild_mondb task

Fixes: http://tracker.ceph.com/issues/17179
Signed-off-by: Kefu Chai <kchai@redhat.com>

suites/fs: pin java test to ubuntu

Workaround for #16640

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1153 from ceph/wip-manager-construct

tasks/ceph: construct CephManager earlier

Merge pull request #1109 from batrick/fs-fuse_default_permissions

fs: fuse_default_permissions = 0 for kernel build test

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #1163 from ceph/wip-kfsx-exclusive-lock

krbd: enable exclusive-lock feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1160 from ceph/wip-no-btrfs

suites: switch filesystem tests from btrfs to xfs

Merge pull request #1165 from tchaikov/wip-16157

rados/singleton-nomsgr/all/11429: wait until clean before checking log

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tasks/ceph_manager: allow ObjectStoreTool to not revive OSD after done with it

rebuild_mondb task is not able to offer OSD with any monitor alive. so
self.manager.revive_osd() will always timeout after calling cot.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rados/singleton-nomsgr/all/11429: wait until clean before checking log

to make sure that load_pgs() is finished before checking its output

Fixes: http://tracker.ceph.com/issues/16157
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1162 from SUSE/wip-make-rpm-os-release

buildpackages/make-rpm.sh: use /etc/os-release

Reviewed-by: Loic Dachary <ldachary@redhat.com>

buildpackages/make-rpm.sh: adjust Source0 spec file line on SUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buildpackages: use unadulterated make-dist

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buildpackages/make-rpm.sh: use /etc/os-release

The lsb_release binary is deprecated and requires installation of packages.

The /etc/os-release file is guaranteed to be present.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

krbd: enable exclusive-lock feature

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1161 from ceph/wip-unmap-separate-client

krbd/unmap: put client.0 on a separate remote

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #1132 from ceph/wip-rgw-remove-btrfs

rgw: we don't support btrfs any more

krbd/unmap: put client.0 on a separate remote

Otherwise a pre-single-major kernel override is a headache,
particularly with non-standard yaml configs.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

suites: switch filesystem tests from btrfs to xfs

btrfs threw up occasional failures, and the long term
target will be bluestore instead of btrfs.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1158 from ceph/wip-fio-unmap-devices

tasks/rbd_fio: unmap rbd devices on cleanup

tasks/rbd_fio: unmap rbd devices on cleanup

Not doing so leads to issues and can interfere with subsequent jobs.
One example is the invocation of vgs(8) during the inital test setup:
it will issue a read to the left-behind rbd device(s) whose backing
cluster is long gone, locking up the job.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

tasks/rbd_fio: don't use sudo unnecessarily

Creating and cloning images doesn't require sudo.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1155 from SUSE/wip-suse-ceph-test

buildpackages: fix make-rpm.sh for SUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>

buildpackages: force ceph-test build on SUSE

The ceph-test package is required for teuthology. It is disabled to speed up
the build in OBS, but here we need it enabled unconditionally.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buildpackages: fix RPM generation script to support opensuse spec file

Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 4692f995ba6967796d5dd2f5fd85fde8705a1ccc)

Merge pull request #1112 from jcsp/wip-16909

tasks/cephfs: test standby replay teardown on rank stop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/ceph: construct CephManager earlier

Previously, if errors occurred during healthy(), then
the finally block would invoke osd_scrub_pgs, which relies
on CephManager being constructed, and it would die, hiding
the original exception.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1149 from ceph/wip-krbd-unmap-fixes

krbd/unmap: assorted pre-single-major fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: add sleep to let the sync agent to init

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

krbd/unmap: override client.0 only

The unmap test uses one remote, so the end result is the same.
However, overriding the most specific role is nicer and allows
scheduling with

  kernel:
    client:
       branch: testing

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd/unmap: set tunables to bobtail

Kernel 3.13, which is used in pre-single-major.yaml test, doesn't
support firefly tunables (default in jewel, up from bobtail tunables).
This went unnoticed for a while because of a kernel task regression -
the pre-single-major override was ignored.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd/unmap: assert that pre-single-major kernel is installed

Similarly to how single-major-off.yaml and single-major-on.yaml check
the value of /sys/module/rbd/parameters/single_major, assert that it's
not there for pre-single-major.yaml.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1146 from dachary/wip-17152-openstack

openstack: add disks where necessary

Reviewed-by: Nathan Cutler <ncutler@suse.com>

openstack: add disks where necessary

Ref: http://tracker.ceph.com/issues/17152

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1141 from rjfd/wip-test-buildpackages

buildpackages: make make-{deb,rpm}.sh aware of cmake

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.cz>

ceph_manager: test offline split via ceph-objectstore-tool

When killing an osd, split all pools with a low threshold.
This will slow down tests, but should not impact correctness.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1142 from dachary/wip-17116-workunit

workunit: allow parallel git clone

Reviewed-by: Nathan Cutler <ncutler@suse.cz>