Merge pull request #1178 from ceph/wip-smoke-krbd-iozone

smoke: mask out unsupported image features for krbd iozone test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

smoke: mask out unsupported image features for krbd iozone test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1127 from ryneli/master

suites/rados: add test case for rockdb's envlibrados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1169 from ceph/wip-kefu-17179

tasks: add rebuild_mondb

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #1136 from ceph/wip-16129

 rgw: add sleep to let the sync agent init

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #978 from ceph/wip-ec-fast-read

test fast ec read

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1171 from dachary/wip-16506-ceph-coverage

upgrade: ceph-test is needed for ceph-coverage

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

upgrade: ceph-test is needed for ceph-coverage

Do not exclude the ceph-test package otherwise the ceph-coverage
executable is not installed.

Fixes: http://tracker.ceph.com/issues/16506
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1118 from ceph/wip-scrub

tasks/scrub_test: use fuse path (do not assume filestore!)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1172 from trociny/wip-16717

suites/upgrade: override rbd_default_features config back to pre-Jewel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

suites/upgrade: override rbd_default_features config back to pre-Jewel

Fixes: http://tracker.ceph.com/issues/16717
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

suites: fix samba xfs symlink

I broke this in bafbeebf

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1124 from ceph/wip-offline-split

ceph_manager: test offline split via ceph-objectstore-tool

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1170 from ceph/wip-16921

rbd: move nbd test workload to separate client host from OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd: move nbd test workload to separate client host from OSDs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tasks/scrub_test: use fuse path (do not assume filestore!)

Signed-off-by: Sage Weil <sage@redhat.com>

tasks: add rebuild_mondb

* tasks/rebuild_mondb.py: this task
  1. removes all store.db on all monitors
  2. rebuild the store.db for the first mon
  3. start the first mon
  4. run mkfs on other mon
  5. and revive them
* suites/rados/singleton/all/rebuild-mon-db.yaml
  1. run rados/test.sh
  2. run rebuild_mondb task

Fixes: http://tracker.ceph.com/issues/17179
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1153 from ceph/wip-manager-construct

tasks/ceph: construct CephManager earlier

Merge pull request #1109 from batrick/fs-fuse_default_permissions

fs: fuse_default_permissions = 0 for kernel build test

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #1163 from ceph/wip-kfsx-exclusive-lock

krbd: enable exclusive-lock feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1160 from ceph/wip-no-btrfs

suites: switch filesystem tests from btrfs to xfs

Merge pull request #1165 from tchaikov/wip-16157

rados/singleton-nomsgr/all/11429: wait until clean before checking log

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tasks/ceph_manager: allow ObjectStoreTool to not revive OSD after done with it

rebuild_mondb task is not able to offer OSD with any monitor alive. so
self.manager.revive_osd() will always timeout after calling cot.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rados/singleton-nomsgr/all/11429: wait until clean before checking log

to make sure that load_pgs() is finished before checking its output

Fixes: http://tracker.ceph.com/issues/16157
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1162 from SUSE/wip-make-rpm-os-release

buildpackages/make-rpm.sh: use /etc/os-release

Reviewed-by: Loic Dachary <ldachary@redhat.com>

buildpackages/make-rpm.sh: adjust Source0 spec file line on SUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buildpackages: use unadulterated make-dist

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buildpackages/make-rpm.sh: use /etc/os-release

The lsb_release binary is deprecated and requires installation of packages.

The /etc/os-release file is guaranteed to be present.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

krbd: enable exclusive-lock feature

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1161 from ceph/wip-unmap-separate-client

krbd/unmap: put client.0 on a separate remote

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #1132 from ceph/wip-rgw-remove-btrfs

rgw: we don't support btrfs any more

krbd/unmap: put client.0 on a separate remote

Otherwise a pre-single-major kernel override is a headache,
particularly with non-standard yaml configs.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

suites: switch filesystem tests from btrfs to xfs

btrfs threw up occasional failures, and the long term
target will be bluestore instead of btrfs.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1158 from ceph/wip-fio-unmap-devices

tasks/rbd_fio: unmap rbd devices on cleanup

tasks/rbd_fio: unmap rbd devices on cleanup

Not doing so leads to issues and can interfere with subsequent jobs.
One example is the invocation of vgs(8) during the inital test setup:
it will issue a read to the left-behind rbd device(s) whose backing
cluster is long gone, locking up the job.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

tasks/rbd_fio: don't use sudo unnecessarily

Creating and cloning images doesn't require sudo.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1155 from SUSE/wip-suse-ceph-test

buildpackages: fix make-rpm.sh for SUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>

buildpackages: force ceph-test build on SUSE

The ceph-test package is required for teuthology. It is disabled to speed up
the build in OBS, but here we need it enabled unconditionally.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

buildpackages: fix RPM generation script to support opensuse spec file

Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 4692f995ba6967796d5dd2f5fd85fde8705a1ccc)

Merge pull request #1112 from jcsp/wip-16909

tasks/cephfs: test standby replay teardown on rank stop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/ceph: construct CephManager earlier

Previously, if errors occurred during healthy(), then
the finally block would invoke osd_scrub_pgs, which relies
on CephManager being constructed, and it would die, hiding
the original exception.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1149 from ceph/wip-krbd-unmap-fixes

krbd/unmap: assorted pre-single-major fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: add sleep to let the sync agent to init

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

krbd/unmap: override client.0 only

The unmap test uses one remote, so the end result is the same.
However, overriding the most specific role is nicer and allows
scheduling with

  kernel:
    client:
       branch: testing

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd/unmap: set tunables to bobtail

Kernel 3.13, which is used in pre-single-major.yaml test, doesn't
support firefly tunables (default in jewel, up from bobtail tunables).
This went unnoticed for a while because of a kernel task regression -
the pre-single-major override was ignored.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

krbd/unmap: assert that pre-single-major kernel is installed

Similarly to how single-major-off.yaml and single-major-on.yaml check
the value of /sys/module/rbd/parameters/single_major, assert that it's
not there for pre-single-major.yaml.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1146 from dachary/wip-17152-openstack

openstack: add disks where necessary

Reviewed-by: Nathan Cutler <ncutler@suse.com>

openstack: add disks where necessary

Ref: http://tracker.ceph.com/issues/17152

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1141 from rjfd/wip-test-buildpackages

buildpackages: make make-{deb,rpm}.sh aware of cmake

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.cz>

ceph_manager: test offline split via ceph-objectstore-tool

When killing an osd, split all pools with a low threshold.
This will slow down tests, but should not impact correctness.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1142 from dachary/wip-17116-workunit

workunit: allow parallel git clone

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

workunit: allow parallel git clone

When multiple client run in parallel on the same machine and they
try to get workunits from a repository that is not github, they must
git clone in a directory that is suffixed as srcdir. Otherwise they
will conflict with each other.

Fixes: http://tracker.ceph.com/issues/17116
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1137 from ceph/wip-fix-console

Fix console handling in tasks

Reviewed-by: Dan Mick <dmick@redhat.com>

Update console checking, and DRY

Signed-off-by: Zack Cerza <zack@redhat.com>

Drop unnecessary console handling code

Remote.console is provided by teuthology now
http://tracker.ceph.com/issues/17124
Fixes: 17124
Signed-off-by: Zack Cerza <zack@redhat.com>

buildpackages: make make-{deb,rpm}.sh aware of cmake

Fixes: http://tracker.ceph.com/issues/16819
Signed-off-by: Ricardo Dias <rdias@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rgw: add debug info when comparing bucket metadata

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #1133 from SUSE/wip-17075

Revert "packages.yaml: reflect python-ceph package split"

Merge pull request #1135 from athanatos/wip-stdout

ceph_manager: do not use StringIO instance as default argument

ceph_manager: do not use StringIO instance as default argument

Otherwise, all invokations share the *same* StringIO instance
leading to some very silly bugs.

Signed-off-by: Samuel Just <sjust@redhat.com>

Revert "packages.yaml: reflect python-ceph package split"

This reverts commit 0216330568fc1dec2e866bbfd1e49dc42266e1ab.

Fixes: http://tracker.ceph.com/issues/17075
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rgw: we don't support btrfs any more

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #1128 from ceph/wip-16805

ceph_manager: lower mon_reweight_min_bytes_per_osd to 10

Reviewed-by: Samuel Just <sjust@redhat.com>

ceph_manager: lower mon_reweight_min_bytes_per_osd to 10

otherwise monitor could reject the command:
```
Refusing to reweight: we only used 588084 kb used across all osds!
```
if the average used space is smaller than
`mon_reweight_min_bytes_per_osd`.

Fixes: http://tracker.ceph.com/issues/16805
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1122 from ceph/wip-fixbiglogs

Fix big logs in rados thrash runs

Reviewed-by: Samuel Just <sjust@redhat.com>

add test case for envlibrados

Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>

Merge pull request #1116 from ceph/wip-16529

suites/upgrade: point-to-point Jewel upgrade workaround

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

revive_osd: Suppress output used to wait for OSD to be fully started

Signed-off-by: David Zafman <dzafman@redhat.com>

Thrashosds: Turn off output when doing dump_ops testing

Needed to add stdout option to pass down to run()

Signed-off-by: David Zafman <dzafman@redhat.com>

suites/upgrade: point-to-point Jewel upgrade workaround

The v10.2.x cls_rbd test case will not pass against a v10.2.0
OSD. Disable the offending test.

Fixes: http://tracker.ceph.com/issues/16529
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1117 from ceph/wip-16975

jewel-x: use jewel-compatible librados tests for upgraded clients

Reviewed-by: Yuri Weinstein <yuri.weinstein@gmail.com>

Merge pull request #1119 from ceph/wip-rgw-17025

rgw: data_extra_pool is unique per zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: data_extra_pool is unique per zone

Fixes: http://tracker.ceph.com/issues/17025
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

jewel-x: use jewel-compatible librados tests for upgraded clients

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1115 from ceph/wip_add_sterr

Added stderr to the log file in addition to stdout

Reviewed-by: Dan Mick <dmick@redhat.com>

Added stderr to the log file in addition to stdout

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #1110 from fullerdj/wip-djf-20160808

cephfs/vstart_runner: add -w option to ps

Reviewed-by: John Spray <john.spray@redhat.com>

tasks/cephfs: test standby replay teardown on rank stop

Fixes: http://tracker.ceph.com/issues/16909
Signed-off-by: John Spray <john.spray@redhat.com>

cephfs/vstart_runner: add -ww option to ps

vstart_runner can't find arguments to ceph daemons to identify them with
ps -x because commands are cut off at terminal width. Add -ww for wide
output.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

fs: fuse_default_permissions = 0 for kernel build test

Ported from Zheng's #22: 2e283ce6d77c66a564d807f0b0909b238c4fb5f0

"This can reduce the test time becuase it avoids sending getattr request
whenever the kernel checks inode permission."

This is part of an effort to eliminate unnecessary differences between
multimds and fs suites.

Signed-off-by: Patrick Donnelly <batrick@batbytes.com>

Merge pull request #1107 from ceph/wip-16879

suites/fs: fix log whitelist for inotable repair

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #1084 from ceph/wip-16640

task/packages: install libcephfs1-dev

suites/fs: fix log whitelist for inotable repair

Fixes: http://tracker.ceph.com/issues/16879
Signed-off-by: John Spray <john.spray@redhat.com>

task/packages: install libcephfs-dev

The way we currently build the cephfs java
bindings depends on the unversioned .so file
which is in the -dev package, not the
main one.

Fixes: http://tracker.ceph.com/issues/16640
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1082 from ceph/wip-mds-minor

tasks/cephfs: check cache size health warning

tasks/cephfs: check cache size health warning

Fortunately we already have a test that creates the condition,
so just tweak it to exceed the 150% threshold for the health warning,
and check that the health message appears.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1106 from ceph/wip-flake8

fix flake8 errors

Reviewed-by: Samuel Just <sjust@redhat.com>

cephfs/filesystem: remove unused var

flake8 passes now

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tox.ini: exclude .tox dir from flake8

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1105 from athanatos/wip-ec-pg-log

update diverge priors tests - this is handled by persisting the missing set now

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

suites: fix asok_dump_tree.yaml

This had an extra ceph and install task.

Signed-off-by: John Spray <john.spray@redhat.com>

cephfs: add test for dump tree admin socket command

Add test_dump_tree to validate the MDS admin socket command
dump tree. Add a parameter for path_to_ino to enable the use of
lstat() instead of stat()

See: http://tracker.ceph.com/issues/11171
See: https://github.com/ceph/ceph/pull/9925
Signed-off-by: Douglas Fuller <dfuller@redhat.com>

suites/fs: log whitelist for inotable repair

Signed-off-by: John Spray <john.spray@redhat.com>

cephfs: Added a inotable repair case to the test

Merge pull request #1099 from ceph/wip-fix-bad-check

Fix rechecking of health in loop

Merge pull request #1098 from smithfarm/wip-16808

packages/packages.yaml: drop ceph-fs-common-dbg

Reviewed-by: John Spray <john.spray@redhat.com>

Fix rechecking of health in loop

Use the newer stdout value for checking
ceph health again.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

packages/packages.yaml: drop ceph-fs-common-dbg

Meant to be merged together with https://github.com/ceph/ceph/pull/10433

Fixes: http://tracker.ceph.com/issues/16808
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1094 from ceph/wip_reduce_rados

Reduce number of jobs for hammer branch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1074 from SUSE/wip-python-split

packages.yaml: reflect python-ceph package split