ceph: wait for CephFS to be healthy before proceeding

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

buildpackages: build ceph packages and upload them

Build Ceph packages from source for the required revision, os_type,
os_version and architecture and upload them to the gitbuilder
repository.

http://tracker.ceph.com/issues/13031 Fixes: #13031

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #560 from dachary/wip-13026-hammer-stop

hammer-x: test CEPH_FEATURE_HAMMER_0_94_4

Merge pull request #564 from ceph/wip-hammer-x

fixes for hammer-x

upgrade/hammer-x/split-*: ignore osdmap encode errors

This is normal during upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/hammer-x: hammer test.sh before and during upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

hammer-x: test CEPH_FEATURE_HAMMER_0_94_4

http://tracker.ceph.com/issues/13026 Refs: #13026

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #563 from ceph/wip-13060

fix test for 11429 (and now 13060)

suites/rados/singleton-nomsgr/all/11429.yaml: double-hop and fix

- simplify this.. lots of extra cruft we don't need
- restart twice at hammer to ensure that we can still load pgs
  post-upgrade
- do the same for the final version.

Fixes: #11429 (again, for ~infernalis)
Fixes: #13060
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #562 from ceph/wip-radosbench-cleanup

tasks/radosbench: allow cleanup: false

tasks/radosbench: allow cleanup: false

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #561 from ceph/wip-sudo

sudo ceph

Merge pull request #559 from ceph/wip-10328

leaks

suites: sudo ceph

Signed-off-by: Sage Weil <sage@redhat.com>

tasks: sudo ceph for cli

/var/run/ceph is 770.  This is mainly necessary for any
interaction with the daemon sockets, but it is what users do
and it may avoid log noise.

Signed-off-by: Sage Weil <sage@redhat.com>

rados/singleton-nomsgr/all/valgrind-leaks: verify leak checking works

Fixes: #10328
Signed-off-by: Sage Weil <sage@redhat.com>

ceph: add option to expect valgind errors and fail if there are none

See #10328
Signed-off-by: Sage Weil <sage@redhat.com>

distros: alias a-supported-distro.yaml to CentOS 7

Some jobs are distribution agnostic and it is meaningless to run them on
all supported distributions. For such jobs it is convient to require
than they run on a supported distribution but we don't care which. This
can be done by including a-supported-distro.yaml in the suite sub
directory instead of linking to distros/supported.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #558 from ceph/wip-12973

rados: 'failed to encode ...' warnings are normal on upgrades

Tested-by: Kefu Chai <kchai@redhat.com>

rados: 'failed to encode ...' warnings are normal on upgrades

Fixes: #12973
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #542 from ceph/wip-12551-cephfs-truncate-sec

Adds test for ceph/pull/5442

Reviewed-by: Sage Weil <sage@redhat.com>

suites/rados/singleton-nomsgr/all/cache-fs-trunc.yaml

Verify the cache tier doesn't clobber truncate_{seq,size}

Merge pull request #557 from ceph/wip-12563

suites/hammer/older: do not upgrade client while running RBD import/export

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #550 from ceph/wip-12806

tasks/qemu: avoid adding duplicated entries to /etc/exports

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

suites/hammer/older: do not upgrade client while running RBD import/export

Between the Giant and Hammer releases, the symbols exported from librados/librbd
were cleaned up.  This creates a short window of time between installing
upgraded packages where symbol lookup errors could occur.

Fixes: #12563
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #556 from ceph/wip-stuck-pg-dump

tasks/ceph_manager: dump pgs when recover times out

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tasks/ceph_manager: dump pgs when recover times out

It is really hard to map a stuck recovery back to the pgs that
are stuck.  This will make it easy.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #541 from XinzeChi/wip-rebug

fix rebug rados

Reviewed-by: Loic Dachary <ldachary@redhat.com>

distros/supported: remove jessie until paramiko issue is resolved

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/wip-ceph-deploy'

Merge pull request #552 from ceph/wip-12878

tasks/scrub_test: sudo ls ...

Reviewed-by: Kefu Chai <kchai@redhat.com>

tasks/divergent_prior[2]: fix rados get syntax

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/reg11184.py: fix rados get syntax

no -o

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #553 from ceph/wip-s3tests

s3tests: use ceph-$foo branches

tasks/s3tests: use ceph-$foo branches

We need to be able to merge things into s3-tests master that
break rgw.  Create ceph-foo branches (ceph-master,
ceph-infernalis, etc.) and use those instead.

Signed-off-by: Sage Weil <sage@redhat.com>

kcephfs: include debug mds 20 everywhere

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

tasks/scrub_test: sudo ls ...

/var/lib/ceph/osd/* is owned by ceph now; ubuntu user can't read
it.

Fixes: #12878
Signed-off-by: Sage Weil <sage@redhat.com>

tasks/ceph_deploy: install ceph-test

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #503 from guangyy/wip-12316-n

erasure coding: LRC thrasher & fast read thrasher

Reviewed-by: David Zafman <dzafman@redhat.com>

suites/rados/singleton-nomsgr/all/11429: upgrade to hammer first

Merge pull request #551 from ceph/wip-12820

tasks/cephfs: fix mount_timeout config loading

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: fix mount_timeout config loading

Previously were defaulting to a string, which
always compared greater than elapsed, so never
timed out.

Fixes: #12820
Signed-off-by: John Spray <john.spray@redhat.com>

tasks/ceph_deploy: work with systemd

This is ugly, but I'm not going to rewrite this right now.

Signed-off-by: Sage Weil <sage@redhat.com>

distros: add debian 8 jessie, remove wheezy from supported list

Wheezy only has experiemntal c++11 support; it's not new enough.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-deploy: install w/ ceph-deploy, not install.py

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #546 from dachary/wip-11881-multipath

ceph-disk workunit

tasks/qemu: avoid adding duplicated entries to /etc/exports

try deleting leftover entries for previous job

Fixes: #12806
Signed-off-by: Yan, Zheng <zyan@redhat.com>

rados/basic/msgr: enable all experimental features if random/async

Signed-off-by: Kefu Chai <kchai@redhat.com>

Revert "rados/basic/msgr: enable all experimental features if random/async"

This reverts commit 40db8376ab42756e7c2f45469153099183439744.

Getting errors trying to schedule with this commit.

Merge remote-tracking branch 'gh/next'

Merge pull request #548 from tchaikov/wip-enable-exp-features

rados/basic/msgr: enable all experimental features if random/async

Reviewed-by: Samuel Just <sjust@redhat.com>

rados/basic/msgr: enable all experimental features if random/async

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #547 from ceph/wip-12822

tasks/cephfs: timeout on wait in test_network_death

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: timeout on wait in test_network_death

So that we can get a failure instead of a hang in

Signed-off-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'origin/next'

Merge pull request #545 from ceph/wip-12777-next

tasks/cephfs: fix test_pool_perm teardown

implement the ceph-disk suite

Run the ceph-disk workunit with a mon setup with ceph-deploy. It is
prefered to the install done by the ceph tasks because it relies on the
init system and udev rules which are going to be tested by the ceph-disk
workunit.

http://tracker.ceph.com/issues/11881 Refs: #11881

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-deploy: implement only_mon

If only_mon is true, do not attempt to deploy osds.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-deploy: implement keep_running

When keep_running is true, do not shutdown the cluster, leave it as it
is for other workunits or tasks to use. This effectively allows the
ceph-deploy task to be used as a helper to deploy clusters.

The call to build_ceph_cluster is simplified by giving it the whole
configuration dictionary instead of re-building one with selected arguments.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-deploy: ceph report on failure

When ceph-deploy fails, run ceph report to get more information about
the state of the cluster at the time of the failure.

Signed-off-by: Loic Dachary <loic@dachary.org>

Update rados_api.yaml(cherry picked from commit 5b9d41e3ea0f07171ce59f0bb1f5636733452062)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Initial check in to add firefly-hammer-x suite #12641

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 998ba8b58c5c70e8a110c3e1cf29cd24ae9692b9)

Merge pull request #528 from ceph/wip-12641

Initial check in to add firefly-hammer-x suite #12641

Merge pull request #534 from ceph/wip-12625-master

Added singleton hammer-x test to address #525

Reviewed-by: Samuel Just <sjust@redhat.com>

tasks/cephfs: fix test_pool_perm teardown

Fixes: #12777
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #544 from ceph/wip-ms-type-async

rados: test both simple, async, and a mix

Reviewed-by: Samuel Just <sjust@redhat.com>

rados: test both simple, async, and a mix

- in general, test simple vs async vs random
- not for msgr-less workloads
- not for thrash-erasure-*.. the regular thrash
should cover it.

Signed-off-by: Sage Weil <sage@redhat.com>

fix rebug rados

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>

Merge pull request #540 from ceph/wip-hobj-sort

suites/rados/thrash: randomize hobject sort order

Reviewed-by: Samuel Just <sjust@redhat.com>

suites/rados/thrash: randomize hobject sort order

Signed-off-by: Sage Weil <sage@redhat.com>

Update rados_api.yaml

Initial check in to add firefly-hammer-x suite #12641

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #538 from ceph/wip-12758

ceph_deploy: if a branch is given, use it when installing rgw

ceph_deploy: if a branch is given, use it when installing rgw

Fixes: http://tracker.ceph.com/issues/12758
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge remote-tracking branch 'gh/next'

Merge pull request #523 from dachary/wip-10887-shec

erasure-code: CEPH_FEATURE_ERASURE_CODE_PLUGINS_V3 integration tests

Reviewed-by: Takeshi Miyamae <miyamae.takeshi@jp.fujitsu.com>

Merge pull request #526 from ceph/wip-stashed-layout

tasks/cephfs: implement TestDataScan.test_stashed_layout

Merge pull request #536 from ceph/wip-11881-multipath

ceph-deploy: hints for OpenStack

update the list of supported distributions

CentOS 6 and Ubuntu 12.04 are not support after Infernalis.
CentOS 7 is supported starting with Infernalis.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #537 from dachary/wip-supported

update the list of supported distributions

Reviewed-by: Sage Weil <sage@redhat.com>

update the list of supported distributions

CentOS 6 and Ubuntu 12.04 are not support after Infernalis.
CentOS 7 is supported starting with Infernalis.

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-deploy: hints for OpenStack

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #531 from ceph/wip-mds-debug

tasks/kcephfs: enable MDS debug

Added singleton hammer-x test to address #12625

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #533 from ceph/revert-525-wip-12625-master

Revert "Added singleton hammer-x test to address #12625"

Revert "Added singleton hammer-x test to address #12625"

Merge pull request #525 from ceph/wip-12625-master

Added singleton hammer-x test to address #12625

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #514 from ceph/wip-cd-add-branch-install

 - Use branch from config for ceph-deploy testing

Reviewed-by: Travis Rhoden <trhoden@redhat.com>

Merge pull request #529 from ceph/wip-calamari-cli

calamari_setup: install "cli" utils on Calamari node

Reviewed-by: Dan Mick <dmick@redhat.com>

tasks/kcephfs: enable MDS debug

To help us debug #11482

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: implement TestDataScan.test_stashed_layout

This is for testing how cephfs-data-scan handles
the new 'layout' xattr when found during recovery.

Signed-off-by: John Spray <john.spray@redhat.com>

calamari_setup: install "cli" utils on Calamari node

In our RHCS 1.3 ceph-deploy docs, we tell users to run "ceph-deploy
install --cli" on their calamari admin node, but our smoke test wasn't
actually doing this.

See https://bugzilla.redhat.com/1252929 , "[Ubuntu 1.3.0] - ceph-deploy
install --no-adjust-repos --cli `hostname` is failing with a Traceback
error"

Merge pull request #527 from dmick/master

calamari_setup: change to use ceph-deploy repo command

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

calamari_setup: change to use ceph-deploy repo command

Fixes: #12643
Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #524 from ceph/wip-kdreyer-calamari-1.2

calamari_setup: handle RHCS 1.2 (missing /mnt/MON)

Reviewed-by: Dan Mick <dmick@redhat.com>

Added singleton hammer-x test to address #12625

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #445 from dachary/wip-11666-erasure-code-lrc

erasure-code: lrc plugin workload

Reviewed-by: Samuel Just <sjust@redhat.com>

erasure-code: CEPH_FEATURE_ERASURE_CODE_PLUGINS_V3 integration tests

http://tracker.ceph.com/issues/10887 Refs: #10887

Signed-off-by: Loic Dachary <ldachary@redhat.com>

fix indentation error in docstring

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit af973259eff9ffbbbe9aa209c3b3dd2d80d7219b)

do not ommit syntax errors in linter checks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit db69142b9078fd9ed35fb3a542fc340dae9be665)