tasks/ceph_manager: enable log for ceph_objectstore_tool

Signed-off-by: Sage Weil <sage@redhat.com>

powercycle: fuse_default_permissions = 0 for kernel build

Signed-off-by: Sage Weil <sage@redhat.com>

Fixing branch assumptions

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #152 from dachary/wip-9608-ec-default-parallel-giant

ec-rados-default is not fit for dumpling parallel upgrade

ec-rados-default is not fit for dumpling parallel upgrade

Because it may race with the upgrade and end up being unable to perform.

http://tracker.ceph.com/issues/9608 Fixes: #9608

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit b77ef02bc5ca623d2c5577c2caafcf3720ddd2ec)

added upgrade suite for dumpling-firefly-giant

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
(cherry picked from commit c7603c0b7871477110946d2d47ea67dda1b7a9bf)

Conflicts:

suites/upgrade/firefly-giant-x/parallel/5-upgrade-sequence/upgrade-by-daemon.yaml
suites/upgrade/firefly-giant-x/parallel/5-upgrade-sequence/upgrade-by-type.yaml

smoke: fix pjd debug conf syntax

Needs the section there or else ceph.py barfs at runtime

Signed-off-by: Sage Weil <sage@redhat.com>

tasks: wait for active after mds restart

May have been causing spurious failures on
trying to read session state after MDS restart (
session list isn't populated until recovery is
complete)

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #147 from dachary/wip-ec-feature-plugin-v2-giant

erasure-code: assert that firefly-x/stress-split cannot load lrc

erasure-code: assert that firefly-x/stress-split cannot load lrc

Because it would crash existing plugins.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit d6f02d604840b448158a1470b6bb1f6d0550e97f)

added workload tag

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #142 from ceph/wip-9284

Tests for #9284, FUSE client capability handling

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #146 from dachary/wip-9298-rgw-ec-starvation-giant

rgw: increase the number of OSDs for erasure coded pools

rgw: increase the number of OSDs for erasure coded pools

So that there never is less than 4 osd which is the minimum when trying
erasure coded workloads that only work with 4 osd.

http://tracker.ceph.com/issues/9298 Fixes: #9298

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

suites: enable mds_client_limits test

Requires same cluster etc as 'recovery', though
we could now do with better name than that.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: wait for mds active before mounting clients

To make the logs clearer when trying to work out
if/when something went wrong, rather than always
having client logs start with some failures.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: rename FuseMount.get_client_id to get_global_id

'client_id' was ambiguous because in other places it
meant the '0' in client.0, whereas here it means
the runtime-generated global ID of the client.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: add mds_client_limits

New CephFS tests for the behaviour of the system while
enforcing its resource limits.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: generalise CephFSTestCase

Some of this stuff could be even more general for embedding
unittest-style suites, but for the moment let's keep the cephfs
stuff in a walled garden.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: generalize config writing for Filesystem

Signed-off-by: John Spray <john.spray@redhat.com>

Update giant.yaml

fixed typos(cherry picked from commit 1f950d2bf4dd76d671ab1c85a87a5f1175305635)

Update giant.yaml(cherry picked from commit 90b6eec23fad6b3af1305de9c59a4759e076ebd3)

Update giant.yaml

Added comments(cherry picked from commit 3dd1314c6fa70fefc4ea8c65c33bac1ad3ba267a)

Create dumpling-giant-x stress-split upgrade suite

Fixes: 9357
Signed-off-by: Warren Usui <warren.usui@inktank.com>
(cherry picked from commit c729372aaae82b8fb0a31a42f24d96924078282f)

Update giant.yaml(cherry picked from commit 35b3f4617c54dcb9705ca31489e2348bdb0e18b5)

dumpling-giant-x: use giant version of workunits

The firefly version of test_librbd_python at least does
not work on giant.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 8755466c4b7c2402e3265ecb04a8300a4d0c146f)

Merge pull request #143 from ceph/wip-migration-test

tasks: more substantial IO for journal migration

Reviewed-by: Greg Farnum <greg@inktank.com>

suites: turn on debugging for smoke cfuse pjd test

Signed-off-by: Greg Farnum <greg@inktank.com>

tasks: escaping '*' when deleting files

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: more substantial IO for journal migration

...so that there will at least be multiple segments
in the log during the rewrite.

Also make the test stricter by checking that
cephfs-journal-tool can happily read the resulting
journal.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #141 from ceph/wip-9423

Fix #9423 - race in mds_client_recovery

Merge pull request #140 from ceph/wip-9177

suites/fs: update to latest mdtest

suites/fs: update to latest mdtest

They appear to have (accidentally?) fixed whatever
was crashing.

Fixes: #9177
Signed-off-by: John Spray <john.spray@redhat.com>

tasks: fix race in test_stale_caps

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: typo in mds_client_recovery

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: handle failure cleanly in test_stale_caps

Previously would fail because the cap waiter
completed too soon, without noticing that the
reason it completed quickly was because it failed.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #119 from ceph/wip_9352

Wip 9352

Merge pull request #118 from dachary/wip-more-osds

add osds to firefly-x upgrade suite

Updated README

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

First draft for a new suite dumpling-giant-x

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

added a test to use single disk for both data and journal.

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

added dmcrypt and separate_journal_disk options

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #112 from ceph/wip-8231-forreview

Wip 8231 forreview

Reviewed-by: Samuel Just <sam.just@inktank.com>

tasks: fix mount race in mds_client_recovery

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #117 from dachary/wip-erasure-code-workloads

erasure-code: do not schedule isa workloads just yet

add osds to firefly-x upgrade suite

so that the lrc plugin has enough to work with

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: do not schedule isa workloads just yet

they should be selectively activated, depending the isa plugin availability

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

ceph_manager: Add test code to use export/import to move a pg

Check for more than 1 osd down and randomize on chance_move_pg (100%)
For now only export from older down osd to newly down osd to avoid missing map

Signed-off-by: David Zafman <david.zafman@inktank.com>

ceph_manager: Implement export/import when thrasher kills an osd

Use list-pgs to avoid races by seeing actual pgs present

Signed-off-by: David Zafman <david.zafman@inktank.com>

ceph_objectstore_tool: Add task for testing of tool of the same name

Based on ceph/src/test/ceph_objectstore_tool.py but only does
replicated pool testing and doesn't test argument validation.

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge pull request #115 from dachary/wip-erasure-code-workloads

erasure-code: do not schedule lrc workloads just yet

erasure-code: do not schedule lrc workloads just yet

testing after renaming is in progress

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #114 from dachary/wip-erasure-code-workloads

DNM: erasure-code: add various erasure-code workloads

erasure-code: add to suites/upgrade/*-x final workloads

And replace the ec-readwrite.yaml custom workloads with links to the
workloads found in the erasure-code directory.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: add various erasure-code workloads

The erasure-code directory is populated with files describing rados
workloads suitable to test the default erasure code plugins. They are
linked to the rados suite. The pre-existing rados workloads from the
rados suite are replaced with those. Two workloads only differed by the
fact that one of them had less operations than the other. The more
complete set of operations is kept and used for all workloads to
maximize coverage.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #113 from ceph/ceph-deploy-dmcrypt

Added dmcrypt option and ability to choose same or different disk for ce...

added config_options to enable adding ceph config file entries to ceph-deploy task

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Added dmcrypt option and ability to choose same or different disk for ceph journal

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #101 from ceph/wip-7810

Wip 7810

Merge pull request #111 from ceph/wip-fix-stress_split

Moved mon.c to the first host with mon.a and mon.b to address the issue

Moved mon.c to the first host with mon.a and mon.b to address the issue
Sage found:
"... supposed to have half dumpling, half x osds. but the steps that upgrade and restart the mons upgrade the packages on the second host (which should remain dumpling w/ osd 3,4,5)"

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #110 from ceph/wip-fix-stress_split

Moved mon.c to teh first host with mon.a and mon.b to address the issue ...

Moved mon.c to teh first host with mon.a and mon.b to address teh issue Sage found
"... supposed to have half dumpling, half x osds.  but the steps that upgrade and restart the mons upgrade the packages on the second host (which should remain dumpling w/ osd 3,4,5)"

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #109 from ceph/wip_fix_firefly-x

Added sequential: clause, so test stop thrashing the mon after it runs t...

Fixed typo

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added sequential: clause, so test stop thrashing the mon after it runs that one workload

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #105 from ceph/wip-boto

tasks/s3tests: push boto config with idle_timeout setting

Merge pull request #106 from ceph/wip-9091-wusui

mplement ceph.created_pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #107 from dachary/wip-9209-ec-workload

erasure-code: add jerasure k=3 m=1 workload

erasure-code: add jerasure k=3 m=1 workload

It validates that http://tracker.ceph.com/issues/9209 is fixed.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

thrashosds: increase osd revive timeout (75s -> 150s)

This is needed when running valgrind.

Signed-off-by: Sage Weil <sage@redhat.com>

mplement ceph.created_pool

ceph.created_pool allows the user (via yaml lines) to add pools
that the ceph_manager knows about.

Fixes: 9091
Signed-off-by: Warren Usui <warren.usui@inktank.com>

set boto timeout, too, for s3tests

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/s3tests: push boto config with idle_timeout setting

Signed-off-by: Sage Weil <sage@redhat.com>

suites/fs: add cephfs_journal_tool test

Fixes: #4886
Signed-off-by: John Spray <john.spray@redhat.com>

gitignore: ignore vim temp files

Signed-off-by: John Spray <john.spray@redhat.com>

suites/fs: add client recovery

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/mds_client_recovery: use existing clients

This will enable using .yaml changes to switch this
guy over to use kcephfs client once the teuthology
code around it supports all the same hooks as I've added
for fuse.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/mds_client_recovery: network freeze test

This is about testing the CephFS client's handling
of losing connectivity to the MDS.

Fixes: #7810
Signed-off-by: John Spray <john.spray@redhat.com>

tasks/ceph_fuse: enable umounting from config

This is for any test config that needs to run
some workunit with clients unmounted.  It allows
you to go toggle the mountedness of a client as
you go up and down the stack list this:

- ceph-fuse:
    client.0:
        mounted: true
- workunit:
    clients:
        client.0:
        - fs/misc/trivial_sync.sh
- ceph-fuse:
    client.0:
        mounted:
            false

The initial use case for this is running the
cephfs_journal_tool_smoke.sh workunit, which
tests administrative operations that are meant
to be run on an unmounted filesystem.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/workunit: fix log message

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/ceph: add ceph.stop task

So that we can explicitly stop daemons on demand.  Useful
for MDS tool tests that want the MDS daemons not to be running,
is this is more solid and explicit than doing e.g. "ceph mds
stop" from within workunits.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #102 from ceph/9171

ignore errors on informational service status

Merge pull request #103 from ceph/9172

use 'mon create-initial' always

use 'mon create-initial' always

But don't error if it fails, as this would mean that the monitors
are just taking longer to form quorum. Go and try the next block which will
wait up to 15 minutes for a successful gatherkeys to happen (that only works
if monitors have formed quorum).

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

ignore errors on informational service status

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>

added a test with erasure_code_profile parameters

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #104 from dachary/wip-8933-rgw-erasure-code-profile

rgw: add erasure_code_profile configuration

rgw: add erasure_code_profile configuration

If erasure_code_profile is present at the same leve as ec-data-pool, it
is used to override the default hard coded profile.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Add centos/rhel7 to distros 'all' folder.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #99 from ceph/wip-hadoop

reorg hadoop tests

Reviewed-by: Greg Farnum <greg@inktank.com>

suites/hadoop: update workunit names

These were renamed/reorged.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: httpd instead of httpd.worker

httpd exists on rhel 6.5 too ...

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: need alll of mod unixd, version, authz

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/rgw: include mod_authz

As per http://www.webhostingtalk.com/showthread.php?t=1173594

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/rgw: get mpm_event frmo mods-available, not mods-enabled

Signed-off-by: Sage Weil <sage@redhat.com>

Add extra conf for Apache 2.4

Inside a conditional to affect only 2.4, set User, Group, and the
module config to load mpm_event.  This is normally done with the
default configuration files, but since this abbreviated conf bypasses
those, we must set them here.

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Merge pull request #96 from dachary/wip-8988-default-idle-timeout

rgw.yaml: keep the client.0 key even if empty

Merge pull request #89 from dachary/wip-9027-create-unique-pool

initialize ctx.manager in ceph.py

Reviewed-by: Sage Weil <sage@redhat.com>

rgw.yaml: keep the client.0 key even if empty

Because rgw.py iterates over it to run the rgw server. If it is removed
the rgw servers are not run and all fails.

Signed-off-by: Loic Dachary <loic@dachary.org>