apache: switch to use the apache worker mpm

Fixes: #9169
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit c3b53c326558dd3dd435c9e3fdbc07d8c15f4e68)

apache: change template to load mpm worker module

in apache 2.4

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 35c9cae84c7964ad96381c7455c40e43dd6b4a8d)

Thrasher: Disable ceph_objectstore_tool tests if old release missing command

Don't need to explicitly turn off the test during some upgrades
Leaving disabled until merge of import/export fixes

Fixes: #9805
Signed-off-by: David Zafman <dzafman@redhat.com>

tasks/thrashosds: support overrides

e.g.,

overrides:
  thrashosds:
    thrash_primary_affinity: false
...
tasks:
- install:
- ceph:
- thrashosds:
- workunit:
...

Needed for #9865

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_manager: ceph_objectstore_tool testing off by default

Signed-off-by: David Zafman <dzafman@redhat.com>

Smarter s3tests branch selection

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
(cherry picked from commit 1b8d31986a5a8bc4cd9fea90eb680db3ca1866c8)

Merge pull request #206 from ceph/wip-9672

Added  timeout: 6h to workunit

Added  timeout: 6h to workunit

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #205 from dachary/wip-erasure-code-isa-giant

erasure-code: isa plugin thrashods suite

Reviewed-by: Samuel Just <sam.just@inktank.com>

erasure-code: isa plugin thrashods suite

Add the distros directory which enumerates the os_type and
os_version on which the suite is to run.

http://tracker.ceph.com/issues/9660 Fixes: #9660

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 16e564ec8dea1649888847102ea39fa082e4d3f2)

Merge pull request #204 from dachary/wip-spurious-erasure-code-workload

erasure-code: dumpling-firefly-x/parallel/6-final-workload is incorrect

erasure-code: dumpling-firefly-x/parallel/6-final-workload is incorrect

6-final-workload/ec-rados-plugin=jerasure-k=2-m=1.yaml was added to the
wrong directory.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Merge pull request #202 from dachary/wip-9841-no-lrc-giant

erasure-code: enclose no-lrc in a task (giant)

erasure-code: enclose no-lrc in a task

http://tracker.ceph.com/issues/9841 Fixes: #9841

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 98ef86a202a26682f74c6d7b7aef4616fcc4b128)

fixed dumpling-firefly-x suite to upgrade to firefly branch during upgrade-sequence0

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

rgw: mark ec related rgw suites as slow backend

We want to avoid tests that time out due to a too slow backend.

Reviwed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 85221bc3258b27f0a6b86a74aa1734dac16afe1d)

Merge pull request #200 from dachary/wip-erasure-code-giant

backport erasure code fixes to giant

upgrade/dumpling-x: do not do ec test on mixed cluster

dumpling doens't support it

Signed-off-by: Sage Weil <sage@redhat.com>

erasure-code: upgrade/firefly-x/stress-split-erasure-code

Mimic upgrade/firefly-x/stress-split only with thrashosd configured with
min_in: 4 for k=3,m=1 . All erasure code workloads are moved in this
dedicated suite.

http://tracker.ceph.com/issues/9652 Fixes: #9652

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Conflicts:
suites/upgrade/firefly-x/stress-split-erasure-code/3-thrash/default.yaml

erasure-code: do not include k=3,m=1 in dumpling-x

Because thrashosd does not have min_in: 4 and that we don't want to set
it because other workloads may be influenced.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: add thrash-erasure-code to suites/rados

Because it needs the thrasher to have min_in: 4 otherewise k=3,m=1 will
block.

http://tracker.ceph.com/issues/9648 Fixes: #9648

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: unfound test needs a non empty file

Other rados put will fail as follows

$ touch /tmp/bar
$ ./rados -p rbd put existing_3 /tmp/bar
$ ./rados -p rbd put existing_3 /tmp/bar
WARNING: could not create object: existing_3
error putting rbd/existing_3: (17) File exists

it should be considered a bug in the rados command line but needs to be
addressed separately.

http://tracker.ceph.com/issues/9387 Fixes: #9387

Signed-off-by: Loic Dachary loic-201408@dachary.org

erasure-code: fix bugous ec-rados-default.yaml in some suites

The ec-rados-default.yaml was linked to suites as if it was a task
although it is a workload intended for parallel upgrade tests.

The ec-rados-plugin=jerasure-k=2-m=1.yaml task is defined and used
instead, where relevant.

http://tracker.ceph.com/issues/9549 Fixes: #9549

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Conflicts:
suites/upgrade/dumpling-firefly-giant/parallel/6-giant-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml
suites/upgrade/dumpling-firefly-x/parallel/7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml
suites/upgrade/firefly-x/stress-split-erasure-code/5-workload/ec-rados-default.yaml

erasure-code: CEPH_FEATURE_ERASURE_CODE_PLUGINS_V2 integration tests

http://tracker.ceph.com/issues/9343 Refs: #9343

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

erasure-code: assert that firefly-x/stress-split cannot load lrc

Because it would crash existing plugins.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

fixed firefly-x suite to not upgrade mon.c twice and optimized the suite as well.
included another upgrade combination to dumpling-firefly-x suite

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

optimized the suite and added tests to upgrade client first

Signed-off-by: tamil <tmuthami@redhat.com>

optimized the suite and added ceph_objectstore_tool: false to thrashosds wherever applicable

Signed-off-by: tamil <tmuthami@redhat.com>

Merge pull request #196 from ceph/wip-9805

Added ceph_objectstore_tool: false, fixes #9805

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Added ceph_objectstore_tool: false, fixes #9805

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #192 from ceph/wip-9729

Wip 9729

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Added print

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added client.0 upgrade to firefly step

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixes #9729, running incompatible tests while the cluster is mixed between firefly and dumpling
Changed to rados/test-upgrade-firefly.sh version

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

tasks/s3tests: add slow backend configurable

Adding this so that we can modify the clients' conf file as needed with slow backend.
This can be achieved by:

overrides:
  s3tests:
    slow_backend: true

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 61409179dfdd6969c624dd741813291332644003)

Merge pull request #188 from ceph/wip-multi-version

multiversion/dumpling-x: remove rgw

multiversion/dumpling-x: remove rgw

firefly radosgw + dumpling osds won't work.  (Nor will anything newer
than firefly).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #187 from ceph/wip-multi-version

multi-version: do not run new ceph_test_rados against dumpling

multi-version: do not run new ceph_test_rados against dumpling

Dumpling does not properly order reads (w/ RWORDERED) vs writes.  The code
controlling that was totally rewritten in firefly.  It's not worth backporting.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #177 from ceph/wip_9691

Wip 9691 - (stable giant releases upgrades suite)

Bring in file to fix missng symlink.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Removed secod ceph-fuse

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge branch 'giant' of https://github.com/ceph/ceph-qa-suite into wip_9691

Merge pull request #183 from ceph/update_upgrade_suits2

Update upgrade suits2

Removing files not in master

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Fixed dirs duplicates

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Synced up d-f-x and f-x suites with the latests from master

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Addded missing ceph task

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Minor modifications, added prints

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Removed dupication of overrides

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Incorporated Tamil's comments

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed typo in client name

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added print statement

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed empty line

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

First draft of stable release upgrade suite for giant

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #168 from ceph/wip-objectstore-log

log ceph_objectstore_tool output

tasks/ceph_manager: enable log for ceph_objectstore_tool

Signed-off-by: Sage Weil <sage@redhat.com>

powercycle: fuse_default_permissions = 0 for kernel build

Signed-off-by: Sage Weil <sage@redhat.com>

Fixing branch assumptions

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #152 from dachary/wip-9608-ec-default-parallel-giant

ec-rados-default is not fit for dumpling parallel upgrade

ec-rados-default is not fit for dumpling parallel upgrade

Because it may race with the upgrade and end up being unable to perform.

http://tracker.ceph.com/issues/9608 Fixes: #9608

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit b77ef02bc5ca623d2c5577c2caafcf3720ddd2ec)

added upgrade suite for dumpling-firefly-giant

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
(cherry picked from commit c7603c0b7871477110946d2d47ea67dda1b7a9bf)

Conflicts:

suites/upgrade/firefly-giant-x/parallel/5-upgrade-sequence/upgrade-by-daemon.yaml
suites/upgrade/firefly-giant-x/parallel/5-upgrade-sequence/upgrade-by-type.yaml

smoke: fix pjd debug conf syntax

Needs the section there or else ceph.py barfs at runtime

Signed-off-by: Sage Weil <sage@redhat.com>

tasks: wait for active after mds restart

May have been causing spurious failures on
trying to read session state after MDS restart (
session list isn't populated until recovery is
complete)

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #147 from dachary/wip-ec-feature-plugin-v2-giant

erasure-code: assert that firefly-x/stress-split cannot load lrc

erasure-code: assert that firefly-x/stress-split cannot load lrc

Because it would crash existing plugins.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit d6f02d604840b448158a1470b6bb1f6d0550e97f)

added workload tag

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #142 from ceph/wip-9284

Tests for #9284, FUSE client capability handling

Reviewed-by: Greg Farnum <greg@inktank.com>

Merge pull request #146 from dachary/wip-9298-rgw-ec-starvation-giant

rgw: increase the number of OSDs for erasure coded pools

rgw: increase the number of OSDs for erasure coded pools

So that there never is less than 4 osd which is the minimum when trying
erasure coded workloads that only work with 4 osd.

http://tracker.ceph.com/issues/9298 Fixes: #9298

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

suites: enable mds_client_limits test

Requires same cluster etc as 'recovery', though
we could now do with better name than that.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: wait for mds active before mounting clients

To make the logs clearer when trying to work out
if/when something went wrong, rather than always
having client logs start with some failures.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: rename FuseMount.get_client_id to get_global_id

'client_id' was ambiguous because in other places it
meant the '0' in client.0, whereas here it means
the runtime-generated global ID of the client.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: add mds_client_limits

New CephFS tests for the behaviour of the system while
enforcing its resource limits.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: generalise CephFSTestCase

Some of this stuff could be even more general for embedding
unittest-style suites, but for the moment let's keep the cephfs
stuff in a walled garden.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: generalize config writing for Filesystem

Signed-off-by: John Spray <john.spray@redhat.com>

Update giant.yaml

fixed typos(cherry picked from commit 1f950d2bf4dd76d671ab1c85a87a5f1175305635)

Update giant.yaml(cherry picked from commit 90b6eec23fad6b3af1305de9c59a4759e076ebd3)

Update giant.yaml

Added comments(cherry picked from commit 3dd1314c6fa70fefc4ea8c65c33bac1ad3ba267a)

Create dumpling-giant-x stress-split upgrade suite

Fixes: 9357
Signed-off-by: Warren Usui <warren.usui@inktank.com>
(cherry picked from commit c729372aaae82b8fb0a31a42f24d96924078282f)

Update giant.yaml(cherry picked from commit 35b3f4617c54dcb9705ca31489e2348bdb0e18b5)

dumpling-giant-x: use giant version of workunits

The firefly version of test_librbd_python at least does
not work on giant.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 8755466c4b7c2402e3265ecb04a8300a4d0c146f)

Merge pull request #143 from ceph/wip-migration-test

tasks: more substantial IO for journal migration

Reviewed-by: Greg Farnum <greg@inktank.com>

suites: turn on debugging for smoke cfuse pjd test

Signed-off-by: Greg Farnum <greg@inktank.com>

tasks: escaping '*' when deleting files

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: more substantial IO for journal migration

...so that there will at least be multiple segments
in the log during the rewrite.

Also make the test stricter by checking that
cephfs-journal-tool can happily read the resulting
journal.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #141 from ceph/wip-9423

Fix #9423 - race in mds_client_recovery

Merge pull request #140 from ceph/wip-9177

suites/fs: update to latest mdtest

suites/fs: update to latest mdtest

They appear to have (accidentally?) fixed whatever
was crashing.

Fixes: #9177
Signed-off-by: John Spray <john.spray@redhat.com>

tasks: fix race in test_stale_caps

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: typo in mds_client_recovery

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: handle failure cleanly in test_stale_caps

Previously would fail because the cap waiter
completed too soon, without noticing that the
reason it completed quickly was because it failed.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #119 from ceph/wip_9352

Wip 9352

Merge pull request #118 from dachary/wip-more-osds

add osds to firefly-x upgrade suite

Updated README

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

First draft for a new suite dumpling-giant-x

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

added a test to use single disk for both data and journal.

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

added dmcrypt and separate_journal_disk options

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

Merge pull request #112 from ceph/wip-8231-forreview

Wip 8231 forreview

Reviewed-by: Samuel Just <sam.just@inktank.com>

tasks: fix mount race in mds_client_recovery

Signed-off-by: John Spray <john.spray@redhat.com>