Merge pull request #66 from ceph/wip_tune_upgrade-firefly

Changed upgrade order and re-enabled rados/tests.sh

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Changed upgrade order and re-enabled rados/tests.sh

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/verify: increase api tests timeout

This has been timing out for weeks to months (hard to say since before that
we were consistently hitting an OSD crash on shutdown).  The runs I've looked
at are making progress and seem fine ... just slow because of valgrind and
because of thrashing.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>

added debug messages for radosbench.yaml

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

clusters/fixed-3-cephfs.yaml: put client.0 on separate node, remove client.1

Only one job needs 2 clients; put them on separate hosts.  They should not
be colocated with the OSDs!

Signed-off-by: Sage Weil <sage@redhat.com>

rados/objectstore: move objectstore related tests from singleton-nomsgr

Just so they're in a nice convenient bunch.

Signed-off-by: Sage Weil <sage@redhat.com>

Removed rados/test.sh for now

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Remove mds role from rados & rgw tests

Follow up to a24efaef, remove the mds from
the (mainly singleton) test configs that
explicitly list it.

Signed-off-by: John Spray <john.spray@redhat.com>

Modified print task

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #64 from ceph/wip_dumpling-firefly-x_upgrade_suite

Wip dumpling firefly x upgrade suite

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

upgrade/dumpling-firefly-x: remove unnecessary files

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

upgrade/dumpling-firefly-x: upgrade clients to firefly

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

upgrade/dumpling-firefly-x: use correct tests against final workload

No reason to run old tests against new code here.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/dumpling-firefly-x: rename final upgrade yamls; expect quorum

The difference is whether we upgrade all at once or each individually.

Also, expect a quorum between a mix of dumpling and firefly monitors.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/dumpling-firefly-x: run firefly tests against firefly

At this point we've upgraded to firefly.

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/dumpling-firefly-x: test rados api

not sure why this was left off?

Signed-off-by: Sage Weil <sage@redhat.com>

upgrade/dumpling-firefly-x: drop useless line

Signed-off-by: Sage Weil <sage@redhat.com>

rados/singleton-nomsgr/all/alloc-hint: fix config syntax

Put in [osd] section.

Signed-off-by: Sage Weil <sage@redhat.com>

Removed upgrade-sequence to avoid failures (becasue 3-firefly-upgrade is downgrading then to firefly causing failures)

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/singleton-nomsgr/all/alloc-hint: enable xfs extsize for test

This is off by default; enable it for this test.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #61 from ceph/wip-no-mds

clusters: Remove mds from default fixed-* configs

Reviewed-by: Greg Farnum <greg@inktank.com>

clusters: Remove mds from default fixed-* configs

Create a separate fixed-3-cephfs cluster config for
use in the filesystem tests.

Signed-off-by: John Spray <john.spray@redhat.com>

Re-enabled rgw tests, #8846 should be fixed now

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed prints
Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added prints

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed missed line

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Removed 'firefly' branch name,  upgrade step will be set to 'next' or 'master' from teuthology-suite comman line

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Again fixed identations

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Fixed indentation

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added print tasks

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Commented out rgw tests

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Disabled rwg tests for now to wait for fix for radosgw crash #8846

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Replace branch dumpling with firefly to fix capitalization issue 2048k vs 2048K

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added client.0 upgrade step

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Commented out test.sh as failed tests: LibRadosMisc.Operate1PP can't work across dumpling and firefly releases

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Enabled tunables

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Changed restart order, mds after osd

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

kclient: remove `kclient` workunit

This only contained the file_layout.sh workunit, which
is going away.

Fixes: #8773
Signed-off-by: John Spray <john.spray@redhat.com>

Trying without tunables again

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #46 from ceph/wip-max-backfills

rados/thrash: set osd_max_backfill = 1 sometimes

Changed restart sequence

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Try diff restart

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added back tunables

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Removed tunables temporarily for testing

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Merge pull request #57 from ceph/wip-test-post-file

test ceph-post-file

Reviewed-by: Greg Farnum <greg@inktank.com>

rados: test ceph-post-file

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #56 from ceph/wip-pjd

multimds: fix pjd fuse_default_permissions

Reviewed-by: Greg Farnum <greg@inktank.com>

marginal: fix pjd overrides for fuse_default_persmissions

Signed-off-by: Sage Weil <sage@redhat.com>

multimds: fix pjd overrides for fuse_default_permissions

Set the default (off) in the ceph task directly, so that there is only a
single overrides field that reenables.  Otherwise, the two overrides
fight each other with undefined precedence.

Signed-off-by: Sage Weil <sage@redhat.com>

fix typos

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added steps to cover for 'ceph osd crush tunables firefly'

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Added dumpling-firefly-x upgrade suite per http://tracker.ceph.com/issues/8796

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

Revert "enable kernel debugging on kclient fsx runs"

This reverts commit c883e310f32509579e827ed03c4ee28b491e5140.
We don't want the extra debugging any more.

Signed-off-by: Greg Farnum <greg@inktank.com>

remove stray dirs

These appear to be copies of samba/*

Signed-off-by: Sage Weil <sage@inktank.com>

remove stray workload dir

Merge pull request #45 from dachary/wip-ec-readwrite-firefly-x

add erasure code workload to upgrade/firefly-x

Merge pull request #47 from ceph/wip-multimds-pjd

marginal/multimds: fuse_default_permissions = 1 for pjd

marginal/multimds: fuse_default_permissions = 1 for pjd

pjd contains lots of test cases for file permission.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

rados/thrash: set osd_max_backfill = 1 sometimes

Hopefully this will help catch leaks in the recovery reservations.

Signed-off-by: Sage Weil <sage@inktank.com>

add erasure code workload to upgrade/firefly-x

copied from suites/rados/thrash/workloads/ec-readwrite.yaml

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #44 from ceph/wip-7032

rados/singleton/all/thrash_cache_writeback_forward_none: thrash cache pool drain

rados/singleton/all/thrash_cache_writeback_forward_none: thrash cache pool drain

Enable/disable cache pool against a running workload.

Backport: firefly
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #43 from dachary/wip-firefly-x

add upgrade/firefly-x based on dumpling-x

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yuri Weinstein <yuri.weinstein@inktank.com>

add upgrade/firefly-x based on dumpling-x

It is a s/dumpling/firefly/ mostly. With the exception of
rados/test-upgrade-firefly.sh which is dumpling specific.

The 'mon warn on legacy crush tunables: false' are dropped because they
no longer apply to firefly.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #42 from ceph/wip-smoke

upgrade: fix dumpling tests

smoke: add rbd fsx + thrashing

Signed-off-by: Sage Weil <sage@inktank.com>

smoke: add a mon thrash test

Signed-off-by: Sage Weil <sage@inktank.com>

smoke: add some rados osd thrashing test

ec, caching, snaps

Signed-off-by: Sage Weil <sage@inktank.com>

smoke: specify fs on each run

Signed-off-by: Sage Weil <sage@inktank.com>

thrashers/mapgap.yaml: ignore osd_map_cache_size warning

Signed-off-by: Samuel Just <sam.just@inktank.com>

smoke-tests -> smoke

Signed-off-by: Sage Weil <sage@redhat.com>

smoke: remove old smoke suite

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "rados: Fix cache_flush, cache_try_flush, cache_evict and add hit_set_list"

cache_evict is kinda broken at the moment due to snapdir nonsense.  I'm reverting this
test for now until it's fixed so it doesn't obscure the test results.

Bug: 8629
This reverts commit 3dae48c7cd9e72af47bc76840df652c1570d7502.

enable kernel debugging on kclient fsx runs

Signed-off-by: Greg Farnum <greg@inktank.com>

Merge pull request #41 from ceph/wip-rados-op-fixes

rados: Fix cache_flush, cache_try_flush, cache_evict and add hit_set_lis...

creating smoke-tests suite to include basic tests from all modules

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

rados: Fix cache_flush, cache_try_flush, cache_evict and add hit_set_list

Signed-off-by: David Zafman <david.zafman@inktank.com>

Merge pull request #40 from ceph/wip-runxfstests

wip-runxfstests

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #38 from ceph/wip-krbd-fsx

krbd: add librbd_fsx -K job

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

krbd: add librbd_fsx -K job

Our fsx fork has recently gained a support for the krbd mode - add it
to the test suite.  Disable discard ops and randomized striping, both
of which aren't ready yet.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

krbd: catch up with run_xfstests.sh changes

In the current xfstests, tests cannot be listed in ranges.  Because of
this, and because our distribution of tests between our three clients
was pretty uneven, I've changed this job to run the whole auto group
(with some tests excluded by run_xfstests_krbd.sh) in random order on
two clients in parallel.  It takes longer to run, but it also spares
a test box and can potentially create a better test workload.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

Merge pull request #39 from ceph/wip-multimds-syntax

multimds: fix fuse_default_permissions syntax

multimds: fix fuse_default_permissions syntax

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #37 from ceph/wip-heartbeat

osd_heartbeat_grace from 20 -> 40 when running valgrind

osd_heartbeat_grace from 20 -> 40 when running valgrind

This will mask occasional 'wrongly marked me down' errors when valgrind
is running (it makes things generally slow).

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #36 from ceph/wip-objectstore

fix objecstore test

Merge pull request #35 from ceph/wip-add-v0.67.9

Added dumpling v0.67.9 version, removed older versions v0.67.2, v0.67.4,...

Added dumpling v0.67.9 version, removed older versions v0.67.2, v0.67.4, v0.67.8
This will reduce number of tests to run, some  may failed on old versions, see http://tracker.ceph.com/issues/8409

Signed-off-by: Yuri Weinstein <yuri.weinstein@inktank.com>

rados/singleton-nomsgr/all/objectstore: skip ceph cluster setup

Not necessary; we just need it installed.

Signed-off-by: Sage Weil <sage@inktank.com>

rados/singleton-nomsgr/all/objectstore: clean up

ceph_test_objectstore scribbles in $CWD.  Keep it inside $TESTDIR and
clean up when we are done.

Fixes: #8395
Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #34 from ceph/wip-fuse

fuse_defautl_permissions = 0;

Reviewed-by: Greg Farnum <greg@inktank.com>

multimds: fuse_default_permissions = 0

Signed-off-by: Sage Weil <sage@inktank.com>

multimds: factor install+ceph tasks out of other collections

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #31 from ceph/wip-fuse-permission

marginal/multimds: fuse_default_permissions = 0 for ceph-fuse

Merge pull request #32 from ceph/wip-8284

Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #33 from ceph/wip-8297-wusui

2-workload testrgw needs to be sequential.

2-workload testrgw needs to be sequential.

Fixes: 8297
Signed-off-by: Warren Usui <warren.usui@inktank.com>

rados/thrash: Add pool_snaps variants

Fixes: #8284
Signed-off-by: David Zafman <david.zafman@inktank.com>

marginal/multimds: fuse_default_permissions = 0 for ceph-fuse

This can reduce the test time becuase it avoids sending getattr
request whenever the kernel checks inode permission.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>

cache-snaps.yaml: set target_max_objects to test snap flush/evict

Signed-off-by: Samuel Just <sam.just@inktank.com>