Fix wrapped line

Fix objectstore name changed to keyvaluestore

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Fix Thrasher kill_osd() to ignore pool going away

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Thrasher fixes to run() calls to set check_status = False

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Re-enable Thrasher ceph-objectstore-tool testing

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: In Thrasher randomly remove past intervals

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: In Thrasher disable ceph_objectstore_tool tests if old release missing command

Require ceph-objectstore-tool to be available on all OSD nodes
Log a message when tool is not available

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #308 from ceph/wip-10579

suites/fs: isolate quota task

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

suites/fs: isolate quota task

Quota script is moved out of fs/misc so that kclient
doesn't try to run it.  Update cfuse suite to find
the new location, and only enable quota for this
one script rather than enabling across
all misc tests.

Fixes: #10579
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #285 from ceph/wip-9883

tasks/mds_journal_repair: create new test

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/mds_journal_repair: add test_table_tool

Explicit test of the tool's various modes, to
complement the places it's used in other tests.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/mds_journal_repair: create new test

This tests the new #9883 repair functionality
where we selectively scrape dentries out of
the journal while the MDS is offline.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #300 from ceph/restore_giant-x_on_master

Restored giant-x suite on master

Restored giant-x suite on master

Signed-off-by: Yuri Weinstein <yuriw@magna002.ceph.redhat.com>

Merge pull request #288 from dmick/master

Tweaks to calamari-setup

Merge pull request #291 from dachary/wip-10018-scrub

add erasure code repair / scrub tests (take 2)

Reviewed-by: David Zafman <dzafman@redhat.com>

repair_test: hinfo_key error injection

Add a function dedicated to erasure coded pools tests, similar to
repair_test_1. Add a corrupter that removes the hinfo_key from the object.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: add ceph-objectstore-tool wrapper

Add the CephManager.objectstore_tool method to encapsulate a call to
ceph-objectstore-tool. The wrapper can convert an object name into the
PG id and figure out the primary OSD. The designated OSD is stopped
before running the command and restarted afterwards.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: add object helpers

Add CephManager.methods to get information regarding the pools and PGs
used to store a given object.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

repair_test: rework indentation to make it pep8 clean

Signed-off-by: Loic Dachary <ldachary@redhat.com>

repair_test: rework to remove levels of indirection

The commit is large but does not introduce any semantic change and
consists primarily in code moving around, re-indented and removed.

Replace functions generating functions by functions and sequentially
iterating over a list of functions with a sequential call to the
functions.

Replace the setup/teardown with an equivalent using a with
statement and the ceph_manager.pool method.

Replace inline code with a call to ceph_manager.wait_for_all_up

It makes it easier to modify the tests, for instance to create erasure
coded pools and tests specific to them.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: rework indentation to make it pep8 clean

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: fix docstring typo

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: define pool context

To create a pool before running a code bloc and remove it after.

   with manager.pool("mypool"):
        mytest..

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #290 from ceph/revert-250-wip-10018-scrub

Revert "add erasure code repair / scrub tests"

Revert "add erasure code repair / scrub tests"

Merge pull request #286 from ceph/wip-10261-wusui

Calamari_setup now allows > 1 mon.

Merge pull request #250 from dachary/wip-10018-scrub

add erasure code repair / scrub tests

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

repair_test: hinfo_key error injection

Add a function dedicated to erasure coded pools tests, similar to
repair_test_1. Add a corrupter that removes the hinfo_key from the object.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: add ceph-objectstore-tool wrapper

Add the CephManager.objectstore_tool method to encapsulate a call to
ceph-objectstore-tool. The wrapper can convert an object name into the
PG id and figure out the primary OSD. The designated OSD is stopped
before running the command and restarted afterwards.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: add object helpers

Add CephManager.methods to get information regarding the pools and PGs
used to store a given object.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

repair_test: rework indentation to make it pep8 clean

Signed-off-by: Loic Dachary <ldachary@redhat.com>

repair_test: rework to remove levels of indirection

The commit is large but does not introduce any semantic change and
consists primarily in code moving around, re-indented and removed.

Replace functions generating functions by functions and sequentially
iterating over a list of functions with a sequential call to the
functions.

Replace the setup/teardown with an equivalent using a with
statement and the ceph_manager.pool method.

Replace inline code with a call to ceph_manager.wait_for_all_up

It makes it easier to modify the tests, for instance to create erasure
coded pools and tests specific to them.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: rework indentation to make it pep8 clean

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: fix docstring typo

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_manager: define pool context

To create a pool before running a code bloc and remove it after.

   with manager.pool("mypool"):
        mytest..

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Fix objectstore name changed to keyvaluestore

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Fix Thrasher kill_osd() to ignore pool going away

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Thrasher fixes to run() calls to set check_status = False

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Re-enable Thrasher ceph-objectstore-tool testing

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: In Thrasher randomly remove past intervals

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: In Thrasher disable ceph_objectstore_tool tests if old release missing command

Require ceph-objectstore-tool to be available on all OSD nodes
Log a message when tool is not available

Signed-off-by: David Zafman <dzafman@redhat.com>

tasks/calamari_setup: build ice-tools virtualenv if building iceball

ice-tools needs a virtualenv populated to properly run to build
an iceball; add the commands to do that.  Also remove the built
iceball when the task exits.

Fixes: #10523
Signed-off-by: Dan Mick <dan.mick@redhat.com>

tasks/calamari_setup: remove HTTP-downloaded iceball, if any

Fixes: #10522
Signed-off-by: Dan Mick <dan.mick@redhat.com>

tasks/calamari_setup: use ice_version to completely specify iceball

Previously, the task would search for the lexicographically-greatest
filename matching ICE-*.tar.gz; now it builds a specific name
ICE-{ice_version}-{ice_distro}.tar.gz

Fixes: #10521
Signed-off-by: Dan Mick <dan.mick@redhat.com>

fs: enable client quota on cfuse_workunit_misc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Calamari_setup now allows > 1 mon.

Signed-off-by: Warren Usui <warren.usui@inktank.com>
Fixes: #10261

Merge pull request #279 from ceph/wip-cephfs-cleanup

Assorted cleanup in the CephFS tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/mds_auto_repair: remove unneeded log settings

The small segments and small segment limit
were used when doing a hacky flush by doing
IO and waiting: now that we have the explicit
'flush journal' asok in use, we can just use
a normal journal configuration.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: remove `config` from Filesystem cons

This was only used in get_first_mon, which doesn't actually
need the parameter itself.  Makes it easier to casually
use Filesystem from any place with a ctx to hand.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: umount unused clients before running

When unused clients were mounted during an fs new,
they would end up in a state where they stalled
on subsequent attempts to umount them (ceph-fuse
stalls on exit if it can't terminate its mds_session)

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: move ROOT_INO into filesystem module

...so that other tests can get at it.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: be more precise in set_clients_block

Instead of blocking the whole port range (which
might make OSDs running on that node collateral
damage), read the MDS's port out of the MDS map
and just block that.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: use set_conf() from test case class

...because this is the one that will store up
changes to roll back during teardown.

Doing this makes it easy to run lots of test cases
togeher in a single teuthology run, raher than
setting up/tearing down the ceph cluster for each
on.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: move common setUp/tearDown to parent

Now that we have more of these cases, there was lots
of duplication in setup and teardown.  For some tests
the "reset everything" setup/teardown is overkill,
but it's harmless.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: move wait_until helpers to testcase

...so that other test cases can use them

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/ceph: update mds auth caps to 'allow *'

Since the new 'tell' for the MDS was introduced,
caps have to have the '*' to permit running remote
administrative commands.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #283 from ceph/issue-10275

Use qemu-kvm in the qemu task for rpm based systems.

Use qemu-kvm in the qemu task for rpm based systems.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #280 from ceph/wip-10387-cleanup

tasks/mds_flush: remove #10387 workaround

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #282 from ceph/wip-10469

Inherit teuthology's log level

Inherit teuthology's log level

Signed-off-by: Zack Cerza <zack@redhat.com>

tasks/mds_flush: remove #10387 workaround

Now that #10387 is fixed in master, we can tighten
up this test to ensure that the expected deletions
are happening.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #278 from ceph/wip-rm-tiobench

#10152: drop tiobench tests

drop tiobench tests

The tiobench software has been abandoned upstream for years. Fedora and
Debian are no longer shipping the tiobench package, so we've had to
carry the package ourselves in the Ceph project, and we're trying to
slim down our dependencies where it makes sense to do so.

Nuke the tiobench tests.

http://tracker.ceph.com/issues/10152 Refs: #10152

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Revert "Move output in task/s3readwrite"

This reverts commit 26a33c3a5aa2aedb52eb5ce140c76503f099b253.

This is tryign to create the archive dir on the remote host:

2014-12-29T12:15:30.213 INFO:teuthology.orchestra.run.plana31:Running: 'mkdir -p /var/lib/teuthworker/archive/sage-2014-12-29_11:40:52-rgw-next---basic-multi/683052'
2014-12-29T12:15:30.231 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/contextutil.py", line 28, in nested
    vars.append(enter())
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/var/lib/teuthworker/src/ceph-qa-suite_next/tasks/s3readwrite.py", line 241, in run_tests
    ctx.cluster.only(client).run(args=['mkdir', '-p', archive_dir])
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 128, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 368, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 106, in wait
    exitstatus=status, node=self.hostname)
CommandFailedError: Command failed on plana31 with status 1: 'mkdir -p /var/lib/teuthworker/archive/sage-2014-12-29_11:40:52-rgw-next---basic-multi/683052'

...but it should only be on the local host.

Revert "tasks/ceph_manager: dump raw cluster command at debug level"

This reverts commit b0ebeba79cf349b9c5a6e387f8246d7b6d0ef576.

Merge pull request #275 from ceph/wip-10404

tasks/ceph_manager: debug raw command output

tasks/ceph_manager: dump raw cluster command at debug level

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #227 from ceph/wip-9881

#9881 MDS flush journal test

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

suites/fs: hook in mds_flush

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: create mds_flush task

This tests:
 * The new 'flush journal' asok command
 * That the resulting on disk structures are as expected
 * That cephfs-journal-tool is happy with the result

Fixes: #9881
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #270 from ceph/wip-cot-rename

ceph_objectstore_tool: Changes for rename of binary ceph-objectstore-too...

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #271 from dachary/wip-cot-rename

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #263 from ceph/wip-calamari-nosetests

Add calamari-nosetests task and calamari suite (and remove obsolete calamari task)

Add calamari suite definitions.

Notes:
- very simple cluster configuration
- selects vps in the actual suite config files
- wheezy is currently disabled

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Add calamari_nosetests task.

Includes all accumulated review commentary and final 'yum clean all' fix.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Remove obsolete tasks/calamari

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph_objectstore_tool: fix pep8 errors

Signed-off-by: Loic Dachary <ldachary@redhat.com>

ceph_objectstore_tool: --op list now prints [pg,object]

The format of the output of --op list was changed to include the PG to
which the object belong. It simplifies the loop in
ceph_objectstore_tool.py.

http://tracker.ceph.com/issues/10376 Fixes: #10376

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rados/singleton: debug watch-notify test

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #225 from ceph/wip-7317

#7317 Tests for ENOSPC and OSD epoch barrier handling

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #269 from ceph/wip-librados-debug

rados/thrash: enable client debug for api tests

rados/thrash: enable client debug for api tests

These will help catch watch/notify issues...

Signed-off-by: Sage Weil <sage@redhat.com>

suites/fs: hook in mds_full task

Signed-off-by: John Spray <john.spray@redhat.com>

tasks: add mds_full task

This is for exercising the CephFS behaviour
when the cluster becomes full.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: let get_config use non-mon services

Previously was always using the default values of things
so querying mon instead of the appropriate service
worked fine.  However, for things we might want to
update on a per-test basis we need to go ask the
correct service what the setting really is.

Needed for osd_mon_report_interval_max in the ENOSPC
testing.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: enable osd epoch get from mounts

Using new hooks for #7317

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/mds_client_limits: fix initial rm -rf

This was previously a no-op because it was trying
to delete "*" instead of *

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/kclient: add 'debug' option

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: enable kclient for mds_* tasks

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #268 from ceph/wip-kcephfs-inline

kcephfs: test inline data

Reviewed-by: John Spray <john.spray@redhat.com>

ceph_objectstore_tool: Changes for rename of binary ceph-objectstore-tool

Signed-off-by: David Zafman <dzafman@redhat.com>

kcephfs: test inline data

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #257 from ceph/wip-fs-creation

Fix PG counts and refactor FS creation during setup

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #266 from ceph/wip-9892-2

radosgw-admin: wait before trimming usage

radosgw-admin: wait before trimming usage

Fixes: #9892
Need to wait through the usage interval before trimming usage, otherwise we might not
remove all pending usage info.

Backport: dumpling, firefly, giant

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit dd09ecbfab8a659f3faaf879a52849caab5e8e8e)

Merge pull request #265 from ceph/wip-test-huge-tickets

krbd: add huge_tickets workunit

krbd: add huge_tickets workunit

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #260 from ceph/wip-mds-readonly

tasks: add test case for readonly MDS

Reviewed-by: Greg Farnum <gfarnum@redhat.com>