tasks/ceph: only run ceph_log and valgrind_post once

These setup and parse logs on all hosts, so they should be run only
for the first cluster setup. This cluster will be torn down last, so
the cleanup happens after all clusters are shutdown as well.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make scrubbing cluster-aware

Simplify implementation by using manager and teuthology.misc methods
instead of reinventing them here.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/watch_notify_same_primary: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/thrash_pool_snaps: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/repair_test: clean up manager usage, adjust to new location

Only repair_test_2 actually uses ctx, the rest of the functions only
need the manager.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/reg11184: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/radosbench: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/rados: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/populate_rbd_pool: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/peering_speed_test: adust for new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/osd_failsafe_enospc: adjust for new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/divergent_priors*: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_objectstore_tool: use existing manager

This is only used after the ceph task, which already createds a
manager for the cluster. Use the default 'ceph' cluster manager for
now.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: make utility_task cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make restart subtask cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make wait_for_mon_quorum cluster-aware

Accept a 'daemons' list like other ceph subtasks, so it can get an
optional 'cluster' setting too.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make wait_for_osds_up cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: update ctx.manager usage to ctx.managers

Not sure this function is ever used (no users in ceph-qa-suite yamls
or tasks.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/workunit: work with roles that include cluster

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks: move find_remote to util, rename and add helper

This is a useful for any cluster-aware task.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/blktrace: make cluster-aware

Need to adjust to lookup devices by role rather than id, so might as
well add a cluster config option while we're at it. Remove the
confusing default config that is not actually used.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: make Thrasher cluster-aware

Just a few spots need to know to lookup only osds in this cluster, or
prefix a filename with the cluster. Use CephManager.find_remote() to
avoid a bunch of repetition and look only in the intended cluster.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: make mount_osd_data() cluster-aware

Use a cluster-specific mount point, and address osds by full role,
rather than just id, in the ctx.disk_config structures.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: add cluster param to write_conf()

Only used by cephfs right now, so don't bother changing callers.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: simplify remote lookup, and make it cluster aware

Re-implement find_remote() using ctx.cluster.only() with a matcher
function that includes the manager's cluster, and use it instead of
miscellaneous ctx.cluster.only() calls elsewhere.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: parameterize CephManager with cluster

Add --cluster arguments, pass cluster to get_daemon() and
iter_daemons_of_role, replace 'ceph' with cluster in paths, and use
ctx.ceph[cluster] instead of ctx.ceph.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/thrashosds: adjust to per-cluster managers

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/thrashosds: add an option to specify the cluster to thrash

Thrashing must be aware of the cluster it is using, since it maintains
some invariants (e.g. enough osds up when waiting to recover).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: store cluster config in a per-cluster dict

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: create a CephManager per cluster

Thrashings tasks will be updated to use ctx.managers indexed by
cluster later.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make healthy() cluster-aware

ceph.healthy may be used as a standalone task, so it may not always
have the cluster name in its configuration.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make cephfs_setup() cluster-aware

Note that cephfs tests using the Filesystem abstractions will need to
be converted to understand multiple clusters later. This just updates
the ceph task portion.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make crush_setup() cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: adapt to cluster-aware daemon.resolve_role_list

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make run_daemon() cluster-aware

Pass --cluster where appropriate and include the full role in file
names.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: convert cluster creation to work with multiple clusters

Add a cluster option to the ceph task, and pass that through to
cluster(). Make sure monitors and clients don't collide by adding
their cluster to paths they use.

This assumes there is one ceph task per cluster, and osds from
multiple clusters do not share hosts (or else the block device
assignment won't work).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_client: add cluster param

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

rados: debug rgw test

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #984 from ceph/wip-name-len

drop broken name length config args

drop broken name length config args

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #982 from ceph/wip-13507

Changes for wip-13507 and test fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #911 from ceph/wip-8885

Add dump_ops_enable and optrack_toggle_delay options
Add noscrub_toggle_delay and flip the noscrub osd flags

Reviewed-by: Kefu Chai <kchai@redhat.com>

copyed upgrade/infernalis suite to master branch

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

repair_test: Disable scheduled scrubbing to not mess up do_pg_scrub()

Fixes: http://tracker.ceph.com/issues/15679
Signed-off-by: David Zafman <dzafman@redhat.com>

scrub_test: Fix swapped assert error messages

Signed-off-by: David Zafman <dzafman@redhat.com>

scrub_test: Fixes need with wip-13507

Accept an attribute if not present as False
Change required due change to dict with "epoch" and "inconsistents"

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #976 from ceph/wip-xfs-hadoop

suites/hadoop: use xfs

Merge pull request #981 from ceph/revert-955-tracker13531

Revert "Rados: Tracker-13531-resolve stuck peering"

Revert "Rados: Tracker-13531-resolve stuck peering"

Merge pull request #955 from shylesh/tracker13531

Rados: Tracker-13531-resolve stuck peering

Reviewed-by: Samuel Just <sjust@redhat.com>

Rados: wip-13531:Resolve stuck peering

Signed-off-by: shylesh kumar <shylesh.mohan@gmail.com>

Merge remote-tracking branch 'gh/jewel'

CephManager: Wait 1 second for pool creation to get far enough along

Fixes: http://tracker.ceph.com/issues/15673
Signed-off-by: David Zafman <dzafman@redhat.com>

CephManager: Maximum 2 minutes for raw cluster commands

Signed-off-by: David Zafman <dzafman@redhat.com>

thrasher: Add noscrub_toggle_delay and flip the noscrub osd flags

Signed-off-by: David Zafman <dzafman@redhat.com>

thrasher: Add dump_ops_enable and optrack_toggle_delay options

Add dump_ops_enable which continuously dumps ops using 3 commands
Add optrack_toggle_delay to alternate op tracking enablement

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph_manager: Add timeout to admin_socket/osd_admin_socket

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #891 from ceph/wip-rgw-new-multisite

Wip rgw new multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>

suites/hadoop: use xfs

This avoids working aroudn ext4 limits.  Whatev.

Signed-off-by: Sage Weil <sage@redhat.com>

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
s/ceph-deploy_hello_world.yaml/ceph-admin-commands.yaml, hence removing ceph-deploy_hello_world.yaml

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
removed ceph-deploy_hello_world.yaml as its been replaced with ceph-admin-commands.yaml

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
removed file ceph-deploy_hello_world.yaml as it is replaced by ceph-admin-commands.yaml

Merge pull request #959 from xiaoxichen/add_ns_check

tasks/cephfs: check data written to NS.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #973 from ceph/jewel

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

rados/monthrash: debug client on librados api tests

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #971 from ceph/wip-kv

rados/objectstore: add ceph_test_keyvaluedb test

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
  Renaming the yaml and workunits for better clarity

(cherry picked from commit dc429cbb06385493c46c8c90f4faf89b2ed6b171)

Merge pull request #972 from ceph/wip-rename-test-in-suite

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
  Renaming the yaml and workunits for better clarity

rados/objectstore: add ceph_test_keyvaluedb test

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: increase sleep to 15 seconds

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #969 from ceph/wip-rgw-xfs

rgw: move xfs to a seperate directory

rgw: move xfs to a seperate directory

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: move xfs to a seperate directory

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: force frontend to be civetweb

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix debug messages

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix zone get with multisite

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: configure users on all clients

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: do not overwrite user_info with zone user_info

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: allow already exist realm

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: increase timeout to wait for master to 20 seconds

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: we can have multicluster without multizone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: no need to do period pull

realm pull already pulls latest period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: non master zones need to be created on the remote gateway

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: mtime is now a time string with microseconds

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: add --rgw-zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: create default zone after zonegroup creation

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: debug zone info

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: set default zonegroup on all clients

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: default zone should be always master zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: different clients have different defaults zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: In multisite configuration start the master gw first

Than pull realm and period configuration from it

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: use master_zonegroup not master_region

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: use zonegroups instead of regions

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: move realm pull and period pull to pull_configuration

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update and commit period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

add realm configuration to all multisite test

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add realm creation

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: always add --rgw-frontends fastcgi

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

use ceph-master branch for s3tests

Signed-off-by: Orit Wasserman <owasserm@redhat.com>