tasks/cephfs: test read-only authorization for volumes

Test whether mount access to volumes can be restricted to read-only.

Fixes: http://tracker.ceph.com/issues/15614
Signed-off-by: Ramana Raja <rraja@redhat.com>

tasks/cephfs: add function to configure guest credentials

In the test_volume_client module, refactor code to add a function
that configures credentials for a guest to mount a volume. The
function could be used by tests to setup a guest client to mount
a volume with fewer lines of code.

Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #1012 from ceph/wip-teuthology-multi-cluster-suite

suites/teuthology/multi-cluster: make them pass again

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

suites/teuthology/multi-cluster: make them pass again

Use xfs to avoid object length limits and be explicit about which
clients to run on, so the workunit git clone doesn't race with another
client on the same machine.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

suites/rbd: added replication stress test workunitReviewed-by: Josh Durgin <jdurgin@redhat.com>

suites/rbd: added replication stress test workunit

suites/rbd: added replication stress test workunit

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1007 from ceph/wip-li-rados

scrub_test: Handle list-inconsistent-obj changes in pull #8983

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1006 from ceph/wip-15754-master

Initial check in for a complete jewel-x suite

Merge pull request #934 from ceph/wip-15266

tasks/cephfs: add TestVolumeClient.test_purge

Reviewed-by: Ramana Raja <rraja@redhat.com>

scrub_test: Handle list-inconsistent-obj changes in pull #8983

Add checking of error in the shards

Signed-off-by: David Zafman <dzafman@redhat.com>

Initial check in for a complete jewel-x suite

Fixes http://tracker.ceph.com/issues/15754
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #974 from ceph/wip-15599

tasks/cephfs: test 'df' output in volumeclient

Reviewed-by: Ramana Raja <rraja@redhat.com>

cron_wrapper: Added echo to log a command name to the log fileReviewed-by: Josh Durgin <jdurgin@redhat.com>

Added echo to log a command name to the log file

Added echo to log a command name to the log file

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #1001 from ceph/wip-15563-master2

Added execute permission to the script

Added execute permission to the script

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #979 from ceph/wip-mon-rocksdb

rados: vary mon kv backend

rados: vary mon kv backend

Use just rocksdb for thrash to avoid blowing up the matrix.  Leave
leveldb for the ec thrash suites.

Signed-off-by: Sage Weil <sage@redhat.com>

make-rpm.sh: support openSUSE

make-rpm.sh: support openSUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>

make-rpm.sh: support openSUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #993 from smithfarm/wip-opensuse-user-data

add user-data for openSUSE 42.1

Reviewed-by: Loic Dachary <ldachary@redhat.com>

add user-data for openSUSE 42.1

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #992 from ceph/wip-14339-master

Added v10.2.0 => jewel upgrade case

Merge pull request #991 from ceph/wip-15857

tasks/cephfs: fix ps usage to not truncate cmds

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: update failover test for standby changesReviewed-by: Greg Farnum <gfarnum@redhat.com>

tasks/cephfs: update failover test for standby changes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #985 from ZhenyuLeng/noneed_op_is_discardable

suites/rados: test osd_discard_disconnected_ops

Test osd_discard_disconnected_ops
Signed-off-by: ZhenyuLeng <lengzhenyu@ict.ac.cn>

Merge pull request #987 from ceph/wip-15563-master

A script to be used in crontab sepia/ovh labs, so emails sent only on errors, logs stored

Reviewed-by: Dan MIck <dmick@redhat.com>

A script to be used in crontab sepia nd ovh lab, so emails sent only on errors

Fixes: http://tracker.ceph.com/issues/15563
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Added v10.2.0 => jewel upgrade case

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

tasks/cephfs: test 'df' output in volumeclient

Fixes: http://tracker.ceph.com/issues/15599
Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix ps usage to not truncate cmds

Now that daemon commandlines include "--cluster" they
easily overflow ps's default behaviour.

Fixes: http://tracker.ceph.com/issues/15857
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #990 from ceph/wip-multi-cluster-fixes

fixes for some issues caused by the multi-cluster branch

tasks: fix {testdir}/data paths

The multi-cluster branch updated these to have a '$cluster.' prefix

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph.healthy: allow None as config

This is used by the pg-removal-interruption test.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/resolve_stuck_peering: fix whitespace and indentation

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/resolve_stuck_peering: adjust to new ctx.manager location

This task was re-added after the multi-cluster branch merged.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #986 from ceph/wip-crush-drop-legacy

rados: test jewel instead of legacy tunables

Reviewed-by: Samuel Just <sjust@redhat.com>

tasks/cephfs: add TestVolumeClient.test_purge

For reproducing error handling paths during purge,
such as #15266 (ASCII exception)

Fixes: http://tracker.ceph.com/issues/15266
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #975 from xiaoxichen/prefix

fs/volume_client: exercise the configurable prefix and ns_prefix.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #968 from ajarr/wip-15045

tasks/cephfs: test volume client eviction

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #989 from ceph/revert-981-revert-955-tracker13531

Revert "Revert "Rados: Tracker-13531-resolve stuck peering""

Reviewed-by: Samuel Just <sjust@redhat.com>

Revert "Revert "Rados: Tracker-13531-resolve stuck peering""

Merge pull request #937 from ceph/wip-multi-cluster

multi-cluster support and basic rbd mirroring test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tasks/create_verify_lfn_objects: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

suites: add multi-cluster tests to the teuthology suite

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks: update ctx.ceph.conf readers to use per-cluster conf

Just use the default cluster name for now, since these tasks aren't used
for multi-cluster tests yet.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

suites/rbd: add basic mirroring suite

Just one workunit to run so far.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks: add an rbd-mirror task

This simply runs the rbd-mirror daemon. Nothing special is needed
since the runtime files (logs, asok, etc) go to the standard ceph
locations.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: ignore EEXIST for the archive data dir creation

With multiple clusters this will be called multiple times.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: pull each mon dir only once

No need to pull all mon dirs for the host for each mon on the host.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: only run ceph_log and valgrind_post once

These setup and parse logs on all hosts, so they should be run only
for the first cluster setup. This cluster will be torn down last, so
the cleanup happens after all clusters are shutdown as well.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make scrubbing cluster-aware

Simplify implementation by using manager and teuthology.misc methods
instead of reinventing them here.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/watch_notify_same_primary: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/thrash_pool_snaps: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/repair_test: clean up manager usage, adjust to new location

Only repair_test_2 actually uses ctx, the rest of the functions only
need the manager.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/reg11184: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/radosbench: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/rados: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/populate_rbd_pool: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/peering_speed_test: adust for new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/osd_failsafe_enospc: adjust for new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/divergent_priors*: adjust to new ctx.manager location

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_objectstore_tool: use existing manager

This is only used after the ceph task, which already createds a
manager for the cluster. Use the default 'ceph' cluster manager for
now.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: make utility_task cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make restart subtask cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make wait_for_mon_quorum cluster-aware

Accept a 'daemons' list like other ceph subtasks, so it can get an
optional 'cluster' setting too.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make wait_for_osds_up cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: update ctx.manager usage to ctx.managers

Not sure this function is ever used (no users in ceph-qa-suite yamls
or tasks.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/workunit: work with roles that include cluster

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks: move find_remote to util, rename and add helper

This is a useful for any cluster-aware task.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/blktrace: make cluster-aware

Need to adjust to lookup devices by role rather than id, so might as
well add a cluster config option while we're at it. Remove the
confusing default config that is not actually used.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: make Thrasher cluster-aware

Just a few spots need to know to lookup only osds in this cluster, or
prefix a filename with the cluster. Use CephManager.find_remote() to
avoid a bunch of repetition and look only in the intended cluster.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: make mount_osd_data() cluster-aware

Use a cluster-specific mount point, and address osds by full role,
rather than just id, in the ctx.disk_config structures.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: add cluster param to write_conf()

Only used by cephfs right now, so don't bother changing callers.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: simplify remote lookup, and make it cluster aware

Re-implement find_remote() using ctx.cluster.only() with a matcher
function that includes the manager's cluster, and use it instead of
miscellaneous ctx.cluster.only() calls elsewhere.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_manager: parameterize CephManager with cluster

Add --cluster arguments, pass cluster to get_daemon() and
iter_daemons_of_role, replace 'ceph' with cluster in paths, and use
ctx.ceph[cluster] instead of ctx.ceph.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/thrashosds: adjust to per-cluster managers

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/thrashosds: add an option to specify the cluster to thrash

Thrashing must be aware of the cluster it is using, since it maintains
some invariants (e.g. enough osds up when waiting to recover).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: store cluster config in a per-cluster dict

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: create a CephManager per cluster

Thrashings tasks will be updated to use ctx.managers indexed by
cluster later.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make healthy() cluster-aware

ceph.healthy may be used as a standalone task, so it may not always
have the cluster name in its configuration.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make cephfs_setup() cluster-aware

Note that cephfs tests using the Filesystem abstractions will need to
be converted to understand multiple clusters later. This just updates
the ceph task portion.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make crush_setup() cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: adapt to cluster-aware daemon.resolve_role_list

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: make run_daemon() cluster-aware

Pass --cluster where appropriate and include the full role in file
names.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph: convert cluster creation to work with multiple clusters

Add a cluster option to the ceph task, and pass that through to
cluster(). Make sure monitors and clients don't collide by adding
their cluster to paths they use.

This assumes there is one ceph task per cluster, and osds from
multiple clusters do not share hosts (or else the block device
assignment won't work).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

tasks/ceph_client: add cluster param

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

rados: test jewel instead of legacy tunables

Legacy tunables semi-frequently turn up crush mappings that only
return 1 instead of 2 results.  Switch to covering jewel tunables
explicitly instead.

Signed-off-by: Sage Weil <sage@redhat.com>

fs/volume_client: exercise the configurable prefix and ns_prefix.

Also ensure namespace_prefix and vol_prefix are set to default values
if they are not explicitly passed.

Fixes #15417

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>

rados: debug rgw test

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #984 from ceph/wip-name-len

drop broken name length config args

drop broken name length config args

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #982 from ceph/wip-13507

Changes for wip-13507 and test fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #911 from ceph/wip-8885

Add dump_ops_enable and optrack_toggle_delay options
Add noscrub_toggle_delay and flip the noscrub osd flags

Reviewed-by: Kefu Chai <kchai@redhat.com>

copyed upgrade/infernalis suite to master branch

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

suites: allow four remote clients for fs/recovery

test_volume_client requires four clients.

Signed-off-by: Ramana Raja <rraja@redhat.com>

tasks/cephfs: test volume client eviction

Test whether the CephFSVolumeClient can evict a client based on its
auth ID and the volume path it has mounted.

Fixes: http://tracker.ceph.com/issues/15045
Signed-off-by: Ramana Raja <rraja@redhat.com>

tasks/cephfs: make mount point more configurable

The mount points of the test clients were created based on the auth
ID they used. So two clients using the same auth ID could not create
different mount points in a straight forward way. Expose the
directory name of the mount point as a public attribute allowing
client mount points to be easily changed whenever needed.

Signed-off-by: Ramana Raja <rraja@redhat.com>

repair_test: Disable scheduled scrubbing to not mess up do_pg_scrub()

Fixes: http://tracker.ceph.com/issues/15679
Signed-off-by: David Zafman <dzafman@redhat.com>