tasks/ceph: make cephfs_setup() cluster-aware

Note that cephfs tests using the Filesystem abstractions will need to
be converted to understand multiple clusters later. This just updates
the ceph task portion.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit b0dd04736eed2822d1584eb8f6f7839e7f02ffc9)

tasks/ceph: make crush_setup() cluster-aware

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 44097101025a3053bb7938062ee527a76cedc29f)

tasks/ceph: adapt to cluster-aware daemon.resolve_role_list

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit e19e9e2fa31bd544894b3e36ade7c09ed8af73d6)

tasks/ceph: make run_daemon() cluster-aware

Pass --cluster where appropriate and include the full role in file
names.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 26b8a1f3acdc53122767918cec86d36fdf74337c)

tasks/ceph: convert cluster creation to work with multiple clusters

Add a cluster option to the ceph task, and pass that through to
cluster(). Make sure monitors and clients don't collide by adding
their cluster to paths they use.

This assumes there is one ceph task per cluster, and osds from
multiple clusters do not share hosts (or else the block device
assignment won't work).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit a54ff597faa379c8fd369352fb924651be269f95)

tasks/ceph_client: add cluster param

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 466678b980e0527947f5c436d83da6933b216a25)

Merge pull request #1000 from smithfarm/wip-opensuse-make-rpm-jewel

make-rpm.sh: support openSUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>

make-rpm.sh: support openSUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b7cbe2f96fa2ebb08f48f1fcb89fa591ec7ea1e2)

Merge pull request #996 from smithfarm/wip-opensuse-user-data-jewel

add user-data for openSUSE 42.1

Reviewed-by: Loic Dachary <ldachary@redhat.com>

add user-data for openSUSE 42.1

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 209ce61788117a9c9b7fa8333504f1e63538f0c2)

suites/hadoop: use xfs

This avoids working aroudn ext4 limits.  Whatev.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2485e7b26302e87903c4951a8c7ceaa13d8a97c6)

Merge remote-tracking branch 'gh/jewel'

Merge pull request #891 from ceph/wip-rgw-new-multisite

Wip rgw new multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
s/ceph-deploy_hello_world.yaml/ceph-admin-commands.yaml, hence removing ceph-deploy_hello_world.yaml

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
removed ceph-deploy_hello_world.yaml as its been replaced with ceph-admin-commands.yaml

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
removed file ceph-deploy_hello_world.yaml as it is replaced by ceph-admin-commands.yaml

Merge pull request #959 from xiaoxichen/add_ns_check

tasks/cephfs: check data written to NS.

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #973 from ceph/jewel

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

rados/monthrash: debug client on librados api tests

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #971 from ceph/wip-kv

rados/objectstore: add ceph_test_keyvaluedb test

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
  Renaming the yaml and workunits for better clarity

(cherry picked from commit dc429cbb06385493c46c8c90f4faf89b2ed6b171)

Merge pull request #972 from ceph/wip-rename-test-in-suite

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>
  Renaming the yaml and workunits for better clarity

rados/objectstore: add ceph_test_keyvaluedb test

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: increase sleep to 15 seconds

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #969 from ceph/wip-rgw-xfs

rgw: move xfs to a seperate directory

rgw: move xfs to a seperate directory

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: move xfs to a seperate directory

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: force frontend to be civetweb

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix debug messages

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: fix zone get with multisite

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: configure users on all clients

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: do not overwrite user_info with zone user_info

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: allow already exist realm

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: increase timeout to wait for master to 20 seconds

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: we can have multicluster without multizone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: no need to do period pull

realm pull already pulls latest period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: non master zones need to be created on the remote gateway

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: mtime is now a time string with microseconds

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: add --rgw-zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: create default zone after zonegroup creation

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: debug zone info

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: set default zonegroup on all clients

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: default zone should be always master zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: different clients have different defaults zone

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: In multisite configuration start the master gw first

Than pull realm and period configuration from it

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: use master_zonegroup not master_region

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: use zonegroups instead of regions

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: move realm pull and period pull to pull_configuration

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: update and commit period

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

add realm configuration to all multisite test

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: add realm creation

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw suite: always add --rgw-frontends fastcgi

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

use ceph-master branch for s3tests

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

suites/krbd/unmap: specify xfs

Signed-off-by: Sage Weil <sage@redhat.com>

tasks/cephfs: check data written to NS.

NS is always used for security isolation.

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>

tasks/ceph: allow set allow_multiple to fail

This will fail on upgrade tests.

Signed-off-by: Sage Weil <sage@redhat.com>

suites/rest: specify xfs

Signed-off-by: Sage Weil <sage@redhat.com>

suites/fs: specify xfs as needed

Signed-off-by: Sage Weil <sage@redhat.com>

suites/rgw: specify xfs as needed

Signed-off-by: Sage Weil <sage@redhat.com>

suites/rbd: specify xfs as needed

Signed-off-by: Sage Weil <sage@redhat.com>

rados: specify short names for misc jobs

These need to specify short names bc they don't have all osds on
a dedicated block device.

Signed-off-by: Sage Weil <sage@redhat.com>

suites/upgrade/hammer-x: specify xfs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #965 from ceph/wip-xfs

rados: specify xfs

Reviewed-by: Samuel Just <sjust@redhat.com>

rados: specify xfs

If no fs is specified, the OSD is created on /, and the scratch
devices aren't used at all.  That means ext4, and the long file
name lameness.

Signed-off-by: Sage Weil <sage@redhat.com>

fs/ext4: longer names

Signed-off-by: Sage Weil <sage@redhat.com>

fs/ext4: max of 256 works better

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #964 from ceph/wip-ext4

set shorter limits when using ext4

Reviewed-by: Samuel Just <sjust@redhat.com>

fs/ext4: set shorter object name limits so that osds can start

Signed-off-by: Sage Weil <sage@redhat.com>

rados/basic: test over xfs (not just btrfs)

Signed-off-by: Sage Weil <sage@redhat.com>

Repleaced ext4 with xfs

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #962 from ceph/wip-rgw-ext4

suites/rgw/multifs: do not test on ext4

suites/rgw/multifs: do not test on ext4

Swift tests fail.  RGW long object names will fail in general.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #948 from ceph/wip-teuthology-master

Fixed distro symlinks

Merge pull request #957 from ceph/wip-teuthology-master2

Added civetweb test case

Merge pull request #936 from ceph/wip-layout-snafu-followup

tasks/cephfs: test health messages from readonly stat

Merge pull request #933 from ceph/wip-fs-lockouts

cephfs: update tests to enable multimds when needed

Reproducer for 15303

Reproducer for http://tracker.ceph.com/issues/15303

Merge pull request #958 from ceph/wip-rgw-prefer-civetweb

rgw: use civetweb if no frontend was configured

rgw: use civetweb if no frontend was configured

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

FIXUP 'cephfs: update tests to enable multimds when needed'

cephfs: enable dirfrags in test_data_scan fragmentation test

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Added civetweb test case

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Fixed distro symblinks

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #944 from athanatos/wip-lfn-upgrade

LFN upgrade tests and a few other misc fixes

Merge pull request #954 from ceph/wip-rgw-fix-zone-get

rgw: add default zone name

Merge pull request #950 from dachary/wip-buildpackages-xenial

buildpackages: xenial support

Merge pull request #945 from dachary/wip-pr-763

buildpackages: openstack image names not unique.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

buildpackages: add build timeouts.

The build/upload process CAN get stuck sometimes. As we are paying
utility rates for the high-power build server, we should timeout when
it's likely the build is stuck.

Use a default timeout just short of 4 hours, as some providers round up
partial instance-hours for billing.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

buildpackages: support ubuntu 12.04.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

buildpackages: fix configure on Ubuntu 12.04.

Ubuntu 12.04/precise has a buggy lttng; it fails our lttng-is-sane
check, as it leaves out headers.

This should not stop building of Ceph, as the automatic build handling
from debian/rules will correctly omit LTTNG on Precise.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

buildpackages: catch VM instances in ERROR state.

It is possible for 'server create' to return success, but then NOT have
a good VM afterwards. The instance will be in state ERROR.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

rgw: add default zone name

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #946 from ceph/wip-rgw-fix-teu-bucket-policy

rgw: policy acl format should be xml

rgw: policy acl format should be xml

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

buildpackages: add ubuntu 16.04 xenial support

Signed-off-by: Loic Dachary <loic@dachary.org>

admin malloc interface testsReviewed-by: Josh Durgin <jdurgin@redhat.com>

admin malloc interface tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rados/singleton-nomsgr: add lfn upgrade tests

Upgrade from hammer/infernalis to x and verify lfn objects are valid
across upgrade.

Signed-off-by: Samuel Just <sjust@redhat.com>

tasks: add create_verify_lfn_objects

Signed-off-by: Samuel Just <sjust@redhat.com>

ceph_manager: return exit status on do_get, do_put, do_rm

Signed-off-by: Samuel Just <sjust@redhat.com>