Merge branch 'wip-rados-reorg' into 'rh-luminous'

Rados reorg for downstream-basic suit

See merge request !5

Add 3.0z and 3.1 version

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

update hadoop version and fix rhbuild config item in overrides

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add options to test nvme as journal

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Remove AWS Credential test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

trial and error for ENAMETOOLONG errors

Signed-off-by: Shylesh Kumar <shylesh.mohan@gmail.com>

rados rebase- change in at-end.yaml

Signed-off-by: Shylesh Kumar <shylesh.mohan@gmail.com>

rados restructure

Signed-off-by: Shylesh Kumar <shylesh.mohan@gmail.com>

set default pg_num and pgp_num as 8

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Client needs both cluster keys, better to overload them

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

qa/workunits/rbd: fixed variable name for resync image id

Fixes: http://tracker.ceph.com/issues/21663
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 22816ed3f48b0a08a6d4e0cf0b0e14dabbb9ce69)

qa/workunits/rbd: simplify split-brain test to avoid potential race

Fixes: http://tracker.ceph.com/issues/22485
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8c2ee6dd0f4c77b9fbbc24a673841b04d3e175d6)

Conflicts:
qa/workunits/rbd/rbd_mirror.sh: trivial resolution

rbd-mirror: sync image metadata when transfering remote image

Fixes: http://tracker.ceph.com/issues/21535
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9aa5dfb8ec46e8babd0f167dcc0e234a7c60a50c)

rbd-mirror: update asok hook name on image rename

Fixes: http://tracker.ceph.com/issues/20860
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 9ddb5da3377029d11cb1089102d7146e9ee2537a)

Merge branch 'wip-sagrawal-rbd' into 'rh-luminous'

workunit for installing operating system on a rbd image

Signed-off-by: Sidhant Agrawal <sagrawal@redhat.com>
See merge request !4

typo correction os_install.py
Signed-off-by: Sidhant Agrawal <sagrawal@redhat.com>

qa/suites/krbd/rbd/tasks:added rbd_os_install.yaml
Signed-off-by: Sidhant Agrawal <sagrawal@redhat.com>

/qa/workunits/rbd:update os_install.py to generate kickstart file automatically
Signed-off-by: Sidhant Agrawal <sagrawal@redhat.com>

qa/workunits/rbd:added script to install operating system on rbd image
Signed-off-by: Sidhant Agrawal <sagrawal@redhat.com>

qa/tests: use norequire_active_mds options only for test kernel mounts

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

sync rgw multi changes

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

sync rbd.py changes

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

sync rados.py changes

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

sync thrash-osd-heath ignore's

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

sync ceph-manager and kernel mount options

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

sync fs script changes

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

mgr changes sync with upstream

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Fix cluster config

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add simple rbd performance tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

use retry instead of bash while loop which can block indefinitly

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

gitlab changed from osas to gitlab.cee.redhat.com

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add support for 3.0 cdn builds

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge branch 'wip_cop_file_err' into 'rh-luminous'

io_info.yaml copy file error fixed

See merge request ceph/ceph!12

removed the rename of put file to master node and renamed the io_info yaml in master node

io_error for io_yaml fixed and test_acls_reset dictionary parse error fixed

io_info file renamed to target_node

reverted to copy_function and extra logs

put file and get file corrected to  put from target to master node

log info added before and after copy

removed wip branch checkout

wip branch added

removed copy_function to copy io_info.yaml file from master to target node

qa: use ceph-luminous branch for s3tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

qa/suites/rados/thrash/d-*/*balancer*: debug osd in mgr

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/thrash: combine balancer with require-luminous

We can only do the balancer stuff if we've done the mkfs-time
require luminous because otherwise the mgr commands aren't proxied via
the mon.

Only do balancer for thrash tests; not for basic, monthrash, verify.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/balancer: enable module by default

It will still be "off".

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 26710f0a9b176289237a52e5fa5894342ad163dc)
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
Conflicts:
src/common/options.cc

qa: fix mgr _load_module helper

I inadvertently broke this with the latest change
to the module ls output.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 4fb3025682d812f86c50fa36532290fc0f8857ae)

qa: fix mgr caps

This was still using Kraken era settings

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 9988ebed9530718b523a23d729d499dedab5eb6d)

qa: expand mgr testing

Some extra coverage of the dashboard, including its standby
redirect mode and the publishing of URIs.

Also invoking the command_spam mode of the selftest module.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 05e648be6a51d3efa110ad9826bbdd0adcc4dd4d)

qa/mgr: fix influx/prometheus test names

This was a typo: they were swapped around.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d96a59e74b6984b77c9f3b15f702e3bf45053590)

qa: add mgr module selftest task

The module self test commands give us a chance to
catch any other ceph changes that change something
that a module was relying on reading.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 99352ceced9d0fe92ddad6b97b1393b41de75d50)

qa/suites/rados/thrash/d-balancer: enable balancer in various modes

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2c9c18d1ec3d33d385adf2a30dc1d3b8e987f9a5)

qa: stop testing deprecated "ceph osd create"

"ceph osd create" is not idempotent, and is considered deprecated.

Fixes: http://tracker.ceph.com/issues/21993
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 371d3f40911917704f4992c48236b4c60905bf4e)

Fix cluster config for octo node

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

qa/tasks: prolong revive_osd() timeout to 6 min

see also #17902

Fixes: http://tracker.ceph.com/issues/21474
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 749bbda07522114b99b17a4a01cfcf019520285d)

qa: do not wait for down/out osd for pg convergence

that osd is not invovlved in the PG state changes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7f549af459b40c5bb51a24a3a443ed7a7116915e)

osd: add max-pg-per-osd limit

osd will refused to create new pgs, until its pg number is lower
than the max-pg-per-osd upper bound setting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4c7df944c7f28232873ba681eedce72cdb062ea5)

qa/tasks: prolong revive_osd() timeout to 6 min

bluestore_fsck_on_mount and bluestore_fsck_on_mount_deep are enabled by
default. and bluestore is used as the default store backend. it takes
longer to perform the deep fsck with verbose log. so prolong the
revive_osd()'s timeout from 150 sec to 360 sec.

Fixes: http://tracker.ceph.com/issues/21474
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42be200c564184feff1056b7771ce3c1b2cc5527)

qa: s/backfill/backfilling/

it's renamed "backfilling" in 4015343f .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e21114274f6a2742748a5a9b965d415241c80f3c)

qa/ceph_manager: check pg state again before timedout

Pg state maybe all in active+clean when no recovering going on,
so check it again before timedout.

Fixes: http://tracker.ceph.com/issues/21294
Signed-off-by: huangjun <huangjun@xsky.com>

Fix node config for octo lab

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge branch 'multisite_tests' into 'rh-luminous'

Multisite tests

See merge request ceph/ceph!11

deleted test_multi.yaml

removed checkout of wip branch of ceph-qe-scripts

deleted unwanted files

cluters dir added

test version with rever yaml added

test suspend versioning

multipart upload download yaml added

multipart upload cancel yaml added

test multipart upload yaml added

 test delete key versions

test bucket delete yaml added

test basic versioning added

test acls reset added

test acls copy obj yaml added

test acls all user yaml added

test acls added

test Mbucket with Nobjects yaml added

test Mbuckets yaml added

fixed some bugs and verification to tests

removed print statement

Remove print statement, needs log object to print to file instead

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

update cluster to use 2 mds and 3 mons

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

update rest module loads via ceph-ansible vars

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

clone branch as is dont try to escape chars

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Debug: for test s3branch  wip-fix-416

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Fix rest test where the global vars is not defined properly

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Fix systemd tests to have more osd's

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add cephfs data/metdata vars and enable rest modules

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Run all tests by default, figure out how to skip landsat tests
in a better way

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

disable 2 size 2 min size to reduce test count

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

whitelist (MDS_CACHE_OVERSIZED)

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Disable for now, mpicc compilation issue on redhat

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

use failure domain as host

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add more osd's for 6 node cluster

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

whitelist TOO_MANY_PGS

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Fix cluster for octo lab

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

ceph-objectstore-tool: Make pg removal require --force

Add new export-remove to combine the 2 operations

Fixes: http://tracker.ceph.com/issues/21272
Signed-off-by: David Zafman <dzafman@redhat.com>

use the --distro option to run on various supported distro

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>