removed cluster names

removed cluster names

removed cluster names

removed cluster names

removed cluster names

removed cluster names

removed cluster names

removed cluster names

Merge pull request #1232 from ceph/wip-haproxy

Adds haproxy support and other cleanup fixes

Removed unused variable 'remote'

Updated docstring to describe haproxy roles

Adds haproxy support and other cleanup fixes

check if osd id is an integer, if so use it and discard any empty or non integer ids.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

fix couple more flake8 issues

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add osd_disk_info to ctx so that any task can access it like ctx.osd_disk_info

Signed-off-by: shylesh kumar <shylesh.mohan@gmail.com>

add ansible 2.6 repo

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

fix rh_install unit test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

exec failure fix

Signed-off-by: Shylesh Kumar <shylesh.mohan@gmail.com>

Fix log remote dir with dual clusters

missing nfss key  added in generate hosts file section

Fix compatibility with single cluster run

All fixes included for running dual cluster with ceph-ansible.

Fix unregister during final cleanup, due to stage cdn task that is not needed now.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

need to debug why check_status is being ignored, for now catch
the command failed error, log and ignore so that tests can pass.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

unsubscribe at the start of the test as well, if nodes are not freshly reimaged
then they may have subscription from old configuration

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

typo fix and ignore status if unregister fails

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Enable standard rhel repos as well after stage cdn configuration

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

subscribe rhel octo nodes using stage cdn from configuration

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add misc cleanup options before we unlock the system

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

osd addition fix in roles and cleanup shipped files

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

ship_utilities import fix

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

ship utilities file during ceph-ansible task

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

detect osd id's and use them in new roles as ansible can change the order
of osd's that are bootstrapped

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Fix roles after cluster is setup for mon/mgr/mds and also
setup daemons for use by ceph_manager

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Fog provides fresh setup so there is no need to do any precleanup

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

fast fix for nvme test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Disable running cm-ansible by default, FOG images have already
run cm-ansible code and it would be redundant to run them again.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

dont poweroff nodes, leave it as is for adhoc debug if needed.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Octo lab has no VPS and its default FOG, so we dont
need any data[os_type] to be sent at all for queue requests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

skip specific os_type/os_version request when FOG reimage in place

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Enable FOG reimage by default for all machine types

the downside: you can't control from teuthology.yaml what
machines can be enabled for FOG

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

run git_ignore_ssl task after ansible setup

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add internal task to ignore ssl errors while
cloning from untrusted gitlab

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add option to schedule jobs using simple subset

Total number of jobs are divided into equal subsets based on
user options and can be scheduled using 1/10, 2/10 etc

If a suite produces 100 jobs, 1/10 would schedule first 10 jobs,
2/10 will schedule next set of 10 jobs and so on.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

use update_latest_rh_kernel for redhat configs

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add option to retry for exec task.

retry and sleep_for_retry options can be used with exec task without
needing to use lengthy shell based while loops for similar
task. Also adds timeout option for commands that can be used
to return within specified time.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

when cm-ansible breaks!, backup to skip cm-ansible and run jobs

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

misc run-rh-task fixes during rhcs 3.0

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

debian repo is setup during internal task and no longer needed.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Some more cleanup before repo setup and skip subscription manager
for beta ISO tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #1209 from smithfarm/wip-sleep-seconds

teuthology/task/sleep: be explicit about seconds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1211 from tchaikov/wip-valgrind-dpdk

teuthology/task/install/valgrind.supp: add suppression for DPDK

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

teuthology/task/install/valgrind.supp: add suppression for DPDK

Signed-off-by: Kefu Chai <kchai@redhat.com>

teuthology/task/sleep: be explicit about seconds

Add the word "seconds" at the end of the log message, since "time.sleep()"
takes a number which is always interpreted as the number of seconds to sleep
for.

Before this commit, the log said:

    INFO:teuthology.task.sleep:Sleeping for 10

After:

    INFO:teuthology.task.sleep:Sleeping for 10 seconds

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1205 from tchaikov/wip-rerun

worker,result: create archive_dir for the first job and mark --{seed,subset} optional

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

results: mark --subset and --seed optional

before d488b9bd, these params were mandatory. after d488b9bd, they are
optional. because

- these parameters passed in only for "first-in-suite" job
- subset is not mandatory even for "first-in-suite", because there is
  chance that user want to run the full combination of the test matrix.

Signed-off-by: Kefu Chai <kchai@redhat.com>

worker: create archive_dir before putting log file in it

before d488b9bd, the memo for rerunning a suite is noted down by the
last-in-suite job. when the last-in-suite job is performed, the
archive_dir has been created by the jobs which performs tests, see
the `Creating archive dir` line in run_job() in teuthology/worker.py .

but after d488b9bd, the memo is logged by the first-in-suite job, by
then, none of the test jobs is performed, so their archive dirs are not
created. i think that's why the first-in-suite job fails to write the
memo to $archive_dir/results.log.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1204 from tchaikov/wip-rerun

 worker: do not pass --timeout to first-in-suite job

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Zack Cerza <zack@redhat.com>

worker: do not pass --timeout to first-in-suite job

likewise, do not pass --seed or --subset to last-in-suite job
otherwise, teuthology/schedule.py will raise a ValueError at seeing
--subset or --seed not coming along with --first-in-suite, or
--email or --timeout not coming along with --last-in-suite.

Signed-off-by: Kefu Chai <kchai@redhat.com>

teuthology/suite: use specified seed and subset if results.log is not readable

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1198 from tchaikov/wip-rerun

script,worker,suite: note down the rerun parameter in the first-job-in-suite

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

suite/run,schedule,result: write rerun memo as the first job in suite

so we don't need to wait for the job to write result to for rerunning
the test suite. without this change, the "result" is normally the last
job in the suite to be scheduled, so it's likely we will not have the
results.log until the suite is almost completed. afer this change,
a "first-in-suite" job is scheduled as the first job to note down
the subset and seed to run the suite.

Signed-off-by: Kefu Chai <kchai@redhat.com>

suite.run: extract write_result() out of prepare_and_schedule()

so it's more modular.

Signed-off-by: Kefu Chai <kchai@redhat.com>

suite: do not bail out if results.log is not around

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1202 from smithfarm/wip-extra_system_packages

task/install: propagate extra_system_packages

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

task/install: propagate extra_system_packages

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1192 from smithfarm/wip-25020

install: make extra_packages distro-agnostic (in a limited way) and support extra_system_packages option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1201 from ceph/wip-syslog-whitelist-info

internal/syslog: whitelist ceph-crash daemon messages

test_install.py: sort package lists before comparing

Signed-off-by: Nathan Cutler <ncutler@suse.com>

internal/syslog: whitelist ceph-crash daemon messages

None of these should cause test failures.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1046 from smithfarm/wip-run-py

run.py: Provide --os-{type,version} values to get_initial_tasks

Merge pull request #1001 from SUSE/wip-18141

platform.linux_distribution() is deprecated; stop using it

Merge pull request #1200 from tchaikov/wip-rerun-fix

suite,report: initialize seed with None

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #1199 from tchaikov/pytest-requires-pluggy-v0.7

setup,py,requirements.txt: add pytest

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

suite,report: initialize seed with None

so we can set it if it is not initialized

Signed-off-by: Kefu Chai <kchai@redhat.com>

setup,py,requirements.txt: add pytest

pytest requires pluggy >= 0.7, while we always use pluggy 0.6, as
specified by requirements.txt. as this version is good enough for
tox. but in tox.ini, we do use pytest, and no version is specified,
so we have good chance running into https://github.com/pytest-dev/pytest/issues/3753

also, remove pytest from tox.ini, as this dependency has been
added in requirements.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1197 from tchaikov/wip-str.replace

task/install: str.replace() does not keyword args

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

task/install: str.replace() does not keyword args

In [8]: ?s.replace
Docstring:
S.replace(old, new[, count]) -> str

Signed-off-by: Kefu Chai <kchai@redhat.com>

install: make exclude_packages distro-agnostic (in a limited way)

Signed-off-by: Nathan Cutler <ncutler@suse.com>

install: support extra_system_packages config option

On DEB systems, packages specified via the extra_packages option are installed
while forcing the same package version number as the version of the project
(i.e. Ceph) under test. So extra_packages can only be used to specify
additional project (Ceph) packages.

If we wanted to specify additional system (non-project, non-Ceph) packages to
install, we were out of luck. This commit implements an extra_system_packages
option for specifying extra non-project packages.

Fixes: http://tracker.ceph.com/issues/25026
Signed-off-by: Nathan Cutler <ncutler@suse.com>

install: make extra_packages distro-agnostic (in a limited way)

Where "distro-agnostic (in a limited way)" means: allow the user to specify
different lists for DEB and RPM.

This fix does not address the case when two RPM distros, or two DEB distros,
have different names for a given package/capability.

Fixes: http://tracker.ceph.com/issues/25020
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1196 from tchaikov/wip-rerun-fix

report: strip space and quotes from parsed message

report: strip space and qutoes from parsed message

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1195 from ceph/revert-1090-wip-upgrade-equal

Revert "[DNM] upgrade check: upgrade to same version is a noop"

Revert "[DNM] upgrade check: upgrade to same version is a noop"

Merge pull request #1090 from smithfarm/wip-upgrade-equal

[DNM] upgrade check: upgrade to same version is a noop

Merge pull request #1193 from adamwg/handle-branches-with-slashes

Handle branch names containing slashes

Handle branch names containing slashes

A branch name containing a slash is perfectly legal in git, but
teuthology uses branch names verbatim in run names, which causes POSTs
to fail when submitting runs to paddles. Replace all '/' in run names
with ':' to allow for branches with slashes in their names.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>

orchestra/opsys: support openSUSE Leap

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests/opsys: add more OS.from_os_release test cases

Signed-off-by: Nathan Cutler <ncutler@suse.com>

orchestra/opsys: add new distros to codenames map

Signed-off-by: Nathan Cutler <ncutler@suse.com>

platform.linux_distribution() is deprecated; stop using it

Fixes: http://tracker.ceph.com/issues/18141
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1191 from ceph/wip-ansible-default

default ansible to 2.5 as per ceph-ansible requirements

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

default ansible to 2.5 as per ceph-ansible requirements for ceph master branch testing.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #1189 from tchaikov/wip-fix-rerun

results,schedule,woker: persist --seed and --subset in results.log

Reviewed-by: Warren Usui <wusui@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1190 from ceph/install_nfs_ganesha_with_ceph-ansible

added nfs-ganesha to roles

added nfs-ganesha to roles

test: '$' is not removed from path even if triggers random tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

results,schedule,woker: persist --seed and --subset in results.log

to create a repeatable test suite, in addition to `--seed <SEED>`, we also
need to pass the same `--subset <SUBSET>` to teuthology-suite when
rerunning the failed tests. but it would be handy if teuthology-suite
could remember these settings and recall them when `--rerun <RUN>`.

in this change, we repurpose the last job sending the email to report
the test result to note down the subset and seed used for scheduling the
test suite. these variables are stored in results.log at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #1186 into master

* refs/pull/1186/head:
ceph.conf.template: drop cephfs configs

Reviewed-by: Zheng Yan <zyan@redhat.com>