teuthology: Add teuthology_yaml_extra

This is an optional var that allows arbitrary yaml fragments to be
included in teuthology.yaml

Signed-off-by: Zack Cerza <zack@redhat.com>

teuthology: Remove bogus default

Signed-off-by: Zack Cerza <zack@redhat.com>

teuthology: Serve logs under /teuthology

Signed-off-by: Zack Cerza <zack@redhat.com>

teuthology: Add tags for config and logs

Signed-off-by: Zack Cerza <zack@redhat.com>

paddles: Restart if prod.py changes

Signed-off-by: Zack Cerza <zack@redhat.com>

paddles: Always call the set_fact plays

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #150 from ceph/wip-fix-users

Don't attempt to add groups that don't exist

Reviewed-by: Dan Mick <dmick@redhat.com>

Don't attempt to add groups that don't exist

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #149 from dmick/master

Create users with same-named primary group

Create users with same-named primary group

If adding sudo, add as secondary group ('groups:').  This is more
standard, and allows for quick enumeration of "who's in sudo" by
examining /etc/group

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #139 from ceph/wip-dban

Add dummy dban-2.3.0 entry

Merge pull request #148 from ceph/wip-teuthology

teuthology: Serve logs over http

teuthology: Serve logs over http

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #147 from ceph/wip-teuthology

teuthology: manage teuthology system users

teuthology: manage teuthology system users

'teuthology' for scheduling and 'teuthworker' for execution

This is intended to mirror our existing setups

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #138 from ceph/wip-puddle-updates

puddle: multiple updates

Merge pull request #146 from ceph/wip-paddles

Add new paddles role

pulpito: Restart if the config changes

Signed-off-by: Zack Cerza <zack@redhat.com>

paddles: Add new paddles role

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #145 from ceph/wip-hostname

testnode: set_hostname should be a global task

Update documentation for lab_domain var

testnode: set_hostname should be a global task

Signed-off-by: dgalloway <dgallowa@REDHAT.COM>

puddle: new RH-COMMON advisory undergoing testing

Adjust the RHEL 6 and 7 RH-COMMON Puddle configs for a new async update
that will be undergoing testing.

puddle: output RHCEPH 1.3 to separate locations

I periodically generate Puddles for both RHCS 1.3.0-async and 1.3.1.
It's confusing when these Puddles exist in the same parent directory.
Split them up into two directories so it is easier to understand what
each Puddle is.

Merge pull request #144 from ceph/wip-13237

Remove custom qemu packages

Remove custom qemu packages

Signed-off-by: dgalloway <dgallowa@REDHAT.COM>

Merge pull request #132 from ceph/wip-magna-smart

Updates to nagios disk monitoring script

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #142 from ceph/wip-12739

Use local epel mirrorlist

Configure yum to use local epel mirrorlists

Signed-off-by: dgalloway <dgallowa@REDHAT.COM>

Merge pull request #141 from ceph/wip-rm-cuttlefish-repo

testnode: rm cuttlefish repository

testnode: rm cuttlefish repository

Prior to this commit, every RHEL 6 and CentOS 6 node would automatically
have the cuttlefish repository enabled.

This hardcoding seems wrong. Now that we've moved the RPMs off of
ceph.com to download.ceph.com, it actually breaks nodes. So let's stop
hard-coding this one altogether.

Reported-by: Warren Usui <wusui@redhat.com>

Merge pull request #140 from ceph/wip-13203-epel-fcgi

testnode: add fcgi to the list of EPEL packages

testnode: add fcgi to the list of EPEL packages

For Ceph upstream, we rely on fcgi from EPEL.

http://tracker.ceph.com/issues/13203 Refs: #13203

Add dummy dban-2.3.0 entry

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #135 from ceph/wip-cobbler-rescue

cobbler_profile: add non-iso image support and inktank-rescue

Make multiline output default

Found typo in conditional

Merge pull request #137 from ceph/rm-ceph-extras

rm ceph-extras repository config

Merge pull request #136 from dmick/master

Some fixes to vmlist.py: update host lists, fix error from missing cloud config items

rm ceph-extras repository config

This repository contains packages that we don't have time to continue
maintaining. In particular, the CentOS 6 QEMU packages were vulnerable
to VENOM (CVE-2015-3456) and no users should rely on them. The other
packages in "ceph-extras" will continue to incur security issues over
time.

With the recent migration from "ceph.com" -> "download.ceph.com", we've
chosen not to copy these "ceph-extras" files over to the new locations.

vmlist.py: default cloud_* so their absence doesn't cause errors

Signed-off-by: Dan Mick <dan.mick@redhat.com>

vmlist.py: senta01 was inexplicably missing (and made me miss ursula!)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

vmlist.py: mira004 is back from the dead

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Add distro_prep tag to distro_prep.yml

Signed-off-by: Zack Cerza <zack@redhat.com>

Add dummy inktank-rescue entry

Signed-off-by: Zack Cerza <zack@redhat.com>

Add inktank-rescue to cobbler.yml

Signed-off-by: Zack Cerza <zack@redhat.com>

cobbler_profile: Import kernel/initrd distros

This will be useful for things like rescue images.

Signed-off-by: Zack Cerza <zack@redhat.com>

cobbler_profile: Split out ISO importing

So that we can have a separate codepath for importing distros from
non-ISO sources.

Signed-off-by: Zack Cerza <zack@redhat.com>

Cobbler: Create other_image_dir

Signed-off-by: Zack Cerza <zack@redhat.com>

Cobbler: Add other_image_dir default

For storing non-iso images for distro/profile creation

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #134 from ceph/wip-fix-distros

cobbler_profile fixes

Ubuntu 14.04.2 -> 14.04.3

14.04.2 is deprecated and ISOs are no longer shipped.

Signed-off-by: Zack Cerza <zack@redhat.com>

CentOS 6.6 -> 6.7

6.6 is deprecated and ISOs are no longer shipped.

Signed-off-by: Zack Cerza <zack@redhat.com>

Add dummy entry for RHEL-6.7

Signed-off-by: Zack Cerza <zack@redhat.com>

Don't attempt to manage tftpd on apt systems

Signed-off-by: Zack Cerza <zack@redhat.com>

Revert "Simplify extra_distros logic"

This reverts commit 0e5eaacba746c698315755b2f8464043d8f37910.

Merge pull request #133 from ceph/wip-pulpito-playbook

Adds the pulpito playbook

Adds the pulpito playbook

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Some useful comments explaining functions

Count num failed drives and output in first line of multiline output

Proceed as JBOD if not RAID

Merge pull request #131 from ceph/wip-pulpito

Pulpito role

Add pulpito role to cephlab.yml

Signed-off-by: Zack Cerza <zack@redhat.com>

Add role to deploy pulpito

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #130 from ceph/wip-12921

downstream-setup: yum_repos will now be removed during cleanup

downstream-setup: yum_repos will now be removed during cleanup

Fixes: http://tracker.ceph.com/issues/12921
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #129 from ceph/wip-smart

smart.pl multiline mode: show failed/missing drives as well

Reviewed-by: David Galloway <dgalloway@redhat.com>

smart.pl: drive numbers are 1-based

Signed-off-by: Dan Mick <dan.mick@redhat.com>

smart.pl multiline mode: show failed/missing drives as well

also, make a global "multiline" flag (since it's now being used
in multiple places)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #127 from ceph/wip-12957

Add support for debian 8

testnode: ensure fuse group exists

On debian 8, uploading the fuse.conf file fails because the fuse group
is no longer present by default. The following debian bug suggests that
testing the fuse group might not be the correct thing to do, but it's
probably best just to keep this as close to debian 7 as possible right
now.

See: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793927

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #128 from ceph/wip-smart

Several bugs/usability things in smart.pl, and a new version with human output

Add -m to smart.pl that prints output for humans

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Find Areca sg device in the documented way; no idea about vsf info

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Output drive letter along with drive number for convenience

Signed-off-by: Dan Mick <dan.mick@redhat.com>

testnode: add a retry to installing the apt keys

Because we're using a mirror that sometimes doesn't respond in a timely
manner, add 2 minutes of retries to overcome transient HTTP issues.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

testnode: add support for debian 8

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

testnode: always update the apt cache

Updating the apt cache is an idempotent task so I see no problem with
always doing this. Also, debian jessie and ubuntu vivid aren't using the
local apt repos, but still need the apt cache updated.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

common: update apt cache on debian before installing kerberos packages

Without updating the apt cache, debian jessie can not find the package
krb5-user.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #124 from ceph/wip-12900

testnode: retry when updating the apt cache to handle http errors

Merge pull request #125 from ceph/wip-puddle-updates

puddle: rh-common-rhel-6 and ceph-1.3.z adjustments

puddle: publish ceph-1.3.z for QE testing

Travis and I think RHCS 1.3.1 looks good enough to give to QE at this
time. Instead of publishing it to the "dev-puddle" website, publish it
to the main puddle site.

puddle: second RHEL 6 async undergoing testing

Adjust the RHEL 6 RH-COMMON Puddle config for a second RHEL 6 async
update that will be undergoing testing.

testnode: retry when updating the apt cache to handle http errors

When updating the apt cache we have to make an http call out to
gitbuilder, if gitbuilder fails for whatever reason we should retry
before failing the playbook.

Also, this removes a 'changed_when: false' on 'Add local apt repos'.
We check it's 'changed' status to know if we need update the cache or
not. We only added that 'changed_when' to avoid seeing this task being
changed each time (because it will always change), but we actually need
that value set correctly because we use it in the 'when' clause of the
updating apt cache stanza.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #123 from ceph/wip-teuth-user-keys-retry

testnode: add a retry when fetching the teuthology_user ssh keys

testnode: add a retry when fetching the teuthology_user ssh keys

We keep seeing transient failures when pulling down ssh keys from
github. This will retry for two minutes before failing to hopefully give
github enough time to recover if it's timing out.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #122 from ceph/wip-12868

users: increase the retry count to 24 for fetching ssh keys from github

users: increase the retry count to 24 for fetching ssh keys from github

We were still getting timeouts when requesting keys from github, so
we're gonna try to just simply increase the number of times it retries
to see if that helps things.

See: http://tracker.ceph.com/issues/12868

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #121 from ceph/wip-12778

common: set timeout in /etc/yum.conf

common: set timeout in /etc/yum.conf

We were getting timeouts when using epel mirrors and the default timeout value
is only 30 seconds which is not long enough for heavily used mirrors to
respond.

See: http://tracker.ceph.com/issues/12778

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #120 from ceph/wip-12794

testnodes: Check for and remove custom apt repos

testnodes: Check for and remove custom apt repos

http://tracker.ceph.com/issues/12794

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #118 from ceph/wip-skip-entitlements-sepia

Common: If no entitlements, skip repo replacement

Common: If no entitlements, skip repo replacement

We were seeing "error while evaluating conditional: replace_repos" when
running against RHEL nodes which were missing entitlements. Even though
that won't be a particularly positive scenario, this was a bug that
should be fixed.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #117 from ceph/wip-epel-mirrorlist

Enable mirrorlist for epel repo

Enable mirrorlist for epel repo

Signed-off-by: dgalloway <dgallowa@REDHAT.COM>

Merge pull request #116 from ceph/wip-12693

downstream-setup: separate cleanup and setup tasks into their own files

downstream-setup: separate cleanup and setup tasks into their own files

This is a refactor that makes this role a bit easier to understand and
use. I've simplified main.yml to only include setup.yml and cleanup.yml.
This separation of cleanup and setup related tasks make it a bit easier
to understand.

Also fixes this bug: http://tracker.ceph.com/issues/12693

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #111 from ceph/wip-12670

testnode: make teuthology_user passwordless

Reviewed-by: Dan Mick <dmick@redhat.com>

testnode: make teuthology_user passwordless

Signed-off-by: Andrew Schoen <aschoen@redhat.com>