testnodes: Disable firewalld service after reboot on RPM-based distros

Fixes: http://tracker.ceph.com/issues/16455
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #256 from ceph/wip-global-cpan

testnodes: Configure cpan and install Amazon::S3 on all OSes

Merge pull request #249 from ceph/wip-gw-networking

Network configuration for gateway role

Reviewed-by: Dan Mick <dmick@redhat.com>

gateway: Add logrotate functionality

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Add README documentation for networking, firewall, fail2ban

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Add fail2ban support

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Configure firewalld

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Enable server network config

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #252 from ceph/wip-cblr-wget

cobbler: testnode post-install rc.local fixes

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Zack Cerza <zack@redhat.com>

testnodes: Configure cpan and install Amazon::S3 on all OSes

Fixes: http://tracker.ceph.com/issues/15316
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #254 from ceph/wip-clone-once

Only clone the keys repo once per execution

Reviewed-by: Dan Mick <dmick@redhat.com>

Only clone the keys repo once per execution

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #253 from ceph/wip-fast-keys

users: Greatly speed up ssh pubkey deployment

Reviewed-by: Dan Mick <dmick@redhat.com>

Put the keys repo in ~/.cache/src/ by default

To avoid multiple invocations by different users on the same host
stepping on each other.

Signed-off-by: Zack Cerza <zack@redhat.com>

Remove unnecessary retry

Since each key isn't being fetched from a remote server any longer, we
can drop the retries.

Signed-off-by: Zack Cerza <zack@redhat.com>

Speed up key deployment by using the git repo

Instead of downloading each key over HTTPS from github.com, we can
simply clone the entire repo (with depth 1) and lookup each key using
the username.

On my laptop, execution time went from 2m49s to 29s.

Signed-off-by: Zack Cerza <zack@redhat.com>

users: Add defaults for keys_repo

Signed-off-by: Zack Cerza <zack@redhat.com>

users: split tasks/main.yml into separate files

Signed-off-by: Zack Cerza <zack@redhat.com>

cobbler: testnode post-install rc.local fixes

- Instead of sleeping for 30 secs and hoping network is up, wait until we can
reliably ping the Cobbler host.
- Give ansible more time to run before wget's built-in 15min timeout is reached.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #251 from ceph/wip-edeploy

users: Set UID_MIN to 1001

Reviewed-by: Dan Mick <dmick@redhat.com>

users: Set UID_MIN to 1001

When building images with edeploy, UID 1000 was being taken by a
"normal" user; we need to reserve this for the teuthology user.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #250 from ceph/wip-no-lsb

Don't rely on LSB when testing for distro version

Reviewed-by: Dan Mick <dmick@redhat.com>

Don't rely on LSB when testing for distro version

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #248 from ceph/wip-xenial

testnodes: Fix resolvconf issues on xenial

Improve network interface discovery

Signed-off-by: Zack Cerza <zack@redhat.com>

Always bounce the main interface on xenial

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #247 from ceph/wip-16119

pcp: Don't fail when setting permissions

pcp: Don't fail when setting permissions

http://tracker.ceph.com/issues/16119
Fixes: 16119
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #246 from dmick/master

smart.sh: default rc to 0 (saw false UNKNOWNs with non-raid)

smart.sh: default rc to 0 (saw false UNKNOWNs with non-raid)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #245 from ceph/wip-newsmart

testnode: Replace old smart.pl with rewritten smart.sh script

Reviewed-by: Dan Mick <dmick@redhat.com>

testnode: Replace old smart.pl with rewritten smart.sh script

Fixes: http://tracker.ceph.com/issues/14682
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #244 from cooboos/centos

testnode: replace hardcoded 'gitbuilder.ceph.com' with {{ gitbuilder_host }}

testnode: replace hardcoded 'gitbuilder.ceph.com' with {{ gitbuilder_host }}

Signed-off-by: wangsongbo <wangsongbo@unitedstack.com>

Merge pull request #243 from ceph/wip-sam-ansible

testnode/tasks/apt/packages.yml: just force everything

Merge pull request #241 from ceph/wip-pcp-settings

pcp: update upstream packages

Reviewed-by: Dan Mick <dmick@redhat.com>

pcp: update upstream packages

PCP's upstream doesn't version its dependencies in all cases. This leads
to situations where, when setting upstream_repo to True, a system ends
up with some old and some new packages. This results in a potentially
broken installation.

This commit causes all installed pcp packages to be upgraded when
upstream_repo is True.

Signed-off-by: Zack Cerza <zack@redhat.com>

testnode/tasks/apt/packages.yml: just force everything

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #239 from ceph/wip-pcp-settings

pcp: retention settings for managers

Make pmlogmerge_reduce configurable

Signed-off-by: Zack Cerza <zack@redhat.com>

Make pmlogmerge_retain configurable

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #240 from ceph/wip-nrpe-load-limits

testnode: Update nagios CPU load thresholds to be less strict

testnode: Update nagios CPU load thresholds to be less strict

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #238 from ceph/wip-pcp-js

pcp: A dynamic dashboard with host- and time-selection

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #237 from ceph/wip-check4-selinux

testnode: Skip NRPE selinux setup if selinux disabled

index.js

testnode: Skip NRPE selinux setup if selinux disabled

Fixes: http://tracker.ceph.com/issues/15675
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #236 from ceph/wip-xenial-nicnames

cobbler: Prevent systemd from renaming NICs

Reviewed-by: Dan Mick <dmick@redhat.com>

cobbler: Prevent systemd from renaming NICs

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #235 from ceph/wip-no-getty

testnode: Remove manual configuration of getty

Reviewed-by: Dan Mick <dmick@redhat.com>

testnode: Remove manual configuration of getty

Manual configuration of the getty service is no longer required for
Trusty or later.  The system-specific "console=ttySX" parameter in
/proc/cmdline is enough to configure the SOL tty during Ubuntu
installation.

See http://tracker.ceph.com/issues/15269 for further explanation.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #234 from ceph/wip-preseed-bootdev

cobbler: Only set bootdev in kickstart if installing Xenial.

Reviewed-by: Dan Mick <dmick@redhat.com>

cobbler: Only set bootdev in kickstart if installing Xenial.

"bootdev string default" is only supported in Xenial.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #233 from ceph/wip-preseed-bootdev

cobbler: Only set bootdev in kickstart if installing Xenial.

cobbler: Only set bootdev in kickstart if installing Xenial.

"bootdev string default" is only supported in Xenial.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #232 from ceph/wip-xenial

Add support for Ubuntu 16.04 Xenial

cobbler: Make Ubuntu kickstart compatible with Xenial

linux-firmware-nonfree package is not available in Xenial

Installer was also stopping after package install asking for
GRUB install destination.  Adding the 'bootdev string default'
line bypasses the message and is already the lab default so
shouldn't affect other distro versions.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Add Xenial-specific packages

Used roles/testnode/vars/ubuntu_14.yml as template.  Differences are:
 - libboost-thread1.54.0 updated to libboost-thread1.58.0
 - libcrypto++9 updated to libcrypto++9v5
 - libssl0.9.8 updated to libssl1.0.0
 - mpich2, libmpich2-dev, and dmapi not available in Xenial

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #229 from ceph/wip-nameserver

A role for an internal master nameserver

Reviewed-by: Dan Mick <dmick@redhat.com>

testnode: Add sshd config for Ubuntu 16.04

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Add Ubuntu 16.04 (Xenial Xerus)

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #231 from ceph/wip-nrpe-selinux

testnode: Update SELinux policy when configuring NRPE

testnode: Update SELinux policy when configuring NRPE

The 'nagios_run_sudo' boolean was added after RHEL7.1 GA.

https://rhn.redhat.com/errata/RHBA-2015-1140.html

Attempting to set the boolean when it's not present causes ansible
to fail.  NRPE will cause SELinux failures in test jobs unless set
so the only other option would be to disable NRPE on
RHEL/CentOS <= 7.1

This change ensures the testnode has the latest version of SELinux
policy installed, and thus, ensuring the boolean exists.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #230 from ceph/wip-pcp-perms

pcp: Correct /var/log/pcp permissions for collectors

Correct /var/log/pcp permissions for collectors

Some hosts were getting a /var/log/pcp/pmmgr that was owned by root:root
for some reason.

Fixes: http://tracker.ceph.com/issues/15575
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #227 from ceph/wip-pcpweb-deb

Add pcp-webapi on ubuntu upstream setups; set pmcd connect timeout for pmmgr

nameserver: Add check for ansible_version.major >= 2

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Add README

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Add ability to set named daemon options

The variable in defaults/main.yml will leave /etc/sysconfig/named
empty which is the package default.  We want IPv4 only however.
This is set in the secrets repo.

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Configure firewalld to allow DNS traffic

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Add configuration task for named service

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Add named service handlers

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Add records task

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Add packages task

Signed-off-by: David Galloway <dgallowa@redhat.com>

nameserver: Initial role structure creation

Signed-off-by: David Galloway <dgallowa@redhat.com>

pcp_manager_package varies depending on the source

Signed-off-by: Zack Cerza <zack@redhat.com>

Fix typo

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #228 from ceph/wip-no-lstn

common: Replace bad epel mirror

common: Replace bad epel mirror

Mirror fedora-epel.mirror.lstn.net is almost a week out of date and
causing yum package transactions to fail due to bad repodata.  Replacing
with a mirror closer to the sepia and octo labs.

Fixes: http://tracker.ceph.com/issues/15539
Signed-off-by: David Galloway <dgallowa@redhat.com>

Set PMCD_CONNECT_TIMEOUT

Signed-off-by: Zack Cerza <zack@redhat.com>

Enable pcp-webapi on Ubuntu with upstream packages

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #226 from ceph/wip-pcp-upstream

pcp: Optionally use upstream package repos

pcp: Optionally use upstream package repos

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #225 from dachary/wip-xenial

testnodes: add ubuntu 16.04 xenial support

Reviewed-by: Dan Mick <dmick@redhat.com>

openstack: console handling is done differently

All hostnames starting with target are created for OpenStack and their
console is accessible via horizon or via other means. There is no need
to configure the serial console in this case.

Signed-off-by: Loic Dachary <loic@dachary.org>

testnodes: add ubuntu 16.04 xenial support

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #219 from dachary/wip-paddles

minor boostrap paddles fixes

Merge pull request #224 from dachary/wip-arch

skip packages / repo unavailable on aarch64

Reviewed-by: Sébastien Han <shan@redhat.com>

skip packages / repo unavailable on aarch64

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #222 from ceph/wip-nagios-load

testnodes: Set sane warn/crit thresholds for nagios check_load

Reviewed-by: Dan Mick <dmick@redhat.com>

Revert "replace hardcoded x86_64 with {{ansible_architecture}}"

This reverts commit 74404d3e9f11c62fddb00fcc1616c02d7f520af3.

Merge pull request #214 from ceph/wip-pcp

A role for PCP

Reviewed-by: Dan Mick <dmick@redhat.com>

Embellish overview documentation

Signed-off-by: Zack Cerza <zack@redhat.com>

Mention support for CentOS 6 and Debian 8

Signed-off-by: Zack Cerza <zack@redhat.com>

Support Fedora as a collector

Signed-off-by: Zack Cerza <zack@redhat.com>

Move pcp toward the beginning of the playbook

Signed-off-by: Zack Cerza <zack@redhat.com>

Don't depend on the users role

It's not a real dependency; if we want to update users on the node we
can run the users.yml playbook.

Signed-off-by: Zack Cerza <zack@redhat.com>

Set PMCD_CONNECT_TIMEOUT in pmmgr.options

With ~400 hosts, polling takes a very long time with default values.
Dropping the connect timeout to 0.1s speeds things way up.

Signed-off-by: Zack Cerza <zack@redhat.com>

Remove lab_domain from target-host

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #223 from ceph/wip-sam-ansible

Don't use sudo for rpm --rebuild-db and remove more packages for testnode