Retry SSH connections five times

This may help with some of the problems we've been seeing.

Related to http://tracker.ceph.com/issues/16826
Inspired by
https://github.com/ansible/ansible/issues/13401#issuecomment-216768025

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #271 from ceph/wip-no-fw

testnode: Disable firewalld and iptables regardless of OS version

testnode: Disable firewalld and iptables regardless of OS version

iptables was recently found installed and running on a RHEL7 system.
Previous testnode playbook runs wouldn't catch this since it shouldn't
be installed in the first place.  This change ensures firewalld and
iptables are stopped on all RPM-based distros.

Fixes: http://tracker.ceph.com/issues/16809
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #270 from ceph/wip-keys-retry

Add a retry when cloning the keys repo

Add a retry when cloning the keys repo

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #269 from ceph/wip-cobbler-gitrace

Use flock to avoid race conditions with git

Use flock to avoid race conditions with git

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #265 from ceph/wip-nagios-common

Move NRPE setup to common role

Merge pull request #267 from ceph/wip-no-gperftools-devel

testnode: drop gperftools-devel

Merge pull request #268 from ceph/wip-failure-log-debug

If a YAML error is hit, log the raw object

Reviewed-by: Dan Mick <dmick@redhat.com>

If a YAML error is hit, log the raw object

We are seeing: "RepresenterError: cannot represent an object: pcp"

Signed-off-by: Zack Cerza <zack@redhat.com>

testnode: drop gperftools-devel

Stop installing the gperftools-devel package on testnodes, for the
following reasons:

1. Ceph does not require gperftools-devel at runtime, so installing it
   unnecessarily just slows everything down at this point.

2. When we want to test newer builds of gperftools (say, from RHEL 7.3),
   if the corresponding "gperftools-devel" package is not available on
   the test node, then yum will fail the transaction.

(Additionally, gperftools-devel is no longer in EPEL 7, and it has moved to
RHEL 7 Optional in RHEL 7.2 (https://bugzilla.redhat.com/1213879,
https://access.redhat.com/errata/RHEA-2015:2293). The "epel" list is no
longer the correct list for this package on redhat_7 and centos_7.)

Merge pull request #266 from ceph/wip-timeout

use 120s instead of default 12s for cmd timeout

common: Update nrpe SELinux policy

This should've been done when smart.sh replaced smart.pl.
I just didn't notice smart.sh was getting denied by SELinux until I
started monitoring disks in Octo.  Evidently the new script requires
much more permission to run.

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Create README

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Install some ansible dependencies during kickstart

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Combine some nrpe-selinux package installations

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Always install disk_monitoring scripts when nagios tag is called

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Move smartmontools to disk_monitoring task

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Enable nrpe installation without epel repo on RHEL/CentOS

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Move nrpe package install to common role

Signed-off-by: David Galloway <dgallowa@redhat.com>

Move NRPE setup to common role

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Configure firewalld (when enabled) for NRPE

Signed-off-by: David Galloway <dgallowa@redhat.com>

use 120s instead of default 12s for cmd timeout

Increase timeout from 12s to 120s for ansible cmds

Signed-off-by: Vasu Kulkarni vasu@redhat.com

Merge pull request #264 from ceph/wip-16615

Don't use a shallow copy of keys.git

Don't use a shallow copy of keys.git

http://tracker.ceph.com/issues/16615
Fixes: 16615
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #263 from ceph/wip-cpan

testnodes: Install perl-CPAN on yum systems

testnodes: Install perl-CPAN on yum systems

perl-CPAN is required to install Amazon::S3 using the 'cpan' command

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #262 from ceph/wip-become

More Ansible v2 fixes

Bracket bare variables for Ansible v2

Signed-off-by: David Galloway <dgallowa@redhat.com>

Add missing 'become' directive to a few more roles

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #261 from ceph/wip-rclocal-msg

cobbler: Output message indicating Ansible is running after firstboot

Reviewed-by: Dan Mick <dmick@redhat.com>

cobbler: Output message indicating Ansible is running after firstboot

Fixes: http://tracker.ceph.com/issues/14297
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #260 from ceph/wip-downstream-fixes

downstream_setup role fixes

downstream_setup: Ensure role is only run on RHEL or CentOS

Signed-off-by: David Galloway <dgallowa@redhat.com>

downstream_setup: Add 'become' privilege escalation directive

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #259 from ceph/wip-entitlements-bools

common: Fix boolean expressions

Fix boolean expressions

Signed-off-by: Zack Cerza <zack@redhat.com>

Revert "Use ansible_user instead of ansible_ssh_user"

This reverts commit 7cc43cf253fc0dceea381ff11ed1d3e546f70ff8.

Merge pull request #258 from ceph/wip-pip

Add missing pip.yml

Add missing pip.yml

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #255 from ceph/wip-ansible-2

DNM: Move to ansible 2

Merge pull request #257 from ceph/wip-disable-firewalld

testnodes: Disable firewalld service after reboot on RPM-based distros

Reviewed-by: Dan Mick <dmick@redhat.com>

testnodes: Disable firewalld service after reboot on RPM-based distros

Fixes: http://tracker.ceph.com/issues/16455
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #256 from ceph/wip-global-cpan

testnodes: Configure cpan and install Amazon::S3 on all OSes

Disable become for repo cloning (2.x method)

Signed-off-by: Zack Cerza <zack@redhat.com>

Remove sudo role; replace with 'become' setting

The old method of setting the default behavior for become/sudo has
changed in 2.x; fortunately the new way is a bit more elegant

Signed-off-by: Zack Cerza <zack@redhat.com>

Quote variables used by with_items

Signed-off-by: Zack Cerza <zack@redhat.com>

cobbler: Install ansible from PyPI

The distros we're using don't ship ansible 2.

Signed-off-by: Zack Cerza <zack@redhat.com>

Locate templates properly

See https://github.com/ansible/ansible/issues/14161

Signed-off-by: Zack Cerza <zack@redhat.com>

Use ansible_user instead of ansible_ssh_user

The option was renamed in 2.0

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #249 from ceph/wip-gw-networking

Network configuration for gateway role

Reviewed-by: Dan Mick <dmick@redhat.com>

gateway: Add logrotate functionality

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Add README documentation for networking, firewall, fail2ban

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Add fail2ban support

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Configure firewalld

Signed-off-by: David Galloway <dgallowa@redhat.com>

gateway: Enable server network config

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #252 from ceph/wip-cblr-wget

cobbler: testnode post-install rc.local fixes

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Zack Cerza <zack@redhat.com>

testnodes: Configure cpan and install Amazon::S3 on all OSes

Fixes: http://tracker.ceph.com/issues/15316
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #254 from ceph/wip-clone-once

Only clone the keys repo once per execution

Reviewed-by: Dan Mick <dmick@redhat.com>

Only clone the keys repo once per execution

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #253 from ceph/wip-fast-keys

users: Greatly speed up ssh pubkey deployment

Reviewed-by: Dan Mick <dmick@redhat.com>

Put the keys repo in ~/.cache/src/ by default

To avoid multiple invocations by different users on the same host
stepping on each other.

Signed-off-by: Zack Cerza <zack@redhat.com>

Remove unnecessary retry

Since each key isn't being fetched from a remote server any longer, we
can drop the retries.

Signed-off-by: Zack Cerza <zack@redhat.com>

Speed up key deployment by using the git repo

Instead of downloading each key over HTTPS from github.com, we can
simply clone the entire repo (with depth 1) and lookup each key using
the username.

On my laptop, execution time went from 2m49s to 29s.

Signed-off-by: Zack Cerza <zack@redhat.com>

users: Add defaults for keys_repo

Signed-off-by: Zack Cerza <zack@redhat.com>

users: split tasks/main.yml into separate files

Signed-off-by: Zack Cerza <zack@redhat.com>

cobbler: testnode post-install rc.local fixes

- Instead of sleeping for 30 secs and hoping network is up, wait until we can
reliably ping the Cobbler host.
- Give ansible more time to run before wget's built-in 15min timeout is reached.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #251 from ceph/wip-edeploy

users: Set UID_MIN to 1001

Reviewed-by: Dan Mick <dmick@redhat.com>

users: Set UID_MIN to 1001

When building images with edeploy, UID 1000 was being taken by a
"normal" user; we need to reserve this for the teuthology user.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #250 from ceph/wip-no-lsb

Don't rely on LSB when testing for distro version

Reviewed-by: Dan Mick <dmick@redhat.com>

Don't rely on LSB when testing for distro version

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #248 from ceph/wip-xenial

testnodes: Fix resolvconf issues on xenial

Improve network interface discovery

Signed-off-by: Zack Cerza <zack@redhat.com>

Always bounce the main interface on xenial

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #247 from ceph/wip-16119

pcp: Don't fail when setting permissions

pcp: Don't fail when setting permissions

http://tracker.ceph.com/issues/16119
Fixes: 16119
Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #246 from dmick/master

smart.sh: default rc to 0 (saw false UNKNOWNs with non-raid)

smart.sh: default rc to 0 (saw false UNKNOWNs with non-raid)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #245 from ceph/wip-newsmart

testnode: Replace old smart.pl with rewritten smart.sh script

Reviewed-by: Dan Mick <dmick@redhat.com>

testnode: Replace old smart.pl with rewritten smart.sh script

Fixes: http://tracker.ceph.com/issues/14682
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #244 from cooboos/centos

testnode: replace hardcoded 'gitbuilder.ceph.com' with {{ gitbuilder_host }}

testnode: replace hardcoded 'gitbuilder.ceph.com' with {{ gitbuilder_host }}

Signed-off-by: wangsongbo <wangsongbo@unitedstack.com>

Merge pull request #243 from ceph/wip-sam-ansible

testnode/tasks/apt/packages.yml: just force everything

Merge pull request #241 from ceph/wip-pcp-settings

pcp: update upstream packages

Reviewed-by: Dan Mick <dmick@redhat.com>

pcp: update upstream packages

PCP's upstream doesn't version its dependencies in all cases. This leads
to situations where, when setting upstream_repo to True, a system ends
up with some old and some new packages. This results in a potentially
broken installation.

This commit causes all installed pcp packages to be upgraded when
upstream_repo is True.

Signed-off-by: Zack Cerza <zack@redhat.com>

testnode/tasks/apt/packages.yml: just force everything

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #239 from ceph/wip-pcp-settings

pcp: retention settings for managers

Make pmlogmerge_reduce configurable

Signed-off-by: Zack Cerza <zack@redhat.com>

Make pmlogmerge_retain configurable

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #240 from ceph/wip-nrpe-load-limits

testnode: Update nagios CPU load thresholds to be less strict

testnode: Update nagios CPU load thresholds to be less strict

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #238 from ceph/wip-pcp-js

pcp: A dynamic dashboard with host- and time-selection

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #237 from ceph/wip-check4-selinux

testnode: Skip NRPE selinux setup if selinux disabled

index.js

testnode: Skip NRPE selinux setup if selinux disabled

Fixes: http://tracker.ceph.com/issues/15675
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #236 from ceph/wip-xenial-nicnames

cobbler: Prevent systemd from renaming NICs

Reviewed-by: Dan Mick <dmick@redhat.com>

cobbler: Prevent systemd from renaming NICs

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #235 from ceph/wip-no-getty

testnode: Remove manual configuration of getty

Reviewed-by: Dan Mick <dmick@redhat.com>

testnode: Remove manual configuration of getty

Manual configuration of the getty service is no longer required for
Trusty or later.  The system-specific "console=ttySX" parameter in
/proc/cmdline is enough to configure the SOL tty during Ubuntu
installation.

See http://tracker.ceph.com/issues/15269 for further explanation.

Signed-off-by: David Galloway <dgallowa@redhat.com>