Merge pull request #346 from ceph/wip-set-hostname

cobbler: Add logic to rc.local for FOG-provisioned machines

Merge pull request #353 from ceph/wip-delete-hostkeys

tools: Delete ssh keys so FOG-imaged machines regenerate on firstboot

tools: Delete ssh keys so FOG-imaged machines regenerate on firstboot

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Regenerate SSH host keys on Ubuntu firstboot if necessary

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #352 from ceph/wip-prep-fog

Tools for FOG image prep

tools: Add script to modify next-server in DHCP

Signed-off-by: David Galloway <dgallowa@redhat.com>

tools: One-off playbook to prep a cobbler-imaged machine w/ FOG

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Configure testnode NIC on firstboot after FOG image

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Set hostname using DHCP every boot on testnodes

This is for use with FOG.  When we create an OS image, the image
inherits the testnode's hostname that was used to create the image.

This rc.local snippet will make sure the testnode's hostname is set to
whatever the IP's PTR record is on every boot.

`rclocal_nameserver` should be set in the inventory groupvars.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #350 from ceph/wip-front-query

vmhost: Allow task to fail but ignore errors

vmhost: Allow task to fail but ignore errors

With a recent update to ansible, the changed task would never return a
'failed' result with `failed_when` set.  `ignore_errors` is what we want
so the task fails but the playbook proceeds.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #349 from ceph/wip-xfs-fix

Omit XFS options on older OSes

Reviewed-by: Dan Mick <dmick@redhat.com>

Omit XFS options on older OSes

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #348 from ceph/wip-var-lib-nofail

testnode: Avoid an unmountable /var/lib/ceph

Reviewed-by: Dan Mick <dmick@redhat.com>

Don't write a version 5 XFS superblock

Signed-off-by: Zack Cerza <zack@redhat.com>

Boot even if /var/lib partition won't mount

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #347 from ceph/wip-add-apt-repo

testnode: Install software-properties-common

testnode: Install software-properties-common

... on Ubuntu, for add-apt-repository

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #340 from ceph/wip-find-fail

testnode: Avoid job failure due to fs loop

testnode: Avoid job failure due to fs loop

Example:

failed: [smithi205.front.sepia.ceph.com] (item=/var/run/) => {"changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004976", "end": "2017-09-29 18:42:23.050412", "failed": true, "failed_when_result": true, "item": "/var/run/", "rc": 1, "start": "2017-09-29 18:42:23.045436", "stderr": "find: File system loop detected; '/var/run/rpc_pipefs/gssd' is part of the same file system loop as '/var/run/rpc_pipefs'.", "stderr_lines": ["find: File system loop detected; '/var/run/rpc_pipefs/gssd' is part of the same file system loop as '/var/run/rpc_pipefs'."], "stdout": "", "stdout_lines": []}

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #345 from ceph/wip-21989

testnode: Be super aggressive with wiping LVM data

testnode: Be super aggressive with wiping LVM data

Fixes: http://tracker.ceph.com/issues/21989
Fixes: http://tracker.ceph.com/issues/21935 (hopefully)
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #344 from ceph/wip-named-ddos

nameserver: Double max amount of concurrent connections

nameserver: Double max amount of concurrent connections

I observed an unintentional DoS on ns1.front last night right as most of
the nightly scheduled jobs started up.  Lots of "nf_conntrack: table
full, dropping packet" messages in the syslog.

Doubling it should be safe.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #343 from ceph/wip-shuffle-tasks

testnode: Shuffle tasks around to make sure packages install first

testnode: Shuffle tasks around to make sure packages install first

I moved lvm.conf in https://github.com/ceph/ceph-cm-ansible/pull/342
because I wanted all the disk management tasks clustered together.  I
failed to take into account the fact that the lvm2 package might not be
installed yet (like on OVH nodes).

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #342 from ceph/wip-var-lib-ceph

testnode: Mount /var/lib/ceph to NVMe partition on smithi

testnode: Mount /var/lib/ceph to NVMe partition on smithi

Fixes: http://tracker.ceph.com/issues/20910
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #341 from ceph/wip-dmsetup

testnode: Update dependencies for zap tasks

testnode: Update dependencies for zap tasks

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #339 from ceph/wip-lvm

Support LVM on testnodes

testnode: Docs for lvm tasks

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Support logical volume management

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #338 from ceph/wip-smithi-part

Re-do how we partition drives

testnode: Add some missing tags

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Add README for partition_dict

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Erase /scratch_devs each run in case the devices changed

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Use new drives_to_partition dict to partition drives

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Create gpt instead of msdos table.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: We repartition on every job after zapping

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #337 from ceph/wip-apt-pinning

testnode: Update apt pinning URL

testnode: Update apt pinning URL

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #336 from ceph/wip-ntp-install

cobbler: Install ntp package on RHEL/CentOS during kickstart

cobbler: Install ntp package on RHEL/CentOS during kickstart

Apparently doesn't come stock with CentOS 7.4 which was causing the
Cobbler ansible-playbook post-install run to fail.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #335 from ceph/wip-pub-ssl

public_facing: Finally get letsencrypt automated renewal  working

public_facing: Finally get letsencrypt automated renewal  working

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #334 from ceph/wip-selinux-modules

Make selinux policy module checking bulletproof

common: Check for *any* empty selinux policy module files

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: reinstall pcpupstream selinux policy module if corrupt

Makes sure policy module is latest and not empty

Fixes: http://tracker.ceph.com/issues/21403
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #333 from ceph/wip-centos74

cobbler: Add CentOS 7.4

cobbler: Add CentOS 7.4

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #326 from ceph/wip-20623

testnodes: Clean up leftover Ceph artifacts from previous jobs

Merge pull request #331 from ceph/wip-cobbler-script

tools: Script to simplify Cobbler user creation

testnodes: Clean up leftover Ceph artifacts from previous jobs

Fixes: http://tracker.ceph.com/issues/20623
Signed-off-by: David Galloway <dgallowa@redhat.com>

tools: Script to simplify Cobbler user creation

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #330 from ceph/wip-sudo-nvme

common: Grant nrpe user access to `sudo nvme`

common: Grant nrpe user access to `sudo nvme`

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #329 from ceph/wip-krbd-lvm-conf

testnode: edit lvm.conf on both ubuntu and centos

testnode: edit lvm.conf on both ubuntu and centos

This modification is needed for krbd xfstests (e.g. generic/081), so
factor out lvm.conf snippet from setup-ubuntu.yml into tasks/.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

testnode: correct krbd max_partitions value in lvm.conf

1024 is clearly a made up number.  16 reflects the current reality,
more or less:

  "Under single_major scheme introduced in kernel 3.14, rbd devices
  support up to 16 partitions.  In pre-single_major mode, rbd driver
  takes up the whole major."

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #327 from ceph/wip-20456

teuthology: minor README update

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #318 from ceph/wip-smart-nvme-binary

Support NVMe drive monitoring via Nagios

Reviewed-by: Dan Mick <dmick@redhat.com>

teuthology: minor README update

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #325 from ceph/wip-nrpe-pidfile

common: Change path to nrpe pid_file on RedHat/CentOS

common: Change path to nrpe pid_file on RedHat/CentOS

I can't find the exact commit in nrpe that caused this to change but the
path has changed and the nrpe service fails to start on CentOS/RHEL now.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #324 from ceph/wip-zap-fix

testnode: Try wipefs without --force if flag is missing

testnode: Try wipefs without --force if flag is missing

Fixes bug introduced in https://github.com/ceph/ceph-cm-ansible/pull/323

The version of wipefs shipped with Trusty lacks the --force flag

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #323 from ceph/wip-20533

Updates to testnode disk zapping

testnode: Retry zapping disk if sgdisk fails the first time

I've seen `sgdisk -Z` fail the first time on RMA replacement drives from HGST but succeed on re-running it.

@alfredodeza recommended we add a retry.  Can't hurt.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Shred any leftover ceph osd metadata on non-root partitions

Fixes: http://tracker.ceph.com/issues/20533
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #322 from ceph/wip-www

public_facing: Create separate task yml for letsencrypt/nginx/ipv6 hosts

public_facing: Create separate task yml for letsencrypt/nginx/ipv6 hosts

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #321 from ceph/wip-download

public_facing: Changes to download.ceph.com crontab

public_facing: Changes to download.ceph.com crontab

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #320 from ceph/wip-rhel74

cobbler: Add RHEL 7.4

Merge pull request #319 from ceph/wip-18126

Repo changes for Ubuntu testnodes

testnode: Only install custom fastcgi repo on Trusty testnodes

Now that we're running apt/repos.yml tasks on Xenial, we can move
the fastcgi repo to ubuntu_14.yml since it's exclusive to that distro
version.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Permanently disable/remove custom blkin repo

See https://github.com/ceph/ceph-cm-ansible/pull/156

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Perform repo tasks on all Ubuntu testnodes

Just realized apt/repos.yml hasn't been getting run on Xenial testnodes.
There are some important tasks in there that should be run (e.g., Removing custom repos from
/etc/apt/sources.list.d).

We aren't using apt-mirror for Xenial so I've limited writing the master
sources.list to Trusty and earlier.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Add custom repo to Xenial to provide newer version of valgrind

Fixes: http://tracker.ceph.com/issues/18126#note-11
Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Add RHEL 7.4

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #317 from ceph/wip-13522

Update libgoogle-perftools4 on Xenial

Update libgoogle-perftools4 on Xenial

Fixes: http://tracker.ceph.com/issues/13522#note-51
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #316 from ceph/wip-retry-unregister

common: Retry unregister beta distro from CDN

common: Retry unregister beta distro from CDN

Seeing intermittent `subscription-manager unregister` failures in Octo.
Since the register task has retries, I see no reason unregistering
can't.

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Add support for NVMe drives in smart.sh

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Install nvme-cli for use with smart.sh

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Don't return WARN if previous disk is CRIT in smart.sh

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #314 from ceph/wip-www-cron

public_facing: Add cronjobs for www.ceph.com

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #315 from ceph/wip-new-ansible

common: Workaround for ansible bug conditional evaluation

Reviewed-by: Dan Mick <dmick@redhat.com>

common: Workaround for new ansible conditional evaluation bug

Prior to ansible v2.2.2.0, if the first conditional was false, the second
conditional wasn't checked.

This change makes sure nrpe-selinux.yml is only included on RHEL/CentOS
which are the only OSes that will have stdout for the selinux_status
var.

Signed-off-by: David Galloway <dgallowa@redhat.com>

public_facing: Add cronjobs for www.ceph.com

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #313 from ceph/wip-fix-312

common: Fix a bug in PR #312

Reviewed-by: Dan Mick <dmick@redhat.com>

common: Fix a bug in PR #312

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #312 from ceph/wip-15439

common: Clean apt cache before updating it

Reviewed-by: Dan Mick <dmick@redhat.com>

common: Clean apt cache before updating it

This is the first apt transaction that gets run in the common role.

Fixes: http://tracker.ceph.com/issues/15439
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #307 from ceph/wip-public-facing

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #311 from ceph/wip-vps-replace

tools: Replace mira091 w mira005 and add missing hv03

Reviewed-by: Dan Mick <dmick@redhat.com>

tools: Replace mira091 w mira005 and add missing hv03

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #310 from ceph/wip-19126

common: Reinstall selinux-policy-targeted if needed

Reviewed-by: Dan Mick <dmick@redhat.com>