testnode: Stop zapping loop devices

Because it's a waste of time

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #359 from ceph/wip-kopts-post

cobbler_systems: Allow setting post install kernel options

cobbler_systems: Allow setting post install kernel options

If defined in group_vars, kernel_options_post will be used.  Otherwise,
it defaults to the kernel_options_post in defaults/main.yml.

This fixes serial output during firstboot in Xenial or later (instead of
the repeated "Ubuntu 16.04 [-1;-1f" garbage.

Running `cobbler system edit` with --kopts-post="" is fine.  It just
leaves that field blank if kernel_options_post is empty.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #357 from ceph/wip-update-xenial

cobbler_profile: Replace 16.04 with 16.04.3 iso

cobbler_profile: Replace 16.04 with 16.04.3 iso

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #356 from ceph/wip-fog-csv

tools: Add ansible role to generate FOG host csv file

tools: Add ansible role to generate FOG host csv file

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #355 from ceph/wip-ethtool

cobbler: Install ethtool on reimage

cobbler: Install ethtool on reimage

ethtool is used in the new rc.local hackery

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #354 from ceph/wip-ubuntu-pkgs

Adjust Cobbler's Ubuntu preseed to work with Artful Aardvark and beyond

cobbler: Install grub to /dev/sda instead of whatever default is

Apparently on some machines (namely bruuni in Octo), the default disk
returns /dev/nvme0n1 which is not where the OS is installed.

Since we already define which disk the OS should be installed on earlier
in the preseed, it only makes sense to install grub on the same disk.

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Set package list for xenial and later instead of just xenial

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #346 from ceph/wip-set-hostname

cobbler: Add logic to rc.local for FOG-provisioned machines

Merge pull request #353 from ceph/wip-delete-hostkeys

tools: Delete ssh keys so FOG-imaged machines regenerate on firstboot

tools: Delete ssh keys so FOG-imaged machines regenerate on firstboot

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Regenerate SSH host keys on Ubuntu firstboot if necessary

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #352 from ceph/wip-prep-fog

Tools for FOG image prep

tools: Add script to modify next-server in DHCP

Signed-off-by: David Galloway <dgallowa@redhat.com>

tools: One-off playbook to prep a cobbler-imaged machine w/ FOG

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Configure testnode NIC on firstboot after FOG image

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Set hostname using DHCP every boot on testnodes

This is for use with FOG.  When we create an OS image, the image
inherits the testnode's hostname that was used to create the image.

This rc.local snippet will make sure the testnode's hostname is set to
whatever the IP's PTR record is on every boot.

`rclocal_nameserver` should be set in the inventory groupvars.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #350 from ceph/wip-front-query

vmhost: Allow task to fail but ignore errors

vmhost: Allow task to fail but ignore errors

With a recent update to ansible, the changed task would never return a
'failed' result with `failed_when` set.  `ignore_errors` is what we want
so the task fails but the playbook proceeds.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #349 from ceph/wip-xfs-fix

Omit XFS options on older OSes

Reviewed-by: Dan Mick <dmick@redhat.com>

Omit XFS options on older OSes

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #348 from ceph/wip-var-lib-nofail

testnode: Avoid an unmountable /var/lib/ceph

Reviewed-by: Dan Mick <dmick@redhat.com>

Don't write a version 5 XFS superblock

Signed-off-by: Zack Cerza <zack@redhat.com>

Boot even if /var/lib partition won't mount

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #347 from ceph/wip-add-apt-repo

testnode: Install software-properties-common

testnode: Install software-properties-common

... on Ubuntu, for add-apt-repository

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #340 from ceph/wip-find-fail

testnode: Avoid job failure due to fs loop

testnode: Avoid job failure due to fs loop

Example:

failed: [smithi205.front.sepia.ceph.com] (item=/var/run/) => {"changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004976", "end": "2017-09-29 18:42:23.050412", "failed": true, "failed_when_result": true, "item": "/var/run/", "rc": 1, "start": "2017-09-29 18:42:23.045436", "stderr": "find: File system loop detected; '/var/run/rpc_pipefs/gssd' is part of the same file system loop as '/var/run/rpc_pipefs'.", "stderr_lines": ["find: File system loop detected; '/var/run/rpc_pipefs/gssd' is part of the same file system loop as '/var/run/rpc_pipefs'."], "stdout": "", "stdout_lines": []}

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #345 from ceph/wip-21989

testnode: Be super aggressive with wiping LVM data

testnode: Be super aggressive with wiping LVM data

Fixes: http://tracker.ceph.com/issues/21989
Fixes: http://tracker.ceph.com/issues/21935 (hopefully)
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #344 from ceph/wip-named-ddos

nameserver: Double max amount of concurrent connections

nameserver: Double max amount of concurrent connections

I observed an unintentional DoS on ns1.front last night right as most of
the nightly scheduled jobs started up.  Lots of "nf_conntrack: table
full, dropping packet" messages in the syslog.

Doubling it should be safe.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #343 from ceph/wip-shuffle-tasks

testnode: Shuffle tasks around to make sure packages install first

testnode: Shuffle tasks around to make sure packages install first

I moved lvm.conf in https://github.com/ceph/ceph-cm-ansible/pull/342
because I wanted all the disk management tasks clustered together.  I
failed to take into account the fact that the lvm2 package might not be
installed yet (like on OVH nodes).

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #342 from ceph/wip-var-lib-ceph

testnode: Mount /var/lib/ceph to NVMe partition on smithi

testnode: Mount /var/lib/ceph to NVMe partition on smithi

Fixes: http://tracker.ceph.com/issues/20910
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #341 from ceph/wip-dmsetup

testnode: Update dependencies for zap tasks

testnode: Update dependencies for zap tasks

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #339 from ceph/wip-lvm

Support LVM on testnodes

testnode: Docs for lvm tasks

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Support logical volume management

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #338 from ceph/wip-smithi-part

Re-do how we partition drives

testnode: Add some missing tags

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Add README for partition_dict

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Erase /scratch_devs each run in case the devices changed

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Use new drives_to_partition dict to partition drives

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Create gpt instead of msdos table.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: We repartition on every job after zapping

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #337 from ceph/wip-apt-pinning

testnode: Update apt pinning URL

testnode: Update apt pinning URL

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #336 from ceph/wip-ntp-install

cobbler: Install ntp package on RHEL/CentOS during kickstart

cobbler: Install ntp package on RHEL/CentOS during kickstart

Apparently doesn't come stock with CentOS 7.4 which was causing the
Cobbler ansible-playbook post-install run to fail.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #335 from ceph/wip-pub-ssl

public_facing: Finally get letsencrypt automated renewal  working

public_facing: Finally get letsencrypt automated renewal  working

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #334 from ceph/wip-selinux-modules

Make selinux policy module checking bulletproof

common: Check for *any* empty selinux policy module files

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: reinstall pcpupstream selinux policy module if corrupt

Makes sure policy module is latest and not empty

Fixes: http://tracker.ceph.com/issues/21403
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #333 from ceph/wip-centos74

cobbler: Add CentOS 7.4

cobbler: Add CentOS 7.4

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #326 from ceph/wip-20623

testnodes: Clean up leftover Ceph artifacts from previous jobs

Merge pull request #331 from ceph/wip-cobbler-script

tools: Script to simplify Cobbler user creation

testnodes: Clean up leftover Ceph artifacts from previous jobs

Fixes: http://tracker.ceph.com/issues/20623
Signed-off-by: David Galloway <dgallowa@redhat.com>

tools: Script to simplify Cobbler user creation

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #330 from ceph/wip-sudo-nvme

common: Grant nrpe user access to `sudo nvme`

common: Grant nrpe user access to `sudo nvme`

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #329 from ceph/wip-krbd-lvm-conf

testnode: edit lvm.conf on both ubuntu and centos

testnode: edit lvm.conf on both ubuntu and centos

This modification is needed for krbd xfstests (e.g. generic/081), so
factor out lvm.conf snippet from setup-ubuntu.yml into tasks/.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

testnode: correct krbd max_partitions value in lvm.conf

1024 is clearly a made up number.  16 reflects the current reality,
more or less:

  "Under single_major scheme introduced in kernel 3.14, rbd devices
  support up to 16 partitions.  In pre-single_major mode, rbd driver
  takes up the whole major."

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #327 from ceph/wip-20456

teuthology: minor README update

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #318 from ceph/wip-smart-nvme-binary

Support NVMe drive monitoring via Nagios

Reviewed-by: Dan Mick <dmick@redhat.com>

teuthology: minor README update

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #325 from ceph/wip-nrpe-pidfile

common: Change path to nrpe pid_file on RedHat/CentOS

common: Change path to nrpe pid_file on RedHat/CentOS

I can't find the exact commit in nrpe that caused this to change but the
path has changed and the nrpe service fails to start on CentOS/RHEL now.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #324 from ceph/wip-zap-fix

testnode: Try wipefs without --force if flag is missing

testnode: Try wipefs without --force if flag is missing

Fixes bug introduced in https://github.com/ceph/ceph-cm-ansible/pull/323

The version of wipefs shipped with Trusty lacks the --force flag

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #323 from ceph/wip-20533

Updates to testnode disk zapping

testnode: Retry zapping disk if sgdisk fails the first time

I've seen `sgdisk -Z` fail the first time on RMA replacement drives from HGST but succeed on re-running it.

@alfredodeza recommended we add a retry.  Can't hurt.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Shred any leftover ceph osd metadata on non-root partitions

Fixes: http://tracker.ceph.com/issues/20533
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #322 from ceph/wip-www

public_facing: Create separate task yml for letsencrypt/nginx/ipv6 hosts

public_facing: Create separate task yml for letsencrypt/nginx/ipv6 hosts

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #321 from ceph/wip-download

public_facing: Changes to download.ceph.com crontab

public_facing: Changes to download.ceph.com crontab

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #320 from ceph/wip-rhel74

cobbler: Add RHEL 7.4

Merge pull request #319 from ceph/wip-18126

Repo changes for Ubuntu testnodes

testnode: Only install custom fastcgi repo on Trusty testnodes

Now that we're running apt/repos.yml tasks on Xenial, we can move
the fastcgi repo to ubuntu_14.yml since it's exclusive to that distro
version.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Permanently disable/remove custom blkin repo

See https://github.com/ceph/ceph-cm-ansible/pull/156

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Perform repo tasks on all Ubuntu testnodes

Just realized apt/repos.yml hasn't been getting run on Xenial testnodes.
There are some important tasks in there that should be run (e.g., Removing custom repos from
/etc/apt/sources.list.d).

We aren't using apt-mirror for Xenial so I've limited writing the master
sources.list to Trusty and earlier.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Add custom repo to Xenial to provide newer version of valgrind

Fixes: http://tracker.ceph.com/issues/18126#note-11
Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Add RHEL 7.4

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #317 from ceph/wip-13522

Update libgoogle-perftools4 on Xenial

Update libgoogle-perftools4 on Xenial

Fixes: http://tracker.ceph.com/issues/13522#note-51
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #316 from ceph/wip-retry-unregister

common: Retry unregister beta distro from CDN

common: Retry unregister beta distro from CDN

Seeing intermittent `subscription-manager unregister` failures in Octo.
Since the register task has retries, I see no reason unregistering
can't.

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Add support for NVMe drives in smart.sh

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Install nvme-cli for use with smart.sh

Signed-off-by: David Galloway <dgallowa@redhat.com>

common: Don't return WARN if previous disk is CRIT in smart.sh

Signed-off-by: David Galloway <dgallowa@redhat.com>