testnode: Update pip tasks to support pip3

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Shuffle package lists around to support Ubuntu 20.04

It appears Focal doesn't come with python2 anything so the python2 packages are no longer "common" across all Ubuntu distros.

I moved them out of `common_packages` and added them to the appropriate package lists for each distro_majorversion.yml back to Xenial.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: SSH config for Ubuntu 20.04

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Add Ubuntu 20.04 support

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #562 from ceph/wip-nrpeopts

common: Also define new NRPE_OPTS var

common: Also define new NRPE_OPTS var

Not sure if this was a change in new NRPE or Ubuntu but let's just set both and cover our bases

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #561 from lxbsz/iozone

testnode: install iozone package from sepia lab repo

testnode: install iozone package from spia lab repo

The iozone package is remove from centos/rhel 8 dues to the license
issue, and this will install the package from the spia lab repo.

Fixes: https://tracker.ceph.com/issues/43600
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #559 from ceph/wip-additional-repos

testnode: Add support for additional yum repos defined in secrets

Merge pull request #558 from ceph/wip-if-inrange

dhcp-server: Only create host declaration if its IP is in subnet we'r…

Merge pull request #560 from ceph/wip-cobbler-mac

cobbler_systems: Only update systems if mac and ip are defined

cobbler_systems: Only update systems if mac and ip are defined

We must have some systems that snuck into cobbler without having a MAC or IP so when this second "update" task runs, it fails

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Add support for additional yum repos defined in secrets

Signed-off-by: David Galloway <dgallowa@redhat.com>

dhcp-server: Only create host declaration if its IP is in subnet we're writing

In the Octo lab, we have multiple subnets with the same domain and VLAN.  This is problematic because in the `dhcp_subnets` dict, we only anticipate there being one cidr.

This change will let you have multiple "subnets" in `dhcp_subnets` without fiddling with the ansible inventory ipvars or macvars.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #557 from ceph/wip-dnf-cache

prep-fog-capture: Disable dnf-makecache service on EL8

prep-fog-capture: Disable dnf-makecache service on EL8

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1814337
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #556 from ceph/wip-rhel82

cobbler: Add RHEL8.2

cobbler: Add RHEL8.2

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #554 from ceph/wip-forwards

nameserver: Support forwarders

nameserver: Support forwarders

When setting up this nameserver role in Octo, recursive lookups were failing.  I suspect maybe BIND is doing an `NS` lookup when it doesn't know about a domain it is asked about.  Red Hat blocks all external DNS queries so I've defined an internal DNS server for the Octo BIND server to forward to.  Now external lookups work.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #553 from ceph/wip-secrets

nameserver: Don't require secrets

nameserver: Don't require secrets

secrets/nameserver.yml is only required if we're using DDNS

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #552 from ceph/wip-ns-updates2

nameserver: Configure selinux only if enabled

nameserver: Configure selinux only if enabled

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #551 from ceph/wip-ns-updates

nameserver: Support python2 and python3 in named.conf template

nameserver: Support python2 and python3 in named.conf template

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #550 from ceph/wip-44501

testnode: Set ansible_python_interpreter for OpenSUSE

testnode: Set ansible_python_interpreter for OpenSUSE

Fixes: https://tracker.ceph.com/issues/44501
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #549 from ceph/wip-cleanup-dnf

prep-fog-capture: Clean yum cache

prep-fog-capture: Clean yum cache

Hopefully fixes: https://github.com/ceph/ceph-cm-ansible/pull/544#issuecomment-599076564

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #544 from ceph/wip-epel-nobaseurl

common: Don't use baseurl for epel at all

common: Don't use baseurl for epel at all

Despite having a mirrorlist present in the repo file, if contacting the baseurl fails, the yum transaction fails which causes our jobs to fail.  Let's try *just* using a mirrorlist.

See https://github.com/ceph/ceph-cm-ansible/pull/542#issuecomment-598924481

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #542 from ceph/wip-centos-mirrorlist

testnode: Use mirrorlists for CentOS 8 repos

testnode: Use mirrorlists for CentOS 8 repos

Fixes ``Failed to download metadata for repo 'extras'`` errors.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #541 from smithfarm/wip-leap-152-no-py3

testnode: use Python 3 in openSUSE Leap 15.2

testnode: use Python 3 in openSUSE Leap 15.2

We should not be using Python 2 at all...

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #540 from ceph/wip-cobbler-centos-aarch64

Adding CentOS-8.1.1911-aarch64 to cobbler

Adding CentOS-8.1.1911-aarch64 to cobbler

Signed-off-by: Adam Kraitman <akraitma@redhat.com>

Merge pull request #539 from ceph/wip-suse-152

Add OpenSUSE 15.2

Add OpenSUSE 15.2

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #537 from ceph/wip-fedora31-cobbler

Adding Fedora 31 support to the common role

Adding Fedora 31 support to the common role

testnodes: add vars for opensuse_leap_15.2

Merge pull request #536 from smithfarm/wip-44292

testnodes: add vars for opensuse_leap_15.2

testnodes: add vars for opensuse_leap_15.2

Fixes: https://tracker.ceph.com/issues/44292
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #535 from ceph/wip-f31-snippet2

cobbler: More kickstart adjustments for Fedora 31

cobbler: Install ethtool on all RPM-based installs

Signed-off-by: David Galloway <dgallowa@redhat.com>

cobbler: Forget about server groupname and just install python3

Let's just hope the basic packages we'd need to at least get ceph-cm-ansible running exist in the base Fedora 31 DVD image.

We can't figure out the @base -> @infrastructure-server-environment -> @???? group name progression/changes so forget it.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #533 from ceph/wip-f31-snippet

cobbler: Update snippet for clarity and Fedora 31

cobbler: Update snippet for clarity and Fedora 31

Changes:
  - @infrastructure-server group name change
  - perl and redhat-lsb-core aren't available in Fedora 31 apparently
  - Added some comments for clarity

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #532 from ceph/wip-fedora31-cobbler

Adding Fedora 31 to sepia cobbler

Adding Fedora 31 to sepia cobbler

Signed-off-by: Adam Kraitman <akraitma@redhat.com>

Merge pull request #530 from ceph/wip-44158

cobbler: Correct ifcfg syntax for OpenSUSE

cobbler: Correct ifcfg syntax for OpenSUSE

Fixes: https://tracker.ceph.com/issues/44158
Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #529 from ceph/wip-updategrub

cobbler_profile: Disable OS prober to prevent update-grub hang

cobbler_profile: Disable OS prober to prevent update-grub hang

The Ubuntu installer appears to at least probe non-root disks during the `update-grub` step.  This was hanging on LVs leftover from previous installations on the smithi NVMe drives.

Settings this kernel option fixes it.

See https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1854839/comments/2

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #528 from ceph/epel-mirrorlist

Use epel mirrors

Revert "common: Update yum metadata from upstream epel and exclude mirrors"

This reverts commit 93e0a4a8f5f34c248d3467f3d1c8300ced0f5148.

Revert "common: rm yum cache that doesn't get cleaned with `yum clean`"

This reverts commit 946bf1576b56c20e5fec42f15a70d921dd440d00.

common: Use global epel_repos var

I diverged a bit when CentOS/RHEL 8 came out and was using metalink when I meant to use mirrorlist.  We're seeing intermittent epel failures which should be resolved by using a mirrorlist instead of relying on dl.fedoraproject.org

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #527 from ceph/wip-no404

common: Add baseurl to epel repo files

common: Add baseurl to epel repo files

Honestly not even sure how this was working at all but in order for https://github.com/ceph/ceph-cm-ansible/pull/487 to work, the repo file needs to actually point to upstream epel to refresh the yum metadata.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #526 from ceph/wip-centos81

Add CentOS 8.1

Add CentOS 8.1

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #525 from ceph/wip-43448

testnode: Move dbench to epel_packages so it gets installed

Merge pull request #524 from ceph/wip-el8

testnode: el8: install gdb for test-crash.sh

testnode: el8: install gdb for test-crash.sh

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #523 from ceph/wip-el7

testnode: Fix typo so task only runs on EL8

testnode: Fix typo so task only runs on EL8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #522 from ceph/wip-bz1782899

testnode: Workaround for BZ 1782899

testnode: Workaround for BZ 1782899

This will stop all the RHEL8 failures.

Signed-off-by: David Galloway <dgallowa@redhat.com>

testnode: Move dbench to epel_packages so it gets installed

Octo lab doesn't use epel by default but by moving this package to `epel_packages`, it will get installed during this task: https://github.com/ceph/ceph-cm-ansible/blob/master/roles/testnode/tasks/yum/packages.yml#L58

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #521 from ceph/wip-exclusive

ansible_managed: Remove unmanaged keys

ansible_managed: Remove unmanaged keys

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #520 from ceph/wip-interpreter

cobbler: Don't set ansible_python_interpreter manually anymore

cobbler: Don't set ansible_python_interpreter manually anymore

Both cobbler hosts (cobbler.front.sepia.ceph.com and magna001.ceph.redhat.com) are now running versions of ansible that handle python interpreter discovery automatically.

Setting this manually was actually causing issues because some ceph-cm-ansible tasks get delegated to localhost (Like 'Clone the keys repo' in the Users role).  So when setting -e ansible_python_interpreter=/usr/bin/python3, ansible looks for that binary on the localhost.  magna001 doesn't have python3 so the playbook run would fail.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #519 from ceph/wip-el8-nfs

testnode: el8: nfs-utils

testnode: el8: nfs-utils

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #517 from ceph/wip-rhel8-kvm

testnode: el8: qemu-kvm-core + qemu-kvm-block-rbd

Merge pull request #518 from tchaikov/python3-nose-for-bionic

testnode/var/ubuntu_18: python3-nose

testnode/var/ubuntu_18: python3-nose

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

testnode: el8: qemu-kvm-core + qemu-kvm-block-rbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #516 from ceph/wip-el8

testnode: el7: install podman-docker

testnode: el7: install podman-docker

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #515 from ceph/wip-el8

testnode: el8: podman-docker

testnode: el8: podman-docker

teuthology's cephadmunit.py needs to send a kill signal to the process,
and doesn't (easily) know whether to use the docker or podman command.
This let's us just use 'docker kill ...' and have it work for both.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #514 from ceph/wip-el8

testnode: el8: fix typo

testnode: el8: fix typo

sigh

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #513 from ceph/wip-el8

testnode: el8: install valgrind

testnode: el8: install valgrind

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #512 from ceph/wip-el8

testnode: el8: dbench, autoconf

testnode: el8: dbench, autoconf

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #511 from ceph/wip-el8

testnode: el8: libacl-devel

testnode: el8: libacl-devel

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #510 from ceph/wip-el8

testnode: el8: drop python3-pytz dep

testnode: el8: drop python3-pytz dep

python3-tempora is now fixed to require this.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #509 from ceph/wip-el8

testnode: el8: qemu-image

testnode: el8: qemu-img

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #508 from ceph/wip-el8

testnode: el8: genisoimage

testnode: el8: genisoimage

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #507 from ceph/wip-el8

testnode: python3-nose