ceph-volume-nightly: adds mimic and luminous to batch mixed type tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1499 from tchaikov/wip-crimson-flavor

scripts/build_utils.sh: install deps for crimson flavor

Reviewed-by: David Galloway <dgallowa@redhat.com>

scripts/build_utils.sh: install deps for crimson flavor

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1498 from ceph/wip-noarm64

Disabling ceph-pull-requests-arm64 until we have packages and reliable hw

Disabling ceph-pull-requests-arm64 unless 'arm64' label is used

CentOS8 is missing dependencies.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1496 from ceph/ceph-medic-pr

ceph-medic-pull-requests: Skip py37 env

ceph-medic-pull-requests: Skip py37 env

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-medic-pull-requests: Replace alfredo

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1494 from ceph/guits-add_job

ceph-ansible: add external ceph clients scenario

ceph-ansible: add external ceph clients scenario

Add a new job in order to test the external ceph clients scenario
testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1493 from ceph/wip-cb-setup

ceph-build: Update setup_rpm for CentOS8 support

ceph-build: Update setup_rpm for CentOS8 support

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1492 from ceph/wip-lol-octopus

ceph-build: Add octopus

ceph-build: Add octopus

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1491 from tchaikov/wip-arm64-on-bionic-only

ceph-pull-requests-arm64: only build on bionic

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-pull-requests-arm64: only build on bionic

ktdreyer/ceph-el8 is amd64 only, so to avoid failures like:

```
No matching package to install: 'python3-cherrypy'
No matching package to install: 'python3-pecan'
No matching package to install: 'python3-routes'
```

we need to disable the "make check" test on the combination of
"arm64+centos8" until these packages are packaged for arm64 or
provided by centos8.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1490 from ceph/guits-add_all_in_one

ceph-ansible-prs: add a new scenario

ceph-ansible-prs: add a new scenario

add an 'all in one' scenario in order to catch more collocation related
issues.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1488 from tchaikov/wip-no-install-deps

ceph-*-setup: do not run install-deps.sh

Merge pull request #1489 from ceph/jan--f-ansible4-nautilus

use ansible 4.0 for nautlius nightlies

use ansible 4.0 for nautlius nightlies

ceph-ansible has dropped centos7 support in master, nautilius however does support it still.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-*-setup: do not run install-deps.sh

this is a follow-up change of #1487. since we don't use autotools for
preparing dist tarball, there is no need to run install-deps.sh for
installing the autotools.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1486 from ceph/ceph_collect_logs_timeout

collect-logs: set connect timeout value

Merge pull request #1487 from tchaikov/wip-no-more-autotools

ceph-*: drop autotools support

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-*: drop autotools support

we don't build ceph with autotools anymore. we switched to cmake since
luminous. see https://github.com/ceph/ceph/pull/9995.

so there is no need to support autotools anymore. also, this avoids
running install-deps.sh simply for running "make dist". as we use
`make-dist` script after switching over to cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>

collect-logs: set connect timeout value

If a job fails and the ceph cluster doesn't respond then the ceph status
task in the collect-logs.yml playbook will wait around 50 mins before
timed out.

  "cmd": [
      "ceph",
      "--cluster",
      "ceph",
      "-s",
      "-f",
      "json"
  ],
  "delta": "0:50:00.197544"
  "end": "2020-01-16 19:45:06.932819"
  "rc": 1,
  "start": "2020-01-16 18:55:06.735275"

  STDERR: [errno 110] RADOS timed out (error connecting to the cluster)

Instead we can use the --connect-timeout paramater to fail after 10s.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1485 from tchaikov/wip-arm64-on-el8

ceph-pull-requests-arm64: only build on centos8 and bionic

ceph-pull-requests-arm64: only build on centos8 and bionic

since we've labeled the slaves correctly. and centos8 is able to build
ceph on arm64. the only pending issue is
https://bugzilla.redhat.com/show_bug.cgi?id=1673990 . but we can fix it
manually.

since there is no trusty or xenial arm64 builders, we can just use
'arm64 && !centos7'

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1484 from ceph/wip-no-double-setup

ceph*build: Don't run pbuilder --clean a second time

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1483 from ceph/wip-no-tar

ceph-build: Apply e8bdd639

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph*build: Don't run pbuilder --clean a second time

It already gets run in build/setup_pbuilder

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-build: Apply e8bdd639

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1481 from ceph/wip-no-hostname

ansible: Set hostname on OpenSUSE

Merge pull request #1482 from tchaikov/wip-cd

ceph-build: apply 604882e9 to ceph-build

ceph-build: apply 604882e9 to ceph-build

to address following failure

+ cd ceph-14.2.6 ceph-14.2.6.tar.bz2 ceph-14.2.6.tar.gz
/tmp/jenkins10747333749307639424.sh: line 1058: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>

ansible: Set hostname on OpenSUSE

Now that we're using system-provided ansible packages (https://github.com/ceph/mita/pull/135), we're at the mercy of the version provided.

OpenSUSE Leap 15.1 gives us ansible 2.8.1 which has this bug: https://github.com/ansible/ansible/issues/42726

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1480 from ceph/guits-add_docker2podman

ceph-ansible: use dedicated tox for docker2podman job

ceph-ansible: use dedicated tox for docker2podman job

This commit makes tox using a dedicated tox ini file for the
docker_to_podman job.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1477 from ceph/wip-suse-sepia

sepia-fog-images: Support openSUSE

Merge pull request #1479 from ceph/guits-add_docker2podman

ceph-ansible-prs: add docker2podman scenario

ceph-ansible-prs: add docker2podman scenario

This commit adds a new 'docker2podman' job for the ceph-ansible CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1478 from ceph/wip-kernel-centos8

kernel: add centos8 builds

kernel: drop invalid axis

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: drop distros we don't build for

We don't use and don't have build agents for any of these.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: add centos8 builds

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

sepia-fog-images: Support openSUSE

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1453 from callithea/wip-dashboard-prs-install-deps

ceph-dashboard-*,ceph-api-nightly: Remove conditional-step; Replaced dir and script name

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-dashboard-*, ceph-api-nightly: Remove conditional-step; Replaced dir name; split-up
scripts

Split up the script which installs dependencies, since e2e and backend API tests require
different packages. Adapt all jobs to use the dedicated scripts.
Also remove the conditional step to not only execute the script that installs dependencies
for e2e test environments

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1474 from ceph/wip-pythonvirt

ansible: Install distro-appropriate virtualenv package version

Merge pull request #1475 from tchaikov/wip-dpkg-source-x

ceph-dev*-build: do not unpack source tar ball

ceph-dev*-build: do not unpack source tar ball

the uncompressed tarball is not used, moreover, the build fails on
bionic when `dpkg-source -x` the source package. because we've been
relying on a bug on xenial. see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=826334;msg=31

and on bionic, the bug was fixed. so `dpkg-source -x` started to
fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ansible: Install distro-appropriate virtualenv package version

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1473 from ceph/wip-noxenial

ceph-dev-new: Don't build master/wip on xenial

ceph-dev-new: Don't build master/wip on xenial

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1471 from ceph/wip-ugh-echo

ceph-dev-new-trigger: "fix" conditional step for all other branches

ceph-dev-new-trigger: "fix" conditional step for all other branches

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1470 from dmick/master

ceph-dev-new-build: allow containers on centos 7 or 8

ceph-dev-new-build: allow containers on centos 7 or 8

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1459 from ceph/wip-jjbyamlwtf

Update build matrices

Merge pull request #1469 from ceph/wip-cpr

ceph-pull-requests: Do NOT run on pbuilder hosts

ceph-pull-requests: Do NOT run on pbuilder hosts

pbuilder hosts are running Xenial.  We still want to be able to build Bionic packages on Xenial but we do not want make check to run on *all* slaves with the "bionic" label.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1468 from ceph/wip-bionic-slaves

ansible: Support Ubuntu Bionic slaves

ceph-pull-requests: Only run make check on Bionic

1) Bionic should have python2 and python3
2) All builds should run on Bionic but no other distro

Signed-off-by: David Galloway <dgallowa@redhat.com>

ansible: Support Ubuntu Bionic slaves

https://askubuntu.com/a/381527/906620

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1467 from ceph/wip-tcmu-runner-el8

tcmu-runner: fixed creation of chroot cfg file

tcmu-runner: fixed creation of chroot cfg file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1466 from ceph/wip-tcmu-runner-el8

tcmu-runner: second attempt at getting mock to use shaman repo

tcmu-runner: second attempt at getting mock to use shaman repo

It was falling back to the BaseOS repo which includes an older
version of librados/librbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1465 from ceph/wip-tcmu-runner-el8

tcmu-runner: fix environment variable typo

tcmu-runner: fix environment variable typo

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1464 from ceph/wip-tcmu-runner-el8

tcmu-runner: copy shaman.repo into mock build chroot

tcmu-runner: copy shaman.repo into mock build chroot

The el8 mock chroot environment templates now use special include
directives which makes it difficult to inject the shaman repos into
them before initializing the chroot environment. Therefore, just
directly copy the repo file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1463 from ceph/wip-c8-master

ceph-dev-build: setup_rpm Support for CentOS8

ceph-dev-build: setup_rpm Support for CentOS8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1462 from ceph/wip-cdlol

ceph-dev-build: Don't try to cd into tarballs

ceph-dev-build: Don't try to cd into tarballs

Fixes:

```
+ cd dist
+ tar xzf ceph_15.0.0-8623-g8a4458f.orig.tar.gz
+ cd ceph-15.0.0-8623-g8a4458f ceph-15.0.0-8623-g8a4458f.tar.bz2 ceph-15.0.0-8623-g8a4458f.tar.gz
/tmp/jenkins8482303231462049247.sh: line 1064: cd: too many arguments
```

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-dev*trigger: Set correct distro/flavor combos based on release

Based on https://pad.ceph.com/p/py3

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1461 from ceph/wip-tcmu-runner-el8

tcmu-runner: add centos8 to the available distros

tcmu-runner: add centos8 to the available distros

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1460 from ceph/wip-master-c8

ceph-dev-build: Build master on centos8

ceph-dev-build: Build ceph.git master on centos8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1458 from ceph/wip-ceph-iscsi-el8

ceph-iscsi: switch to huge centos builders

ceph-iscsi: switch to huge centos builders

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1457 from ceph/wip-ceph-iscsi-el8

ceph-iscsi: add centos8 distro for dev builds

ceph-iscsi: add centos8 distro for dev builds

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1456 from ceph/wip-equals

ansible: Adjust `when` statements to put EL8 repos on EL8 only

ansible: Adjust `when` statements to put EL8 repos on EL8 only

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1455 from ceph/wip-notc-8

ceph-dev{-new}: Build notcmalloc dev packages for centos8

ceph-dev{-new}: Build notcmalloc dev packages for centos8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1454 from ceph/wip-c8-slaves

ansible: Install lab-extras and copr repos on EL8

ansible: Install lab-extras and copr repos on EL8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1450 from ceph/createrepo-flags

scripts: use --no-database flag for createrepo

Merge pull request #1448 from callithea/wip-ceph-dashboard-pull-requests-dependencies

ceph-dashboard-pull-requests: Update dependencies

Reviewed-by: Kefu Chai kchai@redhat.com

Merge pull request #1452 from ceph/centos8-dev-builds

ceph-dev-new-trigger: build centos8 from ceph-ci

ceph-dev-new-trigger: build centos8 from ceph-ci

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1451 from toabctl/ceph-dev-build-opensuse-15.2

ceph-dev-build: Use correct OBS repo octopus on openSUSE Leap

ceph-dev-build: Use correct OBS repo octopus on openSUSE Leap

We want to build against openSUSE Leap 15.2 for octopus. So use the
correct repo.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

scripts: use --no-database flag for createrepo

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1449 from toabctl/chacra-pull-requests-ansible-fix

chacra-pull-requests: Replace ansible sudo with become

chacra-pull-requests: Replace ansible sudo with become

"become" is the correct ansible keyword for priv escalation in newer
versions. So use that.
This will fix at least the ansible-playbook setup.yml call which
previously failed.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>