ceph-volume: use cento8 instead of centos7 for PR jobs

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1510 from rishabh-d-dave/cv-add-scenarios-to-nightly

ceph-volume: add missing scenarios to nightlies

Merge pull request #1505 from rishabh-d-dave/cv-update-batch-functest-scenario-names

ceph-volume: add centos 8 based jobs for master and octopus

ceph-volume: add missing scenarios

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: separate xenial jobs from centos7 jobs

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: add nautilus to jobs for pre-octopus release

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: add centos 8 based jobs for master and octopus

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: add missing scenarios to nightlies

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1508 from ceph/wip-prs-noreboot

ceph-pull-requests: Behave like arm64 job

ceph-pull-requests*: Don't risk exposing jenkins credentials

The `ps` command doesn't even get run currently because the job stops immediately after make check exits but let's not risk it.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-pull-requests: Behave like arm64 job

Debugging unit test failures was difficult on x86 jobs because the slave got rebooted right away.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1509 from toabctl/octopus-on-opensuse-leap-15.2

osc: Build octopus on openSUSE Leap 15.2

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>

osc: Build octopus on openSUSE Leap 15.2

We want to test with the upcoming openSUSE release (15.2) instead of
15.1.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #1506 from ceph/guits-quick_fix

ceph-volume-nightly: fix a typo

ceph-volume-nightly: fix a typo

This fixes a quick typo introduced by e70982ce09a712f1182bc66d381de43a9076ec5f

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1504 from ceph/guits-fix_ceph-container_ci

ceph-container-prs: refact job template

ceph-container-prs: refact job template

This commit refact the job template for ceph-container.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1501 from ceph/jan--f-c-v-nightlies-centos8

Add nightly support for ceph-volume centos8 tests

Merge pull request #1503 from ceph/guits-fix_ceph-container_ci

ceph-container-prs: update scenarios matrix

ceph-container-prs: update scenarios matrix

This commit updates the matrix of scenario tested in ceph-container CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Add nightly support for ceph-volume centos8 tests

Merge pull request #1502 from tchaikov/bionic-for-building-doc

ceph-pr-docs: use bionic for building docs

ceph-pr-docs: use bionic for building docs

xenial comes with python3.5, and our document is renders with a python3
venv created on the builder, also, the python code is also parsed by
the python3 env, so using python3.5 pratically prevents us from using
python3.6 features.

so we can use python3.6 which is offered by bionic.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1472 from ceph/wip-release-dev

ceph-dev: Build containers from ceph.git branches

Merge pull request #1500 from ceph/wip-ceph-volume-tests

ceph-volume-nightly: adds mimic and luminous to batch mixed type tests

ceph-volume-nightly: adds mimic and luminous to batch mixed type tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1499 from tchaikov/wip-crimson-flavor

scripts/build_utils.sh: install deps for crimson flavor

Reviewed-by: David Galloway <dgallowa@redhat.com>

scripts/build_utils.sh: install deps for crimson flavor

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1498 from ceph/wip-noarm64

Disabling ceph-pull-requests-arm64 until we have packages and reliable hw

Disabling ceph-pull-requests-arm64 unless 'arm64' label is used

CentOS8 is missing dependencies.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1496 from ceph/ceph-medic-pr

ceph-medic-pull-requests: Skip py37 env

ceph-medic-pull-requests: Skip py37 env

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-medic-pull-requests: Replace alfredo

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1494 from ceph/guits-add_job

ceph-ansible: add external ceph clients scenario

ceph-ansible: add external ceph clients scenario

Add a new job in order to test the external ceph clients scenario
testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1493 from ceph/wip-cb-setup

ceph-build: Update setup_rpm for CentOS8 support

ceph-build: Update setup_rpm for CentOS8 support

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1492 from ceph/wip-lol-octopus

ceph-build: Add octopus

ceph-build: Add octopus

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1491 from tchaikov/wip-arm64-on-bionic-only

ceph-pull-requests-arm64: only build on bionic

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-pull-requests-arm64: only build on bionic

ktdreyer/ceph-el8 is amd64 only, so to avoid failures like:

```
No matching package to install: 'python3-cherrypy'
No matching package to install: 'python3-pecan'
No matching package to install: 'python3-routes'
```

we need to disable the "make check" test on the combination of
"arm64+centos8" until these packages are packaged for arm64 or
provided by centos8.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1490 from ceph/guits-add_all_in_one

ceph-ansible-prs: add a new scenario

ceph-ansible-prs: add a new scenario

add an 'all in one' scenario in order to catch more collocation related
issues.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1488 from tchaikov/wip-no-install-deps

ceph-*-setup: do not run install-deps.sh

Merge pull request #1489 from ceph/jan--f-ansible4-nautilus

use ansible 4.0 for nautlius nightlies

use ansible 4.0 for nautlius nightlies

ceph-ansible has dropped centos7 support in master, nautilius however does support it still.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-*-setup: do not run install-deps.sh

this is a follow-up change of #1487. since we don't use autotools for
preparing dist tarball, there is no need to run install-deps.sh for
installing the autotools.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1486 from ceph/ceph_collect_logs_timeout

collect-logs: set connect timeout value

Merge pull request #1487 from tchaikov/wip-no-more-autotools

ceph-*: drop autotools support

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-*: drop autotools support

we don't build ceph with autotools anymore. we switched to cmake since
luminous. see https://github.com/ceph/ceph/pull/9995.

so there is no need to support autotools anymore. also, this avoids
running install-deps.sh simply for running "make dist". as we use
`make-dist` script after switching over to cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>

collect-logs: set connect timeout value

If a job fails and the ceph cluster doesn't respond then the ceph status
task in the collect-logs.yml playbook will wait around 50 mins before
timed out.

  "cmd": [
      "ceph",
      "--cluster",
      "ceph",
      "-s",
      "-f",
      "json"
  ],
  "delta": "0:50:00.197544"
  "end": "2020-01-16 19:45:06.932819"
  "rc": 1,
  "start": "2020-01-16 18:55:06.735275"

  STDERR: [errno 110] RADOS timed out (error connecting to the cluster)

Instead we can use the --connect-timeout paramater to fail after 10s.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1485 from tchaikov/wip-arm64-on-el8

ceph-pull-requests-arm64: only build on centos8 and bionic

ceph-pull-requests-arm64: only build on centos8 and bionic

since we've labeled the slaves correctly. and centos8 is able to build
ceph on arm64. the only pending issue is
https://bugzilla.redhat.com/show_bug.cgi?id=1673990 . but we can fix it
manually.

since there is no trusty or xenial arm64 builders, we can just use
'arm64 && !centos7'

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1484 from ceph/wip-no-double-setup

ceph*build: Don't run pbuilder --clean a second time

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1483 from ceph/wip-no-tar

ceph-build: Apply e8bdd639

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph*build: Don't run pbuilder --clean a second time

It already gets run in build/setup_pbuilder

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-build: Apply e8bdd639

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1481 from ceph/wip-no-hostname

ansible: Set hostname on OpenSUSE

Merge pull request #1482 from tchaikov/wip-cd

ceph-build: apply 604882e9 to ceph-build

ceph-build: apply 604882e9 to ceph-build

to address following failure

+ cd ceph-14.2.6 ceph-14.2.6.tar.bz2 ceph-14.2.6.tar.gz
/tmp/jenkins10747333749307639424.sh: line 1058: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>

ansible: Set hostname on OpenSUSE

Now that we're using system-provided ansible packages (https://github.com/ceph/mita/pull/135), we're at the mercy of the version provided.

OpenSUSE Leap 15.1 gives us ansible 2.8.1 which has this bug: https://github.com/ansible/ansible/issues/42726

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1480 from ceph/guits-add_docker2podman

ceph-ansible: use dedicated tox for docker2podman job

ceph-ansible: use dedicated tox for docker2podman job

This commit makes tox using a dedicated tox ini file for the
docker_to_podman job.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1477 from ceph/wip-suse-sepia

sepia-fog-images: Support openSUSE

Merge pull request #1479 from ceph/guits-add_docker2podman

ceph-ansible-prs: add docker2podman scenario

ceph-ansible-prs: add docker2podman scenario

This commit adds a new 'docker2podman' job for the ceph-ansible CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1478 from ceph/wip-kernel-centos8

kernel: add centos8 builds

kernel: drop invalid axis

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: drop distros we don't build for

We don't use and don't have build agents for any of these.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: add centos8 builds

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

sepia-fog-images: Support openSUSE

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1453 from callithea/wip-dashboard-prs-install-deps

ceph-dashboard-*,ceph-api-nightly: Remove conditional-step; Replaced dir and script name

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-dashboard-*, ceph-api-nightly: Remove conditional-step; Replaced dir name; split-up
scripts

Split up the script which installs dependencies, since e2e and backend API tests require
different packages. Adapt all jobs to use the dedicated scripts.
Also remove the conditional step to not only execute the script that installs dependencies
for e2e test environments

Signed-off-by: Laura Paduano <lpaduano@suse.com>

ceph-dev: Build containers from ceph.git branches

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1474 from ceph/wip-pythonvirt

ansible: Install distro-appropriate virtualenv package version

Merge pull request #1475 from tchaikov/wip-dpkg-source-x

ceph-dev*-build: do not unpack source tar ball

ceph-dev*-build: do not unpack source tar ball

the uncompressed tarball is not used, moreover, the build fails on
bionic when `dpkg-source -x` the source package. because we've been
relying on a bug on xenial. see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=826334;msg=31

and on bionic, the bug was fixed. so `dpkg-source -x` started to
fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ansible: Install distro-appropriate virtualenv package version

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1473 from ceph/wip-noxenial

ceph-dev-new: Don't build master/wip on xenial

ceph-dev-new: Don't build master/wip on xenial

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1471 from ceph/wip-ugh-echo

ceph-dev-new-trigger: "fix" conditional step for all other branches

ceph-dev-new-trigger: "fix" conditional step for all other branches

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1470 from dmick/master

ceph-dev-new-build: allow containers on centos 7 or 8

ceph-dev-new-build: allow containers on centos 7 or 8

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1459 from ceph/wip-jjbyamlwtf

Update build matrices

Merge pull request #1469 from ceph/wip-cpr

ceph-pull-requests: Do NOT run on pbuilder hosts

ceph-pull-requests: Do NOT run on pbuilder hosts

pbuilder hosts are running Xenial.  We still want to be able to build Bionic packages on Xenial but we do not want make check to run on *all* slaves with the "bionic" label.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1468 from ceph/wip-bionic-slaves

ansible: Support Ubuntu Bionic slaves

ceph-pull-requests: Only run make check on Bionic

1) Bionic should have python2 and python3
2) All builds should run on Bionic but no other distro

Signed-off-by: David Galloway <dgallowa@redhat.com>

ansible: Support Ubuntu Bionic slaves

https://askubuntu.com/a/381527/906620

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1467 from ceph/wip-tcmu-runner-el8

tcmu-runner: fixed creation of chroot cfg file

tcmu-runner: fixed creation of chroot cfg file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1466 from ceph/wip-tcmu-runner-el8

tcmu-runner: second attempt at getting mock to use shaman repo

tcmu-runner: second attempt at getting mock to use shaman repo

It was falling back to the BaseOS repo which includes an older
version of librados/librbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1465 from ceph/wip-tcmu-runner-el8

tcmu-runner: fix environment variable typo

tcmu-runner: fix environment variable typo

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1464 from ceph/wip-tcmu-runner-el8

tcmu-runner: copy shaman.repo into mock build chroot

tcmu-runner: copy shaman.repo into mock build chroot

The el8 mock chroot environment templates now use special include
directives which makes it difficult to inject the shaman repos into
them before initializing the chroot environment. Therefore, just
directly copy the repo file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1463 from ceph/wip-c8-master

ceph-dev-build: setup_rpm Support for CentOS8

ceph-dev-build: setup_rpm Support for CentOS8

Signed-off-by: David Galloway <dgallowa@redhat.com>