Merge pull request #1521 from ceph/wip-agent-sl

Avoid symlink recursion issue

Avoid symlink recursion issue

Checkouts of the ceph repo contain directories with symlinks that link
to their parent dirs; this causes the file permissions check to fail.
Avoid following symlinks to work around.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #1518 from ceph/wip-noreboot

ceph-pull-requests: Don't require rebootable label

Merge pull request #1516 from ceph/wip-python3dev

Test teuthology PRs on Bionic & install python3-dev

ceph-*: Don't require rebootable label

We got rid of slave reboots for ceph-pull-requests in https://github.com/ceph/ceph-build/pull/1508

The other jobs already didn't reboot on failure.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1517 from ceph/guits-add_braggi_nodes

ceph-ansible: add new slave label 'braggi'

ceph-ansible: add new slave label 'braggi'

This will make ceph-ansible CI using new nodes 'braggi'

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1296 from ceph/wip-jupdates

More Jenkins deployment updates

Merge pull request #1495 from ceph/wip-c8-pull

scripts: Update signing scripts to support CentOS/RHEL8

ansible: Install python3 deps on Bionic and up

Again, I don't know how this was working before but I also had to set `executable:` under the pip tasks in order for them to work on all distros.

Signed-off-by: David Galloway <dgallowa@redhat.com>

teuthology-pull-requests: Test on Bionic only

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1515 from tchaikov/wip-enable-seastar-explicitly

ceph-pull-requests*: enable seastar explicitly

Reviewed-by: David Galloway <dgallowa@redhat.com>

ceph-pull-requests*: enable seastar explicitly

the next step is to disable it by default in ceph/ceph, so it's only
compiled and tested when explictly enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1514 from ceph/wip-notc

ceph-dev*trigger: Drop notcmalloc for centos7 on master/wip

ceph-dev*trigger: Drop notcmalloc for centos7 on master/wip

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1507 from ceph/wip-c8-libvirt

ansible: Update libvirt slave playbooks to support EL8

Merge pull request #1513 from ceph/guits-remove_ubuntu

ceph-ansible: remove ubuntu on nightlies jobs

ceph-ansible: use 'exclude' feature in nightlies definition

This commit refacts the project definitions in ceph-ansible nightlies
jobs, let's use 'exclude' instead of having multiple project blocks.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible: remove ubuntu testing on PRs

This commit removes the ubuntu testing on ceph-ansible PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible: remove ubuntu on nightlies jobs

This commit removes ubuntu testing on ceph-ansible nightlies jobs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1512 from rishabh-d-dave/cv-use-centos8-for-prs

ceph-volume: use cento8 instead of centos7 for PR jobs

ceph-volume: use cento8 instead of centos7 for PR jobs

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1510 from rishabh-d-dave/cv-add-scenarios-to-nightly

ceph-volume: add missing scenarios to nightlies

Merge pull request #1505 from rishabh-d-dave/cv-update-batch-functest-scenario-names

ceph-volume: add centos 8 based jobs for master and octopus

ceph-volume: add missing scenarios

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: separate xenial jobs from centos7 jobs

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: add nautilus to jobs for pre-octopus release

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: add centos 8 based jobs for master and octopus

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume: add missing scenarios to nightlies

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1508 from ceph/wip-prs-noreboot

ceph-pull-requests: Behave like arm64 job

ceph-pull-requests*: Don't risk exposing jenkins credentials

The `ps` command doesn't even get run currently because the job stops immediately after make check exits but let's not risk it.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-pull-requests: Behave like arm64 job

Debugging unit test failures was difficult on x86 jobs because the slave got rebooted right away.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ansible: Update libvirt slave playbooks to support EL8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1509 from toabctl/octopus-on-opensuse-leap-15.2

osc: Build octopus on openSUSE Leap 15.2

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>

osc: Build octopus on openSUSE Leap 15.2

We want to test with the upcoming openSUSE release (15.2) instead of
15.1.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #1506 from ceph/guits-quick_fix

ceph-volume-nightly: fix a typo

ceph-volume-nightly: fix a typo

This fixes a quick typo introduced by e70982ce09a712f1182bc66d381de43a9076ec5f

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1504 from ceph/guits-fix_ceph-container_ci

ceph-container-prs: refact job template

ceph-container-prs: refact job template

This commit refact the job template for ceph-container.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1501 from ceph/jan--f-c-v-nightlies-centos8

Add nightly support for ceph-volume centos8 tests

Merge pull request #1503 from ceph/guits-fix_ceph-container_ci

ceph-container-prs: update scenarios matrix

ceph-container-prs: update scenarios matrix

This commit updates the matrix of scenario tested in ceph-container CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Add nightly support for ceph-volume centos8 tests

Merge pull request #1502 from tchaikov/bionic-for-building-doc

ceph-pr-docs: use bionic for building docs

ceph-pr-docs: use bionic for building docs

xenial comes with python3.5, and our document is renders with a python3
venv created on the builder, also, the python code is also parsed by
the python3 env, so using python3.5 pratically prevents us from using
python3.6 features.

so we can use python3.6 which is offered by bionic.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1472 from ceph/wip-release-dev

ceph-dev: Build containers from ceph.git branches

Merge pull request #1500 from ceph/wip-ceph-volume-tests

ceph-volume-nightly: adds mimic and luminous to batch mixed type tests

ceph-volume-nightly: adds mimic and luminous to batch mixed type tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1499 from tchaikov/wip-crimson-flavor

scripts/build_utils.sh: install deps for crimson flavor

Reviewed-by: David Galloway <dgallowa@redhat.com>

scripts/build_utils.sh: install deps for crimson flavor

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1498 from ceph/wip-noarm64

Disabling ceph-pull-requests-arm64 until we have packages and reliable hw

Disabling ceph-pull-requests-arm64 unless 'arm64' label is used

CentOS8 is missing dependencies.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1496 from ceph/ceph-medic-pr

ceph-medic-pull-requests: Skip py37 env

ceph-medic-pull-requests: Skip py37 env

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-medic-pull-requests: Replace alfredo

Signed-off-by: Zack Cerza <zack@redhat.com>

scripts: Update signing scripts to support CentOS/RHEL8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1494 from ceph/guits-add_job

ceph-ansible: add external ceph clients scenario

ceph-ansible: add external ceph clients scenario

Add a new job in order to test the external ceph clients scenario
testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1493 from ceph/wip-cb-setup

ceph-build: Update setup_rpm for CentOS8 support

ceph-build: Update setup_rpm for CentOS8 support

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1492 from ceph/wip-lol-octopus

ceph-build: Add octopus

ceph-build: Add octopus

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1491 from tchaikov/wip-arm64-on-bionic-only

ceph-pull-requests-arm64: only build on bionic

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-pull-requests-arm64: only build on bionic

ktdreyer/ceph-el8 is amd64 only, so to avoid failures like:

```
No matching package to install: 'python3-cherrypy'
No matching package to install: 'python3-pecan'
No matching package to install: 'python3-routes'
```

we need to disable the "make check" test on the combination of
"arm64+centos8" until these packages are packaged for arm64 or
provided by centos8.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1490 from ceph/guits-add_all_in_one

ceph-ansible-prs: add a new scenario

ceph-ansible-prs: add a new scenario

add an 'all in one' scenario in order to catch more collocation related
issues.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1488 from tchaikov/wip-no-install-deps

ceph-*-setup: do not run install-deps.sh

Merge pull request #1489 from ceph/jan--f-ansible4-nautilus

use ansible 4.0 for nautlius nightlies

use ansible 4.0 for nautlius nightlies

ceph-ansible has dropped centos7 support in master, nautilius however does support it still.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-*-setup: do not run install-deps.sh

this is a follow-up change of #1487. since we don't use autotools for
preparing dist tarball, there is no need to run install-deps.sh for
installing the autotools.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1486 from ceph/ceph_collect_logs_timeout

collect-logs: set connect timeout value

Merge pull request #1487 from tchaikov/wip-no-more-autotools

ceph-*: drop autotools support

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-*: drop autotools support

we don't build ceph with autotools anymore. we switched to cmake since
luminous. see https://github.com/ceph/ceph/pull/9995.

so there is no need to support autotools anymore. also, this avoids
running install-deps.sh simply for running "make dist". as we use
`make-dist` script after switching over to cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>

collect-logs: set connect timeout value

If a job fails and the ceph cluster doesn't respond then the ceph status
task in the collect-logs.yml playbook will wait around 50 mins before
timed out.

  "cmd": [
      "ceph",
      "--cluster",
      "ceph",
      "-s",
      "-f",
      "json"
  ],
  "delta": "0:50:00.197544"
  "end": "2020-01-16 19:45:06.932819"
  "rc": 1,
  "start": "2020-01-16 18:55:06.735275"

  STDERR: [errno 110] RADOS timed out (error connecting to the cluster)

Instead we can use the --connect-timeout paramater to fail after 10s.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1485 from tchaikov/wip-arm64-on-el8

ceph-pull-requests-arm64: only build on centos8 and bionic

ceph-pull-requests-arm64: only build on centos8 and bionic

since we've labeled the slaves correctly. and centos8 is able to build
ceph on arm64. the only pending issue is
https://bugzilla.redhat.com/show_bug.cgi?id=1673990 . but we can fix it
manually.

since there is no trusty or xenial arm64 builders, we can just use
'arm64 && !centos7'

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1484 from ceph/wip-no-double-setup

ceph*build: Don't run pbuilder --clean a second time

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1483 from ceph/wip-no-tar

ceph-build: Apply e8bdd639

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph*build: Don't run pbuilder --clean a second time

It already gets run in build/setup_pbuilder

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-build: Apply e8bdd639

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1481 from ceph/wip-no-hostname

ansible: Set hostname on OpenSUSE

Merge pull request #1482 from tchaikov/wip-cd

ceph-build: apply 604882e9 to ceph-build

ceph-build: apply 604882e9 to ceph-build

to address following failure

+ cd ceph-14.2.6 ceph-14.2.6.tar.bz2 ceph-14.2.6.tar.gz
/tmp/jenkins10747333749307639424.sh: line 1058: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>

ansible: Set hostname on OpenSUSE

Now that we're using system-provided ansible packages (https://github.com/ceph/mita/pull/135), we're at the mercy of the version provided.

OpenSUSE Leap 15.1 gives us ansible 2.8.1 which has this bug: https://github.com/ansible/ansible/issues/42726

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1480 from ceph/guits-add_docker2podman

ceph-ansible: use dedicated tox for docker2podman job

ceph-ansible: use dedicated tox for docker2podman job

This commit makes tox using a dedicated tox ini file for the
docker_to_podman job.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1477 from ceph/wip-suse-sepia

sepia-fog-images: Support openSUSE

Merge pull request #1479 from ceph/guits-add_docker2podman

ceph-ansible-prs: add docker2podman scenario

ceph-ansible-prs: add docker2podman scenario

This commit adds a new 'docker2podman' job for the ceph-ansible CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1478 from ceph/wip-kernel-centos8

kernel: add centos8 builds

kernel: drop invalid axis

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: drop distros we don't build for

We don't use and don't have build agents for any of these.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: add centos8 builds

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

sepia-fog-images: Support openSUSE

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1453 from callithea/wip-dashboard-prs-install-deps

ceph-dashboard-*,ceph-api-nightly: Remove conditional-step; Replaced dir and script name

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-dashboard-*, ceph-api-nightly: Remove conditional-step; Replaced dir name; split-up
scripts

Split up the script which installs dependencies, since e2e and backend API tests require
different packages. Adapt all jobs to use the dedicated scripts.
Also remove the conditional step to not only execute the script that installs dependencies
for e2e test environments

Signed-off-by: Laura Paduano <lpaduano@suse.com>

ceph-dev: Build containers from ceph.git branches

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1474 from ceph/wip-pythonvirt

ansible: Install distro-appropriate virtualenv package version

Merge pull request #1475 from tchaikov/wip-dpkg-source-x

ceph-dev*-build: do not unpack source tar ball

ceph-dev*-build: do not unpack source tar ball

the uncompressed tarball is not used, moreover, the build fails on
bionic when `dpkg-source -x` the source package. because we've been
relying on a bug on xenial. see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=826334;msg=31

and on bionic, the bug was fixed. so `dpkg-source -x` started to
fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>