ansible: fix osc build-root typo for opensuse worker

Fixes: https://tracker.ceph.com/issues/48141
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1685 from ceph/wip-change-osc-builroot

ceph-dev-build: Change build path for Leap builds

Merge pull request #1694 from tchaikov/wip-seastar-allocator

ceph-*-setup: use default allocator for crimson flavor

Reviewed-by: Samuel Just <sjust@redhat.com>

ceph-dev-build: Change build path for Leap builds

Previously, packages were being written to `/var/tmp` which does not live on the intended 1TB SSD.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-*-setup: use default allocator for crimson flavor

since we've merged https://github.com/ceph/ceph/pull/37887

This reverts commit 3aa35a92b466212c0c3c715781a8aab8a5e0136c.

Merge pull request #1628 from tchaikov/wip-images-with-flavors

ceph-*-build: pass CONTAINER_FLAVOR to build-push-ceph-container-imgs.sh

Revert "ceph-dev-*-build: reset CI_CONTAINER when building nautilus"

This reverts commit d86ed63576a7dcc213c211b7678da2af6b6de108.

if we are able to build container images for nautilus on centos7 and
centos8, there is no need to skip this step.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-*-build: pass CONTAINER_FLAVOR to build-push-ceph-container-imgs.sh

so we can specify the distro and its version

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1693 from ceph/ceph_container_flake8_selinux

ceph-container-flake8: add selinux flag to volume

ceph-container-flake8: add selinux flag to volume

Without the selinux flag on the container volume, the flake8 command
fails with:

E902 IOError: [Errno 13] Permission denied: '/envglobals.py'

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1691 from huww98/patch-1

Fix Signed-off-by check with non-ASCII names

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fix Signed-off-by check with non-ASCII names

Blocking https://github.com/ceph/ceph/pull/37810 https://github.com/ceph/ceph/pull/37849

Merge pull request #1690 from ceph/wip-pipeline-master

ceph: multibranch: trust contributors' forks

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph: multibranch: trust contributors' forks

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #1687 from ceph/wip-docsdocs

ceph-pr-render-docs: Use RTD URL

Merge pull request #1689 from ceph/ceph_container_lint_rm_option

ceph-container-lint: use --rm docker option

ceph-container-lint: use --rm docker option

The container lint job, which run shellcheck in a docker container,
doesn't use the --rm docker option so the container aren't removed
after their execution.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

ceph-pr-render-docs: Use RTD URL

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1686 from tchaikov/wip-ceph-pr-commits

ceph-pr-commits: use origin/master as the base commit

ceph-pr-commits: use origin/master as the base commit

instead of using `${sha1}^1...${sha1}^2` use
`origin/master...origin/pr/${ghprbPullId}/head` for getting the range of
commits introduced by a PR. as per `man git-diff`

> "git diff A...B" is equivalent to "git diff $(git-merge-base A B) B".

and `origin/${ghprbTargetBranch}` and `origin/pr/${ghprbPullId}/head` are fetched by
the ghprb-plugin plugin, so it's safe to use them. see
https://plugins.jenkins.io/ghprb/

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1683 from ceph/wip-sha1

ceph-pr-commits: Use sha1 instead of GIT_COMMIT

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-pr-commits: Use sha1 instead of GIT_COMMIT

I think GIT_COMMIT is the merge commit.  Everything Kefu and I could find indicated it should be valid to use here.

Switching to ${sha1} works though.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1682 from tchaikov/wip-ceph-pr-commits-decode

ceph-pr-commits: skip non-ascii chars

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

ceph-pr-commits: skip non-ascii chars

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1681 from ceph/wip-bump-jjb

jjb: Unpin jjb version

jjb: Unpin jjb version

Pinning it keeps biting us more often than fixing/updating job yamls would.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1668 from ceph/wip-pipeline-master

ceph: enable multibranch pipeline

Merge pull request #1678 from tchaikov/wip-drop-bashism

ceph-perf-pull-requests: s/source/./

Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1680 from tchaikov/wip-ceph-pr-commits-junit

ceph-pr-commits: skip test for doc only changes

Reviewed-by: Zack Cerza <zack@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>

ceph-pr-commits: add a doc_test checking title of commit message

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pr-commits: port test_commits.py to py3

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pr-commits: skip "code_test" tests for commits changing "doc/"

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pr-commits: add "code_test" for test_signed_off_by() test

this test should only apply to changes which modifies code. for the doc
only changes, we should skip this test for lowering the bar for
contributors willing to improve the document.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: s/source/./

turns out we have ubuntu test nodes labeled with "performance", and
they use dash for "sh", and jenkins use "sh" for running the embedded
scripts.

let's avoid bashism, and use "." instead.

this change is a follow-up of 4e0a8987c7d19004d33c153b107057f684b547de

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1677 from ceph/wip-dpkg-a

ceph-pr-api: Clean dpkg db on job abort

ceph-pr-api: Clean dpkg db on job abort

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1676 from ceph/wip-xml-signed

ceph-pr-commits: Allow empty junit results

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-pr-commits: Allow empty junit results

report.xml wasn't getting written for docs-only changes because the job `exit 0`s.  The junit plugin was failing the build in those instances but we don't need or want it to.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1675 from ceph/guits-remove_ooo_collocation

ceph-ansible: remove ooo_collocation

ceph-ansible: remove ooo_collocation

This job is not needed anymore, we already have 'collocation' for that.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph: add multibranch pipeline

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #1672 from tchaikov/wip-ceph-deploy-tag

ceph-deploy-tag: use ceph-build:origin/master

Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1674 from dmick/master

ceph-dev{,-new}-build: add SHA1 to build name

ceph-deploy-build: inject chacractl credentials

otherwise we would have

requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://chacra.ceph.com/binaries/ceph-deploy/test/c6ff0a05523cdeda86ebac732dd186620261dc01/centos/7/source/

when posting the created package to chacra

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils: remove duplicated get_rpm_dist()

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-deploy-*: install build deps for python3

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-deploy-build: build on bionic and centos8 for python3 package

* s/xenial/bionic/ for building python3 package
* add centos8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-deploy-tag: use ceph-build:origin/master

otherwise it fails when trying to use the release.yml playbook:

ERROR! the playbook: release.yml could not be found
Build step 'Execute shell' marked build as failure

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1673 from ceph/revert-1648-wip-junit_dashboard-master

Revert "test: analyze JUnit results in PR jobs"

ceph-dev{,-new}-build: add SHA1 to build name

Signed-off-by: Dan Mick <dmick@redhat.com>

Revert "test: analyze JUnit results in PR jobs"

Merge pull request #1671 from ceph/wip-signed-docs

ceph-pr-commits: Don't require signed commits for doc/ dir

ceph-pr-commits: Don't require signed commits for doc/ dir

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1648 from ceph/wip-junit_dashboard-master

test: analyze JUnit results in PR jobs

Merge pull request #1670 from dmick/master

ceph-dev, ceph-dev-new: get shorter branch name, add SHA1 to build name

test: add JUnit result analysis to jobs

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

ceph-dev, ceph-dev-new: add SHA1 to build names

Signed-off-by: Dan Mick <dmick@redhat.com>

ceph-dev, ceph-dev-new: copy branch artifact too for short name

BRANCH is set by the github plugin to, say, origin/master.  The
-setup jobs change that back to just 'master', so, copy their
saved env-setting artifact 'branch' as well so the parent job
can also use the short branch name in its build name.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1669 from dmick/master

ceph-dev, ceph-dev-new: copyartifact permission: no spaces

ceph-dev, ceph-dev-new: copyartifact permission: no spaces

I don't see why from examining the plugin source, but it appears
that if you use both a comma and a space, the second entry will not
be found (as though the space is becoming part of the string for
comparison inside the plugin).  Stopping fighting and just letting it happen.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1667 from dmick/master

Fix intermittent error with last copyartifacts change

Fix intermittent error with last copyartifacts change

Apparently, sometimes, Jenkins can't find a toplevel job by "name", and
one must specify an "absolute path" to the job by prefixing "/".  This
was causing ceph-dev builds to fail to find ceph-dev-setup, but changing
to /ceph-dev-setup fixed the failure-to-find.  It didn't affect ceph-dev-new.
I've no idea why it affected one and not the other.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1665 from dmick/master

Grab SHA1 into parent job's environment for human or program

Grab SHA1 into parent job's environment for human or program

The Ceph SHA1 is divined by the -setup job, and not available in the
parent job.  That's easily fixed by the same method it's passed to the
-build job: grab the file artifact and use it to inject SHA1 into the
environment.  This means the parent job can show the SHA1 in its build
label for humans, and can also be searched by Jenkins API calls.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1663 from ceph/ceph_ansible_nightly_quay_ceph_v2

ceph-ansible-nightly: use latest stable tag

ceph-ansible-nightly: use latest stable tag

Currently the curl command is returning all tags matching the ceph
release so the variable contains multiple values.
We only need to have the latest one.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1662 from dmick/master

ceph-dev{-new}: add build names

ceph-dev{-new}: add build names

Name builds by their build number as well as branch, distros, and flavor,
for easier viewing on the Jenkins UI.  Uses a new-to-us jenkins plugin.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #1661 from ceph/wip-restart-when-necessary

ansible: REstart the jenkins agent service when changes are made

ansible: REstart the jenkins agent service when changes are made

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1652 from ceph/wip-abbrev

ceph*setup: Abbreviate the tag

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1660 from ceph/guits-cv_tox_labels

ceph-volume-pr: change nodes for tox testing

ceph-volume-pr: change nodes for tox testing

We are currently running this job on a ubuntu xenial slave which run
python 3.5 where we need python 3.6.

Switching to these centos7 nodes which use python 3.6 should fix this
issue.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1659 from ceph/wip-dontfollow

ansible: Don't follow symlinks

ansible: Don't follow symlinks

Something in ceph.git creates an infinite loop that this task gets stuck on

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1658 from ceph/wip-unexpose

ansible: Pass API key to slave.jar as file

ansible: Pass API key to slave.jar as file

vstart_runner.py runs 'ps ww' for the jenkins-build user.  This exposes the API key by showing the java.jar process in Jenkins logs.  If we pass the credentials as a file, there's less of a chance of it getting exposed.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1656 from ceph/wip-start-libvirt

ansible: Start libvirt services on slave spinup

Merge pull request #1655 from ceph/ceph_ansible_nightly_quay_ceph

ceph-ansible-nightly: use quay.ceph.io registry

ansible: Start libvirt services on slave spinup

I accidentally left these out when I combined all the slave playbooks.  It wasn't caught till now because I hadn't actually pulled the latest ceph-build.git on prado

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1654 from ceph/wip-kernel-drop-centos7

kernel: drop centos7 builds and arm64 cruft

ceph-ansible-nightly: use quay.ceph.io registry

In order to avoid the docker.io rate limit.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1653 from ceph/wip-cleanup-home

ceph-api-nightly: Clean up after yourself

kernel: drop arm64 cruft

Don't pretend we can build an arm64 kernel: our kernel config is for
x86_64.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: drop centos7 builds

The gcc version requirement was raised to 4.9 in kernel 4.8.  centos7
is on gcc 4.8 and these builds have been failing for the last couple of
months.

While we could switch to devtoolset-7 or devtoolset-8, both of which
are installed on our centos7 build agents, let's just drop them.  For
one-off testing of older releases, trusty and xenial would do.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

ceph-api-nightly: Clean up after yourself

This job has a bunch of templates.  Each one reaches 30-50GB in size which quickly fills up our builder home dirs.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph*setup: Abbreviate the tag

I don't know what changed or when but `git describe` started returning a longer string.  Our tooling relies on there only being 8 characters in the last portion of the version so we need to keep naming our packages that way.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1651 from ceph/wip-timeout

bz-mill: Scrap job

Merge pull request #1649 from ceph/wip-xterm

ansicolor jobs: Set TERM=xterm

bz-mill: Scrap job

Checked with Christina.  She no longer uses this job.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1650 from tspmelo/wip-cypress-libxss1

install-e2e-test-deps.sh: Add libxss1 to debian distros

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

install-e2e-test-deps.sh: Add libxss1 to debian distros

Cypress is failing due to a missing package.

Signed-off-by: Tiago Melo <tmelo@suse.com>

ansicolor jobs: Set TERM=xterm

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1647 from ceph/wip-cancel_on_pr_udpate-master

ceph-*pull-requests*: cancel job on PR update

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-By: Kefu Chai <kchai@redhat.com>

build: cancel job on PR update

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #1646 from tchaikov/wip-ceph-perf-pull-requests-ln

ceph-perf-pull-requests: do not fail if symblink already exists

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

ceph-perf-pull-requests: s/source/./

turns out we have ubuntu test nodes labeled with "performance", and they
use dash for "sh", and jenkins use "sh" for running the embedded
scripts.

let's avoid bashism, and use "." instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: install clang-10 on debian/ubuntu

and always install clang for both classic and crimson job

so we can see the warnings spit by clang when running classic perf tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: do not fail if symblink already exists

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1645 from tchaikov/wip-remove-unexpected-token

ceph-perf-pull-requests: remove superfluous ";;"

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>