Merge pull request #1410 from ceph/guits-ca_pipeline

ceph-ansible-prs: remove non breaking space

ceph-ansible-prs: remove non breaking space

remove non breaking space introduced by 54ed76efcf9b956079daf22669dbfc14b6114224

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1409 from ceph/guits-ca_pipeline

ceph-ansible-prs: don't display podman job on github on stable-3.2

ceph-ansible-prs: don't display podman job on github on stable-3.2

This should avoid to display this job on the github status.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1407 from ceph/guits-ca_pipeline

ceph-ansible: remove stable-3.1 testing

ceph-ansible: don't run podman on stable-3.2

skip podman job when target branch is stable-3.2 since it doesn't
support podman.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible-prs: specify which tox config file is used

When throwing this error, the name can be different according to the
scenario name.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible: remove nightly jobs on stable-3.1

stable-3.1 has no more engineering efforts.
Let's remove testing on this branch.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1406 from ceph/guits-ca_pipeline

ceph-ansible: remove pipeline

ceph-ansible: remove pipeline

The current pipeline is time consuming, it takes ~3h to complete all phases.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1405 from rzarzynski/wip-crimson-use-release-build-type

ceph-perf-pull-requests: use Release builds for crimson perf testing.

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: use Release builds for crimson perf testing.

This patch is mostly intended to build Seastar without `SEASTAR_DEBUG`
flag to avoid impact on performance. There is actually a warnining in
`src/core/app-template.cc`:

  ```cpp
  #ifdef SEASTAR_DEBUG
      fmt::print("WARNING: debug mode. Not for benchmarking or production\n");
  #endif
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #1404 from tchaikov/wip-el8

ceph-dev-new-*: build for CentOS8

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-dev-new-build/build/setup_rpm: install python2 for el7

otherwise we will have

Traceback (most recent call last):
  File "/tmp/venv.pAISkJ09vG/bin/chacractl", line 6, in <module>
    main.ChacraCtl()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 38, in __init__
    self.main(argv)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 82, in main
    parser.dispatch()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/tambo/dispatcher.py", line 21, in dispatch
    result = instance.main()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/api/exists.py", line 62, in main
    return self.head(url)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/util.py", line 65, in inner_wrapper
    for counter in xrange(times):
NameError: name 'xrange' is not defined

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: do not 'cd ceph-*'

on cenos8, we will have

$ cd ceph-15.0.0-5635-gda7c5f3 ceph-15.0.0-5635-gda7c5f3.tar.bz2 ceph-15.0.0-5635-gda7c5f3.tar.gz
/tmp/jenkins8338985937861239310.sh: line 1055: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-*: build for centos8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: enable PowerTools repo for EL8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: get RELEASE early

so we can use it when enabling more repos

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1401 from tchaikov/wip-python3

ceph-*-build: s/python36-devel/python3-devel/

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1403 from ceph/wip-fix-kernel

kernel-trigger: Build any wip branch

Merge pull request #1402 from ceph/wip-kernel-statuses

kernel: Report build statuses to shaman

kernel-trigger: Build any wip branch

I honestly don't know why this works and origin/wip* didn't but here we are.

Signed-off-by: David Galloway <dgallowa@redhat.com>

kernel: Report build status to shaman

Signed-off-by: David Galloway <dgallowa@redhat.com>

kernel: Update shaman status on failed build

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-*-build: s/python36-devel/python3-devel/

to silence the warning of

Package python36-devel is obsoleted by python3-devel, trying to install
python3-devel-3.6.8-10.el7.x86_64

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1399 from ceph/wip-kernel-usercopy-and-ceph-labels

kernel: enable CONFIG_HARDENED_USERCOPY and CONFIG_CEPH_FS_SECURITY_LABEL

kernel: enable CONFIG_CEPH_FS_SECURITY_LABEL

New in kernel 5.3.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: enable CONFIG_HARDENED_USERCOPY

This is something we had to work around in libceph, see linux.git
commit 7e241f647dc7 ("libceph: fall back to sendmsg for slab pages").
It is enabled by default in many distros.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1398 from ceph/wip-build-wips-in-ceph-client

kernel: update the trigger list

kernel: update the trigger list

We didn't trigger on wip* branches like other ceph projects because
there was a lot of old branches matching that mask in the repository.
Now that they are cleaned up, drop the workarounds (testing*/master*)
and add wip*.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1397 from ceph/repo-cleanup

clean up shaman repos left behind after a build

tcmu-runner: remove custom shaman repos added after a build

Signed-off-by: Alfredo Deza <adeza@redhat.com>

nfs-ganesha-stable: remove custom repo/list configuration files added

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1396 from ceph/wip-centos-sclo-rh-source-skip

ceph-*-build: skip unavailable repo when yum-builddep

ceph-*-build: skip unavailable repo when yum-builddep

As centos-sclo-rh-source leads us to 404 at this moment. and we are not
using the source repo for building ceph. so we can just skip any
unavailable repo.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #1394 from ceph/revert-scm-changes

Revert "*/scm: only pull the PR under test"

Revert "*/scm: only pull the PR under test"

This reverts commit 4eae5698a6a1383c69338ec7781e4cee6c087a9f.

It breaks all pull-request jobs for ceph-volume at least, haven't been
able to investigate all other affected

Merge pull request #1393 from tchaikov/wip-crimson-perf

ceph-perf-pull-requests: add ceph-perf-pull-requests

Merge pull request #1392 from tchaikov/wip-scm-refspec

*/scm: only pull the PR under test

ceph-perf-pull-requests: add ceph-perf-pull-requests

for running cbt test against PR and master, then comparing them

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1378 from dmick/wip-dmick

Add container builds using ceph-container.git for centos7

Merge pull request #1391 from hjensas/issue#1390

pr-syntax-check - install all requirements

pr-syntax-check - install all requirements

This change removes the grep for 'ansible' when installing
requirements in ceph-ansible-pr-syntax-check. The grep
cause onlye the ansible requirement to be installed, ignoring
the other packages. This causes an issue in PR[1] which
introduce a filter plugin that depend on the netaddr package.

[1] https://github.com/ceph/ceph-ansible/pull/4339

Closes: #1390
Signed-off-by: Harald Jensås <hjensas@redhat.com>

ceph-build-pull-requests: pass --config-xml to "jenkins-jobs test"

to silence warnings like

WARNING:jenkins_jobs.cli.subcommand.test:(Deprecated) The default output
behavior of `jenkins-jobs test` when given the --output flag will change
in JJB 3.0. Instead of writing jobs to OUTPUT/jobname; they will be
written to OUTPUT/jobname/config.xml. The new behavior can be enabled by
the passing `--config-xml` parameter.

also redirect the test result to /dev/null, otherwise the test will fail
because the output directory may already exist.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-build-pull-requests: ping jjb to 3.0.2

bump jjb's version

Signed-off-by: Kefu Chai <kchai@redhat.com>

*/scm: only pull the PR under test

see
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md#creating-a-job

no need to pull all PR references of the remote repo. it'd be faster
just pull the references of a single PR. actually, what we need is but

- refs/pull/${pr}/head
- refs/pull/${pr}/merge

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1389 from tchaikov/wip-python-rpm-macro

ceph-*build: install python36-devel for rpm macros

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-*build: install python36-devel for rpm macros

see also 100a0fb5
see also https://github.com/ceph/ceph/pull/30190

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>

Conditionally add container builds for wip- branches

New parameter CI_CONTAINER ('true' or 'false') controls behavior;
if set to 'true', ceph-container.git will be used to build a centos7
daemon-base default flavor container and push it to a repo server.
This requires waiting until the shaman repo is built.

New parameters CONTAINER_REPO_{HOSTNAME, ORGANIZATION} set where
to push the resultant container.

If repo doesn't become ready or if the actual build/push fails, don't
fail the overall build.  Maybe we can come up with a better way
to indicate this than just a log notation

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-dev-new-build: add quay.io credentials for cephci repo

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-dev-new-*: add CONTAINER_BRANCH and check it out

In case a container build is needed, check out $CONTAINER_BRANCH
of ceph-container.git to use to build it

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1388 from ceph/prune-fix

scripts: use a loop instead of globbing directly

scripts: use a loop instead of globbing directly

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1387 from tchaikov/wip=40646

Revert "ceph-*build/build/setup_rpm: downgrade to gcc-c++-8.2.1"

Revert "ceph-*build/build/setup_rpm: downgrade to gcc-c++-8.2.1"

This reverts commit cf2c26b181a86738e2a8bb17545f45d6aa1c7c45.

since https://bugzilla.redhat.com/show_bug.cgi?id=1726630 has been
fixed. and we have devtoolset-8-gcc-c++-8.3.1-3.1.el7.x86_64.rpm.

no reason to stick to devtoolset-8-gcc-c++-8.2.1-3 anymore.

Fixes: https://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1386 from ceph/prune-fix

scripts: can't use globbing with quotes

scripts: can't use globbing with quotes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1385 from ceph/prune-fix

scripts: do not specify directories with find -delete, rm the path

Reviewed-by: Kefu Chai <kchai@redhat.com>

scripts: do not specify directories with find -delete, rm the path

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1383 from ceph/prune-fix

scripts: use find's -delete option to prevent 'argument too long'

Reviewed-by: Kefu Chai <kchai@redhat.com>

scripts: use find's -delete option to prevent 'argument too long'

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1382 from ceph/fix-cv-tox

ceph-volume-pr: remove escape which doubles the curly brackets

ceph-volume-pr: remove escape which doubles the curly brackes resulting in syntax errors

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1381 from ceph/submodules-wipe

ceph-pr-submodules: wipe the workspace to prevent repo corruption

ceph-pr-submodules: wipe the workspace to prevent repo corruption

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1380 from ceph/ceph-signed-wipe

ceph-pr-commits: wipe workspace to prevent repo corruption

ceph-pr-commits: wipe workspace to prevent repo corruption

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1377 from ceph/ceph-volume-cleanup

ceph-volume: cleanup workspace before running tests

ceph-volume use wipe-workspace in Git to cleanup before running

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1379 from ceph/wip-signed-master

ceph-pr-commits: add trigger phrase

ceph-pr-commits: add trigger phrase

When Jenkins fails to clone a branch, signed-off checks will fail. If
all other checks have passed, there's no reason to force a full run of
Jenkins jobs again on that PR. This PR introduces `jenkins test signed`
trigger phrase to allow re-running the pr-commit check.

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #1376 from ceph/wip-enable-leap15

enable leap15 for ceph-dev-trigger and ceph-dev-new-trigger

ceph-dev-trigger: enable leap15 distro

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

ceph-dev-new-trigger: enable leap15 distro

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1374 from ceph/wip-scsi-checks

ceph-iscsi-stable: Check for specific package files

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

ceph-iscsi-stable: Check for specific package files

The -f `-*` got expanded too many times if there are multiple matches which broke the `if` statement.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1373 from ceph/wip-configshell-fb

configshell-fb: initial builder for python-configshell packages

configshell-fb: initial builder for python-configshell packages

This updated package is required by ceph-iscsi but won't be available
until CentOS 7.7(ish).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1372 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: fix revision to '-1' if building at tag

ceph-iscsi-stable: fix revision to '-1' if building at tag

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1371 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: inject missing credentials

ceph-iscsi-stable: inject missing credentials

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1370 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: query correct chacra host for uploads

ceph-iscsi-stable: query correct chacra host for uploads

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1369 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: incorporate consolidated repo changes

ceph-iscsi-stable: incorporate consolidated repo changes

ceph-iscsi-config and ceph-iscsi-cli are now merged into
the ceph-iscsi repo.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1368 from ceph/wip-leap15-available-dist

leap15: add leap15 to available dist

leap15: add leap15 to available dist

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1351 from callithea/wip-ceph-dashboard-prs-backend-api

ceph-dashboard-pr-backend: add job config

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-dashboard-pr-backend: add job config

Signed-off-by: Laura Paduano <lpaduano@suse.com>
Signed-off-by: alfonsomthd <almartin@redhat.com>

Merge pull request #1365 from ceph/wip-osc-build-scripts

osc build scripts

Merge pull request #1367 from tchaikov/wip-upload-ddeb-to-chacra

ceph*build/build/build_deb: upload *.ddeb packages

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph*build/build/build_deb: upload *.ddeb packages

ubuntu names *dbgsym* packages .ddeb in hope to put them into a
separated repo. while debian just uses the plain .deb extension for them.
so, to include the dbgsym packages created by ubuntu, we need to upload
the .ddeb packages as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1343 from ceph/github-token

ceph-volume: allow passing GITHUB_TOKEN as a password to manually set github status

ceph-dev-new-build: add osc build scripts

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

ceph-dev-build: add osc build scripts

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

validate_rpm: reuse rpm validation from another project

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

osc: add osc related scripts:
- validate_osc
- setup_osc
- build_osc
Supposed for adding them later to ceph-dev-new-build or ceph-dev-build

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1366 from ceph/wip-jb-group

slave.yml: Add jenkins-build user to jenkins-build group

slave.yml: Add jenkins-build user to jenkins-build group

On ubuntu, at least, with the addition of the "create a jenkins-build group" task, the creation of the user wasn't adding the user to the group.  I discovered this by seeing a docker command fail after the socket was 'chgrp'ed to jenkins-build

```
+ sudo chgrp jenkins-build /var/run/docker.sock
+ do_clean
+ make clean.all
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
```

I logged in to the slave and confirmed this was the case.

Signed-off-by: David Galloway <dgallowa@redhat.com>