ceph-pull-requests-arm64: Don't make check on CentOS8 yet

Until deps are in epel8 or we figure out an alternative

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1421 from ceph/wip-ansible-examples-leap15

ansible/examples/slave: add leap15

ansible/examples/slave: add leap15

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1420 from ceph/wip-mock-slave

Add jenkins-build user to mock group during slave setup

Add jenkins-build user to mock group during slave setup

Instead of during each individual job

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1415 from ceph/wip-centos8

ansible: Update playbooks for CentOS8 compatibility

Merge pull request #1418 from ceph/add_ca_ooo

ceph-ansible-prs: run ooo_collocation on container only

ceph-ansible-prs: run ooo_collocation on container only

Add a dedicated project for ooo_collocation since we don't want it to
run in non_container context.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1417 from ceph/add_ca_ooo

ceph-ansible-prs: re-add ooo_collocation

ceph-ansible-prs: re-add ooo_collocation

This commit adds the ooo_collocation that were removed recently.
Related ceph-ansible PR : https://github.com/ceph/ceph-ansible/pull/4401

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1414 from tchaikov/wip-ceph-pr-submodules

ceph-pr-submodules: do not use shallow-clone

ansible: Update playbooks for CentOS8 compatibility

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1413 from ceph/ceph_ansible_triggered_jobs

ceph-ansible-prs: re-add shrink/rgw_multisite jobs

ceph-pr-submodules: do not use shallow-clone

we use
```
git diff --submodule=log origin/$ghprbTargetBranch...$ghprbActualCommit
```

for checking the differences between the delta since their common
ancestor, so we have to do a non-shallow clone for accessing the
common ancestor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-ansible-prs: re-add shrink/rgw_multisite jobs

Since the pipeline refactoring we're not able to trigger manually the
shrink and rgw_multisite jobs. The shrink jobs were only configured for
ubuntu and rgw_multisite was missing.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1395 from tchaikov/wip-scm-faster-clone

*/scm: faster clone

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1400 from ceph/wip-report-osc-release

osc: report osc build release correctly

Merge pull request #1408 from ceph/wip-add-teuthology-admins

teuthology: add more admins to allow testing pr

Merge pull request #1412 from ceph/guits-ca_pipeline

ceph-ansible-syntax: enable automatic run on PRs

ceph-ansible-syntax: enable automatic run on PRs

Automatically run this job on PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1411 from ceph/guits-ca_pipeline

ceph-ansible-prs: revert commit 54ed76efcf9

ceph-ansible-prs: revert commit 54ed76efcf9

This approach doesn't work as expected, it skips almost all jobs.
Setting back the condition in the build script until we figure out
something better.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1410 from ceph/guits-ca_pipeline

ceph-ansible-prs: remove non breaking space

ceph-ansible-prs: remove non breaking space

remove non breaking space introduced by 54ed76efcf9b956079daf22669dbfc14b6114224

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1409 from ceph/guits-ca_pipeline

ceph-ansible-prs: don't display podman job on github on stable-3.2

ceph-ansible-prs: don't display podman job on github on stable-3.2

This should avoid to display this job on the github status.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

teuthology: add more admins to allow testing pr

Since only one of the admins responds (gregsfortytwo) and
other do not react to new PRs at all, could please add
me (kshtsk) and Nathan (smithfarm) to admin list so we
can aprove pull requests testing for teuthology.

Merge pull request #1407 from ceph/guits-ca_pipeline

ceph-ansible: remove stable-3.1 testing

ceph-ansible: don't run podman on stable-3.2

skip podman job when target branch is stable-3.2 since it doesn't
support podman.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible-prs: specify which tox config file is used

When throwing this error, the name can be different according to the
scenario name.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible: remove nightly jobs on stable-3.1

stable-3.1 has no more engineering efforts.
Let's remove testing on this branch.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1406 from ceph/guits-ca_pipeline

ceph-ansible: remove pipeline

ceph-ansible: remove pipeline

The current pipeline is time consuming, it takes ~3h to complete all phases.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1405 from rzarzynski/wip-crimson-use-release-build-type

ceph-perf-pull-requests: use Release builds for crimson perf testing.

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: use Release builds for crimson perf testing.

This patch is mostly intended to build Seastar without `SEASTAR_DEBUG`
flag to avoid impact on performance. There is actually a warnining in
`src/core/app-template.cc`:

  ```cpp
  #ifdef SEASTAR_DEBUG
      fmt::print("WARNING: debug mode. Not for benchmarking or production\n");
  #endif
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osc: report osc build release correctly

This patch addresses the issue when for osc build
it has been reporting always a distro version of
the host system for any branch. However the target
release should be reported to shaman instead.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>

Merge pull request #1404 from tchaikov/wip-el8

ceph-dev-new-*: build for CentOS8

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-dev-new-build/build/setup_rpm: install python2 for el7

otherwise we will have

Traceback (most recent call last):
  File "/tmp/venv.pAISkJ09vG/bin/chacractl", line 6, in <module>
    main.ChacraCtl()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 38, in __init__
    self.main(argv)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 82, in main
    parser.dispatch()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/tambo/dispatcher.py", line 21, in dispatch
    result = instance.main()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/api/exists.py", line 62, in main
    return self.head(url)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/util.py", line 65, in inner_wrapper
    for counter in xrange(times):
NameError: name 'xrange' is not defined

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: do not 'cd ceph-*'

on cenos8, we will have

$ cd ceph-15.0.0-5635-gda7c5f3 ceph-15.0.0-5635-gda7c5f3.tar.bz2 ceph-15.0.0-5635-gda7c5f3.tar.gz
/tmp/jenkins8338985937861239310.sh: line 1055: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-*: build for centos8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: enable PowerTools repo for EL8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: get RELEASE early

so we can use it when enabling more repos

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1401 from tchaikov/wip-python3

ceph-*-build: s/python36-devel/python3-devel/

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-*/config: add "shalow-clone: true"

for faster git clone

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-*/config/definition: add "honor-refspec: true"

for faster clone

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pull-requests-arm64: use jjb settings instead of raw xml

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-*/config/definition: only pull the PR under test

see
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md#creating-a-job

no need to pull all PR references of the remote repo. it'd be faster
just pull the references of a single PR. actually, what we need is but

- refs/pull/${pr}/head
- refs/pull/${pr}/merge

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1403 from ceph/wip-fix-kernel

kernel-trigger: Build any wip branch

Merge pull request #1402 from ceph/wip-kernel-statuses

kernel: Report build statuses to shaman

kernel-trigger: Build any wip branch

I honestly don't know why this works and origin/wip* didn't but here we are.

Signed-off-by: David Galloway <dgallowa@redhat.com>

kernel: Report build status to shaman

Signed-off-by: David Galloway <dgallowa@redhat.com>

kernel: Update shaman status on failed build

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-*-build: s/python36-devel/python3-devel/

to silence the warning of

Package python36-devel is obsoleted by python3-devel, trying to install
python3-devel-3.6.8-10.el7.x86_64

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1399 from ceph/wip-kernel-usercopy-and-ceph-labels

kernel: enable CONFIG_HARDENED_USERCOPY and CONFIG_CEPH_FS_SECURITY_LABEL

kernel: enable CONFIG_CEPH_FS_SECURITY_LABEL

New in kernel 5.3.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: enable CONFIG_HARDENED_USERCOPY

This is something we had to work around in libceph, see linux.git
commit 7e241f647dc7 ("libceph: fall back to sendmsg for slab pages").
It is enabled by default in many distros.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1398 from ceph/wip-build-wips-in-ceph-client

kernel: update the trigger list

kernel: update the trigger list

We didn't trigger on wip* branches like other ceph projects because
there was a lot of old branches matching that mask in the repository.
Now that they are cleaned up, drop the workarounds (testing*/master*)
and add wip*.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1397 from ceph/repo-cleanup

clean up shaman repos left behind after a build

tcmu-runner: remove custom shaman repos added after a build

Signed-off-by: Alfredo Deza <adeza@redhat.com>

nfs-ganesha-stable: remove custom repo/list configuration files added

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1396 from ceph/wip-centos-sclo-rh-source-skip

ceph-*-build: skip unavailable repo when yum-builddep

ceph-*-build: skip unavailable repo when yum-builddep

As centos-sclo-rh-source leads us to 404 at this moment. and we are not
using the source repo for building ceph. so we can just skip any
unavailable repo.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #1394 from ceph/revert-scm-changes

Revert "*/scm: only pull the PR under test"

Revert "*/scm: only pull the PR under test"

This reverts commit 4eae5698a6a1383c69338ec7781e4cee6c087a9f.

It breaks all pull-request jobs for ceph-volume at least, haven't been
able to investigate all other affected

Merge pull request #1393 from tchaikov/wip-crimson-perf

ceph-perf-pull-requests: add ceph-perf-pull-requests

Merge pull request #1392 from tchaikov/wip-scm-refspec

*/scm: only pull the PR under test

ceph-perf-pull-requests: add ceph-perf-pull-requests

for running cbt test against PR and master, then comparing them

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1378 from dmick/wip-dmick

Add container builds using ceph-container.git for centos7

Merge pull request #1391 from hjensas/issue#1390

pr-syntax-check - install all requirements

pr-syntax-check - install all requirements

This change removes the grep for 'ansible' when installing
requirements in ceph-ansible-pr-syntax-check. The grep
cause onlye the ansible requirement to be installed, ignoring
the other packages. This causes an issue in PR[1] which
introduce a filter plugin that depend on the netaddr package.

[1] https://github.com/ceph/ceph-ansible/pull/4339

Closes: #1390
Signed-off-by: Harald Jensås <hjensas@redhat.com>

ceph-build-pull-requests: pass --config-xml to "jenkins-jobs test"

to silence warnings like

WARNING:jenkins_jobs.cli.subcommand.test:(Deprecated) The default output
behavior of `jenkins-jobs test` when given the --output flag will change
in JJB 3.0. Instead of writing jobs to OUTPUT/jobname; they will be
written to OUTPUT/jobname/config.xml. The new behavior can be enabled by
the passing `--config-xml` parameter.

also redirect the test result to /dev/null, otherwise the test will fail
because the output directory may already exist.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-build-pull-requests: ping jjb to 3.0.2

bump jjb's version

Signed-off-by: Kefu Chai <kchai@redhat.com>

*/scm: only pull the PR under test

see
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md#creating-a-job

no need to pull all PR references of the remote repo. it'd be faster
just pull the references of a single PR. actually, what we need is but

- refs/pull/${pr}/head
- refs/pull/${pr}/merge

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1389 from tchaikov/wip-python-rpm-macro

ceph-*build: install python36-devel for rpm macros

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-*build: install python36-devel for rpm macros

see also 100a0fb5
see also https://github.com/ceph/ceph/pull/30190

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>

Conditionally add container builds for wip- branches

New parameter CI_CONTAINER ('true' or 'false') controls behavior;
if set to 'true', ceph-container.git will be used to build a centos7
daemon-base default flavor container and push it to a repo server.
This requires waiting until the shaman repo is built.

New parameters CONTAINER_REPO_{HOSTNAME, ORGANIZATION} set where
to push the resultant container.

If repo doesn't become ready or if the actual build/push fails, don't
fail the overall build.  Maybe we can come up with a better way
to indicate this than just a log notation

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-dev-new-build: add quay.io credentials for cephci repo

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-dev-new-*: add CONTAINER_BRANCH and check it out

In case a container build is needed, check out $CONTAINER_BRANCH
of ceph-container.git to use to build it

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1388 from ceph/prune-fix

scripts: use a loop instead of globbing directly

scripts: use a loop instead of globbing directly

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1387 from tchaikov/wip=40646

Revert "ceph-*build/build/setup_rpm: downgrade to gcc-c++-8.2.1"

Revert "ceph-*build/build/setup_rpm: downgrade to gcc-c++-8.2.1"

This reverts commit cf2c26b181a86738e2a8bb17545f45d6aa1c7c45.

since https://bugzilla.redhat.com/show_bug.cgi?id=1726630 has been
fixed. and we have devtoolset-8-gcc-c++-8.3.1-3.1.el7.x86_64.rpm.

no reason to stick to devtoolset-8-gcc-c++-8.2.1-3 anymore.

Fixes: https://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1386 from ceph/prune-fix

scripts: can't use globbing with quotes

scripts: can't use globbing with quotes

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1385 from ceph/prune-fix

scripts: do not specify directories with find -delete, rm the path

Reviewed-by: Kefu Chai <kchai@redhat.com>

scripts: do not specify directories with find -delete, rm the path

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1383 from ceph/prune-fix

scripts: use find's -delete option to prevent 'argument too long'

Reviewed-by: Kefu Chai <kchai@redhat.com>

scripts: use find's -delete option to prevent 'argument too long'

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1382 from ceph/fix-cv-tox

ceph-volume-pr: remove escape which doubles the curly brackets

ceph-volume-pr: remove escape which doubles the curly brackes resulting in syntax errors

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1381 from ceph/submodules-wipe

ceph-pr-submodules: wipe the workspace to prevent repo corruption

ceph-pr-submodules: wipe the workspace to prevent repo corruption

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1380 from ceph/ceph-signed-wipe

ceph-pr-commits: wipe workspace to prevent repo corruption

ceph-pr-commits: wipe workspace to prevent repo corruption

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1377 from ceph/ceph-volume-cleanup

ceph-volume: cleanup workspace before running tests

ceph-volume use wipe-workspace in Git to cleanup before running

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1379 from ceph/wip-signed-master

ceph-pr-commits: add trigger phrase

ceph-pr-commits: add trigger phrase

When Jenkins fails to clone a branch, signed-off checks will fail. If
all other checks have passed, there's no reason to force a full run of
Jenkins jobs again on that PR. This PR introduces `jenkins test signed`
trigger phrase to allow re-running the pr-commit check.

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #1376 from ceph/wip-enable-leap15

enable leap15 for ceph-dev-trigger and ceph-dev-new-trigger