Merge pull request #1430 from dmick/quay-pruner

Add "quay-pruner" to prune stale quay.io container images

Add "quay-pruner" to prune stale quay.io container images

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1434 from ceph/wip-waitlonger

sepia-fog-images: Wait longer for lab to quiet down

sepia-fog-images: Wait longer for lab to quiet down

30min wasn't enough for jobs to stop scheduling Deploy tasks.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1433 from ceph/bionic-updates-2

scripts: use main and universe directly

scripts: use main and universe directly

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1432 from ceph/bionic-updates

scripts: add bionic-updates to pbuilderrc for gcc 7.4.0

scripts: add bionic-updates to pbuilderrc for gcc 7.4.0

Fixes: https://tracker.ceph.com/issues/42596
Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1428 from ceph/ceph_ansible_prs_add_mon_osd

ceph-ansible-prs: re-add add_xxx jobs

ceph-ansible-prs: re-add add_xxx jobs

This includes:
  - add_mdss
  - add_mgrs
  - add_mons
  - add_osds
  - add_rbdmirrors
  - add_rgws

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1429 from ceph/fix_ansible_2.9_syntax

ansible: replace sudo by become

ansible: replace sudo by become

Since ansible 2.9 sudo statements have been removed and we should use
become instead.

"All previously deprecated sudo/su and module locale global settings have
been removed." [1]

[1] https://github.com/ansible/ansible/blob/stable-2.9/changelogs/CHANGELOG-v2.9.rst

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1427 from toabctl/paddles-pr-testing

paddles-pull-requests: adds a job to test paddles PRs

paddles-pull-requests: adds a job to test paddles PRs

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #1426 from tchaikov/ceph-perf-pull-requests/pullid

ceph-*-pull-requests*: use ghprbPullId to define branches

ceph-*-pull-requests*: use ghprbPullId to define branches

* add "ghprbPullId" param, simpler to use than "sha1"
* instead of using "sha1", use "ghprbPullId" to define branches to build,
  more consistent this way. as we pull the refs using
  `+refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*`
  and after the change, the ref to build is specified using
  `origin/pr/${ghprbPullId}/merge`. also, by removing "sha1",
  we can specify a single parameter when rebuilding/building a branch
  on jenkins instead of two parameters.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1425 from ceph/cv-rm-xenial

ceph-volume-ansible-prs: remove xenial from PR testing

Merge pull request #1422 from ceph/wip-no-more-dupes

ansible: Just use the example

Merge pull request #1375 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: revert to production chacra instance

Merge pull request #1424 from ceph/wip-pr-docs

ceph-pr-render-docs: Clean up old doc renders

ceph-pr-render-docs: Clean up old doc renders

Since these don't get stored on the larger volume attached to docs.ceph.com, they can fill up the root disk without being cleaned up.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-volume-ansible-prs: remove xenial from PR testing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1423 from ceph/wip-arm64-centos

ceph-pull-requests-arm64: Don't make check on CentOS8 yet

ceph-pull-requests-arm64: Don't make check on CentOS8 yet

Until deps are in epel8 or we figure out an alternative

Signed-off-by: David Galloway <dgallowa@redhat.com>

ansible: Just use the example

In our setup, the example is the canonical playbook that works.  If, for some reason, ceph-build.git is used by someone else, they can rm the symlink and create their own `ansible/slave.yml` but for our purposes, there's no need to keep separate playbooks as it just creates confusion when modifying the playbooks.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1421 from ceph/wip-ansible-examples-leap15

ansible/examples/slave: add leap15

ansible/examples/slave: add leap15

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1420 from ceph/wip-mock-slave

Add jenkins-build user to mock group during slave setup

Add jenkins-build user to mock group during slave setup

Instead of during each individual job

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1415 from ceph/wip-centos8

ansible: Update playbooks for CentOS8 compatibility

Merge pull request #1418 from ceph/add_ca_ooo

ceph-ansible-prs: run ooo_collocation on container only

ceph-ansible-prs: run ooo_collocation on container only

Add a dedicated project for ooo_collocation since we don't want it to
run in non_container context.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1417 from ceph/add_ca_ooo

ceph-ansible-prs: re-add ooo_collocation

ceph-ansible-prs: re-add ooo_collocation

This commit adds the ooo_collocation that were removed recently.
Related ceph-ansible PR : https://github.com/ceph/ceph-ansible/pull/4401

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1414 from tchaikov/wip-ceph-pr-submodules

ceph-pr-submodules: do not use shallow-clone

ansible: Update playbooks for CentOS8 compatibility

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1413 from ceph/ceph_ansible_triggered_jobs

ceph-ansible-prs: re-add shrink/rgw_multisite jobs

ceph-pr-submodules: do not use shallow-clone

we use
```
git diff --submodule=log origin/$ghprbTargetBranch...$ghprbActualCommit
```

for checking the differences between the delta since their common
ancestor, so we have to do a non-shallow clone for accessing the
common ancestor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-ansible-prs: re-add shrink/rgw_multisite jobs

Since the pipeline refactoring we're not able to trigger manually the
shrink and rgw_multisite jobs. The shrink jobs were only configured for
ubuntu and rgw_multisite was missing.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1395 from tchaikov/wip-scm-faster-clone

*/scm: faster clone

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1400 from ceph/wip-report-osc-release

osc: report osc build release correctly

Merge pull request #1408 from ceph/wip-add-teuthology-admins

teuthology: add more admins to allow testing pr

Merge pull request #1412 from ceph/guits-ca_pipeline

ceph-ansible-syntax: enable automatic run on PRs

ceph-ansible-syntax: enable automatic run on PRs

Automatically run this job on PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1411 from ceph/guits-ca_pipeline

ceph-ansible-prs: revert commit 54ed76efcf9

ceph-ansible-prs: revert commit 54ed76efcf9

This approach doesn't work as expected, it skips almost all jobs.
Setting back the condition in the build script until we figure out
something better.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1410 from ceph/guits-ca_pipeline

ceph-ansible-prs: remove non breaking space

ceph-ansible-prs: remove non breaking space

remove non breaking space introduced by 54ed76efcf9b956079daf22669dbfc14b6114224

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1409 from ceph/guits-ca_pipeline

ceph-ansible-prs: don't display podman job on github on stable-3.2

ceph-ansible-prs: don't display podman job on github on stable-3.2

This should avoid to display this job on the github status.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

teuthology: add more admins to allow testing pr

Since only one of the admins responds (gregsfortytwo) and
other do not react to new PRs at all, could please add
me (kshtsk) and Nathan (smithfarm) to admin list so we
can aprove pull requests testing for teuthology.

Merge pull request #1407 from ceph/guits-ca_pipeline

ceph-ansible: remove stable-3.1 testing

ceph-ansible: don't run podman on stable-3.2

skip podman job when target branch is stable-3.2 since it doesn't
support podman.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible-prs: specify which tox config file is used

When throwing this error, the name can be different according to the
scenario name.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible: remove nightly jobs on stable-3.1

stable-3.1 has no more engineering efforts.
Let's remove testing on this branch.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1406 from ceph/guits-ca_pipeline

ceph-ansible: remove pipeline

ceph-ansible: remove pipeline

The current pipeline is time consuming, it takes ~3h to complete all phases.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1405 from rzarzynski/wip-crimson-use-release-build-type

ceph-perf-pull-requests: use Release builds for crimson perf testing.

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: use Release builds for crimson perf testing.

This patch is mostly intended to build Seastar without `SEASTAR_DEBUG`
flag to avoid impact on performance. There is actually a warnining in
`src/core/app-template.cc`:

  ```cpp
  #ifdef SEASTAR_DEBUG
      fmt::print("WARNING: debug mode. Not for benchmarking or production\n");
  #endif
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osc: report osc build release correctly

This patch addresses the issue when for osc build
it has been reporting always a distro version of
the host system for any branch. However the target
release should be reported to shaman instead.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>

Merge pull request #1404 from tchaikov/wip-el8

ceph-dev-new-*: build for CentOS8

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-dev-new-build/build/setup_rpm: install python2 for el7

otherwise we will have

Traceback (most recent call last):
  File "/tmp/venv.pAISkJ09vG/bin/chacractl", line 6, in <module>
    main.ChacraCtl()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 38, in __init__
    self.main(argv)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 82, in main
    parser.dispatch()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/tambo/dispatcher.py", line 21, in dispatch
    result = instance.main()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/api/exists.py", line 62, in main
    return self.head(url)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/util.py", line 65, in inner_wrapper
    for counter in xrange(times):
NameError: name 'xrange' is not defined

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: do not 'cd ceph-*'

on cenos8, we will have

$ cd ceph-15.0.0-5635-gda7c5f3 ceph-15.0.0-5635-gda7c5f3.tar.bz2 ceph-15.0.0-5635-gda7c5f3.tar.gz
/tmp/jenkins8338985937861239310.sh: line 1055: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-*: build for centos8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: enable PowerTools repo for EL8

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: get RELEASE early

so we can use it when enabling more repos

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1401 from tchaikov/wip-python3

ceph-*-build: s/python36-devel/python3-devel/

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-*/config: add "shalow-clone: true"

for faster git clone

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-*/config/definition: add "honor-refspec: true"

for faster clone

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pull-requests-arm64: use jjb settings instead of raw xml

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-*/config/definition: only pull the PR under test

see
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md#creating-a-job

no need to pull all PR references of the remote repo. it'd be faster
just pull the references of a single PR. actually, what we need is but

- refs/pull/${pr}/head
- refs/pull/${pr}/merge

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1403 from ceph/wip-fix-kernel

kernel-trigger: Build any wip branch

Merge pull request #1402 from ceph/wip-kernel-statuses

kernel: Report build statuses to shaman

kernel-trigger: Build any wip branch

I honestly don't know why this works and origin/wip* didn't but here we are.

Signed-off-by: David Galloway <dgallowa@redhat.com>

kernel: Report build status to shaman

Signed-off-by: David Galloway <dgallowa@redhat.com>

kernel: Update shaman status on failed build

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-*-build: s/python36-devel/python3-devel/

to silence the warning of

Package python36-devel is obsoleted by python3-devel, trying to install
python3-devel-3.6.8-10.el7.x86_64

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1399 from ceph/wip-kernel-usercopy-and-ceph-labels

kernel: enable CONFIG_HARDENED_USERCOPY and CONFIG_CEPH_FS_SECURITY_LABEL

kernel: enable CONFIG_CEPH_FS_SECURITY_LABEL

New in kernel 5.3.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: enable CONFIG_HARDENED_USERCOPY

This is something we had to work around in libceph, see linux.git
commit 7e241f647dc7 ("libceph: fall back to sendmsg for slab pages").
It is enabled by default in many distros.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1398 from ceph/wip-build-wips-in-ceph-client

kernel: update the trigger list

kernel: update the trigger list

We didn't trigger on wip* branches like other ceph projects because
there was a lot of old branches matching that mask in the repository.
Now that they are cleaned up, drop the workarounds (testing*/master*)
and add wip*.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #1397 from ceph/repo-cleanup

clean up shaman repos left behind after a build

tcmu-runner: remove custom shaman repos added after a build

Signed-off-by: Alfredo Deza <adeza@redhat.com>

nfs-ganesha-stable: remove custom repo/list configuration files added

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1396 from ceph/wip-centos-sclo-rh-source-skip

ceph-*-build: skip unavailable repo when yum-builddep

ceph-*-build: skip unavailable repo when yum-builddep

As centos-sclo-rh-source leads us to 404 at this moment. and we are not
using the source repo for building ceph. so we can just skip any
unavailable repo.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #1394 from ceph/revert-scm-changes

Revert "*/scm: only pull the PR under test"

Revert "*/scm: only pull the PR under test"

This reverts commit 4eae5698a6a1383c69338ec7781e4cee6c087a9f.

It breaks all pull-request jobs for ceph-volume at least, haven't been
able to investigate all other affected

Merge pull request #1393 from tchaikov/wip-crimson-perf

ceph-perf-pull-requests: add ceph-perf-pull-requests

Merge pull request #1392 from tchaikov/wip-scm-refspec

*/scm: only pull the PR under test

ceph-perf-pull-requests: add ceph-perf-pull-requests

for running cbt test against PR and master, then comparing them

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1378 from dmick/wip-dmick

Add container builds using ceph-container.git for centos7

Merge pull request #1391 from hjensas/issue#1390

pr-syntax-check - install all requirements

pr-syntax-check - install all requirements

This change removes the grep for 'ansible' when installing
requirements in ceph-ansible-pr-syntax-check. The grep
cause onlye the ansible requirement to be installed, ignoring
the other packages. This causes an issue in PR[1] which
introduce a filter plugin that depend on the netaddr package.

[1] https://github.com/ceph/ceph-ansible/pull/4339

Closes: #1390
Signed-off-by: Harald Jensås <hjensas@redhat.com>

ceph-build-pull-requests: pass --config-xml to "jenkins-jobs test"

to silence warnings like

WARNING:jenkins_jobs.cli.subcommand.test:(Deprecated) The default output
behavior of `jenkins-jobs test` when given the --output flag will change
in JJB 3.0. Instead of writing jobs to OUTPUT/jobname; they will be
written to OUTPUT/jobname/config.xml. The new behavior can be enabled by
the passing `--config-xml` parameter.

also redirect the test result to /dev/null, otherwise the test will fail
because the output directory may already exist.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-build-pull-requests: ping jjb to 3.0.2

bump jjb's version

Signed-off-by: Kefu Chai <kchai@redhat.com>

*/scm: only pull the PR under test

see
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md#creating-a-job

no need to pull all PR references of the remote repo. it'd be faster
just pull the references of a single PR. actually, what we need is but

- refs/pull/${pr}/head
- refs/pull/${pr}/merge

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1389 from tchaikov/wip-python-rpm-macro

ceph-*build: install python36-devel for rpm macros

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-*build: install python36-devel for rpm macros

see also 100a0fb5
see also https://github.com/ceph/ceph/pull/30190

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>