Merge pull request #1448 from callithea/wip-ceph-dashboard-pull-requests-dependencies

ceph-dashboard-pull-requests: Update dependencies

Reviewed-by: Kefu Chai kchai@redhat.com

Merge pull request #1452 from ceph/centos8-dev-builds

ceph-dev-new-trigger: build centos8 from ceph-ci

ceph-dev-new-trigger: build centos8 from ceph-ci

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1451 from toabctl/ceph-dev-build-opensuse-15.2

ceph-dev-build: Use correct OBS repo octopus on openSUSE Leap

ceph-dev-build: Use correct OBS repo octopus on openSUSE Leap

We want to build against openSUSE Leap 15.2 for octopus. So use the
correct repo.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #1449 from toabctl/chacra-pull-requests-ansible-fix

chacra-pull-requests: Replace ansible sudo with become

chacra-pull-requests: Replace ansible sudo with become

"become" is the correct ansible keyword for priv escalation in newer
versions. So use that.
This will fix at least the ansible-playbook setup.yml call which
previously failed.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

ceph-dashboard-pull-requests: Update dependencies

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1416 from ceph/wip-no-c8-lum

Build mimic/nautilus on CentOS8

Merge pull request #1447 from ceph/guits-add_job

ceph-ansible-prs: add filestore_to_bluestore job

ceph-ansible-prs: add filestore_to_bluestore job

This commit adds a new job for testing the filestore-to-bluestore.yml
playbook.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1446 from ceph/guits-ceph-disk_3.2

ceph-ansible-prs: introduce back ceph-disk testing (3.2 only)

ceph-ansible-prs: introduce back ceph-disk testing (3.2 only)

Related ceph-ansible PR: ceph/ceph-ansible#4824

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1445 from ceph/pin_ansible_lint_on_stable_3.2

ceph-ansible-pr-syntax-check: pin ansible-lint

ceph-ansible-pr-syntax-check: pin ansible-lint

Latest ansible-lint release (4.2.0) drops ansible 2.6 support so the
syntax-check job isn't compatible with the ceph-ansible stable-3.2
branch which uses ansible 2.6.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1443 from rhcs-dashboard/dashboard-nightly

project config: ceph-api-nightly

Reviewed-by: Kefu Chai kchai@redhat.com
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

project config: ceph-api-nightly

Project: run backend API + E2E tests on a daily basis
against master & nautilus.

No mention to dashboard in project name as API is key
to the whole ceph project.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #1444 from ceph/ceph_ansible_doc_stable_4.0

ceph-ansible-docs: build doc for stable-4.0

ceph-ansible-docs: build doc for stable-4.0

Let's build the upstream documentation for stable-4.0

Closes: https://github.com/ceph/ceph-ansible/issues/4794
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1442 from badone/wip-disable-lizardfs-fsal-follow-up

nfs-ganesha: Remove lizardFSAL patch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1441 from ceph/revert-1440-wip-disable-lizardfs-fsal-follow-up

Revert "nfs-ganesha: dpkg commit source after lizardFSAL patch"

nfs-ganesha: Remove lizardFSAL patch

This is a follow-up to 888793c4 to 'commit' the patch.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Revert "nfs-ganesha: dpkg commit source after lizardFSAL patch"

Merge pull request #1440 from badone/wip-disable-lizardfs-fsal-follow-up

nfs-ganesha: dpkg commit source after lizardFSAL patch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

nfs-ganesha: dpkg commit source after lizardFSAL patch

This is a follow-up to 888793c4 to 'commit' the patch.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #1439 from badone/wip-disable-lizardfs-fsal

nfs-ganesha: Disable LizardFS FSAL for debian

Reviewed-by: Jeff Layton <jlayton@redhat.com>

nfs-ganesha: Disable LizardFS FSAL for debian

LizardFS FSAL has been added in [1] and it's enabled by default. This
is currently failing in the CI for Bionic. 338d18e resolved this for rpm
builds, but not for debs.

CMake Error at CMakeLists.txt:595 (message):
    STRICT_PACKAGE: Cannot find LizardFS client lib.  Disabling lizardfs
    fsal

This patch configures the USE_FSAL_LIZARDFS option to OFF.

[1] nfs-ganesha/nfs-ganesha@884ec97

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #1438 from ceph/guits-add_auto_discovery_prs

ceph-ansible-prs: add lvm_auto_discovery job

ceph-ansible-prs: add lvm_auto_discovery job

Add the possibility to trigger `lvm_auto_discovery` job on ceph-ansible PR

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1437 from dmick/wip-quay-short-tags

quay-pruner/build/prune-quay.py: handle new "short tags"

Merge pull request #1436 from ceph/ceph_ansible_prs_docs_only_condition

ceph-ansible-prs: update docs skip condition

quay-pruner/build/prune-quay.py: handle new "short tags"

There are two new tags proposed for each image: "branchname" and "full
sha1".  Branchname is currently being created.  When pruning image tags,
we need to prune the associated short tags as well.

This also fixes a current bug where, because branchname tags are being
created, the current unaware prune script deletes them because it
treats "can't parse tagname" as equivalent to "can't find in shaman".

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-ansible-prs: update docs skip condition

When a PR can't be merged due to conflicts then we can't check if the
last commit is a merge commit.

Commit message: "Merge xxxx into xxxx"

In this case we only checkout the latest commit in the PR without the
previous one if any.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1435 from dmick/wip-ceph-ci

ceph-dev-new, quay-pruner: cephci/daemon-base -> ceph-ci/ceph

ceph-dev-new, quay-pruner: cephci/daemon-base -> ceph-ci/ceph

name change requested by @liewegas

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1430 from dmick/quay-pruner

Add "quay-pruner" to prune stale quay.io container images

Add "quay-pruner" to prune stale quay.io container images

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #1434 from ceph/wip-waitlonger

sepia-fog-images: Wait longer for lab to quiet down

sepia-fog-images: Wait longer for lab to quiet down

30min wasn't enough for jobs to stop scheduling Deploy tasks.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1433 from ceph/bionic-updates-2

scripts: use main and universe directly

scripts: use main and universe directly

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1432 from ceph/bionic-updates

scripts: add bionic-updates to pbuilderrc for gcc 7.4.0

scripts: add bionic-updates to pbuilderrc for gcc 7.4.0

Fixes: https://tracker.ceph.com/issues/42596
Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1428 from ceph/ceph_ansible_prs_add_mon_osd

ceph-ansible-prs: re-add add_xxx jobs

ceph-ansible-prs: re-add add_xxx jobs

This includes:
  - add_mdss
  - add_mgrs
  - add_mons
  - add_osds
  - add_rbdmirrors
  - add_rgws

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1429 from ceph/fix_ansible_2.9_syntax

ansible: replace sudo by become

ansible: replace sudo by become

Since ansible 2.9 sudo statements have been removed and we should use
become instead.

"All previously deprecated sudo/su and module locale global settings have
been removed." [1]

[1] https://github.com/ansible/ansible/blob/stable-2.9/changelogs/CHANGELOG-v2.9.rst

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1427 from toabctl/paddles-pr-testing

paddles-pull-requests: adds a job to test paddles PRs

paddles-pull-requests: adds a job to test paddles PRs

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #1426 from tchaikov/ceph-perf-pull-requests/pullid

ceph-*-pull-requests*: use ghprbPullId to define branches

ceph-*-pull-requests*: use ghprbPullId to define branches

* add "ghprbPullId" param, simpler to use than "sha1"
* instead of using "sha1", use "ghprbPullId" to define branches to build,
  more consistent this way. as we pull the refs using
  `+refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*`
  and after the change, the ref to build is specified using
  `origin/pr/${ghprbPullId}/merge`. also, by removing "sha1",
  we can specify a single parameter when rebuilding/building a branch
  on jenkins instead of two parameters.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1425 from ceph/cv-rm-xenial

ceph-volume-ansible-prs: remove xenial from PR testing

Merge pull request #1422 from ceph/wip-no-more-dupes

ansible: Just use the example

Merge pull request #1375 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: revert to production chacra instance

Merge pull request #1424 from ceph/wip-pr-docs

ceph-pr-render-docs: Clean up old doc renders

ceph-pr-render-docs: Clean up old doc renders

Since these don't get stored on the larger volume attached to docs.ceph.com, they can fill up the root disk without being cleaned up.

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-volume-ansible-prs: remove xenial from PR testing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1423 from ceph/wip-arm64-centos

ceph-pull-requests-arm64: Don't make check on CentOS8 yet

ceph-pull-requests-arm64: Don't make check on CentOS8 yet

Until deps are in epel8 or we figure out an alternative

Signed-off-by: David Galloway <dgallowa@redhat.com>

ansible: Just use the example

In our setup, the example is the canonical playbook that works.  If, for some reason, ceph-build.git is used by someone else, they can rm the symlink and create their own `ansible/slave.yml` but for our purposes, there's no need to keep separate playbooks as it just creates confusion when modifying the playbooks.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1421 from ceph/wip-ansible-examples-leap15

ansible/examples/slave: add leap15

ansible/examples/slave: add leap15

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1420 from ceph/wip-mock-slave

Add jenkins-build user to mock group during slave setup

Add jenkins-build user to mock group during slave setup

Instead of during each individual job

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1415 from ceph/wip-centos8

ansible: Update playbooks for CentOS8 compatibility

Merge pull request #1418 from ceph/add_ca_ooo

ceph-ansible-prs: run ooo_collocation on container only

ceph-ansible-prs: run ooo_collocation on container only

Add a dedicated project for ooo_collocation since we don't want it to
run in non_container context.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1417 from ceph/add_ca_ooo

ceph-ansible-prs: re-add ooo_collocation

ceph-ansible-prs: re-add ooo_collocation

This commit adds the ooo_collocation that were removed recently.
Related ceph-ansible PR : https://github.com/ceph/ceph-ansible/pull/4401

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Build mimic & nautilus on CentOS8

Signed-off-by: David Galloway <dgallowa@redhat.com>

ceph-dev-new: Don't built luminous on CentOS8

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1414 from tchaikov/wip-ceph-pr-submodules

ceph-pr-submodules: do not use shallow-clone

ansible: Update playbooks for CentOS8 compatibility

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1413 from ceph/ceph_ansible_triggered_jobs

ceph-ansible-prs: re-add shrink/rgw_multisite jobs

ceph-pr-submodules: do not use shallow-clone

we use
```
git diff --submodule=log origin/$ghprbTargetBranch...$ghprbActualCommit
```

for checking the differences between the delta since their common
ancestor, so we have to do a non-shallow clone for accessing the
common ancestor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-ansible-prs: re-add shrink/rgw_multisite jobs

Since the pipeline refactoring we're not able to trigger manually the
shrink and rgw_multisite jobs. The shrink jobs were only configured for
ubuntu and rgw_multisite was missing.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1395 from tchaikov/wip-scm-faster-clone

*/scm: faster clone

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1400 from ceph/wip-report-osc-release

osc: report osc build release correctly

Merge pull request #1408 from ceph/wip-add-teuthology-admins

teuthology: add more admins to allow testing pr

Merge pull request #1412 from ceph/guits-ca_pipeline

ceph-ansible-syntax: enable automatic run on PRs

ceph-ansible-syntax: enable automatic run on PRs

Automatically run this job on PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1411 from ceph/guits-ca_pipeline

ceph-ansible-prs: revert commit 54ed76efcf9

ceph-ansible-prs: revert commit 54ed76efcf9

This approach doesn't work as expected, it skips almost all jobs.
Setting back the condition in the build script until we figure out
something better.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1410 from ceph/guits-ca_pipeline

ceph-ansible-prs: remove non breaking space

ceph-ansible-prs: remove non breaking space

remove non breaking space introduced by 54ed76efcf9b956079daf22669dbfc14b6114224

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1409 from ceph/guits-ca_pipeline

ceph-ansible-prs: don't display podman job on github on stable-3.2

ceph-ansible-prs: don't display podman job on github on stable-3.2

This should avoid to display this job on the github status.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

teuthology: add more admins to allow testing pr

Since only one of the admins responds (gregsfortytwo) and
other do not react to new PRs at all, could please add
me (kshtsk) and Nathan (smithfarm) to admin list so we
can aprove pull requests testing for teuthology.

Merge pull request #1407 from ceph/guits-ca_pipeline

ceph-ansible: remove stable-3.1 testing

ceph-ansible: don't run podman on stable-3.2

skip podman job when target branch is stable-3.2 since it doesn't
support podman.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible-prs: specify which tox config file is used

When throwing this error, the name can be different according to the
scenario name.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible: remove nightly jobs on stable-3.1

stable-3.1 has no more engineering efforts.
Let's remove testing on this branch.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1406 from ceph/guits-ca_pipeline

ceph-ansible: remove pipeline

ceph-ansible: remove pipeline

The current pipeline is time consuming, it takes ~3h to complete all phases.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1405 from rzarzynski/wip-crimson-use-release-build-type

ceph-perf-pull-requests: use Release builds for crimson perf testing.

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-perf-pull-requests: use Release builds for crimson perf testing.

This patch is mostly intended to build Seastar without `SEASTAR_DEBUG`
flag to avoid impact on performance. There is actually a warnining in
`src/core/app-template.cc`:

  ```cpp
  #ifdef SEASTAR_DEBUG
      fmt::print("WARNING: debug mode. Not for benchmarking or production\n");
  #endif
  ```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osc: report osc build release correctly

This patch addresses the issue when for osc build
it has been reporting always a distro version of
the host system for any branch. However the target
release should be reported to shaman instead.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>

Merge pull request #1404 from tchaikov/wip-el8

ceph-dev-new-*: build for CentOS8

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-dev-new-build/build/setup_rpm: install python2 for el7

otherwise we will have

Traceback (most recent call last):
  File "/tmp/venv.pAISkJ09vG/bin/chacractl", line 6, in <module>
    main.ChacraCtl()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 38, in __init__
    self.main(argv)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/main.py", line 82, in main
    parser.dispatch()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/tambo/dispatcher.py", line 21, in dispatch
    result = instance.main()
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/api/exists.py", line 62, in main
    return self.head(url)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/decorators.py", line 68, in newfunc
    return f(*a, **kw)
  File "/tmp/venv.pAISkJ09vG/lib/python3.6/site-packages/chacractl/util.py", line 65, in inner_wrapper
    for counter in xrange(times):
NameError: name 'xrange' is not defined

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-build: do not 'cd ceph-*'

on cenos8, we will have

$ cd ceph-15.0.0-5635-gda7c5f3 ceph-15.0.0-5635-gda7c5f3.tar.bz2 ceph-15.0.0-5635-gda7c5f3.tar.gz
/tmp/jenkins8338985937861239310.sh: line 1055: cd: too many arguments

Signed-off-by: Kefu Chai <kchai@redhat.com>