ceph-iscsi-stable: fix revision to '-1' if building at tag

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1371 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: inject missing credentials

ceph-iscsi-stable: inject missing credentials

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1370 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: query correct chacra host for uploads

ceph-iscsi-stable: query correct chacra host for uploads

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1369 from ceph/wip-ceph-iscsi-stable

ceph-iscsi-stable: incorporate consolidated repo changes

ceph-iscsi-stable: incorporate consolidated repo changes

ceph-iscsi-config and ceph-iscsi-cli are now merged into
the ceph-iscsi repo.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1368 from ceph/wip-leap15-available-dist

leap15: add leap15 to available dist

leap15: add leap15 to available dist

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1351 from callithea/wip-ceph-dashboard-prs-backend-api

ceph-dashboard-pr-backend: add job config

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-dashboard-pr-backend: add job config

Signed-off-by: Laura Paduano <lpaduano@suse.com>
Signed-off-by: alfonsomthd <almartin@redhat.com>

Merge pull request #1365 from ceph/wip-osc-build-scripts

osc build scripts

Merge pull request #1367 from tchaikov/wip-upload-ddeb-to-chacra

ceph*build/build/build_deb: upload *.ddeb packages

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph*build/build/build_deb: upload *.ddeb packages

ubuntu names *dbgsym* packages .ddeb in hope to put them into a
separated repo. while debian just uses the plain .deb extension for them.
so, to include the dbgsym packages created by ubuntu, we need to upload
the .ddeb packages as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1343 from ceph/github-token

ceph-volume: allow passing GITHUB_TOKEN as a password to manually set github status

ceph-dev-new-build: add osc build scripts

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

ceph-dev-build: add osc build scripts

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

validate_rpm: reuse rpm validation from another project

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

osc: add osc related scripts:
- validate_osc
- setup_osc
- build_osc
Supposed for adding them later to ceph-dev-new-build or ceph-dev-build

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1366 from ceph/wip-jb-group

slave.yml: Add jenkins-build user to jenkins-build group

slave.yml: Add jenkins-build user to jenkins-build group

On ubuntu, at least, with the addition of the "create a jenkins-build group" task, the creation of the user wasn't adding the user to the group.  I discovered this by seeing a docker command fail after the socket was 'chgrp'ed to jenkins-build

```
+ sudo chgrp jenkins-build /var/run/docker.sock
+ do_clean
+ make clean.all
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
```

I logged in to the slave and confirmed this was the case.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1361 from ceph/wip-release-from-version

build_utils: add release from version

build_utils: use virtualenv -p python2.7 instead virtualenv

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

build_utils: fix get_rpm_dist openSUSE detection

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

build_utils: add release from version function

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1359 from ceph/wip-leap15-slave

ansible/slave: add opensuse leap15 osc support

ansible/slave: add opensuse leap15 osc support

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1360 from ceph/wip-tserlin-nfs-ganesha-disable-xenial

nfs-ganesha: don't build on xenial

nfs-ganesha: don't build on xenial

Ceph no longer builds its master branch on Xenial. See:
https://github.com/ceph/ceph-build/pull/1345

So we can no longer build the latest nfs-ganesha V2.9-dev on Xenial.

Merge pull request #1357 from tchaikov/wip-ceph-pr-docs/python3

ceph-pr-docs: install python3 packages

Merge pull request #1355 from ceph/fix-validate

ceph-dev-build: fix deb and rpm validate

ceph-pr-docs: install python3 packages

see also https://github.com/ceph/ceph/pull/29528

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-build: fix deb and rpm validate

validate_deb - not redhat does not mean it is deb distro,
there can be other packaging systems
validate_rpm - we do not want to proceed for zypper based,
we only want to proceed for redhat like repo

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #1346 from ceph/nfs_ganesha_disable_lizardfs

nfs-ganesha: Disable LizardFS FSAL

Merge pull request #1350 from ceph/guits-remove-dashboard

ceph-ansible: remove dashboard scenarios

ceph-ansible: remove dashboard scenarios

since dashboard is enabled by default, we test it in all_daemons
scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1349 from tchaikov/wip-grep-cleanup

ceph-dev-new-trigger: only use positive match for filter

Merge pull request #1354 from tchaikov/wip-pin-jjb

ceph-build-pull-requests: ping jjb to 2.10.1

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-build-pull-requests: ping jjb to 2.10.1

jjb 3.0 brings some incompatible behavior:

* it tries to expand the strings like ${FOO} using the parameters,
  so if we continue using include-raw, jjb test will fail like

```
Failure formatting job
....
ERROR:root:Failure formatting job '{
with itself
Traceback (most recent call last):
...
  File
"/tmp/venv.Pa7CUfDCFK/local/lib/python2.7/site-packages/jenkins_jobs/formatter.py",
line 43, in deep_format
    raise JenkinsJobsException(desc)
jenkins_jobs.errors.JenkinsJobsException:
...
...
TEMPVENV=$(mktemp -td venv.XXXXXXXXXX)
```
* also, it fails to pass down some parameters to the build job, for
  instance, the `WORKSPACE` is not passed anymore.

both these change break the "test" job, as we are using `include-raw`
everywhere, and we also use `${WORKSAPCE}` in some yml files.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1352 from tchaikov/wip-include-raw

s/!include-raw/!include-raw:/

s/!include-raw/!include-raw:/

the former is deprecated.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-dev-new-trigger: only use positive match for filter

there is no need to put
```
grep -v '\(jewel\|kraken\|luminous\)' | grep '\(mimic\|nautilus\)'
```

as we are not likely to have both "jewel" and "mimic" in a branch name.

this change is a follow-up of the discussion when reviewing #1345

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1347 from ceph/jan--f-add-notifications

add Jans email for nightly notifications

Merge branch 'master' into jan--f-add-notifications

Merge pull request #1348 from ceph/master-fix

ceph-dev-trigger: add missing escape for grep on master

ceph-dev-trigger: add missing escape for grep on master

Signed-off-by: Alfredo Deza <adeza@redhat.com>

add Jans email for nightly notifications

Merge pull request #1345 from ceph/rm-xenial

ceph-dev-*: remove xenial for master and octopus

ceph-dev-trigger: add conditional step for master and octopus

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-dev-new-trigger: remove xenial for master and development branches

Signed-off-by: Alfredo Deza <adeza@redhat.com>

nfs-ganesha: Disable LizardFS FSAL

LizardFS FSAL has been added in [1] and it's enabled by default. This
is currently failing in the CI.

CMake Error at CMakeLists.txt:595 (message):
  STRICT_PACKAGE: Cannot find LizardFS client lib.  Disabling lizardfs fsal

This patch configures the USE_FSAL_LIZARDFS option to OFF.

[1] https://github.com/nfs-ganesha/nfs-ganesha/commit/884ec97

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1329 from ceph/ceph_container_nightly_devel_build

Add ceph container nightly devel build job

Merge branch 'master' into ceph_container_nightly_devel_build

ceph-volume: allow passing GITHUB_TOKEN as a password to manually set github status

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1342 from ceph/ceph-volume-nautilus

ceph-volume-test: adds on-evaluation-failure to the simple build step

ceph-volume-test: adds on-evaluation-failure to the simple build step

This makes the job skip the simple tests if the condition fails instead
of failing the run.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1339 from ceph/ceph_ansible_nightly_registry_pagesize

ceph-ansible-nightly: update tags pagination

Merge branch 'master' into ceph_ansible_nightly_registry_pagesize

Merge pull request #1341 from ceph/ceph-volume-nautilus

ceph-volume-test: only run simple tests for luminous and mimic

ceph-volume-test: only run simple tests for luminous and mimic

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1340 from ceph/ceph-volume-nautilus

ceph-volume: fix logic when deciding to run simple tests

ceph-volume: fix logic when deciding to run simple tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-ansible-nightly: update tags pagination

Set the page size to 100 instead of 10 (default).
Also we should add an extra page when the remainder of the modulo
operation between the tag count and page size isn't zero.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1337 from ceph/guits-install_jq

jjb: install jq package

jjb: actually fail if a command doesn't return 0

this sets `-euxo pipefail` to avoid this situation :

```
++ sort
++ grep -v jenkins-job-builder
++ jq -r '.jobs[].name'
/tmp/jenkins7521600802673204021.sh: line 1043: jq: command not found
++ curl -s https://2.jenkins.ceph.com/api/json
[PostBuildScript] - Executing post build scripts.
[jenkins-job-builder] $ /bin/sh -xe /tmp/jenkins7309655147676749318.sh
+ rm /home/jenkins-build/.jenkins_jobs.2.jenkins.ceph.com.ini
Finished: SUCCESS
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1338 from ceph/guits-containers_releases

ceph-ansible: make it possible to run nighly job on custom branch

ceph-ansible: make it possible to run nighly job on custom branch

When running manually a nightly containerized job against a custom branch of
ceph-ansible, it fails because `CEPH_DOCKER_IMAGE_TAG` is not set.

Typical error:
```
+ case "$CEPH_ANSIBLE_BRANCH" in
+ CEPH_DOCKER_IMAGE_TAG=
+ '[' -n '' ']'
Build step 'Execute shell' marked build as failure
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1336 from ceph/guits-remove_oldstable_jobs

ceph-ansible: remove old stable jobs

ceph-ansible: remove old stable jobs

This commit removes the (very) old stable jobs.
These jobs can't be launched, we have to use the new matrix.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1335 from ceph/guits-ubuntu_def

ceph-ansible: remove unused ubuntu jobs

ceph-ansible: remove unused ubuntu jobs

This commit removes some ubuntu job definition which are unused.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1333 from ceph/wip-ok-dan

ceph-dev-*: Clarify job descriptions

ceph-dev-*: Clarify job descriptions

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1332 from ceph/ceph-volume-nautilus

ceph-volume: only test simple on mimic and luminous branches

ceph-volume: only test simple on mimic and luminous branches

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1331 from ceph/ceph-volume-nautilus

ceph-volume: add nautilus to nightly testing

ceph-volume: add nautilus to nightly testing

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1330 from tchaikov/wip-40646

ceph-*build/build/setup_rpm: downgrade to gcc-c++-8.2.1

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

ceph-*build/build/setup_rpm: downgrade to gcc-c++-8.2.1

Fixes: http://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>

Add ceph container nightly devel build job

The ceph container images are only built on github event and also based
on stable ceph content.
We can also have nightly builds with ceph developement content from
shaman/chacra.

See https://github.com/ceph/ceph-container/pull/1411

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #1327 from rishabh-d-dave/add-shrink_rgw-to-CI

ceph-ansible: add shrink_rgw scenario test to CI

Merge branch 'master' into add-shrink_rgw-to-CI

Merge pull request #1328 from tchaikov/wip-gcc-9-is-nice-but

scripts/build_utils.sh: only use GCC-9 for building crimson

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

scripts/build_utils.sh: only use GCC-9 for building crimson

using GCC-9 for building mimic and other stable releases has following
consequences:

* it brings runtime dependencies of libstdc++ runtime shipped along with
  GCC-9
* GCC-9 is more picky, so some 3rd party libraries fail to build due to
  -Werror.

let's continue using GCC-8 on bionic when building non-crimson flavors

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-ansible: add shrink_rgw scenario test to CI

Add shrink_rgw scenario to CI so that it can be tested against master
and nautilus.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1326 from rishabh-d-dave/remove-shrink_mds-from-oldstable

ceph-ansible: remove shrink_mds from oldstable

Merge branch 'master' into remove-shrink_mds-from-oldstable

Merge pull request #1325 from rishabh-d-dave/add-shrink_rbdmirror-to-CI

ceph-ansible: add shrink_rbdmirror to CI

ceph-ansible: remove shrink_mds from oldstable

We don't need to run shrink_mds against "oldstable" releases.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-ansible: add shrink_rbdmirror to CI

Add shrink_rbdmirror scenario to CI so that it can be tested against
master and nautilus.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1324 from rishabh-d-dave/add-shrink_mgr-to-CI

ceph-ansible: add shrink_mgr scenario to CI

ceph-ansible: add shrink_mgr scenario to CI

Add shrink_mgr scenario to CI so that it can be tested against master
and nautilus.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1323 from ceph/guits-use_start_tox

ceph-ansible: build job name properly

ceph-ansible: build job name properly

Since tox.ini file has been split, this is needed to be able to run the
individual scenario testing. Those parameters are used to build the job
name passed to start_tox() function.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1322 from ceph/guits-use_start_tox

ceph-ansible: use start_tox function in ansible-scenario job

ceph-ansible: use start_tox function in ansible-scenario job

otherwise it always tries to run scenario using tox.ini only.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1320 from rishabh-d-dave/add-shrink_mds-to-CI

add shrink_mds scenario to CI

add shrink_mds scenario to CI

Add shrink_mds scenario to CI so that it can be tested against master.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #1319 from tchaikov/wip-1315

ceph-dev-new-setup: missing pieces in PR#1315

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

scripts/build_utils.sh: add missing closing quote

Signed-off-by: Kefu Chai <kchai@redhat.com>