nfs-ganesha-stable: various rpm build updates

added a $CEPH_VERSION var to specify rpm installations

changed rpm build type to Release

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #951 from ceph/ceph-volume-nightly

adds nightly functional testing for ceph-volume

adds nightly functional testing for ceph-volume

This will run all our PR tests against the luminous and
master branch of ceph nightly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #950 from ceph/ceph-volume-prs

ceph-volume-ansible-prs: adds simple dmcrypt_plain and dmcrypt_luks

ceph-volume-ansible-prs: adds simple dmcrypt_plain and dmcrypt_luks

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #949 from ceph/ceph-volume-prs

ceph-volume-ansible-prs: add trigger phrase per scenario

ceph-volume-ansible-prs: add trigger phrase per scenario

So we can start all 'create' tests like 'jenkins test ceph-volume lvm
create'

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #948 from ceph/wip-iscsi-rpm-revision

iscsi: ensure initial stable release receives proper RPM revision

iscsi: ensure initial stable release receives proper RPM revision

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #947 from ceph/ceph-volume-prs

ceph-volume-ansible-prs: libvirt cleanup and dmcrypt testing scenarios

ceph-volume-ansible-prs: adds the dmcrypt factor

This adds the dmcrypt testing scenarios for
ceph-volume lvm.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume-ansible-prs: cleanup libvirt and upgrade vagrant box

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #946 from ceph/make-check-trigger

ceph-pull-requests: adds a trigger phrase for make check

ceph-pull-requests: adds a trigger phrase for make check

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #945 from ceph/ansible_prs_fix_start_tox

ceph-ansible: PRs should keep CEPH_DOCKER_IMAGE_TAG set in tox

ceph-ansible: PRs should keep CEPH_DOCKER_IMAGE_TAG set in tox

Setting this variable was originally for nightlies test need.
We should keep `CEPH_DOCKER_IMAGE_TAG` set in tox for PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #944 from ceph/wip-nfs-ganesha-stable

nfs-ganesha-stable: various updates

nfs-ganesha-stable: various updates

- set RADOS_URL cmake options ON in rpm build script
- update various config parameters
- ensure removal of any old leftover ceph rpm packages

Merge pull request #941 from ceph/remote-branches

ceph-volume-ansible-prs: allow to specify branches not in ceph.git

ceph-volume-ansible-prs allow to pass the ghprbSourceBranch if branch is no in ceph.git

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume-ansible-prs fallback on SHA var, report otherwise before failing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #942 from tchaikov/wip-build-utils

script/build_utils.sh: do not download unmirrored arch

*/build_deb: pass env vars with quotes

should double quote env var in command substitution, otherwise shell will
perform field splitting with the substitution, and only the first element in
it will be consumed. that's why CEPH_EXTRA_CMAKE_ARGS will always be
`-DALLOCATOR=libc` when it reaches debian/rules, even if
'CEPH_EXTRA_CMAKE_ARGS' is "-DALLOCATOR=libc -DWITH_STATIC_LIBSTDCXX=ON",
and it is passed to pbuilder like:

sudo CEPH_EXTRA_CMAKE_ARGS=$CEPH_EXTRA_CMAKE_ARGS pbuilder ...

Fixes: http://tracker.ceph.com/issues/22465
Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: use 'D' pbuilder hooks instead of 'E'

to avoid persisting the changes to the base.tgz

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: add security repo

gcc-7 depends on a recent binutils which is offered by security repo

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: fix test for pbuilder base.tar

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: do not download translation index

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: revert to old gcc when not using ppa

Fixes: http://tracker.ceph.com/issues/22465
Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: extract use_ppa() as a function

so it can be reused.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pr-docs/build: nuke ubuntu-toolchain-r.list

* no need to download i18n package index
* and remove ubuntu-toolchain-r.list before `apt-get` update

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/build_utils.sh: do not download unmirrored arch

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #943 from ceph/jewel-nightly

build_utils: set CEPH_STABLE_RELEASE in start_tox

ceph-ansible-nightly: remove lvm tests from the jewel tests

ceph-volume did not exist in the ceph version of jewel

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

build_utils: set CEPH_STABLE_RELEASE in start_tox

This will tell tox which ceph version to install and test. Without this
some of the nightlies advertised that they were testing jewel
but were actually testing luminous.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #940 from tchaikov/wip-apt-get-never-fail

script/build_utils.sh: don't fail if any mirror is not accessible

script/build_utils.sh: don't fail if any mirror is not accessible

also prevent apt-get from downloading i18n index

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #939 from tchaikov/wip-import-ppa-key

script/build_utils.sh: import ppa key into apt's keyring

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

script/build_utils.sh: import ppa key into apt's keyring

to be immune from the MITM attack

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #925 from tchaikov/wip-22301

ceph{-,dev-,dev-new-}build: install centos-release-scl-rh repo for arm64

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph{-,dev-,dev-new-}build: enable testing repo for arm64

the devtoolset-7 for aarch64 is not offered by centos-sclo-rh repo, but
at least centos-sclo-rh-testing has devtoolset-6.

see https://github.com/sclorg/centos-release-scl
see https://buildlogs.centos.org/centos/7/sclo/aarch64/rh/

Fixes: http://tracker.ceph.com/issues/22301
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #937 from tchaikov/wip-gcc-7-on-xenial

use gcc 7 on xenial, and link libstdc++ statically on xenial

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-*-setup/build/build: enable WITH_STATIC_LIBSTDCXX for xenial and trusty for new branchs

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: use ppa on xenial also

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #938 from tchaikov/wip-trigger-notcmalloc

ceph-dev-{,new-}trigger: add FLAVOR=notcmalloc back

ceph-dev-{,new-}trigger: add FLAVOR=notcmalloc back

it's a regression introduced by 3a0b799b

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #929 from ceph/revert_cephstablerelease

ceph-ansible: get CEPH_STABLE_RELEASE back to tox

ceph-ansible: get CEPH_STABLE_RELEASE back to tox

get CEPH_STABLE_RELEASE back to tox.ini so we do not have to set
explicitely CEPH_STABLE_RELEASE when running test locally.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #920 from ceph/add_scenarios

ceph-ansible-prs: Add more scenarios in 'auto'

ceph-ansible-prs: Add more scenarios in 'auto'

We need to get a better coverage for PRs testing.
Adding these 2 purge scenarios will cover osds scenarios testing which
is not the case at the moment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #935 from tchaikov/pass-distros-without-quotes

ceph-dev{,-new}-trigger: pass DISTROS using a list

ceph-dev{,-new}-trigger: pass DISTROS using a list

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #931 from tchaikov/wip-do-not-build-mimic-and-up-on-trusty

ceph-*-build: do not build new branches on trusty

ceph-*-trigger: do not build new branches on trusty

by "new branches", i mean any branch whose name does not contain
"jewel" or "luminous".

see
https://docs.openstack.org/infra/jenkins-job-builder/builders.html#builders.conditional-step
and https://wiki.jenkins.io/display/JENKINS/Conditional+BuildStep+Plugin

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-*-build: use more readable groovy expression

see http://groovy-lang.org/operators.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #934 from ceph/bump-gh-status

ceph-volume-ansible-prs make sure a newer version of github-status is used

ceph-volume-ansible-prs make sure a newer version of github-status is used

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #933 from ceph/ceph-volume-tests

ceph-volume-ansible-prs: remove prepare_activate and add subcommand trigger

ceph-volume-ansible-prs: adds trigger to run all tests by subcommand

This will allow us to run all lvm tests like 'jenkins test ceph-volume
lvm all'.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume-ansible-prs: remove the prepare_activate scenario

This hasn't been created yet so no need to add it to the matrix.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #921 from ceph/manual-github-notification

ceph-volume-ansible-prs: Manual GitHub notification

ceph-volume-ansible-prs set the status with github-status when failure

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume-ansible-prs set the status with github-status when starting

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume-ansible-prs allow Python-based notification to Github

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #932 from ceph/ceph-volume-tests

ceph-volume-ansible-prs: add a trigger phrase to start all jobs

ceph-volume-ansible-prs: add a trigger phrase to start all jobs

This will allow you to comment 'jenkins test ceph-volume all' and
all jobs will trigger.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #930 from ceph/wip-kernel-ceph-iscsi

kernel: added 'ceph-iscsi*' branches to trigger list

kernel: added 'ceph-iscsi*' branches to trigger list

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #928 from tchaikov/wip-only-build-newer-releases-with-gcc-7

setup_pbuilder: only use ppa on newer releases

setup_pbuilder: only use ppa on newer releases

* only use GCC-7 offered by ppa repo on mimic and newer release.
  we are dropping the support of trusty in mimic and newer releases,
  but we still need to use the old toolchain shipped with trusty to
  build jewel and luminous. so, only use GCC-7 on trusty for building
  mimic and up. as it should be safe to use the new GCC ABI on the
  supported distros, like xenial.
* extract the ppa setup related code into build_utils.sh, so the
  setup_pbuilder scripts can reuse it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #927 from ceph/wip-fog-fix

sepia-fog-image: couple fixes

sepia-fog-images: Build workspace for failure too

Signed-off-by: David Galloway <dgallowa@redhat.com>

sepia-fog-image: Remove ubuntu@ for teuthology-update-inventory

Cuz it doesn't work like that.

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #922 from ceph/wip-sepia-fog

sepia-fog-images: Job creation

sepia-fog-images: Job creation

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #926 from ceph/lvm-tests

ceph-ansible-prs: adds lvm tests for the luminous release

ceph-ansible-prs: adds lvm tests for the luminous release

ceph-volume can now be used in luminous, so we should be able to
test PRs for ceph-volume against luminous.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #914 from tchaikov/wip-gcc-7-on-trusty

setup_pbuilder: point gcc to gcc-7 on trusty

setup_pbuilder: point gcc to gcc-7 on trusty

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #924 from ceph/mimic-dev

ceph-dev-trigger support mimic* branches, to include mimic-dev

ceph-dev-trigger support mimic* branches, to include mimic-dev

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #923 from ceph/wip-kernel-install-libelf

kernel: install libelf

kernel: install libelf

ORC unwinder (default on x86 since 4.15) requires it.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #919 from ceph/jewel_stable3

ceph-ansible-prs: enable github-hooks for oldstable-trigger

ceph-ansible-prs: enable github-hooks for oldstable-trigger

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #918 from ceph/jewel_stable3

ceph-ansible-prs: fix wrong syntax in trigger-phrase

ceph-ansible-prs: fix wrong syntax in trigger-phrase

`stable-2.2` is not a correct syntax because it is interpreted as a regexp.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #917 from ceph/jewel_stable3

ceph-ansible-prs: change trigger-phrase

ceph-ansible-prs: change trigger-phrase

fix ceph-ansible-2.2-prs template because it triggers jobs that
shouldn't for backport to stable-2.2 PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #915 from ceph/jewel_stable3

ceph-ansible-nightly: couple of fixes

ceph-ansible-prs: add testing for stable-2.2

When submitting a backport PR to stable-2.2 branch, the jobs that are
auto triggered don't exist in job templates because they have changed
between `stable-2.2` and `stable-3.0`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-ansible-nightly: Remove inconsistency test

This way of testing is a bit dangerous and make the jobs inconsistents.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #916 from ceph/wip-ghprb-vars

ceph-colume-ansible-prs: add injected ghprb vars to help manual triggering

ceph-colume-ansible-prs: add injected ghprb vars to help manual triggering

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #911 from ceph/jewel_stable3

ceph-ansible: set CEPH_STABLE_RELEASE in start_tox()

ceph-ansible: set CEPH_STABLE_RELEASE in start_tox()

according to ceph/ceph-ansible#2165 we need to set `CEPH_STABLE_RELEASE`
in ceph/ceph-build.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #913 from ceph/ceph-volume-tests

ceph-volume-ansible-prs: use subcommand to build the testing path

ceph-volume-ansible-prs: use subcommand to build the testing path

When using test_path jjb was rendering $TEST_PATH as a list. I'm unsure
exactly why, but using $SUBCOMMAND we can easily build the test path.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #910 from ceph/ceph-volume-tests

ceph-volume: adds pr tests for the simple subcommand

ceph-volume: adds pr tests for the simple subcommand

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #909 from tchaikov/wip-scl-for-devtoolset

ceph{-,dev-,dev-new-}build: install centos-release-scl repo