ceph-volume-scenario: allow archiving empty files

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume-scenario: collect ceph-volume logs on failure

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #978 from ceph/kraken-role

ansible: adds a role for deploying kraken

ansible: fix twitter config vars for kraken config

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ansible: remove circus config template from kraken role

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ansible: fix invalid syntax and other errors in the kraken role

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ansible: populate kraken custom_settings

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ansible: adds kraken role and example playbook

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #977 from ceph/tox-manual-trigger

ceph-volume-pr: trigger tox tests manually

ceph-volume-ansible-prs: use github_status_setup from build_utils.sh

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume-pr: adds functionality to trigger tox testing manually

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

build_utils: adds a function to setup things for github-status

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #976 from ceph/shaman-ipv6-2

shaman prs: use a template for pg_hba.conf

shaman-pull-requests use the new template for pg_hba.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>

shaman-pull-requests create a pg_hba.conf template

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #975 from ceph/shaman-pg-ipv6

shaman-pull-requests allow users to connect locally over ipv6 sans password

shaman-pull-requests allow users to connect locally over ipv6 sans password

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #974 from tchaikov/wip-good-old-kraken

ceph-dev{,-new}-trigger: do build kraken w/o ppa and on trusty

ceph-dev{,-new}-trigger: do build kraken w/o ppa and on trusty

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #973 from ceph/build-ceph-img

ceph-container-build-push-imgs: use a script from ceph-container

ceph-container-build-push-imgs: use a script from ceph-container

Let's execute a script in the ceph-container project instead of writting
the script here.
ceph-build does the CI so let's keep it that way.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #971 from ceph/ceph-container-build-push-imgs

ceph-container: add job to build and push images

ceph-container: add job to build and push images

After a PR is merged we trigger a build and a push the newly built
images to the Docker Hub.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #970 from ceph/wip-sepia-space

sepia-fog-images: count words in $remaininghosts

sepia-fog-images: count words in $remaininghosts

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #969 from ceph/wip-fog-typo

sepia-fog-images: Syntax fix and typo correction

sepia-fog-images: Add machine name to unlock command

Signed-off-by: David Galloway <dgallowa@redhat.com>

sepia-fog-images: Remove finished hostnames properly

Before this change, if there were 3+ hosts in $remaininghosts and one of
the hostnames in the middle finished, sed would remove the hostname and
its trailing space which would mash the last two hostnames together.

```
Example:

remaininghosts="mira071 smithi164 mira001"
remaininghosts=$(echo $remaininghosts | sed -e "s/ *smithi164 *//")
$ echo $remaininghosts
mira071mira001
```

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #968 from ceph/wip-rhel-sepia

sepia-fog-images: Add support for RHEL

sepia-fog-images: Add support for RHEL

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #967 from ceph/ceph-volume-emails

ceph-volume-nightly add email notifications

ceph-volume-nightly add email notifications

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #966 from ceph/trim-bz-mill

bz-mill keep just 15 days around of logs

bz-mill keep just 15 days around of logs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #965 from ceph/wip-just-unlock

sepia-fog-images: Don't wait for machines to reboot after capture

sepia-fog-images: Don't wait for machines to reboot after capture

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #960 from ceph/add_ceph-ansible_ooo_scenario

ceph-ansible: add tripleo oriented scenario

ceph-ansible: add tripleo oriented scenario

This commit enable a new 'tripleo deployment' scenario.
ceph-ansible related PR: https://github.com/ceph/ceph-ansible/pull/2407/files

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #963 from ceph/ceph-volume-suite

ceph-volume-test: a job that starts all ceph-volume functional tests

ceph-volume-test: a job that starts all ceph-volume functional tests

This job will allow a user to input ceph branch, ceph sha1 and
ceph-ansible branch and from that will start all ceph-volume tests using
those values.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #961 from ceph/wip-kernel-install-bison

kernel: install flex and bison

Reviewed-by: Alfredo Deza <adeza@redhat.com>

kernel: install flex and bison

Kconfig lexer and parser are built from real .l and .y since 4.16.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #964 from ceph/discard-logs

Discard builds for jobs

ceph-dev-build discard builds after 30 days

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-dev-new-build discard builds after 30 days

Signed-off-by: Alfredo Deza <adeza@redhat.com>

nfs-ganesha discard builds after 90 days

Signed-off-by: Alfredo Deza <adeza@redhat.com>

bz-mill discard builds after 90 days

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #962 from ceph/ceph-ansible-logrotate

ceph-ansible jobs increase daystoKeep for logs to 90

ceph-ansible-nightly increase daystoKeep for logs to 90

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-ansible-prs increase daystoKeep for logs to 90

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #959 from ceph/wip-nfs-ganesha-build-type

nfs-ganesha: change CMAKE_BUILD_TYPE to Maintainer

nfs-ganesha: change CMAKE_BUILD_TYPE to Maintainer

Add yum clean --all before yum install

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #958 from ceph/cdp-prs-python-fix

ceph-deploy-pull-requests: project does not need 2.6 anymore

ceph-deploy-pull-requests: project does not need 2.6 anymore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #957 from ceph/ceph-volume-scenario

ceph-volume-scenario: a new job to run ceph-volume testing scenarios

ceph-volume-scenario: a new job to run ceph-volume testing scenarios

This job can be started manually and allows for ceph branch, ceph sha1,
ceph-volume testing scenario and ceph-ansible branch to be entered
as parameters.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #955 from ceph/wip-tcmu-runner

tcmu-runner: drop libtcmu-devel RPM

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tcmu-runner: drop libtcmu-devel RPM

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #954 from ceph/wip-iscsi-rpm-versions

rtslib-fb/tcmu-runner: strip 'v' prefix from RPM version

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rtslib-fb/tcmu-runner: strip 'v' prefix from RPM version

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #953 from ceph/ceph-volume-nightly

ceph-volume-nightly: fix usage of ceph_branch

ceph-volume-nightly: fix usage of ceph_branch

The jobs will now clone the correct branch of ceph.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #952 from ceph/wip-nfs-ganesha-stable

nfs-ganesha-stable: various rpm build updates

nfs-ganesha-stable: various rpm build updates

added a $CEPH_VERSION var to specify rpm installations

changed rpm build type to Release

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #951 from ceph/ceph-volume-nightly

adds nightly functional testing for ceph-volume

adds nightly functional testing for ceph-volume

This will run all our PR tests against the luminous and
master branch of ceph nightly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #950 from ceph/ceph-volume-prs

ceph-volume-ansible-prs: adds simple dmcrypt_plain and dmcrypt_luks

ceph-volume-ansible-prs: adds simple dmcrypt_plain and dmcrypt_luks

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #949 from ceph/ceph-volume-prs

ceph-volume-ansible-prs: add trigger phrase per scenario

ceph-volume-ansible-prs: add trigger phrase per scenario

So we can start all 'create' tests like 'jenkins test ceph-volume lvm
create'

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #948 from ceph/wip-iscsi-rpm-revision

iscsi: ensure initial stable release receives proper RPM revision

iscsi: ensure initial stable release receives proper RPM revision

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #947 from ceph/ceph-volume-prs

ceph-volume-ansible-prs: libvirt cleanup and dmcrypt testing scenarios

ceph-volume-ansible-prs: adds the dmcrypt factor

This adds the dmcrypt testing scenarios for
ceph-volume lvm.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume-ansible-prs: cleanup libvirt and upgrade vagrant box

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #946 from ceph/make-check-trigger

ceph-pull-requests: adds a trigger phrase for make check

ceph-pull-requests: adds a trigger phrase for make check

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #945 from ceph/ansible_prs_fix_start_tox

ceph-ansible: PRs should keep CEPH_DOCKER_IMAGE_TAG set in tox

ceph-ansible: PRs should keep CEPH_DOCKER_IMAGE_TAG set in tox

Setting this variable was originally for nightlies test need.
We should keep `CEPH_DOCKER_IMAGE_TAG` set in tox for PRs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #944 from ceph/wip-nfs-ganesha-stable

nfs-ganesha-stable: various updates

nfs-ganesha-stable: various updates

- set RADOS_URL cmake options ON in rpm build script
- update various config parameters
- ensure removal of any old leftover ceph rpm packages

Merge pull request #941 from ceph/remote-branches

ceph-volume-ansible-prs: allow to specify branches not in ceph.git

ceph-volume-ansible-prs allow to pass the ghprbSourceBranch if branch is no in ceph.git

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume-ansible-prs fallback on SHA var, report otherwise before failing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #942 from tchaikov/wip-build-utils

script/build_utils.sh: do not download unmirrored arch

*/build_deb: pass env vars with quotes

should double quote env var in command substitution, otherwise shell will
perform field splitting with the substitution, and only the first element in
it will be consumed. that's why CEPH_EXTRA_CMAKE_ARGS will always be
`-DALLOCATOR=libc` when it reaches debian/rules, even if
'CEPH_EXTRA_CMAKE_ARGS' is "-DALLOCATOR=libc -DWITH_STATIC_LIBSTDCXX=ON",
and it is passed to pbuilder like:

sudo CEPH_EXTRA_CMAKE_ARGS=$CEPH_EXTRA_CMAKE_ARGS pbuilder ...

Fixes: http://tracker.ceph.com/issues/22465
Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: use 'D' pbuilder hooks instead of 'E'

to avoid persisting the changes to the base.tgz

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: add security repo

gcc-7 depends on a recent binutils which is offered by security repo

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: fix test for pbuilder base.tar

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: do not download translation index

Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: revert to old gcc when not using ppa

Fixes: http://tracker.ceph.com/issues/22465
Signed-off-by: Kefu Chai <kchai@redhat.com>

scripts/build_utils.sh: extract use_ppa() as a function

so it can be reused.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-pr-docs/build: nuke ubuntu-toolchain-r.list

* no need to download i18n package index
* and remove ubuntu-toolchain-r.list before `apt-get` update

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/build_utils.sh: do not download unmirrored arch

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #943 from ceph/jewel-nightly

build_utils: set CEPH_STABLE_RELEASE in start_tox

ceph-ansible-nightly: remove lvm tests from the jewel tests

ceph-volume did not exist in the ceph version of jewel

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

build_utils: set CEPH_STABLE_RELEASE in start_tox

This will tell tox which ceph version to install and test. Without this
some of the nightlies advertised that they were testing jewel
but were actually testing luminous.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #940 from tchaikov/wip-apt-get-never-fail

script/build_utils.sh: don't fail if any mirror is not accessible

script/build_utils.sh: don't fail if any mirror is not accessible

also prevent apt-get from downloading i18n index

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #939 from tchaikov/wip-import-ppa-key

script/build_utils.sh: import ppa key into apt's keyring

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>