tarball: kill -9 ceph-mon / ceph-osd always succeed

The error code of kill -9 ceph-{mon,osd} must be ignored otherwise the
build may return on error if it fails to kill the process just because
all has been cleaned up as it should.

http://tracker.ceph.com/issues/11929 Fixes: #11929

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #52 from dachary/wip-11929-linger-kill

tarbal: kill -9 lingering ceph-mon / ceph-osd

Reviewed-by: Nathan Cutler <ncutler@suse.com>

tarbal: kill -9 lingering ceph-mon / ceph-osd

be more agressive about killing ceph-mon / ceph-osd as they may be
unresponsive when / if there is a bug.

http://tracker.ceph.com/issues/11929 Fixes: #11929

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #51 from dachary/wip-11929-linger-kill

tarbal: kill lingering ceph-mon / ceph-osd

Reviewed-by: Nathan Cutler <ncutler@suse.com>

tarbal: kill lingering ceph-mon / ceph-osd

When a make check fails it is entirely possible that ceph-osd remain,
because they have their own process group. Kill whatever ceph-osd or
ceph-mon are left behind.

We kill ceph-osd / ceph-mon regardless of the success / failure status
of make check because we also want to address the case where a test
pretends to succeed but left daemons behind.

http://tracker.ceph.com/issues/11929 Fixes: #11929

Signed-off-by: Loic Dachary <ldachary@redhat.com>

fabfile: install more needed rpms

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: centos7 notcmalloc builder

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #48 from ceph/wip-rpm-major

fix rsync-target for rpm systems

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

fabfile: use only major version for rpm systems

centos7, not centos7_1_1503 (or whatever).  This was breaking the
rsync-target for gitbuilder-ceph-rpm-centos7-amd64-basic.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #47 from ceph/wip-bootstrap

bootstrap: copy from teuthology

..with the removal of --system-site-packages

Reviewed-by: Dan Mick <dmick@redhat.com>

bootstrap: copy from teuthology

Drop some dependencies (libevent, mysql) and skip setup.py (which we
don't have).

Drop system-site-packages

Create virtualenv/bin -> bin symlink

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #46 from ceph/wip-cleanup

fix a million things in fabfile, cleanup, etc.

Reviewed-by: Dan Mick dmick@redhat.com

fabfile: centos7 gb instead of rhel7

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: build ceph-ci too for deb and rpm builders

Signed-off-by: Sage Weil <sage@redhat.com>

bootstrap:  what's pip -E?  use virtualenv/pip in now-standard way

Signed-off-by: Dan Mick <dan.mick@redhat.com>

fabfile: drop rhel builders

...except for the rhel7 kernel one, since we don't have a centos7 one up
to take its place.

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: gitbuidler_kernel_{deb,rpm}

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: kill oneiric

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: kill ndn and dho cruft

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: kill apache2 builders

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: kill kmod builder

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: add rpm kernel builders

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: drop rhel7 builders

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #42 from ceph/wip-install-deps

Use install-deps.sh in build-ceph scripts

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #45 from ceph/wip-build-rpm

build-ceph-rpm.sh: fix rpmbuild for ceph-release file

build-ceph-rpm.sh: fix rpmbuild for ceph-release file

We don't want that stray --define arg.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #44 from ceph/wip-el7

fabfile: set $dist to .el7 on centos 7

fabfile: set $dist to .el7 on centos 7

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #43 from dachary/wip-gitbuilder-master

fabfile: always use gitbuilder master

Reviewed-by: Kefu Chai <kchai@redhat.com>

fabfile: always use gitbuilder master

Instead of a hardcoded SHA. If a new version of master at
https://github.com/ceph/gitbuilder/commits/master is available, it needs
to be used because it presumably contains a fix. If there is a need to
update the fabfile to take advantage of that fix, it's like having pull
request review twice on a commit. Once when it is reviewed prior to
being merged in gitbuilder and once when it is reviewed to be used in
the fabfile.

Signed-off-by: Loic Dachary <loic@dachary.org>

Move the installation to a better place and cond-require it on install-deps.sh executability

Use install-deps.sh in build-ceph scripts

Merge pull request #40 from ceph/wip-rpm-unpackaged-files

build-ceph-rpm.sh: terminate on unpackaged files

build-ceph-rpm.sh: terminate on unpackaged files

As Ceph changes and new files get installed by default, we need to
ensure that the commits which create these files also update the
packaging files lists as well.

One way to do this is to tell Gitbuilder to stop skipping unpackaged
files in the RPMs. This will cause Gitbuilder builds to indicate a
failure when new files get added without packaging updates.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

build-ceph: display make check errors when stored in .log

Automake after a certain version started storing the make check logs in
.log files. If a check fails, display the log so the user can figure out
what went wrong.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #39 from ceph/wip-tarballs

build-ceph: run parallel make check when possible

Reviewed-by: Kefu Chai <kchai@redhat.com>

build-ceph: run parallel make check when possible

And timeout after 1h instead of 30mn because giant tests take over 20mn
and it's good to have so spare time in case the host is overloaded and
does not respond as fast as it should.

Signed-off-by: Loic Dachary <loic@dachary.org>

build-ceph: disable static and lttng

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #38 from dachary/wip-sphinx

rhel 6 needs python-sphinx10 & remove obsolete debian tarbals

fabfile: rhel 6 needs python-sphinx10

It is used to build the man pages.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

fabfile: discard squeeze/oneiric tarbals

They no longer exist and attempting to fetch them will fail the
installation.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #37 from dachary/wip-sphinx

fabfile: add python-sphinx to build man pages

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

fabfile: add python-sphinx to build man pages

Now that the man pages are not committed to the repository in their
formatted form (https://github.com/ceph/ceph/pull/4142), we need
python-sphinx at build time.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #31 from dachary/wip-wheezy

try install autobuild-ceph.init and autobuild-ceph.conf

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #32 from dachary/wip-targets

update roles / targets to reflect the current setup

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

update roles / targets to reflect the current setup

The previous update (9c9e68d816bd020d46f45c1a83ef620cd8f5c572) was
wrong: a number of targets that were supposed to create packages were
actually set to create tarballs instead. The update allows to run

       fab fab gitbuilder_auto
       fab fab gitbuilder_ceph_rpm

http://tracker.ceph.com/issues/11052 Fixes: #11052

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #36 from jdurgin/wip-blkin

use CEPH_EXTRA_CONFIGURE_ARGS and add a blkin gitbuilder

Add blkin support and a new gitbuilder for it

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

build: use CEPH_EXTRA_CONFIGURE_ARGS for flags from build-auto.sh

This was the intent, but it was not being used for many of the
build scripts. Factor out the flags that build-auto.sh sets
(tcmalloc and coverage) into defaults in build-ceph*.sh in case
any of them are being run directly, and not via build-auto.sh.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34 from dachary/wip-ssh-loic

add loic ssh key

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

add loic ssh key

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #33 from dachary/wip-make-check

build-ceph.sh always make check

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

build-ceph.sh always make check

Nowadays it's always possible to make check, no need to test for
src/gtest or something.

http://tracker.ceph.com/issues/10766 Fixes: #10766

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #23 from dachary/wip-ceph-disk

fabfile: remove python-flask, add python-requests

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

try install autobuild-ceph.init and autobuild-ceph.conf

It does not hurt to install both. On Debian wheezy, /etc/init and
/etc/init.d exist: if installing only in /etc/init, the script won't end
up in /etc/init.d and won't run.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #30 from dachary/wip-start-rpm

setsid ./loop.sh to not be killed when sudo exits

do not start autobuilder-ceph twice for rpm targets

Signed-off-by: Loic Dachary <loic@dachary.org>

setsid ./loop.sh to not be killed when sudo exits

The sudo() of fabric will kill the process group to cleanup any
leftovers. If the loop.sh daemon does not create a process group it is
also killed. This is not an issue when start/stop is used because a
process group is created implicitly. But when /etc/init.d/autobuild-ceph
is run instead because start/stop does not work, the process group must
be explicitly created.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #29 from dachary/wip-start-rpm

start autobuild-ceph on gitbuilder_ceph_rpm targets

start autobuild-ceph on gitbuilder_ceph_rpm targets

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #27 from dachary/wip-submodules

git reset --hard after removing the modules & do not rely on git deinit

Merge pull request #26 from dachary/wip-hosts

update machine names for ceph and rpm targets

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

git reset --hard after removing the modules

If src/gmock is removed after git reset --hard, it won't be populated if
it is a directory. It will only be populated if it is a submodule
because git submodule update --init follows.

Signed-off-by: Loic Dachary <loic@dachary.org>

Revert "build: use 'git submodule deinit' instead of 'rm -rf'"

This reverts commit a4323ad08fb909bf216fc3ebe0a25cc67d4c0a6c.

reminder about ssh / sudo requirements

Signed-off-by: Loic Dachary <loic@dachary.org>

update machine names for ceph and rpm targets

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #25 from ceph/wip-da-fix-submodules

build: use 'git submodule deinit' instead of 'rm -rf'

Reviewed-by: Loic Dachary <ldachary@redhat.com>

build: use 'git submodule deinit' instead of 'rm -rf'

Fix submodule handling: deinit submodules instead of
removing them and call update with --recursive, needed
to checkout gtest within gmock.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #24 from dachary/wip-gmock

add gmock to the list of directories to remove

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

add gmock to the list of directories to remove

Because gmock is transitioning to a submodule instead of being inlined.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

fabfile: remove python-flask, add python-requests

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge branch 'master' of github.com:ceph/autobuild-ceph

Fixed kernel version string.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #21 from ceph/wip-cryptsetup

fabfile: cryptsetup[-bin] for ceph-disk dmcrypt make check tests

fabfile: cryptsetup[-bin] for ceph-disk dmcrypt make check tests

For RPM distros also need hdparm and python-flask.

Signed-off-by: Sage Weil <sage@redhat.com>

Better detection for what epel to install and new URL.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Add new packages required.

New packages that were missing from the fabfile. Formatting fixes on others
and also some changes to user creation

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #19 from dachary/wip-xmlstarlet

Looks good.

make check will need xmlstarlet

It is an XML command line parser.

Signed-off-by: Loic Dachary <loic@dachary.org>

Use uname string for /version file on rpm kernel gitbuilders.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Merge pull request #18 from dachary/wip-erasure-code-corpus

cleanup the ceph-erasure-code-corpus directory

cleanup the ceph-erasure-code-corpus directory

In the same way it is done for the ceph-object-corpus directory.

http://tracker.ceph.com/issues/9420 Refs: #9420

Signed-off-by: Loic Dachary <loic-201408@dachary.org>

Push to new doc server.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

kernel-config.basic: CONFIG_DYNAMIC_DEBUG=y

Signed-off-by: Sage Weil <sage@redhat.com>

fabfile: update rpm targets

Merge branch 'master' of github.com:ceph/autobuild-ceph

Merge pull request #17 from ceph/wip-rpm-subdir

build-ceph-rpm.sh: copy "*.patch" from rpm/ instead of ./

build-ceph-rpm.sh: copy "*.patch" from rpm/ instead of ./

Signed-off-by: Dan Mick <dan.mick@inktank.com>

Merge pull request #15 from ceph/wip-rpm-patch

build-ceph-rpm.sh: copy "*.patch" from topdir into SOURCES

build-ceph-rpm.sh: copy "*.patch" from topdir into SOURCES

This allows patches to be applied when building the RPMs; the addition
of the ceph patch to disable /etc/init.d/ceph from autostarting Ceph
daemons leads to the desire for a .patch file, conditionally applied
by the .spec.  See the Ceph sources.

Signed-off-by: Dan Mick <dan.mick@inktank.com>

kernel-config: enable CONFIG_DEBUG_ATOMIC_SLEEP

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>

build-ceph.sh: --with-librocksdb-static=maybe

Signed-off-by: Sage Weil <sage@inktank.com>

fabfile: bz2 build dep

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'master' of github.com:ceph/autobuild-ceph

Use auto build script on RPM.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Snappy dev package missing from deb_builder()

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

Rhel7 updates.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>

fabfile: update gitbuilder to clean up out~

Signed-off-by: Sage Weil <sage@inktank.com>

build-*: clean up rocksdb submodule too

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'master' of github.com:ceph/autobuild-ceph