Remove the `FORCE` parameter from the `ceph-windows-installer-build`
Jenkins job, and use it by default.
This job builds the installer from source code, and it bundles
the Ceph binaries and the WNBD driver.
Therefore, the logic to skip the build if the latest installer commit
didn't change is not useful, because the installer might contain
different Ceph binaries and WNBD driver.
David Galloway [Tue, 1 Feb 2022 19:46:50 +0000 (14:46 -0500)]
ceph-dev-container-only: Create job
This job will allow users to build a container for https://quay.ceph.io/repository/ceph-ci/ceph?tab=tags given a branch name. Should work for ceph.git and ceph-ci.git.
This is mostly useful for when packages and a repo successfully get built for a ceph-dev*build job but the container build fails.
Signed-off-by: David Galloway <dgallowa@redhat.com>
Ilya Dryomov [Tue, 1 Feb 2022 18:32:57 +0000 (19:32 +0100)]
kernel: drop local get_rpm_dist() definition
The local definition is missing a couple of things, most notably
CentOSStream addition from commit d4dfd1210124 ("scripts: Support
CentOS Stream"). This fixes repository creation breakage caused
by DIST and DISTRO set to unknown.
David Galloway [Tue, 1 Feb 2022 18:36:23 +0000 (13:36 -0500)]
ceph-dev*build: Stop using docker-mirror
Non-stream CentOS8 containers no longer work because the BaseOS and AppStream repos moved. We now use the centos/centos:stream8 container image as our BASEOS_TAG. Except that container lives on quay.io and not dockerhub.
docker-mirror.front.sepia.ceph.com:5000 can only mirror one registry. We'll keep it as mirroring dockerhub.
Signed-off-by: David Galloway <dgallowa@redhat.com>
Dan Mick [Thu, 20 Jan 2022 21:44:36 +0000 (13:44 -0800)]
ceph-{,dev,dev-new} build: fix conditional for cephadm package test
https://github.com/ceph/ceph-build/pull/1919 added a conditional
test to check for cephadm package existence (so the code to extract
cephadm could work on older builds that did not have a cephadm package).
However, it used bash's [[ operator, and that doesn't expand file globs
in its arguments, so it effectively stopped creating the cephadm
binary altogether.
Ensure version of libarchive is installed due to cmake dependency: https://access.redhat.com/solutions/6068431
Error message encountered:
cmake3: symbol lookup error: cmake3: undefined symbol: archive_write_add_filter_zstd
Related: https://github.com/ceph/ceph/pull/42294
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
In the nightly runs we saw that the ceph_cluster.yml file which we used
to deploy kcli plans is not up-to-date with the master one. It cause
problems in the nightlies. We've narrowed down the problem is with the basedir
and and also with the path difference in the dependency script and the location of
ceph. So changing the workspace path to workspace/ceph.
Ionut Balutoiu [Thu, 23 Dec 2021 11:35:44 +0000 (13:35 +0200)]
Refactor `ceph-windows-test` and `ceph-windows-pull-requests` jobs
Refactor the Ceph Windows Jenkins jobs (`ceph-windows-test` and
`ceph-windows-pull-requests`) with common scripts from
`scripts/ceph-windows` and `scripts/build_utils.sh`.
Ionut Balutoiu [Thu, 18 Nov 2021 14:30:28 +0000 (16:30 +0200)]
[wnbd-build] Add Jenkins job to build the wnbd project
* Add the Jenkins job used to build the `wnbd` project.
* Add the following utils functions: `ssh_exec`, `scp_upload`, `scp_download`.
* Add `./scripts/ceph-windows/cleanup` script that can be used as a
post-build step to clean up the Windows test / build environment.
* Add `./scripts/ceph-windows/setup_libvirt_vm` script used to spawn a
libvirt VM with the Windows test / build environment.
* Upload the build artifact to chacra.
Ionut Balutoiu [Fri, 5 Nov 2021 09:53:17 +0000 (11:53 +0200)]
Update ceph-windows-test job
* Create `scripts/ceph-windows` directory with the common scripts
for the Ceph Windows testing.
* Move `run_tests` script to `scripts/ceph-windows`.
* Remove VM image caching from the `run_tests` script.
* Add `cleanup_tests` script to `scripts/ceph-windows`, and call
this as a post-build step when the job is aborted.
* Add `get_chacra_build` build script, and refactor away this logic
from the `run_tests` script.
* Move cleanup Windows tests function to `scripts/build_utils.sh`.
Ionut Balutoiu [Mon, 18 Oct 2021 16:02:55 +0000 (19:02 +0300)]
Add Ceph Windows testing job
Add a new freestyle Jenkins job that can be used to test a Ceph
Windows build uploaded to Chacra, on the Jenkins agents labeled with
`amd64`, `focal`, `libvirt`.
The Ceph Windows build is identified in Chacra using the following
Jenkins job parameters:
* `CEPH_WINDOWS_VERSION`
* `CEPH_WINDOWS_SHA1`
As a build step, the new job will execute the `./build/run_tests`
bash script. This script sets up a KVM virtual machine with Windows
Server 2019, that it's used to run the Ceph Windows unit tests from
the Chacra build. On top of that, it takes care of the everything
needed to run the Ceph Windows tests.
David Galloway [Mon, 18 Oct 2021 19:41:14 +0000 (15:41 -0400)]
build_utils.sh: Disable CI_CONTAINER for nautilus too
Brad and Yuri are working on something for downstream and need some branches built in ceph-ci.git based on the nautilus branch. The builds keep showing up as failed in shaman because the container is failing to build. This minor change fixes that.
Signed-off-by: David Galloway <dgallowa@redhat.com>
Dan Mick [Thu, 7 Oct 2021 05:18:24 +0000 (22:18 -0700)]
ceph-*-build: only try extracting cephadm if it exists
https://github.com/ceph/ceph-build/pull/1913 added code to extract
the cephadm binary from its package and push it to chacra; however,
older builds don't include a cephadm package, and ceph-build must
work for those as well. Broke for nautilus.
Fixes: https://tracker.ceph.com/issues/52847 Signed-off-by: Dan Mick <dmick@redhat.com>
David Galloway [Fri, 1 Oct 2021 15:07:40 +0000 (11:07 -0400)]
ceph-dev*build: Wait for the appropriate arch's repo to be ready
Also use shaman's search API.
What was happening before this was, for example:
- master branch gets built
- x86_64 job, build, and repo are done way before arm64
- arm64 job finally gets to the point where it's ready to build its container
- arm64 job checks with shaman to ask if the branch's repo is ready
- shaman answers with the x86_64 repo status and says it is ready
- arm64 job proceeds but fails because the arm64 repo is not ready
projects / ceph-build.git / log