John Mulligan [Wed, 12 Mar 2025 20:36:49 +0000 (16:36 -0400)]
ceph-pull-requests-arm64: convert job to run make check in a container
Convert the test script to run make check using the
build-with-container.py tool. Converts env vars that were directly
passed to the script to use a env file.
Remove the redundant n_test_jobs variable since it is *always* the
same as n_build_jobs.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 12 Mar 2025 20:37:19 +0000 (16:37 -0400)]
ceph-pull-requests: convert job to run make check in a container
Convert the test script to run make check using the
build-with-container.py tool. Converts env vars that were directly
passed to the script to use a env file.
Updates the timeout to 4hour (to match the arm64 job) and avoid timeouts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Dan Mick [Thu, 25 Sep 2025 20:58:38 +0000 (13:58 -0700)]
Update jenkins-job-builder spec to released version
jenkins-job-builder finally released 6.4.3, which includes
https://review.opendev.org/c/jjb/jenkins-job-builder/+/941310,
so we can stop using the git url
Dan Mick [Tue, 23 Sep 2025 02:28:14 +0000 (19:28 -0700)]
scripts/build_utils.sh: only do gcc ppa version dance on Ubuntu
This doesn't apply to debian. Maybe debian has similar needs,
but who knows, because we don't test the builds (or indeed
even do them until release time), but this code is definitely
not appropriate for bookworm, so avoid it for now.
We do a shallow clone, but I have seen a few branches that fail because the
clone is _too_ shallow. In rare cases like this we can simply deepen until we
find the tag.
If sccache is supposed to be enabled, but isn't - for example due to the feature
not being backported - we were still attempting to archive, and hanging. Check
for the existence of the file directly.
David Galloway [Wed, 20 Aug 2025 21:57:51 +0000 (17:57 -0400)]
ceph-dev-pipeline: Support for upstream builds
- Variable reading/writing was reordered so that job logic
can override CEPH_REPO, SHA1, and chacra_url.
- For now, --rpm-no-match-sha avoids https://tracker.ceph.com/issues/72626
- Optionally archive sccache artifacts only if we used sccache. Avoids
the job hanging at the end (yes, I know allowEmptyArchive: true should
take care of it.. 2.jenkins didn't care.)
Signed-off-by: David Galloway <david.galloway@ibm.com>
David Galloway [Wed, 20 Aug 2025 21:51:04 +0000 (17:51 -0400)]
setup_chacractl: Only ask shaman for chacra node if chacra_url's unset
In a Release Build, we want to push the packages to chacra.ceph.com.
So we manually define chacra_url=https://chacra.ceph.com in ceph-source-dist
by writing it to dist/other_envvars
Signed-off-by: David Galloway <david.galloway@ibm.com>
David Galloway [Wed, 20 Aug 2025 21:45:14 +0000 (17:45 -0400)]
ceph-source-dist: Support for ceph-release-pipeline
Again, the changes needed here are:
- For a Release Build, we are building a tarball based off a
version commit that has been created by the ceph-tag job
then pushed to ceph-releases.git. So instead of taking
the BRANCH from an upstream Jenkins job, we're reading
the version commit SHA1 from the sha1.txt file that ceph-tag
wrote then checking /that/ out and building the tarball from it.
So some var writing and reading was shuffled around.
- We do not want this job to support checking out from ceph-private.git
or ceph-releases.git directly. Instead, if ceph-release-pipeline
passed RELEASE_BUILD=true, only then can you clone from those private
repos. So we override CEPH_REPO and set chacra_url to the chacra
instance that doesn't automatically prune repos (chacra.ceph.com).
Signed-off-by: David Galloway <david.galloway@ibm.com>
David Galloway [Wed, 20 Aug 2025 21:22:21 +0000 (17:22 -0400)]
ceph-tag: Write a sha1 file for later consumption
In a regular build, ceph-dev-pipeline just takes a branch name and makes a source tarball in ceph-source-dist. ceph-dev-pipeline takes that tarball from the ceph-source-dist job and builds from it so it's able to just get the SHA1 from it.
In a /release/ build, a release branch (e.g., reef-release) is specified and we add an additional version commit on top of it. So we don't know the SHA1 just based on the branch name. My solution was to write the version commit's SHA1 to a file and consume it later.
Signed-off-by: David Galloway <david.galloway@ibm.com>
Zack Cerza [Thu, 21 Aug 2025 17:02:12 +0000 (11:02 -0600)]
ceph-dev-pipeline: Improve podman cleanup
We're seeing a lot of instances of jobs failing to start due to left-over files
from the containerized build. When calling chown, reset the group as well. Also,
mention some details about the build to make log analysis easier.
Ilya Dryomov [Thu, 14 Aug 2025 09:59:58 +0000 (11:59 +0200)]
kernel: install bc
bc has been required for building the Linux kernel for a long time.
It was installed explicitly (albeit from the wrong script) in the deb
case, but not installed in the rpm case:
error: Failed build dependencies:
bc is needed by kernel-6.17.0_rc1_g0dccf5229fc7-1.x86_64
make[2]: *** [scripts/Makefile.package:75: binrpm-pkg] Error 11
Dan Mick [Tue, 29 Jul 2025 18:24:40 +0000 (11:24 -0700)]
Revert "removed build_utils dependency from ceph pull request job"
Jobs are using more than just the 'docs check' functions in
build_utils.sh (ssh_exec, get_nr_build_jobs are two that showed up
after merge). Revert this until it can be more-carefully evaluated.
Dan Mick [Mon, 28 Jul 2025 22:14:37 +0000 (15:14 -0700)]
sync-pull: in the 'ugly loop', skip nonexistent repos
This heuristic remains questionable, but continue it on
by allowing some named distro/version repos to not exist without
stopping sync-pull from working
projects / ceph-build.git / log