Dan Mick [Wed, 1 Oct 2025 02:23:34 +0000 (19:23 -0700)]
builder.yml: make playbook more flexible for finding templates
Many things are relative to 'playbook_dir', but some people
keep a symlink to examples/builder.yml and some don't. I can't
think of a way to fix, say, the module library path 'library/',
but I can make it better for templates.
Dan Mick [Wed, 1 Oct 2025 02:20:44 +0000 (19:20 -0700)]
builder.yml: update to use virtualenv
pip/python no longer like to install modules in global dirs.
Create a virtualenv and use it when running the jenkins_node
module.
This is done by adding a second play to start that creates the
virtualenv and sets ansible_python_interpreter to the interpreter
in the venv. That means it affects the following play in the
playbook, which is the original playbook. So technically the
virtualenv python is used for all plays.
Also, remove the #!/usr/bin/python3 from library/jenkins_node,
which was doing no good, but caused havoc by execing the non-venv
python when running the module.
Dan Mick [Tue, 7 Oct 2025 17:15:49 +0000 (10:15 -0700)]
ceph-build-pull-requsts: unpin urllib3
It probably shouldn't have been pinned to a specific version
anyway, but because it had custom import code, it broke with
the Python in noble (3.13 I think). Removing the pin gets
a version without the problem (I think the custom import
code went away altogether).
Dan Mick [Fri, 3 Oct 2025 23:14:27 +0000 (16:14 -0700)]
ceph-pull-requests*: require node with centos9 or noble
When containerized, we need a newer podman. Limit to these
nodes for now; eventually that will cover almost every node
as we transition to noble from jammy.
John Mulligan [Wed, 12 Mar 2025 20:36:49 +0000 (16:36 -0400)]
ceph-pull-requests-arm64: convert job to run make check in a container
Convert the test script to run make check using the
build-with-container.py tool. Converts env vars that were directly
passed to the script to use a env file.
Remove the redundant n_test_jobs variable since it is *always* the
same as n_build_jobs.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 12 Mar 2025 20:37:19 +0000 (16:37 -0400)]
ceph-pull-requests: convert job to run make check in a container
Convert the test script to run make check using the
build-with-container.py tool. Converts env vars that were directly
passed to the script to use a env file.
Updates the timeout to 4hour (to match the arm64 job) and avoid timeouts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Dan Mick [Thu, 25 Sep 2025 20:58:38 +0000 (13:58 -0700)]
Update jenkins-job-builder spec to released version
jenkins-job-builder finally released 6.4.3, which includes
https://review.opendev.org/c/jjb/jenkins-job-builder/+/941310,
so we can stop using the git url
Dan Mick [Tue, 23 Sep 2025 02:28:14 +0000 (19:28 -0700)]
scripts/build_utils.sh: only do gcc ppa version dance on Ubuntu
This doesn't apply to debian. Maybe debian has similar needs,
but who knows, because we don't test the builds (or indeed
even do them until release time), but this code is definitely
not appropriate for bookworm, so avoid it for now.
We do a shallow clone, but I have seen a few branches that fail because the
clone is _too_ shallow. In rare cases like this we can simply deepen until we
find the tag.
If sccache is supposed to be enabled, but isn't - for example due to the feature
not being backported - we were still attempting to archive, and hanging. Check
for the existence of the file directly.
David Galloway [Wed, 20 Aug 2025 21:57:51 +0000 (17:57 -0400)]
ceph-dev-pipeline: Support for upstream builds
- Variable reading/writing was reordered so that job logic
can override CEPH_REPO, SHA1, and chacra_url.
- For now, --rpm-no-match-sha avoids https://tracker.ceph.com/issues/72626
- Optionally archive sccache artifacts only if we used sccache. Avoids
the job hanging at the end (yes, I know allowEmptyArchive: true should
take care of it.. 2.jenkins didn't care.)
Signed-off-by: David Galloway <david.galloway@ibm.com>
David Galloway [Wed, 20 Aug 2025 21:51:04 +0000 (17:51 -0400)]
setup_chacractl: Only ask shaman for chacra node if chacra_url's unset
In a Release Build, we want to push the packages to chacra.ceph.com.
So we manually define chacra_url=https://chacra.ceph.com in ceph-source-dist
by writing it to dist/other_envvars
Signed-off-by: David Galloway <david.galloway@ibm.com>
David Galloway [Wed, 20 Aug 2025 21:45:14 +0000 (17:45 -0400)]
ceph-source-dist: Support for ceph-release-pipeline
Again, the changes needed here are:
- For a Release Build, we are building a tarball based off a
version commit that has been created by the ceph-tag job
then pushed to ceph-releases.git. So instead of taking
the BRANCH from an upstream Jenkins job, we're reading
the version commit SHA1 from the sha1.txt file that ceph-tag
wrote then checking /that/ out and building the tarball from it.
So some var writing and reading was shuffled around.
- We do not want this job to support checking out from ceph-private.git
or ceph-releases.git directly. Instead, if ceph-release-pipeline
passed RELEASE_BUILD=true, only then can you clone from those private
repos. So we override CEPH_REPO and set chacra_url to the chacra
instance that doesn't automatically prune repos (chacra.ceph.com).
Signed-off-by: David Galloway <david.galloway@ibm.com>
David Galloway [Wed, 20 Aug 2025 21:22:21 +0000 (17:22 -0400)]
ceph-tag: Write a sha1 file for later consumption
In a regular build, ceph-dev-pipeline just takes a branch name and makes a source tarball in ceph-source-dist. ceph-dev-pipeline takes that tarball from the ceph-source-dist job and builds from it so it's able to just get the SHA1 from it.
In a /release/ build, a release branch (e.g., reef-release) is specified and we add an additional version commit on top of it. So we don't know the SHA1 just based on the branch name. My solution was to write the version commit's SHA1 to a file and consume it later.
Signed-off-by: David Galloway <david.galloway@ibm.com>
Zack Cerza [Thu, 21 Aug 2025 17:02:12 +0000 (11:02 -0600)]
ceph-dev-pipeline: Improve podman cleanup
We're seeing a lot of instances of jobs failing to start due to left-over files
from the containerized build. When calling chown, reset the group as well. Also,
mention some details about the build to make log analysis easier.
Ilya Dryomov [Thu, 14 Aug 2025 09:59:58 +0000 (11:59 +0200)]
kernel: install bc
bc has been required for building the Linux kernel for a long time.
It was installed explicitly (albeit from the wrong script) in the deb
case, but not installed in the rpm case:
error: Failed build dependencies:
bc is needed by kernel-6.17.0_rc1_g0dccf5229fc7-1.x86_64
make[2]: *** [scripts/Makefile.package:75: binrpm-pkg] Error 11
projects / ceph-build.git / log