Matan Breizman [Sun, 23 Nov 2025 11:41:42 +0000 (13:41 +0200)]
Introduce "debug" flavor
Currently, the only flavor used for testing is "default".
While this ensures that the tested flavor matches the released flavor, it can also be a limitation.
Introducing a debug flavor would allow us to test branches that require additional or more thorough validation.
The main difference is that built-in assertions would be compiled in.
The reasons for this change are:
a) The Crimson suite uses a crimson-debug flavor for project PR gating.
The next commit will clean up the Crimson flavor entirely,
and the new debug flavor introduced here could be used as its replacement.
b) Good practice: having an additional build with debug enabled is useful
when retesting or performing extra checks.
Initially, the new debug flavor will only apply to centos9 builds.
If it proves valuable, we can expand support to other distros.
Note: The current way to schedule debug builds is by using a *-debug branch name.
Having a dedicated flavor seems more straightforward.
Dan Mick [Sat, 15 Nov 2025 00:00:40 +0000 (16:00 -0800)]
review comments:
- explain --stragglers a bit in help text
- add re match group names and use them
- use variables for URLs in messages
- dryrun -> dry-run
- remove dead code
Dan Mick [Tue, 11 Nov 2025 00:17:18 +0000 (16:17 -0800)]
quay-pruner: completely overhaul prune-quay.py
Pruning had stopped working (up to >26000 image tags), and the
reasons were many; one, pruning's always been less deterministic
than I'd hope; two, when I switched us to ceph.git/container for
building images, I mistakenly changed the format of the 'fulltag'
(no longer has a short sha1 in it) and that was sort of driving
the pruning process. three, I suspect some of the newer flavors
etc. were slipping through the cracks.
So here's an attempt to fix all that by changing the algorithm
fundamentally; now, tags of a certain manifest digest are considered
at the same time, and their sha1 checked in shaman as usual (but
only their sha1); if it's found, the tags are all left, and if not,
they're all removed. This should be cleaner, faster, and more
reliable.
Also refactored a lot of the worker routines to util.py so I could
add some helper/debug/info scripts:
get-tagdates.py generates JSON showing tag-to-age for examining the
state of things
delete-tags.py takes tags on the CLI to delete, or can be invoked
with '--stragglers <age>' to remove anything older than age (as long
as it's not in shaman or seems like it might be a 'distinguished' build
(with recent release names in its name)).
prune-quay.py also now reports summary statistics of its operation.
Dan Mick [Tue, 29 Jul 2025 18:24:40 +0000 (11:24 -0700)]
Revert "removed build_utils dependency from ceph pull request job"
Jobs are using more than just the 'docs check' functions in
build_utils.sh (ssh_exec, get_nr_build_jobs are two that showed up
after merge). Revert this until it can be more-carefully evaluated.
Dan Mick [Mon, 28 Jul 2025 22:14:37 +0000 (15:14 -0700)]
sync-pull: in the 'ugly loop', skip nonexistent repos
This heuristic remains questionable, but continue it on
by allowing some named distro/version repos to not exist without
stopping sync-pull from working
error running container: from /usr/bin/crun creating container for
[<sh command>]: sd-bus call: Interactive authentication required.:
Permission denied
That is thought to be because of the lack of 'linger' on the
jenkins-build user, such that the /run/user directory didn't
persist. The exact mechanism of failure isn't fully understood,
but this script has fixed similar issues for ceph-dev-pipeline.
scripts/build_utils: setup_pbuilder_for_old_gcc() installs requested version
ubuntu noble is special in that we gcc-14 is available but gcc-13 is the
default. before using update-alternatives to prefer gcc-14, we need to
make sure it's installed
copy the relevant bit from setup_pbuilder_for_new_gcc()'s hook
D05install-new-gcc to make sure the requested version is installed
before running the D10update-gcc-alternatives hook
scripts/build_utils: default to gcc-14 for ubuntu noble
noble provides both gcc-13 and 14, but its gcc 13.2.0 suffers from an
LTO bug that causes ceph daemons to crash horribly (tracked in
https://tracker.ceph.com/issues/63867)
This fixes errors like:
RunRoot is pointing to a path (/run/user/$ID/containers) which is not writable. Most likely podman will fail.
Error: creating events dirs: mkdir /run/user/$ID: permission denied
Dan Mick [Wed, 18 Jun 2025 17:10:21 +0000 (10:10 -0700)]
sepia-fog-images: add image test to 'poll for deploy' loop
When checking for scheduled deploy tasks, check not only for existing
deploy tasks but for deploy tasks *for the image we're proposing to
capture*. There are often deploy tasks on a busy fog server, but we
only need to avoid affecting tasks for the image we're updating
Casey Bodley [Wed, 4 Jun 2025 20:57:37 +0000 (16:57 -0400)]
remove 'jaeger' flavor now that it's on by default
jaeger was enabled by default in all ceph builds in
https://github.com/ceph/ceph/pull/44684 which was present in reef and
later releases. so no supported releases need a flavor to turn it on
ceph-dev-new-trigger hadn't been updated from 'DISTROS=centos8 focal'
so is probably not being used
Matan Breizman [Wed, 28 May 2025 08:16:50 +0000 (11:16 +0300)]
ceph-dev-cron: Build both crimson flavors
When building the daily main build, use both crimson flavors.
This is critical for the weekly nightly runs.
Note: Usual builds would still only build a *single* crimson-flavor (debug).
This commit only affects the dev-cron
Matan Breizman [Sun, 8 Jun 2025 07:36:48 +0000 (10:36 +0300)]
Seperate Crimson flavors to crimson-debug/release
The following change would allow us to:
* Keep testing "Debug" builds (crimson-debug) by default.
* When crimson-only is used, crimson-release would also be built.
* `crimson-release` builds would be used in the weekly nightlies and on
selected PR gatings (when crimson-debug is not enough).
Note: Once crimson-release is stable enough we could switch crimson-debug
to actaully be of: "RelWithDebInfo + !NDEBUG + asserts" instead of "Debug".
For now, let's keep it as Debug since we know it's stable.
Seena Fallah [Tue, 27 May 2025 19:03:00 +0000 (21:03 +0200)]
vagrant-libvirt: install the latest version
The pinned version is already too old and is failing in CI due to:
```
Error message given during initialization: Unable to resolve dependency:
user requested 'vagrant-libvirt (= 0.12.2)'
```
Let's try the latest and see if the reason for pinning the old one
still exists.
Dan Mick [Mon, 19 May 2025 20:42:51 +0000 (13:42 -0700)]
ceph-build: remove container build step
This was removed from build_rpm, but then when a reorg happened
to separate the function into a separate script, the fact that
ceph-build doesn't do container builds anymore was missed.
Release container builds now happen after the manual signing
step using job ceph-release-containers
Matan Breizman [Sun, 18 May 2025 08:40:52 +0000 (11:40 +0300)]
scripts/build_utils: remove Debug build from Crimson
See: https://github.com/ceph/ceph-build/pull/2167/files
Builds should be set to Debug only when ending in "-debug".
https://github.com/ceph/ceph-build/pull/1805 set Crimson to always use Debug.
That made sense in earlier development phases.
However, today we should align with Classical builds and use Debug only when needed.
projects / ceph-build.git / log