ceph-trigger-build: Do not trigger on tag pushes

This pipeline doesn't support tags, and it doesn't seem like shaman does either.

Fixes: https://tracker.ceph.com/issues/72693
Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2429 from ceph/shellcheck

Fix more shellcheck errors

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Fix more shellcheck errors

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2428 from ceph/container-fixes

ceph-dev-pipeline: fixes for loginctl and chown

ceph-dev-pipeline: Improve podman cleanup

We're seeing a lot of instances of jobs failing to start due to left-over files
from the containerized build. When calling chown, reset the group as well. Also,
mention some details about the build to make log analysis easier.

Signed-off-by: Zack Cerza <zack@cerza.org>

setup_container_runtime.sh: Always enable linger

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2416 from ceph/shellcheck

ceph-build-pull-requests: Use shellcheck

ceph-build-pull-requests: Use shellcheck_job.py

Signed-off-by: Zack Cerza <zack@cerza.org>

Add scripts/shellcheck_job.py

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2426 from ceph/ctb-cdp

ceph-trigger-build: Default to ceph-dev-pipeline

ceph-trigger-build: Default to ceph-dev-pipeline

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2425 from ceph/wip-kernel-bc-dependency

kernel: install bc

kernel: install bc

bc has been required for building the Linux kernel for a long time.
It was installed explicitly (albeit from the wrong script) in the deb
case, but not installed in the rpm case:

  error: Failed build dependencies:
   bc is needed by kernel-6.17.0_rc1_g0dccf5229fc7-1.x86_64
  make[2]: *** [scripts/Makefile.package:75: binrpm-pkg] Error 11

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #2412 from ceph/specific-sha1

Build specific commits with new build pipelines

Merge pull request #2423 from ceph/no-admins

Remove admin lists from ghprb configs

Remove admin lists from ghprb configs

These will be replaced by proper RABC eventually

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2422 from ceph/fix-bu-sc-errs

build_utils.sh: Fix shellcheck errors

build_utils.sh: Fix shellcheck errors

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2420 from ceph/scr-fixes

setup_container_runtime.sh: Drop bc

setup_container_runtime.sh: Fix shellcheck msgs

Signed-off-by: Zack Cerza <zack@cerza.org>

setup_container_runtime.sh: Drop bc

Use bash instead for math. Thanks to John Mulligan for the suggestion.

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2417 from ceph/builder-wget

builder.yml: Install wget

builder.yml: Install wget

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2415 from ceph/cdnb-scr

ceph-dev-new-build: add setup_container_runtime.sh

ceph-dev-new-build: add setup_container_runtime.sh

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2387 from ceph/wip-builders-upgrade

builder.yml: Add tasks to update builder

Merge pull request #2411 from ceph/cdp-desc-links

ceph-dev-pipeline: Link to branch, commit

Merge pull request #2414 from dmick/revert-build-utils

Revert "removed build_utils dependency from ceph pull request job"

Revert "removed build_utils dependency from ceph pull request job"

Jobs are using more than just the 'docs check' functions in
build_utils.sh (ssh_exec, get_nr_build_jobs are two that showed up
after merge).  Revert this until it can be more-carefully evaluated.

This reverts commit 28871177b6102eb3b17e260341eaa05a861628c1.

Merge pull request #2413 from dmick/sync-pull-skip-nonexistent-deb-repos

sync-pull: in the 'ugly loop', skip nonexistent repos

Merge pull request #2393 from ceph/ceph_pr_job_refactor

removed build_utils dependency from ceph pull request job

sync-pull: in the 'ugly loop', skip nonexistent repos

This heuristic remains questionable, but continue it on
by allowing some named distro/version repos to not exist without
stopping sync-pull from working

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2405 from cbodley/wip-noble-install-gcc14

scripts/build_utils: setup_pbuilder_for_old_gcc() installs requested version

ceph-dev-pipeline: Consume SHA1 parameter

If SHA1 is passed, ensure it is reflected in the consumed artifact.

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-source-dist: Consume SHA1 parameter

If SHA1 is passed, check out the commit directly as opposed to just the branch

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Pass SHA1 param to child job

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2409 from ceph/cdp-rocky9

ceph-dev-pipeline: Add rocky9 target

ceph-dev-pipeline: Link to branch, commit

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2410 from rhcs-dashboard/dashboard-node-install

dashboard-cephadm-e2e: install node via nvm

Reviewed-by: Nizamudeen A <nia@redhat.com>

dashboard-cephadm-e2e: install node via nvm

to avoid pinning the node versions

Signed-off-by: Nizamudeen A <nia@redhat.com>

ceph-dev-pipeline: Add rocky9 target

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2408 from ceph/cdp-rocky

ceph-dev-pipeline: Don't reject rocky

ceph-dev-pipeline: Don't reject rocky

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2406 from dmick/setup-container-runtime

ceph-release-container: add setup_container_runtime.sh

ceph-release-container: add setup_container_runtime.sh

A ceph-release-container run suffered from

error running container: from /usr/bin/crun creating container for
[<sh command>]: sd-bus call: Interactive authentication required.:
Permission denied

That is thought to be because of the lack of 'linger' on the
jenkins-build user, such that the /run/user directory didn't
persist.  The exact mechanism of failure isn't fully understood,
but this script has fixed similar issues for ceph-dev-pipeline.

Also, add an echo of the node for convenience.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

scripts/build_utils: setup_pbuilder_for_old_gcc() installs requested version

ubuntu noble is special in that we gcc-14 is available but gcc-13 is the
default. before using update-alternatives to prefer gcc-14, we need to
make sure it's installed

copy the relevant bit from setup_pbuilder_for_new_gcc()'s hook
D05install-new-gcc to make sure the requested version is installed
before running the D10update-gcc-alternatives hook

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #2404 from cbodley/wip-noble-gcc14

scripts/build_utils: default to gcc-14 for ubuntu noble

scripts/build_utils: default to gcc-14 for ubuntu noble

noble provides both gcc-13 and 14, but its gcc 13.2.0 suffers from an
LTO bug that causes ceph daemons to crash horribly (tracked in
https://tracker.ceph.com/issues/63867)

Signed-off-by: Casey Bodley <cbodley@redhat.com>

removed build_utils dependency from ceph pull request job

Merge pull request #2391 from ceph/doc-trlr-fix

ceph-trigger-build: Clarify build job selection

Merge pull request #2402 from ceph/podman-linger-earlier

Enable linger earlier for podman

Enable linger earlier for podman

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2397 from ceph/cdp-cw

ceph-dev-pipeline: Clean up workspace in post

ceph-dev-pipeline: Clean up workspace in post

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2396 from ceph/loginctl-linger

Enable linger for the Jenkins user

Enable linger for the Jenkins user

This fixes errors like:
RunRoot is pointing to a path (/run/user/$ID/containers) which is not writable. Most likely podman will fail.
Error: creating events dirs: mkdir /run/user/$ID: permission denied

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Clarify build job selection

Signed-off-by: Zack Cerza <zack@cerza.org>

Changes and fixes I have made to the builder.yml
 - Installing java-21 insted of java-17(EOL)
 - Stoping unattended-upgrades to avoid conflicts
 - Refreshes the apt package cache to ensure the system has the latest package metadata before performing upgrades
 - Upgrades all installed packages to their latest versions on debian based systems and cleans up unused dependencies and cached files
 - Updates the package cache and upgrades all packages to their latest versions on redhat based systems
 - Updates the package cache and upgrades all packages on suse based systems
 - Fixing the task - "get github host key from github if necessary" that checks github_host_key == "" but since github_host_key is undefined, ansible throws an error
 - Fixing the taks - "Install the systemd unit files for jenkins" by using playbook_dir for explicit path resolution

Signed-off-by: Adam Kraitman <akraitma@li-8b09b2cc-35b7-11b2-a85c-cd1dbade58f9.ibm.com>

Merge pull request #2384 from ceph/wip-disable-cdnt

ceph-dev-new-trigger: Disable

Merge pull request #2390 from ceph/doc-trlr-fix

ceph-trigger-build: correct git trailer examples

ceph-trigger-build: correct git trailer examples

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2389 from ceph/trigger-pipeline

ceph-trigger-build: Build noble by default

ceph-trigger-build: Build noble by default

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2371 from ceph/pipeline-container

ceph-dev-pipeline: Set CEPH_BUILD_NORMALIZE_PATHS

Merge pull request #2381 from dmick/wip-fog-capture

sepia-fog-images: add image test to 'poll for deploy' loop

Merge pull request #2382 from ceph/wip-noble

Support for Ubuntu Noble

Support for Ubuntu Noble

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2388 from ceph/ctb-crimson

ceph-trigger-build: Correct crimson-only behavior

Merge pull request #2385 from ceph/wip-trailer-docs

ceph-trigger-build: Update docs

ceph-trigger-build: Correct crimson-only behavior

We were accidentally building default instead of crimson-debug

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Update docs

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-trigger-build: build reef on focal

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Remove extra line break

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Add missing def keywords

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-new-trigger: Disable

Replaced by ceph-trigger-build job

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-trigger-build: Fix newlines in job description

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Correct branch

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Remove preserve- from name

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2372 from ceph/ceph-trigger-build

Add ceph-trigger-build job

ceph-dev-pipeline: Matrix entries for noble, el10

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: support new crimson flavors

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-pipeline: support new crimson flavors

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-pipeline: potentially build multiple flavors

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-source-dist: drop unused env vars

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-pipeline: support the crimson flavor

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-pipeline: drop unused setup job params

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-source-dist: drop flavor param

Signed-off-by: Zack Cerza <zack@cerza.org>

Add ceph-trigger-build job

Signed-off-by: Zack Cerza <zack@cerza.org>

sepia-fog-images: add image test to 'poll for deploy' loop

When checking for scheduled deploy tasks, check not only for existing
deploy tasks but for deploy tasks *for the image we're proposing to
capture*.  There are often deploy tasks on a busy fog server, but we
only need to avoid affecting tasks for the image we're updating

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2380 from ceph/dangling-trigger

ceph-dev-new-trigger: Remove dangling job trigger

ceph-dev-new-trigger: Remove dangling job trigger

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2373 from cbodley/wip-rm-jaeger

remove 'jaeger' flavor now that it's on by default

Merge pull request #2379 from dmick/fix-builder-playbook

Fix up ssh client and host key code; add comments on how to run

builder.yml: fix jenkins ssh key and github.com host key code

Previous code was just wrong; 'first_found' can fail, and does
not return file contents; default(URL) just doesn't work

Signed-off-by: Dan Mick <dan.mick@redhat.com>

builder.yml: fix comments, note importance of invocation path

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2377 from ceph/wip-kernel-reduce-distros

kernel: stop building for trusty, xenial and bionic

Merge pull request #2376 from ceph/wip-kernel-dwarf-h

kernel: install elfutils-devel or libdw-dev(el)

kernel: stop building for trusty, xenial and bionic

These are too old and kernel packages are no longer needed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

kernel: install elfutils-devel or libdw-dev(el)

This is a required dependency for CONFIG_GENDWARFKSYMS since 6.15.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

remove 'jaeger' flavor now that it's on by default

jaeger was enabled by default in all ceph builds in
https://github.com/ceph/ceph/pull/44684 which was present in reef and
later releases. so no supported releases need a flavor to turn it on

ceph-dev-new-trigger hadn't been updated from 'DISTROS=centos8 focal'
so is probably not being used

Signed-off-by: Casey Bodley <cbodley@redhat.com>