setup_container_runtime.sh: Version comparison

... using Go templates instead of jq

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2436 from ceph/cdp-kmsgs

ceph-dev-pipeline: Report kernel msgs in post

ceph-dev-pipeline: Report kernel msgs in post

The inspiration is to help us find instances of OOM kills, but may be helpful
with other issues.

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2435 from rhcs-dashboard/fix-nvm

cephadm-dashboard-e2e: use node version from dashboard/frontend

Reviewed-by: Afreen Misbah <afreen@ibm.com>

cephadm-dashboard-e2e: use node version from dashboard/frontend

via .nvmrc

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2418 from bluikko/patch-1

dashboard-cephadm-e2e: fix and silence a few errors

Merge pull request #2431 from ceph/cdp-wo-sha1

ceph-dev-pipeline: Do not require SHA1 property

Merge pull request #2433 from ceph/ctb-no-tags

ceph-trigger-build: Do not trigger on tag pushes

ceph-dev-pipeline: Do not require SHA1 property

dd70a303 inadvertently made the SHA1 property mandatory; correct that.

Fixes: https://tracker.ceph.com/issues/72693
Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Do not trigger on tag pushes

This pipeline doesn't support tags, and it doesn't seem like shaman does either.

Fixes: https://tracker.ceph.com/issues/72693
Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2434 from ceph/cdns

setup_cont_runtime: Move podman debugging info inside command -v

setup_cont_runtime: Move podman debugging info inside command -v

Fails jobs if podman is not installed already

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2427 from ceph/revert-cdp

Revert "ceph-trigger-build: Default to ceph-dev-pipeline"

Merge pull request #2430 from ceph/container-fixes

setup_container_runtime.sh: Reset podman

setup_container_runtime.sh: Reset podman

... if `podman system info` fails.

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2429 from ceph/shellcheck

Fix more shellcheck errors

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Fix more shellcheck errors

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2428 from ceph/container-fixes

ceph-dev-pipeline: fixes for loginctl and chown

ceph-dev-pipeline: Improve podman cleanup

We're seeing a lot of instances of jobs failing to start due to left-over files
from the containerized build. When calling chown, reset the group as well. Also,
mention some details about the build to make log analysis easier.

Signed-off-by: Zack Cerza <zack@cerza.org>

setup_container_runtime.sh: Always enable linger

Signed-off-by: Zack Cerza <zack@cerza.org>

Revert "ceph-trigger-build: Default to ceph-dev-pipeline"

This reverts commit 432cc024fc03daa59b2af0523d333ed654f1a2ac.

Merge pull request #2416 from ceph/shellcheck

ceph-build-pull-requests: Use shellcheck

ceph-build-pull-requests: Use shellcheck_job.py

Signed-off-by: Zack Cerza <zack@cerza.org>

Add scripts/shellcheck_job.py

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2426 from ceph/ctb-cdp

ceph-trigger-build: Default to ceph-dev-pipeline

ceph-trigger-build: Default to ceph-dev-pipeline

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2425 from ceph/wip-kernel-bc-dependency

kernel: install bc

kernel: install bc

bc has been required for building the Linux kernel for a long time.
It was installed explicitly (albeit from the wrong script) in the deb
case, but not installed in the rpm case:

  error: Failed build dependencies:
   bc is needed by kernel-6.17.0_rc1_g0dccf5229fc7-1.x86_64
  make[2]: *** [scripts/Makefile.package:75: binrpm-pkg] Error 11

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #2412 from ceph/specific-sha1

Build specific commits with new build pipelines

Merge pull request #2423 from ceph/no-admins

Remove admin lists from ghprb configs

Remove admin lists from ghprb configs

These will be replaced by proper RABC eventually

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2422 from ceph/fix-bu-sc-errs

build_utils.sh: Fix shellcheck errors

build_utils.sh: Fix shellcheck errors

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2420 from ceph/scr-fixes

setup_container_runtime.sh: Drop bc

setup_container_runtime.sh: Fix shellcheck msgs

Signed-off-by: Zack Cerza <zack@cerza.org>

setup_container_runtime.sh: Drop bc

Use bash instead for math. Thanks to John Mulligan for the suggestion.

Signed-off-by: Zack Cerza <zack@cerza.org>

dashboard-cephadm-e2e: silence a few errors

Change directory before trying to open files from it.
Silences one Jenkins Indication.

Don't try to do docker logs unless docker had something running.
Removes some error lines in Jenkins console log.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>

Merge pull request #2417 from ceph/builder-wget

builder.yml: Install wget

builder.yml: Install wget

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2415 from ceph/cdnb-scr

ceph-dev-new-build: add setup_container_runtime.sh

ceph-dev-new-build: add setup_container_runtime.sh

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2387 from ceph/wip-builders-upgrade

builder.yml: Add tasks to update builder

Merge pull request #2411 from ceph/cdp-desc-links

ceph-dev-pipeline: Link to branch, commit

Merge pull request #2414 from dmick/revert-build-utils

Revert "removed build_utils dependency from ceph pull request job"

Revert "removed build_utils dependency from ceph pull request job"

Jobs are using more than just the 'docs check' functions in
build_utils.sh (ssh_exec, get_nr_build_jobs are two that showed up
after merge).  Revert this until it can be more-carefully evaluated.

This reverts commit 28871177b6102eb3b17e260341eaa05a861628c1.

Merge pull request #2413 from dmick/sync-pull-skip-nonexistent-deb-repos

sync-pull: in the 'ugly loop', skip nonexistent repos

Merge pull request #2393 from ceph/ceph_pr_job_refactor

removed build_utils dependency from ceph pull request job

sync-pull: in the 'ugly loop', skip nonexistent repos

This heuristic remains questionable, but continue it on
by allowing some named distro/version repos to not exist without
stopping sync-pull from working

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2405 from cbodley/wip-noble-install-gcc14

scripts/build_utils: setup_pbuilder_for_old_gcc() installs requested version

ceph-dev-pipeline: Consume SHA1 parameter

If SHA1 is passed, ensure it is reflected in the consumed artifact.

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-source-dist: Consume SHA1 parameter

If SHA1 is passed, check out the commit directly as opposed to just the branch

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Pass SHA1 param to child job

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2409 from ceph/cdp-rocky9

ceph-dev-pipeline: Add rocky9 target

ceph-dev-pipeline: Link to branch, commit

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2410 from rhcs-dashboard/dashboard-node-install

dashboard-cephadm-e2e: install node via nvm

Reviewed-by: Nizamudeen A <nia@redhat.com>

dashboard-cephadm-e2e: install node via nvm

to avoid pinning the node versions

Signed-off-by: Nizamudeen A <nia@redhat.com>

ceph-dev-pipeline: Add rocky9 target

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2408 from ceph/cdp-rocky

ceph-dev-pipeline: Don't reject rocky

ceph-dev-pipeline: Don't reject rocky

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2406 from dmick/setup-container-runtime

ceph-release-container: add setup_container_runtime.sh

ceph-release-container: add setup_container_runtime.sh

A ceph-release-container run suffered from

error running container: from /usr/bin/crun creating container for
[<sh command>]: sd-bus call: Interactive authentication required.:
Permission denied

That is thought to be because of the lack of 'linger' on the
jenkins-build user, such that the /run/user directory didn't
persist.  The exact mechanism of failure isn't fully understood,
but this script has fixed similar issues for ceph-dev-pipeline.

Also, add an echo of the node for convenience.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

scripts/build_utils: setup_pbuilder_for_old_gcc() installs requested version

ubuntu noble is special in that we gcc-14 is available but gcc-13 is the
default. before using update-alternatives to prefer gcc-14, we need to
make sure it's installed

copy the relevant bit from setup_pbuilder_for_new_gcc()'s hook
D05install-new-gcc to make sure the requested version is installed
before running the D10update-gcc-alternatives hook

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #2404 from cbodley/wip-noble-gcc14

scripts/build_utils: default to gcc-14 for ubuntu noble

scripts/build_utils: default to gcc-14 for ubuntu noble

noble provides both gcc-13 and 14, but its gcc 13.2.0 suffers from an
LTO bug that causes ceph daemons to crash horribly (tracked in
https://tracker.ceph.com/issues/63867)

Signed-off-by: Casey Bodley <cbodley@redhat.com>

removed build_utils dependency from ceph pull request job

Merge pull request #2391 from ceph/doc-trlr-fix

ceph-trigger-build: Clarify build job selection

Merge pull request #2402 from ceph/podman-linger-earlier

Enable linger earlier for podman

Enable linger earlier for podman

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2397 from ceph/cdp-cw

ceph-dev-pipeline: Clean up workspace in post

ceph-dev-pipeline: Clean up workspace in post

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2396 from ceph/loginctl-linger

Enable linger for the Jenkins user

Enable linger for the Jenkins user

This fixes errors like:
RunRoot is pointing to a path (/run/user/$ID/containers) which is not writable. Most likely podman will fail.
Error: creating events dirs: mkdir /run/user/$ID: permission denied

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Clarify build job selection

Signed-off-by: Zack Cerza <zack@cerza.org>

Changes and fixes I have made to the builder.yml
 - Installing java-21 insted of java-17(EOL)
 - Stoping unattended-upgrades to avoid conflicts
 - Refreshes the apt package cache to ensure the system has the latest package metadata before performing upgrades
 - Upgrades all installed packages to their latest versions on debian based systems and cleans up unused dependencies and cached files
 - Updates the package cache and upgrades all packages to their latest versions on redhat based systems
 - Updates the package cache and upgrades all packages on suse based systems
 - Fixing the task - "get github host key from github if necessary" that checks github_host_key == "" but since github_host_key is undefined, ansible throws an error
 - Fixing the taks - "Install the systemd unit files for jenkins" by using playbook_dir for explicit path resolution

Signed-off-by: Adam Kraitman <akraitma@li-8b09b2cc-35b7-11b2-a85c-cd1dbade58f9.ibm.com>

Merge pull request #2384 from ceph/wip-disable-cdnt

ceph-dev-new-trigger: Disable

Merge pull request #2390 from ceph/doc-trlr-fix

ceph-trigger-build: correct git trailer examples

ceph-trigger-build: correct git trailer examples

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2389 from ceph/trigger-pipeline

ceph-trigger-build: Build noble by default

ceph-trigger-build: Build noble by default

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2371 from ceph/pipeline-container

ceph-dev-pipeline: Set CEPH_BUILD_NORMALIZE_PATHS

Merge pull request #2381 from dmick/wip-fog-capture

sepia-fog-images: add image test to 'poll for deploy' loop

Merge pull request #2382 from ceph/wip-noble

Support for Ubuntu Noble

Support for Ubuntu Noble

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2388 from ceph/ctb-crimson

ceph-trigger-build: Correct crimson-only behavior

Merge pull request #2385 from ceph/wip-trailer-docs

ceph-trigger-build: Update docs

ceph-trigger-build: Correct crimson-only behavior

We were accidentally building default instead of crimson-debug

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Update docs

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-trigger-build: build reef on focal

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Remove extra line break

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Add missing def keywords

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-new-trigger: Disable

Replaced by ceph-trigger-build job

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-trigger-build: Fix newlines in job description

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Correct branch

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: Remove preserve- from name

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2372 from ceph/ceph-trigger-build

Add ceph-trigger-build job

ceph-dev-pipeline: Matrix entries for noble, el10

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-trigger-build: support new crimson flavors

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-pipeline: support new crimson flavors

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-pipeline: potentially build multiple flavors

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-source-dist: drop unused env vars

Signed-off-by: Zack Cerza <zack@cerza.org>