Merge pull request #2562 from ceph/start-with-libvirt-group

cephadm-e2e: wrap kcli commands in libvirt group

Merge pull request #2567 from shraddhaag/wip-support-debug-rocky10

introduce debug builds for rocky10

introduce debug builds for rocky10

This commit adds debug builds for rocky10. So far we are only building
release builds for rocky10. Debug builds are helpful for testing,
especially for crimson.

Fixes: https://tracker.ceph.com/issues/75952
Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

cephadm-e2e: wrap kcli commands in libvirt group

as per: https://github.com/ceph/ceph-build/pull/2458#discussion_r3009905844

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2565 from dmick/version-only-for-v21

ceph-release-containers: only use --version on v21 and later: fix missing fi

ceph-release-containers: only use --version on v21 and later

This should be temporary and replaced with a proper backport; doing
this to allow v20.2.1 to be released without rebuilding packages

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2564 from dmick/version-only-for-v21

ceph-release-containers: only use --version on v21 and later

ceph-release-containers: only use --version on v21 and later

This should be temporary and replaced with a proper backport; doing
this to allow v20.2.1 to be released without rebuilding packages

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2563 from ceph/el10_functional_test

cephadm-ansible-prs: add el10-functional job

cephadm-ansible-prs: add el10-functional job

Signed-off-by: Teoman ONAY <tonay@ibm.com>

Merge pull request #2561 from ceph/release-tar

ceph-source-dist: Decompress the bzip2 to stdout

ceph-source-dist: Decompress the bzip2 to stdout

The original bunzip didn't like the `ln ceph-$ceph_version_tarball.$extension dist/` hard link.

bunzip2 refuses to decompress in-place when a file has multiple hard links, because doing so would silently corrupt the other link

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2458 from rhcs-dashboard/fix-docker-issue

cephadm-dashboard-e2e: replace docker with podman

cephadm-dashboard-e2e: replace docker with podman

- cleanup the docker installation if its installed through docker-ce

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2560 from ceph/c-o-r-e

ceph-rook-e2e fixes

ceph-rook-e2e: Try podman in cleanup too

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-rook-e2e: Add debug line

Signed-off-by: David Galloway <david.galloway@ibm.com>

rook-e2e: Don't use docker

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-rook-e2e: Fix ghprb

It wasn't possible to run this manually.  These changes allow putting just a PR # in the parameter for hte job.

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2559 from ceph/hash

setup_container_runtime.sh: refresh bash path after podman removal

setup_container_runtime.sh: refresh bash path after podman removal

If we remove podman in the "Found a very old podman; removing" function, we subsequently run `command -v podman` later in the script but we're hitting a stale cache.

e.g.,

```
+ setup_container_runtime
++ id -nu
+ loginctl enable-linger jenkins-build
+ command -v podman
/usr/bin/podman
+ podman system info
++ podman version -f '{{ lt .Client.Version "4" }}'
+ '[' true = true ']'
+ echo 'Found a very old podman; removing'
Found a very old podman; removing
+ command -v dnf
+ command -v apt
/usr/bin/apt
+ sudo apt remove -y podman

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  buildah catatonit cmake cmake-data conmon containernetworking-plugins
  cython3 dh-elpa-helper dh-exec dh-python docutils-common emacsen-common
  fonts-glyphicons-halflings fuse-overlayfs golang golang-1.18 golang-1.18-doc
  golang-1.18-go golang-1.18-src golang-doc
  golang-github-containernetworking-plugin-dnsname
  golang-github-containers-common golang-github-containers-image golang-go
  golang-src gperf javahelper jsonnet junit4 libaio-dev libavahi-glib1
  libbabeltrace-ctf-dev libbabeltrace-dev libbabeltrace1 libblkid-dev
  libc6-dbg libcap-dev libcap-ng-dev libcryptsetup-dev libcunit1 libcunit1-dev
  libcurl4-openssl-dev libdouble-conversion3 libevent-2.1-7 libevent-dev
  libevent-extra-2.1-7 libevent-openssl-2.1-7 libevent-pthreads-2.1-7
  libfmt-dev libfmt8 libfuse-dev libfuse2 libgcrypt20-dev libgmp-dev
  libgmpxx4ldbl libgnutls-dane0 libgnutls-openssl27 libgnutls28-dev
  libgnutlsxx28 libgoogle-perftools-dev libgoogle-perftools4 libgpg-error-dev
  libhamcrest-java libibverbs-dev libidn2-dev libjs-bootstrap libjs-d3
  libjs-eonasdan-bootstrap-datetimepicker libjs-jquery-hotkeys libjs-moment
  libjs-moment-timezone libjs-mustache libjs-rickshaw libjsoncpp25
  libkeyutils-dev libldap-dev libldap2-dev liblmdb-dev liblttng-ust-common1
  liblttng-ust-ctl5 liblttng-ust-dev liblttng-ust-python-agent1 liblttng-ust1
  liblua5.3-0 liblua5.3-dev liblz4-dev libnbd-dev libnbd0 libnl-3-dev
  libnl-genl-3-dev libnl-route-3-dev libnspr4-dev libnss3-dev libnuma-dev
  liboath-dev liboath0 libostree-1-1 libp11-kit-dev libpcre2-16-0
  libpcre2-32-0 libpcre2-dev libpcre2-posix3 libpython3-all-dev libqt5core5a
  libqt5dbus5 libqt5network5 librabbitmq-dev librabbitmq4 librdkafka++1
  librdkafka-dev librdkafka1 librdmacm-dev libre2-dev libreadline-dev
  librhash0 libselinux1-dev libsepol-dev libsnappy-dev libsnappy1v5
  libsqlite3-dev libtasn1-6-dev libtcmalloc-minimal4 libthrift-0.16.0
  libthrift-dev libudev-dev libunbound8 libunwind-dev liburcu-dev
  libutf8proc-dev libutf8proc2 libxmlsec1-dev libxmlsec1-gcrypt
  libxmlsec1-gnutls libxmlsec1-nss libxslt1-dev libyaml-cpp-dev libyaml-cpp0.7
  libzstd-dev nasm nettle-dev ninja-build nlohmann-json3-dev prometheus
  python3-alabaster python3-all python3-all-dev python3-cheroot
  python3-cherrypy3 python3-coverage python3-dateutil python3-docutils
  python3-grpcio python3-imagesize python3-isodate python3-jaraco.classes
  python3-jaraco.collections python3-jaraco.functools python3-jaraco.text
  python3-jmespath python3-lxml python3-natsort python3-onelogin-saml2
  python3-pluggy python3-portend python3-prettytable python3-protobuf
  python3-py python3-roman python3-snowballstemmer python3-sphinx
  python3-tempora python3-toml python3-wcwidth python3-xmlsec
  python3-xmltodict python3-zc.lockfile sgml-base slirp4netns socat
  sphinx-common tox uidmap uuid-dev valgrind xfslibs-dev xml-core xmlstarlet
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  podman
0 upgraded, 0 newly installed, 1 to remove and 6 not upgraded.
After this operation, 36.6 MB disk space will be freed.
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 218554 files and directories currently installed.)
Removing podman (3.4.4+ds1-1ubuntu1.22.04.3) ...
Processing triggers for man-db (2.10.2-1) ...
+ command -v podman
/usr/bin/podman
+ command -v podman
/usr/bin/podman
++ podman version -f '{{ lt .Client.Version "5.6.1" }}'
/tmp/jenkins512750246180841280.sh: line 29: /usr/bin/podman: No such file or directory
+ '[' '' = true ']'
++ podman version -f '{{ ge .Client.Version "4" }}'
/tmp/jenkins512750246180841280.sh: line 36: /usr/bin/podman: No such file or directory
+ '[' '' = true ']'
+ podman rm -f ceph_build
/tmp/jenkins512750246180841280.sh: line 66: /usr/bin/podman: No such file or directory
```

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2558 from ceph/djgalloway-patch-1

Clarify testing changes w/ JJB in README

Clarify testing changes w/ JJB in README

Merge pull request #2556 from dmick/ceph-release-containers-version

ceph-release-containers: Add VERSION parameter to make-manifest-list invocation

Merge pull request #2557 from ceph/r10-for-tent

ceph-trigger-build: Build Rocky 10 by default

ceph-trigger-build: Build Rocky 10 by default

Except squid and reef

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-release-containers: Add VERSION parameter to make-manifest-list invocation

Signed-off-by: Dan Mick <dmick@ibm.com>

Merge pull request #2553 from phlogistonjohn/jjm-refactor-bwc

refactor bwc related code into common functions

xxx: workaround branches with slashes (again)

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>

Merge pull request #2554 from ceph/dist-on-x86

Revert "ceph-source-dist: Run on any architecture"

Revert "ceph-source-dist: Run on any architecture"

This reverts commit 2a944cd9542c5748316910da97d2c10ddd674a43.

Fixes: https://tracker.ceph.com/issues/75442
I don't have the time to dig into which node dependency is the problem on arm64 but ceph-source-dist runs fine on x86 but not arm64.

ceph-pull-requests-arm64: update file to use new bwc functions

Replace the existing "open coded" implementation with calls to the
recently created bwc.sh functions. This avoids duplication and improves
reuse.

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>

ceph-pull-requests: update file to use new bwc functions

Replace the existing "open coded" implementation with calls to the
recently created bwc.sh functions. This avoids duplication and improves
reuse.

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>

scripts: create bwc.sh a library for funcs around build-with-containers

Copy and make generic various bits of code present in
ceph-pull-requests/build/build and ceph-pull-requests-arm64/build/build
such that they can be replaced in the future by calls to functions
in this file.

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>

Merge pull request #2552 from ceph/nofail

ceph-dev-cron: ls-remote failure is not fatal

ceph-dev-cron: ls-remote failure is not fatal

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2551 from ceph/stupid

sign-rpms: extra \n

sign-rpms: extra \n

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2550 from ceph/get

sync-pull: Check the actual repo readiness

sync-pull: Check the actual repo readiness

This has been broken for.. possibly ever.

--head results in 405 Method Not Allowed

Just a GET results in 200 no matter what, whether the repo exists or not.

```
++ curl -fsSo - --connect-timeout 5 --max-time 10 https://shaman.ceph.com/api/repos/ceph/reef/efac5a54607c13fa50d4822e50242b86e6e446df/rocky/10/flavors/default/
++ jq -r '.[].status // empty'
+ status=
+ '[' '' = ready ']'
+ echo 'Shaman repo for rocky/10 is not ready (status: unknown). Skipping.'
Shaman repo for rocky/10 is not ready (status: unknown). Skipping.
```

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2540 from ceph/sign-256

sign-rpms: Sign using SHA256 algo

Merge pull request #2549 from ceph/branch-name

Make get-tarballs.sh executable

Make get-tarballs.sh executable

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2499 from jamiepryde/umbrella-release

Umbrella kickoff!

Merge pull request #2546 from shraddhaag/wip-shraddhaag-improve-ci-for-docs

disable CI checks for PRs with only docs/QA changes

disable CI checks for PRs with only docs/QA changes

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>

Merge pull request #2548 from dmick/add-arm64-c9-main

Add centos9/arm64 to main builds

Add centos9/arm64 to main builds

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-dev-cron: Add umbrella to Jenkinsfile

Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com>

*: Update scripts and triggers for Umbrella

Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com>

Merge pull request #2547 from ceph/wip-revert-make-check-qa-only

ceph-pull-requests: don't skip qa-only PRs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

ceph-pull-requests: don't skip qa-only PRs

This is a partial revert of commit 78849bdf7d0d ("ceph-pull-requests:
Skip qa-only PRs").  Unfortunately "make check" needs to be run even on
qa-only PRs at least once to get linter coverage for files under qa/.

Revert to running "make check" on qa-only PRs for ceph-pull-requests
but continue skipping it for ceph-pull-requests-arm64.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #2545 from ceph/cb-small

ceph-build-pull-requests fixes

Merge pull request #2543 from ceph/arm64

ceph-dev-cron: Build main for arm64

ceph-build-pull-requests: setuptools for pkg_resources

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-build-pull-requests: Don't reuse previous venv

If we add a pkg to pkgs[@], it won't actually get installed in a reused venv if one exists

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-build-pull-requests: Run on small

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-dev-cron: Comment explaining cfg

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2544 from ceph/csd-noarch

ceph-source-dist: Run on any architecture

ceph-source-dist: Run on any architecture

Signed-off-by: Zack Cerza <zack@cerza.org>

ceph-dev-cron: Build main for arm64

Signed-off-by: David Galloway <david.galloway@ibm.com>

scripts/build_utils: Update for Umbrella

Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com>

Merge pull request #2541 from ceph/cron-poll

ceph-dev-cron: Use straight cron/timed trigger

ceph-dev-cron: Use straight cron/timed trigger

PollSCM was polling ceph-build.git for changes instead of ceph.git.

Signed-off-by: David Galloway <david.galloway@ibm.com>

sign-rpms: Sign using SHA256 algo

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2538 from ceph/api

ceph-api: Use modern label. They've been running on Noble for weeks.

Merge pull request #2533 from ceph/cron-pipe

ceph-dev-cron: Convert to groovy, trigger ceph-dev-pipeline instead

ceph-api: Use modern label. They've been running on Noble for weeks.

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-dev-cron: Convert to groovy, trigger ceph-dev-pipeline instead

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2537 from ceph/pipx

builder.yml: Install pipx

builder.yml: Install pipx

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2536 from ceph/fix-uv-installs

setup_chacractl.sh: Use setup_uv.sh

setup_uv.sh: Do not fail needlessly on apt systems

Signed-off-by: Zack Cerza <zack@cerza.org>

setup_chacractl.sh: Use setup_uv.sh

Signed-off-by: Zack Cerza <zack@cerza.org>

Merge pull request #2535 from rhcs-dashboard/qa-not-ignore

ceph-pr-api: don't ignore qa/ folder changes

ceph-pr-api: don't ignore qa/ folder changes

API tests are run from qa/ folder so let's not ignore those changes

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2534 from ceph/fix-rebuild

ceph-tag: Fix condition for RCs

ceph-tag: Fix condition for RCs

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2532 from ceph/trixie

Add Trixie

Add Trixie

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2531 from Matan-B/wip-matanb-debug-container-prune

quay-pruner/build/util.py: handle debug tag suffix

quay-pruner/build/util.py: handle debug tag suffix

Follow-up to: https://github.com/ceph/ceph/pull/67041

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #2530 from rhcs-dashboard/sync-images-revert-fix

sync-images: fix build script

sync-images: fix build script

i accidentally merged the last PR but forgot to push my local changes to
the PR before doing it. I was testing things locally and thought I
pushed it. So adding the fix that were in my local.

I added GODEBUG=http2client=0  to fix the http client related timeouts
that were happening frequently. And also added a retry so that it won't
fail immediately.

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2524 from rhcs-dashboard/sync-image-fix

sync-images: failing due to ratelimiting in dockerhub

Merge pull request #2529 from ceph/libvirt-restart

scripts: Refactor restart_libvirt_services

scripts: Refactor restart_libvirt_services

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2528 from ceph/cephadm-ansible-py3.12

cephadm-ansible-prs: do not pin to pip 22.4.0

cephadm-ansible-prs: do not pin to pip 22.4.0

given that the builder is now using py3.12, we can't pin to 22.4.0 as it
depends on distutils that was dropped from py3.12

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2527 from ceph/cephadm2

cephadm-ansible-prs: Set !centos9 on both labels

cephadm-ansible-prs: Set !centos9 on both labels

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #2526 from ceph/cephadm-prs-update

cephadm-ansible-prs: Don't run on centos9

cephadm-ansible-prs: Don't run on centos9

95% of the builders are Ubuntu Noble and most of them have vagrant & vagrant-libvirt installed

Signed-off-by: David Galloway <david.galloway@ibm.com>

sync-images: failing due to ratelimiting in dockerhub

So trying to authenticate which might increase the ratelimit

https://jenkins.ceph.com/job/sync-images/651/console
```
Copying image sha256:a99b7d9812dbb7527eab8161807e879c3e66951ddccf21c5302061a651289da6 (15/16)
time="2025-12-25T20:16:10Z" level=fatal msg="Error copying ref \"docker://nginx:1.27.4\": copying image 15/16 from manifest list: determining manifest MIME type for docker://nginx:1.27.4: reading manifest sha256:a99b7d9812dbb7527eab8161807e879c3e66951ddccf21c5302061a651289da6 in docker.io/library/nginx: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
Build step 'Execute shell' marked build as failure
```

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2523 from ceph/migrate-cephadm-ansible-jenkins

migrate cephadm-ansible to the main jenkins

migrate cephadm-ansible to the main jenkins

This commit removes this file in order to make cephadm-ansible CI jobs
use the main jenkins.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2520 from ceph/wip-kernel-ice

kernel: compile ice.ko module

kernel: compile ice.ko module

This driver is needed for

01:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-XXV for SFP (rev 02)
01:00.1 Ethernet controller: Intel Corporation Ethernet Controller E810-XXV for SFP (rev 02)

on the new trial nodes.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #2519 from ceph/no-centos-make-cehck

ceph-pull-request: Don't run make check on CentOS9

ceph-pull-request: Don't run make check on CentOS9

Not ideal but easiest for stabilization for now

Signed-off-by: David Galloway <david.galloway@ibm.com>