sync-images: update worker labels

This updates the labels for the `sync-images` job.
(c8->c9)

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2263 from ceph/migrate-cv-to-c9

ceph-volume: update labels

ceph-volume: update labels

These labels must be updated to make ceph-volume jobs use
`centos9` workers.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2262 from ceph/cv-c9

ceph-volume: use centos9 worker

ceph-volume: use centos9 worker

due to recent EOL of CentOS Stream 8, the workers were migrated to
CentOS Stream 9.

This commit reflect that change to the ceph-volume-scenario job.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2261 from ceph/cv-squid

ceph-volume: add squid release to functional test suite

ceph-volume: add squid release to functional test suite

This adds the squid release support to the function test suite

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2258 from ceph/update-sync-images

sync-images: sync nginx images

sync-images: sync nginx images

This commit makes the job sync nginx images from docker.io
to quay.io
Also, two changes:
  - This makes sure the latest stable image of skopeo is pulled from quay.io
  - Use the filter 'by-semver' filter for grafana too.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2257 from dmick/wip-disable-centos8-kernel-and-ganesha

Disable centos8 kernel and ganesha

kernel, nfs-ganesha: remove centos8

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2256 from rhcs-dashboard/e2e-xfvb-fix

scripts/dashboard: kill existing xvfb process

scripts/dashboard: kill existing xvfb process

To fix the below error on starting cypress
```
[STARTED] Task without title.
[FAILED] Xvfb exited with a non zero exit code.
[FAILED]
[FAILED] There was a problem spawning Xvfb.
[FAILED]
[FAILED] This is likely a problem with your system, permissions, or installation of Xvfb.
[FAILED]
[FAILED] ----------
[FAILED]
[FAILED] Error: _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
[FAILED] _XSERVTransMakeAllCOTSServerListeners: server already running
[FAILED] (EE)
[FAILED] Fatal server error:
[FAILED] (EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE)
[FAILED]
[FAILED] ----------
[FAILED]
[FAILED] Platform: linux-x64 (Ubuntu - 22.04)
[FAILED] Cypress Version: 12.17.4
Xvfb exited with a non zero exit code.

There was a problem spawning Xvfb.

This is likely a problem with your system, permissions, or installation of Xvfb.
```

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #2255 from jdurgin/wip-website-pr-branch

ceph-website-prs: disallow branches named 'main'

Reviewed-by: Dan Mick <dan.mick@redhat.com>

ceph-website-prs: disallow branches named 'main'

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #2253 from ceph/wip-java-11-openjdk

Builders need to run java-11-openjdk in order to connect to the master

Merge pull request #2254 from asm0deuz/remove_el8

cephadm-ansible: remove el8 test scenario

cephadm-ansible: remove el8 test scenario

Due to CentOS 8 eol on May 31, 2024*, cephadm-ansible
testing against CentOS 8 is removed

*https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/

Signed-off-by: Teoman ONAY <tonay@ibm.com>

Builders need to run java-11-openjdk in order to connect to the master

Signed-off-by: Adam Kraitman <akraitma@redhat.com>

Merge pull request #2235 from cbodley/wip-main-no-centos8

ceph-dev: stop building centos8 packages/container on quincy/reef/squid/main

ceph-dev: stop building centos8 packages/container on main

Signed-off-by: Casey Bodley <cbodley@redhat.com>

ceph-dev: stop building centos8 packages/container on squid

ceph-dev: stop building centos8 packages/container on reef

ceph-dev: stop building centos8 packages/container on quincy

Reapply "ceph-dev*: remove centos8 crimson builds"

This reverts commit 91858f3b4eb006655062188f11d2efd5992080ab.

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Conflicts: quincy +jammy from 8afdc0521c0d3e09f4b3efa5e76089cc98196440
ceph-dev-new-trigger/config/definitions/ceph-dev-new-trigger.yml

Merge pull request #2251 from ceph/wip-windows-label

ceph-windows-pull-requests: constrain to windows label

ceph-windows-pull-requests: constrain to windows label

Currently, the intersection of focal && libvirt labels returns some
smithi (nproc == 8) and mira (nproc == 4) nodes, which are inadequate
for the two VMs that the Windows job spawns.  It turns we already have
a windows label, which, after dropping it from mira015, seems to be on
point (i.e. would filter those low CPU nodes out).  Let's use it!

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #2250 from dmick/wip-ceph-container-allow-flavor-specification

ceph-container-build-ceph-base-push-imgs*: allow specifying FLAVOR

ceph-container-build-ceph-base-push-imgs*: allow specifying FLAVOR

Add parameters to set variables that the job scripts will look for
to choose which flavors to build.  This will override the defaults in
ceph-container.git; the defaults are set to the same as
ceph-container.git at this moment, but are probably soon to change to
centos,9.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2247 from dmick/wip-fix-include-with-gh-clone

Replace obsolete "include:' in upstream ansible tasks

Merge pull request #2249 from dmick/wip-fix-ceph-container-yml-names

ceph-container-*: make .yml filenames match job names

ceph-container-*: make .yml filenames match job names

Why intentionally confuse people?

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2248 from dmick/wip-fix-container-job-display-names

ceph-container-build-*: make display-name the job name

Merge pull request #2242 from Svelar/extend_arm_build_time

ceph-pull-requests-arm64: extend timeout to 4h

Merge pull request #2243 from dmick/wip-scripts-ceph-iscsi

Modify scripts/{sync-pull,sign-rpms,sync-push} to handle ceph-iscsi

ceph-container-build-*: make display-name the job name

Some of these were using the *same display-name for different jobs*.
O_o

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph-build-pull-requests: listen for a '.*retest.*' PR comment

Signed-off-by: Dan Mick <dmick@redhat.com>

ansible/requirements: Use private github repo to fix include:

ansible has removed include:.  Create clones of the two
repos in sensu-requirements.yml to change it to import_tasks:.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #2239 from dmick/wip-use-base-deb-mirror

build_utils.sh: re: last commit: one must specify a mirror

Merge pull request #2244 from dmick/wip-docs-only-for-admin-dir

scripts/build_utils.sh: set DOCS_ONLY for PRs with only admin/ files,…

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

scripts/build_utils.sh: set DOCS_ONLY for PRs with only admin/ files, too

DOCS_ONLY is set if the filenames all start with docs/, but it's
also true that if the filenames start with admin/, it's strictly
docs related (discovered for https://github.com/ceph/ceph/pull/57649)

Signed-off-by: Dan Mick <dmick@redhat.com>

scripts/sync-push: add projectname to handle ceph-iscsi

ceph-iscsi also needs a repo file created
Drop el7
Add usage

Signed-off-by: Dan Mick <dan.mick@redhat.com>

scripts/sync-pull: changes for ceph-iscsi, drop el7

Several sections are done only for the 'ceph' project (to
accommodate ceph-iscsi, which has no debs, so doesn't need
to find the most-populated distroversion or create debs,
pull a tarball, etc.).

Also drop el7.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

sign-rpms: add 'project' argument

This allows signing ceph-iscsi packages

Signed-off-by: Dan Mick <dan.mick@redhat.com>

scripts: add vim modelines for editing ease

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2241 from dmick/wip-allow-arm64-for-setup

Allow all the ceph -setup jobs to run on arm64.

ceph-pull-requests-arm64: extend timeout to 4h

Because performance of omani series machine are not so well which caused make check often failed, especially when cache hit was low, extend timeout to 4h seems reasonable.

Ref build duration: https://jenkins.ceph.com/job/ceph-pull-requests-arm64/buildTimeTrend

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #2240 from petrutlucian94/apt_timeout

windows: increase package install timeout

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Allow all the ceph -setup jobs to run on arm64.

They only do file operations to build a source tarball and debian source
package things, so the architecture isn't important.

Signed-off-by: Dan Mick <dmick@redhat.com>

windows: increase package install timeout

One "apt-get install" command times out after the default 30 seconds.
We'll increase the timeout to 10m.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

build_utils.sh: re: last commit: one must specify a mirror

To pbuild Debian on Ubuntu (as we do), one must specify a mirror
that contains the Debian environments.  Use the most-generic one.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #2238 from dmick/wip-remove-deb-mirror

build_utils.sh: remove pbuilder's --mirror option for Debian

build_utils.sh: remove pbuilder's --mirror option for Debian

The existing mirror is defunct, and hard-stopping bookworm builds.
For now, let's just revert to not specifying a mirror (as we have
in the past for other repo configurations) and solve any problems
that result if they pop up.

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #2222 from dmick/wip-add-c9-containers

ceph-dev-build,ceph-dev-new-build: build c9 containers

ceph-dev-build,ceph-dev-new-build: build c9 containers

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #2234 from dmick/wip-chacra-postgres-version

chacra-pull-requests: set postgres version for jammy

chacra-pull-requests: set postgres version for jammy

next time: can't we just use a generic "install postgresql"

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #2233 from dmick/wip-deprecated-include

{chacra,shaman}-pull-requests: replace deprecated include:

{chacra,shaman}-pull-requests: replace deprecated include:

Signed-off-by: Dan Mick <dmick@redhat.com>

Merge pull request #2232 from petrutlucian94/log_timeout

windows: increase log collection timeout

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

windows: increase log collection timeout

Some failed Windows jobs are missing the build artifacts and it
seems like the log collection function times out.

https://jenkins.ceph.com/job/ceph-windows-pull-requests/39380/

We'll increase the timeout from 30s to 10m.

While at it, we're updating the scp_download and scp_upload helpers,
checking for timeout errors.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #2231 from ceph/wip-nuke-ceph-doc-releases-rtd

remove ceph-doc-releases-rtd job

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

remove ceph-doc-releases-rtd job

Rebuilding doc/releases in release branches was replaced with
redirects:

  https://github.com/ceph/ceph/pull/48800
  https://github.com/ceph/ceph/pull/48945

Also, this job never actually ran because shallow cloning is
incompatible with included-regions filter.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #2230 from ceph/wip-lusov-build-timestamps

ceph-dev-new-build: add timestamps to the build log

ceph-dev-new-build: add timestamps to the build log

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>

Merge pull request #2229 from petrutlucian94/increase_timeouts

windows: increase timeouts

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

windows: increase timeouts

We've decreased the amount of resources allocated to Windows PR test
jobs but as a result we're now experiencing timeouts.

We'll double the timeouts used by our test scripts.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #2228 from petrutlucian94/win_reduce_vcpus

windows: limit number of Linux VM vcpus

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

windows: limit number of Linux VM vcpus

We reduced the amount of memory allocated to Linux VMs used by the
Windows PR jobs to 16GB but we're now hitting OOM errors while
building Ceph.

To mitigate this, we'll limit the number of parallel jobs and
allocated vcpus.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #2227 from petrutlucian94/reduce_win_mem

windows: reduce Linux VM memory

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

windows: reduce Linux VM memory

There have been a lot of Windows job failures lately due to
qemu errors caused by insufficient Jenkins slave memory:

  ERROR    internal error: process exited while connecting to
  monitor: 2024-04-21T11:37:28.922183Z qemu-system-x86_64: cannot
  set up guest memory 'pc.ram': Cannot allocate memory

Each Windows job spins up two virtual machines: a Linux one with
64GB of ram and a Windows VM with 8GB of ram. Since the Windows job
no longer uses memstore, we can probably reduce the Linux vm memory
to 16GB.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #2226 from ceph/cv-drop-pacific

ceph-volume: drop pacific nightly testing

ceph-volume: drop pacific nightly testing

This drops the pacific nightly ceph-volume testing

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2225 from ceph/cv-test-suite

ceph-volume: update 'test suite' definition

ceph-volume: update 'test suite' definition

corresponding PR: ceph/ceph#56827

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2224 from ceph/ceph-volume_drop_fs

ceph-volume: drop filestore jobs

ceph-volume: drop filestore jobs

filestore is no longer supported.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2223 from ceph/ceph-volume-update-tests

ceph-volume: update jobs definition

ceph-volume: update jobs definition

drop filestore testing
make centos distro value more generic (el8 and el9)

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #2221 from cbodley/wip-squid-no-focal

ceph-dev-new-trigger: stop building focal packages on squid

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #2220 from cbodley/wip-main-no-focal

ceph-dev-new-trigger: stop building focal packages on main

ceph-dev-new-trigger: stop building focal packages on squid

Signed-off-by: Casey Bodley <cbodley@redhat.com>

ceph-dev-new-trigger: stop building focal packages on main

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #2216 from ceph/wip-squid

Add squid to ceph-dev-cron and other jobs (except for ceph-volume)

Reviewed-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #2214 from dmick/wip-add-arm64-centos9

ceph-dev-new-build: allow arm64 builds of centos9

Merge pull request #2206 from cbodley/wip-quincy-on-jammy

ceph-dev-new-trigger: start building quincy packages on jammy

Merge pull request #2215 from Svelar/arm_ci

ceph-pull-requests-arm64: make publishers same as ceph-pull-requests

Merge pull request #2217 from Svelar/zbd

ceph-pull-requests: remove WITH_ZBD

Merge pull request #2218 from Svelar/arm64_RWL

ceph-pull-requests-arm64: enable RWL mode on arm64

ceph-pull-requests-arm64: enable RWL mode on arm64

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

ceph-pull-requests: remove WITH_ZBD

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Update releases - add squid, remove pacific and older

... with the exception of ceph-volume jobs.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

ceph-dev-trigger: fix a typo a regex for reef

It should be .*reef.*, not .*main.*.  This was introduced in commit
a19480349163 ("Update releases - add reef, remove octopus and older"),
but didn't really affect anything because the build matrix happens to
be the same.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

ceph-pull-requests-arm64: add kill tests when aborted to arm64

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

ceph-pull-requests-arm64: add publishers to fix ci result inaccuracy

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #2213 from clwluvw/collect-logs

build_utils: move collect lgos playbook into the root dir

ceph-dev-new-build: allow arm64 builds of centos9

Signed-off-by: Dan Mick <dmick@redhat.com>

build_utils: move collect lgos playbook into the root dir

The playbook needs to be in the root directory so it can discover the roles

Signed-off-by: Seena Fallah <seenafallah@gmail.com>

Merge pull request #2212 from clwluvw/ceph-ansible-collect-logs

build_utils: use custom playbook for collect_ceph_logs

build_utils: use custom playbook for collect_ceph_logs

If provided, use the custom playbook to collect logs.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>