Ionut Balutoiu [Fri, 3 Feb 2023 08:44:40 +0000 (10:44 +0200)]
ceph-windows: Build Ceph on Windows inside libvirt VM
Make sure that the Ceph on Windows build is done inside an Ubuntu
libvirt VM. This ensures that the build is done on a clean environment
on each job run.
After the Windows build is done, the Ubuntu VM is rebuilt to ensure
that Ceph vstart will have a clean environment as well.
Lucian Petrut [Wed, 25 Jan 2023 12:52:45 +0000 (14:52 +0200)]
ceph-windows: update test timeouts
The Python rbd-wnbd tests time out on stable branches. The reason
is that if the test script is missing, we're fetching it from
the main branch and then run the tests individually.
The issue is that while on the main branch the entire suite uses
a 30m timeout, on stable branches we're excuting each individual
test with a 5m timeout, which isn't enough for some of the tests
(e.g. the FIO one).
For this reason, we're going to increase the timeout.
While at it, we're pinning the commit id when fetching the test
script on older branches. That will allow us to move it or
refactor it.
Due to a regression introduced in tox 4.3.1, we have to pin
the version used in the CI in order to avoid the following error:
```
tox.report.HandledError: replace failed in centos8-filestore-create.commands with MatchRecursionError('circular chain between set env VAGRANT_UP_FLAGS, CEPH_DEV_BRANCH, CEPH_DEV_SHA1')
```
Due to a regression introduced in tox 4.3.1, multiple environment variable substitution
is broken.
```
tox.report.HandledError: replace failed in centos-non_container-update.commands with MatchRecursionError('circular chain between set env DOCKER_HUB_USERNAME, DOCKER_HUB_PASSWORD, INVENTORY')
```
Let's pin to the last tox stable version where it used to work fine (4.2.8).
Lucian Petrut [Thu, 24 Nov 2022 08:01:52 +0000 (10:01 +0200)]
ceph-windows: re-enable rbd-wnbd stamp test
We had to disable one of the rbd-wnbd tests while investigating
the reason why disks started coming up as read-only.
This was actually caused by the wnbd bus type, which changed from
virtual to SAS. The default Windows policy (offlineShared) doesn't
automatically bring online such disks, SAS being considered a
shared bus.
While at it, we're also enabling the new *Fs* tests, which use
a Windows partition instead of the raw block devices.
Ionut Balutoiu [Mon, 31 Oct 2022 16:16:56 +0000 (18:16 +0200)]
ceph-windows: Add more Windows testing
Run `test_rbd_wnbd.py` script from the upstream Ceph QA Windows
workunit scripts as part of the `script/ceph-windows/run_tests`.
The new test cases don't take a lot of time to execute (unless
something is wrong with Ceph on Windows or Linux). So, we won't
have any noticeable jobs' execute time penalty.
This adds more Windows tests to the Jenkins jobs:
* `ceph-windows-test`
* `ceph-windows-pull-requests`
* Bump Python to `3.11`.
* Add `fio` as part of Windows CI image setup.
* Add `prettytable` pip dependency. We'll use this for the Python
`test_rbd_wnbd.py` script.
Ilya Dryomov [Tue, 4 Oct 2022 17:58:10 +0000 (19:58 +0200)]
kernel: re-add master branch to the trigger list
Commit 39e983f0198c ("master-> main") shouldn't have touched
kernel-trigger job as ceph-client.git still has the master branch
which feeds into linux-next.git.
Make the Jenkins `ceph-windows-test` similar to `ceph-windows-pull-requests` job.
However, `ceph-windows-test` is meant to be triggered on-demand, and it
exposes more parameters for `win32_build.sh` script, and artifacts collected
at the end of the job run.
The new `ceph-windows-test` Jenkins job will have the following new parameters:
* `CEPH_GIT_REPO` - The Ceph git repo used to clone.
* `CEPH_GIT_BRANCH` - The Ceph git branch name used to clone.
* `CEPH_WIN32_BUILD_FLAGS` - Build flags passed to `win32_build.sh`
besides the defaults.
* `INCLUDE_USERSPACE_CRASH_DUMPS` - Collect Windows userspace crash dumps
with the job artifacts.
* `INCLUDE_CEPH_ZIP` - Collect the `ceph.zip` Windows build with the
job artifacts. This is useful for debugging failures locally.
Also, create `scripts/ceph-windows/win32_build` common script. This is
used in both Jenkins jobs to cross-build Ceph Windows.
Besides this, we collect artifacts with `trap EXIT` instance.
Previously, if the tests failed, the artifacts wouldn't be collected.
Due to the fact that artifacts collected might occupy more space,
we limit `artifact-num-to-keep` to `15`.
ceph-windows: Cleanup logs dir before collecting new logs
We noticed old logs in the Jenkins `ceph-windows-pull-requests` job
artifacts. This is very likely due to the fact that there were
existing logs from previous runs.
Make sure that there are no leftover logs from previous runs by
cleaning up the `$WORKSPACE/logs` dir prior to collecting new logs
from the Windows tests machine.
```
ERROR! the playbook: /home/jenkins-build/build/workspace/ceph-volume-prs-lvm-centos8-bluestore-dmcrypt/tests/functional/collect-logs.yml could not be found
```
Since `collect-logs.yml` is maintained in ceph-ansible repository,
when `collect_ceph_logs` is called from a job different from
ceph-ansible, the path is incorrect.
The idea is to make ceph-volume jobs override this path.
update ceph-volume nightly jobs.
drop nautilus testing as it's EOL for a while.
drop testing against el7
drop testing against xenial (replace with focal)
test against quincy and pacific
The previous commit installed the gperftools-devel package and set the
cmake ALLOCATION variable to tcmalloc but this isn't used at all during
the RPM package build via mock.
We also need those changes in the RPM spec file.
This is a temporary solution until the upstream nfs-ganesha RPM spec file
will support such change.
I swapped the paths in `for endpoint in` because there may not be a centos/9 repo. That means `$version` gets left unset and isn't available to `get-tarballs.sh` later in the script.
Signed-off-by: David Galloway <dgallowa@redhat.com>
Kefu Chai [Sat, 20 Aug 2022 00:17:45 +0000 (08:17 +0800)]
ceph-windows: remove installed packages
`ceph-windows/run_tests` installs client side dependencies using
`cephadm install ceph-common`, which leaves us with a bunch ceph
packages after the test finishes. and they inteferes with succeeding
tests which, for instance, try to import python modules from test's
building directory.
but they end up with importing the python modules installed in system.
in this change, the installed package and its ceph dependencies are
removed in the cleanup script. the list of packages is collected by
inspecting `/var/log/dpkg.log` on the system where the test failures
were spotted and `ceph/debian/control` 's "ceph-common" section.
David Galloway [Wed, 10 Aug 2022 18:41:35 +0000 (14:41 -0400)]
ceph-docs-rtd: Create job to keep releases docs up-to-date
See ceph/ceph#46917
This Jenkins job will receive webhooks from the github plugin. It will then check if the main branch is what was updated and if the `doc/releases` dir was updated.
If true, it will trigger a build for each active Ceph release's docs.
ceph/ceph#47442
ceph/ceph#47443
ceph/ceph#47444
Signed-off-by: David Galloway <dgallowa@redhat.com>
projects / ceph-build.git / log