qa/tasks/cbt: construct venv just for cbt

So we no longer need to install system-wide.

Avoids errors like on Ubuntu 24.04:

    2026-05-24T13:48:19.681 DEBUG:teuthology.orchestra.run.trial043:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt
    2026-05-24T13:48:19.861 INFO:teuthology.orchestra.run.trial043.stderr:error: externally-managed-environment
    2026-05-24T13:48:19.861 INFO:teuthology.orchestra.run.trial043.stderr:
    2026-05-24T13:48:19.861 INFO:teuthology.orchestra.run.trial043.stderr:× This environment is externally managed
    2026-05-24T13:48:19.861 INFO:teuthology.orchestra.run.trial043.stderr:╰─> To install Python packages system-wide, try apt install
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    python3-xyz, where xyz is the package you are trying to
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    install.
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    If you wish to install a non-Debian-packaged Python package,
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    create a virtual environment using python3 -m venv path/to/venv.
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    sure you have python3-full installed.
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    If you wish to install a non-Debian packaged Python application,
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    it may be easiest to use pipx install xyz, which will manage a
    2026-05-24T13:48:19.862 INFO:teuthology.orchestra.run.trial043.stderr:    virtual environment for you. Make sure you have pipx installed.
    2026-05-24T13:48:19.863 INFO:teuthology.orchestra.run.trial043.stderr:
    2026-05-24T13:48:19.863 INFO:teuthology.orchestra.run.trial043.stderr:    See /usr/share/doc/python3.12/README.venv for more information.
    2026-05-24T13:48:19.863 INFO:teuthology.orchestra.run.trial043.stderr:
    2026-05-24T13:48:19.863 INFO:teuthology.orchestra.run.trial043.stderr:note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
    2026-05-24T13:48:19.863 INFO:teuthology.orchestra.run.trial043.stderr:hint: See PEP 668 for the detailed specification.
    2026-05-24T13:48:19.883 DEBUG:teuthology.orchestra.run:got remote process result: 1
    2026-05-24T13:48:19.883 ERROR:teuthology.run_tasks:Saw exception from tasks.
    Traceback (most recent call last):
      File "/home/teuthworker/src/git.ceph.com_teuthology_3686f8793d626abcf5a0018da0a50786e41fed9d/teuthology/run_tasks.py", line 112, in run_tasks
        manager.__enter__()
      File "/home/teuthworker/src/git.ceph.com_teuthology_3686f8793d626abcf5a0018da0a50786e41fed9d/teuthology/task/__init__.py", line 122, in __enter__
        self.setup()
      File "/home/teuthworker/src/github.com_ceph_ceph-c_1bc3c25246d3a6fbc360dc78d9b4b51200743391/qa/tasks/cbt.py", line 173, in setup
        self.install_dependencies()
      File "/home/teuthworker/src/github.com_ceph_ceph-c_1bc3c25246d3a6fbc360dc78d9b4b51200743391/qa/tasks/cbt.py", line 112, in install_dependencies
        self.first_mon.run(args=pip_install_cmd)
      File "/home/teuthworker/src/git.ceph.com_teuthology_3686f8793d626abcf5a0018da0a50786e41fed9d/teuthology/orchestra/remote.py", line 596, in run
        r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/teuthworker/src/git.ceph.com_teuthology_3686f8793d626abcf5a0018da0a50786e41fed9d/teuthology/orchestra/run.py", line 461, in run
        r.wait()
      File "/home/teuthworker/src/git.ceph.com_teuthology_3686f8793d626abcf5a0018da0a50786e41fed9d/teuthology/orchestra/run.py", line 161, in wait
        self._raise_for_status()
      File "/home/teuthworker/src/git.ceph.com_teuthology_3686f8793d626abcf5a0018da0a50786e41fed9d/teuthology/orchestra/run.py", line 181, in _raise_for_status
        raise CommandFailedError(
    teuthology.exceptions.CommandFailedError: Command failed on trial043 with status 1: 'python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt'

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

diff --git a/qa/tasks/cbt.py b/qa/tasks/cbt.py
index 858a34ed17fc5eacce8831384d988a44ffc8c5b9..b85902b55769b85165e8af4eef6b1966b74bfd51 100644 (file)
--- a/qa/tasks/cbt.py
+++ b/qa/tasks/cbt.py
@@ -97,17 +97,20 @@ class CBT(Task):
             )
         else:
             install_cmd = ['sudo', 'apt-get', '-y', '--force-yes', 'install']
-            cbt_depends = ['librbd-dev', 'collectl', 'linux-tools-generic']
+            cbt_depends = ['librbd-dev', 'collectl', 'linux-tools-generic', 'python3-venv']
         self.first_mon.run(args=install_cmd + cbt_depends)
 
-        # Install the python dependencies from the CBT project requirements.txt file
+        # Create a virtual environment for CBT
+        testdir = misc.get_testdir(self.ctx)
+        venv_path = f'{testdir}/cbt-venv'
+        self.first_mon.run(args=['python3', '-m', 'venv', venv_path])
+
+        # Install the python dependencies from the CBT project requirements.txt file into the venv
         pip_install_cmd = [
-            'python3',
-            '-m',
-            'pip',
+            f'{venv_path}/bin/pip',
             'install',
             '-r',
-            f'{misc.get_testdir(self.ctx)}/cbt/requirements.txt',
+            f'{testdir}/cbt/requirements.txt',
         ]
         self.first_mon.run(args=pip_install_cmd)
 
@@ -187,6 +190,7 @@ class CBT(Task):
         )
         self.first_mon.run(
             args=[
+                '{tdir}/cbt-venv/bin/python'.format(tdir=testdir),
                 '{tdir}/cbt/cbt.py'.format(tdir=testdir),
                 '-a', self.cbt_dir,
                 '{cbtdir}/cbt_config.yaml'.format(cbtdir=self.cbt_dir),
@@ -202,6 +206,7 @@ class CBT(Task):
             args=[
                 'rm', '--one-file-system', '-rf', '--',
                 '{tdir}/cbt'.format(tdir=testdir),
+                '{tdir}/cbt-venv'.format(tdir=testdir),
             ]
         )
         benchmark_type = next(iter(self.cbt_config.get('benchmarks').keys()))