cmake: migrate Python module installation from setup.py to pip

Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.

Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:

```
Error compiling Cython file:
------------------------------------------------------------
...
        """
        name = cstr(name, 'name')
        cdef:
            rados_ioctx_t _ioctx = convert_ioctx(ioctx)
            char *_name = name
            librbd_progress_fn_t _prog_cb = &no_op_progress_callback
                                            ^
------------------------------------------------------------

rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```

This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
   with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
   - Regenerates the preprocessed .pyx files
   - Re-runs cythonize() through Cython.Distutils.build_ext
   - Does NOT apply the compiler_directives from setup.py
   - Fails on the regenerated files missing required directives

New Options Explained:

`--use-pep517`:
  Addresses deprecation warning:
  ```
    DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
    mechanism, which will be removed in a future version. pip 25.3 will
    enforce this behaviour change.
  ```
  Uses the modern PEP 517 build backend which:
  - Performs a single build pass with all compiler_directives applied
  - Prevents the implicit rebuild that caused CompileError
  - Future-proofs against pip 25.3+ which will require this

`--no-build-isolation`:
  Ensures that environment variables set by CMake are respected:
  - CC, LDSHARED (compiler toolchain)
  - CPPFLAGS, LDFLAGS (compilation flags)
  - CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)

  Without this flag, pip would create an isolated build environment
  that ignores these critical build settings.

`--no-deps`:
  Prevents pip from attempting to install Python dependencies listed
  in setup.py's install_requires. All dependencies are managed by
  CMake and the distribution's package manager, not pip.

`--ignore-installed`:
  Addresses installation error when DESTDIR is set:
  ```
    ERROR: Could not install packages due to an OSError: [Errno 13]
    Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'

    OSError: [Errno 18] Invalid cross-device link:
    '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
  ```
  This error occurs because pip detects an existing system installation
  and tries to uninstall it before installing to DESTDIR. With
  --ignore-installed, pip skips the uninstall step and directly installs
  to the DESTDIR staging directory, which is the correct behavior for
  packaging.

Removed Options:

`--install-layout=deb`:
  This Debian-specific patch to 'setup.py install' is no longer needed.
  Modern pip automatically detects the distribution and uses the correct
  layout (dist-packages on Debian, site-packages on RPM distros).

`--single-version-externally-managed`:
  This option was specific to 'setup.py install' to prevent egg
  installation. With pip, this is handled automatically.

`--record /dev/null`:
  No longer needed as pip manages installation records internally.

`egg_info --egg-base`:
  Not needed with pip as metadata is generated automatically during
  the build process.

Not added option:
`--root-user-action=ignore`: not added
  In this change, we installing a python module using pip with
  `fakeroot` before packaging it. But pip warned:
  ```
  Error: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  ```
  But we use fakeroot on purpose, this option could have been added  to
  silence this warning. But it is not available in all supported pip
  versions. see
  https://github.com/pypa/pip/commit/2e1112a8141dbdf767505fded918706e9ad61031

New environmental variable:
`DEB_PYTHON_INSTALL_LAYOUT=deb` is conditionally applied when packaging
for debian-derivative distributions. As pip does not support
`--install-layout` option. Since debian patches pip so it installs Python
modules into /usr/local/lib instead of /usr/lib where debian dh_install
helper looks for the content to be packaged, so we have to enforce the
debian layout using the environmental variable.

Working Directory Change:

Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.

Additional Dependencies:

Since the build process uses pip and creates a wheel distribution,
we need to add `pip` and `wheel` Python modules as build dependencies.

Python moduels packaging:

- with `--use-pep517`, pip creates .dist-info directoires as per PEP-517
instead of .egg-info, so we need to package the new metadata directory.

Future Improvements

We considered implementing a custom `build_templates` command or using
setuptools' `sub_commands` mechanism to avoid regenerating `*_processed.pyx`
files on every build (tracking dependencies via file modification times or
hash-based checks). However, to keep `setup.py` simple and maintainable,
we've deferred this optimization for future work. The current solution
using `pip install --use-pep517` ensures correct builds without additional
complexity.

This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.

Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>

diff --git a/ceph.spec.in b/ceph.spec.in
index 9b154a72a9a8988ace80c38ca4c6fb97d15069b8..18cd57b26833e0181627f59ff6f3c901c0b6d298 100644 (file)
--- a/ceph.spec.in
+++ b/ceph.spec.in
@@ -291,6 +291,8 @@ BuildRequires:      python%{python3_pkgversion}
 BuildRequires: python%{python3_pkgversion}-devel
 BuildRequires: python%{python3_pkgversion}-setuptools
 BuildRequires: python%{python3_pkgversion}-Cython
+BuildRequires: python%{python3_pkgversion}-pip
+BuildRequires: python%{python3_pkgversion}-wheel
 BuildRequires: snappy-devel
 BuildRequires: sqlite-devel
 BuildRequires: sudo
@@ -2458,7 +2460,7 @@ fi
 
 %files -n python%{python3_pkgversion}-rados
 %{python3_sitearch}/rados.cpython*.so
-%{python3_sitearch}/rados-*.egg-info
+%{python3_sitearch}/rados-*.dist-info
 
 %files -n libcephsqlite
 %{_libdir}/libcephsqlite.so
@@ -2530,11 +2532,11 @@ fi
 
 %files -n python%{python3_pkgversion}-rgw
 %{python3_sitearch}/rgw.cpython*.so
-%{python3_sitearch}/rgw-*.egg-info
+%{python3_sitearch}/rgw-*.dist-info
 
 %files -n python%{python3_pkgversion}-rbd
 %{python3_sitearch}/rbd.cpython*.so
-%{python3_sitearch}/rbd-*.egg-info
+%{python3_sitearch}/rbd-*.dist-info
 
 %files -n libcephfs2
 %{_libdir}/libcephfs.so.*
@@ -2570,7 +2572,7 @@ fi
 
 %files -n python%{python3_pkgversion}-cephfs
 %{python3_sitearch}/cephfs.cpython*.so
-%{python3_sitearch}/cephfs-*.egg-info
+%{python3_sitearch}/cephfs-*.dist-info
 
 %files -n python%{python3_pkgversion}-ceph-argparse
 %{python3_sitelib}/ceph_argparse.py

diff --git a/cmake/modules/Distutils.cmake b/cmake/modules/Distutils.cmake
index 76b5fb23d91605c221090c64906606b9b462e19a..37b2efa7457c885a7eafb77081295ea0dc79b023 100644 (file)
--- a/cmake/modules/Distutils.cmake
+++ b/cmake/modules/Distutils.cmake
@@ -137,25 +137,27 @@ function(distutils_install_cython_module name)
     set(ENV{CYTHON_BUILD_DIR} \"${CMAKE_CURRENT_BINARY_DIR}\")
     set(ENV{CEPH_LIBDIR} \"${CMAKE_LIBRARY_OUTPUT_DIRECTORY}\")
 
-    set(options --prefix=${CMAKE_INSTALL_PREFIX})
+    set(options
+      --prefix=${CMAKE_INSTALL_PREFIX}
+      --use-pep517
+      --no-build-isolation
+      --no-deps
+      --ignore-installed)
     if(DEFINED ENV{DESTDIR})
       if(EXISTS /etc/debian_version)
-        list(APPEND options --install-layout=deb)
+        list(APPEND env_vars \"DEB_PYTHON_INSTALL_LAYOUT=deb\")
       endif()
       list(APPEND options --root=\$ENV{DESTDIR})
     else()
       list(APPEND options --root=/)
     endif()
     execute_process(
-       COMMAND
-           ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/setup.py
-           build ${maybe_verbose} --build-base ${CYTHON_MODULE_DIR}
-           --build-platlib ${CYTHON_MODULE_DIR}/lib.3
-           build_ext --cython-c-in-temp --build-temp ${CMAKE_CURRENT_BINARY_DIR} --cython-include-dirs ${PROJECT_SOURCE_DIR}/src/pybind/rados
-           install \${options} --single-version-externally-managed --record /dev/null
-           egg_info --egg-base ${CMAKE_CURRENT_BINARY_DIR}
+       COMMAND env \${env_vars}
+           ${Python3_EXECUTABLE} -m pip install
+           \${options}
            ${maybe_verbose}
-       WORKING_DIRECTORY \"${CMAKE_CURRENT_SOURCE_DIR}\"
+           ${CMAKE_CURRENT_SOURCE_DIR}
+       WORKING_DIRECTORY \"${CMAKE_CURRENT_BINARY_DIR}\"
        RESULT_VARIABLE install_res)
     if(NOT \"\${install_res}\" STREQUAL 0)
       message(FATAL_ERROR \"Failed to build and install ${name} python module\")

diff --git a/debian/control b/debian/control
index 18eb8429867b45cbe205467bbeaf8d1e24ca7054..3d4c84984dd0d66c9cca1014fea162021f4ccb98 100644 (file)
--- a/debian/control
+++ b/debian/control
@@ -110,7 +110,9 @@ Build-Depends: automake,
                python3-onelogin-saml2 <pkg.ceph.check>,
                python3-jinja2,
                python3-markupsafe,
+               python3-pip,
                python3-setuptools,
+               python3-wheel,
                python3-sphinx,
                python3-venv,
                python3-yaml,

diff --git a/debian/python3-cephfs.install b/debian/python3-cephfs.install
index 9ac75f5366b49a23ed64684d857b6c45e30df5f2..577236c76a8ba825e036e905b0baf3c2ec5adb32 100644 (file)
--- a/debian/python3-cephfs.install
+++ b/debian/python3-cephfs.install
@@ -1,2 +1,2 @@
-usr/lib/python3*/dist-packages/cephfs-*.egg-info
+usr/lib/python3*/dist-packages/cephfs-*.dist-info
 usr/lib/python3*/dist-packages/cephfs.cpython*.so

diff --git a/debian/python3-rados.install b/debian/python3-rados.install
index 98b5d76cbe7dd681299032dea92ced3154d3d810..84009ee79c58906012ccbd746216f76d286d2c7a 100644 (file)
--- a/debian/python3-rados.install
+++ b/debian/python3-rados.install
@@ -1,2 +1,2 @@
-usr/lib/python3*/dist-packages/rados-*.egg-info
+usr/lib/python3*/dist-packages/rados-*.dist-info
 usr/lib/python3*/dist-packages/rados.cpython*.so

diff --git a/debian/python3-rbd.install b/debian/python3-rbd.install
index 5f4e6e143e8b0e7fb0c9a28fb22a5607f9842835..a447182a56df610265be5c52be1bb48853625263 100644 (file)
--- a/debian/python3-rbd.install
+++ b/debian/python3-rbd.install
@@ -1,2 +1,2 @@
-usr/lib/python3*/dist-packages/rbd-*.egg-info
+usr/lib/python3*/dist-packages/rbd-*.dist-info
 usr/lib/python3*/dist-packages/rbd.cpython*.so

diff --git a/debian/python3-rgw.install b/debian/python3-rgw.install
index 57f4559077102ff4c151a9ee416ecf7d897d3a84..bf6678ad79db2b89c0fb18a2df966c2122ec3eef 100644 (file)
--- a/debian/python3-rgw.install
+++ b/debian/python3-rgw.install
@@ -1,2 +1,2 @@
-usr/lib/python3*/dist-packages/rgw-*.egg-info
+usr/lib/python3*/dist-packages/rgw-*.dist-info
 usr/lib/python3*/dist-packages/rgw.cpython*.so