cmake: migrate Python module installation from setup.py to pip

Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.

Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:

```
Error compiling Cython file:
------------------------------------------------------------
...
        """
        name = cstr(name, 'name')
        cdef:
            rados_ioctx_t _ioctx = convert_ioctx(ioctx)
            char *_name = name
            librbd_progress_fn_t _prog_cb = &no_op_progress_callback
                                            ^
------------------------------------------------------------

rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```

This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
   with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
   - Regenerates the preprocessed .pyx files
   - Re-runs cythonize() through Cython.Distutils.build_ext
   - Does NOT apply the compiler_directives from setup.py
   - Fails on the regenerated files missing required directives

New Options Explained:

--use-pep517:
  Addresses deprecation warning:
  ```
    DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
    mechanism, which will be removed in a future version. pip 25.3 will
    enforce this behaviour change.
  ```
  Uses the modern PEP 517 build backend which:
  - Performs a single build pass with all compiler_directives applied
  - Prevents the implicit rebuild that caused CompileError
  - Future-proofs against pip 25.3+ which will require this

--no-build-isolation:
  Ensures that environment variables set by CMake are respected:
  - CC, LDSHARED (compiler toolchain)
  - CPPFLAGS, LDFLAGS (compilation flags)
  - CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)

  Without this flag, pip would create an isolated build environment
  that ignores these critical build settings.

--no-deps:
  Prevents pip from attempting to install Python dependencies listed
  in setup.py's install_requires. All dependencies are managed by
  CMake and the distribution's package manager, not pip.

--ignore-installed:
  Addresses installation error when DESTDIR is set:
  ```
    ERROR: Could not install packages due to an OSError: [Errno 13]
    Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'

    OSError: [Errno 18] Invalid cross-device link:
    '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
  ```
  This error occurs because pip detects an existing system installation
  and tries to uninstall it before installing to DESTDIR. With
  --ignore-installed, pip skips the uninstall step and directly installs
  to the DESTDIR staging directory, which is the correct behavior for
  packaging.

Removed Options:

--install-layout=deb:
  This Debian-specific patch to 'setup.py install' is no longer needed.
  Modern pip automatically detects the distribution and uses the correct
  layout (dist-packages on Debian, site-packages on RPM distros).

--single-version-externally-managed:
  This option was specific to 'setup.py install' to prevent egg
  installation. With pip, this is handled automatically.

--record /dev/null:
  No longer needed as pip manages installation records internally.

egg_info --egg-base:
  Not needed with pip as metadata is generated automatically during
  the build process.

Working Directory Change:
Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.

This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.

Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>

diff --git a/cmake/modules/Distutils.cmake b/cmake/modules/Distutils.cmake
index 76b5fb23d91605c221090c64906606b9b462e19a..fc8501807a4cacab9ae33d68644dbd568255c051 100644 (file)
--- a/cmake/modules/Distutils.cmake
+++ b/cmake/modules/Distutils.cmake
@@ -137,25 +137,24 @@ function(distutils_install_cython_module name)
     set(ENV{CYTHON_BUILD_DIR} \"${CMAKE_CURRENT_BINARY_DIR}\")
     set(ENV{CEPH_LIBDIR} \"${CMAKE_LIBRARY_OUTPUT_DIRECTORY}\")
 
-    set(options --prefix=${CMAKE_INSTALL_PREFIX})
+    set(options
+      --prefix=${CMAKE_INSTALL_PREFIX}
+      --use-pep517
+      --no-build-isolation
+      --no-deps
+      --ignore-installed)
     if(DEFINED ENV{DESTDIR})
-      if(EXISTS /etc/debian_version)
-        list(APPEND options --install-layout=deb)
-      endif()
       list(APPEND options --root=\$ENV{DESTDIR})
     else()
       list(APPEND options --root=/)
     endif()
     execute_process(
        COMMAND
-           ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/setup.py
-           build ${maybe_verbose} --build-base ${CYTHON_MODULE_DIR}
-           --build-platlib ${CYTHON_MODULE_DIR}/lib.3
-           build_ext --cython-c-in-temp --build-temp ${CMAKE_CURRENT_BINARY_DIR} --cython-include-dirs ${PROJECT_SOURCE_DIR}/src/pybind/rados
-           install \${options} --single-version-externally-managed --record /dev/null
-           egg_info --egg-base ${CMAKE_CURRENT_BINARY_DIR}
+           ${Python3_EXECUTABLE} -m pip install
+           \${options}
            ${maybe_verbose}
-       WORKING_DIRECTORY \"${CMAKE_CURRENT_SOURCE_DIR}\"
+           ${CMAKE_CURRENT_SOURCE_DIR}
+       WORKING_DIRECTORY \"${CMAKE_CURRENT_BINARY_DIR}\"
        RESULT_VARIABLE install_res)
     if(NOT \"\${install_res}\" STREQUAL 0)
       message(FATAL_ERROR \"Failed to build and install ${name} python module\")