debian: add .requires for specifying python3 deps

we use dh_python3 to define subvar of ${python3:Depends} as a part
of the runtime dependencies of python3 packages, like,
ceph-mgr modules named "ceph-mgr-*", python3 bindings named "python3-*".

but unlike python3 bindings of Ceph APIs, the ceph-mgr modules are
not packaged in a typical python way. in other words, they do not
ship a "dist-info" or an "egg-info" directory. instead, we just
install the python scripts into a directory which can be found by
ceph-mgr, by default it is /usr/share/ceph/mgr/dashboard/plugins.

this does not follow the convention of python packaging or
debian packaging policies related to python package. but it
still makes to put these files in this non-convention place, as
they are not supposed to be python packages consumed by the
outer world -- they are but plugins. and should always work
with the same version of ceph-mgr.

the problem is, despite that we have ${python3:Depends} in
the "Depends" field of packages like ceph-mgr-dashboard, dh_python3
is not able to figure out the dependencies by looking at the
installed files. for instance, we have following "Depends" of
ceph-mgr-dashboard:

Depends: ceph-mgr (= 17.0.0-12481-g805d2320-1focal), python3-cherrypy3, python3-jwt, python3-bcrypt, python3-werkzeug, python3-routes

and in the debian/control file we have:

Depends: ceph-mgr (= ${binary:Version}),
         python3-cherrypy3,
         python3-jwt,
         python3-bcrypt,
         python3-werkzeug,
         python3-routes,
         ${misc:Depends},
         ${python:Depends},
         ${shlibs:Depends},

apparently, none of the subvar is materialized to
a non-empty string.

to improve the packaging, in this change:

* drop all subvars from ceph-mgr-*, as they
  are all implemented in pure python.
* add debian/ceph-mgr-*.requires, it's content
  is replicated with the corresponding requirements.txt
  files.
  * add python3-distutils for distutils, as debian
    and its derivatives package non-essetial part of
    distutils into a separate package, see
    https://packages.debian.org/stable/python3-distutils
* add ${python3:Depends} so dh_python3
  can extract the deps from debian/ceph-mgr-*.pydist
* update the rule for "override_dh_python3" target,
  so dh_python3 can pick up the dependencies specified
  in .requires file.
* remove the python3 dependencies not used by
  ceph-mgr from ceph-mgr's "Depends"

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

diff --git a/debian/ceph-mgr-cephadm.requires b/debian/ceph-mgr-cephadm.requires
new file mode 100644 (file)
index 0000000..ec2e22b
--- /dev/null
+++ b/debian/ceph-mgr-cephadm.requires
@@ -0,0 +1,4 @@
+CherryPy
+asyncssh
+cryptography
+Jinja2

diff --git a/debian/ceph-mgr-dashboard.requires b/debian/ceph-mgr-dashboard.requires
new file mode 100644 (file)
index 0000000..14db124
--- /dev/null
+++ b/debian/ceph-mgr-dashboard.requires
@@ -0,0 +1,11 @@
+bcrypt
+CherryPy
+more-itertools
+PyJWT
+pyopenssl
+requests
+Routes
+prettytable
+pytest
+pyyaml
+ceph-common

diff --git a/debian/ceph-mgr-diskprediction-local.requires b/debian/ceph-mgr-diskprediction-local.requires
new file mode 100644 (file)
index 0000000..cb5777a
--- /dev/null
+++ b/debian/ceph-mgr-diskprediction-local.requires
@@ -0,0 +1,3 @@
+numpy
+scipy
+scikit-learn

diff --git a/debian/ceph-mgr-k8sevents.requires b/debian/ceph-mgr-k8sevents.requires
new file mode 100644 (file)
index 0000000..ba84971
--- /dev/null
+++ b/debian/ceph-mgr-k8sevents.requires
@@ -0,0 +1,2 @@
+pyyaml
+kubernetes

diff --git a/debian/ceph-mgr-modules-core.requires b/debian/ceph-mgr-modules-core.requires
new file mode 100644 (file)
index 0000000..afefb97
--- /dev/null
+++ b/debian/ceph-mgr-modules-core.requires
@@ -0,0 +1,6 @@
+natsort
+CherryPy
+pecan
+werkzeug
+requests
+python-dateutil

diff --git a/debian/ceph-mgr-rook.requires b/debian/ceph-mgr-rook.requires
new file mode 100644 (file)
index 0000000..48ae25b
--- /dev/null
+++ b/debian/ceph-mgr-rook.requires
@@ -0,0 +1,2 @@
+kubernetes
+jsonpatch

diff --git a/debian/ceph-mgr.requires b/debian/ceph-mgr.requires
new file mode 100644 (file)
index 0000000..bf334fb
--- /dev/null
+++ b/debian/ceph-mgr.requires
@@ -0,0 +1,5 @@
+pyOpenSSL
+cephfs
+ceph-argparse
+pyyaml
+distutils

diff --git a/debian/control b/debian/control
index e6526b83791a11bc321edad1407d461bc6a3be72..a8e25ec96a57fef60fbe5339b69010cb01c24c91 100644 (file)
--- a/debian/control
+++ b/debian/control
@@ -226,15 +226,6 @@ Package: ceph-mgr
 Architecture: linux-any
 Depends: ceph-base (= ${binary:Version}),
          ceph-mgr-modules-core (= ${binary:Version}),
-         python3-bcrypt,
-         python3-cherrypy3,
-         python3-natsort,
-         python3-distutils,
-         python3-jwt,
-         python3-openssl,
-         python3-pecan,
-         python3-requests,
-         python3-werkzeug,
          ${misc:Depends},
          ${python3:Depends},
          ${shlibs:Depends},
@@ -256,14 +247,7 @@ Description: manager for the ceph distributed storage system
 Package: ceph-mgr-dashboard
 Architecture: all
 Depends: ceph-mgr (= ${binary:Version}),
-         python3-cherrypy3,
-         python3-jwt,
-         python3-bcrypt,
-         python3-werkzeug,
-         python3-routes,
-         ${misc:Depends},
-         ${python:Depends},
-         ${shlibs:Depends},
+         ${python3:Depends}
 Description: dashboard module for ceph-mgr
  Ceph is a massively scalable, open-source, distributed
  storage system that runs on commodity hardware and delivers object,
@@ -279,12 +263,7 @@ Description: dashboard module for ceph-mgr
 Package: ceph-mgr-diskprediction-local
 Architecture: all
 Depends: ceph-mgr (= ${binary:Version}),
-         python3-numpy,
-         python3-scipy,
-         python3-sklearn,
-         ${misc:Depends},
-         ${python:Depends},
-         ${shlibs:Depends},
+         ${python3:Depends}
 Description: diskprediction-local module for ceph-mgr
  Ceph is a massively scalable, open-source, distributed
  storage system that runs on commodity hardware and delivers object,
@@ -297,9 +276,6 @@ Package: ceph-mgr-modules-core
 Architecture: all
 Depends: ${misc:Depends},
          ${python3:Depends},
-         ${shlibs:Depends},
-         python3-dateutil,
-         python3-openssl,
 Replaces: ceph-mgr (<< 15.1.0)
 Breaks: ceph-mgr (<< 15.1.0)
 Description: ceph manager modules which are always enabled
@@ -313,7 +289,6 @@ Description: ceph manager modules which are always enabled
 Package: ceph-mgr-rook
 Architecture: all
 Depends: ceph-mgr (= ${binary:Version}),
-         python3-jsonpatch
          ${misc:Depends},
          ${python3:Depends},
          ${shlibs:Depends},

diff --git a/debian/py3dist-overrides b/debian/py3dist-overrides
index 1cf76162b47c88b690baf914d67ad642b9868d29..b6358cf123efe945a6a896896560ac11264725d5 100644 (file)
--- a/debian/py3dist-overrides
+++ b/debian/py3dist-overrides
@@ -1 +1,4 @@
 cephfs python3-cephfs; PEP386
+ceph_argparse python3-ceph-argparse
+ceph_common python3-ceph-common
+distutils python3-distutils

diff --git a/debian/rules b/debian/rules
index 9f49deabb0e0128d24ce0ca1f3dc08d562bcce58..a3fb66f9e8839abdfb702671c8acf47055854333 100755 (executable)
--- a/debian/rules
+++ b/debian/rules
@@ -142,6 +142,9 @@ override_dh_python3:
        for binding in rados cephfs rbd rgw; do \
          dh_python3 -p python3-$$binding;      \
        done
+       for pkg in $(basename $(notdir $(wildcard debian/*.requires))); do \
+         dh_python3 -p $$pkg --requires=debian/$$pkg.requires;            \
+       done
        dh_python3 -p python3-ceph-argparse
        dh_python3 -p python3-ceph-common
        dh_python3 -p cephfs-shell