requirements.in: add requirements.in back for tracking pinned direct requirements

and update `update-requirements.sh` to use requirements.in for building
requirements.txt

for some install_requires versus requirements.txt, see
https://packaging.python.org/discussions/install-requires-vs-requirements/.
in an ideal world, we should have not put those non-essential dependencies in
``setup.py``.

Signed-off-by: Kefu Chai <kchai@redhat.com>

diff --git a/docs/INSTALL.rst b/docs/INSTALL.rst
index 1e8c8a9e54c68037bfacf19366bca3cfe0288e29..3ecff7c8de2b4d2849cf435274e0d8c500cb8955 100644 (file)
--- a/docs/INSTALL.rst
+++ b/docs/INSTALL.rst
@@ -65,11 +65,11 @@ interacting with the services to schedule tests and to report the test results.
 Update Dependencies
 -------------------
 
-We track the dependencies using ``requirements.txt`` . These packages are
-tested, and should work  with teuthology. But if you want to bump up the
+We track the dependencies using ``requirements.txt``. These packages are
+tested, and should work with teuthology. But if you want to bump up the
 versions of them, please use the following command to update these files::
 
-  ./update-requirements.sh
+  ./update-requirements.sh -P <package-name>
 
 Please upgrade pip-tool using following command ::
 
@@ -83,3 +83,25 @@ if the command above fails like::
   File "/home/kchai/teuthology/virtualenv/local/lib/python2.7/site-packages/piptools/scripts/compile.py", line 11, in <module>
     from pip.req import InstallRequirement, parse_requirements
   ImportError: No module named req
+
+Add Dependencies
+----------------
+
+td,dr: please add the new dependencies in both ``setup.py`` and
+``requirements.in``.
+
+We also use ``pip install <URL>`` to install teuthology in some Ceph's unit
+tests. To cater their needs, some requirements are listed in ``setup.py`` as
+well, so that ``pip install`` can pick them up. We could just avoid duplicating
+the packages specifications in two places by putting::
+
+  -e .[orchestra,test]
+
+in ``requirements.in``. But dependabot includes::
+
+  -e file:///home/dependabot/dependabot-updater/tmp/dependabot_20200617-72-1n8af4b  # via -r requirements.in
+
+in the generated ``requirements.txt``. This renders the created pull request
+useless without human intervention. To appease dependabot, a full-blown
+``requirements.in`` collecting all direct dependencies listed by ``setup.py``
+is used instead.

diff --git a/requirements.in b/requirements.in
new file mode 100644 (file)
index 0000000..2c8a330
--- /dev/null
+++ b/requirements.in
@@ -0,0 +1,42 @@
+# install requires
+apache-libcloud
+gevent
+PyYAML
+argparse>=1.2.1
+configobj
+six>=1.9
+pexpect
+docopt
+netaddr
+paramiko
+psutil >= 2.1.0
+configparser
+ansible>=2.0
+prettytable
+manhole
+humanfriendly
+# orchestra requires
+backports.ssl-match-hostname
+beanstalkc3>=0.4.0
+httplib2
+ndg-httpsclient
+pyasn1
+pyopenssl>=0.13
+python-dateutil
+python-novaclient
+python-openstackclient
+raven
+requests!=2.13.0
+# test requires
+boto>=2.0b4
+boto3
+cryptography>=2.7
+nose
+pip-tools
+pytest
+requests
+tox
+xmltodict
+PyJWT
+ipy
+toml

diff --git a/update-requirements.sh b/update-requirements.sh
index d99a67445a2ad5d684e8e43ee93e4b6271eb17aa..c72e2e8aebad9a3615268eb9330ff61fae6791b7 100755 (executable)
--- a/update-requirements.sh
+++ b/update-requirements.sh
@@ -1,3 +1,3 @@
 #!/bin/bash
 
-echo "-e .[orchestra,test]" | pip-compile $@ - -qo- | sed '/^-e / d' > requirements.txt
+pip-compile $@ requirements.in -qo- | sed -e '/^-e / d' -e 's/-r requirements.in/teuthology/g' > requirements.txt