docs: initial structure for tests docs

Signed-off-by: Alfredo Deza <adeza@redhat.com>

diff --git a/docs/source/testing/contents.rst b/docs/source/testing/contents.rst
new file mode 100644 (file)
index 0000000..1441ef8
--- /dev/null
+++ b/docs/source/testing/contents.rst
@@ -0,0 +1,15 @@
+
+Glossary
+========
+
+.. toctree::
+   :maxdepth: 1
+
+   index
+   running.rst
+   development.rst
+   scenarios.rst
+   modifying.rst
+   layout.rst
+
+

diff --git a/docs/source/testing/development.rst b/docs/source/testing/development.rst
new file mode 100644 (file)
index 0000000..f17031e
--- /dev/null
+++ b/docs/source/testing/development.rst
@@ -0,0 +1,5 @@
+
+.. development:
+
+ceph-ansible testing for development
+====================================

diff --git a/docs/source/testing/layout.rst b/docs/source/testing/layout.rst
new file mode 100644 (file)
index 0000000..9f196a6
--- /dev/null
+++ b/docs/source/testing/layout.rst
@@ -0,0 +1,30 @@
+.. _layout:
+
+Layout and conventions
+----------------------
+Test files and directories follow a few conventions, which makes it easy to
+create (or expect) certain interactions between tests and scenarios.
+
+All tests are in the ``tests`` directory. Scenarios are defined in
+``tests/functional/`` and use the following convention for directory
+structure::
+
+    tests/functional/<distro>/<distro version>/<scenario name>/
+
+For example: ``tests/functional/centos/7/journal-collocation``
+
+Within a test scenario there are a few files that define what that specific
+scenario needs for the tests, like how many OSD nodes or MON nodes. Tls
+
+At the very least, a scenario will need these files:
+
+* ``Vagrantfile``: must be symlinked from the root directory of the project
+* ``hosts``: An Ansible hosts file that defines the machines part of the
+  cluster
+* ``group_vars/all``: if any modifications are needed for deployment, this
+  would override them. Additionally, further customizations can be done. For
+  example, for OSDs that would mean adding ``group_vars/osds``
+* ``vagrant_variables.yml``: Defines the actual environment for the test, where
+  machines, networks, disks, linux distro/version, can be defined.
+
+

diff --git a/docs/source/testing/modifying.rst b/docs/source/testing/modifying.rst
new file mode 100644 (file)
index 0000000..3125535
--- /dev/null
+++ b/docs/source/testing/modifying.rst
@@ -0,0 +1,5 @@
+
+.. _modifying:
+
+Modifying (or adding) tests
+===========================

diff --git a/docs/source/testing/running.rst b/docs/source/testing/running.rst
new file mode 100644 (file)
index 0000000..d18bc72
--- /dev/null
+++ b/docs/source/testing/running.rst
@@ -0,0 +1,147 @@
+
+.. _running_tests:
+
+Running Tests
+=============
+Although tests run continuously in CI, a lot of effort was put into making it
+easy to run in any environment, as long as a couple of requirements are met.
+
+
+.. _dependencies:
+
+Dependencies
+------------
+There are some Python dependencies, which are listed in a ``requirements.txt``
+file within the ``tests/`` directory. These are meant to be installed using
+Python install tools (pip in this case)::
+
+    pip install -r tests/requirements.txt
+
+For virtualization, either libvirt or VirtualBox is needed (there is native
+support from the harness for both). This makes the test harness even more
+flexible as most platforms will be covered by either VirtualBox or libvirt.
+
+
+.. _running_a_scenario:
+
+Running a scenario
+------------------
+Tests are driven by ``tox``, a command line tool to run a matrix of tests defined in
+a configuration file (``tox.ini`` in this case at the root of the project).
+
+For a thorough description of a scenario see :ref:`test_scenarios`.
+
+To run a single scenario, make sure it is available (should be defined from
+``tox.ini``) by listing them::
+
+    tox -l
+
+In this example, we will use the ``kraken-ansible2.2-xenial_cluster`` one.  The
+harness defaults to ``VirtualBox`` as the backend, so if you have that
+installed in your system then this command should just work::
+
+    tox -e kraken-ansible2.2-xenial_cluster
+
+And for libvirt it would be::
+
+    tox -e kraken-ansible2.2-xenial_cluster -- --provider=libvirt
+
+
+.. warning:: Depending on the type of scenario and resources available, running
+    these tests locally in a personal computer can be very resource intensive.
+
+.. note:: Most test runs take between 20 and 40 minutes depending on system
+    resources
+
+The command should bring up the machines needed for the test, provision them
+with ceph-ansible, run the tests, and tear the whole environment down at the
+end.
+
+
+The output would look something similar to this trimmed version::
+
+    kraken-ansible2.2-xenial_cluster create: /Users/alfredo/python/upstream/ceph-ansible/.tox/kraken-ansible2.2-xenial_cluster
+    kraken-ansible2.2-xenial_cluster installdeps: ansible==2.2.2, -r/Users/alfredo/python/upstream/ceph-ansible/tests/requirements.txt
+    kraken-ansible2.2-xenial_cluster runtests: commands[0] | vagrant up --no-provision --provider=virtualbox
+    Bringing machine 'client0' up with 'virtualbox' provider...
+    Bringing machine 'rgw0' up with 'virtualbox' provider...
+    Bringing machine 'mds0' up with 'virtualbox' provider...
+    Bringing machine 'mon0' up with 'virtualbox' provider...
+    Bringing machine 'mon1' up with 'virtualbox' provider...
+    Bringing machine 'mon2' up with 'virtualbox' provider...
+    Bringing machine 'osd0' up with 'virtualbox' provider...
+    ...
+
+
+After all the nodes are up, ceph-ansible will provision them, and run the
+playbook(s)::
+
+    ...
+    PLAY RECAP *********************************************************************
+    client0                    : ok=4    changed=0    unreachable=0    failed=0
+    mds0                       : ok=4    changed=0    unreachable=0    failed=0
+    mon0                       : ok=4    changed=0    unreachable=0    failed=0
+    mon1                       : ok=4    changed=0    unreachable=0    failed=0
+    mon2                       : ok=4    changed=0    unreachable=0    failed=0
+    osd0                       : ok=4    changed=0    unreachable=0    failed=0
+    rgw0                       : ok=4    changed=0    unreachable=0    failed=0
+    ...
+
+
+Once the whole environment is all running the tests will be sent out to the
+hosts, with output similar to this::
+
+    kraken-ansible2.2-xenial_cluster runtests: commands[4] | testinfra -n 4 --sudo -v --connection=ansible --ansible-inventory=/Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster/hosts /Users/alfredo/python/upstream/ceph-ansible/tests/functional/tests
+    ============================ test session starts ===========================
+    platform darwin -- Python 2.7.8, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /Users/alfredo/python/upstream/ceph-ansible/.tox/kraken-ansible2.2-xenial_cluster/bin/python
+    cachedir: ../../../../.cache
+    rootdir: /Users/alfredo/python/upstream/ceph-ansible/tests, inifile: pytest.ini
+    plugins: testinfra-1.5.4, xdist-1.15.0
+    [gw0] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster
+    [gw1] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster
+    [gw2] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster
+    [gw3] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster
+    [gw0] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
+    [gw1] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
+    [gw2] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
+    [gw3] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35)  -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
+    gw0 [154] / gw1 [154] / gw2 [154] / gw3 [154]
+    scheduling tests via LoadScheduling
+
+    ../../../tests/test_install.py::TestInstall::test_ceph_dir_exists[ansible:/mon0]
+    ../../../tests/test_install.py::TestInstall::test_ceph_dir_is_a_directory[ansible:/mon0]
+    ../../../tests/test_install.py::TestInstall::test_ceph_conf_is_a_file[ansible:/mon0]
+    ../../../tests/test_install.py::TestInstall::test_ceph_dir_is_a_directory[ansible:/mon1]
+    [gw2] PASSED ../../../tests/test_install.py::TestCephConf::test_ceph_config_has_mon_host_line[ansible:/mon0]
+    ../../../tests/test_install.py::TestInstall::test_ceph_conf_exists[ansible:/mon1]
+    [gw3] PASSED ../../../tests/test_install.py::TestCephConf::test_mon_host_line_has_correct_value[ansible:/mon0]
+    ../../../tests/test_install.py::TestInstall::test_ceph_conf_is_a_file[ansible:/mon1]
+    [gw1] PASSED ../../../tests/test_install.py::TestInstall::test_ceph_command_exists[ansible:/mon1]
+    ../../../tests/test_install.py::TestCephConf::test_mon_host_line_has_correct_value[ansible:/mon1]
+    ...
+
+Finally the whole environment gets torn down::
+
+    kraken-ansible2.2-xenial_cluster runtests: commands[5] | vagrant destroy --force
+    ==> osd0: Forcing shutdown of VM...
+    ==> osd0: Destroying VM and associated drives...
+    ==> mon2: Forcing shutdown of VM...
+    ==> mon2: Destroying VM and associated drives...
+    ==> mon1: Forcing shutdown of VM...
+    ==> mon1: Destroying VM and associated drives...
+    ==> mon0: Forcing shutdown of VM...
+    ==> mon0: Destroying VM and associated drives...
+    ==> mds0: Forcing shutdown of VM...
+    ==> mds0: Destroying VM and associated drives...
+    ==> rgw0: Forcing shutdown of VM...
+    ==> rgw0: Destroying VM and associated drives...
+    ==> client0: Forcing shutdown of VM...
+    ==> client0: Destroying VM and associated drives...
+
+
+And a brief summary of the scenario(s) that ran is displayed::
+
+    ________________________________________________ summary _________________________________________________
+      kraken-ansible2.2-xenial_cluster: commands succeeded
+      congratulations :)
+

diff --git a/docs/source/testing/scenarios.rst b/docs/source/testing/scenarios.rst
new file mode 100644 (file)
index 0000000..031e1ab
--- /dev/null
+++ b/docs/source/testing/scenarios.rst
@@ -0,0 +1,5 @@
+
+.. _test_scenarios:
+
+Test Scenarios
+==============