From 1f3d78e6c268329e9b161b4b9c70c33c15c42cc4 Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Thu, 6 Apr 2017 13:17:35 -0400 Subject: [PATCH] docs: initial structure for tests docs Signed-off-by: Alfredo Deza --- docs/source/testing/contents.rst | 15 +++ docs/source/testing/development.rst | 5 + docs/source/testing/layout.rst | 30 ++++++ docs/source/testing/modifying.rst | 5 + docs/source/testing/running.rst | 147 ++++++++++++++++++++++++++++ docs/source/testing/scenarios.rst | 5 + 6 files changed, 207 insertions(+) create mode 100644 docs/source/testing/contents.rst create mode 100644 docs/source/testing/development.rst create mode 100644 docs/source/testing/layout.rst create mode 100644 docs/source/testing/modifying.rst create mode 100644 docs/source/testing/running.rst create mode 100644 docs/source/testing/scenarios.rst diff --git a/docs/source/testing/contents.rst b/docs/source/testing/contents.rst new file mode 100644 index 000000000..1441ef8b0 --- /dev/null +++ b/docs/source/testing/contents.rst @@ -0,0 +1,15 @@ + +Glossary +======== + +.. toctree:: + :maxdepth: 1 + + index + running.rst + development.rst + scenarios.rst + modifying.rst + layout.rst + + diff --git a/docs/source/testing/development.rst b/docs/source/testing/development.rst new file mode 100644 index 000000000..f17031e4f --- /dev/null +++ b/docs/source/testing/development.rst @@ -0,0 +1,5 @@ + +.. development: + +ceph-ansible testing for development +==================================== diff --git a/docs/source/testing/layout.rst b/docs/source/testing/layout.rst new file mode 100644 index 000000000..9f196a637 --- /dev/null +++ b/docs/source/testing/layout.rst @@ -0,0 +1,30 @@ +.. _layout: + +Layout and conventions +---------------------- +Test files and directories follow a few conventions, which makes it easy to +create (or expect) certain interactions between tests and scenarios. + +All tests are in the ``tests`` directory. Scenarios are defined in +``tests/functional/`` and use the following convention for directory +structure:: + + tests/functional//// + +For example: ``tests/functional/centos/7/journal-collocation`` + +Within a test scenario there are a few files that define what that specific +scenario needs for the tests, like how many OSD nodes or MON nodes. Tls + +At the very least, a scenario will need these files: + +* ``Vagrantfile``: must be symlinked from the root directory of the project +* ``hosts``: An Ansible hosts file that defines the machines part of the + cluster +* ``group_vars/all``: if any modifications are needed for deployment, this + would override them. Additionally, further customizations can be done. For + example, for OSDs that would mean adding ``group_vars/osds`` +* ``vagrant_variables.yml``: Defines the actual environment for the test, where + machines, networks, disks, linux distro/version, can be defined. + + diff --git a/docs/source/testing/modifying.rst b/docs/source/testing/modifying.rst new file mode 100644 index 000000000..312553587 --- /dev/null +++ b/docs/source/testing/modifying.rst @@ -0,0 +1,5 @@ + +.. _modifying: + +Modifying (or adding) tests +=========================== diff --git a/docs/source/testing/running.rst b/docs/source/testing/running.rst new file mode 100644 index 000000000..d18bc722a --- /dev/null +++ b/docs/source/testing/running.rst @@ -0,0 +1,147 @@ + +.. _running_tests: + +Running Tests +============= +Although tests run continuously in CI, a lot of effort was put into making it +easy to run in any environment, as long as a couple of requirements are met. + + +.. _dependencies: + +Dependencies +------------ +There are some Python dependencies, which are listed in a ``requirements.txt`` +file within the ``tests/`` directory. These are meant to be installed using +Python install tools (pip in this case):: + + pip install -r tests/requirements.txt + +For virtualization, either libvirt or VirtualBox is needed (there is native +support from the harness for both). This makes the test harness even more +flexible as most platforms will be covered by either VirtualBox or libvirt. + + +.. _running_a_scenario: + +Running a scenario +------------------ +Tests are driven by ``tox``, a command line tool to run a matrix of tests defined in +a configuration file (``tox.ini`` in this case at the root of the project). + +For a thorough description of a scenario see :ref:`test_scenarios`. + +To run a single scenario, make sure it is available (should be defined from +``tox.ini``) by listing them:: + + tox -l + +In this example, we will use the ``kraken-ansible2.2-xenial_cluster`` one. The +harness defaults to ``VirtualBox`` as the backend, so if you have that +installed in your system then this command should just work:: + + tox -e kraken-ansible2.2-xenial_cluster + +And for libvirt it would be:: + + tox -e kraken-ansible2.2-xenial_cluster -- --provider=libvirt + + +.. warning:: Depending on the type of scenario and resources available, running + these tests locally in a personal computer can be very resource intensive. + +.. note:: Most test runs take between 20 and 40 minutes depending on system + resources + +The command should bring up the machines needed for the test, provision them +with ceph-ansible, run the tests, and tear the whole environment down at the +end. + + +The output would look something similar to this trimmed version:: + + kraken-ansible2.2-xenial_cluster create: /Users/alfredo/python/upstream/ceph-ansible/.tox/kraken-ansible2.2-xenial_cluster + kraken-ansible2.2-xenial_cluster installdeps: ansible==2.2.2, -r/Users/alfredo/python/upstream/ceph-ansible/tests/requirements.txt + kraken-ansible2.2-xenial_cluster runtests: commands[0] | vagrant up --no-provision --provider=virtualbox + Bringing machine 'client0' up with 'virtualbox' provider... + Bringing machine 'rgw0' up with 'virtualbox' provider... + Bringing machine 'mds0' up with 'virtualbox' provider... + Bringing machine 'mon0' up with 'virtualbox' provider... + Bringing machine 'mon1' up with 'virtualbox' provider... + Bringing machine 'mon2' up with 'virtualbox' provider... + Bringing machine 'osd0' up with 'virtualbox' provider... + ... + + +After all the nodes are up, ceph-ansible will provision them, and run the +playbook(s):: + + ... + PLAY RECAP ********************************************************************* + client0 : ok=4 changed=0 unreachable=0 failed=0 + mds0 : ok=4 changed=0 unreachable=0 failed=0 + mon0 : ok=4 changed=0 unreachable=0 failed=0 + mon1 : ok=4 changed=0 unreachable=0 failed=0 + mon2 : ok=4 changed=0 unreachable=0 failed=0 + osd0 : ok=4 changed=0 unreachable=0 failed=0 + rgw0 : ok=4 changed=0 unreachable=0 failed=0 + ... + + +Once the whole environment is all running the tests will be sent out to the +hosts, with output similar to this:: + + kraken-ansible2.2-xenial_cluster runtests: commands[4] | testinfra -n 4 --sudo -v --connection=ansible --ansible-inventory=/Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster/hosts /Users/alfredo/python/upstream/ceph-ansible/tests/functional/tests + ============================ test session starts =========================== + platform darwin -- Python 2.7.8, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /Users/alfredo/python/upstream/ceph-ansible/.tox/kraken-ansible2.2-xenial_cluster/bin/python + cachedir: ../../../../.cache + rootdir: /Users/alfredo/python/upstream/ceph-ansible/tests, inifile: pytest.ini + plugins: testinfra-1.5.4, xdist-1.15.0 + [gw0] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster + [gw1] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster + [gw2] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster + [gw3] darwin Python 2.7.8 cwd: /Users/alfredo/python/upstream/ceph-ansible/tests/functional/ubuntu/16.04/cluster + [gw0] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] + [gw1] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] + [gw2] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] + [gw3] Python 2.7.8 (v2.7.8:ee879c0ffa11, Jun 29 2014, 21:07:35) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] + gw0 [154] / gw1 [154] / gw2 [154] / gw3 [154] + scheduling tests via LoadScheduling + + ../../../tests/test_install.py::TestInstall::test_ceph_dir_exists[ansible:/mon0] + ../../../tests/test_install.py::TestInstall::test_ceph_dir_is_a_directory[ansible:/mon0] + ../../../tests/test_install.py::TestInstall::test_ceph_conf_is_a_file[ansible:/mon0] + ../../../tests/test_install.py::TestInstall::test_ceph_dir_is_a_directory[ansible:/mon1] + [gw2] PASSED ../../../tests/test_install.py::TestCephConf::test_ceph_config_has_mon_host_line[ansible:/mon0] + ../../../tests/test_install.py::TestInstall::test_ceph_conf_exists[ansible:/mon1] + [gw3] PASSED ../../../tests/test_install.py::TestCephConf::test_mon_host_line_has_correct_value[ansible:/mon0] + ../../../tests/test_install.py::TestInstall::test_ceph_conf_is_a_file[ansible:/mon1] + [gw1] PASSED ../../../tests/test_install.py::TestInstall::test_ceph_command_exists[ansible:/mon1] + ../../../tests/test_install.py::TestCephConf::test_mon_host_line_has_correct_value[ansible:/mon1] + ... + +Finally the whole environment gets torn down:: + + kraken-ansible2.2-xenial_cluster runtests: commands[5] | vagrant destroy --force + ==> osd0: Forcing shutdown of VM... + ==> osd0: Destroying VM and associated drives... + ==> mon2: Forcing shutdown of VM... + ==> mon2: Destroying VM and associated drives... + ==> mon1: Forcing shutdown of VM... + ==> mon1: Destroying VM and associated drives... + ==> mon0: Forcing shutdown of VM... + ==> mon0: Destroying VM and associated drives... + ==> mds0: Forcing shutdown of VM... + ==> mds0: Destroying VM and associated drives... + ==> rgw0: Forcing shutdown of VM... + ==> rgw0: Destroying VM and associated drives... + ==> client0: Forcing shutdown of VM... + ==> client0: Destroying VM and associated drives... + + +And a brief summary of the scenario(s) that ran is displayed:: + + ________________________________________________ summary _________________________________________________ + kraken-ansible2.2-xenial_cluster: commands succeeded + congratulations :) + diff --git a/docs/source/testing/scenarios.rst b/docs/source/testing/scenarios.rst new file mode 100644 index 000000000..031e1abf1 --- /dev/null +++ b/docs/source/testing/scenarios.rst @@ -0,0 +1,5 @@ + +.. _test_scenarios: + +Test Scenarios +============== -- 2.39.5