# General information about the project.
project = u'ceph-ansible'
-copyright = u'2017, Ceph team and individual contributors'
+copyright = u'2017-2018, Ceph team and individual contributors'
author = u'Ceph team and individual contributors'
# The version info for the project you're documenting, acts as replacement for
Contribution Guidelines
=======================
-The repository centralises all the Ansible roles. The roles are all part of the Galaxy.
+The repository centralises all the Ansible roles. The roles are all part of the Ansible Galaxy.
+
We love contribution and we love giving visibility to our contributors, this is why all the **commits must be signed-off**.
Mailing list
IRC
---
-Feel free to join us in the channel #ceph-ansible of the OFTC servers.
+Feel free to join us in the channel ``#ceph-ansible`` of the OFTC servers (https://www.oftc.net).
GitHub
------
$ git commit --amend
$ git push -f origin my-working-branch
-PR Testing
-----------
-Pull Request testing is handled by jenkins. All test must pass before your PR will be merged.
+Pull Request Testing
+--------------------
+
+Pull request testing is handled by Jenkins. All test must pass before your pull request will be merged.
All of tests that are running are listed in the GitHub UI and will list their current status.
-If a test fails and you'd like to rerun it, comment on your PR in the following format:
+If a test fails and you'd like to rerun it, comment on your pull request in the following format:
.. code-block:: none
If a change should be backported to a ``stable-*`` Git branch:
-- Mark your PR with the GitHub label "Backport" so we don't lose track of it.
+- Mark your pull request with the GitHub label "Backport" so we don't lose track of it.
- Fetch the latest updates into your clone: ``git fetch``
- Determine the latest available stable branch:
``git branch -r --list "origin/stable-[0-9].[0-9]" | sort -r | sed 1q``
-- Create a new local branch for your PR, based on the stable branch:
+- Create a new local branch for your pull request, based on the stable branch:
``git checkout --no-track -b my-backported-change origin/stable-3.0``
- Cherry-pick your change: ``git cherry-pick -x (your-sha1)``
- Create a new pull request against the ``stable-3.0`` branch.
-- Ensure that your PR's title has the prefix "backport:", so it's clear
+- Ensure that your pull requests's title has the prefix "backport:", so it's clear
to reviewers what this is about.
-- Add a comment in your backport PR linking to the original (master) PR.
+- Add a comment in your backport pull request linking to the original (master) pull request.
All changes to the stable branches should land in master first, so we avoid
regressions.
+============
ceph-ansible
============
GitHub
------
+
You can install directly from the source on GitHub by following these steps:
- Clone the repository:
$ git checkout $branch
-- Next, use pip and the provided requirements.txt to install ansible and other
- needed python libraries:
+- Next, use pip and the provided requirements.txt to install Ansible and other
+ needed Python libraries:
.. code-block:: console
Ansible on RHEL and CentOS
--------------------------
+
You can acquire Ansible on RHEL and CentOS by installing from `Ansible channel <https://access.redhat.com/articles/3174981>`_.
On RHEL:
branches have been QE tested and sometimes recieve backport fixes throughout their lifecycle.
The ``master`` branch should be considered experimental and used with caution.
-- ``stable-3.0`` Support for ceph versions ``jewel`` and ``luminous``. This branch supports ansible versions
+- ``stable-3.0`` Support for Ceph versions ``jewel`` and ``luminous``. This branch supports Ansible versions
``2.4`` and ``2.5``.
-- ``stable-3.1`` Support for ceph version ``luminous`` and ``mimic``. This branch supports ansible versions
+- ``stable-3.1`` Support for Ceph version ``luminous`` and ``mimic``. This branch supports Ansible versions
``2.4`` and ``2.5``
-- ``master`` Support for ceph versions ``luminous``, and ``mimic``. This branch supports ansible version ``2.5``.
+- ``master`` Support for Ceph versions ``luminous``, and ``mimic``. This branch supports Ansible version ``2.5``.
Configuration and Usage
=======================
-This project assumes you have a basic knowledge of how ansible works and have already prepared your hosts for
-configuration by ansible.
+This project assumes you have a basic knowledge of how Ansible works and have already prepared your hosts for
+configuration by Ansible.
-After you've cloned the ``ceph-ansible`` repository, selected your branch and installed ansible then you'll need to create
-your inventory file, playbook and configuration for your ceph cluster.
+After you've cloned the ``ceph-ansible`` repository, selected your branch and installed Ansible then you'll need to create
+your inventory file, playbook and configuration for your Ceph cluster.
Inventory
---------
-The ansible inventory file defines the hosts in your cluster and what roles each host plays in your ceph cluster. The default
+The Ansible inventory file defines the hosts in your cluster and what roles each host plays in your Ceph cluster. The default
location for an inventory file is ``/etc/ansible/hosts`` but this file can be placed anywhere and used with the ``-i`` flag of
-ansible-playbook. An example inventory file would look like:
+``ansible-playbook``.
+
+An example inventory file would look like:
.. code-block:: ini
.. note::
- For more information on ansible inventories please refer to the ansible documentation: http://docs.ansible.com/ansible/latest/intro_inventory.html
+ For more information on Ansible inventories please refer to the Ansible documentation: http://docs.ansible.com/ansible/latest/intro_inventory.html
Playbook
--------
.. note::
- It's important the playbook you use is placed at the root of the ``ceph-ansible`` project. This is how ansible will be able to find the roles that
+ It's important the playbook you use is placed at the root of the ``ceph-ansible`` project. This is how Ansible will be able to find the roles that
``ceph-ansible`` provides.
Configuration Validation
"changed": false
}
-ceph-ansible - choose installation method
------------------------------------------
+Installation methods
+--------------------
Ceph can be installed through several methods.
installation/methods
-ceph-ansible Configuration
---------------------------
+Configuration
+-------------
-The configuration for your ceph cluster will be set by the use of ansible variables that ``ceph-ansible`` provides. All of these options and their default
+The configuration for your Ceph cluster will be set by the use of ansible variables that ``ceph-ansible`` provides. All of these options and their default
values are defined in the ``group_vars/`` directory at the root of the ``ceph-ansible`` project. Ansible will use configuration in a ``group_vars/`` directory
-that is relative to your inventory file or your playbook. Inside of the ``group_vars/`` directory there are many sample ansible configuration files that relate
-to each of the ceph daemon groups by their filename. For example, the ``osds.yml.sample`` contains all the default configuation for the OSD daemons. The ``all.yml.sample``
+that is relative to your inventory file or your playbook. Inside of the ``group_vars/`` directory there are many sample Ansible configuration files that relate
+to each of the Ceph daemon groups by their filename. For example, the ``osds.yml.sample`` contains all the default configuation for the OSD daemons. The ``all.yml.sample``
file is a special ``group_vars`` file that applies to all hosts in your cluster.
.. note::
- For more information on setting group or host specific configuration refer to the ansible documentation: http://docs.ansible.com/ansible/latest/intro_inventory.html#splitting-out-host-and-group-specific-data
+ For more information on setting group or host specific configuration refer to the Ansible documentation: http://docs.ansible.com/ansible/latest/intro_inventory.html#splitting-out-host-and-group-specific-data
-At the most basic level you must tell ``ceph-ansible`` what version of ceph you wish to install, the method of installation, your clusters network settings and
+At the most basic level you must tell ``ceph-ansible`` what version of Ceph you wish to install, the method of installation, your clusters network settings and
how you want your OSDs configured. To begin your configuration rename each file in ``group_vars/`` you wish to use so that it does not include the ``.sample``
at the end of the filename, uncomment the options you wish to change and provide your own value.
-An example configuration that deploys the upstream ``jewel`` version of ceph with OSDs that have collocated journals would look like this in ``group_vars/all.yml``:
+An example configuration that deploys the upstream ``jewel`` version of Ceph with OSDs that have collocated journals would look like this in ``group_vars/all.yml``:
.. code-block:: yaml
- ``monitor_interface`` or ``monitor_address``
- ``radosgw_interface`` or ``radosgw_address``
-ceph.conf Configuration
------------------------
+``ceph.conf`` Configuration File
+---------------------------------
+
+The supported method for defining your ``ceph.conf`` is to use the ``ceph_conf_overrides`` variable. This allows you to specify configuration options using
+an INI format. This variable can be used to override sections already defined in ``ceph.conf`` (see: ``roles/ceph-config/templates/ceph.conf.j2``) or to provide
+new configuration options.
+
+The following sections in ``ceph.conf`` are supported:
-The supported method for defining your ceph.conf is to use the ``ceph_conf_overrides`` variable. This allows you to specify configuration options using
-an INI format. This variable can be used to override sections already defined in ceph.conf (see: ``roles/ceph-config/templates/ceph.conf.j2``) or to provide
-new configuration options. The following sections in ceph.conf are supported: [global], [mon], [osd], [mds] and [rgw].
+* ``[global]``,
+* ``[mon]``
+* ``[osd]``
+* ``[mds]``
+* ``[rgw]``
An example:
.. note::
- We will no longer accept pull requests that modify the ceph.conf template unless it helps the deployment. For simple configuration tweaks
+ We will no longer accept pull requests that modify the ``ceph.conf`` template unless it helps the deployment. For simple configuration tweaks
please use the ``ceph_conf_overrides`` variable.
-Full documentation for configuring each of the ceph daemon types are in the following sections.
+Full documentation for configuring each of the Ceph daemon types are in the following sections.
OSD Configuration
-----------------
-OSD configuration is set by selecting an osd scenario and providing the configuration needed for
+OSD configuration is set by selecting an OSD scenario and providing the configuration needed for
that scenario. Each scenario is different in it's requirements. Selecting your OSD scenario is done
by setting the ``osd_scenario`` configuration option.
Testing
=======
-Documentation for writing functional testing scenarios for ceph-ansible.
+Documentation for writing functional testing scenarios for ``ceph-ansible``.
* :doc:`Testing with ceph-ansible <testing/index>`
* :doc:`Glossary <testing/glossary>`
====================
The following are all of the available options for the installing Ceph through different channels.
+
We support 3 main installation methods, all managed by the ``ceph_origin`` variable:
-- ``repository``: means that you will get ceph installed through a new repository. Later below choose between ``community``, ``rhcs`` or ``dev``. These options will be exposed through the ``ceph_repository`` variable.
+- ``repository``: means that you will get Ceph installed through a new repository. Later below choose between ``community``, ``rhcs`` or ``dev``. These options will be exposed through the ``ceph_repository`` variable.
- ``distro``: means that no separate repo file will be added and you will get whatever version of Ceph is included in your Linux distro.
-- ``local``: means that the ceph binaries will be copied over from the local machine (not well tested, use at your own risk)
+- ``local``: means that the Ceph binaries will be copied over from the local machine (not well tested, use at your own risk)
Origin: Repository
------------------
This scenario has the following optional configuration options:
- ``osd_objectstore``: defaults to ``filestore`` if not set. Available options are ``filestore`` or ``bluestore``.
- You can only select ``bluestore`` if the ceph release is Luminous or greater.
+ You can only select ``bluestore`` if the Ceph release is Luminous or greater.
- ``dmcrypt``: defaults to ``false`` if not set.
- ``dedicated_devices``: defaults to ``devices`` if not set
- ``osd_objectstore``: defaults to ``filestore`` if not set. Available options are ``filestore`` or ``bluestore``.
- You can only select ``bluestore`` with the ceph release is Luminous or greater.
+ You can only select ``bluestore`` with the Ceph release is Luminous or greater.
- ``dmcrypt``: defaults to ``false`` if not set.
lvm
---
+
This OSD scenario uses ``ceph-volume`` to create OSDs from logical volumes and
-is only available when the ceph release is Luminous or newer.
+is only available when the Ceph release is Luminous or newer.
.. note::
-
-.. development:
+.. _development:
ceph-ansible testing for development
====================================
-
.. _testing:
Testing
=======
-ceph-ansible has the ability to test different scenarios (collocated journals
+
+``ceph-ansible`` has the ability to test different scenarios (collocated journals
or dmcrypt OSDs for example) in an isolated, repeatable, and easy way.
These tests can run locally with VirtualBox or via libvirt if available, which
-
.. _modifying:
Modifying (or adding) tests
Running a scenario
------------------
+
Tests are driven by ``tox``, a command line tool to run a matrix of tests defined in
a configuration file (``tox.ini`` in this case at the root of the project).
resources
The command should bring up the machines needed for the test, provision them
-with ceph-ansible, run the tests, and tear the whole environment down at the
+with ``ceph-ansible``, run the tests, and tear the whole environment down at the
end.
...
-After all the nodes are up, ceph-ansible will provision them, and run the
+After all the nodes are up, ``ceph-ansible`` will provision them, and run the
playbook(s):
.. code-block:: console
Test Scenarios
==============
+
Scenarios are distinct environments that describe a Ceph deployment and
configuration. Scenarios are isolated as well, and define what machines are
-needed aside from any ceph-ansible configuration.
+needed aside from any ``ceph-ansible`` configuration.
.. _scenario_files:
Scenario Files
==============
+
The scenario is described in a ``vagrant_variables.yml`` file, which is
consumed by ``Vagrant`` when bringing up an environment.
There are just a handful of required files, this is the most basic layout.
There are just a handful of required files, these sections will cover the
-required (most basic) ones. Alternatively, other ceph-ansible files can be
+required (most basic) ones. Alternatively, other ``ceph-ansible`` files can be
added to customize the behavior of a scenario deployment.
``vagrant_variables.yml``
-------------------------
+
There are a few sections in the ``vagrant_variables.yml`` file which are easy
to follow (most of them are 1 line settings).
-
-* **docker**: (bool) Indicates if the scenario will deploy docker daemons
+* **docker**: (bool) Indicates if the scenario will deploy Docker daemons
* **VMS**: (int) These integer values are just a count of how many machines will be
needed. Each supported type is listed, defaulting to 0:
* **SUBNETS**: These are used for configuring the network availability of each
server that will be booted as well as being used as configuration for
- ceph-ansible (and eventually ceph). The two values that are **required**:
+ ``ceph-ansible`` (and eventually Ceph). The two values that are **required**:
.. code-block:: yaml
* **vagrant_disable_synced_folder**: (bool) when disabled, it will make
booting machines faster because no files need to be synced over.
-* **os_tuning_params**: These are passed onto ceph-ansible as part of the
+* **os_tuning_params**: These are passed onto ``ceph-ansible`` as part of the
variables for "system tunning". These shouldn't be changed.
``Vagrantfile``
---------------
+
The ``Vagrantfile`` should not need to change, and it is symlinked back to the
``Vagrantfile`` that exists in the root of the project. It is linked in this
way so that a vagrant environment can be isolated to the given scenario.
``hosts``
---------
+
The ``hosts`` file should contain the hosts needed for the scenario. This might
seem a bit repetitive since machines are already defined in
:ref:`vagrant_variables` but it allows granular changes to hosts (for example
defining an interface vs. an IP on a monitor) which can help catch issues in
-ceph-ansible configuration. For example:
+``ceph-ansible`` configuration. For example:
.. code-block:: ini
``group_vars``
--------------
-This directory holds any configuration change that will affect ceph-ansible
+
+This directory holds any configuration change that will affect ``ceph-ansible``
deployments in the same way as if ansible was executed from the root of the
project.
The file that will need to be defined always is ``all`` where (again) certain
values like ``public_network`` and ``cluster_network`` will need to be defined
-along with any customizations that ceph-ansible supports.
+along with any customizations that ``ceph-ansible`` supports.
.. _scenario_wiring:
Scenario Wiring
---------------
+
Scenarios are just meant to provide the Ceph environment for testing, but they
do need to be defined in the ``tox.ini`` so that they are available to the test
framework. To see a list of available scenarios, the following command (ran
Tests
=====
+
Actual tests are written in Python methods that accept optional fixtures. These
fixtures come with interesting attributes to help with remote assertions.
Test Fixtures
=============
+
Test fixtures are a powerful feature of ``py.test`` and most tests depend on
this for making assertions about remote nodes. To request them in a test
method, all that is needed is to require it as an argument.
assert File(node["conf_path"]).contains("^mon initial members = .*$")
-.. node:
+.. _node:
``node`` fixture
----------------
+
The ``node`` fixture contains a few useful pieces of information about the node
where the test is being executed, this is captured once, before tests run:
* ``address``: The IP for the ``eth1`` interface
* ``subnet``: The subnet that ``address`` belongs to
-* ``vars``: all the ansible vars set for the current run
+* ``vars``: all the Ansible vars set for the current run
* ``osd_ids``: a list of all the OSD IDs
* ``num_mons``: the total number of monitors for the current environment
* ``num_devices``: the number of devices for the current node
* ``cluster_address``: the address used for cluster communication. All
environments are set up with 2 interfaces, 1 being used exclusively for the
cluster
-* ``docker``: A boolean that identifies a Ceph docker cluster
-* ``osds``: A list of OSD IDs, unless it is a docker cluster, where it gets the
+* ``docker``: A boolean that identifies a Ceph Docker cluster
+* ``osds``: A list of OSD IDs, unless it is a Docker cluster, where it gets the
name of the devices (e.g. ``sda1``)
Other Fixtures
--------------
+
There are a lot of other fixtures provided by :ref:`testinfra` as well as
``py.test``. The full list of ``testinfra`` fixtures are available in
`testinfra_fixtures`_
When running ``tox`` we've allowed for the usage of environment variables to tweak certain settings
of the playbook run using Ansible's ``--extra-vars``. It's helpful in Jenkins jobs or for manual test
-runs of ceph-ansible.
+runs of ``ceph-ansible``.
The following environent variables are available for use:
-* ``FETCH_DIRECTORY`` : (default: ``changedir``) This would configure the ceph-ansible variable ``fetch_directory``. This defaults to
+* ``FETCH_DIRECTORY`` : (default: ``changedir``) This would configure the ``ceph-ansible`` variable ``fetch_directory``. This defaults to
the ``changedir`` of the given scenario and should not need to be changed.
-* ``CEPH_STABLE_RELEASE``: (default: ``jewel``) This would configure the ceph-ansible variable ``ceph_stable_relese``. This is set
+* ``CEPH_STABLE_RELEASE``: (default: ``jewel``) This would configure the ``ceph-ansible`` variable ``ceph_stable_relese``. This is set
automatically when using the ``jewel-*`` or ``kraken-*`` testing scenarios.
-* ``UPDATE_CEPH_STABLE_RELEASE``: (default: ``kraken``) This would configure the ceph-ansible variable ``ceph_stable_relese`` during an ``update``
+* ``UPDATE_CEPH_STABLE_RELEASE``: (default: ``kraken``) This would configure the ``ceph-ansible`` variable ``ceph_stable_relese`` during an ``update``
scenario. This is set automatically when using the ``jewel-*`` or ``kraken-*`` testing scenarios.
-* ``CEPH_DOCKER_REGISTRY``: (default: ``docker.io``) This would configure the ceph-ansible variable ``ceph_docker_registry``.
+* ``CEPH_DOCKER_REGISTRY``: (default: ``docker.io``) This would configure the ``ceph-ansible`` variable ``ceph_docker_registry``.
-* ``CEPH_DOCKER_IMAGE``: (default: ``ceph/daemon``) This would configure the ceph-ansible variable ``ceph_docker_image``.
+* ``CEPH_DOCKER_IMAGE``: (default: ``ceph/daemon``) This would configure the ``ceph-ansible`` variable ``ceph_docker_image``.
-* ``CEPH_DOCKER_IMAGE_TAG``: (default: ``latest``) This would configure the ceph-ansible variable ``ceph_docker_image_name``.
+* ``CEPH_DOCKER_IMAGE_TAG``: (default: ``latest``) This would configure the ``ceph-ansible`` variable ``ceph_docker_image_name``.
-* ``CEPH_DEV_BRANCH``: (default: ``master``) This would configure the ceph-ansible variable ``ceph_dev_branch`` which defines which branch we'd
+* ``CEPH_DEV_BRANCH``: (default: ``master``) This would configure the ``ceph-ansible`` variable ``ceph_dev_branch`` which defines which branch we'd
like to install from shaman.ceph.com.
-* ``CEPH_DEV_SHA1``: (default: ``latest``) This would configure the ceph-ansible variable ``ceph_dev_sha1`` which defines which sha1 we'd like
+* ``CEPH_DEV_SHA1``: (default: ``latest``) This would configure the ``ceph-ansible`` variable ``ceph_dev_sha1`` which defines which sha1 we'd like
to install from shaman.ceph.com.
-* ``UPDATE_CEPH_DEV_BRANCH``: (default: ``master``) This would configure the ceph-ansible variable ``ceph_dev_branch`` which defines which branch we'd
+* ``UPDATE_CEPH_DEV_BRANCH``: (default: ``master``) This would configure the ``ceph-ansible`` variable ``ceph_dev_branch`` which defines which branch we'd
like to update to from shaman.ceph.com.
-* ``UPDATE_CEPH_DEV_SHA1``: (default: ``latest``) This would configure the ceph-ansible variable ``ceph_dev_sha1`` which defines which sha1 we'd like
+* ``UPDATE_CEPH_DEV_SHA1``: (default: ``latest``) This would configure the ``ceph-ansible`` variable ``ceph_dev_sha1`` which defines which sha1 we'd like
to update to from shaman.ceph.com.
* ``purge`` : This section contains commands that only run for scenarios that purge the cluster and redeploy. You'll see this section being reused in ``testenv``
with the following syntax: ``{[purge]commands}``
-* ``update`` : This section contains commands taht only run for scenarios that deploy a cluster and then upgrade it to another ceph version.
+* ``update`` : This section contains commands taht only run for scenarios that deploy a cluster and then upgrade it to another Ceph version.
* ``testenv`` : This is the main section of the ``tox.ini`` file and is run on every scenario. This section contains many *factors* that define conditional
settings depending on the scenarios defined in the ``envlist``. For example, the factor ``centos7_cluster`` in the ``changedir`` subsection of ``testenv`` sets
in the dynamic matrix that tox creates. Inside of ``{}`` you can include a comma separated list of the *factors*. Do not use a hyphen (``-``) as part
of the *factor* name as those are used by tox as the separator between different factor sets.
-For example, if wanted to add a new test *factor* for the next ceph release of luminious this is how you'd accomplish that. Currently, the first factor set in our ``envlist``
-is used to define the ceph release (``{jewel,kraken,rhcs}-...``). To add luminous you'd change that to look like ``{luminous,kraken,rhcs}-...``. In the ``testenv`` section
+For example, if wanted to add a new test *factor* for the next Ceph release of luminious this is how you'd accomplish that. Currently, the first factor set in our ``envlist``
+is used to define the Ceph release (``{jewel,kraken,rhcs}-...``). To add luminous you'd change that to look like ``{luminous,kraken,rhcs}-...``. In the ``testenv`` section
this is a subsection called ``setenv`` which allows you to provide environment variables to the tox environment and we support an environment variable called ``CEPH_STABLE_RELEASE``. To ensure that all the new tests that are created by adding the luminous *factor* you'd do this in that section: ``luminous: CEPH_STABLE_RELEASE=luminous``.
projects / ceph-ansible.git / commitdiff