testing: use ceph vagrant box on ubuntu xenial mon osd mds rgw scenario

Signed-off-by: Alfredo Deza <adeza@redhat.com>

testing: add comment for ceph vagrant boxes on centos 7 mon osd scenario

Signed-off-by: Alfredo Deza <adeza@redhat.com>

testing: add comment for ceph vagrant boxes on centos 7 mon osd mds rgw scenario

Signed-off-by: Alfredo Deza <adeza@redhat.com>

testing: parse and set DISABLE_SYNCED_FOLDER in vagrant file

Signed-off-by: Alfredo Deza <adeza@redhat.com>

tests: disable sync folder when testing

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1089 from ceph/vagrant-boxes

tests: adds a few more scenarios

tests: adds comments to tox.ini describing each scenario

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: make the centos7-mon-osd scenario use raw_multi_journal

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: adds the centos7-cluster scenario to tox.ini

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: adds a scenario for a centos7 1mon 1osd 1mds 1rgw cluster

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: adds the xenial mon-osd-mds-rgw scenario as xenial-cluster

I tried adding the scenario as xenial-mon-osd-mds-rgw but for
whatever reason tox refused to select that environment. Changing the
name to xenial-cluster fixed that though.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: adds a scenario to test a xenial 1mon 1osd 1mds 1rgw cluster

This scenario duplicates what we are currently doing with our
ceph-ansible testing using OVH and a single node, except now
we are using 4 separate nodes.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: make xenial-mon-osd use raw_multi_journal instead of collocation

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

Merge pull request #1087 from ceph/vagrant-boxes

use vagrant boxes that support libvirt and virtualbox

tests: provide a way use libvirt with vagrant and tox

This allows for:

tox -- --provider=libvirt

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

tests: change the centos box to centos/7

This box supports both libvirt and virtualbox. Eventually we want to
be building our own boxes but this should work in the short term.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

tests: change the xenial box to yk0/ubuntu-xenial

This box supports both virtualbox and libvirt. Eventually
we want to be building our own vagrant boxes, but this might
work for now.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: vagrant#boxes

Merge pull request #1086 from ceph/testing-updates

testing: adds 3 new functional testing scenarios

tests: adds the vagrant_variables.yml that were previously ignored

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

don't ignore vagrant_variables.yml in test directories

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: updates tox to test the new scenarios

This also turns on -vv for ansible and adds an environment for
ansible==1.9.4 testing.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: adds a 1mon 1osd centos7 test scenario

This uses journal collocation for the osd.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: adds a 1mon 1osd xenial test scenario

This sets up the osd using journal collocation

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: ceph.conf tests for the xenial 3 mon cluster

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: adds a xenial 3 mon test scenario

This can be used to test if mon hosts and
mon initial members are being set properly with
multiple hosts.

Also, to verify that monitor_address and monitor_interface
options both work as described.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: ignore folders created by ceph-ansible during testing

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

common: reformat mon_initial_members with indention

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

common: provide a default for monitor_address_block

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

tests: remove the initial_members scenario

This was just a placeholder until we could get more valid scenarios in
place.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

common: check for monitor_interface before monitor_address in ceph.conf

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: testing#updates

Merge pull request #1074 from ceph/test-group-vars

functional test scenarios should include a group_vars dir

tests: require a group_vars folder in the scenario dir

This will allow for no changes needed in the ansible playbook command
when adding new scenarios. Each scenario will just need a hosts file and
a group_vars directory to define how the cluster should be setup.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1058 from ceph/pytest-harness

testing harness with py.test and Vagrant

tests: add a hosts file for the initial_members scenario

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: gitignore the .tox subdir

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: make the tox.ini config call the ssh config helper script

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: add a helper script to generate vagrant ssh configs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: remove the need for a setenv entry per scenario in tox.ini

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: pytest#harness

tests: adds a tox.ini to run the functional tests

This currently tests the initial-members scenario for both ansible 2.1
and ansible 2.2. It assumes a hosts file and vagrant_ssh_config exists
in the scenario directory.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: pytest#harness

tests: mon_initial_members should look for the right hostname

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: fix helper to use the actual lines form ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: use a different vagrant box

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: include the vagrant_variables.yml file specific to this scenario

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: create a README with some explanation on how to use the test harness

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: define a requirements.txt file

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: create an example scenario file

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: create the conftest file that does all the magic

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: add the symlinked Vagrantfile to initial_members

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

create a scenario config file for mon initial members

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: add __init__.py for functional tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: common basic tests for after installation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: add __init__.py for mon tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: create a test to check for mon_initial_members

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

tests: placeholder pytest.ini to define test root dir

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Resolves: pytest#harness

Merge pull request #1073 from ceph/upgrade

rolling_update: add variable to upgrade ceph

rolling_update: add variable to upgrade ceph

My stupid self removed this crucial variable here: 217ce3ca thinking it
was another hard coded variable import where this is actually the
trigger for the upgrade.

Closes: #1071
Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1070 from ceph/RHBZ-1391675

rolling update: fix restarts of radosgw

rolling update: fix restarts of radosgw

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rhbz#1391675
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1069 from ceph/dmycrypt-checks

osd dmcrypt: add ability to check devices

Merge pull request #1065 from ceph/fix-conf-template

various fixes for the ceph.conf template

common: remove underscores in ceph.conf so config names are consistent

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

osd dmcrypt: add ability to check devices

Even for dmcrypt we need to check the "devices" status and
"raw_journal_devices" as well so we can fix them if there is something
wrong with them.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1066 from eduardegorov/egorove_missed_file_for_PR_#1024

Adjust 'devices' list check for being not defined in purge-cluster pl…

Adjust 'devices' list check for being not defined in purge-cluster playbook (see PR #1024)

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

ceph-common: remove unused mon_addr_interface.j2 template

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-common: remove unused mon_addr_address.j2 template

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

fixes monitor_address and monitor_interface features

Before this commit if you had set monitor_interface in your
inventory file for a specific host it would be ignored and the value
in group_vars/all would have been used.

Also, this enables support for monitor_address again as it had been
broken by previous changes to this template.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1024 from eduardegorov/egorove_make_devices_optional

Make {{ devices }} list optional

Merge pull request #1063 from eduardegorov/egorove_purge-cluster_hardcoded_group_names

Fix hard-coded group names in include tasks for group variable file names

Fix typos

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

Fix rbd-mirrors group name

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

Fix hard-coded host group names in include tasks for group variables' file paths.

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

Fix 'raw_journal_devices' to be initialized as a list in ceph-common defaults. Add empty 'devices' list there as well. Adjust 'devices' and 'raw_journal_devices' checks.

This is done for preventing of their use-before-definition for osd scenarios checks (should be removed after a refactor has properly seperated all the checks into appropriate roles).

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

Make {{ raw_journal_devices }} list optional: define it as empty list by default, remove unneccessary 'default([])' checks

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

Make {{ devices }} list optional: define it as empty list by default, remove unneccessary 'default([])' checks

Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>

Merge pull request #1045 from wind0204/pr-monitor_interface_with_hyphens

now you can use a network link name with a hyphen

Merge pull request #1052 from verdurin/revert-1042-centos-extras-ini_file

Revert "Change enabling of extras repo for CentOS to use ini_file"

Merge pull request #1060 from ceph/issue-1054

raw_multi_journal is not required when using dmcrypt_dedicated_journal

Merge pull request #1048 from cloudnull/config_template_v22

Updated config_template for 2.2 compat

raw_multi_journal is not required when using dmcrypt_dedicated_journal

Fixes: https://github.com/ceph/ceph-ansible/issues/1054
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: issue#1054

Merge pull request #1057 from ceph/RHBZ-1389456

rolling_update: define mon_group_name when upgrading the mons

rolling_update: define mon_group_name when upgrading the mons

see: https://bugzilla.redhat.com/show_bug.cgi?id=1389456

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rhbz#1389456

Merge pull request #1050 from ceph/issue-1041

ceph-osd: use ini_file instead of yum_repository to enable Centos extras

Merge pull request #1055 from ceph/update-role-path

rebased rolling_update: fix wrong indent

rolling_update: fix wrong indent

Fixing: https://bugzilla.redhat.com/show_bug.cgi?id=1388295
Also add some notes in the README on how to run infrastructure
playbooks.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1049 from font/rolling_update

Update for infrastructure-playbooks execution

Update for infrastructure-playbooks execution

- Updates to allow running infrastructure-playbooks both from within its
  directory or root directory of ceph-ansible.

Signed-off-by: Ivan Font <ifont@redhat.com>

Merge pull request #1051 from ceph/syntax-fixes

infrastructure-playbooks: fix syntax errors in all playbooks

Revert "Change enabling of extras repo for CentOS to use ini_file"

infrastructure-playbooks: fix syntax errors in all playbooks

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-osd: use ini_file instead of yum_repository to enable Centos extras

Fixes issue #1041

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: issue#1041

Updated config_template for 2.2 compat

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Merge pull request #1029 from badone/wip-python2

site.yml.sample: Attempt to install python2

Merge pull request #1042 from verdurin/centos-extras-ini_file

Change enabling of extras repo for CentOS to use ini_file

now you can use a network link name with a hyphen

Merge pull request #1043 from ceph/rbd-pg

mon: destroy and recreate rbd pool if necessary

mon: destroy and recreate rbd pool if necessary

Users reported that pool_default_pg_num is not honoured for the default
pool 'rbd'. So now we check the pg num value for the RBD pool and if it
does not match pool_default_pg_num then we delete and recreate it.
We also make sure the pool is empty first, just in case someone changed
the value manually and didn't reflect the change in ceph-ansible.

The only issue with this patch is that the pool ID will not be 0 anymore
but more likely 1.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1044 from batrick/no-vagrant-dev-branch

vagrant: use group_vars for ceph pkg source

vagrant: use group_vars for ceph pkg source

This was a holdout from the Linode merge that shouldn't have been
included. The right way to set the installation source is through
group_vars.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Change enabling of extras repo for CentOS to use ini_file

Signed-off-by: Adam Huffman <bloch@verdurin.com>