Merge pull request #185 from zmc/wip-grafana-fixes

Support Grafana 5.1.x

Reviewed-by: Boris Ranto <branto@redhat.com>

ceph-grafana: Restart container when creating

This is necessary so we can move to newer versions
 of the container image.

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Use the correct UID for 5.1.x

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Bump container version

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #167 from zmc/wip-mon-db-host

Monitor the dashboard host

Merge pull request #182 from zmc/wip-bluestore-selinux

ceph-collectd: Only set SEL context on filestore

ceph-collectd: Set SEL context on bluestore

... as well as filestore.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #175 from zmc/wip-classic-fixes

Fixes for classic mode

Merge pull request #180 from zmc/fix-docker-tests

ceph-docker: Update tests for 'containerized' flag

Merge pull request #173 from zmc/wip-cpu-quota

ansible: Set CPU/RAM quotas on containers

ceph-docker: Update tests for 'containerized' flag

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #166 from zmc/wip-prom-pkg

Allow installing prometheus and node_exporter from packages

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #176 from zmc/wip-fix-notification

ceph-grafana: Use 'admin_user', not 'user'

Reviewed-by: Boris Ranto <branto@redhat.com>

ceph-grafana: Use 'admin_user', not 'user'

We missed this when merging #165 & #170

Signed-off-by: Zack Cerza <zack@redhat.com>

tox.init: Unpin ansible for ansible-syntax

Signed-off-by: Zack Cerza <zack@redhat.com>

node_exporter: Fix RPM name

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-prometheus: Allow installation via packages

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-docker: Fail if using classic mode

Signed-off-by: Zack Cerza <zack@redhat.com>

classic: Fix pie charts w/ grafana 5.0

When legends are off, we're seeing "Invalid dimensions for plot, width =
179, height = 0"

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #170 from zmc/wip-notification

ceph-grafana: Add notification channel

Reviewed-by: Boris Ranto <branto@redhat.com>

ansible: Set RAM quotas on containers

We default to 4GB RAM, and we use double that value for swap.

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Set CPU quotas on containers

We want to default to limiting each container to two cores. While docker
has a nicer way to do this, it isn't supported by ansible as of 2.5.2.
It's easy enough to use the slightly more awkward method, however.

Signed-off-by: Zack Cerza <zack@redhat.com>

playbook.yml: Add node_exporter tag

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #165 from ceph/wip-branto

Add a couple of ansible fixes

ceph-grafana: Add notification channel

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-node-exporter: Open firewall ports

Signed-off-by: Zack Cerza <zack@redhat.com>

Add dashboard for monitoring the db host itself

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-prometheus: Add separate job for db host

So that we can monitor the db host's resources, without mixing that data
in with the ceph cluster's metrics.

Signed-off-by: Zack Cerza <zack@redhat.com>

playbook.yml: Deploy node_exporter on db host

Signed-off-by: Zack Cerza <zack@redhat.com>

Add a sample group_vars file

Signed-off-by: Boris Ranto <branto@redhat.com>

Fix grafana admin user/password setting

Signed-off-by: Boris Ranto <branto@redhat.com>

Extend purge.yml

Signed-off-by: Boris Ranto <branto@redhat.com>

Set home dashboard

Signed-off-by: Boris Ranto <branto@redhat.com>

Strip away containerized_deployment

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge containerized variables

This will also help us fix of accessing these variables in ceph-docker
role where they were not defined previously.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #160 from zmc/wip-prometheus-role

Prometheus support via ceph-mgr

Reviewed-by: Boris Ranto <branto@redhat.com>

ceph-prometheus: Rename container to 'prometheus'

This change requires some manual steps:
- docker rename cephmetrics-prometheus prometheus
- mv /etc/systemd/system/cephmetrics-prometheus.service
  /etc/systemd/system/prometheus.service
- systemctl daemon-reload
And finally, rerun the playbook.

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Use light theme

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Ship a minimal grafana.ini

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Use v5.0.4 for containers

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Use ansible's to_json filter

'tojson' is part of jinja2 2.9, which isn't in RHEL.

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-grafana: Fix dashboard_dir in devel_mode

Signed-off-by: Zack Cerza <zack@redhat.com>

Move devel_packages out of defaults

Signed-off-by: Zack Cerza <zack@redhat.com>

tox: Use py.test -v for integration tests

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-graphite: Fix test_metrics_present

If a cluster changes over time, the test broke. When looking at stored
metrics, assert that our inventory hosts are a subset of the stored
metrics as opposed to doing an equality check.

Signed-off-by: Zack Cerza <zack@redhat.com>

ceph-graphite: Skip irrelevant integration tests

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #154 from jeanchlopez/master

Fix a typo in the purge.yml Ansible playbook file

Merge pull request #156 from ceph/wip-req-req

rpm: Require python-requests

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rpm: Require python-requests

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #1 from jeanchlopez/jeanchlopez-patch-purge-playbook-1

Fix a typo in purge.yml

Fix a typo in purge.yml

Author JC Lopez
Fix.       Change /var/lig/graphite-web with /var/lib/graphite-web

Merge pull request #149 from ceph/wip-centos

ansible: Fixes for CentOS deployments

Reviewed-by: Boris Ranto <branto@redhat.com>

Add python-requests to yum packages

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Work with old Django versions

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #148 from zmc/wip-fixes

Minor dashboard fixes

ceph-rgw-workload.json: Set editable=False

BZ1506132

Signed-off-by: Zack Cerza <zack@redhat.com>

osd-node-detail.json: Don't hardcode cluster name

BZ1506200

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #144 from ceph/wip-testinfra

Add some integration tests using testinfra

Merge pull request #147 from ceph/wip-libsemanage-python

Require libsemanage-python on collector nodes

Reviewed-by: Boris Ranto <branto@redhat.com>

Require libsemanage-python on collector nodes

Signed-off-by: Zack Cerza <zack@redhat.com>

Add some integration tests using testinfra

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #141 from ceph/wip-fix-none-osd-colld

collectors: No update on fetch osd_stats failure

Merge pull request #143 from ceph/wip-pyyaml

Ensure PyYAML is installed on the Grafana node

Reviewed-by: Boris Ranto <branto@redhat.com>

Ensure PyYAML is installed on the Grafana node

It's needed by dashUpdater.py. BZ1497198

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #142 from ceph/ui-bz-fixes

Ui bz fixes

Reviewed-by: Boris Ranto <branto@redhat.com>

osd-information: disk size summary timings and metrics updated

Update to the query interval needed on larger configurations to make sure
all samples are present and OSD size table units updated. BZ1496186

osd-node-detail: updated to correct disk units

All disk units now showing as decimal not binary values. In addition
help text updated on the raw capacity panel to better explain how the
value is derived. BZ1496186

ceph-pools : sort and selection fixes

sortByMaxima added to queries and filter (pool_name) added to the
recovery pool overview panel to work the same as the others. BZ1499734

at-a-glance: updates to address OSD UP state issues

OSD Hosts and RGW Hosts panel changed to vonage status panel to better
reflect UP states BZ1498504

Merge pull request #139 from ceph/wip-packaging

Finalize packaging changes

rpm: Move the rpm patch off the gitroot

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #140 from ceph/wip-vonage-color

Tweak Vonage panel color in newer versions

collectors: No update on fetch osd_stats failure

We currently do not check that we were able to successfully fetch the
osd_stas for a given osd_id which can result in collectors being down
for a while. We should check that we were able to fetch them before we
actually try to update them.

Signed-off-by: Boris Ranto <branto@redhat.com>

Tweak Vonage panel color in newer versions

The plugin recently added a system for configurable colors, breaking our
earlier hack. Add a new one.

See: https://github.com/Vonage/Grafana_Status_panel/commit/8633ba23bb2f336342ba3f7002a07be597b6e816

Signed-off-by: Zack Cerza <zack@redhat.com>

rpm: ipaddr ansible filter requires python-netaddr

Signed-off-by: Boris Ranto <branto@redhat.com>

ansible: Disable devel_mode

Signed-off-by: Boris Ranto <branto@redhat.com>

ansible: Fix merge_vars.yml

Currently, we override the variables in merge_vars. However, if we run
the script several times (e.g. a host is a grafana and a collectd node)
then vars[item] is defined but it can be that it is not a mapping. In
this case we redefine the value to empty string breaking the actual
values of the variables.

We redefine e.g. devel_mode or use_epel to empty string this way making
it false for the grafana server node that is both grafana and the
collectd node.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #138 from ceph/dboard-fixes

dashboard updates to address minor usability issues

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #137 from ceph/wip-graphite-migrate

Fix graphite migration

dashboard updates to address minor usability issues

ceph-at-a-glance: Capacity util panel now links directly to the capacity
chart in the ceph-cluster dashboard BZ 1496198

ceph-osd-information: query timespan changed from the default 1hr to 1m
aligning to the OSDs panel query on the 'at-a-glance' dashboard
BZ 1498504

Use --fake-initial for migrations if necessary

Migrations won't work correctly if the db already exists. This manifests
in an error like:
django.db.utils.OperationalError: table "django_content_type" already exists

Signed-off-by: Zack Cerza <zack@redhat.com>

Use 'graphite-manage migrate' instead of 'syncdb'

syncdb is deprecated.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #136 from ceph/wip-db-wait

Add a note about waiting to collect data

Add a note about waiting to collect data

Users might initially be confused that immediately after deployment, the
dashboard looks broken. This is because it doesn't yet have the data it
needs to function.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #135 from ceph/wip-sanity

Check that all hosts are referred to by FQDN

Skip ansible-lint on 'systemctl show'

The linter complains about this, but ansible doesn't have the required
functionality here.

Signed-off-by: Zack Cerza <zack@redhat.com>

Check that all hosts are referred to by FQDN

Signed-off-by: Zack Cerza <zack@redhat.com>

Skip linting on sed call

lineinfile won't necessarily work well for this.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #132 from ceph/wip-rhsm

ansible: Do not enable rhsm repos

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

ansible: Do not enable rhsm repos

We are shipping as a regular product and as such, we cannot enable
additional repos via rhsm. The customers might even not have these repos
installed (especially the storage console repo might not be available to
them).

If we need any of the packages from these repos, we need to cross-ship
them in our product (as we already do downstream).

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #129 from ceph/wip-remove-refs-to-prod-repo

ansible: Remove references to old prod repo

Merge pull request #125 from ceph/wip-collectd-vars

Fix undefined collectd vars

Merge pull request #128 from ceph/wip-prod-repos

Add support for custom repos

Merge pull request #126 from ceph/selinux-fix

selinux policy update

selinux: Update the policy

This policy update should cover all the newly found avc denials.

Signed-off-by: Boris Ranto <branto@redhat.com>

ansible: Remove references to old prod repo

This fixes the build failure after the patch that removed the original
repo file.

Signed-off-by: Boris Ranto <branto@redhat.com>

Add support for custom repos

This is to enable testing packages built from non-master branches with
devel_mode=False

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #124 from ceph/wip-remove-prod-repo

ansible: Remove old production repo

Fix undefined collectd vars

We regressed by moving the definition of a couple of these vars. Set
them in a separate task file to avoid this in the future.

Signed-off-by: Zack Cerza <zack@redhat.com>

ansible: Remove old production repo

We no longer use download.ceph.com as a production repo. We should
be removing instead of installing it.

Signed-off-by: Boris Ranto <branto@redhat.com>