Merge branch 'master' into rm-ooo

Merge pull request #1200 from ceph/syntax-check

ceph-ansible-pr-syntax-check: always fetch origin

ceph-ansible-pr-syntax-check: always fetch origin

We need to fetch origin to always ensure we have the right version.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1199 from ceph/guits_stable3.2

ceph-ansible-pipeline: update pipeline matrix

ceph-ansible-pipeline: update pipeline matrix

We must test corresponding ceph release for each stable branch of
ceph-ansible.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1197 from ceph/container_podman

add new container_podman scenario to ceph-ansible

add new container_podman scenario to ceph-ansible

This scenario allows us testing podman instead of docker and also python
3 for ou rmodules.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1196 from ceph/wip-nfs-ganesha-stable-build-fix

nfs-ganesha: fix debian stable build

Merge branch 'master' into wip-nfs-ganesha-stable-build-fix

Merge pull request #1195 from callithea/wip-ceph-dashboard-pull-requests-switch-node

ceph-dashboard-pull-requests: switch node type from trusty to xenial

Reviewed-by: Kefu Chai <kchai@redhat.com>

nfs-ganesha: fix debian stable build

various fixes to nfs-ganesha-stable deb building
update stable config defaults
add removal of old ceph packages from pbuilder
to normal nfs-ganesha debian job

Signed-off-by: Ali Maredia <amaredia@redhat.com>

ceph-dashboard-pull-requests: switch node type from trusty to xenial

Since there are currently issues with the dashboard job and its dependencies (python-jwt)
when the test node is based on trusty (and Nautilus has dropped the support for trusty anyway)
it makes sense to switch to a more recent Ubuntu version

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1193 from ceph/diff-git

ceph-ansible syntax check

ceph-ansible syntax check

Only check for Modified and Changed content, anything else is irrelevant
especially remove, rename and will make the check fail.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1190 from callithea/wip-ceph-dashboard-pull-requests-jwt-dependency

ceph-dashboard-pull-requests: add new python-jwt dependency

Signed-off-by: Laura Paduano <lpaduano@suse.com>

ceph-dashboard-pull-requests: add new python-jwt dependency

Since the JWT PR (#22833) has been merged  a new dependency
has been introduced which is now missing for the dashboard Jenkins job
Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1192 from ceph/wip-ceph-iscsi

ceph-iscsi: new PR tox and branch trigger builders

ceph-iscsi: new PR tox and branch trigger builders

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1191 from ceph/wip-ceph-iscsi

ceph-iscsi: initial version of combined iSCSI tools repo

ceph-iscsi: initial version of combined iSCSI tools repo

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1189 from callithea/wip-ceph-dashboard-pull-requests-label

Changed config of 'white-list-label'

Reviewed-by: Jos Collin <jcollin@redhat.com>

Changed config of 'white-list-label'

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1188 from callithea/wip-ceph-dashboard-pull-requests-only-trigger-phrase

ceph-dashboard-pull-requests: re-add 'only-trigger-phrase'

Reviewed-by: Jos Collin <jcollin@redhat.com>

ceph-dashboard-pull-requests: re-add 'only-trigger-phrase'

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1187 from ceph/ceph-ansible-deps

ceph-volume ansible jobs need python-netaddr installed in the controlling node

ceph-volume ansible jobs need python-netaddr installed in the controlling node

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #1168 from callithea/wip-ceph-dashboard-pull-requests-trigger

ceph-dashboard-pull-requests: added 'white-list-labels'

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

ceph-dashboard-pull-requests: added 'white-list-labels'
and changed 'only-trigger-phrase' to be able to automatically
start this Job for all dashboard PRs

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1183 from ceph/ceph-ansible-master

ceph-ansible-pipeline: only test master

Merge branch 'master' into ceph-ansible-master

Merge branch 'master' into rm-ooo

Merge pull request #1186 from ceph/guits_ceph-ansible3.2

ceph-ansible: update nightlies job

Merge branch 'master' into ceph-ansible-master

ceph-ansible: update nightlies job

since recent updates and 3.2 is out, we should update the matrix of
testing for nightlies jobs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1178 from ceph/wip-tserlin-nfs-ganesha-stable

nfs-ganesha-stable: update build parameters

Merge branch 'master' into wip-tserlin-nfs-ganesha-stable

Merge pull request #1184 from ceph/wip-nfs-ganesha-build-fix

nfs-ganesha: various nfs-ganesha build fixes

Merge branch 'master' into wip-nfs-ganesha-build-fix

Merge pull request #1185 from ceph/ceph-medic-disable

ceph-medic-tests: disable all child jobs from template

ceph-medic-tests: disable all child jobs from template

Signed-off-by: Alfredo Deza <adeza@redhat.com>

nfs-ganesha: various nfs-ganesha build fixes

- fix nfs-ganesha build for debian
- add missing packages for stable rpm builds
- cleanup for rpm nfs-ganesha build scripts

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge branch 'master' into ceph-ansible-master

Merge pull request #1182 from ceph/rgw-multi-site

Rgw multi site

ceph-ansible-pipeline: only test master

We are now ready to test on master ceph so let's switch all the pipeline
jobs to do so.

Signed-off-by: Sébastien Han <seb@redhat.com>

ceph-ansible-pr: add rgw in devs

Add rgw_multisite_* in devs so we can test this with master upstream.

Signed-off-by: Sébastien Han <seb@redhat.com>

ceph-ansible-pipeline: add rgw multisite

support for testing rgw multisite in the pipeline

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1181 from ceph/ceph-volume-trigger

Ceph volume trigger

ceph-ansible-pipeline: activate batch container

Now batch container is supported so let's activate it.

Signed-off-by: Sébastien Han <seb@redhat.com>

ceph-ansible-pipeline: run osd scenarios when module is modified

When we edit the ceph_volume.py module we should run all the OSD
scenarios.

Signed-off-by: Sébastien Han <seb@redhat.com>

ceph-ansible-pipeline: do not test ooo

Since all the scenario use lvm now, the ooo scenario will always fail
when triggered from the pipeline. The job will remain in stable branches
though.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1179 from ceph/guits_rgw_multisite

ceph-ansible: add rgw_multisite scenario

Merge branch 'master' into guits_rgw_multisite

Merge pull request #1177 from ceph/pipeline

ceph-ansible-pipeline: run all lvm test against ceph@master

ceph-ansible: add rgw_multisite scenario

ceph-ansible corresponding PR:

ceph/ceph-ansible#1944

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

nfs-ganesha-stable: update build parameters

ceph-ansible-pipeline: run all lvm tests against ceph@master

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-ansible-prs: allow all jobs to be tested with dev repos

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1176 from ceph/guits-remove_color

ceph-ansible-syntax: disable syntax coloration in `git_diff_to_head()`

ceph-ansible-syntax: disable syntax coloration in `git_diff_to_head()`

when running this code locally for debugging purposes, the sed in
`match_file()` won't match the pattern because of the ANSI codes inserted by
the git diff (colored) output.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1175 from ceph/guits-revert_4a25fb2b

ceph-ansible-syntax: display matched files in group_vars_check

ceph-ansible-syntax: display matched files in group_vars_check

it can be useful to see which files have matched in `group_vars_check()`
function.
I'm currently facing an issue where the CI complains on that check while
it doesn't on an other environment when trying to reproduce the issue.

on the CI:

```
+ group_vars_check
++ match_file /defaults/main.yml
++ git_diff_to_head
++ wc -l
++ git diff origin/master..HEAD -- . ':(exclude)roles/*/meta/*'
++ sed -n 's|^+++.*\(/defaults/main.yml.*\)|\1|p'
+ nb=1
+ [[ 1 -eq 0 ]]
++ match_file group_vars/
++ wc -l
++ git_diff_to_head
++ git diff origin/master..HEAD -- . ':(exclude)roles/*/meta/*'
++ sed -n 's|^+++.*\(group_vars/.*\)|\1|p'
+ nb_group_vars=0
+ [[ 1 -gt 0 ]]
+ echo 'One or more files containing default variables has/have been modified.'
One or more files containing default variables has/have been modified.
+ echo 'You must run '\''generate_group_vars_sample.sh'\'' to generate the group_vars template files.'
You must run 'generate_group_vars_sample.sh' to generate the group_vars template files.
+ return 1
```

local env:

```
+ group_vars_check
++ match_file /defaults/main.yml
++ wc -l
++ git_diff_to_head
++ sed -n 's|^+++.*\(/defaults/main.yml.*\)|\1|p'
++ git diff origin/master..HEAD -- . ':(exclude)roles/*/meta/*'
+ nb=0
+ [[ 0 -eq 0 ]]
+ echo 'group_vars has not been touched.'
group_vars has not been touched.
+ return 0
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1144 from ceph/guits-remove_capital_check

ceph-ansible-syntax: remove capital letter check

Merge branch 'master' into guits-remove_capital_check

Merge pull request #1174 from ceph/jq-base

ceph-container-build-ceph-base-push-imgs: install jq

ceph-container-build-ceph-base-push-imgs: install jq

We need jq to read json.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1173 from ceph/base-manifest

ceph-container-build-ceph-base-push-imgs: try to build manifest on am…

ceph-container-build-ceph-base-push-imgs: try to build manifest on amd64 too

They are days where arm64 runs before amd64 and the other way around so
let's try to build the manifest also on amd64. This can't not do any
harm.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1172 from ceph/manifest-base

ceph-container-build-ceph-base-push-imgs-arm64: run manifest script

ceph-container-build-ceph-base-push-imgs-arm64: run manifest script

Run manifest script after the arm64 build is complete.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1170 from ceph/ceph-volume-batch-tests

ceph-volume-test: add 'lvm batch' tests

ceph-volume-test: add 'lvm batch' tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1169 from ceph/ceph-volume-smithi

ceph-volume: run all tests on smithi or mira nodes

ceph-volume: run all tests on smithi or mira nodes

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #1165 from tchaikov/ceph-dashboard-pull-requests

ceph-dashboard-pull-requests: install runtime dependencies of dashboard

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1167 from ceph/wip-ormira

ceph-volume-scenario: Also allow building on mira

ceph-volume-scenario: Also allow building on mira

I added a few miras as static libvirt slaves.  This builds on https://github.com/ceph/ceph-build/pull/1166

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #1166 from ceph/ceph-volume-smithi

ceph-volume-scenario: lock tests to smithi nodes

ceph-volume-scenario: lock tests to smithi nodes

We are currently having troubles running test on our OVH nodes. This
locks these tests to smithi nodes for now so we can do some testing for
12.2.9.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-dashboard-pull-requests: install runtime dependencies of dashboard

some of them are already fulfilled by the build time dependencies. and
since mgr is built with python2 runtime, so we don't need to install
python3 modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1141 from ceph/wip-nfs-ganesha-ntirpc-build-patch

nfs-ganesha: apply patch to build ntirpc without lttng

Merge branch 'master' into wip-nfs-ganesha-ntirpc-build-patch

Merge pull request #1164 from ceph/ceph-ansible_add_osds

ceph-ansible: add "add_osds" scenarios to pipeline

ceph-ansible: add "add_osds" scenarios to pipeline

adding `add_osds` and `add_osds_container` to the ceph-ansible pipeline

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

nfs-ganesha: apply patch to build ntirpc without lttng

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #1162 from ceph/guits-add_osds

ceph-ansible: add new scenario 'add_osds'

ceph-ansible: add new scenario 'add_osds'

Adding new scenarios for "day 2 operation" playbook

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1152 from callithea/wip-ceph-dashboard-prs-python3-pkgs

ceph-dashboard-pull-requests: install python3 packages

Reviewed-by: Jos Collin <jcollin@redhat.com>

ceph-dashboard-pull-requests: install python3 packages

Signed-off-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #1161 from ceph/pipeline-sod

ceph-ansible-pipeline: fix wrong condition

ceph-ansible-pipeline: fix wrong condition

if grep report 0 ceph-osd was modified thus we exit

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1160 from ceph/pipeline

ceph-ansible-pipeline osd followup

ceph-ansible-pipeline osd followup

Not sure what's going on, this is mostly a commit to debug

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1159 from ceph/ansible-scenario

ceph-ansible-ci: do not run osd twice

ceph-ansible-ci: do not run osd twice

Do not play twice osds if they have been ack by the first task of the
pipeline.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1143 from ceph/ceph-container-build-ceph-base-push-imgs-arm64

add ceph-container-build-ceph-base-push-imgs-arm64

add ceph-container-build-ceph-base-push-imgs-arm64

Same as ceph-container-build-ceph-base-push-imgs but to build arm64
images.

Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #1156 from ceph/guits-fix_collect_log

ceph-ansible: fix collect ceph config task

ceph-ansible: fix collect ceph config task

`ceph_conf_key_directory` doesn't exist in this playbook.
All our testing is done with '/etc/ceph', let's hardcode it here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #1155 from ceph/guits-syntax-check-ansible

ceph-ansible-pr: fix workspace in syntax check

ceph-ansible-pr: fix workspace in syntax check

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>