Merge pull request #1091 from athanatos/wip-16413

11429: wait_for_clean between restarting the osds and starting the bench

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1090 from athanatos/wip-16416

ec_lost_unfound: set min_size to 2

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1085 from ceph/wip_add_newest

Added option to search for newest good/built commit (7 commits back)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1080 from ajarr/wip-15615

tasks/cephfs/test_volume_client: test authentication metadata

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #1073 from ceph/wip-mds-snap-failover

fs: add snapshot tests to mds thrashing

Added option to search for newest good/built commit (7 commits back)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #1077 from trociny/wip-16386

rbd: tests with rbd_skip_partial_discard option enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: tests with rbd_skip_partial_discard option enabled

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: move copy_on_read overrides to config dir

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

fs: add snapshot tests to mds thrashing

Signed-off-by: Yan, Zheng <zyan@redhat.com>

tasks/cephfs/test_volume_client: test authentication metadata

Test the usecases for the authentication metadata stored
by the volume client:

* Obtain the list of auth IDs having access to a volume.

* Restrict volume access to auth IDs of a single (OpenStack)
  tenant to enforce strong tenant isolation of volumes.

Signed-off-by: Ramana Raja <rraja@redhat.com>

ec_lost_unfound: set min_size to 2

We changed the default to k+1 instead of k.  Adjust test to compensate.

Fixes: http://tracker.ceph.com/issues/16416
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #1072 from trociny/wip-16539

rbd-mirror: remove ceph_test_rbd_mirror_image_replay test case

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1064 from jcsp/wip-vstart-fixes

tasks/cephfs: vstart_runner fixes

Reviewed-by: Ramana Raja <rraja@redhat.com>

vstart_runner: fix misleading log message

Message is logged as, filesystem is mounted,
despite the vstart_runner just trying to mount
at this stage.

Signed-off-by: Ramana Raja <rraja@redhat.com>

tasks/cephfs: rename 'long_running' decorator

Rename 'long_running' decorator as 'for_teuthology',
which is more intuitive.

Signed-off-by: Ramana Raja <rraja@redhat.com>

tasks/cephfs: use "blacklist clear"

... instead of iterating over all blacklist entries.  Fall
back to the old way if the new way doesn't work (i.e. on
old ceph)

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix race while mounting

This could fail if the mount hadn't finished
coming up.

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix vstart_runner with cmake

This needed updating for the CLI living in bin/ now

Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: disable setuid test in vstart environ

``long_running`` needs a better name, it's really just a
marker that says a test is for use in teuthology but not vstart.

Signed-off-by: John Spray <john.spray@redhat.com>

rbd-mirror: remove ceph_test_rbd_mirror_image_replay test case

Fixes: http://tracker.ceph.com/issues/16539
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #1008 from ceph/wip-systemd-tests

Wip systemd tests

Merge pull request #1069 from batrick/i16164

cephfs: test fragment size limit

Reviewed-by: John Spray <john.spray@redhat.com>

Run rados loadgen

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
Conflicts:
suites/smoke/systemd/tasks/systemd.yaml

Few comment fixes

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

cephfs: test fragment size limit

The fragment configuration uses 10000 for the fragment max size. The reason for
this is that many tests add 1000 files to a single directory which will hit
this limit without fragmentation catching up.

The test_dirfrag_limit test confirms:

o That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations).
o That fragmentation (forced) will allow more entries to be created.
o That unlink fails when the stray directory fragment becomes too large and that unlinking may continue once those strays are purged.

Tests: https://github.com/ceph/ceph/pull/9789
Issue: http://tracker.ceph.com/issues/16164

Signed-off-by: Patrick Donnelly <batrick@batbytes.com>

Merge pull request #1068 from smithfarm/wip-drop-ceph-devel

packages.yaml: drop ceph-devel

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

packages.yaml: drop ceph-devel

There are no "-dev" packages listed in the deb section of this file, so it
serves no purpose to have ceph-devel here.

Also, once https://github.com/ceph/ceph/pull/9744 is merged the presence of
"ceph-devel" in this file will break teuthology for rpm targets.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #1020 from ceph/wip-15563-master

Added file locking logic to avoid possible collisions …

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #1065 from ceph/wip-ec-min-size

rados/thrash/workloads/cache-agent-big: set min_size=2 on m=1 k=2 ec pool

rados/thrash/workloads/cache-agent-big: set min_size=2 on m=1 k=2 ec pool

Otherwise min_size will be 3 and we will wedge on a single down
osd.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #1063 from ceph/wip-no-fuse

Revert "tasks/scrub_test: use fuse path (do not assume filestore!)"

Merge pull request #1059 from ceph/wip-s3roundtrip-wusui

Allow force-branch for s3roundtrip tests.

Revert "tasks/scrub_test: use fuse path (do not assume filestore!)"

This reverts commit 9e2b455f45cca2f31b19c73d761661a9028c22ff.

Update cron_wrapper

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #970 from ceph/wip-fix-cd-override

Fix for 'branch' overrides to work

Merge pull request #1058 from ceph/wip-15016

tasks/ceph-deploy: Run on multiple Python versions (2, 3)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #1002 from athanatos/wip-osd-recovery

osd_recovery: add delay to ensure maps propogate

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #1062 from athanatos/wip-resolve-peering

resolve_stuck_peering: reset stats while looping

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #1061 from athanatos/wip-short-log

short_pg_log: double params to avoid failures from dup detection failure

http://tracker.ceph.com/issues/16331

Merge pull request #1060 from athanatos/wip-flush-pg-stats

[ec_]lost_unfound: don't flush_pg_stats at the beginning

Reviewed-by: Sage Weil <sage@redhat.com>

11429: wait_for_clean between restarting the osds and starting the bench

ceph.restart now marks the osds down, so the objects are actually being
created while slowest of the osds boots. That causes a ton of 1 byte
objects to be created in a degraded state and causes the cleanup to take
a long time.  Also, reduce length of bench since it's only being used
to ensure the osds came up correctly.

Signed-off-by: Samuel Just <sjust@redhat.com>

tasks/ceph-deploy: Install appropriate Python version before running

Related to RM-15016

Signed-off-by: Oleh Prypin <oleh@pryp.in>

Allow force-branch for s3roundtrip tests.

Fixes tracker bug 15453
Signed-off-by: Warren Usui <wusui@redhat.com>

Merge pull request #1050 from ceph/wip-fix-scrub-test

test: Eliminate racey scrubbing in scrub_test.py

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #1057 from ceph/wip-scrub

tasks/scrub_test: use fuse path (do not assume filestore!)

Reviewed-by: Samuel Just <sjust@redhat.com>

tasks/scrub_test: use fuse path (do not assume filestore!)

Signed-off-by: Sage Weil <sage@redhat.com>

rados/objectstore/objectstore.yaml: skip bluestore tests

Avoid breaking rados suite while we are developing rapidly.

Signed-off-by: Sage Weil <sage@redhat.com>

Add sleep after stop to avoid checking deactivating state

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

[ec_]lost_unfound: don't flush_pg_stats at the beginning

The upgrade tests restart the daemons right before that part, and the
restart marks the osds down causing the flush_pg_stats to fail.  It's
not necessary anymore anyway.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #1049 from ceph/wip-rbd-nbd

rbd: added rbd-nbd fsx test case

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1053 from ceph/wip-thrash-raise

task/mds_thrash: don't drop exceptions from thrasher

test: Eliminate racey scrubbing in scrub_test.py

Enhance get_pgid() to return passed value if it is already a pgid

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #1036 from david-z/master

Adjust wait_for_health to be more flexible

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #1055 from ceph/wip-fix-core-master

16334 and 16332 fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Fixes http://tracker.ceph.com/issues/16334
Fixes http://tracker.ceph.com/issues/16332
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

tasks/ceph-deploy: Run on multiple Python versions (2, 3)

Related to RM-15016

Signed-off-by: Oleh Prypin <oleh@pryp.in>

Merge pull request #1054 from ceph/wip-16276

suites/fs: use simple messenger some places

suites/fs: use simple messenger some places

Two tests are sensitive to the switch to async msgr
http://tracker.ceph.com/issues/16288
http://tracker.ceph.com/issues/16276

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1051 from gregsfortytwo/wip-fix-mdsthrash

reference the kill_mds_by_rank function in the right location

Reviewed-by: John Spray <john.spray@redhat.com>

tasks/mds_thrash: fix more references to self.manager

Signed-off-by: John Spray <john.spray@redhat.com>

task/mds_thrash: don't drop exceptions from thrasher

Previously, when there was an exception in the thrasher
main loop, it would log it but ultimately "succeed"
the teuthology task.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1046 from batrick/wip-readahead

cephfs: test readahead is working

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

reference the kill_mds_by_rank function in the right location

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

rbd: added rbd-nbd fsx test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #924 from ceph/wip-mountbyfsname

tasks/cephfs: update tests for mount-by-fs-name

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #1018 from ceph/wip-16022

tasks/cephfs: reproducer for #16022

Add xenial for systemd test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

cephfs: test readahead is working

Test case for: https://github.com/ceph/ceph/pull/9374
Issue: http://tracker.ceph.com/issues/16024

Signed-off-by: Patrick Donnelly <batrick@batbytes.com>

Merge pull request #1040 from ceph/wip-libcephfs-posixacl

suites/fs/permission: run qa/workunits/fs/misc/{acl.sh,chmod.sh}

Reviewed-by: John Spray <john.spray@redhat.com>

Merge remote-tracking branch 'upstream/pull/1045/head'

* upstream/pull/1045/head:
  tasks/cephfs: fix vstart runner _get_pid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

tasks/cephfs: fix vstart runner _get_pid

Dropping the 'x' caused the ps lines not to include
the "-i <myid>" part of the command line that we use
to find named daemons' pids.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #1043 from batrick/ps-uid

vstart_runner: only kill/look at user's processes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #1044 from batrick/info-placement

vstart_runner: move info message for loaded mods

Reviewed-by: John Spray <john.spray@redhat.com>

vstart_runner: move info message for loaded mods

The message was printing loaded modules before loading each module.

vstart_runner: only kill/look at user's processes

This prevents vstart_runner.py from trying to kill other user's
ceph-mds/ceph-fuse instances.

suites/fs/permission: run qa/workunits/fs/misc/{acl.sh,chmod.sh}

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Added file locking logic to avoid possible collisions when appending to the log file

Fixes http://tracker.ceph.com/issues/15563
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #1038 from ceph/wip-libcephfs-posixacl

suites/fs: fix config for enabling libcephfs posix ACL

Merge pull request #1039 from ceph/wip-16113

suites/rados: add test for 16113

suites/rados: add test for 16113

Signed-off-by: Samuel Just <sjust@redhat.com>

suites/fs: fix config for enabling libcephfs posix ACL

Signed-off-by: Yan, Zheng <zyan@redhat.com>

Adjust wait_for_health to be more flexsible

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

Merge pull request #1031 from ceph/wip-sleep

tasks: fix non-existent sleep function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Initial systemd test files

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Systemd script to test ceph init files

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

tasks: fix non-existent sleep function

CephManager has no sleep function. Use time.sleep() instead.

Ran into this while testing a branch. Apparently it doesn't happen
much since this hasn't changed in years, but the error was copied
into several tasks.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #1028 from dachary/wip-rbd

rbd: provision volumes to format as XFS

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #1014 from ceph/wip-ceph-restart-wusui

tasks/ceph.restart osd fix

Reviewed-by: Samuel Just <sjust@redhat.com>

rbd: provision volumes to format as XFS

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #1026 from ceph/wip-16064

scrub_test: Shards are not marked in error when osd guesses

Reviewed-by: Samuel Just <sjust@redhat.com>

scrub_test: Shards are not marked in error when osd guesses

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #1017 from dachary/wip-buildpackages

openstack: ovh renamed flavors : s/eg/hg/

Reviewed-by: Nathan Cutler <ncutler@suse.com>

openstack: ovh renamed flavors : s/eg/hg/

http://travaux.ovh.net/?do=details&id=17532

Signed-off-by: Loic Dachary <loic@dachary.org>

tasks/cephfs: reproducer for #16022

Even though check_subs was broken, the multiclient
test was still passing because it didn't test that
clients survived a failover in the multi-filesystem
case.

Fixes: http://tracker.ceph.com/issues/16022
Signed-off-by: John Spray <john.spray@redhat.com>

tasks/ceph.restart osd fix

ceph.restart should mark restarted osds down in order to avoid a
race condition with ceph_manager.wait_for_clean

Fixes: http://tracker.ceph.com/issues/15778
Signed-off-by: Warren Usui <wusui@redhat.com>

Merge pull request #1013 from ceph/wip-15900

tasks/cephfs: fix failure in test_mount_conn_close

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

short_pg_log: double params to avoid failures from dup detection failure

Signed-off-by: Samuel Just <sjust@redhat.com>

tasks/cephfs: find processes correctly

Don't mistake the daemon-helper line for the actual
daemon process.

Fixes: http://tracker.ceph.com/issues/15900
Signed-off-by: John Spray <john.spray@redhat.com>

tasks/cephfs: fix mount wait in test_mount_conn_close

Fixes: http://tracker.ceph.com/issues/15900
Signed-off-by: John Spray <john.spray@redhat.com>