xfstests-dev.git
5 years agogitignore: add src/test-nextquota
Theodore Ts'o [Sun, 21 Feb 2016 21:37:45 +0000 (08:37 +1100)]
gitignore: add src/test-nextquota

Commit 1dfb50585c09 (quota: test Q_GETNEXTQUOTA) added a new binary
without updating .gitignore.  Fix this.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: btrfs/011 add debug logs to seqres.full
Anand Jain [Sun, 21 Feb 2016 21:37:15 +0000 (08:37 +1100)]
fstests: btrfs/011 add debug logs to seqres.full

Signed-off-by: Anand Jain <anand.jain@oracle.com>
5 years agobtrfs/011: use replace_options
Anand Jain [Sun, 21 Feb 2016 21:37:14 +0000 (08:37 +1100)]
btrfs/011: use replace_options

This patch fixes test btrfs/011 which intended to use -r option
but was never used since its associated args 'replace_options'
didn't make it to the cli.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric: test file fsync after rename operation
Filipe Manana [Fri, 19 Feb 2016 01:15:33 +0000 (12:15 +1100)]
generic: test file fsync after rename operation

Test that if we have a file F1 with two links, one in a directory A and
the other in directory B, if we remove the link in directory B, move some
other file F2 from directory B into directory C, fsync inode F1, power
fail and remount the filesystem, file F2 exists and is located only in
directory C.

This is motivated by a bug found in btrfs, which is fixed by the patch
(for the linux kernel) titled:

   "Btrfs: fix file loss on log replay after renaming a file and fsync"

Tested against ext3, ext4, xfs, f2fs and reiserfs.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric: test directory fsync after rename operation
Filipe Manana [Fri, 19 Feb 2016 01:15:17 +0000 (12:15 +1100)]
generic: test directory fsync after rename operation

Test that if we move one file between directories, fsync the parent
directory of the old directory, power fail and remount the filesystem,
the file is not lost and it's located at the destination directory.

This is motivated by a bug found in btrfs, which is fixed by the patch
(for the linux kernel) titled:

  "Btrfs: fix file loss on log replay after renaming a file and fsync"

Tested against ext3, ext4, xfs, f2fs and reiserfs.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon: fix _require_fs_space on remote file systems
Christoph Hellwig [Thu, 18 Feb 2016 23:51:47 +0000 (10:51 +1100)]
common: fix _require_fs_space on remote file systems

Remove the -l flag to df so that it works properly on NFS and co.

This fixes various failures in new COW tests on NFS.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs/24[356]: check for -c switch to xfs_io bmap command
Darrick J. Wong [Thu, 18 Feb 2016 23:51:25 +0000 (10:51 +1100)]
xfs/24[356]: check for -c switch to xfs_io bmap command

Extend _require_xfs_io_command so that we can pass it a command line
argument to look for, and then use new capabililty in the relevant
tests to ensure that bmap knows how to dump CoW fork contents (bmap -c)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agopunch-alternating: use the block size reported by the fs for punching
Darrick J. Wong [Thu, 18 Feb 2016 23:51:17 +0000 (10:51 +1100)]
punch-alternating: use the block size reported by the fs for punching

When we're trying to punch alternating blocks out of a file, use the
bsize reported by fstatfs so that we can punch out single blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/304: fix high offset
Darrick J. Wong [Thu, 18 Feb 2016 23:50:58 +0000 (10:50 +1100)]
generic/304: fix high offset

"Invalid argument" is a better response to an impossibly high offset
dedupe request than "extents don't match", so change the test.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agobtrfs: test directory fsync after deleting snapshots
Filipe Manana [Thu, 18 Feb 2016 23:50:32 +0000 (10:50 +1100)]
btrfs: test directory fsync after deleting snapshots

Test that if we fsync a directory that had a snapshot entry in it that
was deleted and crash, the next time we mount the filesystem, the log
replay procedure will not fail and the snapshot is not present anymore.

This issue is fixed by the following patch for the linux kernel:

  "Btrfs: fix unreplayable log after snapshot delete + parent dir fsync"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs: new test default user/group quota
Zorro Lang [Thu, 18 Feb 2016 23:49:54 +0000 (10:49 +1100)]
xfs: new test default user/group quota

When default quota is set, all different quota types inherits the
same default value, include group quota. So if a user quota limit
larger than the default user quota value, it will still be limited
by the group default quota value.

An upstream patch for this bug:

  xfs: Split default quota limits by quota type

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfstests: remove _need_to_be_root
Theodore Ts'o [Thu, 18 Feb 2016 23:49:17 +0000 (10:49 +1100)]
xfstests: remove _need_to_be_root

The check script requires that it be run as root, so adding
individualized checks for this in each teat is not needed.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfstests: increase tmpfs memory size
Hugh Dickins [Thu, 18 Feb 2016 23:48:48 +0000 (10:48 +1100)]
xfstests: increase tmpfs memory size

512M is not enough for generic/129.  Raise default tmpfs size to 1G.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfstests: add executable permission to tests
Hugh Dickins [Thu, 18 Feb 2016 23:48:38 +0000 (10:48 +1100)]
xfstests: add executable permission to tests

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/079,277: require chattr, not xattrs
Hugh Dickins [Thu, 18 Feb 2016 23:47:55 +0000 (10:47 +1100)]
generic/079,277: require chattr, not xattrs

Add a new helper, _require_chattr, which allows the test to explicitly
check to see if the file system supports a specific chattr flag, as
not all file systems support chattr +A or chattr +i, and the presence
of extended attribute support is has nothing to do with a specific
chattr flag being supported.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/009: require fiemap
Hugh Dickins [Thu, 18 Feb 2016 23:46:27 +0000 (10:46 +1100)]
generic/009: require fiemap

Require xfs_io commands fiemap and falloc as well as fzero: fzero
without falloc is unlikely, but tmpfs may later support fzero, though
probably never fiemap (and in v3.15 wrongly claimed to support fzero).

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/053: do not unmount before calling _check_scratch_fs()
Junho Ryu [Thu, 18 Feb 2016 23:45:55 +0000 (10:45 +1100)]
generic/053: do not unmount before calling _check_scratch_fs()

Fix generic/053 so it works on tmpfs by relying on _check_scratch_fs
to unmount before checking the file system and remounting it
afterwards.  Many other tests rely on this, and since tmpfs does not
have a file system consistency checker, this allows the test to
succeed because the files don't disappear when the tmpfs file system
is unmounted.

Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfstests: do not unmount tmpfs during remount
Junho Ryu [Thu, 18 Feb 2016 23:45:35 +0000 (10:45 +1100)]
xfstests: do not unmount tmpfs during remount

Several tests unmount then re-mount the scratch filesystem, to check
that the content is unchanged; but unmounting a tmpfs is designed to
lose its content, which causes such tests to fail unnecessarily.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoRename _test_mount to _test_cycle_mount
Theodore Ts'o [Thu, 18 Feb 2016 23:45:04 +0000 (10:45 +1100)]
Rename _test_mount to _test_cycle_mount

This makes it clear when we are using "mount ; umount" versus "mount
-o remount" for most file systems.  The reason for this distinction is
(a) tests may want to test the difference between what happens on the
remount versus the munt paths, (b) with tmpfs, "mount ; umount" will
cause the contents of all of the files to disappear which makes many
tests sad, and (c) some mount options may not be changed using "mount
-o remount".

Currently _test_mount performs "_test_mount ; _test_umount"
so mechnically rename this function to _test_cycle_mount.  This was
done mechnically using the script fragment:

git grep -E "_test_remount" | \
awk -F: '{print $1}' | sort -u | grep -v tests/xfs/189 \
xargs sed -i 's/_test_remount/_test_cycle_mount/g'

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoRename _scratch_mount to _scratch_cycle_mount
Theodore Ts'o [Thu, 18 Feb 2016 23:44:53 +0000 (10:44 +1100)]
Rename _scratch_mount to _scratch_cycle_mount

This makes it clear when we are using "mount ; umount" versus "mount
-o remount" for most file systems.  The reason for this distinction is
(a) tests may want to test the difference between what happens on the
remount versus the munt paths, (b) with tmpfs, "mount ; umount" will
cause the contents of all of the files to disappear which makes many
tests sad, and (c) some mount options may not be changed using "mount
-o remount".

Currently _scratch_mount performs "_scratch_mount ; _scratch_umount"
so mechnically rename this function to _scratch_cycle_mount.  This was
done mechnically using the script fragment:

git grep "_scratch_remount" | \
awk -F: '{print $1}' | sort -u | \
xargs sed -i 's/_scratch_remount/_scratch_cycle_mount/g'

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric: add _require_odirect to generic/113 and generic/214
Theodore Ts'o [Thu, 18 Feb 2016 23:44:45 +0000 (10:44 +1100)]
generic: add _require_odirect to generic/113 and generic/214

generic/113 and generic/214 both use O_DIRECT at some stage in their
tests, so check O_DIRECT support before running them.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric: use mount point instead of device name
Hugh Dickins [Thu, 18 Feb 2016 23:44:25 +0000 (10:44 +1100)]
generic: use mount point instead of device name

A tmpfs mount does not involve any block device, its $SCRATCH_DEV is
nothing but a place-holder, so apply 'df' or 'stat' to its mount point
$SCRATCH_MNT instead of to $SCRATCH_DEV.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon: _scratch_mkfs_sized() for tmpfs
Hugh Dickins [Thu, 18 Feb 2016 23:44:11 +0000 (10:44 +1100)]
common: _scratch_mkfs_sized() for tmpfs

Enable _scratch_mkfs_sized() for use with tmpfs, so that tests which
use this helper can now run.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocheck: avoid spurious complaints that tests/$FSTYP/group does not exist
Theodore Ts'o [Thu, 18 Feb 2016 23:43:42 +0000 (10:43 +1100)]
check: avoid spurious complaints that tests/$FSTYP/group does not exist

There are no tmpfs specific tests, so tests/tmpfs does not exist.
This commit avoids printing a spurious error message when running
specifying a group of tests (e.g., "check -g quick"):

  DEVICE: test:/tmp
  MK2FS OPTIONS:
  MOUNT OPTIONS: -o block_validity
  ./check: line 96: tests/tmpfs/group: No such file or directory
  FSTYP         -- tmpfs
  PLATFORM      -- Linux/i686 kvm-xfstests 4.5.0-rc2ext4-00002-g6df2762
  MKFS_OPTIONS  -- test:/scratch
  MOUNT_OPTIONS -- -o size=1G test:/scratch /test/scratch

  generic/001      [10:31:10][    5.811742] run fstests generic/001
      ...

Similar problems have been reported when testing nfs using xfstests.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoext4/001: add output variant for nodelalloc mounts
Theodore Ts'o [Thu, 18 Feb 2016 23:43:34 +0000 (10:43 +1100)]
ext4/001: add output variant for nodelalloc mounts

If the delayed allocation is disabled, we need a slightly different
output for the delayed allocation portion of the tests.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/158: test dedupe with destination offset past EOF
Filipe Manana [Thu, 18 Feb 2016 23:43:04 +0000 (10:43 +1100)]
generic/158: test dedupe with destination offset past EOF

We were testing when the source file offset starts at EOF or beyond,
but not when the destination offset is beyond EOF or when the
destination offset is smaller than EOF but destination offset plus
dedup length is greater than EOF.

This is motivated by a bug in btrfs' extent_same (dedup) ioctl where
we allowed the destination offset to start at EOF and beyond (and
destination offset + length beyond EOF) for the case where the source
and destination files are the same (was not allowed for different
files used as source and destination). This also made the file's
metadata inconsistent when the dedup operation succeeded, which
happened when the source range corresponded to a file hole, prealloc
extent or a data extent filled with zeroes.

The btrfs issue is fixed by the following patch for the linux kernel:

  "Btrfs: fix extent_same allowing destination offset beyond i_size"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
[darrick.wong@oracle.com: fix merge conflicts with latest reflink patchbomb]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoext4: test partial blocksize defrag integrity issue
Eryu Guan [Thu, 18 Feb 2016 23:42:40 +0000 (10:42 +1100)]
ext4: test partial blocksize defrag integrity issue

Calling EXT4_IOC_MOVE_EXTENT on file not aligned with block size and
block size is smaller than page size would cause integrity issue on the
partial-blocksize part when copying data between orign file and donor
file.

This ext4 kernel patch would fix it, titled
"ext4: don't read blocks from disk after extents being swapped in
move_extent_per_page())"

Though this bug only happens in the blocksize smaller than pagesize
case, there's no harm to test on various block size fs, so no block size
is specified in the test, it depends on the test configurations.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs: change return value check to golden image check
Zorro Lang [Thu, 18 Feb 2016 23:41:49 +0000 (10:41 +1100)]
xfs: change return value check to golden image check

xfs/133 and xfs/138 use too much code to do "return value" check,
it's not necessary. For the code can be more readable and clear,
I change "return value" check to golden image check.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/072: limit max cpu number to 8
Eryu Guan [Thu, 18 Feb 2016 23:40:56 +0000 (10:40 +1100)]
generic/072: limit max cpu number to 8

Right now generic/072 scales the loop count based on the cpu count. But
on hosts with many cpus(100+), generic/072 runs for hours and generates
very high system load.

Given that the original bug can be reproduced easily on unpatched
kernel, the great number of loops and long run time are not needed. So
limiting the cpu number to 8 (which gives around 20 seconds run time on
my test vm with 8 vcpus) seems reasonable.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoaiocp: fix the gcc warnings
Darrick J. Wong [Fri, 12 Feb 2016 17:37:29 +0000 (09:37 -0800)]
aiocp: fix the gcc warnings

Eliminate a debug printf that was causing warnings and fix the other two
debug printfs to avoid tripping on return value warnings.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: kick the slow tests out of the 'quick' group
Darrick J. Wong [Thu, 11 Feb 2016 23:23:04 +0000 (15:23 -0800)]
reflink: kick the slow tests out of the 'quick' group

Since 'quick' tests are supposed to run in < 15s, kick out the ones
that can't finish that soon even on fast storage.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: test source CoW across mixed block types with cowextsz set
Darrick J. Wong [Wed, 10 Feb 2016 17:20:45 +0000 (09:20 -0800)]
xfs: test source CoW across mixed block types with cowextsz set

Ensure that we can CoW the source file when the source file consists
of a range of mixed block types and there's a cowextsize hint set.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: ensure that we don't crash when freeing the ag reservations on a ro mount
Darrick J. Wong [Wed, 10 Feb 2016 07:09:15 +0000 (23:09 -0800)]
xfs: ensure that we don't crash when freeing the ag reservations on a ro mount

Christoph Hellwig discovered that the kernel crashed trying to free
the refcount btree per-ag reservation on a ro mount (because we don't
create the reservation except for rw mounts and ro->rw remounts).  So,
test this to make sure we never do that again. :)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test reflink+cow+enospc all at the same time
Darrick J. Wong [Fri, 5 Feb 2016 04:01:41 +0000 (20:01 -0800)]
reflink: test reflink+cow+enospc all at the same time

Set up an impossibly small filesystem and try to reflink and rewrite a
file on it to see what happens when we ENOSPC.  Basically
generic/16[67] but with a constrained fs size.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: test xfs_getbmapx behavior with shared extents
Darrick J. Wong [Wed, 3 Feb 2016 22:18:25 +0000 (14:18 -0800)]
xfs: test xfs_getbmapx behavior with shared extents

Make sure that xfs_getbmapx behaves properly w.r.t. shared extents
and CoW fork reporting.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: aio cow tests
Darrick J. Wong [Mon, 1 Feb 2016 23:04:24 +0000 (15:04 -0800)]
xfs: aio cow tests

Test what happens when AIO writes fail when we have a cowextsize hint
set on the files.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test aio copy on write
Darrick J. Wong [Tue, 2 Feb 2016 00:01:29 +0000 (16:01 -0800)]
reflink: test aio copy on write

Make sure that copy on write works with the AIO path.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: test rmapbt functionality
Darrick J. Wong [Fri, 29 Jan 2016 23:01:43 +0000 (15:01 -0800)]
xfs: test rmapbt functionality

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: test the automatic cowextsize extent garbage collector
Darrick J. Wong [Thu, 28 Jan 2016 23:20:25 +0000 (15:20 -0800)]
xfs: test the automatic cowextsize extent garbage collector

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test CoW across a mixed range of block types with cowextsize set
Darrick J. Wong [Thu, 28 Jan 2016 21:41:24 +0000 (13:41 -0800)]
reflink: test CoW across a mixed range of block types with cowextsize set

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test quota accounting
Darrick J. Wong [Wed, 27 Jan 2016 20:53:02 +0000 (12:53 -0800)]
reflink: test quota accounting

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test xfs cow behavior when the filesystem crashes
Darrick J. Wong [Sat, 23 Jan 2016 00:55:30 +0000 (16:55 -0800)]
reflink: test xfs cow behavior when the filesystem crashes

Use the extent size hint to force leftover CoW reservations then
crash the filesystem to see how recovery works.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: high offset reflink and dedupe tests
Darrick J. Wong [Wed, 20 Jan 2016 23:29:07 +0000 (15:29 -0800)]
reflink: high offset reflink and dedupe tests

Ensure that we can pass absurdly enormous offsets and lengths to
reflink/dedupe and it'll survive.

v2: Ask for dedupe in the dedupe test.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[hch@lst.de: call _require_test_dedupe]
Signed-off-by: Christoph Hellwig <hch@lst.de>
5 years agoxfs: test fragmentation characteristics of copy-on-write
Darrick J. Wong [Thu, 14 Jan 2016 04:33:22 +0000 (20:33 -0800)]
xfs: test fragmentation characteristics of copy-on-write

Perform copy-on-writes at random offsets to stress the CoW allocation
system.  Assess the effectiveness of the extent size hint at
combatting fragmentation via unshare, a rewrite, and no-op after the
random writes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs/122: support refcount/rmap data structures
Darrick J. Wong [Fri, 15 Jan 2016 19:54:22 +0000 (11:54 -0800)]
xfs/122: support refcount/rmap data structures

Include the refcount and rmap structures in the golden output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfs: more reflink tests
Darrick J. Wong [Wed, 13 Jan 2016 19:56:18 +0000 (11:56 -0800)]
xfs: more reflink tests

Create a couple of XFS-specific tests -- one to check that growing
and shrinking the refcount btree works and a second one to check
what happens when we hit maximum refcount.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: ensure that we can handle reflinking a lot of extents
Darrick J. Wong [Thu, 4 Feb 2016 17:26:03 +0000 (09:26 -0800)]
reflink: ensure that we can handle reflinking a lot of extents

Update the existing stress tests to ensure that we can handle
reflinking the same block a million times, and that we can handle
reflinking million different extents.  Add a couple of tests to ensure
that we can ^C and SIGKILL our way out of long-running reflinks.

v2: Don't run the signal tests on NFS, as we cannot interrupt NFS
clone operations.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[hch@lst.de: don't run on NFS]
Signed-off-by: Christoph Hellwig <hch@lst.de>
5 years agoreflink: test CoW operations against the source file
Darrick J. Wong [Wed, 3 Feb 2016 20:59:27 +0000 (12:59 -0800)]
reflink: test CoW operations against the source file

Ensure that CoW operations against shared blocks in the source file
work correctly.

v2: remove filefrag dependencies

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test CoW behavior with IO errors
Darrick J. Wong [Wed, 6 Jan 2016 00:03:30 +0000 (16:03 -0800)]
reflink: test CoW behavior with IO errors

Test various scenarios (with dm-flakey) where we simulate write
failures during CoW, to see if the FS can get through it without
blowing up or corrupting data.  Plumb in a FS-generic method to
sort out repairing filesystems after they get hit by IO errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: test intersecting CoW and falloc/fpunch/fzero/fcollapse/finsert/ftrunc
Darrick J. Wong [Tue, 5 Jan 2016 18:06:37 +0000 (10:06 -0800)]
reflink: test intersecting CoW and falloc/fpunch/fzero/fcollapse/finsert/ftrunc

Ensure that we correctly handle a CoW operation immediately followed
by a truncate, falloc, fpunch, fzero, fcollapse, and finsert operation
in the middle of the CoW'd region before any flush can occur.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agodio: unwritten conversion bug tests
Darrick J. Wong [Tue, 2 Feb 2016 01:50:31 +0000 (17:50 -0800)]
dio: unwritten conversion bug tests

Check that we don't expose old disk contents when a directio write to
an unwritten extent fails due to IO errors.  This primarily affects
XFS and ext4.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: add _require_odirect to the directio tests
Darrick J. Wong [Thu, 11 Feb 2016 17:45:25 +0000 (09:45 -0800)]
reflink: add _require_odirect to the directio tests

For the tests that test O_DIRECT, we need to _require_odirect.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: fix style problems in existing tests
Darrick J. Wong [Wed, 10 Feb 2016 08:33:10 +0000 (00:33 -0800)]
reflink: fix style problems in existing tests

Fix style problems such as unnecessary use of quotes, add helper
variables to reduce visual clutter, and other minor fixes to make the
first batch of tests more closely resemble the second round tests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: refactor mixed block creation code
Darrick J. Wong [Wed, 10 Feb 2016 07:47:22 +0000 (23:47 -0800)]
reflink: refactor mixed block creation code

Refactor the code that creates files with mixed block types that we feed
into CoW tests to make sure that we can tiptoe around that kind of stuff.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agocommon: provide a method to repair the scratch fs
Darrick J. Wong [Tue, 9 Feb 2016 21:59:13 +0000 (13:59 -0800)]
common: provide a method to repair the scratch fs

Create a wrapper function that repairs any damage to the scratch
filesystem and returns a standard result.  We will use this to clean
up after IO error testing and other weird corruption tests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agocommon: create _require_test_program to look for programs in src/
Darrick J. Wong [Tue, 9 Feb 2016 08:22:21 +0000 (00:22 -0800)]
common: create _require_test_program to look for programs in src/

Create a helper that looks for a test program in src/ and fails the
test if it doesn't exist.  Refactor the existing testcases to use it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agocommon/dmerror: add some more dmerror routines
Darrick J. Wong [Tue, 12 Jan 2016 02:04:49 +0000 (18:04 -0800)]
common/dmerror: add some more dmerror routines

Add functions to the dmerror routine so that we can load both the
error table and the linear table.  This will help us with EIO testing
of copy-on-write.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: remove unnecessary umounts from test code
Darrick J. Wong [Wed, 10 Feb 2016 00:44:06 +0000 (16:44 -0800)]
reflink: remove unnecessary umounts from test code

The test harness already takes care of this, so get rid of it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoreflink: remove redundant filesystem checks from the end of the tests
Darrick J. Wong [Tue, 2 Feb 2016 01:57:11 +0000 (17:57 -0800)]
reflink: remove redundant filesystem checks from the end of the tests

Turns out that check already runs _check_filesystems after each test,
so we don't need to do this at the end of each test.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfstests: make _scratch_mkfs_blocksized usable
Darrick J. Wong [Wed, 3 Feb 2016 21:14:01 +0000 (13:14 -0800)]
xfstests: make _scratch_mkfs_blocksized usable

The default mkfs.xfs options contain -b size=4096, so all tests
using _scratch_mkfs_blocksized won't actually run unless those
options are changed.  As we're trying to specificly test 1k
blocks we should always override the default option.

v2: Move the function to common/rc

Signed-off-by: Christoph Hellwig <hch@lst.de>
[darrick.wong@oracle.com: move function to common/rc]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agoxfstests: filter whitespace in 128 and 132
Darrick J. Wong [Wed, 3 Feb 2016 20:59:48 +0000 (12:59 -0800)]
xfstests: filter whitespace in 128 and 132

Seems either I have a different lsattr version, or different mount points
cause differences in the golden output.  Send the lsattr output through
the whitespaces filter so that it works everywhere.

The lsattr output /does/ change depending on mountpoints.  Ick.  I'd
actually changed it to the long format output because line length in
the short format changes every time the flags change.

Signed-off-by: Christoph Hellwig <hch@lst.de>
[darrick.wong@oracle.com: update changelog]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
5 years agogeneric/182: this is a dedupe test, check for dedupe
Darrick J. Wong [Thu, 28 Jan 2016 19:26:13 +0000 (11:26 -0800)]
generic/182: this is a dedupe test, check for dedupe

Since this test examines dedupe behavior, the documentation should
say 'dedupe', not 'reflink'.  Furthermore, the feature checks must
look for working dedupe functionality, not reflink functionality.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[hch@lst.de: add the test for dedupe support]
Signed-off-by: Christoph Hellwig <hch@lst.de>
5 years agoFix prerequisite packages to build fstests on Ubuntu
Satoru Takeuchi [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
Fix prerequisite packages to build fstests on Ubuntu

To build xfstests from minimal Ubuntu, it's necessary to install
some more packages in addition to ones mentioned in README.
Plus, libtool is not needed to specify here since libtool-bin
depends on it.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoaio-dio-eof-race should be ignored by git
Satoru Takeuchi [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
aio-dio-eof-race should be ignored by git

src/aio-dio-regress/aio-dio-eof-race is a binary file built by make.
So it should not be tracked by git.

===============================================================================
$ make clean
...
$ git status
On branch sat-bugfixes
nothing to commit, working directory clean
$ make
...
$ git status
On branch sat-bugfixes
Untracked files:
  (use "git add <file>..." to include in what will be committed)

src/aio-dio-regress/aio-dio-eof-race
===============================================================================

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: btrfs, test for send with clone operations
Filipe Manana [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
fstests: btrfs, test for send with clone operations

Test that an incremental send operation which issues clone operations
works for files that have a full path containing more than one parent
directory component.

This used to fail before the following patch for the linux kernel:

  "[PATCH] Btrfs: send, fix extent buffer tree lock assertion failure"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoreflink: change to relative margins
Christoph Hellwig [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
reflink: change to relative margins

NFS uses 32K blocks and thus the numbers used for sanity checking
various reflink operations are way off.  Change to relative margings
and increase them a bit to make NFS happy.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoreflink: also treat EINVAL as not supported
Christoph Hellwig [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
reflink: also treat EINVAL as not supported

This can be returned by various implementations of the ioctl.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs/139: fix scratch device filtering
Christoph Hellwig [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
xfs/139: fix scratch device filtering

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agotests: use lowercase variables for all the new reflink tests
Darrick J. Wong [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
tests: use lowercase variables for all the new reflink tests

Since $TESTDIR is a local variable, make it lowercase to avoid
confusion with $TEST_DIR.  While we're at it, make all the local
variables lowercase.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agodedupe: record dedupe errors in golden output
Darrick J. Wong [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
dedupe: record dedupe errors in golden output

The xfs_io dedupe command originally didn't print dedupe errors
to stderr like you'd expect.  Since that was fixed, the golden
output should be changed to reflect that.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs/122: update against xfsprogs 4.3
Darrick J. Wong [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
xfs/122: update against xfsprogs 4.3

Make the xfsprogs sb layout and structure size checker work with
modern xfsprogs (i.e. 4.3+ and all the new v5 stuff).  The old test
wasn't picking up structures because they're no longer defined in the
"typedef struct xfs_foo { } xfs_foo_t;" style.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoreflink: fix off-by-one errors when iterating file blocks in a loop
Darrick J. Wong [Sun, 7 Feb 2016 22:27:15 +0000 (09:27 +1100)]
reflink: fix off-by-one errors when iterating file blocks in a loop

When we're iterating file blocks in a loop (via seq), we have to
end at $nr-1, not $nr.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agotools/mvtest: call out to the correct group sort program
Darrick J. Wong [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
tools/mvtest: call out to the correct group sort program

mvtest should call the actual name of the sort-group program.
Fix a cut-paste error in finding the destination group file.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoquota: test Q_GETNEXTQUOTA
Eric Sandeen [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
quota: test Q_GETNEXTQUOTA

The new Q_GETNEXTQUOTA quotactl (not yet merged) is designed
to take an ID as input ala Q_GETQUOTA, and return the quota
for the next active ID >= the input ID.  This lets us quickly
iterate over all existing quotas by leveraging the kernel's
knowledge of which quotas are allocated and active.

The test contains a new helper binary, test-nextquota, which
tests both the "vfs" and "xfs" versions of the quotactl.
It accepts an ID, and outputs the returned ID, ihard, and
isoft values for that quota.  It doesn't return block information
simply because that can vary depending on fs, block size, etc,
and we want something very consistent as output, for verifiation.

The test harness sets quotas for 100 random IDs, remounts,
and uses these quotactls to iterate over all the IDs we set,
using the test binary, making sure we get back what we expect.

Not the prettiest thing, but it works!

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofsx.c: Add header file stdbool.h
Xiao Yang [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
fsx.c: Add header file stdbool.h

Compilation failed on RHEL6.7GA because stdbool.h is not included in
xfs/platform_defs-x86_64.h or xfs/linux.h on RHEL6.7GA, so fix it.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agooverlay: test copy up files from lower dir
Eryu Guan [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
overlay: test copy up files from lower dir

Test copy up files from lower dir that are <, =, > 4G in size.

Kernel commit 0480334fa604 ("ovl: use O_LARGEFILE in ovl_copy_up()")
fixed bug when copying up large file(>4G) from lower dir.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon: export overlay lower/upper/work directories as variables
Eryu Guan [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
common: export overlay lower/upper/work directories as variables

This turns hard coded lower/upper/work dir names to configurable
variables.

Suggested-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon: add overlayfs isupport in _scratch_cleanup_files()
Eryu Guan [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
common: add overlayfs isupport in _scratch_cleanup_files()

All files in lower/upper dirs should be removed for overlayfs in
_scratch_cleanup_files(), not only files in merged dir, otherwise files
from lower dir won't be removed.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agobtrfs: fix test failures after commit 27d077ec0bda
Filipe Manana [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
btrfs: fix test failures after commit 27d077ec0bda

Commit 27d077ec0bda (common: use mount/umount helpers everywhere) made
a few btrfs tests fail (btrfs/003 and btrfs/011). These tests create
filesystems with multiple devices and test the device replace feature,
which need to unmount using the mount path ($SCRATCH_MNT) because
unmounting using one of the devices as an argument ($SCRATCH_DEV) does
not always work - after replace operations we get in /proc/mounts a
device other than $SCRATCH_DEV associated with the mount point
$SCRATCH_MNT (this is mentioned in a comment at btrfs/011 for example),
so we need to pass that other device to the umount program or pass it the
mount point.
Fix this by making _sctatch_unmount() pass $SCRATCH_MNT to umount instead
of $SCRATCH_DEV (when the filesystem being tested is btrfs).

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agonew: create a default .out file
Eryu Guan [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
new: create a default .out file

Creating a default .out file in "new" to ease the new test creation
process.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon: no need to parse config file in new
Eryu Guan [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
common: no need to parse config file in new

I saw "new" failed the TEST_DEV/SCRATCH_DEV validation when TEST_DEV or
SCRATCH_DEV points to directory, which is needed in overlayfs testing.

"new" only generates test template and doesn't need common/config to
parse and validate TEST_DEV/SCRATCH_DEV etc., so just returns after
setting all needed commands.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agogeneric/23[23]: some cleanups
Eryu Guan [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
generic/23[23]: some cleanups

1. remove $seqres.full before test
2. no need to check return values of _fsstress and _check_quota_usage
   calls, the golden image could catch the failures.
3. remove unused FSX related variables

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon: add names device mounted errors
Jia He [Sun, 7 Feb 2016 22:27:14 +0000 (09:27 +1100)]
common: add names device mounted errors

This adds user friendly prompts to output the already mounted point
from _mount.  xfstests will do the cleanup (ie. umount) and user can
not get the mount name information when directory is already
mounted.

Signed-off-by: Jia He <hejianet@gmail.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofsync-tester: reopen files with correct flags
Dave Chinner [Sun, 7 Feb 2016 22:27:10 +0000 (09:27 +1100)]
fsync-tester: reopen files with correct flags

When "drop caches" tests are run, the test_fd is closed and
re-opened. It is reopened O_RDWR, so dropping O_DIRECT and any other
open flag that was set for test_fd. This fixes a random generic/311
failure due to file data corruption I've been seeing on subtests 11
and 13 when direct IO was specified.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocheck: add exclude sections CLI parameter
Dave Chinner [Sun, 7 Feb 2016 22:27:06 +0000 (09:27 +1100)]
check: add exclude sections CLI parameter

When we have a config file with multiple sections, we might want to
exclude certain config sections from running. Rather than specifying
all the section we want to run, add a "-S <section>" option to build
up a list of sections to exclude.

This is useful if a given section config is known to cause a fatal
failure,but you still want to run all the other config sections.

Also add support to the setup program that emits the currently
configured setup for each section in the config file.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agosetup: add section support
Dave Chinner [Sun, 7 Feb 2016 22:27:05 +0000 (09:27 +1100)]
setup: add section support

Make the setup command section aware so that it is easy to test
whether the section config code is generating the correct
configurations or not.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs: support realtime/log device setup changes in config sections
Dave Chinner [Tue, 2 Feb 2016 04:54:43 +0000 (15:54 +1100)]
xfs: support realtime/log device setup changes in config sections

Currently changing the devices used by "USE_EXTERNAL" environmental
variable is not supported by the config section parsing. Add the
functionality so that we can use config sections to test external
device configs successfully.

This required tracking down a bug in _check_xfs_filesystem() which
was causing a log device to be passed to a test device without an
external log device. This was caused by an uninitialised variable in
the function. I also added full output file removals to the first
couple of generic tests that were failing, because that's where the
check failure output ends up in this case.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: canonicalize the mount points by removing the trailing "/"
Jia He [Mon, 11 Jan 2016 04:13:20 +0000 (15:13 +1100)]
fstests: canonicalize the mount points by removing the trailing "/"

removing all the trailing "/"s of mounting point name by readlink

Suggested-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Jia He <hejianet@gmail.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: btrfs, verify fitrim does not discard reserved device area
Filipe Manana [Mon, 11 Jan 2016 04:12:20 +0000 (15:12 +1100)]
fstests: btrfs, verify fitrim does not discard reserved device area

Verify that when a fitrim operation is made against a btrfs filesystem,
the ranges [0, 64Kb[ and [68Kb, 1Mb[ of the device are not discarded,
they remain with the content they had before the fitrim operation. These
regions of the device are reserved for a boot loader to use at its will.

In the 4.3 linux kernel we got a regression that allowed a fitrim
operation to discard these reserved ranges of the device, resulting in
the filesystem becoming unbootable after a fitrim. The issue is fixed
by the following patch (targeted for 4.5 and 4.3/4.4 stable releases):

 "Btrfs: fix fitrim discarding device area reserved for boot loader's use"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoxfs: test XFS torn log write detection
Brian Foster [Mon, 11 Jan 2016 04:11:20 +0000 (15:11 +1100)]
xfs: test XFS torn log write detection

XFS torn log write detection includes a mechanism to inject CRC errors
into log records at runtime and shutdown the fs accordingly. This
ensures that the CRC verification pass on the subsequent mount discovers
an invalid record near the head of the log and considers it a torn
write.

This test runs a workload with error injection enabled and verifies that
the subsequent mount is successful. The test repeats for several
iterations using a random frequency factor for the error event each
time.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agof2fs: add Makefile for f2fs
Xiao Yang [Mon, 11 Jan 2016 04:10:20 +0000 (15:10 +1100)]
f2fs: add Makefile for f2fs

Installation failed due to Makefile omission, so fix this.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agoshared: test mount needs_recovery extN partition with noload option
Xiong Zhou [Mon, 11 Jan 2016 04:09:20 +0000 (15:09 +1100)]
shared: test mount needs_recovery extN partition with noload option

Once upon a time, ext4 encounter NULL pointer dereference under this
situaiton due to jump to a wrong label. Part of this commit fixed
this Oops:

744692d ext4: use ext4_get_block_write in buffer write

We can also run this test on ext2/3.

This case is based on a script from Monakhov Dmitriy @ openvz.

Cc: Monakhov Dmitriy <dmonakhov@openvz.org>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agocommon/scsi_debug: fix removal of scsi_debug module
Eryu Guan [Mon, 11 Jan 2016 04:08:20 +0000 (15:08 +1100)]
common/scsi_debug: fix removal of scsi_debug module

Occasionally scsi_debug cannot be removed because it's still in use and
causes xfs/279 to fail.

Now dryrun the removal by modprobe firstly then do the real rmmod.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: fix cleanup of test btrfs/003
Filipe Manana [Mon, 11 Jan 2016 04:07:20 +0000 (15:07 +1100)]
fstests: fix cleanup of test btrfs/003

If the test fails after removing a device and before adding it back, it
attempts to add back the device in its _cleanup() function. However this
is broken because the device identifier is stored in a variable local to
the function _test_replace() and not in a global variable. So make the
variable global instead of local.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: cleanup test btrfs/031
Filipe Manana [Mon, 11 Jan 2016 04:06:20 +0000 (15:06 +1100)]
fstests: cleanup test btrfs/031

The test was using $SCRATCH_MNT as a mountpoint for $SCRATCH_DEV, which
is counter intuitive and not expected by the fstests framework - this
made the test fail after commit 27d077ec0bda (common: use mount/umount
helpers everywhere). So rewrite the test to use the scratch device for
all data and use a test specific directory inside $TEST_DIR to use as a
mount point for a cross mount of $SCRATCH_DEV.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofstests: cleanup test btrfs/029
Filipe Manana [Mon, 11 Jan 2016 04:05:20 +0000 (15:05 +1100)]
fstests: cleanup test btrfs/029

The test was using $SCRATCH_MNT as a mountpoint for $SCRATCH_DEV, which
is counter intuitive and not expected by the fstests framework - this
made the test fail after commit 27d077ec0bda (common: use mount/umount
helpers everywhere). So rewrite the test to use the scratch device for
all data and use a test specific directory inside $TEST_DIR to use as a
mount point for a cross mount of $SCRATCH_DEV.

This test was also overriding $seqres.full, through the redirect ">"
operator, if a call to cp failed. Fix that by using instead the operator
">>". Also make the test use the function _mount() instead of calling
the mount program directly.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofsx: Add mechanism to replay failed operations
Andreas Gruenbacher [Mon, 11 Jan 2016 03:45:26 +0000 (14:45 +1100)]
fsx: Add mechanism to replay failed operations

Create a $name.fsxops file next to $test.fsxlog.  When a test fails,
dump the operations in the log into that file in a simple, parseable
format like:

  fallocate 0x2e0f2 0xf04a 0x0 keep_size
  truncate 0x0 0x11e00 0x0 *
  write 0x73400 0x6c00 0x11e00
  skip punch_hole 0x71539913 0xdf76 0x7a000 close_open
  mapread 0x56000 0x16d08 0x7a000

Here, each operation is on a separate line. When the first word is
"skip", the operation will be skipped.  The next parameters are offset,
length, and the current file size, followed by optional flags like
keep_size and clode_open.  A trailing asterisk indicates that the
operation overlaps with the operation that has failed.

Add a --replay-ops option that allows to replay the operations recorded
in such a $name.fsxops file.  (The log can be modified to easily narrow
down which operations are causing the failure.)

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofsx: Improve operation logging
Andreas Gruenbacher [Mon, 11 Jan 2016 03:45:13 +0000 (14:45 +1100)]
fsx: Improve operation logging

Add FL_SKIPPED, FL_CLOSE_OPEN, and FL_KEEP_SIZE flags to the log
entries.  Use FL_SKIPPED to indicate that an operation was skipped.  Use
FL_CLOSE_OPEN to encode when an operation is followed by a close/open
cycle.  Use FL_KEEP_SIZE to indicate when the OP_ZERO_RANGE and
OP_FALLOCATE operations should keep the file size unchanged and put the
current file size into args[2] so that we can tell which operation was
actually called from the log.

After that, arg2 of log4 is always either unused or the current file size, so
remove it and unconditionally remember the current file size.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofsx: Generate test parameters in test()
Andreas Gruenbacher [Mon, 11 Jan 2016 03:44:51 +0000 (14:44 +1100)]
fsx: Generate test parameters in test()

Generate all test parameters in test(), including keep_size.

The code is slightly more complicated than it could be to produce the
same sequence of operations for the same random seed.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
5 years agofsx: Report number of successful operations
Andreas Gruenbacher [Mon, 11 Jan 2016 03:44:21 +0000 (14:44 +1100)]
fsx: Report number of successful operations

Report the number of successful operations at the end of an fsx run.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>