xfstests-dev.git
7 years agofstests: allow overlay SCRATCH_DEV to be a base fs mount point
Amir Goldstein [Tue, 28 Feb 2017 12:18:32 +0000 (14:18 +0200)]
fstests: allow overlay SCRATCH_DEV to be a base fs mount point

When configure SCRATCH_DEV to a mount point (and not to a directory therein)
then user will get a false positive error in scratch tests:

 $SCRATCH_DEV=/mnt/base/scratch is mounted but not on $SCRATCH_MNT=/mnt/scratch
 Already mounted result:
 /dev/sda6 on /mnt/base/scratch type xfs (rw,relatime,attr2,inode64,noquota)

This is due to the wrong `grep -F $SCRATCH_DEV` which matches the mount
point instead of the device in that mount.
Fix _check_mounted_on() to grep the pattern "$dev on " and "$dev on $mnt"
instead of just grepping for "$dev" and "$mnt" without the " on " anchor.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agofstests: fix test and scratch filters for overlapping DEV/MNT paths
Amir Goldstein [Tue, 28 Feb 2017 12:18:31 +0000 (14:18 +0200)]
fstests: fix test and scratch filters for overlapping DEV/MNT paths

When configuring overlay base fs, TEST_DEV/DIR and SCRATCH_DEV/MNT
are derived from the base fs mount points, where *_DEV are the
path of the base fs mount point and TEST_DIR/SCRATCH_MNT are
a directory under the base fs mount point.

This means that the overlay DEV paths are prefixes of the overlay
mount points.
Fix the test and sctach filters to check if TEST_DEV/SCRATCH_DEV is
a substring of TEST_DIR/SCRATCH_MNT and try and match the longer
string first.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agofstests: canonicalize mount points on every config section
Amir Goldstein [Tue, 28 Feb 2017 12:18:30 +0000 (14:18 +0200)]
fstests: canonicalize mount points on every config section

Create helper _canonicalize_mountpoint() to check and canonicalize
a mount point path.
Use helper to canonicalize TEST_DIR and SCRATCH_MNT after parse
of every config section.

This is needed for overlay base fs mount.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agofstests: use _test_mount() consistently
Amir Goldstein [Tue, 28 Feb 2017 12:18:29 +0000 (14:18 +0200)]
fstests: use _test_mount() consistently

On start of every test run and on every test, in init_rc() helper,
the test partition is verified to be mounted, or is mounted by
the helper _test_mount().

_test_mount() uses mount options $TEST_FS_MOUNT_OPTS and not
$MOUNT_OPTIONS like _scratch_mount() does.

_test_cycle_mount(), which is called by some tests uses the
_test_mount() helper as well.

Contrary to those cases, in _require_test() helper, if test
partition is not mounted, the helper _mount_or_remount_rw()
is called to mount the test partition with $MOUNT_OPTIONS.
Although this should never happen, because of the test in
init_rc(), this case is inconsistent with the rest of the code,
so it has been changed to use _test_mount() as it should.

When running tests with a multi section configuration, and
either FSTYP or MOUNT_OPTIONS change between sections, the
helper _test_unmount() is called to unmount the old test mount
and then _mount_or_remount_rw() is called to mount it again
with new FSTYP and/or MOUNT_OPTIONS.
This is again inconsistent with the rest of the code, so
was changed to use _test_mount() and instead of checking
if MOUNT_OPTIONS have changed between sections, we check if
TEST_FS_MOUNT_OPTS were changed between sections.
Otherwise, we can leave the test partition mounted.

This change is needed to support overlay base fs mount
and for multi section config files which include overlay FSTYP.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agofstests: sanity check that test partitions are not mounted elsewhere
Amir Goldstein [Tue, 28 Feb 2017 12:18:28 +0000 (14:18 +0200)]
fstests: sanity check that test partitions are not mounted elsewhere

When $TEST_DEV is mounted at a different location then $TEST_DIR,
_require_test() aborts the test with an error:
 TEST_DEV=/dev/sda5 is mounted but not on TEST_DIR=/mnt/test

There are several problems with current sanity check:
1. the output of the error is mixed into out.bad and hard to see
2. the test partition is unmounted at the end of the test regardless
   of the fact that it not pass the sanity that we have exclusivity
3. scratch partition has a similar sanity check in _require_scratch(),
   but we may not get to it, because $SCRATCH_DEV is unmounted prior
   to running the tests (which could unmount another mount point).

To solve all these problems, introduce a helper _check_mounted_on().
It checks if a device is mounted on a given mount point and optionally
checks the mounted fs type.

The sanity checks in _require_scratch() and _require_test() are
converted to use the helper and gain the check for correct fs type.

The helper is used in init_rc() to sanity check both test and scratch
partitions, before tests are run and before $SCRATCH_DEV is unmounted.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/rc: accept "localdomain" to mean YP/NIS disabled
Eric Biggers [Tue, 28 Feb 2017 22:09:36 +0000 (14:09 -0800)]
common/rc: accept "localdomain" to mean YP/NIS disabled

When asked to 'cat' the passwd or group files, currently xfstests runs
the 'domainname' program to get what the kernel thinks the domainname
is.  It seems that traditionally this has been "(none)" if YP/NIS is not
in use.  However, Android devices use a different convention which is to
set the domainname to "localdomain".  Therefore, this patch updates
xfstests to accept both conventions to indicate YP/NIS disabled.  This
stops a number of tests from failing when run on Android.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: stress test xfs_scrub
Darrick J. Wong [Tue, 28 Feb 2017 20:57:10 +0000 (12:57 -0800)]
xfs: stress test xfs_scrub

Three new tests:

- Repair files that are mapped into memory in running programs
- Run scrub -n concurrently with fsstress
- Run scrub -y concurrently with fsstress

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoreflink: test adjacency of reflinked blocks
Darrick J. Wong [Sat, 25 Feb 2017 01:12:57 +0000 (17:12 -0800)]
reflink: test adjacency of reflinked blocks

If we reflink a file with N blocks to another file one block at a time,
does the destination file end up with the same number of extents as the
source file?  In other words, does the filesystem succeed at combining
adjacent mappings into a maximal extents?

Signed-off-by: Darrick J. Wong <djwong@djwong.org>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/004: relax freespace check
Christoph Hellwig [Tue, 28 Feb 2017 00:52:35 +0000 (16:52 -0800)]
xfs/004: relax freespace check

Just check for a greater equals relation so that we don't have
to adjust the test for every new reservation scheme we add..

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: minor fixes to the fsmap tests
Darrick J. Wong [Sat, 25 Feb 2017 01:12:51 +0000 (17:12 -0800)]
xfs: minor fixes to the fsmap tests

Refactor the fsmap tests slightly to use new helpers, and
fix some minor output scanning issues.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/348: classify this as a fuzzer test
Darrick J. Wong [Sat, 25 Feb 2017 01:12:45 +0000 (17:12 -0800)]
xfs/348: classify this as a fuzzer test

xfs/348 is a fuzzer test since it calls xfs_db to break the scratch fs,
so put it in the fuzzers group.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/xfs: fix unexpected failure on real 4k sector device
Zorro Lang [Mon, 27 Feb 2017 08:23:03 +0000 (16:23 +0800)]
common/xfs: fix unexpected failure on real 4k sector device

_xfs_mkfs_validation_check() run "$cmd -s size=2s" to be sure if
xfsprogs support stricter input checks. But there's an unexpected
failure on 4k sector device:

  # blockdev --getbsz --getpbsz --getss $dev
  4096
  4096
  4096
  # mkfs.xfs -f -N -d file,name=$tmpfile,size=1g -s size=2s
  illegal sector size 1024; hw sector is 4096
  ....

So change the sector size from '2s' to '8s' to keep away this
failure.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: accessing stacked overlayfs files
Xiong Zhou [Thu, 23 Feb 2017 09:42:46 +0000 (17:42 +0800)]
overlay: accessing stacked overlayfs files

Lower dir comes from another overlayfs dir, then
accessing files in it should work.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: test flock after copied up
Xiong Zhou [Thu, 23 Feb 2017 09:42:45 +0000 (17:42 +0800)]
overlay: test flock after copied up

Locks of file in lower dir should working after it's
been copied up.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: can not change underlying immutable file
Xiong Zhou [Thu, 23 Feb 2017 09:42:44 +0000 (17:42 +0800)]
overlay: can not change underlying immutable file

Even timestamp change by touch.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: filter out xattr starts with "trusted.overlay."
Xiong Zhou [Thu, 23 Feb 2017 09:42:43 +0000 (17:42 +0800)]
overlay: filter out xattr starts with "trusted.overlay."

Overlayfs should only filter out xattr starting with
"trusted.overlay.", not "trusted.overlay".

Setting xattrs like "trusted.overlay.xxx" is not allowed.
Setting xattrs like "trusted.overlayxxx" is allowed.

v4.8-rc1 introduced a regression that we can't set xattrs
like "trusted.overlayxxx".  Kernel commit below fixed it
in v4.8:
  fe2b75952347 ovl: Fix OVL_XATTR_PREFIX

This case tests both get/set of these 2 kinds of xattrs.

Pattern "trusted.overlay.xxx" should fail, however the
errno returned by set/get varies among kernel versions.

Pattern "trusted.overlayxxx" shold always work.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: permission check with cached acls in tmpfs
Xiong Zhou [Thu, 23 Feb 2017 09:42:42 +0000 (17:42 +0800)]
overlay: permission check with cached acls in tmpfs

tmpfs does not implement ->get_acl method, overlayfs
need to get its cached acls in permission check when
lower or upper fs is tmpfs.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: test workdir cleanup in mounting
Xiong Zhou [Thu, 23 Feb 2017 09:42:41 +0000 (17:42 +0800)]
overlay: test workdir cleanup in mounting

"work" directory in workdir should be cleaned up
and recreated while overlayfs mounting. Or overlayfs
will be mounted read-only.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: work dir in overlay inherits no ACLs
Xiong Zhou [Thu, 23 Feb 2017 09:42:40 +0000 (17:42 +0800)]
overlay: work dir in overlay inherits no ACLs

They should be cleaned while mounting overlayfs.

[eguan: remove uncorrect comments about getfacl filter]

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: disallow overlayfs as upperdir
Xiong Zhou [Thu, 23 Feb 2017 09:42:39 +0000 (17:42 +0800)]
overlay: disallow overlayfs as upperdir

Using overlayfs dir as upperdir to mount another
overlayfs should fail.

CC: Miklos Szeredi <mszeredi@redhat.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/095: fix duplicate variable name
Zorro Lang [Thu, 23 Feb 2017 03:08:21 +0000 (11:08 +0800)]
generic/095: fix duplicate variable name

I think this definitely isn't what we want:
  local warn4="WARNING:.*fs/xfs/xfs_file\.c:.*xfs_file_aio_read.*"
  local warn4="WARNING:.*fs/iomap\.c:.*iomap_dio_rw.*"

The second warn4 will override the first one. So change the second
to warn5.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/133,4: make sure xfs_db sets negative i_size
Xiao Yang [Thu, 23 Feb 2017 03:35:30 +0000 (11:35 +0800)]
xfs/133,4: make sure xfs_db sets negative i_size

1) xfs/133 and xfs/134 work abnornamlly on RHEL6.8GA and RHEL6.9Beta
   because xfs_db fails to set negative i_size and reports "usage:
   write fieldname value".  The special argument "--" is only used to
   end option-scanning in getopt().  So we can run two xfs_db commands
   to set negative i_size regardless of the special argument "--" is
   needed or not.  getopt() has been produced by 'commit c9f5e3db22098
   ("xfs_db: Allow writes of corrupted data")'.

2) xfs/134 passes unexpectedly on RHEL6.8GA due to EINVAL, so we use
   touch command to create 512-aligned test file.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoshared/005,7: fix making sure debugfs sets negative i_size
Xiao Yang [Wed, 22 Feb 2017 08:17:03 +0000 (16:17 +0800)]
shared/005,7: fix making sure debugfs sets negative i_size

If the kernel bug has been fixed, stat command fails to get i_size
and reports "Structure needs cleaning".  So we use debugfs -R "stat"
instead of stat command to make sure debugfs sets negative i_size.
These cases have been broken by commit 0e13e40b247a1 ("shared/005,7:
make sure debugfs sets negative i_size").

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocheck: fix incorrect interrupt detection
Dmitry Monakhov [Wed, 22 Feb 2017 09:48:13 +0000 (13:48 +0400)]
check: fix incorrect interrupt detection

Currently "Interrupted!" string always appeared in check.log because
flag cleared at very end after summary was dumped already.  It looks
very strange, but it was broken from very beginning 27fba05e
(2001-01-15)

- update interrupt flag at the end of a section loop, but before _wrapup
- dump 'Interrupted' to stdout

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric,ext4: add defrag group
Theodore Ts'o [Tue, 21 Feb 2017 00:17:02 +0000 (19:17 -0500)]
generic,ext4: add defrag group

Apparently btrfs already has tests marked as belonging in the defrag
group, but none of the ext4 or generic tests were so marked.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: test mmap io through DAX and non-DAX
Xiong Zhou [Wed, 8 Feb 2017 04:11:18 +0000 (12:11 +0800)]
generic: test mmap io through DAX and non-DAX

Mount TEST_DEV as non-DAX, SCRATCH_DEV as DAX, then
do some IO between them. In this case we use mmap
and dio/buffered IO read/write test programme.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: test per-inode DAX flag by IO
Xiong Zhou [Wed, 8 Feb 2017 04:11:17 +0000 (12:11 +0800)]
xfs: test per-inode DAX flag by IO

In a DAX mountpoint, do IO betwen files with and
without DAX per-inode flag. We do mmap, both
O_DIRECT and buffered read/write IO in this case.
Then test again in the same device without dax
mountoption.

Add help _require_scratch_dax to make sure we can
test DAX feature on SCRATCH_DEV.

Add mmap dio test programme to test read/write
between a mmap area of one file and another file
directly or buffered, with different size.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: test decompression in the middle of large extents
Omar Sandoval [Fri, 17 Feb 2017 06:54:52 +0000 (22:54 -0800)]
btrfs: test decompression in the middle of large extents

This is a regression test for "Btrfs: fix btrfs_decompress_buf2page()".
It fails for zlib on v4.10-rc[1-7].

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/rc: remove unnecessary cat in _ddt
Omar Sandoval [Fri, 17 Feb 2017 02:32:48 +0000 (18:32 -0800)]
common/rc: remove unnecessary cat in _ddt

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: truncate after a mix of dio and buffered writes
Filipe Manana [Tue, 14 Feb 2017 20:36:17 +0000 (20:36 +0000)]
generic: truncate after a mix of dio and buffered writes

Test that if we have a file with a hole, do a mix of direct IO and
buffered writes to it and truncate the file to a size that lies in
the middle of the hole, after unmounting and mounting again the
filesystem, the file has a correct size and no data loss happened.

This test is motivated by a bug found in btrfs when used with the
no-holes feature (i.e. MKFS_OPTIONS="-O no-holes") which is fixed by
the following patch for the linux kernel:

  Btrfs: fix data loss after truncate when using the no-holes feature

[eguan: add _require_odirect]

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: send/receive with sparse files
Filipe Manana [Tue, 14 Feb 2017 20:35:53 +0000 (20:35 +0000)]
btrfs: send/receive with sparse files

Test that both a full and incremental btrfs send operation preserves
file holes.

This used to fail when the filesystem had the NO_HOLES feature enabled,
that is, when the test is run with MKFS_OPTIONS="-O no-holes".

This is fixed by the following patch for the linux kernel:

  "Btrfs: incremental send, fix unnecessary hole writes for sparse files"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: test two vfsmount no peers
Zorro Lang [Tue, 14 Feb 2017 00:55:45 +0000 (19:55 -0500)]
generic: test two vfsmount no peers

This test cover linux commit 7ae8fd0, when mnt_group_id=0, it means
this mount no peers. But this bug treat two zero mnt_group_id as
peers. And it cause a crash by dereference a NULL address.

As below, the crash will happen when mount fs on "B/mnt1/mnt2":

           shared            New FS           shared
     -----------------------[A/mnt1]----------------------
    |                           |                         |
    |                bind       |    bind                 |
[C/mnt1]--[slave C]<------[shared A]------>[slave B]--[B/mnt1]
                                                          |
                                                          |
                                                    [B/mnt1/mnt2]
                                                       (New FS)

Signed-off-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: test mount shared subtrees state transition
Zorro Lang [Tue, 14 Feb 2017 00:55:44 +0000 (19:55 -0500)]
generic: test mount shared subtrees state transition

This case will do function test for mount --make-* operations, it
will verify below state transition:

------------------------------------------------------------------------
|             |make-shared |  make-slave  | make-private |make-unbindab|
--------------|------------|--------------|--------------|-------------|
|shared       |shared      |*slave/private|   private    | unbindable  |
|             |            |              |              |             |
|-------------|------------|--------------|--------------|-------------|
|slave        |shared      |    **slave   |    private   | unbindable  |
|             |and slave   |              |              |             |
|-------------|------------|--------------|--------------|-------------|
|shared       |shared      |    slave     |    private   | unbindable  |
|and slave    |and slave   |              |              |             |
|-------------|------------|--------------|--------------|-------------|
|private      |shared      |  **private   |    private   | unbindable  |
|-------------|------------|--------------|--------------|-------------|
|unbindable   |shared      |**unbindable  |    private   | unbindable  |
------------------------------------------------------------------------

This case uses fsstress to produce a small random load, to make sure
basic operations on the mountpoints won't cause hang or panic etc.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: test bind mount operations
Zorro Lang [Tue, 14 Feb 2017 00:55:43 +0000 (19:55 -0500)]
generic: test bind mount operations

This case will do function test for mount bind operation, it will
verify below semantics:

---------------------------------------------------------------------------
|         BIND MOUNT OPERATION                                            |
|**************************************************************************
|source(A)->| shared       |       private  |       slave    | unbindable |
| dest(B)  |               |                |                |            |
|   |      |               |                |                |            |
|   v      |               |                |                |            |
|**************************************************************************
|  shared  | shared        |     shared     | shared & slave |  invalid   |
|          |               |                |                |            |
|non-shared| shared        |      private   |      slave     |  invalid   |
***************************************************************************

This case usees fsstress to produce a small random load, to make
sure basic operations on the bind mountpoints won't cause hang or
panic etc.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/rc: new functions for multi-level mount/umount operations
Zorro Lang [Tue, 14 Feb 2017 00:55:42 +0000 (19:55 -0500)]
common/rc: new functions for multi-level mount/umount operations

When I try to write cases about mount shared subtrees test, I find I
always need to do many mount operations, then then umount those
mount point one by one.

To make the code clear, I use a stack to save mounted points
sequentially, then I write 3 common functions to operate this stack.

1. The global stack named MOUNTED_POINT_STACK
2. _get_mount() accepts mount parameters like _mount() does, but the
   mountpoint parameter must be the last one. It will run the
   mount operation and push the mountpoint name into stack.
3. _put_mount() doesn't need any parameter. It will pop the newest
   mountpoint name from the stack, and umount it.
4. _clear_mount_stack() doesn't need any parameter either. It will
   umount all mountpoints in the stack sequentially, and set
   MOUNTED_POINT_STACK=""

Generally, the _clear_mount_stack() function also can be used as
_init_mount_stack() at the beginning of a case. Because it will
prepare an empty stack.

[eguan: add comments and fix code style]

Signed-off-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: stress XFS delalloc indirect block reservations
Brian Foster [Mon, 13 Feb 2017 15:15:53 +0000 (10:15 -0500)]
xfs: stress XFS delalloc indirect block reservations

This test is based on generic/033, which originally used zero range
operations to reproduce indlen reservation problems. Zero range now
includes a pagecache flush before it updates extents, which means
generic/033 is no longer able to reproduce the problem it was
originally written to test.

Create a new test that uses an XFS-specific mechanism (in DEBUG
mode) to induce delalloc extent splits and reproduce the problem
originally reproduced by generic/033. In addition, update the test
to include a larger buffered write pattern that is known to
reproduce premature indlen exhaustion on delalloc extents.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoshared/005,7: make sure debugfs sets negative i_size
Xiao Yang [Mon, 13 Feb 2017 11:21:50 +0000 (19:21 +0800)]
shared/005,7: make sure debugfs sets negative i_size

shared/005 and shared/007 work abnornamlly on RHEL6.8GA and
RHEL6.9Beta because debugfs fails to set i_size to -1 or -512
and reports "exceeds field size maximum".  When debugfs fails
to set a negative i_size, we can skip these cases which don't
trigger the kernel bug.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/fuzzy: discard xfs_io stdout in _scratch_fuzz_modify
Xiao Yang [Tue, 7 Feb 2017 09:58:19 +0000 (17:58 +0800)]
common/fuzzy: discard xfs_io stdout in _scratch_fuzz_modify

xfs/083 fails because _scratch_fuzz_modify redirects xfs_io stdout
to $ROUND2_LOG.  _scratch_fuzz_modify function has been modified by
commit 7a7463d362cd3 ("populate: fix some silly errors when
modifying a fs while fuzzing").

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agofstests: btrfs: Use compressible data
Anand Jain [Mon, 6 Feb 2017 07:55:39 +0000 (15:55 +0800)]
fstests: btrfs: Use compressible data

/dev/urandom is incompressible and, /dev/zero is highly compressible,
so both are less effective in testing the compress code logic in btrfs.

This patch introduces a text data generator
 cat /dev/urandom | od
to populate the files where /dev/urandom is currently being used in the
btrfs test cases.

And updates the _populate_fs() with a new option -c, so to instruct
to use the compressible data to populate the file(s).

[eguan: add comments, fix indention]

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/052,4: filter out lost+found when running "ls $SCRATCH_MNT"
Theodore Ts'o [Sun, 5 Feb 2017 06:53:36 +0000 (01:53 -0500)]
generic/052,4: filter out lost+found when running "ls $SCRATCH_MNT"

The generic/052 and generic/054 tests run ls on the root directory,
and on ext4 we have a lost+found directory which is not in the
golden output.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/log: add ext4 support for log state probing
Theodore Ts'o [Sun, 5 Feb 2017 06:53:35 +0000 (01:53 -0500)]
common/log: add ext4 support for log state probing

This enables generic/052, generic/054, and generic/055 for ext4.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: honor the FSSTRESS_AVOID environment variable
Theodore Ts'o [Sun, 5 Feb 2017 06:53:34 +0000 (01:53 -0500)]
generic: honor the FSSTRESS_AVOID environment variable

Make sure all tests which run fsstress, and which do not have a very
specific custom profile of which file system operations to run, honor
the FSSTRESS_AVOID environment variable.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/074: _cleanup potentially removes local FS files
Omer Zilberberg [Sun, 5 Feb 2017 06:58:20 +0000 (08:58 +0200)]
generic/074: _cleanup potentially removes local FS files

fstest_dir is set after _cleanup() is defined and _require_test is
called. If _require_test fails (due to some unrelated bug in tested
FS), _cleanup will attempt to remove $fstest_dir.*, which expands to
.* This has the unfortunate effect of removing xfstests' .git and
.gitignore.

Here is the 074.out.bad file for this case:
QA output created by 074
mount: permission denied
common/rc: retrying test device mount with external set
mount: permission denied
common/rc: could not mount /dev/pmem0 on /mnt
rm: refusing to remove '.' or '..' directory: skipping '.'
rm: refusing to remove '.' or '..' directory: skipping '..'
----

The only other test which could face the same problem is
generic/285.  However, that test defines _cleanup after the
_require* calls, and so on _require_test failure, 285.out.bad
contains the harmless:
./common/rc: line 1: _cleanup: command not found

Signed-off-by: Omer Zilberberg <omzg@plexistor.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: check that destination timestamps are not updated on dedupe
Christoph Hellwig [Fri, 3 Feb 2017 09:58:39 +0000 (10:58 +0100)]
generic: check that destination timestamps are not updated on dedupe

Signed-off-by: Christoph Hellwig <hch@lst.de
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: check that destination timestamps are updated on clone
Christoph Hellwig [Fri, 3 Feb 2017 09:58:38 +0000 (10:58 +0100)]
generic: check that destination timestamps are updated on clone

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: add shutdown group
Theodore Ts'o [Fri, 3 Feb 2017 23:01:37 +0000 (18:01 -0500)]
generic: add shutdown group

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/398: update errors to be returned when doing cross renames w/o keys
Theodore Ts'o [Fri, 3 Feb 2017 16:25:37 +0000 (11:25 -0500)]
generic/398: update errors to be returned when doing cross renames w/o keys

The bug fix that ultimately landed in the fscrypt tree will return
ENOKEY instead of EPERM when doing a cross rename involving a
directory where the key is not available.  So fix up the golden output
for generic/398 accordingly.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Eric Biggers <ebiggers3@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/388: use the src/godown program instead of xfs_io
Theodore Ts'o [Fri, 3 Feb 2017 20:50:37 +0000 (15:50 -0500)]
generic/388: use the src/godown program instead of xfs_io

The "xfs_io -xc shutdown $MNT" command doesn't work on non-xfs file
systems.  So use the equivalent src/godown program instead, which is
how the other tests which use the shutdown command were converted from
xfs-specific tests to generic tests.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/014: fix typo in xfs_quota command
Omar Sandoval [Fri, 3 Feb 2017 23:32:03 +0000 (15:32 -0800)]
xfs/014: fix typo in xfs_quota command

This meant to say "$qa_group", not "$qa_grup".

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/007: cleanup $TEST_DIR/$seq dir
Amir Goldstein [Fri, 3 Feb 2017 11:05:37 +0000 (13:05 +0200)]
generic/007: cleanup $TEST_DIR/$seq dir

Test generic/007 was failing after running test overlay/007,
because the latter left behind a non empty directory named 007
in test dir and the former failed to mkdir a directory with the
same name.

Cleanup $TEST_DIR/$seq both at end of overlay/007 and beginning of
generic/007 to avoid this collision.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/030: ignore lost rmapbt blocks
Darrick J. Wong [Sat, 28 Jan 2017 17:38:25 +0000 (09:38 -0800)]
xfs/030: ignore lost rmapbt blocks

When we destroy the AG0 headers, we also lose track of the rmapbt
blocks, which causes xfs_repair to complain about their unconnectedness.
Filter out the error message from the output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/299: don't chmod the underlying mountpoints prior to mounting
Darrick J. Wong [Sat, 28 Jan 2017 17:38:19 +0000 (09:38 -0800)]
xfs/299: don't chmod the underlying mountpoints prior to mounting

It's silly to make $SCRATCH_MNT world writable just prior to mounting
the scratch device; instead, do it after the mount.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/263: don't hardcode inode numbers in output
Darrick J. Wong [Sat, 28 Jan 2017 17:38:12 +0000 (09:38 -0800)]
xfs/263: don't hardcode inode numbers in output

Since we can't control the inode number assignments when creating files,
don't hardcode them in the golden output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: check btrfs-convert with extent and non-extent source
Lakshmipathi.G [Fri, 3 Feb 2017 04:47:16 +0000 (12:47 +0800)]
btrfs: check btrfs-convert with extent and non-extent source

This is used to check the source which contains combination of Ext3
files in non-extent format and  Ext4 extent-files. And validate the
file md5sums before and after conversion.

btrfs/012: BTRFS_CONVERT_PROG,E2FSCK_PROG definitions reused from
common/config

Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: splitted large dio write could trigger assertion on btrfs
Liu Bo [Wed, 25 Jan 2017 22:13:57 +0000 (14:13 -0800)]
generic: splitted large dio write could trigger assertion on btrfs

On btrfs, if a large dio write (>=128MB) got splitted, the
outstanding_extents assertion would complain.  Note that
CONFIG_BTRFS_ASSERT is required.

Regression test for
  Btrfs: adjust outstanding_extents counter properly when dio write is split

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: setxattr on subvolume directory
Omar Sandoval [Wed, 25 Jan 2017 16:44:01 +0000 (08:44 -0800)]
btrfs: setxattr on subvolume directory

This is a regression test for "Btrfs: disable xattr operations on
subvolume directories". On v4.9, it will result in an aborted
transaction.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/122: reduce xfs_dsb_t size
Darrick J. Wong [Tue, 24 Jan 2017 17:08:19 +0000 (09:08 -0800)]
xfs/122: reduce xfs_dsb_t size

We're not going to do reverse-mapping on the realtime device for
some time, so let's fix xfs/122 in the meantime.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agotools/nextid: pad 0 in next test id seq number
Eryu Guan [Tue, 24 Jan 2017 10:30:04 +0000 (18:30 +0800)]
tools/nextid: pad 0 in next test id seq number

nextid returns the next available seq number, but it doesn't pad id
number with 0, e.g.

./tools/nextid ext4
23

After the fix it returns:

./tools/nextid ext4
023

This eases the process of moving tests around in a script.

Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: test xfs_db source command
Darrick J. Wong [Mon, 23 Jan 2017 21:57:50 +0000 (13:57 -0800)]
xfs: test xfs_db source command

Make sure that the 'source' command works correctly whether supplied
via command line or interactive prompt.

You probably want "xfs_db: fix the 'source' command when passed as a
-c option" in xfsprogs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agomvtest: change "FSQA Test No." when we renumber tests
Darrick J. Wong [Mon, 23 Jan 2017 18:52:24 +0000 (10:52 -0800)]
mvtest: change "FSQA Test No." when we renumber tests

We have a number of tests (and submitters) who write tests
containing "FSQA Test No." (as opposed to "FS QA Test No."), so
update the mvtest script to change both forms.

Also fix a sed bug that deletes too many group lines.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agofstests: fix wrong seq number in test
Xiao Yang [Mon, 23 Jan 2017 03:48:15 +0000 (11:48 +0800)]
fstests: fix wrong seq number in test

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: fuzz every field of every structure
Darrick J. Wong [Sat, 21 Jan 2017 08:11:21 +0000 (00:11 -0800)]
xfs: fuzz every field of every structure

Previously, our XFS fuzzing efforts were limited to using the xfs_db
blocktrash command to scribble garbage all over a block.  This is
pretty easy to discover; it would be far more interesting if we could
fuzz individual fields looking for unhandled corner cases.  Since we
now have an online scrub tool, use it to check for our targeted
corruptions prior to the usual steps of writing to the FS, taking it
offline, repairing, and re-checking.

These tests use the new xfs_db 'fuzz' command to test corner case
handling of every field.  The 'print' command tells us which fields
are available, and the fuzz command can write zeroes or ones to the
field; set the high, middle, or low bit; add or subtract numbers; or
randomize the field.  We loop through all fields and all fuzz verbs to
see if we can trip up the kernel.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/populate: create attrs in different namespaces
Darrick J. Wong [Sat, 21 Jan 2017 08:11:11 +0000 (00:11 -0800)]
common/populate: create attrs in different namespaces

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agopopulate: discover XFS structure fields and fuzz verbs, and use them to fuzz fields
Darrick J. Wong [Sat, 21 Jan 2017 08:11:04 +0000 (00:11 -0800)]
populate: discover XFS structure fields and fuzz verbs, and use them to fuzz fields

Create some routines to help us perform targeted fuzzing of individual
fields in various XFS structures.  Specifically, we want the caller to
drop the xfs_db iocursor on the victim field; from there, the scripts
should discover all available fields and fuzzing verbs, and try each
fuzz verb on every available field.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agopopulate: cache scratch metadata images
Darrick J. Wong [Sat, 21 Jan 2017 08:10:58 +0000 (00:10 -0800)]
populate: cache scratch metadata images

Create a helper function to create a populated FS image and dump the
metadata into a file on the test device, with the purpose of allowing
future (fuzzer) invocations of _populate_fs use the cached metadata to
save time.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/fuzzy: move fuzzing helper functions here
Darrick J. Wong [Sat, 21 Jan 2017 08:10:50 +0000 (00:10 -0800)]
common/fuzzy: move fuzzing helper functions here

Move some fuzzing helper functions into a new common/fuzzy file.
We'll add a lot more fuzzing helpers in subsequent patches.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agopopulate: fix some silly errors when modifying a fs while fuzzing
Darrick J. Wong [Sat, 21 Jan 2017 08:10:44 +0000 (00:10 -0800)]
populate: fix some silly errors when modifying a fs while fuzzing

There were some silly errors in _scratch_fuzz_modify such that it
wasn't really doing much of anything because of undefined variables.
(Sloppy refactoring when converting xfsfuzz.sh into xfstests.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agopopulate: optionally fill the filesystem when populating fs
Darrick J. Wong [Sat, 21 Jan 2017 08:10:38 +0000 (00:10 -0800)]
populate: optionally fill the filesystem when populating fs

Be a little more flexible in how much we fill up a pre-populated
filesystem.  For the field fuzzing tests, we don't need the extra
space/inode usage and therefore won't want much at all.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agopopulate: add _require_populate_commands to check for tools
Darrick J. Wong [Sat, 21 Jan 2017 08:10:31 +0000 (00:10 -0800)]
populate: add _require_populate_commands to check for tools

Back when I created common/populate, I thought it was sufficient to
_require the tools that the populate functions need in the main
file.  This turned out to be a bit sloppy, so move them into a
helper function and call it from everything that uses populate.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agopopulate: create all types of XFS metadata
Darrick J. Wong [Sat, 21 Jan 2017 08:10:25 +0000 (00:10 -0800)]
populate: create all types of XFS metadata

We have three new metadata types -- rmapbt, rtrmapbt, and refcountbt.
Ensure that we populate the scratch fs with all three.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: incremental send after replacing directory
Filipe Manana [Thu, 12 Jan 2017 03:13:50 +0000 (03:13 +0000)]
btrfs: incremental send after replacing directory

Test that an incremental send operation works when in both snapshots
there are two directory inodes that have the same number but
different generations and have an entry with the same name that
corresponds to different inodes in each snapshot.

The btrfs issue is fixed by the following patch for the linux kernel:

  "Btrfs: incremental send, do not issue invalid rmdir operations"

Signed-off-by: Robbie Ko <robbieko@synology.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: incremental send after moving a directory
Filipe Manana [Thu, 12 Jan 2017 03:13:37 +0000 (03:13 +0000)]
btrfs: incremental send after moving a directory

Test that an incremental send operation works after moving a
directory into a new parent directory, deleting its previous parent
directory and creating a new inode that has the same inode number as
the old parent.

This issue is fixed by the following patch for the linux kernel:

  "Btrfs: incremental send, do not delay rename when parent inode is new"

Signed-off-by: Robbie Ko <robbieko@synology.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agobtrfs: incremental send after replacing a top level inode
Filipe Manana [Thu, 12 Jan 2017 03:13:13 +0000 (03:13 +0000)]
btrfs: incremental send after replacing a top level inode

Test that an incremental send operation does not fail when a new
inode replaces an old inode that has the same number but different
generation, and both are direct children of the subvolume/snapshot
root.

This is fixed by the following patch for the linux kernel:

  "Btrfs: send, fix failure to rename top level inode due to name
   collision"

Signed-off-by: Robbie Ko <robbieko@synology.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: test readlink/readdir with malformed inode mode
Amir Goldstein [Wed, 11 Jan 2017 06:41:12 +0000 (08:41 +0200)]
xfs: test readlink/readdir with malformed inode mode

In addition to testing xfs_repair on inodes with malformed mode,
and fstat of those inodes on a mounted fs, try to also list content
of mock directory and readlink of mock symlink.

Readdir of mock directory triggers XFS assertion on kernel 4.9 with
XFS_DEBUG=y

XFS: Assertion failed:
     rval == 0 || args->dp->i_d.di_size == args->geo->blksize,
file: /home/amir/build/src/linux/fs/xfs/libxfs/xfs_dir2.c, line: 634

A fix patch, as suggested by Darrick, changes this ASSERT() to
return -EFSCORRUPTED ("xfs: sanity check directory inode di_size").

Merging this test should be deferred to after fix patch is merged.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: test fstat with malformed inode mode
Amir Goldstein [Mon, 9 Jan 2017 13:30:24 +0000 (15:30 +0200)]
xfs: test fstat with malformed inode mode

In addition to testing xfs_repair on inodes with malformed mode,
also test fstat of those inodes on a mounted fs.

This additional test is quite noisy with dmesg warnings, so
check dmesg has been disabled.

This test fails on kernel 4.9 because a zero size inode is not
identified as malformed dir. A patch has been sent to fix this
("xfs: sanity check directory inode di_size").

This test may be merged before the fix patch.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/405: test mkfs against thin provision device
Boyang Xue [Wed, 18 Jan 2017 09:27:24 +0000 (17:27 +0800)]
generic/405: test mkfs against thin provision device

Test mkfs against thin provision device, which has very small
backing size and very big virtual size. mkfs should return error
when it hits EIO.

Signed-off-by: Boyang Xue <bxue@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/dmthin: call $UDEV_SETTLE_PROG before 'dmsetup remove'
Boyang Xue [Wed, 18 Jan 2017 09:27:23 +0000 (17:27 +0800)]
common/dmthin: call $UDEV_SETTLE_PROG before 'dmsetup remove'

Wait for device to be fully settled so that 'dmsetup remove' doesn't
fail due to EBUSY.

Signed-off-by: Boyang Xue <bxue@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agotools: compare-failures script
Eric Sandeen [Fri, 13 Jan 2017 14:56:15 +0000 (08:56 -0600)]
tools: compare-failures script

This is a simple script to compare failures across runs.

Given files containing stdout from several runs, each of which contains
a Failures: line, it will print a table of all failures for each run.
Test subdir names are abbreviated for compactness, i.e. generic->g.
For 7 results files named test 1 through test 7:

Failures:
g/075 g/082 g/209 g/233 g/270 g/388 x/004 x/073 x/076
-----------------------------------------------------
      g/082       g/233             x/004 x/073       test1
      g/082       g/233             x/004 x/073 x/076 test2
      g/082                         x/004 x/073 x/076 test3
      g/082                   g/388 x/004 x/073       test4
      g/082             g/270       x/004 x/073       test5
      g/082                         x/004 x/073       test6
g/075 g/082 g/209 g/233             x/004 x/073       test7

This lets us easily spot unique failures and outliers.

This could be enhanced to output CSV etc, but for now I think it's
helpful to visualize changes in failures across multiple runs.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/btrfs: add helper to detect dump-super or btrfs-show-super
David Sterba [Mon, 16 Jan 2017 15:35:12 +0000 (16:35 +0100)]
common/btrfs: add helper to detect dump-super or btrfs-show-super

The command btrfs-show-super is not supposed to be distributed but
was useful for testing. The same functionality is now present as
'btrfs inspect-internal dump-super', let's detect which one is
available and use it in btrfs/011 that fails with btrfs-progs 4.8.4+

Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agooverlay: test concurrent copy up
Amir Goldstein [Mon, 16 Jan 2017 12:42:25 +0000 (14:42 +0200)]
overlay: test concurrent copy up

Run 4 process pairs, each competing over copy up of 1K files
in 1 directory. One opponent touches all files in the directory
and the other truncates all files in the directory.

This test does NOT check for concurrent copy up support. It only
fails on unexpected errors of any of the touch/truncate operations.

The test full output should demonstrate the expected results -
for kernel with concurrent copy up support, truncate workers are
not delayed by touch workers. As a result, truncate workers will
finish their work much sooner than a test run without concurrent
copy up support.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoREADME: Fix Quick start typo
Lakshmipathi.G [Mon, 16 Jan 2017 23:13:41 +0000 (04:43 +0530)]
README: Fix Quick start typo

Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/rc: Fix _supports_filetype function
Xiao Yang [Mon, 16 Jan 2017 08:07:20 +0000 (16:07 +0800)]
common/rc: Fix _supports_filetype function

generic/401 failed on RHEL6.8GA because "--output=xxx"
option is not supported by df.  So we remove it.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/004: account for XFS reservations changes in 4.10-rc
Christoph Hellwig [Fri, 13 Jan 2017 17:05:40 +0000 (18:05 +0100)]
xfs/004: account for XFS reservations changes in 4.10-rc

And drop support for some really old kernels to clean things up.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon/rc: use /bin/bash in _user_do()
Christoph Hellwig [Fri, 13 Jan 2017 16:38:16 +0000 (17:38 +0100)]
common/rc: use /bin/bash in _user_do()

Without this the epressions in generic/256 will fail on a system
where /bin/sh is the Default (e.g. modern Debian versions).

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon: remove unused XFSPROGS_VERSION check
David Disseldorp [Thu, 12 Jan 2017 17:18:48 +0000 (18:18 +0100)]
common: remove unused XFSPROGS_VERSION check

This was only used to check for mkfs.xfs -n version=ci support,
which is carried in xfsprogs >= 2.10. _require_xfs_mkfs_ciname() is
now used to explicitly check for the mkfs parameter.

Suggested-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/188: add and use _require_xfs_mkfs_ciname
David Disseldorp [Thu, 12 Jan 2017 17:18:47 +0000 (18:18 +0100)]
xfs/188: add and use _require_xfs_mkfs_ciname

Use an explicit mkfs -n version=ci test to check whether the test
should run, instead of checking the xfsprogs version.

Suggested-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoRevert "generic/38[3-6]: require project quota to be enabled on SCRATCH_DEV"
Eryu Guan [Fri, 16 Dec 2016 04:14:14 +0000 (12:14 +0800)]
Revert "generic/38[3-6]: require project quota to be enabled on SCRATCH_DEV"

Commit 23f60ef304d ("generic/38[3-6]: require project quota to be
enabled on SCRATCH_DEV") introduced a regression that leads
_require_projquota, which uses src/feature to call a quotactl,
operate before the filesystem is mounted, and results in tests not
run on XFS and ext4.

Revert the problematic patch first, because don't want to lose
project quota test coverage on XFS and ext4. We can fix the false
failure on ext3/2 later,

Reported-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: reproduce ext4 bugs in a shift extents logic
Roman Pen [Wed, 11 Jan 2017 09:52:39 +0000 (10:52 +0100)]
generic: reproduce ext4 bugs in a shift extents logic

Regression test which targets two nasty ext4 bugs in a logic which
shifts extents:

1) 14d981f468a1 ("ext4: Include forgotten start block on fallocate insert range")

Test tries to insert many blocks at the same offset to reproduce
the following layout on ext4:

   block #0  block #1
   |ext0 ext1|ext2 ext3 ...|
        ^
     insert of a new block

Because of an incorrect range first block is never reached,
thus ext1 is untouched, resulting to a hole at a wrong offset:

What we got:

   block #0   block #1
   |ext0 ext1|   ext2 ext3 ...|
              ^
              hole at a wrong offset

What we expect:

   block #0    block #1
   |ext0   ext1|ext2 ext3 ...|
        ^
        hole at a correct offset

2) 2b3864b32403 ("ext4: do not polute the extents cache while shifting extents")

Extents status tree is filled in with outdated offsets while doing
extent shift, that leads to wrong data blocks.   That's why md5sum
of a result file is being checked after each block insert.

Signed-off-by: Roman Pen <roman.penyaev@profitbricks.com>
Cc: "Theodore Ts'o <tytso@mit.edu>"
Cc: Eryu Guan <eguan@redhat.com>
Cc: linux-ext4@vger.kernel.org
Cc: fstests@vger.kernel.org
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs: test handling of invalid inode modes
Amir Goldstein [Mon, 9 Jan 2017 13:39:11 +0000 (15:39 +0200)]
xfs: test handling of invalid inode modes

Set all possible file type values for different types of files
and verify that xfs_repair detects the correct errors.

When setting invalid file type values (e.g. core.mode = 0170644),
all files are expected to have been junked by xfs_repair.

When setting valid file type values to non matching file types,
xfs_repair would either detect wrong format and junk the file, e.g.:
  would have junked entry "DATA" in directory PARENT_INO
or detect a ftype mismatch error, e.g.:
  would fix ftype mismatch (5/3) in directory/child PARENT_INO/FIFO_INO

If ftype feature is enabled, when setting file type to one of the
special types (i.e. FIFO(1), CHRDEV(2),BLKDEV(6),SOCKET(14)),
xfs_repair is expected to detect ftype mismatch error. Otherwise,
xfs_repair is not expected to detect ftype mismatch error.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/ext4: check negative inode size
Darrick J. Wong [Mon, 9 Jan 2017 20:55:18 +0000 (12:55 -0800)]
xfs/ext4: check negative inode size

Craft a malicious filesystem image with a negative inode size,
then try to trigger a kernel DoS by appending data to the file.
Ideally this should trigger verifier errors instead of hanging.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agodedupe: fix consistent error message prefixes for dedupe tests
Darrick J. Wong [Mon, 9 Jan 2017 20:54:18 +0000 (12:54 -0800)]
dedupe: fix consistent error message prefixes for dedupe tests

Since we're fixing the xfs_io dedupe command to consistently
print the dedupe ioctl name on error, fix the tests too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoreflink: make error reporting consistent when simulating EIO
Darrick J. Wong [Thu, 5 Jan 2017 01:05:08 +0000 (17:05 -0800)]
reflink: make error reporting consistent when simulating EIO

When we're using dm-error to simulate failed devices, we don't really
know if the write or the fdatasync is going to receive the EIO.  For
tests that make a single (failed) write attempt and never retry, it's
sufficient to check that the file md5 doesn't change after recovery.
For tests that /do/ retry the write, we should capture the entire output
and just look for the word error instead of enshrining the exact perror
message (filename/function call and everything) in the golden output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoreflink: fix quota tests to work properly
Darrick J. Wong [Mon, 9 Jan 2017 20:53:35 +0000 (12:53 -0800)]
reflink: fix quota tests to work properly

Fix the reflink quota tests to su to the fsgqa user so that we actually
test enforcement of quotas.  Seems that XFS enforces user quotas even
if root is writing to a user file, whereas everything else lets root
writes through.  Also clean up some of the variable usage and
_require_user.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoocfs2/reflink: fix file block size reporting
Darrick J. Wong [Thu, 5 Jan 2017 01:04:55 +0000 (17:04 -0800)]
ocfs2/reflink: fix file block size reporting

Some of the reflink tests try to require a specific filesystem block
size so that they can test file block manipulation functions.  That's
straightforward for most filesystems but ocfs2 throws in the additional
twist that data fork block mappings are stored in units of clusters, not
blocks, which causes these reflink tests to fail.

Therefore, introduce a new helper that retrieves the file minimum block
size and adapt the reflink tests to use that instead.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agocommon: add leading underscore to get_block_size
Darrick J. Wong [Thu, 5 Jan 2017 01:04:48 +0000 (17:04 -0800)]
common: add leading underscore to get_block_size

Add a leading underscore to the get_block_size helper since it's a
common function.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoocfs2: test reflinking to inline data files
Darrick J. Wong [Thu, 5 Jan 2017 01:04:42 +0000 (17:04 -0800)]
ocfs2: test reflinking to inline data files

Make sure that we can handle reflinking from and to inline-data files.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/010: explicitly allow corrupted write in xfs_db
Eric Sandeen [Mon, 9 Jan 2017 16:49:21 +0000 (10:49 -0600)]
xfs/010: explicitly allow corrupted write in xfs_db

xfs/010 wants to write corruption and test how xfs_repair
deals, but when:

  xfs: forbid AG btrees with level == 0

is merged to userspace, this new test fails the write verifier
in xfs_db.

Add "-c" to allow the corrupted write, do the corruptions all
in one xfs_db command (so it doesn't have to re-read the
corrupted data on 2nd startup), and filter out the

  "Allowing write of corrupted data and bad CRC"

output from the "write -c" command.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric/081: hack around the lvm unmount race
Christoph Hellwig [Mon, 9 Jan 2017 13:38:18 +0000 (14:38 +0100)]
generic/081: hack around the lvm unmount race

This is butt ugly, but I see no better way than to wait a bit to
work around the race between the weird umount in LVM, and the mount
ismounted checks.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agogeneric: racing getxattr requests against xattr add/remove
Brian Foster [Fri, 6 Jan 2017 19:39:30 +0000 (14:39 -0500)]
generic: racing getxattr requests against xattr add/remove

This test reproduces a bug in XFS where a getxattr of an existing
xattr returns failure due to a race with a setxattr that causes
inode attribute fork conversion.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
7 years agoxfs/118: add to dangerous group
Eric Sandeen [Sat, 7 Jan 2017 16:32:01 +0000 (10:32 -0600)]
xfs/118: add to dangerous group

Failure results in an oops, so add it to dangerous.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>