xfstests-dev.git
23 months agobtrfs/085: check flakey device instead of backend device
Naohiro Aota [Wed, 20 Nov 2019 06:44:06 +0000 (15:44 +0900)]
btrfs/085: check flakey device instead of backend device

btrfs/085 mount and works on $FLAKEY_DEV, but it's inspecting the
underlying $SCRATCH_DEV. Since writing to dm-flakey does not
invalidate page cache of $SCRATCH_DEV, the btrfs command can read a
stalled data from page cache. We should check the flakey device to
avoid such cache inconsistency.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agoext4/031: use _try_scratch_mount instead of _scratch_mount
Xiaoli Feng [Fri, 15 Nov 2019 14:42:48 +0000 (22:42 +0800)]
ext4/031: use _try_scratch_mount instead of _scratch_mount

Here will check the return code of mount option. So update it
to use _try_scratch_mount.

Signed-off-by: Xiaoli Feng <xifeng@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/563: fix the wrong case number in golden image
Zorro Lang [Thu, 14 Nov 2019 14:01:38 +0000 (22:01 +0800)]
generic/563: fix the wrong case number in golden image

The first line of generic/563.out is wrong, the number should be
563, not 011. This mismatch will always cause testing fail.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agooverlay: support timestamp range check
Amir Goldstein [Mon, 11 Nov 2019 07:40:10 +0000 (09:40 +0200)]
overlay: support timestamp range check

Overlayfs timestamp range is the same as base fs timestamp range

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Acked-by: Deepa Dinamani <deepa.kernel@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/050: Consistently remove traling dot from umount output
Jan Kara [Mon, 11 Nov 2019 14:49:53 +0000 (15:49 +0100)]
generic/050: Consistently remove traling dot from umount output

We did not consistently remove trailing dot from umount output which
can presumably lead to false failures with particular versions of
util-linux. Make sure all umount output is properly filtered.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/050: Handle xfs quota special case with different output
Jan Kara [Mon, 11 Nov 2019 14:49:52 +0000 (15:49 +0100)]
generic/050: Handle xfs quota special case with different output

Instead of faking output for the case of XFS with quotas, just use a
different output file with appropriate output.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/235: Fix false failure on ext2
Jan Kara [Fri, 1 Nov 2019 11:00:46 +0000 (12:00 +0100)]
generic/235: Fix false failure on ext2

The test gives false failure on ext2 filesystem as 64k file already
has indirect block and so space usage does not exactly match
expectation.

The test really needs to verify only whether quota accounting got
reenabled so just test using creating another empty file which is
not prone to these problems.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/382: Make test independent of fs allocation
Jan Kara [Fri, 1 Nov 2019 11:00:45 +0000 (12:00 +0100)]
generic/382: Make test independent of fs allocation

Test generic/382 depends on filesystem allocating exactly 30M of blocks
when writing 30M file. This is not true for some filesystems - e.g. for
ext2 due to indirect blocks - while leads to false positive failures.
In this case, the test is not actually interested in comparing exact
usage, rather in verifying the ability to write 30M worth of data. So
instead of comparing 'xfs_quota report' output, just depend on detecting
error when writing files.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/050: Fix test failure for filesystems without journal
Jan Kara [Tue, 5 Nov 2019 13:19:21 +0000 (14:19 +0100)]
generic/050: Fix test failure for filesystems without journal

Filesystems without journal can happily mount unrecovered filesystem
read-only which confuses this test. Handle this by providing
different expected output for filesystems without journal.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agocommon/quota: Fix _qmount_options
Jan Kara [Tue, 5 Nov 2019 13:43:20 +0000 (14:43 +0100)]
common/quota: Fix _qmount_options

_qmount_options didn't properly substitute usrjquota and grpjquota
options. Fix the regular expression.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agofsstress: show the real file id and parid in rmdir() and unlink()
kaixuxia [Tue, 5 Nov 2019 02:41:26 +0000 (10:41 +0800)]
fsstress: show the real file id and parid in rmdir() and unlink()

The source file id and parentid are overwritten by del_from_flist()
call, so should show the actually values.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agobtrfs: test that send can issue clone operations within the same file
Filipe Manana [Tue, 5 Nov 2019 14:57:00 +0000 (14:57 +0000)]
btrfs: test that send can issue clone operations within the same file

Verify that both full and incremental send operations can issue
clone operations when a file has extents that are shared with itself
(at different offsets of course).

This currently fails on btrfs but is addressed by a patch for the
kernel titled:

  "Btrfs: send, allow clone operations within the same file"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agocommon: add helper to count number of exclusive extents in a file
Filipe Manana [Tue, 5 Nov 2019 14:56:40 +0000 (14:56 +0000)]
common: add helper to count number of exclusive extents in a file

Add a new helper that is similar to _count_extents() except that
extents that are shared several times by the file with itself
(reflinked at different file offsets) are accounted 1 time only,
instead of N times.

This is motivated by a subsequent test for btrfs that will use this
new helper, to verify that send streams are issuing reflink
operations.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agocommon: open files in ro mode for extent and hole count helpers
Filipe Manana [Tue, 5 Nov 2019 14:56:22 +0000 (14:56 +0000)]
common: open files in ro mode for extent and hole count helpers

The helper functions _count_extents() and _count_holes() open their
input file in RW mode to call fiemap, however opening it in RO mode
is enough.  By opening them in RW mode it also makes it not possible
to use them against files residing in btrfs readonly snapshots for
example.

So just open the files in RO mode in these functions.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agooverlay/066: adjust test file size && add more test patterns
Chengguang Xu [Thu, 7 Nov 2019 02:59:41 +0000 (10:59 +0800)]
overlay/066: adjust test file size && add more test patterns

Making many small holes in 10M test file seems not very
helpful for test coverage and it takes too much time on
creating test files. In order to improve test speed we
adjust test file size to (10 * iosize) for iosize aligned
hole files and meanwhile add more test patterns for small
random holes and small empty file.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Chengguang Xu <cgxu519@mykernel.net>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agoxfs/207: explicitly test for xflag character
Eric Sandeen [Thu, 7 Nov 2019 15:07:26 +0000 (09:07 -0600)]
xfs/207: explicitly test for xflag character

With a recent change to xfs_io[1], the fsxattr.xflags output line
has added another flag column, so xfs/207 starts failing.

Rather than testing for an exact set of flags, test whether the
specific flag we are interested in is set or unset.

[1] xfs_io/lsattr: expose FS_XFLAG_HASATTR flag

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agoxfs: make sure the kernel and repair tools catch bad names
Darrick J. Wong [Tue, 29 Oct 2019 22:29:48 +0000 (15:29 -0700)]
xfs: make sure the kernel and repair tools catch bad names

Make sure we actually catch bad names in the kernel.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric/317: Use relative paths to avoid traversal permission problems
Zhihao Cheng [Thu, 31 Oct 2019 09:19:17 +0000 (17:19 +0800)]
generic/317: Use relative paths to avoid traversal permission problems

generic/317 will fail because execvp(cmd) is executed without permission,
where cmd is '$here/src/lstat64 $file', which is called by
  $here/src/nsexec -s -U -M "0 $qa_user_id 1000" -G "0 $qa_user_id 1000"\
  $here/src/lstat64 $file

So, you will see following output:
  From user_ns
  ...
  +execvp: Permission denied

nsexec runs the instruction '$here/src/lstat64 $file' as a regular user,
the regular user may not have permission to access path in '$here'.

Actually, it has been fixed in 4818302fbf ("xfstests: generic/317 use
relative paths..."), which then been modified by b7cecbea22 ("fstests:
Add path $here before src/<file>").

Fixes: b7cecbea22 ("fstests: Add path $here before src/<file>")
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agocommon/rc: generalize _get_filesize()
Chao Yu [Fri, 1 Nov 2019 10:28:21 +0000 (18:28 +0800)]
common/rc: generalize _get_filesize()

There are some testcases use below two kind of commands to get file
size, generalize the second way as global function _get_filesize()
to simply codes.

1. ls -l $1 | $AWK_PROG '{print $5}'
2. stat -c %s $1

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric: handle fs.verity.require_signatures being enabled
Eric Biggers [Fri, 1 Nov 2019 23:22:19 +0000 (16:22 -0700)]
generic: handle fs.verity.require_signatures being enabled

Most of the fs-verity tests fail if the fs.verity.require_signatures
sysctl has been set to 1.  Update them to set this sysctl to 0 at the
beginning of the test and restore it to its previous value at the end.

generic/577 intentionally sets this sysctl to 1.  Make it restore the
previous value at the end of the test rather than assuming it was 0.

Also simplify _require_fsverity_builtin_signatures() to just check for
the presence of the file /proc/sys/fs/verity/require_signatures rather
than check whether the fs-verity keyring is listed in /proc/keys.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agogeneric: test the deadlock between the AGI and AGF with RENAME_WHITEOUT
kaixuxia [Thu, 31 Oct 2019 06:41:49 +0000 (14:41 +0800)]
generic: test the deadlock between the AGI and AGF with RENAME_WHITEOUT

There is ABBA deadlock bug in XFS between the AGI and AGF when
performing rename() with RENAME_WHITEOUT flag, and add this testcase
to make sure the rename() call works well.

Though this is a xfs-specific bug, the reproducer has no
xfs-specific part.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agofsstress: add EXCHANGE renameat2 support
kaixuxia [Thu, 31 Oct 2019 06:41:48 +0000 (14:41 +0800)]
fsstress: add EXCHANGE renameat2 support

Support the EXCHANGE renameat2 syscall in fsstress.

In order to maintain filelist/filename integrity, we restrict
rexchange to files of the same type.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agofsstress: add NOREPLACE and WHITEOUT renameat2 support
kaixuxia [Thu, 31 Oct 2019 06:41:47 +0000 (14:41 +0800)]
fsstress: add NOREPLACE and WHITEOUT renameat2 support

Support the renameat2(NOREPLACE and WHITEOUT) syscall in fsstress.

The fent id correlates with filename and the filename correlates
to type in flist, and the RWHITEOUT operation would leave a dev
node around with whatever the name of the source file was, so in
order to maintain filelist/filename integrity, we should restrict
RWHITEOUT source file to device nodes.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agofsstress: show the real file id and parid in rename_f()
kaixuxia [Thu, 31 Oct 2019 06:41:46 +0000 (14:41 +0800)]
fsstress: show the real file id and parid in rename_f()

The source file id and parentid are overwritten by del_from_flist()
call, and should show the actually values.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
23 months agocommon/rc: fix _require_xfs_io_command utimes check
Baihua Lu [Thu, 31 Oct 2019 01:25:53 +0000 (01:25 +0000)]
common/rc: fix _require_xfs_io_command utimes check

There is an error on checking utimes command. Put quotation mark
after four "0".

Signed-off-by: Baihua Lu <lubaihua0331@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: verify ciphertext of v2 encryption policies with Adiantum
Eric Biggers [Tue, 15 Oct 2019 18:16:43 +0000 (11:16 -0700)]
generic: verify ciphertext of v2 encryption policies with Adiantum

Verify ciphertext for v2 encryption policies that use Adiantum to
encrypt file contents and file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: verify ciphertext of v2 encryption policies with AES-128
Eric Biggers [Tue, 15 Oct 2019 18:16:42 +0000 (11:16 -0700)]
generic: verify ciphertext of v2 encryption policies with AES-128

Verify ciphertext for v2 encryption policies that use AES-128-CBC-ESSIV
to encrypt file contents and AES-128-CTS-CBC to encrypt file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: verify ciphertext of v2 encryption policies with AES-256
Eric Biggers [Tue, 15 Oct 2019 18:16:41 +0000 (11:16 -0700)]
generic: verify ciphertext of v2 encryption policies with AES-256

Verify ciphertext for v2 encryption policies that use AES-256-XTS to
encrypt file contents and AES-256-CTS-CBC to encrypt file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: add test for non-root use of fscrypt API additions
Eric Biggers [Tue, 15 Oct 2019 18:16:40 +0000 (11:16 -0700)]
generic: add test for non-root use of fscrypt API additions

Test non-root use of the fscrypt filesystem-level encryption keyring and
v2 encryption policies.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: add basic test for fscrypt API additions
Eric Biggers [Tue, 15 Oct 2019 18:16:39 +0000 (11:16 -0700)]
generic: add basic test for fscrypt API additions

Add a basic test of the fscrypt filesystem-level encryption keyring and
v2 encryption policies.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/encrypt: support verifying ciphertext of v2 encryption policies
Eric Biggers [Tue, 15 Oct 2019 18:16:38 +0000 (11:16 -0700)]
common/encrypt: support verifying ciphertext of v2 encryption policies

Update _verify_ciphertext_for_encryption_policy() to support v2
encryption policies.

This also required adding HKDF-SHA512 support to fscrypt-crypt-util.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/encrypt: support checking for v2 encryption policy support
Eric Biggers [Tue, 15 Oct 2019 18:16:37 +0000 (11:16 -0700)]
common/encrypt: support checking for v2 encryption policy support

Allow passing '-v 2' to _require_scratch_encryption() to check for
v2 encryption policy support on the scratch device, and for xfs_io
support for setting and getting v2 encryption policies.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/encrypt: add helper functions that wrap new xfs_io commands
Eric Biggers [Tue, 15 Oct 2019 18:16:36 +0000 (11:16 -0700)]
common/encrypt: add helper functions that wrap new xfs_io commands

Wrap the new xfs_io commands 'add_enckey', 'rm_enckey', and
'enckey_status' with helper functions.

Also add _user_do_get_encpolicy(), so that all encryption xfs_io
commands have a _user_do() version.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/encrypt: disambiguate session encryption keys
Eric Biggers [Tue, 15 Oct 2019 18:16:35 +0000 (11:16 -0700)]
common/encrypt: disambiguate session encryption keys

Rename the helper functions that add/remove keys from the session
keyring, in order to distinguish them from the helper functions I'll
be adding to add/remove keys from the new filesystem-level keyring.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: add an fs-verity stress test
Eric Biggers [Mon, 14 Oct 2019 22:05:21 +0000 (15:05 -0700)]
generic: add an fs-verity stress test

Add a stress test for fs-verity.  This tests enabling fs-verity on
multiple files concurrently with concurrent readers on those files
(with reads occurring before, during, and after the fs-verity
enablement), while fsstress is also running on the same filesystem.

I haven't seen any failures from running this on ext4 and f2fs.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/rc: skip encrypted direct I/O tests on f2fs
Eric Biggers [Thu, 24 Oct 2019 20:45:29 +0000 (13:45 -0700)]
common/rc: skip encrypted direct I/O tests on f2fs

Skip the O_DIRECT tests on f2fs when the test_dummy_encryption mount
option is given, for the same reason as given for ext4 in commit
9b154b26e4a1 ("common/rc: ext4 doesn't support O_DIRECT with
encryption").

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: trim adjacent extents across adjacent block groups boundary
Qu Wenruo [Thu, 24 Oct 2019 10:16:29 +0000 (18:16 +0800)]
btrfs: trim adjacent extents across adjacent block groups boundary

The test case checks if btrfs can trim adjacent extents across
adjacent block groups boundary correctly.

The test case craft the following extents layout:

         |  BG1 |      BG2        |       BG3            |
 Bytenr: X-8M   X      X+512M     X+1G  X+1G+128M
         |//////|//////|          |     |//|

There is a long existing bug that, when btrfs is trying to trim the
range at [X+512M, X+1G+128M), it will only trim [X+512M, X+1G).

This test case is the regression test for this long existing bug.

It will verify the trimmed bytes by using loopback device backed up
by a file, and checking the bytes used by the file to determine how
many bytes are trimmed.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/{569,570}: use _filter_xfs_io_error to filter pwrite64
Yang Xu [Fri, 25 Oct 2019 09:19:00 +0000 (17:19 +0800)]
generic/{569,570}: use _filter_xfs_io_error to filter pwrite64

xfsprogs prior to commit 2f9a125c3a39 ("xfsprogs: replace
pread64/pwrite64 by equivalent pread/pwrite") will see "pwrite64:
Text file busy" error. Just filter these "pwrite64"s to "pwrite"
with _filter_xfs_io_error.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/casefold: support f2fs
Chao Yu [Tue, 15 Oct 2019 10:09:38 +0000 (18:09 +0800)]
common/casefold: support f2fs

Now, f2fs has ported casefold feature from ext4, let's enable the
testcase.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/defrag: support f2fs
Chao Yu [Tue, 15 Oct 2019 10:09:37 +0000 (18:09 +0800)]
common/defrag: support f2fs

F2FS supports defragment based on file granularity, adjust test suit
to support f2fs.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: test for alien devices
Anand Jain [Tue, 22 Oct 2019 09:24:50 +0000 (17:24 +0800)]
btrfs: test for alien devices

Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians no btrfs magic string.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: test for alien btrfs-devices
Anand Jain [Tue, 22 Oct 2019 09:24:49 +0000 (17:24 +0800)]
btrfs: test for alien btrfs-devices

Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians someother btrfs.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/log-writes: Add new discard check point
Qu Wenruo [Tue, 22 Oct 2019 07:58:05 +0000 (15:58 +0800)]
src/log-writes: Add new discard check point

Despite the existing <number>|fua|flush checkpoint, add a new
discard check point to make sure discard is not screwing up things.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/402: fix for updated behavior of timestamp limits
Deepa Dinamani [Fri, 19 Jul 2019 04:12:31 +0000 (21:12 -0700)]
generic/402: fix for updated behavior of timestamp limits

The mount behavior will not be altered because of the unsupported
timestamps on the filesystems.

Adjust the test accordingly.

You can find the series at
https://git.kernel.org/torvalds/c/cfb82e1df8b7c76991ea12958855897c2fb4debc

Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: copy-up test for variant sparse files
Chengguang Xu [Thu, 24 Oct 2019 12:29:23 +0000 (20:29 +0800)]
overlay: copy-up test for variant sparse files

This is intensive copy-up test for sparse files, these cases will be
mainly used for regression test of copy-up improvement for sparse
files.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Chengguang Xu <cgxu519@mykernel.net>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/435: disable dmesg checks
Darrick J. Wong [Tue, 22 Oct 2019 01:49:46 +0000 (18:49 -0700)]
xfs/435: disable dmesg checks

This test corrupts the filesystem to see what assertions and other
things to get logged, so don't treat that as a failure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofstests: Add path $here before src/<file>
Zhihao Cheng [Mon, 21 Oct 2019 13:35:46 +0000 (21:35 +0800)]
fstests: Add path $here before src/<file>

Calling src/<file> without path '$here' may cause the problem that
the file cannot be found.

For example, Running generic/192 with overlayfs(Let ubifs as base
fs) yields the following output:

  generic/192 - output mismatch
     QA output created by 192
     sleep for 5 seconds
     test
    +./common/rc: line 316: src/t_dir_type: No such file or directory
     delta1 is in range
     delta2 is in range
    ...

When the use case fails, the call stack in generic/192 is:

  local unknowns=$(src/t_dir_type $dir u | wc -l) common/rc
  _supports_filetype common/rc
  _overlay_mount common/overlay
  _overlay_test_mount common/overlay
  _test_mount common/rc
  _test_cycle_mount generic/192

Before _test_cycle_mount() being invoked, generic/192 executed 'cd
/' to change work dir from 'xfstests-dev' to '/', so src/t_dir_type
was not found.

[Eryu: some tests run src/<file> as regular user, don't add $here
prefix in such case, as a regular user may have no search permission
on $here]

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/097: Remove wrong broken assignment operation
Yang Xu [Mon, 7 Oct 2019 07:15:15 +0000 (15:15 +0800)]
xfs/097: Remove wrong broken assignment operation

On old kernel, since commit ded188b8609 ("xfs: Fix the situation
that mount operation rejects corrupted XFS") running this case got
the mismatched output, as below:
-----------------------------------
 + check fs
 + corrupt image
 + mount image && modify files
-broken: 1
+broken: 0
 + repair fs
 + mount image (2)
------------------------------------

It fails because the broken is always equal to 0 when
_try_scratch_mount succeed. So remove this wrong assignment
operation.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/448: define BASE_TEST_FILE before _cleanup
Anna Schumaker [Fri, 18 Oct 2019 17:33:43 +0000 (13:33 -0400)]
generic/448: define BASE_TEST_FILE before _cleanup

NFS v4.2 supports SEEK_DATA and SEEK_HOLE, but earlier versions do
not.  As a result, the test exits and runs the cleanup function
without the $BASE_TEST_FILE variable set and the shell expands it to
"rm -f .*", deleting all hidden files in the current directory.

Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/035: Create new outfile for NFS
Anna Schumaker [Fri, 18 Oct 2019 17:33:42 +0000 (13:33 -0400)]
generic/035: Create new outfile for NFS

Rename-overwrites over NFS work with a "silly rename" over the
network, so the nlink count stays at 1 instead of dropping to 0.
This is expected behavior for NFS, so we should use a different
golden output file to account for this.

 See the NFS faq at: nfs.sourceforge.net/#faq_d2 for more
 information about silly renames.

Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofstests: Add virtio-fs shared file system support
QI Fuli [Tue, 15 Oct 2019 08:32:28 +0000 (17:32 +0900)]
fstests: Add virtio-fs shared file system support

This patch adds support for virtio-fs shared file system that lets
virtual machines access a directory tree on the host.

To run xfstests on it, first, start virtiofsd daemon in host:
 ./virtiofsd -o vhost_user_socket=/tmp/vhostqemu0 -o source=$DIR0 -o cache=always
 ./virtiofsd -o vhost_user_socket=/tmp/vhostqemu1 -o source=$DIR1 -o cache=always

second, launch QEMU with:
 -chardev socket,id=char0,path=/tmp/vhostqemu0
 -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs0
 -chardev socket,id=char1,path=/tmp/vhostqemu1
 -device vhost-user-fs-pci,queue-size=1024,chardev=char1,tag=myfs1
 -m 8G
 -object memory-backend-file,id=mem,size=8G,mem-path=/dev/shm,share=on
 -numa node,memdev=mem

then, inside the VM run xfstests with:
 export TEST_DEV=myfs0
 export TEST_DIR=$TESTDIR
 export SCRATCH_DEV=myfs1
 export SCRATCH_MNT=$SCRATCHMNT
 export MOUNT_OPTIONS=""
 export TEST_FS_MOUNT_OPTS=""

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Misono Tomohiro <misono.tomohiro@fujitsu.com>
Signed-off-by: QI Fuli <qi.fuli@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: check reflink multiple mmap write
Darrick J. Wong [Fri, 11 Oct 2019 21:41:20 +0000 (14:41 -0700)]
generic: check reflink multiple mmap write

Add a test to make sure that we can handle multiple memory mappings to a
physical storage extent shared by multiple files, and that we can handle
the copy on write operation without error.  Make sure we can also handle
mappings at different offsets in the page cache.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agopopulate: punch files after writing to fragment free space properly
Darrick J. Wong [Wed, 9 Oct 2019 18:18:48 +0000 (11:18 -0700)]
populate: punch files after writing to fragment free space properly

The filesystem population code frequently allocates a large file and
punches out every other block ("swiss-cheese files") in an attempt to
cause the creation of a lot of metadata to fill out the btrees.  This
pattern, however, has a subtle bug if the writes to the swiss-cheese
file are not allocated in batches and we're trying to fragment the free
space records in order to achieve a certain metadata btree shape.

This is exactly what happens on a DAX filesystem, since we no longer
have the page cache to stage delalloc writes.  Each xfs_io pwrite call
to the multi-megabyte swiss-chese file turns into multiple 4k pwrites,
which means that file data blocks are allocated 4k at a time.  This can
be fatal to our goal of fragmenting the free space btrees because the
allocator sees a 4k allocation request and uses 4k blocks from the
fragmented parts of the free space to satisfy the "small" request.  When
this happens, the XFS populate function cannot fill out the free space
btree to sufficient height and tests fail.

(In regular delalloc mode we'd cache all those small write() in memory
and try for a single large allocation, which we'd generally get.)

To fix this, we need to force the filesystem to allocate all blocks
before freeing any blocks.  Split the creation of swiss-cheese files
into two parts: (a) writing data to the file to force allocation, and
(b) punching the holes to fragment free space.

This bug affects only XFS but we convert the one ext4 usage anyway.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/263: use _scratch_mkfs_xfs instead of open-coded mkfs call
Darrick J. Wong [Tue, 8 Oct 2019 01:03:23 +0000 (18:03 -0700)]
xfs/263: use _scratch_mkfs_xfs instead of open-coded mkfs call

Fix this test to use _scratch_mkfs_xfs instead of the open-coded mkfs
call.  This is needed to make the test succeed when XFS DAX is enabled
and mkfs enables reflink by default.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/{088, 089, 091}: redirect stderr when writing to corrupt fs
Darrick J. Wong [Tue, 8 Oct 2019 01:03:17 +0000 (18:03 -0700)]
xfs/{088, 089, 091}: redirect stderr when writing to corrupt fs

These tests primarily check that writes to a corrupt fs don't take down
the system, and that running repair will fix them.  Therefore, redirect
stderr to seqres.full so that we don't fail these tests in DAX mode.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/196: check for delalloc blocks after pwrite
Darrick J. Wong [Tue, 8 Oct 2019 01:03:11 +0000 (18:03 -0700)]
xfs/196: check for delalloc blocks after pwrite

This test depends on the pwrite creating delalloc blocks, which doesn't
happen if the scratch fs is mounted in dax mode (or has an extent size
hint applied).  Therefore, check for delalloc blocks and _notrun if we
didn't get any.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/renameat2: remove duplicated renameat2 system call number for i386
kaixuxia [Fri, 11 Oct 2019 07:03:24 +0000 (15:03 +0800)]
src/renameat2: remove duplicated renameat2 system call number for i386

Remove duplicated renameat2 system call number for i386.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/001: remove unnecessary backslash
Murphy Zhou [Wed, 9 Oct 2019 05:34:34 +0000 (13:34 +0800)]
generic/001: remove unnecessary backslash

Awk 5.0.1 is complaining about this backslash and causing false alarm
like this:
    warning: regexp escape sequence '\#' is not a known regexp operator

This fix works fine with Awk 3.1.7 , 4.0.2 , 4.2.1 and 5.0.1. This
backslash is not necessary.

Signed-off-by: Murphy Zhou <jencce.kernel@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: add a test for multi-subvolume fsyncing
Josef Bacik [Mon, 7 Oct 2019 19:33:39 +0000 (15:33 -0400)]
btrfs: add a test for multi-subvolume fsyncing

I discovered a problem in btrfs where we'd end up pointing at a block we
hadn't written out yet.  This is triggered by a race when two different
files on two different subvolumes fsync.  This test exercises this path
with dm-log-writes, and then replays the log at every FUA to verify the
file system is still mountable and the log is replayable.

This test is to verify the fix

btrfs: fix incorrect updating of log root tree

actually fixed the problem.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test the fs-verity built-in signature verification support
Eric Biggers [Mon, 30 Sep 2019 21:15:53 +0000 (14:15 -0700)]
generic: test the fs-verity built-in signature verification support

Add a basic test for the fs-verity built-in signature verification
support, which is an optional feature where the kernel can be configured
to enforce that all verity files are accompanied with a valid signature
by a key that has been loaded into the fs-verity keyring.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test using fs-verity and fscrypt simultaneously
Eric Biggers [Mon, 30 Sep 2019 21:15:52 +0000 (14:15 -0700)]
generic: test using fs-verity and fscrypt simultaneously

This primarily verifies correct ordering of the hooks for each feature:
fscrypt needs to be first.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test that fs-verity is using the correct measurement values
Eric Biggers [Mon, 30 Sep 2019 21:15:51 +0000 (14:15 -0700)]
generic: test that fs-verity is using the correct measurement values

This test verifies that fs-verity is doing its Merkle tree-based hashing
correctly, i.e. that it hasn't been broken by a change.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test corrupting verity files
Eric Biggers [Mon, 30 Sep 2019 21:15:50 +0000 (14:15 -0700)]
generic: test corrupting verity files

This test corrupts various parts of the contents of a verity file, or
parts of its Merkle tree, by writing directly to the block device.  It
verifies that this causes I/O errors when the relevant part of the
contents is later read by any means.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test access controls on the fs-verity ioctls
Eric Biggers [Mon, 30 Sep 2019 21:15:49 +0000 (14:15 -0700)]
generic: test access controls on the fs-verity ioctls

Test access controls on the fs-verity ioctls.  FS_IOC_MEASURE_VERITY is
allowed on any file, whereas FS_IOC_ENABLE_VERITY requires write access.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test general behavior of verity files
Eric Biggers [Mon, 30 Sep 2019 21:15:48 +0000 (14:15 -0700)]
generic: test general behavior of verity files

This is a basic fs-verity test which verifies:

- conditions for enabling verity
- verity files have expected contents and size
- can't change contents of verity files, but can change metadata
- can retrieve a verity file's measurement via FS_IOC_MEASURE_VERITY

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/verity: add common functions for testing fs-verity
Eric Biggers [Mon, 30 Sep 2019 21:15:47 +0000 (14:15 -0700)]
common/verity: add common functions for testing fs-verity

Add common functions for setting up and testing fs-verity.  fs-verity is
an ext4 and f2fs filesystem feature which provides Merkle tree based
hashing (similar to dm-verity) for individual read-only files, mainly
for the purpose of efficient authenticity verification.  Other
filesystems may add fs-verity support later, using the same API.

Running the fs-verity tests will require:

- Kernel v5.4-rc1 or later configured with CONFIG_FS_VERITY=y

- The 'fsverity' utility program from
  https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git

- e2fsprogs v1.45.2 or later for ext4 tests, or f2fs-tools v1.11.0 or
  later for f2fs tests

Some individual tests will have additional requirements.

For more information about fs-verity, see
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/filter: add _filter_xfs_io_fiemap()
Eric Biggers [Mon, 30 Sep 2019 21:15:46 +0000 (14:15 -0700)]
common/filter: add _filter_xfs_io_fiemap()

Add _filter_xfs_io_fiemap() to clean up the output of
'xfs_io -c fiemap'.  This will be used by a function in common/verity.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: Add lease testing
Ira Weiny [Mon, 23 Sep 2019 17:59:59 +0000 (10:59 -0700)]
generic: Add lease testing

The actual lease test code has been added to src/locktest.c (see
previous commits), now create a new lease test script to drive the
test.

Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/locktest: Add common locktest functions
Ira Weiny [Mon, 23 Sep 2019 17:59:58 +0000 (10:59 -0700)]
common/locktest: Add common locktest functions

Move common test functions to common directory and call them from the
lock test.  This is in preparation for adding a separate lease test.

Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Add truncate lease tests
Ira Weiny [Mon, 23 Sep 2019 17:53:34 +0000 (10:53 -0700)]
src/locktest: Add truncate lease tests

Add verification of signal reception on truncate:

/*  9 */"Write lease gets SIGIO on truncate",
/* 10 */"Read lease gets SIGIO on truncate",

Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Add lease testing for basic signal reception
Ira Weiny [Mon, 23 Sep 2019 17:53:33 +0000 (10:53 -0700)]
src/locktest: Add lease testing for basic signal reception

/* 5 */"Read lease gets SIGIO on write open",
/* 6 */"Write lease gets SIGIO on read open",
/* 7 */"Read lease does _not_ get SIGIO on read open",
/* 8 */"Read lease gets SIGIO on write open",

Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Add simple lease testing
Ira Weiny [Mon, 23 Sep 2019 17:53:32 +0000 (10:53 -0700)]
src/locktest: Add simple lease testing

/*  1 */"Take Read Lease",
/*  2 */"Take Write Lease",
/*  3 */"Fail Write Lease if file is open somewhere else",
/*  4 */"Fail Read Lease if opened with write permissions",

Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Add run() function
Ira Weiny [Mon, 23 Sep 2019 17:53:31 +0000 (10:53 -0700)]
src/locktest: Add run() function

Separate the functionality of main() into socket set up and run().  Pass
the test information to run() rather than have it be gloabal.  This is
in preparation for adding a separate lease test array.

Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Audit all debug output
Ira Weiny [Mon, 23 Sep 2019 17:53:30 +0000 (10:53 -0700)]
src/locktest: Audit all debug output

Audit all the debug output to be clear on what failed so that we can
remove the debug flag from the script.

Specifically, remove the need for a debug flag on system call error
output.  This helps to indicate what happened when an individual test
step fails.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Clean up error output
Ira Weiny [Mon, 23 Sep 2019 17:53:29 +0000 (10:53 -0700)]
src/locktest: Clean up error output

Add some debugging
Make sure all errors go to stderr
Clean up formatting and make failures stand out
Report test results for the client not just the server

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/131: Save stderr for debugging
Ira Weiny [Mon, 23 Sep 2019 17:53:28 +0000 (10:53 -0700)]
generic/131: Save stderr for debugging

Details of which internal step failed within a test are lost without
additional debugging output.  Save that output by separating stdout and
stderr.

This allows the server port to be written solely to stdout for
consumption by the script.  Then all error output can be sent to the
seqres.full file in the event of a failure.  Then, depend on the return
code of the server _and_ the client to detect failure and save the error
output for inspection.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/131: Clean up pid variables
Ira Weiny [Mon, 23 Sep 2019 17:53:27 +0000 (10:53 -0700)]
generic/131: Clean up pid variables

Change the name of the variables to reflect the client vs server.  This
will help in future patches.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest.c: Fix return code if last test fails
Ira Weiny [Mon, 23 Sep 2019 17:53:26 +0000 (10:53 -0700)]
src/locktest.c: Fix return code if last test fails

If anything but the first step of the last test fails, the exit code
(fail_count) was not properly set.  Fix this such that follow on patches
will be able to save error output and correctly inform the script that a
failure has occurred rather than just expecting random output to fail
the diff check.

The issue is last_test is not properly tracking which test the loop is
currently on.  Therefore fail_count would not be incremented correctly.

Remove the special case of checking for the end of the steps array and
track last_test properly.  Then adjust test_count to be correct for the
new code.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest.c: Clean up client command passing
Ira Weiny [Mon, 23 Sep 2019 17:53:25 +0000 (10:53 -0700)]
src/locktest.c: Clean up client command passing

The client was using the test index rather than the values being passed
to it for a command.  While this technically worked it is cleaner to
fully initialize the command message and use the values in that message.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Add get_cmd_str
Ira Weiny [Mon, 23 Sep 2019 17:53:24 +0000 (10:53 -0700)]
src/locktest: Add get_cmd_str

Rather than open code printing the strings, use a helper function.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Change command macro names
Ira Weiny [Mon, 23 Sep 2019 17:53:23 +0000 (10:53 -0700)]
src/locktest: Change command macro names

The macro names for commands are not consistent.  Change them such that
they are not confused with other flags and macros within the test.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Remove OPEN macro
Ira Weiny [Mon, 23 Sep 2019 17:53:22 +0000 (10:53 -0700)]
src/locktest: Remove OPEN macro

The code is much cleaner without this macro.  While here add extra debug
output as well.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/locktest: Remove unnecessary sleep
Ira Weiny [Mon, 23 Sep 2019 17:53:21 +0000 (10:53 -0700)]
src/locktest: Remove unnecessary sleep

There is no need to sleep because we asked for debugging.  Remove extra
sleep code.

Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: test balance profile convert functionality
Nikolay Borisov [Tue, 1 Oct 2019 09:04:19 +0000 (12:04 +0300)]
btrfs: test balance profile convert functionality

Add basic test to ensure btrfs conversion functionality is tested.
This test exercies conversion to all possible types of the data
portion. This is sufficient since from the POV of relocation we are
only moving blockgroups.

v5.3 and later kernel needs the following patch to pass the test

btrfs: Fix a regression which we can't convert to SINGLE profile

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocheck: convert spaces to tabs in help msg
Po-Hsu Lin [Fri, 4 Oct 2019 04:04:10 +0000 (12:04 +0800)]
check: convert spaces to tabs in help msg

There are some mix use of spaces and tabs in the help message:
$ ./check --help
Usage: ./check [options] [testlist]

check options
    -nfs                test NFS
    -glusterfs                test GlusterFS
    -cifs               test CIFS
    -9p test 9p
    -overlay test overlay
    -pvfs2          test PVFS2
    -tmpfs              test TMPFS
    -ubifs              test ubifs

unify them with tabs.

Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: Add regression test to check if btrfs can handle high devid
Qu Wenruo [Thu, 3 Oct 2019 07:53:50 +0000 (15:53 +0800)]
btrfs: Add regression test to check if btrfs can handle high devid

Add a regression test to check if btrfs can handle high devid.

The test will add and remove devices to a btrfs fs, so that the devid
will increase to uncommon but still valid values.

The regression is introduced by kernel commit ab4ba2e13346 ("btrfs:
tree-checker: Verify dev item").
The fix is titled "btrfs: tree-checker: Fix wrong check on max devid".

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/493: add missing dedupe group
Chao Yu [Fri, 27 Sep 2019 10:08:10 +0000 (18:08 +0800)]
generic/493: add missing dedupe group

We missed to add the dedupe group, fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: check that we can't write to swap files
Darrick J. Wong [Tue, 24 Sep 2019 16:39:37 +0000 (09:39 -0700)]
generic: check that we can't write to swap files

While active, the media backing a swap file is leased to the kernel.
Userspace has no business writing to it.  Make sure we can't do this.

The two kernel patches titled as below should fix the bug:

mm: set S_SWAPFILE on blockdev swap devices
vfs: don't allow writes to swap files

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/rc: Remove special handing of 'dup' argument for btrfs
Nikolay Borisov [Fri, 27 Sep 2019 10:52:32 +0000 (13:52 +0300)]
common/rc: Remove special handing of 'dup' argument for btrfs

_scratch_pool_mkfs special cases the command executed when 'dup'
option is used when creating a filesystem. This is not true anymore,
since 'dup' works for all profiles and number of devices since
4.5-ish. This is manifested while exercising btrfs' balance argument
combinations test.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: fallocate two bytes at block boundary
Max Reitz [Fri, 27 Sep 2019 10:38:46 +0000 (12:38 +0200)]
generic: fallocate two bytes at block boundary

Allocating two bytes at a block boundary with fallocate should allocate
both blocks involved.  Test this by writing data to both bytes
afterwards and see whether the on-disk size increases (it should not).

This is a regression test for the kernel patch "xfs: Fix tail rounding
in xfs_alloc_file_space()".

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/011: Fill the fs to ensure we have enough data for dev-replace
Qu Wenruo [Fri, 27 Sep 2019 09:58:43 +0000 (17:58 +0800)]
btrfs/011: Fill the fs to ensure we have enough data for dev-replace

[BUG]
When btrfs/011 is executed on a fast enough system (fully memory backed
VM, with test device has unsafe cache mode), the test can fail like
this:

  btrfs/011 43s ... [failed, exit status 1]- output mismatch (see /home/adam/xfstests-dev/results//btrfs/011.out.bad)
    --- tests/btrfs/011.out     2019-07-22 14:13:44.643333326 +0800
    +++ /home/adam/xfstests-dev/results//btrfs/011.out.bad      2019-09-18 14:49:28.308798022 +0800
    @@ -1,3 +1,4 @@
     QA output created by 011
     *** test btrfs replace
    -*** done
    +failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
    +(see /home/adam/xfstests-dev/results//btrfs/011.full for details)
    ...

[CAUSE]
Looking into the full output, it shows:
  ...
  Replace from /dev/mapper/test-scratch1 to /dev/mapper/test-scratch2

  # /usr/bin/btrfs replace start -f /dev/mapper/test-scratch1 /dev/mapper/test-scratch2 /mnt/scratch
  # /usr/bin/btrfs replace cancel /mnt/scratch
  INFO: ioctl(DEV_REPLACE_CANCEL)"/mnt/scratch": not started
  failed: '/usr/bin/btrfs replace cancel /mnt/scratch'

So this means the replace is already finished before we cancel it.
For fast system, it's very common.

[FIX]
In fill_scratch() after all the original file creations, do a timer
based direct IO write.
The extra write will take 2 * $wait_time, utilizing direct IO with 64K
block size, the write performance should be very comparable (although a
little faster) to replace performance.

So later cancel should be able to really cancel the dev-replace without
it finished too early.

Also, do extra check about the above write. If we hit ENOSPC we just
skip the test as the system is really too fast and the fs is not large
enough.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/xfs: wipe the XFS superblock of each AGs
Zorro Lang [Tue, 24 Sep 2019 10:09:19 +0000 (18:09 +0800)]
common/xfs: wipe the XFS superblock of each AGs

xfs/030 always fails after d0e484ac699f ("check: wipe scratch devices
between tests") get merged.

Due to xfs/030 does a sized(100m) mkfs. Before we merge above commit,
mkfs.xfs detects an old primary superblock, it will write zeroes to
all superblocks before formatting the new filesystem. But this won't
be done if we wipe the first superblock(by merging above commit).

That means if we make a (smaller) sized xfs after wipefs, those *old*
superblocks which created by last time mkfs.xfs will be left on disk.
Then when we do xfs_repair, if xfs_repair can't find the first SB, it
will go to find those *old* SB at first. When it finds them,
everyting goes wrong.

So I try to wipe each XFS superblock if there's a XFS ondisk, then
try to erase superblock of each XFS AG by default mkfs.xfs geometry.
Thanks Darrick J. Wong helped to analyze this issue.

Reported-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/036: fix sporadic failures when unmounting scratch filesystem
Filipe Manana [Tue, 24 Sep 2019 09:49:02 +0000 (10:49 +0100)]
btrfs/036: fix sporadic failures when unmounting scratch filesystem

Often this test can fail on unmount because a 'btrfs subvolume snapshot'
command is still running and using the scratch the mount point:

  btrfs/036 168s ... umount: /home/fdmanana/btrfs-tests/scratch_1: target is busy
          (In some cases useful info about processes that
           use the device is found by lsof(8) or fuser(1).)
  _check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
  (see /home/fdmanana/git/hub/xfstests/results//btrfs/036.full for details)

This happens because when we kill the process running the do_snapshots()
function we only kill the subshell and don't wait for any processes it
has spawned to finish before the subshell exits. Fix this by setting a
trap in the do_snapshots() function, so that when the subshell receives
a SIGTERM signal it waits for any running 'btrfs subvolume snapshot'
to finish before exitting. We were also not waiting for the subshell
to exit after sending it the SIGTERM signal, so fix that as well by
calling the 'wait' command after calling 'kill' to send that signal.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: Enable character device to be the base fs partition
Zhihao Cheng [Wed, 25 Sep 2019 06:35:33 +0000 (14:35 +0800)]
overlay: Enable character device to be the base fs partition

When running overlay tests using character devices as base fs partitions,
all overlay usecase results become 'notrun'. Function
'_overay_config_override' (common/config) detects that the current base
fs partition is not a block device and will set FSTYP to base fs. The
overlay usecase will check the current FSTYP, and if it is not 'overlay'
or 'generic', it will skip the execution.

For example, using UBIFS as base fs skips all overlay usecases:

  FSTYP         -- ubifs       # FSTYP should be overridden as 'overlay'
  MKFS_OPTIONS  -- /dev/ubi0_1 # Character device
  MOUNT_OPTIONS -- -t ubifs /dev/ubi0_1 /tmp/scratch

  overlay/001 [not run] not suitable for this filesystem type: ubifs
  overlay/002 [not run] not suitable for this filesystem type: ubifs
  overlay/003 [not run] not suitable for this filesystem type: ubifs

When checking that the base fs partition is a block/character device,
FSTYP is overwritten as 'overlay'. This patch allows the base fs
partition to be a character device that can also execute overlay
usecases (such as ubifs).

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/14[89]: Remove xfs_prepair64 and xfs_prepair tests
Yang Xu [Thu, 19 Sep 2019 10:08:15 +0000 (18:08 +0800)]
xfs/14[89]: Remove xfs_prepair64 and xfs_prepair tests

The two commands have obsoleted long time ago, they don't run on
most systems. So I think we can remove them.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon: check if a given rename flag is supported in _require_renameat2
kaixuxia [Wed, 18 Sep 2019 11:47:47 +0000 (19:47 +0800)]
common: check if a given rename flag is supported in _require_renameat2

Some testcases may require a special rename flag, such as
RENAME_WHITEOUT, so add support check for if a given rename flag is
supported in _require_renameat2.

[Eryu: rename the helper to _require_renameat2 while we're at it,
and add 'exchange' check to generic/398 and generic/419]

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/028: Don't pollute golden output for killing already finished process
Qu Wenruo [Wed, 18 Sep 2019 06:56:25 +0000 (14:56 +0800)]
btrfs/028: Don't pollute golden output for killing already finished process

Sometimes on fast enough test vm, btrfs/028 fails like:

  btrfs/028 31s ... - output mismatch (see /home/adam/xfstests-dev/results//btrfs/028.out.bad)
    --- tests/btrfs/028.out     2019-07-22 14:13:44.646666660 +0800
    +++ /home/adam/xfstests-dev/results//btrfs/028.out.bad      2019-09-18 14:14:45.442131411 +0800
    @@ -1,2 +1,3 @@
     QA output created by 028
    +/home/adam/xfstests-dev/tests/btrfs/028: line 64: kill: (2459) - No such process
     Silence is golden
    ...

It's caused by killing already finished process.

There is no need for kill command to pollute the golden output, so
just redirect all of its stdout and stderr to null.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: Verify falloc on multiple holes won't leak qgroup reserved data space
Qu Wenruo [Sun, 15 Sep 2019 07:22:30 +0000 (15:22 +0800)]
btrfs: Verify falloc on multiple holes won't leak qgroup reserved data space

Add a test case where falloc is called on multiple holes with qgroup
enabled.

This can cause qgroup reserved data space leak and false EDQUOT
error even we're not reaching the limit.

The fix is titled:
"btrfs: qgroup: Fix the wrong target io_tree when freeing
 reserved data space"

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/566: use _require_xfs_quota_foreign and remove -f option
Yang Xu [Mon, 9 Sep 2019 22:11:43 +0000 (06:11 +0800)]
generic/566: use _require_xfs_quota_foreign and remove -f option

xfstests should decide if xfs_quota needs the -f option by
_require_xfs_quota_foreign, not write the -f option after
$XFS_QUOTA_PROG manually. The later way will cause unexpected error
on an old system which xfsprogs doesn't support the -f option.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Tested-by: Murphy Zhou <jencce.kernel@gmail.com>
Acked-by: Murphy Zhou <jencce.kernel@gmail.com>
Acked-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/564: fix copy_range -f availability test
Eric Sandeen [Thu, 12 Sep 2019 16:15:49 +0000 (11:15 -0500)]
generic/564: fix copy_range -f availability test

generic/564 wants to test for copy_range -f, but as it's implemented
it calls copy_range with a length of zero which will silently return
success from the VFS (at least on some kernels) even if the underlying
fs doesn't support it.

So patch this up 2 ways; perform the test with an explicit length
so it's not a no-op, and go ahead test copy_range w/o -f in the test
first just to be on the safe side (and for clearer failure messages.)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>