xfstests-dev.git
2 months agoxfs/007: fix regressions on V4 filesystems
Darrick J. Wong [Wed, 28 Jul 2021 20:00:45 +0000 (13:00 -0700)]
xfs/007: fix regressions on V4 filesystems

Following commit eae40404, I noticed the following regression when
running a V4 fstests run on an 5.13 kernel:

  --- /tmp/fstests/tests/xfs/007.out      2021-05-13 11:47:55.793859995 -0700
  +++ /var/tmp/fstests/xfs/007.out.bad    2021-07-28 09:23:42.856000000 -0700
  @@ -16,4 +16,4 @@
   *** umount
   *** Usage after quotarm ***
   core.nblocks = 0
  -core.nblocks = 0
  +core.nblocks = 1

The underlying cause of this problem is the fact that we now remount the
filesystem with no quota options because that will (soon) become the
only means to turn off quota accounting on XFS.  Because V4 filesystems
don't support simultaneous project and group quotas and play weird
remapping games with the incore superblock field, we actually have to
issue a remove command for the group quota file if we're trying to
truncate the project quota file on a V4 filesystem.

Due to stupid limitations in xfs_quota we actually have to issue a
separate 'remove' command.

Fixes: eae40404 ("xfs/007: unmount after disabling quota")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agoxfs/530: skip test if user MKFS_OPTIONS screw up formatting
Darrick J. Wong [Wed, 28 Jul 2021 20:00:20 +0000 (13:00 -0700)]
xfs/530: skip test if user MKFS_OPTIONS screw up formatting

Skip this test if the user's MKFS_OPTIONS are not compatible with the
realtime parameters that we're injecting in order to test growfs bugs.
Because this test is trying to trigger a specific kernel bug, we need
mkfs to format a filesystem with very specific geometry parameters.

The first problem stems from the fact that the test performs a default
mkfs, computes a suitable realtime geometry from that filesystem, and
then formats a second time with an explicit blocksize option to mkfs.
If the original MKFS_OPTS contained a blocksize directive, the mkfs will
fail because the option was respecified.  The two blocksize options will
be the same, so we drop the explicit blocksize option.

However, this exposes a second problem: MKFS_OPTIONS might contain
options that are not compatible with any realtime filesystem.  If that
happens, _scratch_do_mkfs will "helpfully" drop MKFS_OPTIONS and try
again with only the options specified by the test.  This gets us a
filesystem with the given rt geometry, but it could be missing critical
parameters from MKFS_OPTIONS (like blocksize).  The test will then fail
to exercise the growfs bugfix, so the second part of the fix is to check
that the filesystem we're going to test actually has the geometry
parameters that we require.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agocheck: back off the OOM score adjustment to -500
Darrick J. Wong [Wed, 28 Jul 2021 00:09:59 +0000 (17:09 -0700)]
check: back off the OOM score adjustment to -500

Dave Chinner complained that fstests really shouldn't be running at
-1000 oom score adjustment because that makes it more "important" than
certain system daemons (e.g. journald, udev).  That's true, so increase
it to -500.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agocheck: don't leave the scratch filesystem mounted after _notrun
Darrick J. Wong [Wed, 28 Jul 2021 00:09:48 +0000 (17:09 -0700)]
check: don't leave the scratch filesystem mounted after _notrun

Unmount the scratch filesystem if a test decides to _notrun itself
because _try_wipe_scratch_devs will not be able to wipe the scratch
device prior to the next test run.  We don't want to let scratch state
from one test leak into subsequent tests if we can help it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agoxfs/106: fix golden output regression in quota off test
Darrick J. Wong [Wed, 28 Jul 2021 00:09:43 +0000 (17:09 -0700)]
xfs/106: fix golden output regression in quota off test

In commit 4c5df338, we reshuffled this test a bit in preparation to
disable quotaoff by rearranging the test to testing disabling of quota
by remounting the filesystem.  Unfortunately, extra blank lines were
added to the golden output, leading to test regressions.

The "extra" blank lines are a result of the "echo ; test_off";
test_off() itself doesn't print anything.  Make it print /something/ so
that we know what the test was trying to do when a particular line of
golden output appears, then fix the blank lines.

Fixes: 4c5df338 ("xfs/106: don't test disabling quota accounting")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agomisc: tag all tests that examine crash recovery in a loop
Darrick J. Wong [Thu, 29 Jul 2021 16:53:05 +0000 (09:53 -0700)]
misc: tag all tests that examine crash recovery in a loop

Given all the recent problems that we've been finding with log recovery,
I think it would be useful to create a 'recoveryloop' group so that
developers have a convenient way to run every single test that rolls
around in a fs shutdown loop looking for subtle errors in recovery.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agogeneric: test small swapfile without page-aligned contiguous blocks
Zorro Lang [Mon, 26 Jul 2021 09:46:05 +0000 (17:46 +0800)]
generic: test small swapfile without page-aligned contiguous blocks

If a swapfile doesn't contain even a single page-aligned contiguous
range of blocks, it's an invalid swapfile, and might cause kernel
issue. This case covered commit 5808fecc5723 ("iomap: Fix negative
assignment to unsigned sis->pages in iomap_swapfile_activate").

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Ritesh Harjani <riteshh@linux.ibm.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agogeneric: test for file loss after mix of rename, fsync and inode eviction
Filipe Manana [Tue, 27 Jul 2021 10:24:59 +0000 (11:24 +0100)]
generic: test for file loss after mix of rename, fsync and inode eviction

Test that if we fsync a directory A, evict A's inode, move one file
from directory A to directory B, fsync some other inode that is not
directory A, B or any inode inside these two directories, and then
power fail, the file that was moved is not lost.

This currently fails on btrfs and is fixed by commit ecc64fab7d49
("btrfs: fix lost inode on log replay after mix of fsync, rename and
inode eviction")

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 months agotools: add a dm-logwrites replay tool
Dave Chinner [Wed, 21 Jul 2021 00:13:33 +0000 (10:13 +1000)]
tools: add a dm-logwrites replay tool

Trying to decipher a dm-logwrites replay failure from generic/482 is
difficult. The test destroys all the dm state and devices when the
test fails, leaving nothing behind for post-mortem analysis. This
script is intended to allow replay of the dm-logwrites device one
FUA at a time similar to generic/482, and is used in conjunction
with a modifed g/482 test that does not tear down the dm volumes
or state when the test fails.

This allows the developer to replay the logwrites up to just prior
to failure to examine just the differences between the last good
state and the first failure state, greatly reducing the complexity
of analysing failures.

Instructions for use are in the tools/dm-logwrite-replay script
itself.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
2 months agooverlay: add test for copy up of lower file attributes
Amir Goldstein [Thu, 22 Jul 2021 16:46:34 +0000 (19:46 +0300)]
overlay: add test for copy up of lower file attributes

Overlayfs copies up a subset of lower file attributes since kernel
commits:
173ff5c9ec37 ("ovl: consistent behavior for immutable/append-only inodes")
2e3f6e87c2b0 ("ovl: copy up sync/noatime fileattr flags")

This test verifies this functionality works correctly and that it
survives power failure and/or mount cycle.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agooverlay/077: check for inconsistent d_ino/st_ino
Amir Goldstein [Thu, 22 Jul 2021 17:02:16 +0000 (20:02 +0300)]
overlay/077: check for inconsistent d_ino/st_ino

After unlink of a directory entry, that entry may still apear in getdents
results of an already open directory fd, but it should return a d_ino
value that is consistent with the already observed st_ino of that entry.

Remove redundant break condition from gendents read loop.

For testing of inconsistent d_ino/st_ino we need to unlink an entry
whose st_ino is not that of the upper inode.

In the former merge dir setup we unlink all the files in the lower
dir after copyup, so they all use st_ino of the upper inode.

Let the unlinked file f100 reside in a lower path that is not being
unlinked so it will have the st_ino of the lower inode.

This is a regression test for kernel commit fcb7f373684d
("ovl: skip stale entries in merge dir cache iteration")

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/152: avoid failure when quotaoff is not supported
Darrick J. Wong [Fri, 23 Jul 2021 05:08:59 +0000 (22:08 -0700)]
xfs/152: avoid failure when quotaoff is not supported

Switch the test that removes the quota files to just disable enforcement
and then unmount the file system as disabling quota accounting is about
to go away.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: use $XFS_QUOTA_PROG instead of hardcoding xfs_quota
Christoph Hellwig [Thu, 22 Jul 2021 07:38:32 +0000 (09:38 +0200)]
xfs: use $XFS_QUOTA_PROG instead of hardcoding xfs_quota

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/305: don't turn quota accounting off
Christoph Hellwig [Thu, 22 Jul 2021 07:38:31 +0000 (09:38 +0200)]
xfs/305: don't turn quota accounting off

The test case tests just as much when just testing turning quota
enforcement off, so switch it to that.  This is in preparation for
removing support to turn quota accounting off.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/304: don't turn quota accounting off
Christoph Hellwig [Thu, 22 Jul 2021 07:38:30 +0000 (09:38 +0200)]
xfs/304: don't turn quota accounting off

The test case tests just as much when just testing turning quota
enforcement off, so switch it to that.  This is in preparation for
removing support to turn quota accounting off.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/220: avoid failure when disabling quota accounting is not supported
Christoph Hellwig [Thu, 22 Jul 2021 07:38:29 +0000 (09:38 +0200)]
xfs/220: avoid failure when disabling quota accounting is not supported

Doing a proper _requires for quotaoff support is rather hard, as we need
to test it on a specific file system.  Instead just use sed to remove
the warning and let the test case pass.  Eventually it should just be
removed entirely.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/106: don't test disabling quota accounting
Christoph Hellwig [Thu, 22 Jul 2021 07:38:28 +0000 (09:38 +0200)]
xfs/106: don't test disabling quota accounting

Switch the test that removes the quota files to just disable
enforcement and then unmount the file system as disabling quota
accounting is about to go away.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/007: unmount after disabling quota
Christoph Hellwig [Thu, 22 Jul 2021 07:38:27 +0000 (09:38 +0200)]
xfs/007: unmount after disabling quota

With the pending patches to remove support for disabling quota
accounting on a mounted file system we need to unmount the
file system first before removing the quota files.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/quota: allow removing quota options entirely in _qmount_option
Christoph Hellwig [Thu, 22 Jul 2021 07:38:26 +0000 (09:38 +0200)]
common/quota: allow removing quota options entirely in _qmount_option

Add support for dropping all quota related options instead of only
overriding them with new ones to _qmount_option.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agobtrfs: source common/module when requiring module reload
Marcos Paulo de Souza [Fri, 23 Jul 2021 18:18:00 +0000 (15:18 -0300)]
btrfs: source common/module when requiring module reload

Tests 163, 219, 225 and 242 require the btrfs module to be reloaded
in the middle of the test. If btrfs is compiled builtin, these tests
should be skipped, since we cannot unload it.

Today, this is what happens:

    QA output created by 242
    +./common/btrfs: line 405: _require_loadable_fs_module: command not found

Sourcing common/module in these tests fixes the issue by skipping the
test:
    btrfs/242       [not run] btrfs: must be a module.
    Ran: btrfs/242
    Not run: btrfs/242

Other tests liker btrfs/124 and btrfs/125 already source the same file
for the same reason, so follow the pattern.

Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/encrypt: accept '-' character in no-key names
Eric Biggers [Sun, 18 Jul 2021 19:06:58 +0000 (14:06 -0500)]
common/encrypt: accept '-' character in no-key names

Add the '-' character to the regex that generic/{419,429} use to match
no-key filenames.  This is needed to prevent these tests from failing
after the kernel is changed to use a more standard variant of Base64
(https://lkml.kernel.org/r/20210718000125.59701-1-ebiggers@kernel.org).

Note that despite breaking these tests, the kernel change is not
expected to break any real users, as the fscrypt no-key name encoding
has always been considered an implementation detail.  So it is
appropriate to just update these tests.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/encrypt: add helper function for filtering no-key names
Eric Biggers [Sun, 18 Jul 2021 19:06:57 +0000 (14:06 -0500)]
common/encrypt: add helper function for filtering no-key names

Add a helper function _filter_nokey_filenames() which replaces no-key
filenames with "NOKEY_NAME".  Use it in generic/419 and generic/429.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric: update encryption tests to use term "no-key names"
Eric Biggers [Sun, 18 Jul 2021 19:06:56 +0000 (14:06 -0500)]
generic: update encryption tests to use term "no-key names"

To avoid ambiguity, don't use the terms "ciphertext names" or "encrypted
names" when we really mean "no-key names" (the names that the filesystem
shows when userspace lists an encrypted directory without the
directory's encryption key being present).  This aligns with changes
that have been made to the kernel source code and documentation
(https://lore.kernel.org/r/20200924042624.98439-1-ebiggers@kernel.org).

No change to the actual test logic.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/423: test stx_size of symlink
Eric Biggers [Sun, 18 Jul 2021 15:17:46 +0000 (10:17 -0500)]
generic/423: test stx_size of symlink

Update generic/423 to test that the reported size of a symlink is equal
to the length of the symlink target, as required by POSIX and as
documented in various man pages.

When the test_dummy_encryption mount option is enabled, this serves as a
regression test for the bug that will be fixed by the patch series
"fscrypt: report correct st_size for encrypted symlinks"
(https://lkml.kernel.org/r/20210702065350.209646-1-ebiggers@kernel.org).
However, this test is applicable to all filesystems and mount options.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/561: hide assertions when duperemove is killed
Darrick J. Wong [Tue, 20 Jul 2021 01:08:29 +0000 (18:08 -0700)]
generic/561: hide assertions when duperemove is killed

Use some bash redirection trickery to capture in $seqres.full all of
bash's warnings about duperemove being killed due to assertions
triggering.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agonew: allow users to specify a new test id
Darrick J. Wong [Tue, 20 Jul 2021 01:08:22 +0000 (18:08 -0700)]
new: allow users to specify a new test id

Alter the ./new script so that one can set the test id explicitly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/019: don't dump cores when fio/fsstress hit io errors
Darrick J. Wong [Wed, 7 Jul 2021 00:21:50 +0000 (17:21 -0700)]
generic/019: don't dump cores when fio/fsstress hit io errors

Disable coredumps so that fstests won't mark the test failed when the
EIO injector causes an mmap write to abort with SIGBUS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/371: disable speculative preallocation regressions on XFS
Darrick J. Wong [Wed, 7 Jul 2021 00:21:45 +0000 (17:21 -0700)]
generic/371: disable speculative preallocation regressions on XFS

Once in a very long while, the fallocate calls in this test will fail
due to ENOSPC conditions.  While in theory this test is careful only to
allocate at most 160M of space from a 256M filesystem, there's a twist
on XFS: speculative preallocation.

The first loop in this test is an 80M appending write done in units of
4k.  Once the file size hits 64k, XFS will begin speculatively
preallocating blocks past the end of the file; as the file grows larger,
so will the speculative preallocation.

Since the pwrite/rm loop races with the fallocate/rm loop, it's possible
that the fallocate loop will free that file just before the buffered
write extends the speculative preallocation out to 160MB.  With fs and
log overhead, that doesn't leave enough free space to start the 80MB
fallocate request, which tries to avoid disappointing the caller by
freeing all speculative preallocations.  That fails if the pwriter
thread owns the IOLOCK on $testfile1, so fallocate returns ENOSPC and
the test fails.

The simple solution here is to disable speculative preallocation by
setting an extent size hint if the fs is XFS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/084: fix test program status collection and processing
Darrick J. Wong [Wed, 7 Jul 2021 00:21:39 +0000 (17:21 -0700)]
xfs/084: fix test program status collection and processing

On a test VM with 1.2GB memory, I noticed that the test will
sometimes fail because resvtest leaks too much memory and gets OOM
killed.  It would be useful to _notrun the test when this happens so
that it doesn't appear as an intermittent regression.

The exit code processing in this test is incorrect, since "$?" will
get us the exit status of _filter_resv, not $here/src/resvtest.  Fix
that as part of learning to detect a SIGKILL and skip the test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocheck: run _check_filesystems in an OOM-happy subshell
Darrick J. Wong [Wed, 7 Jul 2021 00:21:34 +0000 (17:21 -0700)]
check: run _check_filesystems in an OOM-happy subshell

While running fstests one night, I observed that fstests stopped
abruptly because ./check ran _check_filesystems to run xfs_repair.
In turn, repair (which inherited oom_score_adj=-1000 from ./check)
consumed so much memory that the OOM killer ran around killing other
daemons, rendering the system nonfunctional.

This is silly -- we set an OOM score adjustment of -1000 on the
./check process so that the test framework itself wouldn't get
OOM-killed, because that aborts the entire run.  Everything else is
fair game for that, including subprocesses started by
_check_filesystems.

Therefore, adapt _check_filesystems (and its children) to run in a
subshell with a much higher oom score adjustment.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agodmthin: erase the metadata device properly before starting
Darrick J. Wong [Wed, 7 Jul 2021 00:21:28 +0000 (17:21 -0700)]
dmthin: erase the metadata device properly before starting

Every now and then I see the following failure when running generic/347:

  --- generic/347.out
  +++ generic/347.out.bad
  @@ -1,2 +1,2 @@
   QA output created by 347
  -=== completed
  +failed to create dm thin pool device

Accompanied by the following dmesg spew:

device-mapper: thin metadata: sb_check failed: blocknr 7016996765293437281: wanted 0
device-mapper: block manager: superblock validator check failed for block 0
device-mapper: thin metadata: couldn't read superblock
device-mapper: table: 253:2: thin-pool: Error creating metadata object
device-mapper: ioctl: error adding target to table

7016996765293437281 is of course the magic number 0x6161616161616161,
which are stale ondisk contents left behind by previous tests that wrote
known tests patterns to files on the scratch device.  This is a bit
surprising, since _dmthin_init supposedly zeroes the first 4k of the
thin pool metadata device before initializing the pool.  Or does it?

dd if=/dev/zero of=$DMTHIN_META_DEV bs=4096 count=1 &>/dev/null

Herein lies the problem: the dd process writes zeroes into the page
cache and exits.  Normally the block layer will flush the page cache
after the last file descriptor is closed, but once in a while the
terminating dd process won't be the only process in the system with an
open file descriptor!

That process is of course udev.  The write() call from dd triggers a
kernel uevent, which starts udev.  If udev is running particularly
slowly, it'll still be running an instant later when dd terminates,
thereby preventing the page cache flush.  If udev is still running a
moment later when we call dmsetup to set up the thin pool, the pool
creation will issue a bio to read the ondisk superblock.  This read
isn't coherent with the page cache, so it sees old disk contents and the
test fails even though we supposedly formatted the metadata device.

Fix this by explicitly flushing the page cache after writing the zeroes.

Fixes: 4b52fffb ("dm-thinp helpers in common/dmthin")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoshared/298: fix random deletion when filenames contain spaces
Darrick J. Wong [Wed, 7 Jul 2021 00:21:23 +0000 (17:21 -0700)]
shared/298: fix random deletion when filenames contain spaces

Correct the deletion loop in this test to work properly when there are
files in $here that have spaces in their name.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/172: disable test when file writes don't use delayed allocation
Darrick J. Wong [Wed, 7 Jul 2021 00:21:12 +0000 (17:21 -0700)]
xfs/172: disable test when file writes don't use delayed allocation

This test tries to exploit an interaction between delayed allocation and
writeback on full filesystems to see if it can trip up the filestreams
allocator.  The behaviors do not present if the filesystem allocates
space at write time, so disable it under these scenarios.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agosrc/seek_sanity_test: test seeks from LLONG_MIN
Matthew Wilcox [Tue, 6 Jul 2021 20:14:46 +0000 (21:14 +0100)]
src/seek_sanity_test: test seeks from LLONG_MIN

We don't currently test seeking from LLONG_MIN, which apparently can
produce an UBSAN warning, although I've been unable to reproduce
that.  In any case, it's a good corner case to test and
straightforward to add.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/459: catch mount failure
Dave Chinner [Fri, 9 Jul 2021 06:10:27 +0000 (16:10 +1000)]
generic/459: catch mount failure

Because if we don't, we freeze the root filesystem instead of
$SCRATCH_MNT and it all goes downhill from there.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/{453,454}: Don't run for FSs restricting names
Pavel Reichl [Thu, 8 Jul 2021 16:35:18 +0000 (18:35 +0200)]
generic/{453,454}: Don't run for FSs restricting names

Running generic/{453,454} tests is probably only useful for filesystems
that allow unrestricted byte streams for names.

Signed-off-by: Pavel Reichl <preichl@redhat.com>
Suggested-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agofstests: install btrfs_crc32c_forged_name.py
Naohiro Aota [Wed, 7 Jul 2021 05:01:41 +0000 (14:01 +0900)]
fstests: install btrfs_crc32c_forged_name.py

btrfs/154 uses btrfs_crc32c_forged_name.py but the file is not
installed in the PKG_LIB_DIR. Also consolidate install target files
into EXTRA_EXECS.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agobtrfs: test fsync after increasing file size with truncate
Filipe Manana [Tue, 6 Jul 2021 14:42:17 +0000 (15:42 +0100)]
btrfs: test fsync after increasing file size with truncate

Test that if we explicitly fsync a file that was previously renamed and
its size was increased through a truncate operation, after a power failure
the file has the size set by the truncate operation. Also, in between the
truncation and the fsync, there was a rename of another file in the same
directory and that file was also fsynced before we fsynced the file that
was truncated.

This currently fails on a 5.13 kernel and on Linus' master branch. It is
fixed by a patch with the following subject:

  "btrfs: fix unpersisted i_size on fsync after expanding truncate"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agobtrfs: test case to fstrim on a degraded filesystem
Anand Jain [Mon, 5 Jul 2021 10:43:31 +0000 (18:43 +0800)]
btrfs: test case to fstrim on a degraded filesystem

Create a degraded btrfs filesystem and run fstrim on it.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoext4/048: skip test of filename wipe if journal checkpoint is not supported
Theodore Ts'o [Mon, 21 Jun 2021 16:48:51 +0000 (12:48 -0400)]
ext4/048: skip test of filename wipe if journal checkpoint is not supported

ext4/048 will fail when running on older kernels that don't support
the filename wipe feature.  The journal checkpoint ioctl is a related
feature, and landed just a little bit after filename wipe feature, so
use support for the journal checkpoint ioctl as a proxy for support
for the filename wipe feature.

Without this change, this test will fail when tesing 5.10, 5.4, and
other LTS kernels.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Leah Rumancik <leah.rumancik@gmail.com>
Tested-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Leah Rumancik <leah.rumancik@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agooverlay/Makefile: Use the unified name rules
Yang Xu [Wed, 30 Jun 2021 01:52:06 +0000 (09:52 +0800)]
overlay/Makefile: Use the unified name rules

Since xfs/ext4 uses XFS_DIR/EXT4_DIR in Makefile(also for other filesystems),
I think we don't have any reason not to use OVERLAY_DIR in here.

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoMakefile: fix bug when running make install under tests/* directory
Yang Xu [Wed, 30 Jun 2021 01:52:05 +0000 (09:52 +0800)]
Makefile: fix bug when running make install under tests/* directory

The TESTS_DIR value is defined in TOPDIR makefile, it is empty here.
So running make install on ext4 directory will get the following info:

../../install-sh -o root -g root -m 755 -d /var/lib/xfstests//ext4

We can see they aren't installed under /var/lib/xfstests/tests/ext4 directory.
Fix this by moving TESTS_DIR definition to include/builddefs.in file.

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agotools: make convert-group ignore converted test files
Darrick J. Wong [Sat, 3 Jul 2021 03:05:48 +0000 (20:05 -0700)]
tools: make convert-group ignore converted test files

Teach the tool that rewrites tests files to use _begin_fstest group
tagging to ignore tests that have already been treated.  This will make
it easier for people to rebase their dev branches.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric: ensure that page is properly filled before write
Jeff Layton [Fri, 2 Jul 2021 13:40:24 +0000 (09:40 -0400)]
generic: ensure that page is properly filled before write

We had a broken optimization in cephfs and netfs lib that could
cause part of a page to be improperly zeroed-out when writing to an
offset that was beyond the EOF but in an existing page.

Add a simple test that would have caught this.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/504: Add scratch_mount before checking for xfs_scrub unicode support
Anju T Sudhakar [Mon, 28 Jun 2021 08:52:57 +0000 (14:22 +0530)]
xfs/504: Add scratch_mount before checking for xfs_scrub unicode support

We may not detect the error `Inappropriate ioctl for device`, while running
`$XFS_IO_PROG -c "scrub probe" "$mountpoint"`, if scratch device is not
mounted before invoking _check_xfs_scrub_does_unicode(). So do
_scratch_mount before checking for xfs_scrub support.

Signed-off-by: Anju T Sudhakar <anju@linux.vnet.ibm.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocheck: exit with exit code 1 after printing the usage message
Theodore Ts'o [Sat, 3 Jul 2021 16:02:47 +0000 (12:02 -0400)]
check: exit with exit code 1 after printing the usage message

If check is passed an invalid command line option, exit with a
non-zero exit code so that a script calling check can detect the
failure.  The check script already performs an "exit 1" if a valid
option has an invalid argument, so this is consistent with existing
practice.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/260: f2fs is also special
Sun Ke [Wed, 30 Jun 2021 11:37:36 +0000 (19:37 +0800)]
generic/260: f2fs is also special

It fail on f2fs:

 [+] Default length with start set (should succeed)
 [+] Length beyond the end of fs (should succeed)
 [+] Length beyond the end of fs with start set (should succeed)
+After the full fs discard 0 bytes were discarded however the file system is 12882804736 bytes long.
+It seems that fs logic handling len argument overflows

The root cause is f2fs can tag a special flag TRIMMED_FLAG to
indicate the whole filesystem is trimmed, so after mkfs/fstrim(),
following fstrim() won't trim any block.

Suggested-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/103: special left calculation for f2fs
Sun Ke [Wed, 30 Jun 2021 11:37:35 +0000 (19:37 +0800)]
generic/103: special left calculation for f2fs

It failed on f2fs:
      QA output created by 103
     +fallocate: No space left on device
      Silence is golden.
     ...

f2fs uses index(radix) tree as mapping metadata, its space overhead
is about one thousandth of the data.

Suggested-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/042: make a bigger img for f2fs
Sun Ke [Wed, 30 Jun 2021 11:37:34 +0000 (19:37 +0800)]
generic/042: make a bigger img for f2fs

f2fs-utils 1.9.0 needs at least 38 MB space for f2fs image. However,
f2fs-utils 1.14.0 needs at least 52 MB. Not sure if it will change
again.  So, just set it to 128M.

Suggested-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/019: fix test cleanup
Dave Chinner [Tue, 29 Jun 2021 00:55:00 +0000 (10:55 +1000)]
generic/019: fix test cleanup

This never worked properly and the failure was exposed by the recent
rework of the fstest setup infrastructure.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agomisc: update documentation to reflect auto-generated group files
Darrick J. Wong [Mon, 21 Jun 2021 16:12:09 +0000 (09:12 -0700)]
misc: update documentation to reflect auto-generated group files

Update the documentation to outline the new requirements for test files
so that we can generate group files during build.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: remove test group management code
Darrick J. Wong [Mon, 21 Jun 2021 16:12:09 +0000 (09:12 -0700)]
fstests: remove test group management code

Remove all the code that manages group files, since we now generate
them at build time.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: remove group files
Darrick J. Wong [Mon, 21 Jun 2021 16:12:00 +0000 (09:12 -0700)]
fstests: remove group files

From: Darrick J. Wong <djwong@kernel.org>

Now that we autogenerate group files, get rid of them in the source
tree.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocheck: use generated group files
Darrick J. Wong [Mon, 21 Jun 2021 15:56:23 +0000 (08:56 -0700)]
check: use generated group files

Convert the ./check script to use the automatically generated group list
membership files, as the transition is now complete.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: adapt the new test script to our new group tagging scheme
Darrick J. Wong [Mon, 21 Jun 2021 15:56:00 +0000 (08:56 -0700)]
fstests: adapt the new test script to our new group tagging scheme

Now that we autogenerate group files, adapt the new test creation script
to use autogenerated group files and to set the group data in the new
test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: convert nextid to use automatic group generation
Darrick J. Wong [Mon, 21 Jun 2021 15:55:39 +0000 (08:55 -0700)]
fstests: convert nextid to use automatic group generation

Convert the nextid script to use the automatic group file generation to
figure out the next available test id.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: automatically generate group files
Darrick J. Wong [Mon, 21 Jun 2021 15:55:09 +0000 (08:55 -0700)]
fstests: automatically generate group files

Now that we've moved the group membership details into the test case
files themselves, automatically generate the group files during build.
The autogenerated files are named "group.list" instead of "group" to
avoid conflicts between generated and (stale) SCM files as everyone
rebases.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: move test group info to test files
Darrick J. Wong [Mon, 21 Jun 2021 15:55:09 +0000 (08:55 -0700)]
fstests: move test group info to test files

Refactor every test in the entire test suite to use the new boilerplate
functions.  This also migrates all the test group information into the
test files.  This patch has been autogenerated via the command:

./tools/convert-group btrfs ceph cifs ext4 f2fs generic nfs ocfs2 overlay perf shared udf xfs

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: clean up open-coded golden output
Darrick J. Wong [Mon, 21 Jun 2021 15:54:54 +0000 (08:54 -0700)]
fstests: clean up open-coded golden output

Fix the handful of tests that open-coded 'QA output created by XXX'.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: add tool migrate group membership data to test files
Darrick J. Wong [Mon, 21 Jun 2021 15:54:54 +0000 (08:54 -0700)]
fstests: add tool migrate group membership data to test files

Create a tool to migrate the mapping of tests <-> groups out of the
group file and into the individual test file as a _begin_fstest
call.  In the next patches we'll rewrite all the test files and auto
generate the group files from the tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: refactor test boilerplate code
Darrick J. Wong [Mon, 21 Jun 2021 15:54:54 +0000 (08:54 -0700)]
fstests: refactor test boilerplate code

Create two new helper functions to deal with boilerplate test code:

A helper function to set the seq and seqnum variables.  We will expand
on this in the next patch so that fstests can autogenerate group files
from now on.

A helper function to register cleanup code that will run if the test
exits or trips over a standard range of signals.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agomisc: move exit status into trap handler
Darrick J. Wong [Mon, 21 Jun 2021 15:54:53 +0000 (08:54 -0700)]
misc: move exit status into trap handler

Move the "exit $status" clause of the _cleanup function into the
argument to the "trap" command so that we can standardize the
registration of the atexit cleanup code in the next few patches.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: fix group check in new script
Darrick J. Wong [Mon, 21 Jun 2021 15:54:53 +0000 (08:54 -0700)]
fstests: fix group check in new script

In the tests/*/group files, group names are found in the Nth columns of
the file, where N > 1.  The grep expression to warn about unknown groups
is not correct (since it currently checks column 1), so fix this.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: test incremental send swapping location of a directory with a file
Filipe Manana [Wed, 9 Jun 2021 10:25:14 +0000 (11:25 +0100)]
btrfs: test incremental send swapping location of a directory with a file

Test that an incremental send operation succeeds, and produces the
correct results, after renaming and moving around directories and files
with multiple hardlinks, in such a way that one of the files gets the old
name and location of a directory and another name (hardlink) with the old
name and location of another file that was located in that same directory.

This currently fails on btrfs but is fixed by a kernel patch with the
following subject:

  "btrfs: send: fix invalid path for unlink operations after parent orphanization"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoext4: test journal checkpoint ioctl
Leah Rumancik [Mon, 7 Jun 2021 21:55:09 +0000 (21:55 +0000)]
ext4: test journal checkpoint ioctl

Test for commit "ext4: add ioctl EXT4_IOC_CHECKPOINT". Tests journal
checkpointing and journal erasing via EXT4_IOC_CHECKPOINT with flag
EXT4_IOC_CHECKPOINT_FLAG_ZEROOUT set.

Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoext4/048: add check for od endian flag
Leah Rumancik [Mon, 7 Jun 2021 21:36:38 +0000 (21:36 +0000)]
ext4/048: add check for od endian flag

od --endian=little is used to read bytes in little endian when on a
big endian machine. Update test ext4/048 to require od endian flag
support on big endian machines, otherwise, skip test.

Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
Reviewed-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/rc: fix _scratch_mkfs_blocksized indentation
Anand Jain [Mon, 7 Jun 2021 12:08:20 +0000 (20:08 +0800)]
common/rc: fix _scratch_mkfs_blocksized indentation

No function change. Fix indentation.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: support other sectorsizes in _scratch_mkfs_blocksized
Anand Jain [Mon, 7 Jun 2021 12:08:19 +0000 (20:08 +0800)]
btrfs: support other sectorsizes in _scratch_mkfs_blocksized

When btrfs supports sectorsize != pagesize it can run these test cases
now,
generic/205 generic/206 generic/216 generic/217 generic/218 generic/220
generic/222 generic/227 generic/229 generic/238

This change is backward compatible for kernels without non pagesize
sectorsize support.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agooverlay/075: add test coverage for clearing immutable/append-only flags
Amir Goldstein [Sun, 6 Jun 2021 15:18:11 +0000 (18:18 +0300)]
overlay/075: add test coverage for clearing immutable/append-only flags

overlay/075 is a variant of check -overlay generic/079.

check -overlay generic/079 does the following operations on overlay fs:
1. Create files
2. Set immutable/append-only flags
3. Verify files/dirs behaving as immutable/append-only
4. Clear immutable/append-only flags
5. Remove files

overlay/075 performs steps 1,2,4,5 on upper and lower layers
and only step 3 is performed on overlay fs (before and after copy up
and mount cycle).

Add also steps 4,5 to be performed on overlay fs to increase the
test coverage of the "merged" inode xflags feature.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/507: support more filesystems
Amir Goldstein [Fri, 11 Jun 2021 13:10:29 +0000 (16:10 +0300)]
generic/507: support more filesystems

The commit message introducing the test says:
"We only check below attribute modification which most filesystem
 supports:
    - no atime updates (A)
    - secure deletion (s)
    - synchronous updates (S)
    - undeletable (u)
"
But in fact, very few filesystems support the (s) and (u) flags.
xfs and btrfs do not support them for example.

The test doesn't need to check those specific flags, so replace those
flags with immutable (i) and append-only (a), which most filesystems
really do support.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/config: remove default 4k blocksize from XFS_MKFS_OPTIONS
Zorro Lang [Tue, 25 May 2021 04:46:41 +0000 (12:46 +0800)]
common/config: remove default 4k blocksize from XFS_MKFS_OPTIONS

xfstests set "-bsize=4k" to XFS_MKFS_OPTIONS by default, then give
it to MKFS_OPTIOPNS. So MKFS_OPTIOPNS always contains "-bsize=4k"
except we set XFS_MKFS_OPTIONS manually.

It's useless to set XFS_MKFS_OPTIONS to "-bsize=4096" by default,
especially that will cause all cases with _scratch_mkfs_blocksized()
always fail as "-b size option respecified", when test on XFS. For
exmaple: generic/222

Signed-off-by: Zorro Lang <zlang@redhat.com>
Acked-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agooverlay/075: fix wrong invocation of t_immutable
Amir Goldstein [Sat, 5 Jun 2021 09:43:26 +0000 (12:43 +0300)]
overlay/075: fix wrong invocation of t_immutable

t_immutable cannot be run twice on the same test directoty, because
append-only directory tests create files in append-only.d and those
file already exist from the first run.

Use separate test directories for the first and second t_immutable runs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs/029: fix the test compatible with older cp(1)
Anand Jain [Fri, 4 Jun 2021 06:25:47 +0000 (14:25 +0800)]
btrfs/029: fix the test compatible with older cp(1)

cp(1) versions 8.30 and 8.32 are compared with its --reflink=always option
and they have different semantic if the target-file (with zero sizes) must be
created when the cp --reflink=alaways fails with a cross-device link
error. As shown below.

$ cp --version | head -1
cp (GNU coreutils) 8.30

$ cp --reflink=always /mnt/scratch/original /mnt/test/test-029/copy
cp: failed to clone '/mnt/test/test-029/copy' from '/mnt/scratch/original': Invalid cross-device link

$ ls -l /mnt/test/test-029/copy
ls: cannot access '/mnt/test/test-029/copy': No such file or directory

$ cp --version | head -1
cp (GNU coreutils) 8.32

$ cp --reflink=always /mnt/scratch/original /mnt/test/test-029/copy;
cp: failed to clone '/mnt/test/test-029/copy' from '/mnt/scratch/original': Invalid cross-device link

$ ls -l /mnt/test/test-029/copy
-rw------- 1 root root 0 Jun  4 13:29 /mnt/test/test-029/copy

Reported-by: Wang Yugui <wangyugui@e16-tech.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoext4: prevent initializing reserved inodes
chenlong [Fri, 28 May 2021 03:13:07 +0000 (11:13 +0800)]
ext4: prevent initializing reserved inodes

Check the block group zero and prevent initializing reserved inodes.
But in some special cases, the reserved inode may not all belong to
the group zero, it may exist into the second group if we formatĀ 
filesystem below.

  mkfs.ext4 -b 4096 -g 8192 -N 1024 -I 4096 /dev/sda

So, it will end up triggering a false positive report of a corrupted
file system

It's a regression test for kernel commit a149d2a5cabb ("ext4: fix
check to prevent false positive report of incorrect used inodes")

Signed-off-by: Chen Long <chenlongcl.chen@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: initial bcachefs support
Kent Overstreet [Tue, 25 May 2021 22:19:53 +0000 (18:19 -0400)]
fstests: initial bcachefs support

To get started, you'll need to get bcachefs from the kernel repository
  https://evilpiepirate.org/git/bcachefs.git/

and tools from
  https://evilpiepirate.org/git/bcache-tools.git/

Build kernel as normal, enabling CONFIG_BCACHEFS_FS (and probably
CONFIG_BCACHEFS_DEBUG), and build and install tools. Then running
fstests is exactly the same as other local filesystems - just set
FSTYP=bcachefs

Also see https://evilpiepirate.org/git/ktest.git/ for a tool for
conveniently building kernel and running fstests inside a qemu VM.

This patch also updates generic/441 to run the more thorough test on
bcachefs, and generic/425 to not run on bcachefs (since bcachefs does
not store xattrs in blocks)

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: improved .gitignore
Kent Overstreet [Tue, 25 May 2021 22:19:52 +0000 (18:19 -0400)]
fstests: improved .gitignore

Ignore dotfiles, tags, and verifier state.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/042: set size to 38M for f2fs
Sun Ke [Tue, 18 May 2021 11:47:00 +0000 (07:47 -0400)]
generic/042: set size to 38M for f2fs

25M is too small for f2fs:

Info: Disable heap-based policy
Info: Debug level = 0
Info: Label =
Info: Trim is enabled
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 51200 (25 MB)
Info: zone aligned segment0 blkaddr: 512
    Error: Device size is not sufficient for F2FS volume
    Error: Failed to prepare a super block!!!
    Error: Could not format the device!!!

So, set size to 38M for f2fs.

Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: add test for multiple fsync with adjacent preallocated extents
Filipe Manana [Mon, 24 May 2021 10:37:21 +0000 (11:37 +0100)]
btrfs: add test for multiple fsync with adjacent preallocated extents

Test a scenario where we do several partial writes into multiple
preallocated extents across two transactions and with several fsyncs
in between. The goal is to check that the fsyncs succeed.

Currently the last fsync fails with an -EIO error, and it aborts the
current transaction. This issue is fixed by a patch with the following
subject:

  "btrfs: fix fsync failure and transaction abort after writes to prealloc extents"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/xfs: Fix _require_scratch_xfs_shrink bug
Yang Xu [Fri, 28 May 2021 09:41:10 +0000 (17:41 +0800)]
common/xfs: Fix _require_scratch_xfs_shrink bug

Since local declaration can't pass function return value, this
helper never detects whether kernel or xfsprogs supports xfs shrink
feature successfully. Fit it by separating declaration and
assignment of local variables.

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs: fix check of whether the mkfs fail
Sun Ke [Fri, 28 May 2021 09:40:46 +0000 (05:40 -0400)]
xfs: fix check of whether the mkfs fail

The return value ("$?") is always the status of the last command in
the pipe.

Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: split out run_test() function
Christian Brauner [Fri, 7 May 2021 15:00:56 +0000 (17:00 +0200)]
idmapped-mounts: split out run_test() function

to make it easier to run subsets of tests.

Cc: fstests@vger.kernel.org
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: add missing newline to print_r()
Christian Brauner [Fri, 7 May 2021 15:00:55 +0000 (17:00 +0200)]
idmapped-mounts: add missing newline to print_r()

The function missed to print a newline making the output difficult to
read when running with DEBUG_TRACE.

Cc: fstests@vger.kernel.org
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: remove unused set_cloexec() helper
Christian Brauner [Fri, 7 May 2021 15:00:54 +0000 (17:00 +0200)]
idmapped-mounts: remove unused set_cloexec() helper

This function has never been used so remove it.

Cc: fstests@vger.kernel.org
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: mmap and copy file data with page overlapping
Zorro Lang [Fri, 21 May 2021 08:01:45 +0000 (16:01 +0800)]
generic: mmap and copy file data with page overlapping

Mmap 2 pages of file, write 64 bytes to the first and second pages,
copy the data from the first page and then second page to the second
page offset with $pagesize - 64. Verify the data at the end.

       +-----------------------+
       |        (copy)         |
       |                       V
    +---------------+---------------+------------
    |AAAA| ........ |AAAA| ... |AAAA|AAAA|
    +---------------+---------------+------------
                       |            ^
                       |   (copy)   |
                       +------------+

This's also a regression test cover kernel commit: 4f06dd92b5d0
("fuse: fix write deadlock")

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4: add test for ext4_dir_entry2 wipe
Leah Rumancik [Mon, 17 May 2021 14:48:49 +0000 (14:48 +0000)]
ext4: add test for ext4_dir_entry2 wipe

Check wiping of dir entry data upon removing a file, converting to an
htree, and splitting htree nodes.

Tests commit 6c0912739699d8e4b6a87086401bf3ad3c59502d ("ext4: wipe
ext4_dir_entry2 upon file deletion").

Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/215: avoid false alert for subpage case
Qu Wenruo [Mon, 17 May 2021 09:29:22 +0000 (17:29 +0800)]
btrfs/215: avoid false alert for subpage case

[BUG]
When running btrfs/215 with 64K page size, 4K sectorsize (subpage RW
support), it fails with the following error:
btrfs/215       [failed, exit status 1]- output mismatch (see ~/xfstests-dev/results//btrfs/215.out.bad)
    --- tests/btrfs/215.out     2021-03-19 16:34:26.069634953 +0800
    +++ ~/xfstests-dev/results//btrfs/215.out.bad      2021-05-17 16:52:34.743514224 +0800
    @@ -1,2 +1,3 @@
     QA output created by 215
    -Silence is golden
    +Errors: 8 expected: 2
    +(see ~/xfstests-dev/results//btrfs/215.full for details)
    ...
    (Run 'diff -u ~/xfstests-dev/tests/btrfs/215.out ~/xfstests-dev/results//btrfs/215.out.bad'  to see the entire diff)

[CAUSE]
For subpage case, btrfs still tries to read the full page, other than
read just one sector for PAGE_SIZE == sectorsize case.

This means for the 2 sectors corrupted case, since they are in the same
page, all the errors will be reported.

[FIX]
Change the following values:
- filesize
  Now it's 8 * pagesize.

- expected error number
  Now it's 2 * sectors_per_page or 6 * sectors_per_page.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoaio-dio-append-write-fallocate-race: fix directio buffer alignment bugs
Darrick J. Wong [Wed, 19 May 2021 23:57:16 +0000 (16:57 -0700)]
aio-dio-append-write-fallocate-race: fix directio buffer alignment bugs

This program fails on filesystems where the stat() block size isn't a
strict power of two because it foolishly feeds that to posix_memalign to
allocate an aligned memory buffer for directio.  posix_memalign requires
the alignment value to be a power of two, so generic/586 fails.

The system page size generally works well for directio buffers, so use
that instead.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/178: fix mkfs success test
Darrick J. Wong [Wed, 19 May 2021 23:57:10 +0000 (16:57 -0700)]
xfs/178: fix mkfs success test

Fix the obviously incorrect code here that wants to fail the test if
mkfs doesn't succeed.  The return value ("$?") is always the status of
the /last/ command in the pipe.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agofsx/fsstress: round blocksize properly
Darrick J. Wong [Wed, 19 May 2021 23:57:04 +0000 (16:57 -0700)]
fsx/fsstress: round blocksize properly

The block sizes reported by stat and DIOINFO aren't required to be
powers of two.  This can happen on an XFS filesystem with a realtime
extent size that isn't a power of two; on such filesystems, certain IO
calls will fail due to alignment issues.  Fix that by providing rounding
helpers that work for all sizes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/117: fix fragility in this fuzz test
Darrick J. Wong [Wed, 19 May 2021 23:56:58 +0000 (16:56 -0700)]
xfs/117: fix fragility in this fuzz test

This fuzz test has some fragility problems -- it doesn't do anything to
guarantee that the inodes that it checks for EFSCORRUPTED are the same
ones that it fuzzed, and it doesn't explicitly try to avoid victimizing
inodes in the same chunk as the root directory.  As a result, this test
fails annoyingly frequently.

Fix both of these problems and get rid of the confusingly named TESTDIR
variable.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs: force file creation to the data device for certain layout tests
Darrick J. Wong [Wed, 19 May 2021 23:56:52 +0000 (16:56 -0700)]
xfs: force file creation to the data device for certain layout tests

I found a bunch more tests in the xfs/ directory that try to create
specific metadata layouts on the data device, either because they're
fuzz tests or because they're testing specific edge cases of the code
base.  Either way, these test need to override '-d rtinherit' in the
MKFS_OPTIONS, so do that with _xfs_force_bdev.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/xfs: refactor commands to select a particular xfs backing device
Darrick J. Wong [Wed, 19 May 2021 23:56:46 +0000 (16:56 -0700)]
common/xfs: refactor commands to select a particular xfs backing device

Refactor all the places where we try to force new file data allocations
to a specific xfs backing device so that we don't end up open-coding the
same xfs_io command lines over and over.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/012: check free size of scratch device before copying files
Sidong Yang [Sun, 23 May 2021 05:08:59 +0000 (05:08 +0000)]
btrfs/012: check free size of scratch device before copying files

This test failed when scratch device don't have enough space for
copying files. This patch gets size of files by du command and
checks if there is enough space in the device.

Signed-off-by: Sidong Yang <realwakka@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs: Add the appropriate _require helper
Jinhui Huang [Mon, 17 May 2021 03:34:13 +0000 (11:34 +0800)]
xfs: Add the appropriate _require helper

1.xfs/162 uses xfs_db -c 'fuzz' but forgets to check if the feature
is supported. This will cause the case to fail on a system without
fuzz support. so we add _require to check if the fuzz is supported.

2.xfs/495 use _require_scratch_xfs_fuzz_fields to check the features
required by field fuzzing, but some of the features are not used in
this case like xfs_scrub, this will cause the case to skip on a system
without xfs_scrub support, even if the features being uesd are supported.
So we just need to use _require to check the features being used.

Signed-off-by: Jinhui Huang <huangjh.jy@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: test log replay after directory fsync and moving a child directory
Filipe Manana [Wed, 12 May 2021 15:27:35 +0000 (16:27 +0100)]
btrfs: test log replay after directory fsync and moving a child directory

Test a particular scenario where we fsync a directory, then move one of
its children directories into another directory and then finally sync the
log trees by fsyncing any other inode. We want to check that after a power
failure we are able to mount the filesystem and that the moved directory
exists only as a child of the directory we moved it into.

This currently fails on a 5.12 kernel (and 5.13-rc1) but is fixed by a
patch with the following subject:

  "btrfs: fix removed dentries still existing after log is synced"

The failure is due to ending up with a directory that has 2 hard links
(two parent directories) as soon as the log replay procedure finishes,
which causes the tree checker to detect the issue and cause the mount
operation to fail with -EIO.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agofsx: fix backwards parameters in complaint about overly long copy
Darrick J. Wong [Wed, 12 May 2021 02:02:19 +0000 (19:02 -0700)]
fsx: fix backwards parameters in complaint about overly long copy

If fsx encounters a situation where copy_file_range reports that it
copied more than it was asked to, we report this as a failure.
Unfortunately, the parameters to the print function are backwards,
leading to this bogus complaint about a short copy:

do_copy_range: asked 28672, copied 24576??

When we really asked to copy 24k but 28k was copied instead.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon: always pass -f to $DUMP_COMPRESSOR
Darrick J. Wong [Wed, 12 May 2021 02:02:08 +0000 (19:02 -0700)]
common: always pass -f to $DUMP_COMPRESSOR

If the test runner gave us the name of a program to use to compress
dumps, always pass -f to overwrite older compressed images, like the
documentation says we do. This prevents the test suite from stalling on
"foo.md.gz exists, overwrite?" prompts.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs: fix old fuzz test invocations of xfs_repair
Darrick J. Wong [Wed, 12 May 2021 02:01:56 +0000 (19:01 -0700)]
xfs: fix old fuzz test invocations of xfs_repair

Some of the older blocktrash-based fuzz tests cause the fs to go down
due to the corrupted image and fail to remount.  Offline repair fails
because _repair_scratch_fs is the helper that is smart enough to call
xfs_repair -L, not _scratch_xfs_repair.  Fix these instances.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/122: add entries for structures added to 5.13
Darrick J. Wong [Wed, 12 May 2021 02:01:45 +0000 (19:01 -0700)]
xfs/122: add entries for structures added to 5.13

Add a new entry for a structure that was changed in 5.13.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs: stress test for shrinking free space in the last AG
Gao Xiang [Tue, 11 May 2021 23:32:28 +0000 (07:32 +0800)]
xfs: stress test for shrinking free space in the last AG

This adds a stress testcase to shrink free space as much as
possible in the last AG with background fsstress workload.

The expectation is that no crash happens with expected output.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>