xfstests-dev.git
2 years agocommon/rc: handle option with argument in _require_xfs_io_command()
Eric Biggers [Mon, 3 Feb 2020 18:18:53 +0000 (10:18 -0800)]
common/rc: handle option with argument in _require_xfs_io_command()

Fix _require_xfs_io_command() to handle options that take arguments when
the argument is shown in the help text.  E.g., it didn't work to run:

_require_xfs_io_command "add_enckey" "-k"

because the relevant line of the help text is:

-k KEY_ID -- ID of fscrypt-provisioning key containing the raw key

... but the grep command only matched "-k --", not "-k KEY_ID --".

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoREADME: fsgqa needs a home directory too
Christian Kujau [Fri, 31 Jan 2020 21:22:49 +0000 (13:22 -0800)]
README: fsgqa needs a home directory too

generic/128 whould fail with:

     QA output created by 128
    +su: warning: cannot change directory to /home/fsgqa: No such file or directory

Adjust README to create a home directory too. And (totally unrelated),
fix the 404 for the udf_test tool.

Signed-off-by: Christian Kujau <lists@nerdbynature.de>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoshared/032: add options for reiser4
Christian Kujau [Fri, 31 Jan 2020 20:55:47 +0000 (12:55 -0800)]
shared/032: add options for reiser4

Last night I had xfstests running and after a while it hung at
shared/032 because I had reiser4progs installed and it was waiting
for input:

-----------------------------------------------------------------
mkfs -t reiser4 /dev/ram0
mkfs.reiser4 1.2.1
Format release: 4.0.2
Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
reiser4progs/COPYING.

Block size 4096 will be used.
Linux 5.5.0-rc5-amd64 is detected.
Uuid 2006a590-832a-49f5-9317-cf91bf866dd7 will be used.
Reiser4 is going to be created on /dev/ram0.
(Yes/No):
-----------------------------------------------------------------

The patch below adds an option (cf. reiserfs the line above) so that
the test continues.

Curiously enough the test still "succeeds" for reiser{fs,4} (and jfs
too) because the userspace tools don't appear to have a routine to
check for "contains an existing filesystem", but that could be
addressed in a different patch.

Signed-off-by: Christian Kujau <lists@nerdbynature.de>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: test unaligned punch hole at ENOSPC
Anand Jain [Fri, 31 Jan 2020 05:09:57 +0000 (13:09 +0800)]
btrfs: test unaligned punch hole at ENOSPC

Try to punch hole with unaligned size and offset when the FS is full
and mounted with nodatacow option.

Mainly holes are punched at locations which are unaligned with the
file extent boundaries when the FS is full by data.  As the punching
holes at unaligned location will involve truncating blocks instead
of just dropping the extents, it shall involve reserving data and
metadata space for delalloc and so data alloc fails as the FS is
full.

btrfs_punch_hole()
 btrfs_truncate_block()
   btrfs_check_data_free_space() <-- ENOSPC

We don't fail punch hole if the holes are aligned with the file
extent boundaries as it shall involve just dropping the related
extents, without truncating data extent blocks.

This test was orignally merged as btrfs/172 (commit 4c2c678cd), then
Iremoved by commit 538d8a4bcc78. But it's decided to bring back this
test case, now the problem is better understood and the fix is
available in the ML as below.

Link: https://patchwork.kernel.org/patch/11357415/
Reviewed-by: Filipe Manana <fdmanana@suse.com>
(cherry picked from commit 4c2c678cd56a81a210cb16f9f9347073e91e2fb0)
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/022: Add debug output
Qu Wenruo [Fri, 7 Feb 2020 01:59:42 +0000 (09:59 +0800)]
btrfs/022: Add debug output

When btrfs/022 fails, its $seqres.full doesn't contain much useful info
to debug.

This patch will add extra debug, including subvolid and full "btrfs
qgroup show" output.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/022: Match qgroup id more correctly
Qu Wenruo [Fri, 7 Feb 2020 01:59:41 +0000 (09:59 +0800)]
btrfs/022: Match qgroup id more correctly

[BUG]
Btrfs/022 sometimes fails with snapshot's reference mismatch with its
source.

[CAUSE]
Since commit fd0830929573 ("fsstress: add the ability to create
snapshots") adds the ability for fsstress to create/delete snapshot and
subvolumes, fsstress will create new subvolumes under test dir.

For example, we could have the following subvolumes created by fsstress:
subvol a id=256
subvol b id=306
qgroupid         rfer         excl
--------         ----         ----
0/5             16384        16384
0/256        13914112        16384
...
0/263         3080192      2306048  << 2 *306* 048
...
0/306        13914112        16384  << 0/ *306

So when we're greping for subvolid 306, it matches qgroup 0/263 first,
which has difference size, and caused false alert.

[FIX]
Instead of greping "$subvolid" blindly, now grep "0/$subvolid" to catch
qgroupid correctly, without hitting rfer/excl values.

Suggested-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: Use word mathcing for _btrfs_get_subvolid()
Qu Wenruo [Fri, 7 Feb 2020 01:59:40 +0000 (09:59 +0800)]
btrfs: Use word mathcing for _btrfs_get_subvolid()

Current _btrfs_get_subvolid() can't handle the following case at all:
  # btrfs subvol list $SCRATCH_MNT
  ID 256 gen 9 top level 5 path subv1
  ID 257 gen 7 top level 256 path subv1/subv2
  ID 258 gen 8 top level 256 path subv1/subv3
  ID 259 gen 9 top level 256 path subv1/subv4

If we call "_btrfs_get_subvolid $SCRATCH_MNT subv1" we will get a list
of all subvolumes, not the subvolid of subv1.

To address this problem, we go egrep to match $name which starts with a
space, and at the end of a line.
So that all other subvolumes won't hit.

Suggested-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofsx: support 64-bit operation counts
Darrick J. Wong [Wed, 5 Feb 2020 00:02:13 +0000 (16:02 -0800)]
fsx: support 64-bit operation counts

Support 64-bit operation counts so that we can run long-soak tests for
more than 2 billion fsxops.

I figured that testcalls and simulatedopcount should both be signed
because numops is also signed.  Granted, I guess numops is signed so
that we can set it to the magic value -1 and have fsx run "forever".

[Eryu: add Darrick's explanation about why we change testcalls to a
signed value.]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/117: fix inode corruption loop
Darrick J. Wong [Wed, 5 Feb 2020 00:02:07 +0000 (16:02 -0800)]
xfs/117: fix inode corruption loop

`seq X Y` will print all numbers between X and Y, including Y.  Since
inode chunks contain inodes numbered from X to X+63, we need to set the
loop variables correctly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/020: fix truncation test
Darrick J. Wong [Wed, 5 Feb 2020 00:01:54 +0000 (16:01 -0800)]
xfs/020: fix truncation test

If we can't create the 60T sparse image for testing repair on a large fs
(such as when running on 32-bit), don't bother running the rest of the
test.  This requires the actual truncate(1) command, because it returns
nonzero if the system call fails.

[Eryu: the original bug was introduced by me when committing the
original patch, which was a correct fix. Sorry about that.. ]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/449: filter out "Discarding..." from output
Darrick J. Wong [Wed, 5 Feb 2020 00:01:48 +0000 (16:01 -0800)]
xfs/449: filter out "Discarding..." from output

xfsprogs 5.4 prints "Discarding..." if the disk supports the trim
command.  Filter this out of the output because xfs_info and friends
won't print that out.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/581: try to avoid flakiness in keys quota test
Eric Biggers [Wed, 29 Jan 2020 00:42:51 +0000 (16:42 -0800)]
generic/581: try to avoid flakiness in keys quota test

generic/581 passes for me, but Murphy Zhou reported that it started
failing for him.  The part that failed is the part that sets the key
quota to the fsgqa user's current number of keys plus 5, then tries to
add 6 filesystem encryption keys as the fsgqa user.  Adding the 6th key
unexpectedly succeeded.

What I think is happening is that because the kernel's keys subsystem
garbage-collects keys asynchronously, the quota may be freed up later
than expected after removing fscrypt keys.  Thus the test is flaky.

It would be nice to fix this in the kernel, but unfortunately there
doesn't seem to be an easy fix, and the keys subsystem has always worked
this way.  And it seems unlikely to cause real-world problems, as the
keys quota really just exists to prevent denial-of-service attacks.

So, for now just try to make the test more reliable by:

(1) Reduce the scope of the modified keys quota to just the part of the
    test that needs it.
(2) Before getting the current number of keys for the purpose of setting
    the quota, wait for any invalidated keys to be garbage-collected.

Tested with a kernel that has a 1 second sleep hacked into the beginning
of key_garbage_collector().  With that, this test fails before this
patch and passes afterwards.

Reported-by: Murphy Zhou <xzhou@redhat.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Acked-by: Murphy Zhou <xzhou@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: add a test for the btrfs file extent gap issue
Josef Bacik [Tue, 4 Feb 2020 14:37:59 +0000 (09:37 -0500)]
btrfs: add a test for the btrfs file extent gap issue

This is a test to validate that we're not adjusting up i_size before
we have the appropriate file extents on disk.  We had a problem
where i_size would be adjusted up without a contiguous range of file
extents, which isn't ok without a special option enabled.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: add test for incremental send for file with shared extents
Filipe Manana [Mon, 3 Feb 2020 16:19:19 +0000 (16:19 +0000)]
btrfs: add test for incremental send for file with shared extents

Test that an incremental send operation works correctly when a file has
shared extents with itself in the send snapshot, with a hole between them,
and the file size has increased in the send snapshot.

This currently fails in 5.5-rc kernels (regression) but is fixed by a
patch that has the following subject:

  Btrfs: send, fix emission of invalid clone operations within the same file

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/527: add additional test including a file with a hardlink
Filipe Manana [Mon, 3 Feb 2020 16:19:28 +0000 (16:19 +0000)]
generic/527: add additional test including a file with a hardlink

Add a similar test to the existing one but with a file that has a
hardlink as well. This is motivated by a bug found in btrfs where
a fsync on a file that has the old name of another file results
in the logging code to hit an infinite loop. The patch that fixes
the bug in btrfs has the following subject:

  "Btrfs: fix infinite loop during fsync after rename operations"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/130: remove the dangerous group
Filipe Manana [Wed, 22 Jan 2020 12:22:33 +0000 (12:22 +0000)]
btrfs/130: remove the dangerous group

As of the linux kernel commit fd0ddbe2509568 ("Btrfs: send, skip
backreference walking for extents with many references"), the test is no
longer dangerous and it's fast (takes 1 to 2 seconds on a modest vm with
a debug kernel). Therefore remove it from the dangerous group and add it
to the auto and quick groups as well.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/402: Make timestamp range check conditional
Deepa Dinamani [Sun, 19 Jan 2020 00:57:44 +0000 (16:57 -0800)]
generic/402: Make timestamp range check conditional

Addition of fs-specific timestamp range checking was added
in 188d20bcd1eb ("vfs: Add file timestamp range support").

Add a check for whether the kernel supports the limits check
before running the associated test.

Based on an off-list discussion, we use a simpler interim approach
until fsinfo syscall would provide fs timestamp limits info.
This isn't perfect, but works for filesystems expiring in 2038.

Suggested-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/020: call _notrun if we can't create a 60t sparse image
Darrick J. Wong [Thu, 16 Jan 2020 05:11:19 +0000 (21:11 -0800)]
xfs/020: call _notrun if we can't create a 60t sparse image

If we can't create the 60T sparse image for testing repair on a large fs
(such as when running on 32-bit), don't bother running the rest of the
test.

[Eryu: use xfs_io instead of truncate]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/108: skip test if we can't initialize scsi_debug
Darrick J. Wong [Thu, 16 Jan 2020 05:11:13 +0000 (21:11 -0800)]
generic/108: skip test if we can't initialize scsi_debug

Correct the logic in this test that detects failed scsi_debug
initializations.  Downgrade the reaction to _notrun since the filesystem
under test did not fail, just our mockup disk.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/rc: fix _get_max_lfs_filesize on 32-bit platforms
Darrick J. Wong [Thu, 16 Jan 2020 05:11:07 +0000 (21:11 -0800)]
common/rc: fix _get_max_lfs_filesize on 32-bit platforms

The 32-bit calculation of the maximum filesize is incorrect.  Replace it
with the formula that the kernel has used since commit 0cc3b0ec23ce
("Clarify (and fix) MAX_LFS_FILESIZE macros").  This fixes a regression
in generic/351 on 32-bit kernels.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/279: skip test if we can't allocate scsi_debug device
Darrick J. Wong [Thu, 16 Jan 2020 05:11:00 +0000 (21:11 -0800)]
xfs/279: skip test if we can't allocate scsi_debug device

Due to the unique structure of xfs/279 running _get_scsi_debug_dev from
a backtick from inside subshell, the "could not get scsi_debug device"
checks do not actually stop the test when modprobe scsi_debug fails.

Therefore, check the value of SCSI_DEBUG_DEV from the subshell and
_notrun the test if we couldn't get memory.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/507: skip if we can't create a large sparse file for testing
Darrick J. Wong [Thu, 16 Jan 2020 05:10:54 +0000 (21:10 -0800)]
xfs/507: skip if we can't create a large sparse file for testing

Skip this test we can't create the large sparse file needed to test
overflows.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/122: add disk dquot structure to the list
Darrick J. Wong [Thu, 16 Jan 2020 05:10:48 +0000 (21:10 -0800)]
xfs/122: add disk dquot structure to the list

Add disk dquot structures to the check list now that we killed the
typedef.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs: add a new test for removing ACLs through the attr interface
Christoph Hellwig [Tue, 7 Jan 2020 16:52:55 +0000 (17:52 +0100)]
xfs: add a new test for removing ACLs through the attr interface

Test that removing the SGI_ACL_FILE attr also removes the cached ACL
used for access control checking.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/520: Remove sync in clean_dir
Yang Xu [Mon, 23 Dec 2019 10:21:39 +0000 (18:21 +0800)]
generic/520: Remove sync in clean_dir

When I test this case on xfs, it may fail as below:
--------------------------------------------
 === link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_MNT/A ===
+umount: /mnt/xfstests/scratch: target is busy.
+        (In some cases useful info about processes that use
+         the device is found by lsof(8) or fuser(1))
---------------------------------------------

It fails because somethings is still using the fs when we call sync and then
try to unmount it. We can simply remove sync as the unmount is supposed to
persist the file/directory removals.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoltp/fsx: drop caches if we're doing closeopen
Josef Bacik [Tue, 7 Jan 2020 16:55:41 +0000 (11:55 -0500)]
ltp/fsx: drop caches if we're doing closeopen

fsx has a closeopen option where it will close and then re-open the
file.  This is handy, but what is really more useful is to drop the file
from cache completely, so add a drop_caches into this operation so that
the file is read back completely from disk to be really evil.

[Eryu: fix drop cache failure number to stay within 190]

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoltp/fsx: do size check after closeopen operation
Josef Bacik [Tue, 7 Jan 2020 16:55:40 +0000 (11:55 -0500)]
ltp/fsx: do size check after closeopen operation

I was running down a i_size problem and was missing the failure until
the next iteration of fsx operations because we do the file size check
_before_ the closeopen operation.  Move it after the closeopen operation
so we can catch problems where the file gets messed up on disk.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: Add check for required keep_size/punch_hole/zero_range/collapse_range
Xiao Yang [Tue, 7 Jan 2020 08:40:53 +0000 (16:40 +0800)]
generic: Add check for required keep_size/punch_hole/zero_range/collapse_range

Tests need the exact operations to reproduce some issues by
--replay-ops so skip tests rather than one operation if a required
operation/flag in tests is not supported.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: test constant ino with nested overlay over non-samefs lower
Amir Goldstein [Mon, 30 Dec 2019 14:14:23 +0000 (16:14 +0200)]
overlay: test constant ino with nested overlay over non-samefs lower

Also test that d_ino of readdir entries and i_ino from /proc/locks are
consistent with st_ino and that inode numbers persist after rename to
new parent, drop caches and mount cycle.

With nested xino configuration, directory st_ino is not persistent and
its st_ino/d_ino/i_ino values are not consistent, so test only non-dir
in this test.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: test constant ino with nested overlay over samefs lower
Amir Goldstein [Mon, 30 Dec 2019 14:14:22 +0000 (16:14 +0200)]
overlay: test constant ino with nested overlay over samefs lower

Also test that d_ino of readdir entries and i_ino from /proc/locks are
consistent with st_ino and that inode numbers persist after rename to
new parent, drop caches and mount cycle.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: test file handles with nested overlay over non-samefs lower
Amir Goldstein [Mon, 30 Dec 2019 14:14:21 +0000 (16:14 +0200)]
overlay: test file handles with nested overlay over non-samefs lower

This is a variant of overlay file handles test for an overlayfs that
is nested over another lower overlayfs on non-samefs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: test file handles with nested overlay over samefs lower
Amir Goldstein [Mon, 30 Dec 2019 14:14:20 +0000 (16:14 +0200)]
overlay: test file handles with nested overlay over samefs lower

This is a variant of overlay file handles test for an overlayfs that
is nested over another lower overlayfs on the same fs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: create the overlay/nested test group
Amir Goldstein [Mon, 30 Dec 2019 14:14:19 +0000 (16:14 +0200)]
overlay: create the overlay/nested test group

For tests that mount an overlayfs over overlayfs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/022: Correct seq number in golden output
Nikolay Borisov [Wed, 8 Jan 2020 10:45:14 +0000 (12:45 +0200)]
btrfs/022: Correct seq number in golden output

The test is merged under index 202 but the golden output contains
201. This makes the test always fail. Correct the number.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: Test unique st_dev;st_ino on non-samefs setup
Amir Goldstein [Sat, 21 Dec 2019 17:04:47 +0000 (19:04 +0200)]
overlay: Test unique st_dev;st_ino on non-samefs setup

Check that files from middle layer on same fs as upper layer
are not allowed to export the real inode st_dev;st_ino.

This is a regression test for kernel commit:
  9c6d8f13e9da ("ovl: fix corner case of non-unique st_dev;st_ino")

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/172: Remove the dead test which we have no plan to fix
Qu Wenruo [Tue, 7 Jan 2020 06:01:43 +0000 (14:01 +0800)]
btrfs/172: Remove the dead test which we have no plan to fix

There is no plan to fix it yet, so remove it.

Cc: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoltp/fsstress: Add header file stdbool.h
Yang Xu [Tue, 7 Jan 2020 06:31:26 +0000 (14:31 +0800)]
ltp/fsstress: Add header file stdbool.h

Compilation failed on Fedora 20 because stdbool.h is not included in
xfs/platform_defs-x86_64.h or xfs/linux.h on Fedora 20.

Also, yang xiao fixed similar problem(commit 234f51ebbd) for fsx.c in
2016.2, but after that, fsstress.c started to use bool variable without
including stdbool.h file. It may fail on old linux distributions, so
fix it.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: regression test for subvol deletion after rename
Josef Bacik [Fri, 20 Dec 2019 14:40:24 +0000 (09:40 -0500)]
btrfs: regression test for subvol deletion after rename

Test removal of a subvolume via rmdir after it has been renamed into a
snapshot of the volume that originally contained the subvolume
reference.

This currently fails on btrfs but is fixed by the patch with the title

  "btrfs: fix invalid removal of root ref"

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofsx: fix range overlap check
Darrick J. Wong [Tue, 24 Dec 2019 04:41:06 +0000 (20:41 -0800)]
fsx: fix range overlap check

On 32-bit systems, the offsets are 'unsigned long' (32-bit) which means
that we must cast the explicitly to unsigned long long before feeding
them to llabs.  Without the type conversion we fail to sign-extend the
llabs parameter, try to make a copy/clone/dedupe call with overlapping
ranges, and fsx aborts and the test fails.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/590: skip test if kernel doesn't support realtime
Yang Xu [Thu, 26 Dec 2019 06:48:36 +0000 (14:48 +0800)]
generic/590: skip test if kernel doesn't support realtime

IMHO, if kernel doesn't supprt realtime, we should skip test.
So add it. Also, when we use _scratch_mkfs on xfs, we will get
the following error:
mkfs failed with extra mkfs options added to "-bsize=4096" by test 590 **

This failure occurs because we have used "export XFS_MKFS_OPTIONS=
${XFS_MKFS_OPTIONS:=-bsize=4096}" in common/config, we don't need to
set it again in extra_options, so remove it.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofstests: verify that xfs_growfs can operate on mounted device node
Eric Sandeen [Tue, 17 Dec 2019 22:34:07 +0000 (16:34 -0600)]
fstests: verify that xfs_growfs can operate on mounted device node

The ability to use a mounted device node as the primary argument
to xfs_growfs was added back in with:
  7e8275f8 xfs_growfs: allow mounted device node as argument
because it was an undocumented behavior that some userspace depended on.
This test exercises that functionality.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/529: use an ACL that doesn't confuse NFS
J. Bruce Fields [Thu, 19 Dec 2019 22:33:36 +0000 (17:33 -0500)]
generic/529: use an ACL that doesn't confuse NFS

For historical reasons having to do with Solaris ACL behavior, the Linux
client treats an ACL like the one used as an example here as equivalent
to a mode, causing listxattr to report that no ACL is set on the file.

(See the comment at the top of fs/nfs_common/nfsacl.c in the kernel
source for details, and the "bogus ACL_MASK entry" comment in the same
source file.)  This causes a spurious generic/529 failure on NFS.

As far as I can tell any ACL should trigger the original XFS problem.
So, modify it so as not to hit this odd NFS corner case.

Signed-off-by: J. Bruce Fields <bfields@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/139: require 2GB scratch dev
Johannes Thumshirn [Fri, 3 Jan 2020 11:29:05 +0000 (12:29 +0100)]
btrfs/139: require 2GB scratch dev

In my testing on 1GB zram devices btrfs/139 usually fails with
ENOSPC.

Add a requirement for 2GB scratch devices (empirically measured).

Signed-off-by: Johannes Thumshirn <jth@kernel.org>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/14[01]: Use proper helper to get both devid and physical for corruption
Qu Wenruo [Thu, 12 Dec 2019 08:31:23 +0000 (16:31 +0800)]
btrfs/14[01]: Use proper helper to get both devid and physical for corruption

[BUG]
With btrfs-progs v5.4, btrfs/140 and btrfs/141 will fail.

[CAUSE]
Both tests are testing re-silvering of RAID1, thus they need to corrupt
on-disk data.

This requires to do manual logical -> physical bytes mapping in the test
case.
However the test case itself uses too many hard coded helper to grab
physical offset, which will change with mkfs.btrfs.

[FIX]
Use more flex helper, to get both devid and physical for such
corruption.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/dmerror: always try to resume device
Darrick J. Wong [Wed, 11 Dec 2019 05:38:31 +0000 (21:38 -0800)]
common/dmerror: always try to resume device

When we're reloading the error-target dm table, always resume the device
even if the reload fails because (a) we shouldn't leave dm state for the
callers to clean up and (b) the caller don't clean up the state which
just leads to the scratch filesystem hanging on the suspended dm error
device.

Resume the dm device when we're cleaning up so that cleaning up the
scratch filesystem won't hang.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: verify ciphertext of IV_INO_LBLK_64 encryption policies
Eric Biggers [Mon, 2 Dec 2019 23:01:55 +0000 (15:01 -0800)]
generic: verify ciphertext of IV_INO_LBLK_64 encryption policies

Verify ciphertext for v2 encryption policies that use the IV_INO_LBLK_64
flag and use AES-256-XTS to encrypt file contents and AES-256-CTS-CBC to
encrypt file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/encrypt: support verifying ciphertext of IV_INO_LBLK_64 policies
Eric Biggers [Mon, 2 Dec 2019 23:01:54 +0000 (15:01 -0800)]
common/encrypt: support verifying ciphertext of IV_INO_LBLK_64 policies

Update _verify_ciphertext_for_encryption_policy() to support encryption
policies with the IV_INO_LBLK_64 flag set.

This flag modifies the encryption to include the inode number in the IVs
and to use a key derived from the tuple [master_key, fs_uuid, mode_num].
Since the file nonce is *not* included in this key derivation, multiple
files can use the same key.

This flag is supported by v2 encryption policies only -- not by v1.

Signed-off-by: Eric Biggers <ebiggers@google.com>
2 years agocommon/encrypt: create named variables for UAPI constants
Eric Biggers [Mon, 2 Dec 2019 23:01:53 +0000 (15:01 -0800)]
common/encrypt: create named variables for UAPI constants

Use named variables rather than hard-coded numbers + comments.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofscrypt-crypt-util: add HKDF context constants
Eric Biggers [Mon, 2 Dec 2019 23:01:52 +0000 (15:01 -0800)]
fscrypt-crypt-util: add HKDF context constants

Use #defines rather than hard-coded numbers + comments.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofscrypt-crypt-util: create key_and_iv_params structure
Eric Biggers [Mon, 2 Dec 2019 23:01:51 +0000 (15:01 -0800)]
fscrypt-crypt-util: create key_and_iv_params structure

In preparation for adding 3 more input parameters to get_key_and_iv(),
create a structure to hold the input parameters so that the code doesn't
get too unwieldy.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/386: fill missing brackets in awk
Yang Xu [Mon, 30 Dec 2019 02:57:57 +0000 (10:57 +0800)]
generic/386: fill missing brackets in awk

Since commit ea726c4a55c98 to check the correct field from df output,
it introduced a syntax error. Fix it.

Reported-by: Feiyu Zhu <zhufy.jy@cn.fujitsu.com>
Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/15[78]: Use proper helper to get both devid and physical offset for corruption
Qu Wenruo [Wed, 11 Dec 2019 10:40:29 +0000 (18:40 +0800)]
btrfs/15[78]: Use proper helper to get both devid and physical offset for corruption

[BUG]
When using btrfs-progs v5.4, btrfs/157 and btrfs/158 will fail:

btrfs/157 1s ... - output mismatch (see xfstests/results//btrfs/157.out.bad)
    --- tests/btrfs/157.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/157.out.bad
2019-12-10 15:35:43.112390076 +0000
    @@ -1,9 +1,9 @@
     QA output created by 157
     wrote 131072/131072 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    +wrote 65536/65536 bytes at offset 22020096
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    ...
    (Run 'diff -u xfstests/tests/btrfs/157.out xfstests/results//btrfs/157.out.bad'  to see the entire diff)
btrfs/158 2s ... - output mismatch (see xfstests/results//btrfs/158.out.bad)
    --- tests/btrfs/158.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/158.out.bad
2019-12-10 15:35:44.844388521 +0000
    @@ -1,9 +1,9 @@
     QA output created by 158
     wrote 131072/131072 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    +wrote 65536/65536 bytes at offset 22020096
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    ...
    (Run 'diff -u xfstests/tests/btrfs/158.out xfstests/results//btrfs/158.out.bad'  to see the entire diff)

[CAUSE]
This two tests use physical offset as golden output, while mkfs.btrfs
can do whatever it likes to arrange its chunk layout, thus physical
offset is never reliable.

And btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") just changed the layout.

So the output mismatch and failed.

[FIX]
In fact, that btrfs-progs commit not only changed offset, but also the
device sequence.

So we can't just simply remove the physical offset, but also need to use
proper helper to get both devid (as its device path) and physical offset
for corruption.

As long as mkfs.btrfs still uses sequential devid, these tests should
handle future chunk layout change without problem.

Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/14[23]: Use proper help to get both devid and physical offset for corruption.
Qu Wenruo [Wed, 11 Dec 2019 10:40:28 +0000 (18:40 +0800)]
btrfs/14[23]: Use proper help to get both devid and physical offset for corruption.

[BUG]
When using btrfs-progs v5.4, btrfs/142 and btrfs/143 will fail:
btrfs/142 1s ... - output mismatch (see xfstests/results//btrfs/142.out.bad)
    --- tests/btrfs/142.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/142.out.bad
2019-12-10 15:35:40.280392626 +0000
    @@ -3,37 +3,37 @@
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
     wrote 65536/65536 bytes
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    ...
    (Run 'diff -u xfstests/tests/btrfs/142.out xfstests/results//btrfs/142.out.bad' to see the entire diff)

[CAUSE]
Btrfs/14[23] test whether a read on corrupted stripe will re-silver
itself.
Such test by its nature will need to modify on-disk data, thus need to
get the btrfs logical -> physical mapping, which is done by near
hard-coded lookup function, which rely on certain stripe:devid sequence.

Recent btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") changes how we use devices in mkfs.btrfs,
this caused a change in chunk layout, and break the hard-coded
stripe:devid sequence.

[FIX]
This patch will do full devid and physical offset lookup, instead of old
physical offset only lookup.

The only assumption made is, mkfs.btrfs assigns devid sequentially for
its devices.
Which means, for "mkfs.btrfs $dev1 $dev2 $dev3", we get devid 1 for $dev1,
devid 2 for $dev2, and so on.

This change will allow btrfs/14[23] to handle even future chunk layout
change. (Although I hope this will never happen again).

This also addes extra debug output (although less than 10 lines) into
$seqres.full, just in case when layout changes and current lookup can't
handle it, developer can still pindown the problem easily.

Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon: Use more accurate kernel config for _require_fail_make_request
Qu Wenruo [Wed, 11 Dec 2019 10:40:27 +0000 (18:40 +0800)]
common: Use more accurate kernel config for _require_fail_make_request

Just enabling CONFIG_FAIL_MAKE_REQUEST will not fulfill
_require_fail_make_request.

It's CONFIG_FAULT_INJECTION_DEBUG_FS.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agosrc/fssum: skip subvolumes when building a sum
Josef Bacik [Tue, 17 Dec 2019 15:25:05 +0000 (10:25 -0500)]
src/fssum: skip subvolumes when building a sum

With the snapshot/subvolume support added to fsstress I've been seeing
random failures with our send/receive related tests.  This is because
fssum is summing the path with the subvolumes for our test fs'es that
are generated by fsstress.  But with send/receive it skips subvolumes,
which makes the sums mismatch.  Fix this by skipping directories that do
not match our st_dev, which is how we differentiate subvolumes in btrfs.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofsstress: allow operations to use either a directory or subvol
Josef Bacik [Thu, 14 Nov 2019 15:58:36 +0000 (10:58 -0500)]
fsstress: allow operations to use either a directory or subvol

Most operations are just looking for a base directory to generate a file
in, they don't actually need a directory specifically.  Add FT_ANYDIR to
cover both directories and subvolumes, and then use this in all the
places where it makes sense.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofsstress: add the ability to create snapshots
Josef Bacik [Thu, 14 Nov 2019 15:58:35 +0000 (10:58 -0500)]
fsstress: add the ability to create snapshots

Snapshots are just fancy subvolumes, add this ability so we can stress
snapshot creation.  We get the deletion with SUBVOL_DELETE.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofsstress: add the ability to create/delete subvolumes
Josef Bacik [Thu, 14 Nov 2019 18:14:15 +0000 (13:14 -0500)]
fsstress: add the ability to create/delete subvolumes

This patch adds support to fsstress for creating and deleting subvolumes
on a btrfs file system.  We link in the libbtrfsutil library to handle
the mechanics of creating and deleting subvolumes instead of duplicating
the ioctl logic.  There is code to check if we're on a btrfs fs at
startup time and if so 0 out the frequency of the btrfs specific
operations.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/386: check the correct field from df output
Theodore Ts'o [Sun, 22 Dec 2019 21:27:30 +0000 (16:27 -0500)]
generic/386: check the correct field from df output

The generic/386 test was checking the "Available" field when it
should have been checking the "1k-blocks" field, which represents
the project quota's hard limit.  On xfs, an empty directory takes no
space, so it doesn't matter.  But for ext4, an empty directory still
takes 4k (or whatever the file system's block size happens to be):

Filesystem           1K-blocks       Used  Available  Use% Pathname
/dev/vdc                512000          4     511996    0% /vdc/test

This causes generic/386 to falsely fail.

This fix is needed to fix generic/386 from failing on ext4:

    hard limit 524283904 bytes, expected 524288000

[Eryu: Used $(NF-4) instead of $2 as Eric Sandeen and Yang Xu
suggested]

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/rc: fix device size parser
Dennis Zhou [Wed, 11 Dec 2019 20:18:30 +0000 (12:18 -0800)]
common/rc: fix device size parser

If you have multiple devices that share the same regex (eg dm-1, dm10),
then _get_device_size() can return "$size\n$size" which causes the
following error for btrfs/011.

    QA output created by 011
    ./common/rc: line 3084: [: too many arguments

So, fix this by making grep check against whole word.

Signed-off-by: Dennis Zhou <dennis@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoRevert "fstests: Don't use gawk's strtonum"
Eryu Guan [Sun, 15 Dec 2019 16:10:55 +0000 (00:10 +0800)]
Revert "fstests: Don't use gawk's strtonum"

This reverts commit 37520a314bd472ed720ed0611c6b69e418be9b61.

This commit has been reported to regress, at least, xfs/139 and
btrfs/09[58]. Let's revert it for now.

Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/187: require 8GB scratch dev
Johannes Thumshirn [Thu, 12 Dec 2019 07:45:43 +0000 (08:45 +0100)]
btrfs/187: require 8GB scratch dev

In my testing on 1GB zram devices btrfs/187 usually fails with
ENOSPC.

Add a requirement for 8GB scratch devices (empirically measured).

Cc: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/590: add _require_xfs_io_command "falloc"
Luis Henriques [Mon, 9 Dec 2019 17:14:52 +0000 (17:14 +0000)]
generic/590: add _require_xfs_io_command "falloc"

Without this check the test will fail with message:

  fallocate: Operation not supported

Instead, with this _require_xfs_io_command, the test will be skipped
instead if falloc isn't supported.

Signed-off-by: Luis Henriques <lhenriques@suse.com>
Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test splice() with pipes
Darrick J. Wong [Wed, 4 Dec 2019 02:37:24 +0000 (18:37 -0800)]
generic: test splice() with pipes

Andreas Gr├╝nbacher reports that on the two filesystems that support
iomap directio, it's possible for splice() to return -EAGAIN (instead of
a short splice) if the pipe being written to has less space available in
its pipe buffers than the length supplied by the calling process.

This is a regression test to check for correct operation. Kernel
needs commit 3253d9d09337 ("splice: only read in as much information
as there is pipe buffer space") to pass.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/148: sort attribute list output
Darrick J. Wong [Wed, 4 Dec 2019 02:36:42 +0000 (18:36 -0800)]
xfs/148: sort attribute list output

Yang Xu reported a test failure in xfs/148 that I think comes from
extended attributes being returned in a different order than they were
set.  Since order isn't important in this test, sort the output to make
it consistent.

Reported-by: Yang Xu <xuyang2018.ky@cn.fujitsu.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofstests: Don't use gawk's strtonum
Kusanagi Kouichi [Mon, 2 Dec 2019 09:43:01 +0000 (18:43 +0900)]
fstests: Don't use gawk's strtonum

It's a gawk extension and not needed. Make tests pass with non-gawk.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: truncating mixed written/unwritten XFS realtime extent
Omar Sandoval [Tue, 3 Dec 2019 23:51:52 +0000 (15:51 -0800)]
generic: truncating mixed written/unwritten XFS realtime extent

The only XFS-specific part of this test is the setup, so we can make
the rest a generic test. It's slow, though, as it needs to write 8GB
to convert a big unwritten extent to written.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/{075,122}: Use $RESULT_DIR instead of $here
Kusanagi Kouichi [Tue, 3 Dec 2019 12:55:54 +0000 (21:55 +0900)]
generic/{075,122}: Use $RESULT_DIR instead of $here

If $here is not writable, tests fail.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/rc: Prevent _scale_fsstress_args from consuming -n
Kusanagi Kouichi [Tue, 3 Dec 2019 12:10:47 +0000 (21:10 +0900)]
common/rc: Prevent _scale_fsstress_args from consuming -n

If first argument is -n, echo consumes it.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/386: Fix _filter_quota_rpt
Kusanagi Kouichi [Sun, 1 Dec 2019 12:52:58 +0000 (21:52 +0900)]
generic/386: Fix _filter_quota_rpt

It outputs nothing because of awk errors.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test mount move semantics
Zorro Lang [Sat, 30 Nov 2019 06:53:24 +0000 (14:53 +0800)]
generic: test mount move semantics

This's a functional test case for mount --move operation, it verifies
below semantics:

  ---------------------------------------------------------------------------
  |         MOVE MOUNT OPERATION                                            |
  |**************************************************************************
  |source(A)->| shared       |       private  |       slave    | unbindable |
  | dest(B)  |               |                |                |            |
  |   |      |               |                |                |            |
  |   v      |               |                |                |            |
  |**************************************************************************
  |  shared  | shared        |     shared     | shared & slave |  invalid   |
  |          |               |                |                |            |
  |non-shared| shared        |      private   |      slave     | unbindable |
  ***************************************************************************
  NOTE: moving a mount residing under a shared mount is invalid.

This case uses fsstress to produce a little random load, to make
sure basic operations won't break the the moved mountpoints.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test journal/log replay when a file with cloned extents was fsync'ed
Filipe Manana [Fri, 29 Nov 2019 15:15:09 +0000 (15:15 +0000)]
generic: test journal/log replay when a file with cloned extents was fsync'ed

Test that if we clone part of an extent from a file to itself at a
different offset, fsync it, rewrite (COW) part of the extent from the
former offset, fsync it again, power fail and then mount the filesystem,
we are able to read the whole file and it has the correct data.

This is motivated by a bug found in btrfs which is fixed by a kernel patch
that has the following subject:

  "Btrfs: fix missing data checksums after replaying a log tree"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/551: Adjust the size argument for aio-dio-write-verify
Masayoshi Mizuma [Mon, 18 Nov 2019 17:40:37 +0000 (12:40 -0500)]
generic/551: Adjust the size argument for aio-dio-write-verify

generic/551 sometimes fails because it's killed by OOM killer.
That is because aio-dio-write-verify, which is called by
generic/551, tries to allocate memory even though the total
allocation size exceeds the available memory.

aio-dio-write-verify allocates memory according to the 'size'
argument, and generic/551 passes the argument.

Stop adding the argument when the total size is exceeds
the available memory.

Signed-off-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test unwritten extent conversion extent mapping quota accounting
Darrick J. Wong [Wed, 13 Nov 2019 02:44:50 +0000 (18:44 -0800)]
generic: test unwritten extent conversion extent mapping quota accounting

Regression test to ensure that dquots are attached to the inode when
we're performing unwritten extent conversion after a directio write
and the extent mapping btree splits.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: test race between appending AIO DIO and fallocate
Darrick J. Wong [Wed, 13 Nov 2019 02:44:16 +0000 (18:44 -0800)]
generic: test race between appending AIO DIO and fallocate

Dave Chinner reports[1] that an appending AIO DIO write to the second
block of a zero-length file and an fallocate request to the first block
of the same file can race to set isize, with the user-visible end result
that the file size is set incorrectly to one block long.  Write a small
test to reproduce the results.

[1] https://lore.kernel.org/linux-xfs/20191029100342.GA41131@bfoster/T/

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofstests: Allow user to keep $seqres.dmesg for all tests
Qu Wenruo [Fri, 29 Nov 2019 04:57:43 +0000 (12:57 +0800)]
fstests: Allow user to keep $seqres.dmesg for all tests

Currently fstests will remove $seqres.dmesg if nothing wrong
happened.  It saves some space, but sometimes it may not provide
good enough history for developers to check.

For example, some unexpected dmesg from fs, but not serious enough
to be caught by current filter.

So instead of deleting the ordinary $seqres.dmesg, provide a new
config: KEEP_DMESG, to allow user to keep the dmesg by setting it to
"yes".

The default value for it is "no", which keeps the existing behavior
by deleting ordinary dmesg.

[Eryu: change it to a "yes"/"no" switch.]

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/050: fix xfsquota configuration failures
Darrick J. Wong [Wed, 27 Nov 2019 16:34:57 +0000 (08:34 -0800)]
generic/050: fix xfsquota configuration failures

The new 'xfsquota' configuration for generic/050 doesn't filter out
SCRATCH_MNT properly and seems to be missing an error message in the
golden output.  Fix both of these problems.

Fixes: e088479871 ("generic/050: Handle xfs quota special case with different output")
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/191-input-validation:Fix issue that the test takes too long
Baihua Lu [Wed, 27 Nov 2019 10:41:31 +0000 (10:41 +0000)]
xfs/191-input-validation:Fix issue that the test takes too long

After this test is passed, checking filesystems will execute. Then
xfs_logprint will be failed and xfs_repair will take a long time
because there is not really creating the file system in this test
and wipefs executes before this test. So there is no need checking
filesystems.

Signed-off-by: Baihua Lu <lubaihua0331@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/558: require scratch device
Darrick J. Wong [Wed, 27 Nov 2019 04:16:02 +0000 (20:16 -0800)]
generic/558: require scratch device

This test formats the scratch device, so require it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: test fsync after hole punching when using the no-holes feature
Filipe Manana [Tue, 19 Nov 2019 12:07:34 +0000 (12:07 +0000)]
btrfs: test fsync after hole punching when using the no-holes feature

Test that when we have the no-holes feature enabled and a specific
metadata layout, if we punch a hole that starts at file offset 0 and
fsync the file, after replaying the log the hole exists.

This currently fails on btrfs but is fixed by a patch for the linux
kernel with the following subject:

 "Btrfs: fix missing hole after hole punching and fsync when using NO_HOLES"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs/085: check flakey device instead of backend device
Naohiro Aota [Wed, 20 Nov 2019 06:44:06 +0000 (15:44 +0900)]
btrfs/085: check flakey device instead of backend device

btrfs/085 mount and works on $FLAKEY_DEV, but it's inspecting the
underlying $SCRATCH_DEV. Since writing to dm-flakey does not
invalidate page cache of $SCRATCH_DEV, the btrfs command can read a
stalled data from page cache. We should check the flakey device to
avoid such cache inconsistency.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoext4/031: use _try_scratch_mount instead of _scratch_mount
Xiaoli Feng [Fri, 15 Nov 2019 14:42:48 +0000 (22:42 +0800)]
ext4/031: use _try_scratch_mount instead of _scratch_mount

Here will check the return code of mount option. So update it
to use _try_scratch_mount.

Signed-off-by: Xiaoli Feng <xifeng@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/563: fix the wrong case number in golden image
Zorro Lang [Thu, 14 Nov 2019 14:01:38 +0000 (22:01 +0800)]
generic/563: fix the wrong case number in golden image

The first line of generic/563.out is wrong, the number should be
563, not 011. This mismatch will always cause testing fail.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay: support timestamp range check
Amir Goldstein [Mon, 11 Nov 2019 07:40:10 +0000 (09:40 +0200)]
overlay: support timestamp range check

Overlayfs timestamp range is the same as base fs timestamp range

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Acked-by: Deepa Dinamani <deepa.kernel@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/050: Consistently remove traling dot from umount output
Jan Kara [Mon, 11 Nov 2019 14:49:53 +0000 (15:49 +0100)]
generic/050: Consistently remove traling dot from umount output

We did not consistently remove trailing dot from umount output which
can presumably lead to false failures with particular versions of
util-linux. Make sure all umount output is properly filtered.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/050: Handle xfs quota special case with different output
Jan Kara [Mon, 11 Nov 2019 14:49:52 +0000 (15:49 +0100)]
generic/050: Handle xfs quota special case with different output

Instead of faking output for the case of XFS with quotas, just use a
different output file with appropriate output.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/235: Fix false failure on ext2
Jan Kara [Fri, 1 Nov 2019 11:00:46 +0000 (12:00 +0100)]
generic/235: Fix false failure on ext2

The test gives false failure on ext2 filesystem as 64k file already
has indirect block and so space usage does not exactly match
expectation.

The test really needs to verify only whether quota accounting got
reenabled so just test using creating another empty file which is
not prone to these problems.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/382: Make test independent of fs allocation
Jan Kara [Fri, 1 Nov 2019 11:00:45 +0000 (12:00 +0100)]
generic/382: Make test independent of fs allocation

Test generic/382 depends on filesystem allocating exactly 30M of blocks
when writing 30M file. This is not true for some filesystems - e.g. for
ext2 due to indirect blocks - while leads to false positive failures.
In this case, the test is not actually interested in comparing exact
usage, rather in verifying the ability to write 30M worth of data. So
instead of comparing 'xfs_quota report' output, just depend on detecting
error when writing files.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/050: Fix test failure for filesystems without journal
Jan Kara [Tue, 5 Nov 2019 13:19:21 +0000 (14:19 +0100)]
generic/050: Fix test failure for filesystems without journal

Filesystems without journal can happily mount unrecovered filesystem
read-only which confuses this test. Handle this by providing
different expected output for filesystems without journal.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/quota: Fix _qmount_options
Jan Kara [Tue, 5 Nov 2019 13:43:20 +0000 (14:43 +0100)]
common/quota: Fix _qmount_options

_qmount_options didn't properly substitute usrjquota and grpjquota
options. Fix the regular expression.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agofsstress: show the real file id and parid in rmdir() and unlink()
kaixuxia [Tue, 5 Nov 2019 02:41:26 +0000 (10:41 +0800)]
fsstress: show the real file id and parid in rmdir() and unlink()

The source file id and parentid are overwritten by del_from_flist()
call, so should show the actually values.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agobtrfs: test that send can issue clone operations within the same file
Filipe Manana [Tue, 5 Nov 2019 14:57:00 +0000 (14:57 +0000)]
btrfs: test that send can issue clone operations within the same file

Verify that both full and incremental send operations can issue
clone operations when a file has extents that are shared with itself
(at different offsets of course).

This currently fails on btrfs but is addressed by a patch for the
kernel titled:

  "Btrfs: send, allow clone operations within the same file"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon: add helper to count number of exclusive extents in a file
Filipe Manana [Tue, 5 Nov 2019 14:56:40 +0000 (14:56 +0000)]
common: add helper to count number of exclusive extents in a file

Add a new helper that is similar to _count_extents() except that
extents that are shared several times by the file with itself
(reflinked at different file offsets) are accounted 1 time only,
instead of N times.

This is motivated by a subsequent test for btrfs that will use this
new helper, to verify that send streams are issuing reflink
operations.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon: open files in ro mode for extent and hole count helpers
Filipe Manana [Tue, 5 Nov 2019 14:56:22 +0000 (14:56 +0000)]
common: open files in ro mode for extent and hole count helpers

The helper functions _count_extents() and _count_holes() open their
input file in RW mode to call fiemap, however opening it in RO mode
is enough.  By opening them in RW mode it also makes it not possible
to use them against files residing in btrfs readonly snapshots for
example.

So just open the files in RO mode in these functions.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agooverlay/066: adjust test file size && add more test patterns
Chengguang Xu [Thu, 7 Nov 2019 02:59:41 +0000 (10:59 +0800)]
overlay/066: adjust test file size && add more test patterns

Making many small holes in 10M test file seems not very
helpful for test coverage and it takes too much time on
creating test files. In order to improve test speed we
adjust test file size to (10 * iosize) for iosize aligned
hole files and meanwhile add more test patterns for small
random holes and small empty file.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Chengguang Xu <cgxu519@mykernel.net>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs/207: explicitly test for xflag character
Eric Sandeen [Thu, 7 Nov 2019 15:07:26 +0000 (09:07 -0600)]
xfs/207: explicitly test for xflag character

With a recent change to xfs_io[1], the fsxattr.xflags output line
has added another flag column, so xfs/207 starts failing.

Rather than testing for an exact set of flags, test whether the
specific flag we are interested in is set or unset.

[1] xfs_io/lsattr: expose FS_XFLAG_HASATTR flag

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agoxfs: make sure the kernel and repair tools catch bad names
Darrick J. Wong [Tue, 29 Oct 2019 22:29:48 +0000 (15:29 -0700)]
xfs: make sure the kernel and repair tools catch bad names

Make sure we actually catch bad names in the kernel.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric/317: Use relative paths to avoid traversal permission problems
Zhihao Cheng [Thu, 31 Oct 2019 09:19:17 +0000 (17:19 +0800)]
generic/317: Use relative paths to avoid traversal permission problems

generic/317 will fail because execvp(cmd) is executed without permission,
where cmd is '$here/src/lstat64 $file', which is called by
  $here/src/nsexec -s -U -M "0 $qa_user_id 1000" -G "0 $qa_user_id 1000"\
  $here/src/lstat64 $file

So, you will see following output:
  From user_ns
  ...
  +execvp: Permission denied

nsexec runs the instruction '$here/src/lstat64 $file' as a regular user,
the regular user may not have permission to access path in '$here'.

Actually, it has been fixed in 4818302fbf ("xfstests: generic/317 use
relative paths..."), which then been modified by b7cecbea22 ("fstests:
Add path $here before src/<file>").

Fixes: b7cecbea22 ("fstests: Add path $here before src/<file>")
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agocommon/rc: generalize _get_filesize()
Chao Yu [Fri, 1 Nov 2019 10:28:21 +0000 (18:28 +0800)]
common/rc: generalize _get_filesize()

There are some testcases use below two kind of commands to get file
size, generalize the second way as global function _get_filesize()
to simply codes.

1. ls -l $1 | $AWK_PROG '{print $5}'
2. stat -c %s $1

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
2 years agogeneric: handle fs.verity.require_signatures being enabled
Eric Biggers [Fri, 1 Nov 2019 23:22:19 +0000 (16:22 -0700)]
generic: handle fs.verity.require_signatures being enabled

Most of the fs-verity tests fail if the fs.verity.require_signatures
sysctl has been set to 1.  Update them to set this sysctl to 0 at the
beginning of the test and restore it to its previous value at the end.

generic/577 intentionally sets this sysctl to 1.  Make it restore the
previous value at the end of the test rather than assuming it was 0.

Also simplify _require_fsverity_builtin_signatures() to just check for
the presence of the file /proc/sys/fs/verity/require_signatures rather
than check whether the fs-verity keyring is listed in /proc/keys.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>