Ritesh Harjani [Thu, 11 Feb 2021 03:31:44 +0000 (09:01 +0530)]
check: add CLI option to repeat and stop tests in case of failure
Currently with -i <n> option the test can run for many iterations,
but in case if we want to stop the iteration in case of a failure,
it is much easier to have such an option which could check the
failed status and stop the test from further proceeding.
This patch adds such an option (-I <n>) thereby extending the -i <n>
option functionality.
Qu Wenruo [Thu, 25 Feb 2021 05:57:17 +0000 (13:57 +0800)]
btrfs: make sure we rescan all devices after unregistering
There are some btrfs test cases utilizing
_btrfs_forget_or_module_reload() to unregister all btrfs devices.
However _btrfs_forget_or_module_reload() will unregister all devices,
meaning if TEST_DEV is part of a multi-device btrfs, after those test
cases TEST_DEV will no longer be mountable.
This patch will introduce a new function, btrfs_rescan_devices() to undo
the unregister, so that all later test cases can mount TEST_DEV without
any problem.
Since we are here, also add a missing
_require_btrfs_forget_or_module_loadable for btrfs/225.
Filipe Manana [Tue, 16 Feb 2021 11:10:15 +0000 (11:10 +0000)]
btrfs: clone a hole post eof when using NO_HOLES feature
Test that when using the NO_HOLES feature, if we truncate down a
file, clone a file range covering only a hole into an offset beyond
the current file size, and then fsync the file, after a power
failure we get the expected file content and we do not get stale
data corresponding to file extents that existed before truncating
the file.
This currently fails on btrfs and is fixed by commit 3660d0bcdb82
("btrfs: fix stale data exposure after cloning a hole with NO_HOLES
enabled")
[Eryu: add the commit id of the patch fixing the bug]
Darrick J. Wong [Wed, 10 Feb 2021 02:56:47 +0000 (18:56 -0800)]
check: run tests in exactly the order specified
Introduce a new --exact-order switch to disable all sorting, filtering
of repeated lines, and shuffling of test order. The goal of this is to
be able to run tests in a specific order, namely to try to reproduce
test failures that could be the result of a -r(andomize) run getting
lucky.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Wed, 10 Feb 2021 02:56:42 +0000 (18:56 -0800)]
check: don't abort on non-existent excluded groups
Don't abort the whole test run if we asked to exclude groups that aren't
included in the candidate group list, since we actually /are/ satisfying
the user's request.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Brian Foster [Wed, 10 Feb 2021 17:06:28 +0000 (12:06 -0500)]
generic: test mapped write after shutdown and failed writeback
XFS has a regression where it failed to check shutdown status in the
write fault path. This produced an iomap warning if the page
happened to recently fail a writeback attempt because writeback
failure can clear Uptodate status on the page.
Add a test for this scenario to help ensure mapped write failures
are handled as expected in the event of filesystem shutdown.
Upstream commit e4826691cc7e ("xfs: restore shutdown check in mapped
write fault path") fixed this bug.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Amir Goldstein [Wed, 10 Feb 2021 19:03:34 +0000 (21:03 +0200)]
overlay: Regression test for deadlock on directory ioctl
Overlayfs added the ability to set inode flags (e.g. chattr +i) in
kernel 5.10 by commit 61536bed2149 ("ovl: support [S|G]ETFLAGS and
FS[S|G]ETXATTR ioctls for directories").
Icenowy Zheng reported [1] a regression in that commit that causes
a deadlock when setting inode flags on lower dir.
The regression was fixed by commit b854cc659dcb ("ovl: avoid deadlock
on directory ioctl") and applied to kernel 5.10.15.
Amir Goldstein [Wed, 10 Feb 2021 19:03:33 +0000 (21:03 +0200)]
overlay: Test lost immutable/append-only flags on copy-up
Chengguang Xu reported [1] that append-only flag is lost on copy-up.
I had noticed that for directories, immutable flag can also be lost
on copy up (when parent is copied up). That's an old overlayfs bug.
Fixing this requires some VFS API changes that Miklos has proposed[2]
Amir Goldstein [Wed, 10 Feb 2021 19:03:30 +0000 (21:03 +0200)]
overlay/030: Update comment w.r.t upstream kernel
commit 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR
ioctls for directories") makes the comment in test header inaccurate.
Fix the comment to include this information.
Darrick J. Wong [Wed, 10 Feb 2021 02:57:06 +0000 (18:57 -0800)]
common: remove _require_no_rtinherit
All the remaining tests that use _require_no_rtinherit can be adapted to
ignore SCRATCH_RTDEV or to force files to be created on the data device.
This makes the helper unnecessary and increases test coverage, so remove
this helper.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Eryu Guan <guaneryu@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Wed, 10 Feb 2021 02:57:00 +0000 (18:57 -0800)]
xfs: fix filestreams tests when rtinherit=1
The filestreams allocator can only be selected for files that reside on
the data volume. In commit ee3e0010, we sprinkled calls to
_require_no_rtinherit in the filestreams tests so that there wouldn't be
regressions reported if the filesystem is formatted with -d rtinherit=1.
This unnecessarily limits test coverage because userspace can control
the device selection parameters quite easily with xfs_io chattr. Make
the filestreams tests unset SCRATCH_RTDEV so that the allocator isn't
thrown off by the rtbitmap consuming space on the data device.
Fixes: ee3e0010 ("xfs/realtime: add _require_no_rtinherit function") Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Eryu Guan <guaneryu@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
David Sterba [Tue, 9 Feb 2021 15:57:15 +0000 (16:57 +0100)]
src/splice-test.c: use memalign instead of aligned_alloc
The build fails on SLE11 as the function aligned_alloc is not
available there. Replace it by memalign that has the same semantics
and is commonly used in fstests code base. aligned_alloc has
additional requirements on the alignment and buffer size but that is
ok as the buffer is defined in multiples of the alignment.
Signed-off-by: David Sterba <dsterba@suse.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Chandan Babu R [Tue, 9 Feb 2021 16:12:52 +0000 (21:42 +0530)]
_scratch_mkfs_geom(): Filter out 'k' suffix from fs block size
If the original value of $MKFS_OPTIONS contained a block size value
having 'k' as a suffix (e.g. -b size=4k), then the newly constructed
value of $MKFS_OPTIONS will have 'k' suffixed to the value of
$blocksize. $blocksize itself is specified in units of bytes. Hence
having 'k' suffixed to this value will result in an incorrect block
size.
This commit fixes the bug by conditionally filtering out the 'k'
suffix from block size option present in the original value of
$MKFS_OPTIONS.
Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Tue, 2 Feb 2021 19:41:58 +0000 (11:41 -0800)]
xfs: test a regression in dquot type checking
This is a regression test for incorrect ondisk dquot type checking that
was introduced in Linux 5.9. The bug is that we can no longer switch a
V4 filesystem from having group quotas to having project quotas (or vice
versa) without logging corruption errors. That is a valid use case, so
add a regression test to ensure this can be done.
[Eryu: add _require_check_dmesg and print the 'corruption' dmesg]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Sun Ke [Mon, 1 Feb 2021 07:57:05 +0000 (15:57 +0800)]
src/swapon.c: initiate p to NULL
when run make, show:
swapon.c:135:3: warning: 'p' may be used uninitialized in this function
[-Wmaybe-uninitialized]
memcpy(p, buf, BUF_SIZE);
^~~~~~~~~~~~~~~~~~~~~~~~
This's a fake uninitialized warning.
gcc think the 'p' can be used uninitialized at here when verb is
TEST_MWRITE_BEFORE_AND_MWRITE_AFTER:
case TEST_MWRITE_BEFORE_AND_MWRITE_AFTER:
memcpy(p, buf, BUF_SIZE);
break;
But, if verb is TEST_MWRITE_BEFORE_AND_MWRITE_AFTER, the 'p' will be
initialized by:
switch (verb) {
case TEST_MWRITE_BEFORE_AND_MWRITE_AFTER:
case TEST_MWRITE_BEFORE:
p = mmap(NULL, BUF_SIZE, PROT_WRITE | PROT_READ, MAP_SHARED,
Silent the warning by initializing p to NULL anyway.
Signed-off-by: Sun Ke <sunke32@huawei.com> Reviewed-by: Eryu Guan <guaneryu@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Sandeen [Thu, 28 Jan 2021 21:31:40 +0000 (15:31 -0600)]
ltp/fsstress: don't fail on io_uring ENOSYS
We might have URING #defined at build time, but be running on a kernel
which does not support it.
For that reason, we should not exit with an error if
io_uring_queue_init() fails with ENOSYS. We can just note the lack of
support and skip all future io_uring operations.
Eric Biggers [Fri, 22 Jan 2021 01:44:34 +0000 (17:44 -0800)]
generic: test for lazytime timestamp updates
Test that when the lazytime mount option is enabled, updates to atime,
mtime, and ctime get persisted in the cases when they should be.
This test currently runs on ext4, f2fs, and xfs, since it's limited to
filesystems that support the shutdown ioctl.
This test currently passes on ext4 and f2fs. On xfs, kernel
commit 1e249cb5b7fc ("fs: fix lazytime expiration handling in
__writeback_single_inode()") is needed for the test to pass, since xfs
had a bug where it didn't persist timestamps when it should have.
Yang Xu [Tue, 19 Jan 2021 06:13:39 +0000 (14:13 +0800)]
ext4/046: skip test when ext4 doesn't support bs < ps with dioread_nolock
When testing arm machine, this case fails because ps > bs and kernel
doesn't introduced commit c8cc88163f40 ("ext4: Add support for
blocksize < pagesize in dioread_nolock"). Only skip this case when
kernel complains about bs!=ps error, so we can find dioread_nolock
mount regression in the future.
Darrick J. Wong [Tue, 27 Oct 2020 20:54:50 +0000 (07:54 +1100)]
xfs: test mkfs.xfs config files
Simple tests of the upcoming mkfs.xfs config file feature. First we
have some simple tests of properly formatted config files, then
improperly formatted config files, and finally we try to spot
conflicts between config file options and the cli.
[dchinner: updated for new libinih-based implementation.]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Qu Wenruo [Tue, 12 Jan 2021 07:40:24 +0000 (15:40 +0800)]
btrfs: check qgroup doesn't crash when beyond limit
There is a bug that, when btrfs is beyond qgroup limit, touching a file
could crash btrfs.
Such beyond limit situation needs to be intentionally created, e.g.
writing 1GiB file, then limit the subvolume to 512 MiB.
As current qgroup works pretty well at preventing us from reaching the
limit.
This makes existing qgroup test cases unable to detect it.
The regression is introduced by commit c53e9653605d ("btrfs: qgroup:
try to flush qgroup space when we get -EDQUOT"), and the fix is
commit 6f23277a49e6 ("btrfs: qgroup: don't commit transaction when
we already hold the handle")
Filipe Manana [Mon, 11 Jan 2021 11:41:54 +0000 (11:41 +0000)]
btrfs: test incremental send after cloning extents from the same file
Test that an incremental send operation correctly issues clone operations
for a file that had different parts of one of its extents cloned into
itself, at different offsets, and a large part of that extent was
overwritten, so all the reflinks only point to subranges of the extent.
This currently fails on btrfs but is fixed by a patch for the kernel that
has the following subject:
"btrfs: send, fix invalid clone operations when cloning from the same file and root"
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Ritesh Harjani [Tue, 5 Jan 2021 14:31:43 +0000 (20:01 +0530)]
generic/496: ext4 and xfs supports swapon on fallocated file
ext4, xfs should not fail swapon on fallocated file. Currently if this
fails the fstst was not returning a failure. Fix those for given
filesystems (for now added ext4/xfs).
There were some regressions which went unnoticed due to this in ext4
tree, which later got fixed as part of this patch [1]
Brian Foster [Tue, 5 Jan 2021 11:58:44 +0000 (06:58 -0500)]
generic/388: randomly recover via read-only mounts
XFS has an issue where superblock counters may not be properly
synced when recovery occurs via a read-only mount. This causes the
filesystem to become inconsistent after unmount. To cover this test
case, update generic/388 to switch between read-only and read-write
mounts to perform log recovery.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Xiao Yang [Wed, 9 Dec 2020 05:08:15 +0000 (13:08 +0800)]
src/dmiperf: Remove obsolete dmiperf
Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Amir Goldstein [Sat, 19 Dec 2020 10:47:27 +0000 (12:47 +0200)]
overlay: run unionmount tests with custom overlay mount options
Assign $OVERLAY_MOUNT_OPTIONS to UNIONMOUNT_MNTOPTIONS and require
that unionmount supports UNIONMOUNT_MNTOPTIONS if OVERLAY_MOUNT_OPTIONS
was provided.
For example, when the mount option metacopy=on is set in
$OVERLAY_MOUNT_OPTIONS, it enables the --meta test option and affects
the test verifications after copy up.
Su Yue [Sat, 19 Dec 2020 09:10:58 +0000 (17:10 +0800)]
shared/032: add options for jffs2
Recently I found that my disk space was eaten by results/shared/032.full
after one round of xfstests running. The file was filled with binary
bytes by the command:
`mkfs -t jffs2 /dev/mapper/test-2 >> results/shared/032.full 2>&1`
Manual of mkfs.jffs2:
========================================================================
The program mkfs.jffs2 creates a JFFS2 (Second Journalling Flash
File System) file system image and writes the resulting image to the
file specified by the -o option or by default to the standard output,
unless the standard output is a terminal device in which case mkfs.jffs2
will abort. The file system image is created using the files and
directories contained in the directory specified by the option -r or
the present directory, if the -r option is not specified.
========================================================================
So the command equals to
`mkfs -t jffs2 -r xfstests-dev /dev/mapper/test-2 \
-o xfstests-dev/results/shared/032.full \
>>xfstests-dev/results/shared/032.full 2>&1`
The output image 032.full is under the directory xfstests-dev.
mkfs.jffs2 keeps reading from 032.full in page size and append bytes
to 032.full until the disk where xfstests-devel located is full.
Fix it by setting @preargs to '-r /proc/fs -o' for jffs2, now the output
is $SCRATCH_DEV. Since mkfs.jffs2 doesn't check overwrite case, just
skip overwrite attempt.
Filipe Manana [Thu, 10 Dec 2020 12:09:17 +0000 (12:09 +0000)]
btrfs: test incremental send after removing a directory and all its files
Test that an incremental send operation succeeds, and produces the
correct results, after removing a directory and all its files, unmounting
the filesystem, mounting the filesystem again and creating a new file (or
directory).
This currently fails on btrfs, but is fixed by a patch that has the
following subject:
btrfs: send, fix wrong file path when there is an inode with a pending rmdir
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
ethanwu [Tue, 15 Dec 2020 03:59:06 +0000 (11:59 +0800)]
btrfs: test if rename handles dir item collision correctly
This is a regression test for the issue fixed by the kernel commit titled
"btrfs: correctly calculate item size used when item key collision happens"
In this case, we'll simply rename many forged filename that cause collision
under a directory to see if rename failed and filesystem is forced readonly.
Feiyu Zhu [Thu, 10 Dec 2020 02:59:54 +0000 (21:59 -0500)]
src/t_enospc.c: Fix an error for the loop initialization declaration
When I compiled xfstests using the gcc(version 4.8.5), the following error occurred:
t_enospc.c: In function 'enospc_test':
t_enospc.c:88:2: error: 'for' loop initial declarations are only allowed in C99 mode
for (int i = 0; i < size; i++) {
^
Brian Foster [Mon, 14 Dec 2020 17:18:54 +0000 (12:18 -0500)]
generic/563: use a loop device to avoid partition incompatibility
cgroup writeback accounting does not track partition level
statistics. Instead, I/O is accounted against the parent device. As
a result, the test fails if the scratch device happens to be a
device partition. Since parent level stats are potentially polluted
by factors external to the test, wrap the scratch device in a
loopback device to guarantee the test always runs on a top-level
block device.
Reported-by: Boyang Xue <bxue@redhat.com> Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Mon, 14 Dec 2020 23:47:20 +0000 (15:47 -0800)]
generic: test for creating duplicate filenames in encrypted dir
Test for a race condition where a duplicate filename could be created in
an encrypted directory while the directory's encryption key was being
added concurrently.
generic/595 was already failing on ubifs due to this bug, but only by
accident. This new test detects the bug on both ext4 and ubifs. I
wasn't able to get it to detect the bug on f2fs.
Xiao Yang [Wed, 9 Dec 2020 05:08:16 +0000 (13:08 +0800)]
src/multi_open_unlink: Stop using attr_set
Avoid the following warning by replacing deprecated
attr_set with setxattr:
---------------------------------------------------
warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations]
---------------------------------------------------
Also remove unneeded flags, <attr/attributes.h> and $(LIBATTR).
Ritesh Harjani [Wed, 9 Dec 2020 04:03:41 +0000 (09:33 +0530)]
generic: Add test to check for mounting a huge sparse dm device
Add this test to check for regression which was reported when ext4 bmap
aops was moved to use iomap APIs. jbd2 calls bmap() kernel function
from fs/inode.c which was failing since iomap_bmap() implementation earlier
returned 0 for block addr > INT_MAX.
This regression was fixed with following kernel commit [1]
commit b75dfde1212991b24b220c3995101c60a7b8ae74
("fibmap: Warn and return an error in case of block > INT_MAX")
[1]: https://patchwork.ozlabs.org/patch/1279914
w/o the kernel fix we get below errors and mount fails
[ 1461.988701] run fstests generic/613 at 2020-10-27 19:57:34
[ 1530.406645] ------------[ cut here ]------------
[ 1530.407332] would truncate bmap result
[ 1530.408956] WARNING: CPU: 0 PID: 6401 at fs/iomap/fiemap.c:116 iomap_bmap_actor+0x43/0x50
[ 1530.410607] Modules linked in:
[ 1530.411024] CPU: 0 PID: 6401 Comm: mount Tainted: G W
<...>
[ 1530.511978] jbd2_journal_init_inode: Cannot locate journal superblock
[ 1530.513310] EXT4-fs (dm-1): Could not load journal inode
Nikolay Borisov [Mon, 7 Dec 2020 09:23:18 +0000 (11:23 +0200)]
btrfs: Update btrfs/215
This patch updates btrfs/215 to work with latest upstream kernel. That's
required since commit 324bcf54c449 ("mm: use limited read-ahead to satisfy read")
changed readahead logic to always issue a read even if the RA pages are
set to 0. This results in 1 extra io being issued so the counts in the
test should be incremented by 1. Also use the opportunity to update the
commit reference since it's been merged in the upstream kernel.
Kaixu Xia [Mon, 7 Dec 2020 11:41:05 +0000 (19:41 +0800)]
xfs/513: fix the regression caused by mount option uqnoenforce
The mount options uqnoenforce and qnoenforce no longer cause 'usrquota'
to be emitted in /proc/mounts, so there is a regression in xfs/513. Fix
it by using proper output option uqnoenforce.
You'll need kernel commit 237d7887ae72 ("xfs: show the proper user
quota options") to pass the test.
[Eryu: add kernel commit info in commit log]
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Kaixu Xia <kaixuxia@tencent.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eryu Guan [Fri, 4 Dec 2020 10:43:53 +0000 (16:13 +0530)]
check: source common/rc again if TEST_DEV was recreated
If TEST_DEV is recreated by check, FSTYP derived from TEST_DEV
previously could be changed too and might not reflect the reality.
So source common/rc again with correct FSTYP to get fs-specific
configs, e.g. common/xfs.
For example, using this config-section config file, and run section
ext4 first then xfs, you can see:
our local _scratch_mkfs routine ...
./common/rc: line 825: _scratch_mkfs_xfs: command not found
check: failed to mkfs $SCRATCH_DEV using specified options
Ira Weiny [Fri, 4 Dec 2020 01:45:50 +0000 (17:45 -0800)]
common/rc: Fix _check_s_dax()
There is a conflict with the user visible statx bits 'mount root' and
'dax'. The kernel is changing the dax bit to correct this conflict.[1]
Adjust _check_s_dax() to use the new bit. Because DAX tests do not run
on root mounts, STATX_ATTR_MOUNT_ROOT should always be 0. Therefore,
check for the old flag and fail the test if that occurs.
generic: ENOSPC regression test in a multi-threaded scenario
Test allocation strategies of the file system and validate space
anomalies as reported by the system versus the allocated by the
program.
The test is motivated by a bug in ext4 systems where-in ENOSPC is
reported by the file system even though enough space for allocations is
available[1].
Linux kernel patch series that fixes the above regression: 53f86b170dfa ("ext4: mballoc: add blocks to PA list under same spinlock
after allocating blocks") cf5e2ca6c990 ("ext4: mballoc: refactor ext4_mb_discard_preallocations()") 07b5b8e1ac40 ("ext4: mballoc: introduce pcpu seqcnt for freeing PA to
improve ENOSPC handling") 8ef123fe02ca ("ext4: mballoc: refactor ext4_mb_good_group()") 993778306e79 ("ext4: mballoc: use lock for checking free blocks while
retrying")
Luis Henriques [Fri, 27 Nov 2020 12:37:42 +0000 (12:37 +0000)]
ceph: add a new test for cross quota realms renames
For the moment cross quota realms renames has been disabled in CephFS
after a bug has been found while renaming files created and truncated.
This allowed clients to easily circumvent quotas.
Link: https://tracker.ceph.com/issues/48203 Signed-off-by: Luis Henriques <lhenriques@suse.de> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Wed, 11 Nov 2020 00:43:33 +0000 (16:43 -0800)]
misc: fix $MKFS_PROG.$FSTYP usage treewide
Replace all the $MKFS_PROG.$FSTYP invocations with $MKFS_PROG -t $FSTYP.
The mkfs wrapper binary knows how to search the user's $PATH to find the
appropriate mkfs delegate, which the author uses to switch between
development and distro versions of various tools.
Unfortunately, using "$MKFS_PROG.$FSTYP" means that the shell only looks
in the same directory as the mkfs wrapper, which means that we can end
up mixing different tool versions when this is the case.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <guaneryu@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Wed, 11 Nov 2020 00:43:16 +0000 (16:43 -0800)]
check: run tests in a systemd scope for mandatory test cleanup
TLDR: If systemd is available, run each test in its own temporary
systemd scope. This enables the test harness to forcibly clean up all
of the test's child processes (if it does not do so itself) so that we
can move into the post-test unmount and check cleanly.
I frequently run fstests in "low" memory situations (2GB!) to force the
kernel to do interesting things. There are a few tests like generic/224
and generic/561 that put processes in the background and occasionally
trigger the OOM killer. Most of the time the OOM killer correctly
shoots down fsstress or duperemove, but once in a while it's stupid
enough to shoot down the test control process (i.e. tests/generic/224)
instead. fsstress is still running in the background, and the one
process that knew about that is dead.
When the control process dies, ./check moves on to the post-test fsck,
which fails because fsstress is still running and we can't unmount.
After fsck fails, ./check moves on to the next test, which fails because
fsstress is /still/ writing to the filesystem and we can't unmount or
format.
The end result is that that one OOM kill causes cascading test failures,
and I have to re-start fstests to see if I get a clean(er) run.
So, the solution I present in this patch is to teach ./check to try to
run the test script in a systemd scope. If that succeeds, ./check will
tell systemd to kill the scope when the test script exits and returns
control to ./check. Concretely, this means that systemd creates a new
cgroup, stuffs the processes in that cgroup, and when we kill the scope,
systemd kills all the processes in that cgroup and deletes the cgroup.
The end result is that fstests now has an easy way to ensure that /all/
child processes of a test are dead before we try to unmount the test and
scratch devices. I've designed this to be optional, because not
everyone does or wants or likes to run systemd, but it makes QA easier.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <guaneryu@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Wed, 11 Nov 2020 00:42:59 +0000 (16:42 -0800)]
common: extract rt extent size for _get_file_block_size
_get_file_block_size is intended to return the size (in bytes) of the
fundamental allocation unit for a file. This is required for remapping
operations like fallocate and reflink, which can only operate on
allocation units. Since the XFS realtime volume can be configure for
allocation units larger than 1 fs block, we need to factor that in here.
Note that ext* with bigalloc does not allocations to be aligned to the
cluster size, so no update is needed there.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Zorro Lang [Sun, 8 Nov 2020 12:26:21 +0000 (20:26 +0800)]
src/feature: add IO_URING feature checking
IO_URING is a new feature for GNU/Linux system, if someone case of
xfstests tests this feature, better to check if current system
supports it, or need _notrun.
Filipe Manana [Mon, 9 Nov 2020 12:01:06 +0000 (12:01 +0000)]
generic: test for non-zero used blocks while writing into a file
Test that if we keep overwriting an entire file, either with buffered
writes or direct IO writes, the number of used blocks reported by stat(2)
is never zero while the writes and writeback are in progress.
This is motivated by a bug in btrfs and currently fails on btrfs only. It
is fixed a patchset for btrfs that has the following patches:
btrfs: fix missing delalloc new bit for new delalloc ranges
btrfs: refactor btrfs_drop_extents() to make it easier to extend
btrfs: fix race when defragging that leads to unnecessary IO
btrfs: update the number of bytes used by an inode atomically
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Sun, 15 Nov 2020 03:15:36 +0000 (19:15 -0800)]
fscrypt-crypt-util: fix maximum IV size
In commit 65cd8e8a8e81 ("fscrypt-crypt-util: fix IV incrementing for
--iv-ino-lblk-32") I mistakenly decreased the size of fscrypt_iv to 24
bytes, which is the most that is explicitly needed by any of the IV
generation methods. However, Adiantum encryption takes a 32-byte IV, so
the buffer still needs to be 32 bytes, with any extra bytes zeroed.
So restore the size to 32 bytes.
This fixes a buffer overread that caused generic/550 and generic/584 to
sometimes fail, depending on the build of the fscrypt-crypt-util binary.
(Most of the time it still worked by chance.)
Fixes: 65cd8e8a8e81 ("fscrypt-crypt-util: fix IV incrementing for --iv-ino-lblk-32") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Xiao Yang [Tue, 10 Nov 2020 02:25:26 +0000 (10:25 +0800)]
generic/426: Remove unused opt variable in create_test_files()
Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Tue, 27 Oct 2020 19:02:02 +0000 (12:02 -0700)]
xfs/327: fix inode reflink flag checking
This is a regression test that tried to make sure that repair correctly
clears the XFS inode reflink flag when it detects files that do not
share any blocks. However, it does this checking by looking at the
(online) lsattr output. This worked fine during development when we
exposed the reflink state via the stat ioctls, but that has long since
been removed. Now the only way to check is via xfs_db, so switch it to
use that.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Tue, 27 Oct 2020 19:01:56 +0000 (12:01 -0700)]
various: replace _get_block_size with _get_file_block_size when needed
The _get_file_block_size helper was added so that tests could find out
the size of a fundamental unit of allocation for a given file, which is
necessary for certain fallocate and clonerange tests.
On certain filesystem configurations (ocfs2 with clusters, xfs with a
large rt extent size), this is /not/ the same as the filesystem block
size, and these tests will fail. Fix them to use the correct helper.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong [Tue, 27 Oct 2020 19:01:42 +0000 (12:01 -0700)]
xfs/520: disable external devices
This is a regression test for a specific bug that requires a specific
configuration of the data device. Realtime volumes and external logs
don't affect the efficacy of the test, but the test can fail mkfs if the
realtime device is very large.
Therefore, unset USE_EXTERNAL so that we always run this regression
test, even if the tester enabled realtime.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Filipe Manana [Wed, 4 Nov 2020 11:13:37 +0000 (11:13 +0000)]
generic: test number of blocks used by a file after mwrite into a hole
Test that after doing a memory mapped write to an empty file, a call to
stat(2) reports a non-zero number of used blocks.
This is motivated by a bug in btrfs where the number of blocks used does
not change. It currenly fails on btrfs and it is fixed by a patch that
has the following subject:
"btrfs: fix missing delalloc new bit for new delalloc ranges"
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Josef Bacik [Wed, 4 Nov 2020 20:38:45 +0000 (15:38 -0500)]
btrfs/220: fix how we tests for mount options
Filipe noticed that btrfs/220 started failing with some mount option
changes I made recently, but upon closer inspection this test actually
fails in a lot of different ways normally, specifically if you specify
MOUNT_OPTIONS, or if you make an fs with the free space tree.
Address all these issues by reworking how we test that the mount options
are what we expect. First get what the default mount options are for a
plain mount of SCRATCH_DEV. This is used as the baseline, so no matter
how the mount options change in the future it will always work properly.
Secondly instead of specifying a rigid order of the mount options we're
testing, which breaks if we adjust the order in /proc/self/mounts,
simply specify the options we're actually interested in checking. Then
in the test function combine the common options with the new options
we're testing, and then combine that with our actual options and use
some sort magic to see if there's any difference. If there's no
difference then we know we have everything set as expected, if not we
fail.
This patch addresses the initial issue that Filipe noticed, but also
fixes the failures when you specified MOUNT_OPTIONS, or if you made the
fs with the free space tree.
Signed-off-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Sun, 1 Nov 2020 17:16:19 +0000 (09:16 -0800)]
tests/ceph: add Makefile
'make install' stopped working because the tests/ceph/ directory is
missing a Makefile. Add it.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Bill O'Donnell <billodo@redhat.com> Reviewed-by: Vladimir Zapolskiy <vladimir@tuxera.com> Tested-by: Vladimir Zapolskiy <vladimir@tuxera.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Sat, 31 Oct 2020 07:23:44 +0000 (00:23 -0700)]
generic: test that encryption nonces are unique and random
Test that encryption nonces are unique and random, where randomness is
approximated as "incompressible by the xz program".
This gets indirectly tested by generic/399, but there are some gaps.
It's good to test for this directly too.
This test runs and passes on ext4 and f2fs. It doesn't currently run on
ubifs because _get_encryption_nonce() isn't implemented for ubifs yet.
(At some point I'll probably switch _get_encryption_nonce() to use
FS_IOC_GET_ENCRYPTION_NONCE, which was added in Linux 5.7. But for now
I'd like to keep the tests using it runnable on older kernels too.)
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Sat, 31 Oct 2020 05:41:41 +0000 (22:41 -0700)]
generic/398: remove workarounds for wrong error codes
generic/398 contains workarounds to allow for renames of encrypted files
to fail with different error codes. However, these error codes were
fixed up by kernel commits f5e55e777cc9 ("fscrypt: return -EXDEV for
incompatible rename or link into encrypted dir") and 0c1ad5242d4f
("ubifs: switch to fscrypt_prepare_rename()").
It's been long enough, so update the test to expect the correct behavior
only, so we don't accidentally reintroduce the wrong behavior.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Sat, 31 Oct 2020 05:40:18 +0000 (22:40 -0700)]
generic/395: remove workarounds for wrong error codes
generic/395 contains workarounds to allow for some of the fscrypt ioctls
to fail with different error codes. However, the error codes were all
fixed up and documented years ago:
- FS_IOC_GET_ENCRYPTION_POLICY on ext4 failed with ENOENT instead of
ENODATA on unencrypted files. Fixed by commit db717d8e26c2
("fscrypto: move ioctl processing more fully into common code").
- FS_IOC_SET_ENCRYPTION_POLICY failed with EINVAL instead of EEXIST
on encrypted files. Fixed by commit 8488cd96ff88 ("fscrypt: use
EEXIST when file already uses different policy").
- FS_IOC_SET_ENCRYPTION_POLICY failed with EINVAL instead of ENOTDIR
on nondirectories. Fixed by commit dffd0cfa06d4 ("fscrypt: use
ENOTDIR when setting encryption policy on nondirectory").
It's been long enough, so update the test to expect the correct behavior
only, so we don't accidentally reintroduce the wrong behavior.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Biggers [Sat, 31 Oct 2020 05:41:29 +0000 (22:41 -0700)]
generic/397: remove workarounds for wrong error codes
generic/397 contains workarounds to allow for kernel bugs where trying
to open or create files in an encrypted directory without the encryption
key failed with ENOENT, EACCES, or EPERM instead of the expected ENOKEY.
However, all these bugs have been fixed. ext4 and f2fs were fixed years
ago by commit 54475f531bb8 ("fscrypt: use ENOKEY when file cannot be
created w/o key"). ubifs was fixed by commit b01531db6cec ("fscrypt:
fix race where ->lookup() marks plaintext dentry as ciphertext").
It's been long enough, so update the test to expect the correct behavior
only, so we don't accidentally reintroduce the wrong behavior.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Filipe Manana [Thu, 29 Oct 2020 12:09:06 +0000 (12:09 +0000)]
fstests: add missing remove of the $seqres.full file for some tests
Some test cases are missing the 'rm -f $seqres.full' line but are appending
to that file, so everytime they run that file gets bigger and bigger (some
of them are using about a dozen megabytes on one of my test boxes).
So just add the 'rm -f $seqres.full' line to them, together with the comment
that the 'new' script generates for new test cases.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Xiao Yang [Tue, 27 Oct 2020 02:52:11 +0000 (10:52 +0800)]
ext4: Add a test for inline_data vs. DAX inode flag
Inline_data is mutually exclusive to DAX inode flag so enabling both
of them is not expected and triggers some errors. It's a regression
test for kernel commit aa2f77920b74 ("ext4: disallow modifying DAX
inode flag if inline_data has been set")
Luis Henriques [Mon, 19 Oct 2020 13:27:49 +0000 (14:27 +0100)]
ceph: test combination of copy_file_range with truncate
This tests a bug found while testing copy_file_range. This bug was an
issue with how the OSDs handled the truncate_seq value, which was being
copied from the original object into the destination object. This test
ensures the kernel client correctly handles fixed/non-fixed OSDs.
Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with different
combinations of both object sizes and copy sizes.
Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with several
combinations of both object sizes and copy sizes. It also uses several
combinations of copy ranges. For example, copying the 1st object in the
src file into:
1) the beginning (1st object) of dst file,
2) the end (last object) of dst file and
3) the middle of the dst file.
Ritesh Harjani [Thu, 22 Oct 2020 12:16:32 +0000 (17:46 +0530)]
ext4: verify unwritten extent conversion in buff-io
There was an issue where with filesize > 4G, map.m_lblk
was getting overflow in buff-IO path while converting unwritten to
written extent with dioread_nolock mount option with bs < ps.
Adding a testcase to test for such regressions with dioread_nolock
mount option. To reproduce the same regression w/o the fix
in the kernel, test with bs < ps config.
The following commit fixed the issue in linux.
id1e18b8824dd ("ext4: fix bs < ps issue reported with dioread_nolock
mount opt")
Filipe Manana [Tue, 20 Oct 2020 14:43:01 +0000 (15:43 +0100)]
btrfs: add test case for rwf_nowait writes
Test several scenarios for RWF_NOWAIT writes, to verify we don't regress
on btrfs specific behaviour (snapshots, cow files, reflinks, holes,
prealloc extent beyond eof).
We had some bugs in the past related to RWF_NOWAIT writes not failing on
btrfs when they should or failing when they shouldn't, these were fixed by
the following kernel commits:
4b1946284dd6 ("btrfs: fix failure of RWF_NOWAIT write into prealloc extent beyond eof") 260a63395f90 ("btrfs: fix RWF_NOWAIT write not failling when we need to cow")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
When nfs_export is enabled, the link count of upper dir
objects are more then the expected number in this testcase.
Because extra index entries are linked to upper inodes.
Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Matthew Wilcox [Mon, 19 Oct 2020 12:58:39 +0000 (13:58 +0100)]
generic/127: to exit if any subtest fails
If one of the subtests of generic/127 fails, we proceed with the
rest of the tests, potentially overwriting useful data. This makes
it stop as soon as any of the subtests fails.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Josef Bacik [Tue, 15 Sep 2020 00:22:43 +0000 (20:22 -0400)]
fstests: drop check.log and check.time into section specific results dir
Right now we only track check.log and check.time globally, it would
be nice to do it per-section as well. This makes it easier to parse
results from systems that run a bunch of different configurations at
once.
Yang Xu [Fri, 16 Oct 2020 09:31:23 +0000 (17:31 +0800)]
xfs/521: use _try_scratch_mount instead of _scratch_mount
Here will check the return code of mount option. So update it
to use _try_scratch_mount.
Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Filipe Manana [Sun, 11 Oct 2020 12:38:52 +0000 (13:38 +0100)]
fstests: add a filter for the new getcap output
Starting with version 2.41 of libcap, the output of the getcap program
changed and therefore some existing tests fail when the installed version
of libcap is >= 2.41 (the latest version available at the moment is 2.44).
The change was made by the following commit of libcap:
A more compact form for the text representation of capabilities.
While this does not change anything about the supported range of
equivalent text specifications for capabilities, as accepted by
cap_from_text(), this does alter the preferred output format of
cap_to_text() to be two characters shorter in most cases. That is,
what used to be summarized as:
"= cap_foo+..."
is now converted to the equivalent text:
"cap_foo=..."
which is also more intuitive.
So add a filter to change the old format to the new one, an helper that
calls getcap with that filter, make existing tests use the new helper and
update their golden output to match the new output format of getcap.
Darrick J. Wong [Thu, 8 Oct 2020 15:05:51 +0000 (08:05 -0700)]
xfs: test running growfs on the realtime volume
Make sure that we can run growfs to expand the realtime volume without
it blowing up. This is a regression test for the following patches:
xfs: Set xfs_buf type flag when growing summary/bitmap files
xfs: Set xfs_buf's b_ops member when zeroing bitmap/summary files
xfs: fix realtime bitmap/summary file truncation when growing rt volume
xfs: make xfs_growfs_rt update secondary superblocks
xfs: annotate grabbing the realtime bitmap/summary locks in growfs
Because the xfs maintainer realized that no, we have no tests for this
particular piece of functionality.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
projects / xfstests-dev.git / log