Darrick J. Wong [Fri, 30 Dec 2022 22:12:53 +0000 (14:12 -0800)]
fuzzy: rework scrub stress output filtering
Rework the output filtering functions for scrub stress tests: first, we
should use _filter_scratch to avoid leaking the scratch fs details to
the output. Second, for scrub and repair, change the filter elements to
reflect outputs that don't indicate failure (such as busy resources,
preening requests, and insufficient space to do anything). Finally,
change the _require function to check that filter functions have been
sourced.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:53 +0000 (14:12 -0800)]
fuzzy: clean up scrub stress programs quietly
In the cleanup function for online fsck stress test common code, send
SIGINT instead of SIGTERM to the fsstress and xfs_io processes to kill
them. bash prints 'Terminated' to the golden output when children die
with SIGTERM, which can make a test fail, and we don't want a regular
cleanup function being the thing that prevents the test from passing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:53 +0000 (14:12 -0800)]
xfs/422: move the fsstress/freeze/scrub racing logic to common/fuzzy
Hoist all this code to common/fuzzy in preparation for making this code
more generic so that we implement a variety of tests that check the
concurrency correctness of online fsck. Do just enough renaming so that
we don't pollute the test program's namespace; we'll fix the other warts
in subsequent patches.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Tue, 10 Jan 2023 22:22:43 +0000 (23:22 +0100)]
common/rc: drop SGI DMF specific _mount_ops_filter
The _mount() helper function is the only caller of _mount_ops_filter(),
which appears to have been used in the past to replace the SGI DMF
specific mtpt= mount option setting.
_mount() invocations could now be replaced with $MOUNT_PROG calls
directly, but I've retained the helper function for readability.
Link: https://irix7.com/techpubs/007-3683-007.pdf Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/012 is requiring ext4 support to test the conversion, but the test
case is only checking if mkfs.ext4 is available, not if the filesystem
driver is actually available on the test host.
Check if the driver is available as well, before trying to run the test.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Thu, 5 Jan 2023 07:18:19 +0000 (15:18 +0800)]
btrfs: add a test case to verify scrub speed throttle works
We introduced scrub speed throttle in commit eb3b50536642 ("btrfs: scrub:
per-device bandwidth control"), but it is not that well documented
(e.g. what's the unit of the sysfs interface), nor tested by any test
case.
This patch will add a test case for this functionality.
The test case itself is pretty straightforward:
- Fill the fs with 2G file as scrub workload
- Scrub without any throttle to grab the initial speed
- Set the throttle to half of the initial speed
- Scrub again and check the speed against the throttle
The test case has an assumption that we can exclusively use all the
performance of the underlying disk.
But for cloud environment it's not ensured 100%, thus the test case is
not included in auto group to avoid false alerts.
Baokun Li [Thu, 29 Dec 2022 13:44:34 +0000 (21:44 +0800)]
overlay: avoid to use NULL OVL_BASE_FSTYP for mounting
Generally, FSTYP is used to specify OVL_BASE_FSTYP. When we specify FSTYP
through an environment variable, it is not converted to OVL_BASE_FSTYP.
In addition, sometimes we do not even specify the file type. For example,
we only use `./check -n -overlay -g auto` to list overlay-related cases.
If OVL_BASE_FSTYP is NULL, mounting fails and the test fails.
To solve this problem, try to assign a value to OVL_BASE_FSTYP when
specifying -overlay. In addition, in the _overlay_base_mount function,
the basic file system type of the overlay is specified only when
OVL_BASE_FSTYP is not NULL.
Reported-by: Murphy Zhou <jencce.kernel@gmail.com> Signed-off-by: Baokun Li <libaokun1@huawei.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:22 +0000 (15:32 -0800)]
generic/575: test 1K Merkle tree block size
In addition to 4K, test 1K Merkle tree blocks. Also always run this
test, regardless of FSV_BLOCK_SIZE, but allow skipping tests of
parameters that are unsupported, unless they are the default.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:21 +0000 (15:32 -0800)]
generic/624: test multiple Merkle tree block sizes
Instead of only testing 4K Merkle tree blocks, test FSV_BLOCK_SIZE, and
also other sizes if they happen to be supported. This allows this test
to run in cases where it couldn't before and improves test coverage in
cases where it did run before.
This required reworking the test to compute the expected digests using
the 'fsverity digest' command, instead of relying on .out file
comparisons. (There isn't much downside to this, since comparison to
known-good file digests already happens in generic/575. So if both the
kernel and fsverity-utils were to be broken in the same way, generic/575
would detect it. generic/624 serves a different purpose.)
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:20 +0000 (15:32 -0800)]
generic/574: test multiple Merkle tree block sizes
Instead of only testing 4K Merkle tree blocks, test FSV_BLOCK_SIZE, and
also other sizes if they happen to be supported. This allows this test
to run in cases where it couldn't before and improves test coverage in
cases where it did run before.
Given that the list of Merkle tree block sizes that will actually work
is not fixed, this required reworking the test to not rely on the .out
file so heavily.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:16 +0000 (15:32 -0800)]
common/verity: add _filter_fsverity_digest()
Add a filter that replaces fs-verity digests with a fixed string. This
is needed because the fs-verity digests that some tests print are going
to start depending on the default Merkle tree block size.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:15 +0000 (15:32 -0800)]
common/verity: use FSV_BLOCK_SIZE by default
Make _fsv_enable() and _fsv_sign() default to FSV_BLOCK_SIZE if no block
size is explicitly specified, so that the individual tests don't have to
do this themselves. This overrides the fsverity-utils default of 4096
bytes, or the page size in older versions of fsverity-utils, both of
which may differ from FSV_BLOCK_SIZE.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:14 +0000 (15:32 -0800)]
common/verity: set FSV_BLOCK_SIZE to an appropriate value
In order to maximize the chance that the verity tests can actually be
run, FSV_BLOCK_SIZE (the default Merkle tree size for the verity tests)
needs to be min(fs_block_size, page_size), not simply page_size. The
only reason that page_size was okay before was because the kernel only
supported merkle_tree_block_size == fs_block_size == page_size anyway.
But that is changing.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:13 +0000 (15:32 -0800)]
common/verity: add and use _fsv_can_enable()
Replace _fsv_have_hash_algorithm() with a more general function
_fsv_can_enable() which checks whether 'fsverity enable' with the given
parameters works. For now it is just used with --hash-alg or with no
parameters, but soon it will be used with --block-size too.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 23 Dec 2022 02:56:42 +0000 (10:56 +0800)]
btrfs/154: migrate to python3
Test case btrfs/154 is still using python2 script, which is already EOL.
Some rolling distros like Archlinux is no longer providing python2
package anymore.
This means btrfs/154 will be harder and harder to run.
To fix the problem, migreate the python script to python3, this involves
the following changes:
- Change common/config to use python3
- Strong type conversion between string and bytes
This means, anything involved in the forged bytes has to be bytes.
And there is no safe way to convert forged bytes into string, unlike
python2.
I guess that's why the author went python2 in the first place.
Thankfully os.rename() still accepts forged bytes.
- Use bytes specific checks for invalid chars.
The updated script can still cause the needed conflicts, can be verified
through "btrfs ins dump-tree" command.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 00:22:02 +0000 (16:22 -0800)]
xfs/179: modify test to trigger refcount update bugs
Upon enabling fsdax + reflink for XFS, this test began to report
refcount metadata corruptions after being run. Specifically, xfs_repair
noticed single-block refcount records that could be combined but had not
been.
The root cause of this is improper MAXREFCOUNT edge case handling in
xfs_refcount_merge_extents. When we're trying to find candidates for a
record merge, we compute the refcount of the merged record, but without
accounting for the fact that once a record hits rc_refcount ==
MAXREFCOUNT, it is pinned that way forever.
Adjust this test to use a sub-filesize write for one of the COW writes,
because this is how we force the extent merge code to run.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 16:57:03 +0000 (08:57 -0800)]
xfs: regression test for writes racing with reclaim writeback
This test uses the new write delay debug knobs to set up a slow-moving
write racing with writeback of an unwritten block at the end of the
file range targetted by the slow write. The test succeeds if the file
does not end up corrupt and if the ftrace log captures a message about
the revalidation occurring.
NOTE: I'm not convinced that madvise actually causes the page to be
removed from the pagecache, which means that this is effectively a
functional test for the invalidation, not a corruption reproducer.
On the other hand, the functional test can be done quickly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 00:21:50 +0000 (16:21 -0800)]
xfs: regression test for writeback corruption bug
This is a regression test for a data corruption bug that existed in XFS'
copy on write code between 4.9 and 4.19. The root cause is a
concurrency bug wherein we would drop ILOCK_SHARED after querying the
CoW fork in xfs_map_cow and retake it before querying the data fork in
xfs_map_blocks. See the test description for a lot more details.
Cc: Wengang Wang <wen.gang.wang@oracle.com> Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 9 Dec 2022 06:05:10 +0000 (14:05 +0800)]
btrfs/220: fix the test failure due to new default mount option
[BUG]
The latest misc-next tree will make test case btrfs/220 fail with the
following error messages:
btrfs/220 15s ... - output mismatch (see ~/xfstests/results//btrfs/220.out.bad)
--- tests/btrfs/220.out 2022-05-11 09:55:30.749999997 +0800
+++ ~/xfstests/results//btrfs/220.out.bad 2022-12-09 13:57:23.706666671 +0800
@@ -1,2 +1,5 @@
QA output created by 220
+Unexepcted mount options, checking for 'rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/' in 'rw,relatime,space_cache=v2,subvolid=5,subvol=/' using 'nodiscard'
+Unexepcted mount options, checking for 'rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/' in 'rw,relatime,space_cache=v2,subvolid=5,subvol=/' using 'nodiscard'
+Unexepcted mount options, checking for 'rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/' in 'rw,relatime,space_cache=v2,subvolid=5,subvol=/' using 'nodiscard'
Silence is golden
...
(Run 'diff -u ~/xfstests/tests/btrfs/220.out ~/xfstests/results//btrfs/220.out.bad' to see the entire diff)
Ran: btrfs/220
Failures: btrfs/220
Failed 1 of 1 tests
[CAUSE]
Since patch "btrfs: auto enable discard=async when possible", which is
already in the maintainer's tree for next merge window, btrfs will
automatically enable asynchronous discard for devices which supports
discard.
This makes our $DEFAULT_OPTS to have "discard=async" in it.
While for "nodiscard" mount option, we will completely disable all
discard, causing the above mismatch.
[FIX]
Fix it by introducing $DEFAULT_NODISCARD_OPTS specifically for
"nodiscard" mount option.
If async discard is not enabled by default, $DEFAULT_NODISCARD_OPTS will
be the same as $DEFAULT_OPTS, thus everything would work as usual.
If async discard is enabled by default, $DEFAULT_NODISCARD_OPTS will
have that removed, so for "nodiscard" we can use $DEFAULT_NODISCARD_OPTS
as expected mount options.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Mon, 19 Dec 2022 20:25:29 +0000 (12:25 -0800)]
btrfs: new test for logical inode resolution panic
If we create a file that has an inline extent followed by a prealloc
extent, then attempt to use the logical to inode ioctl on the prealloc
extent, but in the overwritten range, backref resolution will process
the inline extent. Depending on the leaf eb layout, this can panic.
Add a new test for this condition. In the long run, we can add spew when
we read out-of-bounds fields of inline extent items and simplify this
test to look for dmesg warnings rather than trying to force a fairly
fragile panic (dependent on non-standardized details of leaf layout).
The test causes a kernel panic unless:
btrfs: fix logical_ino ioctl panic
is applied to the kernel.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 00:21:42 +0000 (16:21 -0800)]
xfs/122: fix EFI/EFD log format structure size after flex array conversion
Adjust this test since made EFI/EFD log item format structs proper flex
arrays instead of array[1].
This adjustment was made to the kernel source tree as part of a project
to make the use of flex arrays more consistent throughout the kernel.
Converting array[1] and array[0] to array[] also avoids bugs in various
compiler ports that mishandle the array size computation. Prior to the
introduction of xfs_ondisk.h, these miscomputations resulted in kernels
that would silently write out filesystem structures that would then not
be recognized by more mainstream systems (e.g. x86).
OFC nearly all those reports about buggy compilers are for tiny
architectures that XFS doesn't work well on anyways, so in practice it
hasn't created any user problems (AFAIK).
[zlang: Add more comments to new helpers]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Hironori Shiina [Wed, 21 Dec 2022 22:38:05 +0000 (17:38 -0500)]
xfs: Test bulkstat special query for root inode
This is a test for the fix: bf3cb3944792 xfs: allow single bulkstat of special inodes
This fix added a feature to query the root inode number of a filesystem.
This test creates a file with a lower inode number than the root and run
a query for the root inode.
Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 13 Dec 2022 19:45:33 +0000 (11:45 -0800)]
fuzzy: don't fail on compressed metadumps
This line in __scratch_xfs_fuzz_mdrestore:
test -e "${POPULATE_METADUMP}"
Breaks spectacularly on a setup that uses DUMP_COMPRESSOR to compress
the metadump files, because the metadump files get the compression
program added to the name (e.g. "${POPULATE_METADUMP}.xz"). The check
is wrong, and since the naming policy is an implementation detail of
_xfs_mdrestore, let's get rid of the -e test.
However, we still need a way to fail the test if the metadump cannot be
restored. _xfs_mdrestore returns nonzero on failure, so use that
instead.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Sat, 17 Dec 2022 08:18:34 +0000 (00:18 -0800)]
common/populate: move decompression code to _{xfs,ext4}_mdrestore
Move the metadump decompression code to the per-filesystem mdrestore
commands so that everyone can take advantage of them. This enables the
XFS and ext4 _mdrestore helpers to handle metadata dumps compressed with
their respective _metadump helpers.
In turn, this means that the xfs fuzz tests can now handle the
compressed metadumps created by the _scratch_populate_cached helper.
This is key to unbreaking fuzz testing for xfs.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Thu, 15 Dec 2022 04:21:41 +0000 (12:21 +0800)]
src/vfs/utils: Reset errno to zero when detect O_TMPFILE
For some filesystem that doesn't support O_TMPFILE, it will pass
ENOTSUP errno to upper layer. so it will report the following error:
QA output created by 696
vfstest.c: 1818: setgid_create_umask - Success - failure: is_setgid
vfstest.c: 2421: run_test - Operation not supported - failure...
To fix this, just reset errno before return.
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Leah Rumancik [Wed, 14 Dec 2022 21:59:26 +0000 (13:59 -0800)]
check: wipe tmp.arglist
Make sure tmp.arglist is wiped before each run to avoid
accidentally rerunning tests.
Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Mon, 12 Dec 2022 23:08:20 +0000 (00:08 +0100)]
check: ensure sect_stop is initialized if interrupted
sect_stop is normally set immediately prior to calling _wrapup() via
run_section(). However, when called via a trap signal handler,
sect_stop may be uninitialized, leading to a negative section time
(sect_stop - sect_start) in the xunit report. E.g.
Interrupted!
Passed all 1 tests
Xunit report: /home/david/xfstests/results//result.xml
rapido1:/# head /home/david/xfstests/results//result.xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="xfstests" failures="0" skipped="0" tests="1"
time="-1670885797" ... >
This commit uses the existing $interrupt flag to determine when
sect_stop needs to be initialised.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 13 Dec 2022 19:45:06 +0000 (11:45 -0800)]
xfs/018: fix attr value setting in this test
I was testing a patch to strengthen the buffer length validation of attr
log intent items during log recovery, when I noticed that the lengths of
the logged values were (mostly) a single byte larger than the alleged
attribute value. Upon further investigation, I noticed this code in the
test:
The 'echo' command generally emits a newline before exiting, which
means that the 16-byte "attr16" value was actually storing 17 bytes.
This affects all the test cases except for the attr64k tests, since the
attr(1) command helpfully/silently truncates the value buffer at 65536
bytes.
Fix the test to store values of exactly the length we want, and add a
couple more test cases to check that everything still works when the
value length is not an exact multiple of sizeof(u32).
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
[CAUSE]
$seqres.dmesg is created (and later deleted depending on config) by
_check_dmesg() function.
But if a test case failed by calling _fail, then we no longer call
_check_dmesg(), thus no dmesg will be saved no matter whatever the
config is.
[FIX]
If the test case itself failed, then still call _check_dmesg() to either
save the dmesg unconditionally (KEEP_DMESG=yes case), or save the dmesg
if there is something wrong (default).
The dmesg can be pretty handy debug clue for both cases.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Ziyang Zhang [Mon, 12 Dec 2022 05:56:45 +0000 (13:56 +0800)]
common/populate: Ensure that S_IFDIR.FMT_BTREE is in btree format
Sometimes "$((128 * dblksz / 40))" dirents cannot make sure that
S_IFDIR.FMT_BTREE could become btree format for its DATA fork.
Actually we just observed it can fail after apply our inode
extent-to-btree workaround. The root cause is that the kernel may be
too good at allocating consecutive blocks so that the data fork is
still in extents format.
Therefore instead of using a fixed number, let's make sure the number
of extents is large enough than (inode size - inode core size) /
sizeof(xfs_bmbt_rec_t).
Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Suggested-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Ziyang Zhang <ZiyangZhang@linux.alibaba.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Ziyang Zhang [Mon, 12 Dec 2022 05:56:44 +0000 (13:56 +0800)]
common/xfs: Add a helper to export inode core size
Some xfs test cases need the number of bytes reserved for only the inode
record, excluding the immediate fork areas. Now the value is hard-coded
and it is not a good chioce. Add a helper in common/xfs to export the
inode core size.
Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Ziyang Zhang <ZiyangZhang@linux.alibaba.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Shiyang Ruan [Sat, 10 Dec 2022 02:31:05 +0000 (02:31 +0000)]
fstests: batch update of file mode and untracked files
1. The executable binary in src/ should be in ignore file.
2. tests/xfs/216.out is a symlink generated at runtime, also should be
ignored.
3. tests/generic/692 was created with 644. It should be 755 otherwise
its mode will be changed and dirty the git tree after test.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com>
Qu Wenruo [Fri, 9 Dec 2022 05:34:02 +0000 (13:34 +0800)]
fstests: btrfs/219: remove it from auto group
The test case is to make sure we can mount a fs with older generation
(but with the same fsid/dev uuid).
Normally we will reject such case as btrfs is maintaining an internal
devices list (for multi-device support), and if we find a device
suddenly got an older generation, we will directly reject it.
Although for single device btrfs, we may add an exception for it,
the corresponding kernel patch is never merged.
So for now, just remove the test case from auto group.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 9 Dec 2022 06:19:01 +0000 (14:19 +0800)]
fstests: btrfs/080: fix the stray '\'
[BUG]
The latest grep will report stray '\', causing golden output mismatch
for btrfs/080:
btrfs/080 - output mismatch (see ~/xfstests-dev/results//btrfs/080.out.bad)
--- tests/btrfs/080.out 2022-11-24 19:53:53.137469203 +0800
+++ ~/xfstests-dev/results//btrfs/080.out.bad 2022-12-09 11:41:46.194597311 +0800
@@ -1,2 +1,3 @@
QA output created by 080
+grep: warning: stray \ before -
Silence is golden
...
(Run 'diff -u ~/xfstests-dev/tests/btrfs/080.out ~/xfstests-dev/results//btrfs/080.out.bad' to see the entire diff)
[CAUSE]
Even for regrex of grep, '-' doesn't need special escape, thus
"\bno\-holes\b" indeed has an unnecessary '\' before '-'.
[FIX]
Just remove the stray '\'.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Wed, 30 Nov 2022 17:01:53 +0000 (18:01 +0100)]
generic/273: Limit number of files by available inodes
Test generic/273 is failing for ext4 with 1k blocksize because it is
creating more files than we have available inodes. Just limit the number
of files created to the number of inodes.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Mon, 5 Dec 2022 12:41:44 +0000 (13:41 +0100)]
ext4/044: Fix failure when mount options are incompatible with ext3
There are some mount options that are incompatible with ext3 filesystem
type. If they are used, this test fails because it tries to remount the
filesystem as ext3. The test makes sense even without remounting as ext3
so just make the test silently skip the remount.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 28 Nov 2022 12:07:24 +0000 (12:07 +0000)]
btrfs: test a case with compressed send stream and a shared extent
Test that if we have a snapshot with a compressed extent that is partially
shared between two files, one of them has a size that is not sector size
aligned, we create a v2 send stream for the snapshot with compressed data,
and then apply that stream to another filesystem, the operation succeeds
and no data is missing. Also check that the file that had a reference to
the whole extent gets two compressed extents in the new filesystem, with
only one of them being shared (reflinked).
This tests a recent patch that landed in kernel 6.1-rc7:
a11452a3709e ("btrfs: send: avoid unaligned encoded writes when attempting to clone range")
Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 28 Nov 2022 12:07:23 +0000 (12:07 +0000)]
btrfs/280: also verify that fiemap reports extents as encoded
Now that _filter_fiemap_flags() optionally reports the encoded flag and
since btrfs/280 explicitly uses and tests compression, make it check that
fiemap reports the compressed extents with the encoded flag set.
Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 28 Nov 2022 12:07:22 +0000 (12:07 +0000)]
common: make _filter_fiemap_flags optionally print the encoded flag
We'd like to have some btrfs test cases in the future to verify that
extents are compressed when using fiemap. For that we can just check if
the FIEMAP_EXTENT_ENCODED (0x8) flag is set for an extent. Currently
_filter_fiemap_flags does not print that flag, so this changes it to
print the flag.
However printing the encoded flag is optional, because some tests use
the filter and use its output to match the golden output. So always
printing the flag would make the tests fail on btrfs when they are run
with "-o compress" (or compress-force) set in MOUNT_OPTIONS due to a
mismatch with the golden output. The tests that can be run with or
without compression on btrfs are generic/352, generic/353 and btrfs/279.
Since those tests don't care about the encoded flag, there is no need to
change them, just make the output of the flag optional, and any future
tests that want to check the presence of the encoded flag, will just pass
a parameter to _filter_fiemap_flags to tell it that the encoded flag
should be printed.
Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
fstests: skip btrfs/254 in case MIN_FSSIZE is more than 1G
The test-case btrfs/254 creates a 1G device-mapper setup, but this might
be too small for the filesystem to actually operate (i.e. in case of a
zoned block device which needs at least 5 zones).
Skip the test if MIN_FSSIZE is set to a value above 1G.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Tue, 22 Nov 2022 14:12:19 +0000 (15:12 +0100)]
generic/614: Run test only for filesystems supporting delalloc
Simple filesystems such as ext2 or udf do not support delayed
allocation. Thus they allocate data blocks for mmap writes only during
writeback. This makes test generic/614 fail because it checks exactly
whether the blocks for the write are reserved before writeback happens.
Make the test depend on delayed allocation support.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Tue, 8 Nov 2022 18:32:42 +0000 (02:32 +0800)]
fstests: update group name according to xfs_io command requirement
When a test case requires someone xfs_io command, that nearly means
that case belong that kind of test group. Likes fpunch for punch
group, fcollapse for collapse group, falloc for prealloc group, fzero
for zero group and so on.
Many fstests cases miss some test groups they should belong to, so
this patch trys to supplement this lack, according to the "xxxx"
which required by _require_xfs_io_command "xxxx".
LuÃs Henriques [Tue, 15 Nov 2022 14:45:00 +0000 (14:45 +0000)]
ceph/005: skip test if using "test_dummy_encryption"
When using the "test_dummy_encryption" mount option, new file and directory
names will be encrypted. This means that if using as a mount base directory
a newly created directory, we would have to use the encrypted directory name
instead. For the moment, ceph doesn't provide a way to get this encrypted
file name, thus for now simply skip this test.
Signed-off-by: LuÃs Henriques <lhenriques@suse.de> Reviewed-by: Xiubo Li <xiubli@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiao Yang [Mon, 14 Nov 2022 08:35:03 +0000 (08:35 +0000)]
generic/470: Replace thin volume with blkdiscard -z
generic/470 was original designed to verify mmap(MAP_SYNC) which
is only valid to the DAX capable device(e.g. PMEM). Thin volume[1] was
introduced to fix the inconsistent filesystem issue[2] but it make
the test become not run because it doesn't support DAX. As Darrick
mentioned[3], zeroing the entire mapped range of scartch device
can fix the issue as well, so I try to use blkdiscard -z instead.
Filipe Manana [Wed, 9 Nov 2022 16:44:58 +0000 (16:44 +0000)]
btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
Starting with btrfs-progs v6.0, the defrag command now prints to stdout
the full path of the files it processes. This makes test cases btrfs/021
and btrfs/256 fail because they don't expect any output from the defrag
command.
The change happened with the following commit in btrfs-progs:
dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")
So update the tests to ignore the stdout of the defrag command.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 9 Nov 2022 16:44:57 +0000 (16:44 +0000)]
btrfs/053: fix test failure when running with btrfs-progs v6.0+
In btrfs-progs v6.0 the --leafsize (-l) command line option was removed
from mkfs.btrfs, so btrfs/053 can fail with v6.0+ in case the scratch
device does not have a btrfs filesystem created before running the test,
in which case mounting the scratch device fails.
The change was introduced by the following btrfs-progs commit:
f7a768d62498 ("btrfs-progs: mkfs: remove support for option --leafsize")
Change the test to use --nodesize (-n) instead, since it exists in both
old and new btrfs-progs versions. Also redirect mkfs output to the test's
log file and fail explicitly if mkfs failed.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 9 Nov 2022 16:44:56 +0000 (16:44 +0000)]
btrfs/003: fix failure on new btrfs-progs versions
Starting with btrfs-progs version 5.19, the output of 'filesystem show'
command changed when we have a missing device. The old output was like the
following:
Label: none uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
*** Some devices missing
While the new output (btrfs-progs 5.19+) is like the following:
Label: none uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
devid 2 size 0 used 0 path /dev/loop1 MISSING
More specifically it happened in the following btrfs-progs commit:
957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")
This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep
filter in btrfs/003 so that it works with both output formats.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
[FAILURES]
The following btrfs test cases failed with newer btrfs-progs:
- btrfs/197
- btrfs/198
- btrfs/254
They all fail due to output mismatch like the following:
Label: none uuid: <UUID>
Total devices <NUM> FS bytes used <SIZE>
devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
- *** Some devices missing
+ devid <DEVID> size 0 used 0 path MISSING
[CAUSE]
Since btrfs-progs commit 957a79c9b016 ("btrfs-progs: fi show: print
missing device for a mounted file system"), we output the detailed info
of a missing device if "btrfs filesystem show" is executed using
"-m <mnt>" option.
Such detailed output no longer follows the old format, thus causing the
output mismatch.
[FIX]
Update _filter_btrfs_filesystem_show() to handle detailed missing
device by:
- Buffer the output first
- Output the first two lines
Which is always label/uuid and the total device accounting.
- Replace the detailed missing device line with old output
- Sort (in reverse order) and uniq the device list
By this we can handle both old and new output correctly.
Although this means we lacks the ability to detect mutltiple missing
devices, thankfully the involved test cases are not checking this yet.
[ Zorro: add "rm -f $tmp.btrfs_filesystem_show" ]
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Wed, 9 Nov 2022 13:07:46 +0000 (21:07 +0800)]
generic: shutdown might leave NULL files with nonzero di_size
An old issue might cause on-disk inode sizes are logged prematurely
via the free eofblocks path on file close. Then fs shutdown might
leave NULL files but their di_size > 0.
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 7 Nov 2022 09:38:58 +0000 (09:38 +0000)]
generic: check direct IO writes with io_uring and O_DSYNC are durable
Test that direct IO writes with io_uring and O_DSYNC are durable if a
power failure happens after they complete.
This is motivated by a regression on btrfs, affecting 5.15 stable
kernels and kernels up to 6.0, where often the writes were not
persisted (same behaviour as if O_DSYNC was not provided). This was
recently fixed by the following commit:
8184620ae212 ("btrfs: fix lost file sync on direct IO write with
nowait and dsync iocb")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Whitney [Fri, 21 Oct 2022 21:19:50 +0000 (17:19 -0400)]
generic: add missing $FSX_AVOID to fsx invocations
generic/455 fails when run on an ext4 bigalloc file system. Its
fsx invocations can make insert range and collapse range calls whose
arguments are not cluster aligned, and ext4 will fail those calls for
bigalloc. They can be suppressed by adding the FSX_AVOID environment
variable to the fsx invocation and setting its value appropriately in
the test environment, as is done for other fsx-based tests. This
avoids the need to exclude the test to avoid failures and makes it
possible to take advantage of the remainder of its coverage.
[ Also fix generic/457, as requested by Dave Chinner -- TYT]
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:30 +0000 (13:58 -0700)]
tests: fix some tests for systems with fs.verity.require_signatures=1
Some of the newer verity tests don't work properly on systems where
fs.verity.require_signatures is enabled, either because they forget to
disable it at the beginning of the test, or they forget to re-enable it
afterwards, or both. Fix this.
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:29 +0000 (13:58 -0700)]
generic/577: add missing file removal before empty file test
The fix for _fsv_have_hash_algorithm() exposed a bug where one of the
test cases in generic/577 isn't deleting the file from the previous test
case before it tries to write to it. That causes a failure, since due
to the fix for _fsv_have_hash_algorithm(), the file from the previous
test case now ends up with verity enabled and therefore cannot be
written to. Fix this by deleting the file.
Reported-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:28 +0000 (13:58 -0700)]
common/verity: fix _fsv_have_hash_algorithm() with required signatures
_fsv_have_hash_algorithm() uses _fsv_enable() without a signature, so it
always fails when called while fs.verity.require_signatures=1. This
happens in generic/577, which tests file signing. This wasn't noticed
because it just made part of generic/577 always be skipped.
Fix this by making _fsv_have_hash_algorithm() temporarily set
fs.verity.require_signatures to 0.
Since the previous value needs to be restored afterwards, whether it is
0 or 1, also make some changes to the fs.verity.require_signatures
helper functions to allow the restoration of the previous value, rather
than the value that existed at the beginning of the test.
Finally, make a couple related cleanups: make _fsv_have_hash_algorithm()
always delete the file it works with, and also update the similar code
in _require_scratch_verity().
Reported-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 3 Nov 2022 04:17:06 +0000 (21:17 -0700)]
xfs: test xfs_scrub phase 6 media error reporting
Add new helpers to dmerror to provide for marking selected ranges
totally bad -- both reads and writes will fail. Create a new test for
xfs_scrub to check that it reports media errors in data files correctly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:21 +0000 (11:11 +0000)]
btrfs: test fiemap reports extent as not shared after COWing it in snapshot
Test that if we have a large file, with file extent items spanning several
leaves in the fs tree, and that is shared due to a snapshot, if we COW one
of the extents, doing a fiemap will report the respective file range as
not shared.
This exercises the processing of delayed references for metadata extents
in the backref walking code, used by fiemap to determine if an extent is
shared.
This used to fail until very recently and was fixed by the following
kernel commit that landed in 6.1-rc2:
943553ef9b51 (""btrfs: fix processing of delayed tree block refs during backref walking")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:20 +0000 (11:11 +0000)]
btrfs: test that fiemap reports extent as not shared after deleting file
Test that if we have two files with shared extents, after removing one of
the files, if we do a fiemap against the other file, it does not report
extents as shared anymore.
This exercises the processing of delayed references for data extents in
the backref walking code, used by fiemap to determine if an extent is
shared.
This used to fail until very recently and was fixed by the following
kernel commit that landed in 6.1-rc2:
4fc7b5722824 (""btrfs: fix processing of delayed data refs during backref walking")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:19 +0000 (11:11 +0000)]
common/punch: fix flags printing for filter _filter_fiemap_flags
In the filter _filter_fiemap_flags, if we get a flags field that only has
the 'last' flag set, we end up printing the string "nonelast", which is
ugly and not intuitive.
So fix this by updating the filter's awk code to reset the flags string to
an empty string if we have the "last" flag set and we haven't updated the
flags string before. So now the same test gives the following result:
0: [0..127]: last
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiao Yang [Thu, 27 Oct 2022 10:50:52 +0000 (10:50 +0000)]
common/config: Make test and scratch devices use the same mount options
Some cases(e.g. generic/519) check commands/features on test device but
do tests on scratch device. If some mount options can impact the check
result, these cases may throw error instead if not run when we use
different mount options for test and scratch devices.
Signed-off-by: Xiao Yang <yangx.jy@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Fri, 28 Oct 2022 15:43:37 +0000 (23:43 +0800)]
xfs: new test on xfs with corrupted sb_inopblock
There's a known bug fix 392c6de98af1 ("xfs: sanitize sb_inopblock in
xfs_mount_validate_sb"). So try to corrupt the sb_inopblock of xfs,
to cover this bug.
Signed-off-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Darrick J. Wong [Fri, 28 Oct 2022 17:42:10 +0000 (10:42 -0700)]
common: simplify grep pipe sed interactions
Zorro pointed out that the idiom "program | grep | sed" isn't necessary
for field extraction -- sed is perfectly capable of performing a
substitution and only printing the lines that match that substitution.
Do that for the common helpers.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
There are a lot of places where we open-code detecting the realtime
extent size and extent count of a specific filesystem. Refactor this
into a couple of helpers to clean up the code.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
There are a lot of places where we open-code determining the directory
block size for a specific filesystem. Refactor this into a single
helper to clean up existing tests.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 28 Oct 2022 17:41:53 +0000 (10:41 -0700)]
xfs: refactor filesystem feature detection logic
There are a lot of places where we open-code detecting features of a
specific filesystem. Refactor this into a couple of helpers in
preparation for adding stress tests for online repair and fuzzing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Hironori Shiina [Mon, 17 Oct 2022 15:11:51 +0000 (11:11 -0400)]
xfs: test for fixing wrong root inode number in dump
Test '-x' option of xfsrestore. With this option, a wrong root inode
number in a dump file is corrected. A root inode number can be wrong
in a dump created by problematic xfsdump (v3.1.7 - v3.1.9) with
bulkstat misuse. In this test, a corrupted dump file is created by
overwriting a root inode number in a header.
Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Josef Bacik [Wed, 24 Aug 2022 19:32:10 +0000 (15:32 -0400)]
fstests: get section config after RUN_SECTION checks
While trying to do
./check -s <some section>
I was failing because I had a section defined higher than <some section>
that had TEST_DEV=/some/nonexistent/device, since I was using the other
section to test an experimental drive. This appears to be because we
run through all of the sections, and when getting the section config we
check to see if it's valid, and in this case the section wasn't valid.
The section I was actually trying to use was valid however. Fix check
to see if the section we're trying to run is in our list of sections to
run first, and then if it is get the config at that point.
Signed-off-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 12 Oct 2022 01:45:39 +0000 (18:45 -0700)]
populate: reformat external ext[34] journal devices when restoring a cached image
The fs population code has the ability to save cached metadumps of
filesystems to save time when running fstests. The cached images should
be unmounted cleanly, so we never save the contents of external journal
devices.
Unfortunately, the cache restore code fails to reset the external
journal when restoring a clean image, so we ignore cached images because
the journal doesn't match the filesystem. This makes test runtimes
longer than they need to be.
Solve this by reformatting the external journal to match the filesystem.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 12 Oct 2022 01:45:33 +0000 (18:45 -0700)]
populate: wipe external xfs log devices when restoring a cached image
The fs population code has the ability to save cached metadumps of
filesystems to save time when running fstests. The cached images should
be unmounted cleanly, so we never save the contents of external log
devices.
Unfortunately, the cache restore code fails to wipe the external log
when restoring a clean image, so we end up with strange test failures
because the log doesn't match the filesystem:
Theodore Ts'o [Thu, 13 Oct 2022 14:48:44 +0000 (10:48 -0400)]
ext4/053: remove test for i_version mount option
The i_version mount option is deprecated in favor of iversion and is
advertised to be removed in 5.20. It was removed in commit 1ff20307393e ("ext4: unconditionally enable the i_version counter") in
Linux v6.1. So remove the test from ext4/053.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Sat, 15 Oct 2022 04:57:05 +0000 (12:57 +0800)]
check: optionally compress core dumps
Add a new option, COREDUMP_COMPRESSOR, that will be used to compress
core dumps collected during a fstests run. The program specified must
accept the -f -9 arguments that gzip has.
Darrick J. Wong [Fri, 14 Oct 2022 18:20:20 +0000 (11:20 -0700)]
check: detect and preserve all coredumps made by a test
If someone sets kernel.core_uses_pid (or kernel.core_pattern), any
coredumps generated by fstests might have names that are longer than
just "core". Since the pid isn't all that useful by itself, let's
record the coredumps by hash when we save them, so that we don't waste
space storing identical crash dumps.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Tue, 11 Oct 2022 09:23:39 +0000 (11:23 +0200)]
lsqa.pl: fix --header output
7e98d41a ("fstests: move test group info to test files") replaces the
post-header "seq" assignment with "./common/preamble\n_begin_fstest ..."
Update lsqa.pl to recognise the "./common/preamble" line as a header
divider.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 11 Oct 2022 12:22:03 +0000 (13:22 +0100)]
generic: test fiemap reports extent as shared after cloning it
Test that if we have two consecutive extents and only one of them is
cloned, then fiemap correctly reports which one is shared and reports
the other as not shared.
This currently fails on btrfs for all kernel releases, but is fixed by
a kernel patch that landed in Linus' tree last week:
ac3c0d36a2a2f7 ("btrfs: make fiemap more efficient and accurate reporting extent sharedness")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 7 Oct 2022 13:53:36 +0000 (14:53 +0100)]
fstests: add fiemap group
Add a fiemap group for all tests that exercise fiemap, either directly
through xfs_io, or more indirectly like through filefrag or helpers in
common/* (like _count_extents, _count_holes, etc). This is useful in
order to quickly test changes made to the fiemap implementation of a
filesystem for example.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 7 Oct 2022 13:53:35 +0000 (14:53 +0100)]
fstests: add missing require of xfs_io fiemap command to some tests
btrfs/257, btrfs/258, btrfs/259 and xfs/443 use the fiemap command of
xfs_io but don't do a '_require_xfs_io_command "fiemap"'. So add the
missing requirement.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Due to some braino on my part, _scratch_populate_cached will metadump
the filesystem twice -- once with compression disabled, and again with
it enabled, maybe. Get rid of the first metadump.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 5 Oct 2022 22:30:55 +0000 (15:30 -0700)]
xfs/128: try to force file allocation behavior
Over the years, I've noticed that this test occasionally fails when I've
programmed the allocator to hand out the minimum amount of space with
each allocation or if extent size hints are enabled:
--- /tmp/fstests/tests/xfs/128.out 2022-09-01 15:09:11.506679341 -0700
+++ /var/tmp/fstests/xfs/128.out.bad 2022-10-04 17:32:50.992000000 -0700
@@ -20,7 +21,9 @@ 56ed2f712c91e035adeeb26ed105a982 SCRATCH_MNT/test-128/file3 b81534f439aac5c34ce3ed60a03eba70 SCRATCH_MNT/test-128/file4
Check files
free blocks after creating some reflink copies is in range
free blocks after CoW some reflink copies is in range
-free blocks after defragging all reflink copies is in range
-free blocks after all tests is in range
+free blocks after defragging all reflink copies has value of 8620027
+free blocks after defragging all reflink copies is NOT in range 8651819 .. 8652139
+free blocks after all tests has value of 8620027
+free blocks after all tests is NOT in range 8651867 .. 8652187
It turns out that under the right circumstances, the _pwrite_byte at the
start of this test will end up allocating two extents to file1. This
almost never happens when delalloc is enabled or when the extent size is
large, and is more prone to happening if the extent size is > 1FSB but
small, the allocator hands out small allocations, or if writeback shoots
down pages in random order.
When file1 gets more than 1 extent, problems start to happen. The free
space accounting checks at the end of the test assume that file1 and
file4 still share the same space at the end of the test. This
definitely happens if file1 gets one extent (since fsr ignores
single-extent files), but if there's more than 1, fsr will try to
defragment it. If fsr succeeds in copying the file contents to a temp
file with fewer extents than the source file, it will switch the
contents, but unsharing the contents in the process. This cause the
free space to be lower than expected, and the test fails.
Resolve this situation by preallocating space beforehand to try to set
up file1 with a single space extent. If the test fails and we got more
than one extent, note that in the output.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 5 Oct 2022 22:30:49 +0000 (15:30 -0700)]
xfs/229: do not _xfs_force_bdev on TEST_DIR
In commit ea15099b71, I observed that this test tries to test the
behavior of the extent size hint on the data device. If the test runner
set up MKFS_OPTIONS such that the filesystem gets created with a
realtime section and rtinherit set on the root directory, then the
preconditions of this test (creating files on the data section) is not
satisfied and the results of this test are incorrect. The solution was
to force all files created by this test to be assigned to the data
section.
Unfortunately, the correction that I made has side effects beyond this
test -- by clearing rtinherit on $TEST_DIR, all tests that run after
this one will create files on the data section, because the test
filesystem persists for the duration of the entire test run. This leads
to the wrong things being tested.
Fix this new problem by clearing the rtinherit flag on $TDIR, which
contains the files created by this test and is removed during cleanup,
and leave a few comments celebrating our new discoveries.
Fixes: ea15099b71 ("xfs: force file creation to the data device for certain layout tests") Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / xfstests-dev.git / log