Darrick J. Wong [Wed, 18 Jan 2023 00:44:18 +0000 (16:44 -0800)]
populate: improve attr creation runtime
Replace the file creation loops with a python script that does
everything we want from a single process. This reduces the runtime of
_scratch_xfs_populate substantially by avoiding thousands of execve
overhead. This patch builds on the previous one by reducing the runtime
of xfs/349 from ~45s to ~15s.
For people who don't have python3, use setfattr's "restore" mode to bulk
create xattrs. This reduces runtime to about ~25s.
[zlang: add popattr.py into src/Makefile install list]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 18 Jan 2023 00:44:02 +0000 (16:44 -0800)]
populate: remove file creation loops that take forever
Replace the file creation loops with a perl script that does everything
we want from a single process. This reduces the runtime of
_scratch_xfs_populate substantially by avoiding thousands of execve
overhead. On my system, this reduces the runtime of xfs/349 (with scrub
enabled) from ~140s to ~45s.
[zlang: add popdir.pl into src/Makefile install list]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Dave Chinner [Wed, 18 Jan 2023 00:43:47 +0000 (16:43 -0800)]
populate: ensure btree directories are created reliably
The population function creates an XFS btree format directory by
polling the extent count of the inode and creating new dirents until
the extent count goes over the limit that pushes it into btree
format.
It then removes every second dirent to create empty space in the
directory data to ensure that operations like metadump with
obfuscation can check that they don't leak stale data from deleted
dirents.
Whilst this does not result in directory data blocks being freed, it
does not take into account the fact that the dabtree index has half
the entries removed from it and that can result in btree nodes
merging and extents being freed. This causes the extent count to go
down, and the inode is converted back into extent form. The
population checks then fail because it should be in btree form.
Fix this by counting the number of directory data extents rather than
the total number of extents in the data fork. We can do this simply
by using xfs_bmap and counting the number of extents returned as it
does not report extents beyond EOF (which is where the dabtree is
located). As the number of data blocks does not change with the
dirent removal algorithm used, this will ensure that the inode data
fork remains in btree format.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: make this patch first in line] Signed-off-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 18 Jan 2023 00:43:16 +0000 (16:43 -0800)]
xfs/{080,329,434,436}: add missing check for fallocate support
Don't run this test if the filesystem doesn't support fallocate. This
is only ever the case if always_cow is enabled.
The same logic applies to xfs/329, though it's more subtle because the
test itself does not explicitly invoke fallocate; rather, it is xfs_fsr
that requires fallocate.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 18 Jan 2023 00:43:00 +0000 (16:43 -0800)]
xfs: skip fragmentation tests when alwayscow mode is enabled
If the always_cow debugging flag is enabled, all file writes turn into
copy writes. This dramatically ramps up fragmentation in the filesystem
(intentionally!) so there's no point in complaining about fragmentation.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 18 Jan 2023 00:42:44 +0000 (16:42 -0800)]
xfs/182: fix spurious direct write failure
This test has some weird behavior that causes regressions when fsdax and
reflink are enabled. The goal of this test is to set a cow extent size
hint, perform some random directio writes, perform a directio rewrite of
the entire file, and make sure that the file content (and extent count)
are sane afterwards.
Most of the time, the random directio writes will never touch the
8388609th byte, though if they do randomly select that EOF block, they'd
end up extending the file by $real_blksz bytes and causing spurious test
failures.
Then, the rewrite does this:
pwrite -S 0x63 -b $real_blksz 0 $((filesize + 1))
Note that we previously set filesize=8388608, which means that we're
asking for a series of direct writes that fill the first 8388608 bytes
with 'c'. The last write in the series becomes a single byte direct
write. For regular file access mode, this last write will fail with
EINVAL, since block devices do not support byte granularity writes and
XFS does not fall back to the pagecache for unaligned direct wites.
Hence we never wrote the 8388609th byte of the file.
However, fsdax *does* allow byte-granularity direct writes, which means
that the single-byte write succeeds. There is no EINVAL return code,
and the 8388609th byte of the file is now 'c' instead of 'a'. As a
result, the md5 of file2 is different.
Since fsdax+reflink is the newcomer, amend the direct writes in this
test so that they always end at the 8388608th byte, since we were never
really testing that last byte anyway. This makes the test behavior
consistent across both access modes.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Xiao Yang <yangx.jy@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Ojaswin Mujoo [Wed, 11 Jan 2023 20:58:28 +0000 (12:58 -0800)]
generic/692: generalize the test for non-4K Merkle tree block sizes
Due to the assumption of the Merkle tree block size being 4K, the file
size calculated for the second test case was taking way too long to hit
EFBIG in case of larger block sizes like 64K. Fix this by generalizing
the calculation to any Merkle tree block size >= 1K.
Signed-off-by: Ojaswin Mujoo <ojaswin@linux.ibm.com> Co-developed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Over mutiple kernel releases we have reworked setgid inheritance
significantly due to long-standing security issues, security issues that
were reintroduced after they were fixed, and the subtle and difficult
inheritance rules that plagued individual filesystems. We have lifted
setgid inheritance into the VFS proper in earlier patches. Starting with
kernel v6.2 we have made setgid inheritance consistent between the write
and setattr (ch{mod,own}) paths.
The gist of the requirement is that in order to inherit the setgid bit
the user needs to be in the group of the file or have CAP_FSETID in
their user namespace. Otherwise the setgid bit will be dropped
irregardless of the file's executability. Remove the obsolete tests as
they're not a security issue and will cause spurious warnings on older
distro kernels.
Note, that only with v6.2 setgid inheritance works correctly for
overlayfs in the write path. Before this the setgid bit was always
retained.
Darrick J. Wong [Fri, 30 Dec 2022 22:12:58 +0000 (14:12 -0800)]
xfs: race fsmap with readonly remounts to detect crash or livelock
Add a new test that races the GETFSMAP ioctl with ro/rw remounting to
make sure we don't livelock on the empty transaction that fsmap uses to
avoid deadlocking on rmap btree cycles.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:55 +0000 (14:12 -0800)]
fuzzy: delay the start of the scrub loop when stress-testing scrub
By default, online fsck stress testing kicks off the loops for fsstress
and online fsck at the same time. However, in certain debugging
scenarios it can help if we let fsstress get a head-start in filling up
the filesystem. Plumb in a means to delay the start of the scrub loop.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:55 +0000 (14:12 -0800)]
fuzzy: allow substitution of AG numbers when configuring scrub stress test
Allow the test program to use the metavariable '%agno%' when passing
scrub commands to the scrub stress loop. This makes it easier for tests
to scrub or repair every AG in the filesystem without a lot of work.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:54 +0000 (14:12 -0800)]
fuzzy: clean up frozen fses after scrub stress testing
Some of our scrub stress tests involve racing scrub, fsstress, and a
program that repeatedly freeze and thaws the scratch filesystem. The
current cleanup code suffers from the deficiency that it doesn't
actually wait for the child processes to exit. First, change it to do
that.
However, that exposes a second problem: there's a race condition with a
freezer process that leads to the stress test exiting with a frozen fs.
If the freezer process is blocked trying to acquire the unmount or
sb_write locks, the receipt of a signal (even a fatal one) doesn't cause
it to abort the freeze. This causes further problems with fstests,
since ./check doesn't expect to regain control with the scratch fs
frozen.
Fix both problems by making the cleanup function smarter.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:54 +0000 (14:12 -0800)]
fuzzy: increase operation count for each fsstress invocation
For online fsck stress testing, increase the number of filesystem
operations per fsstress run to 2 million, now that we have the ability
to kill fsstress if the user should push ^C to abort the test early.
This should guarantee a couple of hours of continuous stress testing in
between clearing the scratch filesystem.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:54 +0000 (14:12 -0800)]
fuzzy: clear out the scratch filesystem if it's too full
If the online fsck stress tests run for long enough, they'll fill up the
scratch filesystem completely. While it is interesting to test repair
functionality on a *nearly* full filesystem undergoing a heavy workload,
a totally full filesystem is really only exercising the ENOSPC handlers
in the kernel. That's not what we came here to test, so change the
fsstress loop to detect a nearly full filesystem and erase everything
before starting fsstress again.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:54 +0000 (14:12 -0800)]
fuzzy: abort scrub stress testing if the scratch fs went down
There's no point in continuing a stress test of online fsck if the
filesystem goes down. We can't query that kind of state directly, so as
a proxy we try to stat the mountpoint and interpret any error return as
a sign that the fs is down.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:54 +0000 (14:12 -0800)]
fuzzy: make scrub stress loop control more robust
Currently, each of the scrub stress testing background threads
open-codes logic to decide if it should exit the loop. This decision is
based entirely on TIME_FACTOR*30 seconds having gone by, which means
that we ignore external factors, such as the user pressing ^C, which (in
theory) will invoke cleanup functions to tear everything down.
This is not a great user experience, so refactor the loop exit test into
a helper function and establish a sentinel file that must be present to
continue looping. If the user presses ^C, the cleanup function will
remove the sentinel file and kill the background thread children, which
should be enough to stop everything more or less immediately.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:54 +0000 (14:12 -0800)]
fuzzy: test the scrub stress subcommands before looping
Before we commit to running fsstress and scrub commands in a loop for
some time, we should check that the provided commands actually work on
the scratch filesystem. The _require_xfs_io_command predicate only
detects the presence of the scrub ioctl, not any particular subcommand.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:53 +0000 (14:12 -0800)]
fuzzy: rework scrub stress output filtering
Rework the output filtering functions for scrub stress tests: first, we
should use _filter_scratch to avoid leaking the scratch fs details to
the output. Second, for scrub and repair, change the filter elements to
reflect outputs that don't indicate failure (such as busy resources,
preening requests, and insufficient space to do anything). Finally,
change the _require function to check that filter functions have been
sourced.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:53 +0000 (14:12 -0800)]
fuzzy: clean up scrub stress programs quietly
In the cleanup function for online fsck stress test common code, send
SIGINT instead of SIGTERM to the fsstress and xfs_io processes to kill
them. bash prints 'Terminated' to the golden output when children die
with SIGTERM, which can make a test fail, and we don't want a regular
cleanup function being the thing that prevents the test from passing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:12:53 +0000 (14:12 -0800)]
xfs/422: move the fsstress/freeze/scrub racing logic to common/fuzzy
Hoist all this code to common/fuzzy in preparation for making this code
more generic so that we implement a variety of tests that check the
concurrency correctness of online fsck. Do just enough renaming so that
we don't pollute the test program's namespace; we'll fix the other warts
in subsequent patches.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Tue, 10 Jan 2023 22:22:43 +0000 (23:22 +0100)]
common/rc: drop SGI DMF specific _mount_ops_filter
The _mount() helper function is the only caller of _mount_ops_filter(),
which appears to have been used in the past to replace the SGI DMF
specific mtpt= mount option setting.
_mount() invocations could now be replaced with $MOUNT_PROG calls
directly, but I've retained the helper function for readability.
Link: https://irix7.com/techpubs/007-3683-007.pdf Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/012 is requiring ext4 support to test the conversion, but the test
case is only checking if mkfs.ext4 is available, not if the filesystem
driver is actually available on the test host.
Check if the driver is available as well, before trying to run the test.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Thu, 5 Jan 2023 07:18:19 +0000 (15:18 +0800)]
btrfs: add a test case to verify scrub speed throttle works
We introduced scrub speed throttle in commit eb3b50536642 ("btrfs: scrub:
per-device bandwidth control"), but it is not that well documented
(e.g. what's the unit of the sysfs interface), nor tested by any test
case.
This patch will add a test case for this functionality.
The test case itself is pretty straightforward:
- Fill the fs with 2G file as scrub workload
- Scrub without any throttle to grab the initial speed
- Set the throttle to half of the initial speed
- Scrub again and check the speed against the throttle
The test case has an assumption that we can exclusively use all the
performance of the underlying disk.
But for cloud environment it's not ensured 100%, thus the test case is
not included in auto group to avoid false alerts.
Baokun Li [Thu, 29 Dec 2022 13:44:34 +0000 (21:44 +0800)]
overlay: avoid to use NULL OVL_BASE_FSTYP for mounting
Generally, FSTYP is used to specify OVL_BASE_FSTYP. When we specify FSTYP
through an environment variable, it is not converted to OVL_BASE_FSTYP.
In addition, sometimes we do not even specify the file type. For example,
we only use `./check -n -overlay -g auto` to list overlay-related cases.
If OVL_BASE_FSTYP is NULL, mounting fails and the test fails.
To solve this problem, try to assign a value to OVL_BASE_FSTYP when
specifying -overlay. In addition, in the _overlay_base_mount function,
the basic file system type of the overlay is specified only when
OVL_BASE_FSTYP is not NULL.
Reported-by: Murphy Zhou <jencce.kernel@gmail.com> Signed-off-by: Baokun Li <libaokun1@huawei.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:22 +0000 (15:32 -0800)]
generic/575: test 1K Merkle tree block size
In addition to 4K, test 1K Merkle tree blocks. Also always run this
test, regardless of FSV_BLOCK_SIZE, but allow skipping tests of
parameters that are unsupported, unless they are the default.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:21 +0000 (15:32 -0800)]
generic/624: test multiple Merkle tree block sizes
Instead of only testing 4K Merkle tree blocks, test FSV_BLOCK_SIZE, and
also other sizes if they happen to be supported. This allows this test
to run in cases where it couldn't before and improves test coverage in
cases where it did run before.
This required reworking the test to compute the expected digests using
the 'fsverity digest' command, instead of relying on .out file
comparisons. (There isn't much downside to this, since comparison to
known-good file digests already happens in generic/575. So if both the
kernel and fsverity-utils were to be broken in the same way, generic/575
would detect it. generic/624 serves a different purpose.)
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:20 +0000 (15:32 -0800)]
generic/574: test multiple Merkle tree block sizes
Instead of only testing 4K Merkle tree blocks, test FSV_BLOCK_SIZE, and
also other sizes if they happen to be supported. This allows this test
to run in cases where it couldn't before and improves test coverage in
cases where it did run before.
Given that the list of Merkle tree block sizes that will actually work
is not fixed, this required reworking the test to not rely on the .out
file so heavily.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:16 +0000 (15:32 -0800)]
common/verity: add _filter_fsverity_digest()
Add a filter that replaces fs-verity digests with a fixed string. This
is needed because the fs-verity digests that some tests print are going
to start depending on the default Merkle tree block size.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:15 +0000 (15:32 -0800)]
common/verity: use FSV_BLOCK_SIZE by default
Make _fsv_enable() and _fsv_sign() default to FSV_BLOCK_SIZE if no block
size is explicitly specified, so that the individual tests don't have to
do this themselves. This overrides the fsverity-utils default of 4096
bytes, or the page size in older versions of fsverity-utils, both of
which may differ from FSV_BLOCK_SIZE.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:14 +0000 (15:32 -0800)]
common/verity: set FSV_BLOCK_SIZE to an appropriate value
In order to maximize the chance that the verity tests can actually be
run, FSV_BLOCK_SIZE (the default Merkle tree size for the verity tests)
needs to be min(fs_block_size, page_size), not simply page_size. The
only reason that page_size was okay before was because the kernel only
supported merkle_tree_block_size == fs_block_size == page_size anyway.
But that is changing.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Thu, 29 Dec 2022 23:32:13 +0000 (15:32 -0800)]
common/verity: add and use _fsv_can_enable()
Replace _fsv_have_hash_algorithm() with a more general function
_fsv_can_enable() which checks whether 'fsverity enable' with the given
parameters works. For now it is just used with --hash-alg or with no
parameters, but soon it will be used with --block-size too.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 23 Dec 2022 02:56:42 +0000 (10:56 +0800)]
btrfs/154: migrate to python3
Test case btrfs/154 is still using python2 script, which is already EOL.
Some rolling distros like Archlinux is no longer providing python2
package anymore.
This means btrfs/154 will be harder and harder to run.
To fix the problem, migreate the python script to python3, this involves
the following changes:
- Change common/config to use python3
- Strong type conversion between string and bytes
This means, anything involved in the forged bytes has to be bytes.
And there is no safe way to convert forged bytes into string, unlike
python2.
I guess that's why the author went python2 in the first place.
Thankfully os.rename() still accepts forged bytes.
- Use bytes specific checks for invalid chars.
The updated script can still cause the needed conflicts, can be verified
through "btrfs ins dump-tree" command.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 00:22:02 +0000 (16:22 -0800)]
xfs/179: modify test to trigger refcount update bugs
Upon enabling fsdax + reflink for XFS, this test began to report
refcount metadata corruptions after being run. Specifically, xfs_repair
noticed single-block refcount records that could be combined but had not
been.
The root cause of this is improper MAXREFCOUNT edge case handling in
xfs_refcount_merge_extents. When we're trying to find candidates for a
record merge, we compute the refcount of the merged record, but without
accounting for the fact that once a record hits rc_refcount ==
MAXREFCOUNT, it is pinned that way forever.
Adjust this test to use a sub-filesize write for one of the COW writes,
because this is how we force the extent merge code to run.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 16:57:03 +0000 (08:57 -0800)]
xfs: regression test for writes racing with reclaim writeback
This test uses the new write delay debug knobs to set up a slow-moving
write racing with writeback of an unwritten block at the end of the
file range targetted by the slow write. The test succeeds if the file
does not end up corrupt and if the ftrace log captures a message about
the revalidation occurring.
NOTE: I'm not convinced that madvise actually causes the page to be
removed from the pagecache, which means that this is effectively a
functional test for the invalidation, not a corruption reproducer.
On the other hand, the functional test can be done quickly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 00:21:50 +0000 (16:21 -0800)]
xfs: regression test for writeback corruption bug
This is a regression test for a data corruption bug that existed in XFS'
copy on write code between 4.9 and 4.19. The root cause is a
concurrency bug wherein we would drop ILOCK_SHARED after querying the
CoW fork in xfs_map_cow and retake it before querying the data fork in
xfs_map_blocks. See the test description for a lot more details.
Cc: Wengang Wang <wen.gang.wang@oracle.com> Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 9 Dec 2022 06:05:10 +0000 (14:05 +0800)]
btrfs/220: fix the test failure due to new default mount option
[BUG]
The latest misc-next tree will make test case btrfs/220 fail with the
following error messages:
btrfs/220 15s ... - output mismatch (see ~/xfstests/results//btrfs/220.out.bad)
--- tests/btrfs/220.out 2022-05-11 09:55:30.749999997 +0800
+++ ~/xfstests/results//btrfs/220.out.bad 2022-12-09 13:57:23.706666671 +0800
@@ -1,2 +1,5 @@
QA output created by 220
+Unexepcted mount options, checking for 'rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/' in 'rw,relatime,space_cache=v2,subvolid=5,subvol=/' using 'nodiscard'
+Unexepcted mount options, checking for 'rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/' in 'rw,relatime,space_cache=v2,subvolid=5,subvol=/' using 'nodiscard'
+Unexepcted mount options, checking for 'rw,relatime,discard=async,space_cache=v2,subvolid=5,subvol=/' in 'rw,relatime,space_cache=v2,subvolid=5,subvol=/' using 'nodiscard'
Silence is golden
...
(Run 'diff -u ~/xfstests/tests/btrfs/220.out ~/xfstests/results//btrfs/220.out.bad' to see the entire diff)
Ran: btrfs/220
Failures: btrfs/220
Failed 1 of 1 tests
[CAUSE]
Since patch "btrfs: auto enable discard=async when possible", which is
already in the maintainer's tree for next merge window, btrfs will
automatically enable asynchronous discard for devices which supports
discard.
This makes our $DEFAULT_OPTS to have "discard=async" in it.
While for "nodiscard" mount option, we will completely disable all
discard, causing the above mismatch.
[FIX]
Fix it by introducing $DEFAULT_NODISCARD_OPTS specifically for
"nodiscard" mount option.
If async discard is not enabled by default, $DEFAULT_NODISCARD_OPTS will
be the same as $DEFAULT_OPTS, thus everything would work as usual.
If async discard is enabled by default, $DEFAULT_NODISCARD_OPTS will
have that removed, so for "nodiscard" we can use $DEFAULT_NODISCARD_OPTS
as expected mount options.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Mon, 19 Dec 2022 20:25:29 +0000 (12:25 -0800)]
btrfs: new test for logical inode resolution panic
If we create a file that has an inline extent followed by a prealloc
extent, then attempt to use the logical to inode ioctl on the prealloc
extent, but in the overwritten range, backref resolution will process
the inline extent. Depending on the leaf eb layout, this can panic.
Add a new test for this condition. In the long run, we can add spew when
we read out-of-bounds fields of inline extent items and simplify this
test to look for dmesg warnings rather than trying to force a fairly
fragile panic (dependent on non-standardized details of leaf layout).
The test causes a kernel panic unless:
btrfs: fix logical_ino ioctl panic
is applied to the kernel.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 21 Dec 2022 00:21:42 +0000 (16:21 -0800)]
xfs/122: fix EFI/EFD log format structure size after flex array conversion
Adjust this test since made EFI/EFD log item format structs proper flex
arrays instead of array[1].
This adjustment was made to the kernel source tree as part of a project
to make the use of flex arrays more consistent throughout the kernel.
Converting array[1] and array[0] to array[] also avoids bugs in various
compiler ports that mishandle the array size computation. Prior to the
introduction of xfs_ondisk.h, these miscomputations resulted in kernels
that would silently write out filesystem structures that would then not
be recognized by more mainstream systems (e.g. x86).
OFC nearly all those reports about buggy compilers are for tiny
architectures that XFS doesn't work well on anyways, so in practice it
hasn't created any user problems (AFAIK).
[zlang: Add more comments to new helpers]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Hironori Shiina [Wed, 21 Dec 2022 22:38:05 +0000 (17:38 -0500)]
xfs: Test bulkstat special query for root inode
This is a test for the fix: bf3cb3944792 xfs: allow single bulkstat of special inodes
This fix added a feature to query the root inode number of a filesystem.
This test creates a file with a lower inode number than the root and run
a query for the root inode.
Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 13 Dec 2022 19:45:33 +0000 (11:45 -0800)]
fuzzy: don't fail on compressed metadumps
This line in __scratch_xfs_fuzz_mdrestore:
test -e "${POPULATE_METADUMP}"
Breaks spectacularly on a setup that uses DUMP_COMPRESSOR to compress
the metadump files, because the metadump files get the compression
program added to the name (e.g. "${POPULATE_METADUMP}.xz"). The check
is wrong, and since the naming policy is an implementation detail of
_xfs_mdrestore, let's get rid of the -e test.
However, we still need a way to fail the test if the metadump cannot be
restored. _xfs_mdrestore returns nonzero on failure, so use that
instead.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Sat, 17 Dec 2022 08:18:34 +0000 (00:18 -0800)]
common/populate: move decompression code to _{xfs,ext4}_mdrestore
Move the metadump decompression code to the per-filesystem mdrestore
commands so that everyone can take advantage of them. This enables the
XFS and ext4 _mdrestore helpers to handle metadata dumps compressed with
their respective _metadump helpers.
In turn, this means that the xfs fuzz tests can now handle the
compressed metadumps created by the _scratch_populate_cached helper.
This is key to unbreaking fuzz testing for xfs.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Thu, 15 Dec 2022 04:21:41 +0000 (12:21 +0800)]
src/vfs/utils: Reset errno to zero when detect O_TMPFILE
For some filesystem that doesn't support O_TMPFILE, it will pass
ENOTSUP errno to upper layer. so it will report the following error:
QA output created by 696
vfstest.c: 1818: setgid_create_umask - Success - failure: is_setgid
vfstest.c: 2421: run_test - Operation not supported - failure...
To fix this, just reset errno before return.
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Leah Rumancik [Wed, 14 Dec 2022 21:59:26 +0000 (13:59 -0800)]
check: wipe tmp.arglist
Make sure tmp.arglist is wiped before each run to avoid
accidentally rerunning tests.
Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Mon, 12 Dec 2022 23:08:20 +0000 (00:08 +0100)]
check: ensure sect_stop is initialized if interrupted
sect_stop is normally set immediately prior to calling _wrapup() via
run_section(). However, when called via a trap signal handler,
sect_stop may be uninitialized, leading to a negative section time
(sect_stop - sect_start) in the xunit report. E.g.
Interrupted!
Passed all 1 tests
Xunit report: /home/david/xfstests/results//result.xml
rapido1:/# head /home/david/xfstests/results//result.xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="xfstests" failures="0" skipped="0" tests="1"
time="-1670885797" ... >
This commit uses the existing $interrupt flag to determine when
sect_stop needs to be initialised.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 13 Dec 2022 19:45:06 +0000 (11:45 -0800)]
xfs/018: fix attr value setting in this test
I was testing a patch to strengthen the buffer length validation of attr
log intent items during log recovery, when I noticed that the lengths of
the logged values were (mostly) a single byte larger than the alleged
attribute value. Upon further investigation, I noticed this code in the
test:
The 'echo' command generally emits a newline before exiting, which
means that the 16-byte "attr16" value was actually storing 17 bytes.
This affects all the test cases except for the attr64k tests, since the
attr(1) command helpfully/silently truncates the value buffer at 65536
bytes.
Fix the test to store values of exactly the length we want, and add a
couple more test cases to check that everything still works when the
value length is not an exact multiple of sizeof(u32).
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
[CAUSE]
$seqres.dmesg is created (and later deleted depending on config) by
_check_dmesg() function.
But if a test case failed by calling _fail, then we no longer call
_check_dmesg(), thus no dmesg will be saved no matter whatever the
config is.
[FIX]
If the test case itself failed, then still call _check_dmesg() to either
save the dmesg unconditionally (KEEP_DMESG=yes case), or save the dmesg
if there is something wrong (default).
The dmesg can be pretty handy debug clue for both cases.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Ziyang Zhang [Mon, 12 Dec 2022 05:56:45 +0000 (13:56 +0800)]
common/populate: Ensure that S_IFDIR.FMT_BTREE is in btree format
Sometimes "$((128 * dblksz / 40))" dirents cannot make sure that
S_IFDIR.FMT_BTREE could become btree format for its DATA fork.
Actually we just observed it can fail after apply our inode
extent-to-btree workaround. The root cause is that the kernel may be
too good at allocating consecutive blocks so that the data fork is
still in extents format.
Therefore instead of using a fixed number, let's make sure the number
of extents is large enough than (inode size - inode core size) /
sizeof(xfs_bmbt_rec_t).
Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Suggested-by: "Darrick J. Wong" <djwong@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Ziyang Zhang <ZiyangZhang@linux.alibaba.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Ziyang Zhang [Mon, 12 Dec 2022 05:56:44 +0000 (13:56 +0800)]
common/xfs: Add a helper to export inode core size
Some xfs test cases need the number of bytes reserved for only the inode
record, excluding the immediate fork areas. Now the value is hard-coded
and it is not a good chioce. Add a helper in common/xfs to export the
inode core size.
Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Ziyang Zhang <ZiyangZhang@linux.alibaba.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Shiyang Ruan [Sat, 10 Dec 2022 02:31:05 +0000 (02:31 +0000)]
fstests: batch update of file mode and untracked files
1. The executable binary in src/ should be in ignore file.
2. tests/xfs/216.out is a symlink generated at runtime, also should be
ignored.
3. tests/generic/692 was created with 644. It should be 755 otherwise
its mode will be changed and dirty the git tree after test.
Signed-off-by: Shiyang Ruan <ruansy.fnst@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com>
Qu Wenruo [Fri, 9 Dec 2022 05:34:02 +0000 (13:34 +0800)]
fstests: btrfs/219: remove it from auto group
The test case is to make sure we can mount a fs with older generation
(but with the same fsid/dev uuid).
Normally we will reject such case as btrfs is maintaining an internal
devices list (for multi-device support), and if we find a device
suddenly got an older generation, we will directly reject it.
Although for single device btrfs, we may add an exception for it,
the corresponding kernel patch is never merged.
So for now, just remove the test case from auto group.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Fri, 9 Dec 2022 06:19:01 +0000 (14:19 +0800)]
fstests: btrfs/080: fix the stray '\'
[BUG]
The latest grep will report stray '\', causing golden output mismatch
for btrfs/080:
btrfs/080 - output mismatch (see ~/xfstests-dev/results//btrfs/080.out.bad)
--- tests/btrfs/080.out 2022-11-24 19:53:53.137469203 +0800
+++ ~/xfstests-dev/results//btrfs/080.out.bad 2022-12-09 11:41:46.194597311 +0800
@@ -1,2 +1,3 @@
QA output created by 080
+grep: warning: stray \ before -
Silence is golden
...
(Run 'diff -u ~/xfstests-dev/tests/btrfs/080.out ~/xfstests-dev/results//btrfs/080.out.bad' to see the entire diff)
[CAUSE]
Even for regrex of grep, '-' doesn't need special escape, thus
"\bno\-holes\b" indeed has an unnecessary '\' before '-'.
[FIX]
Just remove the stray '\'.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Wed, 30 Nov 2022 17:01:53 +0000 (18:01 +0100)]
generic/273: Limit number of files by available inodes
Test generic/273 is failing for ext4 with 1k blocksize because it is
creating more files than we have available inodes. Just limit the number
of files created to the number of inodes.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Mon, 5 Dec 2022 12:41:44 +0000 (13:41 +0100)]
ext4/044: Fix failure when mount options are incompatible with ext3
There are some mount options that are incompatible with ext3 filesystem
type. If they are used, this test fails because it tries to remount the
filesystem as ext3. The test makes sense even without remounting as ext3
so just make the test silently skip the remount.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 28 Nov 2022 12:07:24 +0000 (12:07 +0000)]
btrfs: test a case with compressed send stream and a shared extent
Test that if we have a snapshot with a compressed extent that is partially
shared between two files, one of them has a size that is not sector size
aligned, we create a v2 send stream for the snapshot with compressed data,
and then apply that stream to another filesystem, the operation succeeds
and no data is missing. Also check that the file that had a reference to
the whole extent gets two compressed extents in the new filesystem, with
only one of them being shared (reflinked).
This tests a recent patch that landed in kernel 6.1-rc7:
a11452a3709e ("btrfs: send: avoid unaligned encoded writes when attempting to clone range")
Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 28 Nov 2022 12:07:23 +0000 (12:07 +0000)]
btrfs/280: also verify that fiemap reports extents as encoded
Now that _filter_fiemap_flags() optionally reports the encoded flag and
since btrfs/280 explicitly uses and tests compression, make it check that
fiemap reports the compressed extents with the encoded flag set.
Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 28 Nov 2022 12:07:22 +0000 (12:07 +0000)]
common: make _filter_fiemap_flags optionally print the encoded flag
We'd like to have some btrfs test cases in the future to verify that
extents are compressed when using fiemap. For that we can just check if
the FIEMAP_EXTENT_ENCODED (0x8) flag is set for an extent. Currently
_filter_fiemap_flags does not print that flag, so this changes it to
print the flag.
However printing the encoded flag is optional, because some tests use
the filter and use its output to match the golden output. So always
printing the flag would make the tests fail on btrfs when they are run
with "-o compress" (or compress-force) set in MOUNT_OPTIONS due to a
mismatch with the golden output. The tests that can be run with or
without compression on btrfs are generic/352, generic/353 and btrfs/279.
Since those tests don't care about the encoded flag, there is no need to
change them, just make the output of the flag optional, and any future
tests that want to check the presence of the encoded flag, will just pass
a parameter to _filter_fiemap_flags to tell it that the encoded flag
should be printed.
Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
fstests: skip btrfs/254 in case MIN_FSSIZE is more than 1G
The test-case btrfs/254 creates a 1G device-mapper setup, but this might
be too small for the filesystem to actually operate (i.e. in case of a
zoned block device which needs at least 5 zones).
Skip the test if MIN_FSSIZE is set to a value above 1G.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Tue, 22 Nov 2022 14:12:19 +0000 (15:12 +0100)]
generic/614: Run test only for filesystems supporting delalloc
Simple filesystems such as ext2 or udf do not support delayed
allocation. Thus they allocate data blocks for mmap writes only during
writeback. This makes test generic/614 fail because it checks exactly
whether the blocks for the write are reserved before writeback happens.
Make the test depend on delayed allocation support.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Tue, 8 Nov 2022 18:32:42 +0000 (02:32 +0800)]
fstests: update group name according to xfs_io command requirement
When a test case requires someone xfs_io command, that nearly means
that case belong that kind of test group. Likes fpunch for punch
group, fcollapse for collapse group, falloc for prealloc group, fzero
for zero group and so on.
Many fstests cases miss some test groups they should belong to, so
this patch trys to supplement this lack, according to the "xxxx"
which required by _require_xfs_io_command "xxxx".
Luís Henriques [Tue, 15 Nov 2022 14:45:00 +0000 (14:45 +0000)]
ceph/005: skip test if using "test_dummy_encryption"
When using the "test_dummy_encryption" mount option, new file and directory
names will be encrypted. This means that if using as a mount base directory
a newly created directory, we would have to use the encrypted directory name
instead. For the moment, ceph doesn't provide a way to get this encrypted
file name, thus for now simply skip this test.
Signed-off-by: Luís Henriques <lhenriques@suse.de> Reviewed-by: Xiubo Li <xiubli@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiao Yang [Mon, 14 Nov 2022 08:35:03 +0000 (08:35 +0000)]
generic/470: Replace thin volume with blkdiscard -z
generic/470 was original designed to verify mmap(MAP_SYNC) which
is only valid to the DAX capable device(e.g. PMEM). Thin volume[1] was
introduced to fix the inconsistent filesystem issue[2] but it make
the test become not run because it doesn't support DAX. As Darrick
mentioned[3], zeroing the entire mapped range of scartch device
can fix the issue as well, so I try to use blkdiscard -z instead.
Filipe Manana [Wed, 9 Nov 2022 16:44:58 +0000 (16:44 +0000)]
btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
Starting with btrfs-progs v6.0, the defrag command now prints to stdout
the full path of the files it processes. This makes test cases btrfs/021
and btrfs/256 fail because they don't expect any output from the defrag
command.
The change happened with the following commit in btrfs-progs:
dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")
So update the tests to ignore the stdout of the defrag command.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 9 Nov 2022 16:44:57 +0000 (16:44 +0000)]
btrfs/053: fix test failure when running with btrfs-progs v6.0+
In btrfs-progs v6.0 the --leafsize (-l) command line option was removed
from mkfs.btrfs, so btrfs/053 can fail with v6.0+ in case the scratch
device does not have a btrfs filesystem created before running the test,
in which case mounting the scratch device fails.
The change was introduced by the following btrfs-progs commit:
f7a768d62498 ("btrfs-progs: mkfs: remove support for option --leafsize")
Change the test to use --nodesize (-n) instead, since it exists in both
old and new btrfs-progs versions. Also redirect mkfs output to the test's
log file and fail explicitly if mkfs failed.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 9 Nov 2022 16:44:56 +0000 (16:44 +0000)]
btrfs/003: fix failure on new btrfs-progs versions
Starting with btrfs-progs version 5.19, the output of 'filesystem show'
command changed when we have a missing device. The old output was like the
following:
Label: none uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
*** Some devices missing
While the new output (btrfs-progs 5.19+) is like the following:
Label: none uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
devid 2 size 0 used 0 path /dev/loop1 MISSING
More specifically it happened in the following btrfs-progs commit:
957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")
This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep
filter in btrfs/003 so that it works with both output formats.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
[FAILURES]
The following btrfs test cases failed with newer btrfs-progs:
- btrfs/197
- btrfs/198
- btrfs/254
They all fail due to output mismatch like the following:
Label: none uuid: <UUID>
Total devices <NUM> FS bytes used <SIZE>
devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
- *** Some devices missing
+ devid <DEVID> size 0 used 0 path MISSING
[CAUSE]
Since btrfs-progs commit 957a79c9b016 ("btrfs-progs: fi show: print
missing device for a mounted file system"), we output the detailed info
of a missing device if "btrfs filesystem show" is executed using
"-m <mnt>" option.
Such detailed output no longer follows the old format, thus causing the
output mismatch.
[FIX]
Update _filter_btrfs_filesystem_show() to handle detailed missing
device by:
- Buffer the output first
- Output the first two lines
Which is always label/uuid and the total device accounting.
- Replace the detailed missing device line with old output
- Sort (in reverse order) and uniq the device list
By this we can handle both old and new output correctly.
Although this means we lacks the ability to detect mutltiple missing
devices, thankfully the involved test cases are not checking this yet.
[ Zorro: add "rm -f $tmp.btrfs_filesystem_show" ]
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Wed, 9 Nov 2022 13:07:46 +0000 (21:07 +0800)]
generic: shutdown might leave NULL files with nonzero di_size
An old issue might cause on-disk inode sizes are logged prematurely
via the free eofblocks path on file close. Then fs shutdown might
leave NULL files but their di_size > 0.
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 7 Nov 2022 09:38:58 +0000 (09:38 +0000)]
generic: check direct IO writes with io_uring and O_DSYNC are durable
Test that direct IO writes with io_uring and O_DSYNC are durable if a
power failure happens after they complete.
This is motivated by a regression on btrfs, affecting 5.15 stable
kernels and kernels up to 6.0, where often the writes were not
persisted (same behaviour as if O_DSYNC was not provided). This was
recently fixed by the following commit:
8184620ae212 ("btrfs: fix lost file sync on direct IO write with
nowait and dsync iocb")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Whitney [Fri, 21 Oct 2022 21:19:50 +0000 (17:19 -0400)]
generic: add missing $FSX_AVOID to fsx invocations
generic/455 fails when run on an ext4 bigalloc file system. Its
fsx invocations can make insert range and collapse range calls whose
arguments are not cluster aligned, and ext4 will fail those calls for
bigalloc. They can be suppressed by adding the FSX_AVOID environment
variable to the fsx invocation and setting its value appropriately in
the test environment, as is done for other fsx-based tests. This
avoids the need to exclude the test to avoid failures and makes it
possible to take advantage of the remainder of its coverage.
[ Also fix generic/457, as requested by Dave Chinner -- TYT]
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:30 +0000 (13:58 -0700)]
tests: fix some tests for systems with fs.verity.require_signatures=1
Some of the newer verity tests don't work properly on systems where
fs.verity.require_signatures is enabled, either because they forget to
disable it at the beginning of the test, or they forget to re-enable it
afterwards, or both. Fix this.
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:29 +0000 (13:58 -0700)]
generic/577: add missing file removal before empty file test
The fix for _fsv_have_hash_algorithm() exposed a bug where one of the
test cases in generic/577 isn't deleting the file from the previous test
case before it tries to write to it. That causes a failure, since due
to the fix for _fsv_have_hash_algorithm(), the file from the previous
test case now ends up with verity enabled and therefore cannot be
written to. Fix this by deleting the file.
Reported-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:28 +0000 (13:58 -0700)]
common/verity: fix _fsv_have_hash_algorithm() with required signatures
_fsv_have_hash_algorithm() uses _fsv_enable() without a signature, so it
always fails when called while fs.verity.require_signatures=1. This
happens in generic/577, which tests file signing. This wasn't noticed
because it just made part of generic/577 always be skipped.
Fix this by making _fsv_have_hash_algorithm() temporarily set
fs.verity.require_signatures to 0.
Since the previous value needs to be restored afterwards, whether it is
0 or 1, also make some changes to the fs.verity.require_signatures
helper functions to allow the restoration of the previous value, rather
than the value that existed at the beginning of the test.
Finally, make a couple related cleanups: make _fsv_have_hash_algorithm()
always delete the file it works with, and also update the similar code
in _require_scratch_verity().
Reported-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 3 Nov 2022 04:17:06 +0000 (21:17 -0700)]
xfs: test xfs_scrub phase 6 media error reporting
Add new helpers to dmerror to provide for marking selected ranges
totally bad -- both reads and writes will fail. Create a new test for
xfs_scrub to check that it reports media errors in data files correctly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:21 +0000 (11:11 +0000)]
btrfs: test fiemap reports extent as not shared after COWing it in snapshot
Test that if we have a large file, with file extent items spanning several
leaves in the fs tree, and that is shared due to a snapshot, if we COW one
of the extents, doing a fiemap will report the respective file range as
not shared.
This exercises the processing of delayed references for metadata extents
in the backref walking code, used by fiemap to determine if an extent is
shared.
This used to fail until very recently and was fixed by the following
kernel commit that landed in 6.1-rc2:
943553ef9b51 (""btrfs: fix processing of delayed tree block refs during backref walking")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:20 +0000 (11:11 +0000)]
btrfs: test that fiemap reports extent as not shared after deleting file
Test that if we have two files with shared extents, after removing one of
the files, if we do a fiemap against the other file, it does not report
extents as shared anymore.
This exercises the processing of delayed references for data extents in
the backref walking code, used by fiemap to determine if an extent is
shared.
This used to fail until very recently and was fixed by the following
kernel commit that landed in 6.1-rc2:
4fc7b5722824 (""btrfs: fix processing of delayed data refs during backref walking")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:19 +0000 (11:11 +0000)]
common/punch: fix flags printing for filter _filter_fiemap_flags
In the filter _filter_fiemap_flags, if we get a flags field that only has
the 'last' flag set, we end up printing the string "nonelast", which is
ugly and not intuitive.
So fix this by updating the filter's awk code to reset the flags string to
an empty string if we have the "last" flag set and we haven't updated the
flags string before. So now the same test gives the following result:
0: [0..127]: last
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiao Yang [Thu, 27 Oct 2022 10:50:52 +0000 (10:50 +0000)]
common/config: Make test and scratch devices use the same mount options
Some cases(e.g. generic/519) check commands/features on test device but
do tests on scratch device. If some mount options can impact the check
result, these cases may throw error instead if not run when we use
different mount options for test and scratch devices.
Signed-off-by: Xiao Yang <yangx.jy@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Fri, 28 Oct 2022 15:43:37 +0000 (23:43 +0800)]
xfs: new test on xfs with corrupted sb_inopblock
There's a known bug fix 392c6de98af1 ("xfs: sanitize sb_inopblock in
xfs_mount_validate_sb"). So try to corrupt the sb_inopblock of xfs,
to cover this bug.
Signed-off-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
projects / xfstests-dev.git / log