Zorro Lang [Wed, 9 Nov 2022 13:07:46 +0000 (21:07 +0800)]
generic: shutdown might leave NULL files with nonzero di_size
An old issue might cause on-disk inode sizes are logged prematurely
via the free eofblocks path on file close. Then fs shutdown might
leave NULL files but their di_size > 0.
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 7 Nov 2022 09:38:58 +0000 (09:38 +0000)]
generic: check direct IO writes with io_uring and O_DSYNC are durable
Test that direct IO writes with io_uring and O_DSYNC are durable if a
power failure happens after they complete.
This is motivated by a regression on btrfs, affecting 5.15 stable
kernels and kernels up to 6.0, where often the writes were not
persisted (same behaviour as if O_DSYNC was not provided). This was
recently fixed by the following commit:
8184620ae212 ("btrfs: fix lost file sync on direct IO write with
nowait and dsync iocb")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Whitney [Fri, 21 Oct 2022 21:19:50 +0000 (17:19 -0400)]
generic: add missing $FSX_AVOID to fsx invocations
generic/455 fails when run on an ext4 bigalloc file system. Its
fsx invocations can make insert range and collapse range calls whose
arguments are not cluster aligned, and ext4 will fail those calls for
bigalloc. They can be suppressed by adding the FSX_AVOID environment
variable to the fsx invocation and setting its value appropriately in
the test environment, as is done for other fsx-based tests. This
avoids the need to exclude the test to avoid failures and makes it
possible to take advantage of the remainder of its coverage.
[ Also fix generic/457, as requested by Dave Chinner -- TYT]
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:30 +0000 (13:58 -0700)]
tests: fix some tests for systems with fs.verity.require_signatures=1
Some of the newer verity tests don't work properly on systems where
fs.verity.require_signatures is enabled, either because they forget to
disable it at the beginning of the test, or they forget to re-enable it
afterwards, or both. Fix this.
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:29 +0000 (13:58 -0700)]
generic/577: add missing file removal before empty file test
The fix for _fsv_have_hash_algorithm() exposed a bug where one of the
test cases in generic/577 isn't deleting the file from the previous test
case before it tries to write to it. That causes a failure, since due
to the fix for _fsv_have_hash_algorithm(), the file from the previous
test case now ends up with verity enabled and therefore cannot be
written to. Fix this by deleting the file.
Reported-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Fri, 4 Nov 2022 20:58:28 +0000 (13:58 -0700)]
common/verity: fix _fsv_have_hash_algorithm() with required signatures
_fsv_have_hash_algorithm() uses _fsv_enable() without a signature, so it
always fails when called while fs.verity.require_signatures=1. This
happens in generic/577, which tests file signing. This wasn't noticed
because it just made part of generic/577 always be skipped.
Fix this by making _fsv_have_hash_algorithm() temporarily set
fs.verity.require_signatures to 0.
Since the previous value needs to be restored afterwards, whether it is
0 or 1, also make some changes to the fs.verity.require_signatures
helper functions to allow the restoration of the previous value, rather
than the value that existed at the beginning of the test.
Finally, make a couple related cleanups: make _fsv_have_hash_algorithm()
always delete the file it works with, and also update the similar code
in _require_scratch_verity().
Reported-by: Andrey Albershteyn <aalbersh@redhat.com> Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 3 Nov 2022 04:17:06 +0000 (21:17 -0700)]
xfs: test xfs_scrub phase 6 media error reporting
Add new helpers to dmerror to provide for marking selected ranges
totally bad -- both reads and writes will fail. Create a new test for
xfs_scrub to check that it reports media errors in data files correctly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:21 +0000 (11:11 +0000)]
btrfs: test fiemap reports extent as not shared after COWing it in snapshot
Test that if we have a large file, with file extent items spanning several
leaves in the fs tree, and that is shared due to a snapshot, if we COW one
of the extents, doing a fiemap will report the respective file range as
not shared.
This exercises the processing of delayed references for metadata extents
in the backref walking code, used by fiemap to determine if an extent is
shared.
This used to fail until very recently and was fixed by the following
kernel commit that landed in 6.1-rc2:
943553ef9b51 (""btrfs: fix processing of delayed tree block refs during backref walking")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:20 +0000 (11:11 +0000)]
btrfs: test that fiemap reports extent as not shared after deleting file
Test that if we have two files with shared extents, after removing one of
the files, if we do a fiemap against the other file, it does not report
extents as shared anymore.
This exercises the processing of delayed references for data extents in
the backref walking code, used by fiemap to determine if an extent is
shared.
This used to fail until very recently and was fixed by the following
kernel commit that landed in 6.1-rc2:
4fc7b5722824 (""btrfs: fix processing of delayed data refs during backref walking")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Mon, 31 Oct 2022 11:11:19 +0000 (11:11 +0000)]
common/punch: fix flags printing for filter _filter_fiemap_flags
In the filter _filter_fiemap_flags, if we get a flags field that only has
the 'last' flag set, we end up printing the string "nonelast", which is
ugly and not intuitive.
So fix this by updating the filter's awk code to reset the flags string to
an empty string if we have the "last" flag set and we haven't updated the
flags string before. So now the same test gives the following result:
0: [0..127]: last
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiao Yang [Thu, 27 Oct 2022 10:50:52 +0000 (10:50 +0000)]
common/config: Make test and scratch devices use the same mount options
Some cases(e.g. generic/519) check commands/features on test device but
do tests on scratch device. If some mount options can impact the check
result, these cases may throw error instead if not run when we use
different mount options for test and scratch devices.
Signed-off-by: Xiao Yang <yangx.jy@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Fri, 28 Oct 2022 15:43:37 +0000 (23:43 +0800)]
xfs: new test on xfs with corrupted sb_inopblock
There's a known bug fix 392c6de98af1 ("xfs: sanitize sb_inopblock in
xfs_mount_validate_sb"). So try to corrupt the sb_inopblock of xfs,
to cover this bug.
Signed-off-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Darrick J. Wong [Fri, 28 Oct 2022 17:42:10 +0000 (10:42 -0700)]
common: simplify grep pipe sed interactions
Zorro pointed out that the idiom "program | grep | sed" isn't necessary
for field extraction -- sed is perfectly capable of performing a
substitution and only printing the lines that match that substitution.
Do that for the common helpers.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
There are a lot of places where we open-code detecting the realtime
extent size and extent count of a specific filesystem. Refactor this
into a couple of helpers to clean up the code.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
There are a lot of places where we open-code determining the directory
block size for a specific filesystem. Refactor this into a single
helper to clean up existing tests.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 28 Oct 2022 17:41:53 +0000 (10:41 -0700)]
xfs: refactor filesystem feature detection logic
There are a lot of places where we open-code detecting features of a
specific filesystem. Refactor this into a couple of helpers in
preparation for adding stress tests for online repair and fuzzing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Hironori Shiina [Mon, 17 Oct 2022 15:11:51 +0000 (11:11 -0400)]
xfs: test for fixing wrong root inode number in dump
Test '-x' option of xfsrestore. With this option, a wrong root inode
number in a dump file is corrected. A root inode number can be wrong
in a dump created by problematic xfsdump (v3.1.7 - v3.1.9) with
bulkstat misuse. In this test, a corrupted dump file is created by
overwriting a root inode number in a header.
Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Josef Bacik [Wed, 24 Aug 2022 19:32:10 +0000 (15:32 -0400)]
fstests: get section config after RUN_SECTION checks
While trying to do
./check -s <some section>
I was failing because I had a section defined higher than <some section>
that had TEST_DEV=/some/nonexistent/device, since I was using the other
section to test an experimental drive. This appears to be because we
run through all of the sections, and when getting the section config we
check to see if it's valid, and in this case the section wasn't valid.
The section I was actually trying to use was valid however. Fix check
to see if the section we're trying to run is in our list of sections to
run first, and then if it is get the config at that point.
Signed-off-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 12 Oct 2022 01:45:39 +0000 (18:45 -0700)]
populate: reformat external ext[34] journal devices when restoring a cached image
The fs population code has the ability to save cached metadumps of
filesystems to save time when running fstests. The cached images should
be unmounted cleanly, so we never save the contents of external journal
devices.
Unfortunately, the cache restore code fails to reset the external
journal when restoring a clean image, so we ignore cached images because
the journal doesn't match the filesystem. This makes test runtimes
longer than they need to be.
Solve this by reformatting the external journal to match the filesystem.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 12 Oct 2022 01:45:33 +0000 (18:45 -0700)]
populate: wipe external xfs log devices when restoring a cached image
The fs population code has the ability to save cached metadumps of
filesystems to save time when running fstests. The cached images should
be unmounted cleanly, so we never save the contents of external log
devices.
Unfortunately, the cache restore code fails to wipe the external log
when restoring a clean image, so we end up with strange test failures
because the log doesn't match the filesystem:
Theodore Ts'o [Thu, 13 Oct 2022 14:48:44 +0000 (10:48 -0400)]
ext4/053: remove test for i_version mount option
The i_version mount option is deprecated in favor of iversion and is
advertised to be removed in 5.20. It was removed in commit 1ff20307393e ("ext4: unconditionally enable the i_version counter") in
Linux v6.1. So remove the test from ext4/053.
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Sat, 15 Oct 2022 04:57:05 +0000 (12:57 +0800)]
check: optionally compress core dumps
Add a new option, COREDUMP_COMPRESSOR, that will be used to compress
core dumps collected during a fstests run. The program specified must
accept the -f -9 arguments that gzip has.
Darrick J. Wong [Fri, 14 Oct 2022 18:20:20 +0000 (11:20 -0700)]
check: detect and preserve all coredumps made by a test
If someone sets kernel.core_uses_pid (or kernel.core_pattern), any
coredumps generated by fstests might have names that are longer than
just "core". Since the pid isn't all that useful by itself, let's
record the coredumps by hash when we save them, so that we don't waste
space storing identical crash dumps.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
David Disseldorp [Tue, 11 Oct 2022 09:23:39 +0000 (11:23 +0200)]
lsqa.pl: fix --header output
7e98d41a ("fstests: move test group info to test files") replaces the
post-header "seq" assignment with "./common/preamble\n_begin_fstest ..."
Update lsqa.pl to recognise the "./common/preamble" line as a header
divider.
Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 11 Oct 2022 12:22:03 +0000 (13:22 +0100)]
generic: test fiemap reports extent as shared after cloning it
Test that if we have two consecutive extents and only one of them is
cloned, then fiemap correctly reports which one is shared and reports
the other as not shared.
This currently fails on btrfs for all kernel releases, but is fixed by
a kernel patch that landed in Linus' tree last week:
ac3c0d36a2a2f7 ("btrfs: make fiemap more efficient and accurate reporting extent sharedness")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 7 Oct 2022 13:53:36 +0000 (14:53 +0100)]
fstests: add fiemap group
Add a fiemap group for all tests that exercise fiemap, either directly
through xfs_io, or more indirectly like through filefrag or helpers in
common/* (like _count_extents, _count_holes, etc). This is useful in
order to quickly test changes made to the fiemap implementation of a
filesystem for example.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 7 Oct 2022 13:53:35 +0000 (14:53 +0100)]
fstests: add missing require of xfs_io fiemap command to some tests
btrfs/257, btrfs/258, btrfs/259 and xfs/443 use the fiemap command of
xfs_io but don't do a '_require_xfs_io_command "fiemap"'. So add the
missing requirement.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Due to some braino on my part, _scratch_populate_cached will metadump
the filesystem twice -- once with compression disabled, and again with
it enabled, maybe. Get rid of the first metadump.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 5 Oct 2022 22:30:55 +0000 (15:30 -0700)]
xfs/128: try to force file allocation behavior
Over the years, I've noticed that this test occasionally fails when I've
programmed the allocator to hand out the minimum amount of space with
each allocation or if extent size hints are enabled:
--- /tmp/fstests/tests/xfs/128.out 2022-09-01 15:09:11.506679341 -0700
+++ /var/tmp/fstests/xfs/128.out.bad 2022-10-04 17:32:50.992000000 -0700
@@ -20,7 +21,9 @@ 56ed2f712c91e035adeeb26ed105a982 SCRATCH_MNT/test-128/file3 b81534f439aac5c34ce3ed60a03eba70 SCRATCH_MNT/test-128/file4
Check files
free blocks after creating some reflink copies is in range
free blocks after CoW some reflink copies is in range
-free blocks after defragging all reflink copies is in range
-free blocks after all tests is in range
+free blocks after defragging all reflink copies has value of 8620027
+free blocks after defragging all reflink copies is NOT in range 8651819 .. 8652139
+free blocks after all tests has value of 8620027
+free blocks after all tests is NOT in range 8651867 .. 8652187
It turns out that under the right circumstances, the _pwrite_byte at the
start of this test will end up allocating two extents to file1. This
almost never happens when delalloc is enabled or when the extent size is
large, and is more prone to happening if the extent size is > 1FSB but
small, the allocator hands out small allocations, or if writeback shoots
down pages in random order.
When file1 gets more than 1 extent, problems start to happen. The free
space accounting checks at the end of the test assume that file1 and
file4 still share the same space at the end of the test. This
definitely happens if file1 gets one extent (since fsr ignores
single-extent files), but if there's more than 1, fsr will try to
defragment it. If fsr succeeds in copying the file contents to a temp
file with fewer extents than the source file, it will switch the
contents, but unsharing the contents in the process. This cause the
free space to be lower than expected, and the test fails.
Resolve this situation by preallocating space beforehand to try to set
up file1 with a single space extent. If the test fails and we got more
than one extent, note that in the output.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 5 Oct 2022 22:30:49 +0000 (15:30 -0700)]
xfs/229: do not _xfs_force_bdev on TEST_DIR
In commit ea15099b71, I observed that this test tries to test the
behavior of the extent size hint on the data device. If the test runner
set up MKFS_OPTIONS such that the filesystem gets created with a
realtime section and rtinherit set on the root directory, then the
preconditions of this test (creating files on the data section) is not
satisfied and the results of this test are incorrect. The solution was
to force all files created by this test to be assigned to the data
section.
Unfortunately, the correction that I made has side effects beyond this
test -- by clearing rtinherit on $TEST_DIR, all tests that run after
this one will create files on the data section, because the test
filesystem persists for the duration of the entire test run. This leads
to the wrong things being tested.
Fix this new problem by clearing the rtinherit flag on $TDIR, which
contains the files created by this test and is removed during cleanup,
and leave a few comments celebrating our new discoveries.
Fixes: ea15099b71 ("xfs: force file creation to the data device for certain layout tests") Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 5 Oct 2022 22:30:38 +0000 (15:30 -0700)]
generic/092: skip test if file allocation unit isn't aligned
This test exercises allocation behavior when truncating a preallocated
file down to 5M and then up to 7M. If those two sizes aren't aligned
with the file allocation unit length, then the FIEMAP output will show
blocks beyond EOF. That will cause trouble with the golden output, so
skip this test if that will be the case.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Fri, 7 Oct 2022 13:53:34 +0000 (14:53 +0100)]
btrfs: test fiemap on large file with extents shared through a snapshot
Verify that fiemap correctly reports the sharedness of extents for a file
with a very large number of extents, spanning many b+tree leaves in the fs
tree, and when the file's subvolume was snapshoted.
Currently this passes on all kernel releases and its purpose is to prevent
and detect regressions in the future, as this actually happened during
recent development on the btrfs' fiemap related code. With this test we
now have better coverage for fiemap when a file is shared through a
snapshot.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Naohiro Aota [Wed, 5 Oct 2022 05:46:44 +0000 (14:46 +0900)]
btrfs: test active zone tracking
A ZNS device limits the number of active zones, which is the number of
zones can be written at the same time. To deal with the limit, btrfs's
zoned mode tracks which zone (corresponds to a block group on the SINGLE
profile) is active, and finish a zone if necessary.
This test checks if the active zone tracking and the finishing of zones
works properly. First, it fills <number of max active zones> zones
mostly. And, run some data/metadata stress workload to force btrfs to use a
new zone.
This test fails on an older kernel (e.g, 5.18.2) like below.
btrfs/273
[failed, exit status 1]- output mismatch (see /host/btrfs/273.out.bad)
--- tests/btrfs/273.out 2022-09-15 07:52:18.000000000 +0000
+++ /host/btrfs/273.out.bad 2022-09-15 07:59:14.290967793 +0000
@@ -1,2 +1,5 @@
QA output created by 273
-Silence is golden
+stress_data_bgs failed
+stress_data_bgs_2 failed
+failed: '/bin/btrfs subvolume snapshot /mnt/scratch /mnt/scratch/snap825'
+(see /host/btrfs/273.full for details)
...
(Run 'diff -u /var/lib/xfstests/tests/btrfs/273.out /host/btrfs/273.out.bad' to see the entire diff)
The failure is fixed with a series "btrfs: zoned: fix active zone tracking
issues" [1] (upstream commits from 65ea1b66482f ("block: add bdev_max_segments()
helper") to 2ce543f47843 ("btrfs: zoned: wait until zone is finished when
allocation didn't progress")).
Naohiro Aota [Wed, 5 Oct 2022 05:46:43 +0000 (14:46 +0900)]
common: introduce zone_capacity() to return a zone capacity
Introduce _zone_capacity() to return a zone capacity of the given address
in the given device (optional). Move _filter_blkzone_report() for it, and
rewrite btrfs/237 with it.
Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Test creation, modification and deletion of xattr for a BTRFS filesystem
which has the read-only property set to true.
Re-test the same after read-only property is set to false.
This tests the bug for "security.*" modifications which escape
xattr_permission(), because security parameters are usually let through
in xattr_permission, without checking
inode_permission()->btrfs_permission().
This resulted in being able to change a security xattr on a RO subvolume
until it got fixed by kernel commit b51111271b03 ("btrfs: check if root
is readonly while setting security xattr").
Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Pavel Reichl [Thu, 22 Sep 2022 06:38:36 +0000 (08:38 +0200)]
generic: test i_blocks for truncated large files
This is a regression test for an incorrect computation of i_blocks for
truncated files larger than 4 GiB. Bug was filed for exFAT.
Test is based on reproducer provied by Christophe Vu-Brugier as part
of kernel patch-fix submission.
Signed-off-by: Pavel Reichl <preichl@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Pavel Reichl [Thu, 22 Sep 2022 06:38:35 +0000 (08:38 +0200)]
common: new helper to alloacate fixed size files
Helper that creates files of specified size using falloc if supported,
otherwise pwrite is used.
Signed-off-by: Pavel Reichl <preichl@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
This's copied from generic/698, extend it to test overlayfs on top of
idmapped mounts specifically.
When group ownership is changed a caller whose fsuid owns the inode can
change the group of the inode to any group they are a member of. When
searching through the caller's groups we failed to use the gid mapped
according to the idmapped mount otherwise we fail to change ownership.
Add a test for this.
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
When group ownership is changed a caller whose fsuid owns the inode
can change the group of the inode to any group they are a member of.
When searching through the caller's groups we failed to use the gid
mapped according to the idmapped mount otherwise we fail to change
ownership. Add a test for this.
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
We introduced a new type vfs{g,u}id_t into the vfs in b27c82e12965
("attr: port attribute changes to new types"). This type makes sure that
{g,u}ids mapped into an idmapped mount are distinct from regular
k{g,u}id_t. This expands the possible errnos that users may see so we
need to handle both EINVAL and EOVERFLOW.
Cc: Seth Forshee (DigitalOcean) <sforshee@kernel.org> Cc: Christoph Hellwig <hch@lst.de> Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Wed, 31 Aug 2022 07:33:27 +0000 (15:33 +0800)]
vfs: Add new setgid_create_acl test
The current_umask() is stripped from the mode directly in the vfs if the
filesystem either doesn't support acls or the filesystem has been
mounted without posic acl support.
If the filesystem does support acls then current_umask() stripping is
deferred to posix_acl_create(). So when the filesystem calls
posix_acl_create() and there are no acls set or not supported then
current_umask() will be stripped.
If the parent directory has a default acl then permissions are based off
of that and current_umask() is ignored. Specifically, if the ACL has an
ACL_MASK entry, the group permissions correspond to the permissions of
the ACL_MASK entry. Otherwise, if the ACL has no ACL_MASK entry, the
group permissions correspond to the permissions of the ACL_GROUP_OBJ
entry.
Here we only use setfacl to set default acl or add ACL_MASK to check
whether inode strip S_ISGID works correctly.
Like umask test, this patch is also designed to test kernel patchset behaviour
"move S_ISGID stripping into the vfs* helper"
Tested-by: Christian Brauner (Microsoft) <brauner@kernel.org> Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org> Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Wed, 31 Aug 2022 07:33:26 +0000 (15:33 +0800)]
vfs: Add new setgid_create_umask test
The current_umask() is stripped from the mode directly in the vfs if the
filesystem either doesn't support acls or the filesystem has been
mounted without posic acl support.
If the filesystem does support acls then current_umask() stripping is
deferred to posix_acl_create(). So when the filesystem calls
posix_acl_create() and there are no acls set or not supported then
current_umask() will be stripped.
This patch is also designed to test kernel patch behaviour
"move S_ISGID stripping into the vfs* helper"
Here we only use umask(S_IXGRP) to check S_ISGID stripping whether works
correctly and S_IXGRP mode still exists.
Also use noacl mount option to test another kernel bug
fs: Add missing umask strip in vfs_tmpfile
Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org> Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 3 Aug 2022 04:22:40 +0000 (21:22 -0700)]
common: filter internal errors during io error testing
The goal of an EIO shutdown test is to examine the shutdown and recovery
behavior if we make the underlying storage device return EIO. On XFS,
it's possible that the shutdown will come from a thread that cancels a
dirty transaction due to the EIO. This is expected behavior, but
_check_dmesg will flag it as a test failure.
Make it so that we can add simple regexps to the default check_dmesg
filter function, then add the "Internal error" string to filter function
when we invoke an EIO test. This fixes periodic regressions in
generic/019 and generic/475.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 3 Aug 2022 04:22:35 +0000 (21:22 -0700)]
common: disable infinite IO error retry for EIO shutdown tests
This patch fixes a rather hard to hit livelock in the tests that test
how xfs handles shutdown behavior when the device suddenly dies and
starts returing EIO all the time. The livelock happens if the AIL is
stuck retrying failed metadata updates forever, the log itself is not
being written, and there is no more log grant space, which prevents the
frontend from shutting down the log due to EIO errors during
transactions.
While most users probably want the default retry-forever behavior
because EIO can be transient, the circumstances are different here. The
tests are designed to flip the device back to working status only after
the unmount succeeds, so we know there's no point in the filesystem
retrying writes until after the unmount.
This fixes some of the periodic hangs in generic/019 and generic/475.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 3 Aug 2022 04:22:29 +0000 (21:22 -0700)]
common/xfs: fix _reset_xfs_sysfs_error_handling reset to actual defaults
There's a slight mistake in _reset_xfs_sysfs_error_handling: it sets
retry_timeout_seconds to 0, which is not the current default (-1) in
upstream Linux. Fix this.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 1 Sep 2022 16:10:09 +0000 (17:10 +0100)]
generic: test fsync after punching hole adjacent to an existing hole
Test that if we punch a hole adjacent to an existing hole, fsync the file
and then power fail, the new hole exists after mounting again the
filesystem.
This currently fails on btrfs with kernels 5.18 and 5.19 when not using
the "no-holes" feature. The "no-holes" feature is enabled by default at
mkfs time starting with btrfs-progs 5.15, so to trigger the issue with
btrfs-progs 5.15+ and kernel 5.18 or kernel 5.19, one must set
"-O ^no-holes" in the MKFS_OPTIONS environment variable (part of the
btrfs test matrix).
The issue is fixed for btrfs with the following kernel patch:
"btrfs: update generation of hole file extent item when merging holes"
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Chandan Babu R [Thu, 25 Aug 2022 13:00:41 +0000 (18:30 +0530)]
xfs: Check if a direct write can result in a false ENOSPC error
This commit adds a test to check if a direct write on a delalloc extent
present in CoW fork can result in a false ENOSPC error. The bug has been fixed
by upstream commit d62113303d691 ("xfs: Fix false ENOSPC when performing
direct write on a delalloc extent in cow fork").
Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Chandan Babu R <chandan.babu@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/271: use the common fail_request setup helpers
Use the helpers from common/fail_make_request instead of open coding
them. This switches to using a higher error count than the existing
code, which was the intention from the very beginning (and doesn't
actuallt matter for the short sequences in this test).
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
BingJing Chang [Tue, 30 Aug 2022 14:06:59 +0000 (22:06 +0800)]
fstests: btrfs: test incremental send for changed reference paths
Normally btrfs stores file paths in an array of ref items. However, items
for the same parent directory can not exceed the size of a leaf. So btrfs
also store the rest of them in extended ref items alternatively.
In this test, it creates a large number of links under a directory causing
the file paths stored in these two ways to be the parent snapshot. And it
deletes and recreates just an amount of them that can be stored within an
array of ref items to be the send snapshot. Test that an incremental send
operation correctly issues link/unlink operations only against new/deleted
paths, or the receive operation will fail due to a link on an existed path.
This currently fails on btrfs but is fixed by a kernel patch with the
commit 3aa5bd367fa5a3 ("btrfs: send: fix sending link commands for
existing file paths")
Reviewed-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: BingJing Chang <bingjingc@synology.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Wed, 17 Aug 2022 03:10:23 +0000 (11:10 +0800)]
ext4/053: Remove nouser_xattr test
Plan to remove noacl and nouser_xattr mount option in kernel because they
are deprecated[1]. So remove nouser_xattr test in here.
Reported-by: kernel test robot <oliver.sang@intel.com> Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Jan Kara <jack@suse.cz> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Thu, 4 Aug 2022 06:18:52 +0000 (14:18 +0800)]
src/t_ofd_locks.c: Reset errno to zero
It seems I met libcap errno bug again when using libcap-2.48-4.el8.x86_64.
But this time, errno is EINVAL if c program link with lcap.
Lastest upstream libcap doesn't have bug and it should be backport bug.
generic/478 will become not run because of this. To fix this that only
exists on some distributions, reset errno to zero.
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Fri, 19 Aug 2022 19:34:05 +0000 (12:34 -0700)]
fstests: add btrfs fs-verity send/recv test
Test btrfs send/recv support for fs-verity. Includes tests for
signatures, salts, and interaction with chmod/caps. The last of those is
to ensure the various features that go in during inode_finalize interact
properly.
This depends on the kernel patch adding support for send:
btrfs: send: add support for fs-verity
And the btrfs-progs patch adding support for recv:
btrfs-progs: receive: add support for fs-verity
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:01:06 +0000 (14:01 -0700)]
dmerror: support external log and realtime devices
Upgrade the dmerror code to coordinate making external scratch log and
scratch realtime devices error out along with the scratch device. Note
that unlike SCRATCH_DEV, we save the old rt/log devices in a separate
variable and overwrite SCRATCH_{RT,LOG}DEV so that all the helper
functions continue to work properly.
This is very similar to what we did for dm-flakey a while back.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:58 +0000 (14:00 -0700)]
common/ext4: provide custom ext4 scratch fs options
Create a _scratch_options backend for ext* so that we can inject
pathnames to external log devices into the scratch fs mount options.
This enables common/dm* to install block device filters, e.g. dm-error
for stress testing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:52 +0000 (14:00 -0700)]
common/rc: move XFS-specific parts of _scratch_options into common/xfs
Move all the non-XFS code in _scratch_options into a
_scratch_xfs_options helper in common/xfs, in preparation to add ext4
bits.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro zlang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:46 +0000 (14:00 -0700)]
common/rc: move ext4-specific helpers into a separate common/ext4 file
Move the ext4-specific parts of common/rc into a separate file and
source it when we test that.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 9 Aug 2022 21:00:37 +0000 (14:00 -0700)]
xfs/{015,042,076}: fix mkfs failures with nrext64=1
One of my XFS fstests systems is configured with multiple 34G block
devicesm each with a 4k LBA size for the scratch filesystem. If I turn
on large extent counts with -i nrext64=1, I see the following failures
from each of these three tests:
-used inodes is in range
+max log size 4083 smaller than min log size 4287, filesystem is too small
Note that this particular output is dependent on having a recent
xfsprogs with "mkfs: complain about impossible log size constraints"
applied, else you get a far more obscure message about the log being too
small.
It turns out that you can simulate this pretty easily:
# truncate -s 34G /tmp/a; losetup -f -b 4096 /tmp/a
# truncate -s 34G /tmp/b; losetup -f -b 4096 /tmp/b
# mkfs.xfs -f -N /dev/loop0 -r rtdev=/dev/loop1 -d rtinherit=1,size=32m -i nrext64=1
max log size 4083 smaller than min log size 4287, filesystem is too small
So it's pretty obvious what happened here -- the AG size is 4096 blocks.
mkfs has to leave enough free space in each AG to handle the AG space
btree roots, the AG headers, and the AGFL, which means the log can't be
more than 4083 blocks. libxfs computes the minimum log size to be 4287
blocks, which is why the format fails.
Fix this problem by bumping the filesystems up to 96M, which provides
enough space in each AG to handle a ~20M internal log.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 5 Aug 2022 00:55:52 +0000 (17:55 -0700)]
xfs/018: fix LARP testing for small block sizes
Fix this test to work properly when the filesystem block size is less
than 4k. Tripping the error injection points on shape changes in the
xattr structure must be done dynamically.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com> Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 4 Aug 2022 16:26:49 +0000 (09:26 -0700)]
xfs/533: fix false negatives for this test
xfsprogs 5.19 will change the error message that gets printed when the
primary superblock validation fails. Filter the _get_metadata_field
output so that the new message looks like the old message.
While we're at it, _notrun this test on V4 filesystems because the
validation messages are in the V5 superblock validation functions.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Yang Xu <xuyang2018.jy@fujitsu.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Convert this test to use _scratch_xfs_repair, since the only variance
from the standard usage is that it's called against a sparse file into
which the scratch filesystem has been metadumped and mdrestored.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 3 Aug 2022 04:21:26 +0000 (21:21 -0700)]
xfs/432: fix this test when external devices are in use
This program exercises metadump and mdrestore being run against the
scratch device. Therefore, the test must pass external log / rt device
arguments to xfs_repair -n to check the "restored" filesystem. Fix the
incorrect usage, and report repair failures, since this test has been
silently failing for a while now.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Zorro Lang [Sun, 31 Jul 2022 04:46:50 +0000 (12:46 +0800)]
fstests: add more dmerror related test cases into eio group
The g/484, g/628 and g/629 calls _dmerror_init and load error table
for testing, better to count them in eio test group, then we won't
miss them when we want to run all eio related cases.
Signed-off-by: Zorro Lang <zlang@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 28 Jul 2022 18:17:26 +0000 (11:17 -0700)]
seek_sanity_test: use XFS ioctls to determine file allocation unit size
liuyd.fnst@fujitsu.com reported that my recent change to the seek sanity
test broke NFS. I foolishly thought that st_blksize was sufficient to
find the file allocation unit size so that applications could figure out
the SEEK_HOLE granularity. Replace that with an explicit callout to XFS
ioctls so that xfs realtime will work again.
Fixes: e861a302 ("seek_sanity_test: fix allocation unit detection on XFS realtime") Reported-by: liuyd.fnst@fujitsu.com Tested-by: liuyd.fnst@fujitsu.com Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
/dev/block/8:3 corresponds to the SCRATCH_DEV on this system. Given the
failures in 322.out, I think this is caused by generic/322 calling
_init_flakey -> _dmsetup_create -> $DMSETUP_PROG create being unable to
open SCRATCH_DEV exclusively. Add a call to $UDEV_SETTLE_PROG prior to
the creation of the target to try to calm the system down sufficiently
that the test can proceed.
Note that I don't have any hard evidence that it's udev at fault here --
the few times I've caught this thing, udev *has* been active spraying
error messages for nonexistent sysfs paths to journald and adding a
'udevadm settle' seems to fix it... but that's still only
circumstantial. Regardless, it seems to have fixed the test failure.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 26 Jul 2022 19:48:51 +0000 (12:48 -0700)]
dmlogwrites: skip generic tests when external logdev in use
Currently, dm-logwrites and common/dmlogwrites don't seem to have any
means to coordinate the event numbers across multiple devices, and the
fstests setup code is sufficiently intense that it doesn't seem like
there's support for multi-disk filesystems. For now, we'll _notrun the
tests when we have external log devices, even though that seems like
something we'd _really_ want to test.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs: add test case to make sure btrfs can handle one corrupted device
The new test case will verify that btrfs can handle one corrupted device
without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs,
thus btrfs has to utilize its data/metadata checksum to verify which
data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test
- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock
untouched
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Boris Burkov <boris@bur.io> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
btrfs/253: skip on zoned mode as we cannot change the chunk size
On zoned mode, we have a fixed chunk size which is equal to the zone size.
So, we cannot change the chunk size, and running this test results in a
failure with below.
--- tests/btrfs/253.out 2021-12-10 04:33:53.000000000 +0000
+++ /host/results/btrfs/253.out.bad 2022-07-26 05:58:10.000000000 +0000
@@ -2,9 +2,16 @@
Capture default chunk sizes.
First allocation.
Second allocation.
+./common/rc: line 4670: echo: write error: Invalid argument
+./common/rc: line 4670: echo: write error: Invalid argument
Calculate request size so last memory allocation cannot be completely fullfilled.
Third allocation.
...
It is no use to test this feature on zoned mode. So, just skip it.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Yang Xu [Tue, 19 Jul 2022 08:31:13 +0000 (16:31 +0800)]
xfs: test xfs_db whether still can get corrupted field value
This is a regression test that xfs_db code doesn't clean up the
LIBXFS_READBUF_SALVAGE call sites in set_cur so that we can't use the
return value directly and scraping it out later.
Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / xfstests-dev.git / log