Darrick J. Wong [Fri, 30 Dec 2022 22:19:57 +0000 (14:19 -0800)]
xfs: ensure that online directory repairs don't hit EDQUOT
Add a test to ensure that the sysadmin doesn't get EDQUOT if they try to
repair directory metadata when we've already exceeded a quota limit
somewhere.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 1 Mar 2023 02:59:26 +0000 (18:59 -0800)]
generic, xfs: test scatter-gather atomic file updates
Make sure that FILE_SWAP_RANGE_SKIP_FILE1_HOLES does what we want it to
do -- provide a means to implement scatter-gather atomic file writes.
That means we create a temporary file, write whatever sparse bits to it
that we want, and swap the non-hole parts of the temp file.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Thu, 23 Feb 2023 05:10:49 +0000 (13:10 +0800)]
btrfs: add test case for NODATASUM dev-replace
During my development on dev-replace, I made a mistake in RAID56
dev-replace code where it can lead to NODATASUM corruption.
Thankfully such corruption didn't reach upstream.
Inspired by such incident, here comes a new test case for btrfs
dev-replace, the new case would:
- Populate the filesystem with nodatasum mount option
- Run fssum to record the contents
Since the test case only cares about data contents, here we don't
include metadata like uid/gid/timestamp.
- Wipe one device
- Mount the fs with the missing device
- Verify the contents is still correct
- Replace the missing device
- Verify the contents is still correct again
Before the verification, drop all cache to make sure the 2nd
verification is reading from the disks.
For current kernels, the test case should pass as expected.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Thu, 23 Feb 2023 18:01:51 +0000 (10:01 -0800)]
generic: add test for direct io partial writes
btrfs recently had a bug where a direct io partial write resulted in a
hole in the file. Add a new generic test which creates a 2MiB file,
mmaps it, touches the first byte, then does an O_DIRECT write of the
mmapped buffer into a new file. This should result in the mapped pages
being a mix of in and out of page cache and thus a partial write, for
filesystems using iomap and IOMAP_DIO_PARTIAL.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Leah Rumancik [Tue, 21 Feb 2023 22:36:49 +0000 (14:36 -0800)]
selftest: add tests for debugging testing setup
Many people have developed infrastructure around xfstests. In order to
test a setup, it would be helpful to have dummy tests that have
consistent test outcomes. Add a new test folder with the following
tests:
selftest/001 pass
selftest/002 fail from output mismatch
selftest/003 fail via _fail
selftest/004 skip
selftest/005 crash
selftest/006 hang
Also, create two new groups: 'selftest' which includes tests 001-004 and
'dangerous_selftest' which includes tests 005-006. The selftests will
not run unless explicitly specified.
Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Gabriel Niebler [Tue, 31 Jan 2023 12:38:38 +0000 (13:38 +0100)]
README: Add section to install required packages on (open)SUSE
The README already lists how to install the required packages on other popular
distributions. Since a lot of filesystem development happens on SUSE systems and
both the commands as well as the list of packages are slightly different,
it only makes sense to include these as well.
Signed-off-by: Gabriel Niebler <gniebler@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 24 Feb 2023 05:48:11 +0000 (13:48 +0800)]
fuzzy: for fuzzing ag btrees, find the path to the AG header
The fs population code creates various btrees in /some/ allocation group
with at least two levels. These btrees aren't necessarily created in
agno 0, so we need to find it programmatically. While we're at it, fix
a few of the comments that failed to mention when we're fuzzing interior
nodes and not leaves.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:42 +0000 (14:19 -0800)]
xfs: improve metadata array field handling when fuzzing
Currently, we use some gnarly regular expressions to try to constrain
the amount of time we spend fuzzing each element of a metadata array.
This is pretty inflexible (and buggy) since we limit some arrays
(e.g. dir hashes) to the first ten elements and other arrays (e.g.
extent mappings) that use compact index ranges to the first one.
Replace this whole weird mess with logic that can tease out the array
indices, unroll the compact indices if needed, and give the user more
flexible control over which array elements get used.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
fuzzy: dump metadata state before fuzzing
When we start a fuzz test, dump the metadata to stdout so that anyone
analyzing a failure can see what was in the (supposedly) good image, and
what it turns into after fuzzing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common/fuzzy: exercise the filesystem a little harder after repairing
Use fsstress to exercise the filesystem a little more strenuously after
we've run the fuzzing repair strategy, so that we have a better chance
of tripping over corruption problems.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common: check xfs health after doing an online scrub
After we've run xfs_scrub -n to perform a check of a mounted
filesystem's metadata, we should check the health reporting system to
make sure that the results got recorded. Also wire this up to the xfs
fuzz testing helpers.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common/fuzzy: evaluate xfs_check vs xfs_repair
When fuzzing a filesystem and using the offline repair strategy, compare
the outputs of xfs_check against xfs_repair to ensure that the newer
xfs_repair catches at least as many things as xfs_check does.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
xfs/{35[45],455}: fix bogus corruption errors
The AGFL fuzz tests first fuzz the entire block header, and second
extract flfirst from the AGF header to start a second round of targeted
fuzzing of live bno pointers in the AGFL. However, flfirst (and the
AGFL field detection at the start of the second round of fuzzing) are
detected after we've already been fuzz testing, which means that the
AGFL might be garbage because repair failed or was not called. If this
is the case, test will fail because the _scratch_xfs_db -c 'agf 0' -c
'p flfirst' call emits things like this:
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the post-repair fs modification code
While auditing the fuzz tester code, I noticed there were numerous
problems with the code that test-drives the filesystem after we've run
the repair strategy. Now that we've made sure that the repair strategy
checks its own efficacy, we can rearrange this function to try making
mods and then re-check the filesystem afterwards. Also, disable
xfs_repair prefetch to reduce the likelihood of OOM kills.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the online-then-offline repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the online-then-offline repair strategy -- the stages of
the strategy are not consistently logged to the kernel log, some of the
error messages don't identify /which/ scrubber we're calling, we don't
do a pre-repair check to make sure we detect the fuzzed fields, and we
don't actually re-run online scrub after a repair to make sure that it's
ok. Disable xfs_repair prefetch to reduce the possibility of OOM kills.
Rework the error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the no-repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the no repair strategy -- the stages of the strategy
are not consistently logged to the kernel log, and we don't actually
verify that either online or offline scrubs notice the fuzz. Rework the
error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the offline repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the offline repair strategy -- the stages of the strategy
are not consistently logged to the kernel log, we don't actually check
that repair -n finds the fuzzed field, and since this is an offline
test, we don't need or want to mount or try to run the online scrubber.
Also, disable prefetch to reduce the chance of an OOM kill. Rework the
error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the online repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the online repair strategy -- the stages of the strategy
are not consistently logged to the kernel log, some of the error
messages don't identify /which/ scrubber we're calling, and we don't
actually re-run online scrub after a repair to make sure that it's
verification is ok. Disable xfs_repair prefetch to reduce the chances
of an OOM kill, and abort the fuzz test if we can't mount. We also
reorganize the error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:39 +0000 (14:19 -0800)]
fuzzy: don't fuzz xattr namespace flags and values
Extended attribute namespace flags are controlled by userspace, and
there is no validation imposed on the values. Don't bother fuzzing
either of these things.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:39 +0000 (14:19 -0800)]
fuzzy: don't fuzz the log sequence number
Don't bother filtering log sequence numbers since xfs_db doesn't have
the ability to tell us the range of LSNs that would actually cause
validation failures.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:36 +0000 (14:19 -0800)]
populate: take a snapshot of the filesystem if creation fails
There have been a few bug reports filed about people not being able to
use the filesystem metadata population code to create filesystems with
all types of metadata on them. Right now this is super-annoying to
debug because we don't capture a metadump for easy debugging. Fix that.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: race fsstress with online repair for special file metadata
For each XFS_SCRUB_TYPE_* that looks at symbolic link and special file
metadata, create a test that runs that repairer in the foreground and
fsstress in the background.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: ensure that online file data fork repairs don't hit EDQUOT
Add a test to ensure that the sysadmin doesn't get EDQUOT if they try to
repair file data fork metadata when we've already exceeded a quota limit
somewhere.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: race fsstress with online repair for inode and fork metadata
For each XFS_SCRUB_TYPE_* that looks at inode and data/attr/cow fork
metadata, create a test that runs that repairer in the foreground and
fsstress in the background.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: test rebuilding xattrs when the data fork is btree format
Make sure we handle the case of rebuilding extended attributes properly
when the data fork is in btree format and we therefore cannot zap the
attr fork.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:09 +0000 (14:19 -0800)]
fuzzy: use FORCE_REBUILD over injecting force_repair
For stress testing online repair, try to use the FORCE_REBUILD ioctl
flag over the error injection knobs whenever possible because the knobs
are very noisy and are not always available.
[zlang: do not export the SCRUBSTRESS_USE_FORCE_REBUILD]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 16 Feb 2023 16:21:50 +0000 (16:21 +0000)]
generic/604: fix test to actually create dirty inodes
The test case generic/604 aims to test a scenario where at unmount time we
have many dirty inodes, however the test does not actually creates any
files, because it calls xfs_io without the -f argument, so xfs_io fails
but any error is ignored because stderr is redirected to /dev/null.
Fix this by passing -f to xfs_io and also stop redirecting stderr to
/dev/null, so that in case of any unexpected failure creating files, the
test fails.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Fri, 17 Feb 2023 23:57:58 +0000 (07:57 +0800)]
fstests: btrfs/249: add _wants_kernel_commit and _fixed_by_git_commit
Add the _wants_kernel_commit tag for kernel and _fixed_by_git_commit tag
for the btrfs-progs for the benefit of testing on the older kernels and
the older btrfs-progs.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Wed, 15 Feb 2023 07:51:22 +0000 (15:51 +0800)]
fstests: btrfs/185, 198 and 219 add _fixed_by_kernel_commit
Recently, these test cases were added to the auto group. To ensure we have
some clues if they fail in older kernels, add "_fixed_by_kernel_commit"
for the fix and update the test summary.
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
xfs/080 is not dangerous, isn't a known fail and runs very quickly.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
generic/251 isn't dangerous, doesn't takes overly long to run and doesn't
produce spurious failures, so add it to the auto group.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
This test is not dangerous and passes reliably. Add it to the auto
group.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
generic/042 was removed from the auto group in 2015 by commit 7721b8501608 ("generic/042: remove from the 'auto' group") because it
always failed on XFS and wasn't run for other file systems back then.
Since then XFS fixed the problem it reproduces, and ext4 and f2fs
have grown shutdown support and also pass it reliably.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 7 Feb 2023 17:00:12 +0000 (09:00 -0800)]
common/xfs: use whole-word matching for _require_xfsrestore_xflag
On my system, the path to the xfsrestore binary is:
/code/xfsdump/build-x86_64/restore/xfsrestore
The grep command in _require_xfsrestore_xflag matches on the "build-x86"
part, even though my version of xfsrestore does not actually have a -x
flag. Fix the string parsing to match entire words so that we only look
for -x in the help output.
(Maybe someone should patch xfsrestore -h to report basename(argv[0])
instead of argv[0]...)
Fixes: 1ffa16c573 ("xfs: test for fixing wrong root inode number in dump") Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Gabriel Niebler [Fri, 3 Feb 2023 17:54:37 +0000 (18:54 +0100)]
common: Chown mount even if already idmapped to account for remounts
This is a logical consequence of introducing the chown check in _idmapped_mount,
since now a read-only mount can be made idmapped successfully. But if the mount
is then remounted rw the chown never happens, as _idmapped_mount sees that it's
already idmapped and bows out early.
This patch fixes that by simply moving the chown ahead of the idmapped check,
so it will be performed in any case, even on already idmapped mounts.
Signed-off-by: Gabriel Niebler <gniebler@suse.com> Reviewed-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Gabriel Niebler [Fri, 3 Feb 2023 14:35:45 +0000 (15:35 +0100)]
common: Do not chown ro mountpoint when creating idmapped mount
The function _idmapped_mount tries to change the ownership of the mountpoint
for which it aims to create an idmapped mount, to ensure that the mapped UID
and GID can actually create objects within it. Some tests set up a read-only
mount, however, which lets the chown call fail. This patch fixes the
function to check whether the mount is read-only and skip the chown, if so.
Signed-off-by: Gabriel Niebler <gniebler@suse.com> Reviewed-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 2 Feb 2023 15:58:09 +0000 (15:58 +0000)]
btrfs: add a stress test for send v2 streams
Currently we don't have any test case in fstests to do randomized and
stress testing of the send stream v2, added in kernel 6.0 and support for
it in btrfs-progs v5.19. For the send v2 stream, we only have btrfs/281
that exercises a specific scenario which used to trigger a bug.
So add a test that uses fsstress to generate a filesystem and exercise
both full and incremental send operations using the v2 send stream with
compressed extents, and then receive the streams without and with
decompression, to verify they work and produce the same results as in
the original filesystem. This is the same base idea as btrfs/007, but
for the send v2 stream with compressed data.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
generic/676: Unstable d_type handling for NFS READDIR
The NFS client may send READDIR or READDIRPLUS to populate the dentry
cache, and switch between them to optimize for least RPC calls based on the
process' behavior. When using READDIR, dentries will have d_type =
DT_UNKNOWN but with READDIRPLUS d_type will be set from the mode.
This heuristic will cause generic/676 to fail when comparing dentries
cached from one or the other call, since we compare d_type directly. Fix
this by bypassing the comparison of d_type if any entry is loaded with
DT_UNKNOWN.
Signed-off-by: Benjamin Coddington <bcodding@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Tue, 31 Jan 2023 12:39:59 +0000 (13:39 +0100)]
common: Improve blocksize support for udf
Add better support for blocksize selection in _scratch_mkfs_sized
(accept another variant of mkfs options, select correct default block
size if not specified). Also add blocksize selection support to
_scratch_mkfs_blocksized.
For _check_udf_filesystem to keep working when blocksize is not
specified in MKFS_OPTIONS, add detection of blocksize from a mounted
filesystem.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Tue, 31 Jan 2023 12:39:57 +0000 (13:39 +0100)]
common: Provide blocksize and ecclength to udf fsck
udf_test program used for verifying filesystem is not able to determine
filesystem blocksize. Provide it in the options together with disabling
ecclength as it is not used on harddrives.
Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Zorro Lang <zlang@kernel.org>
Make it so that xfs_scrub stress tests can select what kind of fsstress
operations they want to run. This will make it easier for, say,
directory scrubbers to configure fsstress to exercise directory tree
changes while skipping file data updates, because those are irrelevant.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Tue, 7 Feb 2023 17:01:41 +0000 (09:01 -0800)]
fuzzy: add a custom xfs find utility for scrub stress tests
Create a new find(1) like utility that doesn't crash on directory tree
changes (like find does due to bugs in its loop detector) and actually
implements the custom xfs attribute predicates that we need for scrub
stress tests. This program will be needed for a future patch where we
add stress tests for scrub and repair of file metadata.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:06 +0000 (14:19 -0800)]
xfs: race fsstress with online scrubbers for AG and fs metadata
For each XFS_SCRUB_TYPE_* that looks at AG or filesystem metadata,
create a test that runs that scrubber in the foreground and fsstress in
the background.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
xfs/191: fix xattr leaf block emptying on 64k blocksized fses
This test is failing on filesystems with 64k blocksize since the leaf
hdr.firstused field is 16 bit and as such trying to reset it to $dbsize
overflows and is rejected by xfs_db. The leaf is never properly resetted
and the discrepancy is picked up by xfs_repair, thus failing the test.
Fix it by setting it to XFS_ATTR3_LEAF_NULLOFF (0) as this is the proper
on-disk value to indicate an empty leaf on 64k blocksized fses.
Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Anthony Iliopoulos <ailiop@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Anand Jain [Sun, 29 Jan 2023 02:42:31 +0000 (10:42 +0800)]
fstests: fstest.c, fix compile warnings replace sprintf with snprintf
Fixes the buffer overflow warnings, by using snprintf instead of
sprintf.
fstest.c:95:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=]
sprintf(fname, "%s/file%d", dir, fnum);
^~~~~
fstest.c:166:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=]
sprintf(fname, "%s/file%d", dir, fnum);
Signed-off-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 1 Feb 2023 00:51:40 +0000 (16:51 -0800)]
generic/500: skip this test if formatting fails
This testcase exercises what happens when we race a filesystem
perforing discard operations against a thin provisioning device that has
run out of space. To constrain runtime, it creates a 128M thinp volume
and formats it.
However, if that initial format fails because (say) the 128M volume is
too small, then the test fails. This is really a case of test
preconditions not being satisfied, so let's make the test _notrun when
this happens.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 1 Feb 2023 00:51:35 +0000 (16:51 -0800)]
generic/038: set a maximum runtime on this test
This test races multiple FITRIM calls against multiple programs creating
200k small files to ensure that there are no concurrency problems with
the allocator and the FITRIM code. This is not necessarily quick, and
the test itself does not contain any upper bound on the runtime. On my
system that simulates storage with DRAM this takes ~5 minutes to run; on
my cloud system with newly provided discard support, I killed the test
after 27 hours.
Constrain the runtime to about the customary 30s * TIME_FACTOR.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Jan Kara [Tue, 31 Jan 2023 12:54:16 +0000 (13:54 +0100)]
generic/707: Test moving directory while being grown
Test how the filesystem can handle moving a directory to a different
directory (so that parent pointer gets updated) while it is grown. Ext4
and UDF had a bug where if the directory got converted to a different
type due to growth while rename is running, the filesystem got
corrupted.
Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Hironori Shiina [Mon, 30 Jan 2023 22:56:43 +0000 (17:56 -0500)]
xfs: test xfsrestore on multi-level dumpfiles with wrong root
While developing `xfsrestore -x`, we hit an issue at restoring a
renamed file in the cumulative mode (multi-level dumps):
https://lore.kernel.org/linux-xfs/e61ae295-a331-d36a-cae1-646022dc2a6e@gmail.com/
Then, this patch adds test cases where '-x' flag is used in the
cumulative mode with various file operations referring to existing tests.
Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Hironori Shiina [Mon, 30 Jan 2023 22:56:42 +0000 (17:56 -0500)]
xfs: add helper to create fake root inode
xfsdump used to cause a problem when there is an inode whose number is
lower than the root inode number. This patch adds a helper function to
reproduce such a situation for regression tests.
Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Wed, 25 Jan 2023 11:07:54 +0000 (11:07 +0000)]
btrfs: test send optimal cloning behaviour
Test that send operations do the best cloning decisions when we have
extents that are shared but some files refer to the full extent while
others refer to only a section of the extent.
This exercises an optimization that was added to kernel 6.2, by the
following commit:
c7499a64dcf6 ("btrfs: send: optimize clone detection to increase extent sharing")
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 18 Jan 2023 00:44:18 +0000 (16:44 -0800)]
populate: improve attr creation runtime
Replace the file creation loops with a python script that does
everything we want from a single process. This reduces the runtime of
_scratch_xfs_populate substantially by avoiding thousands of execve
overhead. This patch builds on the previous one by reducing the runtime
of xfs/349 from ~45s to ~15s.
For people who don't have python3, use setfattr's "restore" mode to bulk
create xattrs. This reduces runtime to about ~25s.
[zlang: add popattr.py into src/Makefile install list]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 18 Jan 2023 00:44:02 +0000 (16:44 -0800)]
populate: remove file creation loops that take forever
Replace the file creation loops with a perl script that does everything
we want from a single process. This reduces the runtime of
_scratch_xfs_populate substantially by avoiding thousands of execve
overhead. On my system, this reduces the runtime of xfs/349 (with scrub
enabled) from ~140s to ~45s.
[zlang: add popdir.pl into src/Makefile install list]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / xfstests-dev.git / log