Darrick J. Wong [Wed, 29 Mar 2023 00:58:16 +0000 (17:58 -0700)]
xfs/242: fix _filter_bmap for xfs_io bmap that does rt file properly
xfsprogs commit b1faed5f787 ("xfs_io: fix bmap command not detecting
realtime files with xattrs") fixed the xfs_io bmap output to display
realtime file columns for realtime files with xattrs. As a result, the
data and unwritten flags are in column 5 and not column 7.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Thu, 30 Mar 2023 16:46:17 +0000 (09:46 -0700)]
generic/{251,260}: compute maximum fitrim offset
FITRIM is a bizarre ioctl. Callers are allowed to pass in "start" and
"length" parameters, which are clearly some kind of range argument. No
means is provided to discover the minimum or maximum range. Although
regular userspace programs default to (start=0, length=-1ULL), this test
tries to exercise different parameters.
However, the test assumes that the "size" column returned by the df
command is the maximum value supported by the FITRIM command, and is
surprised if the number of bytes trimmed by (start=0, length=-1ULL) is
larger than this size quantity.
This is completely wrong on XFS with realtime volumes, because the
statfs output (which is what df reports) will reflect the realtime
volume if the directory argument is a realtime file or a directory
flagged with rtinherit. This is trivially reproducible by configuring a
rt volume that is much larger than the data volume, setting rtinherit on
the root dir at mkfs time, and running either of these tests.
Refactor the open-coded df logic so that we can determine the value
programmatically for XFS.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Signed-off-by: Zorro Lang <zlang@kernel.org>
common: switch _get_{available,total}_space accounting to units of single bytes
xfs/220 relies on detecting free space changes after truncation of
single filesystem blocks, and this fails when the fs block size is 512b.
By default df is counting 1024b block units and as such is not reporting
any change in the number of available blocks after freeing just a 512b
block.
Switch the _get_available_space df reporting block size from units of
1024b blocks directly to single bytes in order to make free space
accounting fine-grained and independent of any fs block size
assumptions.
Do the same for _get_available_space, since there is no reason for
having an additional conversion step.
Signed-off-by: Anthony Iliopoulos <ailiop@suse.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:55 +0000 (17:53 -0700)]
report: allow test runners to inject arbitrary values
Per Ted's request, add to the test section reporting code the ability
for test runners to point to a file containing colon-separated key value
pairs. These key value pairs will be recorded in the report file as
extra properties.
Requested-by: Theodore Ts'o <tytso@mit.edu> Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:32 +0000 (17:53 -0700)]
report: collect basic information about a test run
Record various generic information about an fstests run when generating
a junit xml report. This includes the cpu architecture, the kernel
revision, the CPU, memory, and numa node counts, and some information
about the block devices passed in.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:27 +0000 (17:53 -0700)]
report: encode xml entities in property values
Avoid trouble with the properties reported in the xml reports by
translating xml-tricky characters in the property values into their xml
entity equivalents.
IOWs, if someone sets a property "NAME" to the value 'BOBBY"; DROP TABLES;',
the xml will be formatted:
<property name="NAME" value="BOBBY"; DROP TABLES;"/>
Thus avoiding XML problems.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:21 +0000 (17:53 -0700)]
report: pass property value to _xunit_add_property
Change this helper to require the caller to pass the value as the second
parameter. This prepares us to start reporting a lot more information
about a test run, not all of which are encoded as bash variables.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:10 +0000 (17:53 -0700)]
report: encode the kernel log as a separate xml element
Record the .dmesg file in a new <kernel-log> element instead of
multiplexing it with <system-err>. This means that the xml report can
now capture kernel log and bad golden output.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:53:04 +0000 (17:53 -0700)]
report: encode cdata sections correctly
The XML report format captures the contents of .full and .out.bad files
in CDATA sections. CDATA sections are supposed to be a stream of
verbatim data, terminated with a "]]>". Hence XML entities such as
quotation marks and angle brackes should not be escaped, and an actual
bracket-bracket-gt sequence in those files /does/ need escaping.
Create a separate filtering function so that these files are encoded
properly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:59 +0000 (17:52 -0700)]
report: record fstests start and report generation timestamps
Report two new timestamps in the xml report: the time that ./check was
started, and the time that the report was generated. We introduce new
timestamps to minimize breakage with parsing scripts.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:53 +0000 (17:52 -0700)]
report: clarify the meaning of the timestamp attribute
We've never specified what the timestamp attribute of the testsuite
element actually means, and it history is rather murky.
Prior to the introduction of the xml report format in commit f9fde7db2f,
the "date_time" variable was used only to scrape dmesg via the /dev/kmsg
device after each test. If /dev/kmsg was not a writable path, the
variable was not set at all. In this case, the report timestamp would
be blank.
In commit ffdecf7498a1, Ted changed the xunit report code to handle
empty date_time values by setting date_time to the time of report
generation. This change was done to handle the case where no tests are
run at all. However, it did not change the behavior that date_time is
not set if /dev/kmsg is not writable.
Clear up all this confusion by defining the timestamp attribute to
reflect the start time of the most recent test, regardless of the state
of /dev/kmsg. If no tests are run, then define the attribute to be the
time of report generation.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:47 +0000 (17:52 -0700)]
report: capture the time zone in the test report timestamp
Make sure we put the time zone of the system running the test in the
timestamp that is recorded in the xunit report. `date "+%F %T"' reports
the local time zone, not UTC.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:42 +0000 (17:52 -0700)]
report: derive an xml schema for the xunit report
The "xunit" report format emits an XML document that more or less
follows the junit xml schema. However, there are two major exceptions:
1. fstests does not emit an @errors attribute on the testsuite element
because we don't have the concept of unanticipated errors such as
"unchecked throwables".
2. The system-out/system-err elements sound like they belong under the
testcase element, though the schema itself imprecisely says "while the
test was executed". The schema puts them under the top-level testsuite
element, but we put them under the testcase element.
Define an xml schema for the xunit report format, and update the xml
headers to link to the schema file. This enables consumers of the
reports to check mechanically that the incoming document follows the
format.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:52:36 +0000 (17:52 -0700)]
check: generate section reports between tests
Generate the section report between tests so that the summary report
always reflects the outcome of the most recent test. Two usecases are
envisioned here -- if a cluster-based test runner anticipates that the
testrun could crash the VM, they can set REPORT_DIR to (say) an NFS
mount to preserve the intermediate results. If the VM does indeed
crash, the scheduler can examine the state of the crashed VM and move
the tests to another VM. The second usecase is a reporting agent that
runs in the VM to upload live results to a test dashboard.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Leah Rumancik <leah.rumancik@gmail.com> Tested-by: Leah Rumancik <leah.rumancik@gmail.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Eric Biggers [Sun, 19 Mar 2023 19:38:46 +0000 (12:38 -0700)]
fscrypt-crypt-util: use OpenSSL EVP API for AES self-tests
OpenSSL 3.0 has deprecated the easy-to-use AES block cipher API in favor
of EVP. EVP is also available in earlier OpenSSL versions. Therefore,
update test_aes_keysize() to use the non-deprecated API to avoid
deprecation warnings when building the algorithm self-tests.
Signed-off-by: Eric Biggers <ebiggers@google.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Khem Raj [Sat, 25 Mar 2023 01:28:58 +0000 (18:28 -0700)]
m4: Check for FTW_ACTIONRETVAL along with nftw
FTW_ACTIONRETVAL is glibc specific extention which is used to implement
xfsfind but it may not be available on other C library implementations on Linux
e.g. musl. Therefore ensure that these defines are available before declaring
nftw() to be usable
Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Darrick J. Wong <djwong@kernel.org> Cc: Zorro Lang <zlang@redhat.com> Acked-by: Darrick J. Wong <djwong@kernel.org>
Qu Wenruo [Wed, 22 Mar 2023 05:52:11 +0000 (13:52 +0800)]
btrfs/246: skip the test if the tested btrfs doesn't support inline extents creation
[FALSE ALERT]
If test case btrfs/246 is executed with a 16K page sized system (like
some aarch64 SoCs) using 4K sector size (would be the new default), the
test case would fail with output mismatch:
btrfs/246 1s ... - output mismatch (see ~/xfstests-dev/results//btrfs/246.out.bad)
--- tests/btrfs/246.out 2022-11-24 19:53:53.158470844 +0800
+++ ~/xfstests-dev/results//btrfs/246.out.bad 2023-03-22 13:27:34.975796048 +0800
@@ -3,3 +3,5 @@ 0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0 SCRATCH_MNT/foobar
sha256sum after mount cycle 0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0 SCRATCH_MNT/foobar
+no inline extent found
+no compressed extent found
...
(Run 'diff -u ~/xfstests-dev/tests/btrfs/246.out ~/xfstests-dev/results//btrfs/246.out.bad' to see the entire diff)
[CAUSE]
For current btrfs subpage support, there are still some limitations:
- No compressed write if the range is not fully page aligned
- No inline extents creation
Reading inline extents is still supported
Thus we won't create such inlined compressed extent at all.
[FIX]
Just skip the test case if we can not even create a regular inline
extent.
This is done by a new require helper,
_require_btrfs_inline_extent_creation(), which would detect if btrfs can
even create an uncompressed inlined extent.
Reported-by: Hector Martin <marcan@marcan.st> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Neal Gompa <neal@gompa.dev> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiubo Li [Sun, 19 Mar 2023 06:29:27 +0000 (14:29 +0800)]
generic/020: fix another really long attr test failure for ceph
If the CONFIG_CEPH_FS_SECURITY_LABEL is disabled the kernel ceph
the 'selinux_size' will be empty and then:
max_attrval_size=$((65536 - $size - $selinux_size - $max_attrval_namelen))
will be:
max_attrval_size=$((65536 - $size - - $max_attrval_namelen))
which equals to:
max_attrval_size=$((65536 - $size + $max_attrval_namelen))
Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 15 Mar 2023 00:58:17 +0000 (17:58 -0700)]
xfs: stress test cycling parent pointers with online repair
Add a couple of new tests to exercise directory and parent pointer
repair against rename() calls moving child subdirectories from one
parent to another. This is a useful test because it turns out that the
VFS doesn't lock the child subdirectory (it does lock the parents), so
repair must be more careful.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 7 Mar 2023 11:38:49 +0000 (11:38 +0000)]
btrfs/286: add missing calls to _scratch_dev_pool_put and _spare_dev_put
The test is doing a _scratch_dev_pool_get, which shrinks the list of
devices in SCRATCH_DEV_POOL, but it's not calling _scratch_dev_pool_put
before it finishes. This will result in subsequent tests (none at the
moment however) getting a reduced list of devices in SCRATCH_DEV_POOL.
The same goes for the spare device, the test calls _spare_dev_get but
it never calls _spare_dev_put.
So add the missing calls.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Tue, 7 Mar 2023 11:38:13 +0000 (11:38 +0000)]
btrfs/284: list a couple btrfs-progs git commits
This test may often fail when running with btrfs-progs versions not very
recent. The corresponding git commits in btrfs-progs that fix issues
uncovered by this test are:
1) 6f4a51886b37 ("btrfs-progs: receive: fix silent data loss after fall back from encoded write")
Introduced in btrfs-progs v6.0.2;
2) e3209f8792f4 ("btrfs-progs: receive: fix a corruption when decompressing zstd extents"")
Introduced in btrfs-progs v6.2.
So add the corresponding _fixed_by_git_commit calls to the test.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Rodrigo Campos [Tue, 14 Mar 2023 11:45:11 +0000 (12:45 +0100)]
vfs: Add tmpfs tests for idmap mounts
This patch calls all tests in the suite s_idmapped_mounts, but with a
tmpfs directory mounted inside a userns. This directory is setup as the
mount point for the test that runs nested.
This excercises that tmpfs mounted inside a userns works as expected
regarding idmap mounts.
Signed-off-by: Rodrigo Campos <rodrigo@sdfg.com.ar> Reviewed-by: Zorro Lang <zlang@redhat.com> Acked-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Rodrigo Campos [Tue, 14 Mar 2023 11:45:07 +0000 (12:45 +0100)]
vfs: Make switch_userns set PR_SET_DUMPABLE
We need PR_SET_DUMPABLE in order to write the mapping files when
creating a userns. From prctl(2) PR_SET_DUMPABLE is reset when the
process's effective user or group ID is changed.
As we are changing the EUID here, we also reset it to allow creating
nested userns with subsequent switch_users() calls.
This was not causing any issues because we weren't using switch_users()
to create nested userns. Nested userns were created with
userns_fd_cb()/create_userns_hierarchy() that set PR_SET_DUMPABLE.
Future patches will rely on switch_users() to create nested userns. So
this patch fixes that.
Signed-off-by: Rodrigo Campos <rodrigo@sdfg.com.ar> Reviewed-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Rodrigo Campos [Tue, 14 Mar 2023 11:45:06 +0000 (12:45 +0100)]
vfs: Fix race condition on get_userns_fd()
There is a race when we clone: we call a function that just returns
while at the same time we try to get the userns via /proc/pid/ns/user.
The thing is that when the function returns, in the kernel do_exit()
from kernel/exit.c is called, which calls exit_task_namespaces() to destroy
the namespaces.
So, let's wait indefinitely there and add an _exit() call to avoid
warnings. We are already sending a SIGKILL to this pid, so nothing else
remaining to not leak the process.
Signed-off-by: Rodrigo Campos <rodrigo@sdfg.com.ar> Reviewed-by: Christian Brauner <brauner@kernel.org> Signed-off-by: Zorro Lang <zlang@kernel.org>
Xiubo Li [Mon, 27 Feb 2023 04:13:58 +0000 (12:13 +0800)]
generic/020: fix really long attr test failure for ceph
If the CONFIG_CEPH_FS_SECURITY_LABEL is enabled the kernel ceph
itself will set the security.selinux extended attribute to MDS.
And it will also eat some space of the total size.
Fixes: https://tracker.ceph.com/issues/58742 Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
common/rc: don't clear superblock for zoned scratch pools
_require_scratch_dev_pool() zeros the first 100 sectors of each device to
clear eventual remains of older filesystems.
On zoned devices this won't work as a plain dd will end up creating
unaligned write errors failing all subsequent actions on the device.
For zoned devices it is enough to simply reset the first two zones of the
device to achieve the same result.
Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:57 +0000 (14:19 -0800)]
xfs: ensure that online directory repairs don't hit EDQUOT
Add a test to ensure that the sysadmin doesn't get EDQUOT if they try to
repair directory metadata when we've already exceeded a quota limit
somewhere.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Wed, 1 Mar 2023 02:59:26 +0000 (18:59 -0800)]
generic, xfs: test scatter-gather atomic file updates
Make sure that FILE_SWAP_RANGE_SKIP_FILE1_HOLES does what we want it to
do -- provide a means to implement scatter-gather atomic file writes.
That means we create a temporary file, write whatever sparse bits to it
that we want, and swap the non-hole parts of the temp file.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Qu Wenruo [Thu, 23 Feb 2023 05:10:49 +0000 (13:10 +0800)]
btrfs: add test case for NODATASUM dev-replace
During my development on dev-replace, I made a mistake in RAID56
dev-replace code where it can lead to NODATASUM corruption.
Thankfully such corruption didn't reach upstream.
Inspired by such incident, here comes a new test case for btrfs
dev-replace, the new case would:
- Populate the filesystem with nodatasum mount option
- Run fssum to record the contents
Since the test case only cares about data contents, here we don't
include metadata like uid/gid/timestamp.
- Wipe one device
- Mount the fs with the missing device
- Verify the contents is still correct
- Replace the missing device
- Verify the contents is still correct again
Before the verification, drop all cache to make sure the 2nd
verification is reading from the disks.
For current kernels, the test case should pass as expected.
Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Boris Burkov [Thu, 23 Feb 2023 18:01:51 +0000 (10:01 -0800)]
generic: add test for direct io partial writes
btrfs recently had a bug where a direct io partial write resulted in a
hole in the file. Add a new generic test which creates a 2MiB file,
mmaps it, touches the first byte, then does an O_DIRECT write of the
mmapped buffer into a new file. This should result in the mapped pages
being a mix of in and out of page cache and thus a partial write, for
filesystems using iomap and IOMAP_DIO_PARTIAL.
Signed-off-by: Boris Burkov <boris@bur.io> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Leah Rumancik [Tue, 21 Feb 2023 22:36:49 +0000 (14:36 -0800)]
selftest: add tests for debugging testing setup
Many people have developed infrastructure around xfstests. In order to
test a setup, it would be helpful to have dummy tests that have
consistent test outcomes. Add a new test folder with the following
tests:
selftest/001 pass
selftest/002 fail from output mismatch
selftest/003 fail via _fail
selftest/004 skip
selftest/005 crash
selftest/006 hang
Also, create two new groups: 'selftest' which includes tests 001-004 and
'dangerous_selftest' which includes tests 005-006. The selftests will
not run unless explicitly specified.
Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Gabriel Niebler [Tue, 31 Jan 2023 12:38:38 +0000 (13:38 +0100)]
README: Add section to install required packages on (open)SUSE
The README already lists how to install the required packages on other popular
distributions. Since a lot of filesystem development happens on SUSE systems and
both the commands as well as the list of packages are slightly different,
it only makes sense to include these as well.
Signed-off-by: Gabriel Niebler <gniebler@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 24 Feb 2023 05:48:11 +0000 (13:48 +0800)]
fuzzy: for fuzzing ag btrees, find the path to the AG header
The fs population code creates various btrees in /some/ allocation group
with at least two levels. These btrees aren't necessarily created in
agno 0, so we need to find it programmatically. While we're at it, fix
a few of the comments that failed to mention when we're fuzzing interior
nodes and not leaves.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:42 +0000 (14:19 -0800)]
xfs: improve metadata array field handling when fuzzing
Currently, we use some gnarly regular expressions to try to constrain
the amount of time we spend fuzzing each element of a metadata array.
This is pretty inflexible (and buggy) since we limit some arrays
(e.g. dir hashes) to the first ten elements and other arrays (e.g.
extent mappings) that use compact index ranges to the first one.
Replace this whole weird mess with logic that can tease out the array
indices, unroll the compact indices if needed, and give the user more
flexible control over which array elements get used.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
fuzzy: dump metadata state before fuzzing
When we start a fuzz test, dump the metadata to stdout so that anyone
analyzing a failure can see what was in the (supposedly) good image, and
what it turns into after fuzzing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common/fuzzy: exercise the filesystem a little harder after repairing
Use fsstress to exercise the filesystem a little more strenuously after
we've run the fuzzing repair strategy, so that we have a better chance
of tripping over corruption problems.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common: check xfs health after doing an online scrub
After we've run xfs_scrub -n to perform a check of a mounted
filesystem's metadata, we should check the health reporting system to
make sure that the results got recorded. Also wire this up to the xfs
fuzz testing helpers.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common/fuzzy: evaluate xfs_check vs xfs_repair
When fuzzing a filesystem and using the offline repair strategy, compare
the outputs of xfs_check against xfs_repair to ensure that the newer
xfs_repair catches at least as many things as xfs_check does.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
xfs/{35[45],455}: fix bogus corruption errors
The AGFL fuzz tests first fuzz the entire block header, and second
extract flfirst from the AGF header to start a second round of targeted
fuzzing of live bno pointers in the AGFL. However, flfirst (and the
AGFL field detection at the start of the second round of fuzzing) are
detected after we've already been fuzz testing, which means that the
AGFL might be garbage because repair failed or was not called. If this
is the case, test will fail because the _scratch_xfs_db -c 'agf 0' -c
'p flfirst' call emits things like this:
Darrick J. Wong [Fri, 30 Dec 2022 22:19:41 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the post-repair fs modification code
While auditing the fuzz tester code, I noticed there were numerous
problems with the code that test-drives the filesystem after we've run
the repair strategy. Now that we've made sure that the repair strategy
checks its own efficacy, we can rearrange this function to try making
mods and then re-check the filesystem afterwards. Also, disable
xfs_repair prefetch to reduce the likelihood of OOM kills.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the online-then-offline repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the online-then-offline repair strategy -- the stages of
the strategy are not consistently logged to the kernel log, some of the
error messages don't identify /which/ scrubber we're calling, we don't
do a pre-repair check to make sure we detect the fuzzed fields, and we
don't actually re-run online scrub after a repair to make sure that it's
ok. Disable xfs_repair prefetch to reduce the possibility of OOM kills.
Rework the error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the no-repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the no repair strategy -- the stages of the strategy
are not consistently logged to the kernel log, and we don't actually
verify that either online or offline scrubs notice the fuzz. Rework the
error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the offline repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the offline repair strategy -- the stages of the strategy
are not consistently logged to the kernel log, we don't actually check
that repair -n finds the fuzzed field, and since this is an offline
test, we don't need or want to mount or try to run the online scrubber.
Also, disable prefetch to reduce the chance of an OOM kill. Rework the
error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:40 +0000 (14:19 -0800)]
common/fuzzy: fix some problems with the online repair strategy
While auditing the fuzz tester code, I noticed there were numerous
problems with the online repair strategy -- the stages of the strategy
are not consistently logged to the kernel log, some of the error
messages don't identify /which/ scrubber we're calling, and we don't
actually re-run online scrub after a repair to make sure that it's
verification is ok. Disable xfs_repair prefetch to reduce the chances
of an OOM kill, and abort the fuzz test if we can't mount. We also
reorganize the error messages to make reading the golden output easier.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:39 +0000 (14:19 -0800)]
fuzzy: don't fuzz xattr namespace flags and values
Extended attribute namespace flags are controlled by userspace, and
there is no validation imposed on the values. Don't bother fuzzing
either of these things.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:39 +0000 (14:19 -0800)]
fuzzy: don't fuzz the log sequence number
Don't bother filtering log sequence numbers since xfs_db doesn't have
the ability to tell us the range of LSNs that would actually cause
validation failures.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:36 +0000 (14:19 -0800)]
populate: take a snapshot of the filesystem if creation fails
There have been a few bug reports filed about people not being able to
use the filesystem metadata population code to create filesystems with
all types of metadata on them. Right now this is super-annoying to
debug because we don't capture a metadump for easy debugging. Fix that.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: race fsstress with online repair for special file metadata
For each XFS_SCRUB_TYPE_* that looks at symbolic link and special file
metadata, create a test that runs that repairer in the foreground and
fsstress in the background.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: ensure that online file data fork repairs don't hit EDQUOT
Add a test to ensure that the sysadmin doesn't get EDQUOT if they try to
repair file data fork metadata when we've already exceeded a quota limit
somewhere.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: race fsstress with online repair for inode and fork metadata
For each XFS_SCRUB_TYPE_* that looks at inode and data/attr/cow fork
metadata, create a test that runs that repairer in the foreground and
fsstress in the background.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:18 +0000 (14:19 -0800)]
xfs: test rebuilding xattrs when the data fork is btree format
Make sure we handle the case of rebuilding extended attributes properly
when the data fork is in btree format and we therefore cannot zap the
attr fork.
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Darrick J. Wong [Fri, 30 Dec 2022 22:19:09 +0000 (14:19 -0800)]
fuzzy: use FORCE_REBUILD over injecting force_repair
For stress testing online repair, try to use the FORCE_REBUILD ioctl
flag over the error injection knobs whenever possible because the knobs
are very noisy and are not always available.
[zlang: do not export the SCRUBSTRESS_USE_FORCE_REBUILD]
Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Zorro Lang <zlang@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
Filipe Manana [Thu, 16 Feb 2023 16:21:50 +0000 (16:21 +0000)]
generic/604: fix test to actually create dirty inodes
The test case generic/604 aims to test a scenario where at unmount time we
have many dirty inodes, however the test does not actually creates any
files, because it calls xfs_io without the -f argument, so xfs_io fails
but any error is ignored because stderr is redirected to /dev/null.
Fix this by passing -f to xfs_io and also stop redirecting stderr to
/dev/null, so that in case of any unexpected failure creating files, the
test fails.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com> Signed-off-by: Zorro Lang <zlang@kernel.org>
projects / xfstests-dev.git / log