Xiao Yang [Mon, 3 Jul 2017 07:35:59 +0000 (15:35 +0800)]
generic/446: add __xfs_get_blocks() into _filter_xfs_dmesg
On xfs filesystem, the following patch fixed system crash caused by
this race, but it introduced the __xfs_get_blocks() warning when the
race occurred: 04197b3 ("xfs: don't BUG() on mixed direct and mapped I/O")
On upstream kernel, the fix patch was cleared by: acdda3a ("xfs: use iomap_dio_rw")
When the fix patch was applied and not cleared(e.g, on RHEL7.4),
this case triggered the __xfs_get_blocks() warning as expected.
Moreover, generic/095 may reproduce the same warning occasionally.
So we could add __xfs_get_blocks() into _filter_xfs_dmesg.
Darrick J. Wong [Wed, 21 Jun 2017 21:58:22 +0000 (14:58 -0700)]
xfs: scrub while appending to a file
It turns out that there was a bug in xfs_bmap_count_blocks wherein
the block count returned would count or not count delalloc blocks
depending on the fork format. This is a bug that is easily exposed
via scrub, so check the output of that function here -- the buggy
version of the function produces online fsck errors.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Darrick J. Wong [Fri, 30 Jun 2017 04:12:39 +0000 (21:12 -0700)]
common/rc: test that the xfs_io scrub/repair commands actually work
When we call _require_xfs_io_command for the scrub ioctl, we have to
actually try calling the ioctl to make sure that the ioctl is
present on the running kernel.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Record the external log and realtime device configurations when we
create a sample filesystem. ext4 tightly binds to external logs,
so we have to preserve that too.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
overlay/031: add tests of whiteouts in lowerdir and both dirs
In overlay/031, it only cover the test case of whiteouts in
origined upper dir. This patch add two cases cover the other
two situations:
1) Lower origined dir have whiteouts;
2) Both upper and lower origined dirs have whiteouts (although
this case is pass now, still add this to cover all three kinds
of situations).
Darrick J. Wong [Wed, 21 Jun 2017 21:58:11 +0000 (14:58 -0700)]
xfs: test freeze/rmap repair race
The rmapbt repair code plays some dirty tricks with the fs freezer
to avoid running afoul of regular xfs locking requirements. Add a
test to check that filesystem write activities do not deadlock with
the repair program.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Xiao Yang [Thu, 29 Jun 2017 06:36:41 +0000 (14:36 +0800)]
generic: test a race between dio reads and mapped writes
This test reproduces a race between a direct I/O read and
a mapped write to a hole in a file. On xfs filesystem, it
will trigger a BUG_ON(), and this XFS bug has been fixed by:
04197b3 ("xfs: don't BUG() on mixed direct and mapped I/O")
[ eguan: umount before check dmesg log, and add rw group ]
Brian Foster [Wed, 28 Jun 2017 14:14:42 +0000 (10:14 -0400)]
generic/247: filter out expected XFS warnings for mixed mmap/direct I/O
generic/247 reproduces some of the same, expected warnings from XFS
as generic/095. These warnings occur due to mixed buffered/mapped
I/O racing with direct I/O to the same file.
generic/095 contains a custom dmesg filter to prevent test failure
in the event of such warnings. Lift the helper from generic/095 to
common/xfs and reuse it in generic/247 to implement the same
behavior.
Luis Henriques [Wed, 28 Jun 2017 11:20:51 +0000 (12:20 +0100)]
common/rc: handle the case when syscall isn't available
_require_xfs_io_command() isn't handling the case where the
requested syscall isn't available. To fix this simply check the
error returned by xfs_io.
Usually the userspace should be synced with kernelspace, the case
that a syscall is supported by userspace tool but not kernelspace
should not happen, but in rare test setups it's possible, e.g.
building an initramfs that contains the testing tools from a recent
distro, and running the tests against an old kernel (which does not
include such syscall).
[ eguan: it's not copy_file_range syscall specific issue, update
summary and commit log, and provide more background information ]
Lu Fengqi [Wed, 28 Jun 2017 05:45:31 +0000 (13:45 +0800)]
btrfs/027: reorder the arguments of btrfs replace
The option parser only accept options before non-option argument.
Although David Sterba has submitted commit df8c7225ba00 ("btrfs:
reorder arguments so that options come first"), but this case seems
to be left out.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Filipe Manana [Sun, 25 Jun 2017 23:13:39 +0000 (00:13 +0100)]
btrfs: test incremental send after replacing directory with a file
Test that an incremental send/receive operation works correctly after
moving some directory inode A, renaming a regular file inode B into the
old name of inode A and finally creating a new hard link for inode B at
directory inode A.
This issue is fixed by the following patch for the linux kernel:
"Btrfs: incremental send, fix invalid path for link commands"
src/seek_sanity_test: Fix for filesystems without unwritten extent support
src/seek_sanity_test assumes that after preallocating space in a
file with fallocate, fseek SEEK_HOLE / SEEK_DATA will still report
the allocated space as a hole. On filesystems without unwritten
extent support, that space will be reported as data, though. On
such filesystems, skip the unwritten extent tests.
Tested on ext4, xfs, and gfs2 + patches for fseek SEEK_HOLE /
SEEK_DATA support.
Both ext4 and xfs have a bug in the page cache scanning code for
SEEK_HOLE / SEEK_DATA in unwritten extents: the start offset isn't
taken into account when scanning a page, so seeking can fail on
filesystems with a block size less than half of the page size. For
example, the following command fails on a filesystem with a block
size of 1k:
seek_sanity_test: Report the actual allocation size
Instead of reporting the preferred block size for I/O as returned by
stat(2) as the allocation size, report the actual allocation size.
Clean things up a little in the process.
Darrick J. Wong [Wed, 21 Jun 2017 21:57:39 +0000 (14:57 -0700)]
xfs/040: use compare-libxfs in xfsprogs
xfsprogs now ships with a tool to compare its libxfs against a kernel
libxfs. Since the old srcdiff tool assumes dmapi.h (IRIX only) and the
pre-libxfs directory tree layout, fix the test and remove the old tool.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Eric Sandeen digged into it and find a good reproducer.
The problem comes when the several IO vectors are copied in, and
it runs into page faults, which stops the copy before all vectors
are copied. XFS sees this as a failed/short write, and so tries
to unmap the blocks & truncate away the pages in xfs_vm_write_end.
generic_perform_write is looping, and comes back around for the
other iovecs, but the page is still there, the buffer head is still
mapped, and so a new delalloc block isn't allocated - and ends up
being allocated at writeback time, despite the fact that we "should"
have accounted for it all at delalloc write time, and trips the
assert.
Jeff Layton [Fri, 16 Jun 2017 19:36:19 +0000 (15:36 -0400)]
btrfs: make a btrfs version of writeback error reporting test
For btrfs, we can test how it reports data writeback errors on fsync by
implementing a suggestion from Chris Mason:
Build a filesystem with 2 devices that stripes the data across
both devices, but mirrors metadata across both. Then, make one
of the devices fail and test what it does.
[eguan: add comments about creating btrfs with "-d raid0 -m raid1"]
Cc: Chris Mason <clm@fb.com> Signed-off-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Jeff Layton [Fri, 16 Jun 2017 19:36:17 +0000 (15:36 -0400)]
generic: add a writeback error handling test
I'm working on a set of kernel patches to change how writeback errors
are handled and reported in the kernel. Instead of reporting a
writeback error to only the first fsync caller on the file, it has
the the kernel report them once on every file description that was
open at the time of the error.
This patch adds a test for this new behavior. Basically, open many fds
to the same file, turn on dm_error, write to each of the fds, and then
fsync them all to ensure that they all get an error back.
To do that, I'm adding a new tools/dmerror script that the C program
can use to load the error table from the script. It's also suitable for
setting up, frobbing and tearing down a dmerror device for by-hand testing.
For now, only ext2/3/4 and xfs are whitelisted on this test, since those
filesystems are included in the initial patchset. We can add to that as
we convert filesystems, and eventually make it a more general test.
Jeff Layton [Fri, 16 Jun 2017 19:36:15 +0000 (15:36 -0400)]
ext4: allow ext4 to use $SCRATCH_LOGDEV
The writeback error handling test requires that you put the journal on a
separate device. This allows us to use dmerror to simulate data
writeback failure, without affecting the journal.
xfs already has infrastructure for this (a'la $SCRATCH_LOGDEV), so wire
up the ext4 code so that it can do the same thing when _scratch_mkfs is
called.
Signed-off-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Darrick J. Wong [Mon, 19 Jun 2017 20:24:02 +0000 (13:24 -0700)]
xfs: don't allow realtime swap files
Linux swapfiles use bmap which has no ability to tell the swap code
that the blocks its reporting aren't actually on the data device.
Therefore, swapon on a realtime file could corrupt random blocks on
the data device, so we must ensure that swapon cannot happen.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eryu Guan <eguan@redhat.com>
zhangyi (F) [Mon, 19 Jun 2017 03:53:39 +0000 (11:53 +0800)]
overlay: test whiteout check in impure dir
The unmerged and impure upper directories may contain invalid
whiteouts when we umount && modify lowerdir(e.g. clean up dir) and
remount overlay. This may lead to whiteouts exposure and rmdir
failure.
This case test impure dir's whiteouts check in ovl_iterate() and
ovl_remove_xxx().
Eric Biggers [Mon, 12 Jun 2017 21:15:28 +0000 (14:15 -0700)]
generic/397: be compatible with ignored SIGPIPE
If generic/397 is executed in an environment with SIGPIPE ignored,
it fails because the 'yes' program prints an error message:
yes: standard output: Broken pipe
yes: write error
This can be reproduced with:
trap '' SIGPIPE; ./check generic/397
Fix it by generating the string of 255 y's using just 'head' and
'tr' instead of 'yes', 'head', and 'tr'.
Although it's not really a good idea to execute xfstests with
SIGPIPE ignored, this is the only test I've noticed where it causes
a problem, so it might as well be fixed in the test.
It would be much nicer to prevent this problem for all tests by
making the 'check' script restore the default SIGPIPE handler. But
that isn't straightforward because bash's 'trap' builtin doesn't
allow un-ignoring signals that were ignored on entry to the shell.
[ eguan added more background infomation to commit log, which is
also from Eric.
I think it's an easy problem for others to run into, since sometimes
processes ignore SIGPIPE because they want to get write errors
instead, but then when doing fork() + exec() they forget to reset
the SIGPIPE handler. Notably, Python got this wrong and it wasn't
fixed until Python 3, so any programs executing the 'check' script
from a Python 2 script will usually get this wrong (see:
https://bugs.python.org/issue1652). And usually everything works
fine but every once in a while there is a weird problem like this
which has to be debugged. ]
Filipe Manana [Tue, 13 Jun 2017 15:06:39 +0000 (16:06 +0100)]
btrfs: incremental send after renaming a file and a directory
Test that an incremental send works if we rename some directory inode A
and then rename some file inode B to the name inode A had, for the case
where the directory inode A is an ancestor of inode B in the parent
snapshot.
This issue is fixed by the following patch for the linux kernel:
"Btrfs: incremental send, fix invalid path for unlink commands"
Jan Kara [Wed, 14 Jun 2017 07:19:20 +0000 (09:19 +0200)]
generic/360: Create symlink with valid path
A test is creating symlink with a path containing name 1023 characters
long. Such file name is invalid for most filesystems (the limit on file
name lenght is mostly 255 characters) and UDF actually complains about
this and so the test fails. Since the point of this test is to verify
storage of the symlink, change the test to use a valid path where each
component name has only 254 characters.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Eric Biggers [Fri, 9 Jun 2017 22:36:33 +0000 (15:36 -0700)]
generic: test for buggy fscrypt context consistency check
Add a regression test for a bug where ->lookup() in an encrypted
directory would incorrectly return EPERM, depending on which inodes
happened to have their keys still cached in memory following removal of
the keyring key. This bug was fixed in v4.12-rc1, v4.9.29, and v4.4.70.
Filipe Manana [Wed, 7 Jun 2017 15:36:31 +0000 (16:36 +0100)]
btrfs: test incremental send after renaming and linking file
Test that an incremental send operation works correctly when an inode A
is renamed, a new hard link added to it and some other inode B is renamed
to the old name of inode A.
The btrfs bug is fixed by the following patch for the linux kernel:
"Btrfs: send, fix invalid path after renaming and linking file"
fstests: Fix block device requirements and manual scratch mounts
Some tests require the scratch volume to be a block device,
although they don't do anything block device specific like
the device mapper tests. They actually only require a local
device and fail with network or overlay pseudo mount devices.
The same tests also try to mount the scratch device manually
after running _scratch_mkfs. For UBIFS, _scratch_mkfs simply
truncates the UBI volume and relies on the kernel to re-format
the volume on the next mount. This fails if the fs type is not
specified explicitly when mounting.
This patch replaces the block device requirement with a
generic check for a local device and adds a filesystem
specification to all manual mounts of the scratch device
to a mount point.
generic/398: Accept failing with EPERM in addition to ENOKEY for rename without key
Some filesystems (e.g. UBIFS) fail with EPERM when trying to move a
file into an encrypted directory via cross rename, without having
access to the encryption key.
Since rename is perfectly allowed to return EPERM, which is also
tested for, and no precise specification seems to exist that
clarifies when to expect EPERM and when ENOKEY, this patch modifies
the generic/398 test to accept both, for the test case where the key
is not available.
UBIFS is a filesystem for unmanaged flash memory devices. It works
on top of UBI (Unsorted Block Images) which is a wear leveling and
volume management layer on top of flash memory devices, which are
handled by the MTD subsystem (memory technology device).
Since the semantics of flash devices are drastically different from
regular block devices (blocks or "pages" must be erased before
writing, only larger groups of pages or "erase blocks" can be erased
at once, page write must be in order within an erase block, etc...)
it was decided to expose MTD devices as character devices with
ioctls for operations like erase.
Since erasing a flash erase block causes physical wear on the
device, eventually causing the erase blocks to go bad, the UBI layer
provides mainly transparent wear leveling on top of MTD devices. UBI
does not attempt to emulate a regular block device, but rather
something like a flash memory with idealized characteristics that
can be partitioned into multiple UBI volumes in a fashion somewhat
similar to LVM. UBI volumes are also exposed to user space as
character devices.
This patch mainly deals with some quirks of UBIFS like working on
top of character devices instead of block devices. Also UBIFS
automatically formats UBI devices when trying to mount an empty
device. The mkfs.ubifs program is mainly used for creating images.
This patch changes _scratch_mkfs and _scratch_mkfs_encrypted to
truncate the UBI volume instead, relying on the kernel to reformat
it on the next mount.
For _scratch_mkfs_encrypted this is actually required to get the
encryption tests to run, because mkfs.ubifs, at the time of writing
this, the kernel support for UBIFS encryption is fairly recent and
mkfs.ubifs does not have proper support yet.
The necessity of an additional -ubifs switch was discussed but auto
detection of UBIFS formated UBI devices could not be reproduced on
my end and is unlikely to work with empty UBI volumes anyway.
Omar Sandoval [Wed, 7 Jun 2017 06:57:10 +0000 (23:57 -0700)]
btrfs: test Btrfs delalloc accounting overflow
This is a regression test for patch "Btrfs: fix delalloc accounting
leak caused by u32 overflow". It creates a bunch of delalloc extents
and merges them together to make sure the accounting is done right.
Nikolay Borisov [Tue, 6 Jun 2017 13:37:22 +0000 (16:37 +0300)]
generic/019: Silence possible kill failure
When the filesystem is force-killed in generic/019 and fsstress is
compiled with DEBUG then it's possible for check_cwd() to fail. In
this case fsstress would just die and invoking kill on its pid would
produce an error to the output file:
+./tests/generic/019: line 157: kill: (32750) - No such process
Fix this possibility by ignoring the result of the output command,
this won't affect the test coverage.
Filipe Manana [Tue, 30 May 2017 04:52:50 +0000 (05:52 +0100)]
generic: hole punching followed by writes in the same range
Test that if we punch a hole in a file, with either a range that goes
beyond the file's size or covers a file range that is already a hole,
and that if after we do some buffered write operations that cover
different parts of the hole, no warnings are emmitted in syslog/dmesg
and the file's content is correct after remounting the filesystem.
This test is motivated by a bug in btrfs that is manifested in kernel
4.12-rc1 onwards (the bug existed long time ago but was not so easy
to expose before 4.12-rc1). The btrfs patch that fixes the issue is
titled: "Btrfs: fix invalid extent maps due to hole punching".
Nikolay Borisov [Tue, 30 May 2017 14:10:07 +0000 (17:10 +0300)]
src/listxattr: Fix reading past the end of the user buffer
listxattr reaturns a null-terminated list of entries that represent
the xattr names. However, if it is passed larger buffer than it
requires it won't zero-out the rest of the memory. The way the
loop iterator in listxattr.c is written makes it go print every
null-terminated entry up to bufsize (which is user passed parameter).
This can lead to a situation where listxattr users N bytes out of
M bytes big buffer ( M > N). This will leave the rest (M-N)
as garbage, which in turn will be printed by listxattr. Fix this
by converting the 'for' loop to 'while' and properly ensuring
we are reading at most howevermany elements the syscall reported
it returned
Eryu Guan [Tue, 16 May 2017 07:39:35 +0000 (15:39 +0800)]
build: workaround build failures with old autoconf version
Xiao Yang reported that fstests failed to build on RHEL6.9 hosts due
to old autoreconf didn't pass -I to aclocal -I. (This was fixed by
autoconf commit 44fbeef86d03 ("Pass autoreconf -I to aclocal -I"),
but not on RHEL6.9).
So call aclocal, autoheader and autoconf directly instead of
autoreconf, as what's done in xfsprogs Makefile.
Reported-by: Xiao Yang <yangx.jy@cn.fujitsu.com> Reviewed-by: Xiao Yang <yangx.jy@cn.fujitsu.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Eryu Guan [Tue, 16 May 2017 11:35:15 +0000 (19:35 +0800)]
nfs: test nfs4_getfacl near page size ACL from server
Test nfs4_getfacl gets ACL list correctly from server when the ACL
length is close enough to the end of a page. On buggy NFS client
getxattr could return ERANGE. Upstream commit ed92d8c137b7 ("NFSv4:
fix getacl ERANGE for some ACL buffer sizes") fixed this bug in 4.11
kernel.
Note that this reproducer was originally written by J. Bruce Fields.
Cc: J. Bruce Fields <bfields@redhat.com> Cc: Weston Andros Adamson <dros@primarydata.com> Cc: linux-nfs@vger.kernel.org Reviewed-by: J. Bruce Fields <bfields@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Theodore Ts'o [Thu, 25 May 2017 17:41:34 +0000 (13:41 -0400)]
build: Stop relying on OpenSSL
The OpenSSL dependency was added for one program, fssum, and it needs
it only because it needs a md5 implementation. Use Solar Designer's
openssl compatible implementation of md5 so we no longer need to
depend on OpenSSL.
Since the OpenSSL libraries are not always available, we had to add
extra complexity to test to see whether fssum exists.
The other problem with depending on the OpenSSL libraries is that
shared library compatibility situation is terrible; a fssum binary
built on a system using libssl1.0.0 is *NOT* run on a system with
libssl1.0.2, since the shared libraries are incompatible even across a
minor version bump. (Sigh.)
Nikolay Borisov [Thu, 25 May 2017 09:08:46 +0000 (12:08 +0300)]
generic/108: Fix return value check from _get_scsi_debug_dev
_get_scsi_debug_dev is supposed to return a "/dev/$device".
However, in case the scsi device is not mapped to a disk, hence
/dev/sd* doesn't exist, then get_scsi_debug_dev would return only
the "/dev/" string. In generic/108 we check whether return value is
"" and only then consider it a failure. This behavior allows the
test to erroneously consider _get_scsi_debug_dev succeeded even if
it returned a malformed string. Fix this by correctly checking
whether the return value is "/dev/"
Zorro Lang [Thu, 25 May 2017 08:38:38 +0000 (16:38 +0800)]
xfs/288: filter out extra mkfs warning
From xfsprogs v4.7, mkfs.xfs add respecification detection by
commit 9090e18. Then mkfs will fail and return if we run it
as below:
mkfs.xfs -m crc=1,finobt=1 -m crc=0 ....
Then _scratch_mkfs_xfs can deal with this problem. But for old
xfsprogs ( < v4.7), it replace the first "crc=1" with the second
"crc=0". Then "crc=0,finobt=1" cause a warning, but keep running:
"warning: finobt not supported without CRC support, disabled."
This extra warning breaks the golden image of xfs/288, so filter
it out in case.
Zorro Lang [Wed, 24 May 2017 14:52:58 +0000 (22:52 +0800)]
xfs/196: fallback to fail_writes for old kernel
linux XFS rename all "fail_writes" references to "drop_writes" in
v4.11. Some old kernel still use the name "fail_writes", e.g.
RHEL-7. For testing on old kernel, we need to fallback to
"fail_writes".
Filesystesm with the "default behavior" will always return the
offset of the end of the file when lseek'ing with SEEK_HOLE. This
test does the following:
Nikolay Borisov [Tue, 23 May 2017 14:16:40 +0000 (17:16 +0300)]
xfs/293: Make 'man' hard requirement
If xfs/293 is run on a system which doesn't have 'man' installed
it will hang the due to $CAT waiting for input indefinitely. Also
create an entry for $MAN_PROG and use the cached $MANPAGE instead
of repeatedy calling $MAN_PROG --page
Theodore Ts'o [Mon, 22 May 2017 00:29:12 +0000 (20:29 -0400)]
src: fix compiler warnings
Most of the fixes are printf format type warnings, but apparently GCC
6 is smart enough to realize is that if you don't do proper error
checking with posix_memalign, the resulting pointer can be undefined,
and whines about it. So while fixing this in aio-dio-fcntl-race, I
also cleaned up the error checking and reporting.
Eric Biggers [Thu, 18 May 2017 22:49:10 +0000 (15:49 -0700)]
fstests: skip AIO-related tests when CONFIG_AIO=n
When running xfstests on a kernel configured with CONFIG_AIO=n, all
AIO-related tests fail, often due to an error similar to the
following:
error Function not implemented during io_setup
This affected at least the following tests: generic/036,
generic/112, generic/113, generic/198, generic/207, generic/208,
generic/210, generic/211, generic/239, generic/323, generic/427,
xfs/240, xfs/241.
Fix this by enhancing the 'feature' program to allow testing for
asynchronous I/O support, then skipping all AIO-related tests when
AIO is unsupported.
This change is useful because CONFIG_AIO is sometimes disabled to
reduce the kernel's attack surface (e.g. see
https://android-review.googlesource.com/#/c/292158/).
Liu Bo [Wed, 17 May 2017 22:36:10 +0000 (16:36 -0600)]
btrfs: regression test for nocsum buffered read's repair
This is to test whether buffered read retry-repair code is able to
work in raid1 case as expected.
Please note that without checksum, btrfs doesn't know if the data
used to repair is correct, so repair is more of resync which makes
sure that both of the copy has the same content.
Commit 20a7db8ab3f2 ("btrfs: add dummy callback for
readpage_io_failed and drop checks") introduced the regression.
The upstream fix is commit 9d0d1c8b1c9d ("Btrfs: bring back repair
during read")
Signed-off-by: Liu Bo <bo.li.liu@oracle.com> Reviewed-by: Eryu Guan <eguan@redhat.com> Reviewed-by: Filipe Manana <fdmanana@gmail.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Zorro Lang [Wed, 17 May 2017 15:48:47 +0000 (23:48 +0800)]
fsstress: cleanup flist with test directory together
The "-c" option of fsstress will clean up the test directory after
each run. But it only does "rm -rf $dir". If run fsstress likes:
fsstress -d $test_dir -n 1000 -p 10 -l 0 -c
fsstress will remove all test directories at the end of each run,
but the flist still save those *deleted* entries. It'll cause
more and more useless ENOENT failures. So we need to release all
entries in flist too.
The above patches fix two related PMD vs PTE races in the DAX code.
These can both be easily triggered by having two threads reading and
writing simultaneously to the same private mapping, with the key
being that private mapping reads can be handled with PMDs but
private mapping writes are always handled with PTEs so that we can
COW.
Without this 2-patch kernel series, the newly added test will result
in the following errors:
run fstests generic/437 at 2017-05-16 16:53:43
mm/pgtable-generic.c:39: bad pmd ffff8808daa49b88(84000001006000a5)
... a bunch of the bad pmd messages ...
BUG: Bad rss-counter state mm:ffff8800a8c1b700 idx:1 val:1
BUG: non-zero nr_ptes on freeing mm: 38
XFS (pmem0p1): Unmounting Filesystem
Jan Kara [Wed, 17 May 2017 12:04:05 +0000 (14:04 +0200)]
generic: Add more SEEK_HOLE tests
Add tests for bugs found in ext4 & xfs SEEK_HOLE implementations
fixed by following patches:
xfs: Fix missed holes in SEEK_HOLE implementation
ext4: Fix SEEK_HOLE
We add tests to seek_sanity_test as it is easiest to reuse its
infrastructure for seek tests, however not to regress generic/285
which uses seek_sanity_test we don't run new tests by default.
Instead we add options to select a range of tests to run and run new
tests from this new test.
[eguan: add $tmp definition and cleanup $tmp.* on exit]
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Eryu Guan <eguan@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Eric Biggers [Tue, 16 May 2017 22:46:15 +0000 (15:46 -0700)]
generic: test that encrypted filenames are presented without collisions
Add a test which creates many similarly-named files in an encrypted
directory, then verifies they can be deleted without access to the
encryption key. This is a regression test for two related bugs which
caused presented names to "collide" and point to the wrong inodes.
These bugs were present in the original versions of ext4 and f2fs
encryption, and they were fixed in v4.12-rc1.
Xiao Yang [Wed, 17 May 2017 01:42:33 +0000 (09:42 +0800)]
common: cleanup _require_xfs_io_command
We don't need to check specific flags at the end of this function
if we have checked them before. e.g, generic/071 and generic/422
are marked as notrun unexpectedly because xfs_io doesn't support
long-format help for falloc before xfsprogs v4.9. Actually, xfs_io
has supported falloc, so these case should not be marked as notrun.
[eguan: declare local vars as local, rename param_check to
param_checked]
Xiao Yang [Wed, 17 May 2017 01:42:32 +0000 (09:42 +0800)]
common: use _require_xfs_io_command() directly to check fiemap
1) _require_fiemap and _require_xfs_io_command "fiemap" do the
same thing, but some test cases use the former and some use
the latter, so i feel they should be unified.
2) The number of helpers like this is slowly growing, but it's
easy to simply use _require_xfs_io_command directly and just
specify the command we want to check.
Amir Goldstein [Thu, 11 May 2017 06:55:09 +0000 (09:55 +0300)]
overlay/017: test consistent st_ino/d_ino for hardlinks
Currently hardlinks do not preserve the inode number across copy up,
so hardlinks did not participate in this test so far.
Stay honest and let the test verify what is was meant to verify and
let it fail because of the fact that hardlinks inode numbers are not
constant across copy up.
Amir Goldstein [Thu, 11 May 2017 06:55:08 +0000 (09:55 +0300)]
overlay/017: use t_dir_type to find file by d_ino
'find -ino' is this test was supposed to filter files by inode
number that was recorded with 'ls -i' to compare st_ino returned by
stat(2) with d_ino returned by getdents64(2).
It turns out that on some systems, 'find -ino' uses stat(2) for
filtering by inode number, which is not what we want.
Use the auxiliary program t_dir_type to filter files by inode number
instead.
Anna Schumaker [Wed, 10 May 2017 17:46:28 +0000 (13:46 -0400)]
generic: Add a copy test for invalid input
This test passes invalid argumnt combinations to the copy_file_range()
system call to test that input is verified before attempting to copy.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Anna Schumaker [Wed, 10 May 2017 17:46:27 +0000 (13:46 -0400)]
generic: Add a copy test for overwriting small amounts of data
This test is similar to the previous one, except that it copies one
byte at a time to make sure that this case works as expected.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Anna Schumaker [Wed, 10 May 2017 17:46:26 +0000 (13:46 -0400)]
generic: Add copy test that overwrites data
Using copy to overwrite data in the destination file is perfectly
valid, so let's make sure this case works as expected.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Anna Schumaker [Wed, 10 May 2017 17:46:25 +0000 (13:46 -0400)]
generic: Add small copies to new file test
This test copies single bytes from a source file into various new
files just to make sure that we can handle very small copies.
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Anna Schumaker [Wed, 10 May 2017 17:46:24 +0000 (13:46 -0400)]
generic: Add copy to new file test
This test copies data from various points in a source file to a new
file. This is useful for testing the basics of copy_file_range().
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
Luis Henriques [Mon, 8 May 2017 15:12:27 +0000 (16:12 +0100)]
src/seek_sanity_test: ensure file size is big enough
Tests test07, test08, and test09 preallocate a file and assume the
file size used is bigger than 10xbufsz (100xbufsz for test09). This
patch adjusts the file size so this assumption is always true.
As an example, here's test07 output for cephfs, where the allocation
size is set to 4194304, and the output is (4194304 * 10 + 4194304)
07. Test file with unwritten extents, only have dirty pages
07.01 SEEK_HOLE expected 0 or 4194304, got 46137344. FAIL
07.02 SEEK_HOLE expected 1 or 4194304, got 46137344. FAIL
Eric Biggers [Sat, 6 May 2017 00:19:33 +0000 (17:19 -0700)]
common/config: implement set_prog_path() using 'type -P'
Bash's 'type -P' builtin is equivalent to 'which', but it's more
efficient because it doesn't involve executing an external binary.
Because set_prog_path() is executed 60+ times in common/config,
which is sourced by common/rc, which in turn is sourced by every
test, switching to 'type -P' actually can make a noticeable
performance improvement for short-running or skipped tests. For
example:
Before:
# time ./check generic/002
...
Passed all 1 tests
real 0m1.365s
user 0m0.746s
sys 0m0.644s
After:
# time ./check generic/002
...
Passed all 1 tests
Eric Biggers [Sat, 6 May 2017 00:19:32 +0000 (17:19 -0700)]
common/config: make set_prog_path() accept one argument only
All callers of set_prog_path() pass it only one argument, the
program to find on the $PATH. Therefore, to simplify things remove
the unused code which allowed fallback paths to be specified in the
remaining arguments.
Eric Biggers [Thu, 4 May 2017 21:55:48 +0000 (14:55 -0700)]
generic: test revalidation of encrypted dentries
Add a test which verifies that dentries in an encrypted directory
are invalidated when an encryption key is added --- which should
cause the plaintext filenames to be visible and accessible,
replacing the encoded ciphertext filenames and any negative dentries
for the plaintext names. This primarily tests for a bug which was
fixed in the v4.5 kernel, plus a v4.6 fix for incorrect RCU usage in
the earlier fix.
Luis Henriques [Wed, 3 May 2017 10:54:13 +0000 (11:54 +0100)]
attr: add support for cephfs
Block size for cephfs is 4M, which makes generic/020 test fail as the
value for MAX_ATTRS and MAX_ATTRVAL_SIZE will be too high. Restrict these
two variables to sane values for this FSTYP.
Bill O'Donnell [Thu, 27 Apr 2017 18:31:10 +0000 (13:31 -0500)]
xfs: xfs_growfs target path must be an active xfs mountpoint
xfs_growfs manpage clearly states that the target path must be an
active xfs mountpoint. This is a test to ensure that if the target
path isn't an active xfs mountpoint, the command is rejected. The
purpose is to check the command response, but not necessarily the
functionality of xfs_growfs. Test cases include absolute paths,
relative paths, symbolic links, and bind mounts.
Zorro Lang [Wed, 26 Apr 2017 16:23:36 +0000 (00:23 +0800)]
generic: test eofblocks race with file extending aio dio writes
It's possible for post-eof blocks to end up being used for direct
I/O writes. dio write performs an upfront unwritten extent
allocation, sends the dio and then updates the inode size (if
necessary) on write completion. If a file release occurs while a
file extending dio write is in flight, it is possible to mistake the
post-eof blocks for speculative preallocation and incorrectly
truncate them from the inode. This means that the resulting dio
write completion can discover a hole and allocate new blocks rather
than perform unwritten extent conversion.
A kernel warning can be reproduced by generic/299 on XFS:
XFS: Assertion failed: tp->t_blk_res_used <= tp->t_blk_res, \
file: fs/xfs//xfs_trans.c, line: 309
The root cause is that xfs_free_eofblocks() uses i_size to truncate
post-eof blocks from the inode, but async, file extending direct
writes do not update i_size until write completion, long after inode
locks are dropped. Therefore, xfs_free_eofblocks() effectively
truncates the inode to the incorrect size.
Besides reproduce above kernel warning, the verification of written
data is an important distinction between this test and generic/299.
For cover this filesystem corruption testing, write this new case to
check data integrality manually, not only depend on a kernel
warning.
To increase the test stress of aio-dio-eof-race, add two arguments
to this source code to change the file size will be written.
Signed-off-by: Zorro Lang <zlang@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Eryu Guan <eguan@redhat.com>
- Request LIBTOOL to be used
- Set topbuildir based on a Makefile variable to call libtool
- Use /usr/local instead of /var for xfstest final location
- Move macros from aclocal.m4 to acinclude.m4, aclocal.m4 is autogenerated.
- Use autoconf variables @prefix@, @exec_prefix@.
The regular way of compiling xfstests - make - remains.
But it now runs autoreconf and libtoolize -i to produce a valid
configure.
Verified with 'make install --dry-run' that files are installed at the
same place.
Verified compiling in chromeOS chroot works as well.
[eguan: resolve merge conflicts and update .gitignore and remove
generated files by realclean]
Zorro Lang [Thu, 13 Apr 2017 07:31:09 +0000 (15:31 +0800)]
xfs: xfs_repair should junk empty attribute leaf blocks
There was a bug during log replay, the attr/attr3 leaf verifier
reported corruption when encountering a leaf attribute with a
count of 0 in the header, as below:
Metadata corruption detected at xfs_attr3_leaf block 0x480988/0x1000
commit f714016 from xfsprogs has fixed this bug. This test case
will emulate this corruption by xfs_db and use xfs_repair to fix
it.
Filipe Manana [Fri, 21 Apr 2017 15:00:52 +0000 (16:00 +0100)]
btrfs: fix local array declarations
We were declaring local arrays using a notation that does not seem to be
standard resulting in failures on some systems, like for example in a
Debian Stretch installation with bash version 4.4.11(1)-release:
btrfs/003 45s ... [failed, exit status 1] - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/003.out.bad)
--- tests/btrfs/003.out 2016-08-23 10:17:35.027012095 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/003.out.bad 2017-04-21 15:53:58.807366940 +0100
@@ -1,2 +1,4 @@
QA output created by 003
-Silence is golden
+./tests/btrfs/003: line 102: devs[]: bad array subscript
+dev balance failed
+(see /home/fdmanana/git/hub/xfstests/results//btrfs/003.full for details)
...
(Run 'diff -u tests/btrfs/003.out /home/fdmanana/git/hub/xfstests/results//btrfs/003.out.bad' to see the entire diff)
btrfs/027 7s ... [failed, exit status 1] - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/027.out.bad)
--- tests/btrfs/027.out 2016-08-23 10:17:35.035012077 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/027.out.bad 2017-04-21 15:53:59.835367271 +0100
@@ -1,2 +1,7 @@
QA output created by 027
Silence is golden
+./common/rc: line 935: devs[]: bad array subscript
+./common/rc: line 893: devs[]: bad array subscript
+mkfs -m raid1 -d raid1 failed
+Bug: str empty, must call _spare_dev_get before its put
+(see /home/fdmanana/git/hub/xfstests/results//btrfs/027.full for details)
...
(Run 'diff -u tests/btrfs/027.out /home/fdmanana/git/hub/xfstests/results//btrfs/027.out.bad' to see the entire diff)
Ran: btrfs/003 btrfs/027
Failures: btrfs/003 btrfs/027
Failed 2 of 2 tests
So fix this by changing the declaration pattern "local dev[]=..." to the
standard way of "local -a dev=...".
Xiao Yang [Fri, 21 Apr 2017 10:10:39 +0000 (18:10 +0800)]
ext4: check mount's handling for very large s_first_meta_bg
On ext4 filesystem, the kernel carshes at mount time when
s_first_meta_bg's value exceeds the largest possible meta_bg
number. This kernel bug has been fixed in:
3a4b77c ext4: validate s_first_meta_bg at mount time
projects / xfstests-dev.git / log