xfstests-dev.git
3 months agofstests: faster group file creation
Dave Chinner [Fri, 6 May 2022 05:10:17 +0000 (15:10 +1000)]
fstests: faster group file creation

We don't need to execute every test just to check it's groups are
valid. Just grab all the groups with grep, pull out the unique ones,
then check them.

This also avoids the problem of editor swap files being present in
the test directory and breaking the build because they are not
executable.

Building on a clean, already built tree so it only builds the
group lists:

$ time make
....
Building udf
 [GROUP] /home/dave/src/xfstests-dev/tests/udf/group.list
Building xfs
 [GROUP] /home/dave/src/xfstests-dev/tests/xfs/group.list

real    0m36.917s
user    0m15.032s
sys     0m26.219s
$

Patched:

$ time make
....
Building udf
 [GROUP] /home/dave/src/xfstests-dev/tests/udf/group.list
Building xfs
 [GROUP] /home/dave/src/xfstests-dev/tests/xfs/group.list
groups "frobnozzle" not mentioned in documentation.
gmake[3]: *** [../../include/buildgrouplist:8: group.list] Error 1
gmake[2]: *** [../include/buildrules:31: xfs] Error 2
gmake[1]: *** [include/buildrules:31: tests] Error 2
make: *** [Makefile:51: default] Error 2

real    0m1.751s
user    0m0.863s
sys     0m1.067s

$

Just a little bit faster, and as you can see that it still detects
groups that are not documented. There was also an open .001.swp file
in the XFS directory and that doesn't throw a failure anymore,
either.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Zorro Lang <zlang@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agocommon: print hints for reasons of test failures
Amir Goldstein [Tue, 19 Apr 2022 12:56:37 +0000 (15:56 +0300)]
common: print hints for reasons of test failures

Introduce helpers _fixed_by_{kernel,git}_commit() and
_fixed_in_{kernel_,}version() that can be used to hint testers why a
test might be failing and aid in auto-generating of expunge lists for
downstream kernel testing.

A test may be annotated with multiple hints, for example:

_fixed_by_kernel_commit 09889695864 xfs: foo
_fixed_by_kernel_commit 46464565465 ext4: bar
_fixed_in_version xfsprogs v5.15

Annotate fix kernel commits for some overlayfs tests.
Annotate fix kernel version for some overlayfs tests testing
for legacy behavior whose fixes are not likely to be backported
to stable kernels.

This is modeled after LTP's 'make filter-known-fails' and
print_failure_hints() using struct tst_tag annotations.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agocommon: support black listing fs in _supported_fs()
Amir Goldstein [Tue, 19 Apr 2022 12:56:36 +0000 (15:56 +0300)]
common: support black listing fs in _supported_fs()

For example:
_supported_fs ^xfs

There is no need to specify "generic" when using a block list
"all other fs are supported" is implied.

Converted some generic tests that open code this condition without
emitting a meaningful reason.

More generic test like generic/186,187 could use a block list, but
were not converted because they emit some meaningful reason:
  _notrun "Can't fragment free space on btrfs."

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agofstests: add some missing _require_loop's v2022.05.01
Josef Bacik [Thu, 28 Apr 2022 15:05:00 +0000 (11:05 -0400)]
fstests: add some missing _require_loop's

Got a new box running overnight fstests and noticed a couple of failures
because I forgot to enable loop device support.  Fix these two tests to
have _require_loop so they don't fail if there's no loop device support.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agogeneric/673: Add separate sgid stripping sub-tests
Yang Xu [Tue, 26 Apr 2022 08:55:45 +0000 (16:55 +0800)]
generic/673: Add separate sgid stripping sub-tests

Even kernel doesn't get ATTR_KILL_SGID mask and get ATTR_KILL_SUID mask,
we still can strip S_ISGID mode in setattr_prepare and setattr_copy.

We should check separate sgid stripping logic whether works well
on different filesystems.

Also fix comments error.

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agofsstress: remove ALLOCSP and FREESP operations entirely
Zorro Lang [Tue, 26 Apr 2022 06:24:11 +0000 (14:24 +0800)]
fsstress: remove ALLOCSP and FREESP operations entirely

Due to upstream linux has removed ALLOCSP/FREESP ioctls by commit:
4d1b97f9ce7c0 ("xfs: kill the XFS_IOC_{ALLOC,FREE}SP* ioctls"), so
let's remove ALLOCSP/FREESP testing from fsstress, to avoid more
mismatch problems.

Due to g/070 specified "-f allocsp" and "-f freesp=0", so remove
these two lines too.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agofstests: btrfs: test setting compression via xattr on nodatacow files
Chung-Chiang Cheng [Mon, 25 Apr 2022 04:22:26 +0000 (12:22 +0800)]
fstests: btrfs: test setting compression via xattr on nodatacow files

Compression and nodatacow are mutually exclusive. Besides ioctl, there
is another way to setting compression via xattrs, and shouldn't produce
invalid combinations.

Signed-off-by: Chung-Chiang Cheng <cccheng@synology.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agofstests: remove xfs/144
Eric Sandeen [Tue, 26 Apr 2022 05:14:53 +0000 (00:14 -0500)]
fstests: remove xfs/144

This test was designed to validate the quota warning limit, which in
theory was supposed to migrate from a soft quota to hard enforcement
after a certain number of warnings. However, the xfs kernel commit
which incremented the warning counter was reverted; see:

xfs: revert "xfs: actually bump warning counts when we send warnings"

in the kernel tree for an explanation of why. Due to that revert,
which removed this functionality, remove this test.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agogeneric/681, generic/682: add debugging information to $seqreq.full
Theodore Ts'o [Mon, 25 Apr 2022 21:14:50 +0000 (17:14 -0400)]
generic/681, generic/682: add debugging information to $seqreq.full

These two tests are checking whether a non-privileged user causing a
block allocation while expanding a directory block when over quota
will fail with an EDQUOT error.  There are three reasons why this can
fail.

* Aa test bug, where if the file system is using cluster allocation
  (for example, ext4 bigalloc) the test doesn't add enough directory
  entries to actually force directory grwoth.

* A file system bug, where the file system allocates blocks but for
  some reason isn't charging the space quota correctly (which
  currently seems to be the case in ext4 with fscrypt).

* A file system bug, where the file system is correctly charging the
  space quota to the unprivileged user, but isn't failing the system
  call with EDQUOT.

By adding some additional debugging information about whether
directory has grown or not (in addition to the existing repquota
output) to the the $seqres.full, it makes easier for the file system
developer to disambiguate between these possibilities.  It's cheap to
do this, and it could save developer time when trying to root cause
the failure.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agochown: fix ownership format string
Murphy Zhou [Mon, 25 Apr 2022 08:20:51 +0000 (16:20 +0800)]
chown: fix ownership format string

After coreutils rebasing to 9.1, chown(1) behavior changes:
"
   chown and chroot now warn about usages like "chown root.root f",
   which have the nonstandard and long-obsolete "." separator that
   causes problems on platforms where user names contain ".".
   Applications should use ":" instead of ".".
"

https://lwn.net/Articles/891574/

With this behavior change, old format of ownership string will cause
warning like this:
"
+chown: warning: '.' should be ':': '1000.1000'
+.chown: warning: '.' should be ':': '1100.1100'
+.chown: warning: '.' should be ':': '1200.1200'
+.chown: warning: '.' should be ':': '1300.1300'
+.chown: warning: '.' should be ':': '1400.1400'
"

The new format works fine with old versions of coreutils.

Signed-off-by: Murphy Zhou <jencce.kernel@gmail.com>
Reviewed-by: Zorro Lang <zlang@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agogeneric: add test for tmpfs POSIX ACLs
Christian Brauner [Wed, 20 Apr 2022 17:52:22 +0000 (19:52 +0200)]
generic: add test for tmpfs POSIX ACLs

Add a regression test for commit 705191b03d50 ("fs: fix acl translation").
This tests whether setting POSIX ACLs on a tmpfs mounted in a
non-initial user and mount namespace works as expected.

Note, once again the idmapped mount testsuite is grossly misnamed at
this point. It has morphed into a full-blown generic vfs feature
testsuite.

Cc: Eryu Guan <guaneryu@gmail.com>
Cc: Seth Forshee <sforshee@digitalocean.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Zorro Lang <zlang@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Reviewed-by: Zorro Lang <zlang@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agogeneric: ensure we drop suid after fallocate
Darrick J. Wong [Tue, 19 Apr 2022 17:32:21 +0000 (10:32 -0700)]
generic: ensure we drop suid after fallocate

fallocate changes file contents, so make sure that we drop privileges
and file capabilities after each fallocate operation.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agoxfs: make sure syncfs(2) passes back super_operations.sync_fs errors
Darrick J. Wong [Tue, 19 Apr 2022 17:32:15 +0000 (10:32 -0700)]
xfs: make sure syncfs(2) passes back super_operations.sync_fs errors

This is a regression test to make sure that nonzero error returns from
a filesystem's ->sync_fs implementation are actually passed back to
userspace when the call stack involves syncfs(2).

[zlang@ add '_supported_fs xfs' in case]

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agocephfs: add ceph-fuse suport for ceph-fuse
Rishabh Dave [Tue, 19 Apr 2022 17:58:11 +0000 (23:28 +0530)]
cephfs: add ceph-fuse suport for ceph-fuse

Currently tests in xfstests-dev can be executed against CephFS only by
mounting CephFS using kernel driver. Attempting to run tests against
CephFS using FUSE doesn't work because xfstests-dev would remount CephFS
using kernel. This patch adds the ability for xfstest-dev code to mount
CephFS using FUSE.

[Zorro add missed ";;" in common/rc]

Fixes: https://tracker.ceph.com/issues/55354
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agogeneric/019: fix incorrect unset statements
Darrick J. Wong [Tue, 19 Apr 2022 17:32:06 +0000 (10:32 -0700)]
generic/019: fix incorrect unset statements

Fix incorrect usage of unset -- one passes the name of the variable, not
the *value* contained within it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agoxfs/019: fix golden output for files created in setgid dir
Darrick J. Wong [Tue, 19 Apr 2022 17:32:00 +0000 (10:32 -0700)]
xfs/019: fix golden output for files created in setgid dir

A recent change to xfs/019 exposed a long-standing bug in mkfs where
it would always set the gid of a new child created in a setgid directory
to match the gid parent directory instead of what's in the protofile.

Ignoring the user's directions is not the correct behavior, so update
this test to reflect that.  Also don't erase the $seqres.full file,
because that makes forensic analysis pointlessly difficult.

Cc: Catherine Hoang <catherine.hoang@oracle.com>
Fixes: 7834a740 ("xfs/019: extend protofile test")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agoxfs: test xfsdump when an inode < root inode is present
Eric Sandeen [Mon, 18 Apr 2022 17:03:26 +0000 (01:03 +0800)]
xfs: test xfsdump when an inode < root inode is present

This tests a longstanding bug where xfsdumps are not properly
created when an inode is present on the filesytsem which has
a lower number than the root inode.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
3 months agofstests: remove _wipe_fs from dump testing entirely
Zorro Lang [Wed, 20 Apr 2022 16:07:56 +0000 (00:07 +0800)]
fstests: remove _wipe_fs from dump testing entirely

The _wipe_fs function in common/dump is a historical remnant of
xfstests, it's easy to cause confusion now. Now xfstests tend to
call `require_scratch && scratch_mkfs && scratch_mount` in each case
itself, we don't need to use a function to do that specially, so
remove _wipe_fs entirely.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
3 months agogeneric/633: Avoid failure without CONFIG_USER_NS
Jan Kara [Thu, 31 Mar 2022 11:19:20 +0000 (13:19 +0200)]
generic/633: Avoid failure without CONFIG_USER_NS

Some tests in idmapped_mounts fail without CONFIG_USER_NS because they
have implicit dependence on user namespaces and these tests are run
despite idmapped mount support not being detected. Detect whether at
least user namespaces are supported and skip tests needing them when
they are not.

Signed-off-by: Jan Kara <jack@suse.cz>
Tested-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agoidmapped_mounts: Prepare for support for more features
Jan Kara [Thu, 31 Mar 2022 11:19:19 +0000 (13:19 +0200)]
idmapped_mounts: Prepare for support for more features

Currently idmapped_mounts tests fail for kernels without CONFIG_USER_NS
because some tests are run despite missing support for idmapped mounts
and they implicitely require user namespace support. Prepare
idmapped_mounts to support more features a test may require to reliably
run.

Signed-off-by: Jan Kara <jack@suse.cz>
Tested-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Zorro Lang <zlang@kernel.org>
3 months agogeneric: test that renaming into a directory fails with EDQUOT
Darrick J. Wong [Mon, 11 Apr 2022 22:54:54 +0000 (15:54 -0700)]
generic: test that renaming into a directory fails with EDQUOT

Add a regression test to make sure that unprivileged userspace renaming
within a directory fails with EDQUOT when the directory quota limits have
been exceeded.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric: test that linking into a directory fails with EDQUOT
Darrick J. Wong [Mon, 11 Apr 2022 22:54:48 +0000 (15:54 -0700)]
generic: test that linking into a directory fails with EDQUOT

Add a regression test to make sure that unprivileged userspace linking
into a directory fails with EDQUOT when the directory quota limits have
been exceeded.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/019: kill background processes on interrupt
Dave Chinner [Mon, 11 Apr 2022 05:48:33 +0000 (15:48 +1000)]
generic/019: kill background processes on interrupt

If you ctrl-c generic/019, it leaves fsstress processes running.
Kill them in the cleanup function so that they don't have to be
manually killed after interrupting the test.

While touching the _cleanup() function, make it do everything that
the generic _cleanup function it overrides does and fix the
indenting.

[Eryu: unset fs_pid and fio_pid after wait]

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/216: handle larger log sizes
Darrick J. Wong [Fri, 15 Apr 2022 15:04:58 +0000 (08:04 -0700)]
xfs/216: handle larger log sizes

mkfs will soon refuse to format a log smaller than 64MB, so update this
test to reflect the new log sizing calculations.

[Eryu: add xfs/216.cfg and use _link_out_file_name to choose .out file]

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: test mkfs.xfs config file stack corruption issues
Darrick J. Wong [Mon, 11 Apr 2022 22:55:08 +0000 (15:55 -0700)]
xfs: test mkfs.xfs config file stack corruption issues

Add a new regression test for a stack corruption problem uncovered in
the mkfs config file parsing code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/rc: let xfs_scrub tell us about its unicode checker
Darrick J. Wong [Mon, 11 Apr 2022 22:55:02 +0000 (15:55 -0700)]
common/rc: let xfs_scrub tell us about its unicode checker

Now that xfs_scrub can report whether or not it was built with the
Unicode name checker, rewrite _check_xfs_scrub_does_unicode to take
advantage of that.  This supersedes the old method of trying to observe
dynamic library linkages and grepping the binary, neither of which
worked very well.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoceph/001: add extra check for remote object copies
Luís Henriques [Wed, 23 Mar 2022 16:09:25 +0000 (16:09 +0000)]
ceph/001: add extra check for remote object copies

Ceph kernel client now has a facility to check stats for certain
operations.  One of these operations is the 'copyfrom', the
operation that is used to offload to the OSDs the copy of objects
from, for example, the copy_file_range() syscall.

This patch changes ceph/001 to add an extra check to verify that the
copies performed by the test are _really_ remote copies and not
simple read+write operations.

Signed-off-by: Luís Henriques <lhenriques@suse.de>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/filter: extend _filter_xfs_io to match -nan
Eric Whitney [Thu, 14 Apr 2022 14:22:58 +0000 (10:22 -0400)]
common/filter: extend _filter_xfs_io to match -nan

When run on ext4 with sufficiently fast x86_64 hardware, generic/130
sometimes fails because xfs_io can report rate values as -nan:
0.000000 bytes, 0 ops; 0.0000 sec (-nan bytes/sec and -nan ops/sec)

_filter_xfs_io matches the strings 'inf' or 'nan', but not '-nan'.  In
that case it fails to convert the actual output to a normalized form
matching generic/130's golden output.  Extend the regular expression
used to match xfs_io's output to fix this.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/encrypt: use a sub-keyring within the session keyring
Eric Biggers [Thu, 14 Apr 2022 07:19:32 +0000 (00:19 -0700)]
common/encrypt: use a sub-keyring within the session keyring

Make the encryption tests create and use a named keyring "xfstests" in
the session keyring that the tests happen to be running under, rather
than replace the session keyring using 'keyctl new_session'.
Unfortunately, the latter doesn't work when the session keyring is owned
by a non-root user, which (depending on the Linux distro) can happen if
xfstests is run in a sudo "session" rather than in a real root session.

This isn't a great solution, as the lifetime of the keyring will no
longer be tied to the tests as it should be, but it should work.  The
alternative would be the weird hack of making the 'check' script
re-execute itself using something like 'keyctl session - $0 $@'.

Reported-by: Ritesh Harjani <ritesh.list@gmail.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
3 months agocommon/rc: Skip virtiofs when _require_exportfs
Liu Yiding [Thu, 14 Apr 2022 02:34:47 +0000 (10:34 +0800)]
common/rc: Skip virtiofs when _require_exportfs

open_by_handle_at(2) is not supported by virtio-fs.
Reference:
https://gitlab.com/virtio-fs/qemu/-/issues/10

Signed-off-by: Liu Yiding <liuyd.fnst@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/187: don't rely on FSCOUNTS for free space data
Darrick J. Wong [Mon, 11 Apr 2022 22:54:22 +0000 (15:54 -0700)]
xfs/187: don't rely on FSCOUNTS for free space data

Currently, this test relies on the XFS_IOC_FSCOUNTS ioctl to return
accurate free space information.  It doesn't.  Convert it to use statfs,
which uses the accurate versions of the percpu counters.  Obviously,
this only becomes a problem when we convert the free rtx count to use
(sloppier) percpu counters instead of the (more precise and previously
buggy) ondisk superblock counts.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs/507: add test to auto group
Darrick J. Wong [Mon, 11 Apr 2022 22:54:28 +0000 (15:54 -0700)]
xfs/507: add test to auto group

Add this regression test to the auto group now that it's been quite a
while since the fix patches went upstream.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/020: fix max_attrval_size output filter
David Disseldorp [Wed, 13 Apr 2022 16:44:22 +0000 (18:44 +0200)]
generic/020: fix max_attrval_size output filter

The current attr -g "$max_attrval_name" output filter is broken if
$max_attrval_size isn't 16-byte aligned, due to od's duplicate
suppression behaviour.
Fix it by having od only dump one byte per line.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/020: fix max_attrval_size for XFS, UDF, Btrfs and NFS
David Disseldorp [Wed, 13 Apr 2022 16:44:21 +0000 (18:44 +0200)]
generic/020: fix max_attrval_size for XFS, UDF, Btrfs and NFS

As found by Dave Chinner, fff4359d ("020: make this xattr test generic")
unintentionally changed the long attribute value length from 100K to 64
*bytes* for XFS, UDF and Btrfs.
Update XFS and UDF to use a 64K $max_attrval_size value. For Btrfs, use
the nodesize, xattr length and tree entry overhead sizes to calculate
the maximum.
NFS doesn't provide a way to find out the $max_attrval_size for the
underlying filesystem on the server, so just use a rough 1K limit.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/020: add and use _attr_get_max()
David Disseldorp [Wed, 13 Apr 2022 16:44:20 +0000 (18:44 +0200)]
generic/020: add and use _attr_get_max()

In preparation for taking into account the attr name length when
calculating $MAX_ATTRVAL_SIZE, put the current logic in a
_attr_get_max() helper function and set local $max_attrval_size /
$max_attrs variables instead of using export.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/020: move MAX_ATTRS and MAX_ATTRVAL_SIZE logic
David Disseldorp [Wed, 13 Apr 2022 16:44:19 +0000 (18:44 +0200)]
generic/020: move MAX_ATTRS and MAX_ATTRVAL_SIZE logic

No functional change. MAX_ATTRS and MAX_ATTRVAL_SIZE are only used
within generic/020, so move the logic for determining these values
over there.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/rc: fix _try_scratch_mount() and _test_mount() when mount fails
Filipe Manana [Mon, 11 Apr 2022 14:08:38 +0000 (15:08 +0100)]
common/rc: fix _try_scratch_mount() and _test_mount() when mount fails

After the recent commit 4a7b35d7a76cd9 ("common: allow to run all tests
on idmapped mounts"), some test that use _try_scratch_mount started to
fail. For example:

$ ./check btrfs/131 btrfs/220
FSTYP         -- btrfs
PLATFORM      -- Linux/x86_64 debian9 5.17.0-rc8-btrfs-next-114 (...)
MKFS_OPTIONS  -- /dev/sdc
MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1

btrfs/131 2s ... - output mismatch (see .../results//btrfs/131.out.bad)
    --- tests/btrfs/131.out 2020-06-10 19:29:03.818519162 +0100
    +++ /home/fdmanana/git/hub/xfstests/results//btrfs/131.out.bad (...)
    @@ -6,8 +6,6 @@
     Disabling free space cache and enabling free space tree
     free space tree is enabled
     Trying to mount without free space tree
    -mount failed
    -mount failed
     Mounting existing free space tree
     free space tree is enabled
    ...
    (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/131.out ...
btrfs/220 7s ... - output mismatch (see .../results//btrfs/220.out.bad)
    --- tests/btrfs/220.out 2020-10-16 23:13:46.802162554 +0100
    +++ /home/fdmanana/git/hub/xfstests/results//btrfs/220.out.bad (...)
    @@ -1,2 +1,32 @@
     QA output created by 220
    +Option fragment=invalid should fail to mount
    +umount: /home/fdmanana/btrfs-tests/scratch_1: not mounted.
    +Option nologreplay should fail to mount
    +umount: /home/fdmanana/btrfs-tests/scratch_1: not mounted.
    +Option norecovery should fail to mount
    +umount: /home/fdmanana/btrfs-tests/scratch_1: not mounted.
    ...
    (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/220.out ...
Ran: btrfs/131 btrfs/220
Failures: btrfs/131 btrfs/220
Failed 2 of 2 tests

The reason is that if _try_scratch_mount() fails to mount the filesystem,
we don't return the failure, instead we call _idmapped_mount(), which
can succeed and make _try_scratch_mount() return 0 (success). The same
happens for _test_mount(), however a quick search revealed no tests
currently relying on the return value of _test_mount().

So fix that by making _try_scratch_mount() return immediately if it gets
a mount failure. Also do the same for _test_mount().

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Acked-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agofstests: test dirty pipe vulnerability issue of CVE-2022-0847
Zorro Lang [Mon, 11 Apr 2022 16:37:10 +0000 (00:37 +0800)]
fstests: test dirty pipe vulnerability issue of CVE-2022-0847

Test for the Dirty Pipe vulnerability (CVE-2022-0847) caused by an
uninitialized  "pipe_buffer.flags" variable. The bug cause a file
can be overwritten even if a user/process is not permitted to write
it. It's fixed by 9d2231c5d74e ("lib/iov_iter: initialize "flags" in
new pipe_buffer").

Cc: Max Kellermann <max.kellermann@ionos.com>
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agofstests: replace hexdump with od command
Zorro Lang [Mon, 11 Apr 2022 16:37:09 +0000 (00:37 +0800)]
fstests: replace hexdump with od command

The "od" is one of the most fundamental commands in GNU/Linux and
most Unix-like systems. So we nearly always can count on it, don't
need to check if it's installed.

The "hexdump" isn't such fundamental as "od", some systems don't
install it by default. And as "od" nearly can replace all functions
of "hexdump", so let's use an unified command "od" to do the hexdump
job in fstests cases.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon: allow to run all tests on idmapped mounts
Christian Brauner [Thu, 7 Apr 2022 11:21:57 +0000 (13:21 +0200)]
common: allow to run all tests on idmapped mounts

In addition to the generic and filesystem-specific idmapped mount
testsuites that already exist upstream today add simple
infrastructure so any test can be run on idmapped mounts simply by
setting IDMAPPED_MOUNTS=true in the config file or section. The main
user for now will be overlay to verify it works correctly on
idmapped mounts.

Note that the infrastructure is completely generic so every
filesystem that supports idmapped mounts can simply run all of their
tests idmapped. But note that not all ways to create a mount have
been converted yet. That includes e.g. _dmthin_mount and direct
calls to _mount in various tests.

In addition, there will be corner-cases. For example, xfs doesn't
allow bulkstat on idmapped mounts because it is a filesystem wide
operation, i.e. you can retrieve information for any inode in the
filesystem so the operation cannot be scoped reasonably under a
single mount. So xfstests testing bulkstat will fail as it's
blocked. Similar for some btrfs ioctl()s.

While we could of course restrict this testmode to -overlay for
which we know things work correctly we should not do this. It would
mean that people won't start using it and so we won't see issues
unless someone sits down and goes through more than 1000 tests and
figures out for each individual one whether it needs to be skipped
or not.

So instead allow this mode but for all filesystems so people can
start running and reporting failures and we can fix them up or block
them as we detect them.

Cc: Eryu Guan <guaneryu@gmail.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: <fstests@vger.kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/encrypt: allow the use of 'fscrypt:' as key prefix
Luís Henriques [Tue, 5 Apr 2022 09:46:33 +0000 (10:46 +0100)]
common/encrypt: allow the use of 'fscrypt:' as key prefix

fscrypt keys have used the $FSTYP as prefix.  However this format is being
deprecated and newer kernels are expected to use the generic 'fscrypt:'
prefix instead.  This patch adds support for this new prefix, and only
uses $FSTYP on filesystems that didn't initially supported it, i.e. ext4 and
f2fs.  This will allow old kernels to be tested.

Signed-off-by: Luís Henriques <lhenriques@suse.de>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: drop no-op count=$(( count-- ))
David Disseldorp [Wed, 30 Mar 2022 11:45:40 +0000 (13:45 +0200)]
btrfs: drop no-op count=$(( count-- ))

This line doesn't result in a changed count value. The tests don't
appear to rely on a decremented value, so remove the line.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofsx: remove XFS_IOC_ALLOCSP operation entirely
Zorro Lang [Wed, 30 Mar 2022 05:25:06 +0000 (13:25 +0800)]
fsx: remove XFS_IOC_ALLOCSP operation entirely

Due to XFS_IOC_ALLOCSP is nearly retired from linux xfs, and the
ALLOCSP testing in fsx easily trigger errors. So compare with
disable it by default, I'd like to remove it entirely. So this patch
revert two patches:

cd99a499 ("fsx: disable allocsp_calls if -F is specified")
a32fb1bb ("fsx: add support for XFS_IOC_ALLOCSP")

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs/237: Use zone cap instead of zone size in fill_size and rest calculation
Pankaj Raghav [Fri, 25 Mar 2022 10:19:08 +0000 (11:19 +0100)]
btrfs/237: Use zone cap instead of zone size in fill_size and rest calculation

This test will break when zone capacity != zone size because the
calculation of the size to be filled is done using zone_size instead
of the actual capacity available per zone. Fix it by using zone
capacity.

As a zoned device can have variable capacity, use the btrfs utility
to get the zone capacity from the first data block group that the
test will be performed on.

The support to extract zone capacity was added to blkzone only from
version 2.37. So zcap will be used only when the blkzone version is
greater or equal to 2.37.

Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs/194: allow it to be run on systems with page size larger than 4K
Qu Wenruo [Tue, 8 Mar 2022 13:18:49 +0000 (21:18 +0800)]
btrfs/194: allow it to be run on systems with page size larger than 4K

Since btrfs has subpage support since v5.15, there is no need to
require such hard requirement on 4K page size.

We only need to check if the current system support 4K as
sectorsize.

Now the test case can pass on aarch64 with 64K page size.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/btrfs: don't skip the test if BTRFS_PROFILE_CONFIGS contains unsupported profile
Qu Wenruo [Tue, 8 Mar 2022 13:03:42 +0000 (21:03 +0800)]
common/btrfs: don't skip the test if BTRFS_PROFILE_CONFIGS contains unsupported profile

[BUG]
Sometimes the tester wants to use $BTRFS_PROFILE_CONFIGS to limit the
tests to certain profiles.

This is especially useful for btrfs subpage tests, as we don't yet
support RAID56 for it yet.

But unfortunately, if we specify $BTRFS_PROFILE_CONFIGS with the
following content:

 export BTRFS_PROFILE_CONFIGS="single:single dup:single raid0:raid0 \
       raid1:raid1 raid10:raid10"

A lot of tests will not run, like:

  btrfs/064 30s ... [not run] Profile dup not supported for replace
  btrfs/065 26s ... [not run] Profile dup not supported for replace
  btrfs/066 27s ...  14s
  btrfs/069 25s ... [not run] Profile dup not supported for replace
  btrfs/070 59s ... [not run] Profile dup not supported for replace
  btrfs/071 25s ... [not run] Profile dup not supported for replace

[CAUSE]
Those test cases uses _btrfs_get_profile_configs() to grab the profiles
which support given workload (like replace/repace-missing).

But _btrfs_get_profile_configs() will behave different based on whether
BTRFS_PROFILE_CONFIGS is defined.

If not defined, it goes with default profiles, and just skip those
unsupported.
This is what we want.

But if the environment variable is defined, it will not run if there is
any unsupported profile in it.

[FIX]
Unify the behaivor by always skip the unsupported profiles, no matter if
$BTRFS_PROFILE_CONFIGS is defined or not.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs/019: extend protofile test
Catherine Hoang [Thu, 17 Mar 2022 23:24:08 +0000 (23:24 +0000)]
xfs/019: extend protofile test

This test creates an xfs filesystem and verifies that the filesystem
matches what is specified by the protofile.

This patch extends the current test to check that a protofile can
specify setgid mode on directories. Also, check that the created
symlink isn’t broken.

Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs/029: change the cross vfsmount reflink test
Josef Bacik [Thu, 24 Mar 2022 20:24:34 +0000 (16:24 -0400)]
btrfs/029: change the cross vfsmount reflink test

We now allow cross vfsmount reflinks, change this test to make sure
we pass the cross-vfsmount reflink.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/374: validate cross-vfsmount dedupe
Josef Bacik [Thu, 24 Mar 2022 20:24:33 +0000 (16:24 -0400)]
generic/374: validate cross-vfsmount dedupe

We allow for cross-vfsmount dedupes now, change this test to
validate dedupe works properly cross-vfsmount.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/373: change test to validate cross-vfsmount reflink
Josef Bacik [Thu, 24 Mar 2022 20:24:32 +0000 (16:24 -0400)]
generic/373: change test to validate cross-vfsmount reflink

We now allow cross-vfsmount reflinks so change the test to validate
that cross-vfsmount reflinks work.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/674: replace _require_scratch_reflink with _require_scratch_dedupe
Boyang Xue [Wed, 30 Mar 2022 06:03:39 +0000 (14:03 +0800)]
generic/674: replace _require_scratch_reflink with _require_scratch_dedupe

Signed-off-by: Boyang Xue <bxue@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/066: attr1 is still there after log replay on f2fs
Sun Ke [Wed, 23 Mar 2022 08:37:24 +0000 (16:37 +0800)]
generic/066: attr1 is still there after log replay on f2fs

The test fail on f2fs:
     xattr names and values after second fsync log replay:
     # file: SCRATCH_MNT/foobar
    +user.attr1="val1"
     user.attr3="val3"

attr1 is still there after log replay. f2fs doesn't support fs-op level
transaction functionality. so it have no way to persist all metadata
updates in one transaction.  We can use "fastboot" mountoption for this
case, so during last fsync on qwerty, f2fs can trigger a checkpoint
which will persist all metadata updates before fsync().

Suggested-by: Chao Yu <chao@kernel.org>
Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/631: use trusted.* extended attributes to filter NFS filesystem
Dai Shili [Fri, 25 Mar 2022 20:04:12 +0000 (16:04 -0400)]
generic/631: use trusted.* extended attributes to filter NFS filesystem

Since commit 2461e8901c ("generic/631: Add a check for extended
attributes"), check of user.* extended attributes was added to avoid
running this tests for filesystems that are not supported.

But it still fails on NFSv4.2 when mount overlayfs with the
following error.

/var/mnt/scratch/merged0: wrong fs type, bad option, bad superblock
on overlay, missing codepage or helper program, or other error.

dmesg log reports the following -
overlayfs: upper fs does not support xattr, falling back to index=off and metacopy=off.

As per the linux patch series(https://lwn.net/Articles/799185/ ), the
support of user extended attributes were added to NFSv4.2.  So this
case passed the check of extended attributes by _require_attrs.

As per the overlayfs documentation -

"The upper filesystem will normally be writable and if it is it must
support the creation of trusted.* and/or user.* extended attributes,
and must provide valid d_type in readdir responses, so NFS is not
suitable."

So we can replace user.* extended attributes check with trusted.* to
filter NFS filesystem.

Signed-off-by: Dai Shili <daisl.fnst@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/108: use blockdev utility to calculate physical and logical block size
Nitesh Shetty [Wed, 23 Mar 2022 18:06:43 +0000 (23:36 +0530)]
generic/108: use blockdev utility to calculate physical and logical block size

Reading sysfs to get physical and logical block fails,
if SCRATCH_DEV is partitioned device.

Signed-off-by: Nitesh Shetty <nj.shetty@samsung.com>
Reported-by: Darrick J. Wong <djwong@kernel.org>
Reported-by: Wang Yugui <wangyugui@e16-tech.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: fallocate against a file range with a mix of holes and extents
Filipe Manana [Thu, 17 Mar 2022 16:34:32 +0000 (16:34 +0000)]
generic: fallocate against a file range with a mix of holes and extents

Test that if we call fallocate against a file range that has a mix
of holes and written extents, the fallocate succeeds if the
filesystem has enough free space to allocate extents for the holes.

This test currently fails on btrfs and is fixed by a patch that has the
following subject:

    "btrfs: only reserve the needed data space amount during fallocate"

The test also fails on xfs, and after some discussion with Darrick,
it seems it's due to technical reasons that would require a
significant effort to xfs's implementation, and at the moment there
isn't enough motivation to do such change. The relevent thread is at:

    https://lore.kernel.org/linux-btrfs/20220315164011.GF8241@magnolia/

Therefore the test is intentionally skipped on xfs only. Ext4 and
f2fs pass this test.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/encrypt: verify the key identifiers
Eric Biggers [Sun, 13 Mar 2022 01:05:59 +0000 (17:05 -0800)]
common/encrypt: verify the key identifiers

As part of all the ciphertext verification tests, verify that the
filesystem correctly computed the key identifier from the key the test
generated.  This uses fscrypt-crypt-util to compute the key identifier.

Previously this was only being tested indirectly, via the tests that
happen to use the hardcoded $TEST_RAW_KEY and $TEST_KEY_IDENTIFIER.
The new check provides better coverage.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/encrypt: log full ciphertext verification params
Eric Biggers [Sun, 13 Mar 2022 01:05:58 +0000 (17:05 -0800)]
common/encrypt: log full ciphertext verification params

To help with debugging, log some additional information to $seqres.full.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofscrypt-crypt-util: add support for dumping key identifier
Eric Biggers [Sun, 13 Mar 2022 01:05:57 +0000 (17:05 -0800)]
fscrypt-crypt-util: add support for dumping key identifier

Add an option to fscrypt-crypt-util to make it compute the key
identifier for the given key.  This will allow testing the correctness
of the filesystem's key identifier computation.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofscrypt-crypt-util: refactor get_key_and_iv()
Eric Biggers [Sun, 13 Mar 2022 01:05:56 +0000 (17:05 -0800)]
fscrypt-crypt-util: refactor get_key_and_iv()

Split get_key_and_iv() into two distinct parts: (1) deriving the key and
(2) generating the IV.  Also, check for the presence of needed options
just before they are used rather than doing it all up-front.

These changes should make this code much easier to understand.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofscrypt-crypt-util: use an explicit --direct-key option
Eric Biggers [Sun, 13 Mar 2022 01:05:55 +0000 (17:05 -0800)]
fscrypt-crypt-util: use an explicit --direct-key option

Make fscrypt-crypt-util use an option --direct-key to specify the use of
the DIRECT_KEY method for key derivation and IV generation.  Previously,
this method was implicitly detected via --mode-num being given without
either --iv-ino-lblk-64 or --iv-ino-lblk-32, or --kdf=none being given
in combination with --file-nonce.

The benefit of this change is that it makes the various options to
fscrypt-crypt-util behave more consistently.  --direct-key,
--iv-ino-lblk-64, and --iv-ino-lblk-32 now all work similarly (they
select a key derivation and IV generation method); likewise, --mode-num,
--file-nonce, --inode-number, and --fs-uuid now all work similarly (they
provide information that key derivation and IV generation may need).

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs/17[013]: fix intermittent failures when filesystem metadata gets large
Darrick J. Wong [Wed, 16 Mar 2022 22:13:26 +0000 (15:13 -0700)]
xfs/17[013]: fix intermittent failures when filesystem metadata gets large

These tests check that the filestreams allocator never shares an AG
across multiple streams when there's plenty of space in the filesystem.
Recent increases in metadata overhead for newer features (e.g. bigger
logs due to reflink) can throw this off, so add another AG to the
formatted filesystem to encourage it to avoid the AG with the log.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs/076: only create files on the data device
Darrick J. Wong [Wed, 16 Mar 2022 22:12:32 +0000 (15:12 -0700)]
xfs/076: only create files on the data device

This test checks that filesystems with sparse inode support can continue
to allocate inodes when free space gets fragmented.  Inodes only exist
on the data device, so we need to ensure that realtime is not enabled on
the filesystem so that the rt metadata doesn't mess with the inode usage
percentage and cause a test failure.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/673: fix golden output to reflect vfs setgid behavior
Darrick J. Wong [Wed, 16 Mar 2022 03:30:25 +0000 (20:30 -0700)]
generic/673: fix golden output to reflect vfs setgid behavior

Filipe Manana pointed out[1] that the setgid dropping behavior encoded
in this generic test is based on some outdated XFS code, and not based
on what the VFS inode attribute change functions actually do.  Now that
we're working on fixing that, we should update the golden output to
reflect what all filesystems are supposed to be doing.

[1] https://lore.kernel.org/linux-xfs/CAL3q7H47iNQ=Wmk83WcGB-KBJVOEtR9+qGczzCeXJ9Y2KCV25Q@mail.gmail.com/

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs/420: fix occasional test failures due to pagecache readahead
Darrick J. Wong [Wed, 16 Mar 2022 03:30:20 +0000 (20:30 -0700)]
xfs/420: fix occasional test failures due to pagecache readahead

Every now and then, this test fails with this golden output:

  --- xfs/420.out
  +++ xfs/420.out.bad
  @@ -29,7 +29,7 @@
   Whence Result
   DATA   0
   HOLE   131072
  -DATA   196608
  +DATA   192512
   HOLE   262144
   Compare files
   c2803804acc9936eef8aab42c119bfac  SCRATCH_MNT/test-420/file1

Curiously, the file checksums always match, and it's not *forbidden* for
the page cache to have a page backing an unwritten extent that hasn't
been written.

The condition that this test cares about is that block 3 (192k-256k) are
reported by SEEK_DATA as data even if the data fork has a hole and the
COW fork has an unwritten extent.  Matthew Wilcox thinks this is a side
effect of readahead.

To fix this occasional false failure, call SEEK_DATA and SEEK_HOLE only
on the offsets that we care about.

Suggested-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/xfs: fix broken code in _check_xfs_filesystem
Darrick J. Wong [Wed, 16 Mar 2022 03:30:14 +0000 (20:30 -0700)]
common/xfs: fix broken code in _check_xfs_filesystem

Fix some problems with undefined variables in the scrub control code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/459: ensure that the lvm devices have been created
Darrick J. Wong [Wed, 16 Mar 2022 03:30:09 +0000 (20:30 -0700)]
generic/459: ensure that the lvm devices have been created

Once in a /very/ long while this test fails because _mkfs_dev can't find
the LVM thinp volume after it's been created.  Since the "solution" du
jour seems to be to sprinkle udevadm settle calls everywhere, do that
here in the hopes that will fix it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: read multiple extents with io_uring
Filipe Manana [Thu, 3 Mar 2022 11:36:42 +0000 (11:36 +0000)]
generic: read multiple extents with io_uring

Test doing a read, with io_uring, over a file range that includes
multiple extents. The read operation triggers page faults when
accessing all pages of the read buffer except for the pages
corresponding to the first extent.  We want to check that the
operation results in reading all the extents and that it returns the
correct data.

This is motivated by an issue found with MariaDB when using io_uring
and running on btrfs. There's a patch for btrfs to address the issue

ca93e44bfb5f ("btrfs: fallback to blocking mode when doing async dio
over multiple extents")

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs: test xfsdump with bind-mounted filesystem
Masayoshi Mizuma [Wed, 16 Feb 2022 14:55:20 +0000 (09:55 -0500)]
xfs: test xfsdump with bind-mounted filesystem

commit 25195eb ("xfsdump: handle bind mount target") introduced
a bug of xfsdump which doesn't store the files to the dump file
correctly when the root inode number is changed.

The commit 25195eb is reverted, and commit 0717c1c ("xfsdump: intercept
bind mount targets") which is in xfsdump v3.1.10 fixes the bug to reject
the filesystem if it's bind-mounted.

Test that xfsdump can reject the bind-mounted filesystem.

Signed-off-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoext4: Test to ensure resize with sparse_super2 is handled correctly
Ojaswin Mujoo [Tue, 22 Feb 2022 17:50:53 +0000 (23:20 +0530)]
ext4: Test to ensure resize with sparse_super2 is handled correctly

Kernel currently doesn't support resize of EXT4 mounted with
sparse_super2 option enabled. Earlier, it used to leave the resize
incomplete and the fs would be left in an inconsistent state,
however commit b1489186cc83[1] fixed this to avoid the fs corruption
by clearly returning -EOPNOTSUPP.

Test to ensure that kernel handles resizing with sparse_super2
correctly. Run resize for multiple iterations because this sometimes
leads to kernel crash due to fs corruption, which we want to detect.

Related commit in mainline:

[1] commit b1489186cc8391e0c1e342f9fbc3eedf6b944c61

ext4: add check to prevent attempting to resize an fs with sparse_super2

Signed-off-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agosrc/ext4_resize.c: Refactor code and ensure accurate errno is returned
Ojaswin Mujoo [Tue, 22 Feb 2022 17:50:52 +0000 (23:20 +0530)]
src/ext4_resize.c: Refactor code and ensure accurate errno is returned

The current implementation of ext4_resize returned 1 whenever
there was an error. Modify this to return the correct error code.
This is important for tests that rely on correct error reporting, by
the kernel, for ext4 resize functionality.

Additionaly, perform some code cleanup.

Signed-off-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Reviewed-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoREADME: restructure & format building manual
Andrey Albershteyn [Mon, 21 Mar 2022 09:19:21 +0000 (10:19 +0100)]
README: restructure & format building manual

The commands for package managers for both Ubuntu and RHEL weren't
up-to-date. A few packages doesn't exist anymore or missing from the
latest repositories (e.g. liburing-devel, btrfs-progs-devel).
Ubuntu's list missed a few packages listed in RHEL's list. There are
a few repeating steps. The indent of avaliable environment variables
is not clear.

This patch:
- Update package dependencies for Ubuntu/Debian/RHEL/CentOS/Fedora
- Unify list of packages between Ubuntu/Debian and RHEL/CentOS/Fedora
- Add list of tool packages for other FS
- Add links to tools not in the standard repository for RHEL/CentOS
- Drop xfsprogs-qa-devel reference for old systems
- Replace note about EPEL with installation step with link to Fedora
  manual
- Add configuration examples in 'Setup Environment'
- Small clarification details, such as size of the partition and
  KCONFIG_PATH description
- Removal of repeating steps (install administrative tools,
  partitions content)
- Restructuring and formatting of "BUILDING THE FSQA SUITE" section
- Drop IRIXDEV references in variable set up
- Variables in 'Additional setup' divided into groups
- Move fsqa user/group creation and udf_test to 'Setup Environment'
  section

Signed-off-by: Andrey Albershteyn <aalbersh@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: test that autdodefrag does not rewrite single extents
Qu Wenruo [Tue, 15 Mar 2022 12:00:20 +0000 (20:00 +0800)]
btrfs: test that autdodefrag does not rewrite single extents

There is a report that btrfs autodefrag is defragging extents which
only have one single sector.

Such defragging will not reduce the number of extents, but only
waste IO.

The fix for it is titled:

  btrfs: avoid defragging extents whose next extents are not targets

Here we add a test case, which will create an inode with the
following layout:

  0                16K   20K                  64K
  |<-- Extent A -->|<-B->|<----- Extent C --->|
  |Gen 7           |Gen 9|Gen 7               |

And we trigger autodefrag with newer_than = 8, which means it will
only defrag extents newer than or equal to generation 8.

Currently only Extent B meets the condition, but it can not be
merged with Extent A nor Extent C, as they don't meet the generation
requirement.

Unpatched kernel will defrag only Extent B, resulting no change in
fragmentation, while costs extra IO.

Patched kernel will not defrag anything.

Although this is still not the ideal case, as we can defrag the
whole 64K range, but that's not what autodefrag can do with its
generation limitation.

And such "perfect" defrag can cause way more IO than some users can
stand.

At least we should not only defrag extent B.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/rc: avoid mixed mode for zoned btrfs
Shin'ichiro Kawasaki [Tue, 22 Feb 2022 04:19:08 +0000 (13:19 +0900)]
common/rc: avoid mixed mode for zoned btrfs

When the helper function _scratch_mkfs_sized is called with small
filesystem size and FSTYP=btrfs, it calls mkfs.btrfs with --mixed option
to enable mixed mode. However, mkfs.btrfs with --mixed option fails for
zoned block devices since btrfs does not support mixed mode together
with zoned mode. To avoid this failure, do not set --mixed option when
the scratch device is a zoned block device.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: move test case btrfs/261 into the generic group
Filipe Manana [Thu, 3 Mar 2022 11:40:21 +0000 (11:40 +0000)]
generic: move test case btrfs/261 into the generic group

The test case btrfs/261, part from its comments, doesn't really exercise
any behaviour that is btrfs specific, so, as Dave Chinner pointed out, it
can be moved into the generic group.

This change moves that test into the generic group and slightly adjust the
comments to make it clear which parts are btrfs specific.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: add test for enable/disable quota and create/destroy qgroup repeatedly
Sidong Yang [Wed, 2 Mar 2022 14:05:48 +0000 (14:05 +0000)]
btrfs: add test for enable/disable quota and create/destroy qgroup repeatedly

Test enable/disable quota and create/destroy qgroup repeatedly in
parallel and confirm it does not cause kernel hang. It only happens
in kernel staring with kernel 5.17-rc3. This is a regression test
for the problem reported to linux-btrfs list [1].

The hang was recreated using the test case and fixed by kernel patch
titled

  btrfs: qgroup: fix deadlock between rescan worker and remove qgroup

[1] https://lore.kernel.org/linux-btrfs/20220228014340.21309-1-realwakka@gmail.com/

Signed-off-by: Sidong Yang <realwakka@gmail.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/108: use sysfs values for logical,physical block size in scsi_debug
Nitesh Shetty [Tue, 1 Mar 2022 21:29:47 +0000 (02:59 +0530)]
generic/108: use sysfs values for logical,physical block size in scsi_debug

scsi_debug device used for test, is created with assumption of 512
bytes logical and physical block size.

This causes error in lvcreate step, when SCRATCH_DEV device lba is
not 512 bytes. This can be solved by reading block size from sysfs
of device.  If sysfs is missing fallback to 512 bytes as default.

Signed-off-by: Nitesh Shetty <nj.shetty@samsung.com>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: Add test for seekdir
Jan Kara [Tue, 22 Feb 2022 10:36:58 +0000 (11:36 +0100)]
generic: Add test for seekdir

Add test checking functionality of seekdir. We check whether seekdir
gets us back to the directory entry it should and also whether
seeking to random positions does not crash the filesystem.

Unlike test generic/310 which also tests seeking, this test checks both
glibc readdir() function as well as getdents64() syscall directly. This
is because glibc readdir() implementation does a lot of caching and
processing internally thus hiding kernel from some possible problems.
Also test wider range of random offsets to have better chance of
hitting out of bound accesses or other bugs.

This is a regression test for a48fc69fe658 ("udf: Fix crash after
seekdir")

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: add support for capturing metadumps of corrupted fses
Anthony Iliopoulos [Wed, 16 Feb 2022 19:03:34 +0000 (20:03 +0100)]
btrfs: add support for capturing metadumps of corrupted fses

Add the capability to capture btrfs metadumps when filesystem checks
fail, so that they can be used for further debugging. This is useful
for tests that _require_test and/or _require_scratch for which
filesystem checkers will run after a test completes and may
occasionally pick up inconsistencies.

Signed-off-by: Anthony Iliopoulos <ailiop@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: log replay after fsync of file with prealloc extents beyond eof
Filipe Manana [Thu, 17 Feb 2022 12:14:21 +0000 (12:14 +0000)]
btrfs: log replay after fsync of file with prealloc extents beyond eof

Test that after a full fsync of a file with preallocated extents
beyond the file's size, if a power failure happens, the preallocated
extents still exist after we mount the filesystem.

This test currently fails and there is a patch for btrfs that fixes
this issue and has the following subject:

  "btrfs: fix lost prealloc extents beyond eof after full fsync"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon: factor out xfs unique part from _filter_mkfs
Shin'ichiro Kawasaki [Fri, 18 Feb 2022 07:31:56 +0000 (16:31 +0900)]
common: factor out xfs unique part from _filter_mkfs

Most of the code in the function _filter_mkfs is xfs unique. This is
misleading that the function would be dedicated for xfs. Clean up the
function by factoring out xfs unique part to _xfs_filter_mkfs in
common/xfs. While at the same time, fix indent in _xfs_filter_mkfs to be
consistent with other functions in common/xfs.

Suggested-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/015: check _scratch_mkfs_sized return code
Shin'ichiro Kawasaki [Fri, 18 Feb 2022 07:31:55 +0000 (16:31 +0900)]
xfs/015: check _scratch_mkfs_sized return code

The test cases xfs/015 calls _scratch_mkfs before _scratch_mkfs_sized,
and does not check return code of _scratch_mkfs_sized. Even if
_scratch_mkfs_sized failed, _scratch_mount after it cannot detect the
sized mkfs failure because _scratch_mkfs already created a file system
on the device. This results in unexpected test condition.

To avoid the unexpected test condition, check return code of
_scratch_mkfs_sized.

Suggested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4/021: check _scratch_mkfs_sized return code
Shin'ichiro Kawasaki [Fri, 18 Feb 2022 07:31:54 +0000 (16:31 +0900)]
ext4/021: check _scratch_mkfs_sized return code

The test cases ext4/021 calls _scratch_mkfs before _scratch_mkfs_sized,
and does not check return code of _scratch_mkfs_sized. Even if
_scratch_mkfs_sized failed, _scratch_mount after it cannot detect the
sized mkfs failure because _scratch_mkfs already created a file system
on the device. This results in unexpected test condition.

To avoid the unexpected test condition, check return code of
_scratch_mkfs_sized.

Suggested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric/{171,172,173,174}: check _scratch_mkfs_sized return code
Shin'ichiro Kawasaki [Fri, 18 Feb 2022 07:31:53 +0000 (16:31 +0900)]
generic/{171,172,173,174}: check _scratch_mkfs_sized return code

The test cases generic/{171,172,173,174} call _scratch_mkfs before
_scratch_mkfs_sized, and they do not check return code of
_scratch_mkfs_sized. Even if _scratch_mkfs_sized failed, _scratch_mount
after it cannot detect the sized mkfs failure because _scratch_mkfs
already created a file system on the device. This results in unexpected
test condition of the test cases.

To avoid the unexpected test condition, check return code of
_scratch_mkfs_sized in the test cases.

Suggested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric/204: remove unnecessary _scratch_mkfs call
Shin'ichiro Kawasaki [Fri, 18 Feb 2022 07:31:52 +0000 (16:31 +0900)]
generic/204: remove unnecessary _scratch_mkfs call

The test case generic/204 calls _scratch_mkfs to get data block size and
i-node size of the filesystem and obtained data block size is passed to
the following _scratch_mfks_sized call as an option. However, the
_scratch_mkfs call is unnecessary since the sizes can be obtained by
_scratch_mkfs_sized call without the data block size option.

Also the _scratch_mkfs call is harmful when the _scratch_mkfs succeeds
and the _scratch_mkfs_sized fails. In this case, the _scratch_mkfs
leaves valid working filesystem on scratch device then following mount
and IO operations can not detect the failure of _scratch_mkfs_sized.
This results in the test case run with unexpected test condition.

Hence, remove the _scratch_mkfs call and the data block size option for
_scratch_mkfs_sized call.

Suggested-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agotests/btrfs: use $AWK_PROG instead of awk
Luis Chamberlain [Tue, 15 Feb 2022 01:48:45 +0000 (17:48 -0800)]
tests/btrfs: use $AWK_PROG instead of awk

Use $AWK_PROG instead of awk in case the default awk
program is not called awk.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Adam Manzanares <a.manzanares@samsung.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/btrfs: use $AWK_PROG
Luis Chamberlain [Tue, 15 Feb 2022 01:48:44 +0000 (17:48 -0800)]
common/btrfs: use $AWK_PROG

Use $AWK_PROG instead of just awk in case the preferred awk
program is different.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Adam Manzanares <a.manzanares@samsung.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/237: adjust uuid heuristics
Luis Chamberlain [Tue, 15 Feb 2022 01:48:43 +0000 (17:48 -0800)]
btrfs/237: adjust uuid heuristics

Using $(findmnt -n -o UUID mount-directory) only works on
util-linux v2.38, not on 2.37. Even debian-testing has util-linux
v2.37, so use a mechanism which has worked for longer to fetch the
UUID.

Without this the test fails on permission failure on accessing
the file /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold as $uuid is
empty. So while at it, add a check to ensure the UUID is never
empty.

Use $AWK_PROG in case the preferred awk program is not the
preferred awk program.

Cc: Karel Zak <kzak@redhat.com>
Cc: Pankaj Raghav <p.raghav@samsung.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Tested-by: Adam Manzanares <a.manzanares@samsung.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/config: fix use of MODPROBE_RM_PATIENT_TIMEOUT_ARGS
Luis Chamberlain [Tue, 8 Feb 2022 21:37:54 +0000 (13:37 -0800)]
common/config: fix use of MODPROBE_RM_PATIENT_TIMEOUT_ARGS

When patient module removal support is detected on kmod
(note that this is not yet even merged onto kmod yet but
work in progress) the arguments required for it are not being
set yet. This was a typo, fix this.

This issue was spotted using shellcheck from blktests when adding
patient module removal support there. I'll post a patch next to
let folks evaluate if we should embrace shellcheck on fstests as
well.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/attr: adbjust acl_max of f2fs
Sun Ke [Tue, 8 Feb 2022 07:16:24 +0000 (15:16 +0800)]
common/attr: adbjust acl_max of f2fs

f2fs has set inline_xattr as a default option, and introduced a new
option named 'noinline_xattr' for disabling default inline_xattr
option.  So in _acl_get_max we need to check 'noinline_xattr' string
in fs option, otherwise we may select the wrong max acl number since
we always found the string 'inline_xattr' in fs option.

Additionally, f2fs has changed disk layout of xattr block a bit, so
will contain one more entry in both inline and noinline xattr inode,
this patch will modify the max acl number to adjust it.

Suggested-by: Chao Yu <chao@kernel.org>
Signed-off-by: Sun Ke <sunke32@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agosrc/fssum: use newer EVP_* interface to replace deprecated MD5_* interace
Qu Wenruo [Wed, 2 Feb 2022 04:29:29 +0000 (12:29 +0800)]
src/fssum: use newer EVP_* interface to replace deprecated MD5_* interace

In OpenSSL 3.0, MD_Init/Update/Final() interfaces are marked
deprecated, and we have to go EVP_DigestInit/Update/Final() instead.

Personally I'm not a fan of this, especially the new EVP_MD_CTX
structure can no longer be stack allocated, thus we have to
dynamically allocate and free EVP_MD_CTX in sum_init() and sum_free().

Hopes this is proper way to go and would solve the problem until
OpenSSL chooses to change their interface again.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agobtrfs: verify "btrfs filesystem defragment -c" behavior
Qu Wenruo [Wed, 2 Feb 2022 11:15:08 +0000 (19:15 +0800)]
btrfs: verify "btrfs filesystem defragment -c" behavior

Despite the regular file defragging, "btrfs filesystem defragment"
provides an option, -c, to convert all data extents (except holes
and preallocated ranges) to a new compression algorithm.

The special behavior here is, unlike regular defrag which is not
going to touch extents which are adjacent to preallocated/hole
ranges, with -c, all non-hole/non-preallocated extents should be
defragged and converted to the new compression algorithm.

This test case will ensure the old behavior is properly kept.

Currently both old kernels (v5.15 and older) and newer kernel with
refactored defrag (v5.16 and newer) can pass the tests.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agobtrfs: defrag with compressed extents
Qu Wenruo [Fri, 28 Jan 2022 00:27:01 +0000 (08:27 +0800)]
btrfs: defrag with compressed extents

There is a long existing bug in btrfs defrag code that it will
always try to defrag compressed extents, even they are already at
max capacity.

This will not reduce the number of extents, but only waste IO/CPU.

The kernel fix is titled:

  btrfs: defrag: don't defrag extents which is already at its max capacity

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agobtrfs: autodefrag with regular and hole extents
Qu Wenruo [Fri, 28 Jan 2022 00:27:00 +0000 (08:27 +0800)]
btrfs: autodefrag with regular and hole extents

In v5.11~v5.15 kernels, there is a regression in autodefrag that if
a cluster (up to 256K in size) has even a single hole, the whole
cluster will be rejected.

This will greatly reduce the efficiency of autodefrag.

The behavior is fixed in v5.16 by a full rework, although the rework
itself has other problems, it at least solves the problem.

Here we add a test case to reproduce the case, where we have a 128K
cluster, the first half is fragmented extents which can be
defragged.  The second half is hole.

Make sure autodefrag can defrag the 64K part.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agobtrfs: defrag with regular and preallocated extents
Qu Wenruo [Fri, 28 Jan 2022 00:26:59 +0000 (08:26 +0800)]
btrfs: defrag with regular and preallocated extents

Recent v5.16 has some regression around btrfs autodefrag mount
option, and the extra scrutiny around defrag code exposes some
questionable behavior from the old code.

One behavior is to defrag extents along with the next preallocated
extent.

This behavior will cause extra IO and convert all the preallocated
extent to regular zero filled extents, rendering the preallocated
extent useless.

The kernel fix is titled:

  btrfs: defrag: don't try to merge regular extents with preallocated extents

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agofsx: disable allocsp_calls if -F is specified
Zorro Lang [Sat, 29 Jan 2022 04:44:31 +0000 (12:44 +0800)]
fsx: disable allocsp_calls if -F is specified

As the fsx.c source code says:
  int     fallocate_calls = 1;            /* -F flag disables */
  int     allocsp_calls = 1;              /* -F flag disables */

The allocsp_calls and fallocate_calls should be disabled, if the -F
option is specified. But current fsx forgets to disable allocsp_calls
as is documented.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agocommon/rc: set maximum label length for ext4
Lukas Czerner [Tue, 23 Nov 2021 10:11:19 +0000 (11:11 +0100)]
common/rc: set maximum label length for ext4

Set maximum label length for ext4 in _label_get_max() to be able to test
online file system label set/get ioctls.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agofstests: add more tests to the metadump group
Dave Chinner [Wed, 2 Feb 2022 07:38:31 +0000 (18:38 +1100)]
fstests: add more tests to the metadump group

Because they use metadump and I want to test metadump more easily.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agocommon: handle old versions of modprobe which don't support --help
Theodore Ts'o [Tue, 1 Feb 2022 23:19:36 +0000 (18:19 -0500)]
common: handle old versions of modprobe which don't support --help

There exists versions of modprobe out there which are so old that
modprobe --help isn't a thing.  They're certainly not going to support
modprobe --remove-patiently, so test to make sure modprobe --help
works to avoid causing all tests to fail due to the error message
to stderr showing up in $seq.out.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fixes: 405c21d40aa1 ("common/module: add patient module rmmod support")
Reported-by: Leah Rumancik <leah.rumancik@gmail.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agofstests: skip tests that require XFS_IOC_ALLOCSP
Darrick J. Wong [Wed, 26 Jan 2022 02:11:54 +0000 (18:11 -0800)]
fstests: skip tests that require XFS_IOC_ALLOCSP

Deprecating this, so turn off the tests that require it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agogeneric: test suid/sgid behavior with reflink and dedupe
Darrick J. Wong [Thu, 27 Jan 2022 01:27:01 +0000 (17:27 -0800)]
generic: test suid/sgid behavior with reflink and dedupe

Make sure that we drop the setuid and setgid bits any time reflink or
dedupe change the file contents.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>