xfstests-dev.git
3 months agotools: add missing license tags to my scripts
Darrick J. Wong [Fri, 17 Sep 2021 00:39:41 +0000 (17:39 -0700)]
tools: add missing license tags to my scripts

I forgot to add spdx license tags and copyright statements to some of
the tools that I've contributed to fstests.  Fix this to be explicit.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agotools: make sure that test groups are described in the documentation
Darrick J. Wong [Fri, 17 Sep 2021 00:39:35 +0000 (17:39 -0700)]
tools: make sure that test groups are described in the documentation

Create a file to document the purpose of each test group that is
currently defined in fstests, and change mkgroupfile to check that every
group mentioned in the tests is also mentioned in the documentation.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric/631: change this test to use the 'whiteout' group
Darrick J. Wong [Fri, 17 Sep 2021 00:39:30 +0000 (17:39 -0700)]
generic/631: change this test to use the 'whiteout' group

This test isn't really an overlay test; it's a regression test for a bug
that someone found in xfs handling of whiteout files.  Since the
'overlay' group has one member, let's move it to 'whiteout'.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agobtrfs: fix incorrect subvolume test group name
Darrick J. Wong [Fri, 17 Sep 2021 00:39:24 +0000 (17:39 -0700)]
btrfs: fix incorrect subvolume test group name

The group for testing subvolume functionality is 'subvol', not
'subvolume'.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: fix incorrect fuzz test group name
Darrick J. Wong [Fri, 17 Sep 2021 00:39:19 +0000 (17:39 -0700)]
xfs: fix incorrect fuzz test group name

The group name for fuzz tests is 'fuzzers'.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: move reflink tests into the clone group
Darrick J. Wong [Fri, 17 Sep 2021 00:39:13 +0000 (17:39 -0700)]
xfs: move reflink tests into the clone group

"clone" is the group for tests that exercise FICLONERANGE, so move these
tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoceph: re-tag copy_file_range as being in the copy_range group
Darrick J. Wong [Fri, 17 Sep 2021 00:39:08 +0000 (17:39 -0700)]
ceph: re-tag copy_file_range as being in the copy_range group

copy_range is the group name for copy_file_range tests, so reclassify
these tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agogeneric: fsstress with cpu offlining
Darrick J. Wong [Wed, 15 Sep 2021 23:42:34 +0000 (16:42 -0700)]
generic: fsstress with cpu offlining

Exercise filesystem operations when we're taking CPUs online and offline
throughout the test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: test adding realtime sections to filesystem
Darrick J. Wong [Wed, 15 Sep 2021 23:42:25 +0000 (16:42 -0700)]
xfs: test adding realtime sections to filesystem

Add a functional test to exercise using "xfs_growfs -e XXX -r" to add a
realtime section to a filesystem while changing the extent size.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: test correct propagation of rt extent size hints on rtinherit dirs
Darrick J. Wong [Wed, 15 Sep 2021 23:42:20 +0000 (16:42 -0700)]
xfs: test correct propagation of rt extent size hints on rtinherit dirs

This is a regression test for the following fixes:

 xfs: standardize extent size hint validation
 xfs: validate extsz hints against rt extent size when rtinherit is set
 mkfs: validate rt extent size hint when rtinherit is set

These patches fix inadequate rtextsize alignment validation of extent
size hints on directories with the rtinherit and extszinherit flags set.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agoxfs: test fsx with extent size hints set on a realtime file
Darrick J. Wong [Wed, 15 Sep 2021 23:42:14 +0000 (16:42 -0700)]
xfs: test fsx with extent size hints set on a realtime file

This is a regression test for the two realtime allocator bug fixes:

xfs: adjust rt allocation minlen when extszhint > rtextsize
xfs: retry allocations when locality-based search fails

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/rc: use directio mode for the loop device when possible
Darrick J. Wong [Fri, 17 Sep 2021 00:48:29 +0000 (17:48 -0700)]
common/rc: use directio mode for the loop device when possible

Recently, I've been observing very high runtimes of tests that
format a filesystem atop a loop device and write enough data to fill
memory, such as generic/590 and generic/361.  Logging into the test
VMs, I noticed that the writes to the file on the upper filesystem
started fast, but soon slowed down to about 500KB/s and stayed that
way for nearly 20 minutes.  Looking through the D-state processes on
the system revealed:

/proc/4350/comm = xfs_io
/proc/4350/stack : [<0>] balance_dirty_pages+0x332/0xda0
[<0>] balance_dirty_pages_ratelimited+0x304/0x400
[<0>] iomap_file_buffered_write+0x1ab/0x260
[<0>] xfs_file_buffered_write+0xba/0x330 [xfs]
[<0>] new_sync_write+0x119/0x1a0
[<0>] vfs_write+0x274/0x310
[<0>] __x64_sys_pwrite64+0x89/0xc0
[<0>] do_syscall_64+0x35/0x80
[<0>] entry_SYSCALL_64_after_hwframe+0x44/0xae

Here's the xfs_io process performing a buffered write to the file on the
upper filesystem, which at this point has dirtied enough pages to be
ratelimited.

/proc/28/comm = u10:0+flush-8:80
/proc/28/stack : [<0>] blk_mq_get_tag+0x11c/0x280
[<0>] __blk_mq_alloc_request+0xce/0xf0
[<0>] blk_mq_submit_bio+0x139/0x5b0
[<0>] submit_bio_noacct+0x3ba/0x430
[<0>] iomap_submit_ioend+0x4b/0x70
[<0>] xfs_vm_writepages+0x86/0x170 [xfs]
[<0>] do_writepages+0xcc/0x200
[<0>] __writeback_single_inode+0x3d/0x300
[<0>] writeback_sb_inodes+0x207/0x4a0
[<0>] __writeback_inodes_wb+0x4c/0xe0
[<0>] wb_writeback+0x1da/0x2c0
[<0>] wb_workfn+0x2ad/0x4f0
[<0>] process_one_work+0x1e2/0x3d0
[<0>] worker_thread+0x53/0x3c0
[<0>] kthread+0x149/0x170
[<0>] ret_from_fork+0x1f/0x30

This is a flusher thread that has invoked writeback on the upper
filesystem to try to clean memory pages.

/proc/89/comm = u10:7+loop0
/proc/89/stack : [<0>] balance_dirty_pages+0x332/0xda0
[<0>] balance_dirty_pages_ratelimited+0x304/0x400
[<0>] iomap_file_buffered_write+0x1ab/0x260
[<0>] xfs_file_buffered_write+0xba/0x330 [xfs]
[<0>] do_iter_readv_writev+0x14f/0x1a0
[<0>] do_iter_write+0x7b/0x1c0
[<0>] lo_write_bvec+0x62/0x1c0
[<0>] loop_process_work+0x3a4/0xba0
[<0>] process_one_work+0x1e2/0x3d0
[<0>] worker_thread+0x53/0x3c0
[<0>] kthread+0x149/0x170
[<0>] ret_from_fork+0x1f/0x30

Here's the loop device worker handling the writeback IO submitted by the
flusher thread.  Unfortunately, the loop device is using buffered write
mode, which means that /writeback/ is dirtying pages and being throttled
for that.  This is stupid.

Fix this by trying to enable "directio" mode on the loop device, which
delivers two performance benefits: setting directio mode also enables
async io mode, which will allow multiple IOs at once; and using directio
nearly eliminates the chance that writeback will get throttled.

On the author's system with fast storage, this reduces the runtime of
g/590 from 20 minutes to 12 seconds, and g/361 from ~30s to ~3s.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/rc: re-fix detection of device-mapper/persistent memory incompatibility
Darrick J. Wong [Wed, 15 Sep 2021 23:42:06 +0000 (16:42 -0700)]
common/rc: re-fix detection of device-mapper/persistent memory incompatibility

In commit e05491b3, I tried to resolve false test failures that were
a result of device mapper refusing to change access modes on a block
device that supports the FSDAX access mode.  Unfortunately, I did
not realize that there are two ways that fsdax support can be
detected via sysfs: /sys/block/XXX/queue/dax and /sys/block/XXX/dax/,
so I only added a test for the latter.

As of 5.15-rc1 this doesn't seem to work anymore for some reason.  I
don't know enough about the byzantine world of pmem device driver
initialization, but fsdax mode actually does work even though the
/sys/block/XXX/dax/ path went away.  So clearly we have to detect it
via the other sysfs path.

Fixes: e05491b3 ("common/rc: fix detection of device-mapper/persistent memory incompatibility")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agobtrfs/179: fix test failure when there are no snapshots to delete
Filipe Manana [Fri, 24 Sep 2021 14:06:32 +0000 (15:06 +0100)]
btrfs/179: fix test failure when there are no snapshots to delete

It's very rare, but we can end up in a situation where there are no
snapshots to delete, in which case the $victim variable of the function
delete_workload() ends up being assigned with an empty string. When
that happens we end up running the command:

   btrfs subvolume delete "$SCRATCH_MNT/snapshots/"

Which fails since the argument is not a subvolume or a snapshot.
This causes the test to fail due to an unexpected error message from
the subvolume delete command:

  btrfs/179 129s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/179.out.bad)
      --- tests/btrfs/179.out 2020-10-16 23:13:46.546152332 +0100
      +++ /home/fdmanana/git/hub/xfstests/results//btrfs/179.out.bad 2021-09-24 11:15:01.404863801 +0100
      @@ -1,2 +1,3 @@
       QA output created by 179
      +ERROR: Not a Btrfs subvolume: Invalid argument
       Silence is golden
      ...

Fix that by making the delete_workload() loop skip the deletion attempt
when there are no snapshots.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agofsstress: fix bogus compile warning do_renameat2()
Luis Chamberlain [Tue, 21 Sep 2021 17:50:59 +0000 (10:50 -0700)]
fsstress: fix bogus compile warning do_renameat2()

gcc complains with:

fsstress.c:4629:4: warning: 'oldparid' may be used uninitialized in this function [-Wmaybe-uninitialized]
 4629 |    printf("%d/%d: rename source entry: id=%d,parent=%d\n",
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 4630 |     procid, opno, oldid, oldparid);
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
fsstress.c:4629:4: warning: 'oldid' may be used uninitialized in this function [-Wmaybe-uninitialized]

But the varaibles are sure to be initialized, it is just that the
heuristics are broken since another check is used later which confuses
gcc. So just initialize the variables, to shup the compile warning.

Cc: kaixuxia <kaixuxia@tencent.com>
Cc: Brian Foster <bfoster@redhat.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
3 months agocommon/module: use patient module removal
Luis Chamberlain [Tue, 21 Sep 2021 17:49:52 +0000 (10:49 -0700)]
common/module: use patient module removal

Now that scsi_debug has been using the patient module removal
for a while, let's generalize its use for the other use cases.
This likey will fix some odd false positives due to races.

Suggested-by: Eryu Guan <guan@eryu.me>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs: regresion test for fsmap problems with realtime
Darrick J. Wong [Wed, 1 Sep 2021 00:12:26 +0000 (17:12 -0700)]
xfs: regresion test for fsmap problems with realtime

This is a regression test for:

c02f6529864a ("xfs: make xfs_rtalloc_query_range input parameters const")
9ab72f222774 ("xfs: fix off-by-one error when the last rt extent is in use")
7e1826e05ba6 ("xfs: make fsmap backend function key parameters const")

In which we synthesize an XFS with a realtime volume and a special
realtime volume to trip the bugs fixed by all three patches that
resulted in incomplete fsmap output.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs/108: sync filesystem before querying quota
Darrick J. Wong [Wed, 1 Sep 2021 00:12:20 +0000 (17:12 -0700)]
xfs/108: sync filesystem before querying quota

The new deferred inactivation code is lazy about deallocating
deleted files, which means that we need to be more proactive about
syncing the filesystem after deleting things.  When reporting quotas,
XFS only flushes the deferred work if we query quota id 0, so we
need the explicit sync to ensure the quota numbers are not affected
by laziness.

[Eryu: replace the global sync with xfs_io syncfs command]

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs: test DONTCACHE behavior with the inode cache
Darrick J. Wong [Wed, 1 Sep 2021 00:12:15 +0000 (17:12 -0700)]
xfs: test DONTCACHE behavior with the inode cache

Basic testing that DONTCACHE affects the XFS inode cache in the manner
that we expect.  The only way we can do that (for XFS, anyway) is to
play around with the BULKSTAT ioctl.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: regression test for a FALLOC_FL_UNSHARE bug in XFS
Darrick J. Wong [Wed, 1 Sep 2021 00:12:10 +0000 (17:12 -0700)]
generic: regression test for a FALLOC_FL_UNSHARE bug in XFS

This is a regression test for commit 72a048c1056a ("xfs: only set
IOMAP_F_SHARED when providing a srcmap to a write").

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agooverlay: fix documented kernel commit ids
Amir Goldstein [Sun, 12 Sep 2021 09:34:18 +0000 (12:34 +0300)]
overlay: fix documented kernel commit ids

Fix documented commit ids for test overlay/078 following rebase of
overlayfs-next branch before merge to v5.15-rc1.

Document an additional kernel fix commit id for test overlay/077.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs/146: require scratch dev and pool before _require_dm_target
Shin'ichiro Kawasaki [Wed, 8 Sep 2021 08:37:14 +0000 (17:37 +0900)]
btrfs/146: require scratch dev and pool before _require_dm_target

The test case btrfs/146 calls _require_dm_target which depends on
SCRATCH_DEV. The test case assumes that valid devices are set in
SCRATCH_DEV_POOL, and one of the devices is propagated to
SCRATCH_DEV.  However, when SCRATCH_DEV_POOL is not set, valid value
is not propagated to SCRATCH_DEV and _require_dm_target causes
unexpected test case failure.

To avoid the failure, move _require_scratch and
_require_scratch_dev_pool calls before _require_dm_target call to
detect invalid SCRATCH_DEV_POOL beforehand.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/{628,629}: fix order of _require_scratch* and _require_dm_target
Shin'ichiro Kawasaki [Wed, 8 Sep 2021 08:37:13 +0000 (17:37 +0900)]
generic/{628,629}: fix order of _require_scratch* and _require_dm_target

Test cases with dm_target shall call _require_scratch* before
_require_dm_target to ensure that valid SCRATCH_DEV is available for
dm_target. However, the test cases generic/628 and generic/629 call
_require_dm_target before _require_scratch*, then unexpected failure
was reported when SCRATCH_DEV is not specified. Fix the order of
function calls.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/361: require sparse file support and fix mounting the volume.
Ari Sundholm [Fri, 10 Sep 2021 11:53:55 +0000 (14:53 +0300)]
generic/361: require sparse file support and fix mounting the volume.

The test case generic/361 fails on multiple filesystem
implementations due to two issues:

1. A file is created on a 512 MB volume by truncating it to
   1 GB. For this to work, the filesystem implementation must
   support sparse files. However, the test case does not check
   for sparse file support.
2. A loop device, backed by the 1 GB file, is created, mkfs is
   run on it and it is mounted. However, the mount command does
   not specify the filesystem type. This doesn't always work
   properly, as the kernel may pick the wrong driver if there
   are multiple implementations for a particular filesystem,
   or not find a driver at all in some cases.

To fix the behavior of the test case, this commit adds a requirement
for sparse file support to the test case, and explicitly specifies
the filesystem type as $FSTYP when mounting the loop device.

Signed-off-by: Ari Sundholm <ari@tuxera.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoxfs/449: filter out deprecation warnings from mkfs
Darrick J. Wong [Wed, 1 Sep 2021 00:12:01 +0000 (17:12 -0700)]
xfs/449: filter out deprecation warnings from mkfs

To avoid regressing this test when testing XFS v4 when mkfs is new
enough to whine about creating new deprecated filesystems, filter out
the deprecation warning.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: test shutdowns of a nested filesystem
Darrick J. Wong [Wed, 1 Sep 2021 00:11:55 +0000 (17:11 -0700)]
generic: test shutdowns of a nested filesystem

generic/475, but we're running fsstress on a disk image inside the
scratch filesystem

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoidmapped-mounts: Fix build error because of undefined reallocarray
Yang Xu [Wed, 1 Sep 2021 12:43:26 +0000 (20:43 +0800)]
idmapped-mounts: Fix build error because of undefined reallocarray

On old glibc, reallocarray was not introduced, so this case compiles
failed.  We should use reallocarray if glibc supports and use
realloc if glibcs doesn't support reallocarray.

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoidmapped-mounts: Use renameat instead of renameat2
Yang Xu [Wed, 1 Sep 2021 12:43:25 +0000 (20:43 +0800)]
idmapped-mounts: Use renameat instead of renameat2

We don't add fallback for renameat2, so it build failed on old
kernel. So use renameat instead.

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoconfigure.ac: Fix uncorrect detection because of linux/btrfs.h
Yang Xu [Wed, 1 Sep 2021 12:43:24 +0000 (20:43 +0800)]
configure.ac: Fix uncorrect detection because of linux/btrfs.h

On old distros, we can't detect some btrfs structs because of
undeclared 'NULL' in btrfs_err_str function. This function has been
removed after kernel commit 68598d2ea8863 ("btrfs: remove
btrfs_err_str function from uapi/linux/btrfs.h").

Fix this bug in xfstests layer by adding stddef.h because NULL is
defined in stddef.h(it is in /usr/lib/gcc*/ directory).

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agosrc/idmapped-mounts: Fix an error for the loop initialization declaration
Yang Xu [Wed, 1 Sep 2021 12:43:23 +0000 (20:43 +0800)]
src/idmapped-mounts: Fix an error for the loop initialization declaration

Signed-off-by: Yang Xu <xuyang2018.jy@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: test for rename/exchange behavior between subvolumes
Nikolay Borisov [Mon, 30 Aug 2021 15:36:41 +0000 (18:36 +0300)]
btrfs: test for rename/exchange behavior between subvolumes

This tests ensures that renames/exchanges across subvolumes work
only for other subvolumes and are otherwise forbidden and fail.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/renameat2: Make _rename_tests_source_dest take flags as 4th arguement
Nikolay Borisov [Mon, 30 Aug 2021 12:23:05 +0000 (15:23 +0300)]
common/renameat2: Make _rename_tests_source_dest take flags as 4th arguement

Currently this function takes the flags parameter to be passed to the
renameat program by assuming there exists a 'flags' variable. Instead,
make the flags being passed as 4th argument to the function.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/rc: fix MKSWAP_PROG quoting
Xiong Zhou [Wed, 1 Sep 2021 09:40:24 +0000 (17:40 +0800)]
common/rc: fix MKSWAP_PROG quoting

After commit
  0e4dd8b9 common/rc: fix ignoring of errors on
we are getting this error message when running swapfiles tests:
  +./common/rc: line 2553: MKSWAP_PROG: command not found

Rewrite the line.

Signed-off-by: Murphy Zhou <xzhou@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/643: fix weird problems on 64k-page arm systems
Darrick J. Wong [Wed, 1 Sep 2021 00:11:21 +0000 (17:11 -0700)]
generic/643: fix weird problems on 64k-page arm systems

I noticed the following regression on an arm64 system with 64k pages:

  --- generic/643.out
  +++ generic/643.out.bad
  @@ -1,2 +1,3 @@
   QA output created by 643
  +swapon added 960 pages, expected 896
   Silence is golden

Evidently mkswap writes the swapfile header advertising one memory page
less than the size of the file, and on some architectures the kernel
can sometimes grab one page less than what's advertised.  This variance
is weird but tolerable; we simply don't want to see the page count
doubling when the file size doubles.

While we're at it, include the commit id of the fix in the commit
message.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/xfs: skip xfs_check unless the test runner forces us to
Darrick J. Wong [Wed, 1 Sep 2021 00:11:16 +0000 (17:11 -0700)]
common/xfs: skip xfs_check unless the test runner forces us to

At long last I've completed my quest to ensure that every corruption
found by xfs_check can also be found by xfs_repair.  Since xfs_check
uses more memory than repair and has long been obsolete, let's stop
running it automatically from _check_xfs_filesystem unless the test
runner makes us do it.

Tests that explicitly want xfs_check can call it via _scratch_xfs_check
or _xfs_check; that part doesn't go away.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agofstests: which has been deprecated by Debian
Dave Chinner [Mon, 30 Aug 2021 01:14:01 +0000 (11:14 +1000)]
fstests: which has been deprecated by Debian

This patch is brought to you by the Department That Hates Users.

The latest debian unstable release is now causing a bunch of new
test failures because they have deprecated the which command.

Rather than make everyone jump through hoops chasing problems with
'which' while debian decides to how to package at least three
variants of 'which' - each which will different semantics, behaviour
and support - as alternatives users then have to opt into, let's
just remove the remaining uses of the shell independent 'which'
command and replace them with bash builtin 'type -P' operations.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric/081: clean up snapshot devices correctly
Dave Chinner [Mon, 30 Aug 2021 01:13:24 +0000 (11:13 +1000)]
generic/081: clean up snapshot devices correctly

I recently updated the userspace on a DAX-capable test VM, and no
g/081 fails to release the SCRATCH_DEV correctly. g/081 fails on dax
capable devices because DM rejects one of the setup operations:

The 081.full output file shows setup failure and then snapshot
creation failure:

  Wiping xfs signature on /dev/pmem1.
  Physical volume "/dev/pmem1" successfully created.
  Volume group "vg_081" successfully created
  Logical volume "base_081" created.
  device-mapper: reload ioctl on  (251:0) failed: Invalid argument
  Failed to suspend logical volume vg_081/base_081.
  Device vg_081-base_081-real (251:1) is used by another device.
  Failed to revert logical volume vg_081/base_081.
  Aborting. Manual intervention required.
Failed to create snapshot
.....

And dmesg tells us the reload ioctl failed because:

[ 2977.522306] device-mapper: ioctl: can't change device type (old=3 vs new=1) after initial table load.

DM table types are:

enum dm_queue_mode {
        DM_TYPE_NONE             = 0,
        DM_TYPE_BIO_BASED        = 1,
        DM_TYPE_REQUEST_BASED    = 2,
        DM_TYPE_DAX_BIO_BASED    = 3,
};

which indicates that the original table is DAX_BIO based (correct,
this is a pmem device) but then one of the setup operations is for
a type that is not DAX capable. Hence it fails and then things go
bad.

But the test has actually created the snapshot device and it's COW
target, so now when the _cleanup() function is called, the vgremove
command fails because the VG is busy (still has snapshot devices
attached to it) and at this point $SCRATCH_DEV is now unusable
without manual cleanup.

Manual cleanup just needs to remove the snapshot and logical base
volumes before trying to remove the vg and pv.  With this change, we
now see the teardown do:

....
Failed to create snapshot
umount: /mnt/test/mnt_081: not mounted.
  Logical volume "snap_081" successfully removed
  Logical volume "base_081" successfully removed
  Volume group "vg_081" successfully removed
  Labels on physical volume "/dev/pmem1" successfully wiped.

And future tests are able to use the SCRATCH_DEV normally.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agobtrfs: make sure btrfs can create compressed inline extent
Qu Wenruo [Thu, 26 Aug 2021 05:34:32 +0000 (13:34 +0800)]
btrfs: make sure btrfs can create compressed inline extent

Btrfs has the ability to inline small file extents into its metadata,
and such inlined extents can be further compressed if needed.

The new test case is for a regression caused by commit f2165627319f
("btrfs: compression: don't try to compress if we don't have enough
pages").

That commit prevents btrfs from creating compressed inline extents, even
"-o compress,max_inline=2048" is specified, only uncompressed inline
extents can be created.

The test case will make sure that the content of the small file is
consistent between cycle mount, then use "btrfs inspect dump-tree" to
verify the created extent is both inlined and compressed.

The regression is fixed by commit 4e9655763b82 ("Revert "btrfs:
compression: don't try to compress if we don't have enough pages"")

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: Test page faults during read and write
Andreas Gruenbacher [Fri, 27 Aug 2021 21:02:09 +0000 (23:02 +0200)]
generic: Test page faults during read and write

Some filesystems have problems when the buffer passed to read or
write is memory-mapped to the file being read from or written to,
and the buffer needs to be faulted in during the read or write.
That's not common, but filesystems are still required to cope with
it, and if they fail this test, then they will also fail more
complex scenarios that involve multiple files.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agogeneric: check log recovery with readonly mount
Murphy Zhou [Wed, 25 Aug 2021 03:26:30 +0000 (11:26 +0800)]
generic: check log recovery with readonly mount

And followed by a rw mount. After log recovery by these 2 mounts,
the filesystem should be in a consistent state.

Suggested-by: Donald Douwsma <ddouwsma@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Murphy Zhou <xzhou@redhat.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agocommon/rc: fix ignoring of errors on
Luis Chamberlain [Wed, 25 Aug 2021 21:15:20 +0000 (14:15 -0700)]
common/rc: fix ignoring of errors on

Commit 0c95c8acb ("common: hide permision warning from mkswap for
exfat") and commit 08e93fa7c ("generic/554: hide permision warning on
exfat") removed a warning by grepping an error out. It however broke
use on older versions of bash such as 4.4.23:

./common/rc: line 2554: syntax error near unexpected token `>'
./common/rc: line 2554: `       MKSWAP_PROG "$fname" 2> >(grep -v 'insecure permission' >&2) >> $seqres.full'

Fix both instances by using a $(foo) to run the executable.

Cc: Pavel Reichl <preichl@redhat.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoidmap: override btrfs_ioctl_vol_args_v2 if system header doesn't have subvolid
Darrick J. Wong [Mon, 23 Aug 2021 18:21:38 +0000 (11:21 -0700)]
idmap: override btrfs_ioctl_vol_args_v2 if system header doesn't have subvolid

The struct btrfs_ioctl_vol_args_v2 in /usr/include/linux/btrfs.h on
my system predates the inclusion of the "subvolid" field.  This
causes the following build failure:

idmapped-mounts.c: In function 'btrfs_delete_subvolume_id':
idmapped-mounts.c:9730:6: error: 'struct btrfs_ioctl_vol_args_v2' has no member named 'subvolid'
 9730 |  args.subvolid = subvolid;
      |      ^

Since this source file contains its own more uptodate copy of that
structure, add some more autoconf/cpp magic so that we can override
the struct definition if the system header doesn't have the desired
field.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Acked-by: Christian Brauner <christian.brauner@ubuntu.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Anju T Sudhakar<anju@linux.vnet.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
4 months agoext4: add test to validate the large_dir feature
Theodore Ts'o [Mon, 23 Aug 2021 20:22:31 +0000 (16:22 -0400)]
ext4: add test to validate the large_dir feature

The ext4 large_dir feature supports a directory hash tree with a depth
greater than 2.  Reuse the dirstress program to create a sufficiently
large directory in order to exercise the large_dir code paths.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: introduce btrfs specific idmapped mounts tests
Christian Brauner [Sat, 14 Aug 2021 10:48:05 +0000 (12:48 +0200)]
btrfs: introduce btrfs specific idmapped mounts tests

While core vfs functionality that btrfs implements is completely covered
by the generic test-suite the btrfs specific ioctls are not.
This patch expands the test-suite to cover btrfs specific ioctls that
required changes to work on idmapped mounts. We deliberately don't use
the libbtrfsutil library as we need to know exactly what ioctl's are
issued and we need to be in control of all privileges at all times. This
test-suite currently tests:

- BTRFS_IOC_{SNAP,SUBVOL}_CREATE_V2
  - subvolume creation on idmapped mounts where the fsids do have a
    mapping in the superblock
  - snapshot creation on idmapped mounts where the fsids do have a
    mapping in the superblock
  - subvolume creation on idmapped mounts where the fsids do not have a
    mapping in the superblock
  - snapshot creation on idmapped mounts where the fsids do not have
    a mapping in the superblock
  - subvolume creation on idmapped mounts where the caller is
    located in a user namespace and the fsids do have a mapping
    in the superblock
  - snapshot creation on idmapped mounts where the caller is located
    in a user namespace and the fsids do have a mapping in the
    superblock
  - subvolume creation on idmapped mounts where the caller is
    located in a user namespace and the fsids do not have a mapping
    in the superblock
  - snapshot creation on idmapped mounts where the caller is located
    in a user namespace and the fsids do not have a mapping in the
    superblock

- BTRFS_IOC_SNAP_DESTROY_V2
  - subvolume deletion on idmapped mounts where the fsids do have a
    mapping in the superblock
  - snapshot deletion on idmapped mounts where the fsids do have a
    mapping in the superblock
  - subvolume deletion on idmapped mounts where the fsids do not have a
    mapping in the superblock
  - snapshot deletion on idmapped mounts where the fsids do not have
    a mapping in the superblock
  - subvolume deletion on idmapped mounts where the caller is
    located in a user namespace and the fsids do have a mapping
    in the superblock
  - snapshot deletion on idmapped mounts where the caller is located
    in a user namespace and the fsids do have a mapping in the
    superblock
  - subvolume deletion on idmapped mounts where the caller is
    located in a user namespace and the fsids do not have a mapping
    in the superblock
  - snapshot deletion on idmapped mounts where the caller is located
    in a user namespace and the fsids do not have a mapping in the
    superblock
  - unprivileged subvolume deletion on idmapped mounts where the fsids
    do have a mapping in the superblock and the filesystem is mounted
    with "user_subvol_rm_allowed"
  - unprivileged snapshot deletion on idmapped mounts where the fsids do
    have a mapping in the superblock and the filesystem is mounted with
    "user_subvol_rm_allowed"
  - subvolume deletion on idmapped mounts where the caller is
    located in a user namespace and the fsids do have a mapping
    in the superblock and the filesystem is mounted with
    "user_subvol_rm_allowed"
  - snapshot deletion on idmapped mounts where the caller is located
    in a user namespace and the fsids do have a mapping in the
    superblock and the filesystem is mounted with
    "user_subvol_rm_allowed"

- BTRFS_IOC_SUBVOL_SETFLAGS
  - subvolume flags on idmapped mounts where the fsids do have a mapping
    in the superblock
  - snapshot flags on idmapped mounts where the fsids do have a mapping
    in the superblock
  - subvolume flags on idmapped mounts where the fsids do not have a
    mapping in the superblock
  - snapshot flags on idmapped mounts where the fsids do not have a
    mapping in the superblock
  - subvolume flags on idmapped mounts where the caller is located in a
    user namespace and the fsids do have a mapping in the superblock
  - snapshot flags on idmapped mounts where the caller is located in a
    user namespace and the fsids do have a mapping in the superblock
  - subvolume flags on idmapped mounts where the caller is located in a
    user namespace and the fsids do not have a mapping in the superblock
  - snapshot flags on idmapped mounts where the caller is located in a
    user namespace and the fsids do not have a mapping in the superblock

- BTRFS_IOC_INO_LOOKUP_USER
  - subvolume lookup on idmapped mounts where the fsids do have a mapping
    in the superblock
  - subvolume lookup on idmapped mounts where the fsids do not have a
    mapping in the superblock
  - subvolume lookup on idmapped mounts where the caller is located in a
    user namespace and the fsids do have a mapping in the superblock
  - subvolume lookup on idmapped mounts where the caller is located in a
    user namespace and the fsids do not have a mapping in the superblock

Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chris Mason <clm@fb.com>
Cc: David Sterba <dsterba@suse.com>
Cc: fstests@vger.kernel.org
Cc: linux-btrfs@vger.kernel.org
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: add nested user namespace tests
Christian Brauner [Sat, 14 Aug 2021 10:48:04 +0000 (12:48 +0200)]
generic: add nested user namespace tests

Test ownership and ownership changes in a complex user namespace
hierarchy.

Cc: fstests@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: add nested userns creation helpers
Christian Brauner [Sat, 14 Aug 2021 10:48:03 +0000 (12:48 +0200)]
idmapped-mounts: add nested userns creation helpers

Add a helper to create a nested userns hierarchy. This will be used in
follow-up tests.

Cc: fstests@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: refactor helpers
Christian Brauner [Sat, 14 Aug 2021 10:48:02 +0000 (12:48 +0200)]
idmapped-mounts: refactor helpers

Make all userns creation helpers share a commond codebase and move a
bunch of code into utils.{c,h}. This simplifies a bunch of things and
makes it easier to create nested user namespaces in follow up patches.

Cc: fstests@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: add fscaps regression test
Christian Brauner [Sat, 14 Aug 2021 10:48:01 +0000 (12:48 +0200)]
generic: add fscaps regression test

Add a test to verify that setting a v3 fscap from an idmapped mount
works as expected. This and other related use-cases were regressed by
commit [1] which was reverted in [2] and the proper fix merged right
before v5.12 was released in [3].

[1]: commit 3b0c2d3eaa83 ("Revert 95ebabde382c ("capabilities: Don't allow writing ambiguous v3 file capabilities")")
[2]: commit 95ebabde382c ("capabilities: Don't allow writing ambiguous v3 file capabilities")
[3]: commit db2e718a4798 ("capabilities: require CAP_SETFCAP to map uid 0")

Cc: fstests@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: introduce an explicit command line switch for testsuite
Christian Brauner [Sat, 14 Aug 2021 10:48:00 +0000 (12:48 +0200)]
idmapped-mounts: introduce an explicit command line switch for testsuite

Introduce an explicit command line switch to runs the basic test suite.
This prepares for the introduction of additional command line switches
to run additional tests.

Cc: fstests@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: switch to getopt_long_only()
Christian Brauner [Sat, 14 Aug 2021 10:47:59 +0000 (12:47 +0200)]
idmapped-mounts: switch to getopt_long_only()

We're not using the shortopts anywhere anyway  and shouldn't encourage
using shortopts.  It's much more descriptive to see:

$here/src/idmapped-mounts/idmapped-mounts \
        --test-btrfs \
        --device "$TEST_DEV" \
        --mountpoint "$TEST_DIR" \
        --scratch-device "$SCRATCH_DEV" \
        --scratch-mountpoint "$SCRATCH_MNT"
        --fstype "$FSTYP"

in a test than it is to see:

$here/src/idmapped-mounts/idmapped-mounts \
        -b
        -d "$TEST_DEV" \
        -m "$TEST_DIR" \
        -s "$SCRATCH_DEV" \
        -a "$SCRATCH_MNT" \
        -f "$FSTYP"

In the second case one has to go consult the source code to make sure
that the correct option is passed. In the first case one can just see it
directly.

Cc: fstests@vger.kernel.org
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoidmapped-mounts: use die() helper
Christian Brauner [Sat, 14 Aug 2021 10:47:58 +0000 (12:47 +0200)]
idmapped-mounts: use die() helper

Use the dedicated helper to report an error and exit with failure
instead of hand-rolling it.

Cc: fstests@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/scsi_debug: use the patient module remover
Luis Chamberlain [Fri, 20 Aug 2021 01:04:02 +0000 (18:04 -0700)]
common/scsi_debug: use the patient module remover

If you try to run tests such as generic/108 in a loop
you'll eventually see a failure, but the failure can
be a false positive and the test was just unable to remove
the scsi_debug module.

We need to give some time for the refcnt to become 0. For
instance for the test generic/108 the refcnt lingers between
2 and 1. It should be 0 when we're done but a bit of time
seems to be required. The chance of us trying to run rmmod
when the refcnt is 2 or 1 is low, about 1/30 times if you
run the test in a loop on linux-next today.

Likewise, even when its 0 we just need a tiny breather before
we can remove the module (sleep 10 suffices) but this is
only required on older kernels. Otherwise removing the module
will just fail.

Some of these races are documented on the korg#212337, and
Doug Gilbert has posted at least one patch attempt to try
to help with this [1]. The patch does not resolve all the
issues though, it helps though.

This let's us remove the cheesy try loop. We keep the
udevadm settle call as it can help salvage buggy tests
which forgot to call it.

We also special-case where MODPROBE_PATIENT_RM_TIMEOUT_SECONDS
is set to "forever" and the initial module check finds its
in use, for that case we just try removing the module once
since fstests would not be the one leaving modues lingering
around, and waiting forever could mean you won't discover
the issue for a while.

[0] https://bugzilla.kernel.org/show_bug.cgi?id=212337
[1] https://lkml.kernel.org/r/20210508230745.27923-1-dgilbert@interlog.com

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/module: add patient module rmmod support
Luis Chamberlain [Fri, 20 Aug 2021 01:04:01 +0000 (18:04 -0700)]
common/module: add patient module rmmod support

When we call rmmod it will fail if the refcnt is greater than 0.
This is expected, however, if using test modules such as scsi_debug,
userspace tests may expect that once userspace is done issuing out
commands it can safely remove the module, and the module will be
removed.

This is not true for few reasons. First, a module might take a while
to quiesce after its used. This varies module by module. For example,
at least for scsi_debug there is one patch to help with this but
that is not sufficient to address all the removal issues, it just helps
quiesce the module faster. If something like LVM pvremove is used, as in
the case of generic/108, it may take time before the module's refcnt goes
to 0 even if DM_DEFERRED_REMOVE is *not* used and even if udevadm settle
is used. Even *after* all this... the module refcnt is still very
fickle. For example, any blkdev_open() against a block device will bump
a module refcnt up and we have little control over stopping these
sporadic userspace calls after a test. A failure on module removal then
just becomes an inconvenience on false positives.

This was first observed on scsi_debug [0]. Doug worked on a patch to
help the driver quiesce [1]. Later the issue has been determined to be
generic [2]. The only way to properly resolve these issues is with a
patient module remover. The kernel used to support a wait for the
delete_module() system call, however this was later deprecated into
kmod with a 10 second userspace sleep. That 10 second sleep is long gone
from kmod now though. I've posted patches now for a kmod patient module
remover then [3], in light of the fact that this issue is generic and
the only way to then properly deal with this is implementing a userspace
patient module remover.

Use the kmod patient module remover when supported, otherwise we open
code our own solution inside fstests. We default to a timeout of 100
seconds. Each test can override the timeout by setting the variable
MODPROBE_PATIENT_RM_TIMEOUT_SECONDS or setting it to "forever" if they
wish for the patience to be infinite.

This uses kmod's patient module remover if you have that feature,
otherwise we open code a solution in fstests which is a simplified
version of what has been proposed for kmod.

[0] https://bugzilla.kernel.org/show_bug.cgi?id=212337
[1] https://lore.kernel.org/linux-scsi/20210508230745.27923-1-dgilbert@interlog.com/
[2] https://bugzilla.kernel.org/show_bug.cgi?id=214015
[3] https://lkml.kernel.org/r/20210810051602.3067384-1-mcgrof@kernel.org

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric/457: post-check fs on thin-vol device instead of scratch device
Kai-Wen Hu [Fri, 20 Aug 2021 12:10:31 +0000 (20:10 +0800)]
generic/457: post-check fs on thin-vol device instead of scratch device

Recently, we found that this test running with btrfs didn't
pass the post-check fsck on scratch device.  This issue occured
due to the commit 5faafc77f ("dm space maps: don't reset space map
allocation cursor when committing") and the use of SSD as scratch
device.

In this test, it does not directly use the filesystem on scratch
device. It runs test on dm-thin volume tying two linear-mapping
devices, and those two linear devices are created from the scratch
device.  When the test runs, it will write data on dm-thin volume,
and dirty the underlying scratch device.

Since the commit (5faafc77f) changes the dm space allocation policy,
it may cause more dirty blocks on underlying scratch device when we
write data on thin volume.  So it makes more chance to overwrite the
position of btrfs-meta on the scratch device.  Besides, our scratch
device is SSD, so there is only one copy of btrfs meta on the scratch
device.  Once the btrfs-meta is overwritten, the btrfs volume will
crash, and cannot pass the fsck check.

Because, this test won't directly use the fs on scratch device.  We
just disable the post-check fsck on scratch device, instead, we run
post-check fsck on thin-vol device.

Reviewed-by: Chung-Chiang Cheng <cccheng@synology.com>
Reviewed-by: Robbie Ko <robbieko@synology.com>
Signed-off-by: Kai-Wen Hu <kevinhu@synology.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/176: fix the group name
Darrick J. Wong [Tue, 17 Aug 2021 23:53:05 +0000 (16:53 -0700)]
xfs/176: fix the group name

Filesystem shrink tests for xfs are supposed to be in the 'shrinkfs'
group, not 'shrink'.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/187: remove the test
Dave Chinner [Thu, 19 Aug 2021 06:00:44 +0000 (16:00 +1000)]
xfs/187: remove the test

Upstream changes mean that "-o noattr2" no longer removes the
on-disk ATTR2 feature bit from the superblock and so the failure
case this test exercises is no longer guaranteed to be a failure
case. Hence the test is now useless as a regression test, so remove
it.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4: regression test for "tune2fs -l" after ext4 shutdown
Boyang Xue [Thu, 19 Aug 2021 08:07:51 +0000 (16:07 +0800)]
ext4: regression test for "tune2fs -l" after ext4 shutdown

Regression test for e2fsprogs commit:

e905fbe3fd0f ext4: Fix tune2fs checksum failure for mounted filesystem

This test runs "tune2fs -l" after ext4 shutdown. tune2fs reads
superblock checksum from the buffer cache. On unfixed kernels, the
checksum is incorrect until the writeout happens, so tune2fs fails
with "superblock checksum does not match" in this case.

Signed-off-by: Boyang Xue <bxue@redhat.com>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: add swapfile maxpages regression test
Darrick J. Wong [Thu, 19 Aug 2021 01:43:26 +0000 (18:43 -0700)]
generic: add swapfile maxpages regression test

Add regression test for "mm/swap: consider max pages in
iomap_swapfile_add_extent".

Cc: Gang Deng <gavin.dg@linux.alibaba.com>
Cc: Xu Yu <xuyu@linux.alibaba.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: add checks for zoned block device
Naohiro Aota [Mon, 16 Aug 2021 11:35:10 +0000 (20:35 +0900)]
generic: add checks for zoned block device

Modify generic tests to require non-zoned block device

generic/108 is disabled on zoned block device because the LVM device
not always aligned to the zone boundary.

generic/471 is disabled because we cannot enable NoCoW on zoned
btrfs.

generic/570 is disabled because swap file which require nocow is not
usable on zoned btrfs.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: add checks for zoned block device
Naohiro Aota [Mon, 16 Aug 2021 11:35:09 +0000 (20:35 +0900)]
btrfs: add checks for zoned block device

Modify btrfs tests to require non-zoned block device or limit some
part of tests not to be run on zone block devices.

Modified tests by the reasons:

* Mixed BG
  - btrfs/011
* Non-single profile
  - btrfs/003
  - btrfs/011
  - btrfs/023
  - btrfs/124
  - btrfs/195
  - btrfs/197
  - btrfs/198
  - and these are restricted indirectly by "_require_btrfs_fs_feature raid56"
    - btrfs/125
    - btrfs/148
    - btrfs/157
    - btrfs/158
* Convert from ext4
  - btrfs/012
  - btrfs/136
* nodatacow
  - btrfs/236
* inode cache
  - btrfs/049
* space cache (v1)
  - btrfs/131
* write outside of FS code
  - btrfs/116
  - btrfs/140
  - btrfs/215
* verbose output
  - btrfs/194

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon: add zoned block device checks
Naohiro Aota [Mon, 16 Aug 2021 11:35:08 +0000 (20:35 +0900)]
common: add zoned block device checks

dm-error and dm-snapshot does not have DM_TARGET_ZONED_HM nor
DM_TARGET_MIXED_ZONED_MODEL feature and does not implement
.report_zones(). So, it cannot pass the zone information from the down
layer (zoned device) to the upper layer.

Loop device also cannot pass the zone information.

This patch requires non-zoned block device for the tests using these
ones.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs: test regression in shrink when the new EOFS splits a sparse inode cluster
Darrick J. Wong [Wed, 28 Jul 2021 00:10:35 +0000 (17:10 -0700)]
xfs: test regression in shrink when the new EOFS splits a sparse inode cluster

This is a targeted regression test for commit da062d16a897 ("xfs:
check for sparse inode clusters that cross new EOAG when shrinking"),
which was found by running the random-loopy shrink stresser xfs/168.

The original shrink implementation assumed that if we could allocate
the last free extent in the filesystem, it was ok to proceed with
the fs shrink.  Unfortunately, this isn't quite the case -- if
there's a sparse inode cluster such that the blocks at the end of
the cluster are free, it is not ok to shrink the fs to the point
that part of the cluster hangs off the end of the filesystem.  Doing
so results in repair and scrub marking the filesystem corrupt, so we
must not.

(EOFS == "end of filesystem"; EOAG == "end of allocation group")

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: test xattr operations only
Darrick J. Wong [Wed, 28 Jul 2021 00:10:24 +0000 (17:10 -0700)]
generic: test xattr operations only

Exercise extended attribute operations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/220: clear_cache fix for older kernel
Anand Jain [Fri, 13 Aug 2021 01:59:35 +0000 (09:59 +0800)]
btrfs/220: clear_cache fix for older kernel

mount option -o clear_cache shown in /proc/self/mounts isn't
supported in the newer kernel, make this test case older kernel
compatible by checking if clear_cache is shown in the
/proc/self/mounts.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/220: nologreplay support older kernel
Anand Jain [Fri, 13 Aug 2021 01:59:34 +0000 (09:59 +0800)]
btrfs/220: nologreplay support older kernel

mount option -o rescue=nologreplay isn't supported on the older
kernel, make this test case older kernel compatible by checking if
the mount option is supported.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/220: discard=sync support older kernel
Anand Jain [Fri, 13 Aug 2021 01:59:33 +0000 (09:59 +0800)]
btrfs/220: discard=sync support older kernel

mount option -o discard=sync isn't supported on the older kernel,
make this test case older kernel compatible by checking if the mount
option is supported.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/rc: debug add _scratch_mount_options to the _scratch_mount
Anand Jain [Fri, 13 Aug 2021 01:59:32 +0000 (09:59 +0800)]
common/rc: debug add _scratch_mount_options to the _scratch_mount

When the _scratch_mount fails, there isn't any clue on why it failed
as of now, change this and add the $(_scratch_mount_options $*) to
the _fail.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/220: fix clear_cache and inode_cache option tests
Boris Burkov [Tue, 12 Jan 2021 21:17:47 +0000 (13:17 -0800)]
btrfs/220: fix clear_cache and inode_cache option tests

I recently changed clear_cache to not appear in mount options, as it
has one shot semantics, which was breaking this test. Test
explicitly that it _doesn't_ appear, which properly fails on old
filesystems and passes on misc-next.

The patch that changed this behavior was:
8b228324a8ce btrfs: clear free space tree on ro->rw remount

Separately, inode_cache is deprecated and will never appear in mount
options; remove it entirely.

Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoshared/032: add check for zoned block device
Naohiro Aota [Wed, 11 Aug 2021 15:12:30 +0000 (00:12 +0900)]
shared/032: add check for zoned block device

Mkfs on zoned block device won't work on most filesystem. Let's
disable the test.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: add minimal file system size check
Naohiro Aota [Wed, 11 Aug 2021 15:12:28 +0000 (00:12 +0900)]
btrfs: add minimal file system size check

Some btrfs tests call _scratch_pool_mkfs or _scratch_mkfs by themselves to
specify file system size limit. It slips through the check in
_scratch_mkfs_sized(). Let's add size check call for each of them.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/057: use _scratch_mkfs_sized to set filesystem size
Naohiro Aota [Wed, 11 Aug 2021 15:12:27 +0000 (00:12 +0900)]
btrfs/057: use _scratch_mkfs_sized to set filesystem size

Btrfs/057 is using _scratch_mkfs directly to set filesystem size. This
can be _scratch_mkfs_sized instead, to go through several
checks (e.g., minimal filesystem size check).

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/rc: fix blocksize detection for btrfs
Naohiro Aota [Wed, 11 Aug 2021 15:12:26 +0000 (00:12 +0900)]
common/rc: fix blocksize detection for btrfs

Mkfs.btrfs's "-b" does not specify the block size but the file system size.
Use "-s" to detect it instead.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/rc: introduce minimal fs size check
Naohiro Aota [Wed, 11 Aug 2021 15:12:25 +0000 (00:12 +0900)]
common/rc: introduce minimal fs size check

_scratch_mkfs_sized() create a file system with specified size
limit. It can, however, too small for certain kind of devices. For
example, zoned btrfs requires at least 5 zones to make a file system.

This commit introduces MIN_FSSIZE, which specify the minimum size of the
possible file system. We can set this variable e.g. $ZONE_SIZE *
$MIN_ZONE_COUNT.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agofstests: use udevadm settle after pvremove
Luis Chamberlain [Wed, 11 Aug 2021 15:45:10 +0000 (08:45 -0700)]
fstests: use udevadm settle after pvremove

As with creation, we also need to use udevadm settle
when removing a pv, otherwise we can trip on races with
module removals for the block devices in use.

This reduces the amount of time in which a block device
module refcnt for test modules such as scsi_debug spends
outside of 0.

Races with the refcnt being greater than 0 means module
removal can fail causing false positives. This helps
ensure that the pv is really long gone. These issues
are tracked for scsi_debug [0] and later found to be a
generic issue regardless of filesystem with pvremove [1].

Using udevadm settle *helps*, it does not address all
possible races with the refcnt as noted in the generic
bug entry [1].

[0] https://bugzilla.kernel.org/show_bug.cgi?id=212337
[1] https://bugzilla.kernel.org/show_bug.cgi?id=214015

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs/177: Handle the different fi resize output formats
Marcos Paulo de Souza [Thu, 12 Aug 2021 11:30:48 +0000 (08:30 -0300)]
btrfs/177: Handle the different fi resize output formats

Commit 78aa1d95dd99 ("btrfs-progs: fi resize: make output more
readable") added the device id of the resized fs along with a pretty
printed size. Create a new function to simplify the output message
using size in bytes.

Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agodmerror: export configuration so that subprograms don't have to reinit
Darrick J. Wong [Tue, 20 Jul 2021 01:09:13 +0000 (18:09 -0700)]
dmerror: export configuration so that subprograms don't have to reinit

Export the dmerror configuration variables so that subprograms don't
have to reinitialize the configuration in their own subprograms before
calling the helpers.  In the next patch (where we allow dmerror for log
and rt devices) it will become important to avoid these
reinitializations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agodmflakey: support external log and realtime devices
Darrick J. Wong [Tue, 20 Jul 2021 01:09:08 +0000 (18:09 -0700)]
dmflakey: support external log and realtime devices

Upgrade the dmflakey code to coordinate making external scratch log and
scratch realtime devices flakey along with the scratch device.  Note
that unlike SCRATCH_DEV, we save the old rt/log devices as separate
variables and replace SCRATCH_{RT,LOG}DEV so that helper functions
continue to work without modification.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agooverlay/078: Fix cleanup with unpatched kernel
Amir Goldstein [Tue, 3 Aug 2021 07:07:58 +0000 (10:07 +0300)]
overlay/078: Fix cleanup with unpatched kernel

Darrick wrote: "...
I noticed a massive regression with this week's fstests,
probably because something didn't get cleaned up properly:
...
+rm: cannot remove '/opt/ovl-upper/testfile': Operation not permitted
...

and then the tests after it (e.g. generic/030) fail with:

+mount: /opt/ovl-mnt: mount(2) system call failed: Stale file handle.
..."

Link: https://lore.kernel.org/fstests/20210802230727.GC3601425@magnolia/
Reported-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs: quota state command should report upg grace times separately
Zorro Lang [Fri, 6 Aug 2021 05:10:12 +0000 (13:10 +0800)]
xfs: quota state command should report upg grace times separately

Regression test for xfsprogs d8a94546 ("xfs_quota: state command
should report ugp grace times"). When give "-ugp" or "-a" options to
xfs_quota state command, it should report grace times for all three
types separately.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agobtrfs: verify the behavior of deleting non-existing device
Qu Wenruo [Fri, 6 Aug 2021 11:33:33 +0000 (19:33 +0800)]
btrfs: verify the behavior of deleting non-existing device

There is a kernel regression for btrfs, that when passing
non-existing devid to "btrfs device remove" command, kernel will
crash due to NULL pointer dereference.

The test case is for such regression, it will:

- Create and mount an empty single-device btrfs
- Try to remove devid 3, which doesn't exist for above fs
- Make sure the command exits properly with expected error message

The kernel fix is titled "btrfs: fix NULL pointer dereference when
deleting device by invalid id".

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/rc: only force nfs4.2 non-default SEEK_HOLE behaviour
Jeffle Xu [Thu, 29 Jul 2021 04:47:58 +0000 (12:47 +0800)]
common/rc: only force nfs4.2 non-default SEEK_HOLE behaviour

Only NFSv4.2 supports non-defautl SEEK_HOLE behaviour. Thus default
SEEK_HOLE behaviour shall be allowed for NFSv4.0/4.1, or it will fail
generic/285, generic/448, generic/490 on NFSv4.0/4.1, complaining they
should support non-default SEEK_HOLE behaviour.

The *.full log is like:
File system supports the default behavior.
Default behavior is not allowed. Aborting.

Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/attr: Reduce MAX_ATTRS to leave some overhead for 64K blocksize
Ritesh Harjani [Wed, 21 Jul 2021 05:28:02 +0000 (10:58 +0530)]
common/attr: Reduce MAX_ATTRS to leave some overhead for 64K blocksize

Test generic/020 fails for ext4 with 64K blocksize.
This adds changes in common/attr for MAX_ATTRS calculations for
ext2|ext3|ext4 along with comments explaining the calculations.

Suggested-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/attr: Cleanup end of line whitespaces issues
Ritesh Harjani [Wed, 21 Jul 2021 05:28:01 +0000 (10:58 +0530)]
common/attr: Cleanup end of line whitespaces issues

This patch clears the end of line whitespace issues in this file.
Mostly since many kernel developers also keep this editor config to clear
any end of line whitespaces on file save.

Reviewed-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4/022: exclude this test for dax config on 64KB pagesize platform
Ritesh Harjani [Wed, 21 Jul 2021 05:27:57 +0000 (10:57 +0530)]
ext4/022: exclude this test for dax config on 64KB pagesize platform

This test case assumes blocksize to be 4KB and hence it fails
to mount with "-o dax" option on a 64kb pagesize platform (e.g. PPC64).
This leads to test case reported as failed with dax config on PPC64.

This patch exclude this test when pagesize is 64KB and for dax config.

Reviewed-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4/306: Add -b blocksize parameter too to avoid failure with DAX config
Ritesh Harjani [Wed, 21 Jul 2021 05:27:56 +0000 (10:57 +0530)]
ext4/306: Add -b blocksize parameter too to avoid failure with DAX config

mkfs.ext4 by default uses 4K blocksize. On DAX config with a 64K
pagesize platform (PPC64), this will fail to mount since DAX requires bs
== ps.
Hence add the -b blocksize paramter in ext4/306.

Reviewed-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4/027: Correct the right code of block and inode bitmap
Ritesh Harjani [Wed, 21 Jul 2021 05:27:55 +0000 (10:57 +0530)]
ext4/027: Correct the right code of block and inode bitmap

Observed occasional failure of this test sometimes say with 64k config
and small device size. Reason is we were grepping for wrong values for
inode and block bitmap.

Correct those values according to [1] to fix this test.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/ext4/fsmap.h#n53

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoext4/003: Fix this test on 64K platform for dax config
Ritesh Harjani [Wed, 21 Jul 2021 05:27:54 +0000 (10:57 +0530)]
ext4/003: Fix this test on 64K platform for dax config

mkfs.ext4 by default uses 4K blocksize which doesn't mount when
testing with dax config and the test fails. This patch fixes it.

Signed-off-by: Ritesh Harjani <riteshh@linux.ibm.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/scsi_debug: use udevadm settle instead of sleeping
Luis Chamberlain [Tue, 27 Jul 2021 20:10:43 +0000 (13:10 -0700)]
common/scsi_debug: use udevadm settle instead of sleeping

The variable UDEV_SETTLE_PROG is already defined and used for
lvm to either use `udevadm settle` in case it is available, and
if not, use 'sleep 1' otherwise (ancient distros or CONFIG_NET
is missing).

Use it on scsi_debug to replace the sleep calls sprinkled in
place after module removal. The correct thing to do is to
just use udevadm settle when available, and only fall back to
calling sleep when udevadm is not available or CONFIG_NET
is disabled.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocommon/config: disable udevadm settle if CONFIG_NET is disabled
Luis Chamberlain [Tue, 27 Jul 2021 20:10:42 +0000 (13:10 -0700)]
common/config: disable udevadm settle if CONFIG_NET is disabled

If CONFIG_NET is disabled kobject_uevent_net_broadcast() will be a no-op
and so no uevent are sent and so 'udevadm settle' won't really do
anything for you.

We check for /proc/net to see if CONFIG_NET was enabled.

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/007: fix regressions on V4 filesystems
Darrick J. Wong [Wed, 28 Jul 2021 20:00:45 +0000 (13:00 -0700)]
xfs/007: fix regressions on V4 filesystems

Following commit eae40404, I noticed the following regression when
running a V4 fstests run on an 5.13 kernel:

  --- /tmp/fstests/tests/xfs/007.out      2021-05-13 11:47:55.793859995 -0700
  +++ /var/tmp/fstests/xfs/007.out.bad    2021-07-28 09:23:42.856000000 -0700
  @@ -16,4 +16,4 @@
   *** umount
   *** Usage after quotarm ***
   core.nblocks = 0
  -core.nblocks = 0
  +core.nblocks = 1

The underlying cause of this problem is the fact that we now remount the
filesystem with no quota options because that will (soon) become the
only means to turn off quota accounting on XFS.  Because V4 filesystems
don't support simultaneous project and group quotas and play weird
remapping games with the incore superblock field, we actually have to
issue a remove command for the group quota file if we're trying to
truncate the project quota file on a V4 filesystem.

Due to stupid limitations in xfs_quota we actually have to issue a
separate 'remove' command.

Fixes: eae40404 ("xfs/007: unmount after disabling quota")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/530: skip test if user MKFS_OPTIONS screw up formatting
Darrick J. Wong [Wed, 28 Jul 2021 20:00:20 +0000 (13:00 -0700)]
xfs/530: skip test if user MKFS_OPTIONS screw up formatting

Skip this test if the user's MKFS_OPTIONS are not compatible with the
realtime parameters that we're injecting in order to test growfs bugs.
Because this test is trying to trigger a specific kernel bug, we need
mkfs to format a filesystem with very specific geometry parameters.

The first problem stems from the fact that the test performs a default
mkfs, computes a suitable realtime geometry from that filesystem, and
then formats a second time with an explicit blocksize option to mkfs.
If the original MKFS_OPTS contained a blocksize directive, the mkfs will
fail because the option was respecified.  The two blocksize options will
be the same, so we drop the explicit blocksize option.

However, this exposes a second problem: MKFS_OPTIONS might contain
options that are not compatible with any realtime filesystem.  If that
happens, _scratch_do_mkfs will "helpfully" drop MKFS_OPTIONS and try
again with only the options specified by the test.  This gets us a
filesystem with the given rt geometry, but it could be missing critical
parameters from MKFS_OPTIONS (like blocksize).  The test will then fail
to exercise the growfs bugfix, so the second part of the fix is to check
that the filesystem we're going to test actually has the geometry
parameters that we require.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocheck: back off the OOM score adjustment to -500
Darrick J. Wong [Wed, 28 Jul 2021 00:09:59 +0000 (17:09 -0700)]
check: back off the OOM score adjustment to -500

Dave Chinner complained that fstests really shouldn't be running at
-1000 oom score adjustment because that makes it more "important" than
certain system daemons (e.g. journald, udev).  That's true, so increase
it to -500.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agocheck: don't leave the scratch filesystem mounted after _notrun
Darrick J. Wong [Wed, 28 Jul 2021 00:09:48 +0000 (17:09 -0700)]
check: don't leave the scratch filesystem mounted after _notrun

Unmount the scratch filesystem if a test decides to _notrun itself
because _try_wipe_scratch_devs will not be able to wipe the scratch
device prior to the next test run.  We don't want to let scratch state
from one test leak into subsequent tests if we can help it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agoxfs/106: fix golden output regression in quota off test
Darrick J. Wong [Wed, 28 Jul 2021 00:09:43 +0000 (17:09 -0700)]
xfs/106: fix golden output regression in quota off test

In commit 4c5df338, we reshuffled this test a bit in preparation to
disable quotaoff by rearranging the test to testing disabling of quota
by remounting the filesystem.  Unfortunately, extra blank lines were
added to the golden output, leading to test regressions.

The "extra" blank lines are a result of the "echo ; test_off";
test_off() itself doesn't print anything.  Make it print /something/ so
that we know what the test was trying to do when a particular line of
golden output appears, then fix the blank lines.

Fixes: 4c5df338 ("xfs/106: don't test disabling quota accounting")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agomisc: tag all tests that examine crash recovery in a loop
Darrick J. Wong [Thu, 29 Jul 2021 16:53:05 +0000 (09:53 -0700)]
misc: tag all tests that examine crash recovery in a loop

Given all the recent problems that we've been finding with log recovery,
I think it would be useful to create a 'recoveryloop' group so that
developers have a convenient way to run every single test that rolls
around in a fs shutdown loop looking for subtle errors in recovery.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: test small swapfile without page-aligned contiguous blocks
Zorro Lang [Mon, 26 Jul 2021 09:46:05 +0000 (17:46 +0800)]
generic: test small swapfile without page-aligned contiguous blocks

If a swapfile doesn't contain even a single page-aligned contiguous
range of blocks, it's an invalid swapfile, and might cause kernel
issue. This case covered commit 5808fecc5723 ("iomap: Fix negative
assignment to unsigned sis->pages in iomap_swapfile_activate").

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Ritesh Harjani <riteshh@linux.ibm.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agogeneric: test for file loss after mix of rename, fsync and inode eviction
Filipe Manana [Tue, 27 Jul 2021 10:24:59 +0000 (11:24 +0100)]
generic: test for file loss after mix of rename, fsync and inode eviction

Test that if we fsync a directory A, evict A's inode, move one file
from directory A to directory B, fsync some other inode that is not
directory A, B or any inode inside these two directories, and then
power fail, the file that was moved is not lost.

This currently fails on btrfs and is fixed by commit ecc64fab7d49
("btrfs: fix lost inode on log replay after mix of fsync, rename and
inode eviction")

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
5 months agotools: add a dm-logwrites replay tool
Dave Chinner [Wed, 21 Jul 2021 00:13:33 +0000 (10:13 +1000)]
tools: add a dm-logwrites replay tool

Trying to decipher a dm-logwrites replay failure from generic/482 is
difficult. The test destroys all the dm state and devices when the
test fails, leaving nothing behind for post-mortem analysis. This
script is intended to allow replay of the dm-logwrites device one
FUA at a time similar to generic/482, and is used in conjunction
with a modifed g/482 test that does not tear down the dm volumes
or state when the test fails.

This allows the developer to replay the logwrites up to just prior
to failure to examine just the differences between the last good
state and the first failure state, greatly reducing the complexity
of analysing failures.

Instructions for use are in the tools/dm-logwrite-replay script
itself.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
5 months agooverlay: add test for copy up of lower file attributes
Amir Goldstein [Thu, 22 Jul 2021 16:46:34 +0000 (19:46 +0300)]
overlay: add test for copy up of lower file attributes

Overlayfs copies up a subset of lower file attributes since kernel
commits:
173ff5c9ec37 ("ovl: consistent behavior for immutable/append-only inodes")
2e3f6e87c2b0 ("ovl: copy up sync/noatime fileattr flags")

This test verifies this functionality works correctly and that it
survives power failure and/or mount cycle.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
6 months agooverlay/077: check for inconsistent d_ino/st_ino
Amir Goldstein [Thu, 22 Jul 2021 17:02:16 +0000 (20:02 +0300)]
overlay/077: check for inconsistent d_ino/st_ino

After unlink of a directory entry, that entry may still apear in getdents
results of an already open directory fd, but it should return a d_ino
value that is consistent with the already observed st_ino of that entry.

Remove redundant break condition from gendents read loop.

For testing of inconsistent d_ino/st_ino we need to unlink an entry
whose st_ino is not that of the upper inode.

In the former merge dir setup we unlink all the files in the lower
dir after copyup, so they all use st_ino of the upper inode.

Let the unlinked file f100 reside in a lower path that is not being
unlinked so it will have the st_ino of the lower inode.

This is a regression test for kernel commit fcb7f373684d
("ovl: skip stale entries in merge dir cache iteration")

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>