git.apps.os.sepia.ceph.com Git - xfstests-dev.git/log

common/encrypt: create named variables for UAPI constants

Use named variables rather than hard-coded numbers + comments.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fscrypt-crypt-util: add HKDF context constants

Use #defines rather than hard-coded numbers + comments.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fscrypt-crypt-util: create key_and_iv_params structure

In preparation for adding 3 more input parameters to get_key_and_iv(),
create a structure to hold the input parameters so that the code doesn't
get too unwieldy.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/386: fill missing brackets in awk

Since commit ea726c4a55c98 to check the correct field from df output,
it introduced a syntax error. Fix it.

Reported-by: Feiyu Zhu <zhufy.jy@cn.fujitsu.com>
Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs/15[78]: Use proper helper to get both devid and physical offset for corruption

[BUG]
When using btrfs-progs v5.4, btrfs/157 and btrfs/158 will fail:

btrfs/157 1s ... - output mismatch (see xfstests/results//btrfs/157.out.bad)
    --- tests/btrfs/157.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/157.out.bad
2019-12-10 15:35:43.112390076 +0000
    @@ -1,9 +1,9 @@
     QA output created by 157
     wrote 131072/131072 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    +wrote 65536/65536 bytes at offset 22020096
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    ...
    (Run 'diff -u xfstests/tests/btrfs/157.out xfstests/results//btrfs/157.out.bad'  to see the entire diff)
btrfs/158 2s ... - output mismatch (see xfstests/results//btrfs/158.out.bad)
    --- tests/btrfs/158.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/158.out.bad
2019-12-10 15:35:44.844388521 +0000
    @@ -1,9 +1,9 @@
     QA output created by 158
     wrote 131072/131072 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    +wrote 65536/65536 bytes at offset 22020096
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    ...
    (Run 'diff -u xfstests/tests/btrfs/158.out xfstests/results//btrfs/158.out.bad'  to see the entire diff)

[CAUSE]
This two tests use physical offset as golden output, while mkfs.btrfs
can do whatever it likes to arrange its chunk layout, thus physical
offset is never reliable.

And btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") just changed the layout.

So the output mismatch and failed.

[FIX]
In fact, that btrfs-progs commit not only changed offset, but also the
device sequence.

So we can't just simply remove the physical offset, but also need to use
proper helper to get both devid (as its device path) and physical offset
for corruption.

As long as mkfs.btrfs still uses sequential devid, these tests should
handle future chunk layout change without problem.

Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs/14[23]: Use proper help to get both devid and physical offset for corruption.

[BUG]
When using btrfs-progs v5.4, btrfs/142 and btrfs/143 will fail:
btrfs/142 1s ... - output mismatch (see xfstests/results//btrfs/142.out.bad)
    --- tests/btrfs/142.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/142.out.bad
2019-12-10 15:35:40.280392626 +0000
    @@ -3,37 +3,37 @@
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
     wrote 65536/65536 bytes
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    -XXXXXXXX:  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
    ...
    (Run 'diff -u xfstests/tests/btrfs/142.out xfstests/results//btrfs/142.out.bad' to see the entire diff)

[CAUSE]
Btrfs/14[23] test whether a read on corrupted stripe will re-silver
itself.
Such test by its nature will need to modify on-disk data, thus need to
get the btrfs logical -> physical mapping, which is done by near
hard-coded lookup function, which rely on certain stripe:devid sequence.

Recent btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") changes how we use devices in mkfs.btrfs,
this caused a change in chunk layout, and break the hard-coded
stripe:devid sequence.

[FIX]
This patch will do full devid and physical offset lookup, instead of old
physical offset only lookup.

The only assumption made is, mkfs.btrfs assigns devid sequentially for
its devices.
Which means, for "mkfs.btrfs $dev1 $dev2 $dev3", we get devid 1 for $dev1,
devid 2 for $dev2, and so on.

This change will allow btrfs/14[23] to handle even future chunk layout
change. (Although I hope this will never happen again).

This also addes extra debug output (although less than 10 lines) into
$seqres.full, just in case when layout changes and current lookup can't
handle it, developer can still pindown the problem easily.

Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common: Use more accurate kernel config for _require_fail_make_request

Just enabling CONFIG_FAIL_MAKE_REQUEST will not fulfill
_require_fail_make_request.

It's CONFIG_FAULT_INJECTION_DEBUG_FS.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

src/fssum: skip subvolumes when building a sum

With the snapshot/subvolume support added to fsstress I've been seeing
random failures with our send/receive related tests.  This is because
fssum is summing the path with the subvolumes for our test fs'es that
are generated by fsstress.  But with send/receive it skips subvolumes,
which makes the sums mismatch.  Fix this by skipping directories that do
not match our st_dev, which is how we differentiate subvolumes in btrfs.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: allow operations to use either a directory or subvol

Most operations are just looking for a base directory to generate a file
in, they don't actually need a directory specifically. Add FT_ANYDIR to
cover both directories and subvolumes, and then use this in all the
places where it makes sense.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: add the ability to create snapshots

Snapshots are just fancy subvolumes, add this ability so we can stress
snapshot creation. We get the deletion with SUBVOL_DELETE.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: add the ability to create/delete subvolumes

This patch adds support to fsstress for creating and deleting subvolumes
on a btrfs file system. We link in the libbtrfsutil library to handle
the mechanics of creating and deleting subvolumes instead of duplicating
the ioctl logic. There is code to check if we're on a btrfs fs at
startup time and if so 0 out the frequency of the btrfs specific
operations.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/386: check the correct field from df output

The generic/386 test was checking the "Available" field when it
should have been checking the "1k-blocks" field, which represents
the project quota's hard limit.  On xfs, an empty directory takes no
space, so it doesn't matter.  But for ext4, an empty directory still
takes 4k (or whatever the file system's block size happens to be):

Filesystem           1K-blocks       Used  Available  Use% Pathname
/dev/vdc                512000          4     511996    0% /vdc/test

This causes generic/386 to falsely fail.

This fix is needed to fix generic/386 from failing on ext4:

    hard limit 524283904 bytes, expected 524288000

[Eryu: Used $(NF-4) instead of $2 as Eric Sandeen and Yang Xu
suggested]

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/rc: fix device size parser

If you have multiple devices that share the same regex (eg dm-1, dm10),
then _get_device_size() can return "$size\n$size" which causes the
following error for btrfs/011.

QA output created by 011
./common/rc: line 3084: [: too many arguments

So, fix this by making grep check against whole word.

Signed-off-by: Dennis Zhou <dennis@kernel.org>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

Revert "fstests: Don't use gawk's strtonum"

This reverts commit 37520a314bd472ed720ed0611c6b69e418be9b61.

This commit has been reported to regress, at least, xfs/139 and
btrfs/09[58]. Let's revert it for now.

Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs/187: require 8GB scratch dev

In my testing on 1GB zram devices btrfs/187 usually fails with
ENOSPC.

Add a requirement for 8GB scratch devices (empirically measured).

Cc: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/590: add _require_xfs_io_command "falloc"

Without this check the test will fail with message:

fallocate: Operation not supported

Instead, with this _require_xfs_io_command, the test will be skipped
instead if falloc isn't supported.

Signed-off-by: Luis Henriques <lhenriques@suse.com>
Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test splice() with pipes

Andreas Grünbacher reports that on the two filesystems that support
iomap directio, it's possible for splice() to return -EAGAIN (instead of
a short splice) if the pipe being written to has less space available in
its pipe buffers than the length supplied by the calling process.

This is a regression test to check for correct operation. Kernel
needs commit 3253d9d09337 ("splice: only read in as much information
as there is pipe buffer space") to pass.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/148: sort attribute list output

Yang Xu reported a test failure in xfs/148 that I think comes from
extended attributes being returned in a different order than they were
set. Since order isn't important in this test, sort the output to make
it consistent.

Reported-by: Yang Xu <xuyang2018.ky@cn.fujitsu.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fstests: Don't use gawk's strtonum

It's a gawk extension and not needed. Make tests pass with non-gawk.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: truncating mixed written/unwritten XFS realtime extent

The only XFS-specific part of this test is the setup, so we can make
the rest a generic test. It's slow, though, as it needs to write 8GB
to convert a big unwritten extent to written.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/{075,122}: Use $RESULT_DIR instead of $here

If $here is not writable, tests fail.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/rc: Prevent _scale_fsstress_args from consuming -n

If first argument is -n, echo consumes it.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/386: Fix _filter_quota_rpt

It outputs nothing because of awk errors.

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test journal/log replay when a file with cloned extents was fsync'ed

Test that if we clone part of an extent from a file to itself at a
different offset, fsync it, rewrite (COW) part of the extent from the
former offset, fsync it again, power fail and then mount the filesystem,
we are able to read the whole file and it has the correct data.

This is motivated by a bug found in btrfs which is fixed by a kernel patch
that has the following subject:

"Btrfs: fix missing data checksums after replaying a log tree"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/551: Adjust the size argument for aio-dio-write-verify

generic/551 sometimes fails because it's killed by OOM killer.
That is because aio-dio-write-verify, which is called by
generic/551, tries to allocate memory even though the total
allocation size exceeds the available memory.

aio-dio-write-verify allocates memory according to the 'size'
argument, and generic/551 passes the argument.

Stop adding the argument when the total size is exceeds
the available memory.

Signed-off-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test unwritten extent conversion extent mapping quota accounting

Regression test to ensure that dquots are attached to the inode when
we're performing unwritten extent conversion after a directio write
and the extent mapping btree splits.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test race between appending AIO DIO and fallocate

Dave Chinner reports[1] that an appending AIO DIO write to the second
block of a zero-length file and an fallocate request to the first block
of the same file can race to set isize, with the user-visible end result
that the file size is set incorrectly to one block long. Write a small
test to reproduce the results.

[1] https://lore.kernel.org/linux-xfs/20191029100342.GA41131@bfoster/T/

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fstests: Allow user to keep $seqres.dmesg for all tests

Currently fstests will remove $seqres.dmesg if nothing wrong
happened. It saves some space, but sometimes it may not provide
good enough history for developers to check.

For example, some unexpected dmesg from fs, but not serious enough
to be caught by current filter.

So instead of deleting the ordinary $seqres.dmesg, provide a new
config: KEEP_DMESG, to allow user to keep the dmesg by setting it to
"yes".

The default value for it is "no", which keeps the existing behavior
by deleting ordinary dmesg.

[Eryu: change it to a "yes"/"no" switch.]

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/050: fix xfsquota configuration failures

The new 'xfsquota' configuration for generic/050 doesn't filter out
SCRATCH_MNT properly and seems to be missing an error message in the
golden output. Fix both of these problems.

Fixes: e088479871 ("generic/050: Handle xfs quota special case with different output")
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/191-input-validation:Fix issue that the test takes too long

After this test is passed, checking filesystems will execute. Then
xfs_logprint will be failed and xfs_repair will take a long time
because there is not really creating the file system in this test
and wipefs executes before this test. So there is no need checking
filesystems.

Signed-off-by: Baihua Lu <lubaihua0331@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/558: require scratch device

This test formats the scratch device, so require it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs: test fsync after hole punching when using the no-holes feature

Test that when we have the no-holes feature enabled and a specific
metadata layout, if we punch a hole that starts at file offset 0 and
fsync the file, after replaying the log the hole exists.

This currently fails on btrfs but is fixed by a patch for the linux
kernel with the following subject:

"Btrfs: fix missing hole after hole punching and fsync when using NO_HOLES"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs/085: check flakey device instead of backend device

btrfs/085 mount and works on $FLAKEY_DEV, but it's inspecting the
underlying $SCRATCH_DEV. Since writing to dm-flakey does not
invalidate page cache of $SCRATCH_DEV, the btrfs command can read a
stalled data from page cache. We should check the flakey device to
avoid such cache inconsistency.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

ext4/031: use _try_scratch_mount instead of _scratch_mount

Here will check the return code of mount option. So update it
to use _try_scratch_mount.

Signed-off-by: Xiaoli Feng <xifeng@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/563: fix the wrong case number in golden image

The first line of generic/563.out is wrong, the number should be
563, not 011. This mismatch will always cause testing fail.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

overlay: support timestamp range check

Overlayfs timestamp range is the same as base fs timestamp range

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Acked-by: Deepa Dinamani <deepa.kernel@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/050: Consistently remove traling dot from umount output

We did not consistently remove trailing dot from umount output which
can presumably lead to false failures with particular versions of
util-linux. Make sure all umount output is properly filtered.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/050: Handle xfs quota special case with different output

Instead of faking output for the case of XFS with quotas, just use a
different output file with appropriate output.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/235: Fix false failure on ext2

The test gives false failure on ext2 filesystem as 64k file already
has indirect block and so space usage does not exactly match
expectation.

The test really needs to verify only whether quota accounting got
reenabled so just test using creating another empty file which is
not prone to these problems.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/382: Make test independent of fs allocation

Test generic/382 depends on filesystem allocating exactly 30M of blocks
when writing 30M file. This is not true for some filesystems - e.g. for
ext2 due to indirect blocks - while leads to false positive failures.
In this case, the test is not actually interested in comparing exact
usage, rather in verifying the ability to write 30M worth of data. So
instead of comparing 'xfs_quota report' output, just depend on detecting
error when writing files.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/050: Fix test failure for filesystems without journal

Filesystems without journal can happily mount unrecovered filesystem
read-only which confuses this test. Handle this by providing
different expected output for filesystems without journal.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/quota: Fix _qmount_options

_qmount_options didn't properly substitute usrjquota and grpjquota
options. Fix the regular expression.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: show the real file id and parid in rmdir() and unlink()

The source file id and parentid are overwritten by del_from_flist()
call, so should show the actually values.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs: test that send can issue clone operations within the same file

Verify that both full and incremental send operations can issue
clone operations when a file has extents that are shared with itself
(at different offsets of course).

This currently fails on btrfs but is addressed by a patch for the
kernel titled:

"Btrfs: send, allow clone operations within the same file"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common: add helper to count number of exclusive extents in a file

Add a new helper that is similar to _count_extents() except that
extents that are shared several times by the file with itself
(reflinked at different file offsets) are accounted 1 time only,
instead of N times.

This is motivated by a subsequent test for btrfs that will use this
new helper, to verify that send streams are issuing reflink
operations.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common: open files in ro mode for extent and hole count helpers

The helper functions _count_extents() and _count_holes() open their
input file in RW mode to call fiemap, however opening it in RO mode
is enough. By opening them in RW mode it also makes it not possible
to use them against files residing in btrfs readonly snapshots for
example.

So just open the files in RO mode in these functions.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

overlay/066: adjust test file size && add more test patterns

Making many small holes in 10M test file seems not very
helpful for test coverage and it takes too much time on
creating test files. In order to improve test speed we
adjust test file size to (10 * iosize) for iosize aligned
hole files and meanwhile add more test patterns for small
random holes and small empty file.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Chengguang Xu <cgxu519@mykernel.net>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/207: explicitly test for xflag character

With a recent change to xfs_io[1], the fsxattr.xflags output line
has added another flag column, so xfs/207 starts failing.

Rather than testing for an exact set of flags, test whether the
specific flag we are interested in is set or unset.

[1] xfs_io/lsattr: expose FS_XFLAG_HASATTR flag

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs: make sure the kernel and repair tools catch bad names

Make sure we actually catch bad names in the kernel.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/317: Use relative paths to avoid traversal permission problems

generic/317 will fail because execvp(cmd) is executed without permission,
where cmd is '$here/src/lstat64 $file', which is called by
  $here/src/nsexec -s -U -M "0 $qa_user_id 1000" -G "0 $qa_user_id 1000"\
  $here/src/lstat64 $file

So, you will see following output:
  From user_ns
  ...
  +execvp: Permission denied

nsexec runs the instruction '$here/src/lstat64 $file' as a regular user,
the regular user may not have permission to access path in '$here'.

Actually, it has been fixed in 4818302fbf ("xfstests: generic/317 use
relative paths..."), which then been modified by b7cecbea22 ("fstests:
Add path $here before src/<file>").

Fixes: b7cecbea22 ("fstests: Add path $here before src/<file>")
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/rc: generalize _get_filesize()

There are some testcases use below two kind of commands to get file
size, generalize the second way as global function _get_filesize()
to simply codes.

1. ls -l $1 | $AWK_PROG '{print $5}'
2. stat -c %s $1

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: handle fs.verity.require_signatures being enabled

Most of the fs-verity tests fail if the fs.verity.require_signatures
sysctl has been set to 1. Update them to set this sysctl to 0 at the
beginning of the test and restore it to its previous value at the end.

generic/577 intentionally sets this sysctl to 1. Make it restore the
previous value at the end of the test rather than assuming it was 0.

Also simplify _require_fsverity_builtin_signatures() to just check for
the presence of the file /proc/sys/fs/verity/require_signatures rather
than check whether the fs-verity keyring is listed in /proc/keys.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test the deadlock between the AGI and AGF with RENAME_WHITEOUT

There is ABBA deadlock bug in XFS between the AGI and AGF when
performing rename() with RENAME_WHITEOUT flag, and add this testcase
to make sure the rename() call works well.

Though this is a xfs-specific bug, the reproducer has no
xfs-specific part.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: add EXCHANGE renameat2 support

Support the EXCHANGE renameat2 syscall in fsstress.

In order to maintain filelist/filename integrity, we restrict
rexchange to files of the same type.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: add NOREPLACE and WHITEOUT renameat2 support

Support the renameat2(NOREPLACE and WHITEOUT) syscall in fsstress.

The fent id correlates with filename and the filename correlates
to type in flist, and the RWHITEOUT operation would leave a dev
node around with whatever the name of the source file was, so in
order to maintain filelist/filename integrity, we should restrict
RWHITEOUT source file to device nodes.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fsstress: show the real file id and parid in rename_f()

The source file id and parentid are overwritten by del_from_flist()
call, and should show the actually values.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/rc: fix _require_xfs_io_command utimes check

There is an error on checking utimes command. Put quotation mark
after four "0".

Signed-off-by: Baihua Lu <lubaihua0331@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: verify ciphertext of v2 encryption policies with Adiantum

Verify ciphertext for v2 encryption policies that use Adiantum to
encrypt file contents and file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: verify ciphertext of v2 encryption policies with AES-128

Verify ciphertext for v2 encryption policies that use AES-128-CBC-ESSIV
to encrypt file contents and AES-128-CTS-CBC to encrypt file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: verify ciphertext of v2 encryption policies with AES-256

Verify ciphertext for v2 encryption policies that use AES-256-XTS to
encrypt file contents and AES-256-CTS-CBC to encrypt file names.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: add test for non-root use of fscrypt API additions

Test non-root use of the fscrypt filesystem-level encryption keyring and
v2 encryption policies.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: add basic test for fscrypt API additions

Add a basic test of the fscrypt filesystem-level encryption keyring and
v2 encryption policies.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/encrypt: support verifying ciphertext of v2 encryption policies

Update _verify_ciphertext_for_encryption_policy() to support v2
encryption policies.

This also required adding HKDF-SHA512 support to fscrypt-crypt-util.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/encrypt: support checking for v2 encryption policy support

Allow passing '-v 2' to _require_scratch_encryption() to check for
v2 encryption policy support on the scratch device, and for xfs_io
support for setting and getting v2 encryption policies.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/encrypt: add helper functions that wrap new xfs_io commands

Wrap the new xfs_io commands 'add_enckey', 'rm_enckey', and
'enckey_status' with helper functions.

Also add _user_do_get_encpolicy(), so that all encryption xfs_io
commands have a _user_do() version.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/encrypt: disambiguate session encryption keys

Rename the helper functions that add/remove keys from the session
keyring, in order to distinguish them from the helper functions I'll
be adding to add/remove keys from the new filesystem-level keyring.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: add an fs-verity stress test

Add a stress test for fs-verity. This tests enabling fs-verity on
multiple files concurrently with concurrent readers on those files
(with reads occurring before, during, and after the fs-verity
enablement), while fsstress is also running on the same filesystem.

I haven't seen any failures from running this on ext4 and f2fs.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/rc: skip encrypted direct I/O tests on f2fs

Skip the O_DIRECT tests on f2fs when the test_dummy_encryption mount
option is given, for the same reason as given for ext4 in commit
9b154b26e4a1 ("common/rc: ext4 doesn't support O_DIRECT with
encryption").

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs: trim adjacent extents across adjacent block groups boundary

The test case checks if btrfs can trim adjacent extents across
adjacent block groups boundary correctly.

The test case craft the following extents layout:

         |  BG1 |      BG2        |       BG3            |
Bytenr: X-8M   X      X+512M     X+1G  X+1G+128M
         |//////|//////|          |     |//|

There is a long existing bug that, when btrfs is trying to trim the
range at [X+512M, X+1G+128M), it will only trim [X+512M, X+1G).

This test case is the regression test for this long existing bug.

It will verify the trimmed bytes by using loopback device backed up
by a file, and checking the bytes used by the file to determine how
many bytes are trimmed.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/{569,570}: use _filter_xfs_io_error to filter pwrite64

xfsprogs prior to commit 2f9a125c3a39 ("xfsprogs: replace
pread64/pwrite64 by equivalent pread/pwrite") will see "pwrite64:
Text file busy" error. Just filter these "pwrite64"s to "pwrite"
with _filter_xfs_io_error.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/casefold: support f2fs

Now, f2fs has ported casefold feature from ext4, let's enable the
testcase.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/defrag: support f2fs

F2FS supports defragment based on file granularity, adjust test suit
to support f2fs.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs: test for alien devices

Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians no btrfs magic string.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs: test for alien btrfs-devices

Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians someother btrfs.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

src/log-writes: Add new discard check point

Despite the existing <number>|fua|flush checkpoint, add a new
discard check point to make sure discard is not screwing up things.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/402: fix for updated behavior of timestamp limits

The mount behavior will not be altered because of the unsupported
timestamps on the filesystems.

Adjust the test accordingly.

You can find the series at
https://git.kernel.org/torvalds/c/cfb82e1df8b7c76991ea12958855897c2fb4debc

Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

overlay: copy-up test for variant sparse files

This is intensive copy-up test for sparse files, these cases will be
mainly used for regression test of copy-up improvement for sparse
files.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Chengguang Xu <cgxu519@mykernel.net>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/435: disable dmesg checks

This test corrupts the filesystem to see what assertions and other
things to get logged, so don't treat that as a failure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fstests: Add path $here before src/<file>

Calling src/<file> without path '$here' may cause the problem that
the file cannot be found.

For example, Running generic/192 with overlayfs(Let ubifs as base
fs) yields the following output:

  generic/192 - output mismatch
     QA output created by 192
     sleep for 5 seconds
     test
    +./common/rc: line 316: src/t_dir_type: No such file or directory
     delta1 is in range
     delta2 is in range
    ...

When the use case fails, the call stack in generic/192 is:

  local unknowns=$(src/t_dir_type $dir u | wc -l) common/rc
  _supports_filetype common/rc
  _overlay_mount common/overlay
  _overlay_test_mount common/overlay
  _test_mount common/rc
  _test_cycle_mount generic/192

Before _test_cycle_mount() being invoked, generic/192 executed 'cd
/' to change work dir from 'xfstests-dev' to '/', so src/t_dir_type
was not found.

[Eryu: some tests run src/<file> as regular user, don't add $here
prefix in such case, as a regular user may have no search permission
on $here]

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/097: Remove wrong broken assignment operation

On old kernel, since commit ded188b8609 ("xfs: Fix the situation
that mount operation rejects corrupted XFS") running this case got
the mismatched output, as below:
-----------------------------------
+ check fs
+ corrupt image
+ mount image && modify files
-broken: 1
+broken: 0
+ repair fs
+ mount image (2)
------------------------------------

It fails because the broken is always equal to 0 when
_try_scratch_mount succeed. So remove this wrong assignment
operation.

Signed-off-by: Yang Xu <xuyang2018.jy@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/448: define BASE_TEST_FILE before _cleanup

NFS v4.2 supports SEEK_DATA and SEEK_HOLE, but earlier versions do
not. As a result, the test exits and runs the cleanup function
without the $BASE_TEST_FILE variable set and the shell expands it to
"rm -f .*", deleting all hidden files in the current directory.

Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/035: Create new outfile for NFS

Rename-overwrites over NFS work with a "silly rename" over the
network, so the nlink count stays at 1 instead of dropping to 0.
This is expected behavior for NFS, so we should use a different
golden output file to account for this.

See the NFS faq at: nfs.sourceforge.net/#faq_d2 for more
information about silly renames.

Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

fstests: Add virtio-fs shared file system support

This patch adds support for virtio-fs shared file system that lets
virtual machines access a directory tree on the host.

To run xfstests on it, first, start virtiofsd daemon in host:
./virtiofsd -o vhost_user_socket=/tmp/vhostqemu0 -o source=$DIR0 -o cache=always
./virtiofsd -o vhost_user_socket=/tmp/vhostqemu1 -o source=$DIR1 -o cache=always

second, launch QEMU with:
-chardev socket,id=char0,path=/tmp/vhostqemu0
-device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs0
-chardev socket,id=char1,path=/tmp/vhostqemu1
-device vhost-user-fs-pci,queue-size=1024,chardev=char1,tag=myfs1
-m 8G
-object memory-backend-file,id=mem,size=8G,mem-path=/dev/shm,share=on
-numa node,memdev=mem

then, inside the VM run xfstests with:
export TEST_DEV=myfs0
export TEST_DIR=$TESTDIR
export SCRATCH_DEV=myfs1
export SCRATCH_MNT=$SCRATCHMNT
export MOUNT_OPTIONS=""
export TEST_FS_MOUNT_OPTS=""

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Misono Tomohiro <misono.tomohiro@fujitsu.com>
Signed-off-by: QI Fuli <qi.fuli@fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: check reflink multiple mmap write

Add a test to make sure that we can handle multiple memory mappings to a
physical storage extent shared by multiple files, and that we can handle
the copy on write operation without error. Make sure we can also handle
mappings at different offsets in the page cache.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

populate: punch files after writing to fragment free space properly

The filesystem population code frequently allocates a large file and
punches out every other block ("swiss-cheese files") in an attempt to
cause the creation of a lot of metadata to fill out the btrees.  This
pattern, however, has a subtle bug if the writes to the swiss-cheese
file are not allocated in batches and we're trying to fragment the free
space records in order to achieve a certain metadata btree shape.

This is exactly what happens on a DAX filesystem, since we no longer
have the page cache to stage delalloc writes.  Each xfs_io pwrite call
to the multi-megabyte swiss-chese file turns into multiple 4k pwrites,
which means that file data blocks are allocated 4k at a time.  This can
be fatal to our goal of fragmenting the free space btrees because the
allocator sees a 4k allocation request and uses 4k blocks from the
fragmented parts of the free space to satisfy the "small" request.  When
this happens, the XFS populate function cannot fill out the free space
btree to sufficient height and tests fail.

(In regular delalloc mode we'd cache all those small write() in memory
and try for a single large allocation, which we'd generally get.)

To fix this, we need to force the filesystem to allocate all blocks
before freeing any blocks.  Split the creation of swiss-cheese files
into two parts: (a) writing data to the file to force allocation, and
(b) punching the holes to fragment free space.

This bug affects only XFS but we convert the one ext4 usage anyway.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/263: use _scratch_mkfs_xfs instead of open-coded mkfs call

Fix this test to use _scratch_mkfs_xfs instead of the open-coded mkfs
call. This is needed to make the test succeed when XFS DAX is enabled
and mkfs enables reflink by default.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/{088, 089, 091}: redirect stderr when writing to corrupt fs

These tests primarily check that writes to a corrupt fs don't take down
the system, and that running repair will fix them. Therefore, redirect
stderr to seqres.full so that we don't fail these tests in DAX mode.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

xfs/196: check for delalloc blocks after pwrite

This test depends on the pwrite creating delalloc blocks, which doesn't
happen if the scratch fs is mounted in dax mode (or has an extent size
hint applied). Therefore, check for delalloc blocks and _notrun if we
didn't get any.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

src/renameat2: remove duplicated renameat2 system call number for i386

Remove duplicated renameat2 system call number for i386.

Signed-off-by: kaixuxia <kaixuxia@tencent.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic/001: remove unnecessary backslash

Awk 5.0.1 is complaining about this backslash and causing false alarm
like this:
warning: regexp escape sequence '\#' is not a known regexp operator

This fix works fine with Awk 3.1.7 , 4.0.2 , 4.2.1 and 5.0.1. This
backslash is not necessary.

Signed-off-by: Murphy Zhou <jencce.kernel@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

btrfs: add a test for multi-subvolume fsyncing

I discovered a problem in btrfs where we'd end up pointing at a block we
hadn't written out yet. This is triggered by a race when two different
files on two different subvolumes fsync. This test exercises this path
with dm-log-writes, and then replays the log at every FUA to verify the
file system is still mountable and the log is replayable.

This test is to verify the fix

btrfs: fix incorrect updating of log root tree

actually fixed the problem.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test the fs-verity built-in signature verification support

Add a basic test for the fs-verity built-in signature verification
support, which is an optional feature where the kernel can be configured
to enforce that all verity files are accompanied with a valid signature
by a key that has been loaded into the fs-verity keyring.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test using fs-verity and fscrypt simultaneously

This primarily verifies correct ordering of the hooks for each feature:
fscrypt needs to be first.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test that fs-verity is using the correct measurement values

This test verifies that fs-verity is doing its Merkle tree-based hashing
correctly, i.e. that it hasn't been broken by a change.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test corrupting verity files

This test corrupts various parts of the contents of a verity file, or
parts of its Merkle tree, by writing directly to the block device. It
verifies that this causes I/O errors when the relevant part of the
contents is later read by any means.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test access controls on the fs-verity ioctls

Test access controls on the fs-verity ioctls. FS_IOC_MEASURE_VERITY is
allowed on any file, whereas FS_IOC_ENABLE_VERITY requires write access.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

generic: test general behavior of verity files

This is a basic fs-verity test which verifies:

- conditions for enabling verity
- verity files have expected contents and size
- can't change contents of verity files, but can change metadata
- can retrieve a verity file's measurement via FS_IOC_MEASURE_VERITY

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/verity: add common functions for testing fs-verity

Add common functions for setting up and testing fs-verity.  fs-verity is
an ext4 and f2fs filesystem feature which provides Merkle tree based
hashing (similar to dm-verity) for individual read-only files, mainly
for the purpose of efficient authenticity verification.  Other
filesystems may add fs-verity support later, using the same API.

Running the fs-verity tests will require:

- Kernel v5.4-rc1 or later configured with CONFIG_FS_VERITY=y

- The 'fsverity' utility program from
  https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git

- e2fsprogs v1.45.2 or later for ext4 tests, or f2fs-tools v1.11.0 or
  later for f2fs tests

Some individual tests will have additional requirements.

For more information about fs-verity, see
https://www.kernel.org/doc/html/latest/filesystems/fsverity.html

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>

common/filter: add _filter_xfs_io_fiemap()

Add _filter_xfs_io_fiemap() to clean up the output of
'xfs_io -c fiemap'. This will be used by a function in common/verity.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>