lib/tlibio: Fix a build warning

Remove the following warning message.

  ===============================
  tlibio.c: In function 'lio_set_debug':
  tlibio.c:212:1: warning: type of 'level' defaults to 'int' [-Wimplicit-int]
   lio_set_debug(level)
   ^
  ==============================

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

overlay/001: change supported fs from generic to overlay

The supported fs was set to generic by mistake, so fix it by setting it
to the proper type 'overlay'.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

check: catch general protection fault traces in _check_dmesg

While writing a test case for overlayfs I noticed that _check_dmesg()
did not detect general protection fault traces like the following:

[15277.026108] run fstests overlay/002 at 2016-03-21 18:42:27
[15277.933179] general protection fault: 0000 [#11] PREEMPT SMP DEBUG_PAGEALLOC
[15277.935427] Modules linked in: btrfs overlay ppdev crc32c_generic evdev xor raid6_pq psmouse pcspkr sg serio_raw acpi_cpufreq parport_pc parport tpm_tis i2c_piix4
[15277.936011] CPU: 15 PID: 22762 Comm: xfs_io Tainted: G      D         4.5.0-rc6-btrfs-next-26+ #1
[15277.936011] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS by qemu-project.org 04/01/2014
[15277.936011] task: ffff88001f6f1340 ti: ffff88000a3f8000 task.ti: ffff88000a3f8000
[15277.936011] RIP: 0010:[<ffffffffa030b7ab>]  [<ffffffffa030b7ab>] btrfs_sync_file+0x11b/0x3e9 [btrfs]
[15277.936011] RSP: 0018:ffff88000a3fbe40  EFLAGS: 00010286
[15277.936011] RAX: 0000000080000000 RBX: ffff8800b0152c88 RCX: 0000000000000001
[15277.936011] RDX: 0000000000000001 RSI: ffffffff8148fec0 RDI: 00000000ffffffff
[15277.936011] RBP: ffff88000a3fbec0 R08: 0000000000000001 R09: 0000000000000000
[15277.936011] R10: ffff88000a3fbe40 R11: 0000000000000246 R12: 6e756f635f657269
[15277.936011] R13: 0000000000000000 R14: 00000000014655a0 R15: ffff88017dd25e40
[15277.936011] FS:  00007f27f0e1f700(0000) GS:ffff88023ede0000(0000) knlGS:0000000000000000
[15277.936011] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15277.936011] CR2: 00007f27f0e2f000 CR3: 000000013aadf000 CR4: 00000000000006e0
[15277.936011] Stack:
[15277.936011]  ffffffff8108b5cc ffff88000a3fbec0 0000000000000246 ffff880125abaed0
[15277.936011]  ffff8800b0152d60 8000000000000000 7fffffffffffffff 0000000000000246
[15277.936011]  0000000000000246 ffffffff81074f9b ffffffff8104357c ffff88017dd25e40
[15277.936011] Call Trace:
[15277.936011]  [<ffffffff8108b5cc>] ? arch_local_irq_save+0x9/0xc
[15277.936011]  [<ffffffff81074f9b>] ? ___might_sleep+0xce/0x217
[15277.936011]  [<ffffffff8104357c>] ? __do_page_fault+0x3c0/0x43a
[15277.936011]  [<ffffffff811a2351>] vfs_fsync_range+0x8c/0x9e
[15277.936011]  [<ffffffff811a237f>] vfs_fsync+0x1c/0x1e
[15277.936011]  [<ffffffff811a24d6>] do_fsync+0x31/0x4a
[15277.936011]  [<ffffffff811a2700>] SyS_fsync+0x10/0x14
[15277.936011]  [<ffffffff81493617>] entry_SYSCALL_64_fastpath+0x12/0x6b
[15277.936011] Code: 85 c0 0f 85 e2 02 00 00 48 8b 45 b0 31 f6 4c 29 e8 48 ff c0 48 89 45 a8 48 8d 83 d8 00 00 00 48 89 c7 48 89 45 a0 e8 fc 43 18 e1 <f0> 41 ff 84 24
[15277.936011] RIP  [<ffffffffa030b7ab>] btrfs_sync_file+0x11b/0x3e9 [btrfs]
[15277.936011]  RSP <ffff88000a3fbe40>
[15277.974598] ---[ end trace 721e14960eb939c1 ]---

So teach it to look out for such traces.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

overlayfs: test fsync on merged directory

Test that calling fsync against a file using the merged directory does
not result in a crash nor fails unexpectedly.

This is motivated by a change in overlayfs that resulted in a crash
(invalid memory access) when the lower or upper directory belonged to
a btrfs file system. The overlayfs change came in commit 4bacc9c9234
(overlayfs: Make f_path always point to the overlay and f_inode to the
underlay). At the moment there are two patches in the linux-fsdevel
and linux-btrfs mailing lists to fix this problem:

  * vfs: add file_dentry()
  * Btrfs: fix crash/invalid memory access on fsync when using overlayfs

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

btrfs: 'subvolume stress' tasks need to exit gracefully

The following scenario can occur when running btrfs/066,

  Task A                                Task B                     Task C

  run_test()
  - Execute _btrfs_stress_subvolume()
    in a background shell.
                                        _btrfs_stress_subvolme()
                                          ...
                                        - fork & exec "mount"
                                                          Mount subvolume on directory in $TEST_DIR
  - Wait for fsstress to finish                                    do_mount()
  - kill shell process executing                                   - btrfs_mount()
    _btrfs_stress_subvolume()
    i.e. Task B.
  - Init process becomes the parent
    of "subvolume mount" task
    i.e. Task C.
  - In case subvolume is mounted
    (which is not the case),
    unmount it.
                                                                   - Complete mounting subvolume

Hence on the completion of one iteration of run_test(), the subvolume
created inside the filesystem on $SCRATCH_DEV continues to be mounted on
$TEST_DIR/$seq.mnt. Subsequent invocations of run_test() (called for
remaining Btrfs profile configs) fail during _scratch_pool_mkfs.

Instead of killing the 'subvolume stress' task this commit makes
_btrfs_stress_subvolume() to break out of the loop when a file exists
on the filesystem. The commit also makes relevant changes to other
users of _btrfs_stress_subvolume() i.e. btrfs/060, btrfs/065,
btrfs/067 & btrfs/068.

Suggested-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/073: avoid finobt warning from mkfs

When testing xfs/073 with MKFS_OPTIONS="-m crc=1,finobt=1" set, it fails
due to extra warning about disabling finobt feature:

 +warning: finobt not supported without CRC support, disabled.

Because xfs/073 disables crc unconditionally and finobt can not be
enabled either.

Fix it by explicitly disabling finobt as well.

Also remove all meta related mkfs options in _scratch_mkfs_xfs_opts() if
mkfs.xfs has no metadata support, not only the crc option. So that test
still passes on distros with such old binaries.

Suggested-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/191: Remove obsolete nfs4acl tests

The nfs4acl tests don't make sense anymore as they have been obsoleted
by richacls.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

Rename output file templates to match TEST.out*

Rename the expected output files so that they match "$TEST_NAME.out*";
this makes the file names a bit more consistent.

Add $TEST_NAME.out and similar symlinks to .gitignore.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

fstests: make xfs/006 generic

xfs/006 has no requirements that are specific to XFS, so make it generic
and other filesystems could get some coverage too.

Along with the movement, I also added a test that removes all created
dirs, as that's how the original bug was found.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

btrfs/059: add a filter for btrfs compression property

btrfs/059.out should not be hardcoded to zlib, if compression method
is lzo, this case will fail wrongly, so here add a filter.

Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic: test I/O on dm error device

This is a test that performs simple I/O on dm error device, which
returns EIO on all I/O request.

This is motivated by an ext4 bug that crashes kernel on error path when
trying to update atime. Following kernel patch should fix the issue

  ext4: fix NULL pointer dereference in ext4_mark_inode_dirty()

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/209: filter scratch dir properly

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs: remove NOCOW_FL testing from test

Since XFS dropped support for the NOCOW_FL flag, cut it out of the tests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs: test per-ag allocation accounting during truncate-caused refcountbt expansion

Ensure that refcountbt allocations during truncate operations come
from the per-AG reservation and are not charged to the transaction.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/122: support rmapxbt

Support the extended rmap btree key structure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/122: define _GNU_SOURCE when compiling test program

This makes it so we don't get warnings about off64_t not being
defined when compiling the test program.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/073: fix output

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs/030: fix output on newer filesystems

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/206: fix output when mkfs knows about reflink

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/23[3-6]: don't source common/xfs, it doesn't exist

Don't source common/xfs, since it doesn't (yet) exist.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

reflink: fix fragmentation tests to work on >4k block size filesystems

For tests that only overwrite part of a file, only consider the number
of extents in the *rewritten* blocks when deciding if the FS
fragmentation performance is satisfactory.

(Also slip in a fix for xfs/127 so that it formats correctly when we
specify big blocksizes via MKFS_OPTIONS.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs: test copy-on-write leftover recovery

Test recovery of CoW leftovers in xfs_repair.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/207: fix golden output to match FS_IOC_FSSETXATTR hoist

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

btrfs/048: user filter for property error messages

Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

filter.btrfs: new filter for property messages

The error message in newer btrfs utilities contains name of the
property, we want to filter it out so it matches golden output. There's
an optional parameter with property name that is matched against the
command output.

Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

btrfs/118: remove call to _need_to_be_root

The function _need_to_be_root does not exists anymore as of commit
56ff01f471c9 ("xfstests: remove _need_to_be_root").

A v2 of the patch that added test btrfs/118 without calling this
function was sent but not picked [1], instead v1 was picked.

So fix this now.

[1] https://patchwork.kernel.org/patch/8354831/

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/137: require scratch to be CRC enabled XFS

This test is only valid for CRC enabled XFS, and test fails with
MKFS_OPTIONS="-m crc=0" being set (no mount failure detected).

Fix it by adding _require_scratch_xfs_crc, which ensures scratch device
has CRC support and enabled.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

ext4: defrag not supported with DAX

As of this kernel commit:

commit 73f34a5e2ced ("ext4: online defrag not supported with DAX")

online defrag operations for ext4 are disallowed when the filesystem is
mounted with the DAX option.

This causes several xfstests to fail because they expect the defrag
operation to change the file layout:

ext4/308  [failed, exit status 1] - output mismatch (see /root/xfstests/results//ext4/308.out.bad)
    --- tests/ext4/308.out 2015-10-02 10:19:36.791795792 -0600
    +++ /root/xfstests/results//ext4/308.out.bad 2016-02-17 16:20:52.330454602 -0700
    @@ -23,659 +23,5 @@
     50f924a5dc9b03609a4577f9f961414b  SCRATCH_MNT/test.10
     Perform compacting
     50f924a5dc9b03609a4577f9f961414b  SCRATCH_MNT/test.10
    -Perform compacting, second pass
    -50f924a5dc9b03609a4577f9f961414b  SCRATCH_MNT/test.10
    -Create file with 20 * 2 fragments
    -wrote 1234/1234 bytes at offset 0
    ...
    (Run 'diff -u tests/ext4/308.out /root/xfstests/results//ext4/308.out.bad'  to see the entire diff)
generic/018 1s ... [failed, exit status 1] - output mismatch (see /root/xfstests/results//generic/018.out.bad)
    --- tests/generic/018.out 2016-02-17 16:02:40.103656140 -0700
    +++ /root/xfstests/results//generic/018.out.bad 2016-02-17 16:20:53.117459173 -0700
    @@ -10,10 +10,6 @@
     After: 1
     Write backwards sync, but contiguous - should defrag to 1 extent
     Before: in_range(5, 10)
    -After: 1
    -Write backwards sync leaving holes - defrag should do nothing
    -Before: 16
    -After: 16
    ...
    (Run 'diff -u tests/generic/018.out /root/xfstests/results//generic/018.out.bad'  to see the entire diff)

Avoid this by skipping over defrag tests if we are using ext4 + DAX.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

common: filter more messages from _scratch_mkfs_ext4

Commit a94334afa2ff ("common: filter warning messages from mkfs.ext4")
filtered warning messages from mke2fs stderr output but missed the
mke2fs version infomation message, and generic/135 still fails because
of this

  QA output created by 135
 +mke2fs 1.42.9 (28-Dec-2013)
  0000000 1212 1212 1212 1212 1212 1212 1212 1212

So also filter out this message in _scratch_mkfs_ext4.

Reported-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

attr: add maximum acl count for f2fs

Add a max acl count into common/attr for f2fs in order to make
f2fs passing generic/026 test case.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

fstests: update autoconf rules to define _GNU_SOURCE where needed

Starting from xfsprogs commit cb898f157f84 ("linux.h: Use off64_t
instead of loff_t") in v4.5-rc1, fstests failed to build because of
configure error:

    checking xfs/xfs.h usability... no
    checking xfs/xfs.h presence... yes
    configure: WARNING: xfs/xfs.h: present but cannot be compiled
    configure: WARNING: xfs/xfs.h:     check for missing prerequisite headers?
    configure: WARNING: xfs/xfs.h: see the Autoconf documentation
    configure: WARNING: xfs/xfs.h:     section "Present But Cannot Be Compiled"
    configure: WARNING: xfs/xfs.h: proceeding with the compiler's result
    checking for xfs/xfs.h... no

    FATAL ERROR: cannot find a valid <xfs/xfs.h> header file.
    Run "make install-qa" from the xfsprogs source.

and that's because off64_t is not defined without _GNU_SOURCE being set.

Fix it by updating the autoconf rules adding _GNU_SOURCE define for the
following checks:

xfs.h libxfs.h xfs_log_format.h xlog_assign_lsn xqm.h xfsctl

Suggested-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

fstests: generic test for listing an inode's xattrs

Test that the filesystem's implementation of the listxattrs system call
lists all the xattrs an inode has.

This test is motivated by a bug found in btrfs, which is fixed by the
following patch for the linux kernel:

  "Btrfs: fix listxattrs not listing all xattrs packed in the same item"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

common: filter warning messages from mkfs.ext4

Commit 4a5cda8102828 ("xfstests: do not unmount tmpfs during remount")
changed generic/135 to no longer redirect stderr to /dev/null when
running _scratch_mkfs.  This caused ext4 using a 1k block size to fail
when running this test because mkfs.ext4 would issue a warning message
about the use of a non-block size that wasn't expected by golden
output for the test.  Fix this by filtering out warning messages in
_scratch_mkfs_ext4.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

defrag: require extents support for ext4 defrag

The e4defrag program requires the use of fallocate, which in turn
means that the file system must have extents.  Enforce this
requirement so we don't get test failure noise when testing ext4's
ext3 compatibility.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/125: add _require_odirect

Also remove generic/125 from the auto group, and add it to the new
pnfs group.  This is to document where this test might be useful; it's
not really going to be useful for most normal on-disk file systems, so
remove it from the auto group.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/312: remove dependency on /proc/partitions

We don't need to do this test at all, since _scratch_mkfs_sized will
do this check for us.   This allows this test to work with tmpfs.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

gitignore: add src/test-nextquota

Commit 1dfb50585c09 (quota: test Q_GETNEXTQUOTA) added a new binary
without updating .gitignore.  Fix this.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

fstests: btrfs/011 add debug logs to seqres.full

Signed-off-by: Anand Jain <anand.jain@oracle.com>

btrfs/011: use replace_options

This patch fixes test btrfs/011 which intended to use -r option
but was never used since its associated args 'replace_options'
didn't make it to the cli.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic: test file fsync after rename operation

Test that if we have a file F1 with two links, one in a directory A and
the other in directory B, if we remove the link in directory B, move some
other file F2 from directory B into directory C, fsync inode F1, power
fail and remount the filesystem, file F2 exists and is located only in
directory C.

This is motivated by a bug found in btrfs, which is fixed by the patch
(for the linux kernel) titled:

   "Btrfs: fix file loss on log replay after renaming a file and fsync"

Tested against ext3, ext4, xfs, f2fs and reiserfs.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic: test directory fsync after rename operation

Test that if we move one file between directories, fsync the parent
directory of the old directory, power fail and remount the filesystem,
the file is not lost and it's located at the destination directory.

This is motivated by a bug found in btrfs, which is fixed by the patch
(for the linux kernel) titled:

  "Btrfs: fix file loss on log replay after renaming a file and fsync"

Tested against ext3, ext4, xfs, f2fs and reiserfs.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

common: fix _require_fs_space on remote file systems

Remove the -l flag to df so that it works properly on NFS and co.

This fixes various failures in new COW tests on NFS.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs/24[356]: check for -c switch to xfs_io bmap command

Extend _require_xfs_io_command so that we can pass it a command line
argument to look for, and then use new capabililty in the relevant
tests to ensure that bmap knows how to dump CoW fork contents (bmap -c)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

punch-alternating: use the block size reported by the fs for punching

When we're trying to punch alternating blocks out of a file, use the
bsize reported by fstatfs so that we can punch out single blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/304: fix high offset

"Invalid argument" is a better response to an impossibly high offset
dedupe request than "extents don't match", so change the test.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

btrfs: test directory fsync after deleting snapshots

Test that if we fsync a directory that had a snapshot entry in it that
was deleted and crash, the next time we mount the filesystem, the log
replay procedure will not fail and the snapshot is not present anymore.

This issue is fixed by the following patch for the linux kernel:

  "Btrfs: fix unreplayable log after snapshot delete + parent dir fsync"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs: new test default user/group quota

When default quota is set, all different quota types inherits the
same default value, include group quota. So if a user quota limit
larger than the default user quota value, it will still be limited
by the group default quota value.

An upstream patch for this bug:

  xfs: Split default quota limits by quota type

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfstests: remove _need_to_be_root

The check script requires that it be run as root, so adding
individualized checks for this in each teat is not needed.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfstests: increase tmpfs memory size

512M is not enough for generic/129.  Raise default tmpfs size to 1G.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfstests: add executable permission to tests

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/079,277: require chattr, not xattrs

Add a new helper, _require_chattr, which allows the test to explicitly
check to see if the file system supports a specific chattr flag, as
not all file systems support chattr +A or chattr +i, and the presence
of extended attribute support is has nothing to do with a specific
chattr flag being supported.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/009: require fiemap

Require xfs_io commands fiemap and falloc as well as fzero: fzero
without falloc is unlikely, but tmpfs may later support fzero, though
probably never fiemap (and in v3.15 wrongly claimed to support fzero).

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/053: do not unmount before calling _check_scratch_fs()

Fix generic/053 so it works on tmpfs by relying on _check_scratch_fs
to unmount before checking the file system and remounting it
afterwards.  Many other tests rely on this, and since tmpfs does not
have a file system consistency checker, this allows the test to
succeed because the files don't disappear when the tmpfs file system
is unmounted.

Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfstests: do not unmount tmpfs during remount

Several tests unmount then re-mount the scratch filesystem, to check
that the content is unchanged; but unmounting a tmpfs is designed to
lose its content, which causes such tests to fail unnecessarily.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

Rename _test_mount to _test_cycle_mount

This makes it clear when we are using "mount ; umount" versus "mount
-o remount" for most file systems.  The reason for this distinction is
(a) tests may want to test the difference between what happens on the
remount versus the munt paths, (b) with tmpfs, "mount ; umount" will
cause the contents of all of the files to disappear which makes many
tests sad, and (c) some mount options may not be changed using "mount
-o remount".

Currently _test_mount performs "_test_mount ; _test_umount"
so mechnically rename this function to _test_cycle_mount.  This was
done mechnically using the script fragment:

git grep -E "_test_remount" | \
awk -F: '{print $1}' | sort -u | grep -v tests/xfs/189 \
xargs sed -i 's/_test_remount/_test_cycle_mount/g'

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

Rename _scratch_mount to _scratch_cycle_mount

This makes it clear when we are using "mount ; umount" versus "mount
-o remount" for most file systems.  The reason for this distinction is
(a) tests may want to test the difference between what happens on the
remount versus the munt paths, (b) with tmpfs, "mount ; umount" will
cause the contents of all of the files to disappear which makes many
tests sad, and (c) some mount options may not be changed using "mount
-o remount".

Currently _scratch_mount performs "_scratch_mount ; _scratch_umount"
so mechnically rename this function to _scratch_cycle_mount.  This was
done mechnically using the script fragment:

git grep "_scratch_remount" | \
awk -F: '{print $1}' | sort -u | \
xargs sed -i 's/_scratch_remount/_scratch_cycle_mount/g'

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic: add _require_odirect to generic/113 and generic/214

generic/113 and generic/214 both use O_DIRECT at some stage in their
tests, so check O_DIRECT support before running them.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic: use mount point instead of device name

A tmpfs mount does not involve any block device, its $SCRATCH_DEV is
nothing but a place-holder, so apply 'df' or 'stat' to its mount point
$SCRATCH_MNT instead of to $SCRATCH_DEV.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

common: _scratch_mkfs_sized() for tmpfs

Enable _scratch_mkfs_sized() for use with tmpfs, so that tests which
use this helper can now run.

Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Junho Ryu <jayr@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

check: avoid spurious complaints that tests/$FSTYP/group does not exist

There are no tmpfs specific tests, so tests/tmpfs does not exist.
This commit avoids printing a spurious error message when running
specifying a group of tests (e.g., "check -g quick"):

  DEVICE: test:/tmp
  MK2FS OPTIONS:
  MOUNT OPTIONS: -o block_validity
  ./check: line 96: tests/tmpfs/group: No such file or directory
  FSTYP         -- tmpfs
  PLATFORM      -- Linux/i686 kvm-xfstests 4.5.0-rc2ext4-00002-g6df2762
  MKFS_OPTIONS  -- test:/scratch
  MOUNT_OPTIONS -- -o size=1G test:/scratch /test/scratch

  generic/001      [10:31:10][    5.811742] run fstests generic/001
      ...

Similar problems have been reported when testing nfs using xfstests.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

ext4/001: add output variant for nodelalloc mounts

If the delayed allocation is disabled, we need a slightly different
output for the delayed allocation portion of the tests.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Jan Kara <jack@suse.cz>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/158: test dedupe with destination offset past EOF

We were testing when the source file offset starts at EOF or beyond,
but not when the destination offset is beyond EOF or when the
destination offset is smaller than EOF but destination offset plus
dedup length is greater than EOF.

This is motivated by a bug in btrfs' extent_same (dedup) ioctl where
we allowed the destination offset to start at EOF and beyond (and
destination offset + length beyond EOF) for the case where the source
and destination files are the same (was not allowed for different
files used as source and destination). This also made the file's
metadata inconsistent when the dedup operation succeeded, which
happened when the source range corresponded to a file hole, prealloc
extent or a data extent filled with zeroes.

The btrfs issue is fixed by the following patch for the linux kernel:

  "Btrfs: fix extent_same allowing destination offset beyond i_size"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
[darrick.wong@oracle.com: fix merge conflicts with latest reflink patchbomb]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

ext4: test partial blocksize defrag integrity issue

Calling EXT4_IOC_MOVE_EXTENT on file not aligned with block size and
block size is smaller than page size would cause integrity issue on the
partial-blocksize part when copying data between orign file and donor
file.

This ext4 kernel patch would fix it, titled
"ext4: don't read blocks from disk after extents being swapped in
move_extent_per_page())"

Though this bug only happens in the blocksize smaller than pagesize
case, there's no harm to test on various block size fs, so no block size
is specified in the test, it depends on the test configurations.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

xfs: change return value check to golden image check

xfs/133 and xfs/138 use too much code to do "return value" check,
it's not necessary. For the code can be more readable and clear,
I change "return value" check to golden image check.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

generic/072: limit max cpu number to 8

Right now generic/072 scales the loop count based on the cpu count. But
on hosts with many cpus(100+), generic/072 runs for hours and generates
very high system load.

Given that the original bug can be reproduced easily on unpatched
kernel, the great number of loops and long run time are not needed. So
limiting the cpu number to 8 (which gives around 20 seconds run time on
my test vm with 8 vcpus) seems reasonable.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>

aiocp: fix the gcc warnings

Eliminate a debug printf that was causing warnings and fix the other two
debug printfs to avoid tripping on return value warnings.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: kick the slow tests out of the 'quick' group

Since 'quick' tests are supposed to run in < 15s, kick out the ones
that can't finish that soon even on fast storage.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: test source CoW across mixed block types with cowextsz set

Ensure that we can CoW the source file when the source file consists
of a range of mixed block types and there's a cowextsize hint set.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: ensure that we don't crash when freeing the ag reservations on a ro mount

Christoph Hellwig discovered that the kernel crashed trying to free
the refcount btree per-ag reservation on a ro mount (because we don't
create the reservation except for rw mounts and ro->rw remounts).  So,
test this to make sure we never do that again. :)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test reflink+cow+enospc all at the same time

Set up an impossibly small filesystem and try to reflink and rewrite a
file on it to see what happens when we ENOSPC.  Basically
generic/16[67] but with a constrained fs size.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: test xfs_getbmapx behavior with shared extents

Make sure that xfs_getbmapx behaves properly w.r.t. shared extents
and CoW fork reporting.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: aio cow tests

Test what happens when AIO writes fail when we have a cowextsize hint
set on the files.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test aio copy on write

Make sure that copy on write works with the AIO path.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: test rmapbt functionality

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: test the automatic cowextsize extent garbage collector

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test CoW across a mixed range of block types with cowextsize set

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test quota accounting

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test xfs cow behavior when the filesystem crashes

Use the extent size hint to force leftover CoW reservations then
crash the filesystem to see how recovery works.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: high offset reflink and dedupe tests

Ensure that we can pass absurdly enormous offsets and lengths to
reflink/dedupe and it'll survive.

v2: Ask for dedupe in the dedupe test.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[hch@lst.de: call _require_test_dedupe]
Signed-off-by: Christoph Hellwig <hch@lst.de>

xfs: test fragmentation characteristics of copy-on-write

Perform copy-on-writes at random offsets to stress the CoW allocation
system.  Assess the effectiveness of the extent size hint at
combatting fragmentation via unshare, a rewrite, and no-op after the
random writes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs/122: support refcount/rmap data structures

Include the refcount and rmap structures in the golden output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfs: more reflink tests

Create a couple of XFS-specific tests -- one to check that growing
and shrinking the refcount btree works and a second one to check
what happens when we hit maximum refcount.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: ensure that we can handle reflinking a lot of extents

Update the existing stress tests to ensure that we can handle
reflinking the same block a million times, and that we can handle
reflinking million different extents.  Add a couple of tests to ensure
that we can ^C and SIGKILL our way out of long-running reflinks.

v2: Don't run the signal tests on NFS, as we cannot interrupt NFS
clone operations.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[hch@lst.de: don't run on NFS]
Signed-off-by: Christoph Hellwig <hch@lst.de>

reflink: test CoW operations against the source file

Ensure that CoW operations against shared blocks in the source file
work correctly.

v2: remove filefrag dependencies

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test CoW behavior with IO errors

Test various scenarios (with dm-flakey) where we simulate write
failures during CoW, to see if the FS can get through it without
blowing up or corrupting data.  Plumb in a FS-generic method to
sort out repairing filesystems after they get hit by IO errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: test intersecting CoW and falloc/fpunch/fzero/fcollapse/finsert/ftrunc

Ensure that we correctly handle a CoW operation immediately followed
by a truncate, falloc, fpunch, fzero, fcollapse, and finsert operation
in the middle of the CoW'd region before any flush can occur.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

dio: unwritten conversion bug tests

Check that we don't expose old disk contents when a directio write to
an unwritten extent fails due to IO errors.  This primarily affects
XFS and ext4.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: add _require_odirect to the directio tests

For the tests that test O_DIRECT, we need to _require_odirect.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: fix style problems in existing tests

Fix style problems such as unnecessary use of quotes, add helper
variables to reduce visual clutter, and other minor fixes to make the
first batch of tests more closely resemble the second round tests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: refactor mixed block creation code

Refactor the code that creates files with mixed block types that we feed
into CoW tests to make sure that we can tiptoe around that kind of stuff.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

common: provide a method to repair the scratch fs

Create a wrapper function that repairs any damage to the scratch
filesystem and returns a standard result.  We will use this to clean
up after IO error testing and other weird corruption tests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

common: create _require_test_program to look for programs in src/

Create a helper that looks for a test program in src/ and fails the
test if it doesn't exist.  Refactor the existing testcases to use it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

common/dmerror: add some more dmerror routines

Add functions to the dmerror routine so that we can load both the
error table and the linear table.  This will help us with EIO testing
of copy-on-write.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: remove unnecessary umounts from test code

The test harness already takes care of this, so get rid of it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

reflink: remove redundant filesystem checks from the end of the tests

Turns out that check already runs _check_filesystems after each test,
so we don't need to do this at the end of each test.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfstests: make _scratch_mkfs_blocksized usable

The default mkfs.xfs options contain -b size=4096, so all tests
using _scratch_mkfs_blocksized won't actually run unless those
options are changed.  As we're trying to specificly test 1k
blocks we should always override the default option.

v2: Move the function to common/rc

Signed-off-by: Christoph Hellwig <hch@lst.de>
[darrick.wong@oracle.com: move function to common/rc]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

xfstests: filter whitespace in 128 and 132

Seems either I have a different lsattr version, or different mount points
cause differences in the golden output.  Send the lsattr output through
the whitespaces filter so that it works everywhere.

The lsattr output /does/ change depending on mountpoints.  Ick.  I'd
actually changed it to the long format output because line length in
the short format changes every time the flags change.

Signed-off-by: Christoph Hellwig <hch@lst.de>
[darrick.wong@oracle.com: update changelog]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

generic/182: this is a dedupe test, check for dedupe

Since this test examines dedupe behavior, the documentation should
say 'dedupe', not 'reflink'.  Furthermore, the feature checks must
look for working dedupe functionality, not reflink functionality.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[hch@lst.de: add the test for dedupe support]
Signed-off-by: Christoph Hellwig <hch@lst.de>

Fix prerequisite packages to build fstests on Ubuntu

To build xfstests from minimal Ubuntu, it's necessary to install
some more packages in addition to ones mentioned in README.
Plus, libtool is not needed to specify here since libtool-bin
depends on it.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>

aio-dio-eof-race should be ignored by git

src/aio-dio-regress/aio-dio-eof-race is a binary file built by make.
So it should not be tracked by git.

===============================================================================
$ make clean
...
$ git status
On branch sat-bugfixes
nothing to commit, working directory clean
$ make
...
$ git status
On branch sat-bugfixes
Untracked files:
  (use "git add <file>..." to include in what will be committed)

src/aio-dio-regress/aio-dio-eof-race
===============================================================================

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>