tools: compare-failures script
authorEric Sandeen <sandeen@sandeen.net>
Fri, 13 Jan 2017 14:56:15 +0000 (08:56 -0600)
committerEryu Guan <eguan@redhat.com>
Wed, 18 Jan 2017 07:59:33 +0000 (15:59 +0800)
commit2ad51674753978db5c1a036eaf08b00994a218eb
tree00d5a14269bee65fbfe81ffcc8326ae20ecab11a
parentacb8acd2367ac8b9d3f9a471b9bd18e0e6d83eda
tools: compare-failures script

This is a simple script to compare failures across runs.

Given files containing stdout from several runs, each of which contains
a Failures: line, it will print a table of all failures for each run.
Test subdir names are abbreviated for compactness, i.e. generic->g.
For 7 results files named test 1 through test 7:

Failures:
g/075 g/082 g/209 g/233 g/270 g/388 x/004 x/073 x/076
-----------------------------------------------------
      g/082       g/233             x/004 x/073       test1
      g/082       g/233             x/004 x/073 x/076 test2
      g/082                         x/004 x/073 x/076 test3
      g/082                   g/388 x/004 x/073       test4
      g/082             g/270       x/004 x/073       test5
      g/082                         x/004 x/073       test6
g/075 g/082 g/209 g/233             x/004 x/073       test7

This lets us easily spot unique failures and outliers.

This could be enhanced to output CSV etc, but for now I think it's
helpful to visualize changes in failures across multiple runs.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
README
tools/compare-failures [new file with mode: 0755]