Some unit tests fail sometimes. 

I have seen failures for "checking that merges are valid" and for "checking
other inverse being valid". I attached two failing data structures for the
latter case.

Attachments

• checking-other-inverse-being-valid-fail-case-1.txt

Sounds like something that should be high up on our priority list, even if it's
just to find out that our tests are too strict.

Now I see a failure in 'checking commutex either way'. Failing data attached.

Attachments

• checking-commutex-either-way-fail.txt

Is there a good procedure for shrinking these down, perhaps automatically or
semi-automatically?

If, with the new capability of choosing the QuickCheck seed and size, you do a:

dist/build/unit/unit -r '1387922338 2147483372,86'

you get no less than five failures:

87 tests passed.Failing tests:
    Checking commutex either way... :
        Falsifiable (seed: 1814836306 2147483325, size: 86)
    Checking other inverse being valid... :
        Falsifiable (seed: 1387922338 2147483372, size: 86)
    Checking inverses being valid... :
        Falsifiable (seed: 1387922338 2147483372, size: 86)
    Checking that merges are valid... :
        Falsifiable (seed: 1814836306 2147483325, size: 86)
    Checking that commutes are equivalent... :
        Falsifiable (seed: 1814836306 2147483325, size: 86)

This hints that this QuickCheck seed and size combination makes QuickCheck
generate a very evil patch.

We have reproducibility!  Thanks for the new framework.  Is there some sort of
useful textual representation of the patch (and the history) that we could save
and translate into something human-readable?

On Friday 05 June 2009 09:04:08 Eric Kow wrote:
> Eric Kow <kowey@darcs.net> added the comment:
>
> We have reproducibility!  Thanks for the new framework.  Is there some sort
> of useful textual representation of the patch (and the history) that we
> could save and translate into something human-readable?

The next step, I guess, is to introduce some trace statements in the test that 
print the patch using the show instance.  And introduce more statements to see 
where exactly it goes wrong. I'm going to publish testrunner to hackage first, 
and then I'll attack this issue if no-one else has taken it up yet.

BTW, QuickCheck already dumps the whole failing patch to the terminal so you can
already examine the offending patch without modifying a letter of code. But it
is quite big iirc.

The following patch updated the status of issue1412 to be resolved:

* resolve issue1412: also adjust maxline in PatchCheck upon insert_line 
Ignore-this: e7d84f5c6bc0175fff252296189abe1f