Currently, all the tests in tests/ run directly in tests-foo/ and many
have a prelude along the lines of

    rm -rf test                 # cleanup after a previous run
    mkdir test
    cd test

This is stupid for two reasons: firstly, it is duplication of code.
Secondly, screw-ups are not obvious because a previous test script
might be inadvertently "fixing" the broken script.  We found two such
bugs merely by reversing the order in which test scripts ran!

Instead, the script harness should create a completely clean scratch
directory for each test, and cd into it before running the test.  It
should also automatically remove this scratch directory at the end of
the test -- UNLESS the test failed, in which case the scratch
directory might be left around as a diagnostic tool.

I propose that "cabal test" continue to use tests.<repo format>/
directories as it does now, but within that directory create a <script
name>/ directory for each test.

PS: when a test fails, perhaps its stderr and stdout could also be
placed in this test-specific directory, rather than printing it to
standard output (which tends to be overwhelming when you initially
just want a list of failing tests).

Thanks Trent. I whole-heartedly agree with these suggestions. You seem to have 
assigned yourself to this task (?), so perhaps you would take the trouble, 
whenever suitable, to change the status to "in-progress".

Thanks again and best regards
Thorkil

This isn't as easy as I first expected, and I've made no progress on it.  Unassigning.

Sounds like Christian might have some thoughts about this.
I'll assign it in case he's interested in looking into the issue.

P.S. Resent because everytime I try to use the email interface, I get the
syntax wrong and get rebuffed by roundup.  Sticking to ru and to the web
interface instead.
-- 
Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
PGP Key ID: 08AC04F9

I think Christian may be too busy for this, and our need for this has 
been crystal clear.

Perhaps we should work on this during the 2010-10 sprint

BTS training: what's the difference between feature/wishlist? Mostly a 
notion of importance.

I'll look into this as soon as I'm done with darcs-benchmark refactor.

Implemented by Petr.  Hooray!