darcs

Patch 799 Implement exponential backoff test strategy

Title Implement exponential backoff test strategy
Superseder Nosy List mndrix
Related Issues
Status accepted Assigned To
Milestone

Created on 2012-04-06.22:41:49 by mndrix, last changed 2012-04-24.14:14:47 by gh.

Files
File name Status Uploaded Type Edit Remove
patch-preview.txt mndrix, 2012-04-06.22:41:47 text/x-darcs-patch
style-and-haddock-for-darcs_commands_test.dpatch mndrix, 2012-04-06.22:41:47 application/x-darcs-patch
unnamed mndrix, 2012-04-06.22:41:47
See mailing list archives for discussion on individual patches.
Messages
msg15539 (view) Author: mndrix Date: 2012-04-06.22:41:47
11 patches for repository http://darcs.net/screened:

This patch series defines a new "darcs test" search strategy named
--backoff.  It's aimed at finding regressions which are closer to the
head than to the middle of the repository.  Under these conditions, in
the screened repository, it performs between 30 and 70% faster than
bisect.  It will perform relatively better on repositories with more
patches.

The search strategy starts by unapplying 4 patches (chosen to match
the break-even point with linear search).  If the test fails, it
unapplies 8 more patches and tests again.  On each test failure, it
removes twice the number of patches as before.  Once the test passes,
it bisects the patches it just skipped. 

--backoff was the best name I could think of.   Any better ideas?

The main patch is "Implement exponential backoff test strategy".
Everything else is refactoring in preparation for that patch.  If the
new feature turns out to be more controversial than I anticipated,
that patch should be easy to rollback, leaving only the valuable clean
up patches.


Tue Apr  3 13:43:50 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Style and Haddock for Darcs.Commands.Test

Tue Apr  3 14:00:13 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Use safeApply throughout Darcs.Commands.Test
  Provides consistent error handling and cleaner code.

Tue Apr  3 14:01:50 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Unify darcs test strategy type signatures
  Using consistent type signatures opens additional clean up
  possibilities.

Tue Apr  3 14:27:14 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Factor out Strategy type

Tue Apr  3 14:28:06 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Clearer test strategy selection
  This also makes it easier to add other search strategies later.

Wed Apr  4 08:28:08 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Consistent test strategy function naming
  trackFoo is the initial function while trackNextFoo is the
  follow-up function.  Only bisect strategy needs the latter.

Wed Apr  4 08:56:17 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Correct typos in trackdown-bisect test comments

Fri Apr  6 11:46:09 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Improve darcs test descriptive help
  This removes a redundant sentence starting with "Given two
  arguments".  It also offers some guidance on selecting between
  the linear and bisect search strategies.

Fri Apr  6 16:04:37 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Factor out initial test run
  It removes redundant code and makes test strategy definitions simpler.

Fri Apr  6 16:04:52 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Factor out function to build bisect strategy initial state

Fri Apr  6 16:07:13 MDT 2012  Michael Hendricks <michael@ndrix.org>
  * Implement exponential backoff test strategy
  Use exponential backoff to locate a subset of patches on which to
  perform a bisect.  For large repositories or recent regressions,
  this strategy is faster than either linear or bisect.
Attachments
msg15540 (view) Author: mndrix Date: 2012-04-06.22:45:26
Self-screening because it's mostly clean up patches with one feature I expect not 
to be controversial.  If I was wrong to self-screen, please let me know so I can 
improve my mental model of when that process is acceptable (and so I can rollback 
the one relevant patch).
msg15541 (view) Author: ganesh Date: 2012-04-07.18:54:14
I think this strategy is a great idea!

BTW it's fine for you to self-screen (i.e. push to screened) anything 
you like. Reviewing, i.e.  pushing from screened->reviewed, would 
ideally be done by someone else, but it's fine to self-accept trivial 
patches and also patches that have been awaiting review for a while.
msg15613 (view) Author: gh Date: 2012-04-24.14:14:47
> Style and Haddock for Darcs.Commands.Test
Small style tweaks.

> Use safeApply throughout Darcs.Commands.Test
Good idea.

> Unify darcs test strategy type signatures
Useful for next patch (could have been merged?).

> Factor out Strategy type
OK.

> Clearer test strategy selection
Good.

> Consistent test strategy function naming
Naming cleanup.

> Correct typos in trackdown-bisect test comments
OK.

> Improve darcs test descriptive help
OK.

> Factor out initial test run
Nice one, every test strategy does the test on the current state of the
repo, and then chooses proceeds (or not, as it is the case for --once).

> Factor out function to build bisect strategy initial state
Why not.

> Implement exponential backoff test strategy
The new strategy is well commented, thanks for that.

Tests pass, so I'm accepting the bundle. Thanks!

This new strategy needs a high-level explanation on the wiki (like the
one for bisect).
History
Date User Action Args
2012-04-06 22:41:49mndrixcreate
2012-04-06 22:45:26mndrixsetstatus: needs-screening -> needs-review
messages: + msg15540
title: Style and Haddock for Darcs.Commands.Test (and 10 more) -> Implement exponential backoff test strategy
2012-04-07 18:54:15ganeshsetmessages: + msg15541
2012-04-24 14:14:47ghsetstatus: needs-review -> accepted
messages: + msg15613