Zandr was setting up a new branch of tahoe for allmydata.com, and he  
did a "darcs push" over ssh from a repository containing the tahoe  
source (http://allmydata.org/source/tahoe/server-hashedformat ) to an  
empty repository.  Time passed, and Zandr and I gave up and assumed  
it was broken, killed it, experimented, experimented some more, and  
finally copied the whole thing over by making a tarball on one side  
and scp'ing it over and untarring it.

Later, I figured out that if we had waited for 37 minutes, it would  
have worked.

I ran "darcs optimize" in the repo, and then the push --all takes  
"only" 17 minutes.

However, 17 minutes is probably about 10 times as slow as competing  
revision control tools.  For comparison the following script takes  
between 60 and 120 seconds:

tar cjf server-hashedformat.tar.b2 server-hashedformat
scp server-hashedformat.tar.bz2 zooko@allmydata.org:
ssh zooko@allmydata.org 'tar xjf server-hashedformat.tar.bz2'

Could we add this repository to the darcs benchmarking zoo and start  
generating automated benchmarks of how long it takes to push all  
these patches into an empty repository?

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

Presumably you were trying to do push --all because of other problems with darcs
put?  I recall that darcs put has a bug where it wasn't propagating flags correctly.

We could certainly use more help with benchmarking!  We now have a sort of
benchmarking harness, but we don't have the infrastructure to use it
automatically.  Infrastructure would probably consist of dedicated hardware and
scripts.  Any help would be appreciated.

Note: Thorkil, I'm not benchmarking push --all is exactly the wish we want

Also it's worth comparing your numbers with the same command on the local hard disk.

Right -- issue1255 is why I didn't suggest that Zandr try "darcs put".

A "darcs push -a ../targetemptyrepo" takes 7m28s.  (Cache is enabled.)

Memory usage gets very large during these experiments -- more than 1 GB.

Summing up for anybody following this:

- original push time (remote): 37 min
- optimized push time (remote): 17 min
- optimized (?) push time (local): 7 min

Yes the local push was optimized.

And, to sum up for anybody following this:

- suggested target for excellent performance improvement: 120 sec

:-)

Benchmarking push --all is probably a little silly, but it would make sense to
benchmark darcs apply.

Variants:
- lots of patches
- one big named patch (with lots of pieces?)