When I try and clone a repository like so 'darcs get foo foo-cloned' with my
current largish and unfortunately non-OSS-licensed repository, I get an 'out of
memory' error.

Strangely enough, 'darcs get http://my-darcs-repo-machine/foo' foo-cloned works
just fine.  Somehow, 'darcs get' over HTTP does not exhibit the issue, but
'darcs get' on the local file system does.

This happens on 2.0.0pre3 and current unstable head. I have confirmed that this
issue DOES NOT occur with darcs 1.0.9rc1 (what comes with Ubuntu)

James, to debug this further:

Would you have any preferences in ~/.darcs/prefs which could be affecting this? 

You mentioned it happens with Darcs2. How about if you use "darcs get --hashed" ? 

also, you can try this:

strace darcs get foo foo-cloned. 

That will spew all the low level system calls that are happening. (There will be
many!). Although the output will be verbose, you should be able to perhaps get
an idea of what's going on when the system runs out of memory, and/or how the
local case differs from the http case.

James, 

I'm also curios to know: What happens if you try this with Darcs2:

darcs convert foo foo-d2
darcs get foo-d2 foo-cloned

 ?

  Mark

My ~/.darcs/defaults looks like this:

get --set-scripts-executable
pull --set-scripts-executable
send --sign

I'll let you know how the tests with strace go.

On 28/01/2008, Mark Stosberg <bugs@darcs.net> wrote:
> James, to debug this further:
>
> Would you have any preferences in ~/.darcs/prefs which could be affecting this?
>
> You mentioned it happens with Darcs2. How about if you use "darcs get --hashed" ?
>
> also, you can try this:
>
> strace darcs get foo foo-cloned.
>
> That will spew all the low level system calls that are happening. (There will be
> many!). Although the output will be verbose, you should be able to perhaps get
> an idea of what's going on when the system runs out of memory, and/or how the
> local case differs from the http case.
>
> ----------
> nosy: +markstos
>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue623>
> __________________________________
>

James, could you give us some further information about the repository that
triggers this issue?

I presume from what you've said that it's in darcs1 format.  Here's some other
information that would be helpful in trying to reproduce this:

How many patches are in it?

What is the total size of _darcs/patches (e.g. `du -sh _darcs/patches`)?

What's the total size of the repository contents (e.g. du -sh _darcs/pristine)?

How much memory and swap do you have? Similarly, if you watch top, how much
memory do you see darcs using before it dies?

Do you have many (or large) binary files in the repository?

Thanks for your bug report!

David

Hi David, 

The repository is in darcs1 format, yes.
> How many patches are in it?
2112

> What is the total size of _darcs/patches (e.g. `du -sh _darcs/patches`)?
619M

> What's the total size of the repository contents (e.g. du -sh _darcs/pristine)?
342M

> How much memory and swap do you have? Similarly, if you watch top, how much
memory do you see darcs using before it dies?
My machine has 4GB, but due to irritating BIOS/motherboard limitation, I can
only see 3.2GB.  At the time I made the error report I was running 32bit Ubuntu
Gutsy.  My machine was configured with 4GB swap.  Using top, darcs's resident
set was about 2.7GB, and it died trying to allocate another ~700MB.  I don't
recall the size of the non-resident set, but I expect that the 700MB allocation
attempt would have surpassed the 4GB address space for 32bit processes.

> Do you have many (or large) binary files in the repository?
Unfortunately, yes (358 binaries - basically all the dependencies for our
different modules).  There are about 10 3MB binaries and the rest are
progressively smaller.  There is also a single very large text file ~30MB.

I realize that putting all of those binaries under source control wasn't the
smartest thing to do - it was the result of a hasty decision, what can I say..

It would be great if somebody could write a small script to reproduce conditions
like those in James's repository (and James, that you could keep your repo
around for future testing).  I think Jason has done some work in this area.

Note particularly the failing on local get and not on remote get.