Here is a bash script which, on my Mac OS X machine, with darcs-2  
(recent vintage), yields the following output:

darcs: failed to read patch:
Tue Jan  8 11:47:49 MST 2008  zooko@zooko.com
   tagged allmydata-tahoe-0.7.0
user error (Couldn't fetch `cc623980b78aec5413b56d51f7fba803672e6194'
in subdir patches from sources:

thisrepo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/d2b/trunk- 
d2f-gotten
repo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/d2b/trunk-d2f
)

$ time darcs get http://allmydata.org/source/tahoe/trunk
$ darcs convert trunk trunk-d2f
$ /bin/rm -rf ./trunk
$ cd trunk-d2f
$ darcs optimize --modernize-patches --reorder-patches --uncompress

Zooko,

I'm unable to reproduce this bug using the command you give.  Perhaps you left
something out? It seems suspicious that your error message mentions

thisrepo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/d2b/trunk- 
d2f-gotten
repo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/d2b/trunk-d2f

when your script doesn't mention trunk-d2f-gotten.  Perhaps you left out a step?

David

Let me also mention that I suspect that this error message will be improved by a
change I recently made.  So once we can reproduce it, that will be worth seeing.

David

> I'm unable to reproduce this bug using the command you give.   
> Perhaps you left
> something out? It seems suspicious that your error message mentions
>
> thisrepo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/d2b/trunk-
> d2f-gotten
> repo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/d2b/trunk-d2f
>
> when your script doesn't mention trunk-d2f-gotten.  Perhaps you  
> left out a step?

I changed the names when I typed the script into the bug report.   
I'll do a true cut and paste this time.  I suspect this might be a  
Mac OS X-specific bug if it didn't reproduce for you.

Here I go:

Hm... It didn't happen this time...

I'll be back.

Regards,

Zooko

You were right, I forgot a step.  Here's a real cut and paste  
transcript.  Sorry for the incomplete bug report earlier.

(You might also be interested in the output from "time".)

Regards,

Zooko

DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp$  
time darcs get http://allmydata.org/source/tahoe/trunk
Copying patch 1970 of 1970... done.
Applying patch 1970 of 1970... done.
Finished getting.

real    12m5.427s
user    0m4.618s
sys     0m5.879s
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp$  
time darcs convert trunk trunk-d2f
^[[AFinished converting.

real    0m31.634s
user    0m17.463s
sys     0m5.478s
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp$  
time darcs get trunk-d2f trunk-d2f-gotten
Finished getting.

real    0m9.896s
user    0m0.338s
sys     0m0.280s
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp$  
time /bin/rm -rf ./trunk-d2f

real    0m3.376s
user    0m0.007s
sys     0m0.374s
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp$  
cd trunk-d2f-gotten/
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$ time darcs optimize --modernize-patches --reorder- 
patches
  --uncompress &> darcs_opt.log.txt

real    0m0.142s
user    0m0.013s
sys     0m0.014s
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$ cat darcs_opt.log.txt
darcs: failed to read patch:
Tue Jan  8 11:47:49 MST 2008  zooko@zooko.com
   tagged allmydata-tahoe-0.7.0
user error (Couldn't fetch `cc623980b78aec5413b56d51f7fba803672e6194'
in subdir patches from sources:

thisrepo:/Users/wonwinmcbrootles/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten
repo:/Users/wonwinmcbrootles/playground/darcs/bugfind/rmuncomp/trunk-d2f
)
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$
DARC wonwin-mcbrootles-computer:~/playground/darcs/bugfind/rmuncomp/ 
trunk-d2f-gotten$

Zooko, could you try compiling... oh, this is windows? Well, anyhow, I'd be
interested in the results if you could compile darcs linked against libwww
(which requires getting libwww and configure ---with-libwww.  This should make
that initial get much faster.

David

> Zooko, could you try compiling... oh, this is windows? Well,  
> anyhow, I'd be
> interested in the results if you could compile darcs linked against  
> libwww
> (which requires getting libwww and configure ---with-libwww.  This  
> should make
> that initial get much faster.

No, this is Mac.

I'll probably try that sometime.

Regards,

Zooko

The Darcs2 page on the wiki says: "if the source repository disappears, or you
lose network connectivity, some operations may fail. "

Could this be one those cases? It does say explicitly in the error message that
is trying to read from repo that no longer exists:

repo:/Users/wonwinmcbrootles/playground/darcs/bugfind/rmuncomp/trunk-d2f

    Mark

I see the problem in this bug, it's the lazy patch reading.  darcs get
doesn't create a complete repository, but instead includes a reference
to the original repository.  Thus when you remove the original
repository, the gotten one is incomplete.

The question is how much we should change this.  One option, of
course, would be to default to strict getting of repositories, and say
that folks who use --partial get what they ask for.  I'd prefer to
keep the lazy getting behavior, at least by default, since in most
cases it seems to be what folks want (being far faster for large
repositories), and when getting from a remote repository that is not
removed, it's quite safe.

We could add special logic for local gets, which could create hard
links to all the patch files, but this would make a local get more
expensive, and would also complicate the behavior (and complicated
behavior has a tendancy to confuse folks... which could be used as an
argument against lazy get in the first place).

I should perhaps note that a non-lazy get is easily achieveable by
doing a darcs get followed by "darcs changes everything" or "darcs
changes -s".

David

Please see also:

http://lists.osuosl.org/pipermail/darcs-devel/2008-January/007160.html

In which I point out that strict get is what is expected by current users, and
that if they misunderstand what is going on they might delete their entire history.

In some sense the laziness breaks decentralization because it adds a dependency
on the remote repository, and sets up the potential for a data-loss case where
remote node disappears, the local darcs can't continue as usefully as before. 

The remote dependency I'm not as concerned about as I am the data-loss case. 

I think that's a case for making strict the default, but still possibly hyping
"--lazy" as a Darcs-2 feature to significantly speed up common cases, as long as
you know the the side effects are. (Maybe "--partial" could start to mean
"--lazy" ? That would work for me. )

Okay, I'm convinced.  I mostly created the lazy default because it seemed like
almost every project in existence that uses darcs has instructions to get the
repository with

darcs get --partial http://...

But I guess that proves that overriding a default isn't hard.

David