I had what I thought was a complete copy of the darcs.net repository.          
                                         
When I did darcs changes src/Lcs.lhs, I got (after a while...)                 
                                         
                                                                               
                                         
darcs: bug in darcs!                                                           
                                         
URL.urlThread: same URLs with different parameters                             
                                         
+http://darcs.net/_darcs/patches/0000000443-a2841e7f2a87564fff172cfe174ede03e332db60b5cd62b2fce80a684c1821a5
./darcs44946
+at src/URL.hs:127 compiled Aug 14 2008 12:40:05                               
                                         
I'm unable to check http://darcs.net/maintenance to see if this version is
supported.                                    
If it is supported, please report this to bugs@darcs.net                       
                                         
If possible include the output of 'darcs --exact-version'.    

This happens with libcurl 7.18.2 (and 7.15.5, for what it's worth)

Two other points: 

First, I don't think I see this error message unless I run --debug (it just sits
there).

Second, prior to this I had moved away my ~/.darcs/sources, effectively
disabling my global cache.  Moving it back helped, although it still had to
fetch some patches from the website.  I'm a bit puzzled by this.  Hopefully, it
just means that I am confused and had gotten darcs lazily the first time.

Sigh, no I cannot reproduce this, partly because I deleted my old repository
(for some unknown reason).

With darcs 2.0.2+129, I
1. moved away my .darcs/sources
2. darcs get --lazy http://darcs.net
3. darcs changes src/Lcs.lhs

It seems to work, although it does spit out this message at the end
darcs:
/private/tmp/darcs.net/_darcs/patches/0000002351-c317febc7ddc250737b6be5b1eead7edc273345ef0babf87a0cf0ccfd1382a55-new:
renameFile: does not exist (No such file or directory)

I was not able to reproduce this with darcs 2.0.2+132 on debian with libcurl.
But I am behind few proxies... Will try at home, where pipelining works.

Can you attach --debug output?

Regards,
  Dmitry

Log attached, will also attach the one for the lazy get

Attachments

• debug-log-2.gz

I have looked at the log. And here is my understanding of what is going on:

1. Darcs speculates on a patch.
2. urlThread gets to it, and puts into waitToStart queue.
3. urlThread starts downloading the patch, blocks in waitNextUrl.
4. Darcs calls copyUrl for the same patch. This puts download request into the
URL channel, but urlThread did not read it yet.
5. Darcs calls waitUrl for the patch.
6. Patch is successfully downloaded, urlThread unblocks, waitUrl unblocks.
7. urlThread reads the second request for the patch from the channel. Since the
previous URL is already removed from the map, a new download starts.
8. urlThread blocks in waitNextUrl and waiting for the patch to be downloaded
for the second time.
9. Darcs exits.

I am not sure what happens when haskell program exits. Looks like curl download
is interrupted but haskell code is still executed until renameFile is called
(and exception is thrown, since download is not complete and file does not exist
yet). Any comments if this makes sense?

Besides the fact that there is a (mystery) renameFile call, there is a race
condition that makes a patch downloaded twice. One solution for this would be
keeping map of all URLs we have downloaded. But I think this can have
performance impact if we have a huge repo... My proposal is to check if target
file exists before starting download. That will not help with the case when we
download to temporary files, but should work fine for patches. I will send a
patch for Eric to test.

Regards,
  Dmitry

Hi Eric.

Can you please test with the attach patch and send me (--debug --debug-http) output?

Regards,
  Dmitry

Attachments

• formating-and-minor-refactoring-in-url_urlthread_.dpatch

Glancing at my logs, this does seem to fix the issue!

* http://code.haskell.org/~kowey/tmp/debug-log-b-1.bz2
* http://code.haskell.org/~kowey/tmp/debug-log-b-2.bz2

Thanks.

I've changed the title of this bug to reflect what you've fixed.  I hope the
URL.urlThread: same URLs with different parameters one has been solved by some
of the other HTTP activity :-), but I guess we'll have to open a new issue if
anybody else sees this in the wild.

Thanks Eric!

I will do darcs send as soon as I get email access. And close the bug after it
is applied.

Regards,
  Dmitry

Fixed in darcs unstable by:

Fri Sep 12 09:32:36 MSD 2008  Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
  * Do not download URL we have speculated before.

Regards,
  Dmitry