On MacOS X 

darcs get --hashed http://darcs.haskell.org/ghc --debug

eventually leads to this

URL.waitNextUrl failed: http://darcs.haskell.org/ghc/_darcs/inventories/20080601155241-3fd76-13b3337cedf6323aab252f8035c39326411e3614.gz ./darcs7859 curl_multi_info_read() failed
URL.copyUrl (http://darcs.haskell.org/ghc/_darcs/patches/20080719132217-3fd76-01a5008e6cab0e4a746ac65d175246a818554c50.gz
          -> ./darcs7860)
URL.copyUrl succeeded
darcs: bug in darcs!
Possible bug in URL.waitNextUrl:  at src/URL.hs:109 compiled Jul 31 2008 08:18:30
I'm unable to check http://darcs.net/maintenance to see if this version is supported.
If it is is supported, please report this to bugs@darcs.net
If possible include the output of 'darcs --exact-version'.

Dmitry: could I ask you to have a look at this please?  People on all platforms
seem to be experiencing an infinite loop fetching patches.

Lele [who is on Linux], could we have your curl version, please?  I'll dig mine
up when I get home.

On Wed, 06 Aug 2008 09:04:13 -0000
Eric Kow <bugs@darcs.net> wrote:

> Lele [who is on Linux], could we have your curl version, please?
> I'll dig mine up when I get home.

I'm experiencing the problem on an up-to-date Ubuntu Hardy:

$ curl --version
curl 7.18.0 (x86_64-pc-linux-gnu) libcurl/7.18.0 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.1
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

As mentioned (maybe just on IRC), on a different system lacking curl
completely (at least "ldd darcs" does not show any refs to that
library) darcs 2.0.2+2 does not exhibit the problem.

ciao, lele.
-- 
nickname: Lele Gaifax    | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas    | comincerò ad aver paura di chi mi copia.
lele@nautilus.homeip.net |                 -- Fortunato Depero, 1929.

Hi Eric.

On Wed, Aug 6, 2008 at 1:04 PM, Eric Kow <bugs@darcs.net> wrote:
>
> Eric Kow <eric.kow@gmail.com> added the comment:
>
> Dmitry: could I ask you to have a look at this please?  People on all platforms
> seem to be experiencing an infinite loop fetching patches.

I was able to reproduce this. Not infinite loop but
"curl_multi_info_read() failed".

Versions:

darcs 2.0.2 (+ 25 patches)
curl 7.18.2 (debian unstable package)
linux amd64

I will look into this as soon as I can.

Regards,
  Dmitry

>
> Lele [who is on Linux], could we have your curl version, please?  I'll dig mine
> up when I get home.
>
> ----------
> assignedto:  -> dmitry.kurochkin
> nosy: +dmitry.kurochkin, lele
> priority:  -> bug
> status: unread -> chatting
>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue977>
> __________________________________
>

I think I have found the root of our problems - multi threading!
I added myThreadId printing in URL.waitUrl and there are 2 different threads
calling curl functions.

There is no synchronization done neither in C nor in Haskell part of curl/libwww
modules. Now the question is if we want to add multi thread support in URL
module(s) or fix the code to make calls from only one thread.

I do not know darcs code and can not say what is the best way to fix this. If
someone can point me where to look (or what doc to read) I will try to look more
on this tomorrow.

Oh, and infinite loop is a separate bug. I will send a patch for it soon.

Regards,
  Dmitry

At the risk of saying something irrelevant, maybe the new curl binding on
hackage has better support for calling curl functions from multiple threads?

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/curl

It may be worthwhile to discuss this with the Galois guys.

Attached is a patch that adds a basic thread synchronization to URL module. It
is not a complete solution. It is possible that URL is requested on one thread
but wait is done on another...

Still it looks like 'darcs get --hashed http://darcs.haskell.org/ghc' works now.

I am thinking about a proper solution in the meantime. And I was trying to find
the actual place where multiple threads are used for repo download. But I could
not. If you can point me to the right direction that would be nice :)

And I have looked at curl bindings. Looks like there is only "easy" API. We need
"multi". Besides curl itself does not provide any multi threading support by
itself, so we would have the same issues.

Regards,
  Dmitry

Attachments

• url_sync.dpatch

> And I was trying to find
> the actual place where multiple threads are used for repo download. But I could
> not. If you can point me to the right direction that would be nice :)

David: any advice for Dmitry?

For me 'darcs get --hashed http://darcs.haskell.org/ghc' works fine with the
latest darcs. But Gaëtan reports a failure with 'darcs get --lazy
http://darcs.net' (works for me as well).

I would appreciate some testing.

I believe this bug is fixed in the latest darcs now.

No one complained (or no one tested). So closing the bug.

Hello, I'm afraid that I get this same failure on Solaris when using 2.0.2+133
patches.

Here is the transcript of the failing darcs get:

http://allmydata.org/buildbot-pycryptopp/builders/solaris-amd64-nexenta-nooxie/builds/34/steps/darcs/logs/stdio

Here is the "darcs --version":

http://allmydata.org/buildbot-pycryptopp/builders/solaris-amd64-nexenta-nooxie/builds/34/steps/darcs-version/logs/stdio

Ah, actually this may have been a different problem (with similar results). 
There was a problem in a patch that Jason Dagit committed.  When I applied his
fix, this problem stopped (sporadically) happening:

http://lists.osuosl.org/pipermail/darcs-users/2008-August/013248.html

So as far as I know we can re-close this ticket.

Actually, the log shows select() error. I wouldn't expect it to be related to
Jason's work. Besides I have changed how select() is called recently. So it *is*
possible that there is a bug. But I do not see it on linux.

I would prefer to close this bug and, if you see select() failures again, open a
new one. Perhaps some more debug prints should be added for such errors...

Regards,
  Dmitry