More trouble with darcs get, this time on Linux X86_64 machine
                                                               
libcurl 7.18.0                                                                 
darcs 2.0.2+126
using a proxy server                                                           

Interestingly, it does not appear to hang in the same place each time.         
                                               
Maybe it's getting further along each time (I do have the global cache enabled
on this machine, if this explains anything)                            
                                               
                                                                               
                                               
I will attach a list of patches from Dmitry since 2.0.2 along with the --debug
--timings logs

[by the way, Dmitry, I've added an HTTP topic to the bugtracker; maybe you
should consider editing your details so that you are nosy on that topic]

Attachments

• darcs-curl-log-2008-08-25.tgz

I'm attaching the debug http output trying to do darcs get.  Note that this is a
different repository, on doing a get on a different machine.  Same darcs and
libcurl configuration though.  Doing the same get with --disable-pipelining
seems to work.

Attachments

• debug-http-tahoe.gz

Unfortunately I do not see anything interesting in the log. At the moment I
suspect a curl problem, since you say it works with --disable-pipelining.
Ideas/questions:

1. There is a proxy server. Can you try running on the same machine but without
a proxy? If it works without a proxy it is clearly a curl problem.

2. Can you upgrade to curl 7.18.2 and/or nightly?

BTW On my debian x64 'darcs get http://allmydata.org/source/tahoe/prod' runs
fine. curl 7.18.2, pipelining enabled, no proxy.

Regards,
  Dmitry

I am forced to use a proxy, sadly.

But I can work on getting a recent libcurl on my machine to see.

If it turns out that having libcurl < 7.18.2 is the problem, we may have to
alter the configure scripts not to enable pipelining for these inferior versions :-(

Eric, today curl 7.19.0 was released with more pipelining fixes. I think it is
worth trying if 7.18.2 does not work.

Regards,
  Dmitry

Upgrading to curl 7.19.0 does not appear to help, sadly.

After looking at your logs and code again, I think I have found one problem. I
hope this problem causes the bug.

The problem is that waitUrl reads notification MVar, deletes it from the map and
starts waiting on it. At the same time another thread can call copyUrl, which
creates a new notify MVar and puts it into the map. Next waitUrl hangs...

Will attach a patch for testing soon.

Is the bug always reproducible? On any repo?

Regards,
  Dmitry

Cool!

On Wed, Sep 03, 2008 at 19:34:52 -0000, dmitry.kurochkin wrote:
> Is the bug always reproducible? On any repo?

It does seem to always hang, although I'm only testing by getting
the darcs.net repo

The patch is attached. Hopefully it fixes the problem...

Regards,
  Dmitry

Attachments

• url_hs_-fix-race-condition-in-copyurlwithpriority-and-waiturl_.dpatch

On Wed, Sep 03, 2008 at 20:19:43 -0000, dmitry.kurochkin wrote:
> The patch is attached. Hopefully it fixes the problem...

Sadly, not :-(

Log attached

Attachments

• darcs-log-2008-09-03.gz

Eric, in the last log I see many "Couldn't handle interrupt since darcs was in a
sensitive job.". Did you hit C-c?

Is it possible to get traffic dump?

Regards,
  Dmitry

Another thought, how long do you wait befor killing darcs? When connection is
lost curl can wait for timeout for quite some time... And from logs it looks
like URL thread is blocked in hscurl.c.

I will try getting through proxy with pipelining enabled and disabled. And
compare my logs with yours.

Regards,
  Dmitry

On Thu, Sep 04, 2008 at 08:28:39 -0000, dmitry.kurochkin wrote:
> Eric, in the last log I see many "Couldn't handle interrupt since darcs was in a
> sensitive job.". Did you hit C-c?

Yes.

> Is it possible to get traffic dump?

I don't really know how to give you one.  Googling, I see there is a
tcpdump command, which I might try later.

Responding to your next comment

On Thu, Sep 04, 2008 at 08:34:13 -0000, dmitry.kurochkin wrote:
> Another thought, how long do you wait befor killing darcs? When connection is
> lost curl can wait for timeout for quite some time... And from logs it looks
> like URL thread is blocked in hscurl.c.

I don't remember.  I'm pretty sure I just went away, did something else,
and came back, maybe 5 or 10 minutes?

> I will try getting through proxy with pipelining enabled and disabled. And
> compare my logs with yours.

Thanks

On Thu, Sep 4, 2008 at 1:21 PM, Eric Kow <bugs@darcs.net> wrote:
>
> Eric Kow <kowey@darcs.net> added the comment:
>
> On Thu, Sep 04, 2008 at 08:28:39 -0000, dmitry.kurochkin wrote:
>> Eric, in the last log I see many "Couldn't handle interrupt since darcs was in a
>> sensitive job.". Did you hit C-c?
>
> Yes.

So it really hangs not at the end but before you hit C-c.

>
>> Is it possible to get traffic dump?
>
> I don't really know how to give you one.  Googling, I see there is a
> tcpdump command, which I might try later.

Try:

# tcpdump -w darcs.pcap -s 0 host darcs.net

And make sure that no other programs are talking to darcs.net.

>
> Responding to your next comment
>
> On Thu, Sep 04, 2008 at 08:34:13 -0000, dmitry.kurochkin wrote:
>> Another thought, how long do you wait befor killing darcs? When connection is
>> lost curl can wait for timeout for quite some time... And from logs it looks
>> like URL thread is blocked in hscurl.c.
>
> I don't remember.  I'm pretty sure I just went away, did something else,
> and came back, maybe 5 or 10 minutes?

Understood.

Regards,
  Dmitry

I was able to reproduce the bug! I have setup with transparent proxy. Running
darcs with empty http_proxy goes fine (using transparent proxy). But when I
specify http_proxy explicitly it hangs.

This is clearly curl issue. I will take a look at what is going on there, but
this can take time.

BTW I believe my race condition patch is relevant, just fixes another bug. So I
will submit it.

Regards,
  Dmitry

I have reproduced the bug outside of darcs and reported it to libcurl ML.

Unfortunately it looks like we will have to fix this bug by increasing minimal
curl version for pipelining to 7.19.1. Which will be released around November.

Regards,
  Dmitry

I sent this message yesterday, but it never got on the bugtracker.
Resending.

On Thu, Sep 04, 2008 at 14:47:12 -0000, dmitry.kurochkin wrote:
> I have reproduced the bug outside of darcs and reported it to libcurl ML.

Great work.

> Unfortunately it looks like we will have to fix this bug by increasing minimal
> curl version for pipelining to 7.19.1. Which will be released around November.

Ok, so 2.0.3 will ship without pipelining by default, thanks!

Earlier offlist, I had suggested to Dmitry that we compile it, but turn
it off by default on runtime (--{no,}-http-pipelining).  We could
consider this as an alternative.

Darcs now has pipelining disabled by default for curl < 7.19.1:

Mon Sep  8 21:57:26 MSD 2008  Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
  * Add -[-no]-http-pipelining options, disable pipelining by default for curl <
7.19.1.

My fix for the curl bug has been applied upstream. You can find it in CVS or
daily snapshots starting from curl-7.19.1-20080909.

Regards,
  Dmitry

The same issue with proxy here. Are there any 2.1.0 binaries to test on windows?

Do you mean the issue is not resolved?

Regards,
  Dmitry

There aren't any working binaries for windows where I can say that this issue is
fixed. Until then I still can't use Darcs.