darcs

Issue 1770 Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles

Title Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles
Priority bug Status resolved
Milestone Resolved in 2.5.0
Superseder Nosy List dmitry.kurochkin, kowey, twb
Assigned To
Topics HTTP

Created on 2010-03-18.04:19:55 by twb, last changed 2010-06-19.22:32:56 by kowey.

Messages
msg10262 (view) Author: twb Date: 2010-03-18.04:19:50
When, and only when, Darcs uses polipo as its proxy, and polipo uses
squid as its parent proxy, do I get the following behaviour:

    $ rm -rf ~/.cache/darcs/* ~/.darcs/cache/*
    $ with-temp-dir darcs get http://darcs.debian.org/pkg-haskell/haskell-hashed-storage
    with-temp-dir: entering directory `/tmp/with-temp-dir.sLbr9S'
    darcs: bug at src/URL.hs:246 compiled Nov 16 2009 21:34:38
    Another possible bug in URL.waitNextUrl:  curl_multi_perform() - no running handles
    See http://wiki.darcs.net/index.html/BugTrackerHowto for help on bug reporting.
    Copying pristine 2/10 : series   -- it hung here --

Here are the versions I'm using (on Debian unstable/amd64):

    $ COLUMNS=72 dpkg -l polipo libcurl3-gnutls darcs | tail -3
    ii  darcs          2.3.0-3        a distributed, interactive, smart revision c
    ii  libcurl3-gnutl 7.20.0-1       Multi-protocol file transfer library (GnuTLS
    ii  polipo         1.0.4-3        a small, caching web proxy

The parent Squid proxy (http://proxy:8080/) is squid/2.5.STABLE9,
running on FC3.

My darcs was almost certainly built with curl-pipelining enabled.

UPDATE: wait, no, it SOMETIMES happens even without polipo:

    $ rm -rf ~/.cache/darcs/* ~/.darcs/cache/*
    $ http_proxy=http://proxy:8080/ with-temp-dir darcs get http://darcs.debian.org/pkg-haskell/haskell-hashed-storage
    with-temp-dir: entering directory `/tmp/with-temp-dir.LKPesp'
    darcs: bug at src/URL.hs:246 compiled Nov 16 2009 21:34:38
    Another possible bug in URL.waitNextUrl:  curl_multi_perform() - no running handles
    See http://wiki.darcs.net/index.html/BugTrackerHowto for help on bug reporting.
    Copying pristine 3/10 : mmap_0.5_compat.patch   -- it hung here --

AFAICT that squid doesn't speak HTTP/1.1[0], so pipelining can't be
the issue. (Indeed, AFAICT *no* squid speaks HTTP/1.1 yet:
http://wiki.squid-cache.org/Features/HTTP11)
msg10267 (view) Author: kowey Date: 2010-03-18.08:20:51
Does this happen frequently enough that you have a chance of reproducing it?

Variables so far:
- proxy
- pipelining (even if we think there is no reason for this to be the
problem)

Presumably the curl-free Darcs is fine?
msg10268 (view) Author: twb Date: 2010-03-18.08:30:22
Eric Kow wrote:
> Does this happen frequently enough that you have a chance of reproducing it?
>
> Variables so far:
> - proxy
> - pipelining (even if we think there is no reason for this to be the
> problem)

AFAIK, I started getting it today, and I could reproduce it about four
times in every five.

This is also the first time I'm running Darcs on my new system:

    Debian unstable, amd64, btrfs, SSD

My old system was

    Debian testing, i386, ext2, SSD

> Presumably the curl-free Darcs is fine?

I haven't got such a build handy.  I'll try to make one Real Soon Now;
this issue is not high on my priority list.
msg10413 (view) Author: kowey Date: 2010-03-22.13:36:38
Just linking to curl page on multi_perform:

 http://curl.haxx.se/libcurl/c/curl_multi_perform.html

Looks like what we really need is for somebody to stare at the code a
bit and identify what the warning really means (Darcs thinks there are
no transfers in progress?) and if it actually is a bug...
msg10469 (view) Author: kowey Date: 2010-03-23.19:34:46
Note that in issue1368, we have a sporadic occurrence of this error
message, without any proxy servers whatsoever...
msg10635 (view) Author: kowey Date: 2010-04-02.16:50:20
Actually, another thing that may be worth trying is to pass
--no-http-pipelining to the get on runtime.
msg10695 (view) Author: twb Date: 2010-04-08.05:08:39
Eric Kow wrote:
> Actually, another thing that may be worth trying is to pass
> --no-http-pipelining to the get on runtime.

No noticable change.

$ darcs pull
Pulling from "http://repos.mornfall.net/hashed-storage"...
No remote changes to pull in!
$ darcs pull
Pulling from "http://repos.mornfall.net/hashed-storage"...
No remote changes to pull in!
$ darcs pull
Pulling from "http://repos.mornfall.net/hashed-storage"...
darcs: bug at src/URL.hs:246 compiled Apr  1 2010 06:54:05
Another possible bug in URL.waitNextUrl:  curl_multi_perform() - no
running handles
See http://wiki.darcs.net/BugTracker/Reportingfor help on bug reporting.
^CwithSignalsHandled: Interrupted!

$ darcs pull --no-http-pipelining
Pulling from "http://repos.mornfall.net/hashed-storage"...
No remote changes to pull in!
$ darcs pull --no-http-pipelining
Pulling from "http://repos.mornfall.net/hashed-storage"...
darcs: bug at src/URL.hs:246 compiled Apr  1 2010 06:54:05
Another possible bug in URL.waitNextUrl:  curl_multi_perform() - no
running handles
See http://wiki.darcs.net/BugTracker/Reportingfor help on bug reporting.
^CwithSignalsHandled: Interrupted!

$ darcs --version
2.4 (+ 151 patches)
$ darcs --exact-version | sed 1,/Compiled.with:/d | fmt

HTTP-4000.0.6 array-0.3.0.0 base-4.2.0.0 bytestring-0.9.1.5
containers-0.3.0.0 directory-1.0.1.0 extensible-exceptions-0.1.1.1
filepath-1.1.0.3 hashed-storage-0.4.10 haskeline-0.6.2.2 html-1.0.1.2
mmap-0.4.1 mtl-1.1.0.2 network-2.2.1.7 old-time-1.0.0.3 parsec-2.1.0.1
process-1.0.1.2 random-1.0.0.2 regex-compat-0.92 tar-0.3.1.0
terminfo-0.3.0.2 text-0.7.1.0 unix-2.4.0.0 zlib-0.5.2.0
msg11105 (view) Author: kowey Date: 2010-05-24.07:10:47
Fixed in HEAD by Dmitry's patch?

Thu Apr 15 23:47:39 BST 2010  Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
  * Fix hscurl.c when URL is downloaded during the first call to
curl_multi_perform.
  Turns out that the first call to curl_multi_perform() can fetch the URL or
  result in error. I can easily reproduce this using HTTP server on
localhost.
  This means that situation when running_handles is zero is valid, so
remove the
  error and handle it correctly.
History
Date User Action Args
2010-03-18 04:19:55twbcreate
2010-03-18 08:20:56koweysetstatus: unknown -> waiting-for
priority: bug
nosy: + kowey
messages: + msg10267
topic: + HTTP
assignedto: twb
2010-03-18 08:30:24twbsetmessages: + msg10268
title: Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles -> Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles
2010-03-22 13:36:41koweysetstatus: waiting-for -> needs-reproduction
assignedto: twb ->
messages: + msg10413
2010-03-23 19:34:48koweysetmessages: + msg10469
2010-04-02 16:50:21koweysetmessages: + msg10635
2010-04-08 05:08:40twbsetmessages: + msg10695
2010-04-08 11:03:12koweysetstatus: needs-reproduction -> unknown
2010-05-24 07:10:47koweysetstatus: unknown -> resolved
nosy: - darcs-devel
messages: + msg11105
2010-05-26 18:04:54koweylinkissue1850 superseder
2010-06-19 22:32:56koweysetresolvedin: 2.5.0
2010-06-19 22:36:04koweylinkissue1876 superseder
2010-11-17 09:50:52koweylinkissue1995 superseder