darcs

Issue 1368 Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles

Title Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles
Priority bug Status resolved
Milestone Resolved in
Superseder Nosy List darcs-devel, dmitry.kurochkin, fplccl, kowey, sionescu, thorkilnaur, twb, zooko
Assigned To fplccl
Topics HTTP

Created on 2009-02-23.20:29:30 by zooko, last changed 2010-06-05.14:48:23 by fplccl.

Files
File name Uploaded Type Edit Remove
dev.txt.bz2 zooko, 2009-02-23.20:29:28 application/octet-stream
Messages
msg7348 (view) Author: zooko Date: 2009-02-23.20:29:28
As shown on the tahoe buildbot:

http://allmydata.org/buildbot-tahoe-server/builders/gutsy/builds/31/ 
steps/darcs/logs/stdio

The command-line

darcs get --verbose --partial --repo-name build http://allmydata.org/ 
source/tahoe/server-hashedformat

yields this output

"""
darcs: bug at src/URL.hs:243 compiled Feb 23 2009 11:44:07
Another possible bug in URL.waitNextUrl:  curl_multi_perform() - no  
running handles
I'm unable to check http://darcs.net/maintenance to see if this  
version is supported.
If it is supported, please report this to bugs@darcs.net
If possible include the output of 'darcs --exact-version'.
"""

darcs --exact-version starts with:

darcs compiled on Feb 23 2009, at 11:51:58
# configured Mon Feb 23 11:41:39 PST 2009
./configure /usr/local/stow/darcs-2.2.1/share/config.site /usr/local/ 
stow/darcs-2.2.1/etc/config.site

Context:

[Explain #ifdef HAVE_HTTP in check for _darcs/prefs/post.
Eric Kow <kowey@darcs.net>**20090221141553
  Ignore-this: 1e75f1230c4bfc79f3c7a83f364ed003
]

[show patch names on push/pull only when using 'l'
Christian Kellermann <Christian.Kellermann@nefkom.net>**20090220124322
  Ignore-this: ac72d63dd8880257d851f99c7b8682ff
]

[Remove "unused import/definition" warnings with -fzlib.
Trent W. Buck <trentbuck@gmail.com>**20090220073544
  Ignore-this: 55400327516c0a2b3fcd95f911bd187a
]

[Only import things if they are needed.
Trent W. Buck <trentbuck@gmail.com>**20090220033734
  Ignore-this: ab0558b58e4ccf6cd4deba1e66ba2df
]

[Typo: have_HTTP should be HAVE_HTTP.
Trent W. Buck <trentbuck@gmail.com>**20090220033456
  Ignore-this: a9f418e6fe39d705e994b96e90d5711f
]

[Add a sunset notice to our UTF8 module
Eric Kow <kowey@darcs.net>**20090204165922
  Ignore-this: e29bed6a8fa818419eee49ea17e9caa6
]

The full output of darcs exact version is attached in bzip2.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig
Attachments
msg7349 (view) Author: zooko Date: 2009-02-23.22:14:46
I then tried to work-around this problem by building darcs-2.2.0 release and got
the same error.  However, this error is sporadic, not reproducible.
msg7354 (view) Author: kowey Date: 2009-02-24.11:57:05
Dmitry, any comments, please?
msg7369 (view) Author: dmitry.kurochkin Date: 2009-03-01.17:11:07
Hi Zooko.

Can you please give me some more info. Is pipelining enabled, what version of
libcurl darcs was compiled with? Can you provide --debug-http log?

How easily you can reproduce the bug? I mean how many times you need to run the
command to trigger the bug.

Regards,
  Dmitry
msg10466 (view) Author: kowey Date: 2010-03-23.18:00:33
Zooko, is the buildslave in question behind a proxy server?
This error message has reappeared in issue1770.  Perhaps there's
something in common between the two?
msg10468 (view) Author: zooko Date: 2010-03-23.19:25:51
No it was not using an HTTP proxy. My assumption is just that the
buildslave was doing darcs get enough times that it uncovered a bug that
occurs sporadically. I think someone (I'm sorry, I forget who)
reproduced this bug just by running while [ 1 ] ; do darcs get ; done.
Maybe that was me who did that.
msg10470 (view) Author: kowey Date: 2010-03-23.19:38:26
Would you mind attempting to produce this with Darcs 2.4 (using your
while loop method, maybe with a counter)

If you can still reproduce this, we'd like to know:

 - libcurl version
 - is pipelining enabled?
 - if you can reproduce it also while using --debug-http

Thanks!
msg10512 (view) Author: kowey Date: 2010-03-25.11:49:19
(still waiting on input from Zooko).

I just want to point out that we're actually seeing several instances of
this error message on the tracker.  issue1489 has another instance,
although there, the specific problem was that the receiving directory
did not have sufficient permissions (which doesn't seem to explain the
intermittent nature of this).  Anyway, it could be a useful clue for
anybody that wants to look into this.
msg10634 (view) Author: kowey Date: 2010-04-02.15:37:37
I've moved msg10620, msg10629, msg10630, msg10632 back to issue1880 (my
merge was premature).  Got my hopes up! (--debug-http output from Zooko
would still be good when he gets a round tuit)
msg11101 (view) Author: zooko Date: 2010-05-24.04:26:43
I'm not going to be able to reproduce this -- that buildslave long since
disappeared from the Net and it isn't coming back.
msg11103 (view) Author: kowey Date: 2010-05-24.07:08:54
Actually, I think this is fixed in this patch:

Thu Apr 15 23:47:39 BST 2010  Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
  * Fix hscurl.c when URL is downloaded during the first call to
curl_multi_perform.
  Turns out that the first call to curl_multi_perform() can fetch the URL or
  result in error. I can easily reproduce this using HTTP server on
localhost.
  This means that situation when running_handles is zero is valid, so
remove the
  error and handle it correctly.
msg11227 (view) Author: fplccl Date: 2010-06-03.22:10:58
Same bug. With specified patches are applied.

user@host $ darcs --exact-version
darcs compiled on Jun  3 2010, at 21:02:56

Context:

[TAG 2.4.4
Eric Kow <kowey@darcs.net>**20100515090819
 Ignore-this: 7d1a0e6a17c2be314f6ab1607bbcac13
] 

/usr/portage/dev-vcs/darcs/darcs-2.4.4.ebuild:
src_configure() {
	...
        cabal_src_configure \
                --flags=curl \
                --flags=-http \
                --flags=curl-pipelining \
                --flags=color \
                --flags=terminfo \
                --flags=mmap \
                $threaded_flag \
                $(cabal_flag test)
}

net-misc/curl-7.20.0-r2

Reproducing.

user@host $ darcs get --lazy
"http://code.haskell.org/gentoo/gentoo-haskell/"

This is the gentoo-haskell darcs overlay.

Please report bugs in the IRC channel #gentoo-haskell on the freenode
network, or send mail to haskell@gentoo.org.

Please do *not* report bugs from this overlay at bugs.gentoo.org.
**********************
darcs: bug at src/URL.hs:246 compiled Jun  3 2010 21:02:56                  
Another possible bug in URL.waitNextUrl:  curl_multi_perform() - no
running handles
msg11233 (view) Author: kowey Date: 2010-06-03.22:48:43
I'm likely abusing this ticket by making it a catch-all for the
curl_multi_perform bugs.  I've copied fplccl's msg11227 here from issue1808.

Fplccl: you said with the specified patches applied.  Are you
specifically talking about Dmitry's patch? "Fix hscurl.c when URL is
downloaded during the first call to curl_multi_perform."

Your darcs --exact-version output (always helpful!) seems to indicate
you have a vanilla darcs 2.4.4, in which case that's no surprise to us
(the fix has not made it to the 2.4 line).

Anyway thanks for your report!  Unfortunately, we won't be issuing a fix
for this until the Darcs 2.5 release (which will include this patch). 
But if you could just confirm that you were just using Darcs 2.4.4
without Dmitry's patch, I'll go back to marking this bug resolved.

If I don't hear from you within a week, I'll eventually just mark this
bug resolved again.  Thanks!
msg11234 (view) Author: dmitry.kurochkin Date: 2010-06-03.22:56:59
I am sure that the darcs version Fplccl used did not have the patch.

The patch *completely* removes this error message. So there is simply no 
way you can get this error with it.

Regards,
  Dmitry
msg11235 (view) Author: fplccl Date: 2010-06-03.23:29:02
Reply to http://bugs.darcs.net/msg11232 (kowey)

>Meanwhile, I'm re-setting this to resolved on the expectation that the
>patch in Darcs HEAD (not 2.4.4, but the unstable Darcs) solves the
>problem.  More later.  If anything else goes wrong, with that patch in
>place. I'd like Stelian to report a new bug.

Maybe it is my mistake.
The humor is in that I have tried to install haskell overlay in Gentoo
for learning Darcs. And have been stopped by this bug.

It seems I have gotten tangled in darcs branches.
I just have done "darcs changes --context" with darcs.net HEAD, have
seen that messages about 2.4.4 are later than those patches. And so I
decided that those patches are already in 2.4.4 tarball.

If there are not those patches in 2.4.4 then it have to delete my messages.

I should try to compile Darcs from darcs. :)
msg11236 (view) Author: fplccl Date: 2010-06-03.23:36:52
Sorry. Mistake with users 'naur' and 'thorkilnaur' in list.
msg11237 (view) Author: kowey Date: 2010-06-04.06:46:05
Fplcc, no problem (Dmitry's point is why I checked; no matter what it
should be impossible to get that particular error message with Darcs
HEAD).  Marking this resolved then.  Good luck!
msg11245 (view) Author: fplccl Date: 2010-06-05.14:48:23
Yes, it works!

Darcs from HEAD is needed in additional dependencies, which absent even
in overlay.
But I have found a pretty command for correcting gentoo ebulds - "darcs
diff --diff-opts=-u".
And also after I have understood the "--match" option, the life has
become much better. :)

So, the Darcs 2.4.4 from the tarball with these two patches gets fine
all tested repos!
History
Date User Action Args
2009-02-23 20:29:30zookocreate
2009-02-23 22:14:49zookosetpriority: bug
nosy: kowey, zooko, simon, thorkilnaur, dmitry.kurochkin
status: unread -> unknown
messages: + msg7349
2009-02-24 11:56:36koweysettopic: + HTTP
nosy: kowey, zooko, simon, thorkilnaur, dmitry.kurochkin
2009-02-24 11:57:07koweysetnosy: kowey, zooko, simon, thorkilnaur, dmitry.kurochkin
messages: + msg7354
assignedto: dmitry.kurochkin
2009-03-01 17:11:10dmitry.kurochkinsetnosy: kowey, zooko, simon, thorkilnaur, dmitry.kurochkin
messages: + msg7369
2009-08-11 00:45:52koweysetstatus: unknown -> waiting-for
nosy: kowey, zooko, simon, thorkilnaur, dmitry.kurochkin
assignedto: dmitry.kurochkin -> zooko
2009-08-25 17:41:15adminsetnosy: + darcs-devel, - simon
2009-08-27 14:22:38adminsetnosy: kowey, darcs-devel, zooko, thorkilnaur, dmitry.kurochkin
2010-03-23 18:00:35koweysetmessages: + msg10466
title: darcs: bug at src/URL.hs:243 -> Another possible bug in URL.waitNextUrl: curl_multi_perform() - no running handles
2010-03-23 19:25:53zookosetmessages: + msg10468
2010-03-23 19:38:28koweysetmessages: + msg10470
2010-03-25 11:49:22koweysetmessages: + msg10512
2010-03-30 13:18:29koweylinkissue1420 superseder
2010-04-02 09:36:46koweylinkissue1808 superseder
2010-04-02 09:39:24koweysetnosy: + sionescu
messages: + msg10620
assignedto: zooko -> sionescu
2010-04-02 13:22:15sionescusetmessages: + msg10629
2010-04-02 13:58:48koweysetnosy: + twb
messages: + msg10630
2010-04-02 15:01:26sionescusetnosy: + naur
messages: + msg10632
2010-04-02 15:34:56koweyunlinkissue1808 superseder
2010-04-02 15:35:28koweysetmessages: - msg10620
2010-04-02 15:35:38koweysetmessages: - msg10629
2010-04-02 15:35:52koweysetmessages: - msg10630
2010-04-02 15:36:02koweysetmessages: - msg10632
2010-04-02 15:37:38koweysetassignedto: sionescu -> zooko
messages: + msg10634
2010-05-24 04:26:44zookosetmessages: + msg11101
2010-05-24 07:08:54koweysetstatus: waiting-for -> resolved
messages: + msg11103
2010-06-03 22:18:09fplcclsetstatus: resolved -> waiting-for
assignedto: zooko -> kowey
messages: + msg11229
nosy: + fplccl
2010-06-03 22:42:52adminsetmessages: + msg11227
2010-06-03 22:44:54fplcclsetmessages: - msg11229
2010-06-03 22:48:44koweysetnosy: - naur
messages: + msg11233
assignedto: kowey -> fplccl
2010-06-03 22:57:00dmitry.kurochkinsetmessages: + msg11234
2010-06-03 23:29:02fplcclsetnosy: - thorkilnaur
messages: + msg11235
2010-06-03 23:36:53fplcclsetnosy: + thorkilnaur
messages: + msg11236
2010-06-04 06:46:06koweysetstatus: waiting-for -> resolved
assignedto: fplccl -> (no value)
messages: + msg11237
2010-06-05 14:37:08fplcclsetstatus: resolved -> waiting-for
assignedto: fplccl
messages: + msg11243
2010-06-05 14:45:59fplcclsetstatus: waiting-for -> resolved
messages: + msg11244
2010-06-05 14:47:43fplcclsetstatus: resolved -> unknown
messages: - msg11244
2010-06-05 14:47:54fplcclsetmessages: - msg11243
2010-06-05 14:48:23fplcclsetstatus: unknown -> resolved
messages: + msg11245