darcs

Issue 736 incorrect "Not a repository" error with mysteriously failed curl download

Title incorrect "Not a repository" error with mysteriously failed curl download
Priority bug Status resolved
Milestone Resolved in
Superseder Nosy List SamB, darcs-devel, dmitry.kurochkin, kowey, thorkilnaur, tommy
Assigned To
Topics

Created on 2008-03-06.15:04:41 by SamB, last changed 2009-08-27.14:09:23 by admin.

Files
File name Uploaded Type Edit Remove
curlm_call_multi_perform.dpatch dmitry.kurochkin, 2008-03-07.02:32:48 text/x-darcs-patch
Messages
msg3780 (view) Author: SamB Date: 2008-03-06.15:04:39
This has been happenning for me for the past few days at least whenever I tried
to use the latest from unstable over HTTP, regardless of repository format:

naesten@hydrogen:~/hacking/haskell/darcs/unstable% ./darcs pull

darcs failed:  Not a repository: http://darcs.net/repos/unstable (Failed to
start download URL http://darcs.net/repos/unstable/_darcs/inventory: please call
curl_multi_perform() soon)
msg3782 (view) Author: droundy Date: 2008-03-06.16:48:27
On Thu, Mar 06, 2008 at 03:04:41PM -0000, Samuel Bronson wrote:
> darcs failed:  Not a repository: http://darcs.net/repos/unstable (Failed to
> start download URL http://darcs.net/repos/unstable/_darcs/inventory: please call
> curl_multi_perform() soon)

Dmitry, do you have any idea what might be triggering this? Sounds like
curl has some sort of limit that we're running into when trying to queue up
many files to grab in parallel.

Samuel, could you rerun this with --debug, and send us the output?

And thanks for the bug report!
-- 
David Roundy
Department of Physics
Oregon State University
msg3785 (view) Author: dmitry.kurochkin Date: 2008-03-06.17:49:54
On Thu, Mar 6, 2008 at 7:43 PM, David Roundy <droundy@darcs.net> wrote:
>  On Thu, Mar 06, 2008 at 03:04:41PM -0000, Samuel Bronson wrote:
>  > darcs failed:  Not a repository: http://darcs.net/repos/unstable (Failed to
>  > start download URL http://darcs.net/repos/unstable/_darcs/inventory: please call
>  > curl_multi_perform() soon)
>
>  Dmitry, do you have any idea what might be triggering this? Sounds like
>  curl has some sort of limit that we're running into when trying to queue up
>  many files to grab in parallel.

At the moment I have no idea of what is happening. I will take a look
at this later today.

I am surprised no one got this before since new curl code has been in
unstable for a while.
It should not be a curl limit issue because:
1. We give curl only 1 or 5 (if pipelining enabled) files at the same time.
2. It fails to download inventory, and it is downloaded before patches
and many other files (I can be mistaking here).

Anyway, I will try to reproduce this and hopefully will be able to
tell you more soon.

Regards,
  Dmitry

>
>  Samuel, could you rerun this with --debug, and send us the output?
>
>  And thanks for the bug report!
>  --
>  David Roundy
>  Department of Physics
>  Oregon State University
>
msg3799 (view) Author: SamB Date: 2008-03-07.01:48:04
On 3/6/08, David Roundy <bugs@darcs.net> wrote:
>  Samuel, could you rerun this with --debug, and send us the output?

naesten@hydrogen:~/hacking/haskell/darcs/unstable.curl% ./darcs pull --debug
Beginning identifying repository .
Done identifying repository .
Beginning identifying repository http://darcs.net/repos/unstable
URL.copyUrl (http://darcs.net/repos/unstable/_darcs/format
          -> /home/naesten/hacking/haskell/darcs/unstable.curl/darcs6QO1sG)
Failed to start download URL
http://darcs.net/repos/unstable/_darcs/format: please call
curl_multi_perform() soon
URL.waitNextUrl succeeded:
http://darcs.net/repos/unstable/_darcs/format
/home/naesten/hacking/haskell/darcs/unstable.curl/darcs6QO1sG
URL.copyUrl (http://darcs.net/repos/unstable/_darcs/inventory
          -> /home/naesten/hacking/haskell/darcs/unstable.curl/darcsQ1rwDL)
Failed to start download URL
http://darcs.net/repos/unstable/_darcs/inventory: please call
curl_multi_perform() soon
Done identifying repository http://darcs.net/repos/unstable

darcs failed:  Not a repository: http://darcs.net/repos/unstable
(Failed to start download URL
http://darcs.net/repos/unstable/_darcs/inventory: please call
curl_multi_perform() soon)
naesten@hydrogen:~/hacking/haskell/darcs/unstable.curl%
msg3800 (view) Author: dmitry.kurochkin Date: 2008-03-07.02:32:48
I was not able to reproduce the bug. But I think I know where the
problem is: any curl multi function can return
CURLM_CALL_MULTI_PERFORM code, not just curl_multi_perform().

Samuel, can you please test the attached patch?

Regards,
  Dmitry
Attachments
msg3826 (view) Author: droundy Date: 2008-03-07.17:20:28
On Fri, Mar 07, 2008 at 05:27:33AM +0300, Dmitry Kurochkin wrote:
> I was not able to reproduce the bug. But I think I know where the
> problem is: any curl multi function can return
> CURLM_CALL_MULTI_PERFORM code, not just curl_multi_perform().

Dmitry, are you happy with this patch? Should I apply it? And thanks for
looking into this!
-- 
David Roundy
Department of Physics
Oregon State University
msg3833 (view) Author: dmitry.kurochkin Date: 2008-03-07.17:28:48
On Fri, Mar 7, 2008 at 8:20 PM, David Roundy <bugs@darcs.net> wrote:
>
>
>  On Fri, Mar 07, 2008 at 05:27:33AM +0300, Dmitry Kurochkin wrote:
>  > I was not able to reproduce the bug. But I think I know where the
>  > problem is: any curl multi function can return
>  > CURLM_CALL_MULTI_PERFORM code, not just curl_multi_perform().
>
>  Dmitry, are you happy with this patch? Should I apply it? And thanks for
>  looking into this!

I am pretty sure hscurl.c changes are good. But I would appreciate if
you look at haskell code. I am not experienced here, comments are very
welcome :)

Anyway, I think patch does not introduce any regression. So it can go in.

Regards,
  Dmitry

>  --
>
> David Roundy
>  Department of Physics
>  Oregon State University
>
>
>
> __________________________________
>  Darcs bug tracker <bugs@darcs.net>
>  <http://bugs.darcs.net/issue736>
>  __________________________________
>
History
Date User Action Args
2008-03-06 15:04:41SamBcreate
2008-03-06 16:48:29droundysetstatus: unread -> unknown
nosy: + dmitry.kurochkin
messages: + msg3782
2008-03-06 17:49:56dmitry.kurochkinsetnosy: droundy, tommy, beschmi, kowey, SamB, dmitry.kurochkin
messages: + msg3785
2008-03-07 01:48:05SamBsetnosy: droundy, tommy, beschmi, kowey, SamB, dmitry.kurochkin
messages: + msg3799
2008-03-07 02:32:49dmitry.kurochkinsetfiles: + curlm_call_multi_perform.dpatch
nosy: droundy, tommy, beschmi, kowey, SamB, dmitry.kurochkin
messages: + msg3800
2008-03-07 17:20:32droundysetnosy: droundy, tommy, beschmi, kowey, SamB, dmitry.kurochkin
messages: + msg3826
2008-03-07 17:28:49dmitry.kurochkinsetnosy: droundy, tommy, beschmi, kowey, SamB, dmitry.kurochkin
messages: + msg3833
2008-03-09 03:26:46SamBsetstatus: unknown -> resolved-in-unstable
nosy: droundy, tommy, beschmi, kowey, SamB, dmitry.kurochkin
2008-09-04 21:32:54adminsetstatus: resolved-in-unstable -> resolved
nosy: + dagit
2009-08-06 17:56:02adminsetnosy: + markstos, jast, Serware, darcs-devel, zooko, mornfall, simon, thorkilnaur, - droundy, SamB
2009-08-06 21:00:00adminsetnosy: - beschmi
2009-08-10 22:17:30adminsetnosy: + SamB, - markstos, darcs-devel, zooko, jast, Serware, mornfall
2009-08-11 00:08:21adminsetnosy: - dagit
2009-08-25 18:07:19adminsetnosy: + darcs-devel, - simon
2009-08-27 14:09:23adminsetnosy: tommy, kowey, darcs-devel, SamB, thorkilnaur, dmitry.kurochkin