darcs

Issue 920 darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining)

Title darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining)
Priority bug Status resolved
Milestone 2.1.x Resolved in
Superseder get => Possible bug in URL.waitNextUrl (2.0.2 + 20)
View: 977
Nosy List darcs-devel, dmitry.kurochkin, eivuokko, jaredj, kirby, kowey, marlowsd, mornfall, thorkilnaur, tommy, wglozer, zooko
Assigned To
Topics HTTP, Windows

Created on 2008-06-13.09:36:27 by kowey, last changed 2010-06-15.21:48:04 by admin.

Files
File name Uploaded Type Edit Remove
get-hinderer.log kowey, 2008-06-20.16:50:19 application/octet-stream
log-fetching-ndp.bz2 kowey, 2008-06-13.09:36:24 application/octet-stream
Messages
msg5032 (view) Author: kowey Date: 2008-06-13.09:36:24
On Windows, I have a pretty systematic problem trying to darcs get
http://darcs.haskell.org/packages/ndp
It's systematic in that darcs always stops printing things at that last patch,
and then the computer starts to work very hard (the fan starts whirring angrily)

The workaround is simple: do not enable --with-curl-pipelining

Here is the tail of the debug log.  Full log attached
Fri Jun 13 10:31:52 GMT Daylight Time 2008: URL.waitNextUrl succeeded:
http://darcs.haskell.org/packages/ndp/_darcs/patches/20060314143402-6295e-28ca67f2a4ddc04528a0844ea9e318629391028e.gz
./_darcs/patches/20060314143402-6295e-28ca67f2a4ddc04528a0844ea9e318629391028e.gz
Fri Jun 13 10:31:52 GMT Daylight Time 2008: URL.copyUrl
(http://darcs.haskell.org/packages/ndp/_darcs/patches/20060313231445-6295e-30c68bd46752f4aba04a990871c3f0393d381a91.gz
          ->
./_darcs/patches/20060313231445-6295e-30c68bd46752f4aba04a990871c3f0393d381a91.gz)
Fri Jun 13 10:31:52 GMT Daylight Time 2008: URL.copyUrl succeeded
Fri Jun 13 10:32:16 GMT Daylight Time 2008: URL.waitNextUrl succeeded:
http://darcs.haskell.org/packages/ndp/_darcs/patches/20060313231445-6295e-30c68bd46752f4aba04a990871c3f0393d381a91.gz
./_darcs/patches/20060313231445-6295e-30c68bd46752f4aba04a990871c3f0393d381a91.gz
Fri Jun 13 10:32:16 GMT Daylight Time 2008: URL.copyUrl
(http://darcs.haskell.org/packages/ndp/_darcs/patches/20060313110056-760e2-120f94c56a7dc2e96a9b78d51e1e7e359cd8db1c.gz
          ->
./_darcs/patches/20060313110056-760e2-120f94c56a7dc2e96a9b78d51e1e7e359cd8db1c.gz)
Fri Jun 13 10:32:16 GMT Daylight Time 2008: URL.copyUrl succeeded
Fri Jun 13 10:32:16 GMT Daylight Time 2008: URL.waitNextUrl succeeded:
http://darcs.haskell.org/packages/ndp/_darcs/patches/20060313110056-760e2-120f94c56a7dc2e96a9b78d51e1e7e359cd8db1c.gz
./_darcs/patches/20060313110056-760e2-120f94c56a7dc2e96a9b78d51e1e7e359cd8db1c.gz
Attachments
msg5035 (view) Author: dmitry.kurochkin Date: 2008-06-13.12:01:13
Hi Eric.

It seems I can not reproduce this on debian. Both debian darcs 2.0.0 (with
pipelining) and darcs 2.0.1rc2 (also with pipelining) work fine. Printing stops
near the end (few patches before) and I see network activity. But it stops soon
and darcs get succeeds. I did not check if this freeze at the end is related to
pipelining.

Few questions.

1. What curl version are you using? 7.18.2 was released recently and it has some
pipelining fixes. If you are not using the latest libcurl you should really
upgrade (especially for pipelining).

2. Can you reproduce this on linux or this is windows specific?

3. What do you mean by "init then pull  -a"? You init a fresh darcs repo and
then do pull from ndp repo, I suppose? Can you give exact commands please.

Regards,
  Dmitry
msg5036 (view) Author: kowey Date: 2008-06-13.12:47:23
Hi,

> It seems I can not reproduce this on debian. Both debian darcs 2.0.0 (with
> pipelining) and darcs 2.0.1rc2 (also with pipelining) work fine. Printing stops
> near the end (few patches before) and I see network activity. But it stops soon
> and darcs get succeeds. I did not check if this freeze at the end is related to
> pipelining.

Thanks!

> 1. What curl version are you using? 7.18.2 was released recently and it has some
> pipelining fixes. If you are not using the latest libcurl you should really
> upgrade (especially for pipelining).

7.18.2

> 2. Can you reproduce this on linux or this is windows specific?

It is Windows-specific.  I was always able to fetch this repository on Linux.

> 3. What do you mean by "init then pull  -a"? You init a fresh darcs repo and
> then do pull from ndp repo, I suppose? Can you give exact commands please.

Yes.

mkdir foo
cd foo
darcs init
darcs pull http://darcs.haskell.org/packages/ndp --debug-verbose --timing
msg5069 (view) Author: dmitry.kurochkin Date: 2008-06-16.20:28:03
I am trying to build darcs on windows in virtualbox. And I have encountered few
problems.

1. GHC can not find zlib. I set LDFLAGS=/usr/lib and ghc is run with
-optl-L/usr/lib but linker still fails. If ghc is run with -L instead of -optl-L
it works fine. So I patched configure and makefile to leave -L option as is. Am
I the only one having this problem? Or I just do it wrong on windows?

2. Compilation seems to go fine, but make fails when list_authors is run. It
fails with:

Segmentation fault/access violation in generated code

darcs binary prints help when run with no parameters. But when run with whats
argument it also segfaults. Any idea what the problem can be?

Regards,
  Dmitry
msg5071 (view) Author: kowey Date: 2008-06-16.21:08:22
On Mon, Jun 16, 2008 at 20:28:05 -0000, dmitry.kurochkin wrote:
> I am trying to build darcs on windows in virtualbox. And I have encountered few
> problems.

For what it's worth, the instructions I use are
  http://wiki.darcs.net/index.html/BuildingUnderWindows

Do you think you could update that page if you ever figure out what's
going on

> 1. GHC can not find zlib. I set LDFLAGS=/usr/lib and ghc is run with
> -optl-L/usr/lib but linker still fails. If ghc is run with -L instead of -optl-L
> it works fine. So I patched configure and makefile to leave -L option as is. Am
> I the only one having this problem? Or I just do it wrong on windows?

I think I just copied them to places so that I wouldn't have to set any
compiler flags

> 2. Compilation seems to go fine, but make fails when list_authors is run. It
> fails with:
> 
> Segmentation fault/access violation in generated code

Err... not sure!  I have experienced something like this, but then I did
make clean, and everything worked
msg5076 (view) Author: dmitry.kurochkin Date: 2008-06-16.21:36:56
On Tue, Jun 17, 2008 at 1:08 AM, Eric Kow <bugs@darcs.net> wrote:
>
> Eric Kow <eric.kow@gmail.com> added the comment:
>
> On Mon, Jun 16, 2008 at 20:28:05 -0000, dmitry.kurochkin wrote:
>> I am trying to build darcs on windows in virtualbox. And I have encountered few
>> problems.
>
> For what it's worth, the instructions I use are
>  http://wiki.darcs.net/index.html/BuildingUnderWindows
>
> Do you think you could update that page if you ever figure out what's
> going on
>
>> 1. GHC can not find zlib. I set LDFLAGS=/usr/lib and ghc is run with
>> -optl-L/usr/lib but linker still fails. If ghc is run with -L instead of -optl-L
>> it works fine. So I patched configure and makefile to leave -L option as is. Am
>> I the only one having this problem? Or I just do it wrong on windows?
>
> I think I just copied them to places so that I wouldn't have to set any
> compiler flags

Yes, the wiki page tells you to copy libs to ghc. This solution seems
to be really bad to me (but maybe that is ok for windows :)).

>
>> 2. Compilation seems to go fine, but make fails when list_authors is run. It
>> fails with:
>>
>> Segmentation fault/access violation in generated code
>
> Err... not sure!  I have experienced something like this, but then I did
> make clean, and everything worked

That did not help. I ran it from explorer and it showed me a missing
dll error, I copied it and now it is working. I am trying to get
make_changelog to work now. I know there were a windows tool similar
to ldd which shows which dll's are required... I should google for it.

It seems that I will get a working build without library copying soon.
After that I want to get a static windows build, and check that darcs
still builds on linux with my changes. Then I will submit the patch
for -optl-L handling.

Actually I am puzzled why ghc does not handle -optl-L properly. Maybe
there is a better solution than handling -L separately...

Regards,
  Dmitry

>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue920>
> __________________________________
>
msg5081 (view) Author: simonmar Date: 2008-06-17.10:12:56
Dmitry Kurochkin wrote:
> On Tue, Jun 17, 2008 at 1:08 AM, Eric Kow <bugs@darcs.net> wrote:
>> Eric Kow <eric.kow@gmail.com> added the comment:
>>
>> On Mon, Jun 16, 2008 at 20:28:05 -0000, dmitry.kurochkin wrote:
>>> I am trying to build darcs on windows in virtualbox. And I have encountered few
>>> problems.
>> For what it's worth, the instructions I use are
>>  http://wiki.darcs.net/index.html/BuildingUnderWindows
>>
>> Do you think you could update that page if you ever figure out what's
>> going on
>>
>>> 1. GHC can not find zlib. I set LDFLAGS=/usr/lib and ghc is run with
>>> -optl-L/usr/lib but linker still fails. If ghc is run with -L instead of -optl-L
>>> it works fine. So I patched configure and makefile to leave -L option as is. Am
>>> I the only one having this problem? Or I just do it wrong on windows?
>> I think I just copied them to places so that I wouldn't have to set any
>> compiler flags
> 
> Yes, the wiki page tells you to copy libs to ghc. This solution seems
> to be really bad to me (but maybe that is ok for windows :)).

I do it without copying, but the darcs build system isn't set up well for 
this scenario.  I do the following:

1. comment out the line 'CURLLIBS=' in configure.ac
2. autoconf
3. run this script (adjust the paths accordingly)

export CURLCFLAGS="-Ic:/builds/curl-7.16.0/include -DCURL_STATICLIB"
export CURLLIBS="-Lc:/builds/curl-7.16.0 -lcurl -lwinmm -lws2_32 -lws2_32"
export CPPFLAGS=-Ic:/builds/zlib-1.2.3
export LDFLAGS=-Lc:/builds/zlib-1.2.3
export CC=c:/mingw/bin/gcc

./configure --without-libcurl --target=mingw

4. build as normal

Cheers,
	Simon
msg5084 (view) Author: dmitry.kurochkin Date: 2008-06-17.10:46:26
On Tue, Jun 17, 2008 at 2:21 PM, Simon Marlow <marlowsd@gmail.com> wrote:
> I do it without copying, but the darcs build system isn't set up well for
> this scenario.  I do the following:
>
> 1. comment out the line 'CURLLIBS=' in configure.ac
> 2. autoconf
> 3. run this script (adjust the paths accordingly)
>
> export CURLCFLAGS="-Ic:/builds/curl-7.16.0/include -DCURL_STATICLIB"
> export CURLLIBS="-Lc:/builds/curl-7.16.0 -lcurl -lwinmm -lws2_32 -lws2_32"
> export CPPFLAGS=-Ic:/builds/zlib-1.2.3
> export LDFLAGS=-Lc:/builds/zlib-1.2.3
> export CC=c:/mingw/bin/gcc
>
> ./configure --without-libcurl --target=mingw
>
> 4. build as normal

I was able to build statically linked darcs:
1. Build zlib from source from official site (not mingw port).
   ./configure --prefix=/usr
   make
   make install
2. Build curl from source from official site.
   ./configure --prefix=/usr
   make
   make install
3. Build darcs
   CPPFLAGS=-I/usr/include LDFLAGS=-L/usr/lib ./configure
--prefix=/usr --with-static-libs --with-curl-pipelining
   make

There was only one problem. LDFLAGS -L option is converted to -optl-L
ghc option. And, by unknown reason, linker can not find libraries in
that directory. When run with -L option it works fine. Tried on ghc
6.8.2 and 6.8.3.

But it seems you do not have such problems, do you? Any ideas what
should be done about this? Is this a ghc problem or I should submit
patch for darcs which leaves -L options unchanged?

Regards,
  Dmitry

>
> Cheers,
>        Simon
>
msg5085 (view) Author: simonmar Date: 2008-06-17.11:12:53
dmitry.kurochkin wrote:
> dmitry.kurochkin <dmitry.kurochkin@gmail.com> added the comment:
> 
> On Tue, Jun 17, 2008 at 2:21 PM, Simon Marlow <marlowsd@gmail.com> wrote:
>> I do it without copying, but the darcs build system isn't set up well for
>> this scenario.  I do the following:
>>
>> 1. comment out the line 'CURLLIBS=' in configure.ac
>> 2. autoconf
>> 3. run this script (adjust the paths accordingly)
>>
>> export CURLCFLAGS="-Ic:/builds/curl-7.16.0/include -DCURL_STATICLIB"
>> export CURLLIBS="-Lc:/builds/curl-7.16.0 -lcurl -lwinmm -lws2_32 -lws2_32"
>> export CPPFLAGS=-Ic:/builds/zlib-1.2.3
>> export LDFLAGS=-Lc:/builds/zlib-1.2.3
>> export CC=c:/mingw/bin/gcc
>>
>> ./configure --without-libcurl --target=mingw
>>
>> 4. build as normal
> 
> I was able to build statically linked darcs:
> 1. Build zlib from source from official site (not mingw port).
>    ./configure --prefix=/usr
>    make
>    make install
> 2. Build curl from source from official site.
>    ./configure --prefix=/usr
>    make
>    make install

make install is scary.  Are you using MSYS or Cygwin?  There's no guarantee 
that GHC can find /usr, because that's a magic path only known to MSYS or 
Cygwin, and GHC is a native Windows program.  (hence why I didn't install 
my zlib or libcurl, incedentally).

> 3. Build darcs
>    CPPFLAGS=-I/usr/include LDFLAGS=-L/usr/lib ./configure
> --prefix=/usr --with-static-libs --with-curl-pipelining
>    make
> 
> There was only one problem. LDFLAGS -L option is converted to -optl-L
> ghc option. And, by unknown reason, linker can not find libraries in
> that directory. When run with -L option it works fine. Tried on ghc
> 6.8.2 and 6.8.3.

Maybe it's the /usr problem above.  You could try with --prefix=c:/lib, or 
something.

Cheers,
	Simon
msg5087 (view) Author: dmitry.kurochkin Date: 2008-06-17.11:49:12
On Tue, Jun 17, 2008 at 3:21 PM, Simon Marlow <marlowsd@gmail.com> wrote:
> dmitry.kurochkin wrote:
>>
>> dmitry.kurochkin <dmitry.kurochkin@gmail.com> added the comment:
>>
>> On Tue, Jun 17, 2008 at 2:21 PM, Simon Marlow <marlowsd@gmail.com> wrote:
>>>
>>> I do it without copying, but the darcs build system isn't set up well for
>>> this scenario.  I do the following:
>>>
>>> 1. comment out the line 'CURLLIBS=' in configure.ac
>>> 2. autoconf
>>> 3. run this script (adjust the paths accordingly)
>>>
>>> export CURLCFLAGS="-Ic:/builds/curl-7.16.0/include -DCURL_STATICLIB"
>>> export CURLLIBS="-Lc:/builds/curl-7.16.0 -lcurl -lwinmm -lws2_32
>>> -lws2_32"
>>> export CPPFLAGS=-Ic:/builds/zlib-1.2.3
>>> export LDFLAGS=-Lc:/builds/zlib-1.2.3
>>> export CC=c:/mingw/bin/gcc
>>>
>>> ./configure --without-libcurl --target=mingw
>>>
>>> 4. build as normal
>>
>> I was able to build statically linked darcs:
>> 1. Build zlib from source from official site (not mingw port).
>>   ./configure --prefix=/usr
>>   make
>>   make install
>> 2. Build curl from source from official site.
>>   ./configure --prefix=/usr
>>   make
>>   make install
>
> make install is scary.  Are you using MSYS or Cygwin?  There's no guarantee
> that GHC can find /usr, because that's a magic path only known to MSYS or
> Cygwin, and GHC is a native Windows program.  (hence why I didn't install my
> zlib or libcurl, incedentally).

I use msys. make install is scary indeed. I would have never done make
install in /usr on my debian system :) But in msys it is all in a
single directory. And there is no package manager anyway. So make
install in /usr does not seem so bad to me.

>
>> 3. Build darcs
>>   CPPFLAGS=-I/usr/include LDFLAGS=-L/usr/lib ./configure
>> --prefix=/usr --with-static-libs --with-curl-pipelining
>>   make
>>
>> There was only one problem. LDFLAGS -L option is converted to -optl-L
>> ghc option. And, by unknown reason, linker can not find libraries in
>> that directory. When run with -L option it works fine. Tried on ghc
>> 6.8.2 and 6.8.3.
>
> Maybe it's the /usr problem above.  You could try with --prefix=c:/lib, or
> something.

You are absolutely right! If I set path to -optl-Lc:/msys/1.0/lib it
works. So the final configure command line looks like:

CPPFLAGS=-Ic:/msys/1.0/include LDFLAGS=-Lc:/msys/1.0/lib ./configure
--prefix=/usr --with-static-libs --with-curl-pipelining

Now I did few more tests to understand -L vs -optl-L difference. It
turns out that when run with -L ghc runs collect2.exe with -L path
"converted" by msys(?): from /usr/lib to C:/msys/1.0/lib. And collect2
finds libs with no problems. But if run with -optl-L option, the path
is not converted. And collect2.exe is run with -L/usr/lib...

I am not an expert in all this "unix on windows" stuff. Do you think
this is a ghc bug and should be fixed there or just leave it as is?

Anyway I guess there is no point in making darcs to handle -L options specially.

Thanks,
  Dmitry

>
> Cheers,
>        Simon
>
msg5089 (view) Author: simonmar Date: 2008-06-17.13:03:31
dmitry.kurochkin wrote:
> dmitry.kurochkin <dmitry.kurochkin@gmail.com> added the comment:
> 
> On Tue, Jun 17, 2008 at 3:21 PM, Simon Marlow <marlowsd@gmail.com> wrote:
>> dmitry.kurochkin wrote:
>>> dmitry.kurochkin <dmitry.kurochkin@gmail.com> added the comment:
>>>
>>> On Tue, Jun 17, 2008 at 2:21 PM, Simon Marlow <marlowsd@gmail.com> wrote:
>>>> I do it without copying, but the darcs build system isn't set up well for
>>>> this scenario.  I do the following:
>>>>
>>>> 1. comment out the line 'CURLLIBS=' in configure.ac
>>>> 2. autoconf
>>>> 3. run this script (adjust the paths accordingly)
>>>>
>>>> export CURLCFLAGS="-Ic:/builds/curl-7.16.0/include -DCURL_STATICLIB"
>>>> export CURLLIBS="-Lc:/builds/curl-7.16.0 -lcurl -lwinmm -lws2_32
>>>> -lws2_32"
>>>> export CPPFLAGS=-Ic:/builds/zlib-1.2.3
>>>> export LDFLAGS=-Lc:/builds/zlib-1.2.3
>>>> export CC=c:/mingw/bin/gcc
>>>>
>>>> ./configure --without-libcurl --target=mingw
>>>>
>>>> 4. build as normal
>>> I was able to build statically linked darcs:
>>> 1. Build zlib from source from official site (not mingw port).
>>>   ./configure --prefix=/usr
>>>   make
>>>   make install
>>> 2. Build curl from source from official site.
>>>   ./configure --prefix=/usr
>>>   make
>>>   make install
>> make install is scary.  Are you using MSYS or Cygwin?  There's no guarantee
>> that GHC can find /usr, because that's a magic path only known to MSYS or
>> Cygwin, and GHC is a native Windows program.  (hence why I didn't install my
>> zlib or libcurl, incedentally).
> 
> I use msys. make install is scary indeed. I would have never done make
> install in /usr on my debian system :) But in msys it is all in a
> single directory. And there is no package manager anyway. So make
> install in /usr does not seem so bad to me.
> 
>>> 3. Build darcs
>>>   CPPFLAGS=-I/usr/include LDFLAGS=-L/usr/lib ./configure
>>> --prefix=/usr --with-static-libs --with-curl-pipelining
>>>   make
>>>
>>> There was only one problem. LDFLAGS -L option is converted to -optl-L
>>> ghc option. And, by unknown reason, linker can not find libraries in
>>> that directory. When run with -L option it works fine. Tried on ghc
>>> 6.8.2 and 6.8.3.
>> Maybe it's the /usr problem above.  You could try with --prefix=c:/lib, or
>> something.
> 
> You are absolutely right! If I set path to -optl-Lc:/msys/1.0/lib it
> works. So the final configure command line looks like:
> 
> CPPFLAGS=-Ic:/msys/1.0/include LDFLAGS=-Lc:/msys/1.0/lib ./configure
> --prefix=/usr --with-static-libs --with-curl-pipelining
> 
> Now I did few more tests to understand -L vs -optl-L difference. It
> turns out that when run with -L ghc runs collect2.exe with -L path
> "converted" by msys(?): from /usr/lib to C:/msys/1.0/lib. And collect2
> finds libs with no problems. But if run with -optl-L option, the path
> is not converted. And collect2.exe is run with -L/usr/lib...
> 
> I am not an expert in all this "unix on windows" stuff. Do you think
> this is a ghc bug and should be fixed there or just leave it as is?

This is just an MSYS-ism, I think.  When you run a command under MSYS, it 
applies path conversion to paths it finds on the command line according to 
a bunch of heuristics.  Apparently it recognises -L/usr/lib as a flag with 
a path embedded in it, but not -optl-L/usr/lib.  Yet another reason not to 
use MSYS, IMO (or at least, not to mix MSYS paths with GHC).

Cheers,
	Simon
msg5117 (view) Author: kowey Date: 2008-06-20.16:50:19
For what it's worth, I had trouble getting a repository under Linux with
curl-pipelining.

The repository is http://www.loria.fr/~hinderer/these

And the debug-verbose/timings output is attached.  The same symptoms apply; it
stays 'stuck' on copying the first patch.  Darcs init and darcs pull -a lets me
fetch all the patches, as does using a version without pipelining.  Note,
however, that this is with an older libcurl (7.18.0)... but hopefully this
report will give some ideas
Attachments
msg5118 (view) Author: kowey Date: 2008-06-20.16:53:59
Note also that if you do a grep and wc on this log (of copyUrl and succeeded),
it appears to successfully retrieve all the files it wants... just not notice
that it has done so.
msg5125 (view) Author: dmitry.kurochkin Date: 2008-06-20.18:33:27
Hi Eric.

On Fri, Jun 20, 2008 at 8:50 PM, Eric Kow <bugs@darcs.net> wrote:
>
> Eric Kow <eric.kow@gmail.com> added the comment:
>
> For what it's worth, I had trouble getting a repository under Linux with
> curl-pipelining.
>
> The repository is http://www.loria.fr/~hinderer/these
>
> And the debug-verbose/timings output is attached.  The same symptoms apply; it
> stays 'stuck' on copying the first patch.  Darcs init and darcs pull -a lets me
> fetch all the patches, as does using a version without pipelining.  Note,
> however, that this is with an older libcurl (7.18.0)... but hopefully this
> report will give some ideas

I will try to reproduce it. I was not able to reproduce stuck on
windows with the first repo. Run it few times and it seems to work
fine.

In your original report you said that it stucks on the last patch. Now
you say it hangs on the first one. Is that a mistake?

If I am still not able to reproduce we can try to nail it with curl
debug output enabled.

Regards,
  Dmitry

>
> ----------
> nosy:  -marlowsd
> title: darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining, Windows) -> darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining)
>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue920>
> __________________________________
msg5127 (view) Author: kowey Date: 2008-06-20.21:18:22
On Fri, Jun 20, 2008 at 22:33:15 +0400, Dmitry Kurochkin wrote:
> In your original report you said that it stucks on the last patch. Now
> you say it hangs on the first one. Is that a mistake?

Huh... actually, I think the progress report always stays stuck on
'copying patch 1/x', but the debug-verbose stuff stops printing
at the last patch...

> If I am still not able to reproduce we can try to nail it with curl
> debug output enabled.

I don't know if this matters, but I'm also behind a proxy server...
msg5129 (view) Author: dmitry.kurochkin Date: 2008-06-20.22:27:02
On Sat, Jun 21, 2008 at 1:18 AM, Eric Kow <bugs@darcs.net> wrote:
>
> Eric Kow <eric.kow@gmail.com> added the comment:
>
> On Fri, Jun 20, 2008 at 22:33:15 +0400, Dmitry Kurochkin wrote:
>> In your original report you said that it stucks on the last patch. Now
>> you say it hangs on the first one. Is that a mistake?
>
> Huh... actually, I think the progress report always stays stuck on
> 'copying patch 1/x', but the debug-verbose stuff stops printing
> at the last patch...

Ok. I am still not able to reproduce. From log it looks like curl does
not get the second patch.
This is most likely a curl bug. It would really help if you update to
latest libcurl release. Or even better to HEAD/daily snapshot. AFAIK
there was at least one pipelining related bug fixed since 7.18.2.

Then I have just sent a patch which adds --enable-curl-debug configure
option. Build darcs with it and send the log.

Also it would help if you can send a traffic dump. Somthing like
tcpdump -w darcs.pcap port <proxy port> should do the trick. And try
to stop other proxy-using apps during the test.

One thing that bothers me is why darcs pull succeeds? AFAIK patch
getting code is the same for get/pull. Do you have an idea?

>
>> If I am still not able to reproduce we can try to nail it with curl
>> debug output enabled.
>
> I don't know if this matters, but I'm also behind a proxy server...

That probably matters...

Regards,
  Dmitry

>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue920>
> __________________________________
>
msg5139 (view) Author: kowey Date: 2008-06-23.13:42:00
On Linux, it appears to succeed with curl 7.19 nightly (20080623).  Do you still
want the other information?

I will try and see if I get the same better results on Windows.
msg5140 (view) Author: dmitry.kurochkin Date: 2008-06-23.13:50:06
On Mon, Jun 23, 2008 at 5:42 PM, Eric Kow <bugs@darcs.net> wrote:
>
> Eric Kow <eric.kow@gmail.com> added the comment:
>
> On Linux, it appears to succeed with curl 7.19 nightly (20080623).  Do you still
> want the other information?

Good news! No info is needed in this case.

>
> I will try and see if I get the same better results on Windows.

That would be great. If it still fails then I need additional info
(curl debug log and traffic dump) from the windows machine. BTW is
this windows box in the same network as the linux machine (uses the
same proxy)?

Regards,
  Dmitry

>
> ----------
> nosy:  -marlowsd
>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue920>
> __________________________________
>
msg5142 (view) Author: kowey Date: 2008-06-23.14:46:24
Hmm :-(

Some more details on this.  It's still broken under Windows.

Also, under Linux, I remember that I have recently upgraded to GHC 6.8.3 (from
6.8.2)... I tried again with the old libcurl 7.18.0 and it still works!  (the
darcs that was built with GHC 6.8.2 does not).

So I'll first have to try rebuilding darcs on Windows with the old libcurl and
the new GHC.

Confusing!
msg5445 (view) Author: mornfall Date: 2008-08-12.22:25:16
I might be overestimating the threading + curl problems, but I wouldn't be that
surprised if this is another manifestation of that (see issue876 and issue977).

Fri May  9 21:58:33 CEST 2008  David Roundy <droundy@darcs.net>
  * resolve Issue739: compile with threaded runtime by default.
  It turns out that the tempfile-removal cleanup wasn't particularly feasible
  without the threaded runtime.     

As far as I can gather, all the reports about hanging/livelocking get are with
darcs versions newer than this change. 2.0.1rc1 includes this as well.
msg5656 (view) Author: kowey Date: 2008-08-22.17:41:47
I need a volunteer to check that darcs with curl-pipelining can download this
repository on Windows:

  http://darcs.haskell.org/packages/ndp

If so, we can likely close this bug.
msg5709 (view) Author: simonmar Date: 2008-08-26.13:48:56
Eric Kow wrote:
> Eric Kow <eric.kow@gmail.com> added the comment:
> 
> I need a volunteer to check that darcs with curl-pipelining can download this
> repository on Windows:
> 
>   http://darcs.haskell.org/packages/ndp
> 
> If so, we can likely close this bug.

I managed to get that repo using my build of darcs-2.0.2 (+75) on Windows. 
  It has curl pipelining enabled, but I'm not sure how to tell whether 
pipelining was being used or not.

Cheers,
	Simon
msg5710 (view) Author: dmitry.kurochkin Date: 2008-08-26.13:57:10
Simon, take a look at my comments for issue987 - there are instructions on how
to see if pipelining is actually used. You can just make a pcap traffic dump
(IIRC ndp repo is not very big) and send it to me.

BTW do you use HTTP proxy servers?

Regards,
  Dmitry
msg5933 (view) Author: kowey Date: 2008-09-08.07:40:21
Simon can get this repository, so I'd be happy to close this bug.
History
Date User Action Args
2008-06-13 09:36:27koweycreate
2008-06-13 12:01:15dmitry.kurochkinsetstatus: unread -> unknown
nosy: + dmitry.kurochkin
messages: + msg5035
2008-06-13 12:47:24koweysetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd
messages: + msg5036
title: darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining, Windows) -> darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining, Windows)
2008-06-16 20:28:05dmitry.kurochkinsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd
messages: + msg5069
2008-06-16 21:08:24koweysetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd
messages: + msg5071
2008-06-16 21:37:00dmitry.kurochkinsetnosy: + marlowsd
messages: + msg5076
2008-06-17 10:12:58marlowsdsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5081
2008-06-17 10:46:28dmitry.kurochkinsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5084
2008-06-17 11:12:55marlowsdsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5085
2008-06-17 11:49:15dmitry.kurochkinsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5087
2008-06-17 13:03:33marlowsdsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5089
2008-06-20 16:50:22koweysetfiles: + get-hinderer.log
nosy: - marlowsd
messages: + msg5117
title: darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining, Windows) -> darcs get hangs; init then pull -a works (2.0.1rc1+, curl-pipelining)
2008-06-20 16:54:02koweysetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd
messages: + msg5118
2008-06-20 18:33:31dmitry.kurochkinsetnosy: + marlowsd
messages: + msg5125
2008-06-20 21:18:26koweysetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5127
2008-06-20 22:27:06dmitry.kurochkinsetnosy: tommy, beschmi, kowey, wglozer, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, marlowsd
messages: + msg5129
2008-06-23 13:42:02koweysetnosy: - marlowsd
messages: + msg5139
2008-06-23 13:50:10dmitry.kurochkinsetnosy: + marlowsd
messages: + msg5140
2008-06-23 14:46:27koweysetnosy: - marlowsd
messages: + msg5142
2008-08-12 22:25:19mornfallsetnosy: + mornfall, marlowsd, - marlowsd
messages: + msg5445
2008-08-22 17:41:50koweysetstatus: unknown -> needs-reproduction
nosy: + darcs-devel
superseder: + get => Possible bug in URL.waitNextUrl (2.0.2 + 20)
messages: + msg5656
2008-08-22 17:42:36koweysettopic: + Target-2.1
nosy: tommy, beschmi, kowey, wglozer, darcs-devel, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, mornfall
2008-08-25 09:31:16koweysettopic: + HTTP
nosy: tommy, beschmi, kowey, wglozer, darcs-devel, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, mornfall
2008-08-26 13:48:58marlowsdsetnosy: tommy, beschmi, kowey, wglozer, darcs-devel, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, mornfall
messages: + msg5709
2008-08-26 13:57:13dmitry.kurochkinsetnosy: tommy, beschmi, kowey, wglozer, darcs-devel, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, mornfall
messages: + msg5710
2008-09-08 07:40:23koweysetstatus: needs-reproduction -> resolved-in-unstable
nosy: tommy, beschmi, kowey, wglozer, darcs-devel, zooko, eivuokko, dagit, jaredj, dmitry.kurochkin, marlowsd, mornfall
messages: + msg5933
2009-04-22 03:29:55twbsetstatus: resolved-in-unstable -> resolved
nosy: + simon, thorkilnaur
2009-08-06 21:07:57adminsetnosy: - beschmi
2009-08-11 00:17:07adminsetnosy: - dagit
2009-08-25 17:44:05adminsetnosy: - simon
2009-08-27 14:18:14adminsetnosy: tommy, kowey, wglozer, darcs-devel, zooko, eivuokko, thorkilnaur, jaredj, dmitry.kurochkin, marlowsd, mornfall
2010-06-15 21:48:02adminsetmilestone: 2.1.x
2010-06-15 21:48:04adminsettopic: - Target-2.1
nosy: + kirby