darcs

Issue 696 darcs transfer-mode not working?

Title darcs transfer-mode not working?
Priority bug Status given-up
Milestone Resolved in
Superseder Nosy List darcs-devel, dmitry.kurochkin, kowey, manveru, thorkilnaur, tommy
Assigned To
Topics Darcs2, Performance

Created on 2008-02-16.16:31:39 by manveru, last changed 2009-08-27.14:12:44 by admin.

Messages
msg3494 (view) Author: manveru Date: 2008-02-16.16:31:38
I've just tried out latest darcs from the unstable and converted our repo to hashed format: http://darcs.ramaze.net/ramaze.hashed

For some reason, get --lazy via ssh takes an hour, while the same from http takes 7 minutes. A complete `scp -r` is in between. Wouldn't it be possible to improve speed of get, maybe by using a remote command to tar.(bz2|gz) the whole repo and d/l that or using scp -r?

I tried following:
scp -r ramaze.net:ramaze.hashed .  1.21s user 8.50s system 0% cpu 38:51.73 total
darcs get --lazy ramaze.net:ramaze.hashed ramaze  6.08s user 4.97s system 0% cpu 59:59.34 total
darcs get --lazy http://darcs.ramaze.net/ramaze.hashed ramaze.hashed.lazy.http  1.09s user 2.41s system 0% cpu 7:33.57 total
msg3580 (view) Author: droundy Date: 2008-02-19.15:36:42
On Sat, Feb 16, 2008 at 04:31:39PM -0000, Michael Fellinger wrote:
> I've just tried out latest darcs from the unstable and converted our repo
> to hashed format: http://darcs.ramaze.net/ramaze.hashed
> 
> For some reason, get --lazy via ssh takes an hour, while the same from
> http takes 7 minutes. A complete `scp -r` is in between. Wouldn't it be
> possible to improve speed of get, maybe by using a remote command to
> tar.(bz2|gz) the whole repo and d/l that or using scp -r?
> 
> I tried following:
> scp -r ramaze.net:ramaze.hashed .  1.21s user 8.50s system 0% cpu 38:51.73 total
> darcs get --lazy ramaze.net:ramaze.hashed ramaze  6.08s user 4.97s system 0% cpu 59:59.34 total
> darcs get --lazy http://darcs.ramaze.net/ramaze.hashed ramaze.hashed.lazy.http  1.09s user 2.41s system 0% cpu 7:33.57 total

I believe we talked on IRC about this a bit, and you had installed darcs2
on the remote server, but had trouble getting the transfer-mode connection
to set up (you posted a transcript from darcs get --lazy --debug).  I've
added some better debug output, so would you mind trying the darcs get
--lazy --debug again and post the transcript? It looked like the pipe
wasn't getting set up properly, and I'd like to track down why.

I've added a number extra debug messages when things fail, in particular
for when the transfer-mode is not working, so hopefully this will let us
track down the problem.

Thanks for your bug report, by the way!
-- 
David Roundy
Department of Physics
Oregon State University
msg3589 (view) Author: manveru Date: 2008-02-20.04:48:19
I did as you asked, pulled latest darcs-unstable and tried again:
http://p.ramaze.net/584
I cannot seem to find the relevant parts though.
msg4679 (view) Author: kowey Date: 2008-05-14.13:17:19
Assigning to David so that he can look at the log that manveru helpfully produced.
msg5615 (view) Author: kowey Date: 2008-08-20.13:38:24
Hi Michael, are you still experiencing the problem of darcs get over ssh being
very slow?
msg5760 (view) Author: kowey Date: 2008-08-28.12:20:36
No word from Michael since February.  Marking presumed-dead (but please do feel
free to re-open).  Thanks!
msg5761 (view) Author: kowey Date: 2008-08-28.12:21:23
Whoops! I just realised that my last contact was on 20 August.  Sorry, Michael,
re-opening.
msg6294 (view) Author: kowey Date: 2008-10-09.19:19:46
I've taken a look at this log and don't see any clues about what happened with
darcs transfer mode.  It just says that we tried to invoke it, and then moved on
to using scp a lot.

I know David has recently fixed http://bugs.darcs.net/issue1003 but it does not
seem to be related as far as I can tell (the repository being fetched is hashed).

Michael: it would be good if you could test this with darcs 2.1.0 on both the
server and client side, but since we still haven't heard from you in a while, I
am going to mark this presumed-dead (whilst still hoping you can get back in
touch with us).  Thanks!
History
Date User Action Args
2008-02-16 16:31:39manverucreate
2008-02-16 18:50:56markstossetstatus: unread -> deferred
topic: + Performance, Darcs2
title: 'get --lazy' vs 'scp -r' -> wish: improve performance of "get --lazy"
nosy: droundy, tommy, beschmi, kowey, manveru
2008-02-19 15:36:44droundysetnosy: droundy, tommy, beschmi, kowey, manveru
messages: + msg3580
title: wish: improve performance of "get --lazy" -> 'get --lazy' vs 'scp -r'
2008-02-20 04:48:20manverusetnosy: droundy, tommy, beschmi, kowey, manveru
messages: + msg3589
2008-05-14 13:17:20koweysetstatus: deferred -> unknown
title: 'get --lazy' vs 'scp -r' -> darcs transfer-mode not working?
nosy: + dagit
messages: + msg4679
priority: wishlist -> bug
assignedto: droundy
2008-08-20 13:38:27koweysetstatus: unknown -> waiting-for
nosy: droundy, tommy, beschmi, kowey, dagit, manveru
messages: + msg5615
assignedto: droundy ->
2008-08-28 12:20:39koweysetstatus: waiting-for -> given-up
nosy: droundy, tommy, beschmi, kowey, dagit, manveru
messages: + msg5760
2008-08-28 12:21:25koweysetstatus: given-up -> unknown
nosy: droundy, tommy, beschmi, kowey, dagit, manveru
messages: + msg5761
2008-08-28 12:21:50koweysetstatus: unknown -> waiting-for
nosy: droundy, tommy, beschmi, kowey, dagit, manveru
2008-10-09 19:19:48koweysetnosy: + dmitry.kurochkin, simon, thorkilnaur
messages: + msg6294
2008-10-09 19:22:08koweysetstatus: waiting-for -> given-up
nosy: droundy, tommy, beschmi, kowey, dagit, simon, thorkilnaur, manveru, dmitry.kurochkin
2009-08-06 17:54:41adminsetnosy: + markstos, jast, Serware, darcs-devel, zooko, mornfall, - droundy, manveru
2009-08-06 20:58:29adminsetnosy: - beschmi
2009-08-10 22:15:39adminsetnosy: + manveru, - markstos, darcs-devel, zooko, jast, Serware, mornfall
2009-08-11 00:07:21adminsetnosy: - dagit
2009-08-25 17:30:45adminsetnosy: + darcs-devel, - simon
2009-08-27 14:12:44adminsetnosy: tommy, kowey, darcs-devel, thorkilnaur, manveru, dmitry.kurochkin