darcs

Issue 550 pull via http takes a very long time (possibly improved in Darcs2)

Title pull via http takes a very long time (possibly improved in Darcs2)
Priority bug Status resolved
Milestone Resolved in
Superseder Nosy List darcs-devel, dmitry.kurochkin, kowey, markstos, nejucomo, thorkilnaur, tommy
Assigned To nejucomo
Topics Darcs2

Created on 2007-10-18.15:01:42 by nejucomo, last changed 2009-08-27.13:58:42 by admin.

Messages
msg2170 (view) Author: nejucomo Date: 2007-10-18.15:01:41
Hello,

I seem to have encountered an efficiency bug.  Can anyone suggest ways
to investigate the cause?

When I say darcs pull in a certain project it says there are 260
patches, and if I say "a" for all, my cpu gets pegged for a long time.
 I've been doing darcs pull,
then accepting a few at a time, but it'll take forever that way.

I ran darcs pull -a overnight to see what happens, and it is still
running a large fraction (95% according to top) of cpu.  The memory
consumption seems relatively flat at just under 60 megs.

For now I'll leave it running, just for good measure.

Thanks,
Nathan Wilcox
msg2173 (view) Author: nejucomo Date: 2007-10-24.03:39:12
Here's an update:

On 10/18/07, Nathan <nejucomo@gmail.com> wrote:
> Hello,
>
> I seem to have encountered an efficiency bug.  Can anyone suggest ways
> to investigate the cause?
>

...

> I ran darcs pull -a overnight to see what happens, and it is still
> running a large fraction (95% according to top) of cpu.  The memory
> consumption seems relatively flat at just under 60 megs.
>
> For now I'll leave it running, just for good measure.
>

It succeeded after a long while with exit status 0!  Here is the output:

$ time darcs pull -a
Pulling from "http://allmydata.org/source/tahoe/trunk"...
We have conflicts in the following files:
./src/allmydata/client.py ./src/allmydata/node.py
./src/allmydata/test/check_memory.py
./src/allmydata/test/test_runner.py
Warning: Not deleting ./src/foolscap/foolscap/slicers because it is non-empty.
Warning: Not deleting ./src/foolscap/foolscap because it is non-empty.
Warning: Not deleting ./src/foolscap because it is non-empty.
Warning: Not deleting ./src/simplejson/ez_setup because it is non-empty.
Warning: Not deleting ./src/simplejson because it is non-empty.
Warning: Not deleting ./src/zfec because it is non-empty.
Finished pulling and applying.

real    665m14.215s
user    633m39.716s
sys     1m41.388s
msg2552 (view) Author: droundy Date: 2008-01-17.20:31:57
I'm not sure what triggered this, but I've just implemented a number of
improvements for darcs pull in the darcs 2 branch, one of which changes the
scaling properties.  Darcs 2 is not always faster on pull than darcs 1, but
sometimes it is.

Any chance you can try this same pull with darcs 2 (actually, the latest version
from http://darcs.net/repos/darcs-unstable), and then try it again using a
hashed repository?

David
msg2566 (view) Author: markstos Date: 2008-01-19.02:00:44
I'm marking this as "deferred" until we get more feedback about how the
performance compares with Darcs 2.
msg3501 (view) Author: markstos Date: 2008-02-16.18:53:19
Hello Nathan, 

Have you had any better luck with pull performing better when you try darcs 2?

  Mark
msg3587 (view) Author: nejucomo Date: 2008-02-19.18:15:06
Hi Mark,

Thanks for the feedback.  I haven't tried darcs 2 yet, because I
typically rely on Ubuntu packages, (which is currently darcs 1.0.9rc1
on my system).  Next time I have some slack time in a project, I'll
install darcs 2 and get back to you.

Better yet, I'll continue using darcs 1.0.9rc1 until I trigger that
long hang situation, and that will serve as a reminder to compare
darcs 2's behavior.

Regards,
Nathan

On Feb 16, 2008 10:53 AM, Mark Stosberg <bugs@darcs.net> wrote:
>
>
> Hello Nathan,
>
> Have you had any better luck with pull performing better when you try darcs 2?
>
>   Mark
>
> ----------
> assignedto:  -> nejucomo
> title: pull takes a very long time (possibly improved in Darcs2) -> pull via http takes a very long time (possibly improved in Darcs2)
>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue550>
> __________________________________
>
msg4625 (view) Author: kowey Date: 2008-05-10.13:38:06
Hi Nathan,

I'm going to mark this one resolved with the darcs 2 release (I keep hearing of
better performance on the mailing lists).  Please file a new report if you run
into more slowness.
History
Date User Action Args
2007-10-18 15:01:43nejucomocreate
2007-10-24 03:39:13nejucomosetstatus: unread -> unknown
nosy: kowey, beschmi, droundy, nejucomo, tommy
messages: + msg2173
2007-11-26 23:19:39jchsetnosy: kowey, beschmi, droundy, tommy, nejucomo
2008-01-17 20:31:58droundysetnosy: droundy, tommy, beschmi, kowey, nejucomo
messages: + msg2552
2008-01-19 02:00:46markstossetstatus: unknown -> deferred
nosy: + markstos
topic: + Darcs2
messages: + msg2566
title: Pulling many patches pegs my cpu. -> pull takes a very long time (possibly improved in Darcs2)
2008-02-16 18:53:20markstossetnosy: droundy, tommy, beschmi, kowey, markstos, nejucomo
assignedto: nejucomo
messages: + msg3501
title: pull takes a very long time (possibly improved in Darcs2) -> pull via http takes a very long time (possibly improved in Darcs2)
2008-02-19 18:15:08nejucomosetnosy: droundy, tommy, beschmi, kowey, markstos, nejucomo
messages: + msg3587
2008-05-10 13:38:08koweysetstatus: deferred -> resolved
nosy: + dagit
messages: + msg4625
2009-08-06 17:43:39adminsetnosy: + jast, Serware, dmitry.kurochkin, darcs-devel, zooko, mornfall, simon, thorkilnaur, - droundy, nejucomo
2009-08-06 20:40:34adminsetnosy: - beschmi
2009-08-10 22:09:08adminsetnosy: + nejucomo, - darcs-devel, zooko, jast, Serware, mornfall
2009-08-11 00:02:52adminsetnosy: - dagit
2009-08-25 17:56:44adminsetnosy: + darcs-devel, - simon
2009-08-27 13:58:42adminsetnosy: tommy, kowey, markstos, darcs-devel, nejucomo, thorkilnaur, dmitry.kurochkin