darcs

Issue 123 darcs pull memory and cpu usage

Title darcs pull memory and cpu usage
Priority bug Status resolved
Milestone Resolved in
Superseder Nosy List darcs-devel, dmitry.kurochkin, freyther, kowey, thorkilnaur, tommy
Assigned To
Topics Performance

Created on 2006-02-01.17:16:51 by freyther, last changed 2009-08-27.13:50:01 by admin.

Messages
msg451 (view) Author: freyther Date: 2006-02-01.17:16:49
Accidently I have misused tailor and ended up creating a new tree instead adding 
the changes to the existing one.
The initial patch of the new tree is 21mb small and if I try to push or pull it 
to the old one CPU usage goes to 99% and memory usage to 75% (2GB of memory 
used). It sits there for many many hours without being able to apply the patch.
Actually the merging of these two tree should not bring any errors as they are 
only one revision away from each other.
If you want to try it I can give you the URLs of the two trees.
msg452 (view) Author: dagit Date: 2006-02-01.19:32:24
Quoting Holger Freyther <bugs@darcs.net>:

>
> New submission from Holger Freyther <freyther@handhelds.org>:
>
> Accidently I have misused tailor and ended up creating a new tree instead
> adding
> the changes to the existing one.
> The initial patch of the new tree is 21mb small and if I try to push or pull
> it
> to the old one CPU usage goes to 99% and memory usage to 75% (2GB of memory
> used). It sits there for many many hours without being able to apply the
> patch.
> Actually the merging of these two tree should not bring any errors as they
> are
> only one revision away from each other.
> If you want to try it I can give you the URLs of the two trees.

The URLs would certainly be handy for checking.  I think I know what is going
wrong here. If I'm right, then fixing this would require an extensive rewrite
of some of the data handling code.  Hopefully I'm wrong and you're hitting a
corner case that can be fixed more easily.

Not sure when would be able to look at this though.

Thanks,
Jason
msg454 (view) Author: freyther Date: 2006-02-03.09:59:08
The two tress can be found on:
http://ewi546.ewi.utwente.nl/OE/other_scms/darcs/ The .orig tree contains the 
changes until the 11ntg of january and the .new tree contains all changes since 
then.
My naive expecatation is I can pull the .new tree back into the .orig tree. Due 
the nature of the tailor conversion this pull should be even conflict free and 
it should be a simple merge.
msg462 (view) Author: tommy Date: 2006-02-05.12:52:27
On Wed, Feb 01, 2006 at 05:16:52PM +0000, Holger Freyther wrote:
> Accidently I have misused tailor and ended up creating a new tree instead adding 
> the changes to the existing one.
> The initial patch of the new tree is 21mb small and if I try to push or pull it 
> to the old one CPU usage goes to 99% and memory usage to 75% (2GB of memory 
> used). It sits there for many many hours without being able to apply the patch.
> 
> Actually the merging of these two tree should not bring any errors as they are 
> only one revision away from each other.

This is not how darcs works.  Every repo starts with an empty
tree.  The first patch in each of your repos populates this
empty three with lots of dirs and files.  They are not the
same patch.  They both need an empty tree to perform their
changes on, because that's how they were constructed during
recording.  And when they meet they will fight badly about
every little file and dir they both want to to put in the
empty tree.  There will be no winner and the result will be
sort of undecided, which causes further problems for patches
that follows with changes to these files.  You can't use
darcs to merge versions like this.  Even if darcs eventually
completes, the result will not be what you wish for.

What you could do is replay the changes from repo.new one
by one and record them as new patches in repo.old.  It is
scriptable, but I think move and replace patches get lost
(turns into rm/add and hunks) this way.
msg464 (view) Author: droundy Date: 2006-02-05.13:16:17
On Fri, Feb 03, 2006 at 09:59:09AM +0000, Holger Freyther wrote:
> My naive expecatation is I can pull the .new tree back into the .orig
> tree. Due the nature of the tailor conversion this pull should be even
> conflict free and it should be a simple merge.

Tommy's right.  This isn't going to work like you expect.  Everything will
conflict, and the result will not be desirable, even if it were to complete
(which it probably won't).
-- 
David Roundy
http://www.darcs.net
msg468 (view) Author: freyther Date: 2006-02-05.23:29:44
Well, from a naive point of view I have a linearized history with rev0,rev1,...
Now I have rev0 to rev1300 in one tree and rev1301.. in another one.

If I'm in the tree with rev1300 and applying the patch of rev1301 no conflicts
should be created as well. From a user point of view I do not see any reason why 
this should not work. From a implementation point of view I see the issue now.

Somehow git, svk, hg are quite well capable of handling this situation (e.g hg 
warns about merging two foreign trees). 
Anyway as this is not the way darcs is working and this is likely to be not a 
typical use case I guess when can close this issue.

Could you please give me a hint on how to replay the change (e.g. which commands 
it involves)
msg471 (view) Author: droundy Date: 2006-02-06.12:41:33
On Sun, Feb 05, 2006 at 11:29:46PM +0000, Holger Freyther wrote:
> Well, from a naive point of view I have a linearized history with
> rev0,rev1,...  Now I have rev0 to rev1300 in one tree and rev1301.. in
> another one.

Darcs doesn't store a sequence of versions, it stores a sequence of
changes, and the second tree has one big change that conflicts with all the
changes in the first tree.

> Somehow git, svk, hg are quite well capable of handling this situation
> (e.g hg warns about merging two foreign trees).  Anyway as this is not
> the way darcs is working and this is likely to be not a typical use case
> I guess when can close this issue.

This is because git, svk, etc are all version-based, not change-based.

> Could you please give me a hint on how to replay the change (e.g. which
> commands it involves)

To get a sequence of trees, you could pull one patch at a time into an
empty repository.  But really, this is the sort of work that tailor is
designed to handle, so that's what I'd recommend using (although I'm not
sufficiently familiar with tailor to know how exactly to do this).
-- 
David Roundy
http://www.darcs.net
History
Date User Action Args
2006-02-01 17:16:51freythercreate
2006-02-01 19:32:26dagitsetstatus: unread -> unknown
nosy: + dagit
messages: + msg452
2006-02-03 09:59:09freythersetnosy: droundy, tommy, dagit, freyther
messages: + msg454
2006-02-05 12:52:28tommysetnosy: droundy, tommy, dagit, freyther
messages: + msg462
2006-02-05 13:16:17droundysetnosy: droundy, tommy, dagit, freyther
messages: + msg464
2006-02-05 23:29:46freythersetnosy: droundy, tommy, dagit, freyther
messages: + msg468
2006-02-06 12:41:33droundysetnosy: droundy, tommy, dagit, freyther
messages: + msg471
2006-02-06 12:43:11droundysetstatus: unknown -> resolved
nosy: droundy, tommy, dagit, freyther
2007-07-16 09:27:04koweylinkissue471 superseder
2007-07-16 09:27:16koweysettopic: + Performance
nosy: + kowey, beschmi
2009-08-06 17:49:00adminsetnosy: + markstos, jast, Serware, dmitry.kurochkin, darcs-devel, zooko, mornfall, simon, thorkilnaur, - droundy, freyther
2009-08-06 20:48:45adminsetnosy: - beschmi
2009-08-10 21:43:36adminsetnosy: + freyther, - markstos, darcs-devel, zooko, jast, Serware, mornfall
2009-08-10 23:52:09adminsetnosy: - dagit
2009-08-25 18:00:50adminsetnosy: + darcs-devel, - simon
2009-08-27 13:50:01adminsetnosy: tommy, kowey, darcs-devel, freyther, thorkilnaur, dmitry.kurochkin