By default, darcs send seem to not take advantage of tags in order to
reduce context size of patch bundles.

Running "darcs optimize --reorder" before "send" solves the problem, so
perhaps this could be done automatically?

My only concerns would be if darcs optimize --reorder takes a long time, 
or if the person was doing darcs send from a public repository (this may 
be a bit far-fetched, maybe imagine some hypothetical future darcsden 
with some kind of send button?).

Basically trying to see if this should lead to any surprise.  The sort 
of rough goal I'm aiming for is a sense of transparency, in the sense 
that actions should not have any unexpected side-effects, that things 
seem to happen for a reason (X because I did Y).  Hmm, I wish I could 
think a bit more systematically about this...

On 02/17/11 09:38, Eric Kow wrote:
>
> Eric Kow<kowey@darcs.net>  added the comment:
>
> My only concerns would be if darcs optimize --reorder takes a long time,
> or if the person was doing darcs send from a public repository (this may
> be a bit far-fetched, maybe imagine some hypothetical future darcsden
> with some kind of send button?).
>
> Basically trying to see if this should lead to any surprise.  The sort
> of rough goal I'm aiming for is a sense of transparency, in the sense
> that actions should not have any unexpected side-effects, that things
> seem to happen for a reason (X because I did Y).  Hmm, I wish I could
> think a bit more systematically about this...

First I thought, on send, ask user: "reorder first? [yn]" with 
explanation.  Of course that's a terrible idea.

Is reorder interruptible?  Start doing the reorder in the background and 
tell the user "if you're bored, type X and we'll send a bigger patch 
bundle, but you won't have to wait for it".  Where "X" is probably along 
the lines of y/n or ctrl-C (a la darcs get with ctrl-C-->lazy repo).

That doesn't address the "public repo" or other effects of "You should 
not run --reorder-patches on a repository from which someone may be 
simultaneously pulling or getting, as this could lead to repository 
corruption."  If we have no way of addressing that, we really can't run 
reorder silently :-/

-Isaac

So reordering at the moment of "darcs send" causes two problems:

1) creating race conditions if people get or pull at the same time.
2) taking a long time when running "send", making the command look
broken/weird from the user point of view.

I think 1) is not a problem: you usually don't send from a public,
central repository. The race condition problem can already happen when
you run optimize in a repository. I think it's good to know the race
problem can happen, but it's not a blocker, IMHO.

2) seems more important. "optimize --reorder" can easily take 30s on a
repo of reasonable size, while it is usually almost instant between
local repositories.

Something that can be done now:
* update the help of "darcs send" to tell users they can run "optimize
--reorder" to reduce the size of bundle contexts.
* also update the FAQ on the wiki

A more global issue is: can we maintain inventories (eg, when we pull a
tag) in an efficient way so that they are usually efficiently ordered?
Who know more about this? I think galbolle told me he knew about this,
I'm adding him as nosy on this issue.

In Darcs.Commands.Optimize, there is the following piece of comments:

-- FIXME: someone needs to grovel through the source and determine
-- just how optimizeInventory differs from doReorder.  The following
-- is purely speculation. --twb, 2009-04

So, should we recommend just "optimize" or "optimize --reorder" to
produce small bundle files?

On Thu, Feb 17, 2011 at 17:16:15 -0500, Isaac Dupree wrote:
> Is reorder interruptible?  Start doing the reorder in the background
> and tell the user "if you're bored, type X and we'll send a bigger
> patch bundle, but you won't have to wait for it".  Where "X" is
> probably along the lines of y/n or ctrl-C (a la darcs get with
> ctrl-C-->lazy repo).

Thanks, Isaac.

> That doesn't address the "public repo" or other effects of "You
> should not run --reorder-patches on a repository from which someone
> may be simultaneously pulling or getting, as this could lead to
> repository corruption."  If we have no way of addressing that, we
> really can't run reorder silently :-/

OK, trusting Guillaume's argument that nobody actually sends from
public repositories, and that the possible race condition issue is no
worse than now.

Here's a third proposal: Could we establish some sort of arbitrary
criterion for what constitutes a "large" bundle, for example, one whose
context is >N KiB and X times larger than the number of patches sent?
Darcs could tend go the "helpful afterthought" route by saying

  HINT: That was a big patch bundle! (X KiB).
        Do optimize --reorder now if you want smaller bundles

Or is that making darcs too damned chatty again?

-- 
Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
For a faster response, try +44 (0)1273 64 2905 or
xmpp:kowey@jabber.fr (Jabber or Google Talk only)

kowey> It would not work that well since the repositories may not be
tagged at all, or with a too old tag. And yes, that would be yet another
message.

I suggest maybe something like:

Optimizing for smaller patch bundles (Hit Ctrl-C to send without trying 
to do this)

If not feasible, or not the right design, might be good to set to needs-
design

I think the specification of this feature is not exactly what we want, 
in order to address the issue of patch bundles with irrelevant patches 
in context. Irrelevant patches are typically present *after* the last 
tag.

So I think the approach of http://bugs.darcs.net/issue1514 is more 
appropriate. We could mitigate the computational costs by only looking 
at patches after the last tag.