I went with the proposal #2 of http://bugs.darcs.net/issue2438.
That is, patch index is only created:

* when running annotate on a file or dir
* when running non-interactive log on a file
* for the above two cases, only if repository
  lock can be taken, otherwise PI creation is
  not attempted
* on init, clone, convert with --patch-index

This makes patch index creation happen only when the user runs
a command that uses it. In the future, if we change our mind
(for instance if we find a way to accelerate PI building)
we may come back to the previous default behaviour without
much hassle.

I'll wait until tomorrow to screen this bundle, in case
there is something very wrong with it.

3 patches for repository http://darcs.net:

patch a3465a6ac8ed406bf40c2f477fca7fcbf4305f9b
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Wed Feb 25 11:39:30 ART 2015
  * in annotate command, push call to withRepository deeper

patch 324bbe2cc34e68e3d83b142dbfe4c4831a0b1de1
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Wed Feb 25 15:01:15 ART 2015
  * resolve issue2438: no longer build patch index by default on cloning
  * build automatically only with annotate or non-interactive `log file`
  * patch index creation only occurs if the lock can be taken
  * PI creation only happens on init, clone and convert if
    --enable-patch-index is passed, disabled by default
  * PI creation no longer done when finalizing a repo job
  * the only way to have the file `_darcs/no_patch_index` created,
    is to run `optimize disable-patch-index`, or to ctrl-c
    PI creation (during annotate, log, init, clone or convert)

patch 56466a3b4ccae2959aef39e65ac02ba60afeace5
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Wed Feb 25 15:02:23 ART 2015
  * fix patch-index tests after defaults change

Attachments

• in-annotate-command_-push-call-to-withrepository-deeper.dpatch
• patch-preview.txt
• unnamed

Renamed the flag --disable-patch-index to --no-patch-index,
to make it look more like other init/clone flags, eg
--no-packs, --no-working-dir, etc.

This is possible since we switched from
"optimize --disable-patch-index" to "optimize disable-patch-index".

3 patches for repository http://darcs.net:

patch a3465a6ac8ed406bf40c2f477fca7fcbf4305f9b
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Wed Feb 25 11:39:30 ART 2015
  * in annotate command, push call to withRepository deeper

patch 6bccee4155bfacc4bdf807da51e6df296007d87e
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Thu Feb 26 14:50:31 ART 2015
  * resolve issue2438: no longer build patch index by default on cloning
  * build automatically only with annotate or non-interactive `log file`
  * patch index creation only occurs if the lock can be taken
  * PI creation only happens on init, clone and convert if
    --patch-index is passed
  * PI creation no longer done when finalizing a repo job
  * the only way to have the file `_darcs/no_patch_index` created,
    is to run `optimize disable-patch-index`, or to ctrl-c
    PI creation (during annotate, log, init, clone or convert)

patch ebf492982be16cf7cab3215e1e81e33fef6574aa
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Thu Feb 26 14:52:40 ART 2015
  * fix patch-index tests after defaults change

Attachments

• in-annotate-command_-push-call-to-withrepository-deeper.dpatch
• patch-preview.txt
• unnamed

Screening it.

One more patch, that enables the user to prevent log and annotate
to create patch-index by passing --no-patch-index.

One scenario resolved by this change would be a read-only repo
(in which once cannot run `darcs optimize disable-patch-index`)
with a very big first patch for which PI creation would instantly
hang darcs.

I had to rename --patch-index to --with-patch-index since its
presence prevented the use of the --patch flag (used in some
tests for log).  The --with-xxx idiom is already used for
--with-working-dir.

1 patch for repository http://darcs.net:

patch cb5848e8c9bd11b101a47d6018ba86ab825e9d40
Author: Guillaume Hoffmann <guillaumh@gmail.com>
Date:   Sun Mar  8 16:50:19 ART 2015
  * allow flag to prevent patch index creation with log and annotate
  ename --patch-index to --with-patch-index to avoid conflict
  with existing --patch flag used with log and annotate

Attachments

• allow-flag-to-prevent-patch-index-creation-with-log-and-annotate.dpatch
• patch-preview.txt
• unnamed

> patch a3465a6ac8ed406bf40c2f477fca7fcbf4305f9b
> Author: Guillaume Hoffmann <guillaumh@gmail.com>
> Date:   Wed Feb 25 11:39:30 ART 2015
>   * in annotate command, push call to withRepository deeper

Refactoring only, looks fine.

> patch 6bccee4155bfacc4bdf807da51e6df296007d87e
> Author: Guillaume Hoffmann <guillaumh@gmail.com>
> Date:   Thu Feb 26 14:50:31 ART 2015
>   * resolve issue2438: no longer build patch index by default on cloning
>   * build automatically only with annotate or non-interactive `log file`
>   * patch index creation only occurs if the lock can be taken
>   * PI creation only happens on init, clone and convert if
>     --patch-index is passed
>   * PI creation no longer done when finalizing a repo job
>   * the only way to have the file `_darcs/no_patch_index` created,
>     is to run `optimize disable-patch-index`, or to ctrl-c
>     PI creation (during annotate, log, init, clone or convert)

Adds a 'withRepoLockCanFail' function for doing things with the lock only if 
available. Uses it in log and annotate as described. The calls to 
withRepoLockCanFail are alongside those to withRepository (i.e. not inside or 
outside).

Looks good.

> patch ebf492982be16cf7cab3215e1e81e33fef6574aa
> Author: Guillaume Hoffmann <guillaumh@gmail.com>
> Date:   Thu Feb 26 14:52:40 ART 2015
>   * fix patch-index tests after defaults change

> patch cb5848e8c9bd11b101a47d6018ba86ab825e9d40
> Author: Guillaume Hoffmann <guillaumh@gmail.com>
> Date:   Sun Mar  8 16:50:19 ART 2015
>   * allow flag to prevent patch index creation with log and annotate
>   ename --patch-index to --with-patch-index to avoid conflict
>   with existing --patch flag used with log and annotate

These look fine too.

With regard to withRepoLockCanFail you might also want to look at patch1312.