darcs

Issue 642 wish: Automatic detection of file renames

Title wish: Automatic detection of file renames
Priority wishlist Status resolved
Milestone Resolved in 2.10.0
Superseder Nosy List Serware, alex1, darcs-devel, dmitry.kurochkin, jaredj, jlneder, kowey, lele, markstos, riffraff, thorkilnaur, tommy
Assigned To
Topics ProbablyEasy, UI

Created on 2008-02-03.23:31:08 by riffraff, last changed 2014-01-20.19:04:23 by noreply.

Messages
msg3085 (view) Author: riffraff Date: 2008-02-03.23:31:06
Using darcs in conjunction with other SCM without explicit import (i.e. I use
darcs on a project where the vendor/ directory contains an svn checkout) it is
difficult to keep track of file renames, because darcs sees a file rename in the
svn checkout just as a missing file and a new non-added one.

It would be nice if darcs, at record time, could look for renames (possibly as a
command line option, on the lines of --look-for-adds).
If file FOO is missing but there are new files BAR and BAZ and one of those has
the same content of a missing file darcs could suggest including it in the patch
and thus recording a single `move` patch instead of rmfile+addfile+content.
msg3094 (view) Author: lele Date: 2008-02-04.15:25:49
On Sun, 03 Feb 2008 23:31:08 -0000
gabriele renzi <bugs@darcs.net> wrote:

> It would be nice if darcs, at record time, could look for renames
> (possibly as a command line option, on the lines of --look-for-adds).
> If file FOO is missing but there are new files BAR and BAZ and one of
> those has the same content of a missing file darcs could suggest
> including it in the patch and thus recording a single `move` patch
> instead of rmfile+addfile+content.

Given that a svn rename may be effectively a rename+edit, I doubt
there's a [simple] heuristic that could help here. Isn't what you are
seeking better handled by tools like tailor?

ciao, lele.
-- 
nickname: Lele Gaifax    | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas    | comincerò ad aver paura di chi mi copia.
lele@nautilus.homeip.net |                 -- Fortunato Depero, 1929.
msg3097 (view) Author: droundy Date: 2008-02-04.15:31:37
I think this is a reasonable request.  We needn't look for *identical* files,
but just for substantially-similar non-boring files.  And we've got functions to
compare similar files, so it shouldn't too hard.  It's very often that I
accidentally use ordinary mv to rename a file, then kick myself, and have to mv
it back again and then do a darcs mv.

David
msg3108 (view) Author: droundy Date: 2008-02-04.16:17:48
On Mon, Feb 04, 2008 at 03:44:13PM -0000, Lele Gaifax wrote:
> David Roundy <bugs@darcs.net> wrote:
> > I think this is a reasonable request.  We needn't look for
> > *identical* files, but just for substantially-similar non-boring
> > files.  And we've got functions to compare similar files, so it
> > shouldn't too hard.  It's very often that I accidentally use ordinary
> > mv to rename a file, then kick myself, and have to mv it back again
> > and then do a darcs mv.
> 
> That's great to hear. BTW, you don't actually need to move that back,
> as you can issue "darcs mv a b" *after* a "mv a b"... I still think
> it's something [what originally asked] that can maybe be alleviated,
> but not solved completely (consider directory renames, or a
> complicated reorg of the sources). ... OTOH, I think this is something
> git does/tries to do, doesn't it?

You're right, directory renames are trickier, but even there, we've got
diff code for directories, so we can quantify relatively easily how much
they've changed.  For complicated reorganizations (e.g. filenames change
and file contents get intermingled) it'd still be possible to do something
reasonable, I suspect.
-- 
David Roundy
Department of Physics
Oregon State University
msg3114 (view) Author: alex1 Date: 2008-02-04.18:34:44
On Feb 4, 2008 5:17 PM, David Roundy <bugs@darcs.net> wrote:
> You're right, directory renames are trickier, but even there, we've got
> diff code for directories, so we can quantify relatively easily how much
> they've changed.  For complicated reorganizations (e.g. filenames change
> and file contents get intermingled) it'd still be possible to do something
> reasonable, I suspect.

I would love to see someone use kernel facilities such as FSEvents [1]
on OS X and inotify [2] on Linux to accomplish this. (There are also
others such as fam [3], and Windows has its own easy-to-use API.)

These mechanisms provide low-overhead notifications about file changes
that alleviates the need to explicitly check file contents -- instead
of these expensive brute-force checks and imprecise heuristics, the
kernel does the work and gives you semi-real-time callbacks whenever
something actually changes.

FSEvents has the advantage that the listening process need not be
running in order to subscribe to events -- FSEvents has a persistent
mode where events are logged to disk and survive reboots. It's this
engine serves as the underpinnings of Leopard's Time Machine backup
system.

For the other APIs, a helper daemon running in the background would be
needed to receive notifications; it would be as simple as logging the
events sequentially to a text file in _darcs, which the Darcs
command-line tool could then read and process.

Depending on such an external tool wouldn't be a big deal because
people are surviving just fine today without such clever change
detection. It would purely be a convenience for those willing to run
the tool. And it could be fairly transparent: If Darcs finds stuff in
_darcs/events, then it can use it to glean structural repo changes; if
this file is not available, it needs to rely on the old behaviour --
namely "darcs mv" and "--look-for-adds".

I imagine you would have a config file ~/.darcs/monitor.conf listing
monitored repos, and a support command in Darcs itself:

$ darcs monitor --enable
Change monitor not running. Starting.
Enabled monitoring of /home/alex/work/importantproject.
$ darcs monitor --disable
Disabled monitoring of /home/alex/work/importantproject.

[1] http://developer.apple.com/documentation/Darwin/Conceptual/FSEvents_ProgGuide/TechnologyOverview/chapter_3_section_1.html
[2] http://en.wikipedia.org/wiki/Inotify
[3] http://en.wikipedia.org/wiki/File_alteration_monitor

Alexander.
msg3123 (view) Author: droundy Date: 2008-02-05.15:27:06
On Mon, Feb 04, 2008 at 07:30:33PM +0100, Alexander Staubo wrote:
> On Feb 4, 2008 5:17 PM, David Roundy <bugs@darcs.net> wrote:
> > You're right, directory renames are trickier, but even there, we've got
> > diff code for directories, so we can quantify relatively easily how
> > much they've changed.  For complicated reorganizations (e.g. filenames
> > change and file contents get intermingled) it'd still be possible to do
> > something reasonable, I suspect.
> 
> I would love to see someone use kernel facilities such as FSEvents [1] on
> OS X and inotify [2] on Linux to accomplish this. (There are also others
> such as fam [3], and Windows has its own easy-to-use API.)

One nice thing about this feature is that as far as I can tell, there's
essentially no benefit to implementing it in darcs.  You could implement it
in any language for any platform, and all you need to do is call darcs mv
whenever you detect a file rename in a darcs repository.
-- 
David Roundy
Department of Physics
Oregon State University
msg3141 (view) Author: alex1 Date: 2008-02-05.23:01:32
On Feb 5, 2008 4:22 PM, David Roundy <droundy@darcs.net> wrote:
> On Mon, Feb 04, 2008 at 07:30:33PM +0100, Alexander Staubo wrote:
> > I would love to see someone use kernel facilities such as FSEvents [1] on
> > OS X and inotify [2] on Linux to accomplish this. (There are also others
> > such as fam [3], and Windows has its own easy-to-use API.)
>
> One nice thing about this feature is that as far as I can tell, there's
> essentially no benefit to implementing it in darcs.  You could implement it
> in any language for any platform, and all you need to do is call darcs mv
> whenever you detect a file rename in a darcs repository.

True, but tighter integration with Darcs would mean a smoother user
experience. I would rather have such a monitoring system be
subservient of Darcs rather than an active player, which makes the
separation of concerns cleaner: the monitor exists to gather
information, and Darcs uses that information; and I would prefer to
know about the existence of the monitor as long as Darcs is able to
control it. But you're right, it could be implemented either way.

Alexander.
msg3175 (view) Author: markstos Date: 2008-02-07.04:58:12
I have also accidentally done a plain "mv" numerous times when I should have
done a "darcs mv". My suggestions fo r implementing this would be:

- "darcs mv" with no arguments could be interactive...it would look for likely
"moves" and then prompt the user to confirm each one. I suppose could you also
have an "--all" flag to accept all the recommendations automatically. 

- "darcs record --look-for-moves" would do the same kind of thing, the way
"--look-for-adds" complements "darcs add".
msg7051 (view) Author: thorkilnaur Date: 2009-01-12.10:11:18
Setting status deferred, as this is a new feature.

Best regards
Thorkil
msg7077 (view) Author: thorkilnaur Date: 2009-01-13.11:13:41
Sorry, deferring was a mistake. Setting status need-info instead, asking for a 
discussion of whether we should work towards supporting this feature.

Thanks and best regards
Thorkil
msg7082 (view) Author: markstos Date: 2009-01-13.14:20:30
On Tue, 13 Jan 2009 11:13:52 -0000
Thorkil Naur <bugs@darcs.net> wrote:

> 
> Thorkil Naur <naur@post11.tele.dk> added the comment:
> 
> Sorry, deferring was a mistake. Setting status need-info instead, asking for a 
> discussion of whether we should work towards supporting this feature.

I agree this is nice, but would be OK with rejecting it, or setting it as very
low priority. To accept it, I'd like to better understand the kind of
complexity involved in implementing it, and decide if we want to introduce that
into the codebase.

    Mark
msg17133 (view) Author: noreply Date: 2014-01-20.19:04:22
The following patch sent by Jose Luis Neder <jlneder@gmail.com> updated issue issue642 with
status=resolved;resolvedin=2.10.0 HEAD

* resolve issue642: Automatic detection of file renames 
Ignore-this: ef673bf6a47a65355ca96efe3a6c6d8
History
Date User Action Args
2008-02-03 23:31:08riffraffcreate
2008-02-04 15:25:50lelesetstatus: unread -> unknown
nosy: + darcs-devel, lele
messages: + msg3094
2008-02-04 15:31:41droundysettopic: + Confirmed, ProbablyEasy
nosy: + jaredj
messages: + msg3097
2008-02-04 15:44:13lelesetnosy: droundy, tommy, beschmi, kowey, darcs-devel, lele, jaredj, riffraff
messages: + msg3102
2008-02-04 16:17:50droundysetnosy: droundy, tommy, beschmi, kowey, darcs-devel, lele, jaredj, riffraff
messages: + msg3108
2008-02-04 18:34:49alex1setnosy: + alex1
messages: + msg3114
2008-02-05 15:27:07droundysetnosy: droundy, tommy, beschmi, kowey, darcs-devel, lele, alex1, jaredj, riffraff
messages: + msg3123
2008-02-05 23:01:34alex1setnosy: droundy, tommy, beschmi, kowey, darcs-devel, lele, alex1, jaredj, riffraff
messages: + msg3141
2008-02-06 14:48:10riffraffsetnosy: droundy, tommy, beschmi, kowey, darcs-devel, lele, alex1, jaredj, riffraff
messages: - msg3102
2008-02-07 04:58:16markstossetnosy: + markstos
messages: + msg3175
title: Automatic detection of file renames -> wish: Automatic detection of file renames
2008-02-11 01:04:18markstossetstatus: unknown -> deferred
nosy: droundy, tommy, beschmi, kowey, markstos, darcs-devel, lele, alex1, jaredj, riffraff
2008-05-09 17:15:09koweysetpriority: wishlist -> feature
status: deferred -> unknown
nosy: + Serware, dagit
2009-01-12 10:11:28thorkilnaursetstatus: unknown -> deferred
nosy: + dmitry.kurochkin, simon, thorkilnaur
messages: + msg7051
2009-01-13 11:13:52thorkilnaursetstatus: deferred -> waiting-for
nosy: droundy, tommy, beschmi, kowey, markstos, darcs-devel, dagit, lele, simon, alex1, thorkilnaur, jaredj, riffraff, dmitry.kurochkin, Serware
messages: + msg7077
2009-01-13 14:20:32markstossetnosy: droundy, tommy, beschmi, kowey, markstos, darcs-devel, dagit, lele, simon, alex1, thorkilnaur, jaredj, riffraff, dmitry.kurochkin, Serware
messages: + msg7082
2009-08-06 17:53:12adminsetnosy: + jast, zooko, mornfall, - droundy, lele, alex1, jaredj, riffraff
2009-08-06 20:56:28adminsetnosy: - beschmi
2009-08-10 22:13:12adminsetnosy: + riffraff, lele, alex1, jaredj, - zooko, jast, mornfall
2009-08-11 00:06:00adminsetnosy: - dagit
2009-08-25 17:38:46adminsetnosy: - simon
2009-08-27 14:14:59adminsetnosy: tommy, kowey, markstos, darcs-devel, lele, alex1, thorkilnaur, jaredj, riffraff, dmitry.kurochkin, Serware
2009-09-06 20:52:55koweysetpriority: feature -> wishlist
status: waiting-for -> needs-implementation
topic: - Confirmed, Darcs2
nosy: tommy, kowey, markstos, darcs-devel, lele, alex1, thorkilnaur, jaredj, riffraff, dmitry.kurochkin, Serware
2009-10-23 22:34:15adminsetnosy: + alexander, - alex1
2009-10-23 22:42:48adminsetnosy: + serware, - Serware
2009-10-23 23:28:43adminsetnosy: + Serware, - serware
2009-10-23 23:43:14adminsetnosy: + alex1, - alexander
2013-05-29 22:01:32ghsetnosy: + jlneder
2014-01-20 19:04:23noreplysetstatus: needs-implementation -> resolved
messages: + msg17133
resolvedin: 2.10.0