darcs

Issue 1159 unionCache does not adjust cache paths wrt remote repository location

Title unionCache does not adjust cache paths wrt remote repository location
Priority bug Status resolved
Milestone Resolved in 2.5.0
Superseder Nosy List Serware, darcs-devel, dmitry.kurochkin, kirby, kowey, thorkilnaur, zooko
Assigned To
Topics Hashed

Created on 2008-10-23.16:57:35 by kowey, last changed 2010-06-15.22:29:10 by dmitry.kurochkin.

Messages
msg6400 (view) Author: kowey Date: 2008-10-23.16:57:31
We need to take remote repository paths into account when doing unionCaches. 
Otherwise we get meaningless lines in your caches

# server
mkdir /path/on/server/aaa
cd /path/on/server/aaa
darcs init
cd ..
darcs get aaa bbb

# local
darcs get server:/path/on/server/bbb
grep repo:/path/on/server/aaa _darcs/prefs/sources
# should be repo:server:/path/on/server/aaa if anything
msg6401 (view) Author: kowey Date: 2008-10-23.17:01:07
Note: this bug inspired by Zooko's issue1153.  His Windows repository had this
in its _darcs/prefs/sources.  The /Users lines are nonsensical (it must have
been copied from a MacOS X repository)

thisrepo:c:/playground/allmydata/tahoe/trunk
cache:c:\Documents and Settings\Zooko Brillnonywonx\.darcs\cache
repo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/trunk-hf
thisrepo:/Users/wonwinmcbrootles/playground/allmydata/tahoe/trunk
cache:/Users/wonwinmcbrootles/.darcs/cache
msg6435 (view) Author: droundy Date: 2008-10-24.20:13:35
I'm not sure how to solve this problem.  Unioning with the cache we got from is
an important feature, since it enables you to do something like

darcs get --lazy http://darcs.net foo
darcs get --lazy foo bar
cd bar
darcs changes -s

and have it actually work.  I'm pretty certain there isn't a solution that
allows us to always find as many sources as possible, but am not sure where to
draw the line.  Of course, eliminating unionCaches would fix the problem, but
I'm not sure that it's the best approach.

David
msg6436 (view) Author: zooko Date: 2008-10-24.20:46:38
For what it is worth, I don't think the first thing we need to do here is try to
reduce the incidence of non-functioning entries in the _darcs/prefs/sources. 
More urgent is to make it so that if there is a non-functioning entry in there,
such as the IP address of a machine whose IP address has changed (for example
because it uses DHCP and it has rebooted), then this doesn't cause your darcs
client to hang.

Currently, and I've seen this one Mac, as well as Windows, the presence of
"123.456.789.12" or some such bogus IP address in _darcs/prefs/repos will cause
your darcs to slow to a crawl.  It looks like it waits for 60 seconds in between
fetching each individual patch, checking to see if that server is going to
respond in 60 seconds.  Something like that.

See the first log file -- the one from 2008-10-21 -- over in issue1153 for an
example.
msg8224 (view) Author: kowey Date: 2009-08-18.00:04:32
I think a fancier unionCache which takes into account whether or not we're
unioning a local or remote cache is possible, but I would agree that being able
to notice bad entries is probably a higher priority.
msg10515 (view) Author: kowey Date: 2010-03-25.14:26:33
I talked about this in Zurich with Salvatore (and Guillaume and Luca),
so I'll just mention his simple proposal: filter out all local caches. 
This would not help for the case of darcs get --lazy /local/foo bar (and
subsequent remote darcs get from bar), but it might save some pain.
msg10781 (view) Author: dmitry.kurochkin Date: 2010-04-21.11:21:56
The following patch updated the status of issue1159 to be resolved:

* Resolve issue1159: smart caches union. 
Ignore-this: 608859997eb69ac9263d29333f8b4dba
Try to do better than just blindly copying remote cache entries:

* If remote repository is accessed through network, do not copy any cache
  entries from it. Taking local entries does not make sense and using network
  entries can lead to darcs hang when it tries to get to unaccessible host.

* If remote repositoty is local, copy all network cache entries. For local
  cache entries if the cache directory exists and is writable it is added as
  writable cache, if it exists but is not writable it is added as read-only
  cache.

This approach should save us from bogus cache entries. One case it does not
work very well is when you fetch from partial repository over network.
Hopefully this is not a common case.
msg11438 (view) Author: dmitry.kurochkin Date: 2010-06-15.22:29:09
The following patch updated issue issue1159 with status=resolved;resolvedin=2.5.0 (current)

* Resolve issue1159: smart caches union. 
Ignore-this: 608859997eb69ac9263d29333f8b4dba
Try to do better than just blindly copying remote cache entries:

* If remote repository is accessed through network, do not copy any cache
  entries from it. Taking local entries does not make sense and using network
  entries can lead to darcs hang when it tries to get to unaccessible host.

* If remote repositoty is local, copy all network cache entries. For local
  cache entries if the cache directory exists and is writable it is added as
  writable cache, if it exists but is not writable it is added as read-only
  cache.

This approach should save us from bogus cache entries. One case it does not
work very well is when you fetch from partial repository over network.
Hopefully this is not a common case.
History
Date User Action Args
2008-10-23 16:57:35koweycreate
2008-10-23 17:01:09koweysetstatus: unread -> unknown
nosy: droundy, kowey, zooko, dagit, simon, thorkilnaur, dmitry.kurochkin, Serware
messages: + msg6401
2008-10-24 20:13:37droundysetnosy: droundy, kowey, zooko, dagit, simon, thorkilnaur, dmitry.kurochkin, Serware
messages: + msg6435
2008-10-24 20:46:42zookosetnosy: droundy, kowey, zooko, dagit, simon, thorkilnaur, dmitry.kurochkin, Serware
messages: + msg6436
2009-08-06 18:01:24adminsetnosy: + markstos, jast, darcs-devel, mornfall, tommy, beschmi, - droundy
2009-08-06 21:13:37adminsetnosy: - beschmi
2009-08-10 21:50:07adminsetnosy: - tommy, markstos, darcs-devel, jast, mornfall
2009-08-10 23:48:54adminsetnosy: - dagit
2009-08-18 00:04:34koweysetstatus: unknown -> needs-implementation
nosy: kowey, zooko, simon, thorkilnaur, dmitry.kurochkin, Serware
topic: - Darcs2
messages: + msg8224
2009-08-25 17:31:52adminsetnosy: + darcs-devel, - simon
2009-08-27 09:03:44koweysetnosy: kowey, darcs-devel, zooko, thorkilnaur, dmitry.kurochkin, Serware
2009-08-27 14:25:53adminsetnosy: kowey, darcs-devel, zooko, thorkilnaur, dmitry.kurochkin, Serware
2009-09-07 04:31:19koweysettopic: + Hashed
nosy: kowey, darcs-devel, zooko, thorkilnaur, dmitry.kurochkin, Serware
2009-10-23 22:45:21adminsetnosy: + serware, - Serware
2009-10-23 23:30:57adminsetnosy: + Serware, - serware
2010-03-25 14:26:35koweysetnosy: + kirby
messages: + msg10515
2010-04-21 11:21:57dmitry.kurochkinsetstatus: needs-implementation -> resolved
messages: + msg10781
2010-06-15 22:29:10dmitry.kurochkinsetmessages: + msg11438
resolvedin: 2.5.0