darcs

Issue 2421 Couldn't fetch ... while pulling (was: cloning) darcs repo

Title Couldn't fetch ... while pulling (was: cloning) darcs repo
Priority critical Status resolved
Milestone 2.10.0 Resolved in 2.10.0
Superseder Nosy List reilithion
Assigned To
Topics

Created on 2014-11-20.04:32:35 by reilithion, last changed 2015-06-21.01:18:59 by gh.

Messages
msg17849 (view) Author: reilithion Date: 2014-11-20.04:32:32
1. Summarise the issue (what were doing, what went wrong?)
I attempted to clone the entire darcs-2.8 repo:
${here}/dist/build/darcs/darcs get http://darcs.net/releases/branch-2.8
$tmpdir

Note the lack of --lazy

I got the following output:

Copying patches, to get lazy repository hit ctrl-C...
darcs: user error (Couldn't fetch
`0000049111-0bd34d8a802beae0c587e68de8c2ea1b2691b7270ea7f534716cec6ea2fee843'
in subdir pristine.hashed from sources:

thisrepo:/tmp/branch-2.8
cache:/home/lucas/.cache/darcs
readonly:/home/lucas/.darcs/cache
repo:http://darcs.net/releases/branch-2.8
)
Finished cloning.

The repo is actually cloned correctly, it just displays this cryptic
error message that seems to claim it wasn't.

2. What behaviour were you expecting instead?
No error message, as occurs with darcs 2.8.5

3. What darcs version are you using? (Try: darcs --exact-version)
darcs screened at the following patch:
Fri Nov 14 11:29:40 MST 2014  Ben Franksen
<benjamin.franksen@helmholtz-berlin.de>
  * In CommandControl, push the existential down into the alternatives

I used darcs test to try to find the first patch that printed this
error, but ran into problems between 64 and 128 patches back. (cabal
sandbox did stuff I didn't understand)

4. What operating system are you running?
Arch Linux, with fairly recently updated packages.
msg17919 (view) Author: bf Date: 2014-12-29.17:11:26
I can confirm this, happened to me, too.
msg18277 (view) Author: bf Date: 2015-03-09.10:27:54
It just happened to me again:

franksen@elbe: ...tools/darcs/screened > darcs pull
Pulling from "http://darcs.net"...
Welcome to the darcs screened repository.

If you would like to contribute, please read our guide for contributors:
http://wiki.darcs.net/Development/GettingStarted

Thanks and happy hacking!
**********************
patch 4aa26ae6f5dd3692280d9f947b14a3a301e88ea4
Author: Ganesh Sittampalam <ganesh@earth.li>
Date:   Tue Mar  3 20:06:41 CET 2015
  * ErrorT is deprecated, switch to ExceptT
  
  The two have different 'fail' implementations, but as far as
  I can see we don't have any explicit calls to that or any
  inexhaustive pattern matches in the relevant code.
  
Shall I pull this patch? (1/8)  [ynW...], or ? for more options: a
darcs: failed to read patch:
patch d1e8c83ae1ec11914390f83bd658b7918b09a338
Author: Ganesh Sittampalam <ganesh@earth.li>
Date:   Wed Mar  4 23:41:39 CET 2015
  * replace pointless import of Control.Monad.Except
  
  This fixes the build with mtl<2.2.1
  
Couldn't fetch
`0000000308-dc89aa8e56af324cf8c3cfc654b9b505ef152b15d0e1f9e75dfd78736e101f39'
in subdir patches from sources:

thisrepo:/srv/projects/ctl/franksen/tools/darcs/screened
cache:/home/franksen/.cache/darcs
cache:/opt/csr/share/cache/darcs
cache:/tmp/darcs-franksen/cache
readonly:/home/franksen/.darcs/cache
repo:http://darcs.net

franksen@elbe: ...tools/darcs/screened > lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 7.8 (wheezy)
Release:        7.8
Codename:       wheezy
franksen@elbe: ...tools/darcs/screened > darcs --version
2.11.0 (unknown)
franksen@elbe: ...tools/darcs/screened > darcs --exact-version 
darcs compiled on Mar  4 2015, at 14:35:16

Context:

[silence warning (ghc<7.8) about the MINIMAL pragma
Ganesh Sittampalam <ganesh@earth.li>**20150303075653
 Ignore-this: e316d3b3254981b270f14b6655c7f1ea
] 

[...]

Compiled with:

HTTP-4000.2.3
array-0.4.0.0
attoparsec-0.11.1.0
base-4.5.0.0
base16-bytestring-0.1.1.6
binary-0.5.1.0
bytestring-0.9.2.1
containers-0.4.2.1
cryptohash-0.11.6
data-ordlist-0.4.7.0
dataenc-0.14.0.7
deepseq-1.3.0.0
directory-1.1.0.2
filepath-1.3.0.0
hashable-1.2.3.1
haskeline-0.7.1.3
html-1.0.1.2
mmap-0.5.9
mtl-2.1.1
network-2.3.0.13
old-locale-1.0.0.4
old-time-1.1.0.0
parsec-3.1.2
process-1.1.0.1
random-1.0.1.1
regex-applicative-0.3.1
regex-compat-tdfa-0.95.1.4
tar-0.4.1.0
terminfo-0.3.2.6
text-0.11.2.0
time-1.4
transformers-0.3.0.0
unix-2.5.1.0
unix-compat-0.4.1.4
utf8-string-0.3.8
vector-0.10.12.2
zip-archive-0.2.3.7
zlib-0.5.3.3
FindBin-0.0.5
HUnit-1.2.4.2
QuickCheck-2.4.2
cmdlib-0.3.5
shellish-0.1.4
split-0.2.2
test-framework-0.8.1.1
test-framework-hunit-0.3.0.1
test-framework-quickcheck2-0.3.0.3
msg18278 (view) Author: bf Date: 2015-03-09.10:40:11
In my last case it was a pull, not a clone, so not *exactly* the same bug.
msg18279 (view) Author: bf Date: 2015-03-09.10:43:39
More importantly, I had the "same" error immediately before this one
while pulling from a *ssh* remote repo. My nose tells me that both
problems (pull via http, pull via ssh) have a closely related cause.
msg18280 (view) Author: bf Date: 2015-03-09.10:45:55
Marking as critical and milestone 2.10. This problem makes Darcs
unusable in practice.
msg18281 (view) Author: bf Date: 2015-03-09.11:54:40
It may be related or not, but on the machine I have this problem, I get
this when compiling darcs:

[178 of 238] Compiling Darcs.Repository.PatchIndex (
src/Darcs/Repository/PatchIndex.hs,
dist/build/Darcs/Repository/PatchIndex.o )
match_co baling out
    <darcs-2.11.0:Darcs.Patch.PatchInfoAnd.PatchInfoAnd{tc r2Kvc}
       ( p{tv a48NG} [tv] :: ghc-prim:GHC.Prim.*{(w) tc 34d}
                             -> ghc-prim:GHC.Prim.*{(w) tc 34d}
                             -> ghc-prim:GHC.Prim.*{(w) tc 34d} )>

The same message gets repeated for 15 times. but compilatioon does not
actually fail.
msg18282 (view) Author: bf Date: 2015-03-09.12:22:55
Strange. I have re-compiled darcs and now I can pull the darcs repo
(screened) via http. Then I re-compiled darcs again.

But I still cannot pull via ssh:

franksen@elbe: ...epics/support/bessyRules > darcs pull
Pulling from
"rcsadm@repo.acc.bessy.de:darcs-repos/epics/support/bessyRules"...
patch 8214f934eccaa11abdac22a3d3b94151ff18b8d1
Author: Ben Franksen <benjamin.franksen@helmholtz-berlin.de>
Date:   Sat Feb 28 17:55:42 CET 2015
  * removed make-rsync-dist.html DOC target
Shall I pull this patch? (1/5)  [ynW...], or ? for more options: a
darcs: failed to read patch:
patch 8adcb83cc9b9dcc77f0ea4b67afb2a9aa5f0d225
Author: Ben Franksen <benjamin.franksen@helmholtz-berlin.de>
Date:   Tue Mar  3 18:24:26 CET 2015
  tagged R2-18
Couldn't fetch
`0000000957-0f0f2694fc1210c36ddd1f074371dc024c666bc0fe673c97d653ccfd05f07801'
in subdir patches from sources:

thisrepo:/srv/projects/ctl/franksen/epics/support/bessyRules
cache:/home/franksen/.cache/darcs
cache:/opt/csr/share/cache/darcs
cache:/tmp/darcs-franksen/cache
readonly:/home/franksen/.darcs/cache
repo:rcsadm@repo.acc.bessy.de:darcs-repos/epics/support/bessyRules
msg18283 (view) Author: bf Date: 2015-03-09.13:22:20
Commenting out the catchall handler in
Darcs.Repository.Cache.fetchFileUsingCachePrivate that printed the error
message finally gave me the diagnostics I needed to track this down. The
problem was that I had disabled writing to ~/.cache/darcs, for teh same
reason I had done so with ~/.darcs/cache: the home directories on this
machine have a pretty low quota which tends to fill quickly when used to
cache patches. One solution is to sym-link ~/.cache/darcs to a directory
under /tmp. Another is to pass --no-cache (I checked that this works).

Lesson1: Never use catchall unless you know *exactly* what you are
doing. Otherwise you'll just loose useful diagnostics. I would have seen
immediately what the problem is if the catchall handler hadn't hidden it
from me.

Lesson2: Darcs should (always) fall back to --no-cache behaviour if none
of the cache directory candidates work (i.e. they all result in some IO
error). This would resolve the issue in the most useful way.
msg18298 (view) Author: noreply Date: 2015-03-10.19:43:30
The following patch sent by Ben Franksen <benjamin.franksen@helmholtz-berlin.de> updated issue issue2421 with
status=resolved;resolvedin=2.10.0 HEAD

* resolve issue2421: commandGetArgPossibilities for apply should not list all files in the repo 
Ignore-this: 94a95ddca1903ef788f3832203d9832e
msg18299 (view) Author: gh Date: 2015-03-10.19:51:30
(Opening again, was wrongly closed.)
msg18320 (view) Author: gh Date: 2015-03-19.15:07:37
This is an important bug. We want darcs to work for users that do not
know about --no-cache (even less how to set it in defaults) and that
work on machines where ~/.cache/ is somehow disabled.
msg18343 (view) Author: gh Date: 2015-04-05.04:42:20
An easy way to reproduce this bug is to "chmod -rwx ~/.cache" before
cloning a repository. Looks like the culprit is the call to
createDirectoryIfMissing at
http://hub.darcs.net/darcs/darcs-reviewed/browse/src/Darcs/Repository/Cache.hs#423
  that should be inside of the following `catchall`.
msg18366 (view) Author: noreply Date: 2015-04-13.18:41:24
The following patch sent by Guillaume Hoffmann <guillaumh@gmail.com> updated issue issue2421 with
status=resolved;resolvedin=2.10.0 HEAD

* resolve issue2421: don't fail if caches are not writeable 
Ignore-this: 8ab0b04779dada9c402fb7bf5d0b8c1f
msg18569 (view) Author: bf Date: 2015-06-21.00:58:44
It happened to me again:

ben@sarun[1]: .../src/darcs > darcs --version
2.10.0 (release)

ben@sarun[1]: .../src/darcs > darcs get http://darcs.net/screened bla 
Welcome to the darcs screened repository.

If you would like to contribute, please read our guide for contributors:
http://wiki.darcs.net/Development/GettingStarted

Thanks and happy hacking!
**********************
Copying patches, to get lazy repository hit ctrl-C...        
darcs: user error (Couldn't fetch
`0000017073-ac60cee205c110af2f54efcc0ca27ce2b2e04cc045d03f06bc89d4d75ddf7594'
in subdir pristine.hashed from sources:

thisrepo:/home/ben/src/darcs/bla
cache:/home/ben/.cache/darcs
readonly:/home/ben/.darcs/cache
repo:http://darcs.net/screened
)
Finished cloning.
msg18570 (view) Author: gh Date: 2015-06-21.01:01:15
There is another, older issue for this, please look it up and close this one.

2015-06-20 21:58 GMT-03:00 Ben Franksen <bugs@darcs.net>:
>
> Ben Franksen <benjamin.franksen@helmholtz-berlin.de> added the comment:
>
> It happened to me again:
>
> ben@sarun[1]: .../src/darcs > darcs --version
> 2.10.0 (release)
>
> ben@sarun[1]: .../src/darcs > darcs get http://darcs.net/screened bla
> Welcome to the darcs screened repository.
>
> If you would like to contribute, please read our guide for contributors:
> http://wiki.darcs.net/Development/GettingStarted
>
> Thanks and happy hacking!
> **********************
> Copying patches, to get lazy repository hit ctrl-C...
> darcs: user error (Couldn't fetch
> `0000017073-ac60cee205c110af2f54efcc0ca27ce2b2e04cc045d03f06bc89d4d75ddf7594'
> in subdir pristine.hashed from sources:
>
> thisrepo:/home/ben/src/darcs/bla
> cache:/home/ben/.cache/darcs
> readonly:/home/ben/.darcs/cache
> repo:http://darcs.net/screened
> )
> Finished cloning.
>
> ----------
> resolvedin: 2.10.0 ->
> status: resolved -> unknown
>
> __________________________________
> Darcs bug tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue2421>
> __________________________________
> _______________________________________________
> darcs-devel mailing list
> darcs-devel@darcs.net
> http://lists.osuosl.org/mailman/listinfo/darcs-devel
msg18571 (view) Author: gh Date: 2015-06-21.01:18:58
As I understoof it, this ticket is about the ``couldn t fetch┬┤┬┤ bug 
provoked by unwriteable caches. Now, this precise cause is fixed but 
the message still occurs for some other cause (IIRC dowload 
speculation, something like that). Let us use 
http://bugs.darcs.net/issue2400 instead.
History
Date User Action Args
2014-11-20 04:32:35reilithioncreate
2014-12-29 17:11:28bfsetstatus: unknown -> (no value)
messages: + msg17919
2015-03-09 10:27:56bfsetstatus: unknown
messages: + msg18277
title: user error (Couldn't fetch ... while cloning large repo -> Couldn't fetch ... while cloning darcs repo
2015-03-09 10:37:24bfsettitle: Couldn't fetch ... while cloning darcs repo -> Couldn't fetch ... while cloning/pulling darcs repo
2015-03-09 10:40:12bfsetmessages: + msg18278
2015-03-09 10:43:40bfsetmessages: + msg18279
2015-03-09 10:45:56bfsetpriority: critical
title: Couldn't fetch ... while cloning/pulling darcs repo -> Couldn't fetch ... while pulling (was: cloning) darcs repo
messages: + msg18280
milestone: 2.10.0
2015-03-09 11:54:41bfsetmessages: + msg18281
2015-03-09 12:22:57bfsetmessages: + msg18282
2015-03-09 13:22:22bfsetstatus: unknown -> needs-implementation
messages: + msg18283
2015-03-10 19:43:32noreplysetstatus: needs-implementation -> resolved
messages: + msg18298
resolvedin: 2.10.0
2015-03-10 19:51:31ghsetstatus: resolved -> needs-implementation
messages: + msg18299
2015-03-19 15:07:39ghsetmessages: + msg18320
2015-04-05 04:42:21ghsetmessages: + msg18343
2015-04-13 18:41:25noreplysetstatus: needs-implementation -> resolved
messages: + msg18366
2015-06-21 00:58:46bfsetstatus: resolved -> unknown
messages: + msg18569
resolvedin: 2.10.0 -> (no value)
2015-06-21 01:01:16ghsetmessages: + msg18570
2015-06-21 01:18:59ghsetstatus: unknown -> resolved
messages: + msg18571
resolvedin: 2.10.0