Issue 2556 apply fails with 'hGetBuf: invalid argument (Invalid argument)' for large patch bundle

Title apply fails with 'hGetBuf: invalid argument (Invalid argument)' for large patch bundle
Priority Status unknown
Milestone Resolved in
Superseder Nosy List mihaigiurgeanu
Assigned To

Created on 2017-10-10.11:01:53 by mihaigiurgeanu, last changed 2017-11-03.10:44:35 by bfrk.

msg19722 (view) Author: mihaigiurgeanu Date: 2017-10-10.11:01:52
darcs apply bundle.dpatch 

fails with the error 'hGetBuf: invalid argument (Invalid argument) for very 
large patch bundle (my patch bundle has 2.1G)

If I send the patch bundle on the standard input to 'darcs apply', everything 

    type bundle.dpatch | darcs apply

I used darcs 12.2.5, 64 bit on Windows 10
msg19724 (view) Author: bfrk Date: 2017-10-10.13:42:28
Thanks for the bug report.

The sources for 2.12.5 to not contain any call to hGetBuf, so I guess
this comes from a library call.

Can you attach the output of

  darcs apply bundle.dpatch --debug

(might be large, so you should redirect it to a file). Not sure if this
helps but it may give us an idea where to look.
msg19747 (view) Author: mihaigiurgeanu Date: 2017-10-12.10:47:43
Unfortunately, there is not match in the debug output. It immediately fails with the above error:

    D:\Projects\OPTIMIX\OPTIMIX>darcs apply --debug decompile-produitsimulationdaoimpl.dpatch
    Beginning identifying repository .
    Done identifying repository .
    Identified darcs-2 repo: D:/Projects/OPTIMIX/OPTIMIX
    darcs: D:/Projects/OPTIMIX/OPTIMIX/decompile-produitsimulationdaoimpl.dpatch: hGetBuf: invalid argument (Invalid argument)
msg19757 (view) Author: bfrk Date: 2017-10-16.09:28:59
The fact that there is not much in the debug output does in fact help.
It clearly shows that the error happens right when the patch bundle is
being read from disk. The function that does this is
Darcs.Util.External.gzFetchFilePS. We have to find out why it fails.
msg19799 (view) Author: bfrk Date: 2017-11-03.10:44:33
Sorry for the latency, I was pretty much distracted by other stuff.

gzFetchFilePS reads the whole file into a strict ByteString. It uses
mmapFilePS unless mingw32_HOST_OS is true as a preprocessor symbol.
Looking at the docs for System.IO.MMap I see that Windows is supported
as well. Perhaps we should remove the #if here and use mmap unconditionally?

There is one more detail where reading from stdin differs: in isGZFile
we seek and read off the number of bytes from the last 4 bytes in the
file and return an Int. I think this fails if the size is greater than
2^31. This is the code:

       else do hSeek h SeekFromEnd (-4)
               len <- hGetLittleEndInt h
Date User Action Args
2017-10-10 11:01:53mihaigiurgeanucreate
2017-10-10 13:42:30bfrksetmessages: + msg19724
2017-10-12 10:47:44mihaigiurgeanusetmessages: + msg19747
2017-10-16 09:29:00bfrksetmessages: + msg19757
2017-11-03 10:44:35bfrksetmessages: + msg19799