darcs

Patch 1157 implement doFastZip to create zip archive from pristin...

Title implement doFastZip to create zip archive from pristin...
Superseder Nosy List darcs-devel, gh, mdiaz, simon
Related Issues
Status accepted Assigned To
Milestone

Created on 2014-05-15.19:57:36 by gh, last changed 2014-06-05.18:00:50 by gh.

Files
File name Status Uploaded Type Edit Remove
implement-dofastzip-to-create-zip-archive-from-pristine-tree.dpatch gh, 2014-05-15.19:57:35 application/x-darcs-patch
implement-dofastzip-to-create-zip-archive-from-pristine-tree.dpatch gh, 2014-05-15.21:14:30 application/x-darcs-patch
implement-dofastzip-to-create-zip-archive-from-pristine-tree.dpatch gh, 2014-05-16.19:11:59 application/x-darcs-patch
patch-preview.txt gh, 2014-05-15.19:57:35 text/x-darcs-patch
patch-preview.txt gh, 2014-05-15.21:14:30 text/x-darcs-patch
patch-preview.txt gh, 2014-05-16.19:11:59 text/x-darcs-patch
unnamed gh, 2014-05-15.19:57:35
unnamed gh, 2014-05-15.21:14:30
unnamed gh, 2014-05-16.19:11:59
See mailing list archives for discussion on individual patches.
Messages
msg17453 (view) Author: gh Date: 2014-05-15.19:57:35
(Work in progress, comments are welcome)

The aim of this patch is to implement a function that builds a
zip archive from the pristine tree of a repository.

This would enable darcsden to have a "Download ZIP" button, like Github
currently has (see for instance <https://github.com/jgm/zip-archive>).
The idea is to help people who do not have darcs installed to
easily retrieve the last version of any repository hosted on darcsden,
thus reducing lock-in and frustration.

We already have a `darcs dist` command that generates a tar.gz
archive from a repository. It accepts matchers (--tag for instance)
and the --set-scripts-executable flag.

However in the case of "Download ZIP", reusing the code of darcs dist
to generate a zip file did not seem good to me because:

* `darcs dist` creates a temporary working copy and then
  compresses it, this is quite inefficient (for a website that
  may host many and big repoitories)
* a "download zip" button would only care about the last version
* zip files do not store executable flags anyway

So I implemented two functions: Darcs.Repository.HashedIO.pathsAndContents
that returns a list of pairs (FilePath,B.ByteString) of the whole
pristine tree (with paths rooted at a path given as argument),
and Darcs.UI.Commands.Dist.doFastZip that generates the zip file
from a repository.

The patch introduces a dependency on zip-archive, which seems to
be the best haskell module to create zip files. It is written by
John MacFarlane who uses it for its program Pandoc. The new dependency
on time is needed to get the current time when creating zip file.
I introduced it after looking at pandoc's code.

Things are not perfect, here are some issues:

* Can we get rid of the new time dependency?
* I just opened <https://github.com/jgm/zip-archive/issues/19>
  since zip-archive seems to generate incorrect zip files.
  This is not our bug but it would be nice to see if it gets fixed.
* The new pathsAndContents function of HashedIO is quite ad-hoc.
* pathAndContents returns strict bytestrings, is that a problem?
* do we really want doFastZip inside of darcs, or in darcsden? 
  I'd rather have it inside of darcs (for responsibility and
  maintainability sake). Then a --zip flag for `darcs dist` would
  make sense.

To try the patch, apply it, replace `commandCommand = distCmd`
by `commandCommand = doFastZip`, compile and run `darcs dist` in
some repository.

1 patch for repository http://darcs.net:

Thu May 15 16:35:14 ART 2014  Guillaume Hoffmann <guillaumh@gmail.com>
  * implement doFastZip to create zip archive from pristine tree
Attachments
msg17454 (view) Author: gh Date: 2014-05-15.21:14:30
New version.

John MacFarlane fixed zip-archive's bug so I bumped the dependency.

Got rid of time dependency (used datetime, a dependency we introduced
when porting fast-export/import code).

Also I thing the strict bytestrings of pathAndContents are not
a problem since the list returned by this function *is* lazy.

I've added a --zip flag to `darcs dist` so to easily try the feature.

Remaining issues:

* The new pathsAndContents function of HashedIO is quite ad-hoc.
* do we really want doFastZip inside of darcs, or in darcsden? 

1 patch for repository http://darcs.net:

Thu May 15 18:07:00 ART 2014  Guillaume Hoffmann <guillaumh@gmail.com>
  * implement doFastZip to create zip archive from pristine tree
Attachments
msg17455 (view) Author: ganesh Date: 2014-05-15.22:12:20
I think it belongs in darcs.
msg17456 (view) Author: gh Date: 2014-05-16.19:11:59
Updated help string of dist, and added zip archive check to
shell test dist.sh. I'm screening it.

1 patch for repository http://darcs.net:

Fri May 16 16:09:46 ART 2014  Guillaume Hoffmann <guillaumh@gmail.com>
  * implement doFastZip to create zip archive from pristine tree
Attachments
msg17522 (view) Author: gh Date: 2014-06-05.18:00:50
No regrets, I'm self-accepting the patch.
History
Date User Action Args
2014-05-15 19:57:36ghcreate
2014-05-15 19:58:22ghsetnosy: + darcs-devel, simon, mdiaz
2014-05-15 21:14:31ghsetfiles: + patch-preview.txt, implement-dofastzip-to-create-zip-archive-from-pristine-tree.dpatch, unnamed
messages: + msg17454
2014-05-15 22:12:20ganeshsetmessages: + msg17455
2014-05-16 19:12:00ghsetfiles: + patch-preview.txt, implement-dofastzip-to-create-zip-archive-from-pristine-tree.dpatch, unnamed
messages: + msg17456
2014-05-16 19:13:28ghsetstatus: needs-screening -> needs-review
2014-06-05 18:00:50ghsetstatus: needs-review -> accepted
messages: + msg17522