When running "make test" in an AFS directory for darcs (both 1.0.5 and
the current stable repository), pull.pl has a failure:

pull......................ok 2/0
#     Failed test (pull.pl at line 56)
#                   'Pulling from "../temp2"...
# Finished pulling and applying.
# '
#     doesn't match '(?i-xsm:one/date\.t.+: permission denied)'
pull......................ok 9/0# Looks like you failed 1 test of 14.
pull......................dubious
       Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 3
       Failed 1/14 tests, 92.86% okay

This is the test where the harness calls "chmod -w" on the client repo
and then checks to make sure that it can't still pull into it.
However, on AFS, the standard Unix permissions are not as important as
the "acl" permissions, and chmod -w on a directory is not actually
enough to stop the user from making new files in it, so this test
fails.

I'm not sure what the best solution here is: probably just ignoring
this test on AFS like it does on Windows, but I'm not positive how to
test this appropriately.

I'm happy to test out any patch that others might come up with.

[I guess this should still be unread since the only "chat" is the original 
message.]

Darcs works (and is safe w.r.t. multiple accesses) on POSIX filesystems and on
NFS.  Nobody has reviewed the code for work on AFS.

By using Darcs on AFS, you're playing with fire; there's absolutely no guarantee
that Darcs will not eat your repository.  Unfortunately, as none of the Darcs
developers has a clue about AFS, that's not likely to change soon.

Patches to make sure Darcs is safe over AFS will be gratefully accepted.

I'm renaming the bug, and marking it wishlist.

Ah, ok.

I'm much more of a user of AFS than a developer, but I do have access to people
with more AFS clue than I do.  I doubt I'll have time to work on this any time
soon, but perhaps at some point I will -- I'd love to help the darcs project on
this.

I take it the issue is mostly locking?

> I take it the issue is mostly locking?

There's certainly locking, but I hear that AFS also has some weird
semantics w.r.t. rename.

                                        Juliusz

> By using Darcs on AFS, you're playing with fire; there's absolutely no
> guarantee that Darcs will not eat your repository.  Unfortunately, as
> none of the Darcs developers has a clue about AFS, that's not likely
> to change soon.

AFS supports flock() and implements it properly.  Darcs should not be relying on
something other than flock() for file locking.  So long as it isn't, darcs will
work with AFS.

Could somebody confirm/deny if darcs uses flock()?

AFS is far less likely to result in data corruption than NFS, since you don't 
see changes until a close().

Note that ALL afs clients do "silly rename" the way the Linux NFS client does:

  http://nfs.sourceforge.net/#faq_d2

This can result in .__afsXXXX droppings, but you would get the same thing with
NFS.

sorry, I thought the "title" field was supposed to be the title of my comment, not 
the title of the bug.

There's a nice summary of the semantic differences in AFS here:

 http://www.angelfire.com/hi/plutonic/afs-faq.html#sub2.01

The ones that might tangentially affect darcs:

  - file ownership+permissions work differently
    (IMHO these should not be darcs' concern)

  - save-on-close
    (only an issue with multiple concurrent writers, which
     should not happen if flock() is used properly)

  - no byte range locking
    (does darcs use byte-range locking? I doubt it.)

On Wed, May 31, 2006 at 02:00:12AM +0000, Adam Megacz wrote:
> The ones that might tangentially affect darcs:
> 
>   - file ownership+permissions work differently
>     (IMHO these should not be darcs' concern)
> 
>   - save-on-close
>     (only an issue with multiple concurrent writers, which
>      should not happen if flock() is used properly)
> 
>   - no byte range locking
>     (does darcs use byte-range locking? I doubt it.)

I don't see how these would cause trouble with darcs, but Juliusz is
our FS expert and would know better.
-- 
David Roundy

> AFS supports flock() and implements it properly.  Darcs should not
> be relying on something other than flock() for file locking.  So
> long as it isn't, darcs will work with AFS.

Darcs doesn't use flock(2), it uses lockfiles and link(2).  This
relies on the protocol being strongly consistent w.r.t. i-node
creation, which is the case with NFS.

What are the consistency properties guaranteed by AFS?

> Note that ALL afs clients do "silly rename"

That's okay.  But I recall that AFS has some weird semantics
w.r.t. rename, although I don't remember the details.

Adam, I really need authoritative documentation on

  - consistency properties of AFS;
  - restrictions of Unix system calls on AFS.

Unless you can provide us with pointers to the above, Darcs will not
run over AFS.

                                        Julek

Adam, can you provide me with an account on a machine with AFS and GHC
installed?  (If so, I'll send you a public key.)

> There's a nice summary of the semantic differences in AFS here:
>
>  http://www.angelfire.com/hi/plutonic/afs-faq.html#sub2.01

Thanks, although I'd prefer authoritative docs to a FAQ entry.

Here's the issues that I notice:

> Save on close:                                          [ Programmer ]
> 
>       AFS Cache Manager does not send file modifications to a file server
>       until the close() or fsync() system call.

That's okay -- it's no different from NFS.  (Off topic, but the Sprite
project has shown how to efficiently provide better semantics than
that more than 20 years ago.)

> Hard links:                                             [ User ]
> 
>       In AFS, hard links (eg: ln old new) are only valid within a directory.

This will definitely break ``darcs get'' and ``optimize --relink''
(anything else?).  We can work around the issue, but you'll have to
tell us in what way link(2) fails when the above constraint is
violated.

> Changing file protection by moving a file:              [ User ]
>  
>       Moving a file to a different directory will change the protection
>       of a file

I have no idea what that implies w.r.t. Darcs, but it sounds worrying.

The entry doesn't answer the following questions:

  - how does open(O_CREAT | O_EXCL) work?
  - is link(2) consistent w.r.t. link and open?
  - is rename(2) consistent w.r.t. rename and open?

We can work around any of the above, but we need to know.

And one off-topic criticism:

      AFS does not support byte-range locking within a file,
      although lockf() and fcntl() calls will return 0 (success).

This is careless.  I fully agree that SVR4-style locks are brain-
damaged beyond hope, but fcntl(2) over AFS should fail with ENOSYS
rather than returning success!

                                        Juliusz

> Adam, can you provide me with an account on a machine with AFS and GHC
> installed?  (If so, I'll send you a public key.)

I can provide you immediately with an account on an AFS *server* (/afs/
research.cs.berkeley.edu/) if you are willing to install the OpenAFS client on 
one of your machines (Mac, Windows, nearly every UNIX)...  I wrote up a quick 
guide here:

  http://research.cs.berkeley.edu/doc/afs/

I can make you an account on this cell very easily (send me a password using my 
GPG sig, which is in the MIT keyserver under adam@megacz.com).  Otherwise I can
check with the security people about what it takes to get you an ssh account.

> Thanks, although I'd prefer authoritative docs to a FAQ entry.

Yeah, this annoys me about AFS as well.  The following documentation was 
authoritative at the time that IBM open-source AFS in 2000:

  http://openafs.org/pages/doc/

The OpenAFS people are now standardizing the AFS protocol (and file semantics), 
but that's going to take a few months.

  http://michigan-openafs-lists.central.org/mailman/listinfo/afs3-
standardization

> > Changing file protection by moving a file:              [ User ]

> >       Moving a file to a different directory will change the protection
> >       of a file

> I have no idea what that implies w.r.t. Darcs, but it sounds worrying.

It's not what it sounds like.  In AFS, access control is at the directory level;
people who use AFS know this and don't rely on per-file permissions.  Moving
directories does not change their access controls.

> And one off-topic criticism:
> 
>       AFS does not support byte-range locking within a file,
>       although lockf() and fcntl() calls will return 0 (success).
>
> This is careless.  I fully agree that SVR4-style locks are brain-
> damaged beyond hope, but fcntl(2) over AFS should fail with ENOSYS
> rather than returning success!

I'm almost certain this has been fixed.  You're definately right, it's insane.

True byte-range locking is implemented in the "unstable" OpenAFS 1.5.0, but 
nobody should be using that for anything important yet.

  - a

Adam,

> - how does open(O_CREAT | O_EXCL) work?
> - is link(2) consistent w.r.t. link and open?
> - is rename(2) consistent w.r.t. rename and open?

Could you please find out the answers to these questions?  I'd rather
not declare Darcs safe to use on AFS before I convince myself it is.

                                        Juliusz

> - is link(2) consistent w.r.t. link and open?

BTW, this one is sort of not applicable since hardlinks in AFS can't be created
across directories.  I'm researching solid answers for the other two points.

> - is rename(2) consistent w.r.t. rename and open?

By this I assume you want to know:

  1. if two processes both try to rename the same file, does AFS guarantee
     that exactly one of them succeeds

  2. if one process open()s a file and another rename()s it, can the first
     process keep reading/writing the file?

I know the answer to #2 is "yes".

Okay, I got an answer from Jeffrey Altman, who is one of the AFS Elders (the 
group of nine people that decides "what AFS is":

  http://lists.openafs.org/pipermail/openafs-info/2006-June/022800.html

I know it's not as authoritative as you had hoped, but the basic upshot is that 
you can in fact rely on all of these synchronization mechanisms (except link,
since AFS doesn't support hardlinks across directories).

You bet.  I'll do this right away.

  - a

On 6/7/06, Juliusz Chroboczek <bugs@darcs.net> wrote:
>
> Juliusz Chroboczek <jch@pps.jussieu.fr> added the comment:
>
> Adam,
>
> > - how does open(O_CREAT | O_EXCL) work?
> > - is link(2) consistent w.r.t. link and open?
> > - is rename(2) consistent w.r.t. rename and open?
>
> Could you please find out the answers to these questions?  I'd rather
> not declare Darcs safe to use on AFS before I convince myself it is.
>
>                                         Juliusz
>
> ____________________________________
> Darcs issue tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue117>
> ____________________________________
>

Thanks to a bunch of information provided by Adam, I'm now fairly positive that
Darcs is safe to use over AFS.  Of course, this information is provided with no
guarantee, except that if it breaks you get to keep both pieces.

Juliusz, you are my hero. =)

  - a

On 7/13/06, Juliusz Chroboczek <bugs@darcs.net> wrote:
>
> Juliusz Chroboczek <jch@pps.jussieu.fr> added the comment:
>
> Thanks to a bunch of information provided by Adam, I'm now fairly positive that
> Darcs is safe to use over AFS.  Of course, this information is provided with no
> guarantee, except that if it breaks you get to keep both pieces.
>
> ----------
> status: chatting -> resolved
>
> ____________________________________
> Darcs issue tracker <bugs@darcs.net>
> <http://bugs.darcs.net/issue117>
> ____________________________________
>