darcs

Patch 200 Resolve issue1756: new h-s handles emptied directories...

Title Resolve issue1756: new h-s handles emptied directories...
Superseder Nosy List darcs-users, ganesh, kowey, mornfall
Related Issues moving files between directories breaks index
View: 1756
Status accepted Assigned To mornfall
Milestone

Created on 2010-03-21.14:56:34 by mornfall, last changed 2011-05-10.22:07:04 by darcswatch. Tracked on DarcsWatch.

Files
File name Status Uploaded Type Edit Remove
resolve-issue1756_-new-h_s-handles-emptied-directories-correctly-in-index_.dpatch mornfall, 2010-03-21.14:56:32 text/x-darcs-patch
resolve-issue1756_-new-h_s-handles-emptied-directories-correctly-in-index_.dpatch mornfall, 2010-03-21.15:05:37 text/x-darcs-patch
resolve-issue1756_-new-h_s-handles-file-removals-correctly-in-index_.dpatch mornfall, 2010-03-24.21:28:23 text/x-darcs-patch
unnamed mornfall, 2010-03-21.14:56:33 text/plain
unnamed mornfall, 2010-03-21.15:05:37 text/plain
unnamed mornfall, 2010-03-24.21:28:23 text/plain
See mailing list archives for discussion on individual patches.
Messages
msg10393 (view) Author: mornfall Date: 2010-03-21.14:56:33
Here.

15:55:25 | morn@twi:~/dev/darcs/hashed-storage-0.4.9 -> darcs diff --from-tag 0.4.8
Sun Mar 21 15:54:09 CET 2010  Petr Rockai <me@mornfall.net>
  tagged 0.4.9
Sun Mar 21 15:54:03 CET 2010  Petr Rockai <me@mornfall.net>
  * Bump version to 0.4.9.
Sun Mar 21 15:48:50 CET 2010  Petr Rockai <me@mornfall.net>
  * Make Index not break itself when a new empty directory emerges.
Sat Mar 20 15:21:08 CET 2010  Petr Rockai <me@mornfall.net>
  tagged 0.4.8
diff -rN -u -p old-hashed-storage-0.4.9/hashed-storage.cabal new-hashed-storage-0.4.9/hashed-storage.cabal
--- old-hashed-storage-0.4.9/hashed-storage.cabal       2010-03-21 15:55:50.000000000 +0100
+++ new-hashed-storage-0.4.9/hashed-storage.cabal       2010-03-21 15:55:50.000000000 +0100
@@ -1,5 +1,5 @@
 name:          hashed-storage
-version:       0.4.8
+version:       0.4.9
 synopsis:      Hashed file storage support code.
 
 description:   Support code for reading and manipulating hashed file storage
diff -rN -u -p old-hashed-storage-0.4.9/Storage/Hashed/Index.hs new-hashed-storage-0.4.9/Storage/Hashed/Index.hs
--- old-hashed-storage-0.4.9/Storage/Hashed/Index.hs    2010-03-21 15:55:50.000000000 +0100
+++ new-hashed-storage-0.4.9/Storage/Hashed/Index.hs    2010-03-21 15:55:50.000000000 +0100
@@ -271,9 +271,8 @@ readDir index state item =
        inferiors <- if want then subs $ start substate
                             else return []
 
-       let we_changed = or [ changed x | (_, x) <- inferiors ] || nullleaf
-           nullleaf = null inferiors && oldhash == nullsha
-           nullsha = SHA256 (BS.replicate 32 0)
+       let we_changed = or [ changed x | (_, x) <- inferiors ] || leaf
+           leaf = null inferiors
            tree' = makeTree [ (n, fromJust $ treeitem s) | (n, s) <- inferiors, isJust $ treeitem s ]
            treehash = we_changed ? (hashtree index tree', oldhash)
            tree = tree' { treeHash = treehash }

1 patch for repository http://darcs.net:

Sun Mar 21 15:46:46 CET 2010  Petr Rockai <me@mornfall.net>
  * Resolve issue1756: new h-s handles emptied directories correctly in Index.
Attachments
msg10395 (view) Author: mornfall Date: 2010-03-21.15:05:37
Forgotten comma in there. ... : - (
1 patch for repository darcs-unstable@darcs.net:darcs:

Sun Mar 21 16:02:03 CET 2010  Petr Rockai <me@mornfall.net>
  * Resolve issue1756: new h-s handles emptied directories correctly in Index.
Attachments
msg10433 (view) Author: kowey Date: 2010-03-23.01:37:57
I'd hate to add to Ganesh's already burgeoning review pile, but as we're
trying to get darcs 2.4.1 out the door, I think he's the best/fastest
person to look into this...
msg10472 (view) Author: ganesh Date: 2010-03-23.22:11:31
We record a change in a dir if

(a) any of the children have changed
(b) the dir is now empty

I think this still leaves a hole when a file is removed from a directory 
but the directory is not then empty. A simple amendment to the attached 
test - moving e/f back into d - seems to confirm this.

(That said, this patch is clearly an improvement on the previous 
behaviour)
msg10483 (view) Author: mornfall Date: 2010-03-24.08:00:14
Ganesh Sittampalam <bugs@darcs.net> writes:

> Ganesh Sittampalam <ganesh@earth.li> added the comment:
>
> We record a change in a dir if
>
> (a) any of the children have changed
> (b) the dir is now empty
>
> I think this still leaves a hole when a file is removed from a directory 
> but the directory is not then empty. A simple amendment to the attached 
> test - moving e/f back into d - seems to confirm this.
>
> (That said, this patch is clearly an improvement on the previous 
> behaviour)
Oh, I see. Mostly means I screwed up the testcase and got lulled by it
into thinking this was it. I'll look into this some more, then. :\

Yours,
   Petr.
msg10484 (view) Author: mornfall Date: 2010-03-24.09:40:00
Petr Rockai <me@mornfall.net> writes:

> Oh, I see. Mostly means I screwed up the testcase and got lulled by it
> into thinking this was it. I'll look into this some more, then. :\

This should be a proper fix... (also rolls back the previous version).

diff -rN -u -p old-hashed-storage-0.4/Storage/Hashed/Index.hs new-hashed-storage-0.4/Storage/Hashed/Index.hs
--- old-hashed-storage-0.4/Storage/Hashed/Index.hs      2010-03-24 10:38:08.000000000 +0100
+++ new-hashed-storage-0.4/Storage/Hashed/Index.hs      2010-03-24 10:38:08.000000000 +0100
@@ -261,9 +261,7 @@ readDir index state item =
            subs off | off < following = do
              result <- readItem index $ substate { start = off }
              rest <- subs $ next result
-             case treeitem result of
-               Nothing -> return $! rest
-               Just _  -> return $! (name (resitem result) $ dirlength substate, result) : rest
+             return $! (name (resitem result) $ dirlength substate, result) : rest
            subs coff | coff == following = return []
                      | otherwise = fail $ "Offset mismatch at " ++ show coff ++
                                           " (ends at " ++ show following ++ ")"
@@ -271,8 +269,9 @@ readDir index state item =
        inferiors <- if want then subs $ start substate
                             else return []
 
-       let we_changed = or [ changed x | (_, x) <- inferiors ] || leaf
-           leaf = null inferiors
+       let we_changed = or [ changed x | (_, x) <- inferiors ] || nullleaf
+           nullleaf = null inferiors && oldhash == nullsha
+           nullsha = SHA256 (BS.replicate 32 0)
            tree' = makeTree [ (n, fromJust $ treeitem s) | (n, s) <- inferiors, isJust $ treeitem s ]
            treehash = we_changed ? (hashtree index tree', oldhash)
            tree = tree' { treeHash = treehash }

Yours,
   Petr.
msg10497 (view) Author: ganesh Date: 2010-03-24.20:52:57
Looks ok to me, but we really need better test coverage of this area..
msg10499 (view) Author: ganesh Date: 2010-03-24.21:07:36
Just realised we still need an amended version of the patch to bump to h-s 
0.4.10, and I think you mentioned tests too. Please go ahead and push 
yourself once ready.
msg10500 (view) Author: mornfall Date: 2010-03-24.21:28:23
Fixed up.

1 patch for repository darcs-unstable@darcs.net:darcs:

Wed Mar 24 22:24:53 CET 2010  Petr Rockai <me@mornfall.net>
  * Resolve issue1756: new h-s handles file removals correctly in Index.
Attachments
msg10501 (view) Author: darcswatch Date: 2010-03-24.21:45:39
This patch bundle (with 1 patches) was just applied to the repository http://darcs.net/.
This message was brought to you by DarcsWatch
http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-7aa3ca11c9a3009034ccd57f987108cc7d5046ec
msg14226 (view) Author: darcswatch Date: 2011-05-10.19:38:07
This patch bundle (with 1 patches) was just applied to the repository http://darcs.net/reviewed.
This message was brought to you by DarcsWatch
http://darcswatch.nomeata.de/repo_http:__darcs.net_reviewed.html#bundle-7aa3ca11c9a3009034ccd57f987108cc7d5046ec
History
Date User Action Args
2010-03-21 14:56:34mornfallcreate
2010-03-21 14:58:08darcswatchsetdarcswatchurl: http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-eac38453256086ee92da7a03c244bcefdc13b506
2010-03-21 15:05:38mornfallsetfiles: + resolve-issue1756_-new-h_s-handles-emptied-directories-correctly-in-index_.dpatch, unnamed
messages: + msg10395
2010-03-21 15:08:59darcswatchsetdarcswatchurl: http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-eac38453256086ee92da7a03c244bcefdc13b506 -> http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-a7c2ab5c60d2208a6e1c942e4400a83cd64d8471
2010-03-23 01:37:57koweysetnosy: + kowey, ganesh
messages: + msg10433
assignedto: ganesh
2010-03-23 22:11:31ganeshsetstatus: needs-review -> followup-requested
assignedto: ganesh -> mornfall
messages: + msg10472
2010-03-23 22:14:09ganeshsetissues: + moving files between directories breaks index
2010-03-24 08:00:15mornfallsetmessages: + msg10483
2010-03-24 09:40:03mornfallsetmessages: + msg10484
2010-03-24 20:52:58ganeshsetstatus: followup-requested -> accepted-pending-tests
assignedto: mornfall -> ganesh
messages: + msg10497
2010-03-24 21:07:37ganeshsetstatus: accepted-pending-tests -> followup-in-progress
assignedto: ganesh -> mornfall
messages: + msg10499
2010-03-24 21:28:24mornfallsetfiles: + resolve-issue1756_-new-h_s-handles-file-removals-correctly-in-index_.dpatch, unnamed
messages: + msg10500
2010-03-24 21:29:29darcswatchsetdarcswatchurl: http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-a7c2ab5c60d2208a6e1c942e4400a83cd64d8471 -> http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-7aa3ca11c9a3009034ccd57f987108cc7d5046ec
2010-03-24 21:45:41darcswatchsetstatus: followup-in-progress -> accepted
messages: + msg10501
2011-05-10 19:05:51darcswatchsetdarcswatchurl: http://darcswatch.nomeata.de/repo_http:__darcs.net_.html#bundle-7aa3ca11c9a3009034ccd57f987108cc7d5046ec -> http://darcswatch.nomeata.de/repo_http:__darcs.net_reviewed.html#bundle-a7c2ab5c60d2208a6e1c942e4400a83cd64d8471
2011-05-10 19:38:07darcswatchsetmessages: + msg14226
2011-05-10 22:07:04darcswatchsetdarcswatchurl: http://darcswatch.nomeata.de/repo_http:__darcs.net_reviewed.html#bundle-a7c2ab5c60d2208a6e1c942e4400a83cd64d8471 -> http://darcswatch.nomeata.de/repo_http:__darcs.net_reviewed.html#bundle-eac38453256086ee92da7a03c244bcefdc13b506