darcs

Patch 1731 store rebase patch at the repo layer

Title store rebase patch at the repo layer
Superseder Nosy List bf
Related Issues
Status needs-review Assigned To
Milestone

Created on 2018-09-21.06:35:54 by bf, last changed 2018-09-25.20:20:24 by bf.

Files
File name Status Uploaded Type Edit Remove
cleanups-in-d_p_named_wrapped-and-d_p_rebase_container.dpatch bf, 2018-09-25.20:20:24 application/x-darcs-patch
move-more-repo-paths-to-d_r_paths.dpatch bf, 2018-09-21.06:35:53 application/x-darcs-patch
move-more-repo-paths-to-d_r_paths.dpatch bf, 2018-09-25.18:52:03 application/x-darcs-patch
patch-preview.txt bf, 2018-09-21.06:35:53 text/x-darcs-patch
patch-preview.txt bf, 2018-09-25.18:52:03 text/x-darcs-patch
patch-preview.txt bf, 2018-09-25.20:20:24 text/x-darcs-patch
unnamed bf, 2018-09-21.06:35:53 text/plain
unnamed bf, 2018-09-25.18:52:03 text/plain
unnamed bf, 2018-09-25.20:20:24 text/plain
See mailing list archives for discussion on individual patches.
Messages
msg20316 (view) Author: bf Date: 2018-09-21.06:35:53
As announced on @darcs-devel here is my bundle for the rebase-storage
refactor. Note this doesn't touch the internal workings of rebase at all. It
is purely about how the rebase patch is stored in the repo. The on-disk
format is exactly the same as before, minus the meta data.

TODO: give the remains of Darcs.Patch.Named.Wrapped a better name and place
in the module hierarchy.

8 patches for repository http://darcs.net/screened:

patch 288d5f53f83e2fd1b72e66a0c37f4c729b531ef0
Author: Ben Franksen <ben.franksen@online.de>
Date:   Mon Jul  2 18:58:26 CEST 2018
  * move more repo paths to D.R.Paths

patch 5aa080f8f1bf87912d49ef4803a1f8ef07f14ef0
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Sep 18 19:00:40 CEST 2018
  * store rebase patch at the repo layer instead of mixing it with normal patches
  
  This does not yet do away with the WrappedNamed layer and the
  RepoType/PatchType cruft, which will be done in a second and third step.
  Some tests now fail which is due to bugs which are only weakly related to
  the change made here, so will be fixed in a follow-up patch.
  
  Note that this changes is incompatible in that previous versions of darcs
  can't handle a repo with a new-style rebase in progress and vice versa. This
  is something we cannot avoid unless we keep all the old code around, which
  would reap us us of most of the benefits we get from this change.

patch f0e13101eb205765ade027749c5af445e79e5580
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Sep 18 04:10:46 CEST 2018
  * two fixes in clone and convert import commands
  
  The bug was in both cases that finalizeRepositoryChanges was not correctly
  paired with revertRepositoryChanges. This was exposed by the new way of
  storing the rebase patch, which crashes when it tries to rename the
  tentative rebase patch back to its final version.

patch cb9d99d39ac946fbbb931464c24154be80e05290
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Sep 18 19:13:23 CEST 2018
  * remove the WrappedNamed layer

patch 404143fadc2551ac409bdf850d9a10f7075cd939
Author: Ben Franksen <ben.franksen@online.de>
Date:   Wed Sep 19 11:56:55 CEST 2018
  * add PatchInfoAndG which is polymorphic in the named patch type
  
  The standard PatchInfoAnd is now a type synonym that fixes the named patch
  type as 'Named'. Unfortunately this required the addition of Eq2 constraints
  in lots of places.
  
  The goal of this generalization is to be able to convert old-style rebasing
  repos, for which we need to read PatchInfoAndG with a simplified version of
  the old WrappedNamed as the named patch type.

patch efa36333ad5e9073488930c0dd9912b47c68bb23
Author: Ben Franksen <ben.franksen@online.de>
Date:   Thu Sep 20 01:10:29 CEST 2018
  * use englishNum for correct grammar in rebase status line

patch 49d7d124a3bd725dff62d22f989679fdef4dc30c
Author: Ben Franksen <ben.franksen@online.de>
Date:   Wed Sep 19 22:11:51 CEST 2018
  * add command 'rebase upgrade'
  
  This required a few additional refactorings mostly in D.R.Hashed. We lift a
  local function to the top level that (lazily) reads the patches from a
  single inventory. Since this does not return a PatchSet but only an RL of
  PatchInfoAnd, we can generalize it to return PatchInfoAndG and so can be
  used with WrappedNamed instead of Named. The WrappedNamed has been
  resurrected and largely cut down to what is needed for this single purpose.

patch 4a12994d2b9d97fd0b81b87014a4ead7c0f95223
Author: Ben Franksen <ben.franksen@online.de>
Date:   Thu Sep 20 15:56:15 CEST 2018
  * reliably fail if we detect that an old-style rebase is in progress
  
  The trick is to check if the repo type is tagged with SIsRebase, which means
  that the repo format has rebase-in-progress, and then count the suspended
  patches in the new-style rebase patch. If it is zero we can assume that we
  have an old-style rebase in progress.
  
  While the check itself is simple, making sure it is called with the right
  parameters at the right time is not. One problem is that we must make an
  exception for the 'rebase upgrade' command. This is achieved by adding a new
  kind of RepoJob (OldRebaseJob) just for this command. A further complication
  arises because startRebaseJob is called with an SIsRebase-typed repo
  regardless of whether a rebase was in progress initially or not. In this
  case we cannot decide whether to run the check based on the repo type alone,
  but instead have to (re-)test the format stored in repository token.
Attachments
msg20318 (view) Author: bf Date: 2018-09-21.07:39:22
I am not screening this bundle yet. While it passes all tests, 
including a new one I added (see the last patch in the bundle), this 
is a breaking change that can cause previous darcs versions to 
"crash"  i.e. throw an unhandled exception (though this happens only 
if you invoke it when a new-style rebase is in progress).

I want everyone to think about whether this is a price you would be 
willing to pay for this cleanup.
msg20323 (view) Author: bf Date: 2018-09-22.21:11:28
One solution to avoid unhandled exceptions is to introduce a new 
format, say new-syle-rebase-in-progress. This gives a slightly more 
decent error message:

"""
ben@juliana[1]:.../darcs/store-rebase-at-repo-layer>darcs what
Unable to "darcs whatsnew" here.

Can't understand repository format: Unknown format: new-style-
rebase-in-progress
"""

However, this is now an error for /every/ command, whereas the crash 
happens only for commands that modify the repo.
msg20324 (view) Author: ganesh Date: 2018-09-23.15:58:52
Sorry, still failing to keep up and reply intelligently to all the 
great work you're doing :-) My immediate assumption was that a new 
repository format was the right way to express a new rebase format. And  
I think we *want* old darcs to refuse to touch such a repo, no matter 
what command it's running.
msg20325 (view) Author: bf Date: 2018-09-23.17:59:37
Thanks Ganesh. Yes, I came to the same conclusion. One further 
advantage of a new format flag is that we can now move patches 
between repos regardless of whether there is a (new-style) rebase in 
progress. This limitation has bitten me a few times in the past and 
I am glad we can get rid of it now and without any additional 
complications.
msg20326 (view) Author: bf Date: 2018-09-23.18:04:43
Ganesh, do you think I should screen this bundle? I have a number of 
follow-ups waiting...
msg20329 (view) Author: ganesh Date: 2018-09-25.05:46:28
I have no objections to screening it. The approach with just 
updating the rebase context in tentativelyAdd/tentativelyRemove does 
look a lot simpler and I'm trying to remember why I didn't do it - I 
do now vaguely remember trying it and giving up. But if the tests 
pass I can't think what the problem could be, perhaps I was just too 
nervous or the repository code was more complicated then.
msg20331 (view) Author: bf Date: 2018-09-25.18:52:03
Here is an extended bundle (same patches as before plus new ones). It is
mostly about compatibility stuff like adding a new format property, tests,
error messages etc, and one or two minor cleanups they depend upon.

14 patches for repository http://darcs.net/screened:

patch 288d5f53f83e2fd1b72e66a0c37f4c729b531ef0
Author: Ben Franksen <ben.franksen@online.de>
Date:   Mon Jul  2 18:58:26 CEST 2018
  * move more repo paths to D.R.Paths

patch 5aa080f8f1bf87912d49ef4803a1f8ef07f14ef0
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Sep 18 19:00:40 CEST 2018
  * store rebase patch at the repo layer instead of mixing it with normal patches
  
  This does not yet do away with the WrappedNamed layer and the
  RepoType/PatchType cruft, which will be done in a second and third step.
  Some tests now fail which is due to bugs which are only weakly related to
  the change made here, so will be fixed in a follow-up patch.
  
  Note that this changes is incompatible in that previous versions of darcs
  can't handle a repo with a new-style rebase in progress and vice versa. This
  is something we cannot avoid unless we keep all the old code around, which
  would reap us us of most of the benefits we get from this change.

patch f0e13101eb205765ade027749c5af445e79e5580
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Sep 18 04:10:46 CEST 2018
  * two fixes in clone and convert import commands
  
  The bug was in both cases that finalizeRepositoryChanges was not correctly
  paired with revertRepositoryChanges. This was exposed by the new way of
  storing the rebase patch, which crashes when it tries to rename the
  tentative rebase patch back to its final version.

patch cb9d99d39ac946fbbb931464c24154be80e05290
Author: Ben Franksen <ben.franksen@online.de>
Date:   Tue Sep 18 19:13:23 CEST 2018
  * remove the WrappedNamed layer

patch 404143fadc2551ac409bdf850d9a10f7075cd939
Author: Ben Franksen <ben.franksen@online.de>
Date:   Wed Sep 19 11:56:55 CEST 2018
  * add PatchInfoAndG which is polymorphic in the named patch type
  
  The standard PatchInfoAnd is now a type synonym that fixes the named patch
  type as 'Named'. Unfortunately this required the addition of Eq2 constraints
  in lots of places.
  
  The goal of this generalization is to be able to convert old-style rebasing
  repos, for which we need to read PatchInfoAndG with a simplified version of
  the old WrappedNamed as the named patch type.

patch efa36333ad5e9073488930c0dd9912b47c68bb23
Author: Ben Franksen <ben.franksen@online.de>
Date:   Thu Sep 20 01:10:29 CEST 2018
  * use englishNum for correct grammar in rebase status line

patch 49d7d124a3bd725dff62d22f989679fdef4dc30c
Author: Ben Franksen <ben.franksen@online.de>
Date:   Wed Sep 19 22:11:51 CEST 2018
  * add command 'rebase upgrade'
  
  This required a few additional refactorings mostly in D.R.Hashed. We lift a
  local function to the top level that (lazily) reads the patches from a
  single inventory. Since this does not return a PatchSet but only an RL of
  PatchInfoAnd, we can generalize it to return PatchInfoAndG and so can be
  used with WrappedNamed instead of Named. The WrappedNamed has been
  resurrected and largely cut down to what is needed for this single purpose.

patch 4a12994d2b9d97fd0b81b87014a4ead7c0f95223
Author: Ben Franksen <ben.franksen@online.de>
Date:   Thu Sep 20 15:56:15 CEST 2018
  * reliably fail if we detect that an old-style rebase is in progress
  
  The trick is to check if the repo type is tagged with SIsRebase, which means
  that the repo format has rebase-in-progress, and then count the suspended
  patches in the new-style rebase patch. If it is zero we can assume that we
  have an old-style rebase in progress.
  
  While the check itself is simple, making sure it is called with the right
  parameters at the right time is not. One problem is that we must make an
  exception for the 'rebase upgrade' command. This is achieved by adding a new
  kind of RepoJob (OldRebaseJob) just for this command. A further complication
  arises because startRebaseJob is called with an SIsRebase-typed repo
  regardless of whether a rebase was in progress initially or not. In this
  case we cannot decide whether to run the check based on the repo type alone,
  but instead have to (re-)test the format stored in repository token.

patch 70e7eb2a2f540344931740f1fa70d84133f6d64a
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sun Sep 23 15:07:10 CEST 2018
  * adapt rebase tests to new style of storing rebase patch
  
  It is now okay to transfer patches between repos with new-style rebase in
  progress; these operations ignore the rebase patch.

patch 18a8aee9ee409ad91985dbdc39095c69f373f9ec
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sat Sep 22 14:28:39 CEST 2018
  * avoid needless String/ByteString conversions when reading format file

patch d51fae030fb6090bce28e9c1fb682334b04be0aa
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sat Sep 22 17:36:12 CEST 2018
  * factor yet another bunch of repo paths to D.R.Paths

patch 1ca018fa626ee2d62d1a3a79fa7e5c686f4037c9
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sat Sep 22 23:29:54 CEST 2018
  * add new repo format new-style-rebase-in-progress

patch 25b1da56c6c7a3fa7ad8acb0dc630b67642de88d
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sun Sep 23 19:19:27 CEST 2018
  * fix error messages for clone/transfer from repo with old-style rebase in progress

patch 33515bbbf9d2c7da70adf5a04441528c87c92805
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sun Sep 23 19:14:42 CEST 2018
  * include clone command in test for new-style rebase
Attachments
msg20332 (view) Author: bf Date: 2018-09-25.18:53:57
Thanks Ganesh, I am screening them now.
msg20337 (view) Author: bf Date: 2018-09-25.20:20:24
Yet another follow-up patch for the rebase-storage refactor.

1 patch for repository http://darcs.net/screened:

patch b59a2d9bd57ad82c29c5bcbf1108688706a23c3a
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sun Sep 23 18:52:57 CEST 2018
  * cleanups in D.P.Named.Wrapped and D.P.Rebase.Container
  
  This mostly removes or fixes outdated docs.
Attachments
History
Date User Action Args
2018-09-21 06:35:55bfcreate
2018-09-21 07:39:22bfsetmessages: + msg20318
2018-09-22 21:11:28bfsetmessages: + msg20323
2018-09-23 15:58:53ganeshsetmessages: + msg20324
2018-09-23 17:59:37bfsetmessages: + msg20325
2018-09-23 18:04:43bfsetmessages: + msg20326
2018-09-25 05:46:29ganeshsetmessages: + msg20329
2018-09-25 18:52:05bfsetfiles: + patch-preview.txt, move-more-repo-paths-to-d_r_paths.dpatch, unnamed
messages: + msg20331
2018-09-25 18:53:57bfsetstatus: needs-screening -> needs-review
messages: + msg20332
2018-09-25 20:20:24bfsetfiles: + patch-preview.txt, cleanups-in-d_p_named_wrapped-and-d_p_rebase_container.dpatch, unnamed
messages: + msg20337