I'm seeing a behavior where changes on lines that end in "?" are not 
recordable. When such a change is made, "darcs diff" shows the change, but 
"darcs whatsnew" and "darcs record" act like there is no change. This occurs in 
all versions of Darcs I tried, including the latest unstable. Here's a simple 
script that demonstrated the bug:


#!/bin/bash
darcs init --darcs-2

echo "U?" > fil
darcs add fil
darcs record -A me -a -m patch0 

echo "J?" > fil
darcs whatsnew
darcs diff
darcs record

On Mon, Oct 20, 2008 at 6:12 PM, dkogan <bugs@darcs.net> wrote:
>
> New submission from dkogan <dkogan@cds.caltech.edu>:
>
> I'm seeing a behavior where changes on lines that end in "?" are not
> recordable. When such a change is made, "darcs diff" shows the change, but
> "darcs whatsnew" and "darcs record" act like there is no change. This occurs in
> all versions of Darcs I tried, including the latest unstable. Here's a simple
> script that demonstrated the bug:
>
>
> #!/bin/bash
> darcs init --darcs-2
>
> echo "U?" > fil
> darcs add fil
> darcs record -A me -a -m patch0
>
> echo "J?" > fil
> darcs whatsnew
> darcs diff
> darcs record

I tried to reproduce this.  I saved your script as, no-changes-bug.sh:
$ ./no-changes-bug.sh
Finished recording patch 'patch0'
hunk ./fil 1
-U?
+J?
diff -rN old-bug/fil new-bug/fil
1c1
< U?
---
> J?

hunk ./fil 1
-U?
+J?
Shall I record this change? (1/1)  [ynWsfvpxdaqjk], or ? for help:

So, it seems to be working here on OSX using a darcs that is more or
less the current stable.

Jason

I just confirmed the issue on Mac OS X using 2.1.0 (+ 96 patches), which I
happened to build yesterday (Oct 19, 2008):

$ darcs diff
diff -rN -u old-ddd/fil new-ddd/fil
--- old-ddd/fil	2008-10-20 19:05:48.000000000 -0700
+++ new-ddd/fil	2008-10-20 19:05:48.000000000 -0700
@@ -1 +1 @@
-U?
+J?

$ darcs record
No changes!

However, 2.0.2 (release) seems to work on my linux machine (Debian, using the
darcs 2.0.2-2 package).

On Mon, Oct 20, 2008 at 7:14 PM, David <bugs@darcs.net> wrote:
>
> David <david+darcs@porkrind.org> added the comment:
>
> I just confirmed the issue on Mac OS X using 2.1.0 (+ 96 patches), which I
> happened to build yesterday (Oct 19, 2008):
>
> $ darcs diff
> diff -rN -u old-ddd/fil new-ddd/fil
> --- old-ddd/fil 2008-10-20 19:05:48.000000000 -0700
> +++ new-ddd/fil 2008-10-20 19:05:48.000000000 -0700
> @@ -1 +1 @@
> -U?
> +J?
>
> $ darcs record
> No changes!
>
> However, 2.0.2 (release) seems to work on my linux machine (Debian, using the
> darcs 2.0.2-2 package).

Do you think this could be related to the way darcs was built instead
of the exact version?  I guess we could use trackdown and see what it
finds.

Jason

On Mon, Oct 20, 2008 at 6:59 PM, Dima Kogan <dkogan@cds.caltech.edu> wrote:
> Interesting. Yesterday when I was trying to get some things recorded
> that were affected by this, it seemed like the bug didn't occur every
> time, and it took a few attempts to get the patch recorded properly.
> Today, when I looked for the bug to submit the report, it seemed much
> more consistent. In either case, I have two amd64 linux boxes running
> Debian, and they both exhibit the problem consistently. I did try both
> the darcs in Debian (2.0.2), and the new version from the repository (
> 2.1.0 + 139 patches). How can I help track this down? Don't know any
> Haskell, unfortunately.

I think the configure log would be handy.  That way we could see which
libraries and which versions of those libraries you're using.

I don't know if your locale would be useful, but I suppose it couldn't hurt.

What version of GHC did you use?

Thanks!
Jason

I did a darcs trackdown and it started working after unpulling this patch:

Sat Oct  4 16:48:06 PDT 2008  Eric Kow <E.Y.Kow@brighton.ac.uk>
  * Skip filepath test which does not work on Windows.

Hmmm. That's just a shell script regression test--that can't be it. Maybe a ghc
bug? I just got the latest ghc binary from http://haskell.org/ghc/ (6.8.3).

I'm attaching the config.log. GHC version 6.8.2 from the amd64 debian package 
version 6.8.2-7. The locale command returns everything as "en_US.UTF-8". Also, 
I just proved that this is a heisenbug. I removed the "darcs record" line from 
the original script I submitted, and called it darcstest. Running the following 
ugly line of bash

i=0; while true; do echo -n "$i "; rm -rf *; ../darcstest >&1 | grep  changes 
> /dev/null || echo "correct"; i=$(($i+1)); done

you should get the script running repeatedly, printing the iteration count 
followed by "correct" if the correct behavior is seen, and nothing if the 
erroneous behaviour is seen. I get

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 correct
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 correct
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 correct
59 60 61 62 63 64 65 66 67 68 correct
69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 correct

So I see the bug most of the time, but not all the time.

Attachments

• config.log

This seems to be a race condition of some sort. "sleep 1" after the "darcs 
record" in the script makes the bug go away consistently. I just made two 
copies of the repo containing the one patch and the one modification. One is 
good (darcs whatsnew sees the change) and one is bad. It's not obvious why the 
broken one doesn't work, but I'm attaching both of these

Attachments

• darcstest.tar.gz

On Mon, Oct 20, 2008 at 9:57 PM, dkogan <bugs@darcs.net> wrote:
>
> dkogan <dkogan@cds.caltech.edu> added the comment:
>
> I'm attaching the config.log. GHC version 6.8.2 from the amd64 debian package
> version 6.8.2-7. The locale command returns everything as "en_US.UTF-8". Also,
> I just proved that this is a heisenbug. I removed the "darcs record" line from
> the original script I submitted, and called it darcstest. Running the following
> ugly line of bash
>
> i=0; while true; do echo -n "$i "; rm -rf *; ../darcstest >&1 | grep  changes
>> /dev/null || echo "correct"; i=$(($i+1)); done
>
> you should get the script running repeatedly, printing the iteration count
> followed by "correct" if the correct behavior is seen, and nothing if the
> erroneous behaviour is seen. I get
>
> 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 correct
> 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 correct
> 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 correct
> 59 60 61 62 63 64 65 66 67 68 correct
> 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 correct
>
> So I see the bug most of the time, but not all the time.

Ah ha!  Thanks for this little script.  After some trial and error I
noticed a pattern.

When fil is the same size before and after, it's unpredictable.  But
when fil is a different size before and after it's always correct.

For example, this has the heisenbug:
\begin{bash}
#!/bin/bash
darcs init --darcs-2

echo "aaaaa" > fil
echo "aaaaa" >> fil
echo "aaaaa" >> fil
darcs add fil
darcs record -A me -a -m patch0

echo "bbbbb" > fil
echo "bbbbb" >> fil
echo "bbbbb" >> fil
darcs whats
darcs diff
\end{bash}

And this one does not:
\begin{bash}
#!/bin/bash
darcs init --darcs-2

echo "aaaaa" > fil
echo "aaaaa" >> fil
echo "aaaaa" >> fil
darcs add fil
darcs record -A me -a -m patch0

echo "bbbbb" > fil
echo "bbbbb" >> fil
darcs whats
darcs diff
\end{bash}

So it seems that this bug is triggered by same length diffs some how.
The next easiest step in the process is to rebuild darcs with hpc,
then run a good test case in one directory then run a bad test case in
a different directory.  Once you've done that we can look at the
results made by HPC to compare the code paths.  You'll need ghc 6.8.3
to do this though.  So I may have to do it actually.  But, for anyone
reading along at home, here are the directions:
1) In the darcs repository, configure with --enable-hpc
2) build darcs as normal and set $DARCS to point to it
3) mkdir -p ~/tmp/good-case/bug; mkdir -p ~/tmp/bad-case/bug
4) copy this script to good-case as test.sh
\begin{bash}
#!/bin/bash
darcs init --darcs-2

echo "a" > fil
darcs add fil
darcs record -A me -a -m patch0

echo "ab" > fil
$DARCS whats
\end{bash}

5) copy this script to bad-case as test.sh
\begin{bash}
#!/bin/bash
darcs init --darcs-2

echo "a" > fil
darcs add fil
darcs record -A me -a -m patch0

echo "b" > fil
$DARCS whats
\end{bash}

6) cd ~/tmp/good-case/bug;
i=0; while true; do echo -n "$i "; rm -rf *; ../test.sh >&1 | grep
changes >/dev/null || echo "correct"; i=$(($i+1)); done

7) repeat in ~/tmp/bad-case/bug

8) Run this command in each case:
hpc sum --union --exclude=Main --exclude=Context --exclude=Autoconf
--exclude=Workaround *.tix > darcs-$CASE.tix
hpc markup darcs-$CASE.tix

I'm not sure if the hpc sum is needed.

That roughly should give us two html files, one with all the good code
pathes and one with all the bad code pathes.

Jason

On Mon, Oct 20, 2008 at 10:29 PM, dkogan <bugs@darcs.net> wrote:
>
> dkogan <dkogan@cds.caltech.edu> added the comment:
>
> This seems to be a race condition of some sort. "sleep 1" after the "darcs
> record" in the script makes the bug go away consistently. I just made two
> copies of the repo containing the one patch and the one modification. One is
> good (darcs whatsnew sees the change) and one is bad. It's not obvious why the
> broken one doesn't work, but I'm attaching both of these

A race condition, or some other dependence on timing could help
explain what I noticed about files of equal size.

Jason

On Mon, Oct 20, 2008 at 10:33 PM, Jason Dagit <bugs@darcs.net> wrote:
>
> Jason Dagit <dagit@codersbase.com> added the comment:
>
> On Mon, Oct 20, 2008 at 10:29 PM, dkogan <bugs@darcs.net> wrote:
>>
>> dkogan <dkogan@cds.caltech.edu> added the comment:
>>
>> This seems to be a race condition of some sort. "sleep 1" after the "darcs
>> record" in the script makes the bug go away consistently. I just made two
>> copies of the repo containing the one patch and the one modification. One is
>> good (darcs whatsnew sees the change) and one is bad. It's not obvious why the
>> broken one doesn't work, but I'm attaching both of these
>
> A race condition, or some other dependence on timing could help
> explain what I noticed about files of equal size.

Yes, indeed,
darcs whats --ignore-times

Works every time even without the sleep.  So it would seem that our
mtime optimization when diffing files is biting us.

Jason

dkogan,

I'm setting this as wont-fix on the basis that --ignore-times lets you work
around it and it's a rather useful optimization 99% of the time.  If you'd like
a different resolution feel free to reopen this bug.

Thanks!

I see you've closed this already, but I wanted to put one last piece of info out:

Using Dima's wile loop test I did a binary search and found this patch seemed to
cause the problem:

Fri Mar 21 14:35:18 PDT 2008  David Roundy <droundy@darcs.net>
  * write sha256 hashes.
  Since this means we no longer take sha1 hashes of large data, I've removed
  some of the sha1-related config options.

If I unpull that patch (and everything that came after it) then Dima's script
works without the sleep. Maybe sha-256 hashes take a little bit longer to
compute which makes the mtimes line up and cause the problem?

Whoops. I didn't mean to change the status.

On Mon, Oct 20, 2008 at 9:12 PM, dkogan <bugs@darcs.net> wrote:
> echo "J?" > fil
> darcs whatsnew
> darcs diff
> darcs record

This  is truly bizarre.  Thanks for the  easy-to-user bug report,
complete with handy script!

I haven't tracked down the cause of this, but have seen that
increasing the length of the line makes it disappear so it's
definitely  something more tricky than just the question marks
(unsurprisingly)...

David

Thanks a lot for looking into this, dagit.  Now that I know about --ignore-
times, I'm perfectly happy to just use it. Is there anything that can be done 
to make the workaround more discoverable to anybody else who would stumble upon 
it in the future? I ran into this when I was running a script to do many "darcs 
record"s in a row, and I can easily imagine this coming up again. Maybe "darcs 
check" could detect this scenario? This would've saved my time and the time of 
everybody reading this thread.

Interesting. Yesterday when I was trying to get some things recorded
that were affected by this, it seemed like the bug didn't occur every
time, and it took a few attempts to get the patch recorded properly.
Today, when I looked for the bug to submit the report, it seemed much
more consistent. In either case, I have two amd64 linux boxes running
Debian, and they both exhibit the problem consistently. I did try both
the darcs in Debian (2.0.2), and the new version from the repository (
2.1.0 + 139 patches). How can I help track this down? Don't know any
Haskell, unfortunately.

On Mon, 20 Oct 2008 18:40:24 -0700
"Jason Dagit" <dagit@codersbase.com> wrote:

> On Mon, Oct 20, 2008 at 6:12 PM, dkogan <bugs@darcs.net> wrote:
> >
> > New submission from dkogan <dkogan@cds.caltech.edu>:
> >
> > I'm seeing a behavior where changes on lines that end in "?" are not
> > recordable. When such a change is made, "darcs diff" shows the
> > change, but "darcs whatsnew" and "darcs record" act like there is
> > no change. This occurs in all versions of Darcs I tried, including
> > the latest unstable. Here's a simple script that demonstrated the
> > bug:
> >
> >
> > #!/bin/bash
> > darcs init --darcs-2
> >
> > echo "U?" > fil
> > darcs add fil
> > darcs record -A me -a -m patch0
> >
> > echo "J?" > fil
> > darcs whatsnew
> > darcs diff
> > darcs record
> 
> I tried to reproduce this.  I saved your script as, no-changes-bug.sh:
> $ ./no-changes-bug.sh
> Finished recording patch 'patch0'
> hunk ./fil 1
> -U?
> +J?
> diff -rN old-bug/fil new-bug/fil
> 1c1
> < U?
> ---
> > J?
> 
> hunk ./fil 1
> -U?
> +J?
> Shall I record this change? (1/1)  [ynWsfvpxdaqjk], or ? for help:
> 
> So, it seems to be working here on OSX using a darcs that is more or
> less the current stable.
> 
> Jason