darcs

Issue 739 compile with threaded runtime for more responsive progress reports

Title compile with threaded runtime for more responsive progress reports
Priority feature Status resolved
Milestone Resolved in
Superseder Nosy List darcs-devel, dmitry.kurochkin, kowey, thorkilnaur, tommy, zooko
Assigned To
Topics

Created on 2008-03-08.23:57:00 by zooko, last changed 2009-08-27.14:09:25 by admin.

Messages
msg3847 (view) Author: zooko Date: 2008-03-08.23:56:59
David Reid reported on IRC:

"(I ran darcs get http://allmydata.org/source/tahoe/trunk tahoe)
... there is no status ..."

A minute later he said "Oh, now there is status.".

However, I believe that the current UI principle for darcs is that  
there should *always* be status, so perhaps this is a bug.

Regards,

Zooko
msg3947 (view) Author: droundy Date: 2008-03-21.15:44:10
What is status?
msg4018 (view) Author: zooko Date: 2008-03-26.03:13:55
Oh, I'm sorry I didn't understand your question when you posted it on the 21st,
and just now re-reading this ticket, I understand that you didn't understand the
bug report!

"Status" is an indication of what darcs is trying to do, or a measure of
progress.  It's the thing that you added to darcs-2 -- quick notification to the
user of why the user is waiting.
msg4023 (view) Author: droundy Date: 2008-03-26.14:10:50
Okay.  I suspect the only option for providing feedback in this case is to
compile darcs with the -threaded flag to ghc.  This changes the semantics of ghc
considerably, and so it won't happen (at least as the default) until after the
2.0 release.

In particular, the problem is that darcs is taking a long time while doing the
download.  But the downloading is all done by a FFI call (to a C library, I
presume... it would depend on how darcs was compiled), and we can't do anything
while an FFI call is running unless we run with multiple threads.  I'd like to
do that, as a simple compile flag ought to give us a much more
responsive-appearing system (and could even allow us to marginally take
advantage of multiple cores), but I've been told that it's a big change in the
RTS that is rather risky.  It also may expose bugs in older (or newer?) versions
of ghc, so even if I make the change and test it thoroughly myself, it could
still bite our users.  And right now the test suite has very little that would
stress this sort of system, partly because we wrote the test suite to run
quickly, an partly because we don't do much sshing and downloading in the test
suite (for both efficiency and portability).

David
msg4027 (view) Author: zooko Date: 2008-03-26.15:43:53
I strongly agree that using pre-emptive multithreading is a bad idea  
-- it leads to unreproducible, untestable behavior.

So, what about emitting a string to the user saying "Downloading..."  
before making that FFI call?

David Reid's complaint wasn't that it took a long time, it was that  
it didn't show any current status for a long time.

Regards,

Zooko
msg4029 (view) Author: droundy Date: 2008-03-26.16:58:49
On Wed, Mar 26, 2008 at 08:37:50AM -0700, zooko wrote:
> I strongly agree that using pre-emptive multithreading is a bad idea  
> -- it leads to unreproducible, untestable behavior.

I don't think it's a bad idea, I just think that it's better to wait.  As
long as we don't do stupid things, there's no reason (given that we're
writing in Haskell with a decent API that means unsafe things are hard to
do) why it shouldn't be perfectly safe.  It's only that we don't *know*
that it's safe.

> So, what about emitting a string to the user saying "Downloading..."  
> before making that FFI call?
>
> David Reid's complaint wasn't that it took a long time, it was that  
> it didn't show any current status for a long time.

The problem with that is that I want darcs to produce no output for
commands that run very quickly.  I think it's much nicer to not go flashing
things on the screen all the time.  It may be that the trick we use for
progress reports (which involves overwriting them with spaces that aren't
flushed to the screen) would keep this from being too annoying, but I still
would rather keep the current approach of printing progress reports only
so often (about once a second).
-- 
David Roundy
Department of Physics
Oregon State University
msg4033 (view) Author: zooko Date: 2008-03-26.17:15:30
>
>> So, what about emitting a string to the user saying "Downloading..."
>> before making that FFI call?
>>
>> David Reid's complaint wasn't that it took a long time, it was that
>> it didn't show any current status for a long time.
>
> The problem with that is that I want darcs to produce no output for
> commands that run very quickly.  I think it's much nicer to not go  
> flashing
> things on the screen all the time.  It may be that the trick we use  
> for
> progress reports (which involves overwriting them with spaces that  
> aren't
> flushed to the screen) would keep this from being too annoying, but  
> I still
> would rather keep the current approach of printing progress reports  
> only
> so often (about once a second).

Instead of waiting a second, and then printing a progress report,  
which means that if your attempt to wait a second actually takes many  
seconds due to FFI not returning, why not print a progress report,  
and then refrain from printing another one if less than one second  
has passed?

Regards,

Zooko
msg4036 (view) Author: droundy Date: 2008-03-26.17:31:16
On Wed, Mar 26, 2008 at 10:09:28AM -0700, zooko wrote:
> >>So, what about emitting a string to the user saying "Downloading..."
> >>before making that FFI call?
> >>
> >>David Reid's complaint wasn't that it took a long time, it was that
> >>it didn't show any current status for a long time.
> >
> >The problem with that is that I want darcs to produce no output for
> >commands that run very quickly.  I think it's much nicer to not go
> >flashing things on the screen all the time.  It may be that the trick we
> >use for progress reports (which involves overwriting them with spaces
> >that aren't flushed to the screen) would keep this from being too
> >annoying, but I still would rather keep the current approach of printing
> >progress reports only so often (about once a second).
> 
> Instead of waiting a second, and then printing a progress report,  
> which means that if your attempt to wait a second actually takes many  
> seconds due to FFI not returning, why not print a progress report,  
> and then refrain from printing another one if less than one second  
> has passed?

Because I'd rather print zero bug reports when darcs finishes quickly.
-- 
David Roundy
Department of Physics
Oregon State University
msg4039 (view) Author: zooko Date: 2008-03-26.18:01:04
> On Wed, Mar 26, 2008 at 10:09:28AM -0700, zooko wrote:
> Because I'd rather print zero bug reports when darcs finishes quickly.

You would rather that darcs sometimes waits without informing the  
user, rather than thta it always prints a progress report, even when  
it runs quickly?

That's a reasonable preference for you to have, but I will point out  
that there are two different bug reports, by Rob Kinninmont and by  
David Reid, complaining about the former behavior, not counting my  
own bug reports about it.

Maybe we could try doing it the other way and see if any users  
complain about the issue of "too many flashing messages".

Regards,

Zooko
msg4055 (view) Author: droundy Date: 2008-03-27.18:28:27
I've just pushed a configure --threaded option, which results in consistently
responsive progress reports.  I've left it as non-default, but I see no reason
why it should be unsafe.
msg4847 (view) Author: kowey Date: 2008-05-23.11:08:59
Resolved with

Fri May  9 20:58:33 BST 2008  David Roundy <droundy@darcs.net>
  * resolve Issue739: compile with threaded runtime by default.
  It turns out that the tempfile-removal cleanup wasn't particularly feasible
  without the threaded runtime.
History
Date User Action Args
2008-03-08 23:57:00zookocreate
2008-03-09 01:46:22SamBlinkissue740 superseder
2008-03-21 15:44:11droundysetpriority: bug
nosy: droundy, tommy, beschmi, kowey, zooko
status: unread -> unknown
messages: + msg3947
2008-03-26 03:13:57zookosetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4018
2008-03-26 14:10:52droundysetpriority: bug -> feature
status: unknown -> deferred
messages: + msg4023
nosy: droundy, tommy, beschmi, kowey, zooko
title: nothing happened (at first) -> compile with threaded runtime for more responsive progress reports
2008-03-26 15:43:54zookosetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4027
2008-03-26 16:58:51droundysetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4029
2008-03-26 17:15:32zookosetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4033
2008-03-26 17:31:18droundysetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4036
2008-03-26 18:01:06zookosetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4039
2008-03-27 18:28:28droundysetnosy: droundy, tommy, beschmi, kowey, zooko
messages: + msg4055
2008-05-07 15:25:41droundysetstatus: deferred -> unknown
nosy: + dagit
2008-05-15 16:03:12koweysetstatus: unknown -> testing
nosy: droundy, tommy, beschmi, kowey, zooko, dagit
2008-05-23 11:09:01koweysetstatus: testing -> resolved-in-unstable
nosy: droundy, tommy, beschmi, kowey, zooko, dagit
messages: + msg4847
2008-09-04 21:32:56adminsetstatus: resolved-in-unstable -> resolved
nosy: droundy, tommy, beschmi, kowey, zooko, dagit
2009-08-06 17:56:12adminsetnosy: + markstos, jast, Serware, dmitry.kurochkin, darcs-devel, mornfall, simon, thorkilnaur, - droundy
2009-08-06 21:00:12adminsetnosy: - beschmi
2009-08-10 22:17:37adminsetnosy: - markstos, darcs-devel, jast, Serware, mornfall
2009-08-11 00:08:27adminsetnosy: - dagit
2009-08-25 18:07:25adminsetnosy: + darcs-devel, - simon
2009-08-27 14:09:25adminsetnosy: tommy, kowey, darcs-devel, zooko, thorkilnaur, dmitry.kurochkin