See msg8396

Now we need to test (A) if this affects the web interface and (B) if it affects
the email submissions now that we've cut the content stripping

Testing the web interface

From FOO BAR
From: FOO BAR

Expecting "From FOO BAR" and "From: FOO BAR"

On Sat, Oct 31, 2009 at 08:42:59 +0000, Eric Kow wrote:
> Testing the web interface
> 
> From FOO BAR
> From: FOO BAR
> 
> Expecting "From FOO BAR" and "From: FOO BAR"

Seems to work.

And now testing the email interface:

From QUUX POP
From: QUUX POP

Expecting "From QUUX POP" and "From: QUUX POP"

From: FOO Bar

what if the message starts with such a leading line? From: FOO Bar (web)

From: QUUX PLOP

One more test using "From: QUUX PLOP" (via email)

>From QUUX PLOP

Sorry, just one more test using "From QUUX PLOP" (via email)

Look, how interesting! (msg9128).  It looks like roundup quotes the line with a
leading From.  

I bet that's why it was getting eaten earlier (the subsequent email content filter).

Looks like we'll need to submit a nice bug report to roundup.

Excerpts from Eric Kow's message of Sat Oct 31 18:53:18 +0100 2009:
> 
> Eric Kow <kowey@darcs.net> added the comment:
> 
> Look, how interesting! (msg9128).  It looks like roundup quotes the line with a
> leading From.  
> 
> I bet that's why it was getting eaten earlier (the subsequent email content filter).
> 
> Looks like we'll need to submit a nice bug report to roundup.

Extra quoting is not dangerous, what is the real problem?

-- 
Nicolas Pouillard
http://nicolaspouillard.fr

On Sat, Oct 31, 2009 at 21:45:56 +0100, Nicolas Pouillard wrote:
> > I bet that's why it was getting eaten earlier (the subsequent email content filter).
> > 
> > Looks like we'll need to submit a nice bug report to roundup.
> 
> Extra quoting is not dangerous, what is the real problem?

The extra quoting in itself is fine.

What I meant is that in issue1596, we were having problems where roundup
would automatically trim content that appears to excess quoting, or
signatures etc.  This was causing cases where bits of messages submitted
via the email interface were getting eaten.

So I think that *that* mechanism clashed with the roundup mechanism to
deliberately add extra quoting on the 'From ' line.  Ie. (1) roundup
inserts extra quoting (2) roundup gets confused and removes the line.

Since we've configured (2) away, this should no longer be a problem, but
the roundup people should know about it.

(actually, to be thorough, we should temporarily configure roundup to
trim content again and then perform the test a second time)

Excerpts from Eric Kow's message of Sat Oct 31 21:59:13 +0100 2009:
> 
> Eric Kow <kowey@darcs.net> added the comment:
> 
> On Sat, Oct 31, 2009 at 21:45:56 +0100, Nicolas Pouillard wrote:
> > > I bet that's why it was getting eaten earlier (the subsequent email content filter).
> > > 
> > > Looks like we'll need to submit a nice bug report to roundup.
> > 
> > Extra quoting is not dangerous, what is the real problem?
> 
> The extra quoting in itself is fine.
> 
> What I meant is that in issue1596, we were having problems where roundup
> would automatically trim content that appears to excess quoting, or
> signatures etc.  This was causing cases where bits of messages submitted
> via the email interface were getting eaten.
> 
> So I think that *that* mechanism clashed with the roundup mechanism to
> deliberately add extra quoting on the 'From ' line.  Ie. (1) roundup
> inserts extra quoting (2) roundup gets confused and removes the line.
> 
> Since we've configured (2) away, this should no longer be a problem, but
> the roundup people should know about it.
> 
> (actually, to be thorough, we should temporarily configure roundup to
> trim content again and then perform the test a second time)

OK, so for me the trimmer is the buggy one.

-- 
Nicolas Pouillard
http://nicolaspouillard.fr

I don't think Roundup itself will do From-stuffing (it might if it uses the Python 
email package to manipulate the message).  More likely it is an intermediate MTA.  
The point is that you probably can't fix the From stuffing completely; you'll have 
to make sure the quote stripper doesn't strip that.

Right: we don't want to fix the quote-stuffing.  I'm just saying I should let
the roundup people know about this interaction.

Which I have done here:
http://issues.roundup-tracker.org/issue2550597

So I think the fact that we've disabled email trimming due to issue1596 and that
we are unlikely to follow up on the roundup issue (at least, as far as the darcs
tracker is concerned), we can safely mark this resolved.

>From the first

Testing the behaviour... there should be a "From" line above me

>From here

Testing some more... there should be a "From" line above me

(The roundup guys cannot reproduce the behaviour.  Here is their test)

-- 
Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
PGP Key ID: 08AC04F9

>From foo:
>From bar:
 From quux:
ZFrom quuux:
This is the last line of message body.

The same message, via c.c.a instead of g.c.

>From foo:
>From bar:
 From quux:
ZFrom quuux:
This is the last line of message body.

Eric Kow wrote:
> (The roundup guys cannot reproduce the behaviour.  Here is their test)

You're right.  I can't reproduce this now.  My test was:

    $ msmtp -a cybersource -oi -t -d <<EOF
    To: bugs@darcs.net
    From: trentbuck@gmail.com
    Subject: [issue1667]

    The same message, via c.c.a instead of g.c.

    From foo:
    >From bar:
     From quux:
    ZFrom quuux:
    This is the last line of message body.
    EOF

My msmtprc was http://twb.ath.cx/Preferences/.msmtprc.

I wonder if the behaviour was differegnt before when you (Eric) were
fiddling with the pre/post-processing options?

On Tue, Feb 02, 2010 at 09:00:16 +0000, Trent W. Buck wrote:
> I wonder if the behaviour was differegnt before when you (Eric) were
> fiddling with the pre/post-processing options?

I belatedly realised that my test was sort of silly because we have
already disabled the email trimming on our bugtracker.

I guess we'd need to turn it back on temporarily and then repeat.

-- 
Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
PGP Key ID: 08AC04F9