[4.5.0.19] Forwarding email can insert extra `\r\n` in header data

Bug Reports
#1

REPRODUCIBILITY: 100%
OS VERSION: 4.5.0.19
HARDWARE: Sony Xperia 10 III
UI LANGUAGE: Finnish
REGRESSION: Yes

DESCRIPTION:

Forwarding an email doesn’t forward the full email as an attachment, but removes some data in the process. This used to work before, but I’m not sure at which release it got broken.

PRECONDITIONS:

  • Have an (IMAP) email account in your phone

STEPS TO REPRODUCE:

  • Receive spam
  • Forward the mail to SpamCop
  • Wait for response

EXPECTED RESULT:

  • SpamCop processes the email and you can send the reports

ACTUAL RESULT:

  • SpamCop tells me that the headers of the forwarded email are incomplete

MODIFICATIONS:

None

ADDITIONAL INFORMATION:

When I forward an email to SpamCop using an SFOS device, I get this reply when I enter the reporting page from the reply link:

This header is incomplete. Please supply the full headers of the spam you’re trying to report.
No source IP address found, cannot proceed.

When I forward the same email from Thunderbird or webmail, it gets processed properly.

Suggestion: Often times I can detect the spam from the title and/or preview text, so it would be helpful to have “Forward” option in the folder view long-press context menu, which would send the full, raw email as attachment.

If required, I have the headers as SpamCop reports them available and can provide them privately if needed.

3 Likes
Community meeting on IRC 17th August 2023
Community meeting on IRC 14th September 2023
#2

That part sounds like a feature; so that the recipient can actually read it.

#3

If it worked before it should be a regression. Are we sure it worked before?

#4

In any case, if the full headers are not supplied, it would automatically end up being rejected by my rspamd setup. But I’ll have to test it. You can always re-write the headers so that it is not apparent.

#5

Hmmm. The test I just performed provides an attachment (Content-Type: multipart/mixed;
boundary="[)}<qmf:1931167021EEzvaoGvnGp23FQbFe9p7Q==)}<]") with the complete, identical headers. So, maybe your mail server is in question?

#6

It’s one thing to reply with the text part only, and I understand stripping the headers too (as there can be a lot of headers!) but it defeats the use case of reporting soam, which explicitly relies on intact headers.

Currently I have K9 Mail for this purpose only, so this is not a blocker to me. And I can always use webmail, if all else fails!

I’m using the same server settings elsewhere, that shouldn’t be it.

I’m quite sure it used to work, at least I have finished several spam reports on my SFOS device. I’ll double check this with my 3.4 device tomorrow.

1 Like
#7

TL;DR I suspect Jolla Email re-parses the DKIM-Signature headers wrongly back together and inserts an extra newline which is not supposed to go there.

I forwarded the same spam mail to SpamCop from my Jolla* (SFOS 3.4) and X10III (SFOS 4.5) and they are identical. Then I forwarded the spam from Thunderbird, and the file is indeed different!

Difference 1: different folding of headers (I don’t think this is it, as RFCs say folding/unfolding is rather flexible, and parser shouldn’t distinguish these)

Thunderbird:

X-Rspam-Report: Action: add header
 Symbol: HAS_REPLYTO(0.00)
 Symbol: R_SPF_ALLOW(0.00)
 Symbol: REPLYTO_DN_EQ_FROM_DN(0.00)
 Symbol: TO_DN_NONE(0.00)
 Symbol: NEURAL_SPAM(0.00)
 Symbol: URI_COUNT_ODD(1.00)
 [...]

Jolla Email:

X-Rspam-Report: Action: add header Symbol: HAS_REPLYTO(0.00) Symbol:
 R_SPF_ALLOW(0.00) Symbol: REPLYTO_DN_EQ_FROM_DN(0.00) Symbol:
 TO_DN_NONE(0.00) Symbol: NEURAL_SPAM(0.00) Symbol: URI_COUNT_ODD(1.00)
 [...]

Difference 2: Different location (and content) of MIME-Version header (I don’t think this is it either):

Thunderbird: MIME-Version: 1.0 “in the middle” of headers
Jolla: Right before the first segment separator:

MIME-Version: 1.0

This is a multipart message in Mime 1.0 format

Difference 3: Extra newline in DKIM-signature (bingo!)

Thunderbird:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=mail; d=yyy.tld;
 h=Message-ID:Reply-To:From:To:Subject:Date:MIME-Version:Content-Type;
 i=aaa@yyy.tld;
 bh=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=;
 b=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
   bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
   bbbbbbbbbbbbbbbbbbb=
Message-ID: <xxx@yyy.tld>

Jolla Email:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=mail;
 d=yyy.tld;
 h=Message-ID:Reply-To:From:To:Subject:Date:MIME-Version:Content-Type;
 i=aaa@yyy.tld; bh=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=;
 b=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb   bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb   bbbbbbbbbbbbbbbbbbb=
 
Message-ID: <xxx@yyy.tld>

Also, the folding is different for the b= parameter, I can’t tell if it’s incorrect or not (my guess is…yes?)

So I was wrong in two ways:

  • the headers are not stripped
  • this was not a regression

That also explains how it may have worked before; if the DKIM header is the cause here, and it wasn’t present, then the email would be parsed correctly. That’s my best educated guess, at least.

I would edit the opening post, but I can’t do that any more. I’ll change the title though.

Could someone change the opening post to a wiki so I could update it?

*) It turns out that SFOS 3.4 can’t connect to my email service provider without enabling self-signed certs, but that’s expected since the OS has received no certificate nor OpenSSL updates in a long while.

3 Likes
#8

This is great. Your analysis really pushed this forward! The last remaining question is, if the header parser is to strict or jolla hurting the spec. This should be easy to figure out. (in any case it might be more sensible if sfos just behaves like thunderbird)

1 Like
#9

Interesting, I’ll test this in QMF. But one question before: QMF is creating a header with a long line (b=…) like it is shown in your message or is it just \n\r that are represented by space in the formatting ?

#10

I’m not sure I understand the relationship between the DKIM fields and the initial message from spamcop:

No source IP address found, cannot proceed.

I did some tests where ALL original headers were stripped, which I assume has to do with the content (the attaching of the original message as BASE64 attached). The more middle-men, the less faith I have. However, I’d always expect to see:

Received: from mout.telta.de (mout.telta.de [80.78.160.55])
	by mail.ffba.info (Postfix) with ESMTPS id 1235C5XXXXXX
	for <blueprint@poetaster.de>; Mon,  4 Sep 2023 14:34:11 +0200 (CEST)

...
Received: from srv-mail1.svsrb.local ([fe80::ef7d:41d1:c10f:e829]) by
 srv-mail1.svsrb.local ([fe80::ef7d:41d1:c10f:e829%4]) with mapi id
 15.02.0986.042; Mon, 4 Sep 2023 14:34:10 +0200

Or the like.

#11

AFAICT Jolla Email inserts an extra \r\n after parsing the DKIM header, which signals “end of headers” to SpamCop parser. In all cases I have tested, DKIM is quite early on in the headers. The rest of the headers are there, but seemingly the parser has stopped there, losing most of the headers.

Hence the No source IP address found, cannot proceed message.

1 Like
#12

This does not sound plausible since the first Received: preceeds DKIM signature in all cases I tested. I also can’t confirm that extra line feeds are inserted, given that in numerous tests none of the original headers are there. For instance I just forwarded a letsencrypt automated mail with a very large number of headers and none of those were forwarded. Just the contents. This sounds like the very opposite of what you are seeing!

It’s a bit of a tricky call what the client should do. If you consider the ‘audit trail’ question, you want the headers. On the other hand, after passing through a veritable zoo of proxies and gateways (microsoft exchange, anyone) you can’t trust the headers anyway. What I’m seeing suggests the headers are generally discarded. Or maybe I’ve mangled the jolla mail client (possible, @nephros has a plain text hack that I had installed on some devices, but not on this one, I think.)

But, that doesn’t mean the headers aren’t mangled under some circumstances. I haven’t used Spamcop in many years (for diverse reasons) so I have no recent experience with reporting.

If you’re up for it, PM me a mail address for testing and I’ll do some testing from my mail setup to you / vice versa.

#13

Just testing with another convoluted mail and I see that the References are all there and the message IDs are all correct (so that I can retrieve the entire back and forth of all the messages) but the headers of the forwarded mails are otherwise gone. Well, that’s interesting.

#14

Email Plain Text View only displays plain text content (if available). The view does not allow to reply, forward etc. any messages. You have to do that from the usual places.
Data/content is not altered.

1 Like
#15

Thanks for clarifying! I doubted it had anything to do with it, and it appears that patch is not active on the phone I was doing these tests with.

#16

Thanks for the report, however I was trying to reproduce this and as a reference I forwarded one spam message with laptop/gmail to spamcop and that didn’t work either. Just trying to figure out how severe issue this is.