Issue 6143 in pharo: MIME decoding and Unicode not working

Status: New
Owner: ----
Labels: Milestone-1.4 Type-Bug

New issue 6143 by [hidden email]: MIME decoding and Unicode not working
http://code.google.com/p/pharo/issues/detail?id=6143

This sample test for decoding MIME-headers fails
       
        | mimeHeader expected multiStream characters|
        characters := #[16rBE 16rFD 16r5F 16rE1 16r2E 16rC8].
        mimeHeader := '=?ISO-8859-2?Q?=BE=FD=5F=E1=2E=C8?=' decodeMimeHeader.

        multiStream:=MultiByteBinaryOrTextStream
                on: String new
                encoding: 'ISO-8859-2'.
        multiStream nextPutAll: characters.

        expected := multiStream reset contents.
       
        self assert: mimeHeader = expected

Fix attached, including 2 tests

Attachments:
        FIX-MIME-Unicode.cs  1.8 KB


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Updates:
        Status: Workneeded

Comment #1 on issue 6143 by [hidden email]: MIME decoding and Unicode not  
working
http://code.google.com/p/pharo/issues/detail?id=6143

Switching nextPut:toStream to do
converter encoding -> internal instead of
internal -> converter encoding  is not a good idea.

The escaped characters in the header are not Unicode values that should be  
converted to ISO-8859-2 to get the internal string representation, it's the  
other way around.

That is what decodeMimeHeader does though.
Instead, it should first do mime-decoding of escaped characters to end up  
with an ISO-8859-2 string, THEN use the converter to convert said string  
from 8859-2 to Unicode.

The multistream contents it compares against are fatally flawed as well, as  
nextPut:'ing integer values rather than character does no conversion as  
all, so you basically end up with the (latin1) version of the characters  
array, and might as well have written asString.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Updates:
        Status: FixReviewNeeded
        Labels: Milestone-2.0

Comment #2 on issue 6143 by [hidden email]: MIME decoding and Unicode not  
working
http://code.google.com/p/pharo/issues/detail?id=6143

Here's a .cs with fixed decodeMimeHeader, and some tests, as loadable .cs  
since it's a bug that applies to both 1.4 and 2.0.

Attachments:
        mimeHeaderDecoder.1.cs  4.4 KB


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Updates:
        Status: Integrated

Comment #3 on issue 6143 by [hidden email]: MIME decoding and  
Unicode not working
http://code.google.com/p/pharo/issues/detail?id=6143

in 2.0 166


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Updates:
        Status: FixToInclude
        Labels: -Milestone-2.0

Comment #4 on issue 6143 by [hidden email]: MIME decoding and  
Unicode not working
http://code.google.com/p/pharo/issues/detail?id=6143

TODO: 1.4


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Updates:
        Status: Integrated

Comment #5 on issue 6143 by [hidden email]: MIME decoding and  
Unicode not working
http://code.google.com/p/pharo/issues/detail?id=6143

in 1.4 446


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker