[BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Janko Mivšek
Hi guys,

When trying to run Aida on Pharo 1.2 from Hudson build [1] on Windows
XP, I got this error immediatelly after start, when Pharo tries to
initialize all podsystems and in SecurityManager tries to initialize its
directory, which in my machine looks like:

  'D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek'

As you see it takes my Win account name (Janko Mivšek) and it seems
incorectly translate to the filename.

Problem is that image cannot be saved at all on that Windoze.

Reported as issue http://code.google.com/p/pharo/issues/detail?id=3721

[1]
https://pharo-ic.lille.inria.fr/hudson/job/Aida%20Pharo%201.2/lastSuccessfulBuild/artifact/AidaOneClickPharo1.2.zip

Best regards
Janko

16 February 2011 9:53:41 pm

VM: Win32 - IX86 - 5.1 - Croquet Closure Cog VM [CoInterpreter
VMMaker-oscog.35]
Image: Pharo1.2rc2 [Latest update: #12335]

SecurityManager state:
Restricted: false
FileAccess: true
SocketAccess: true
Working Dir D:\AidaOneClickPharo1.2.app\Contents\Resources
UTF8TextConverter(Object)>>error:
        Receiver: an UTF8TextConverter
        Arguments and temporary variables:
                aString: ''Invalid utf8 input detected''
        Receiver''s instance variables:
                latin1Map: #[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 ...etc...
                latin1Encodings: #(nil nil nil nil nil nil nil nil nil nil nil nil
nil nil nil ...etc...

UTF8TextConverter>>errorMalformedInput
        Receiver: an UTF8TextConverter
        Arguments and temporary variables:

        Receiver''s instance variables:
                latin1Map: #[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 ...etc...
                latin1Encodings: #(nil nil nil nil nil nil nil nil nil nil nil nil
nil nil nil ...etc...

UTF8TextConverter>>nextFromStream:
        Receiver: an UTF8TextConverter
        Arguments and temporary variables:
                aStream: a ReadStream
''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Miv...etc...
                character1: $š
                value1: 154
                character2: $e
                value2: 101
                unicode: nil
                character3: $k
                value3: 107
                character4: nil
                value4: nil
        Receiver''s instance variables:
                latin1Map: #[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 ...etc...
                latin1Encodings: #(nil nil nil nil nil nil nil nil nil nil nil nil
nil nil nil ...etc...

ByteString(String)>>convertFromWithConverter:
        Receiver: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
        Arguments and temporary variables:
                converter: an UTF8TextConverter
                readStream: a ReadStream
''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko ...etc...
                writeStream: a WriteStream
''D:\AidaOneClickPharo1.2.app\Contents\Resources\Jank...etc...
                c: $v
        Receiver''s instance variables:
''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''

FilePath>>pathName:isEncoded:
        Receiver: <<error during printing>>
        Arguments and temporary variables:
                p: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
                isEncoded: true
        Receiver''s instance variables:
                squeakPathName: nil
                vmPathName: nil
                converter: an UTF8TextConverter

FilePath class>>pathName:isEncoded:
        Receiver: FilePath
        Arguments and temporary variables:
                pathName: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
                aBoolean: true
        Receiver''s instance variables:
                superclass: Object
                methodDict: a
MethodDictionary(#asSqueakPathName->(FilePath>>#asSqueakPathName ...etc...
                format: 136
                instanceVariables: #(''squeakPathName'' ''vmPathName'' ''converter'')
                organization: (''conversion'' asSqueakPathName asString asVmPathName
coverter: pa...etc...
                subclasses: nil
                name: #FilePath
                classPool: nil
                sharedPools: nil
                environment: a SystemDictionary(lots of globals)
                category: #''Files-Directories''
                traitComposition: nil
                localSelectors: nil

SecurityManager>>secureUserDirectory
        Receiver: a SecurityManager
        Arguments and temporary variables:
                dir: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
        Receiver''s instance variables:
                privateKeyPair: nil
                trustedKeys: #()
                keysFileName: ''Pharo.keys''
                securityChecksEnabled: false
                showSecurityStatus: false
                startInUntrustedDirectory: false
                warnAboutInsecureContent: nil
                automaticKeyGeneration: false

SecurityManager>>loadSecurityKeys
        Receiver: a SecurityManager
        Arguments and temporary variables:
                fd: nil
                loc: nil
                file: nil
                keys: nil
        Receiver''s instance variables:
                privateKeyPair: nil
                trustedKeys: #()
                keysFileName: ''Pharo.keys''
                securityChecksEnabled: false
                showSecurityStatus: false
                startInUntrustedDirectory: false
                warnAboutInsecureContent: nil
                automaticKeyGeneration: false

SecurityManager>>startUp
        Receiver: a SecurityManager
        Arguments and temporary variables:

        Receiver''s instance variables:
                privateKeyPair: nil
                trustedKeys: #()
                keysFileName: ''Pharo.keys''
                securityChecksEnabled: false
                showSecurityStatus: false
                startInUntrustedDirectory: false
                warnAboutInsecureContent: nil
                automaticKeyGeneration: false

SecurityManager class>>startUp
        Receiver: SecurityManager
        Arguments and temporary variables:

        Receiver''s instance variables:
                superclass: Object
                methodDict: a
MethodDictionary(#addTrustedKey:->(SecurityManager>>#addTrustedKe...etc...
                format: 146
                instanceVariables: #(''privateKeyPair'' ''trustedKeys''
''keysFileName'' ''securityChe...etc...
                organization: (''accessing'' addTrustedKey: keysFileName
keysFileName: primSecure...etc...
                subclasses: ni...etc...

--
Janko Mivšek
Svetovalec za informatiko
Eranova d.o.o.
Ljubljana, Slovenija
www.eranova.si
tel:  01 514 22 55
faks: 01 514 22 56
gsm: 031 674 565

Reply | Threaded
Open this post in threaded view
|

Re: [BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Stéphane Ducasse
thanks janko for the report.
I remember seeing passing a problem with filename encoding but I forgot.
Can you open a ticket?

Stef

> Hi guys,
>
> When trying to run Aida on Pharo 1.2 from Hudson build [1] on Windows
> XP, I got this error immediatelly after start, when Pharo tries to
> initialize all podsystems and in SecurityManager tries to initialize its
> directory, which in my machine looks like:
>
>  'D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek'
>
> As you see it takes my Win account name (Janko Mivšek) and it seems
> incorectly translate to the filename.
>
> Problem is that image cannot be saved at all on that Windoze.
>
> Reported as issue http://code.google.com/p/pharo/issues/detail?id=3721
>
> [1]
> https://pharo-ic.lille.inria.fr/hudson/job/Aida%20Pharo%201.2/lastSuccessfulBuild/artifact/AidaOneClickPharo1.2.zip
>
> Best regards
> Janko
>
> 16 February 2011 9:53:41 pm
>
> VM: Win32 - IX86 - 5.1 - Croquet Closure Cog VM [CoInterpreter
> VMMaker-oscog.35]
> Image: Pharo1.2rc2 [Latest update: #12335]
>
> SecurityManager state:
> Restricted: false
> FileAccess: true
> SocketAccess: true
> Working Dir D:\AidaOneClickPharo1.2.app\Contents\Resources
> UTF8TextConverter(Object)>>error:
> Receiver: an UTF8TextConverter
> Arguments and temporary variables:
> aString: ''Invalid utf8 input detected''
> Receiver''s instance variables:
> latin1Map: #[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 ...etc...
> latin1Encodings: #(nil nil nil nil nil nil nil nil nil nil nil nil
> nil nil nil ...etc...
>
> UTF8TextConverter>>errorMalformedInput
> Receiver: an UTF8TextConverter
> Arguments and temporary variables:
>
> Receiver''s instance variables:
> latin1Map: #[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 ...etc...
> latin1Encodings: #(nil nil nil nil nil nil nil nil nil nil nil nil
> nil nil nil ...etc...
>
> UTF8TextConverter>>nextFromStream:
> Receiver: an UTF8TextConverter
> Arguments and temporary variables:
> aStream: a ReadStream
> ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Miv...etc...
> character1: $š
> value1: 154
> character2: $e
> value2: 101
> unicode: nil
> character3: $k
> value3: 107
> character4: nil
> value4: nil
> Receiver''s instance variables:
> latin1Map: #[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 ...etc...
> latin1Encodings: #(nil nil nil nil nil nil nil nil nil nil nil nil
> nil nil nil ...etc...
>
> ByteString(String)>>convertFromWithConverter:
> Receiver: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
> Arguments and temporary variables:
> converter: an UTF8TextConverter
> readStream: a ReadStream
> ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko ...etc...
> writeStream: a WriteStream
> ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Jank...etc...
> c: $v
> Receiver''s instance variables:
> ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
>
> FilePath>>pathName:isEncoded:
> Receiver: <<error during printing>>
> Arguments and temporary variables:
> p: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
> isEncoded: true
> Receiver''s instance variables:
> squeakPathName: nil
> vmPathName: nil
> converter: an UTF8TextConverter
>
> FilePath class>>pathName:isEncoded:
> Receiver: FilePath
> Arguments and temporary variables:
> pathName: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
> aBoolean: true
> Receiver''s instance variables:
> superclass: Object
> methodDict: a
> MethodDictionary(#asSqueakPathName->(FilePath>>#asSqueakPathName ...etc...
> format: 136
> instanceVariables: #(''squeakPathName'' ''vmPathName'' ''converter'')
> organization: (''conversion'' asSqueakPathName asString asVmPathName
> coverter: pa...etc...
> subclasses: nil
> name: #FilePath
> classPool: nil
> sharedPools: nil
> environment: a SystemDictionary(lots of globals)
> category: #''Files-Directories''
> traitComposition: nil
> localSelectors: nil
>
> SecurityManager>>secureUserDirectory
> Receiver: a SecurityManager
> Arguments and temporary variables:
> dir: ''D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek''
> Receiver''s instance variables:
> privateKeyPair: nil
> trustedKeys: #()
> keysFileName: ''Pharo.keys''
> securityChecksEnabled: false
> showSecurityStatus: false
> startInUntrustedDirectory: false
> warnAboutInsecureContent: nil
> automaticKeyGeneration: false
>
> SecurityManager>>loadSecurityKeys
> Receiver: a SecurityManager
> Arguments and temporary variables:
> fd: nil
> loc: nil
> file: nil
> keys: nil
> Receiver''s instance variables:
> privateKeyPair: nil
> trustedKeys: #()
> keysFileName: ''Pharo.keys''
> securityChecksEnabled: false
> showSecurityStatus: false
> startInUntrustedDirectory: false
> warnAboutInsecureContent: nil
> automaticKeyGeneration: false
>
> SecurityManager>>startUp
> Receiver: a SecurityManager
> Arguments and temporary variables:
>
> Receiver''s instance variables:
> privateKeyPair: nil
> trustedKeys: #()
> keysFileName: ''Pharo.keys''
> securityChecksEnabled: false
> showSecurityStatus: false
> startInUntrustedDirectory: false
> warnAboutInsecureContent: nil
> automaticKeyGeneration: false
>
> SecurityManager class>>startUp
> Receiver: SecurityManager
> Arguments and temporary variables:
>
> Receiver''s instance variables:
> superclass: Object
> methodDict: a
> MethodDictionary(#addTrustedKey:->(SecurityManager>>#addTrustedKe...etc...
> format: 146
> instanceVariables: #(''privateKeyPair'' ''trustedKeys''
> ''keysFileName'' ''securityChe...etc...
> organization: (''accessing'' addTrustedKey: keysFileName
> keysFileName: primSecure...etc...
> subclasses: ni...etc...
>
> --
> Janko Mivšek
> Svetovalec za informatiko
> Eranova d.o.o.
> Ljubljana, Slovenija
> www.eranova.si
> tel:  01 514 22 55
> faks: 01 514 22 56
> gsm: 031 674 565
>


Reply | Threaded
Open this post in threaded view
|

Re: [BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Janko Mivšek
On 16. 02. 2011 23:02, Stéphane Ducasse wrote:
> thanks janko for the report.
> I remember seeing passing a problem with filename encoding but I forgot.
> Can you open a ticket?

Already done: http://code.google.com/p/pharo/issues/detail?id=3721

Janko

>> When trying to run Aida on Pharo 1.2 from Hudson build [1] on Windows
>> XP, I got this error immediatelly after start, when Pharo tries to
>> initialize all podsystems and in SecurityManager tries to initialize its
>> directory, which in my machine looks like:
>>
>>  'D:\AidaOneClickPharo1.2.app\Contents\Resources\Janko Mivšek'
>>
>> As you see it takes my Win account name (Janko Mivšek) and it seems
>> incorectly translate to the filename.
>>
>> Problem is that image cannot be saved at all on that Windoze.
>>
>> Reported as issue http://code.google.com/p/pharo/issues/detail?id=3721
>>
>> [1]
>> https://pharo-ic.lille.inria.fr/hudson/job/Aida%20Pharo%201.2/lastSuccessfulBuild/artifact/AidaOneClickPharo1.2.zip
>>
>> Best regards
>> Janko

--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si

Reply | Threaded
Open this post in threaded view
|

Re: [BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Stéphane Ducasse
**tx**
>
> Can you open a ticket?
>
> Already done: http://code.google.com/p/pharo/issues/detail?id=3721
>
> Janko

Igor could you have a look?

Stef

Reply | Threaded
Open this post in threaded view
|

Re: [BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Igor Stasenko
On 17 February 2011 18:20, Stéphane Ducasse <[hidden email]> wrote:

> **tx**
>>
>> Can you open a ticket?
>>
>> Already done: http://code.google.com/p/pharo/issues/detail?id=3721
>>
>> Janko
>
> Igor could you have a look?
>
yes.

> Stef
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Henrik Sperre Johansen
On 17.02.2011 22:44, Igor Stasenko wrote:

> On 17 February 2011 18:20, Stéphane Ducasse<[hidden email]>  wrote:
>> **tx**
>>> Can you open a ticket?
>>>
>>> Already done: http://code.google.com/p/pharo/issues/detail?id=3721
>>>
>>> Janko
>> Igor could you have a look?
>>
> yes.
I've been looking into it, it's the SecurityPlugin sources used by Cog
not doing conversion from system codepage to utf8.
The one in standard vm does it, but it's still suboptimal compared to
using GetUserNameW.
Hopefully find time to finish up and post proposal code to vmdev tomorrow.

Cheers,
Henry.

Reply | Threaded
Open this post in threaded view
|

Re: [BUG] 1.2 UTF8TextConverter: Invalid utf8 input detected

Stéphane Ducasse
thanks a lot henrik.

Stef

On Feb 18, 2011, at 12:11 AM, Henrik Sperre Johansen wrote:

> On 17.02.2011 22:44, Igor Stasenko wrote:
>> On 17 February 2011 18:20, Stéphane Ducasse<[hidden email]>  wrote:
>>> **tx**
>>>> Can you open a ticket?
>>>>
>>>> Already done: http://code.google.com/p/pharo/issues/detail?id=3721
>>>>
>>>> Janko
>>> Igor could you have a look?
>>>
>> yes.
> I've been looking into it, it's the SecurityPlugin sources used by Cog not doing conversion from system codepage to utf8.
> The one in standard vm does it, but it's still suboptimal compared to using GetUserNameW.
> Hopefully find time to finish up and post proposal code to vmdev tomorrow.
>
> Cheers,
> Henry.
>