Smalltalk › Gemtalk › GLASS

[Glass] MultibyteString>>encodeUsing: bug

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

Johan Brichau-2

[Glass] MultibyteString>>encodeUsing: bug

Hi,

I stumbled upon a situation where the code in MultiByteString>>encodeUsing: is being executed (the primitive has failed?).

This code does a "MultibyteString new" but that is a subclass responsibility:

WriteStream on: MultibyteString new.

I assume the code should be:

WriteStream on: self species new.

A couple of questions:
- it seems I need to login as SystemUser to recompile this method. I get an error only a SystemUser can change a primitive ? Anything specific to watch out for when doing that?
- is there a way to know why the primitive failed?

regards
Johan
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Dale Henrichs-3

Re: [Glass] MultibyteString>>encodeUsing: bug

Johan,

Could you share some more details about the bug?

GemStone will automatically coerce a `String new` into the appropriate *String class when a multi-byte character is added to the String, so presumably `String new` shouldn't be a problem.

I don't know where the `MutiByteString new` is being executed (perhaps not necessary?)

Regarding primitive failures, there is typically failure code written in Smalltalk to throw an explicit error upon primitive failure, but that code is clearly missing in the #encodeUsing: method ... From the comment:

If the ascii value is >= size of the encodingArray, nil is returned.

If the value from the encodingArray is a Character with value > 16rFF

or is not a kind of String, nil is returned.

So there should some Smalltalk code checking for those conditions ... also it looks like MutiByteStrings will be problematic since they would involve characters > 16rFF, so I will have to check into that ... If MultiByteStrings are supported by the primitive, then perhaps moving the primitive call to the concrete subclasses of MultiByteString will do the trick ...

Finally, for security reasons compilation of methods with primitives must be done by SystemUser. It is possible to give a user (like Datacurator) the CompilePrimitives privilege to allow primitive compilation. Something like the following:

GsCurrentSession currentSession userProfile

addPrivilege: #'CompilePrimitives';

yourself.

System commitTransaction.

I'll look into the details of the primitive and let you know if I find anything new...

Dale

On Mon, Apr 21, 2014 at 2:29 AM, Johan Brichau <[hidden email]> wrote:

Hi,

I stumbled upon a situation where the code in MultiByteString>>encodeUsing: is being executed (the primitive has failed?).

This code does a "MultibyteString new" but that is a subclass responsibility:

WriteStream on: MultibyteString new.

I assume the code should be:

WriteStream on: self species new.

A couple of questions:
- it seems I need to login as SystemUser to recompile this method. I get an error only a SystemUser can change a primitive ? Anything specific to watch out for when doing that?
- is there a way to know why the primitive failed?

regards
Johan
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Dale Henrichs-3

Re: [Glass] MultibyteString>>encodeUsing: bug

Johan,

I looked into the primitive call and there are a number of possible failures:

1. lookup character value beyond end of array

2. the character in the table is larger than will fit in receiver (automatic coercion

not supported in this primitive)

3. non-character in the table

Presumably the failure code should implement the encodeUsing: in Smalltalk so that we can get a more meaningful error (or take advantage of automatic coercion)...

I've submitted a bug against 3.2.

Still curious about your failure mode, tho...

Dale

On Mon, Apr 21, 2014 at 12:39 PM, Dale Henrichs <[hidden email]> wrote:

Johan,

Could you share some more details about the bug?

GemStone will automatically coerce a `String new` into the appropriate *String class when a multi-byte character is added to the String, so presumably `String new` shouldn't be a problem.

I don't know where the `MutiByteString new` is being executed (perhaps not necessary?)

Regarding primitive failures, there is typically failure code written in Smalltalk to throw an explicit error upon primitive failure, but that code is clearly missing in the #encodeUsing: method ... From the comment:

If the ascii value is >= size of the encodingArray, nil is returned.
If the value from the encodingArray is a Character with value > 16rFF
or is not a kind of String, nil is returned.

So there should some Smalltalk code checking for those conditions ... also it looks like MutiByteStrings will be problematic since they would involve characters > 16rFF, so I will have to check into that ... If MultiByteStrings are supported by the primitive, then perhaps moving the primitive call to the concrete subclasses of MultiByteString will do the trick ...

Finally, for security reasons compilation of methods with primitives must be done by SystemUser. It is possible to give a user (like Datacurator) the CompilePrimitives privilege to allow primitive compilation. Something like the following:

GsCurrentSession currentSession userProfile
addPrivilege: #'CompilePrimitives';
yourself.
System commitTransaction.

I'll look into the details of the primitive and let you know if I find anything new...

Dale

On Mon, Apr 21, 2014 at 2:29 AM, Johan Brichau <[hidden email]> wrote:

Hi,

I stumbled upon a situation where the code in MultiByteString>>encodeUsing: is being executed (the primitive has failed?).

This code does a "MultibyteString new" but that is a subclass responsibility:

WriteStream on: MultibyteString new.

I assume the code should be:

WriteStream on: self species new.

A couple of questions:
- it seems I need to login as SystemUser to recompile this method. I get an error only a SystemUser can change a primitive ? Anything specific to watch out for when doing that?
- is there a way to know why the primitive failed?

regards
Johan
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Johan Brichau-2

Re: [Glass] MultibyteString>>encodeUsing: bug

In reply to this post by Dale Henrichs-3

Hi Dale,

The issue is that the String has characters with codePoint > 65356 but the table that gets passed as argument (Seaside xmlEncoder table) does not include anything beyond that code point.

So, that is the primary cause of the failure, but the fallback code for the primitive would best be fixed.

I wanted to check if the 'String new' rather than 'MultibyteString new' works as say by changing the method code, but I do have trouble trying to login as SystemUser using Gemtools, or setting the #CompilePrimitives privilege. Trying to set the user to SystemUser in Gemtools results in a compile error when logging in and setting the privilege for DataCurator says that it's not a valid privilege (see screenshots attached).

The error occurred in a 2.4 stone. Maybe I should move to a 3.1 stone to do those things?

On 21 Apr 2014, at 21:39, Dale Henrichs <[hidden email]> wrote:

Johan,

Could you share some more details about the bug?

GemStone will automatically coerce a `String new` into the appropriate *String class when a multi-byte character is added to the String, so presumably `String new` shouldn't be a problem.

I don't know where the `MutiByteString new` is being executed (perhaps not necessary?)

Regarding primitive failures, there is typically failure code written in Smalltalk to throw an explicit error upon primitive failure, but that code is clearly missing in the #encodeUsing: method ... From the comment:

If the ascii value is >= size of the encodingArray, nil is returned.
If the value from the encodingArray is a Character with value > 16rFF
or is not a kind of String, nil is returned.

So there should some Smalltalk code checking for those conditions ... also it looks like MutiByteStrings will be problematic since they would involve characters > 16rFF, so I will have to check into that ... If MultiByteStrings are supported by the primitive, then perhaps moving the primitive call to the concrete subclasses of MultiByteString will do the trick ...

Finally, for security reasons compilation of methods with primitives must be done by SystemUser. It is possible to give a user (like Datacurator) the CompilePrimitives privilege to allow primitive compilation. Something like the following:

GsCurrentSession currentSession userProfile
addPrivilege: #'CompilePrimitives';
yourself.
System commitTransaction.

I'll look into the details of the primitive and let you know if I find anything new...

Dale

On Mon, Apr 21, 2014 at 2:29 AM, Johan Brichau <[hidden email]> wrote:
Hi,

I stumbled upon a situation where the code in MultiByteString>>encodeUsing: is being executed (the primitive has failed?).

This code does a "MultibyteString new" but that is a subclass responsibility:

WriteStream on: MultibyteString new.

I assume the code should be:

WriteStream on: self species new.

A couple of questions:
- it seems I need to login as SystemUser to recompile this method. I get an error only a SystemUser can change a primitive ? Anything specific to watch out for when doing that?
- is there a way to know why the primitive failed?

regards
Johan
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass