The Trunk: Multilingual-jr.218.mcz

Patrick Rein uploaded a new version of Multilingual to project The Trunk:
http://source.squeak.org/trunk/Multilingual-jr.218.mcz

==================== Summary ====================

Name: Multilingual-jr.218
Author: jr
Time: 19 January 2017, 5:14:23.763655 pm
UUID: 36416c42-a4b4-554f-8203-aba25eee794f
Ancestors: Multilingual-tfel.217

support 'iso-8859-1' and do not let UTF8TextConverter expect that its input stream  returns Characters from basicNext

A stream implementation might always return bytes from basicNext and expect the conversion to Character to be done solely by the TextConverter, so use asInteger instead of asciiValue to support both cases. Convert back with asCharacter.

=============== Diff against Multilingual-tfel.217 ===============

Item was changed:
  ----- Method: Latin1TextConverter class>>encodingNames (in category 'utilities') -----
  encodingNames
 
+ ^ #('latin-1' 'latin1' 'iso-8859-1') copy.
- ^ #('latin-1' 'latin1') copy.
  !

Item was changed:
  ----- Method: UTF8TextConverter>>nextFromStream: (in category 'conversion') -----
  nextFromStream: aStream
 
  | char1 value1 char2 value2 unicode char3 value3 char4 value4 |
  aStream isBinary ifTrue: [^ aStream basicNext].
  char1 := aStream basicNext.
  char1 ifNil:[^ nil].
+ value1 := char1 asInteger.
- value1 := char1 asciiValue.
  value1 <= 127 ifTrue: [
  "1-byte char"
+ ^ char1 asCharacter
- ^ char1
  ].
 
  "at least 2-byte char"
  char2 := aStream basicNext.
+ char2 ifNil:[^self errorMalformedInput: (String with: char1 asCharacter)].
+ value2 := char2 asInteger.
- char2 ifNil:[^self errorMalformedInput: (String with: char1)].
- value2 := char2 asciiValue.
 
  (value1 bitAnd: 16rE0) = 192 ifTrue: [
  ^ Unicode value: ((value1 bitAnd: 31) bitShift: 6) + (value2 bitAnd: 63).
  ].
 
  "at least 3-byte char"
  char3 := aStream basicNext.
+ char3 ifNil:[^self errorMalformedInput: (String with: char1 asCharacter with: char2 asCharacter)].
+ value3 := char3 asInteger.
- char3 ifNil:[^self errorMalformedInput: (String with: char1 with: char2)].
- value3 := char3 asciiValue.
  (value1 bitAnd: 16rF0) = 224 ifTrue: [
  unicode := ((value1 bitAnd: 15) bitShift: 12) + ((value2 bitAnd: 63) bitShift: 6)
  + (value3 bitAnd: 63).
  ].
 
  (value1 bitAnd: 16rF8) = 240 ifTrue: [
  "4-byte char"
  char4 := aStream basicNext.
+ char4 ifNil:[^self errorMalformedInput: (String with: char1 asCharacter with: char2 asCharacter with: char3 asCharacter)].
+ value4 := char4 asInteger.
- char4 ifNil:[^self errorMalformedInput: (String with: char1 with: char2 with: char3)].
- value4 := char4 asciiValue.
  unicode := ((value1 bitAnd: 16r7) bitShift: 18) +
  ((value2 bitAnd: 63) bitShift: 12) +
  ((value3 bitAnd: 63) bitShift: 6) +
  (value4 bitAnd: 63).
  ].
 
+ unicode ifNil:[^self errorMalformedInput: (String with: char1 asCharacter with: char2 asCharacter with: char3 asCharacter)].
- unicode ifNil:[^self errorMalformedInput: (String with: char1 with: char2 with: char3)].
  unicode > 16r10FFFD ifTrue: [
+ ^self errorMalformedInput: (String with: char1 asCharacter with: char2 asCharacter with: char3 asCharacter).
- ^self errorMalformedInput: (String with: char1 with: char2 with: char3).
  ].
 
  unicode = 16rFEFF ifTrue: [^ self nextFromStream: aStream].
  ^ Unicode value: unicode.
  !