The Trunk: Multilingual-ul.133.mcz

Levente Uzonyi uploaded a new version of Multilingual to project The Trunk:
http://source.squeak.org/trunk/Multilingual-ul.133.mcz

==================== Summary ====================

Name: Multilingual-ul.133
Author: ul
Time: 24 January 2011, 3:40:52.355 am
UUID: e8de6a53-b209-c442-911b-4f711ca326aa
Ancestors: Multilingual-mtf.132, Multilingual-sn.131

- merged Multilingual-sn.131 to fix #defaultSystemConverter if using Latin1Environment
- make sure that the converter (#defaultSystemConverter) and line end convention (crlf for windows, lf for other platforms) is properly initialized for multi-byte stdio streams
- fixed MultiByteFileStream >> #setConverterForCode which tried to save and restore the stream's state while the converter (which decides how to store the state) was changed
- use the converter variable instead of the message with the same name from MultiByteFileStream
- deprecated #bareNext, inlined it to it's two senders which were also refactored a bit
- initialize MultiByteFileStream instances for stdio via #newForStdio instead of #openOnHandle:name:forWrite: which is removed.

=============== Diff against Multilingual-mtf.132 ===============

Item was changed:
  ----- Method: Latin1Environment class>>fileNameConverterClass (in category 'subclass responsibilities') -----
  fileNameConverterClass
 
+ ^ self systemConverterClass
- ^ Latin1TextConverter
  !

Item was changed:
  ----- Method: Latin1Environment class>>inputInterpreterClass (in category 'subclass responsibilities') -----
  inputInterpreterClass
  | platformName osVersion |
  platformName := SmalltalkImage current platformName.
  osVersion := SmalltalkImage current osVersion.
  (platformName = 'Win32' and: [osVersion ~= 'CE'])
  ifTrue: [^ (self win32VMUsesUnicode) ifTrue: [UTF32InputInterpreter] ifFalse: [MacRomanInputInterpreter]].
  platformName = 'Mac OS'
+ ifTrue: [^ MacUnicodeInputInterpreter].
- ifTrue: [^ MacRomanInputInterpreter].
  platformName = 'unix'
  ifTrue: [^ UTF32InputInterpreter].
+ ^ MacUnicodeInputInterpreter!
- ^ MacRomanInputInterpreter!

Item was changed:
  ----- Method: Latin1Environment class>>systemConverterClass (in category 'subclass responsibilities') -----
  systemConverterClass
 
+ | platformName osVersion |
+ platformName := SmalltalkImage current platformName.
+ osVersion := SmalltalkImage current getSystemAttribute: 1002.
+ (platformName = 'Win32'
+ and: [osVersion = 'CE'])
+ ifTrue: [^ MacRomanTextConverter].
+ platformName = 'Win32'
+ ifTrue: [^ (self win32VMUsesUnicode) ifTrue: [UTF8TextConverter] ifFalse: [Latin1TextConverter]].
+ platformName = 'Mac OS'
+ ifTrue: [^ ('10*' match: SmalltalkImage current osVersion)
+ ifTrue: [UTF8TextConverter]
+ ifFalse: [MacRomanTextConverter]].
+ platformName = 'unix'
+ ifTrue: [^ UTF8TextConverter].
+ ^ MacRomanTextConverter!
- ^ Latin1TextConverter.
- !

Item was changed:
  ----- Method: MacUnicodeInputInterpreter>>initialize (in category 'initialization') -----
  initialize
  | satisfiesVersion |
  satisfiesVersion := self
  majorMinorBuildFrom: SmalltalkImage current vmVersion
+ satisfies: [:major :minor :build |
+ major >= 4 or: [
+ major >= 3 and: [minor >= 8 and: [build >= 7]]]].
- satisfies: [:major :minor :build |  (major >= 3
- and: [minor >= 8
- and: [build >= 7]]) or: [major >= 4]].
-
  satisfiesVersion
  ifTrue: [keyValueIndex := 6]
  ifFalse: [keyValueIndex := 3]!

Item was added:
+ ----- Method: MultiByteFileStream class>>newForStdio (in category 'as yet unclassified') -----
+ newForStdio
+ "Use crlf as line end convention on windows, lf on all other platforms. Also make sure that the converter is initialized."
+
+ | lineEndConvention |
+ lineEndConvention := self lineEndDefault.
+ lineEndConvention == #crlf ifFalse: [
+ lineEndConvention := #lf ].
+ ^self new
+ lineEndConvention: lineEndConvention;
+ initializeConverter;
+ yourself!

Item was changed:
  ----- Method: MultiByteFileStream>>bareNext (in category 'crlf private') -----
  bareNext
 
+ self deprecated: 'Don''t use this method anymore.'.
+ ^converter nextFromStream: self.
- ^ self converter nextFromStream: self.
  !

Item was changed:
  ----- Method: MultiByteFileStream>>converter (in category 'accessing') -----
  converter
 
+ ^converter ifNil: [
+ self initializeConverter.
+ converter ]!
- converter ifNil: [self converter: TextConverter defaultSystemConverter].
- ^ converter
- !

Item was added:
+ ----- Method: MultiByteFileStream>>initializeConverter (in category 'initialize-release') -----
+ initializeConverter
+
+ self converter: TextConverter defaultSystemConverter!

Item was changed:
  ----- Method: MultiByteFileStream>>next (in category 'public') -----
  next
 
+ | char |
- | char secondChar state |
  char := converter nextFromStream: self.
+ "#doConversion is inlined here"
+ (wantsLineEndConversion == true and: [ lineEndConvention notNil ]) ifTrue: [
+ char == Cr ifTrue: [
+ | state |
+ state := converter saveStateOf: self.
+ (converter nextFromStream: self) ifNotNil: [ :secondChar |
+ secondChar == Lf ifFalse: [
+ converter restoreStateOf: self with: state ] ].
+ ^Cr ].
+ char == Lf ifTrue: [
+ ^Cr ] ].
- (wantsLineEndConversion == true and: [ lineEndConvention notNil ]) "#doConversion is inlined here"
- ifTrue: [
- char == Cr ifTrue: [
- state := converter saveStateOf: self.
- secondChar := self bareNext.
- secondChar ifNotNil: [
- secondChar == Lf ifFalse: [ converter restoreStateOf: self with: state ] ].
- ^Cr ].
- char == Lf ifTrue: [
- ^Cr ] ].
  ^char.
 
  !

Item was changed:
  ----- Method: MultiByteFileStream>>next:innerFor: (in category 'crlf private') -----
  next: n innerFor: aString
 
- | peekChar state |
- "if we just read a CR, and the next character is an LF, then skip the LF"
  aString size = 0 ifTrue: [^ aString].
+ "if we just read a CR, and the next character is an LF, then skip the LF"
+ aString last == Cr ifTrue: [
+ | state |
- (aString last = Character cr) ifTrue: [
  state := converter saveStateOf: self.
+ (converter nextFromStream: self) ifNotNil: [ :peekChar |
+ peekChar == Lf ifFalse: [
+ converter restoreStateOf: self with: state ] ] ].
+ ^aString withSqueakLineEndings
- peekChar := self bareNext. "super peek doesn't work because it relies on #next"
- (peekChar notNil and: [peekChar ~= Character lf]) ifTrue: [
- converter restoreStateOf: self with: state.
- ].
- ].
-  
- ^ aString withSqueakLineEndings.
  !

Item was changed:
  ----- Method: MultiByteFileStream>>nextPut: (in category 'public') -----
  nextPut: aCharacter
 
  aCharacter isInteger ifTrue: [ ^super nextPut: aCharacter ].
  (wantsLineEndConversion == true and: [ lineEndConvention notNil ]) "#doConversion is inlined here"
  ifTrue: [
+ aCharacter == Cr
- aCharacter = Cr
  ifTrue: [
+ converter
+ nextPutAll: (LineEndStrings at: lineEndConvention)
+ toStream: self ]
- (LineEndStrings at: lineEndConvention) do: [ :each |
- converter nextPut: each toStream: self ] ]
  ifFalse: [
  converter nextPut: aCharacter toStream: self ].
  ^aCharacter ].
+ converter nextPut: aCharacter toStream: self.
+ ^aCharacter!
- self converter nextPut: aCharacter toStream: self.
- ^aCharacter
- !

Item was changed:
  ----- Method: MultiByteFileStream>>nextPutAll: (in category 'public') -----
  nextPutAll: aCollection
 
  (self isBinary or: [aCollection class == ByteArray]) ifTrue: [
  ^ super nextPutAll: aCollection.
  ].
+ converter nextPutAll: aCollection toStream: self.
- self converter nextPutAll: aCollection toStream: self.
  ^aCollection!

Item was removed:
- ----- Method: MultiByteFileStream>>openOnHandle:name:forWrite: (in category 'open/close') -----
- openOnHandle: aFileID name: streamName forWrite: writeMode
-
- super openOnHandle: aFileID name: streamName forWrite: writeMode.
- self converter "Make sure that converter is initialized."!

Item was changed:
  ----- Method: MultiByteFileStream>>setConverterForCode (in category 'private') -----
  setConverterForCode
 
+ | currentPosition |
- | current |
  (SourceFiles at: 2)
  ifNotNil: [self fullName = (SourceFiles at: 2) fullName ifTrue: [^ self]].
+ currentPosition := self position.
- current := self converter saveStateOf: self.
  self position: 0.
  self binary.
  ((self next: 3) = #[ 16rEF 16rBB 16rBF ]) ifTrue: [
  self converter: UTF8TextConverter new
  ] ifFalse: [
  self converter: MacRomanTextConverter new.
  ].
+ self position: currentPosition.
- converter restoreStateOf: self with: current.
  self text.
  !