The Trunk: Compiler-ar.107.mcz

Andreas Raab uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ar.107.mcz

==================== Summary ====================

Name: Compiler-ar.107
Author: ar
Time: 22 December 2009, 12:38:17 pm
UUID: 46345836-169c-a245-aa07-b546b826fdc4
Ancestors: Compiler-jcg.106

CompiledMethodTrailer phase 1: Preparations.

=============== Diff against Compiler-jcg.106 ===============

Item was changed:
  ----- Method: MethodNode>>rawSourceRangesAndMethodDo: (in category 'source mapping') -----
  rawSourceRangesAndMethodDo: aBinaryBlock
  "Evaluate aBinaryBlock with the rawSourceRanges and method generated from the receiver."
 
  | methNode method |
  methNode := encoder classEncoding parserClass new
  encoderClass: encoder class;
  parse: (sourceText "If no source, use decompile string as source to map from"
  ifNil: [self decompileString]
  ifNotNil: [sourceText])
  class: self methodClass.
+ method := methNode generate.  "set bytecodes to map to"
- method := methNode generate: #(0 0 0 0).  "set bytecodes to map to"
  ^aBinaryBlock
  value: methNode encoder rawSourceRanges
  value: method!

Item was changed:
  ----- Method: ClosureCompilerTest>>testInjectIntoDecompiledDebugs (in category 'tests') -----
  testInjectIntoDecompiledDebugs
  "Test various debugs of the decompiled form debug correctly."
  "self new testInjectIntoDecompiledDebugs"
  | source |
  source := (Collection sourceCodeAt: #inject:into:) asString.
  { Encoder.
    EncoderForV3PlusClosures. EncoderForLongFormV3PlusClosures } do:
  [:encoderClass| | method |
  method := (Parser new
  encoderClass: encoderClass;
  parse: source
  class: Collection)
+ generate.
- generate: #(0 0 0 0).
  self supportTestSourceRangeAccessForDecompiledInjectInto: method source: method decompileString]!

Item was changed:
  ----- Method: ClosureCompilerTest>>testBlockNumberingForInjectInto (in category 'tests') -----
  testBlockNumberingForInjectInto
  "Test that the compiler and CompiledMethod agree on the block numbering of Collection>>inject:into:
  and that temp names for inject:into: are recorded."
  "self new testBlockNumberingForInjectInto"
  | methodNode method tempRefs |
  methodNode := Parser new
  encoderClass: EncoderForV3PlusClosures;
  parse: (Collection sourceCodeAt: #inject:into:)
  class: Collection.
+ method := methodNode generate.
- method := methodNode generate: #(0 0 0 0).
  tempRefs := methodNode encoder blockExtentsToTempsMap.
  self assert: tempRefs keys asSet = method startpcsToBlockExtents values asSet.
  self assert: ((tempRefs includesKey: (0 to: 6))
  and: [(tempRefs at: (0 to: 6)) hasEqualElements: #(('thisValue' 1) ('binaryBlock' 2) ('nextValue' (3 1)))]).
  self assert: ((tempRefs includesKey: (2 to: 4))
  and: [(tempRefs at: (2 to: 4)) hasEqualElements: #(('each' 1) ('binaryBlock' 2) ('nextValue' (3 1)))])!

Item was changed:
  ----- Method: BytecodeAgnosticMethodNode>>schematicTempNamesString (in category 'debugger support') -----
  schematicTempNamesString
  "Answer the temp names for the current method node in a form that captures
  temp structure.  The temps at each method and block scope level occur
  space-separated, with any indirect temps enclosed in parentheses.  Each block
  level is enclosed in square brackets.  e.g.
  'method level temps (indirect temp)[block args and temps (indirect)]'
  This representation can be reconstituted into a blockExtentsToTempsMap
  by a CompiledMethod that has been copied with the schematicTempNamesString."
  encoder hasGeneratedMethod ifFalse:
  ["create the encoder's blockExtentsToLoals map, except if the method is quick
   in which case it has no temps."
+ (self generate) isQuick ifTrue:
- (self generate: #(0 0 0 0)) isQuick ifTrue:
  [^'']].
  ^encoder schematicTempNamesString!

Item was changed:
  ----- Method: BytecodeAgnosticMethodNode>>blockExtentsToTempsMap (in category 'debugger support') -----
  blockExtentsToTempsMap
  "Answer a Dictionary of blockExtent to temp locations for the current method.
  This is used by the debugger to locate temp vars in contexts.  A temp map
  entry is a pair of the temp's name and its index, where an index is either an
  integer for a normal temp or a pair of the index of the indirect temp vector
  containing  the temp and the index of the temp in its indirect temp vector."
 
  ^encoder blockExtentsToTempsMap ifNil:
  [| methNode |
  methNode := encoder classEncoding parserClass new
  encoderClass: encoder class;
  parse: (sourceText ifNil: [self decompileString])
  class: self methodClass.
  "As a side effect generate: creates data needed for the map."
+ methNode generate.
- methNode generate: #(0 0 0 0).
  methNode encoder blockExtentsToTempsMap]!

Item was changed:
  ----- Method: ClosureCompilerTest>>testBlockNumbering (in category 'tests') -----
  testBlockNumbering
  "Test that the compiler and CompiledMethod agree on the block numbering of a substantial doit."
  "self new testBlockNumbering"
  | methodNode method tempRefs |
  methodNode :=
  Parser new
  encoderClass: EncoderForV3PlusClosures;
  parse: 'foo
  | numCopiedValuesCounts |
  numCopiedValuesCounts := Dictionary new.
  0 to: 32 do: [:i| numCopiedValuesCounts at: i put: 0].
  Transcript clear.
  Smalltalk allClasses remove: GeniePlugin; do:
  [:c|
  {c. c class} do:
  [:b|
  Transcript nextPut: b name first; endEntry.
  b selectorsAndMethodsDo:
  [:s :m| | pn |
  m isQuick not ifTrue:
  [pn := b parserClass new
  encoderClass: EncoderForV3PlusClosures;
  parse: (b sourceCodeAt: s)
  class: b.
+ pn generate.
- pn generate: #(0 0 0 0).
  [pn accept: nil]
  on: MessageNotUnderstood
  do: [:ex| | msg numCopied |
  msg := ex message.
  (msg selector == #visitBlockNode:
  and: [(msg argument instVarNamed: ''optimized'') not]) ifTrue:
  [numCopied := (msg argument computeCopiedValues: pn) size.
  numCopiedValuesCounts
  at: numCopied
  put: (numCopiedValuesCounts at: numCopied) + 1].
  msg setSelector: #==.
  ex resume: nil]]]]].
  numCopiedValuesCounts'
  class: Object.
+ method := methodNode generate.
- method := methodNode generate: #(0 0 0 0).
  tempRefs := methodNode encoder blockExtentsToTempsMap.
  self assert: tempRefs keys asSet = method startpcsToBlockExtents values asSet!

Item was changed:
  ----- Method: ClosureCompilerTest>>testMethodAndNodeTempNames (in category 'tests') -----
  testMethodAndNodeTempNames
  "self new testMethodAndNodeTempNames"
  "Test that BytecodeAgnosticMethodNode>>blockExtentsToTempRefs answers the same
  structure as CompiledMethod>>blockExtentsToTempRefs when the method has been
  copied with the appropriate temps.  This tests whether doit methods are debuggable
  since they carry their own temps."
  self closureCases do:
  [:source| | mn om m mbe obe |
  mn := source first isLetter
  ifTrue:
  [self class compilerClass new
  compile: source
  in: self class
  notifying: nil
  ifFail: [self error: 'compilation error']]
  ifFalse:
  [self class compilerClass new
  compileNoPattern: source
  in: self class
  context: nil
  notifying: nil
  ifFail: [self error: 'compilation error']].
+ m := (om := mn generate) copyWithTempsFromMethodNode: mn.
- m := (om := mn generate: #(0 0 0 0)) copyWithTempsFromMethodNode: mn.
  self assert: m holdsTempNames.
  self assert: m endPC = om endPC.
  mbe := m blockExtentsToTempsMap.
  obe := mn blockExtentsToTempsMap.
  self assert: mbe keys asSet = obe keys asSet.
  (mbe keys intersection: obe keys) do:
  [:interval|
  self assert: (mbe at: interval) = (obe at: interval)]]!

Item was added:
+ ----- Method: BytecodeAgnosticMethodNode>>generateWithTempNames (in category 'code generation (new scheme)') -----
+ generateWithTempNames
+ "Answer a CompiledMethod with temps names encoded in trailer"
+ ^ self generate: (CompiledMethodTrailer new tempNames: self schematicTempNamesString).
+ !

Item was changed:
  ----- Method: ClosureCompilerTest>>testDecompiledDoitMethodTempNames (in category 'tests') -----
  testDecompiledDoitMethodTempNames
  "self new testDecompiledDoitMethodTempNames"
  "Test that a decompiled doit that has been copied with temps decompiles to the input"
  | removeComments |
  removeComments := [:n| n comment: nil].
  self closureCases do:
  [:source| | mns m mps mnps |
  "Need to compare an ungenerated tree with the generated method's methodNode
  because generating code alters the tree when it introduces remote temp vectors."
  mns := #(first last) collect:
  [:ignored|
  source first isLetter
  ifTrue:
  [self class compilerClass new
  compile: source
  in: self class
  notifying: nil
  ifFail: [self error: 'compilation error']]
  ifFalse:
  [self class compilerClass new
  compileNoPattern: source
  in: self class
  context: nil
  notifying: nil
  ifFail: [self error: 'compilation error']]].
+ m := (mns last generateWithTempNames).
- m := (mns last generate: #(0 0 0 0)) copyWithTempsFromMethodNode: mns last.
  removeComments value: mns first.
  mns first nodesDo: removeComments.
  self assert: (mnps := mns first printString) = (mps := m methodNode printString)]!

Item was changed:
  ----- Method: ClosureCompilerTest>>testTempNameAccessForInjectInto (in category 'tests') -----
  testTempNameAccessForInjectInto
  "self new testTempNameAccessForInjectInto"
  | methodNode method evaluationCount block debuggerMap |
  methodNode := Parser new
  encoderClass: EncoderForV3PlusClosures;
  parse: (Collection sourceCodeAt: #inject:into:)
  class: Collection.
+ method := methodNode generate.
- method := methodNode generate: #(0 0 0 0).
  debuggerMap := DebuggerMethodMap forMethod: method methodNode: methodNode.
  evaluationCount := 0.
  block := [:prev :each| | theContext tempNames |
  evaluationCount := evaluationCount + 1.
  theContext := thisContext sender.
  tempNames := debuggerMap tempNamesForContext: theContext.
  self assert: (tempNames hasEqualElements: tempNames).
  #('thisValue' 'each' 'binaryBlock' 'nextValue')
  with: { 0. each. block. prev}
  do: [:tempName :value|
  self assert: (debuggerMap namedTempAt: (tempNames indexOf: tempName) in: theContext) == value.
  tempName ~= 'each' ifTrue:
  [self assert: (debuggerMap namedTempAt: (tempNames indexOf: tempName) in: theContext home) == value]]].
  (1 to: 10) withArgs: { 0. block } executeMethod: method.
  self assert: evaluationCount = 10!

Item was changed:
  ----- Method: DecompilerTests>>checkDecompileMethod: (in category 'utilities') -----
  checkDecompileMethod: oldMethod
 
  | cls selector oldMethodNode methodNode newMethod oldCodeString newCodeString |
  cls := oldMethod methodClass.
  selector := oldMethod selector.
  oldMethodNode := cls decompilerClass new
  decompile: selector
  in: cls
  method: oldMethod.
  [oldMethodNode properties includesKey: #warning] whileTrue:
  [oldMethodNode properties removeKey: #warning].
  oldCodeString := oldMethodNode decompileString.
  methodNode := [cls compilerClass new
  compile: oldCodeString
  in: cls
  notifying: nil
  ifFail: []]
  on: SyntaxErrorNotification
  do: [:ex|
  ex errorMessage = 'Cannot store into' ifTrue:
  [ex return: #badStore].
  ex pass].
  "Ignore cannot store into block arg errors; they're not our issue."
  methodNode ~~ #badStore ifTrue:
+ [newMethod := methodNode generate.
- [newMethod := methodNode generate: #(0 0 0 0).
  newCodeString := (cls decompilerClass new
  decompile: selector
  in: cls
  method: newMethod) decompileString.
  "(StringHolder new textContents:
  (TextDiffBuilder buildDisplayPatchFrom: oldCodeString to: newCodeString))
  openLabel: 'Decompilation Differences for ', cls name,'>>',selector"
  "(StringHolder new textContents:
  (TextDiffBuilder buildDisplayPatchFrom: oldMethod abstractSymbolic to: newMethod abstractSymbolic))
  openLabel: 'Bytecode Differences for ', cls name,'>>',selector"
  self assert: oldCodeString = newCodeString
  description: cls name asString, ' ', selector asString
  resumable: true]!

Item was changed:
  ----- Method: ClosureCompilerTest>>testInjectIntoDecompilations (in category 'tests') -----
  testInjectIntoDecompilations
  "Test various compilations decompile to the same code for a method sufficiently
  simple that this is possible and sufficiently complex that the code generated
  varies between the compilations."
  "self new testInjectIntoDecompilations"
  | source |
  source := (Collection sourceCodeAt: #inject:into:) asString.
  { Encoder.
    EncoderForV3. EncoderForLongFormV3.
    EncoderForV3PlusClosures. EncoderForLongFormV3PlusClosures } do:
  [:encoderClass| | method |
  method := (Parser new
  encoderClass: encoderClass;
  parse: source
  class: Collection)
+ generate.
- generate: #(0 0 0 0).
  self assert: (Scanner new scanTokens: method decompileString)
  = #(inject: t1 into: t2
  | t3 |
  t3 ':=' t1 .
  self do: [ ':t4' | t3 ':=' t2 value: t3 value: t4 ] .
  ^ t3)]!