[PATCH] stinst: Parse lowercase number literals from Squeak/Pharo

> From: Holger Hans Peter Freyther <[hidden email]>
>
> Be able to parse literals like 16rabcdef from Squeak/Pharo source.
> Introduce RBScanner>>#digitValue: and reimplement >>#isDigit:base:
> and #digitValue: in the SqueakFileInScanner class.
> ---
>  ChangeLog                              |    5 +++++
>  packages/stinst/parser/ChangeLog       |    7 +++++++
>  packages/stinst/parser/RBParser.st     |    9 +++++++--
>  packages/stinst/parser/SqueakParser.st |   12 ++++++++++++
>  tests/stcompiler.ok                    |    3 +++
>  tests/stcompiler.st                    |   11 +++++++++++
>  6 files changed, 45 insertions(+), 2 deletions(-)
>
> diff --git a/ChangeLog b/ChangeLog
> index 7353dcc..694ec33 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,3 +1,8 @@
> +2012-09-29  Holger Freyther  <[hidden email]>
> +
> + * tests/stcompiler.st: Add testcase for parsing 16rff.
> + * tests/stcompiler.ok: Update the test result.
> +
>  2012-09-09  Holger Freyther  <[hidden email]>
>  
>   * tests/stcompiler.st: Add testcase for article.
> diff --git a/packages/stinst/parser/ChangeLog b/packages/stinst/parser/ChangeLog
> index c49df53..31bb50c 100644
> --- a/packages/stinst/parser/ChangeLog
> +++ b/packages/stinst/parser/ChangeLog
> @@ -1,3 +1,10 @@
> +2012-09-29  Holger Freyther  <[hidden email]>
> +
> + * RBParser.st: Add RBScanner>>#digitValue:.
> + * SqueakParser.st: Reimplement SqueakFileInScanner>>#digitValue:,
> + SqueakFileInScanner>>#isDigit:base:. Parsing of 16rff is now
> + possible.
> +
>  2012-09-09  Holger Freyther  <[hidden email]>
>  
>   * STLoaderObjs.st: Add LoadedBehavior>>#article.
> diff --git a/packages/stinst/parser/RBParser.st b/packages/stinst/parser/RBParser.st
> index f40356b..2700977 100644
> --- a/packages/stinst/parser/RBParser.st
> +++ b/packages/stinst/parser/RBParser.st
> @@ -976,6 +976,11 @@ Stream subclass: RBScanner [
>      ifFalse: [aChar isDigit]
>      ]
>  
> +    digitValue: aChar [
> + <category: 'private-scanning numbers'>
> +        ^ aChar digitValue
> +    ]
> +
>      scanDigits: ch base: base [
>   <category: 'private-scanning numbers'>
>   | c num |
> @@ -987,7 +992,7 @@ Stream subclass: RBScanner [
>   c := currentCharacter].
>   c notNil and: [self isDigit: c base: base]]
>   whileTrue:
> -    [num := num * base + c digitValue.
> +    [num := num * base + (self digitValue: c).
>      self step.
>      c := currentCharacter].
>   ^num
> @@ -1016,7 +1021,7 @@ Stream subclass: RBScanner [
>   c := currentCharacter].
>   c notNil and: [self isDigit: c base: base]]
>   whileTrue:
> -    [result := result * base + c digitValue.
> +    [result := result * base + (self digitValue: c).
>      self step.
>      c := currentCharacter.
>      scale := scale - 1].
> diff --git a/packages/stinst/parser/SqueakParser.st b/packages/stinst/parser/SqueakParser.st
> index e223e66..6be255d 100644
> --- a/packages/stinst/parser/SqueakParser.st
> +++ b/packages/stinst/parser/SqueakParser.st
> @@ -76,6 +76,18 @@ STFileScanner subclass: SqueakFileInScanner [
>      <comment: nil>
>      <category: 'Refactory-Parser'>
>  
> +    isDigit: aChar base: aBase [
> +        <category: 'private-scanning numbers'>
> +        "Pharo/Squeak allows 16rff as literal."
> +        ^ super isDigit: aChar asUppercase base: aBase.
> +    ]
> +
> +    digitValue: aChar [
> +        <category: 'private-scanning numbers'>
> +        "Pharo/Squeak allows 16rff as literal."
> +        ^ super digitValue: aChar asUppercase
> +    ]
> +
>      on: aStream [
>   <category: 'accessing'>
>   super on: aStream.
> diff --git a/tests/stcompiler.ok b/tests/stcompiler.ok
> index f3acb52..dc54a2c 100644
> --- a/tests/stcompiler.ok
> +++ b/tests/stcompiler.ok
> @@ -46,4 +46,7 @@ Execution begins...
>  returned value is OrderedSet new: 32 "<0>"
>  
>  Execution begins...
> +returned value is OrderedSet new: 32 "<0>"
> +
> +Execution begins...
>  returned value is 'an'
> diff --git a/tests/stcompiler.st b/tests/stcompiler.st
> index ef44d8d..084f5f9 100644
> --- a/tests/stcompiler.st
> +++ b/tests/stcompiler.st
> @@ -91,6 +91,17 @@ asParser
>  ]
>  
>  Eval [
> +    | squeak |
> +    "Test literal parsing"
> +    squeak := '!String methodsFor: ''*unit-test'' stamp: ''lr 11/7/2009 13:32''!
> +literalValueFoo12345
> +    ^ 16rabcdef! !'.
> +
> +    STInST.STClassLoader new parseSmalltalkStream: squeak readStream
> +    with: STInST.SqueakFileInParser.
> +]
> +
> +Eval [
>      | classes |
>  
>      classes := STInST.STClassLoader new
>