Smalltalk › Squeak › Squeak - Dev

testrun results

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

11 messages Options

laza

testrun results

Frank Shearar

Re: testrun results

Many tests below fail for the same reason. As such I've categorised them
as [1], [2], etc. and given a description of the failure under the table.

[hidden email] wrote:
> All Tests that failed or have errors:
>
> class | selector | result
> --------------------------+--------------------------------+---------
<snip>

<snip>

[1] tests fail because variables are being "named" (I don't know the
correct terminology) in a different order. For instance:

old AA->AM
'at: t1 ifAbsentPut: t2
| t4 |
1
to: self basicSize
do: [:t3 | (t4 := self basicAt: t3) key == t1
ifTrue: [^ t4 isVariableBinding
ifTrue: [t4 value]
ifFalse: [t4]]].
^ method propertyValueAt: t1 put: t2 value'

new AA->AM
'at: t1 ifAbsentPut: t2
| t3 |
1
to: self basicSize
do: [:t4 | (t3 := self basicAt: t4) key == t1
ifTrue: [^ t3 isVariableBinding
ifTrue: [t3 value]
ifFalse: [t3]]].
^ method propertyValueAt: t1 put: t2 value'

The block temps get named before the local variables, or a block temp
skips a number: GNtoGZ's oldCodeString uses "t7" as a block temp name,
while newCodeString uses "t8".

[2] fails because the Symbols are now being output as #Foo not Foo
inside literal ByteArrays.

[3] has a Syntax Error in, for example, EventSensor>>eventTickler -
"Period or right bracket expected ->_r2 := Time millisecondClockValue -
lastEventPoll." Maybe it's that underscore?

[4] fails in a similar way to [1] tests, but here the old code string
has, for instance, "t4" and the new code string has "t3". A temp name's
skipped in the oldCodeString.

[5] oldCodeString for HttpUrl>>checkAuthorization:retry:,
PNGReadWriter>>copyPixelsGray: returns nil while newCodeString returns
(by not having a return) self.

[6] is also a naming issue. The oldCodeString for MVCProject>>textWindow
shows two differently named block temps (t9, t10) for the two
#detect:ifNone: calls, while the newCodeString shows the two block temps
with the same name (t10).

frank

Levente Uzonyi-2

Re: testrun results

On Mon, 5 Apr 2010, Frank Shearar wrote:

> Many tests below fail for the same reason. As such I've categorised them as
> [1], [2], etc. and given a description of the failure under the table.
>
> [hidden email] wrote:
>> All Tests that failed or have errors:
>>
>> class | selector | result
>> --------------------------+--------------------------------+---------
> <snip>
>> DecompilerTests | testDecompilerInClassesAAtoAM | failure [1]
>> DecompilerTests | testDecompilerInClassesBAtoBM | failure [1]
>> DecompilerTests | testDecompilerInClassesCAtoCM | failure [1]
>> DecompilerTests | testDecompilerInClassesCNtoCZ | failure [1]
>> DecompilerTests | testDecompilerInClassesDAtoDM | failure [2]
>> DecompilerTests | testDecompilerInClassesENtoEZ | error [3]
>> DecompilerTests | testDecompilerInClassesFAtoFM | failure [1]
>> DecompilerTests | testDecompilerInClassesFNtoFZ | failure [4]
>> DecompilerTests | testDecompilerInClassesGNtoGZ | failure [1]
>> DecompilerTests | testDecompilerInClassesHNtoHZ | failure [5]
>> DecompilerTests | testDecompilerInClassesINtoIZ | failure [1]
>> DecompilerTests | testDecompilerInClassesJNtoJZ | failure [1]
>> DecompilerTests | testDecompilerInClassesLNtoLZ | failure [4]
>> DecompilerTests | testDecompilerInClassesMAtoMM | failure [1]
>> DecompilerTests | testDecompilerInClassesMNtoMZ | failure [6]
>> DecompilerTests | testDecompilerInClassesNAtoNM | failure [1]
>> DecompilerTests | testDecompilerInClassesOAtoOM | failure [4]
>> DecompilerTests | testDecompilerInClassesPAtoPM | failure [1]
>> DecompilerTests | testDecompilerInClassesPNtoPZ | failure [5]
>> DecompilerTests | testDecompilerInClassesRAtoRM | failure [1]
>> DecompilerTests | testDecompilerInClassesSAtoSM | error [3]
>> DecompilerTests | testDecompilerInClassesSNtoSZ | failure [1]
>> DecompilerTests | testDecompilerInClassesTAtoTM | failure [1]
>> DecompilerTests | testDecompilerInClassesTNtoTZ | failure [1]
>> DecompilerTests | testDecompilerInClassesWAtoWM | failure [4]
> <snip>
>
> [1] tests fail because variables are being "named" (I don't know the correct
> terminology) in a different order. For instance:
>
> old AA->AM
> 'at: t1 ifAbsentPut: t2
> | t4 |
> 1
> to: self basicSize
> do: [:t3 | (t4 := self basicAt: t3) key == t1
> ifTrue: [^ t4 isVariableBinding
> ifTrue: [t4 value]
> ifFalse: [t4]]].
> ^ method propertyValueAt: t1 put: t2 value'
>
> new AA->AM
> 'at: t1 ifAbsentPut: t2
> | t3 |
> 1
> to: self basicSize
> do: [:t4 | (t3 := self basicAt: t4) key == t1
> ifTrue: [^ t3 isVariableBinding
> ifTrue: [t3 value]
> ifFalse: [t3]]].
> ^ method propertyValueAt: t1 put: t2 value'
>
> The block temps get named before the local variables, or a block temp skips a
> number: GNtoGZ's oldCodeString uses "t7" as a block temp name, while
> newCodeString uses "t8".
>
> [2] fails because the Symbols are now being output as #Foo not Foo inside
> literal ByteArrays.
>
> [3] has a Syntax Error in, for example, EventSensor>>eventTickler - "Period
> or right bracket expected ->_r2 := Time millisecondClockValue -
> lastEventPoll." Maybe it's that underscore?

It's a decompiler bug. Eliot has fixes for these issues but I don't know
if he has time to add them to the Trunk.

Levente

>
> [4] fails in a similar way to [1] tests, but here the old code string has,
> for instance, "t4" and the new code string has "t3". A temp name's skipped in
> the oldCodeString.
>
> [5] oldCodeString for HttpUrl>>checkAuthorization:retry:,
> PNGReadWriter>>copyPixelsGray: returns nil while newCodeString returns (by
> not having a return) self.
>
> [6] is also a naming issue. The oldCodeString for MVCProject>>textWindow
> shows two differently named block temps (t9, t10) for the two #detect:ifNone:
> calls, while the newCodeString shows the two block temps with the same name
> (t10).
>
> frank
>
>

Nicolas Cellier

Re: testrun results

2010/4/5 Levente Uzonyi <[hidden email]>:

> On Mon, 5 Apr 2010, Frank Shearar wrote:
>
>> Many tests below fail for the same reason. As such I've categorised them
>> as [1], [2], etc. and given a description of the failure under the table.
>>
>> [hidden email] wrote:
>>>
>>> All Tests that failed or have errors:
>>>
>>> class | selector | result
>>> --------------------------+--------------------------------+---------
>>
>> <snip>
>>>
>>> DecompilerTests | testDecompilerInClassesAAtoAM | failure [1]
>>> DecompilerTests | testDecompilerInClassesBAtoBM | failure [1]
>>> DecompilerTests | testDecompilerInClassesCAtoCM | failure [1]
>>> DecompilerTests | testDecompilerInClassesCNtoCZ | failure [1]
>>> DecompilerTests | testDecompilerInClassesDAtoDM | failure [2]
>>> DecompilerTests | testDecompilerInClassesENtoEZ | error [3]
>>> DecompilerTests | testDecompilerInClassesFAtoFM | failure [1]
>>> DecompilerTests | testDecompilerInClassesFNtoFZ | failure [4]
>>> DecompilerTests | testDecompilerInClassesGNtoGZ | failure [1]
>>> DecompilerTests | testDecompilerInClassesHNtoHZ | failure [5]
>>> DecompilerTests | testDecompilerInClassesINtoIZ | failure [1]
>>> DecompilerTests | testDecompilerInClassesJNtoJZ | failure [1]
>>> DecompilerTests | testDecompilerInClassesLNtoLZ | failure [4]
>>> DecompilerTests | testDecompilerInClassesMAtoMM | failure [1]
>>> DecompilerTests | testDecompilerInClassesMNtoMZ | failure [6]
>>> DecompilerTests | testDecompilerInClassesNAtoNM | failure [1]
>>> DecompilerTests | testDecompilerInClassesOAtoOM | failure [4]
>>> DecompilerTests | testDecompilerInClassesPAtoPM | failure [1]
>>> DecompilerTests | testDecompilerInClassesPNtoPZ | failure [5]
>>> DecompilerTests | testDecompilerInClassesRAtoRM | failure [1]
>>> DecompilerTests | testDecompilerInClassesSAtoSM | error [3]
>>> DecompilerTests | testDecompilerInClassesSNtoSZ | failure [1]
>>> DecompilerTests | testDecompilerInClassesTAtoTM | failure [1]
>>> DecompilerTests | testDecompilerInClassesTNtoTZ | failure [1]
>>> DecompilerTests | testDecompilerInClassesWAtoWM | failure [4]
>>
>> <snip>
>>
>> [1] tests fail because variables are being "named" (I don't know the
>> correct terminology) in a different order. For instance:
>>
>> old AA->AM
>> 'at: t1 ifAbsentPut: t2
>> | t4 |
>> 1
>> to: self basicSize
>> do: [:t3 | (t4 := self basicAt: t3) key == t1
>> ifTrue: [^ t4 isVariableBinding
>> ifTrue: [t4 value]
>> ifFalse: [t4]]].
>> ^ method propertyValueAt: t1 put: t2 value'
>>
>> new AA->AM
>> 'at: t1 ifAbsentPut: t2
>> | t3 |
>> 1
>> to: self basicSize
>> do: [:t4 | (t3 := self basicAt: t4) key == t1
>> ifTrue: [^ t3 isVariableBinding
>> ifTrue: [t3 value]
>> ifFalse: [t3]]].
>> ^ method propertyValueAt: t1 put: t2 value'
>>
>> The block temps get named before the local variables, or a block temp
>> skips a number: GNtoGZ's oldCodeString uses "t7" as a block temp name, while
>> newCodeString uses "t8".
>>
>> [2] fails because the Symbols are now being output as #Foo not Foo inside
>> literal ByteArrays.
>>
>> [3] has a Syntax Error in, for example, EventSensor>>eventTickler -
>> "Period or right bracket expected ->_r2 := Time millisecondClockValue -
>> lastEventPoll." Maybe it's that underscore?
>
> It's a decompiler bug. Eliot has fixes for these issues but I don't know if
> he has time to add them to the Trunk.
>
>
> Levente
>

For example see
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-March/145405.html

>>
>> [4] fails in a similar way to [1] tests, but here the old code string has,
>> for instance, "t4" and the new code string has "t3". A temp name's skipped
>> in the oldCodeString.
>>
>> [5] oldCodeString for HttpUrl>>checkAuthorization:retry:,
>> PNGReadWriter>>copyPixelsGray: returns nil while newCodeString returns (by
>> not having a return) self.
>>
>> [6] is also a naming issue. The oldCodeString for MVCProject>>textWindow
>> shows two differently named block temps (t9, t10) for the two
>> #detect:ifNone: calls, while the newCodeString shows the two block temps
>> with the same name (t10).
>>
>> frank
>>
>>
>
>

Andreas.Raab

Re: testrun results

In reply to this post by Levente Uzonyi-2

On 4/5/2010 4:04 AM, Levente Uzonyi wrote:
> It's a decompiler bug. Eliot has fixes for these issues but I don't know
> if he has time to add them to the Trunk.

I just spoke with Eliot and it seems unlikely that he'll have much time
to work on this. Also, there are cases that the decompiler handles
correctly, but differently from the compiler.

Eliot's recommendation to extent DecompilerTests>>decompilerFailures
with the failing sites.

Cheers,
- Andreas

Eliot Miranda-2

Re: testrun results

On Mon, Apr 5, 2010 at 10:37 AM, Andreas Raab <[hidden email]> wrote:

On 4/5/2010 4:04 AM, Levente Uzonyi wrote:

It's a decompiler bug. Eliot has fixes for these issues but I don't know
if he has time to add them to the Trunk.

I just spoke with Eliot and it seems unlikely that he'll have much time to work on this. Also, there are cases that the decompiler handles correctly, but differently from the compiler.

Eliot's recommendation to extent DecompilerTests>>decompilerFailures with the failing sites.

Some saintly person should go through the failures and add to decompilerFailures all those that fail because of known limitations in the decompiler. These limitations are things like

Floating point reader precision, e.g. the source specifies more precision than is supported and so the compilation of the decompilation is slightly different.

Unreachable statements. e.g. foo ifTrue: [^bar] ifFalse: [^baz]. ^huh? decompiles as foo ifTrue: [^bar] ifFalse: [^baz]. and so ^'huh?' is missing.

Null statements. e.g. foo ifTrue: []. decompiles as foo. which, if foo is an inst var, compiles as empty.

These are limitations in the decompiler we simply have to tolerate. decompilerFailures is a list that documents the reasons for failure.

Of course another, also valid, approach is to fix the code in the failures so that they don't contain unreachable or null statements.

HTH

Eliot

Cheers,
- Andreas

Levente Uzonyi-2

Re: testrun results

On Mon, 5 Apr 2010, Eliot Miranda wrote:

> On Mon, Apr 5, 2010 at 10:37 AM, Andreas Raab <[hidden email]> wrote:
>
>> On 4/5/2010 4:04 AM, Levente Uzonyi wrote:
>>
>>> It's a decompiler bug. Eliot has fixes for these issues but I don't know
>>> if he has time to add them to the Trunk.
>>>
>>
>> I just spoke with Eliot and it seems unlikely that he'll have much time to
>> work on this. Also, there are cases that the decompiler handles correctly,
>> but differently from the compiler.
>>
>> Eliot's recommendation to extent DecompilerTests>>decompilerFailures with
>> the failing sites.
>>
>
> Some saintly person should go through the failures and add to
> decompilerFailures all those that fail because of known limitations in the
> decompiler. These limitations are things like
>
> Floating point reader precision, e.g. the source specifies more precision
> than is supported and so the compilation of the decompilation is slightly
> different.
>
> Unreachable statements. e.g. foo ifTrue: [^bar] ifFalse: [^baz]. ^huh?
> decompiles as foo ifTrue: [^bar] ifFalse: [^baz]. and so ^'huh?' is missing.
>
> Null statements. e.g. foo ifTrue: []. decompiles as foo. which, if foo is
> an inst var, compiles as empty.
>
> These are limitations in the decompiler we simply have to tolerate.
> decompilerFailures is a list that documents the reasons for failure.
>
> Of course another, also valid, approach is to fix the code in the failures
> so that they don't contain unreachable or null statements.

What about the "real" bugs when the decompiler cannot reproduce the
declaration of temporiaries (for example EventSensor >> #eventTickler)?

Levente

>
> HTH
> Eliot
>
>
>
>
>
>>
>> Cheers,
>> - Andreas
>>
>>
>

Frank Shearar

Re: testrun results

In reply to this post by Eliot Miranda-2

Eliot Miranda wrote:

>
>
> On Mon, Apr 5, 2010 at 10:37 AM, Andreas Raab <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> On 4/5/2010 4:04 AM, Levente Uzonyi wrote:
>
> It's a decompiler bug. Eliot has fixes for these issues but I
> don't know
> if he has time to add them to the Trunk.
>
>
> I just spoke with Eliot and it seems unlikely that he'll have much
> time to work on this. Also, there are cases that the decompiler
> handles correctly, but differently from the compiler.
>
> Eliot's recommendation to extent DecompilerTests>>decompilerFailures
> with the failing sites.
>
>
> Some saintly person should go through the failures and add to
> decompilerFailures all those that fail because of known limitations in
> the decompiler. These limitations are things like
>
> Floating point reader precision, e.g. the source specifies more
> precision than is supported and so the compilation of the decompilation
> is slightly different.
>
> Unreachable statements. e.g. foo ifTrue: [^bar] ifFalse: [^baz]. ^huh?
> decompiles as foo ifTrue: [^bar] ifFalse: [^baz]. and so ^'huh?' is missing.
>
> Null statements. e.g. foo ifTrue: []. decompiles as foo. which, if foo
> is an inst var, compiles as empty.
>
> These are limitations in the decompiler we simply have to tolerate.
> decompilerFailures is a list that documents the reasons for failure.
>
> Of course another, also valid, approach is to fix the code in the
> failures so that they don't contain unreachable or null statements.

Done, in the form of Tests-fbs.68.mcz. The errors that I looked into (as
detailed in a previous mail) were mostly harmless: variables looked to
be named in a slightly different order. Because we check the old code
string against the new code string we get a false negative: a test that
fails but the behaviour's correct.

At worst, I hope my work at least narrows down the field of inquiry for
someone else.

frank

Eliot Miranda-2

Re: testrun results

In reply to this post by Levente Uzonyi-2

On Mon, Apr 5, 2010 at 12:04 PM, Levente Uzonyi <[hidden email]> wrote:

On Mon, 5 Apr 2010, Eliot Miranda wrote:

On Mon, Apr 5, 2010 at 10:37 AM, Andreas Raab <[hidden email]> wrote:

On 4/5/2010 4:04 AM, Levente Uzonyi wrote:

It's a decompiler bug. Eliot has fixes for these issues but I don't know
if he has time to add them to the Trunk.

I just spoke with Eliot and it seems unlikely that he'll have much time to
work on this. Also, there are cases that the decompiler handles correctly,
but differently from the compiler.

Eliot's recommendation to extent DecompilerTests>>decompilerFailures with
the failing sites.

Some saintly person should go through the failures and add to
decompilerFailures all those that fail because of known limitations in the
decompiler. These limitations are things like

Floating point reader precision, e.g. the source specifies more precision
than is supported and so the compilation of the decompilation is slightly
different.

Unreachable statements. e.g. foo ifTrue: [^bar] ifFalse: [^baz]. ^huh?
decompiles as foo ifTrue: [^bar] ifFalse: [^baz]. and so ^'huh?' is missing.

Null statements. e.g. foo ifTrue: []. decompiles as foo. which, if foo is
an inst var, compiles as empty.

These are limitations in the decompiler we simply have to tolerate.
decompilerFailures is a list that documents the reasons for failure.

Of course another, also valid, approach is to fix the code in the failures
so that they don't contain unreachable or null statements.

What about the "real" bugs when the decompiler cannot reproduce the declaration of temporiaries (for example EventSensor >> #eventTickler)?

There are two tasks here. One is to collect the set of genuine decompiler failures. The other is to compare the current trunk code with my Teleplace decompiler state and do a merge, hopefully eliminating some of the genuine failures. Unfortunately I don't have any space cycles right now. So if you're interested I can supply the Teleplace decompiler code to anyone that wants to take a look.

best

Eliot

Levente

HTH
Eliot

Cheers,
- Andreas

Juan Vuletich-4

Re: testrun results

Eliot Miranda wrote:

>
>
> On Mon, Apr 5, 2010 at 12:04 PM, Levente Uzonyi <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> ...
> What about the "real" bugs when the decompiler cannot reproduce
> the declaration of temporiaries (for example EventSensor >>
> #eventTickler)?
>
>
> There are two tasks here. One is to collect the set of genuine
> decompiler failures. The other is to compare the current trunk code
> with my Teleplace decompiler state and do a merge, hopefully
> eliminating some of the genuine failures. Unfortunately I don't have
> any space cycles right now. So if you're interested I can supply the
> Teleplace decompiler code to anyone that wants to take a look.
>
> best
> Eliot

Hi Eliot,

I'd love to receive that code and integrate it into Cuis. Thanks!

If someone wants to do the same for Squeak, we can compare notes.

Cheers,
Juan Vuletich

Nicolas Cellier

Re: testrun results

2010/4/6 Juan Vuletich <[hidden email]>:

> Eliot Miranda wrote:
>>
>>
>> On Mon, Apr 5, 2010 at 12:04 PM, Levente Uzonyi <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> ...
>> What about the "real" bugs when the decompiler cannot reproduce
>> the declaration of temporiaries (for example EventSensor >>
>> #eventTickler)?
>>
>>
>> There are two tasks here. One is to collect the set of genuine decompiler
>> failures. The other is to compare the current trunk code with my Teleplace
>> decompiler state and do a merge, hopefully eliminating some of the genuine
>> failures. Unfortunately I don't have any space cycles right now. So if
>> you're interested I can supply the Teleplace decompiler code to anyone that
>> wants to take a look.
>>
>> best
>> Eliot
>
> Hi Eliot,
>
> I'd love to receive that code and integrate it into Cuis. Thanks!
>
> If someone wants to do the same for Squeak, we can compare notes.
>
> Cheers,
> Juan Vuletich
>

If no other hand up, I can try and integrate in squeak (maybe end of this week).
Eliot, can you send me a copy at this gmail address ?

Nicolas