Optimization bug in Pharo 64-bit Mac VM [was BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)]

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Optimization bug in Pharo 64-bit Mac VM [was BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)]

Eliot Miranda-2
 
Hi All, but especially Nicolas and C experts,

    if you build a Mac 64-bit Pharo VM in production and assert configurations and run the below

wget files.pharo.org/get-files/60/pharo64.zip
wget files.pharo.org/get-files/60/sources.zip
unzip pharo64.zip
unzip sources.zip
pharo64cfvm Pharo64-60438.image eval "VGTigerDemo runDemo" &
pharo64cavm Pharo64-60438.image eval "VGTigerDemo runDemo"

you'll see that in the Assert VM the screen is clean and the tiger is displayed within a white rectangle, but in the production VM there are lots of garbage pixels at the bottom and bottom right of the screen.  So somewhere in the display pipeline we have an optimization-related bug.  The Assert VM is optimized at -O1 and the production VM at -Os (favour space over time).

On Fri, Mar 10, 2017 at 12:13 PM, Eliot Miranda <[hidden email]> wrote:
Hi Esteban,

    first of all hanks so much for the instructions.  It's so nice to just run the commands below and be able to reproduce immediately instead of having to root around through build servers, following links.  Thanks, it saves me a huge amount of time.  i really appreciate the effort.

Second, I find I can make the primitive fail much sooner if I pick up the main window and move it.  I was running it on one machine, wanted to run it on another with a different VM but the main window was in the way of the command.  So I moved the main window so I could read the command line and lo and behold it failed immediately.  So if I start the demo,and then pick up the window and move it, the primitive fails within very few seconds.  That could be a clue.

On Fri, Mar 10, 2017 at 7:35 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I’m tumbling into an error in Pharo, because we use callbacks intensively, in Athens(cairo)-to-World conversion in particular, and people is sending always their crash reports… we made the whole conversion a lot more robust since problems started to arise, but now I hit a wall I cannot solve: I think problem is in something in callbacks.

And problem is showing very easy on 64bits (while in 32bits it takes time and is more random).

Here is the easiest way to reproduce it (in mac):

wget files.pharo.org/get-files/60/pharo64-mac-latest.zip
wget files.pharo.org/get-files/60/pharo64.zip
wget files.pharo.org/get-files/60/sources.zip
unzip pharo64-mac-latest.zip
unzip pharo64.zip
unzip sources.zip
./Pharo.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval "VGTigerDemo runDemo"

eventually (like 5-6 seconds after, if not immediately), you will have a stack like this:

SmallInteger(Object)>>primitiveFailed:
SmallInteger(Object)>>primitiveFailed
SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext:
GrafPort>>copyBits
GrafPort>>image:at:sourceRect:rule:
FormCanvas>>image:at:sourceRect:rule:
FormCanvas(Canvas)>>drawImage:at:sourceRect:
FormCanvas(Canvas)>>drawImage:at:
VGTigerDemo>>runDemo
VGTigerDemo class>>runDemo
UndefinedObject>>DoIt
OpalCompiler>>evaluate
OpalCompiler(AbstractCompiler)>>evaluate:
[ result := Smalltalk compiler evaluate: aStream.
self hasSessionChanged
        ifFalse: [ self stdout
                        print: result;
                        lf ] ] in EvaluateCommandLineHandler>>evaluate: in Block: [ result := Smalltalk compiler evaluate: aStream....
BlockClosure>>on:do:
EvaluateCommandLineHandler>>evaluate:
EvaluateCommandLineHandler>>evaluateArguments
EvaluateCommandLineHandler>>activate
EvaluateCommandLineHandler class(CommandLineHandler class)>>activateWith:
[ aCommandLinehandler activateWith: commandLine ] in PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: in Block: [ aCommandLinehandler activateWith: commandLine ]
BlockClosure>>on:do:
PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand:
PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand
PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument:
[ self
        handleArgument:
                (self arguments
                        ifEmpty: [ '' ]
                        ifNotEmpty: [ :arguments | arguments first ]) ] in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in Block: [ self...
BlockClosure>>on:do:
PharoCommandLineHandler(BasicCommandLineHandler)>>activate
PharoCommandLineHandler>>activate
PharoCommandLineHandler class(CommandLineHandler class)>>activateWith:
[ super activateWith: aCommandLine ] in PharoCommandLineHandler class>>activateWith: in Block: [ super activateWith: aCommandLine ]

Any idea?

thanks!
Esteban



--
_,,,^..^,,,_
best, Eliot



--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Optimization bug in Pharo 64-bit Mac VM [was BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)]

Nicolas Cellier
 


2017-03-10 21:23 GMT+01:00 Eliot Miranda <[hidden email]>:
 
Hi All, but especially Nicolas and C experts,

    if you build a Mac 64-bit Pharo VM in production and assert configurations and run the below


Or curl -O if you don't want to bother installing wget

unzip pharo64.zip
unzip sources.zip
pharo64cfvm Pharo64-60438.image eval "VGTigerDemo runDemo" &
pharo64cavm Pharo64-60438.image eval "VGTigerDemo runDemo"

you'll see that in the Assert VM the screen is clean and the tiger is displayed within a white rectangle, but in the production VM there are lots of garbage pixels at the bottom and bottom right of the screen.  So somewhere in the display pipeline we have an optimization-related bug.  The Assert VM is optimized at -O1 and the production VM at -Os (favour space over time).

Nope.
The first two times I tried, there were the same artefacts with assert vm than with fast vm...

../Smalltalk/OpenSmalltalk/opensmalltalk-vm/build.macos64x64/pharo.cog.spur/PharoAssert.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval "VGTigerDemo runDemo"

lisp < lifp 22113

lifp > lisp 22113
a VGTigerDemo

The 3rd time, the artefact had disappeared...


Then I tried moving the window and I could reproduce the failure.

../Smalltalk/OpenSmalltalk/opensmalltalk-vm/build.macos64x64/pharo.cog.spur/PharoAssert.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval "VGTigerDemo runDemo"

lisp < lifp 22113

lifp > lisp 22113
PrimitiveFailed: primitive #primSignal:andReturnAs:fromContext: in SmallInteger failed

(ReceiverIndex + ((sp >> 3))) < (lengthOf(aContext)) 63951
SmallInteger(Object)>>primitiveFailed:
SmallInteger(Object)>>primitiveFailed
SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext:
GrafPort>>copyBits
GrafPort>>image:at:sourceRect:rule:
...snip...

(ReceiverIndex + ((sp >> 3))) < (lengthOf(aContext)) 63951

(ReceiverIndex + ((sp >> 3))) < (lengthOf(aContext)) 63951

(ReceiverIndex + ((sp >> 3))) < (lengthOf(aContext)) 63951

(ReceiverIndex + ((sp >> 3))) < (lengthOf(aContext)) 63951

(ReceiverIndex + ((sp >> 3))) < (lengthOf(aContext)) 63951



Then I restarted the test, saw the artefacts, stopped the demo, resize the window, saved the image.
Now I got the PrimitiveFailed: primitive #primSignal:andReturnAs:fromContext: in SmallInteger failed at each start (before the demo starts).

If I start the saved image without eval demo, then it starts normally --- errr, exept it displays nothing (light gray background) untill I click inside.

Ah not allways...

I also get the artefacts with pharo.stack.spur, but the first time I tried, the space outer white demo square was black.

It looks to much like some sort of buffer overrun. Then anything can happen.


 
On Fri, Mar 10, 2017 at 12:13 PM, Eliot Miranda <[hidden email]> wrote:
Hi Esteban,

    first of all hanks so much for the instructions.  It's so nice to just run the commands below and be able to reproduce immediately instead of having to root around through build servers, following links.  Thanks, it saves me a huge amount of time.  i really appreciate the effort.

Second, I find I can make the primitive fail much sooner if I pick up the main window and move it.  I was running it on one machine, wanted to run it on another with a different VM but the main window was in the way of the command.  So I moved the main window so I could read the command line and lo and behold it failed immediately.  So if I start the demo,and then pick up the window and move it, the primitive fails within very few seconds.  That could be a clue.

On Fri, Mar 10, 2017 at 7:35 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

I’m tumbling into an error in Pharo, because we use callbacks intensively, in Athens(cairo)-to-World conversion in particular, and people is sending always their crash reports… we made the whole conversion a lot more robust since problems started to arise, but now I hit a wall I cannot solve: I think problem is in something in callbacks.

And problem is showing very easy on 64bits (while in 32bits it takes time and is more random).

Here is the easiest way to reproduce it (in mac):

wget files.pharo.org/get-files/60/pharo64-mac-latest.zip
wget files.pharo.org/get-files/60/pharo64.zip
wget files.pharo.org/get-files/60/sources.zip
unzip pharo64-mac-latest.zip
unzip pharo64.zip
unzip sources.zip
./Pharo.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval "VGTigerDemo runDemo"

eventually (like 5-6 seconds after, if not immediately), you will have a stack like this:

SmallInteger(Object)>>primitiveFailed:
SmallInteger(Object)>>primitiveFailed
SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext:
GrafPort>>copyBits
GrafPort>>image:at:sourceRect:rule:
FormCanvas>>image:at:sourceRect:rule:
FormCanvas(Canvas)>>drawImage:at:sourceRect:
FormCanvas(Canvas)>>drawImage:at:
VGTigerDemo>>runDemo
VGTigerDemo class>>runDemo
UndefinedObject>>DoIt
OpalCompiler>>evaluate
OpalCompiler(AbstractCompiler)>>evaluate:
[ result := Smalltalk compiler evaluate: aStream.
self hasSessionChanged
        ifFalse: [ self stdout
                        print: result;
                        lf ] ] in EvaluateCommandLineHandler>>evaluate: in Block: [ result := Smalltalk compiler evaluate: aStream....
BlockClosure>>on:do:
EvaluateCommandLineHandler>>evaluate:
EvaluateCommandLineHandler>>evaluateArguments
EvaluateCommandLineHandler>>activate
EvaluateCommandLineHandler class(CommandLineHandler class)>>activateWith:
[ aCommandLinehandler activateWith: commandLine ] in PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: in Block: [ aCommandLinehandler activateWith: commandLine ]
BlockClosure>>on:do:
PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand:
PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand
PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument:
[ self
        handleArgument:
                (self arguments
                        ifEmpty: [ '' ]
                        ifNotEmpty: [ :arguments | arguments first ]) ] in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in Block: [ self...
BlockClosure>>on:do:
PharoCommandLineHandler(BasicCommandLineHandler)>>activate
PharoCommandLineHandler>>activate
PharoCommandLineHandler class(CommandLineHandler class)>>activateWith:
[ super activateWith: aCommandLine ] in PharoCommandLineHandler class>>activateWith: in Block: [ super activateWith: aCommandLine ]

Any idea?

thanks!
Esteban



--
_,,,^..^,,,_
best, Eliot



--
_,,,^..^,,,_
best, Eliot