bugs loading GL extensions

I keep running into a problem when a Croquet world starts up: a call to
glLoadTransposeMatrixf produces a does not understand.  This appears to
be by design; methods are defined on the class side and then faulted in
by OGLExtManager>>doesNotUnderstand:.  It looks as if all methods are
loaded whenever any of them hits, but I'm not sure.  OGLExtManager
attempts to load the extension, but in OpelGL>>extensions
(self glGetString: GLExtensions) fromCString
fails because the value returned by glGetString: is (apparently) not a
legitimate CString.  I think it's null; the ExternData handle is @
16r00000000.

In at least one previous report I attempted manuallying editing the
references to glLoadTransposeMatrixf, as suggested by squeak bug 6338,
but I have not done so here.  My libGL.so appears to have both
glLoadTransposeMatrixf and glLoadTransposeMatrixfARB, so I don't think
it's a simple case of the functions not being there.

This is on Linux with cobalt20081125 + some freecad loads.

29 December 2008 5:26:51 pm

VM: unix - a SmalltalkImage
Image: Croquet1.0beta [latest update: #2]

SecurityManager state:
Restricted: false
FileAccess: true
SocketAccess: true
Working Dir /usr/local/src/CAD/cobalt-base-current-build-20081125
Trusted Dir /usr/local/src/CAD/cobalt-base-current-build-20081125/secure
Untrusted Dir /usr/local/src/CAD/cobalt-base-current-build-20081125/My
Squeak

ExternalAddress(Object)>>error:
        Receiver: @ 16r00000000
        Arguments and temporary variables:
                aString: 'a primitive has failed'
        Receiver's instance variables:
@ 16r00000000

ExternalAddress(Object)>>primitiveFailed
        Receiver: @ 16r00000000
        Arguments and temporary variables:

        Receiver's instance variables:
@ 16r00000000

ExternalAddress(ByteArray)>>integerAt:size:signed:
        Receiver: @ 16r00000000
        Arguments and temporary variables:
                byteOffset: 1
                nBytes: 1
                aBoolean: false
        Receiver's instance variables:
@ 16r00000000

ExternalAddress(ByteArray)>>unsignedByteAt:
        Receiver: @ 16r00000000
        Arguments and temporary variables:
                byteOffset: 1
        Receiver's instance variables:
@ 16r00000000


--- The full stack ---
ExternalAddress(Object)>>error:
ExternalAddress(Object)>>primitiveFailed
ExternalAddress(ByteArray)>>integerAt:size:signed:
ExternalAddress(ByteArray)>>unsignedByteAt:
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ExternalAddress(ByteArray)>>unsignedCharAt:
ExternalData>>fromCString
OGLUnixX11LE(OpenGL)>>extensions
*OGLExtManager(OGLExtManager)>>loadExtension:
*OGLExtManager(OGLExtManager)>>doesNotUnderstand:
OGLUnixX11LE(OpenGL)>>glLoadTransposeMatrixf:
TSpace>>renderSpace:port:depth:ghostFrame:
TSpace>>renderSpace:
TCamera>>renderView:space:overlay:
TSpace(TFrame)>>renderView:overlay:
[] in TFarRef>>syncSend:withArguments: {[result := myValue perform:
selector withArguments: args]}
BlockContext>>ensure:
TFarRef>>syncSend:withArguments:
TFarRef>>syncSend:
TMessageMaker>>doesNotUnderstand:
[] in TFarRef>>renderView:overlay: {[:vp | vp renderView: ogl overlay:
ov]}
TFarRef>>send:
TFarRef>>renderView:overlay:
TPortal>>renderView:with:
[] in TFarRef>>syncSend:withArguments: {[result := myValue perform:
selector withArguments: args]}
BlockContext>>ensure:
TFarRef>>syncSend:withArguments:
TFarRef>>syncSend:
TMessageMaker>>doesNotUnderstand:
[] in MyHarness(CroquetHarness)>>renderWorld {[:vp | vp renderView: ogl
with: {systemOverlayPortal}]}
TFarRef>>send:
[] in MyHarness(CroquetHarness)>>renderWorld {[readyToRender wait.
randomList := (controllers     select: [:cc | cc islan...]}
TMutexSet>>pvtCritical:startingAt:
[] in TMutexSet>>pvtCritical:startingAt: {[self pvtCritical: aBlock
startingAt: index + 1]}
BlockContext>>ensure:
[] in TMutex>>critical: {[owner := activeProcess.  aBlock   ensure:
[owner := nil]]}
[] in Semaphore>>critical: {[blockValue := mutuallyExcludedBlock value]}
BlockContext>>ensure:
Semaphore>>critical:
TMutex>>critical:
TMutexSet>>pvtCritical:startingAt:
[] in TMutexSet>>pvtCritical:startingAt: {[self pvtCritical: aBlock
startingAt: index + 1]}
BlockContext>>ensure:
[] in TMutex>>critical: {[owner := activeProcess.  aBlock   ensure:
[owner := nil]]}
[] in Semaphore>>critical: {[blockValue := mutuallyExcludedBlock value]}
BlockContext>>ensure:
Semaphore>>critical:
TMutex>>critical:
TMutexSet>>pvtCritical:startingAt:
TMutexSet>>critical:
MyHarness(CroquetHarness)>>renderWorld
[] in MyHarness(CroquetHarness)>>renderProcess {[self renderWorld]}
BlockContext>>ensure:
MyHarness(CroquetHarness)>>renderProcess
[] in MyHarness(Object)>>fork:at: {[self perform: aSelector]}
[] in BlockContext>>newProcess {[self value.  Processor
terminateActive]}

OGLUnixX11LE(OpenGL)>>glGetString:
says it is an automatically generated method, with
<apicall: byte* 'glGetString' (ulong) module: 'opengl32.dll'>

Since my system uses libGL.so, not dll's, that doesn't look as if it should
work.

OGLUnix* should't be trying to load a windows dll.  I don't understand the
automatic generation process enough to know how it should interact with the
class hieararchy--i.e., whether changing something down in OGLUnix* would
actually make a difference.

OGLUnix>>openGLLibraryName does return 'GL'.

P.S. I wish I could get copy and paste to work between croquet and the rest of
my desktop.

When a Croquet image starts up, it munges the compiled code for the
auto-generated OpenGL methods to point to the appropriate library for
the current platform.  In other words, the compiled code is slightly
different from what the source code would compile to.

This mainly happens in OpenGL#privateInstallLibrary:, with helper
methods such as #openGLLibraryName.

Cheers,
Josh


[hidden email] wrote:

On 30.12.2008, at 03:04, [hidden email] wrote:

> P.S. I wish I could get copy and paste to work between croquet and  
> the rest of
> my desktop.


Should work - use Alt-X/C/V (or enable either  
duplicateControlAndAltKeys or swapControlAndAltKeys).

- Bert -

On 30.12.2008, at 02:39, [hidden email] wrote:

> I keep running into a problem when a Croquet world starts up: a call  
> to
> glLoadTransposeMatrixf produces a does not understand.  This appears  
> to
> be by design; methods are defined on the class side and then faulted  
> in
> by OGLExtManager>>doesNotUnderstand:.  It looks as if all methods are
> loaded whenever any of them hits, but I'm not sure.

This part of Croquet uses serious meta programming magic. It is  
actually using an instance of an anonymous subclass of what appears to  
get used, and the dosNotUnderstand mechanism triggers on-the-fly  
extension loading and installing of extension methods in that  
subclass, so that on the second call the method is actually there and  
does not trigger the DNU again.

- Bert -

On Tue, 2008-12-30 at 01:29 -0800, Josh Gargus wrote: That's consistent with the byte code.  That eliminates one theory.

So what could be causing glGetString to return a null pointer?

I checked open files for the process:
$ lsof -p 30945 | grep GL
squeak  30945 ross  mem    REG      254,2   397196
12304 /usr/lib/nvidia/libGL.so.1.2.xlibmesa
squeak  30945 ross  mem    REG      254,2 13886208
45121 /usr/lib/libGLcore.so.173.14.09
squeak  30945 ross  mem    REG      254,2   671684
45120 /usr/lib/libGL.so.173.14.09

Maybe the fact that it's got 2 libGL's open is causing trouble?  My
packaging system says that even /usr/lib/libGL.so.173.14.09 is provided
by nvidia.

[hidden email] wrote:
> I keep running into a problem when a Croquet world starts up: a call to
> glLoadTransposeMatrixf produces a does not understand.  This appears to
> be by design; methods are defined on the class side and then faulted in
> by OGLExtManager>>doesNotUnderstand:.  It looks as if all methods are
> loaded whenever any of them hits, but I'm not sure.

That is correct. All the functions for a particular extension are loaded
when the extension is loaded.

> OGLExtManager
> attempts to load the extension, but in OpelGL>>extensions
> (self glGetString: GLExtensions) fromCString
> fails because the value returned by glGetString: is (apparently) not a
> legitimate CString.  I think it's null; the ExternData handle is @
> 16r00000000.

Yes, that would be NULL. It's odd though - I didn't think this function
can fail at all in that way.

Cheers,
   - Andreas

[hidden email] wrote:
> OGLUnixX11LE(OpenGL)>>glGetString:
> says it is an automatically generated method, with
> <apicall: byte* 'glGetString' (ulong) module: 'opengl32.dll'>
>
> Since my system uses libGL.so, not dll's, that doesn't look as if it should
> work.

That's okay. The module gets initialized in OpenGL>>privateInstallLibrary:.

> P.S. I wish I could get copy and paste to work between croquet and the rest of
> my desktop.

Go to SystemDictionary>>setPlatformPreferences and change the line saying:

        platform = 'unix' ifTrue:[
                specs := #(
                                        (usePrimitiveClipboard false)  "would otherwise cause VM crash"
                                )].

Then, save your image. After the next restart you should be able to
copy/paste between the OS and your image.

Cheers,
   - Andreas

On 30.12.2008, at 18:32, Andreas Raab wrote:


Do we have reason to believe OpenGL works correctly on this machine?

- Bert -

> > [hidden email] wrote:
> > > OGLUnixX11LE(OpenGL)>>glGetString:
> > > says it is an automatically generated method, with
> > > <apicall: byte* 'glGetString' (ulong) module: 'opengl32.dll'>
According to http://wiki.squeak.org/squeak/2426 apicall doesn't work on
Unix:
Using 'apicall:' instead of 'cdecl:' indicates a different calling
convention. I haven't been able to determine the actual intended
difference. On Macintosh, the two forms are equivalent. On Unix, only
'cdecl:' is supported. I can't find the Windows source code, so I don't
know what it does. [Jörn Eyrich: apicall specifies the Pascal calling
convention (push arguments on stack left to right, callee removes
arguments) which seems to be standard on Windows, and cdecl secifies the
C calling conventions (push right to left, caller removes)]

However, this has worked intermittently for me with other images.

On Tue, 2008-12-30 at 09:36 -0800, Andreas Raab wrote: Thanks; that worked.  I assume you meant change to usePrimitiveClipboard
true. The comment after is little worrying, though...
Ross

Ross Boylan wrote:
> Thanks; that worked.  I assume you meant change to usePrimitiveClipboard
> true. The comment after is little worrying, though...

The comment applies to headless operations. When you launch the VM
-headless and connect via VNC for example, using Alt-C/X/P will make the
VM explode.

Cheers,
   - Andreas

I'm not sure if the question is does OpenGL work for other applications on my
machine--I think so--or whether OpenGL and squeak are compatible (perhaps even
whether external C calls are working in general).  For the latter, there's some
evidence for a "No" answer.

In an earlier post I cited a page saying that <apicall:...> didn't work on
Unix.

On 03.01.2009, at 00:12, [hidden email] wrote:

> I'm not sure if the question is does OpenGL work for other  
> applications on my
> machine--I think so

Can you attach the output of glxinfo?

> --or whether OpenGL and squeak are compatible (perhaps even
> whether external C calls are working in general).  For the latter,  
> there's some
> evidence for a "No" answer.

FFI works, you would have got another error otherwise.

> In an earlier post I cited a page saying that <apicall:...> didn't  
> work on
> Unix.

And as mentioned previously this is a red herring - the FFI  
declaration is replaced at runtime with the one appropriate for your  
system.

- Bert -

On Sat, 2009-01-03 at 13:41 +0100, Bert Freudenberg wrote:
> On 03.01.2009, at 00:12, [hidden email] wrote:
>
> > I'm not sure if the question is does OpenGL work for other  
> > applications on my
> > machine--I think so
>
> Can you attach the output of glxinfo?
>
Attached.

glxgears gives around 2500 FPS.

Ross

On Sat, 2009-01-03 at 13:41 +0100, Bert Freudenberg wrote:
> On 03.01.2009, at 00:12, [hidden email] wrote:
>
...
>
> > In an earlier post I cited a page saying that <apicall:...> didn't  
> > work on
> > Unix.
>
> And as mentioned previously this is a red herring - the FFI  
> declaration is replaced at runtime with the one appropriate for your  
> system.
>
The previous discussion concerned the name of the library, not the use
of apicall.  The bytecode of OpenGL>>glGetString: shows as
<apicall: byte* 'glGetString' (ulong) module: 'GL'>
21 <70> self
22 <D1> send: externalCallFailed
23 <7C> returnTop

This is after the failure, so any rewriting should have been done.
module: 'opengl32.dll' in the source has been changed to module: 'GL'
above.  Also, should it be 'GL' or 'GL.so', 'libGL.so' or some other
variant?

Actually on unix if the GL library is not found it automatically looks  
for libGL, GL.so, GL.dylib, libGL.so, libGL.dylib  so you don't need  
to code that in the module name,
mind extra work then is required by the VM to guess what the proper  
name is.

It also looks in the SQUEAK_PLUGIN_PATH, VM_LIBDIR, LD_LIBRARY_PATH,  
working directory, maybe the VM_X11DIR. oh and likely some other places.

On 3-Jan-09, at 12:51 PM, Ross Boylan wrote:

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================