OpenGLES2 binding available!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

OpenGLES2 binding available!

Thibault Raffaillac
Hi all,

My tiny binding for OpenGLES2 is ready :)
http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/

It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.

A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
Tested only on Mac OSX, patches welcome!

Cheers,
Thibault

Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

stepharo

Thanks for publishing it.

I would love to see an athens back-end for it.


Stef


Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :
Hi all,

My tiny binding for OpenGLES2 is ready :)
http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/

It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.

A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
Tested only on Mac OSX, patches welcome!

Cheers,
Thibault


Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

stepharo
In reply to this post by Thibault Raffaillac
Hi thibault


I took some times to look at your project. Nice.

Now the GLW example is not working.

I understand that this is a binding. I do not want to have to work that
low level.

I'm curious to see what people will be able to build on top of it.


Stef


Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :

> Hi all,
>
> My tiny binding for OpenGLES2 is ready :)
> http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/
>
> It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.
>
> A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
> Tested only on Mac OSX, patches welcome!
>
> Cheers,
> Thibault
>
>


Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

abergel
In reply to this post by stepharo

I would love to see an athens back-end for it.

Isn't it what Ronie is working on?

Alexandre


Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :
Hi all,

My tiny binding for OpenGLES2 is ready :)
http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/

It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.

A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
Tested only on Mac OSX, patches welcome!

Cheers,
Thibault



Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

Ronie Salgado
Isn't it what Ronie is working on?
I am not working with OpenGL or OpenGL ES anymore. I am moving all of my efforts into Woden 2, the AbstractGPU (an abstraction layer for Vulkan, Direct3D 12 and Metal), and Dastrel (Data Stream Language) a custom shader language whose compiler I implemented in Pharo, with a backend for SpirV, GLSL and C++. I still have to implement the backends for Metal and HLSL.

Thibault is interested more on being able to use OpenGL ES 2.0 directly in the playground, for teaching purposes without any abstraction layer underlying. I do not agree with many of his approaches, too low level, and many of themp do not work well with Pharo VM. One of the biggest issue of OpenGL is related with vsync, which is an operation that can block all of the Pharo processes, until we get a VM with the threaded FFI, which will have in the near future. It will use the UnifiedFFI API, so we will not have to further changes to take advantage of the threaded ffi in the user code.

A far worse problem with OpenGL is that the current OpenGL context is a thread local variable, which do not interact at all with Pharo processes. This is the reason of why I added the OSWindowRenderThread class for serializing OSWindow and OpenGL animations. SDL2 renderers are usually implemented using OpenGL. They are two correct solution for this problem:
- Having a VM that maps each Pharo Process into an operating system thread, 
- Not using OpenGL at all.

The others low level graphics API (Vulkan, Direct 3D and Metal) do not rely on a global thread local storage variable. These APIs are object oriented, so handles are passed in each API call.

Currently, I have an Athens backend above Woden 2, which is only supporting Vulkan for now. Since this backend is for Woden 2, it has some dependencies in the Woden 2 Math and the Lowcode NativeStructure. This means that the Woden 2 Athens backend has a strong dependency in the the modified OpalCompiler that uses the Lowcode extended byte code set, and the the modified PharoVM that implements these extra bytecode in the just in time compiler.

Anyway, implementing a Athens backend from scratch is a really hard problem, because paths can be concave, curve and self intersecting. Drawing a path, unlike drawing a triangle is a global operation, for which GPUs were not designed for. The easiest and most robust technique for drawing a path using the GPU is to flatten the path using De Casteljau's algorithm and then using the stencil buffer technique for drawing concave polygons that was described in the OpenGL red book ( http://www.glprogramming.com/red/chapter14.html#name13 ). The problem of this technique is that involves many state changes(glStencil*)  and drawing commands(glDrawElements) per path, which are the most expensive operations when using OpenGL. The alternative is to perform a triangulation in the CPU of paths, or to use one of the newer low level APIs such as Vulkan and Direct 3D 12 which were designed to reduce or eliminate the cost of state changes and drawing commands.

Path stroking is an even harder problem, which I have not solved yet. The standard approach is to convert a stroked path into a path that can be filled. The problem is that bezier offset curves are not bezier curves, so they have to be approximated. Fortunately, since this operation is backend agnostic, it can be used and reused by many Athens backends. Since stroking is not critical for my needs, I do not think I will implement it soon properly. Currently I am just drawing a 1 pixel thick line, which can look horrible, but is better than nothing. One algorithm that seems to be simple to implement is one described in a paper from the Cairo people, which uses the Minkowksi  Sum between the outline of the path to be stroked, and shape of the pen that is stroking

Soon I will merge the Lowcode instructions into the SqueakVM/Cog VM to integrate these bytecodes into the mainline VM.

I will not publish Woden 2 until the dependencies and Woden 2 itself can be loaded easily.  Soon I will publish properly the Dastrel shader language and the Slovim backend. Slovim name comes from Smalltalk Low levelish VIrtual Machine, an intermediate representation heavily modeled after the one used in LLVM, implemented in Pharo, some very basic optimizations(basic constant folding, inlining and some dead code generation), and a code generator backend for SpirV, GLSL and C++. My biggest issue before publishing this compiler are:
- Implementing a command line interface.
- Doing an easy to install and use package for Linux, OS X and Windows.
- Documenting the language and the compiler.

Best regards,
Ronie

2016-07-28 19:31 GMT+02:00 Alexandre Bergel <[hidden email]>:

I would love to see an athens back-end for it.

Isn't it what Ronie is working on?

Alexandre


Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :
Hi all,

My tiny binding for OpenGLES2 is ready :)
http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/

It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.

A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
Tested only on Mac OSX, patches welcome!

Cheers,
Thibault




Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

stepharo

Thanks ronie for your analysis (especially the thread local point).

Stef


Le 28/7/16 à 20:36, Ronie Salgado a écrit :
Isn't it what Ronie is working on?
I am not working with OpenGL or OpenGL ES anymore. I am moving all of my efforts into Woden 2, the AbstractGPU (an abstraction layer for Vulkan, Direct3D 12 and Metal), and Dastrel (Data Stream Language) a custom shader language whose compiler I implemented in Pharo, with a backend for SpirV, GLSL and C++. I still have to implement the backends for Metal and HLSL.

Thibault is interested more on being able to use OpenGL ES 2.0 directly in the playground, for teaching purposes without any abstraction layer underlying. I do not agree with many of his approaches, too low level, and many of themp do not work well with Pharo VM. One of the biggest issue of OpenGL is related with vsync, which is an operation that can block all of the Pharo processes, until we get a VM with the threaded FFI, which will have in the near future. It will use the UnifiedFFI API, so we will not have to further changes to take advantage of the threaded ffi in the user code.

A far worse problem with OpenGL is that the current OpenGL context is a thread local variable, which do not interact at all with Pharo processes. This is the reason of why I added the OSWindowRenderThread class for serializing OSWindow and OpenGL animations. SDL2 renderers are usually implemented using OpenGL. They are two correct solution for this problem:
- Having a VM that maps each Pharo Process into an operating system thread, 
- Not using OpenGL at all.

The others low level graphics API (Vulkan, Direct 3D and Metal) do not rely on a global thread local storage variable. These APIs are object oriented, so handles are passed in each API call.

Currently, I have an Athens backend above Woden 2, which is only supporting Vulkan for now. Since this backend is for Woden 2, it has some dependencies in the Woden 2 Math and the Lowcode NativeStructure. This means that the Woden 2 Athens backend has a strong dependency in the the modified OpalCompiler that uses the Lowcode extended byte code set, and the the modified PharoVM that implements these extra bytecode in the just in time compiler.

Anyway, implementing a Athens backend from scratch is a really hard problem, because paths can be concave, curve and self intersecting. Drawing a path, unlike drawing a triangle is a global operation, for which GPUs were not designed for. The easiest and most robust technique for drawing a path using the GPU is to flatten the path using De Casteljau's algorithm and then using the stencil buffer technique for drawing concave polygons that was described in the OpenGL red book ( http://www.glprogramming.com/red/chapter14.html#name13 ). The problem of this technique is that involves many state changes(glStencil*)  and drawing commands(glDrawElements) per path, which are the most expensive operations when using OpenGL. The alternative is to perform a triangulation in the CPU of paths, or to use one of the newer low level APIs such as Vulkan and Direct 3D 12 which were designed to reduce or eliminate the cost of state changes and drawing commands.

Path stroking is an even harder problem, which I have not solved yet. The standard approach is to convert a stroked path into a path that can be filled. The problem is that bezier offset curves are not bezier curves, so they have to be approximated. Fortunately, since this operation is backend agnostic, it can be used and reused by many Athens backends. Since stroking is not critical for my needs, I do not think I will implement it soon properly. Currently I am just drawing a 1 pixel thick line, which can look horrible, but is better than nothing. One algorithm that seems to be simple to implement is one described in a paper from the Cairo people, which uses the Minkowksi  Sum between the outline of the path to be stroked, and shape of the pen that is stroking

Soon I will merge the Lowcode instructions into the SqueakVM/Cog VM to integrate these bytecodes into the mainline VM.

I will not publish Woden 2 until the dependencies and Woden 2 itself can be loaded easily.  Soon I will publish properly the Dastrel shader language and the Slovim backend. Slovim name comes from Smalltalk Low levelish VIrtual Machine, an intermediate representation heavily modeled after the one used in LLVM, implemented in Pharo, some very basic optimizations(basic constant folding, inlining and some dead code generation), and a code generator backend for SpirV, GLSL and C++. My biggest issue before publishing this compiler are:
- Implementing a command line interface.
- Doing an easy to install and use package for Linux, OS X and Windows.
- Documenting the language and the compiler.

Best regards,
Ronie

2016-07-28 19:31 GMT+02:00 Alexandre Bergel <[hidden email]>:

I would love to see an athens back-end for it.

Isn't it what Ronie is working on?

Alexandre


Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :
Hi all,

My tiny binding for OpenGLES2 is ready :)
http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/

It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.

A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
Tested only on Mac OSX, patches welcome!

Cheers,
Thibault





Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

stepharo
In reply to this post by Ronie Salgado


Anyway, implementing a Athens backend from scratch is a really hard problem, because paths can be concave, curve and self intersecting. Drawing a path, unlike drawing a triangle is a global operation, for which GPUs were not designed for. The easiest and most robust technique for drawing a path using the GPU is to flatten the path using De Casteljau's algorithm and then using the stencil buffer technique for drawing concave polygons that was described in the OpenGL red book ( http://www.glprogramming.com/red/chapter14.html#name13 ). The problem of this technique is that involves many state changes(glStencil*)  and drawing commands(glDrawElements) per path, which are the most expensive operations when using OpenGL. The alternative is to perform a triangulation in the CPU of paths, or to use one of the newer low level APIs such as Vulkan and Direct 3D 12 which were designed to reduce or eliminate the cost of state changes and drawing commands.

Path stroking is an even harder problem, which I have not solved yet. The standard approach is to convert a stroked path into a path that can be filled. The problem is that bezier offset curves are not bezier curves, so they have to be approximated. Fortunately, since this operation is backend agnostic, it can be used and reused by many Athens backends. Since stroking is not critical for my needs, I do not think I will implement it soon properly. Currently I am just drawing a 1 pixel thick line, which can look horrible, but is better than nothing. One algorithm that seems to be simple to implement is one described in a paper from the Cairo people, which uses the Minkowksi  Sum between the outline of the path to be stroked, and shape of the pen that is stroking

Thanks because indeed this is a lot of work. Thanks for the reality check.



Soon I will merge the Lowcode instructions into the SqueakVM/Cog VM to integrate these bytecodes into the mainline VM.

I will not publish Woden 2 until the dependencies and Woden 2 itself can be loaded easily.  Soon I will publish properly the Dastrel shader language and the Slovim backend. Slovim name comes from Smalltalk Low levelish VIrtual Machine, an intermediate representation heavily modeled after the one used in LLVM, implemented in Pharo, some very basic optimizations(basic constant folding, inlining and some dead code generation), and a code generator backend for SpirV, GLSL and C++. My biggest issue before publishing this compiler are:
- Implementing a command line interface.
- Doing an easy to install and use package for Linux, OS X and Windows.
- Documenting the language and the compiler.

Best regards,
Ronie

2016-07-28 19:31 GMT+02:00 Alexandre Bergel <[hidden email]>:

I would love to see an athens back-end for it.

Isn't it what Ronie is working on?

Alexandre


Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :
Hi all,

My tiny binding for OpenGLES2 is ready :)
http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/

It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.

A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
Tested only on Mac OSX, patches welcome!

Cheers,
Thibault





Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

abergel
In reply to this post by stepharo
+1

> On Jul 28, 2016, at 3:52 PM, stepharo <[hidden email]> wrote:
>
> Thanks ronie for your analysis (especially the thread local point).
> Stef
>
> Le 28/7/16 à 20:36, Ronie Salgado a écrit :
>> Isn't it what Ronie is working on?
>> I am not working with OpenGL or OpenGL ES anymore. I am moving all of my efforts into Woden 2, the AbstractGPU (an abstraction layer for Vulkan, Direct3D 12 and Metal), and Dastrel (Data Stream Language) a custom shader language whose compiler I implemented in Pharo, with a backend for SpirV, GLSL and C++. I still have to implement the backends for Metal and HLSL.
>>
>> Thibault is interested more on being able to use OpenGL ES 2.0 directly in the playground, for teaching purposes without any abstraction layer underlying. I do not agree with many of his approaches, too low level, and many of themp do not work well with Pharo VM. One of the biggest issue of OpenGL is related with vsync, which is an operation that can block all of the Pharo processes, until we get a VM with the threaded FFI, which will have in the near future. It will use the UnifiedFFI API, so we will not have to further changes to take advantage of the threaded ffi in the user code.
>>
>> A far worse problem with OpenGL is that the current OpenGL context is a thread local variable, which do not interact at all with Pharo processes. This is the reason of why I added the OSWindowRenderThread class for serializing OSWindow and OpenGL animations. SDL2 renderers are usually implemented using OpenGL. They are two correct solution for this problem:
>> - Having a VM that maps each Pharo Process into an operating system thread,
>> - Not using OpenGL at all.
>>
>> The others low level graphics API (Vulkan, Direct 3D and Metal) do not rely on a global thread local storage variable. These APIs are object oriented, so handles are passed in each API call.
>>
>> Currently, I have an Athens backend above Woden 2, which is only supporting Vulkan for now. Since this backend is for Woden 2, it has some dependencies in the Woden 2 Math and the Lowcode NativeStructure. This means that the Woden 2 Athens backend has a strong dependency in the the modified OpalCompiler that uses the Lowcode extended byte code set, and the the modified PharoVM that implements these extra bytecode in the just in time compiler.
>>
>> Anyway, implementing a Athens backend from scratch is a really hard problem, because paths can be concave, curve and self intersecting. Drawing a path, unlike drawing a triangle is a global operation, for which GPUs were not designed for. The easiest and most robust technique for drawing a path using the GPU is to flatten the path using De Casteljau's algorithm and then using the stencil buffer technique for drawing concave polygons that was described in the OpenGL red book ( http://www.glprogramming.com/red/chapter14.html#name13 ). The problem of this technique is that involves many state changes(glStencil*)  and drawing commands(glDrawElements) per path, which are the most expensive operations when using OpenGL. The alternative is to perform a triangulation in the CPU of paths, or to use one of the newer low level APIs such as Vulkan and Direct 3D 12 which were designed to reduce or eliminate the cost of state changes and drawing commands.
>>
>> Path stroking is an even harder problem, which I have not solved yet. The standard approach is to convert a stroked path into a path that can be filled. The problem is that bezier offset curves are not bezier curves, so they have to be approximated. Fortunately, since this operation is backend agnostic, it can be used and reused by many Athens backends. Since stroking is not critical for my needs, I do not think I will implement it soon properly. Currently I am just drawing a 1 pixel thick line, which can look horrible, but is better than nothing. One algorithm that seems to be simple to implement is one described in a paper from the Cairo people, which uses the Minkowksi  Sum between the outline of the path to be stroked, and shape of the pen that is stroking
>> the path: http://keithp.com/~keithp/talks/cairo2003.pdf
>>
>> Soon I will merge the Lowcode instructions into the SqueakVM/Cog VM to integrate these bytecodes into the mainline VM.
>>
>> I will not publish Woden 2 until the dependencies and Woden 2 itself can be loaded easily.  Soon I will publish properly the Dastrel shader language and the Slovim backend. Slovim name comes from Smalltalk Low levelish VIrtual Machine, an intermediate representation heavily modeled after the one used in LLVM, implemented in Pharo, some very basic optimizations(basic constant folding, inlining and some dead code generation), and a code generator backend for SpirV, GLSL and C++. My biggest issue before publishing this compiler are:
>> - Implementing a command line interface.
>> - Doing an easy to install and use package for Linux, OS X and Windows.
>> - Documenting the language and the compiler.
>>
>> Best regards,
>> Ronie
>>
>> 2016-07-28 19:31 GMT+02:00 Alexandre Bergel <[hidden email]>:
>>> I would love to see an athens back-end for it.
>>>
>> Isn't it what Ronie is working on?
>>
>> Alexandre
>>
>>>
>>> Le 19/7/16 à 13:19, Thibault Raffaillac a écrit :
>>>> Hi all,
>>>>
>>>> My tiny binding for OpenGLES2 is ready :)
>>>>
>>>> http://smalltalkhub.com/#!/~ThibaultRaffaillac/OpenGLES2/
>>>>
>>>>
>>>> It takes a different direction than that of NBOpenGL. I found the support for all versions of OpenGL overwhelming, both for beginners and to maintain. With a limited number of functions I could carefully name all messages.
>>>>
>>>> A demo using either SDL2 or GLFW (faster) is included, supporting VSync and retina displays.
>>>> Tested only on Mac OSX, patches welcome!
>>>>
>>>> Cheers,
>>>> Thibault
>>>>
>>>>
>>>
>>
>>
>

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.




Reply | Threaded
Open this post in threaded view
|

Re: OpenGLES2 binding available!

Thibault Raffaillac
In reply to this post by Thibault Raffaillac
+1 for your analysis!

> One of the biggest issue of OpenGL is related with vsync, which is an operation that can block all of the Pharo processes.

It does indeed, but is one way to sync with display rate (until we get threaded FFI of course then a thread can handle the wait). Currently we have to way to know the ideal time to start redraw, so as to minimise the delay between processing an event and displaying its feedback. Standard way is redraw with fixed independent frequency, which incurs an average feedback delay of (display period+redraw period)/2, and a waste of processing time if faster than display rate.

VSync in video games has a bad press because people redraw right after it returns:
_ Process events
_ Draw scene
_ Swap buffers <- pauses until scene is displayed
_ Loop back

That gives a worse feedback delay, but we could to pause a bit before processing stuff:
_ Delay(display period - last redraw duration - redraw jitter)
_ Process events
_ Draw scene
_ Swap buffers
_ Loop back

I find it very hard to find good info on VSync, and low latency event handling on PC in general, so please don't pan this idea too hard, this is just for experiment fun :)

> A far worse problem with OpenGL is that the current OpenGL context is a
> thread local variable, which do not interact at all with Pharo processes.
> This is the reason of why I added the OSWindowRenderThread class for
> serializing OSWindow and OpenGL animations. SDL2 renderers are usually
> implemented using OpenGL. They are two correct solution for this problem:
> - Having a VM that maps each Pharo Process into an operating system thread,
> - Not using OpenGL at all.
>
> The others low level graphics API (Vulkan, Direct 3D and Metal) do not rely
> on a global thread local storage variable. These APIs are object oriented,
> so handles are passed in each API call.

Yes, this is a major flaw in OpenGL. A GL context is global to all Pharo processes, meaning one Process could draw in an other Process area if preempted at the wrong moment. But a simple way to circumvent this is giving all display Processes (using OpenGL) the same priority.

> Anyway, implementing a Athens backend from scratch is a really hard problem.

Agreed. The difficulty with a hardware backend for Athens is it comes from software world. Some primitives are a nightmare to accelerate, resulting in usually incomplete hardware backends (https://developer.android.com/guide/topics/graphics/hardware-accel.html#unsupported). This is annoying for users because some primitives may turn off acceleration entirely, and force you to dig into the internal GL mud.
To me the use of hardware drawing is either explicit or none. If appended to Athens it should be very clear what goes to the GPU and what not.