Smalltalk › Pharo › Pharo Smalltalk Developers

Some new bindings for native emulation stuff

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

6 messages Options

Guillermo Polito

Some new bindings for native emulation stuff

Hi everybody,

I’ve been playing around with machine code simulation this last week and I’ve made bindings for the unicorn library and the llvm disassembler:

https://github.com/guillep/pharo-unicorn

https://github.com/guillep/pharo-llvmDisassembler

Funny thing: both support lots of platforms (x86 and arm both 32 and 64 bits and more…). So out of the box we can simulate and disassemble lots of platforms.

And in one afternoon I’ve played with them to do a native debugger with Spec2 just for fun.

Hope this evolves a bit more soon, and that it helps somebody.

Guille

NorbertHartl

Re: Some new bindings for native emulation stuff

That is pretty awesome. Couldn‘t be an constraint umbrella for VMMaker. This way it would be possible to write tests that check for resulting code. Or writing tests for the JIT (does sista has something like this). Could be one step closer to a deterministic code generation, no?

Norbert

Am 02.08.2019 um 18:09 schrieb Guillermo Polito <[hidden email]>:

Hi everybody,

I’ve been playing around with machine code simulation this last week and I’ve made bindings for the unicorn library and the llvm disassembler:

https://github.com/guillep/pharo-unicorn
https://github.com/guillep/pharo-llvmDisassembler

Funny thing: both support lots of platforms (x86 and arm both 32 and 64 bits and more…). So out of the box we can simulate and disassemble lots of platforms.

And in one afternoon I’ve played with them to do a native debugger with Spec2 just for fun.
Hope this evolves a bit more soon, and that it helps somebody.

Guille

<PastedGraphic-5.png>

ducasse

Re: Some new bindings for native emulation stuff

That is pretty awesome. Couldn‘t be an constraint umbrella for VMMaker.

Guille is doing that to see if we can reduce the number of plugin and how to build tools to support assembly generation :)

And also to be able to write much nicer tests (I mean with real assertions and not transcript show: ;)

This way it would be possible to write tests that check for resulting code. Or writing tests for the JIT (does sista has something like this). Could be one step closer to a deterministic code generation, no?

Stef

Norbert

Am 02.08.2019 um 18:09 schrieb Guillermo Polito <[hidden email]>:

Hi everybody,

I’ve been playing around with machine code simulation this last week and I’ve made bindings for the unicorn library and the llvm disassembler:

https://github.com/guillep/pharo-unicorn
https://github.com/guillep/pharo-llvmDisassembler

Funny thing: both support lots of platforms (x86 and arm both 32 and 64 bits and more…). So out of the box we can simulate and disassemble lots of platforms.

And in one afternoon I’ve played with them to do a native debugger with Spec2 just for fun.
Hope this evolves a bit more soon, and that it helps somebody.

Guille

<PastedGraphic-5.png>

melkyades

Re: Some new bindings for native emulation stuff

In reply to this post by Guillermo Polito

Just in time :P This is really cool!

On Fri, Aug 2, 2019 at 1:10 PM Guillermo Polito <[hidden email]> wrote:

Hi everybody,

I’ve been playing around with machine code simulation this last week and I’ve made bindings for the unicorn library and the llvm disassembler:

https://github.com/guillep/pharo-unicorn
https://github.com/guillep/pharo-llvmDisassembler

Funny thing: both support lots of platforms (x86 and arm both 32 and 64 bits and more…). So out of the box we can simulate and disassemble lots of platforms.

And in one afternoon I’ve played with them to do a native debugger with Spec2 just for fun.
Hope this evolves a bit more soon, and that it helps somebody.

Guille

Javier Pimás
Ciudad de Buenos Aires

Guillermo Polito

Re: Some new bindings for native emulation stuff

In reply to this post by NorbertHartl

El 3 ago 2019, a las 13:59, Norbert Hartl <[hidden email]> escribió:

That is pretty awesome. Couldn‘t be an constraint umbrella for VMMaker. This way it would be possible to write tests that check for resulting code. Or writing tests for the JIT (does sista has something like this). Could be one step closer to a deterministic code generation, no?

Just to be fair, there are in Cog machine code simulation capabilities by using Bochs for intel and gdb 7.10 for arm32.

And there is also a starting point for testing the JIT.

However, we have seen that for this the opensmalltalk-vm contains a copy of bochs and gdb source code

https://github.com/OpenSmalltalk/opensmalltalk-vm/tree/Cog/processors

And with Pablo we have thought this was a suboptimal (both regarding building and licencing…), so we started to look at alternatives.

That’s why we took a look at unicorn and llvm. Unicorn is ultimately based on qemu, which is fairly mature from our point of view.

There is not much to say about llvm :).

What is nice is that with a single set of bindings we can cover lots of platforms. E.g., no need for new bindings for arm64.

Aaand, with some little adjustments here and there, we have ~85% of 114 existing tests passing for arm32, x86 and x86-64.

Of course there are not many tests, and they are only testing the basic compiler (just instructions) and not the Jitting of methods.

But it’s a good start.

Now, also we were a bit picky about licensing:

- The llvm disassembler bindings are MIT.

- We have licensed the unicorn bindings as LGPL because Unicorn is GPL, and this would allow people to use the bindings without any requirements on licensing. However, any modifications to the bindings or unicorn itself should be further published as LGPL or GPL.

Guille

Norbert

Am 02.08.2019 um 18:09 schrieb Guillermo Polito <[hidden email]>:

Hi everybody,

I’ve been playing around with machine code simulation this last week and I’ve made bindings for the unicorn library and the llvm disassembler:

https://github.com/guillep/pharo-unicorn
https://github.com/guillep/pharo-llvmDisassembler

Funny thing: both support lots of platforms (x86 and arm both 32 and 64 bits and more…). So out of the box we can simulate and disassemble lots of platforms.

And in one afternoon I’ve played with them to do a native debugger with Spec2 just for fun.
Hope this evolves a bit more soon, and that it helps somebody.

Guille

<PastedGraphic-5.png>

NorbertHartl

Re: Some new bindings for native emulation stuff

There is much sense in this mail, I like it. The unicorn GPL shouldn‘t be an issue in this usage scenario.

Well done,

Norbert

Am 05.08.2019 um 11:48 schrieb Guillermo Polito <[hidden email]>:

El 3 ago 2019, a las 13:59, Norbert Hartl <[hidden email]> escribió:

That is pretty awesome. Couldn‘t be an constraint umbrella for VMMaker. This way it would be possible to write tests that check for resulting code. Or writing tests for the JIT (does sista has something like this). Could be one step closer to a deterministic code generation, no?

Just to be fair, there are in Cog machine code simulation capabilities by using Bochs for intel and gdb 7.10 for arm32.
And there is also a starting point for testing the JIT.

However, we have seen that for this the opensmalltalk-vm contains a copy of bochs and gdb source code
https://github.com/OpenSmalltalk/opensmalltalk-vm/tree/Cog/processors

And with Pablo we have thought this was a suboptimal (both regarding building and licencing…), so we started to look at alternatives.
That’s why we took a look at unicorn and llvm. Unicorn is ultimately based on qemu, which is fairly mature from our point of view.
There is not much to say about llvm :).

What is nice is that with a single set of bindings we can cover lots of platforms. E.g., no need for new bindings for arm64.

Aaand, with some little adjustments here and there, we have ~85% of 114 existing tests passing for arm32, x86 and x86-64.
<PastedGraphic-6.png>

Of course there are not many tests, and they are only testing the basic compiler (just instructions) and not the Jitting of methods.
But it’s a good start.

Now, also we were a bit picky about licensing:
- The llvm disassembler bindings are MIT.
- We have licensed the unicorn bindings as LGPL because Unicorn is GPL, and this would allow people to use the bindings without any requirements on licensing. However, any modifications to the bindings or unicorn itself should be further published as LGPL or GPL.

Guille

Norbert

Am 02.08.2019 um 18:09 schrieb Guillermo Polito <[hidden email]>:

Hi everybody,

I’ve been playing around with machine code simulation this last week and I’ve made bindings for the unicorn library and the llvm disassembler:

https://github.com/guillep/pharo-unicorn
https://github.com/guillep/pharo-llvmDisassembler

Funny thing: both support lots of platforms (x86 and arm both 32 and 64 bits and more…). So out of the box we can simulate and disassemble lots of platforms.

And in one afternoon I’ve played with them to do a native debugger with Spec2 just for fun.
Hope this evolves a bit more soon, and that it helps somebody.

Guille

<PastedGraphic-5.png>