Need your help filling a FAQ

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Need your help filling a FAQ

Igor Stasenko
Hello guys,

i'm currently writing a documentation for my project - NativeBoost.

The work is based on a simple plugin which i wrote, which allows to
run a native code from Squeak.
The native code is attached to a compiled method itself (into its
trailer), and so lives within an image.
The plugin can run this code through a single primitive, which knows
how to do it right.

Actually VM requires very small changes to support that. The bulk of
the work lies at the language side.
I'm currently working on making FFI callout support interface which
will support many cool things, comparing to usual FFIPlugin. :)

I'd like to know, what details you want to know about it (tautology ;)
, so i will compose the FAQ along with other details about project.


P.S. Google became too smart :) Once i clicked 'send' button, its
shows a message box:
     Did you mean to attach files?
     You wrote "is attached" in your message, but there are no files
attached.  Send anyway?

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Need your help filling a FAQ

Stéphane Ducasse
super cool :)
Questions:
        - does it work on mac, linux..?
        - how to enable it?
        - why did you choose ffi and not alien?
        - does it work on pharo?
        - what is the representation that you use to convert it to assembly?
        byteocde or ast
        if AST which one? RBAst?
       
Stef

On Apr 25, 2010, at 10:28 AM, Igor Stasenko wrote:

> Hello guys,
>
> i'm currently writing a documentation for my project - NativeBoost.
>
> The work is based on a simple plugin which i wrote, which allows to
> run a native code from Squeak.
> The native code is attached to a compiled method itself (into its
> trailer), and so lives within an image.
> The plugin can run this code through a single primitive, which knows
> how to do it right.
>
> Actually VM requires very small changes to support that. The bulk of
> the work lies at the language side.
> I'm currently working on making FFI callout support interface which
> will support many cool things, comparing to usual FFIPlugin. :)
>
> I'd like to know, what details you want to know about it (tautology ;)
> , so i will compose the FAQ along with other details about project.
>
>
> P.S. Google became too smart :) Once i clicked 'send' button, its
> shows a message box:
>     Did you mean to attach files?
>     You wrote "is attached" in your message, but there are no files
> attached.  Send anyway?
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 25 April 2010 15:04, Stéphane Ducasse <[hidden email]> wrote:
> super cool :)
> Questions:
>        - does it work on mac, linux..?

Currently supported is only Win32 platform.
It should be easy to port plugin on any other platform, which based on
x86 architecture.
Porting plugin on x64 or other archs will require additional work.
Also, it requires a change in VM virtual memory allocation (a memory,
which used for squeak object memory),
to allow execution of native code.
It won't work on platforms, where VM will be unable to control what
regions of memory is executable and what is not (as well as any other
JIT implementation).

>        - how to enable it?

1. Build a VM with NativeBoost plugin, and you have it. Also, plugin
can be made external, so you can use it as any other plugin.
2. On image side , it requires a small change to
CompiledMethodTrailer, to add a new trailer kind for methods, to allow
methods to hold a native code and to easily work with it
(encode/decode etc).
You can put any native code (bytes) into a method's trailer from any
sourse. Just make sure that your method using
a NativeBoost's primitive:
 <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>

>        - why did you choose ffi and not alien?

You don't need either of them to run native code.
FFI stands for Foreign Function Interface, and NativeBoost project
will have own
interface for generating foreign function calls using native code.
It having dependencies from FFI-Kernel package, but ultimately, they
can be avoided.
Its using FFI to simply not duplicate the code and for some testing.

>        - does it work on pharo?

It can be used in any fork , where we already having a compiled
methods trailers adopted.
Pharo is supported.

>        - what is the representation that you use to convert it to assembly?
>        byteocde or ast
>        if AST which one? RBAst?
>

A NativeBoost package using AsmJit library
(http://www.squeaksource.com/AsmJit) for generating native code.
Its been ported from AsmJit project (http://code.google.com/p/asmjit/)
written in C.
By using a small AsmJit library you can emit almost any x86 instruction directly
to generate native code from them.
So, there currently no any platform-neutral DSL (and therefore AST)
for representing a native code in more
platform-neutral and abstract way.
An AsmJit can be used as a basis for making a more advanced/abstract
compiler/parser, as well as one can use
any other JIT implementation which can translate some intermediate
instructions into native code.

> Stef
>

Thanks for questions!

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Stéphane Ducasse
>
>>        - what is the representation that you use to convert it to assembly?
>>       byteocde or ast
>>       if AST which one? RBAst?
>>
>
> A NativeBoost package using AsmJit library
> (http://www.squeaksource.com/AsmJit) for generating native code.
> Its been ported from AsmJit project (http://code.google.com/p/asmjit/)
> written in C.

I was more thinking if you analyze the method AST and automatically generate x86.
But I think that you do it by hand right :)



_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 25 April 2010 20:46, Stéphane Ducasse <[hidden email]> wrote:

>>
>>>        - what is the representation that you use to convert it to assembly?
>>>       byteocde or ast
>>>       if AST which one? RBAst?
>>>
>>
>> A NativeBoost package using AsmJit library
>> (http://www.squeaksource.com/AsmJit) for generating native code.
>> Its been ported from AsmJit project (http://code.google.com/p/asmjit/)
>> written in C.
>
> I was more thinking if you analyze the method AST and automatically generate x86.
> But I think that you do it by hand right :)

Yup.
Translating smalltalk code into native code requires an abstraction
layer, i'm yet missing :)

I know what you asking, like, if you can code like this:

myNativeMethod: foo with: bar
<native>

  ^  foo readByte + bar * self

(I following this way , but in another project - Moebius ;) ).
Sure, it can be possible to use it. And i slowly moving in that direction.
The infrastructure in Moebius is following:

(a)(Smalltalk parser) -> (b)(Native intermediate instructions
generator (compiler)) -> (c)(Native code translator)
-> (d)(AsmJit)

I having a, b and d , but c is still not complete.
And sure thing, one can use it for own purposes , since Moebius
implemented purely in smalltalk,
and works in Squeak/Pharo, so potentially it can be retargeted to anything else.



> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Stéphane Ducasse
>
>
> (a)(Smalltalk parser) ->

could you plug the IRBuilder

> (b)(Native intermediate instructions
> generator (compiler)) -> (c)(Native code translator)
> -> (d)(AsmJit)
>
> I having a, b and d , but c is still not complete.
> And sure thing, one can use it for own purposes , since Moebius
> implemented purely in smalltalk,
> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>
>
>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>
>>
>> (a)(Smalltalk parser) ->
>
> could you plug the IRBuilder

yes i can! The Moebius parser designed by taking in mind, that it can
be used as a backend
for any kind of encoding. It recognizing a smalltalk syntax and
semantic elements and then passing the messages to encoder.
And encoder can be anything. It can encode the parsed data into any
kind of AST , its just needs to conform with parser's protocol,
but can use arbitrary data structures for building the method's AST.
Parser don't have _any_ assumptions, into what form an encoder
translating the parsed data.

So, for instance, i can implement a syntax highlighting encoder, based
on this parser, without a need of having separate parser targeted only
for syntax highlighting (like SHParserST80)

>
>> (b)(Native intermediate instructions
>> generator (compiler)) -> (c)(Native code translator)
>> -> (d)(AsmJit)
>>
>> I having a, b and d , but c is still not complete.
>> And sure thing, one can use it for own purposes , since Moebius
>> implemented purely in smalltalk,
>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>
>>
>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Adrian Lienhard
Hi Igor,

One question I have is how your work compares to Exupery?

Adrian

On Apr 25, 2010, at 22:21 , Igor Stasenko wrote:

> On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>>
>>>
>>> (a)(Smalltalk parser) ->
>>
>> could you plug the IRBuilder
>
> yes i can! The Moebius parser designed by taking in mind, that it can
> be used as a backend
> for any kind of encoding. It recognizing a smalltalk syntax and
> semantic elements and then passing the messages to encoder.
> And encoder can be anything. It can encode the parsed data into any
> kind of AST , its just needs to conform with parser's protocol,
> but can use arbitrary data structures for building the method's AST.
> Parser don't have _any_ assumptions, into what form an encoder
> translating the parsed data.
>
> So, for instance, i can implement a syntax highlighting encoder, based
> on this parser, without a need of having separate parser targeted only
> for syntax highlighting (like SHParserST80)
>
>>
>>> (b)(Native intermediate instructions
>>> generator (compiler)) -> (c)(Native code translator)
>>> -> (d)(AsmJit)
>>>
>>> I having a, b and d , but c is still not complete.
>>> And sure thing, one can use it for own purposes , since Moebius
>>> implemented purely in smalltalk,
>>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>>
>>>
>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 25 April 2010 23:45, Adrian Lienhard <[hidden email]> wrote:
> Hi Igor,
>
> One question I have is how your work compares to Exupery?
>

Well, i missing the good Exupery description , on squeak source it says:
A bytecode compiler for Squeak, still an alpha project. It doesn't yet
do anything useful but it can compile most bytecodes.
Or from wiki page:
Exupery is a compiler written in Smalltalk that compiles bytecodes to
machine code.

So, as you can see, an Exupery aims mainly towards turning a bytecode
into a native code, i.e. towards implementing a JIT.
It is important to understand that NativeBoost project having a bit
orthogonal (or more complementary) aim - to allow you to run an
arbitrary native code and speak directly to hardware, OS etc.



> Adrian
>
> On Apr 25, 2010, at 22:21 , Igor Stasenko wrote:
>
>> On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>>>
>>>>
>>>> (a)(Smalltalk parser) ->
>>>
>>> could you plug the IRBuilder
>>
>> yes i can! The Moebius parser designed by taking in mind, that it can
>> be used as a backend
>> for any kind of encoding. It recognizing a smalltalk syntax and
>> semantic elements and then passing the messages to encoder.
>> And encoder can be anything. It can encode the parsed data into any
>> kind of AST , its just needs to conform with parser's protocol,
>> but can use arbitrary data structures for building the method's AST.
>> Parser don't have _any_ assumptions, into what form an encoder
>> translating the parsed data.
>>
>> So, for instance, i can implement a syntax highlighting encoder, based
>> on this parser, without a need of having separate parser targeted only
>> for syntax highlighting (like SHParserST80)
>>
>>>
>>>> (b)(Native intermediate instructions
>>>> generator (compiler)) -> (c)(Native code translator)
>>>> -> (d)(AsmJit)
>>>>
>>>> I having a, b and d , but c is still not complete.
>>>> And sure thing, one can use it for own purposes , since Moebius
>>>> implemented purely in smalltalk,
>>>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko AKA sig.
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 26 April 2010 00:06, Igor Stasenko <[hidden email]> wrote:

> On 25 April 2010 23:45, Adrian Lienhard <[hidden email]> wrote:
>> Hi Igor,
>>
>> One question I have is how your work compares to Exupery?
>>
>
> Well, i missing the good Exupery description , on squeak source it says:
> A bytecode compiler for Squeak, still an alpha project. It doesn't yet
> do anything useful but it can compile most bytecodes.
> Or from wiki page:
> Exupery is a compiler written in Smalltalk that compiles bytecodes to
> machine code.
>
> So, as you can see, an Exupery aims mainly towards turning a bytecode
> into a native code, i.e. towards implementing a JIT.
> It is important to understand that NativeBoost project having a bit
> orthogonal (or more complementary) aim - to allow you to run an
> arbitrary native code and speak directly to hardware, OS etc.
>

Ohh.. i guess you asked about Moebius project. Sorry.

Moebius is much more ambitious project. It is a smalltalk environment
, fully implemented in itself, i.e.
which requires no VM and minimal glue code to run on target platform.

While Exupery, is still using a 'VM' concept which interpreting the
methods, while additionally providing facilities to optimize the
system speed by jitting some methods.
In moebius, all methods are initially contain a native code. There is
no interpreter. Sure it can be added later (as for any other language
interpreter btw), but that will not change the fact that system can
sustain itself and run without a need of having VM.

>
>
>> Adrian
>>
>> On Apr 25, 2010, at 22:21 , Igor Stasenko wrote:
>>
>>> On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> (a)(Smalltalk parser) ->
>>>>
>>>> could you plug the IRBuilder
>>>
>>> yes i can! The Moebius parser designed by taking in mind, that it can
>>> be used as a backend
>>> for any kind of encoding. It recognizing a smalltalk syntax and
>>> semantic elements and then passing the messages to encoder.
>>> And encoder can be anything. It can encode the parsed data into any
>>> kind of AST , its just needs to conform with parser's protocol,
>>> but can use arbitrary data structures for building the method's AST.
>>> Parser don't have _any_ assumptions, into what form an encoder
>>> translating the parsed data.
>>>
>>> So, for instance, i can implement a syntax highlighting encoder, based
>>> on this parser, without a need of having separate parser targeted only
>>> for syntax highlighting (like SHParserST80)
>>>
>>>>
>>>>> (b)(Native intermediate instructions
>>>>> generator (compiler)) -> (c)(Native code translator)
>>>>> -> (d)(AsmJit)
>>>>>
>>>>> I having a, b and d , but c is still not complete.
>>>>> And sure thing, one can use it for own purposes , since Moebius
>>>>> implemented purely in smalltalk,
>>>>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>>>>
>>>>>
>>>>>
>>>>>> _______________________________________________
>>>>>> Pharo-project mailing list
>>>>>> [hidden email]
>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko AKA sig.
>>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Adrian Lienhard
Thanks for the explanation.

It might make sense to contact Bryce [1] since Exupery probably has very similar mechanism to execute native methods (I think he is only irregularly reading this list). Maybe it makes sense to base Exupery on NativeBoost?

Cheers,
Adrian

[1] Bryce Kampjes <[hidden email]>

On Apr 25, 2010, at 23:14 , Igor Stasenko wrote:

> On 26 April 2010 00:06, Igor Stasenko <[hidden email]> wrote:
>> On 25 April 2010 23:45, Adrian Lienhard <[hidden email]> wrote:
>>> Hi Igor,
>>>
>>> One question I have is how your work compares to Exupery?
>>>
>>
>> Well, i missing the good Exupery description , on squeak source it says:
>> A bytecode compiler for Squeak, still an alpha project. It doesn't yet
>> do anything useful but it can compile most bytecodes.
>> Or from wiki page:
>> Exupery is a compiler written in Smalltalk that compiles bytecodes to
>> machine code.
>>
>> So, as you can see, an Exupery aims mainly towards turning a bytecode
>> into a native code, i.e. towards implementing a JIT.
>> It is important to understand that NativeBoost project having a bit
>> orthogonal (or more complementary) aim - to allow you to run an
>> arbitrary native code and speak directly to hardware, OS etc.
>>
>
> Ohh.. i guess you asked about Moebius project. Sorry.
>
> Moebius is much more ambitious project. It is a smalltalk environment
> , fully implemented in itself, i.e.
> which requires no VM and minimal glue code to run on target platform.
>
> While Exupery, is still using a 'VM' concept which interpreting the
> methods, while additionally providing facilities to optimize the
> system speed by jitting some methods.
> In moebius, all methods are initially contain a native code. There is
> no interpreter. Sure it can be added later (as for any other language
> interpreter btw), but that will not change the fact that system can
> sustain itself and run without a need of having VM.
>
>>
>>
>>> Adrian
>>>
>>> On Apr 25, 2010, at 22:21 , Igor Stasenko wrote:
>>>
>>>> On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> (a)(Smalltalk parser) ->
>>>>>
>>>>> could you plug the IRBuilder
>>>>
>>>> yes i can! The Moebius parser designed by taking in mind, that it can
>>>> be used as a backend
>>>> for any kind of encoding. It recognizing a smalltalk syntax and
>>>> semantic elements and then passing the messages to encoder.
>>>> And encoder can be anything. It can encode the parsed data into any
>>>> kind of AST , its just needs to conform with parser's protocol,
>>>> but can use arbitrary data structures for building the method's AST.
>>>> Parser don't have _any_ assumptions, into what form an encoder
>>>> translating the parsed data.
>>>>
>>>> So, for instance, i can implement a syntax highlighting encoder, based
>>>> on this parser, without a need of having separate parser targeted only
>>>> for syntax highlighting (like SHParserST80)
>>>>
>>>>>
>>>>>> (b)(Native intermediate instructions
>>>>>> generator (compiler)) -> (c)(Native code translator)
>>>>>> -> (d)(AsmJit)
>>>>>>
>>>>>> I having a, b and d , but c is still not complete.
>>>>>> And sure thing, one can use it for own purposes , since Moebius
>>>>>> implemented purely in smalltalk,
>>>>>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pharo-project mailing list
>>>>>>> [hidden email]
>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>> Igor Stasenko AKA sig.
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pharo-project mailing list
>>>>>> [hidden email]
>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko AKA sig.
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 26 April 2010 11:17, Adrian Lienhard <[hidden email]> wrote:
> Thanks for the explanation.
>
> It might make sense to contact Bryce [1] since Exupery probably has very similar mechanism to execute native methods (I think he is only irregularly reading this list). Maybe it makes sense to base Exupery on NativeBoost?
>
You are right , of course there are some synergy. Both projects in one
or another way dealing with native code.

But if you dig a bit deeper , you'll find some differences.

1.
- NativeBoost plugin requires very small changes to existing VMs.
I mentioned only one change: allow a code execution for object memory.
To enable it, it requires changing 2 lines of code in sqWin32Alloc.c file

  -- VirtualAlloc(pageBase, commit, MEM_COMMIT, PAGE_READWRITE)
  ++ VirtualAlloc(pageBase, commit, MEM_COMMIT, PAGE_EXECUTE_READWRITE)

(similar amount of changes required for other platforms).

A plugin, which is a very small thing (only 2 primitives) can be
shipped as external one.
done!

- Exupery requires a lot more changes to VM. Its even having own VMMaker.

2.
- Exupery allocates and uses a separate memory region for storing a
native code there.
So, it requires and additional efforts to manage this memory.

- NativeBoost uses an object memory (a CompiledMethods for storing a
native code).
And so, it requres no additional efforts to manage a memory. GC is
your friend :)

3.
- (last time i seen) Exupery compiler produces a location-dependent
code. So, if you relocate the native code, you have to alter all call
labels, literals and so on. This adds an additional logic to code, in
order to handle that. Sure it saves some CPU cycles (relative calls is
a bit cheaper), except from cases when you moving the code regularily
;)

- NativeBoost is intentionally puts you in costraints that you should
not use/produce a location-dependent code i.e.
all jumps should be relative and local to same routine, all calls
should be absolute.
NativeBoost handles the native code relocation automagically and don't
needs an extra effort.

> Cheers,
> Adrian
>
> [1] Bryce Kampjes <[hidden email]>
>
> On Apr 25, 2010, at 23:14 , Igor Stasenko wrote:
>
>> On 26 April 2010 00:06, Igor Stasenko <[hidden email]> wrote:
>>> On 25 April 2010 23:45, Adrian Lienhard <[hidden email]> wrote:
>>>> Hi Igor,
>>>>
>>>> One question I have is how your work compares to Exupery?
>>>>
>>>
>>> Well, i missing the good Exupery description , on squeak source it says:
>>> A bytecode compiler for Squeak, still an alpha project. It doesn't yet
>>> do anything useful but it can compile most bytecodes.
>>> Or from wiki page:
>>> Exupery is a compiler written in Smalltalk that compiles bytecodes to
>>> machine code.
>>>
>>> So, as you can see, an Exupery aims mainly towards turning a bytecode
>>> into a native code, i.e. towards implementing a JIT.
>>> It is important to understand that NativeBoost project having a bit
>>> orthogonal (or more complementary) aim - to allow you to run an
>>> arbitrary native code and speak directly to hardware, OS etc.
>>>
>>
>> Ohh.. i guess you asked about Moebius project. Sorry.
>>
>> Moebius is much more ambitious project. It is a smalltalk environment
>> , fully implemented in itself, i.e.
>> which requires no VM and minimal glue code to run on target platform.
>>
>> While Exupery, is still using a 'VM' concept which interpreting the
>> methods, while additionally providing facilities to optimize the
>> system speed by jitting some methods.
>> In moebius, all methods are initially contain a native code. There is
>> no interpreter. Sure it can be added later (as for any other language
>> interpreter btw), but that will not change the fact that system can
>> sustain itself and run without a need of having VM.
>>
>>>
>>>
>>>> Adrian
>>>>
>>>> On Apr 25, 2010, at 22:21 , Igor Stasenko wrote:
>>>>
>>>>> On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>>>>>>
>>>>>>>
>>>>>>> (a)(Smalltalk parser) ->
>>>>>>
>>>>>> could you plug the IRBuilder
>>>>>
>>>>> yes i can! The Moebius parser designed by taking in mind, that it can
>>>>> be used as a backend
>>>>> for any kind of encoding. It recognizing a smalltalk syntax and
>>>>> semantic elements and then passing the messages to encoder.
>>>>> And encoder can be anything. It can encode the parsed data into any
>>>>> kind of AST , its just needs to conform with parser's protocol,
>>>>> but can use arbitrary data structures for building the method's AST.
>>>>> Parser don't have _any_ assumptions, into what form an encoder
>>>>> translating the parsed data.
>>>>>
>>>>> So, for instance, i can implement a syntax highlighting encoder, based
>>>>> on this parser, without a need of having separate parser targeted only
>>>>> for syntax highlighting (like SHParserST80)
>>>>>
>>>>>>
>>>>>>> (b)(Native intermediate instructions
>>>>>>> generator (compiler)) -> (c)(Native code translator)
>>>>>>> -> (d)(AsmJit)
>>>>>>>
>>>>>>> I having a, b and d , but c is still not complete.
>>>>>>> And sure thing, one can use it for own purposes , since Moebius
>>>>>>> implemented purely in smalltalk,
>>>>>>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pharo-project mailing list
>>>>>>>> [hidden email]
>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best regards,
>>>>>>> Igor Stasenko AKA sig.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pharo-project mailing list
>>>>>>> [hidden email]
>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pharo-project mailing list
>>>>>> [hidden email]
>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko AKA sig.
>>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Need your help filling a FAQ

Igor Stasenko
On 26 April 2010 11:58, Igor Stasenko <[hidden email]> wrote:

> On 26 April 2010 11:17, Adrian Lienhard <[hidden email]> wrote:
>> Thanks for the explanation.
>>
>> It might make sense to contact Bryce [1] since Exupery probably has very similar mechanism to execute native methods (I think he is only irregularly reading this list). Maybe it makes sense to base Exupery on NativeBoost?
>>
> You are right , of course there are some synergy. Both projects in one
> or another way dealing with native code.
>
> But if you dig a bit deeper , you'll find some differences.
>
> 1.
> - NativeBoost plugin requires very small changes to existing VMs.
> I mentioned only one change: allow a code execution for object memory.
> To enable it, it requires changing 2 lines of code in sqWin32Alloc.c file
>
>  -- VirtualAlloc(pageBase, commit, MEM_COMMIT, PAGE_READWRITE)
>  ++ VirtualAlloc(pageBase, commit, MEM_COMMIT, PAGE_EXECUTE_READWRITE)
>
> (similar amount of changes required for other platforms).
>
> A plugin, which is a very small thing (only 2 primitives) can be
> shipped as external one.
> done!
>
> - Exupery requires a lot more changes to VM. Its even having own VMMaker.
>
> 2.
> - Exupery allocates and uses a separate memory region for storing a
> native code there.
> So, it requires and additional efforts to manage this memory.
>
> - NativeBoost uses an object memory (a CompiledMethods for storing a
> native code).
> And so, it requres no additional efforts to manage a memory. GC is
> your friend :)
>
> 3.
> - (last time i seen) Exupery compiler produces a location-dependent
> code. So, if you relocate the native code, you have to alter all call
> labels, literals and so on. This adds an additional logic to code, in
> order to handle that. Sure it saves some CPU cycles (relative calls is
> a bit cheaper), except from cases when you moving the code regularily
> ;)
>
> - NativeBoost is intentionally puts you in costraints that you should
> not use/produce a location-dependent code i.e.
> all jumps should be relative and local to same routine, all calls
> should be absolute.
> NativeBoost handles the native code relocation automagically and don't
> needs an extra effort.
>


4.
-(last time i seen) Exupery flushing the native code periodically.
Actually its using a memory for native code in a cache-like fashion
- NativeBoost flushing the native code only once: at image startup. Or
on user request.

there can be more differences but i think, i presented enough facts ,
which explaining why i invented own wheel instead of using existing
one.

>> Cheers,
>> Adrian
>>
>> [1] Bryce Kampjes <[hidden email]>
>>
>> On Apr 25, 2010, at 23:14 , Igor Stasenko wrote:
>>
>>> On 26 April 2010 00:06, Igor Stasenko <[hidden email]> wrote:
>>>> On 25 April 2010 23:45, Adrian Lienhard <[hidden email]> wrote:
>>>>> Hi Igor,
>>>>>
>>>>> One question I have is how your work compares to Exupery?
>>>>>
>>>>
>>>> Well, i missing the good Exupery description , on squeak source it says:
>>>> A bytecode compiler for Squeak, still an alpha project. It doesn't yet
>>>> do anything useful but it can compile most bytecodes.
>>>> Or from wiki page:
>>>> Exupery is a compiler written in Smalltalk that compiles bytecodes to
>>>> machine code.
>>>>
>>>> So, as you can see, an Exupery aims mainly towards turning a bytecode
>>>> into a native code, i.e. towards implementing a JIT.
>>>> It is important to understand that NativeBoost project having a bit
>>>> orthogonal (or more complementary) aim - to allow you to run an
>>>> arbitrary native code and speak directly to hardware, OS etc.
>>>>
>>>
>>> Ohh.. i guess you asked about Moebius project. Sorry.
>>>
>>> Moebius is much more ambitious project. It is a smalltalk environment
>>> , fully implemented in itself, i.e.
>>> which requires no VM and minimal glue code to run on target platform.
>>>
>>> While Exupery, is still using a 'VM' concept which interpreting the
>>> methods, while additionally providing facilities to optimize the
>>> system speed by jitting some methods.
>>> In moebius, all methods are initially contain a native code. There is
>>> no interpreter. Sure it can be added later (as for any other language
>>> interpreter btw), but that will not change the fact that system can
>>> sustain itself and run without a need of having VM.
>>>
>>>>
>>>>
>>>>> Adrian
>>>>>
>>>>> On Apr 25, 2010, at 22:21 , Igor Stasenko wrote:
>>>>>
>>>>>> On 25 April 2010 23:03, Stéphane Ducasse <[hidden email]> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> (a)(Smalltalk parser) ->
>>>>>>>
>>>>>>> could you plug the IRBuilder
>>>>>>
>>>>>> yes i can! The Moebius parser designed by taking in mind, that it can
>>>>>> be used as a backend
>>>>>> for any kind of encoding. It recognizing a smalltalk syntax and
>>>>>> semantic elements and then passing the messages to encoder.
>>>>>> And encoder can be anything. It can encode the parsed data into any
>>>>>> kind of AST , its just needs to conform with parser's protocol,
>>>>>> but can use arbitrary data structures for building the method's AST.
>>>>>> Parser don't have _any_ assumptions, into what form an encoder
>>>>>> translating the parsed data.
>>>>>>
>>>>>> So, for instance, i can implement a syntax highlighting encoder, based
>>>>>> on this parser, without a need of having separate parser targeted only
>>>>>> for syntax highlighting (like SHParserST80)
>>>>>>
>>>>>>>
>>>>>>>> (b)(Native intermediate instructions
>>>>>>>> generator (compiler)) -> (c)(Native code translator)
>>>>>>>> -> (d)(AsmJit)
>>>>>>>>
>>>>>>>> I having a, b and d , but c is still not complete.
>>>>>>>> And sure thing, one can use it for own purposes , since Moebius
>>>>>>>> implemented purely in smalltalk,
>>>>>>>> and works in Squeak/Pharo, so potentially it can be retargeted to anything else.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Pharo-project mailing list
>>>>>>>>> [hidden email]
>>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> Igor Stasenko AKA sig.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pharo-project mailing list
>>>>>>>> [hidden email]
>>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pharo-project mailing list
>>>>>>> [hidden email]
>>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>> Igor Stasenko AKA sig.
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pharo-project mailing list
>>>>>> [hidden email]
>>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko AKA sig.
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project