Need your help filling a FAQ

Hello guys,

i'm currently writing a documentation for my project - NativeBoost.

The work is based on a simple plugin which i wrote, which allows to
run a native code from Squeak.
The native code is attached to a compiled method itself (into its
trailer), and so lives within an image.
The plugin can run this code through a single primitive, which knows
how to do it right.

Actually VM requires very small changes to support that. The bulk of
the work lies at the language side.
I'm currently working on making FFI callout support interface which
will support many cool things, comparing to usual FFIPlugin. :)

I'd like to know, what details you want to know about it (tautology ;)
, so i will compose the FAQ along with other details about project.


P.S. Google became too smart :) Once i clicked 'send' button, its
shows a message box:
     Did you mean to attach files?
     You wrote "is attached" in your message, but there are no files
attached.  Send anyway?

--
Best regards,
Igor Stasenko AKA sig.

super cool :)
Questions:
        - does it work on mac, linux..?
        - how to enable it?
        - why did you choose ffi and not alien?
        - does it work on pharo?
        - what is the representation that you use to convert it to assembly?
        byteocde or ast
        if AST which one? RBAst?
       
Stef

On Apr 25, 2010, at 10:28 AM, Igor Stasenko wrote:

What does this do that I didn't know I wanted to do? 

Hi Igor,

thanks for asking. :-)

On Sun, Apr 25, 2010 at 10:28 AM, Igor Stasenko <[hidden email]> wrote:
> I'd like to know, what details you want to know about it (tautology ;)

All of them. :-P (No, seriously, all of them.)
Please forgive I'm not directly answering the FAQ question. Only time
will show which questions arise frequently.

Still, if I may ask for some things ...

User documentation:
* quick intro "howto" style tutorial
* complete and consistent API documentation
* also: what needs to be taken care of wrt. Smalltalk side ./. native
side (read: how to deal with low-level internals)
* perhaps 1-2 bigger-scale tutorials

Developer documentation:
* how does it work?
-> shape of the interface one uses
-> how is native code actually invoked
-> what is the extension to the VM (places, purposes)
-> what do I need to know / where would I have to look if I wanted to
extend this

I recall you wrote some things on this project earlier, and I read
them with great interest. Please, if you assemble documentation,
include those texts in the docs, instead of pointing to mailing list
archives. :-)

Best,

Michael

Hello Igor

Three more questions....

1) What are example applications of NativeBoost?
2) How much faster are they compared to regular Smalltalk code?
3) How do I write the native code?

Hannes

On 4/25/10, Michael Haupt <[hidden email]> wrote:

On 25 April 2010 15:04, Stéphane Ducasse <[hidden email]> wrote:
> super cool :)
> Questions:
>        - does it work on mac, linux..?

Currently supported is only Win32 platform.
It should be easy to port plugin on any other platform, which based on
x86 architecture.
Porting plugin on x64 or other archs will require additional work.
Also, it requires a change in VM virtual memory allocation (a memory,
which used for squeak object memory),
to allow execution of native code.
It won't work on platforms, where VM will be unable to control what
regions of memory is executable and what is not (as well as any other
JIT implementation).

>        - how to enable it?

1. Build a VM with NativeBoost plugin, and you have it. Also, plugin
can be made external, so you can use it as any other plugin.
2. On image side , it requires a small change to
CompiledMethodTrailer, to add a new trailer kind for methods, to allow
methods to hold a native code and to easily work with it
(encode/decode etc).
You can put any native code (bytes) into a method's trailer from any
sourse. Just make sure that your method using
a NativeBoost's primitive:
 <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>

>        - why did you choose ffi and not alien?

You don't need either of them to run native code.
FFI stands for Foreign Function Interface, and NativeBoost project
will have own
interface for generating foreign function calls using native code.
It having dependencies from FFI-Kernel package, but ultimately, they
can be avoided.
Its using FFI to simply not duplicate the code and for some testing.

>        - does it work on pharo?

It can be used in any fork , where we already having a compiled
methods trailers adopted.
Pharo is supported.

>        - what is the representation that you use to convert it to assembly?
>        byteocde or ast
>        if AST which one? RBAst?
>

A NativeBoost package using AsmJit library
(http://www.squeaksource.com/AsmJit) for generating native code.
Its been ported from AsmJit project (http://code.google.com/p/asmjit/)
written in C.
By using a small AsmJit library you can emit almost any x86 instruction directly
to generate native code from them.
So, there currently no any platform-neutral DSL (and therefore AST)
for representing a native code in more
platform-neutral and abstract way.
An AsmJit can be used as a basis for making a more advanced/abstract
compiler/parser, as well as one can use
any other JIT implementation which can translate some intermediate
instructions into native code.

> Stef
>

Thanks for questions!

--
Best regards,
Igor Stasenko AKA sig.

On 25 April 2010 17:18, Chris Cunnington <[hidden email]> wrote:
> <stevejobs>
> What does this do that I didn't know I wanted to do?

You can use it for pretty much anything, where you need to access your
host CPU directly,
without intermediate layers (OS, VM, compilers etc)

> What problem does this solve? I already have OSProcess to talk to the
> operating system.

OSProcess gives you only a small bit of functionality provided by host
platform - execution and interfacing
with other processes. With NativeBoost you can have any of it :)

> How can I use this for a project I'm already working on?

It can be used for binding with existing libraries (like OpenGL),
or implementing own primitives, without a need to do it in C language
and without a need to compile it and ship
separate binaries with your project.

> Does this mean I can execute code in a language the underlying OS
> understands? And then I get the results in a Smalltalk useable form?

If by assembler, you mean 'a language that underlying OS understands'
, then yes :)
There is an interface for marchalling data between C world and
Smalltalk objects and making a foreign calls.

> </stevejobs>
> Chris
>


--
Best regards,
Igor Stasenko AKA sig.

On 25 April 2010 19:00, Michael Haupt <[hidden email]> wrote: yes yes yes. :)

Yes. I planning to document all of the things you may need for using it.
Thanks for giving a scheme which i can follow for writing it.

> Best,
>
> Michael
>
>



--
Best regards,
Igor Stasenko AKA sig.

On 25 April 2010 20:08, Hannes Hirzel <[hidden email]> wrote:
> Hello Igor
>
> Three more questions....
>
> 1) What are example applications of NativeBoost?

Case 1) dynamic FFI callout generation:

loadLibrary: libName
        <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>

        ^ NBFFICallout apiCall: #( long 'LoadLibraryA' ( String )) module: #Kernel32

Case 2) dynamic primitive generation

readSignedInt32From: integerAddress

        <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>
        "
        Emulate the C function of form:
                long readSignedInt32From(long* ptr) {
                        return *ptr;
                }
       
        we don't need to call anything but just read the int32 value from given address
        and let a callout generator care about value coercion
        "
        ^ NBFFICallout cdecl: #(long long)
                emitCall: [:gen | | asm |
                                asm := gen asm.
                                asm
                                        mov: (asm mem: ESP) to: EAX;
                                        mov: (asm mem: EAX) to: EAX
                ].

Case 3) an OpenGL binding i working on.. (its a bit too big to fit in there) ;)

> 2) How much faster are they compared to regular Smalltalk code?

A question is a bit vague. There is a many ways how to answer it.
In short: Smalltalk code is interpreted by VM. Native code is
interpreted by CPU of your computer.
In smalltalk, you code fast, but running slow.
In assembler, you code slow, but running fast.

> 3) How do I write the native code?
>
Sorry, don't understand, what exactly you want to know.
There is a helper classes, which allow you to do it more or less
without much hurdles.
So, if you know assembler you can write it right away.
And sure thing, i will post an examples how to use it in my docs.

> Hannes
>
> On 4/25/10, Michael Haupt <[hidden email]> wrote:




--
Best regards,
Igor Stasenko AKA sig.

On 4/25/10, Igor Stasenko <[hidden email]> wrote:
This question was answered indirectly by the examples of point one.
You have implemented an assembler in Smalltalk. A code snippet looks like

    mov: (asm mem: ESP) to: EAX;
  mov: (asm mem: EAX) to: EAX

--Hannes

Hi Igor,

On Sun, Apr 25, 2010 at 8:03 PM, Igor Stasenko <[hidden email]> wrote:
> yes yes yes. :)

equally many thank yous! :-)

> Yes. I planning to document all of the things you may need for using it.
> Thanks for giving a scheme which i can follow for writing it.

Sure. If you want me to comment on drafts, throw them at me. I'm your
client, you see. :-)

Best,

Michael

On 25 April 2010 21:58, Michael Haupt <[hidden email]> wrote: Kewl. I'll let you know, once i'll have something to play with and to read :)

> Best,
>
> Michael
>
>



--
Best regards,
Igor Stasenko AKA sig.