Using my asm-generator with Exypery.

Hi everyone, i'm very interesting in Exupery progress, so i decided to
join this list.

If you may know i implemented a small parser, which translates a
smalltalk source into asm instructions.
(if you missed my post on squeak-dev, you can read it here:
http://computeradvenrutes.blogspot.com/)

I have read some threads in archive and come to conclusion, that
Exupery itself can make a good use of my parser. Many primitives,
which handle smallints/largeints can be coded in smalltalk and then
translated using asm-parser to become low-level primitives.

Then they can be placed in code cache as standalone functions, or what
is more nicer - to be inlined by Exupery when compiling bytecodes of
smalltalk methods!

What is more important - it become easier to manage your low-level
code, and keep all things in same place, without need of recompiling
VM again and again.

All what you need then is to compile source code of some class, get
intermediate form of methods - and use these pieces of code for
inlining when generating a method from bytecode, or put a near/far
calls to it.

Consider following example:

you write the code:
------
PrimitiveSmallIntegers>>add: a with: b
| result |
   self pragma: #inline.  "tell the parser to not generate prologue/epilogue"

" adding pair of smallintegers (assuming a and b is smallintegers!!) "

^ ( result := a + b ) ifNotOverflow: [ result - 1 ] else: [ self call:
#coerceToBigIntsAndAdd:with: with: a with: b ]
-------
and then in

IntermediateGenerator>>generate:add:
^ self parsedMethod: #add:with:  "return inlined form for adding two operands"

It provides an easy way for extending Exupery with all basic numerics
functionality (like big integers/floats), without jungles of hardly to
write and hardly to manage mid-level code.

Do you like my idea?

--
regards
 sig.
_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

Excellent news, I could use some of that.
I am trying to write a new ST implementation, compiling ST methods to
machine code with exupery. But some parts (ie primitives) are still in C,
may be I can use some of your code and translate my kernel to ST.

What is the state of your project? (unfinished, testing, etc)
If I understood alright, your project is useful to translate Interpreter
(including primitives) and object memory, ok? (maybe translating  all the
VM to asm instead of C?)
How can we see it in action?

Cheers, Guille


_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

On 09/05/07, Guillermo Adrián Molina <[hidden email]> wrote:
> Excellent news, I could use some of that.
> I am trying to write a new ST implementation, compiling ST methods to
> machine code with exupery. But some parts (ie primitives) are still in C,
> may be I can use some of your code and translate my kernel to ST.
>
There are assembler instructions which currently not supported by
Exupery, so it may be problematic to get fully equivalent code in asm
for some primitives.
For instance, mov instructions support only 32 mem/reg operand size.
Also, NOT, XOR is missing (without these instruction i can't invert
operand, because logical set of instructions is not fully defined. If
there will be NOT, i can use it to do XOR, if there will be XOR i can
do NOT, but i cannot invert without at least one of them).
I would also like to see support of calls with relative offset, so i
can link all compiled functions in single byte array and properly set
relative calls where its needed.

I found only one way how to load a compiled code into executable
memory: its a 'loadCode' primitive.
It loads a given bytearray to code cache and returns a pointer to it.
The problem is, that i don't see a way how i can modify bytes in
already loaded chunk of code (for relocating calls).

BAH! I just thought, that problem can be simply solved by loading at
startup a small asm subroutine which will allow me to do that :)
simple assembly function accepting 2 arguments - pointer and value.. :)
See, how life can be easier: i have a way for extending functionality
based on what i have! I don't need to climb down to C and write
another plugin/primitive :)

> What is the state of your project? (unfinished, testing, etc)

Current state is unfinished. I must polish some things and add extra
functionality.
Parser is functional (at least by my analysis it generates correct
intermediate for given methods). To test it in action i'll need to
install exupery plugin. Btw, if someone can share a ready to use
squeak win32 executable with it, things will go faster..

> If I understood alright, your project is useful to translate Interpreter
> (including primitives) and object memory, ok? (maybe translating  all the
> VM to asm instead of C?)

No. And yes.
In other words, its not dictates you what will you do with your code.
You can run it, or simply fileout. Do anything you want :)
Of course, if you wish to replace some primitives with asm, this can
be done easily.
_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

sig writes:
 > On 09/05/07, Guillermo Adrián Molina <[hidden email]> wrote:
 > There are assembler instructions which currently not supported by
 > Exupery, so it may be problematic to get fully equivalent code in asm
 > for some primitives.
 > For instance, mov instructions support only 32 mem/reg operand size.
 > Also, NOT, XOR is missing (without these instruction i can't invert
 > operand, because logical set of instructions is not fully defined. If
 > there will be NOT, i can use it to do XOR, if there will be XOR i can
 > do NOT, but i cannot invert without at least one of them).
 > I would also like to see support of calls with relative offset, so i
 > can link all compiled functions in single byte array and properly set
 > relative calls where its needed.

If you want to add the missing instructions feel free, I'll merge
them into Exupery so long as they're written in a similar style and
have similar test coverage.

 > I found only one way how to load a compiled code into executable
 > memory: its a 'loadCode' primitive.
 > It loads a given bytearray to code cache and returns a pointer to it.
 > The problem is, that i don't see a way how i can modify bytes in
 > already loaded chunk of code (for relocating calls).
 >
 > BAH! I just thought, that problem can be simply solved by loading at
 > startup a small asm subroutine which will allow me to do that :)
 > simple assembly function accepting 2 arguments - pointer and value.. :)
 > See, how life can be easier: i have a way for extending functionality
 > based on what i have! I don't need to climb down to C and write
 > another plugin/primitive :)

Have a look at Exupery>>relocateMethodAt:

Exupery does relocation after loading code.

 > > What is the state of your project? (unfinished, testing, etc)
 >
 > Current state is unfinished. I must polish some things and add extra
 > functionality.
 > Parser is functional (at least by my analysis it generates correct
 > intermediate for given methods). To test it in action i'll need to
 > install exupery plugin. Btw, if someone can share a ready to use
 > squeak win32 executable with it, things will go faster..

There's both a Windows and a Linux VM here:

http://wiki.squeak.org/squeak/3945

Bryce
_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

sig writes:
 > Consider following example:
 >
 > you write the code:
 > ------
 > PrimitiveSmallIntegers>>add: a with: b
 > | result |
 >    self pragma: #inline.  "tell the parser to not generate prologue/epilogue"
 >
 > " adding pair of smallintegers (assuming a and b is smallintegers!!) "
 >
 > ^ ( result := a + b ) ifNotOverflow: [ result - 1 ] else: [ self call:
 > #coerceToBigIntsAndAdd:with: with: a with: b ]
 > -------
 > and then in
 >
 > IntermediateGenerator>>generate:add:
 > ^ self parsedMethod: #add:with:  "return inlined form for adding two operands"
 >
 > It provides an easy way for extending Exupery with all basic numerics
 > functionality (like big integers/floats), without jungles of hardly to
 > write and hardly to manage mid-level code.

Generating intermediate is not hard, and has good support for unit
testing.

For use in Exupery primitives must be implemented in the mid level
code because that exists to allow for easy optimization of waste
across bytecodes (or primitives).

For instance "a < b ifTrue: [...]"

Will remove the conversion to then from a boolean object for the
SmallInteger case. Only if a floating point < primitive was inlined
into medium level intermediate could this optimisation be done.

Which is similar to removing the temporary float created in "1.0 + 2.0
+ 3.0".

There are several different kinds of primitive:
* Small frequently called ones. e.g. #at: and #at:put:
* Larger important clean primitives (possibly large integer stuff)
* Larger or infrequent unclean primitives
* Interfacing to C
* Performance optimizations.

The small frequently called primitives are compiled into code
customized for each receiver class. There's only a handful of them
and most of them are inlined into interpret() in the interpreter.
interpret () accounts for 70% of the time in the benchmarks I've
analyzed so performance wise it's the area to focus on.

The larger clean primitives could be quickly implemented by calling
a C helper function. These could be either compiled as part of
their methods or inlined.

The larger infrequent unclean primitives are operations that may
change the active context. Here Exupery would need to save all
the state back into the context, call the C helper function, then
load it back and potentially enter a different method. The only
overhead that can be saved is some of the message sends by using
PICs.

Interfacing to C could use some support. I haven't looked at it.

Some of the performance optimizations should be replaced with
normal Smalltalk as Exupery get's faster. Some should stay as
tolerable interpreted performance is still nice.


To be inlined a primitive must be able to execute in both it's
senders context (in case it isn't inlined) and in it's senders
context (in case it is inlined). If you send, or GC from your
own context you must save the stack pointer and context state
while if you're in the senders context you mustn't save the
stack pointer. This is easy to implement using generated code
but not using a template.

Bryce
_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

On 08/05/07, [hidden email] <[hidden email]> wrote:
Yes, this is what you called high level itermediate, which is then
expanded to low level intermedite by doing optimizations before
entering low-level.
At this low-level point you already working with registers, not
objects, so you can inline asm-code generated by asm-generator.
I think that all IntermediateEmitter>>expanded:.... can be replaced by
inlining asm-methods which will have clean and easy to read syntax.

I am sure, that when you going to C level, you can use asm-generator
instead. For fully equivalent usage potential we need, of course, some
pointers from VM to be able calling another VM functions or access
globals.
i'm not sure what you talking about. A compiled asm primitive acts in
same manner as C function, so when you need to call GC, you call it
without asking anyone.
For inlining a med-code generated by asm-generator you can add a
special MedGC (or whatever) which expands to noop when its inlined and
expands to some calls when its standalone.
For example, parser puts MedScopeArgument at point where your
asm-method accessing the method argument. Then, when you generating
inline code, this med is replaced by supplied register, and for
standalone function, it replaced with instruction(s) which loading
value from stack, or from anywhere you want.
A low level med-code generated by parser is much like a #define macro
in C to me.
_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

in addition, i can add a pragma to check for inlining, so developper
could write:

myAsmMethod: arg
| a |
a:= self ifInlined: [ 10 ] else: [ 0 ]
^ arg + a
_______________________________________________
Exupery mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery