Re: [Pharo-dev] [ANN] PharoNOS

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Eliot Miranda-2
 
Hi Jan,

On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
Hi Stef,
what I want, is NativeBoost for Pharo 5+. Because we could do this:

primInPortByte: portNumber 
	<primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin' error: errorCode>

	^ self nbCallout function: #(uint8 ( uint16 portNumber )) emit: [:gen | 
			| asm result proxy | 
			asm := gen asm.
			proxy := gen proxy.
			result := gen reserveTemp.
			asm
				pop: asm EDX;	"EDX = portNumber"
				push: asm EDI;	"save EDI"
				mov: result to: asm EDI; "EDI = temp address for result"
				db: 16rEC;		"IN AL, DX"
				pop: asm EDI.  "restore EDI"
			gen releaseTemps: 1.
		].

This is a rewrite of one method from C source(calling asm) in internal SqueakNOSPlugin into NativeBoost Smalltalk code. It calls privileged (kernel mode only) instruction to directly access HW I/O - disk access.
I have tried to run it, and it worked fine. However there is one little problem. NativeBoost needs method source, which is stored where? On the disk it wants to access... Solution - the source code can be embedded into compiled method trailer - this is only way to make it work. The first run will generate native code, and replaces the embedded source, so you can't touch the method anymore (at runtime)... so it is not perfect.
This is how SqueakNOSPlugin can be removed entirelly. But I'm afraid nobody can play with it in Pharo7 images...

Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
 
Save The World!


View this message in context: Re: [ANN] PharoNOS



--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

timrowledge
 

> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

melkyades
 
Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one. 

Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
 In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.

I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.

I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available

Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?

Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.

Eliot: does UFFI allow execution of arbitrary assembly?

Cheers!
Pocho

On Mon, Mar 20, 2017 at 6:57 PM, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"





--
Javier Pimás
Ciudad de Buenos Aires
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Igor Stasenko
In reply to this post by timrowledge
 

On 20 March 2017 at 23:57, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...


we've got ARM assembler in pharo. We even got virtual CPU that depending on platform could generate ARM/X86/x64 code
and then we got working alpha version of nativeboost that was using it.
but then everything vent to /dev/null since nobody wants to maintain it and keep working on it, including me, of course.
and i get tired advocating to people why it is so cool being able to directly control your hardware. They prefer to rely on C, 
even in places where C is completely inadequate for their tasks (like the above example with using privileged instruction(s))

 
tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"


--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Igor Stasenko
In reply to this post by melkyades
 


On 21 March 2017 at 15:34, Javier Pimás <[hidden email]> wrote:
 
Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one. 

Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
 In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.

I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.

I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available

Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?

Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.

Eliot: does UFFI allow execution of arbitrary assembly?

yes, if you write it in C :)
you can have a bike of any color, as long as its black :)
 

Cheers!
Pocho

On Mon, Mar 20, 2017 at 6:57 PM, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"





--
Javier Pimás
Ciudad de Buenos Aires




--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Eliot Miranda-2
In reply to this post by melkyades
 
Hi Javier,

On Tue, Mar 21, 2017 at 6:34 AM, Javier Pimás <[hidden email]> wrote:
 
Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one. 

Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
 In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.

I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.

I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available

See the work Clément and I have done on inline primitives and Ronie Salgado's work on Lowcode.  Here we extend the byte code set with low-level in-line primitives, but they are still cross-platform.  These primitives are unsafe; they do no checking, and expect particular kinds of operands.  The JIT translates them into the relevant machine instructions.

Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?

:-)

Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.

Ronie's work is probably the most relevant.
 

Eliot: does UFFI allow execution of arbitrary assembly?

No. But we have support for x86, x86_64 & ARM call-outs and callbacks.


Can you talk about things like the implementation of page fault interrupt handlers, etc?

Cheers!
Pocho

On Mon, Mar 20, 2017 at 6:57 PM, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"





--
Javier Pimás
Ciudad de Buenos Aires




--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Ben Coman
 
On Wed, Mar 22, 2017 at 7:28 AM, Eliot Miranda <[hidden email]> wrote:

>
> Hi Javier,
>
> On Tue, Mar 21, 2017 at 6:34 AM, Javier Pimás <[hidden email]> wrote:
>>
>>
>> Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one.
>>
>> Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
>>  In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.
>>
>> I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.
>>
>> I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available
>
>
> See the work Clément and I have done on inline primitives and Ronie Salgado's work on Lowcode.  Here we extend the byte code set with low-level in-line primitives, but they are still cross-platform.  These primitives are unsafe; they do no checking, and expect particular kinds of operands.  The JIT translates them into the relevant machine instructions.
>
>> Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?
>
>
> :-)
>
>> Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.
>
>
> Ronie's work is probably the most relevant.

For easy reference...
http://www.esug.org/data/ESUG2016/IWST/Papers/IWST_2016_paper_16.pdf
https://github.com/OpenSmalltalk/opensmalltalk-vm/search?p=1&q=lowcode&type=Commits&utf8=%E2%9C%93

build.linux32x86/
  + pharo.stack.spur.lowcode
  + pharo.cog.spur.lowcode

cheers -ben
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

melkyades
In reply to this post by Eliot Miranda-2
 


On Tue, Mar 21, 2017 at 8:28 PM, Eliot Miranda <[hidden email]> wrote:
 
Hi Javier,

On Tue, Mar 21, 2017 at 6:34 AM, Javier Pimás <[hidden email]> wrote:
 
Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one. 

Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
 In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.

I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.

I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available

See the work Clément and I have done on inline primitives and Ronie Salgado's work on Lowcode.  Here we extend the byte code set with low-level in-line primitives, but they are still cross-platform.  These primitives are unsafe; they do no checking, and expect particular kinds of operands.  The JIT translates them into the relevant machine instructions.

That may work if arm has in and out instructions as x86. But the thing is that the kind of code involved is hyper architecture specific. For example this is used to access keyboard and pic following protocols that I think apply only to IBM/PC.


Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?

:-)

Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.

Ronie's work is probably the most relevant.

We should have a look then.
 
 

Eliot: does UFFI allow execution of arbitrary assembly?

No. But we have support for x86, x86_64 & ARM call-outs and callbacks.


Can you talk about things like the implementation of page fault interrupt handlers, etc?

That was a nice thing we did with Guido. And regarding it there are some modifications we noticed we'll have to do with current code, where we'll need your help.

In x86 you have the IDT which is a table of interrupt descriptors, basically the addresses of the handlers and some extra bits. Most of SqueakNOS handlers are very similar: a piece of assembly code that saves all registers, signals a smalltalk semaphore (from native world), restores regs and quits. Actual handling of the interrupt is delayed: for each interrupt there's a high priority smalltalk process blocked waiting on its corresponding semaphore, so when reentering smalltalk world, the sleeping process is quickly awoken and handles the interrupt.

In the case of page-faults, interrupt handling cannot be delayed (you cannot return to the code that pagefaulted), so the handler saves vm status and reenters the VM immediately to deal with the interrupt. To be able to do so, VM status has to be saved, in order to restore it after the int has been handled. This works basically because the code that causes page faults is mostly controlled: pf are raised at smalltalk code, by hand (it'd probably crash for example if the VM itself generated a page fault). 

Status saving code was written by us by hand, maybe in the future this could be reified in a method of the interpreter that does the job. Going back to the thing that seems to have changed: vm status is stored in a bunch of variables in interp.c, that are now declared as static. Probably this was not the case before, and because of the static we cannot access it from other files. We have to find a solution for that (for now it is appending saving status funcs to interp.c).

I had forgotten how all this irq handling mechanism worked, it is nice to have it back in my mindmap.

Cheers!


Cheers!
Pocho

On Mon, Mar 20, 2017 at 6:57 PM, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"





--
Javier Pimás
Ciudad de Buenos Aires




--
_,,,^..^,,,_
best, Eliot




--
Javier Pimás
Ciudad de Buenos Aires
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

EstebanLM
In reply to this post by melkyades
 

On 21 Mar 2017, at 14:34, Javier Pimás <[hidden email]> wrote:

Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one. 

Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
 In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.

I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.

I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available

Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?

Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.

Eliot: does UFFI allow execution of arbitrary assembly?

that was more a question for me ;)
UFFI is just an FFI, it does not provides all the power NB provided. 

Then yes, it applies Eliot answer: you can call any library as long as they fulfils the ABI (which usually means: wrote in C). 

Esteban


Cheers!
Pocho

On Mon, Mar 20, 2017 at 6:57 PM, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"





--
Javier Pimás
Ciudad de Buenos Aires

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

EstebanLM
 

On 22 Mar 2017, at 09:56, Esteban Lorenzano <[hidden email]> wrote:


On 21 Mar 2017, at 14:34, Javier Pimás <[hidden email]> wrote:

Wow! it is really encouraging to see that you kept the candle burning, and the interest of the community. We have to integrate both what you and we did into one. 

Our approach started very recently and for now has only been focused in getting the VM compile and link with the upstream sources, to be able to run with current versions of pharo, without touching upstream files.
 In case any modifications are needed they can either be asked to integrate upstream o either be generated automatically against current sources via scripts. I believe this is the right way to do it, otherwise the code gets rotten really soon, and this is what we had started to do.

I see that we both took the same direction about using musl. Using parts of libc from system directories was a pain. We are able to cross-compile from 64-bits systems: ubuntu and osx.

I agree that having something like NativeBoost would be nice to be able to code all things that require assembly, specially io. It isn't a huge difference for now at it replaces a plugin that just executes in and out x86 instructions, but it'd make a nice plus to have available

Esteban: when you say "to make the memory executable" you forget this is squeaknos :). I mean, maybe we only need the plugin to work in squeaknos platform, not the others, would that make the task easier?

Tim: this is already architecture specific code, which already exists but at VM level; I would rather have it coded in Smalltalk.

Eliot: does UFFI allow execution of arbitrary assembly?

that was more a question for me ;)
UFFI is just an FFI, it does not provides all the power NB provided. 

Then yes, it applies Eliot answer: you can call any library as long as they fulfils the ABI (which usually means: wrote in C). 
sorry, Igor’s answer :)


Esteban


Cheers!
Pocho

On Mon, Mar 20, 2017 at 6:57 PM, tim Rowledge <[hidden email]> wrote:


> On 20-03-2017, at 2:48 PM, Eliot Miranda <[hidden email]> wrote:
>
>> Hi Jan,
>>
>> On Sun, Mar 19, 2017 at 1:46 PM, jan.struz <[hidden email]> wrote:
>> Hi Stef,
>> what I want, is NativeBoost for Pharo 5+. Because we could do this:
>>
>
> Can you not use the new UnifiedFFI which uses the ThreadedFFIPlugin at the bottom level?
>

Not to mention that native boost excludes ARM cpus. Which are the most numerous by quite a large factor...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 2) "My program has just dumped Stova Core!"





-- 
Javier Pimás
Ciudad de Buenos Aires

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Ronie Salgado
In reply to this post by Eliot Miranda-2
 
Hi,

With the experimental Lowcode backend for the UFFI, it is possible to call functions with almost arbitrary calling conventions. For some of the special assembly instructions that are required by this project, I could add the following extra instruction:
 - storeIntoSpecialRegister: {CR0, CR1, CR2, CR3, CR4, CR5, GDT, LDT, IDT)
 - loadFromSpecialRegister:
 - loadFromIOPortU?Int{8,16,32}:
 - storeIntoIOPortU?Int{8,16,32}:

As for the interruptions, there are far trickier because they can be triggered in any time. They probably would require some special trampoline for storing/restoring the VM state, something similar to the callbacks.

Currently I am finishing a refactoring of the Lowcode register allocator, so the Lowcode VM is out of service for this week :) .

Hey, guys, i'd like to stress, that AsmJit 
is quite standalone part of NB, and nothing prevents you to use it in own way, not the way NB was.. 
Because NB was born to make nice integration between machine code and VM , and providing FFI,
and as a demonstration that generating machine code IS possible and quite doable from within an image.

But there's a tons of ways how you could generate machine code, and then use it outside the scope of FFI.
You could just make own llittle primitive, that just makes a call on arbitrary address, and put your generated code on it.. Voila.
Again, you could store generated code not in method(s) trailer , but somewhere else.. it just up to you, after all.
For instance i had parts that were storing generated code directly in COG's JITter, replacing its primitive section.
So, that make a nice integration with JIT, that just using your code as a primitive for given method..
All is doable, when you working at lowest level, you just need to think how to expose it in convenient way.
 
For GPU compilation, I implemented a SSA based intermediate representation very similar to the one used by LLVM. As a experiment, I did a very basic machine code generator which can produce relocatable ELF32 and ELF64 object files, which can be linked with any C library or program. With this I managed to implement a basic ahead of time compiled Smalltalk runtime.

Before starting that backend, I took a look on AsmJit and the VirtualCPU and I did not like the dependencies and some details about the stack. However, now I do not like the low level backend so I started refactoring into a separate package, but I still think that I should take yet another look to AsmJit and the VirtualCPU, so I am wondering where is actually located the latest version of these two projects.

Best regards,
Ronie

2017-03-22 14:29 GMT-03:00 Igor Stasenko <[hidden email]>:

Hey, guys, i'd like to stress, that AsmJit 
is quite standalone part of NB, and nothing prevents you to use it in own way, not the way NB was.. 
Because NB was born to make nice integration between machine code and VM , and providing FFI,
and as a demonstration that generating machine code IS possible and quite doable from within an image.

But there's a tons of ways how you could generate machine code, and then use it outside the scope of FFI.
You could just make own llittle primitive, that just makes a call on arbitrary address, and put your generated code on it.. Voila.
Again, you could store generated code not in method(s) trailer , but somewhere else.. it just up to you, after all.
For instance i had parts that were storing generated code directly in COG's JITter, replacing its primitive section.
So, that make a nice integration with JIT, that just using your code as a primitive for given method..
All is doable, when you working at lowest level, you just need to think how to expose it in convenient way.

--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Pharo-dev] [ANN] PharoNOS

Igor Stasenko
 


On 23 March 2017 at 00:03, Ronie Salgado <[hidden email]> wrote:
 
Hi,

With the experimental Lowcode backend for the UFFI, it is possible to call functions with almost arbitrary calling conventions. For some of the special assembly instructions that are required by this project, I could add the following extra instruction:
 - storeIntoSpecialRegister: {CR0, CR1, CR2, CR3, CR4, CR5, GDT, LDT, IDT)
 - loadFromSpecialRegister:
 - loadFromIOPortU?Int{8,16,32}:
 - storeIntoIOPortU?Int{8,16,32}:

As for the interruptions, there are far trickier because they can be triggered in any time. They probably would require some special trampoline for storing/restoring the VM state, something similar to the callbacks.

Currently I am finishing a refactoring of the Lowcode register allocator, so the Lowcode VM is out of service for this week :) .

Hey, guys, i'd like to stress, that AsmJit 
is quite standalone part of NB, and nothing prevents you to use it in own way, not the way NB was.. 
Because NB was born to make nice integration between machine code and VM , and providing FFI,
and as a demonstration that generating machine code IS possible and quite doable from within an image.

But there's a tons of ways how you could generate machine code, and then use it outside the scope of FFI.
You could just make own llittle primitive, that just makes a call on arbitrary address, and put your generated code on it.. Voila.
Again, you could store generated code not in method(s) trailer , but somewhere else.. it just up to you, after all.
For instance i had parts that were storing generated code directly in COG's JITter, replacing its primitive section.
So, that make a nice integration with JIT, that just using your code as a primitive for given method..
All is doable, when you working at lowest level, you just need to think how to expose it in convenient way.
 
For GPU compilation, I implemented a SSA based intermediate representation very similar to the one used by LLVM. As a experiment, I did a very basic machine code generator which can produce relocatable ELF32 and ELF64 object files, which can be linked with any C library or program. With this I managed to implement a basic ahead of time compiled Smalltalk runtime.

Before starting that backend, I took a look on AsmJit and the VirtualCPU and I did not like the dependencies and some details about the stack. However, now I do not like the low level backend so I started refactoring into a separate package, but I still think that I should take yet another look to AsmJit and the VirtualCPU, so I am wondering where is actually located the latest version of these two projects.

IIRC, the stack (but more precisely - temporaries) management in AsmJIT are implemented in own classes, that doing the job of allocating stack space and assigning each temporary to specific place on stack.
As well, as calling convention. Nothing prevents you from adding own by implementing their protocol(s).
Or you may leave it as a whole, except that then you'll have to handle it by yourself. And of course, it is already there, since initial version has no such functionality at all,
all you have to do is to tell:
asm noStackFrame.
And so, it will leave you without it. 

As for where it located - see at smalltalkhub , if i remember Max  were working on that and made quite complete coverage of ARM instruction set. (http://smalltalkhub.com/#!/~maxmattone/AsmJit-ARM)
I don't remember if we merged the ARM support into main branch.. But i think you can figure it out.
 
P.S. ohh, you see.. it just never ending quest to keep dispelling people's perception, that there's something impossible or wrong with these things
or that's there are something that won't make things possible.. So, they choose to go with C :)

Best regards,
Ronie


--
Best regards,
Igor Stasenko.
Loading...