Compiling Pharo to C++
Locked
 
|
So doing further research into Unreal , I have reached the conclusion that the best way to integrate Pharo is make a compiler that takes Pharo code and compiles it to C++.
I have barely scratch such area when I was playing around with Pharo AST nodes. Any advice / ideas how to accomplish this ? Currently my goal is to start super simple, with compiling small pharo code fragments to C++ code and then much later on even introduce extensions to the pharo language to accommodate for C++ features like templates , pointers (FFI probably already cover this) , static types etc. Obviously if I can keep the pharo syntax intact would be the ideal option. |
|
Are you sure C++ compilation is the best way ? There are a lot of flaws: Smalltalk needs to be restrictive or will be compiled to very slow code, you will loose debugging features, etc. I would try to integrate Pharo the same way lua is integrated in engines like Source2 from valve: you run your Unreal engine and have it call Pharo as a scripting language to code some features. In such case, one needs to make the VM embeddable in a C++ app, but on the vm-dev mailing list people can help you to do so. Then you need to define some APIs based on what exist in the interpreter proxy to easily call Pharo methods and share struct / objects from C++. Lastly, you may consider adding a few settings in FFI. By default the Smalltalk stack and the C stack are separated to to Smalltalk specific stack page handling. I guess with some annotation in Pharo API code called from C++ you could define the case where the stack could be shared or not. While compiling to C++, how to you plan your compiler to manage memory ? 2016-02-16 9:52 GMT+01:00 kilon.alios <[hidden email]>: So doing further research into Unreal , I have reached the conclusion that |
Re: Compiling Pharo to C++
|
|
In reply to this post by kilon.alios
I think you totally underestimate the task.
Yes, Pharo (Smalltalk) is a pretty simple language (by design), so given a parsed AST (thanks Opal) it will not be so hard to transpile into some other language. But Pharo (Smalltalk) is much, much more: you need a sophisticated runtime (basically the VM) and a large part of the core libraries. Many people have worked very hard for decades to give us what we have today, this represents a huge amount of developer years. > On 16 Feb 2016, at 09:52, kilon.alios <[hidden email]> wrote: > > So doing further research into Unreal , I have reached the conclusion that > the best way to integrate Pharo is make a compiler that takes Pharo code and > compiles it to C++. > > I have barely scratch such area when I was playing around with Pharo AST > nodes. Any advice / ideas how to accomplish this ? > > Currently my goal is to start super simple, with compiling small pharo code > fragments to C++ code and then much later on even introduce extensions to > the pharo language to accommodate for C++ features like templates , pointers > (FFI probably already cover this) , static types etc. Obviously if I can > keep the pharo syntax intact would be the ideal option. > > > > -- > View this message in context: http://forum.world.st/Compiling-Pharo-to-C-tp4877775.html > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. > |
|
|
In reply to this post by kilon.alios
2016-02-16 9:52 GMT+01:00 kilon.alios <[hidden email]>: So doing further research into Unreal , I have reached the conclusion that even for C, there is no compiler that takes Pharo code and compiles it to C - binary code. what slang and vmmaker is doing, is more like a preprocessor, generating C - source code. Compiling C to code is done by an ordinary c-compiler, not smalltalk.
AST for pharo code is not used in any step generating vm code or FFI glue code AFAIK
The attractiveness of NativeBoost was, that it can generate (written in smalltalk) glue code for interacting with C-libraries. Call functions, provide (C-Type)-arguments and process return values. "Mirror" C-Structures in the pharo image side and generate accessors for objects living in the heap memory. And the new FFI can do this (differently) too. But this is again only for C-libraries. If Unity does not provide a c-binding, it is really difficult to interact with it through a FF-interface. C++ code uses (theoretically every c++ runtime its own) different naming scheme for functions. You can not load and call c++ code the same way as for C.
|
|
|
In reply to this post by Clément Béra
Ok first my apology to people because I see that many have misunderstood my motivation.
So let me be crystal clear. This IS NOT an effort to make Pharo "super cool". "Look mom Pharo compiles to C++ magically, it runs blazzing fast, it can use C++ libraries with zero efforts" and similar none sense. Unfortunately I cannot use Pharo as a scripting language that would be awesome but I afraid not possible. Compiling Unreal as a set of dlls is somewhat possible but the problem is that iOS does not allow dlls, iOS support is super crucial for me so that goes out of the window. Embeding Pharo in Unreal is no go either since pharo does not support this. I am not looking into bringing Pharo features to C++. For example I dont care about dynamic types because Unreal depends a lot on templates which are very dynamic, I dont care about Pharo GC because unreal has its own GC, I dont care about Pharo reflection because Unreal also gives C++ reflection features, I dont even care about Pharo live coding because Unreal also allow to hot reload on the fly when you recompile C++ code. WARNING !!! I am not saying that coding C++ in Unreal is better , more dynamic or more live than Pharo. Pharo is awesome. So my thinking is like this, I cannot avoid C++. So since I cannot avoid it then maybe I could try to make it more enjoyable by making it move closer to pharo syntax. That means my implementation will do a lot of nasty compromises, meaning manual memory managment, disallow dynamic types (though later on with the help of templates maybe I can make it more dynamic) , have to include headers etc. So in general write C++ code using something that is close to pharo syntax. My goal is not an ideal solution but a practical solution something that is relative simple to make for a single developer. It wont be a tool you guys will be using because it will be too tied to Unreal anyway and it will contain the nasty parts of C++. The idea will be similar to what Cython is for python, start prototyping in pure python with all the great features it has, then when code works as you expect start inserting static types, manual memory management etc to turn it to C++. I hope now I am much more clear what I aiming for |
|
|
Hi Kilon, the way your are explaining it, it seems you are trying to build a Smalltalk-like syntax in C++ via templates? It could make moving a software architecture from Smalltalk to C++ significantly easier (and open maybe some opportunities). Thierry 2016-02-16 11:00 GMT+01:00 kilon.alios <[hidden email]>: Ok first my apology to people because I see that many have misunderstood my |
|
|
In reply to this post by Nicolai Hess-3-2
"Are you sure C++ compilation is the best way ? There are a lot of flaws: Smalltalk needs to be restrictive or will be compiled to very slow code, you will loose debugging features, etc.
I would try to integrate Pharo the same way lua is integrated in engines like Source2 from valve: you run your Unreal engine and have it call Pharo as a scripting language to code some features. In such case, one needs to make the VM embeddable in a C++ app, but on the vm-dev mailing list people can help you to do so. Then you need to define some APIs based on what exist in the interpreter proxy to easily call Pharo methods and share struct / objects from C++. Lastly, you may consider adding a few settings in FFI. By default the Smalltalk stack and the C stack are separated to to Smalltalk specific stack page handling. I guess with some annotation in Pharo API code called from C++ you could define the case where the stack could be shared or not. While compiling to C++, how to you plan your compiler to manage memory ? " I am not able to use Pharo as scripting language for Unreal I am afraid the easy way. I cannot use FFI for reason I describe later on. How I plan my compiler to manage memory , it wont, the compiler will expect very close to C++ syntax and features, it will do minimal work at its side so it will be up to the coder. "I think you totally underestimate the task. Yes, Pharo (Smalltalk) is a pretty simple language (by design), so given a parsed AST (thanks Opal) it will not be so hard to transpile into some other language. But Pharo (Smalltalk) is much, much more: you need a sophisticated runtime (basically the VM) and a large part of the core libraries. Many people have worked very hard for decades to give us what we have today, this represents a huge amount of developer years. " I think you overestimate what I am trying to do, I am not trying to recreate Pharo. There wont be a reimplementation of the VM , or the pharo libraries. This is a simple tool that take pharo-like and turns it to C++ making zero assumptions. That means it will be up to the coder to make sure that what it makes is proper C++. It wont be a magical tool that takes pharo code and turns it to C++ no matter what. There will be a ton of limitations on how you write the code to make sure it produces readable and proper C++ code. The advantage of using it over pure Pharo code ? NONE !!! and NOT RECOMMENDED !!! The advantage of using it over pure C++ code ? a) prototype in pure Pharo code using all the advantages and tools of Pharo IDE b) turn it to pharo like C++ code without having to rewrite your code from scratch in C++ "even for C, there is no compiler that takes Pharo code and compiles it to C - binary code. what slang and vmmaker is doing, is more like a preprocessor, generating C - source code. Compiling C to code is done by an ordinary c-compiler, not smalltalk." we call it "compiler" when we have a tool that take one input and turns it to another output. I am well aware that the compiling is done by ordinary C compiler to machine code. I did not mean that I want to compile pharo code to machine code, but pharo code to C++ code and to be precise Unreal friendly C++ code. "AST for pharo code is not used in any step generating vm code or FFI glue code AFAIK" The way I understand it is that AST is used to take pharo code and brake it down to individual part that is the feed into bytecode compiler and the vm then turns bytecode to machine JIT or interprets it to machine depending on the optimisation. Again I am not concerned with the binary part this is taking one source code and turning to another source code, so it has nothing to do with VM, FFI and machine code. "If Unity does not provide a c-binding, it is really difficult to interact with it through a FF-interface. C++ code uses (theoretically every c++ runtime its own) different naming scheme for functions. You can not load and call c++ code the same way as for C. " My engine is Unreal not Unity. Unreal assumes that you either write C++ code or using their own scripting language "Blueprint". Its not designed to be used as a library so making wrappers for Pharo via FFI is not possible. The main reason is that iOS allows only for static executables. |
|
|
In reply to this post by Thierry Goubier
"Hi Kilon,
the way your are explaining it, it seems you are trying to build a Smalltalk-like syntax in C++ via templates? It could make moving a software architecture from Smalltalk to C++ significantly easier (and open maybe some opportunities). Thierry" Bingo ! You get it ! Thats what is all about about moving code from Pharo to C++ and NOT reimplementing Pharo in C++ as some people assumed. Yes if I can go for smalltalk-syntax that would be a fair compromise, ideally i want to keep Pharo syntax intact and introduce C++ features like static types , pointers etc through the use of pharo objects. For example I can make an object to represent a C++ int, what stops me from taking pharo class variable make it initialise with that int object then later on use a string instead ? Nothing, my compiler wont even complain, but when you turn it to C++ code and try to compile it you will get C++ errors. On the subject of templates, again there wont be magical transormation of dynamic types to C++ templates at least not in the start since C++ expect to be explicity about the use of a template that will carry on to my tool as well. That means that the pharo code will have to be explicit and leave nothing to chance like a C++ coder does. Essentially you coding C++ using pharo syntax or pharo like syntax depending on how succesful I will be with this. Also creating readable code is a must do. For example I have taken a look at chicken and gambit scheme that take scheme (lisp variant) and turn to C and then call the c compiler to make the machine code executable. The problem is that C code is unreeadable because it tries to emulate dynamic types etc. My goal is so not to retain Pharo features and dynamism but rather to produce C++ that is not only readable but proper C++ code with comments , good names, etc. My thinking however is like this: 1) Prototype in Pharo and make sure it works well using all the features and tools of pharo (dynamic types, GC, live coding, live debugging etc) 2) convert your code to my language that looks a lot like pharo but with all the C++ features leaving no ambiguities for the compiler 3) use my compiler to turn a specific fragment of code to C++ 4) go back to to (1) 3) |
|
|
Ok. The slang compiler parses the code using the Smalltalk Compiler parser, then translate the Smalltalk AST to its own AST, do some manipulation based on pragmas available in Slang methods and inlines different things based on a closed world (you can know what method is called at each send site) / frozen state assumptions (The value of literal variable will not change). Some selectors are specifically mapped to C instructions / macros, for example #cppIf:ifTrue: maps to #IFDEF, or bitShift: maps to signed long bitShift. Comment are translated from Smalltalk to C and they are really useful. Another interesting thing is that due to inlining part of the allocation / deallocation can be avoided. You can't really use Slang directly as its implementation is tied with the VM (each time we have a slang bug in the vm code, we choose to either change the slang compiler or the slang code depending on how much work is each task). I am not saying that slang is a nice language and that you should do something similar. Slang is horrible. But in the case of the VM is has some significant advantages over C++ and method templates. 2016-02-16 11:30 GMT+01:00 kilon.alios <[hidden email]>: "Hi Kilon, |
|
|
In reply to this post by Clément Béra
> Unreal assumes that you either write C++ code
> or using their own scripting language "Blueprint". Not entirely true [1].... [1] https://forums.unrealengine.com/showthread.php?3958-Scripting-Language-extensions-via-plugins On Tue, Feb 16, 2016 at 5:42 PM, Clément Bera <[hidden email]> wrote: > Are you sure C++ compilation is the best way ? There are a lot of flaws: > Smalltalk needs to be restrictive or will be compiled to very slow code, you > will loose debugging features, etc. > > I would try to integrate Pharo the same way lua is integrated in engines > like Source2 from valve: you run your Unreal engine and have it call Pharo > as a scripting language to code some features. Something like [2], scrolling down to the #include ? [2] http://accu.org/index.php/journals/351 What would be the Cog equivalent of...? lua_State* L = lua_open(); and what would be a good starting point to look at this? main() and fakevm() in threadValidate.c [3] seem to lay out a basic structure. [3] https://git.io/vg7bn I see interpret() called from: * platforms/unix/vm/sqUnixMainNoDisplay.c [4] and * iOS/vm/Common/Classes/sqSqueakMainApplication.m [5] but can't see where this is implemented. [4] https://git.io/vg5fB [5] https://git.io/vg5JS I guess the new SDL derived headless mode mentioned recently would help with embedding Pharo in another program? cheers -ben > In such case, one needs to > make the VM embeddable in a C++ app, but on the vm-dev mailing list people > can help you to do so. Then you need to define some APIs based on what exist > in the interpreter proxy to easily call Pharo methods and share struct / > objects from C++. Lastly, you may consider adding a few settings in FFI. By > default the Smalltalk stack and the C stack are separated to to Smalltalk > specific stack page handling. I guess with some annotation in Pharo API code > called from C++ you could define the case where the stack could be shared or > not. > > While compiling to C++, how to you plan your compiler to manage memory ? > > > 2016-02-16 9:52 GMT+01:00 kilon.alios <[hidden email]>: >> >> So doing further research into Unreal , I have reached the conclusion that >> the best way to integrate Pharo is make a compiler that takes Pharo code >> and >> compiles it to C++. >> >> I have barely scratch such area when I was playing around with Pharo AST >> nodes. Any advice / ideas how to accomplish this ? >> >> Currently my goal is to start super simple, with compiling small pharo >> code >> fragments to C++ code and then much later on even introduce extensions to >> the pharo language to accommodate for C++ features like templates , >> pointers >> (FFI probably already cover this) , static types etc. Obviously if I can >> keep the pharo syntax intact would be the ideal option. |
|
|
In reply to this post by Clément Béra
"Then I guess you can do something like Slang, which is used for the VM. Slang is a restrictive Smalltalk compiling to C. "
I dont think Slang is close to what I am trying to do, last time I checked Slang out it appeared to me that Slang is trying to create as much smalltalk syntax as possible. My goal is the other way around to create C++ code by sticking as close to C++ syntax as possible but still using syntax that will make porting pharo code easier. " The slang compiler parses the code using the Smalltalk Compiler parser, then translate the Smalltalk AST to its own AST, " Yeap thats the approach I am interested into as well. Take the Smalltalk AST and translate to an AST very close to C++. I am not interested into creating an in between language. "I am not saying that slang is a nice language and that you should do something similar. Slang is horrible. " Why Slang is horrible according to you ? "But in the case of the VM is has some significant advantages over C++ and method templates." Indeed the VM is awesome but as I said my main goal is not to keep the pharo features but rather to produce very readable C++ code. So I have little problem sacrificing those features. I probably end up making my own variant of the Pharo language that can happily coexist with the existing pharo language. "Not entirely true [1].... [1] https://forums.unrealengine.com/showthread.php?3958-Scripting-Language-extensions-via-plugins " Still Pharo is no Lua and it cannot embed itself. What I meant was that its not actively supported. Also its not that simple because one has to expose C++ to the scripting language. I rather have something that produces C++ and I can easily integrate. Or maybe there is something I am missing here and its far easier than I assume. Even without your link embeding lua and python in existing C++ executable is not that hard. Those language offer good support for this and they are heavily used for this task. I also spoke with Esteban about embedding pharo and he told me that pharo has a long way to go before it is able to do this. So for the time being I will start playing around with the Pharo AST and see if I can start generate super simple C++ code out of it and see how hard really it is and move step by step, thank you all for the suggestions. |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |