Hi All, today I committed the threaded FFI plugin along with the threaded VM and hope to be able to support people in doing threaded FFI work on the x86 platforms really soon now (Esteban, amongst others, this means you ;) ). But to put the image-level code that uses the new FFI plugin into production the standard VM needs to be upgraded to not crash with the ExternalFunction image changes the new threaded FFI plugin wants. The main change is that ExternalFunction changes size, gaining a stackSize inst var which is used to tell the FFI plugin how much stack spacer to reserve for marshalling (its computed on first use). The best way for this to happen is to start using the ThreadedFFIPlugin as the standard plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler than the old FFI plugin (no assembler support code) and better (fully reentrant). But for this to happen we may need to implement the platform=specific parts of the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC and ARM (and perhaps others, SPARC??). I understand what has to be done and I can explain it to anyone interested in working on the problem, but I don't have any hardware with which I can test the results so its pretty pointless my doing the work and producing something inevitably broken. So are there people out there who would be interested in doing the port to PPC and ARM? It means fleshing out the classes
ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC and having a solid understanding of how alloca works to do stack allocation and how the ABIs on ARM and PPC pass registers.
here's hoping there are a few good low-level people who would be interested. Let me know. --
best, Eliot |
On 22 July 2011 02:34, Eliot Miranda <[hidden email]> wrote: > Hi All, > today I committed the threaded FFI plugin along with the threaded VM and > hope to be able to support people in doing threaded FFI work on the x86 > platforms really soon now (Esteban, amongst others, this means you ;) ). > But to put the image-level code that uses the new FFI plugin into > production the standard VM needs to be upgraded to not crash with the > ExternalFunction image changes the new threaded FFI plugin wants. The main > change is that ExternalFunction changes size, gaining a stackSize inst var > which is used to tell the FFI plugin how much stack spacer to reserve for > marshalling (its computed on first use). The best way for this to happen is > to start using the ThreadedFFIPlugin as the standard plugin in the > Interpreter VM. The ThreadedFFIPlugin overall is simpler than the old FFI > plugin (no assembler support code) and better (fully reentrant). But for > this to happen we may need to implement the platform=specific parts of > the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC and ARM > (and perhaps others, SPARC??). I understand what has to be done and I can > explain it to anyone interested in working on the problem, but I don't have > any hardware with which I can test the results so its pretty pointless my > doing the work and producing something inevitably broken. So are there > people out there who would be interested in doing the port to PPC and ARM? > It means fleshing out the classes > ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM > ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC > and having a solid understanding of how alloca works to do stack allocation > and how the ABIs on ARM and PPC pass registers. > here's hoping there are a few good low-level people who would be interested. > Let me know. I wonder if you find anyone today using these archs, not saying using squeak/pharo on them + knowing low-level details + wanting to help :) > -- > best, > Eliot > -- Best regards, Igor Stasenko AKA sig. |
In reply to this post by Eliot Miranda-2
+1 On Jul 22, 2011, at 10:07 AM, Janko Mivšek wrote: > Wow, Eliot, thanks for that so long waited function a lot! > > Threaded FFI is namelly a must for Squeak/Pharo to be used seriously > everywhere where a connection to the other worlds is needed. Which is a > common situation when working on "enterprise" level information systems. > > Thanks again! Thanks also to all investors/donators who invested to this > so important work! > > Janko > > S, Eliot Miranda piše: >> Hi All, >> >> today I committed the threaded FFI plugin along with the threaded VM >> and hope to be able to support people in doing threaded FFI work on the >> x86 platforms really soon now (Esteban, amongst others, this means you >> ;) ). But to put the image-level code that uses the new FFI plugin into >> production the standard VM needs to be upgraded to not crash with the >> ExternalFunction image changes the new threaded FFI plugin wants. The >> main change is that ExternalFunction changes size, gaining a stackSize >> inst var which is used to tell the FFI plugin how much stack spacer to >> reserve for marshalling (its computed on first use). The best way for >> this to happen is to start using the ThreadedFFIPlugin as the standard >> plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler >> than the old FFI plugin (no assembler support code) and better (fully >> reentrant). But for this to happen we may need to implement the >> platform=specific parts of the ThreadedFFIPlugin (still entirely >> Smalltalk code) for PowerPC and ARM (and perhaps others, SPARC??). I >> understand what has to be done and I can explain it to anyone interested >> in working on the problem, but I don't have any hardware with which I >> can test the results so its pretty pointless my doing the work and >> producing something inevitably broken. So are there people out there >> who would be interested in doing the port to PPC and ARM? It means >> fleshing out the classes >> ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM >> ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC >> and having a solid understanding of how alloca works to do stack >> allocation and how the ABIs on ARM and PPC pass registers. >> >> here's hoping there are a few good low-level people who would be interested. >> Let me know. >> -- >> best, >> Eliot >> > > -- > Janko Mivšek > Aida/Web > Smalltalk Web Application Server > http://www.aidaweb.si > |
In reply to this post by Eliot Miranda-2
In september Guido and Javier will be visiting us for 3 months and one of them will be payed to work on the project we exchanged about some months ago (header C parser and ....) So at least we will get that. Stef On Jul 22, 2011, at 1:34 AM, Eliot Miranda wrote: > Hi All, > > today I committed the threaded FFI plugin along with the threaded VM and hope to be able to support people in doing threaded FFI work on the x86 platforms really soon now (Esteban, amongst others, this means you ;) ). But to put the image-level code that uses the new FFI plugin into production the standard VM needs to be upgraded to not crash with the ExternalFunction image changes the new threaded FFI plugin wants. The main change is that ExternalFunction changes size, gaining a stackSize inst var which is used to tell the FFI plugin how much stack spacer to reserve for marshalling (its computed on first use). The best way for this to happen is to start using the ThreadedFFIPlugin as the standard plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler than the old FFI plugin (no assembler support code) and better (fully reentrant). But for this to happen we may need to implement the platform=specific parts of the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC and ARM (and perhaps others, SPARC??). I understand what has to be done and I can explain it to anyone interested in working on the problem, but I don't have any hardware with which I can test the results so its pretty pointless my doing the work and producing something inevitably broken. So are there people out there who would be interested in doing the port to PPC and ARM? It means fleshing out the classes > ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM > ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC > and having a solid understanding of how alloca works to do stack allocation and how the ABIs on ARM and PPC pass registers. > > here's hoping there are a few good low-level people who would be interested. > Let me know. > -- > best, > Eliot > > |
In reply to this post by Igor Stasenko
On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran <[hidden email]> wrote: Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time. But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
-- best, Eliot |
On Jul 22, 2011, at 6:25 PM, Eliot Miranda wrote: > > > On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran <[hidden email]> wrote: > Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time. > > But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on. > > So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :) Personally we would love but to do that you need to know. This is the key question behind increasing the mass of people knowing. This is why we organize the School and why the blog of mariano is good or the effort of igor to make everybody been able to compile the vm. It is a slow and long process. There is a difference between will and can. At least to me there are plenty of things I would like to do but I'm quite sure I will not be able to do them in this life. Stef |
In reply to this post by Eliot Miranda-2
On Fri, Jul 22, 2011 at 1:25 PM, Eliot Miranda <[hidden email]> wrote:
Hi Eliot, I would love to help but I don't have the hardware nor the knowledge to do it. Sorry... Cheers, Richo
|
In reply to this post by Eliot Miranda-2
On Fri, 22 Jul 2011, Eliot Miranda wrote: > But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some > way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me > put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI > as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need > help on the platforms I can't justify working on. > > So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. > Let's defer gratification :) QEMU can be used for emulating various CPUs, so if you don't have an ARM, PPC or SPARC hardware, you can still develop/test FFI with it. Threaded FFI is great, it opens new possibilities, but nowadays the feature I miss the most is the ability to call functions in 64-bit (x86_64) libraries (from a 32-bit VM/image). Levente |
On Sat, Jul 23, 2011 at 01:50:49PM +0200, Levente Uzonyi wrote: > > Threaded FFI is great, it opens new possibilities, but nowadays the > feature I miss the most is the ability to call functions in 64-bit > (x86_64) libraries (from a 32-bit VM/image). As far as I know, mixing 32-bit and 64-bit libraries in a single executable (VM) is not feasible, at least not on Linux. Maybe it would work in the case of statically linked libraries, I'm not certain. The traditional FFI does not work for 64-bit VMs, and I expect that this needs to be resolved for any new FFI implementation as well, because some of the issues were in the interpreter itself, not just the FFI plugin. The issue also may affect other things, e.g. Igor's proposed #primLoadModule primitive crashes the VM if compiled for 64 bit. I had intended to push the fixes for 64-bit FFI as a VM team initiative, but it's a complex set of changes that require coordination of VMMaker (oscog plus traditional), as well as of all the platforms (oscog, trunk, all platforms) so IMO this would be a lot to take on right now. But I guess that I should ask - is there interest in implementing fixes for 64-bit FFI? Some background: Mantis report and patches to implement the fixes, circa 2008. I assume some bit rot will have set in by this time: <http://bugs.squeak.org/view.php?id=7237> Status of the fixes, as tested on various platforms: <http://lists.squeakfoundation.org/pipermail/vm-dev/2008-May/001945.html> The last discussion relative to VM team priorities: <http://lists.squeakfoundation.org/pipermail/vm-dev/2009-July/002853.html> Dave |
Free forum by Nabble | Edit this page |