Re: Sista alternate bytecodes + Java Bytecode ?
Locked
Re: Sista alternate bytecodes + Java Bytecode ?
 
|
Hi Ben, in addition to Clément's observations, you should not forget that much existing JVM bytecode depends on the multi-threading capabilities of any contemporary JVM implementation and its supporting platform libraries (e.g., java.util.concurrent.* for concurrency frameworks, java.util.stream.* for parallel streams, etc.) So, besides the humongous work mentioned by Clément, I can only imagine how hyper-humongous a work would it be to make the current OpenSmalltalk VM multi-threaded to accommodate the JVM computational model inherent in almost any real-world Java bytecode. Think only of the impact on the garbage collectors, for example. (Or on how to present multi-threading to Smalltalkers accustomed to the vastly simpler shared memory model of Process, where the happens-before relationship is trivial.) Greetings Raffaello On 2017-10-17 16:12, Raffaello Giulietti wrote: > Hi, > > That question should be asked on vm-dev. > > Just a detail, multibytecode set support is prior to the Sista > implementation, it was used for example in the context of Newspeak (Squeak > and Newspeak bytecodes supported together). Sista uses that feature, which > makes it look like new in the Pharo community since afaik no one used it > before in the context of Pharo. > > Then yes it is feasible to have the VM process Java bytecode, but it's > humongous work. On the top my head, we would need to support typed > bytecodes for Numbers and exceptions at the bytecode level (which in itself > is month of work, imagine raising an exception from Java/ST and catch it > from the other runtime, with multiple ensure/try finally in between) > > Note that supporting the Java bytecode is far from enough to be able to run > Java code (We also need core Java libraries support at least, including > exceptions/stack reification model that should be able to interact with > each other) > > You can have a look at Smalltalk/X where the VM can run Java, C and > Smalltalk. > > Regards > > > On Tue, Oct 3, 2017 at 1:57 PM, Ben Coman <btc at openinworld.com> wrote: > > > Just wondering about an exotic idea, with Sista facilitating alternate > > bytecodes (btw, is that multiple bytecode sets within one Image?), how > > feasible would it be to have the VM process Java bytecode? > > > > cheers -ben > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20171003/4c21a419/attachment.html> > > |
|
|
Hi there, On Tue, 2017-10-17 at 16:53 +0200, Raffaello Giulietti wrote: > > Hi Ben, > > in addition to Clément's observations, you should not forget that > much > existing JVM bytecode depends on the multi-threading capabilities of > any > contemporary JVM implementation and its supporting platform > libraries > (e.g., java.util.concurrent.* for concurrency frameworks, > java.util.stream.* for parallel streams, etc.) > > So, besides the humongous work mentioned by Clément, I can only > imagine > how hyper-humongous a work would it be to make the current > OpenSmalltalk > VM multi-threaded to accommodate the JVM computational model inherent > in > almost any real-world Java bytecode. In my opinion, the "vastly simpler" model of current smalltalk VMs makes the implementation of Java execution on top of them simpler, not more difficult. As for the real-world code, I do not recall any difficulties w.r.t. smalltalk's "vastly simpler" model. Jan > > > > On 2017-10-17 16:12, Raffaello Giulietti wrote: > > Hi, > > > > That question should be asked on vm-dev. > > > > Just a detail, multibytecode set support is prior to the Sista > > implementation, it was used for example in the context of Newspeak > > (Squeak > > and Newspeak bytecodes supported together). Sista uses that > > feature, which > > makes it look like new in the Pharo community since afaik no one > > used it > > before in the context of Pharo. > > > > Then yes it is feasible to have the VM process Java bytecode, but > > it's > > humongous work. On the top my head, we would need to support typed > > bytecodes for Numbers and exceptions at the bytecode level (which > > in itself > > is month of work, imagine raising an exception from Java/ST and > > catch it > > from the other runtime, with multiple ensure/try finally in > > between) > > > > Note that supporting the Java bytecode is far from enough to be > > able to run > > Java code (We also need core Java libraries support at least, > > including > > exceptions/stack reification model that should be able to interact > > with > > each other) > > > > You can have a look at Smalltalk/X where the VM can run Java, C and > > Smalltalk. > > > > Regards > > > > > > On Tue, Oct 3, 2017 at 1:57 PM, Ben Coman <btc at openinworld.com> > > wrote: > > > > > Just wondering about an exotic idea, with Sista facilitating > > alternate > > > bytecodes (btw, is that multiple bytecode sets within one > > Image?), how > > > feasible would it be to have the VM process Java bytecode? > > > > > > cheers -ben > > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/201 > > 71003/4c21a419/attachment.html> > > > > > > |
Re: Sista alternate bytecodes + Java Bytecode ?
|
|
Hi Jan, I'm not arguing whether the complex JVM model (java.lang.Thread) is better than the simpler Smalltalk model (Process). What I mean is that to execute existing JVM bytecode, the VM must also support genuine multi-threading, with all the related intricacies stemming from its memory model. Without MT, many useful, mature, maintained JVM libraries out there cannot work, as they depend, whether knowingly or not, whether directly or indirectly, on MT support from the VM. I understand Ben's dream to have a Smalltalk VM able to execute JVM bytecode. But as long as one cannot execute arbitrary JVM code, I wonder what the value of extending the Smalltalk VM to support it would be, given the huge efforts this task would require even without MT support. Raffaello On 2017-10-17 18:04, Jan Vrany wrote: > > Hi there, > > On Tue, 2017-10-17 at 16:53 +0200, Raffaello Giulietti wrote: >> >> Hi Ben, >> >> in addition to Clément's observations, you should not forget that >> much >> existing JVM bytecode depends on the multi-threading capabilities of >> any >> contemporary JVM implementation and its supporting platform >> libraries >> (e.g., java.util.concurrent.* for concurrency frameworks, >> java.util.stream.* for parallel streams, etc.) >> >> So, besides the humongous work mentioned by Clément, I can only >> imagine >> how hyper-humongous a work would it be to make the current >> OpenSmalltalk >> VM multi-threaded to accommodate the JVM computational model inherent >> in >> almost any real-world Java bytecode. > > In my opinion, the "vastly simpler" model of current smalltalk VMs > makes the implementation of Java execution on top of them simpler, not > more difficult. > > As for the real-world code, I do not recall any difficulties w.r.t. > smalltalk's "vastly simpler" model. > > Jan > >> >> >> >> On 2017-10-17 16:12, Raffaello Giulietti wrote: >>> Hi, >>> >>> That question should be asked on vm-dev. >>> >>> Just a detail, multibytecode set support is prior to the Sista >>> implementation, it was used for example in the context of Newspeak >>> (Squeak >>> and Newspeak bytecodes supported together). Sista uses that >>> feature, which >>> makes it look like new in the Pharo community since afaik no one >>> used it >>> before in the context of Pharo. >>> >>> Then yes it is feasible to have the VM process Java bytecode, but >>> it's >>> humongous work. On the top my head, we would need to support typed >>> bytecodes for Numbers and exceptions at the bytecode level (which >>> in itself >>> is month of work, imagine raising an exception from Java/ST and >>> catch it >>> from the other runtime, with multiple ensure/try finally in >>> between) >>> >>> Note that supporting the Java bytecode is far from enough to be >>> able to run >>> Java code (We also need core Java libraries support at least, >>> including >>> exceptions/stack reification model that should be able to interact >>> with >>> each other) >>> >>> You can have a look at Smalltalk/X where the VM can run Java, C and >>> Smalltalk. >>> >>> Regards >>> >>> >>> On Tue, Oct 3, 2017 at 1:57 PM, Ben Coman <btc at openinworld.com> >>> wrote: >>> >>> > Just wondering about an exotic idea, with Sista facilitating >>> alternate >>> > bytecodes (btw, is that multiple bytecode sets within one >>> Image?), how >>> > feasible would it be to have the VM process Java bytecode? >>> > >>> > cheers -ben >>> > >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: >>> <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/201 >>> 71003/4c21a419/attachment.html> >>> >>> >> >> |
Re: Sista alternate bytecodes + Java Bytecode ?
|
|
> On 18. Oct 2017, at 01:09, Raffaello Giulietti <[hidden email]> wrote: > > Hi Jan, Hey! > I'm not arguing whether the complex JVM model (java.lang.Thread) is better than the simpler Smalltalk model (Process). > > What I mean is that to execute existing JVM bytecode, the VM must also support genuine multi-threading, with all the related intricacies stemming from its memory model. Without MT, many useful, mature, maintained JVM libraries out there cannot work, as they depend, whether knowingly or not, whether directly or indirectly, on MT support from the VM. my cents... Many people confuse (including myself) concurrency with parallelism. There is concurrency in Smalltalk and Java but sadly the current VM offers[1] no parallelism. I haven't looked at Java bytecode > 1.5 but what instructions require _parallelism_? Last time (university) I checked the Java memory model allowed "delays" of when a store becomes visible in other threads but didn't require it. What OpensmalltalkVM has is that one running Process can be interrupted because of IO or other events. Could you elaborate on what runtime behavior Java libraries expect? In the past Paolo Bonzini added Java support[2] to GNU Smalltalk using GNU Classpath by implementing the primitives in Smalltalk, a bytecode converted and custom class loader. Last but not least. I would love to have parallelism in Pharo but the "share everything" approach (and put a giant lock everywhere, then more granular) seems to not be the future. E.g. the Erlang actors (yeah coming from somewhere else with Smalltalk legacy) or CSP as used in Go might be better? And to completely derail this thread (none of this being my idea)... maybe the future is in a meta-vm... dispatching[3] tiny images over a number of CPU cores and machines(!) sending messages and objects from one image to another (with VM support to avoid the copying of them). Atomically replacing one image (and externalizing/internalizing the state) with a metaBecome:. regards holger [1] Leaving threaded alien aside [2] http://git.savannah.gnu.org/cgit/smalltalk.git/tree/packages/java?h=3.2.5&id=4dc033eb76d82ff22266e7689c82a0ecd17ae3c6 [3] We have already seen running multiple images on one VM, and spawning new images from one. |
Re: Sista alternate bytecodes + Java Bytecode ?
|
Well technically its feasible outside the standard scope of the VM, we have afterall multithreaded FFI (OS threads are parrarel), I assume, though never used it that it requires for all threads to join the main one in order to be used. Maybe something could be done similarly for JVM as well, join its threads before its passes bytecode to Smalltalk VM ? On Wed, Oct 18, 2017 at 1:39 PM Holger Freyther <[hidden email]> wrote:
|
|
|
In reply to this post by raffaello.giulietti
Hi Raffaello,
Thanks for your perspective. I guess also supporting the semantics of "Java Primitives" is more work than just executing "Java Bytecode". cheers -ben On Tue, Oct 17, 2017 at 10:53 PM, Raffaello Giulietti <[hidden email]> wrote:
|
Re: Sista alternate bytecodes + Java Bytecode ?
|
|
Since we’ve dropped into the whole thread thing - I’d like to mention that I’ve done a lot of iPhone and some Mac programming with Grand Central Dispatch (GCD - open source versions available as libdispatch) and have found that model easy to use and powerful. The key trick being to only manipulate the GUI and stateful entities on the main thread/queue while shuffling the heavy lifting off to lower priority queues via closures that steer clear of data that isn’t thread safe to access. It works very well and this idea of mini worker images sounds like it might be a good fit for that model. It would take some work to pull in libdispatch and restructure the vm but I feel like that might well be the way forward.
Sent from the road
|
Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)
|
|
> On 19-10-2017, at 7:31 AM, Todd Blanchard <[hidden email]> wrote: > > It works very well and this idea of mini worker images sounds like it might be a good fit for that model. > One thing I noticed with surprise was how astonishingly fast Dave’s OSProcess could spawn an entire image, even on a Pi. So it might be worth experimenting with spawning a child image to do some processing and maybe transmit results back via sockets or writing a project file or.. well, whatever. You might think that on a tiny thing like a Pi you would rapidly use up all memory and start disastrous paging to SD (ouch!) but a Pi has a Gb or ram and my 6.0alpha development image uses less than 9% of that. It’s a potentially simple way to make some use of multiple cores. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Hex dump: Where witches put used curses... |
|
|
On Thursday 19 October 2017 10:16 PM, tim Rowledge wrote: > > >> On 19-10-2017, at 7:31 AM, Todd Blanchard<[hidden email]> >> wrote: >> >> It works very well and this idea of mini worker images sounds like >> it might be a good fit for that model. >> > One thing I noticed with surprise was how astonishingly fast Dave’s > OSProcess could spawn an entire image, even on a Pi. I second this idea. LatexMorph is a good example of that. It uses OSProcess to pipe LaTeX text through latex engine and Ghostscript into an image and import that image back into Squeak. The pipeline was fast enough for live rendering of image as the text was typed on laptops way back in 2010. Regards .. Subbu |
Re: Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)
|
|
K, so if you were to marry that with Craig Latta's micro images and "imprinting" strategy, offloading work to other "queues" might be a really cool way to chuck over a minimal amount of code associated with a task - where each work queue is just another micro worker image. Just thinking out loud. > On Oct 19, 2017, at 10:09 AM, K K Subbu <[hidden email]> wrote: > > On Thursday 19 October 2017 10:16 PM, tim Rowledge wrote: >>> On 19-10-2017, at 7:31 AM, Todd Blanchard<[hidden email]> >>> wrote: >>> It works very well and this idea of mini worker images sounds like >>> it might be a good fit for that model. >> One thing I noticed with surprise was how astonishingly fast Dave’s >> OSProcess could spawn an entire image, even on a Pi. > > I second this idea. LatexMorph is a good example of that. It uses OSProcess to pipe LaTeX text through latex engine and Ghostscript into an image and import that image back into Squeak. The pipeline was fast enough for live rendering of image as the text was typed on laptops way back in 2010. > > Regards .. Subbu |
Re: Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)
|
|
> On 19-10-2017, at 10:20 AM, Todd Blanchard <[hidden email]> wrote: > > > K, so if you were to marry that with Craig Latta's micro images and "imprinting" strategy, offloading work to other "queues" might be a really cool way to chuck over a minimal amount of code associated with a task - where each work queue is just another micro worker image. Something like that; if you spawn an image to do a job it doesn’t need to have that code in it at spawn-time, just enough to know how to load it. Which means your ‘master control program’ doesn’t need to be large either. For example, Sista could spawn an image to do its optimising work and return the results to the ‘real’ image. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Do files get embarrassed when they get unzipped? |
|
|
In reply to this post by tblanchard
This was my thinking on eLinda, whose combination of evaluating in Linda TupleSpace Subspaces and also the eventual sending based upon Erights.org ELib. I added event registrations of Tuples with ETupleSpace default eventuallyDo: [:tuple | count := count + 1]. Every eval afterwards would count. That eventual reactor runs on the PriorityVat' s Process. If tuple evals in the subspace started using the Vat's process, then it would be single-threaded, multitasking. I feel it could be a monitor of task dispersal. Here is the latest eLinda, for Squeak and Pharo: http://www.squeaksource.com/Cryptography/elinda-HenryHouse.3.mcz I also think it would make a good parsing engine for scripting ai computations. I was looking at PeitiParser and thinking of changing their baskends to generate Scripts that could run through this linda-vat engine. As you have said, just thinking out loud. - HH
|
|
|
In reply to this post by timrowledge
On Fri, Oct 20, 2017 at 12:46 AM, tim Rowledge <[hidden email]> wrote:
Are talking about booting up a new image? What about native-forking a running image. Linux default copy-on-write (IIUC) should help limit memory usage, with full access to existing objects without needing to worry about multi-threaded access to objectspace. The trick to work out would the join mechanism. Perhaps returned objects would be limited to instance variables containing only immediate types and plain arrays of the same. Maybe the join mechanism would need to groom out non-compliant objects at the VM level. Or the join returns a STON representation. cheers -ben |
Re: Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)
|
|
On Fri, Oct 20, 2017 at 06:40:47AM +0800, Ben Coman wrote: > > On Fri, Oct 20, 2017 at 12:46 AM, tim Rowledge <[hidden email]> wrote: > > > > > > > > On 19-10-2017, at 7:31 AM, Todd Blanchard <[hidden email]> wrote: > > > > > > It works very well and this idea of mini worker images sounds like it > > might be a good fit for that model. > > > > > > > One thing I noticed with surprise was how astonishingly fast Dave???s > > OSProcess could spawn an entire image, even on a Pi. So it might be worth > > experimenting with spawning a child image to do some processing and maybe > > transmit results back via sockets or writing a project file or.. well, > > whatever. You might think that on a tiny thing like a Pi you would rapidly > > use up all memory and start disastrous paging to SD (ouch!) but a Pi has a > > Gb or ram and my 6.0alpha development image uses less than 9% of that. > > > > It???s a potentially simple way to make some use of multiple cores. > > > > Are talking about booting up a new image? > What about native-forking a running image. Linux default copy-on-write > (IIUC) should help limit memory usage, with full access to existing objects > without needing to worry about multi-threaded access to objectspace. > The trick to work out would the join mechanism. Perhaps returned objects > would be limited to instance variables containing only immediate types and > plain arrays of the same. Maybe the join mechanism would need to groom out > non-compliant objects at the VM level. Or the join returns a STON > representation. > > cheers -ben Yes that is exactly what Tim is talking about. The unix fork mechanism is suprisingly efficient when applied to a VM process running a Smalltalk image. Fuel works nicely for returning objects to the parent image, although I'm sure STON would work fine also. The Spur image format allows object memory to be segmented such that it will support fast and efficient forking of images even for extremely large images. I don't have any real world use cases to prove the point, but would be quite interested in finding some. The join mechanism can be very simple. The child image serializes the result of whatever work it has done, and writes it to a pipe connected to the parent image. The parent deserializes the next object from the pipe, and that is the result. As long as the pipe read does not block the VM (that is an OSProcess thing), you can put any number of child readers into separate Smalltalk processes. Background information is at http://wiki.squeak.org/squeak/6176. Dave |
|
|
On Fri, Oct 20, 2017 at 9:05 AM, David T. Lewis <[hidden email]> wrote:
I forgot Fuel. I guess its binary nature would make it faster. although I'm sure STON would work fine also. Would it be efficient enough to benefit forking off the display rendering half of the world loop? One thing I've seen in the past with Pharo with attempts to multi-thread UI components is that #changed: is unwittingly called somewhere deep in morph code away from the level the components are put together, and the rendering code is written assuming its the only thread, particularly with String producing race conditions like... bufferCopy := Array new: buffer length. 1 to: buffer length do: [ :i | bufferCopy at: i put: (buffer at: i) ]. which goes bang when another thread causes buffer gets longer between lines 1 & 2, or gets shorter between lines 2 & 3. And I sometimes wonder if the rendering occurred in a fork, then its copy of buffer would be fixed and the race conditions could not occur. The rendering-fork normally wouldn't even need to hand anything back in a join. It would just update the display and die quietly. How to keep debugging integrated would be interesting. Maybe Fuel helps there anyhow. cheers -ben I don't have any real world use cases to prove the point, but would be quite |
Re: Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)
|
|
On Fri, Oct 20, 2017 at 02:39:51PM +0800, Ben Coman wrote: > > On Fri, Oct 20, 2017 at 9:05 AM, David T. Lewis <[hidden email]> wrote: > > > > > The Spur image format allows object memory to be segmented such that it will > > support fast and efficient forking of images even for extremely large images. > > > > Would it be efficient enough to benefit forking off the display rendering > half of the world loop? No, it is much too slow for that. Reading the results back into the parent image takes time. But it might be good enough for things like background generation of Sista bytecodes to be used in future method calls (I think Tim had that in mind). I think that forkSqueak opens some interesting possibilities for multiprocessing, but I would expect it to be useful only for coarse-grained problems, possibly such as a numeric method that can be partitioned easily. I doubt that it will prove useful for small or short processing where the complexity would outweigh any possible benefits. It does make for a good parlor trick though. Try doing this for a completely useless but otherwise impressive example: (RemoteTask do: [2 + 2]) inspect Dave |
Re: Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)
|
|
> On 20-10-2017, at 6:59 AM, David T. Lewis <[hidden email]> wrote: > > > On Fri, Oct 20, 2017 at 02:39:51PM +0800, Ben Coman wrote: >> >> On Fri, Oct 20, 2017 at 9:05 AM, David T. Lewis <[hidden email]> wrote: >> >>> >>> The Spur image format allows object memory to be segmented such that it will >>> support fast and efficient forking of images even for extremely large images. >>> >> >> Would it be efficient enough to benefit forking off the display rendering >> half of the world loop? > > No, it is much too slow for that. Reading the results back into the parent image > takes time. But it might be good enough for things like background generation of > Sista bytecodes to be used in future method calls (I think Tim had that in mind). Maybe think of it the other way round; fork off the stuff that does the ‘work’ and have it send back answers to the root image for display. And for some cases - say where we are actually using the ancient and venerable Ffenestri stuff? - you could indeed spawn an image to display things in a separate window. And since we have image segments (I think they’re working again?) maybe use them to pass objects back and forth. And maybe even shared memory might do some good here. The VM object memory has segments so maybe build a way to return some handle-thing to a segment that has been built with an ImageSegment in it, thus saving the actual file-writing/reading time. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: PUS: PUrge System |
|
|
In reply to this post by David T. Lewis
Somehow I'm reminded of Igor Stasenko work on the Hydra VM. http://squeakvm.org/~sig/hydravm/devnotes.html
I did do a port for Mac OS back in 2008. Also btw for the OSX browser Squeak plugin work I did 10 years back we would fork the VM and setup a shared memory location which shared the Screen bitmap between the running background only Squeak VM process and the plugin code in the browser that would get screen draw requests via a pipe from the Squeak VM process and with the use of a shared semaphore latch to prevent the Squeak VM from burying the Safari plugin with draw requests. For this shared memory worked nicely as we didn't pipe MB of memory for a screen draw back to the Safari plugin. All the safari plugin knew was to copy the bit rectangle to the screen as ordered and signal the semaphore to let the VM continue running after it had done a screen sync cmd. Morphic in this millennium (not the last) generally issues a screen flush at the end of a 'drawing cycle' At that point the bits should go to the device that shows them to the user. Intermediate draws aren't particularly important but you can modify that visual aspect by forcing an update every 50 fps. Also maybe handy for last millennium code and obviously do the draw only if the bitmap has changed. On Thu, Oct 19, 2017 at 6:05 PM, David T. Lewis <[hidden email]> wrote:
=========================================================================== John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk =========================================================================== |
Hydra VM (was: Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?))
|
|
On Sat, Oct 21, 2017 at 01:37:57PM -0700, John McIntosh wrote: > > Somehow I'm reminded of Igor Stasenko work on the Hydra VM. > http://squeakvm.org/~sig/hydravm/devnotes.html > I did do a port for Mac OS back in 2008. > I believe that Igor (CCed) is still on this list, although I don't know if he keeps up with it. He does pop up on the Pharo lists from time :-) I would be interested to know if there were some general conclusions from the Hydra VM work. Did this prove to be a productive direction that should be pursued further? A lot has changed in the world of computing since Igor created Hydra VM. I wonder if it was a project just a few years ahead of its time. Dave |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |