Running the VM in another thread

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Running the VM in another thread

Pierce Ng-3
 
Hi Eliot and VM hackers,

I wrote the simplest pthread C program that invokes osvm_main() in a
separate thread.

On Linux Ubuntu 18.04, with minheadless.pharo.cog.spur's
libPharoVMCore.a and a Pharo 8 image, the program ran to completion
printing Pharo's command line handler help on stdout.

On MacOS Mojave, the same C program separately linked to
PharoVMCore.framework and SqueakVMCore.framework crash at the line
bzero(abstractOpcodes, allocBytes) in compileCogMethod() in
cogitX64SysV.c

Is this A or B?

A - Not meant to work. The VM must run on the main thread. On Linux the
program only ran by happenstance.

B - Meant to work. Crashing on MacOS is a bug.

Thanks. Cheers.

Reply | Threaded
Open this post in threaded view
|

Re: Running the VM in another thread

Tobias Pape
 

> On 02.02.2020, at 06:28, Pierce Ng <[hidden email]> wrote:
>
>
> Hi Eliot and VM hackers,
>
> I wrote the simplest pthread C program that invokes osvm_main() in a
> separate thread.
>
> On Linux Ubuntu 18.04, with minheadless.pharo.cog.spur's
> libPharoVMCore.a and a Pharo 8 image, the program ran to completion
> printing Pharo's command line handler help on stdout.
>
> On MacOS Mojave, the same C program separately linked to
> PharoVMCore.framework and SqueakVMCore.framework crash at the line
> bzero(abstractOpcodes, allocBytes) in compileCogMethod() in
> cogitX64SysV.c
>
> Is this A or B?
>
> A - Not meant to work. The VM must run on the main thread. On Linux the
> program only ran by happenstance.
>
> B - Meant to work. Crashing on MacOS is a bug.
>
>

I'd say Neither.
macOS has certain restrictions on what has to run on the main thread, namely gui stuff, but certainly other things, too…

But that's just my quick guess.

-t


Reply | Threaded
Open this post in threaded view
|

Re: Running the VM in another thread

Eliot Miranda-2
In reply to this post by Pierce Ng-3
 
Hi Pierce,

> On Feb 1, 2020, at 9:28 PM, Pierce Ng <[hidden email]> wrote:
>
> 
> Hi Eliot and VM hackers,
>
> I wrote the simplest pthread C program that invokes osvm_main() in a
> separate thread.
>
> On Linux Ubuntu 18.04, with minheadless.pharo.cog.spur's
> libPharoVMCore.a and a Pharo 8 image, the program ran to completion
> printing Pharo's command line handler help on stdout.
>
> On MacOS Mojave, the same C program separately linked to
> PharoVMCore.framework and SqueakVMCore.framework crash at the line
> bzero(abstractOpcodes, allocBytes) in compileCogMethod() in
> cogitX64SysV.c
>
> Is this A or B?
>
> A - Not meant to work. The VM must run on the main thread. On Linux the
> program only ran by happenstance.
>
> B - Meant to work. Crashing on MacOS is a bug.

C. There are restrictions on interacting with the GUI on MacOS. I’m not an expert here. John McIntosh can state what the restrictions are.

I’m curious why you want to run the vm in another thread.  I’m curious what your goals are here.  There are a number of potential viable paths to take wrt the vm and threading.

>
> Thanks. Cheers.
>
Reply | Threaded
Open this post in threaded view
|

Re: Running the VM in another thread

Pierce Ng-3
 
On Sun, Feb 02, 2020 at 11:25:07PM -0800, Eliot Miranda wrote:
> > Is this A or B?
> >
> > A - Not meant to work. The VM must run on the main thread. On Linux the
> > program only ran by happenstance.
> >
> > B - Meant to work. Crashing on MacOS is a bug.
>
> C. There are restrictions on interacting with the GUI on MacOS. I’m
> not an expert here. John McIntosh can state what the restrictions are.

I should've been clearer. The program is command line only, there is no
GUI. It is really just one step beyond hello world. I've attached it in
this mail.

On MacOS I created an Xcode project to link with the Pharo/Squeak VM
framework because it is quicker that way.

> I’m curious why you want to run the vm in another thread.  I’m curious
> what your goals are here.  There are a number of potential viable
> paths to take wrt the vm and threading.

In the headless branch of Pharo's fork, the Smalltalk image access
routines have been made pluggable. Pablo Tesone created an example that
reads the image from a Windows resource that is embedded into a compiled
host program.

I implemented the image-in-Windows-resource access routines in Free
Pascal as that functionality comes for free with Free Pascal and is
cross-platform.

With a couple of FFI, I then have a single-source Pascal program that
builds on Linux, MacOS and Windows, that runs a Pharo image embedded in
a Windows resource embedded in said Pascal program.

Then I rewrote the Pascal program to use pthreads. It works on Linux but
crashes on MacOS, so I wrote the C pthread program mentioned in this
mail.

My goal is to determine feasibility of building GUIs with Free Pascal /
Lazarus, while running Pharo on another thread handling app logic, with
its image embedded and read-only, and have the two communicate via IPC.

Actually, on desktops, running the VM on another thread is not that
important, because I can and should follow the Firefox/Chrome
multi-process architecture, with GUI and embedded Pharo as separate
compiled programs running in separate OS processes.

On iOS/Android, being able to run the VM in another thread might still
be desirable.

Some links:

- https://github.com/tesonep/pharo-vm-embedded-example (Pablo's example)
- https://github.com/PierceNg/pharo-vm-embedded-pascal (My Pascal experimentation)
- https://www.lazarus-ide.org/ (Lazarus)
- https://www.freepascal.org (Free Pascal)

Pierce

runosvm.c (676 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Running the VM in another thread

tesonep@gmail.com
 
Hi Pierce,
    sorry for the late reply, this weekend I have been quite occupied
with the presence in FOSDEM.

We have managed to successfully run the VM in a different thread than
the main thread.
I am working in a new version of the VM that allows us to separate the
processing of events from the VM thread to another thread.
This is available in https://github.com/pharo-project/opensmalltalk-vm/tree/idle
Also, this VM has a better implementation of the relinquish of the
processor, semaphore signaling and sockets.
As Cocoa, and other UI frameworks, require the event loop to run in
the main thread, we have moved the VM to run in a different thread.
Also, this VM does not work with normal headful images. To work with
headful images we have implemented in the image side the logic to use
GTK as a backend polling the events in the main loop.
If you require a headful VM I can help you to build the required image.

Basically, the problems we have solved (and you will find) are:

- The stack size of the new threads. As the stack pages used by the VM
are allocated in the stack, it is required that the stack is big
enough.
- The VM should not perform any UI call to Cocoa!
- In Linux, if the heap for the image is created from another thread,
it should give an initial hint to mmap to allocate the image in the
correct position. If the hint is not given, mmap will allocate the
first segment of the heap in a high address close to the stack,
limiting the ability to add new segments. As new segments should
always have a bigger address than the previous segments.

If you check the function vm_main in client.c you can see how we are
launching the thread. Pay attention that some initialization have to
be done before creating the new thread.

Any doubt, please ask us, I will answer faster I hope...

Cheers,

On Mon, Feb 3, 2020 at 12:12 PM Pierce Ng <[hidden email]> wrote:

>
>  On Sun, Feb 02, 2020 at 11:25:07PM -0800, Eliot Miranda wrote:
> > > Is this A or B?
> > >
> > > A - Not meant to work. The VM must run on the main thread. On Linux the
> > > program only ran by happenstance.
> > >
> > > B - Meant to work. Crashing on MacOS is a bug.
> >
> > C. There are restrictions on interacting with the GUI on MacOS. I’m
> > not an expert here. John McIntosh can state what the restrictions are.
>
> I should've been clearer. The program is command line only, there is no
> GUI. It is really just one step beyond hello world. I've attached it in
> this mail.
>
> On MacOS I created an Xcode project to link with the Pharo/Squeak VM
> framework because it is quicker that way.
>
> > I’m curious why you want to run the vm in another thread.  I’m curious
> > what your goals are here.  There are a number of potential viable
> > paths to take wrt the vm and threading.
>
> In the headless branch of Pharo's fork, the Smalltalk image access
> routines have been made pluggable. Pablo Tesone created an example that
> reads the image from a Windows resource that is embedded into a compiled
> host program.
>
> I implemented the image-in-Windows-resource access routines in Free
> Pascal as that functionality comes for free with Free Pascal and is
> cross-platform.
>
> With a couple of FFI, I then have a single-source Pascal program that
> builds on Linux, MacOS and Windows, that runs a Pharo image embedded in
> a Windows resource embedded in said Pascal program.
>
> Then I rewrote the Pascal program to use pthreads. It works on Linux but
> crashes on MacOS, so I wrote the C pthread program mentioned in this
> mail.
>
> My goal is to determine feasibility of building GUIs with Free Pascal /
> Lazarus, while running Pharo on another thread handling app logic, with
> its image embedded and read-only, and have the two communicate via IPC.
>
> Actually, on desktops, running the VM on another thread is not that
> important, because I can and should follow the Firefox/Chrome
> multi-process architecture, with GUI and embedded Pharo as separate
> compiled programs running in separate OS processes.
>
> On iOS/Android, being able to run the VM in another thread might still
> be desirable.
>
> Some links:
>
> - https://github.com/tesonep/pharo-vm-embedded-example (Pablo's example)
> - https://github.com/PierceNg/pharo-vm-embedded-pascal (My Pascal experimentation)
> - https://www.lazarus-ide.org/ (Lazarus)
> - https://www.freepascal.org (Free Pascal)
>
> Pierce



--
Pablo Tesone.
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Running the VM in another thread

Pierce Ng-3
 
On Mon, Feb 03, 2020 at 02:26:51PM +0100, [hidden email] wrote:
>     sorry for the late reply, this weekend I have been quite occupied
> with the presence in FOSDEM.

No problem at all. Thanks for replying.

> We have managed to successfully run the VM in a different thread than
> the main thread.
> I am working in a new version of the VM that allows us to separate the
> processing of events from the VM thread to another thread.
> This is available in https://github.com/pharo-project/opensmalltalk-vm/tree/idle
>
> [very cool stuff]
>
> Any doubt, please ask us, I will answer faster I hope...

This is great! I will try it out and will sure to ask questions if I run
into problems.

Thanks again!

Pierce