Status of Alien
Locked
 
|
What is the status of Alien.
According to http://bugs.squeak.org/view.php?id=7475 there is no one who confirmed it working on Linux. Any news here? Any ideas on FFI/Alien successor? Thanks T. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl |
|
|
Torsten,
I have been under the impression that it has never worked on Linux - it would be great to be wrong on that. Does anyone have Alien on Linux waiting for someone to try it? One thing slowing my search for FFI/Alien improvements has been that I was initially dependent on some vm modifications that I made. Some off-line correspondence with Dave Lewis and the benefit of some time might have yielded a solution that will let me return to stock vms. FFI has generally worked for me. I am doing a horrible job of handling double arrays, but I am at least able to use them. My "solution" makes assumptions about byte order and I have declare double* as void*. There was mention of extending FFI with callbacks. Given that capability, the only thing missing would then to be make calls on OS threads to avoid blocking the image (but blocking the calling Smalltalk Process for the duration of the call). Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Torsten Bergmann [[hidden email]] Sent: Tuesday, February 08, 2011 4:04 AM To: [hidden email] Subject: [Pharo-project] Status of Alien What is the status of Alien. According to http://bugs.squeak.org/view.php?id=7475 there is no one who confirmed it working on Linux. Any news here? Any ideas on FFI/Alien successor? Thanks T. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl |
|
|
On Tue, Feb 8, 2011 at 4:16 AM, Schwab,Wilhelm K <[hidden email]> wrote: Torsten, I would be very surprised if it didn't work. We at least use the data manipulation side of things routinely at Teleplace.
I need to integrate my new callback support and the ReentrantFFIPlugin into the standard interpreter as soon as possible. The ReentrantFFIPlugin provides compatibility with the old squeak FFI plugin but also allows aliens to be passed as arguments, at least to the extent of passing Alien Callbacks. This means its possible to discard the Alien callout support and use the FFI instead (but just to reiterate to keep the Alien data representation and callback facilities). The project for the next few weekends has to be harmonising the primitive set and functionality of Cog and the standard interpreter now that the StackToRegisterMappingCogit looks complete.
I have this in prototype and want to release soon. But there is too much else going on right now. Patience please. best Eliot
|
|
|
Eliot,
No lack of patience implied: just mentioning some things that I have picked up as off in the future and that would make us all stronger. Re the calls on OS threads, that is GREAT news. What strategy are you using to assign calls to the threads? I ask because there is a tradeoff between using a pool and taking what is ready to go, and coping with thread affinity. Example: OpenSSL is "easy" to call if one uses blocking calls and simply moves them out of the main thread (at least it worked for me on Dolphin). When I did this, I had to write a wrapper DLL to combined calls (connect, read, send, etc.) and checks for error status because otherwise, Dolphin would typically use different threads for the calls and the error test call was meaningless. IIRC, now they always use the same OS thread for any given Smalltalk Process - I think. I fixed a few more things that have been pinning me to my hacked vms, so I will hopefully be able to more easily experiment. I will look around for Alien on Linux and give it a try if I find it. I am also looking forward to running my "real" images through Cog. Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Eliot Miranda [[hidden email]] Sent: Tuesday, February 08, 2011 11:19 AM To: [hidden email] Cc: Squeak Virtual Machine Development Discussion Subject: Re: [Pharo-project] Status of Alien On Tue, Feb 8, 2011 at 4:16 AM, Schwab,Wilhelm K <[hidden email]<mailto:[hidden email]>> wrote: Torsten, I have been under the impression that it has never worked on Linux - it would be great to be wrong on that. Does anyone have Alien on Linux waiting for someone to try it? I would be very surprised if it didn't work. We at least use the data manipulation side of things routinely at Teleplace. One thing slowing my search for FFI/Alien improvements has been that I was initially dependent on some vm modifications that I made. Some off-line correspondence with Dave Lewis and the benefit of some time might have yielded a solution that will let me return to stock vms. I need to integrate my new callback support and the ReentrantFFIPlugin into the standard interpreter as soon as possible. The ReentrantFFIPlugin provides compatibility with the old squeak FFI plugin but also allows aliens to be passed as arguments, at least to the extent of passing Alien Callbacks. This means its possible to discard the Alien callout support and use the FFI instead (but just to reiterate to keep the Alien data representation and callback facilities). The project for the next few weekends has to be harmonising the primitive set and functionality of Cog and the standard interpreter now that the StackToRegisterMappingCogit looks complete. FFI has generally worked for me. I am doing a horrible job of handling double arrays, but I am at least able to use them. My "solution" makes assumptions about byte order and I have declare double* as void*. There was mention of extending FFI with callbacks. Given that capability, the only thing missing would then to be make calls on OS threads to avoid blocking the image (but blocking the calling Smalltalk Process for the duration of the call). I have this in prototype and want to release soon. But there is too much else going on right now. Patience please. best Eliot Bill ________________________________________ From: [hidden email]<mailto:[hidden email]> [[hidden email]<mailto:[hidden email]>] On Behalf Of Torsten Bergmann [[hidden email]<mailto:[hidden email]>] Sent: Tuesday, February 08, 2011 4:04 AM To: [hidden email]<mailto:[hidden email]> Subject: [Pharo-project] Status of Alien What is the status of Alien. According to http://bugs.squeak.org/view.php?id=7475 there is no one who confirmed it working on Linux. Any news here? Any ideas on FFI/Alien successor? Thanks T. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl |
|
|
On Tue, Feb 8, 2011 at 5:45 PM, Schwab,Wilhelm K <[hidden email]> wrote: Eliot, There is no pool. Instead, arbitrary threads can run the VM, but only one at a time. So the strategy to control calls is to decide which processes are bound to which threads. A new Smalltalk process can be executed by any thread. There is a well-known thread id for "the GUI thread" and it may be necessary to have more. So to arrange that a call is made on the GUI thread one arranges that the cal is made from a Smalltalk process that is bound to the GUI thread. To arrange that calls are made form some specific thread once again one binds one or more processes to that thread and makes calls from those processes. The VM's Smalltalk scheduler is modified to do a thread switch when a process switch is made to a process bound to other than the current thread (i.e. it is now a classic two-level scheduler). Hence scheduling semantics don't change; it is still a real-time scheduler that always runs the highest-priority runnable Smalltalk process. Its just that under the covers an implicit thread switch might also occur when a process switch occurs. Make sense?
best Eliot
|
|
|
Eliot,
I'm impressed :) To say that I truly understand would be dishonest, but I think I more or less get it. How is the binding of a Process to a thread done - I guess I am really asking what it looks like in code? One choice would be class side messages to ProcessScheduler or similar that return handles to or wrappers for the threads (#gui and #newThread come to mind), and those could be provided to a new process to bind it. Am I close? It sounds like a really nice solution to the problem. Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Eliot Miranda [[hidden email]] Sent: Tuesday, February 08, 2011 8:57 PM To: [hidden email] Subject: Re: [Pharo-project] Status of Alien On Tue, Feb 8, 2011 at 5:45 PM, Schwab,Wilhelm K <[hidden email]<mailto:[hidden email]>> wrote: Eliot, No lack of patience implied: just mentioning some things that I have picked up as off in the future and that would make us all stronger. Re the calls on OS threads, that is GREAT news. What strategy are you using to assign calls to the threads? I ask because there is a tradeoff between using a pool and taking what is ready to go, and coping with thread affinity. Example: OpenSSL is "easy" to call if one uses blocking calls and simply moves them out of the main thread (at least it worked for me on Dolphin). When I did this, I had to write a wrapper DLL to combined calls (connect, read, send, etc.) and checks for error status because otherwise, Dolphin would typically use different threads for the calls and the error test call was meaningless. IIRC, now they always use the same OS thread for any given Smalltalk Process - I think. There is no pool. Instead, arbitrary threads can run the VM, but only one at a time. So the strategy to control calls is to decide which processes are bound to which threads. A new Smalltalk process can be executed by any thread. There is a well-known thread id for "the GUI thread" and it may be necessary to have more. So to arrange that a call is made on the GUI thread one arranges that the cal is made from a Smalltalk process that is bound to the GUI thread. To arrange that calls are made form some specific thread once again one binds one or more processes to that thread and makes calls from those processes. The VM's Smalltalk scheduler is modified to do a thread switch when a process switch is made to a process bound to other than the current thread (i.e. it is now a classic two-level scheduler). Hence scheduling semantics don't change; it is still a real-time scheduler that always runs the highest-priority runnable Smalltalk process. Its just that under the covers an implicit thread switch might also occur when a process switch occurs. Make sense? best Eliot I fixed a few more things that have been pinning me to my hacked vms, so I will hopefully be able to more easily experiment. I will look around for Alien on Linux and give it a try if I find it. I am also looking forward to running my "real" images through Cog. Bill ________________________________________ From: [hidden email]<mailto:[hidden email]> [[hidden email]<mailto:[hidden email]>] On Behalf Of Eliot Miranda [[hidden email]<mailto:[hidden email]>] Sent: Tuesday, February 08, 2011 11:19 AM To: [hidden email]<mailto:[hidden email]> Cc: Squeak Virtual Machine Development Discussion Subject: Re: [Pharo-project] Status of Alien On Tue, Feb 8, 2011 at 4:16 AM, Schwab,Wilhelm K <[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>> wrote: Torsten, I have been under the impression that it has never worked on Linux - it would be great to be wrong on that. Does anyone have Alien on Linux waiting for someone to try it? I would be very surprised if it didn't work. We at least use the data manipulation side of things routinely at Teleplace. One thing slowing my search for FFI/Alien improvements has been that I was initially dependent on some vm modifications that I made. Some off-line correspondence with Dave Lewis and the benefit of some time might have yielded a solution that will let me return to stock vms. I need to integrate my new callback support and the ReentrantFFIPlugin into the standard interpreter as soon as possible. The ReentrantFFIPlugin provides compatibility with the old squeak FFI plugin but also allows aliens to be passed as arguments, at least to the extent of passing Alien Callbacks. This means its possible to discard the Alien callout support and use the FFI instead (but just to reiterate to keep the Alien data representation and callback facilities). The project for the next few weekends has to be harmonising the primitive set and functionality of Cog and the standard interpreter now that the StackToRegisterMappingCogit looks complete. FFI has generally worked for me. I am doing a horrible job of handling double arrays, but I am at least able to use them. My "solution" makes assumptions about byte order and I have declare double* as void*. There was mention of extending FFI with callbacks. Given that capability, the only thing missing would then to be make calls on OS threads to avoid blocking the image (but blocking the calling Smalltalk Process for the duration of the call). I have this in prototype and want to release soon. But there is too much else going on right now. Patience please. best Eliot Bill ________________________________________ From: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>> [[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>] On Behalf Of Torsten Bergmann [[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>] Sent: Tuesday, February 08, 2011 4:04 AM To: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>> Subject: [Pharo-project] Status of Alien What is the status of Alien. According to http://bugs.squeak.org/view.php?id=7475 there is no one who confirmed it working on Linux. Any news here? Any ideas on FFI/Alien successor? Thanks T. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl |
|
|
On Tue, Feb 8, 2011 at 7:01 PM, Schwab,Wilhelm K <[hidden email]> wrote: Eliot, There's an instance variable in Process, threadId, that is nil (unbound) or holds the threadId of the thread it can run on. There is one primitive to bind it to the current thread and one to unbind. It is unsafe to manipulate the threadId inst var directly.
The overall scheme or the thread-to-process mapping? best, Eliot
|
|
|
Eliot,
I was referring to the overall design; so far it seems like a nice way to put the image in charge of thread management. The details of binding would not matter too much; I'm simply trying to get a handle on how it will look. Having a primitive to manipulate the threadid makes sense. Are there constraints on when it can be invoked? Can the Process be running at the time? Thinking in terms of how the Smalltalk code will look, I am wondering whether there should be a #forkAt:threadID: to allow a new process to "wake up" with the correct threadid?? How does one start a new thread? Is the result represented only by ID, or is it wrapped by something that is finalized? What about image saves and startup/shutdown while a bound process is running? In fairness, I am not certain I ever asked those questions about Dolphin's implementation. Putting calls on a background thread was mostly a way to protect my software from being frozen by long operations that either finished during the session or did not - it happened mostly in deployed executables so, when the background calls were needed, there was no concept of saving an image. A Process being unbound means it will run on any thread? Have you thought about binding everything to the GUI thread by default? Maybe I did Windows programming for too long, but undocumented thread affinities might not matter in that case, and if there's little harm, it might be a safer default. You and Andreas are probably the ones to hash that out though. Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Eliot Miranda [[hidden email]] Sent: Tuesday, February 08, 2011 11:28 PM To: [hidden email] Subject: Re: [Pharo-project] Status of Alien On Tue, Feb 8, 2011 at 7:01 PM, Schwab,Wilhelm K <[hidden email]<mailto:[hidden email]>> wrote: Eliot, I'm impressed :) To say that I truly understand would be dishonest, but I think I more or less get it. How is the binding of a Process to a thread done - I guess I am really asking what it looks like in code? One choice would be class side messages to ProcessScheduler or similar that return handles to or wrappers for the threads (#gui and #newThread come to mind), and those could be provided to a new process to bind it. Am I close? There's an instance variable in Process, threadId, that is nil (unbound) or holds the threadId of the thread it can run on. There is one primitive to bind it to the current thread and one to unbind. It is unsafe to manipulate the threadId inst var directly. It sounds like a really nice solution to the problem. The overall scheme or the thread-to-process mapping? best, Eliot Bill ________________________________________ From: [hidden email]<mailto:[hidden email]> [[hidden email]<mailto:[hidden email]>] On Behalf Of Eliot Miranda [[hidden email]<mailto:[hidden email]>] Sent: Tuesday, February 08, 2011 8:57 PM To: [hidden email]<mailto:[hidden email]> Subject: Re: [Pharo-project] Status of Alien On Tue, Feb 8, 2011 at 5:45 PM, Schwab,Wilhelm K <[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>> wrote: Eliot, No lack of patience implied: just mentioning some things that I have picked up as off in the future and that would make us all stronger. Re the calls on OS threads, that is GREAT news. What strategy are you using to assign calls to the threads? I ask because there is a tradeoff between using a pool and taking what is ready to go, and coping with thread affinity. Example: OpenSSL is "easy" to call if one uses blocking calls and simply moves them out of the main thread (at least it worked for me on Dolphin). When I did this, I had to write a wrapper DLL to combined calls (connect, read, send, etc.) and checks for error status because otherwise, Dolphin would typically use different threads for the calls and the error test call was meaningless. IIRC, now they always use the same OS thread for any given Smalltalk Process - I think. There is no pool. Instead, arbitrary threads can run the VM, but only one at a time. So the strategy to control calls is to decide which processes are bound to which threads. A new Smalltalk process can be executed by any thread. There is a well-known thread id for "the GUI thread" and it may be necessary to have more. So to arrange that a call is made on the GUI thread one arranges that the cal is made from a Smalltalk process that is bound to the GUI thread. To arrange that calls are made form some specific thread once again one binds one or more processes to that thread and makes calls from those processes. The VM's Smalltalk scheduler is modified to do a thread switch when a process switch is made to a process bound to other than the current thread (i.e. it is now a classic two-level scheduler). Hence scheduling semantics don't change; it is still a real-time scheduler that always runs the highest-priority runnable Smalltalk process. Its just that under the covers an implicit thread switch might also occur when a process switch occurs. Make sense? best Eliot I fixed a few more things that have been pinning me to my hacked vms, so I will hopefully be able to more easily experiment. I will look around for Alien on Linux and give it a try if I find it. I am also looking forward to running my "real" images through Cog. Bill ________________________________________ From: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>> [[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>] On Behalf Of Eliot Miranda [[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>] Sent: Tuesday, February 08, 2011 11:19 AM To: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>> Cc: Squeak Virtual Machine Development Discussion Subject: Re: [Pharo-project] Status of Alien On Tue, Feb 8, 2011 at 4:16 AM, Schwab,Wilhelm K <[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>><mailto:[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>>> wrote: Torsten, I have been under the impression that it has never worked on Linux - it would be great to be wrong on that. Does anyone have Alien on Linux waiting for someone to try it? I would be very surprised if it didn't work. We at least use the data manipulation side of things routinely at Teleplace. One thing slowing my search for FFI/Alien improvements has been that I was initially dependent on some vm modifications that I made. Some off-line correspondence with Dave Lewis and the benefit of some time might have yielded a solution that will let me return to stock vms. I need to integrate my new callback support and the ReentrantFFIPlugin into the standard interpreter as soon as possible. The ReentrantFFIPlugin provides compatibility with the old squeak FFI plugin but also allows aliens to be passed as arguments, at least to the extent of passing Alien Callbacks. This means its possible to discard the Alien callout support and use the FFI instead (but just to reiterate to keep the Alien data representation and callback facilities). The project for the next few weekends has to be harmonising the primitive set and functionality of Cog and the standard interpreter now that the StackToRegisterMappingCogit looks complete. FFI has generally worked for me. I am doing a horrible job of handling double arrays, but I am at least able to use them. My "solution" makes assumptions about byte order and I have declare double* as void*. There was mention of extending FFI with callbacks. Given that capability, the only thing missing would then to be make calls on OS threads to avoid blocking the image (but blocking the calling Smalltalk Process for the duration of the call). I have this in prototype and want to release soon. But there is too much else going on right now. Patience please. best Eliot Bill ________________________________________ From: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>><mailto:[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>> [[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>><mailto:[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>>] On Behalf Of Torsten Bergmann [[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>><mailto:[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>>] Sent: Tuesday, February 08, 2011 4:04 AM To: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>><mailto:[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>> Subject: [Pharo-project] Status of Alien What is the status of Alien. According to http://bugs.squeak.org/view.php?id=7475 there is no one who confirmed it working on Linux. Any news here? Any ideas on FFI/Alien successor? Thanks T. -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |