Re: [squeak-dev] Criteria For Plugin Compatibility
Locked
Re: [squeak-dev] Criteria For Plugin Compatibility
 
|
Hi Chris,
On Fri, Sep 11, 2015 at 9:01 AM, Chris Cunnington <[hidden email]> wrote:
The problems I see with this are a) a plugin compiled for Spur may not work with V3 or vice verse. The issue is the header size of an object. Some plugins, but not all, use this define, and the header sizes between Spur and V3 are incompatible. b) 32-bit plugins won't work with 64-bit VMs, and 64-bit plugins won't work with 32-bit VMs. Period. Now there are platform-level packaging technologies, such as fat binaries on Mac OS X, that allow one to construct plugins that contain more than one binary. But this is a lot of work to build and maintain. So personally I wouldn't put much effort into this level of drag-and-drop compatibility. Instead I'd put energy into good error messages so that when plugins don't work the user can find out why, and that when the wrong kind of plugin is used the VM doesn't just stumble along, maybe producing incorrect results, but instead puts the plugin out with a comprehensible complaint. Does this make sense? I know its a downer, but what you propose is, IMO, not affordable given our resources. And I must say, *this is a VM-DEV discussion, not a general purpose Squeak discussion*, yes?
_,,,^..^,,,_ best, Eliot |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
Eliot, I agree. Indeed, even with an interpreter VM it has long been the case that a plugin compiled as 64 bit will not work on a VM compiled for 64 bit and vice versa. With the number of variations of 32/64, Spur/V3, and so forth, I don't think that this is a problem worth worrying about. Better to just be a aware that plugins are not readily interchangeable in those cases. Dave > Hi Chris, > > On Fri, Sep 11, 2015 at 9:01 AM, Chris Cunnington <[hidden email]> > wrote: > >> I had the idea that a plugin from one VM could be dragged into another >> VM's directory and that it could be used just by starting up the image. >> I've done a little experimenting and it seems more of an idea of a >> reality. >> The functionality is there, but VM developers over the years have not >> seen >> this as a priority. Typically plugins and their VMs are compiled >> together. >> Or a person adds one by recompiling a VM compilation rig they already >> have. >> > > The problems I see with this are > > a) a plugin compiled for Spur may not work with V3 or vice verse. The > issue is the header size of an object. Some plugins, but not all, use > this > define, and the header sizes between Spur and V3 are incompatible. > > b) 32-bit plugins won't work with 64-bit VMs, and 64-bit plugins won't > work > with 32-bit VMs. Period. > > Now there are platform-level packaging technologies, such as fat binaries > on Mac OS X, that allow one to construct plugins that contain more than > one > binary. But this is a lot of work to build and maintain. > > So personally I wouldn't put much effort into this level of drag-and-drop > compatibility. Instead I'd put energy into good error messages so that > when plugins don't work the user can find out why, and that when the wrong > kind of plugin is used the VM doesn't just stumble along, maybe producing > incorrect results, but instead puts the plugin out with a comprehensible > complaint. > > Does this make sense? I know its a downer, but what you propose is, IMO, > not affordable given our resources. > > > And I must say, *this is a VM-DEV discussion, not a general purpose Squeak > discussion*, yes? > > As I had a rig for the Interpreter VM I decided to compile a plugin and > see >> if I could drag it around to other VM directories for use. Most times it >> didn't work. >> >> I compiled a TheUniversalAnswer plugin which returns 42. I moved it from >> a >> 4.14.1-3430 VM to a 4.14.1-3414 VM. I could get it to work if I did not >> use >> the squeak.sh start script. That is, I dragged so.vm-sound-null, >> so.TheUniversalAnswer and so.vm-display-X11 into the same directory as >> the >> VM binary. That worked. >> >> The only way to see the external plugins is with #listLoadedModules. >> But, >> irritatingly, modules are loaded as needed, so once you have proof of >> using >> the primitive from the plugin, yea, it will appear as a result of that >> Smalltalk listLoadedModules message. So, it's not that useful. >> >> I dragged so.TheUniversalAnswer around to other VMs such as 4.0.3-2202 >> and >> 4.13.10-3268 without success. And the unload selectors #forgetModule: >> and >> unloadModule:, which both use primitive 571 don't appear to work on my >> Ubuntu 15.04. >> >> So, I don't know if a community of plugins passed around is in the >> offing. >> What I have learned is that many of the answers are in >> sqUnixExternalPrims.c. The issues are really where does the VM look for >> primitives and what does it do when it finds one. It appears to me, >> after >> reading both the non-Cog and Cog versions of that file, that this an >> interesting area poorly documented that VM developers alter in haste to >> get >> on to something else. >> >> It's pretty important stuff, though. Do you start the VM with the binary >> on the path? Do you use squeak.sh? Where is the VM looking for stuff. >> Where >> will FFI look for stuff. And so on. >> >> Chris >> >> >> >> > > > -- > _,,,^..^,,,_ > best, Eliot > |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
In reply to this post by Eliot Miranda-2
On 11-09-2015, at 12:00 PM, Eliot Miranda <[hidden email]> wrote: > a) a plugin compiled for Spur may not work with V3 or vice verse. The issue is the header size of an object. Some plugins, but not all, use this define, and the header sizes between Spur and V3 are incompatible. > > b) 32-bit plugins won't work with 64-bit VMs, and 64-bit plugins won't work with 32-bit VMs. Period. At the very least we should make sure the version checking that was designed specifically to help handle this is actually getting used properly. That ought to prevent crashes and help with providing some decent error messages. > > Now there are platform-level packaging technologies, such as fat binaries on Mac OS X, that allow one to construct plugins that contain more than one binary. But this is a lot of work to build and maintain. I suspect we could avoid that by using the versioning stuff to indicate which plugin file(s) the user/system fetches? As I (re)discovered this week, the unix vm plugin/library loading is amazingly convoluted and looks in more directories than I think my Pi actually has. And it uses the name you give it prepended with ‘lib’ and appended with ‘.so’ and ‘dylib’, so searching for a plugin/lib involves an awful lot of checking and file testing. Looking in the platforms/unix/vm/sqUnixExternalPrims.c you can see what is being done. Note that ther eare two completely different approaches in that code and the Cog VM is using (so far as I can tell) the NOT USE_SIMPLIFIED_PLUGIN_LOGIC branch. The (somewhat old) copy of the plain interpreter VM equivalent I have around DOES use the simplified logic. We should probably clean that up... tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Fractured Idiom:- AMICUS PURIAE - Platonic friend |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
> > > On 11-09-2015, at 12:00 PM, Eliot Miranda <[hidden email]> wrote: >> a) a plugin compiled for Spur may not work with V3 or vice verse. The >> issue is the header size of an object. Some plugins, but not all, use >> this define, and the header sizes between Spur and V3 are incompatible. >> >> b) 32-bit plugins won't work with 64-bit VMs, and 64-bit plugins won't >> work with 32-bit VMs. Period. > > At the very least we should make sure the version checking that was > designed specifically to help handle this is actually getting used > properly. That's a good point, although it probably will not help with a many of these cases. For example, with an interpreter VM the various permutations of 32/64 bit host and 32/64 bit object memory are all compiled from a single identical code base, but the plugins cannot be mixed among them. On the other hand, it might be helpful for handling cases such as someone trying to run a Cog plugin with a Spur VM, or trying to mix 32 bit plugins with a 64 bit Spur VM. > That ought to prevent crashes and help with providing some decent error > messages. > In the case of mixing 32/64 bit compiled modules, the OS loader will not let this happen, so the only issue is that the plugin (quite rightly) does not load. But there might be other mixes of VM and plugin that could result in a crash. Dave >> >> Now there are platform-level packaging technologies, such as fat >> binaries on Mac OS X, that allow one to construct plugins that contain >> more than one binary. But this is a lot of work to build and maintain. > > I suspect we could avoid that by using the versioning stuff to indicate > which plugin file(s) the user/system fetches? > > > As I (re)discovered this week, the unix vm plugin/library loading is > amazingly convoluted and looks in more directories than I think my Pi > actually has. And it uses the name you give it prepended with lib and > appended with .so and dylib, so searching for a plugin/lib involves an > awful lot of checking and file testing. Looking in the > platforms/unix/vm/sqUnixExternalPrims.c you can see what is being done. > Note that ther eare two completely different approaches in that code and > the Cog VM is using (so far as I can tell) the NOT > USE_SIMPLIFIED_PLUGIN_LOGIC branch. The (somewhat old) copy of the plain > interpreter VM equivalent I have around DOES use the simplified logic. We > should probably clean that up... > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Fractured Idiom:- AMICUS PURIAE - Platonic friend > > |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
On 11-09-2015, at 1:15 PM, David T. Lewis <[hidden email]> wrote: >> At the very least we should make sure the version checking that was >> designed specifically to help handle this is actually getting used >> properly. > > That's a good point, although it probably will not help with a many of > these cases. For example, with an interpreter VM the various permutations > of 32/64 bit host and 32/64 bit object memory are all compiled from a > single identical code base, but the plugins cannot be mixed among them. Sounds like maybe modifying the version system to include some signifier for that might help? tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: RG: Read inter-record Gap |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
> > > On 11-09-2015, at 1:15 PM, David T. Lewis <[hidden email]> wrote: >>> At the very least we should make sure the version checking that was >>> designed specifically to help handle this is actually getting used >>> properly. >> >> That's a good point, although it probably will not help with a many of >> these cases. For example, with an interpreter VM the various >> permutations >> of 32/64 bit host and 32/64 bit object memory are all compiled from a >> single identical code base, but the plugins cannot be mixed among them. > > Sounds like maybe modifying the version system to include some signifier > for that might help? It would be a good thing on general principles to make sure the versioning system is sane (I have not paid attention to it). But it can't handle the case of mixing 32 and 64 bit compiled modules, because the modules are not going to load in the first place (the OS will see to that). I might be the only person around who regularly uses the V3 64 bit object memory, so you don't need to worry about protecting me from mixing the plugins, I already know not to do that ;-) It might be helpful for protecting against the case of mixing up 32-bit compiled Cog and Spur modules. Dave |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
In reply to this post by Eliot Miranda-2
On 2015-09-11 3:00 PM, Eliot Miranda
wrote:
I looked up some old Igor pdfs and I think I see what you mean. #primtitiveFailFor: and #initializePrimitiveErrorCodes. Yea, that looks like a better focus of energy. Agreed. Yes. Chris
|
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
In reply to this post by timrowledge
On 2015-09-11 3:18 PM, tim Rowledge wrote: > > > On 11-09-2015, at 12:00 PM, Eliot Miranda <[hidden email]> wrote: >> a) a plugin compiled for Spur may not work with V3 or vice verse. The issue is the header size of an object. Some plugins, but not all, use this define, and the header sizes between Spur and V3 are incompatible. >> >> b) 32-bit plugins won't work with 64-bit VMs, and 64-bit plugins won't work with 32-bit VMs. Period. > At the very least we should make sure the version checking that was designed specifically to help handle this is actually getting used properly. > That ought to prevent crashes and help with providing some decent error messages. > >> Now there are platform-level packaging technologies, such as fat binaries on Mac OS X, that allow one to construct plugins that contain more than one binary. But this is a lot of work to build and maintain. > I suspect we could avoid that by using the versioning stuff to indicate which plugin file(s) the user/system fetches? > > > As I (re)discovered this week, the unix vm plugin/library loading is amazingly convoluted and looks in more directories than I think my Pi actually has. And it uses the name you give it prepended with ‘lib’ and appended with ‘.so’ and ‘dylib’, so searching for a plugin/lib involves an awful lot of checking and file testing. Looking in the platforms/unix/vm/sqUnixExternalPrims.c you can see what is being done. Note that ther eare two completely different approaches in that code and the Cog VM is using (so far as I can tell) the NOT USE_SIMPLIFIED_PLUGIN_LOGIC branch. The (somewhat old) copy of the plain interpreter VM equivalent I have around DOES use the simplified logic. We should probably clean that up... implementations of ioLoadModule(). I think Cog ends up leaning on tryLoading() while the Interpreter leans on tryLoadModule(). It's appalling. Just count the number of uses of dlopen(). I'm no C programmer and following this stuff through step by step with gdb was needless torture. It's the reason my 32-bit Cog/Stack builds fail on Ubuntu 15.04. **shivers** I think the reason is that nobody wants to offend their forebearers. The three of you attend Squeak Board meetings. Come to some kind of agreement on what can be ripped out and then send Ian an nice note. What's most revolting is there are about five systems of tests looking for every possible combination (i.e. so.vm-display-X11, vm-display-X11.so, vm-display-X11.dylib, etc.) and they all fail. Only the last one works or something like that. **shivers like Bart Simpson** Chris > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Fractured Idiom:- AMICUS PURIAE - Platonic friend > > |
Re: [squeak-dev] Criteria For Plugin Compatibility
|
|
On Fri, Sep 11, 2015 at 07:29:11PM -0400, Chris Cunnington wrote: > > On 2015-09-11 3:18 PM, tim Rowledge wrote: > > > >As I (re)discovered this week, the unix vm plugin/library loading is > >amazingly convoluted and looks in more directories than I think my Pi > >actually has. And it uses the name you give it prepended with ?lib? and > >appended with ?.so? and ?dylib?, so searching for a plugin/lib involves an > >awful lot of checking and file testing. Looking in the > >platforms/unix/vm/sqUnixExternalPrims.c you can see what is being done. > >Note that ther eare two completely different approaches in that code and > >the Cog VM is using (so far as I can tell) the NOT > >USE_SIMPLIFIED_PLUGIN_LOGIC branch. The (somewhat old) copy of the plain > >interpreter VM equivalent I have around DOES use the simplified logic. We > >should probably clean that up... > > Oh my. Both the Cog and Interpreter sqUnixExternalPrims.c have two > implementations of ioLoadModule(). I think Cog ends up leaning on > tryLoading() while the Interpreter leans on tryLoadModule(). It's > appalling. Just count the number of uses of dlopen(). I'm no C > programmer and following this stuff through step by step with gdb was > needless torture. It's the reason my 32-bit Cog/Stack builds fail on > Ubuntu 15.04. **shivers** > > I think the reason is that nobody wants to offend their forebearers. The > three of you attend Squeak Board meetings. Come to some kind of > agreement on what can be ripped out and then send Ian an nice note. :-) I am quite confident that Ian would not be unhappy if someone were to send him a nice note. That does not happen often enough. Ian is the author of the Unix VM platform code, and a number of other things that the Squeak/Pharo community rely on every day. I think that Andreas Raab was also responsible for developing the module lookup strategy. As far as I am aware, the module search logic in SVN trunk would be the latest and presumably best version of Ian's implementation. The oscog branch was forked at an earlier time, and may have differences that reflect an earlier version of Ian's work, at least with respect to module lookup logic. Unless there is some compelling reason to look elsewhere, the latest version of Ian's work is in SVN trunk, and that should serve as the reference implementation with respect to the module lookup strategy for plugin modules and VM loadable modules (another of Ian's related contributions). Dave |
|
|
In reply to this post by David T. Lewis
On Sat, Sep 12, 2015 at 5:16 AM, David T. Lewis <[hidden email]> wrote: > >> >> >> On 11-09-2015, at 1:15 PM, David T. Lewis <[hidden email]> wrote: >>>> At the very least we should make sure the version checking that was >>>> designed specifically to help handle this is actually getting used >>>> properly. >>> >>> That's a good point, although it probably will not help with a many of >>> these cases. For example, with an interpreter VM the various >>> permutations >>> of 32/64 bit host and 32/64 bit object memory are all compiled from a >>> single identical code base, but the plugins cannot be mixed among them. >> >> Sounds like maybe modifying the version system to include some signifier >> for that might help? > > It would be a good thing on general principles to make sure the versioning > system is sane (I have not paid attention to it). But it can't handle the > case of mixing 32 and 64 bit compiled modules, because the modules are not > going to load in the first place (the OS will see to that). I might be the > only person around who regularly uses the V3 64 bit object memory, so you > don't need to worry about protecting me from mixing the plugins, I already > know not to do that ;-) > > It might be helpful for protecting against the case of mixing up 32-bit > compiled Cog and Spur modules. Just a random idea that may not be practical or worth the effort, you might have a system where a plugin can be dragged onto the Image desktop, and something in the binary header not to hard to analyse could report whether the plugin is compatible and suggest where to put it. cheers -ben |
re: Criteria For Plugin Compatibility
|
|
In reply to this post by Chris Cunnington-4
> Oh my. Both the Cog and Interpreter sqUnixExternalPrims.c have two > implementations of ioLoadModule(). I think Cog ends up leaning on > tryLoading() while the Interpreter leans on tryLoadModule(). It's > appalling. Just count the number of uses of dlopen(). I'm no C > programmer and following this stuff through step by step with gdb was > needless torture. It's the reason my 32-bit Cog/Stack builds fail on > Ubuntu 15.04. **shivers** > > What's most revolting is there are about five systems of tests looking > for every possible combination (i.e. so.vm-display-X11, > vm-display-X11.so, vm-display-X11.dylib, etc.) and they all fail. Only > the last one works or something like that. **shivers like Bart > Simpson** > > I think the reason is that nobody wants to offend their forebearers. > The [four] of you attend Squeak Board meetings. Come to some kind of > agreement on what can be ripped out and then send Ian an nice note. Well said. Yes, let's. -C -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS) |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |