[squeak-dev] Better VM <-> plugin API
Locked
12
12
[squeak-dev] Better VM <-> plugin API
 
|
I was pretty unhappy with some places in VM, where it loads a function
from plugin by calling ioLoadFunction... Also, even more painful to see when one plugin wants to call another plugin function. IMO things would be much better , if we formalize these things in VM. An idea is simple and easy to implement: We need only few functions implemented in VM: sqInt makeAtom(char * name); "registers new atom or returns id of already existing one" sqInt registerService (sqInt atom, void * service); "associate a value with service id" sqInt unregisterService(sqInt atom); "clear association with service id (make value=0)" And finally, void * getService(sqInt atom); Now, plugins first, should declare the atoms they would want to use or provide. This can be done once at plugin initialization stage, for instance: static sqInt bitBlitAtom = makeAtom('ioBitBlt'); Now, after acquiring atom, plugin can register a service fn under this atom: registerService(bitBlitAtom, (void*) bitBlt); and upon unloading unregisterService(bitBlitAtom); now, any other plugin can do: bitBltFn = getService(bitBlitAtom); if (bitBltFn) { bitBltFn( a, b,c blabla); } VM maintains a simple list of atom values and list of atom names and their numeric correspondence. A getService(bitBlitAtom) is very fast, because its simple access by index: getService(sqInt atom) { if (atom>=0 && atom<atomsCount) return atoms[atom]; return 0; } -- Best regards, Igor Stasenko AKA sig. |
[squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
Igor Stasenko wrote:
> I was pretty unhappy with some places in VM, where it loads a function > from plugin by calling ioLoadFunction... > Also, even more painful to see when one plugin wants to call another > plugin function. Why would it be advantageous to write: static sqInt bitBlitAtom = makeAtom('ioBitBlt'); registerService(bitBlitAtom, (void*) bitBlt); bitBltFn = getService(bitBlitAtom); instead of using bitBltFn = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin"); Is there any reason for making things even more lengthy than they are already? Cheers, - Andreas > IMO things would be much better , if we formalize these things in VM. > An idea is simple and easy to implement: > > We need only few functions implemented in VM: > > sqInt makeAtom(char * name); "registers new atom or returns id of > already existing one" > > sqInt registerService (sqInt atom, void * service); "associate a > value with service id" > sqInt unregisterService(sqInt atom); "clear association with service > id (make value=0)" > > And finally, > > void * getService(sqInt atom); > > > Now, plugins first, should declare the atoms they would want to use or > provide. This can be done once at plugin initialization stage, for > instance: > > static sqInt bitBlitAtom = makeAtom('ioBitBlt'); > > Now, after acquiring atom, plugin can register a service fn under this atom: > > registerService(bitBlitAtom, (void*) bitBlt); > and upon unloading > unregisterService(bitBlitAtom); > > now, any other plugin can do: > > bitBltFn = getService(bitBlitAtom); > > if (bitBltFn) { bitBltFn( a, b,c blabla); } > > VM maintains a simple list of atom values and list of atom names and > their numeric correspondence. > A getService(bitBlitAtom) is very fast, because its simple access by index: > > getService(sqInt atom) > { > if (atom>=0 && atom<atomsCount) > return atoms[atom]; > return 0; > } > > |
Re: [squeak-dev] Better VM <-> plugin API
|
|
In reply to this post by Igor Stasenko
On 21-Nov-2008, at 11:50 AM, Igor Stasenko wrote: > I was pretty unhappy with some places in VM, where it loads a function > from plugin by calling ioLoadFunction... > Also, even more painful to see when one plugin wants to call another > plugin function. I know very little about the VM, but I agree what you describe is ugly and inelegant, even at a very generic level. > Now, plugins first, should declare the atoms they would want to use or > provide. This can be done once at plugin initialization stage, for > instance: > > static sqInt bitBlitAtom = makeAtom('ioBitBlt'); > > Now, after acquiring atom, plugin can register a service fn under > this atom: > > registerService(bitBlitAtom, (void*) bitBlt); > and upon unloading > unregisterService(bitBlitAtom); > > now, any other plugin can do: > > bitBltFn = getService(bitBlitAtom); How is another plugin supposed to access this 'bitBlitAtom' handle that is effectively always in the first plugin's private storage and literally (as you've declared it above) also in the other plugin module's private symbol namespace? (i.e. if it's compiled as a "static" variable in the first plugin module then there's no way the second plugin can use the same symbol to refer to the same storage) Perhaps you meant to add a getAtom() function to your API? Then the second plugin would first do: static sqInt theAtom; theAtom = getAtom("ioBitBlt"); And then it could do the call above as: if (theAtom != 0) bitBltFn = getService(theAtom); > if (bitBltFn) { bitBltFn( a, b,c blabla); } -- Greg A. Woods; Planix, Inc. <[hidden email]> |
PGP.sig (193 bytes)
[squeak-dev] Better VM <-> plugin API
|
|
---------- Forwarded message ----------
From: Igor Stasenko <[hidden email]> Date: 2008/11/21 Subject: Re: [squeak-dev] Better VM <-> plugin API To: "Greg A. Woods, Planix, Inc." <[hidden email]> 2008/11/21 Greg A. Woods; Planix, Inc. <[hidden email]>: > > On 21-Nov-2008, at 11:50 AM, Igor Stasenko wrote: > >> I was pretty unhappy with some places in VM, where it loads a function >> from plugin by calling ioLoadFunction... >> Also, even more painful to see when one plugin wants to call another >> plugin function. > > I know very little about the VM, but I agree what you describe is ugly and > inelegant, even at a very generic level. > >> Now, plugins first, should declare the atoms they would want to use or >> provide. This can be done once at plugin initialization stage, for >> instance: >> >> static sqInt bitBlitAtom = makeAtom('ioBitBlt'); >> >> Now, after acquiring atom, plugin can register a service fn under this >> atom: >> >> registerService(bitBlitAtom, (void*) bitBlt); >> and upon unloading >> unregisterService(bitBlitAtom); >> >> now, any other plugin can do: >> >> bitBltFn = getService(bitBlitAtom); > > Everything looked good until I got here. > > How is another plugin supposed to access this 'bitBlitAtom' handle that is > effectively always in the first plugin's private storage and literally (as > you've declared it above) also in the other plugin module's private symbol > namespace? (i.e. if it's compiled as a "static" variable in the first > plugin module then there's no way the second plugin can use the same symbol > to refer to the same storage) Its not in a private storage. When you telling makeAtom(), you receiving an atom id for given name which is shared among all plugins. To be more clear, i think there is no need in register/unregister functions, instead there could be only single function: void * setAtomValue(int atom, void* value); which assigns atom new value and returns old one. Atom values are global and seen throughout VM. There is no privacy of any kind - one plugin can assign value to atom, and other can read value from it, or vice versa. So, there is 3 functions: makeAtom setAtomValue getAtomValue if we take an analogy from smalltalk, then its similar to system dictionary with 3 operations: atom := 'something' asSymbol. "intern new symbol, or get existing one" value := Smalltalk at: atom. "get value" Smalltalk at: atom put: value. "set value" > > Perhaps you meant to add a getAtom() function to your API? Then the second > plugin would first do: > > static sqInt theAtom; > > theAtom = getAtom("ioBitBlt"); > > And then it could do the call above as: > > if (theAtom != 0) > bitBltFn = getService(theAtom); > >> if (bitBltFn) { bitBltFn( a, b,c blabla); } > > -- > Greg A. Woods; Planix, Inc. > <[hidden email]> > > -- Best regards, Igor Stasenko AKA sig. -- Best regards, Igor Stasenko AKA sig. |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
In reply to this post by Andreas.Raab
2008/11/21 Andreas Raab <[hidden email]>:
> Igor Stasenko wrote: >> >> I was pretty unhappy with some places in VM, where it loads a function >> from plugin by calling ioLoadFunction... >> Also, even more painful to see when one plugin wants to call another >> plugin function. > > Why would it be advantageous to write: > > static sqInt bitBlitAtom = makeAtom('ioBitBlt'); > registerService(bitBlitAtom, (void*) bitBlt); > bitBltFn = getService(bitBlitAtom); > > instead of using > > bitBltFn = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin"); > > Is there any reason for making things even more lengthy than they are > already? > First, you are statically associating functionality with specific module ("ioBitBlt" in your example), while using atoms i don't care which module may set it , i care only about specific functionality. Second, with ioLoadFunctionFrom you can retrieve function, not arbitrary value(s). Third - you can change atom value dynamically , while ioLoadFunctionFrom doomed to return same value all the time - null if no module found or no such function in module , or function address on success. Fourth, there are many cases where i wouldn't want to expose functions of my module directly (by exporting them), only indirectly. > Cheers, > - Andreas > >> IMO things would be much better , if we formalize these things in VM. >> An idea is simple and easy to implement: >> >> We need only few functions implemented in VM: >> >> sqInt makeAtom(char * name); "registers new atom or returns id of >> already existing one" >> >> sqInt registerService (sqInt atom, void * service); "associate a >> value with service id" >> sqInt unregisterService(sqInt atom); "clear association with service >> id (make value=0)" >> >> And finally, >> >> void * getService(sqInt atom); >> >> >> Now, plugins first, should declare the atoms they would want to use or >> provide. This can be done once at plugin initialization stage, for >> instance: >> >> static sqInt bitBlitAtom = makeAtom('ioBitBlt'); >> >> Now, after acquiring atom, plugin can register a service fn under this >> atom: >> >> registerService(bitBlitAtom, (void*) bitBlt); >> and upon unloading >> unregisterService(bitBlitAtom); >> >> now, any other plugin can do: >> >> bitBltFn = getService(bitBlitAtom); >> >> if (bitBltFn) { bitBltFn( a, b,c blabla); } >> >> VM maintains a simple list of atom values and list of atom names and >> their numeric correspondence. >> A getService(bitBlitAtom) is very fast, because its simple access by >> index: >> >> getService(sqInt atom) >> { >> if (atom>=0 && atom<atomsCount) >> return atoms[atom]; >> return 0; >> } >> >> > > -- Best regards, Igor Stasenko AKA sig. |
[squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
Igor Stasenko wrote:
>> Why would it be advantageous to write: >> >> static sqInt bitBlitAtom = makeAtom('ioBitBlt'); >> registerService(bitBlitAtom, (void*) bitBlt); >> bitBltFn = getService(bitBlitAtom); >> >> instead of using >> >> bitBltFn = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin"); >> >> Is there any reason for making things even more lengthy than they are >> already? > > First, you are statically associating functionality with specific > module ("ioBitBlt" in your example), while using atoms i don't care > which module may set it , i care only about specific functionality. I see. This wasn't clear in your example. What use cases do you have in mind? I have only once needed something like this which was during an interim period where we supported two BitBlt variants and I wanted to be able to use either one from Balloon. Also, keep in mind that the unstructured namespace you are proposing can easily lead to conflicts - one of the reasons why ioLoadFunctionFrom is explicit about providing the plugin name is because of all of these "primitiveVersion" implementations out there. If you use the flat namespace throughout the system you'll have to have a need to disambiguate somehow. > Second, with ioLoadFunctionFrom you can retrieve function, not > arbitrary value(s). ioLoadFunctionFrom retrieves pointers and they can of course point to values. For example, the Windows VM exports references to the main window ("stWindow") and message hooks. Check out <platform>/vm/sq<Plat>Exports.c for examples of platform specific exports. > Third - you can change atom value dynamically , while > ioLoadFunctionFrom doomed to return same value all the time - null if > no module found or no such function in module , or function address on > success. Well, of course for changing values, you export functions ;-) Seems like the most obvious reason for exporting a function instead of a value. Though either way will work. > Fourth, there are many cases where i wouldn't want to expose functions > of my module directly (by exporting them), only indirectly. Again, I'd be interested in knowing more about your use cases. It is certainly doable to do this in the VM but we should be careful about adding duplicate functionality. Cheers, - Andreas |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
2008/11/22 Andreas Raab <[hidden email]>:
> Igor Stasenko wrote: >>> >>> Why would it be advantageous to write: >>> >>> static sqInt bitBlitAtom = makeAtom('ioBitBlt'); >>> registerService(bitBlitAtom, (void*) bitBlt); >>> bitBltFn = getService(bitBlitAtom); >>> >>> instead of using >>> >>> bitBltFn = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin"); >>> >>> Is there any reason for making things even more lengthy than they are >>> already? >> >> First, you are statically associating functionality with specific >> module ("ioBitBlt" in your example), while using atoms i don't care >> which module may set it , i care only about specific functionality. > > I see. This wasn't clear in your example. What use cases do you have in > mind? I have only once needed something like this which was during an > interim period where we supported two BitBlt variants and I wanted to be > able to use either one from Balloon. > > Also, keep in mind that the unstructured namespace you are proposing can > easily lead to conflicts - one of the reasons why ioLoadFunctionFrom is > explicit about providing the plugin name is because of all of these > "primitiveVersion" implementations out there. If you use the flat namespace > throughout the system you'll have to have a need to disambiguate somehow. > The main reason why to use the flat namespace is to allow sharing state identified by name between modules without exact knowledge where this state comes from. This, of course implies some contracts about meaning of symbols , e.g. "bitblt" is a function pointer which takes N arguments and so on (but hey, you implying the same contract anyways, when calling ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") - just with one exception, that you have to explicitly specify the source module where it comes from). This common namespace can act as meeting point , provided by VM, where plugins can share data without knowing exact source of it. As for ambiguous names: we have not much plugins to care about this problem so far :) In any case - this problem can be addressed in same way as currently in squeak e.g. define 'abcValue' instead of 'Value'. Or define 'mymodule.myname' .. any form which you may prefer. But in latter case, 'mymodule.myname' is nothing more better than ioLoadFunctionFrom("myname", "mymodule") and simply indicates that you abusing the global namespace by defining too specific values. And finally, if we care so much about ambiguousness , one can always register symbols like 'myModule.version' so others could check if version is ok. >> Second, with ioLoadFunctionFrom you can retrieve function, not >> arbitrary value(s). > > ioLoadFunctionFrom retrieves pointers and they can of course point to > values. For example, the Windows VM exports references to the main window > ("stWindow") and message hooks. Check out <platform>/vm/sq<Plat>Exports.c > for examples of platform specific exports. > >> Third - you can change atom value dynamically , while >> ioLoadFunctionFrom doomed to return same value all the time - null if >> no module found or no such function in module , or function address on >> success. > > Well, of course for changing values, you export functions ;-) Seems like the > most obvious reason for exporting a function instead of a value. Though > either way will work. > >> Fourth, there are many cases where i wouldn't want to expose functions >> of my module directly (by exporting them), only indirectly. > > Again, I'd be interested in knowing more about your use cases. It is > certainly doable to do this in the VM but we should be careful about adding > duplicate functionality. > One case, as you pointed out, is when you having multiple versions of same plugin and want to swap them on the fly. Smalltalk unloadModule: 'OldBlt'. Smalltalk loadModule: 'NewBlt'. Another case, is when you want to overlook some activity by installing a proxy which will log all calls to some functions. Smalltalk loadModule: 'SecurityLogger'. ... do some testing/debugging etc .. Smalltalk unloadModule: 'SecurityLogger' Third case: Smalltalk unloadModule: 'UnwantedStuff'. if there another hideous plugin (or VM) which using ioLoadFunctionFrom(), it would be very hard to prevent from loading it again :) Another one, as variant of first case (OldBlt/NewBlt), suppose you have module 'fooBar' which using functionality of some 'defaultFoo' module. But as well, you have a 'highEndFoo' module, loaded as well which having same interface as defaultFoo, but slightly different implementation: Smalltalk lowerCPUUsage: bool bool ifTrue: [ HighEndFoo deactivate ] ifFalse: [ HighEndFoo reactivate ] Note, that here, #deactivate wont lead to unloading a module. It stays in memory, ready to be used later, but simply rerouting things back to 'defaultFoo'. I'm not saying that there's much pressing need in such use cases. But these 3 functions will give us more flexibility , security and modularity. To be fair , i'd make all plugins to export only single function, like setInterpreter(), which then tells VM what things it wants to expose. Currently, all plugins exposing functions and primitives in static manner - and have no control at run time what to expose or not. But that's another story :) > Cheers, > - Andreas -- Best regards, Igor Stasenko AKA sig. |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
If we follow further this road, potentially, we could associate each
primitive with atom. E.g. instead of holding direct function pointers in external primitive table, we register an atom/symbol with moduleName.primitiveName and put pointer there. Now new module can selectively replace existing primitive with new one, even #primitiveAdd, with own , better faster implementation, if it makes any sense :) -- Best regards, Igor Stasenko AKA sig. |
[squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
In reply to this post by Igor Stasenko
Igor Stasenko wrote:
> The main reason why to use the flat namespace is to allow sharing > state identified by name between modules without exact knowledge where > this state comes from. This, of course implies some contracts about > meaning of symbols , e.g. "bitblt" is a function pointer which takes N > arguments and so on (but hey, you implying the same contract anyways, > when calling ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") - just > with one exception, that you have to explicitly specify the source > module where it comes from). Well, what's interesting about this is that it becomes much more of an issue if this mechanism is widely used to share function pointers. Right now, 99% of the entry points into plugins are primitives which use the Smalltalk stack and do their own argument checking. There is only a tiny number of functions that is used directly but once you start widening that interface it probably becomes more of an issue. Just something to consider. > One case, as you pointed out, is when you having multiple versions of > same plugin and want to swap them on the fly. Yeah, but that use case is the least convincing to me. We can do pretty much what you describe below (with the minor variation that it has to be an external plugin) and generally, the replacement of plugins has historically not played much of a role. For all practical intents and purposes it seems safer to subclass/switch the primitives in the image instead of replacing the plugin. > Another case, is when you want to overlook some activity by installing > a proxy which will log all calls to some functions. Yes, I could see doing that for debugging/auditing/tracing/profiling purposes. That's definitely a more interesting use case. > Third case: > > Smalltalk unloadModule: 'UnwantedStuff'. > > if there another hideous plugin (or VM) which using > ioLoadFunctionFrom(), it would be very hard to prevent from loading it > again :) I don't get this use case. Can you elaborate? > Another one, as variant of first case (OldBlt/NewBlt), suppose you > have module 'fooBar' which using functionality of some 'defaultFoo' > module. > But as well, you have a 'highEndFoo' module, loaded as well which > having same interface as defaultFoo, but slightly different > implementation: > > Smalltalk lowerCPUUsage: bool > bool ifTrue: [ HighEndFoo deactivate ] > ifFalse: [ HighEndFoo reactivate ] > > Note, that here, #deactivate wont lead to unloading a module. It stays > in memory, ready to be used later, but simply rerouting things back to > 'defaultFoo'. Again, I find this particular use case not very convincing. In a situation like the above I would prefer having two classes in the image referring to the plugins explicitly and have my code use whichever it finds appropriate - because in this case you can run both the low-end as well as the high-end side-by-side and code won't stomp on each other by one deciding it wants to use the low-end and the other one deciding to use the high-end version. > I'm not saying that there's much pressing need in such use cases. But > these 3 functions will give us more flexibility , security and > modularity. How so? More flexibility, yes probably. More security? Most definitely not - plugins run at a trusted level already so an evil plugin can do bad things and nothing of what you're describing will prevent that. Modularity I'd say is arguable; but given how badly Smalltalk fares with a flat global namespace I'd say that you'll be hard pressed to explain how using a single flat namespace is an improvement over a per-plugin namespace ;-) > To be fair , i'd make all plugins to export only single function, like > setInterpreter(), which then tells VM what things it wants to expose. > Currently, all plugins exposing functions and primitives in static > manner - and have no control at run time what to expose or not. > But that's another story :) Well, actually it's intrinsically related. Part of that story is that plugins use dlopen/dlsym for the lookup from an external plugin. It's because of that that the exports have to be static. So in order to make this dynamic you would have to have a single entry exporting it. Cheers, - Andreas |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
2008/11/22 Andreas Raab <[hidden email]>:
> Igor Stasenko wrote: >> >> The main reason why to use the flat namespace is to allow sharing >> state identified by name between modules without exact knowledge where >> this state comes from. This, of course implies some contracts about >> meaning of symbols , e.g. "bitblt" is a function pointer which takes N >> arguments and so on (but hey, you implying the same contract anyways, >> when calling ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") - just >> with one exception, that you have to explicitly specify the source >> module where it comes from). > > Well, what's interesting about this is that it becomes much more of an issue > if this mechanism is widely used to share function pointers. Right now, 99% > of the entry points into plugins are primitives which use the Smalltalk > stack and do their own argument checking. There is only a tiny number of > functions that is used directly but once you start widening that interface > it probably becomes more of an issue. Just something to consider. > Of course you right. What i proposed is simplest possible enhancement. A more secure design can be employed, if there such need. But there always a risk of abuse regardless how much obstacles you put - its C with direct memory access after all :) All you need to do to be in safe is to use correct function prototypes. But same applies to functions which you importing from dynamic library. >> One case, as you pointed out, is when you having multiple versions of >> same plugin and want to swap them on the fly. > > Yeah, but that use case is the least convincing to me. We can do pretty much > what you describe below (with the minor variation that it has to be an > external plugin) and generally, the replacement of plugins has historically > not played much of a role. For all practical intents and purposes it seems > safer to subclass/switch the primitives in the image instead of replacing > the plugin. > >> Another case, is when you want to overlook some activity by installing >> a proxy which will log all calls to some functions. > > Yes, I could see doing that for debugging/auditing/tracing/profiling > purposes. That's definitely a more interesting use case. > >> Third case: >> >> Smalltalk unloadModule: 'UnwantedStuff'. >> >> if there another hideous plugin (or VM) which using >> ioLoadFunctionFrom(), it would be very hard to prevent from loading it >> again :) > > I don't get this use case. Can you elaborate? > module loaded , it tries to load it first. >> Another one, as variant of first case (OldBlt/NewBlt), suppose you >> have module 'fooBar' which using functionality of some 'defaultFoo' >> module. >> But as well, you have a 'highEndFoo' module, loaded as well which >> having same interface as defaultFoo, but slightly different >> implementation: >> >> Smalltalk lowerCPUUsage: bool >> bool ifTrue: [ HighEndFoo deactivate ] >> ifFalse: [ HighEndFoo reactivate ] >> >> Note, that here, #deactivate wont lead to unloading a module. It stays >> in memory, ready to be used later, but simply rerouting things back to >> 'defaultFoo'. > > Again, I find this particular use case not very convincing. In a situation > like the above I would prefer having two classes in the image referring to > the plugins explicitly and have my code use whichever it finds appropriate - > because in this case you can run both the low-end as well as the high-end > side-by-side and code won't stomp on each other by one deciding it wants to > use the low-end and the other one deciding to use the high-end version. > >> I'm not saying that there's much pressing need in such use cases. But >> these 3 functions will give us more flexibility , security and >> modularity. > > How so? More flexibility, yes probably. More security? Most definitely not - > plugins run at a trusted level already so an evil plugin can do bad things > and nothing of what you're describing will prevent that. > Modularity I'd say > is arguable; but given how badly Smalltalk fares with a flat global > namespace I'd say that you'll be hard pressed to explain how using a single > flat namespace is an improvement over a per-plugin namespace ;-) > By more security, i meant a little feature, that since getting atom value is inexpensive operation, you can load value identified by atom and check for non-null value each time before calling function. Old scheme, needs to check for non-null as well, but in addition you need to be careful to clean out this pointer when you get notification that module, from where you taken this pointer is unloaded. I'm not sure what you mean by per-plugin namespace. And how much difference in namespaces between this: bitblt = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") and this: atom = makeAtom("BitBltPlugin.ioBitBlt"); bitBlt = getAtomValue(atom); The difference that first binds symbol at compile/link time, while second - at run time. As for modularity, let me illustrate what i meant. There are many primitives in VM, which look like: primitiveFoo: x with: y ^ self cCode: 'ioFoo(x,y)'. Obviously, when you generate this code using VMMaker, it wont compile unless you having ioFoo() function implemented somewhere. And you have a little choice where to put this implementation - in one of internal plugins or in platform-specific part of VM. Now, lets consider, if i refactor this code to use atoms (i skipping details like declarations etc) : primitiveFoo: x with: y ioFoo := getAtomValue: AtomIoFoo. ioFoo notNil ifTrue: [ ^ self cCode: 'ioFoo(x,y)' ]. ^ self primitiveFail. 1. the code will compile regardless i declared a function or not. 2. i'm free in choice where to put the implementation of this function, or even build VM initially w/o this function. 3. across different platforms, one can use same VMMaker to generate sources , and it will always compile & link. doesn't it look like more modular? Now, take a look at a sqWin32Stubs.c - how it would look if we would use shared namespace? It would look as zero length file. Because all we have to do in platform code is just initialize pointers: pointers = { "ioFoo" , ioFoo #ifdef NO_SOME_STUFF "ioBar" , ioBar #endif ... }; in this way, if i don't want some stuff , i can simply comment out a line in this table and let code rot in sources :) I'm currently trying to make a HostWindowsPlugin and want to put all windowing and i/o stuff in it from VM platform sources. It would be much a cut'n'paste experience to me, if all callouts in VM would use atoms - because then i don't need to touch headers & sources and many different places to make compiler content. Because there would be no need in exporting function in C-like manner. And lastly, remember InterpreterProxy thing? Each time you want to introduce new functionality , you have to extend the structure and increase version number. But what is it? Its just a list of pointers! Isn't it would be simpler for plugin to just lookup a function by its name - and use it if such function exists. Its pretty much same as dynamic linking, just s/dlsym/getAtomValue/ and don't let your code be constrained by compiler/linker anymore :) >> To be fair , i'd make all plugins to export only single function, like >> setInterpreter(), which then tells VM what things it wants to expose. >> Currently, all plugins exposing functions and primitives in static >> manner - and have no control at run time what to expose or not. >> But that's another story :) > > Well, actually it's intrinsically related. Part of that story is that > plugins use dlopen/dlsym for the lookup from an external plugin. It's > because of that that the exports have to be static. So in order to make this > dynamic you would have to have a single entry exporting it. > Actually the idea to use it came to my mind after reading this: http://tronche.com/gui/x/icccm/sec-1.html#s-1 . They calling it inter-client communications :) As well, as hated SystemDictionary does pretty same thing :) > Cheers, > - Andreas > -- Best regards, Igor Stasenko AKA sig. |
[squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
Igor Stasenko wrote:
> By more security, i meant a little feature, that since getting atom > value is inexpensive operation, you can > load value identified by atom and check for non-null value each time > before calling function. Oh, I think you mean it can be safer to use (with which I would agree) not necessarily more secure. > Old scheme, needs to check for non-null as well, but in addition you > need to be careful to clean out this pointer when you get notification > that module, from where you taken this pointer is unloaded. Yes. Although, you won't get around doing something about unloading either - we spent today learning about the intricacies of unloading OpenAL32.dll and it turned out to be absolutely crucial to be able to unload our own plugins. A flat shared namespace might have caused some serious issues here. > I'm not sure what you mean by per-plugin namespace. > And how much difference in namespaces between this: > bitblt = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") > and this: > atom = makeAtom("BitBltPlugin.ioBitBlt"); > bitBlt = getAtomValue(atom); > > The difference that first binds symbol at compile/link time, while > second - at run time. It would be quite possible to bind this at runtime, too. But what I mean by per-plugin namespace is that the *export* isn't done into a flat namespace but rather into a structured one. And yes, one could conceivably use a naming scheme that is equivalent in practice but unless that's automated (the current scheme is fully automated) it seems error-prone and one of these things were people are simply too lazy in practice to utilize it correctly (if that were different I would expect people to use class and selector prefixes consistently which they don't). > As for modularity, let me illustrate what i meant. > There are many primitives in VM, which look like: > > primitiveFoo: x with: y > > ^ self cCode: 'ioFoo(x,y)'. > > Obviously, when you generate this code using VMMaker, it wont compile > unless you having ioFoo() function implemented somewhere. > And you have a little choice where to put this implementation - in one > of internal plugins or in platform-specific part of VM. > > Now, lets consider, if i refactor this code to use atoms (i skipping > details like declarations etc) : > > primitiveFoo: x with: y > > ioFoo := getAtomValue: AtomIoFoo. > ioFoo notNil ifTrue: [ ^ self cCode: 'ioFoo(x,y)' ]. > ^ self primitiveFail. > > 1. the code will compile regardless i declared a function or not. > 2. i'm free in choice where to put the implementation of this > function, or even build VM initially w/o this function. > 3. across different platforms, one can use same VMMaker to generate > sources , and it will always compile & link. > > doesn't it look like more modular? No. It looks utterly pointless to me. You introduce a plugin that does nothing but looking up and call an atom; what good is that plugin? If you generalize that just a little you have the FFI where you might declare ioFoo() directly and call it. Which of course could be done via atom table too, but I still fail to see how that would be more modular. > Now, take a look at a sqWin32Stubs.c - how it would look if we would > use shared namespace? It would look as zero length file. > Because all we have to do in platform code is just initialize pointers: > > pointers = { > "ioFoo" , ioFoo > #ifdef NO_SOME_STUFF > "ioBar" , ioBar > #endif > ... > }; And if you stick this in sqWin32Stubs.c (or its equivalent) you end up with a non-empty stubs file. In other words, you are replacing one set of stubs with another one. Not much of an improvement. > I'm currently trying to make a HostWindowsPlugin and want to put all > windowing and i/o stuff in it from VM platform sources. Why do another one? Is there something that the current host window plugin doesn't address? > It would be much a cut'n'paste experience to me, if all callouts in VM > would use atoms - because then i don't need to touch headers & sources > and many different places to make compiler content. Because there > would be no need in exporting function in C-like manner. Ah. But that's a fallacy. That you don't need to "touch" these places doesn't mean you don't need to know about them. In fact, having to touch them, having the compiler complain about them is a great way to learn and know and understand all the areas you need to touch - if the VM would randomly crash on you because you have replaced one but not another function you'd be in for a hellish experience. Yes, the C compiler can be notorious but it can also be a pretty good teacher. > And lastly, remember InterpreterProxy thing? Each time you want to > introduce new functionality , you have to extend the structure and > increase version number. But what is it? Its just a list of pointers! No, it's not. It's an interface (a contract) between the VM and the plugin. > Isn't it would be simpler for plugin to just lookup a function by its > name - and use it if such function exists. Its pretty much same as > dynamic linking, just s/dlsym/getAtomValue/ and don't let your code be > constrained by compiler/linker anymore :) I *very much* doubt that it would be simpler for each plugin to look up the function every time they use it and test for its existence every time it is used. Consider this primitive: primitiveStringClone interpreterProxy methodArgument = 1 ifFalse:[^interpreterProxy primitiveFail]. arg := interpreterProxy stackValue: 0. (interpreterProxy isBytes: arg) ifFalse:[^interpreterProxy primitiveFail]. clone := interpreterProxy clone: arg. interpreterProxy pop: 2. interpreterProxy push: clone. Now let's rewrite this in pseudo-code to see what it would look like without an interface: primitiveStringClone ((interpreterProxy has: #methodArgumentCount) and:[interpreterProxy methodArgument = 1]) ifFalse:[^interpreterProxy primitiveFail]. (interpreterProxy has: #stackValue) ifFalse:[^interpreterProxy primitiveFail]. arg := interpreterProxy stackValue: 0. (interpreterProxy has: #isBytes) ifFalse:[^interpreterProxy primitiveFail]. (interpreterProxy isBytes: arg) ifFalse:[^interpreterProxy primitiveFail]. (interpreterProxy has: #clone) ifFalse:[^interpreterProxy primitiveFail]. clone := interpreterProxy clone: arg. (interpreterProxy has: #pop) ifFalse:[^interpreterProxy primitiveFail]. interpreterProxy pop: 2. (interpreterProxy has: #push) ifFalse:[^interpreterProxy primitiveFail]. interpreterProxy push: clone. Simpler? You've got to be kidding me ;-) Cheers, - Andreas |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
2008/11/22 Andreas Raab <[hidden email]>:
> Igor Stasenko wrote: >> >> By more security, i meant a little feature, that since getting atom >> value is inexpensive operation, you can >> load value identified by atom and check for non-null value each time >> before calling function. > > Oh, I think you mean it can be safer to use (with which I would agree) not > necessarily more secure. > >> Old scheme, needs to check for non-null as well, but in addition you >> need to be careful to clean out this pointer when you get notification >> that module, from where you taken this pointer is unloaded. > > Yes. Although, you won't get around doing something about unloading either - > we spent today learning about the intricacies of unloading OpenAL32.dll and > it turned out to be absolutely crucial to be able to unload our own plugins. > A flat shared namespace might have caused some serious issues here. > use it wisely (Class become: nil). Its not an argument not to use it. >> I'm not sure what you mean by per-plugin namespace. >> And how much difference in namespaces between this: >> bitblt = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") >> and this: >> atom = makeAtom("BitBltPlugin.ioBitBlt"); >> bitBlt = getAtomValue(atom); >> >> The difference that first binds symbol at compile/link time, while >> second - at run time. > > It would be quite possible to bind this at runtime, too. But what I mean by > per-plugin namespace is that the *export* isn't done into a flat namespace > but rather into a structured one. > > And yes, one could conceivably use a naming scheme that is equivalent in > practice but unless that's automated (the current scheme is fully automated) > it seems error-prone and one of these things were people are simply too lazy > in practice to utilize it correctly (if that were different I would expect > people to use class and selector prefixes consistently which they don't). > >> As for modularity, let me illustrate what i meant. >> There are many primitives in VM, which look like: >> >> primitiveFoo: x with: y >> >> ^ self cCode: 'ioFoo(x,y)'. >> >> Obviously, when you generate this code using VMMaker, it wont compile >> unless you having ioFoo() function implemented somewhere. >> And you have a little choice where to put this implementation - in one >> of internal plugins or in platform-specific part of VM. >> >> Now, lets consider, if i refactor this code to use atoms (i skipping >> details like declarations etc) : >> >> primitiveFoo: x with: y >> >> ioFoo := getAtomValue: AtomIoFoo. >> ioFoo notNil ifTrue: [ ^ self cCode: 'ioFoo(x,y)' ]. >> ^ self primitiveFail. >> >> 1. the code will compile regardless i declared a function or not. >> 2. i'm free in choice where to put the implementation of this >> function, or even build VM initially w/o this function. >> 3. across different platforms, one can use same VMMaker to generate >> sources , and it will always compile & link. >> >> doesn't it look like more modular? > > No. It looks utterly pointless to me. You introduce a plugin that does > nothing but looking up and call an atom; what good is that plugin? If you > generalize that just a little you have the FFI where you might declare > ioFoo() directly and call it. Which of course could be done via atom table > too, but I still fail to see how that would be more modular. > Not a plugin. I mentioned VM code in interp.c. I see little need in having such constructs in plugin. In VM core, however, there is always a bit of uncertainty - VM forced to provide primitive by default (because its a standart one), but on different platforms, depending on their capabilities or your intent, you may omit putting functionality of some primitives into VM. This is where such scheme is can be quite useful - instead of stubbing function in sources, or exploring what function has to return to make primitive fail , just remove it from build. >> Now, take a look at a sqWin32Stubs.c - how it would look if we would >> use shared namespace? It would look as zero length file. >> Because all we have to do in platform code is just initialize pointers: >> >> pointers = { >> "ioFoo" , ioFoo >> #ifdef NO_SOME_STUFF >> "ioBar" , ioBar >> #endif >> ... >> }; > > And if you stick this in sqWin32Stubs.c (or its equivalent) you end up with > a non-empty stubs file. In other words, you are replacing one set of stubs > with another one. Not much of an improvement. > Its not the same thing. Currently you have to define a function - with implementation or with empty body to make compiler content. In example i showed - you can do simpler - just omit binding a function to a symbol. Suppose we having a well structured organization of VM platform sources, then it might look like: #ifdef SUPPORT_FILES #include "file_functions.inc" /// or put all functions here w/o using #include #endif instead of something like: #ifdef SUPPORT_FILES #include "file_functions.inc" /// or put all functions here w/o using #include #else #include "file_functions_stubs.inc" #endif >> I'm currently trying to make a HostWindowsPlugin and want to put all >> windowing and i/o stuff in it from VM platform sources. > > Why do another one? Is there something that the current host window plugin > doesn't address? Well, there's many things. Don't want to OT here, just simple example: when image fires up, i want to show a splash screen, and then, when user clicks on it, or some time passed , hide it and show the "main" window. ALL is controlled from image, of course , e.g. one might want to show splash screen and when user clicks on it - just quit the squeak :) > >> It would be much a cut'n'paste experience to me, if all callouts in VM >> would use atoms - because then i don't need to touch headers & sources >> and many different places to make compiler content. Because there >> would be no need in exporting function in C-like manner. > > Ah. But that's a fallacy. That you don't need to "touch" these places > doesn't mean you don't need to know about them. In fact, having to touch > them, having the compiler complain about them is a great way to learn and > know and understand all the areas you need to touch - if the VM would > randomly crash on you because you have replaced one but not another function > you'd be in for a hellish experience. Yes, the C compiler can be notorious > but it can also be a pretty good teacher. > The same self-fallacy is to be confident, that if your code compiles ok it doesn't crash in some random place :) >> And lastly, remember InterpreterProxy thing? Each time you want to >> introduce new functionality , you have to extend the structure and >> increase version number. But what is it? Its just a list of pointers! > > No, it's not. It's an interface (a contract) between the VM and the plugin. > >> Isn't it would be simpler for plugin to just lookup a function by its >> name - and use it if such function exists. Its pretty much same as >> dynamic linking, just s/dlsym/getAtomValue/ and don't let your code be >> constrained by compiler/linker anymore :) > > I *very much* doubt that it would be simpler for each plugin to look up the > function every time they use it and test for its existence every time it is > used. Consider this primitive: > > primitiveStringClone > interpreterProxy methodArgument = 1 > ifFalse:[^interpreterProxy primitiveFail]. > arg := interpreterProxy stackValue: 0. > (interpreterProxy isBytes: arg) > ifFalse:[^interpreterProxy primitiveFail]. > clone := interpreterProxy clone: arg. > interpreterProxy pop: 2. > interpreterProxy push: clone. > > Now let's rewrite this in pseudo-code to see what it would look like without > an interface: > > primitiveStringClone > ((interpreterProxy has: #methodArgumentCount) > and:[interpreterProxy methodArgument = 1]) > ifFalse:[^interpreterProxy primitiveFail]. > (interpreterProxy has: #stackValue) > ifFalse:[^interpreterProxy primitiveFail]. > arg := interpreterProxy stackValue: 0. > (interpreterProxy has: #isBytes) > ifFalse:[^interpreterProxy primitiveFail]. > (interpreterProxy isBytes: arg) > ifFalse:[^interpreterProxy primitiveFail]. > (interpreterProxy has: #clone) > ifFalse:[^interpreterProxy primitiveFail]. > clone := interpreterProxy clone: arg. > (interpreterProxy has: #pop) > ifFalse:[^interpreterProxy primitiveFail]. > interpreterProxy pop: 2. > (interpreterProxy has: #push) > ifFalse:[^interpreterProxy primitiveFail]. > interpreterProxy push: clone. > > > Simpler? You've got to be kidding me ;-) > This counter-example missing a point :) Take a look at Hydra code, where InterpreterProxy has left with only 3 functions, and will stay to have 3 functions forever without need in extending protocol , because rest functions is obtained dynamicaly using name lookup. A plugin simply refuse to start without having all requested VM functions be non-null. But the trick is, that VM not forced anymore to support obsolete stuff of ever-growing InterpreterProxy protocol. > Cheers, > - Andreas > -- Best regards, Igor Stasenko AKA sig. |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
In reply to this post by Igor Stasenko
On Sat, Nov 22, 2008 at 09:14:57AM +0200, Igor Stasenko wrote:
> > I'm currently trying to make a HostWindowsPlugin and want to put all > windowing and i/o stuff in it from VM platform sources. > It would be much a cut'n'paste experience to me, if all callouts in VM > would use atoms - because then i don't need to touch headers & sources > and many different places to make compiler content. Because there > would be no need in exporting function in C-like manner. I don't see how changing the interface to plugins helps with this problem. Any way you look at it, you have to reorganize the current one-window implementation in the platform support code as well as in the plugin. Changing the way you call plugins or obtain function addresses does not make that problem any easier. It seems to me that it adds a layer of complexity without helping to solve the problem at hand. > Actually the idea to use it came to my mind after reading this: > http://tronche.com/gui/x/icccm/sec-1.html#s-1 . > They calling it inter-client communications :) There are a lot of good ideas you can borrow from X11, but complexity of implemention is not one of them. Dave p.s. To quote from Wikipedia: "The ICCCM is notorious for being ambiguous and difficult to correctly implement". |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
In reply to this post by Igor Stasenko
On Sat, Nov 22, 2008 at 9:34 AM, Igor Stasenko <[hidden email]> wrote:
I like this very much. Not totally, just very much :) What's good here is that the plugin can access functions directly in the VM and not go through the slow interpreterProxy. Because C supports syntax for calls through function pointers that looks just like normal functions calls wouldn't look very different to normal calls. The function pointers simply need to be decared.
But what I don't like is having interpreterProxy persist at all. Why not get rid of it except for initialization and pass it in through setInterpreter? What do you need interpreterProxy to persist for?
So Slang generates a standard prolog for all the functions used in a plugin at the beginning of the file (nice documentation; lists exactly the functions the plugin uses). setInterpreter (better called initializePlugin?) takes an argument that provides a function to lookup function pointers by name. Slang generates the code to initialize these fnction pointers. Uses of the function pointers look just like normal calls.
Then internal plugins can be linked directly against the VM. e.g. #if EXTERNAL_PLUGIN static void (*popthenPush(sqInt,sqInt); static void (*primitiveFailed)(void);
#endif ... void somePrimitive() { .... if (!ok) { primitiveFailed(); return; } popthenPush(result);
} ... #if EXTERNAL_PLUGIN static sqInt setInterpreter(InterpreterProxy *interpreterProxy) { if (interpreterProxy.oopSize() != 4) return InitErrorWrongOopSize;
if (!interpreterProxy.getFunction("popthenPush", &popthenPush) || !interpreterProxy.getFunction("primitiveFailed", & primitiveFailed)) return InitErrorMissingFunction;
return 0; } #endif /* EXTERNAL_PLUGIN */
|
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
2008/11/22 Eliot Miranda <[hidden email]>:
> > > On Sat, Nov 22, 2008 at 9:34 AM, Igor Stasenko <[hidden email]> wrote: >> >> 2008/11/22 Andreas Raab <[hidden email]>: >> > Igor Stasenko wrote: >> >> >> >> By more security, i meant a little feature, that since getting atom >> >> value is inexpensive operation, you can >> >> load value identified by atom and check for non-null value each time >> >> before calling function. >> > >> > Oh, I think you mean it can be safer to use (with which I would agree) >> > not >> > necessarily more secure. >> > >> >> Old scheme, needs to check for non-null as well, but in addition you >> >> need to be careful to clean out this pointer when you get notification >> >> that module, from where you taken this pointer is unloaded. >> > >> > Yes. Although, you won't get around doing something about unloading >> > either - >> > we spent today learning about the intricacies of unloading OpenAL32.dll >> > and >> > it turned out to be absolutely crucial to be able to unload our own >> > plugins. >> > A flat shared namespace might have caused some serious issues here. >> > >> Almost anything "might have cause some serious issues", if you don't >> use it wisely (Class become: nil). >> Its not an argument not to use it. >> >> >> I'm not sure what you mean by per-plugin namespace. >> >> And how much difference in namespaces between this: >> >> bitblt = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin") >> >> and this: >> >> atom = makeAtom("BitBltPlugin.ioBitBlt"); >> >> bitBlt = getAtomValue(atom); >> >> >> >> The difference that first binds symbol at compile/link time, while >> >> second - at run time. >> > >> > It would be quite possible to bind this at runtime, too. But what I mean >> > by >> > per-plugin namespace is that the *export* isn't done into a flat >> > namespace >> > but rather into a structured one. >> > >> > And yes, one could conceivably use a naming scheme that is equivalent in >> > practice but unless that's automated (the current scheme is fully >> > automated) >> > it seems error-prone and one of these things were people are simply too >> > lazy >> > in practice to utilize it correctly (if that were different I would >> > expect >> > people to use class and selector prefixes consistently which they >> > don't). >> > >> It could be automated or used manually. Its open for any intrusion :) >> >> >> As for modularity, let me illustrate what i meant. >> >> There are many primitives in VM, which look like: >> >> >> >> primitiveFoo: x with: y >> >> >> >> ^ self cCode: 'ioFoo(x,y)'. >> >> >> >> Obviously, when you generate this code using VMMaker, it wont compile >> >> unless you having ioFoo() function implemented somewhere. >> >> And you have a little choice where to put this implementation - in one >> >> of internal plugins or in platform-specific part of VM. >> >> >> >> Now, lets consider, if i refactor this code to use atoms (i skipping >> >> details like declarations etc) : >> >> >> >> primitiveFoo: x with: y >> >> >> >> ioFoo := getAtomValue: AtomIoFoo. >> >> ioFoo notNil ifTrue: [ ^ self cCode: 'ioFoo(x,y)' ]. >> >> ^ self primitiveFail. >> >> >> >> 1. the code will compile regardless i declared a function or not. >> >> 2. i'm free in choice where to put the implementation of this >> >> function, or even build VM initially w/o this function. >> >> 3. across different platforms, one can use same VMMaker to generate >> >> sources , and it will always compile & link. >> >> >> >> doesn't it look like more modular? >> > >> > No. It looks utterly pointless to me. You introduce a plugin that does >> > nothing but looking up and call an atom; what good is that plugin? If >> > you >> > generalize that just a little you have the FFI where you might declare >> > ioFoo() directly and call it. Which of course could be done via atom >> > table >> > too, but I still fail to see how that would be more modular. >> > >> >> Not a plugin. I mentioned VM code in interp.c. I see little need in >> having such constructs in plugin. >> In VM core, however, there is always a bit of uncertainty - VM forced >> to provide primitive by default (because its a standart one), but on >> different platforms, depending on their capabilities or your intent, >> you may omit putting functionality of some primitives into VM. >> This is where such scheme is can be quite useful - instead of stubbing >> function in sources, or exploring what function has to return to make >> primitive fail , just remove it from build. >> >> >> Now, take a look at a sqWin32Stubs.c - how it would look if we would >> >> use shared namespace? It would look as zero length file. >> >> Because all we have to do in platform code is just initialize pointers: >> >> >> >> pointers = { >> >> "ioFoo" , ioFoo >> >> #ifdef NO_SOME_STUFF >> >> "ioBar" , ioBar >> >> #endif >> >> ... >> >> }; >> > >> > And if you stick this in sqWin32Stubs.c (or its equivalent) you end up >> > with >> > a non-empty stubs file. In other words, you are replacing one set of >> > stubs >> > with another one. Not much of an improvement. >> > >> >> Its not the same thing. Currently you have to define a function - with >> implementation or with empty body to make compiler content. >> In example i showed - you can do simpler - just omit binding a >> function to a symbol. >> Suppose we having a well structured organization of VM platform >> sources, then it might look like: >> >> #ifdef SUPPORT_FILES >> #include "file_functions.inc" >> /// or put all functions here w/o using #include >> #endif >> >> instead of something like: >> >> #ifdef SUPPORT_FILES >> #include "file_functions.inc" >> /// or put all functions here w/o using #include >> #else >> #include "file_functions_stubs.inc" >> #endif >> >> >> I'm currently trying to make a HostWindowsPlugin and want to put all >> >> windowing and i/o stuff in it from VM platform sources. >> > >> > Why do another one? Is there something that the current host window >> > plugin >> > doesn't address? >> >> Well, there's many things. >> Don't want to OT here, just simple example: when image fires up, i >> want to show a splash screen, and then, when user clicks on it, or >> some time passed , hide it and show the "main" window. ALL is >> controlled from image, of course , e.g. one might want to show splash >> screen and when user clicks on it - just quit the squeak :) >> >> > >> >> It would be much a cut'n'paste experience to me, if all callouts in VM >> >> would use atoms - because then i don't need to touch headers & sources >> >> and many different places to make compiler content. Because there >> >> would be no need in exporting function in C-like manner. >> > >> > Ah. But that's a fallacy. That you don't need to "touch" these places >> > doesn't mean you don't need to know about them. In fact, having to touch >> > them, having the compiler complain about them is a great way to learn >> > and >> > know and understand all the areas you need to touch - if the VM would >> > randomly crash on you because you have replaced one but not another >> > function >> > you'd be in for a hellish experience. Yes, the C compiler can be >> > notorious >> > but it can also be a pretty good teacher. >> > >> >> The same self-fallacy is to be confident, that if your code compiles >> ok it doesn't crash in some random place :) >> >> >> And lastly, remember InterpreterProxy thing? Each time you want to >> >> introduce new functionality , you have to extend the structure and >> >> increase version number. But what is it? Its just a list of pointers! >> > >> > No, it's not. It's an interface (a contract) between the VM and the >> > plugin. >> > >> >> Isn't it would be simpler for plugin to just lookup a function by its >> >> name - and use it if such function exists. Its pretty much same as >> >> dynamic linking, just s/dlsym/getAtomValue/ and don't let your code be >> >> constrained by compiler/linker anymore :) >> > >> > I *very much* doubt that it would be simpler for each plugin to look up >> > the >> > function every time they use it and test for its existence every time it >> > is >> > used. Consider this primitive: >> > >> > primitiveStringClone >> > interpreterProxy methodArgument = 1 >> > ifFalse:[^interpreterProxy primitiveFail]. >> > arg := interpreterProxy stackValue: 0. >> > (interpreterProxy isBytes: arg) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > clone := interpreterProxy clone: arg. >> > interpreterProxy pop: 2. >> > interpreterProxy push: clone. >> > >> > Now let's rewrite this in pseudo-code to see what it would look like >> > without >> > an interface: >> > >> > primitiveStringClone >> > ((interpreterProxy has: #methodArgumentCount) >> > and:[interpreterProxy methodArgument = 1]) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > (interpreterProxy has: #stackValue) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > arg := interpreterProxy stackValue: 0. >> > (interpreterProxy has: #isBytes) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > (interpreterProxy isBytes: arg) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > (interpreterProxy has: #clone) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > clone := interpreterProxy clone: arg. >> > (interpreterProxy has: #pop) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > interpreterProxy pop: 2. >> > (interpreterProxy has: #push) >> > ifFalse:[^interpreterProxy primitiveFail]. >> > interpreterProxy push: clone. >> > >> > >> > Simpler? You've got to be kidding me ;-) >> > >> >> This counter-example missing a point :) >> Take a look at Hydra code, where InterpreterProxy has left with only 3 >> functions, and will stay to have 3 functions forever without need in >> extending protocol , because rest functions is obtained dynamicaly >> using name lookup. >> A plugin simply refuse to start without having all requested VM >> functions be non-null. But the trick is, that VM not forced anymore to >> support obsolete stuff of ever-growing InterpreterProxy protocol. > > I like this very much. Not totally, just very much :) > What's good here is that the plugin can access functions directly in the VM > and not go through the slow interpreterProxy. Because C supports syntax for > calls through function pointers that looks just like normal functions calls > wouldn't look very different to normal calls. The function pointers simply > need to be decared. right! > But what I don't like is having interpreterProxy persist at all. Why not get > rid of it except for initialization and pass it in through setInterpreter? > What do you need interpreterProxy to persist for? right! > So Slang generates a standard prolog for all the functions used in a plugin > at the beginning of the file (nice documentation; lists exactly the > functions the plugin uses). setInterpreter (better called > initializePlugin?) takes an argument that provides a function to lookup > function pointers by name. Slang generates the code to initialize these > fnction pointers. Uses of the function pointers look just like normal > calls. Right! > Then internal plugins can be linked directly against the VM. e.g. > Things works exactly in that way in Hydra. :) > #if EXTERNAL_PLUGIN > static void (*popthenPush(sqInt,sqInt); > static void (*primitiveFailed)(void); > #endif > ... > void somePrimitive() > { > .... > if (!ok) { > primitiveFailed(); > return; > } > popthenPush(result); > } > ... > #if EXTERNAL_PLUGIN > static sqInt > setInterpreter(InterpreterProxy *interpreterProxy) > { > if (interpreterProxy.oopSize() != 4) > return InitErrorWrongOopSize; > if (!interpreterProxy.getFunction("popthenPush", &popthenPush) > || !interpreterProxy.getFunction("primitiveFailed", & primitiveFailed)) > return InitErrorMissingFunction; > return 0; > } > #endif /* EXTERNAL_PLUGIN */ >> >> >> > Cheers, >> > - Andreas >> > >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> > > > > > -- Best regards, Igor Stasenko AKA sig. |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
OK, good :) So Igor, what are the three functions interpreterProxy still has?
TIA (P.S. this can be backward-compatible, right? We can still provide the old nterface for a while to allow external plugins to still load)
On Sat, Nov 22, 2008 at 11:06 AM, Igor Stasenko <[hidden email]> wrote: 2008/11/22 Eliot Miranda <[hidden email]>: |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
2008/11/22 Eliot Miranda <[hidden email]>:
> OK, good :) So Igor, what are the three functions interpreterProxy still > has? Here the part of sqVirtualMachine.h typedef struct ObjVirtualMachine { sqInt (*minorVersion)(void); sqInt (*majorVersion)(void); /* IMPORTANT!!! * The rest of functions can be obtained by plugin by calling a getVMFunctionPointerBySelector function. * The need in defining additional functions in this struct is gone forever */ void * (*getVMFunctionPointerBySelector)(char * selector); } ObjVirtualMachine; > TIA > > (P.S. this can be backward-compatible, right? We can still provide the old > nterface for a while to allow external plugins to still load) Yes, first it calls setInterpreter() with new struct. Old plugin checks version and refuses to work. Then VM calls setInterpreter() with old InterpreterProxy, to make plugin happy. > On Sat, Nov 22, 2008 at 11:06 AM, Igor Stasenko <[hidden email]> wrote: >> -- Best regards, Igor Stasenko AKA sig. |
[squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
In reply to this post by Igor Stasenko
Igor Stasenko wrote:
>> Yes. Although, you won't get around doing something about unloading either - >> we spent today learning about the intricacies of unloading OpenAL32.dll and >> it turned out to be absolutely crucial to be able to unload our own plugins. >> A flat shared namespace might have caused some serious issues here. >> > Almost anything "might have cause some serious issues", if you don't > use it wisely (Class become: nil). > Its not an argument not to use it. Certainly. And it wasn't an argument not to use it, merely a practical point about something that comes up with a flat shared namespace (cleaning up that namespace in the face of unloading modules and possibly overwriting entries that are now owned by other modules). >>> doesn't it look like more modular? >> No. It looks utterly pointless to me. You introduce a plugin that does >> nothing but looking up and call an atom; what good is that plugin? If you >> generalize that just a little you have the FFI where you might declare >> ioFoo() directly and call it. Which of course could be done via atom table >> too, but I still fail to see how that would be more modular. > > Not a plugin. I mentioned VM code in interp.c. I see little need in > having such constructs in plugin. > In VM core, however, there is always a bit of uncertainty - VM forced > to provide primitive by default (because its a standart one), but on > different platforms, depending on their capabilities or your intent, > you may omit putting functionality of some primitives into VM. Well, that's why we came up with plugins to begin with ;-) So that different platforms and different builds can have different sets of functionality. > Its not the same thing. Currently you have to define a function - with > implementation or with empty body to make compiler content. > In example i showed - you can do simpler - just omit binding a > function to a symbol. Sure. But it's still a completely contrived example. The stubs are only there because once upon a time, in an age long past there weren't VM plugins and you couldn't leave out entire plugins from the generated VM so the stubs were a necessity. I need to remove these stubs - they are completely pointless in this day and age. Thanks for reminding me. > Take a look at Hydra code, where InterpreterProxy has left with only 3 > functions, and will stay to have 3 functions forever without need in > extending protocol , because rest functions is obtained dynamicaly > using name lookup. > A plugin simply refuse to start without having all requested VM > functions be non-null. But the trick is, that VM not forced anymore to > support obsolete stuff of ever-growing InterpreterProxy protocol. I see - I had understood you wrongly. I thought you were explicitly trying to back out of the role of InterpreterProxy as the contract (which is not fulfilled if the VM doesn't export all the required functions in Hydra) and rather have the plugin code deal with the necessary lookups manually. The way Hydra does the proxy lookup is just fine as far as I am concerned. There is no difference from the client's point of view - for the plugin writer the proxy remains the contract and the ability to selectively allow and deny certain entry points can in fact be helpful since it allows a more flexible set of definitions of what is part of a particular proxy interface. Cheers, - Andreas |
Re: [squeak-dev] Re: [Vm-dev] Better VM <-> plugin API
|
|
2008/11/22 Andreas Raab <[hidden email]>:
> Igor Stasenko wrote: >>> >>> Yes. Although, you won't get around doing something about unloading >>> either - >>> we spent today learning about the intricacies of unloading OpenAL32.dll >>> and >>> it turned out to be absolutely crucial to be able to unload our own >>> plugins. >>> A flat shared namespace might have caused some serious issues here. >>> >> Almost anything "might have cause some serious issues", if you don't >> use it wisely (Class become: nil). >> Its not an argument not to use it. > > Certainly. And it wasn't an argument not to use it, merely a practical point > about something that comes up with a flat shared namespace (cleaning up that > namespace in the face of unloading modules and possibly overwriting entries > that are now owned by other modules). > I agree, it is not flawless, as well as current implementation. I just looking for a ways to relax interdependency between different modules or between VM and modules. A shared namespace fits good for some cases listed, and not so good for others. No wonder, there can't be single tool to make everything :) If you see better alternatives (to shared namespace), i'll listen with interest. >>>> doesn't it look like more modular? >>> >>> No. It looks utterly pointless to me. You introduce a plugin that does >>> nothing but looking up and call an atom; what good is that plugin? If you >>> generalize that just a little you have the FFI where you might declare >>> ioFoo() directly and call it. Which of course could be done via atom >>> table >>> too, but I still fail to see how that would be more modular. >> >> Not a plugin. I mentioned VM code in interp.c. I see little need in >> having such constructs in plugin. >> In VM core, however, there is always a bit of uncertainty - VM forced >> to provide primitive by default (because its a standart one), but on >> different platforms, depending on their capabilities or your intent, >> you may omit putting functionality of some primitives into VM. > > Well, that's why we came up with plugins to begin with ;-) So that different > platforms and different builds can have different sets of functionality. > Yeah, but still there are places which is quite rigid and causing headache when you have to deal with them. For instance, try building VM without including SecurityPlugin: //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Intel.c:1128: undefined reference to `ioInitSecurity' ./obj/vm/sqWin32Prefs.o: In function `SetAllowFileAccess': //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:96: undefined reference to `ioHasFileAccess' ./obj/vm/sqWin32Prefs.o: In function `SetAllowImageWrite': //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:101: undefined reference to `ioCanWriteImage' ./obj/vm/sqWin32Prefs.o: In function `SetAllowSocketAccess': //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:106: undefined reference to `ioHasSocketAccess' ./obj/vm/sqWin32Prefs.o: In function `HandlePrefsMenu': //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:383: undefined reference to `ioHasFileAccess' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:383: undefined reference to `_ioSetFileAccess' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:387: undefined reference to `ioCanWriteImage' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:387: undefined reference to `_ioSetImageWrite' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:391: undefined reference to `ioHasSocketAccess' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:391: undefined reference to `_ioSetSocketAccess' ------- Without SocketPlugin: ./obj/vm/sqWin32Prefs.o: In function `HandlePrefsMenu': //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Prefs.c:400: undefined reference to `win32DebugPrintSocketState' ------- Without FilePlugin (this list quite long, so i just leave a one error per single .c file): //F/projects/pharo-freetype/src32/winbuild/./src/vm/gnu-interp.c:3242: undefined reference to `sqImageFileSeek' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Args.c:26: undefined reference to `sqImageFileOpen' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32Intel.c:1116: undefined reference to `sqImageFileOpen' //F/projects/pharo-freetype/src32/winbuild/../platforms/win32/vm/sqWin32PluginSupport.c:252: undefined reference to `fileRecordSize' ./obj/vm/AsynchFilePlugin.lib(sqWin32AsyncFilePrims.o): In function `asyncFileValid': //F/projects/pharo-freetype/src32/winbuild/obj/AsynchFilePlugin/../../../platforms/Win32/plugins/AsynchFilePlugin/sqWin32AsyncFilePrims.c:167: undefined reference to `thisSession' ./obj/vm/DropPlugin.lib(sqWin32Drop.o): In function `dropRequestFileHandle': //F/projects/pharo-freetype/src32/winbuild/obj/DropPlugin/../../../platforms/Win32/plugins/DropPlugin/sqWin32Drop.c:745: undefined reference to `fileRecordSize' --------------- I cannot accept any reasoning, why VM would be considered defunc without any of the above plugins, and therefore they ought to be included in build. In the end it defeats the purpose of plugins: be able to plug-in or plug-out at will. >> Its not the same thing. Currently you have to define a function - with >> implementation or with empty body to make compiler content. >> In example i showed - you can do simpler - just omit binding a >> function to a symbol. > > Sure. But it's still a completely contrived example. The stubs are only > there because once upon a time, in an age long past there weren't VM plugins > and you couldn't leave out entire plugins from the generated VM so the stubs > were a necessity. I need to remove these stubs - they are completely > pointless in this day and age. Thanks for reminding me. > >> Take a look at Hydra code, where InterpreterProxy has left with only 3 >> functions, and will stay to have 3 functions forever without need in >> extending protocol , because rest functions is obtained dynamicaly >> using name lookup. >> A plugin simply refuse to start without having all requested VM >> functions be non-null. But the trick is, that VM not forced anymore to >> support obsolete stuff of ever-growing InterpreterProxy protocol. > > I see - I had understood you wrongly. I thought you were explicitly trying > to back out of the role of InterpreterProxy as the contract (which is not > fulfilled if the VM doesn't export all the required functions in Hydra) and > rather have the plugin code deal with the necessary lookups manually. > > The way Hydra does the proxy lookup is just fine as far as I am concerned. > There is no difference from the client's point of view - for the plugin > writer the proxy remains the contract and the ability to selectively allow > and deny certain entry points can in fact be helpful since it allows a more > flexible set of definitions of what is part of a particular proxy interface. > > Cheers, > - Andreas > > -- Best regards, Igor Stasenko AKA sig. |
Re: [squeak-dev] Better VM <-> plugin API
|
|
In reply to this post by Igor Stasenko
On 21-Nov-2008, at 4:30 PM, Igor Stasenko wrote: > > 2008/11/21 Greg A. Woods; Planix, Inc. <[hidden email]>: >> >> How is another plugin supposed to access this 'bitBlitAtom' handle >> that is >> effectively always in the first plugin's private storage and >> literally (as >> you've declared it above) also in the other plugin module's private >> symbol >> namespace? (i.e. if it's compiled as a "static" variable in the >> first >> plugin module then there's no way the second plugin can use the >> same symbol >> to refer to the same storage) > > Its not in a private storage. When you telling makeAtom(), you > receiving an atom id for given name which is shared among all plugins. what you meant? The code you wrote explicitly defines a _static_ (i.e. private) symbol for storage that only the one compiled source file will be able to see the referenced storage, and only within the scope of the definition. But I think Andreas Raab hit the nail on the head with his suggested ioLoadFunctionFrom() idea. -- Greg A. Woods; Planix, Inc. <[hidden email]> |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |