Making a pluggable primitive - basic questions

Hi

I'm trying to build a plug-in and getting pretty lost at the moment. I
have found a few useful references but they are not complete enough for
me to follow. I have a simple piece of 'C' I want to integrate first
off, at least, the interface is simple. A couple of the methods don't
even have parameters so thought I would try those first. As I understand
it the process is this.

1. Create the smalltalk class to call the primitive. Something like
this:
        run
  <primitive: 'primitiveRun' module:'PhasingDSPPlugin'>
  ^ false

2. Create the primitive code calling code in Slang dialect. Something
like this (I'm sure this is very incomplete/plain wrong):
        primitiveRun
                SmartSyntaxInterpreterPlugin doPrimitive: #StartStream
withArguments: #().

        Should I be using interpreterProxy here. Where does the
interpreterProxy instance get created.

3. Run Slang to convert (2) to 'C'. I know I need to use VMMaker and
should only need to convert this one method. I have looked at
VMMakerTool but don't know if this is the right thing to use. If it is
it's not clear to me how one gets a new plugin into the 'plugins not
built' list and the ones listed are not draggable anyway. Does my code
need to go into VMMaker-Plugins or is this just a convention? Also tried
cCodeGenerator cCodeForMethod but couldn't get it to work.

4 Assuming someone can put me right on (3) I then have to compile and
link (on XP by the way) for which I can use VisualStudio or Borland.
What do I compile/link against? I don't see any header files or libs in
the squeak distribution.

5. Having got my compiled/linked code, not sure what form that should
take, is it object format, linked to a .lib or .dll. How do I register
it with Squeak so it knows how to play.

Sorry for so many questions but any help in moving forward, even a
little, would be greatly appreciated.

Thanks

Bob
*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

On 15-Jan-06, at 10:20 AM, Cowdery, Bob [UK] wrote:

> Hi
>
> I'm trying to build a plug-in and getting pretty lost at the moment.
Sounds like you haven't been pointed to the swiki pages yet: - Start  
at http://minnow.cc.gatech.edu/squeak/2105 and particularly follow  
links to NamedPrimitives and similar.
>
> 1. Create the smalltalk class to call the primitive. Something like
> this:
> run
>   <primitive: 'primitiveRun' module:'PhasingDSPPlugin'>
>   ^ false
>
That's a reasonable start. You might want to return 'self  
primitiveFailed' instead of false or you might have some way to check  
the various parameters and settings relevant to the prim and retry.  
You might look at some of the primitive calling methods in BitBlt to  
see examples.


> 2. Create the primitive code calling code in Slang dialect. Something
> like this (I'm sure this is very incomplete/plain wrong):
> primitiveRun
> SmartSyntaxInterpreterPlugin doPrimitive: #StartStream
> withArguments: #().
>
> Should I be using interpreterProxy here. Where does the
> interpreterProxy instance get created.
Here you've clearly been handed the wrong end of an inappropriate  
stick. No need for anything like this and you don't have to worry  
about any interpreterProxy.
Your method in 1. will do all that is needed to actually call the  
prim when it exists. The #doPrimitive:withArguments: stuff is to do  
with simulating the plugin code and I think it is not fully  
functional right now anyway.

>
> 3. Run Slang to convert (2) to 'C'.
Not quite. You need to create a plugin class as a subclass of  
SmartSyntaxInterpreterPlugin (bad name I know but better than  
'TestInterpreterPlugin' at least) and implement your function.
By default the plugin will be named the same as your class but you  
can override that (see implementors of #moduleName I think) if you  
like. Remember that the actual generated plugin name needs to match  
what you put in the method in 1. above! Don't Ask Me How I Know.

> I know I need to use VMMaker and
> should only need to convert this one method.
Well, one plugin rather than one method.


> I have looked at
> VMMakerTool but don't know if this is the right thing to use. If it is
> it's not clear to me how one gets a new plugin into the 'plugins not
> built' list and the ones listed are not draggable anyway. Does my code
> need to go into VMMaker-Plugins or is this just a convention? Also  
> tried
> cCodeGenerator cCodeForMethod but couldn't get it to work.
Sounds to me like you really need to read the swiki page(s) mentioned  
above. Try that and 'call us back'.

>
> 4 Assuming someone can put me right on (3) I then have to compile and
> link (on XP by the way) for which I can use VisualStudio or Borland.
> What do I compile/link against? I don't see any header files or  
> libs in
> the squeak distribution.
The win32 VM is built with gcc and some tools Andreas has packaged up  
somewhere. I'm not sure where but somebody will know. The swiki page  
on building a windows vm seems to point to a dead server right now.
>
> 5. Having got my compiled/linked code, not sure what form that should
> take, is it object format, linked to a .lib or .dll. How do I register
> it with Squeak so it knows how to play.
Once it is built it can go in a variety of places, somewhat dependent  
on the platform. Pretty much all of them are ok with the plugin (it's  
a .dll on window) in the same directory as the vm for testing  
purposes. When you run you method to call the prim it will lookup the  
plugin and load it and cache the function pointer(s) and run it. Next  
time you run it  the pointer can be used directly of course.
>


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
BASIC is to computer programming as QWERTY is to typing.

Tim
Thanks for the pointers. There seems to be a problem with the swiki but I will take a look when it's up. A few comments in-line.

Bob

> Hi
>
> I'm trying to build a plug-in and getting pretty lost at the moment.
Sounds like you haven't been pointed to the swiki pages yet: - Start 
at http://minnow.cc.gatech.edu/squeak/2105 and particularly follow 
links to NamedPrimitives and similar.
>
> 1. Create the smalltalk class to call the primitive. Something like
> this:
>       run
>               <primitive: 'primitiveRun' module:'PhasingDSPPlugin'>
>               ^ false
>
That's a reasonable start. You might want to return 'self 
primitiveFailed' instead of false or you might have some way to check 
the various parameters and settings relevant to the prim and retry. 
You might look at some of the primitive calling methods in BitBlt to 
see examples.

> 2. Create the primitive code calling code in Slang dialect. Something
> like this (I'm sure this is very incomplete/plain wrong):
>       primitiveRun
>               SmartSyntaxInterpreterPlugin doPrimitive: #StartStream
> withArguments: #().
>
>       Should I be using interpreterProxy here. Where does the
> interpreterProxy instance get created.
>> Here you've clearly been handed the wrong end of an inappropriate 
>> stick. No need for anything like this and you don't have to worry 
>> about any interpreterProxy.
>> Your method in 1. will do all that is needed to actually call the 
>> prim when it exists. The #doPrimitive:withArguments: stuff is to do 
>> with simulating the plugin code and I think it is not fully 
>> functional right now anyway.

I was looking at various code snippets, shows how you can be misled.

>
> 3. Run Slang to convert (2) to 'C'.
>> Not quite. You need to create a plugin class as a subclass of 
>> SmartSyntaxInterpreterPlugin (bad name I know but better than 
>> 'TestInterpreterPlugin' at least) and implement your function.
>> By default the plugin will be named the same as your class but you 
>> can override that (see implementors of #moduleName I think) if you 
>> like. Remember that the actual generated plugin name needs to match 
>> what you put in the method in 1. above! Don't Ask Me How I Know.

Ok, I've done that. I'm not sure what 'implement your function' means. I already have my function, it's in 'C' and I want to bridge to it. I guess the code here makes that bridge but not sure what I need to write. Guess that's on the Swiki somewhere.

 
> I know I need to use VMMaker and
> should only need to convert this one method.
>> Well, one plugin rather than one method.

> I have looked at
> VMMakerTool but don't know if this is the right thing to use. If it is
> it's not clear to me how one gets a new plugin into the 'plugins not
> built' list and the ones listed are not draggable anyway. Does my code
> need to go into VMMaker-Plugins or is this just a convention? Also 
> tried
> cCodeGenerator cCodeForMethod but couldn't get it to work.
>> Sounds to me like you really need to read the swiki page(s) mentioned 
>> above. Try that and 'call us back'.

Will do.

>
> 4 Assuming someone can put me right on (3) I then have to compile and
> link (on XP by the way) for which I can use VisualStudio or Borland.
> What do I compile/link against? I don't see any header files or 
> libs in
> the squeak distribution.
>> The win32 VM is built with gcc and some tools Andreas has packaged up 
>> somewhere. I'm not sure where but somebody will know. The swiki page 
>> on building a windows vm seems to point to a dead server right now.
>
> 5. Having got my compiled/linked code, not sure what form that should
> take, is it object format, linked to a .lib or .dll. How do I register
> it with Squeak so it knows how to play.
>> Once it is built it can go in a variety of places, somewhat dependent 
>> on the platform. Pretty much all of them are ok with the plugin (it's 
>> a .dll on window) in the same directory as the vm for testing 
>> purposes. When you run you method to call the prim it will lookup the 
>> plugin and load it and cache the function pointer(s) and run it. Next 
>> time you run it  the pointer can be used directly of course.
>

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
BASIC is to computer programming as QWERTY is to typing.

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

[hidden email] wrote:

> Tim
> Thanks for the pointers. There seems to be a problem with the swiki
> but I will take a look when it's up. A few comments in-line.
>
bob,
I am in NO WAY an expert here, but this helped me:
http://bike-nomad.com/squeak/writingPlugins.html

and this helps from the new blue book with this chapter:
http://coweb.cc.gatech.edu:8888/squeakbook/uploads/greenberg.pdf


--

*Brad Fuller*
+1 (408) 799-6124
*Sonaural Audio Studio*
See Us At GDC 2006
<http://www.cmpevents.com/GD06/a.asp?option=C&V=11&SessID=1954>
Hear us online: www.Sonaural.com <http://www.sonaural.com/>
See me on O'Reilly <http://www.oreillynet.com/pub/au/2184>

> Tim
> Thanks for the pointers. There seems to be a problem with the swiki
> but I will take a look when it's up. A few comments in-line.
>
>> bob,
>> I am in NO WAY an expert here, but this helped me:
>> http://bike-nomad.com/squeak/writingPlugins.html

>> and this helps from the new blue book with this chapter:
>> http://coweb.cc.gatech.edu:8888/squeakbook/uploads/greenberg.pdf

Brad

Thanks for the pointers. I was trying to follow your first ref which led
me into trouble as it missed out some things like what to inherit from.
The PDF however was useful and I have managed to create myself a 'C'
file. Of course like everything it's out of data as it inherits from
InterpreterPlugin not SmartSyntaxInterpreterPlugin and the method it
uses, 'translate' no longer exists but I found 'translateInDirectory'
seems to be the one to use. I guess having got my 'C' I now need to
figure out how to build it. We is getting there though..

Bob
--

*Brad Fuller*
+1 (408) 799-6124
*Sonaural Audio Studio*
See Us At GDC 2006
<http://www.cmpevents.com/GD06/a.asp?option=C&V=11&SessID=1954>
Hear us online: www.Sonaural.com <http://www.sonaural.com/>
See me on O'Reilly <http://www.oreillynet.com/pub/au/2184>


*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

Cowdery, Bob [UK] wrote:

oh yeah... I believe  TestInterpreterPlugin has been renamed
SmartSyntaxInterpreterPlugin.

Configuring and building the VM was the hardest thing for me to get
going. I first tried to build the whole VM before I tackled creating a
plugin. Then, when I got the VM process building down, I moved on to a
plugin. I took little steps. And.. I still ain't walking.

Brad Fuller wrote:

> Configuring and building the VM was the hardest thing for me to get
> going. I first tried to build the whole VM before I tackled creating a
> plugin. Then, when I got the VM process building down, I moved on to a
> plugin. I took little steps. And.. I still ain't walking.

I 'm not going to try and build the VM, just my plugin. First thing I now come up against is where are the header files like sqVirtualMachine.h. I have read everything I can get hold of but no clues so far. The closest I get is something called InterpreterSupportCode but my image doesn't seem to have that class. I guess it should be exportable from the image - but how -.

Thanks
Bob

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

On Mon, Jan 16, 2006 at 01:30:12PM -0000, [hidden email] wrote:
For developing your plugin, there is no need to use the latest
and greatest VM source or VMMaker. You may find it easier to
just get a stable combination of slightly out-of-date Squeak
and source code.

For stable source, look at www.squeakvm.org. For example, if you
are building something for unix/linux, you can use
  http://www.squeakvm.org/unix/release/Squeak-3.7-7.src.tar.gz
This contains the platform sources, including sqVirtualMachine.h.

Then just use a stable Squeak 3.7 full image, which will include
an out-of-date but perfectly servicable VMMaker. The combination
should work fine for building plugins. And you *should* build a
complete VM, not just your plugin. It will only take a few minutes
extra, and it makes it possible to use all the existing configure
and make procedures. Once you've done this, you can use VMMaker
to regenerate your plugin as you develop it, and the make procedure
will rebuild just your plugin as you would expect.

The only caution I would add is to be aware that the up-to-date
sources are intended to support both 64 bit and 32 bit images, so
everything that defaulted to type "int" in the older sources is
now defined as type "sqInt", which might be either 32 bits or
64 bits. Just don't assume that anything defaults to an int, and
read http://www.squeakvm.org/squeak64/ if you want some more
background. But don't lose any sleep over the issue, it's fairly
easy to update your plugin for 64 bit later on if you don't feel
like worrying about it right now.

Dave

David Lewis wrote:

> For developing your plugin, there is no need to use the latest
> and greatest VM source or VMMaker. You may find it easier to
> just get a stable combination of slightly out-of-date Squeak
> and source code.

> For stable source, look at www.squeakvm.org. For example, if you
> are building something for unix/linux, you can use
>   http://www.squeakvm.org/unix/release/Squeak-3.7-7.src.tar.gz
> This contains the platform sources, including sqVirtualMachine.h.

> Then just use a stable Squeak 3.7 full image, which will include
> an out-of-date but perfectly servicable VMMaker. The combination
> should work fine for building plugins. And you *should* build a
> complete VM, not just your plugin. It will only take a few minutes
> extra, and it makes it possible to use all the existing configure
> and make procedures. Once you've done this, you can use VMMaker
> to regenerate your plugin as you develop it, and the make procedure
> will rebuild just your plugin as you would expect.

> The only caution I would add is to be aware that the up-to-date
> sources are intended to support both 64 bit and 32 bit images, so
> everything that defaulted to type "int" in the older sources is
> now defined as type "sqInt", which might be either 32 bits or
> 64 bits. Just don't assume that anything defaults to an int, and
> read http://www.squeakvm.org/squeak64/ if you want some more
> background. But don't lose any sleep over the issue, it's fairly
> easy to update your plugin for 64 bit later on if you don't feel
> like worrying about it right now.

Dave

Thanks very much for the help. I am absolutely staggered that I managed to do this and in less than an hour. Not sure I did everything correct so please double check me.

1. Downloaded the 3.7.1 sources and win32 tools.
2. Ran the make on the sources, completed without error.
3. Now the dodgy bit. Thought I would try the new Squeak.exe it built with the 3.8 image and change set. It ran up and appeared to work fine.

4. Filed-in my plugin class from my 3.8 system.
5. Ran VMMakerTool and dragged my SDRPhasingDSPPlugin into the ExternalPlugins. Hit GenerateExternalPlugins and it dropped the 'C' code into the plugins directory.

6. Ran make again and it built just the plugin.
7. Copied the dll over to the root of my 3.8 system and as far as I can tell it does call it because I can change the smalltalk plugin code and it still executes the code I built.

Now I need to build the real thing with all my C code (which includes PortAudio) hanging off the plugin.

Regards
Bob

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

[hidden email] wrote:
you can build just the one plugin if you select it, right click and
choose "generate plugin"

> 6. Ran make again and it built just the plugin.
> 7. Copied the dll over to the root of my 3.8 system and as far as I
> can tell it does call it because I can change the smalltalk plugin
> code and it still executes the code I built.
>
cool.

> Now I need to build the real thing with all my C code (which includes
> PortAudio) hanging off the plugin
>
are we both building a PortAudio plugin? What's the extent of your
PortAudio work?

> Dave
>
> Thanks very much for the help. I am absolutely staggered that I
> managed to do this and in less than an hour. Not sure I did everything
> correct so please double check me.
>
> 1. Downloaded the 3.7.1 sources and win32 tools.
> 2. Ran the make on the sources, completed without error.
> 3. Now the dodgy bit. Thought I would try the new Squeak.exe it built
> with the 3.8 image and change set. It ran up and appeared to work fine.
>
> 4. Filed-in my plugin class from my 3.8 system.
> 5. Ran VMMakerTool and dragged my SDRPhasingDSPPlugin into the
> ExternalPlugins. Hit GenerateExternalPlugins and it dropped the 'C'
> code into the plugins directory.
>
you can build just the one plugin if you select it, right click and
choose "generate plugin"

> 6. Ran make again and it built just the plugin.
> 7. Copied the dll over to the root of my 3.8 system and as far as I
> can tell it does call it because I can change the smalltalk plugin
> code and it still executes the code I built.
>
cool.

> Now I need to build the real thing with all my C code (which includes
> PortAudio) hanging off the plugin
>

Brad Fuller wrote:
>> are we both building a PortAudio plugin? What's the extent of your
>> PortAudio work?

Not exactly, at least not yet. I am building a software radio which at the moment is just front end stuff. The signal processing is all written in C and there are various sources of it, my own stuff, some from the Flex Radio open source project as well as GNURadio. The processing essentially processes an audio stream so the stream does not need to touch Squeak although there might be some advantages if it did so. At the moment I just have a simple phasing demodulator I am trying to hook in. The interface to it is just a control interface, so pretty simple.

Bob

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

Yep, I think you've got it. Have fun!

Dave

On Mon, Jan 16, 2006 at 09:03:09PM -0000, [hidden email] wrote:

[hidden email] wrote:
I don't know how it works in the windows tree, but in the unix tree the
makefile wil be updated if you run config/configure, i believe.