Re: Replacement for DLLCC front-end...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Replacement for DLLCC front-end...

Michael Lucas-Smith-2
Thanks Antony, this is fantastic and exactly what I suggested we could
do - it's good to see it turn in to a reality. The best thing is, it's
better than SWIG and Boost in that you don't have to recompile the
libraries -at all-.

I'll have to find some time to look further in to this and play with it.

Cheers,
Michael

Antony Blakey wrote:

>
> Hi,
>
> I've built a replacement for the DLLCC front-end using GCCXML, and
> built a simple mechanism for dealing with library differences between
> platforms. I've got one hurdle before it's done (well, releasable),
> and that's bitfield handling. It really needs to be handled in the
> DLLCC machinery i.e. alignment/access, but it seems not to be, at
> least upon my inspection of the code. I'll describe what I've built,
> and what I'm hoping is that someone with expertise in DLLCC internals
> will see enough benefit to do the DLLCC bitfield stuff.
>
> What I've built uses ExternalProcessStreams to execute gccxml (twice)
> to pick up #defines and the parsed infoset of C/C++ headers. So gccxml
> needs to be installed, and I've not done windows, although apparently
> gccxml works there as well (maybe gccxml could be a library - I don't
> know). The resulting #defines/infoset is reified in a typed tree
> structure in VW. You can then manipulate the tree fairly trivially to
> do a number of things such as:
>
>  - Replace pointer-to-functions with typedefs for use in CCallback
> casting.
>  - Build wrapper classes for structs as I've seen in CairoGraphics
>  - Add _threaded to some functions
>  - Convert #defines to/from enums
>  - Resolve system-specific (or all) typedefs so you don't see stuff
> like __darwin_fsfilcnt_t
>
>  ... and basically anything else you can imagine.
>
> Then you can generate the methods into a platform specific
> ExternalInterface sub(sub*)class, but you only need to do it for a
> specified set of function/structs etc and it will compute the
> transitive closure from the roots and only generate what you need.
>
> All of this is programatic and requires no stuffing around with C
> files or hand-editing i.e. you're in VW the entire time. It's a proper
> parser, and handles C++, including giving the mangled names using a
> variety of platform/compiler strategies, so you can link to C++
> (although I haven't reified the *entire* infoset as typed classes
> yet). It's repeatable, and can be simple re-executed on different
> platforms to build platform-specific library wrappers. A simple
> mechanism based on External-Interface-Pragmas makes
> platform-distinctions both at generation time and load time automatic
> and trivial, whilst still allowing custom code both at a library and
> per-platform level, without losing code when regenerating (it will
> delete stuff it knows it generated when regenerating).
>
> Why don't I just drop it in Store and let the market decide? Because
> I'm hoping to motive the bitfield stuff. I've got a hack but it's
> based on pre-computing all the alignment details for a given platform
> and transforming the field and replacing bitfield members with a
> padder of the appropriate size i.e. losing their declaration in the
> struct. I could automatically generate getter/setter methods for
> bitfields, but that smells bad to me and in the case of FUSE, the
> three bitfields aren't of that much interest to me. Furthermore, it
> means the library client has to know which fields are bitfields.
>
> I'm using this to generate an interface to the FUSE library which
> allows me to write user space filesystem handlers in VW (FUSE is
> OSX/Linux only). It worked fine on OSX but when I went to deploy on
> Linux the work of generating the library class and hand-editing
> preprocessed header file output again was just too much bullshit, so I
> wrote this instead.
>
> [ BTW: This is the second time I've sent this because it's not showing
> up in the lists ]
>
> Antony Blakey
> --------------------------
> CTO, Linkuistics Pty Ltd
> Ph: 0438 840 787
>
> It is as useless to argue with those who have renounced the use of
> reason as to administer medication to the dead.
>   -- Thomas Jefferson
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Replacement for DLLCC front-end...

Eliot Miranda-2
I don't mean to be a wet blanket but some obvious questions need answering.

- what happens on platforms where gcc is not the default compiler (i.e. Windows, Solaris, IRIX, AIX, to a lesser extent MacOS 9)?

- how (and as importantly who) is gcc extended such that it obeys Visual C++'s pragmas and hence parses Windows include files correctly?

- Does the VM team have to maintain and distribute a version of gcc with the xml extension on all supported platforms?

On 6/26/07, Michael Lucas-Smith <[hidden email]> wrote:
Thanks Antony, this is fantastic and exactly what I suggested we could
do - it's good to see it turn in to a reality. The best thing is, it's
better than SWIG and Boost in that you don't have to recompile the
libraries -at all-.

I'll have to find some time to look further in to this and play with it.

Cheers,
Michael

Antony Blakey wrote:

>
> Hi,
>
> I've built a replacement for the DLLCC front-end using GCCXML, and
> built a simple mechanism for dealing with library differences between
> platforms. I've got one hurdle before it's done (well, releasable),
> and that's bitfield handling. It really needs to be handled in the
> DLLCC machinery i.e. alignment/access, but it seems not to be, at
> least upon my inspection of the code. I'll describe what I've built,
> and what I'm hoping is that someone with expertise in DLLCC internals
> will see enough benefit to do the DLLCC bitfield stuff.
>
> What I've built uses ExternalProcessStreams to execute gccxml (twice)
> to pick up #defines and the parsed infoset of C/C++ headers. So gccxml
> needs to be installed, and I've not done windows, although apparently
> gccxml works there as well (maybe gccxml could be a library - I don't
> know). The resulting #defines/infoset is reified in a typed tree
> structure in VW. You can then manipulate the tree fairly trivially to
> do a number of things such as:
>
>  - Replace pointer-to-functions with typedefs for use in CCallback
> casting.
>  - Build wrapper classes for structs as I've seen in CairoGraphics

>  - Add _threaded to some functions
>  - Convert #defines to/from enums
>  - Resolve system-specific (or all) typedefs so you don't see stuff
> like __darwin_fsfilcnt_t
>
>  ... and basically anything else you can imagine.
>
> Then you can generate the methods into a platform specific
> ExternalInterface sub(sub*)class, but you only need to do it for a
> specified set of function/structs etc and it will compute the
> transitive closure from the roots and only generate what you need.
>
> All of this is programatic and requires no stuffing around with C
> files or hand-editing i.e. you're in VW the entire time. It's a proper
> parser, and handles C++, including giving the mangled names using a
> variety of platform/compiler strategies, so you can link to C++
> (although I haven't reified the *entire* infoset as typed classes
> yet). It's repeatable, and can be simple re-executed on different
> platforms to build platform-specific library wrappers. A simple
> mechanism based on External-Interface-Pragmas makes
> platform-distinctions both at generation time and load time automatic
> and trivial, whilst still allowing custom code both at a library and
> per-platform level, without losing code when regenerating (it will
> delete stuff it knows it generated when regenerating).
>
> Why don't I just drop it in Store and let the market decide? Because
> I'm hoping to motive the bitfield stuff. I've got a hack but it's
> based on pre-computing all the alignment details for a given platform
> and transforming the field and replacing bitfield members with a
> padder of the appropriate size i.e. losing their declaration in the
> struct. I could automatically generate getter/setter methods for
> bitfields, but that smells bad to me and in the case of FUSE, the
> three bitfields aren't of that much interest to me. Furthermore, it
> means the library client has to know which fields are bitfields.
>
> I'm using this to generate an interface to the FUSE library which
> allows me to write user space filesystem handlers in VW (FUSE is
> OSX/Linux only). It worked fine on OSX but when I went to deploy on
> Linux the work of generating the library class and hand-editing
> preprocessed header file output again was just too much bullshit, so I
> wrote this instead.
>
> [ BTW: This is the second time I've sent this because it's not showing
> up in the lists ]
>
> Antony Blakey
> --------------------------
> CTO, Linkuistics Pty Ltd
> Ph: 0438 840 787
>
> It is as useless to argue with those who have renounced the use of
> reason as to administer medication to the dead.
>   -- Thomas Jefferson
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Replacement for DLLCC front-end...

Christian Schuckmann
In reply to this post by Michael Lucas-Smith-2

From time to time I have to use dllcc - usually to make
some MS-Windows functionality available. And usually it is
quite a pain to do so. I start with a bunch of header files
and manually assemble a simple header file that can be
understood by the dllcc parser.
It would really be great to have a mechanism that can
parse the original definitions and transport them without
loss to the Smalltalk side.
Even if it is not the "works everywhere for everybody"
solution - it is in many cases better than the current solution.
So: Go ahead Antony! (and Cincom!!!)
Hopefully that thing will be available when I have to do the
dirty C work again ;-)

-Christian

--
Christian Schuckmann                           intelligent views gmbh
Development Manager Core Technologies          Julius-Reiber-Str. 17
Tel/Fax ++49 6151 5006 - 240/138               64293 Darmstadt
e-mail: [hidden email]                http://www.i-views.de/


-----Ursprüngliche Nachricht-----
Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Antony Blakey
Gesendet: Dienstag, 3. Juli 2007 11:55
An: Eliot Miranda
Cc: VWDEV List; VWNC; Jaroslaw Podgajny
Betreff: Re: Replacement for DLLCC front-end...



Firstly - http://www.gccxml.org/ for those who are interested.

On 30/06/2007, at 5:02 AM, Eliot Miranda wrote:

> - what happens on platforms where gcc is not the default compiler
> (i.e. Windows, Solaris, IRIX, AIX, to a lesser extent MacOS 9)?

I know there are versions of gccxml available for Linux, OSX,  
Windows, IRIX64, SunOS and generic Unix. It's a patch to the gcc c++  
parser, so I imagine compiling under Solaris/IRIX/AIX is not really  
an issue. I'm afraid you're just SOL on OS9.

> - how (and as importantly who) is gcc extended such that it obeys
> Visual C++'s pragmas and hence parses Windows include files correctly?

It has an ability to simulate other parsers (specifically VC) - I'm  
not sure how well it does, but remember it's only parsing the source  
- it's not interested in struct layout etc. I'm also not sure about  
the calling conventions annotations - is that still used on recent  
Windows versions? In any case, stuff like layout is really the job of  
an ABI modeling layer, which the guts of DLLCC sort-of-is.

> - Does the VM team have to maintain and distribute a version of gcc
> with the xml extension on all supported platforms?

GCCXML is separate from GCC - it uses some GCC sources, but is not 'a  
version of gcc with the xml extension'.

I'm not proposing that this be a 'replacement' in the sense that it  
would have to work for everyone, rather it would be an enhancement  
for some platforms. I hoped that it would be useful for enough people  
that it's worth Cincom's while to fix the bitfield support in DLLCC.

OSX gccxml is in DarwinPorts. Ubuntu gccxml is an apt-gettable  
package. On Windows it would need an installer because currently you  
have to compile it yourself to get a recent version. I started the  
process but really my Windows interest is pretty low. I could try  
Solaris/x86.

One question is whether it's more effort to follow through on this or  
to fix DLLCC's front end, which is broken. Comparing efforts needs to  
compare like vs. like i.e. a working DLLCC vs. a working GCCXML.

One benefit is that the gccxml work was done under contract. The same  
people (kitware, who host it) might be available to package it for  
the platforms Cincom's interested in. That might be an attractive  
option financially.

IMO DLLCC should be an absolute bedrock feature. There is SO much  
useful stuff you can do that isn't currently within the Smalltalk  
Ivory Tower. And considering Cincom's resources, it's not reasonable  
to expect that situation to change.

Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

A priest, a minister and a rabbi walk into a bar. The bartender says  
"What is this, a joke?"