testing MT and reentrant

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

testing MT and reentrant

EstebanLM

Hi,
I'm trying to build latest oscog... and, well, I wonder how can I test the MT FFI? Or the reentrant FFI?

Thanks,
Esteban
Reply | Threaded
Open this post in threaded view
|

Re: testing MT and reentrant

EstebanLM

about this... what I wonder is if regular FFI package should work with new ThreadedIA32FFIPlugin... or I need something new?

El 23/03/2011, a las 10:44a.m., Esteban Lorenzano escribió:

> Hi,
> I'm trying to build latest oscog... and, well, I wonder how can I test the MT FFI? Or the reentrant FFI?
>
> Thanks,
> Esteban

Reply | Threaded
Open this post in threaded view
|

Re: testing MT and reentrant

Eliot Miranda-2
 


On Wed, Mar 23, 2011 at 2:36 PM, Esteban Lorenzano <[hidden email]> wrote:

about this... what I wonder is if regular FFI package should work with new ThreadedIA32FFIPlugin... or I need something new?

I have some image-level changes that I'll try and commit asap, or at least give you as some inbox packages
- parsing FFI method annotations to discover the #threaded keyword that sets the "go ahead and thread this call" bit in an FFI spec
- an additional inst var in ExternalFunction (IIRC) that allows the VM to record how much stack space to reserve when marshalling an FFI call
- the new callback marshalling machinery that provides platform-independence

I just need to fid the time to push the code to you.

Now on testing I use
a) an image with a "listener" that reads and writes from/to stdin/out while allowing one to interact with the image
b) a native Mac OS file dialog that uses a threaded call to invoke without blocking the image and threaded callbacks to determine which files to show (filter callback) and what the accept action is.  I've of course lost my workspace containing this and so have to ferret out the doits from my changes file (I feel such a fool!).  Alas this will need work as it used an extension to a Teleplace native file dialog plugin.  But posting it to FFI will be good; it will test the FFI further.
c) a threaded version of the ODBC connect that hasn't really been tested

etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages, changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible and don't break trunk I'm going ot be much slower.

best regards (albeit feeling a little frazzled :-) ),
Eliot

El 23/03/2011, a las 10:44a.m., Esteban Lorenzano escribió:

> Hi,
> I'm trying to build latest oscog... and, well, I wonder how can I test the MT FFI? Or the reentrant FFI?
>
> Thanks,
> Esteban


Reply | Threaded
Open this post in threaded view
|

Re: testing MT and reentrant

Igor Stasenko

On 24 March 2011 19:18, Eliot Miranda <[hidden email]> wrote:
>
>
> etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages, changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible and don't break trunk I'm going ot be much slower.

Another alternative is to create a gitorious account and publish your
intermediate hacks there, and then once you ready, commit to blessed
repository.
In this way you can make anything available, without endangering main branch :)

And don't worry, we have a client for you who can test new
functionality deeply - an Ocean project which implements networking
stack from scratch and it badly needs threaded FFI.

> best regards (albeit feeling a little frazzled :-) ),
> Eliot



--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Levente Uzonyi-2
In reply to this post by Eliot Miranda-2
 
On Thu, 24 Mar 2011, Eliot Miranda wrote:

(pine can't quote your mail, sorry)

"I have some image-level changes that I'll try and commit asap, or at
least give you as some inbox packages
- parsing FFI method annotations to discover the #threaded keyword that
sets the "go ahead and thread this call" bit in an FFI spec
- an additional inst var in ExternalFunction (IIRC) that allows the VM to
record how much stack space to reserve when marshalling an FFI call
- the new callback marshalling machinery that provides
platform-independence

I just need to fid the time to push the code to you.

Now on testing I use
a) an image with a "listener" that reads and writes from/to stdin/out
while allowing one to interact with the image
b) a native Mac OS file dialog that uses a threaded call to invoke without
blocking the image and threaded callbacks to determine which files to
show (filter callback) and what the accept action is.  I've of course lost
my workspace containing this and so have to ferret out the doits from my
changes file (I feel such a fool!).  Alas this will need work as it used
an extension to a Teleplace native file dialog plugin.  But posting it to
FFI will be good; it will test the FFI further.
c) a threaded version of the ODBC connect that hasn't really been tested

etting you and others to start pounding on these would be fantastic.  Just
a matter of finding time.  f you're happy for me to push inbox packages,
changesets, workspace contents then I can provide something quickly.  If
you want me to commit canges that I've made sure are backward compatible
and don't break trunk I'm going ot be much slower.

best regards (albeit feeling a little frazzled :-) ),
Eliot"

Isn't it the best time to migrate the syntax of FFI calls to Pragmas?
I know it takes some time to implement the pragma support, but IIRC Lukas
did that a few years ago, so dusting it off and adding support for
threaded calls (which I didn't see yet) shouldn't be that hard.
Both formats could be supported for a while, then we could leave the
current format behind and simplify the parsers/compilers, etc.


Levente
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Eliot Miranda-2
 


2011/3/24 Levente Uzonyi <[hidden email]>
 
On Thu, 24 Mar 2011, Eliot Miranda wrote:

(pine can't quote your mail, sorry)

:)  No problem.
 

"I have some image-level changes that I'll try and commit asap, or at least give you as some inbox packages
- parsing FFI method annotations to discover the #threaded keyword that sets the "go ahead and thread this call" bit in an FFI spec
- an additional inst var in ExternalFunction (IIRC) that allows the VM to record how much stack space to reserve when marshalling an FFI call
- the new callback marshalling machinery that provides platform-independence

I just need to fid the time to push the code to you.

Now on testing I use
a) an image with a "listener" that reads and writes from/to stdin/out while allowing one to interact with the image
b) a native Mac OS file dialog that uses a threaded call to invoke without blocking the image and threaded callbacks to determine which files to
show (filter callback) and what the accept action is.  I've of course lost my workspace containing this and so have to ferret out the doits from my
changes file (I feel such a fool!).  Alas this will need work as it used an extension to a Teleplace native file dialog plugin.  But posting it to
FFI will be good; it will test the FFI further.
c) a threaded version of the ODBC connect that hasn't really been tested

etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages,
changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible
and don't break trunk I'm going ot be much slower.

best regards (albeit feeling a little frazzled :-) ),
Eliot"

Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.

SPunds good.  Got any URLs?  Lukas, got any code?
 
Both formats could be supported for a while, then we could leave the current format behind and simplify the parsers/compilers, etc.

Indeed.  In fact, nothing is set in stone here.  I desperately want an accurate C signature, not the quick hack we have now (a good thing we have a quick hack, but its a long way short of copy/pasting C into the pragma).  The interpreted marshalling scheme Andreas did (ffi specs etc) can serve for the moment.  The main thing before getting rid of that is a design for e.g. a NativeBoost based marshalling code generator that is cross-platform, cross-ABI and works with the interpreted VMs.  I think this is therefore a good way off, and there are people like Esteban who need to get going now, so we can't let the perfect be the enemy of the good and need to provide something functional asap, and that may mean living with the current ffi annotation for a few months.

best,
Eliot



Levente

Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Igor Stasenko

On 24 March 2011 19:43, Eliot Miranda <[hidden email]> wrote:

>
>
>
> 2011/3/24 Levente Uzonyi <[hidden email]>
>>
>>
>> On Thu, 24 Mar 2011, Eliot Miranda wrote:
>>
>> (pine can't quote your mail, sorry)
>
> :)  No problem.
>
>>
>> "I have some image-level changes that I'll try and commit asap, or at least give you as some inbox packages
>> - parsing FFI method annotations to discover the #threaded keyword that sets the "go ahead and thread this call" bit in an FFI spec
>> - an additional inst var in ExternalFunction (IIRC) that allows the VM to record how much stack space to reserve when marshalling an FFI call
>> - the new callback marshalling machinery that provides platform-independence
>>
>> I just need to fid the time to push the code to you.
>>
>> Now on testing I use
>> a) an image with a "listener" that reads and writes from/to stdin/out while allowing one to interact with the image
>> b) a native Mac OS file dialog that uses a threaded call to invoke without blocking the image and threaded callbacks to determine which files to
>> show (filter callback) and what the accept action is.  I've of course lost my workspace containing this and so have to ferret out the doits from my
>> changes file (I feel such a fool!).  Alas this will need work as it used an extension to a Teleplace native file dialog plugin.  But posting it to
>> FFI will be good; it will test the FFI further.
>> c) a threaded version of the ODBC connect that hasn't really been tested
>>
>> etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages,
>> changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible
>> and don't break trunk I'm going ot be much slower.
>>
>> best regards (albeit feeling a little frazzled :-) ),
>> Eliot"
>>
>> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.
>
+1000.

> SPunds good.  Got any URLs?  Lukas, got any code?
>
>>
>> Both formats could be supported for a while, then we could leave the current format behind and simplify the parsers/compilers, etc.
>
+1

> Indeed.  In fact, nothing is set in stone here.  I desperately want an accurate C signature, not the quick hack we have now (a good thing we have a quick hack, but its a long way short of copy/pasting C into the pragma).  The interpreted marshalling scheme Andreas did (ffi specs etc) can serve for the moment.  The main thing before getting rid of that is a design for e.g. a NativeBoost based marshalling code generator that is cross-platform, cross-ABI and works with the interpreted VMs.  I think this is therefore a good way off, and there are people like Esteban who need to get going now, so we can't let the perfect be the enemy of the good and need to provide something functional asap, and that may mean living with the current ffi annotation for a few months.

In NativeBoost i made a separate class for parsing a C function
declaration. It supports two forms of declaration:
 - named function, i.e.
   int foo (int bar, float zork)
and anonymous one:
  int (int bar, float zork)

as input, parser takes a literal array.

So its quite easy to integrate it into compiler/pragma parser, we only
need to find out what format will work for us.

so, it could be something like:

<cdecl: #( int foo (int bar , float baz )) module: #myModule>

or:
<ffi: #( cdecl int foo ... ) module: ... >

The class for parsing function prototype can be found in SqS/NativeBoost
in class NBFnSpecParser
and tests are in:
NBFnSpecParserTest in NativeBoost-Test package.
(some of them are NB-specific) but its easy to rip out extra functionality :)

Also, there is a wiki page explaining this format:
http://code.google.com/p/nativeboost/wiki/NBFnSpecParser

So, if we agree with plan, i could port this class and adopt it for
compiler (and yes , by taking Lukas code as well)

--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Eliot Miranda-2
 


On Thu, Mar 24, 2011 at 12:02 PM, Igor Stasenko <[hidden email]> wrote:

On 24 March 2011 19:43, Eliot Miranda <[hidden email]> wrote:
>
>
>
> 2011/3/24 Levente Uzonyi <[hidden email]>
>>
>>
>> On Thu, 24 Mar 2011, Eliot Miranda wrote:
>>
>> (pine can't quote your mail, sorry)
>
> :)  No problem.
>
>>
>> "I have some image-level changes that I'll try and commit asap, or at least give you as some inbox packages
>> - parsing FFI method annotations to discover the #threaded keyword that sets the "go ahead and thread this call" bit in an FFI spec
>> - an additional inst var in ExternalFunction (IIRC) that allows the VM to record how much stack space to reserve when marshalling an FFI call
>> - the new callback marshalling machinery that provides platform-independence
>>
>> I just need to fid the time to push the code to you.
>>
>> Now on testing I use
>> a) an image with a "listener" that reads and writes from/to stdin/out while allowing one to interact with the image
>> b) a native Mac OS file dialog that uses a threaded call to invoke without blocking the image and threaded callbacks to determine which files to
>> show (filter callback) and what the accept action is.  I've of course lost my workspace containing this and so have to ferret out the doits from my
>> changes file (I feel such a fool!).  Alas this will need work as it used an extension to a Teleplace native file dialog plugin.  But posting it to
>> FFI will be good; it will test the FFI further.
>> c) a threaded version of the ODBC connect that hasn't really been tested
>>
>> etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages,
>> changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible
>> and don't break trunk I'm going ot be much slower.
>>
>> best regards (albeit feeling a little frazzled :-) ),
>> Eliot"
>>
>> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.
>
+1000.

> SPunds good.  Got any URLs?  Lukas, got any code?
>
>>
>> Both formats could be supported for a while, then we could leave the current format behind and simplify the parsers/compilers, etc.
>
+1

> Indeed.  In fact, nothing is set in stone here.  I desperately want an accurate C signature, not the quick hack we have now (a good thing we have a quick hack, but its a long way short of copy/pasting C into the pragma).  The interpreted marshalling scheme Andreas did (ffi specs etc) can serve for the moment.  The main thing before getting rid of that is a design for e.g. a NativeBoost based marshalling code generator that is cross-platform, cross-ABI and works with the interpreted VMs.  I think this is therefore a good way off, and there are people like Esteban who need to get going now, so we can't let the perfect be the enemy of the good and need to provide something functional asap, and that may mean living with the current ffi annotation for a few months.

In NativeBoost i made a separate class for parsing a C function
declaration. It supports two forms of declaration:
 - named function, i.e.
  int foo (int bar, float zork)
and anonymous one:
 int (int bar, float zork)

as input, parser takes a literal array.

So its quite easy to integrate it into compiler/pragma parser, we only
need to find out what format will work for us.

so, it could be something like:

<cdecl: #( int foo (int bar , float baz )) module: #myModule>

or:
<ffi: #( cdecl int foo ... ) module: ... >

This is such a fascinating hack.  e.g.

#(int (*compar)(const void *, const void *))
    #(#int #(#* #compar) #(#const #void #*, #const #void #*))

#(int (struct { field : 8; fence : 8; gate: 16; } arg))
    #(#int #(#struct #'{' #field #':' 8 #';' #fence #':' 8 #';' #gate: 16 #';' #'}' #arg))

Can anyone prove that Igor's approach works for all C signatures?  What about other languages?

I find this is like really dirty sex, repellent and irresistible at one and the same time.


The class for parsing function prototype can be found in SqS/NativeBoost
in class NBFnSpecParser
and tests are in:
NBFnSpecParserTest in NativeBoost-Test package.
(some of them are NB-specific) but its easy to rip out extra functionality :)

Also, there is a wiki page explaining this format:
http://code.google.com/p/nativeboost/wiki/NBFnSpecParser

So, if we agree with plan, i could port this class and adopt it for
compiler (and yes , by taking Lukas code as well)

--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: testing MT and reentrant

Mariano Martinez Peck
In reply to this post by Igor Stasenko
 


On Thu, Mar 24, 2011 at 7:27 PM, Igor Stasenko <[hidden email]> wrote:

On 24 March 2011 19:18, Eliot Miranda <[hidden email]> wrote:
>
>
> etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages, changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible and don't break trunk I'm going ot be much slower.

Another alternative is to create a gitorious account and publish your
intermediate hacks there, and then once you ready, commit to blessed
repository.
In this way you can make anything available, without endangering main branch :)

And don't worry, we have a client for you who can test new
functionality deeply - an Ocean project which implements networking
stack from scratch and it badly needs threaded FFI.


Don't forget the new team for DBXTalk ;)

 
> best regards (albeit feeling a little frazzled :-) ),
> Eliot



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Igor Stasenko
In reply to this post by Eliot Miranda-2

On 24 March 2011 20:10, Eliot Miranda <[hidden email]> wrote:

>
>
> On Thu, Mar 24, 2011 at 12:02 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> On 24 March 2011 19:43, Eliot Miranda <[hidden email]> wrote:
>> >
>> >
>> >
>> > 2011/3/24 Levente Uzonyi <[hidden email]>
>> >>
>> >>
>> >> On Thu, 24 Mar 2011, Eliot Miranda wrote:
>> >>
>> >> (pine can't quote your mail, sorry)
>> >
>> > :)  No problem.
>> >
>> >>
>> >> "I have some image-level changes that I'll try and commit asap, or at
>> >> least give you as some inbox packages
>> >> - parsing FFI method annotations to discover the #threaded keyword that
>> >> sets the "go ahead and thread this call" bit in an FFI spec
>> >> - an additional inst var in ExternalFunction (IIRC) that allows the VM
>> >> to record how much stack space to reserve when marshalling an FFI call
>> >> - the new callback marshalling machinery that provides
>> >> platform-independence
>> >>
>> >> I just need to fid the time to push the code to you.
>> >>
>> >> Now on testing I use
>> >> a) an image with a "listener" that reads and writes from/to stdin/out
>> >> while allowing one to interact with the image
>> >> b) a native Mac OS file dialog that uses a threaded call to invoke
>> >> without blocking the image and threaded callbacks to determine which files
>> >> to
>> >> show (filter callback) and what the accept action is.  I've of course
>> >> lost my workspace containing this and so have to ferret out the doits from
>> >> my
>> >> changes file (I feel such a fool!).  Alas this will need work as it
>> >> used an extension to a Teleplace native file dialog plugin.  But posting it
>> >> to
>> >> FFI will be good; it will test the FFI further.
>> >> c) a threaded version of the ODBC connect that hasn't really been
>> >> tested
>> >>
>> >> etting you and others to start pounding on these would be fantastic.
>> >>  Just a matter of finding time.  f you're happy for me to push inbox
>> >> packages,
>> >> changesets, workspace contents then I can provide something quickly.
>> >>  If you want me to commit canges that I've made sure are backward compatible
>> >> and don't break trunk I'm going ot be much slower.
>> >>
>> >> best regards (albeit feeling a little frazzled :-) ),
>> >> Eliot"
>> >>
>> >> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I
>> >> know it takes some time to implement the pragma support, but IIRC Lukas did
>> >> that a few years ago, so dusting it off and adding support for threaded
>> >> calls (which I didn't see yet) shouldn't be that hard.
>> >
>> +1000.
>>
>> > SPunds good.  Got any URLs?  Lukas, got any code?
>> >
>> >>
>> >> Both formats could be supported for a while, then we could leave the
>> >> current format behind and simplify the parsers/compilers, etc.
>> >
>> +1
>>
>> > Indeed.  In fact, nothing is set in stone here.  I desperately want an
>> > accurate C signature, not the quick hack we have now (a good thing we have a
>> > quick hack, but its a long way short of copy/pasting C into the pragma).
>> >  The interpreted marshalling scheme Andreas did (ffi specs etc) can serve
>> > for the moment.  The main thing before getting rid of that is a design for
>> > e.g. a NativeBoost based marshalling code generator that is cross-platform,
>> > cross-ABI and works with the interpreted VMs.  I think this is therefore a
>> > good way off, and there are people like Esteban who need to get going now,
>> > so we can't let the perfect be the enemy of the good and need to provide
>> > something functional asap, and that may mean living with the current ffi
>> > annotation for a few months.
>>
>> In NativeBoost i made a separate class for parsing a C function
>> declaration. It supports two forms of declaration:
>>  - named function, i.e.
>>   int foo (int bar, float zork)
>> and anonymous one:
>>  int (int bar, float zork)
>>
>> as input, parser takes a literal array.
>>
>> So its quite easy to integrate it into compiler/pragma parser, we only
>> need to find out what format will work for us.
>>
>> so, it could be something like:
>>
>> <cdecl: #( int foo (int bar , float baz )) module: #myModule>
>>
>> or:
>> <ffi: #( cdecl int foo ... ) module: ... >
>
> This is such a fascinating hack.  e.g.
> #(int (*compar)(const void *, const void *))
>     #(#int #(#* #compar) #(#const #void #*, #const #void #*))
> #(int (struct { field : 8; fence : 8; gate: 16; } arg))
>     #(#int #(#struct #'{' #field #':' 8 #';' #fence #':' 8 #';' #gate: 16
> #';' #'}' #arg))
> Can anyone prove that Igor's approach works for all C signatures?

After i did it, i were able to copy-paste functions declarations
straightly from
windoze documentation, without making any changes! :)

>What about other languages?

You can always put a string literal inside literal array.. it will
cost you just extra quotes and extra parsing effort..

> I find this is like really dirty sex, repellent and irresistible at one and
> the same time.

:)))

>>
>> The class for parsing function prototype can be found in SqS/NativeBoost
>> in class NBFnSpecParser
>> and tests are in:
>> NBFnSpecParserTest in NativeBoost-Test package.
>> (some of them are NB-specific) but its easy to rip out extra functionality
>> :)
>>
>> Also, there is a wiki page explaining this format:
>> http://code.google.com/p/nativeboost/wiki/NBFnSpecParser
>>
>> So, if we agree with plan, i could port this class and adopt it for
>> compiler (and yes , by taking Lukas code as well)
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>


--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax

Henrik Sperre Johansen
In reply to this post by Eliot Miranda-2
 
On 24.03.2011 20:10, Eliot Miranda wrote:
 




On Thu, Mar 24, 2011 at 12:02 PM, Igor Stasenko <[hidden email]> wrote:

On 24 March 2011 19:43, Eliot Miranda <[hidden email]> wrote:
>
>
>
> 2011/3/24 Levente Uzonyi <[hidden email]>
>>
>>
>> On Thu, 24 Mar 2011, Eliot Miranda wrote:
>>
>> (pine can't quote your mail, sorry)
>
> :)  No problem.
>
>>
>> "I have some image-level changes that I'll try and commit asap, or at least give you as some inbox packages
>> - parsing FFI method annotations to discover the #threaded keyword that sets the "go ahead and thread this call" bit in an FFI spec
>> - an additional inst var in ExternalFunction (IIRC) that allows the VM to record how much stack space to reserve when marshalling an FFI call
>> - the new callback marshalling machinery that provides platform-independence
>>
>> I just need to fid the time to push the code to you.
>>
>> Now on testing I use
>> a) an image with a "listener" that reads and writes from/to stdin/out while allowing one to interact with the image
>> b) a native Mac OS file dialog that uses a threaded call to invoke without blocking the image and threaded callbacks to determine which files to
>> show (filter callback) and what the accept action is.  I've of course lost my workspace containing this and so have to ferret out the doits from my
>> changes file (I feel such a fool!).  Alas this will need work as it used an extension to a Teleplace native file dialog plugin.  But posting it to
>> FFI will be good; it will test the FFI further.
>> c) a threaded version of the ODBC connect that hasn't really been tested
>>
>> etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages,
>> changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible
>> and don't break trunk I'm going ot be much slower.
>>
>> best regards (albeit feeling a little frazzled :-) ),
>> Eliot"
>>
>> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.
>
+1000.

> SPunds good.  Got any URLs?  Lukas, got any code?
>
>>
>> Both formats could be supported for a while, then we could leave the current format behind and simplify the parsers/compilers, etc.
>
+1

> Indeed.  In fact, nothing is set in stone here.  I desperately want an accurate C signature, not the quick hack we have now (a good thing we have a quick hack, but its a long way short of copy/pasting C into the pragma).  The interpreted marshalling scheme Andreas did (ffi specs etc) can serve for the moment.  The main thing before getting rid of that is a design for e.g. a NativeBoost based marshalling code generator that is cross-platform, cross-ABI and works with the interpreted VMs.  I think this is therefore a good way off, and there are people like Esteban who need to get going now, so we can't let the perfect be the enemy of the good and need to provide something functional asap, and that may mean living with the current ffi annotation for a few months.

In NativeBoost i made a separate class for parsing a C function
declaration. It supports two forms of declaration:
 - named function, i.e.
  int foo (int bar, float zork)
and anonymous one:
 int (int bar, float zork)

as input, parser takes a literal array.

So its quite easy to integrate it into compiler/pragma parser, we only
need to find out what format will work for us.

so, it could be something like:

<cdecl: #( int foo (int bar , float baz )) module: #myModule>

or:
<ffi: #( cdecl int foo ... ) module: ... >

This is such a fascinating hack.  e.g.

#(int (*compar)(const void *, const void *))
    #(#int #(#* #compar) #(#const #void #*, #const #void #*))

#(int (struct { field : 8; fence : 8; gate: 16; } arg))
    #(#int #(#struct #'{' #field #':' 8 #';' #fence #':' 8 #';' #gate: 16 #';' #'}' #arg))

Can anyone prove that Igor's approach works for all C signatures?  What about other languages?

I find this is like really dirty sex, repellent and irresistible at one and the same time.
Thank you for putting words to my feeling at the time I suggested it :)
Don't know if it works for every signature though, only the examples he gave of the old format :/

Cheers,
Henry
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Igor Stasenko
In reply to this post by Eliot Miranda-2

On 24 March 2011 20:58, Igor Stasenko <[hidden email]> wrote:

> On 24 March 2011 20:56, Igor Stasenko <[hidden email]> wrote:
>> On 24 March 2011 20:47, Eliot Miranda <[hidden email]> wrote:
>>>
>>>>
>>>> You can always put a string literal inside literal array.. it will
>>>> cost you just extra quotes and extra parsing effort..
>>>
>>> Yes, but that would be a failure.  If yu ever need a string literal I'd
>>> argue you're better off just using a string literal.  But I think you could
>>> be OK. I think your hack could work for everything.  I just want a proof.
>>
>> it won't parse only following declarations:
>>
>> #( int foo( int arg[] ) )
>>
> but we could modify compiler to treat [ and ] inside a literal array as
> #'['
> and
> #']'
>
> :)
>
oops, sorry, my bad..
it actually works...

#( int foo ( int arg[] ) )  ->

 #(#int #foo #(#int #arg #'[' #']'))

#( int foo ( int arg[10] ) )  ->

#(#int #foo #(#int #arg #'[' 10 #']'))

There is a bug in parser, which fails to parse literal arrays if you
don't put space between function name and its arguments.

#( int foo( int arg[] ) )
- fails

#( int foo ( int arg[] ) )
- works


>> which of course could be turned into just:
>>
>> #( int foo( int * arg ) )
>>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>



--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Lukas Renggli
In reply to this post by Eliot Miranda-2

>> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.
>
> SPunds good.  Got any URLs?  Lukas, got any code?

Sorry, I don't have that code anymore. I used the existing FFI parser
code within a string literal in the pragma.

Lukas

--
Lukas Renggli
www.lukas-renggli.ch
Reply | Threaded
Open this post in threaded view
|

Re: testing MT and reentrant

Igor Stasenko
In reply to this post by Mariano Martinez Peck

On 24 March 2011 20:25, Mariano Martinez Peck <[hidden email]> wrote:

>
>
>
> On Thu, Mar 24, 2011 at 7:27 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> On 24 March 2011 19:18, Eliot Miranda <[hidden email]> wrote:
>> >
>> >
>> > etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages, changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible and don't break trunk I'm going ot be much slower.
>>
>> Another alternative is to create a gitorious account and publish your
>> intermediate hacks there, and then once you ready, commit to blessed
>> repository.
>> In this way you can make anything available, without endangering main branch :)
>>
>> And don't worry, we have a client for you who can test new
>> functionality deeply - an Ocean project which implements networking
>> stack from scratch and it badly needs threaded FFI.
>>
>
> Don't forget the new team for DBXTalk ;)
>

Yeah... so Eliot, as you can see you have customers lined up at your
office already :)

>
>>
>> > best regards (albeit feeling a little frazzled :-) ),
>> > Eliot
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>
>



--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Eliot Miranda-2
In reply to this post by Lukas Renggli
 


On Thu, Mar 24, 2011 at 1:10 PM, Lukas Renggli <[hidden email]> wrote:

>> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.
>
> SPunds good.  Got any URLs?  Lukas, got any code?

Sorry, I don't have that code anymore. I used the existing FFI parser
code within a string literal in the pragma.

So what do you think of the literal array hack.  And remember, this is for posterity so be honest.
 

Lukas

--
Lukas Renggli
www.lukas-renggli.ch

Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

Lukas Renggli

>> >> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I
>> >> know it takes some time to implement the pragma support, but IIRC Lukas did
>> >> that a few years ago, so dusting it off and adding support for threaded
>> >> calls (which I didn't see yet) shouldn't be that hard.
>> >
>> > SPunds good.  Got any URLs?  Lukas, got any code?
>>
>> Sorry, I don't have that code anymore. I used the existing FFI parser
>> code within a string literal in the pragma.
>
> So what do you think of the literal array hack.  And remember, this is
> for posterity so be honest.

I don't like it to have methods polluted with meaningless symbols
(that will also show up in senders/implementors).

I doubt that the symbol hack will make parsing much simpler, because
you still need to analyze the token sequence.

Lukas

--
Lukas Renggli
www.lukas-renggli.ch
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)

stephane ducasse-2
In reply to this post by Levente Uzonyi-2

>
>
> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I know it takes some time to implement the pragma support, but IIRC Lukas did that a few years ago, so dusting it off and adding support for threaded calls (which I didn't see yet) shouldn't be that hard.
> Both formats could be supported for a while, then we could leave the current format behind and simplify the parsers/compilers, etc.
>
>
> Levente

YES!!!

Reply | Threaded
Open this post in threaded view
|

Re: testing MT and reentrant

EstebanLM
In reply to this post by Mariano Martinez Peck
 

El 24/03/2011, a las 4:25p.m., Mariano Martinez Peck escribió:



On Thu, Mar 24, 2011 at 7:27 PM, Igor Stasenko <[hidden email]> wrote:

On 24 March 2011 19:18, Eliot Miranda <[hidden email]> wrote:
>
>
> etting you and others to start pounding on these would be fantastic.  Just a matter of finding time.  f you're happy for me to push inbox packages, changesets, workspace contents then I can provide something quickly.  If you want me to commit canges that I've made sure are backward compatible and don't break trunk I'm going ot be much slower.

Another alternative is to create a gitorious account and publish your
intermediate hacks there, and then once you ready, commit to blessed
repository.
In this way you can make anything available, without endangering main branch :)

And don't worry, we have a client for you who can test new
functionality deeply - an Ocean project which implements networking
stack from scratch and it badly needs threaded FFI.


Don't forget the new team for DBXTalk ;)

Also, if you give me a hint, I can migrate Mars to reentrant FFI (yes, again... and I honestly hope this is the last time I migrate it, I'm starting to hate the process :P )



 
> best regards (albeit feeling a little frazzled :-) ),
> Eliot



--
Best regards,
Igor Stasenko AKA sig.


Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax

Andreas.Raab
In reply to this post by Levente Uzonyi-2
 
On 3/24/2011 19:35, Levente Uzonyi wrote:
> Isn't it the best time to migrate the syntax of FFI calls to Pragmas?
> I know it takes some time to implement the pragma support, but IIRC
> Lukas did that a few years ago, so dusting it off and adding support
> for threaded calls (which I didn't see yet) shouldn't be that hard.

Sorry, but that's a decidedly bad idea. It breaks each and every use of
the FFI out there. The FFI syntax and support is what it is, so please
leave it alone. If you want to implement something different, then
implement something different and do it in a form that doesn't conflict
with the FFI so that both can live side by side.

Cheers,
   - Andreas

> Both formats could be supported for a while, then we could leave the
> current format behind and simplify the parsers/compilers, etc.
>
>
> Levente
Reply | Threaded
Open this post in threaded view
|

Re: FFI syntax

Levente Uzonyi-2
 
On Fri, 25 Mar 2011, Andreas Raab wrote:

>
> On 3/24/2011 19:35, Levente Uzonyi wrote:
>> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I
>> know it takes some time to implement the pragma support, but IIRC Lukas did
>> that a few years ago, so dusting it off and adding support for threaded
>> calls (which I didn't see yet) shouldn't be that hard.
>
> Sorry, but that's a decidedly bad idea. It breaks each and every use of the
> FFI out there. The FFI syntax and support is what it is, so please leave it
> alone. If you want to implement something different, then implement something
> different and do it in a form that doesn't conflict with the FFI so that both
> can live side by side.

That's the idea:
"Both formats could be supported for a while, then we could leave the
current format behind and simplify the parsers/compilers, etc."

Even tools can be built to convert from one syntax to the other.


Levente

>
> Cheers,
>  - Andreas
>
>> Both formats could be supported for a while, then we could leave the
>> current format behind and simplify the parsers/compilers, etc.
>>
>>
>> Levente
>
12