NativeBoost

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

NativeBoost

Markus Fritsche-4
Hello,

tl;dr: what's the reason to get aNBExternalHandle even if you expect a
MySuperHandle object from an nbCall?

I am trying to build my first DLL wrapper with NativeBoost. I am having
a problem that instead of returning a handle object (as I thought), my
"primitive" throws NBExternalHandles at me.

The API I am trying to wrap tries to be somewhat object oriented by
(mostly) having a handle of some sort.

My thinking was to attach all DLL primitives to the respective Handle
Objects they take as a first parameter and go from there.

Some workspace code:

"Supplementary DLLs"
Tm1ULibDllLibraryLoader loadLibrary.
LibIbmCogEayLibraryLoader loadLibrary.
SslIbmCogEayLibraryLoader loadLibrary.
Log4CxxLibraryLoader loadLibrary.
"DLL to wrap"
TM1LibraryLoader loadTM1Library.

t := TM1Api new.
t tm1APIInitialize.
hUser := t tm1SystemOpen. "Returns a proper 'TM1UHandle' object'"
hUser tm1SystemAdminHostSet: 'localhost'.
hPool1 := hUser tm1ValPoolCreate. "Returns NBExternalHandle"
hPool1 inspect.
hUser tm1SystemClose.
t tm1APIFinalize.

TM1Api>>#tm1SystemOpen
     <primitive: #primitiveNativeCall
      module: #NativeBoostPlugin>
     ^ self nbCall: #(TM1UHandle TM1SystemOpen(void))

TM1UHandle>>#tm1ValPoolCreate
     <primitive: #primitiveNativeCall
      module: #NativeBoostPlugin>
     ^ self nbCall: #(#TM1PHandle TM1ValPoolCreate ( TM1UHandle self ) )

Both, TM1UHandle class and TM1PHandle class have the method
asNBExternalType: gen
     "use handle ivar to hold my instance (TM1x)"
     ^ NBExternalObjectType objectClass: self

and are subclasses of object. What's my mistake? Can you tell with the
information I provided?

Best regards,
   Markus

Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Igor Stasenko



On 26 April 2014 09:44, Markus Fritsche <[hidden email]> wrote:
Hello,

tl;dr: what's the reason to get aNBExternalHandle even if you expect a MySuperHandle object from an nbCall?

I am trying to build my first DLL wrapper with NativeBoost. I am having a problem that instead of returning a handle object (as I thought), my "primitive" throws NBExternalHandles at me.

The API I am trying to wrap tries to be somewhat object oriented by (mostly) having a handle of some sort.

My thinking was to attach all DLL primitives to the respective Handle Objects they take as a first parameter and go from there.

Some workspace code:

"Supplementary DLLs"
Tm1ULibDllLibraryLoader loadLibrary.
LibIbmCogEayLibraryLoader loadLibrary.
SslIbmCogEayLibraryLoader loadLibrary.
Log4CxxLibraryLoader loadLibrary.
"DLL to wrap"
TM1LibraryLoader loadTM1Library.

t := TM1Api new.
t tm1APIInitialize.
hUser := t tm1SystemOpen. "Returns a proper 'TM1UHandle' object'"
hUser tm1SystemAdminHostSet: 'localhost'.
hPool1 := hUser tm1ValPoolCreate. "Returns NBExternalHandle"
hPool1 inspect.
hUser tm1SystemClose.
t tm1APIFinalize.

TM1Api>>#tm1SystemOpen
    <primitive: #primitiveNativeCall
     module: #NativeBoostPlugin>
    ^ self nbCall: #(TM1UHandle TM1SystemOpen(void))

TM1UHandle>>#tm1ValPoolCreate
    <primitive: #primitiveNativeCall
     module: #NativeBoostPlugin>
    ^ self nbCall: #(#TM1PHandle TM1ValPoolCreate ( TM1UHandle self ) )

sidenote: when using 'self', you can use it without type. just say 'self', since it refers to
the instance of method's class, which is enough to determine the type to use for marshalling.
 
Both, TM1UHandle class and TM1PHandle class have the method
asNBExternalType: gen
    "use handle ivar to hold my instance (TM1x)"
    ^ NBExternalObjectType objectClass: self

and are subclasses of object. What's my mistake? Can you tell with the information I provided?


This is strange.. should work fine.
There's no way how NBExternalObjectType could return an instance of NBExternalHandle.
Please check your code again.

Best regards,
  Markus




--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Markus Fritsche-4

On 28.04.2014 13:30, Igor Stasenko wrote:
> This is strange.. should work fine. There's no way how
> NBExternalObjectType could return an instance of NBExternalHandle.
> Please check your code again.
Been there, done that => thrown everything away and restarted the code
from scratch... Sourcecode attached (the 32bit libraries are in the bin
subdirectory, there's another bin64 directory) - is there anything
obvious wrong with it?.

Workspace:
====================
TM1LibraryLoader loadTM1Library.

TM1Library tm1APIInitialize.
hUser := TM1Library tm1SystemOpen.
hPool1 := hUser tm1ValPoolCreate.
hPool1 inspect.
hUser tm1SystemClose.
TM1Library tm1APIFinalize.
====================

The Inspector I get shows
NBExternalHandle
self    @ 16r74CF4A8
(apparently it's fine for TM1U, otherwise tm1ValPoolCreate wouldn't be
found)

From the headers:

#define TM1API __stdcall
#ifndef TM1IMPORT
  #define TM1IMPORT __declspec( dllimport )
#endif

typedef void         * TM1U;           // user  handle
typedef void         * TM1P;           // pool  handle  
TM1IMPORT void TM1API TM1APIInitialize( void );
TM1IMPORT TM1U TM1API TM1SystemOpen( void );
TM1IMPORT TM1P TM1API TM1ValPoolCreate( TM1U hUser );


In order to debug further I will have to redo the Windows VM.

Cognos-TM1-Api.st (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Igor Stasenko

Object subclass: #TM1U
    uses: TTM1ApiLibrary
    instanceVariableNames: 'handle'
    classVariableNames: ''
    poolDictionaries: ''
    category: 'Cognos-TM1-Api'

hmm.. why just don't subclass from NBExternalObject?
If you don't need to subclass from own base class, you can just subclass from
NBExternalObject, which means you already having handle ivar, and #asNBExternalType: inherited at class side, tested and working..?


So, try

NBFFICallout new resolveType: #TM1U

what it gives?

On 28 April 2014 23:44, Markus Fritsche <[hidden email]> wrote:

On 28.04.2014 13:30, Igor Stasenko wrote:
> This is strange.. should work fine. There's no way how
> NBExternalObjectType could return an instance of NBExternalHandle.
> Please check your code again.
Been there, done that => thrown everything away and restarted the code
from scratch... Sourcecode attached (the 32bit libraries are in the bin
subdirectory, there's another bin64 directory) - is there anything
obvious wrong with it?.

Workspace:
====================
TM1LibraryLoader loadTM1Library.

TM1Library tm1APIInitialize.
hUser := TM1Library tm1SystemOpen.
hPool1 := hUser tm1ValPoolCreate.
hPool1 inspect.
hUser tm1SystemClose.
TM1Library tm1APIFinalize.
====================

The Inspector I get shows
NBExternalHandle
self    @ 16r74CF4A8
(apparently it's fine for TM1U, otherwise tm1ValPoolCreate wouldn't be
found)

could it be inspector screwed?
what
hPool1 class
gives?
 
From the headers:

#define TM1API __stdcall
#ifndef TM1IMPORT
  #define TM1IMPORT __declspec( dllimport )
#endif

typedef void         * TM1U;           // user  handle
typedef void         * TM1P;           // pool  handle
TM1IMPORT void TM1API TM1APIInitialize( void );
TM1IMPORT TM1U TM1API TM1SystemOpen( void );
TM1IMPORT TM1P TM1API TM1ValPoolCreate( TM1U hUser );


 i don't see where you are using stdcall calling convention option?
 you should specify it either in code:

self nbCallout
   stdCall
   function: #( TM1U  TM1SystemOpen( ) ) module:

or in the class with callout, implementing

ffiCalloutOptions
^ #( + optStdcall )

(i think you can just put it into trait)


In order to debug further I will have to redo the Windows VM.



--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Markus Fritsche-4
Hello Igor,

I did change my class (tbh, I was just looking at the Cairo stuff and
did some cargo cult programming).

NBFFICallout new resolveType: #TM1U prints TM1U

NBFFICallout new resolveType: #TM1P prints TM1P

- my DoIt still gives NBExternalType

I attached the resulting CompiledMethod as a fuel serialized file.

What can I look at more deeply?


Thank you for your patience,
  Markus

On 29.04.2014 00:36, Igor Stasenko wrote:

> Object subclass: #TM1U
>     uses: TTM1ApiLibrary
>     instanceVariableNames: 'handle'
>     classVariableNames: ''
>     poolDictionaries: ''
>     category: 'Cognos-TM1-Api'
>
> hmm.. why just don't subclass from NBExternalObject?
> If you don't need to subclass from own base class, you can just subclass
> from
> NBExternalObject, which means you already having handle ivar, and
> #asNBExternalType: inherited at class side, tested and working..?
>
>
> So, try
>
> NBFFICallout new resolveType: #TM1U
>
> what it gives?
>
> On 28 April 2014 23:44, Markus Fritsche <[hidden email]> wrote:
>
>>
>> On 28.04.2014 13:30, Igor Stasenko wrote:
>>> This is strange.. should work fine. There's no way how
>>> NBExternalObjectType could return an instance of NBExternalHandle.
>>> Please check your code again.
>> Been there, done that => thrown everything away and restarted the code
>> from scratch... Sourcecode attached (the 32bit libraries are in the bin
>> subdirectory, there's another bin64 directory) - is there anything
>> obvious wrong with it?.
>>
>> Workspace:
>> ====================
>> TM1LibraryLoader loadTM1Library.
>>
>> TM1Library tm1APIInitialize.
>> hUser := TM1Library tm1SystemOpen.
>> hPool1 := hUser tm1ValPoolCreate.
>> hPool1 inspect.
>> hUser tm1SystemClose.
>> TM1Library tm1APIFinalize.
>> ====================
>>
>> The Inspector I get shows
>> NBExternalHandle
>> self    @ 16r74CF4A8
>> (apparently it's fine for TM1U, otherwise tm1ValPoolCreate wouldn't be
>> found)
>>
>> could it be inspector screwed?
> what
> hPool1 class
> gives?
>
>
>> From the headers:
>>
>> #define TM1API __stdcall
>> #ifndef TM1IMPORT
>>   #define TM1IMPORT __declspec( dllimport )
>> #endif
>>
>> typedef void         * TM1U;           // user  handle
>> typedef void         * TM1P;           // pool  handle
>> TM1IMPORT void TM1API TM1APIInitialize( void );
>> TM1IMPORT TM1U TM1API TM1SystemOpen( void );
>> TM1IMPORT TM1P TM1API TM1ValPoolCreate( TM1U hUser );
>>
>>
>  i don't see where you are using stdcall calling convention option?
>  you should specify it either in code:
>
> self nbCallout
>    stdCall
>    function: #( TM1U  TM1SystemOpen( ) ) module:
>
> or in the class with callout, implementing
>
> ffiCalloutOptions
> ^ #( + optStdcall )
>
> (i think you can just put it into trait)
>
>
>> In order to debug further I will have to redo the Windows VM.
>>
>
>
>

CompiledMethod.fuel (110 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Igor Stasenko

On 29 April 2014 20:54, Markus Fritsche <[hidden email]> wrote:
Hello Igor,

I did change my class (tbh, I was just looking at the Cairo stuff and
did some cargo cult programming).

NBFFICallout new resolveType: #TM1U prints TM1U

NBFFICallout new resolveType: #TM1P prints TM1P

- my DoIt still gives NBExternalType

I attached the resulting CompiledMethod as a fuel serialized file.

What can I look at more deeply?

i don't know what is going on..
everything looks correct..

as a workaround, you can change the return type of offending call to return uint,
and then you can manually convert it to instance of TM1P with corresponding
handle value.

Also, i doubt it, but try change the alignment
NativeBoostWin32>>stackAlignment
 to use 16


P.S. the fuel file with compiled method is not really helpful - it gives me an error during materialization.


Thank you for your patience,
  Markus

On 29.04.2014 00:36, Igor Stasenko wrote:
> Object subclass: #TM1U
>     uses: TTM1ApiLibrary
>     instanceVariableNames: 'handle'
>     classVariableNames: ''
>     poolDictionaries: ''
>     category: 'Cognos-TM1-Api'
>
> hmm.. why just don't subclass from NBExternalObject?
> If you don't need to subclass from own base class, you can just subclass
> from
> NBExternalObject, which means you already having handle ivar, and
> #asNBExternalType: inherited at class side, tested and working..?
>
>
> So, try
>
> NBFFICallout new resolveType: #TM1U
>
> what it gives?
>
> On 28 April 2014 23:44, Markus Fritsche <[hidden email]> wrote:
>
>>
>> On 28.04.2014 13:30, Igor Stasenko wrote:
>>> This is strange.. should work fine. There's no way how
>>> NBExternalObjectType could return an instance of NBExternalHandle.
>>> Please check your code again.
>> Been there, done that => thrown everything away and restarted the code
>> from scratch... Sourcecode attached (the 32bit libraries are in the bin
>> subdirectory, there's another bin64 directory) - is there anything
>> obvious wrong with it?.
>>
>> Workspace:
>> ====================
>> TM1LibraryLoader loadTM1Library.
>>
>> TM1Library tm1APIInitialize.
>> hUser := TM1Library tm1SystemOpen.
>> hPool1 := hUser tm1ValPoolCreate.
>> hPool1 inspect.
>> hUser tm1SystemClose.
>> TM1Library tm1APIFinalize.
>> ====================
>>
>> The Inspector I get shows
>> NBExternalHandle
>> self    @ 16r74CF4A8
>> (apparently it's fine for TM1U, otherwise tm1ValPoolCreate wouldn't be
>> found)
>>
>> could it be inspector screwed?
> what
> hPool1 class
> gives?
>
>
>> From the headers:
>>
>> #define TM1API __stdcall
>> #ifndef TM1IMPORT
>>   #define TM1IMPORT __declspec( dllimport )
>> #endif
>>
>> typedef void         * TM1U;           // user  handle
>> typedef void         * TM1P;           // pool  handle
>> TM1IMPORT void TM1API TM1APIInitialize( void );
>> TM1IMPORT TM1U TM1API TM1SystemOpen( void );
>> TM1IMPORT TM1P TM1API TM1ValPoolCreate( TM1U hUser );
>>
>>
>  i don't see where you are using stdcall calling convention option?
>  you should specify it either in code:
>
> self nbCallout
>    stdCall
>    function: #( TM1U  TM1SystemOpen( ) ) module:
>
> or in the class with callout, implementing
>
> ffiCalloutOptions
> ^ #( + optStdcall )
>
> (i think you can just put it into trait)
>
>
>> In order to debug further I will have to redo the Windows VM.
>>
>
>
>



--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Markus Fritsche-4
Hello Igor,

> Also, i doubt it, but try change the alignment
> NativeBoostWin32>>stackAlignment
>  to use 16

That did the trick - it was ^ 1 - changed it to ^ 16 and now the DoIt
opens an inspector on a TM1P object.

Should this be reported on fogbugz?

Thank you
  Markus

Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Igor Stasenko


On 29 April 2014 22:51, Markus Fritsche <[hidden email]> wrote:
Hello Igor,

> Also, i doubt it, but try change the alignment
> NativeBoostWin32>>stackAlignment
>  to use 16

That did the trick - it was ^ 1 - changed it to ^ 16 and now the DoIt
opens an inspector on a TM1P object.

 
This is weird..
 
Can you try one thing:

change:

NBFFICallout>>foreignCall: aBlock

    callInfo := self newCallInfo.
    callInfo alignment: 16.
    asm performingCall: callInfo in: aBlock.


while use:
NativeBoostWin32>>stackAlignment
  ^ 1

and then swap values. And tell which case works and which is not (they both may not work though). Don't forget to restart image before trying to reset all generated code.
 
(you can also use #optNoAlignment option while calling external function e.g.

self nbcall: blah options: #(+optNoAlignment).

I wanna know, who corrupts stack: external function or VM itself.

Should this be reported on fogbugz?

Thank you
  Markus




--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Markus Fritsche-4
On 2014-04-30 04:58, Igor Stasenko wrote:
> NBFFICallout>>foreignCall: aBlock

> callInfo := self newCallInfo.
> callInfo alignment: 16.
> asm performingCall: callInfo in: aBlock.

> while use:
> NativeBoostWin32>>stackAlignment
> ^ 1

This seems to work - however, I did not to subsequent calls.

> and then swap values.

That crashes instantaneously.

Best regards,
   Markus


Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Igor Stasenko



On 30 April 2014 11:09, Markus Fritsche <[hidden email]> wrote:
On 2014-04-30 04:58, Igor Stasenko wrote:
NBFFICallout>>foreignCall: aBlock

callInfo := self newCallInfo.
callInfo alignment: 16.
asm performingCall: callInfo in: aBlock.

while use:
NativeBoostWin32>>stackAlignment
^ 1

This seems to work - however, I did not to subsequent calls.

did not what?
 
and then swap values.

That crashes instantaneously.


So, it seems that it is DLL function wants stack alignment.
 
Best regards,
  Markus




--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Markus Fritsche-4
On 2014-04-30 11:35, Igor Stasenko wrote:

> So, it seems that it is DLL function wants stack alignment.

So does that mean (doing it the right way) that I will have to change my
stackAlignment method or do I have to use different nbCall-sonventions?

Best regards,
   Markus

Reply | Threaded
Open this post in threaded view
|

Re: NativeBoost

Igor Stasenko



On 30 April 2014 11:46, Markus Fritsche <[hidden email]> wrote:
On 2014-04-30 11:35, Igor Stasenko wrote:

So, it seems that it is DLL function wants stack alignment.

So does that mean (doing it the right way) that I will have to change my stackAlignment method or do I have to use different nbCall-sonventions?


It is safer to change it for whole platform, e.g in:
NativeBoostWin32>>stackAlignment
 
Best regards,
  Markus




--
Best regards,
Igor Stasenko.