Smalltalk › Pharo › Pharo Smalltalk Users

uFFI ExternalAddress challenges

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

20 messages Options

eftomi

uFFI ExternalAddress challenges

Dear all,

I'm struggling with passing an ExternalAddress to a c function call with uFFI. Firstly, I get the ExternalAddress from this method:

myFFI>>create: aString
^ self
ffiCall: #(void * CreateObject (String aString) )

[in c:

void* CreateObject(char* szProgId)]

The method that uses the resulting ExternalAddress is defined as:

myFFI>>ask: anExternalAddress
^ self
ffiCall: #(void CallMethod (void * anExternalAddress) )

[in c:

void CallMethod(void* myObj)]

In playground I have

| w |

w := myFFI create: 'Word.Application'.

myFFI ask: w .

w gets a "nice", properly looking external address, however the last line crushes Pharo, its window gets closed.

I went through the uFFI book, however I cannot find the answer.

I'm on Windows 10 x64, Pharo 7.0.4 32-bit.

Best wishes,

Tomaz

Guillermo Polito

Re: uFFI ExternalAddress challenges

Hi Thomas,

Can you share more about the definition of your CallMethod function in C?

From the uFFI point of view the binding looks ok…

How are you opening Pharo? From the command line or the pharo launcher?

Are you on cygwin/mingw? One other possibility (since you’re playing with C libraries) is to launch Pharo inside a gdb/lldb so you can check the reason behind the error when it crashes.

Keep us posted,

Guille

El 10 sept 2019, a las 19:53, Tomaž Turk <[hidden email]> escribió:

Dear all,

I'm struggling with passing an ExternalAddress to a c function call with uFFI. Firstly, I get the ExternalAddress from this method:

myFFI>>create: aString
^ self
ffiCall: #(void * CreateObject (String aString) )

[in c:
void* CreateObject(char* szProgId)]

The method that uses the resulting ExternalAddress is defined as:

myFFI>>ask: anExternalAddress
^ self
ffiCall: #(void CallMethod (void * anExternalAddress) )

[in c:
void CallMethod(void* myObj)]

In playground I have

| w |
w := myFFI create: 'Word.Application'.
myFFI ask: w .

w gets a "nice", properly looking external address, however the last line crushes Pharo, its window gets closed.

I went through the uFFI book, however I cannot find the answer.

I'm on Windows 10 x64, Pharo 7.0.4 32-bit.

Best wishes,
Tomaz

eftomi

Re: uFFI ExternalAddress challenges

Hi, thanks for your quick reply. I found the culprint while writing this email :-)

Generally, what I'm trying to do is to make a very basic package to create and communicate with COM components on Windows, mostly as an exercise :-) I found this library: http://disphelper.sourceforge.net/ as a great start, so you don't have to fiddle with OLE/COM headaches.

The problematic function in C is defined as:

__declspec(dllexport) void CallMethod(void* myObj) {

...

}

It takes a pointer to a COM object, which was previously returned by

__declspec(dllexport) void* CreateObject(char* szProgId) {

...

IDispatch * myObj = NULL

...

return &myObj;

}

So, CreateObject() creates COM object and returns a pointer to it, CallMethod() takes this pointer and tries to communicate with it.

My error was in returning a pointer to a pointer ... since IDispatch * myObj = NULL was actually hidden in a macro and I missed its true definition, huh.

I'm running Pharo from PharoLauncher directly on Win10, without cygwin/mingw. For DLL I use Visual Studio 2019. I was not sure whether the pointer survives from one FFI call to another, or how Pharo loads/unloads DLLs - but it works as expected.

Thanks again and best wishes,

Tomaz

------ Original Message ------

From: "Guillermo Polito" <[hidden email]>

To: "Tomaž Turk" <[hidden email]>; "Any question about pharo is welcome" <[hidden email]>

Sent: 11. 09. 2019 10:29:34

Subject: Re: [Pharo-users] uFFI ExternalAddress challenges

Hi Thomas,

Can you share more about the definition of your CallMethod function in C?
From the uFFI point of view the binding looks ok…

How are you opening Pharo? From the command line or the pharo launcher?
Are you on cygwin/mingw? One other possibility (since you’re playing with C libraries) is to launch Pharo inside a gdb/lldb so you can check the reason behind the error when it crashes.

Keep us posted,
Guille

Guillermo Polito

Re: uFFI ExternalAddress challenges

El 11 sept 2019, a las 11:07, Tomaž Turk <[hidden email]> escribió:

Hi, thanks for your quick reply.

no problem ;)

I found the culprint while writing this email :-)

Cool!

Generally, what I'm trying to do is to make a very basic package to create and communicate with COM components on Windows, mostly as an exercise :-) I found this library: http://disphelper.sourceforge.net/ as a great start, so you don't have to fiddle with OLE/COM headaches.

The problematic function in C is defined as:

__declspec(dllexport) void CallMethod(void* myObj) {
...
}

It takes a pointer to a COM object, which was previously returned by

__declspec(dllexport) void* CreateObject(char* szProgId) {
...
IDispatch * myObj = NULL
...
return &myObj;
}

So, CreateObject() creates COM object and returns a pointer to it, CallMethod() takes this pointer and tries to communicate with it.

My error was in returning a pointer to a pointer ... since IDispatch * myObj = NULL was actually hidden in a macro and I missed its true definition, huh.

Yes, looking at the code above it seems strange returning the &myObj, being myObj a local variable.

Because that means that you would be returning a pointer to the stack, from a frame that already returned...

I was not sure whether the pointer survives from one FFI call to another

Well, the pointer survives, the area of memory pointed by that pointer will be recycled with subsequent calls in general.

, or how Pharo loads/unloads DLLs - but it works as expected.

In general Pharo will load the library on usage, so you don’t have to care about it.

I’m sure the library is not unloaded, unless it is done explicitly (you can check VirtualMachine >> unloadModule:)

Thanks again and best wishes,
Tomaz

Cheers!

eftomi

Re: uFFI ExternalAddress challenges

> Well, the pointer survives, the area of memory pointed by that pointer will be recycled with subsequent calls in general.

I'll pay special attention to that, but it's probably a matter of properly creating and releasing COM objects.

I have another question - in FFI call, how would one send a wide string, as in

void CallMethod(void * COMObj, wchar_t * MethodName)

I might have additional questions about UTF-8 and other code page nuances ...

Best wishes,

Tomaz

Guillermo Polito

Re: uFFI ExternalAddress challenges

El 11 sept 2019, a las 11:43, Tomaž Turk <[hidden email]> escribió:

> Well, the pointer survives, the area of memory pointed by that pointer will be recycled with subsequent calls in general.

I'll pay special attention to that, but it's probably a matter of properly creating and releasing COM objects.

I have another question - in FFI call, how would one send a wide string, as in

void CallMethod(void * COMObj, wchar_t * MethodName)

I might have additional questions about UTF-8 and other code page nuances …

:D Windows is particularly different than other platforms.

In *nixes most APIs require a char* with the encoded bytes, which we can encode in Pharo doing something like

‘my string’ utf8Encoded => a byte array with the encoded string

But for windows, I think the best is to use their encoding APIs.

You’ll find the image already has some support for that, that we use to access environment variables and the working directory string.

Check the class Win32WideString and its class comment ;)

Best wishes,
Tomaz

eftomi

Re: uFFI ExternalAddress challenges

Excellent, thanks!

eftomi

Re: uFFI ExternalAddress challenges

In reply to this post by eftomi

> We are interested in it :)

> If I recall well Pablo got one version somewhere.

Me too :-) Unfortunately, I only have time for 'weekend' projects, and any help is appreciated :-)

Regarding the uFFI calls, is it possible to pass Win32WideString in a similar fashion as a String - or where to look to implement that?

Best wishes,

Tomaz

Torsten Bergmann

Re: uFFI ExternalAddress challenges

From your snippets it looks like you want to do COM Automation on Windows.

Maybe you should have a look at: https://github.com/tesonep/pharo-com

Bye

Gesendet: Donnerstag, 12. September 2019 um 11:08 Uhr
Von: "Tomaž Turk" <[hidden email]>
An: "Any question about pharo is welcome" <[hidden email]>
Betreff: Re: [Pharo-users] uFFI ExternalAddress challenges

> We are interested in it :)

> If I recall well Pablo got one version somewhere.

Me too :-) Unfortunately, I only have time for 'weekend' projects, and any help is appreciated :-)

Regarding the uFFI calls, is it possible to pass Win32WideString in a similar fashion as a String - or where to look to implement that?

Best wishes,

Tomaz

Ben Coman

Re: uFFI ExternalAddress challenges

On Thu, 12 Sep 2019 at 18:08, Torsten Bergmann <[hidden email]> wrote:

From your snippets it looks like you want to do COM Automation on Windows.

Maybe you should have a look at: https://github.com/tesonep/pharo-com

Does that by chance include an implementation of "MS-CFB" Microsoft Compound File Binary

https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-cfb/50708a61-81d9-49c8-ab9c-43c98a795242

I was wanting to write a utility to parse headers of MSG files saved from Outlook,

to process a for a project email archive by bulk renaming each MSG file with its transmit-date & subject.

cheers -ben

eftomi

Re: uFFI ExternalAddress challenges

In reply to this post by Torsten Bergmann

> Maybe you should have a look at: https://github.com/tesonep/pharo-com

Thanks for the link, I'll check it out - that's probably the work Steph has mentioned.

Best wishes,

Tomaz

eftomi

Re: uFFI ExternalAddress challenges

In reply to this post by Torsten Bergmann

OK, I'm proceeding with the library that I mentioned (http://disphelper.sourceforge.net/) and I got nice results. Pablo's package goes directly into the core of OLE/COM automation and it would be too hard to for me to study it and continue with it at this stage.

An example below loads Word for Windows, makes it visible, creates a new document and writes some text:

| w |

COMEngineAlpha initializeCOM .

w := COMEngineAlpha createObjectByName: 'Word.Application'.

COMEngineAlpha callObject: w setProperty: '.Visible = %b' withInteger: 1.

COMEngineAlpha callObject: w method: '.Documents.Add'.

COMEngineAlpha callObject: w method: '.Selection.TypeText(%S)' withString: 'This is a message to appear in Word document.'.

COMEngineAlpha safeReleaseObject: w.

COMEngineAlpha uninitializeCOM.

However, I'd like to marshall a WideString through uFFI so that I won't loose UTF-8 support, of course the plain (String aString) signature doesn't work ("Cannot coerce arguments"). Is there anything I can do myself (i.e. without poking into Pharo virtual machine complexity) to solve this challenge?

Best wishes,

Tomaz

------ Original Message ------

From: "Torsten Bergmann" <[hidden email]>

To: [hidden email]; [hidden email]

Cc: "Any question about pharo is welcome" <[hidden email]>

Sent: 12. 09. 2019 12:07:34

Subject: Aw: Re: [Pharo-users] uFFI ExternalAddress challenges

From your snippets it looks like you want to do COM Automation on Windows.

Maybe you should have a look at: https://github.com/tesonep/pharo-com

Bye

T.

Gesendet: Donnerstag, 12. September 2019 um 11:08 Uhr
Von: "Tomaž Turk" <[hidden email]>
An: "Any question about pharo is welcome" <[hidden email]>
Betreff: Re: [Pharo-users] uFFI ExternalAddress challenges

> We are interested in it

> If I recall well Pablo got one version somewhere.

Me too Unfortunately, I only have time for 'weekend' projects, and any help is appreciated

Regarding the uFFI calls, is it possible to pass Win32WideString in a similar fashion as a String - or where to look to implement that?

Best wishes,

Tomaz

eftomi

Re: uFFI ExternalAddress challenges

A rudimentary disphelper library "approach" is working OK, I succeeded to
create a connection to SQL Server through ADODB, too, which is rather cool,
since I've been looking for this functionality for too long.

I re-checked Pablo's pharo-com package and it is rather comprehensive - it
successfully creates COM servers and more. However, when trying to invoke
writing to COM server properties, as in:

wordObj propertyNamed: 'Visible' put: true.

the call fails with:

Primitive failed: primitive #integerAt:size:signed: in ExternalAddress

Problematic method uses a primitive

<primitive: 'primitiveFFIIntegerAt' module:'SqueakFFIPrims'>

which probably doesn't exist. Does anybody know about the purpose/intent and
the history of this primitive in 'SqueakFFIPrims'?

I think it would be a pitty that we leave Pablo's work unfinished.

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

eftomi

Re: uFFI ExternalAddress challenges

It seems that there's nothing wrong with the primitive, but with Win32Variant
creation in ExternalStructure class>>fromHandle:, specifically with basicNew
and Win32Variant class as receiver.

In what circumstances the debugger/inspector shows 'error printing' message
(on the result of basicNew)?

Sorry for newbie questions :-)

Best wishes,
Tomaz

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Guillermo Polito

Re: uFFI ExternalAddress challenges

In reply to this post by eftomi

Hi Tomaz,

Have you checked Win32WideString in Pharo7/8 and its users?

You can do

aWindowsWideString := ‘a wide string’ asWin32WideString.

And then check the senders of Win32WideString, you will find you can declare ffi signatures like this

removeEnvironmentVariable: nameString

^ self ffiCall: #( int SetEnvironmentVariableW ( Win32WideString nameString, 0 ) )

Isn’t this working for you?

If not, I’d like to have more details on why it does not work, so we can figure out a solution :)

Cheers,

Guille

El 13 sept 2019, a las 15:22, Tomaž Turk <[hidden email]> escribió:

OK, I'm proceeding with the library that I mentioned (http://disphelper.sourceforge.net/) and I got nice results. Pablo's package goes directly into the core of OLE/COM automation and it would be too hard to for me to study it and continue with it at this stage.

An example below loads Word for Windows, makes it visible, creates a new document and writes some text:

| w |
COMEngineAlpha initializeCOM .
w := COMEngineAlpha createObjectByName: 'Word.Application'.
COMEngineAlpha callObject: w setProperty: '.Visible = %b' withInteger: 1.
COMEngineAlpha callObject: w method: '.Documents.Add'.
COMEngineAlpha callObject: w method: '.Selection.TypeText(%S)' withString: 'This is a message to appear in Word document.'.
COMEngineAlpha safeReleaseObject: w.
COMEngineAlpha uninitializeCOM.

However, I'd like to marshall a WideString through uFFI so that I won't loose UTF-8 support, of course the plain (String aString) signature doesn't work ("Cannot coerce arguments"). Is there anything I can do myself (i.e. without poking into Pharo virtual machine complexity) to solve this challenge?

Best wishes,
Tomaz

------ Original Message ------
From: "Torsten Bergmann" <[hidden email]>
To: [hidden email]; [hidden email]
Cc: "Any question about pharo is welcome" <[hidden email]>
Sent: 12. 09. 2019 12:07:34
Subject: Aw: Re: [Pharo-users] uFFI ExternalAddress challenges

From your snippets it looks like you want to do COM Automation on Windows.

Maybe you should have a look at: https://github.com/tesonep/pharo-com

Bye
T.

Gesendet: Donnerstag, 12. September 2019 um 11:08 Uhr
Von: "Tomaž Turk" <[hidden email]>
An: "Any question about pharo is welcome" <[hidden email]>
Betreff: Re: [Pharo-users] uFFI ExternalAddress challenges
> We are interested in it <emoticon_smiley.png>
> If I recall well Pablo got one version somewhere.

Me too <emoticon_smiley.png> Unfortunately, I only have time for 'weekend' projects, and any help is appreciated <emoticon_smiley.png>

Regarding the uFFI calls, is it possible to pass Win32WideString in a similar fashion as a String - or where to look to implement that?

Best wishes,
Tomaz

eftomi

Re: uFFI ExternalAddress challenges

OK, great, I'll try it out.

Thanks,

Tomaz

------ Original Message ------

From: "Guillermo Polito" <[hidden email]>

To: "Tomaž Turk" <[hidden email]>

Sent: 17.9.2019 11:56:21

Subject: Re: [Pharo-users] uFFI ExternalAddress challenges

I don’t know if there is support for that (I don’t have the time to check it in detail now).
But, as a workaround, you may try

- return a void*

myFunction
^self ffiCall: #(void* myFunction ( void ) )

- and then do the transformation yourself using Win32WideString>>#fromHandle:

Stephan Eggermont-3

Re: uFFI ExternalAddress challenges

Tomaž Turk <[hidden email]> wrote:
> OK, great, I'll try it out.

Hi Tomaž,

What you are discovering here would be very useful as a section/chapter in
the uFFI booklet. Just posting it as text here will do if you don’t feel up
to translating to Pillar.

Cheers
Stephan

eftomi

Re: uFFI ExternalAddress challenges

Hi,

> I don’t know if there is support for that (I don’t have the time to check it in detail now).

> But, as a workaround, you may try

> - return a void*

> myFunction

> ^self ffiCall: #(void* myFunction ( void ) )

> - and then do the transformation yourself using Win32WideString>>#fromHandle:

I checked, Win32WideString class>>#asExternalTypeOn: is defined, and it creates FFIExternalObjectType(Win32WideString) nicely with

^self ffiCall: #(Win32WideString myFunction ( void ) )

I also tried the other approach:

^self ffiCall: #(void* myFunction ( void ) )

and the transformation Win32WideString>>#fromHandle

Interestingly, in both cases I get

however I cannot pinpoint the reason for this.

> What you are discovering here would be very useful as a section/chapter in

> the uFFI booklet.

Thanks for the invitation, yes, it would be nice to participate, I'll try to prepare something when I get to the bottom of the challenge :-)

Thanks and best wishes,

Tomaz

eftomi

Re: uFFI ExternalAddress challenges

Hi, after some more investigation the easiest way to get Win32WideString from
external module is to pass it as a parameter, like in:

^self ffiCall: #( void myFunction ( Win32WideString aWin32WideString ) )

and the external module can access it directly to do r/w.

There are other possibilities as well - like adapting the method
Win32WideString>>#fromHandle: to translate from ExternalData to
Win32WideString, creating new ExternalType, or even creating a new
FFIWin32WideString. I wonder what would be the best approach :-)

Best wishes,
Tomaz

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

eftomi

Re: uFFI ExternalAddress challenges

Hi,

as a wrap-up for this forum thread, I found an elegant way to take a result
as C wchar_t* from an FFI call on Windows, based on the idea of
ExternalData>>#fromCString. If anybody needs it:
https://github.com/eftomi/pharo-uFFI-readWin32WideString.

Best wishes,
Tomaz

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html