Smalltalk › Squeak › Squeak VM

FFIExternalStructureReferenceHandle and pushing it to the stack

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

5 messages Options

Holger Freyther

FFIExternalStructureReferenceHandle and pushing it to the stack

In addition to my mail from yesterday to vm-dev I want to share the following example to illustrate the problem. I have taken StructA and StructNested from the FFIExternalStructureReferenceHandle class comment and put it into Unclassified.st that is attached to this mail. I am also making a call to LibC's getpid and try to push StructNested to the stack. The ffiCall is failing before getpid is called (Bad argument to external function).

The reproducer is:

StructA new theNest anyCall.

StructA has a ByteArray handle of the full struct. theNest returns an instance of StructNested with a FFIExternalStructureReferenceHandle. This makes sense as well. It can't be a copy of the ByteArray (or else updates would be invisible to the outer struct) and it can't be an ExternalAddress as StructA's handle might move in the Smalltalk heap.

I am not entirely sure how this is should work but what do you think of:

1st) Extending ExternalStructure class comment to list FFIExternalStructureReferenceHandle as possible handle type? I think we would only describe the reality.

2nd) ThreadedFFIPlugin>>#ffiPushStructureContentsOf: should be aware of FFIExternalStructureReferenceHandle and know how to reach the handle. Or should there be something in the image converting this to a ByteArray when being pushed as value in a ffiCall?

comments/ideas?

Unclassified.st (7K) Download Attachment

Holger Freyther

Re: FFIExternalStructureReferenceHandle and pushing it to the stack

> On 4. Apr 2019, at 08:36, Holger Freyther <[hidden email]> wrote:

And I have a stand-a-lone testcase for Pharo as well:

https://github.com/zecke/pharo/commit/390f81241a7dfee546784453fa426bc1c218b39e

Any hint of how I can test (simulate?) and generate changes to ThreadedX64SysVFFIPlugin from Pharo (or how you normally do it in Squeak).

cheers
holger

> In addition to my mail from yesterday to vm-dev I want to share the following example to illustrate the problem. I have taken StructA and StructNested from the FFIExternalStructureReferenceHandle class comment and put it into Unclassified.st that is attached to this mail. I am also making a call to LibC's getpid and try to push StructNested to the stack. The ffiCall is failing before getpid is called (Bad argument to external function).
>
> The reproducer is:
>
> StructA new theNest anyCall.
>
>
> StructA has a ByteArray handle of the full struct. theNest returns an instance of StructNested with a FFIExternalStructureReferenceHandle. This makes sense as well. It can't be a copy of the ByteArray (or else updates would be invisible to the outer struct) and it can't be an ExternalAddress as StructA's handle might move in the Smalltalk heap.
>
> I am not entirely sure how this is should work but what do you think of:
>
> 1st) Extending ExternalStructure class comment to list FFIExternalStructureReferenceHandle as possible handle type? I think we would only describe the reality.
>
>
> 2nd) ThreadedFFIPlugin>>#ffiPushStructureContentsOf: should be aware of FFIExternalStructureReferenceHandle and know how to reach the handle. Or should there be something in the image converting this to a ByteArray when being pushed as value in a ffiCall?
>
>
> comments/ideas?
>
>
> <Unclassified.st>

Eliot Miranda-2

Re: FFIExternalStructureReferenceHandle and pushing it to the stack

Hi Holger,

On Thu, Apr 4, 2019 at 4:49 AM Holger Freyther <[hidden email]> wrote:

> On 4. Apr 2019, at 08:36, Holger Freyther <[hidden email]> wrote:

And I have a stand-a-lone testcase for Pharo as well:

https://github.com/zecke/pharo/commit/390f81241a7dfee546784453fa426bc1c218b39e

Any hint of how I can test (simulate?) and generate changes to ThreadedX64SysVFFIPlugin from Pharo (or how you normally do it in Squeak).

If I wanted to simulate I would:

- create a 64-bit VMMaker image using the scripts in the image directory (32bit works, but 64-bits is faster)

- prepare the Pharo image (32 or 64-bit) containing the FFI call so that either it is snapshotted so that it will perform the call immediately, or that a script can be used at start-up to initiate the call

- load this image in the simulator, e.g.

| cos |

cos := StackInterpreterSimulator newWithOptions: #(ObjectMemory Spur32BitMemoryManager PharoVM true).

cos systemAttributes at: 2 put: 'segload.st'.

cos setBreakSelector: #primitiveCallout.

cos openAsMorph; run

- then, when I had fixed ThreadedFFIPlugin to function as I wanted I would either submit VMMaker.oscog to http://source.squeak.org/VMMakerInbox or commit directly to http://source.squeak.org/VMMaker

Now, the only problem here is that simulation can only proceed up until the point of actually trying to make the call. Currently the simulator isn't clever enough to actually simulate the call. But at least you'll be able to examine the state of the C stack, and the point of calling the foreign function in the simulator.

Holder, since much of this is probably very unfamiliar to you I would happily try and pair with you while you attempted it. You can make your Squeak VMMaker image first of course.

cheers
holger

> In addition to my mail from yesterday to vm-dev I want to share the following example to illustrate the problem. I have taken StructA and StructNested from the FFIExternalStructureReferenceHandle class comment and put it into Unclassified.st that is attached to this mail. I am also making a call to LibC's getpid and try to push StructNested to the stack. The ffiCall is failing before getpid is called (Bad argument to external function).
>
> The reproducer is:
>
> StructA new theNest anyCall.
>
>
> StructA has a ByteArray handle of the full struct. theNest returns an instance of StructNested with a FFIExternalStructureReferenceHandle. This makes sense as well. It can't be a copy of the ByteArray (or else updates would be invisible to the outer struct) and it can't be an ExternalAddress as StructA's handle might move in the Smalltalk heap.
>
> I am not entirely sure how this is should work but what do you think of:
>
> 1st) Extending ExternalStructure class comment to list FFIExternalStructureReferenceHandle as possible handle type? I think we would only describe the reality.
>
>
> 2nd) ThreadedFFIPlugin>>#ffiPushStructureContentsOf: should be aware of FFIExternalStructureReferenceHandle and know how to reach the handle. Or should there be something in the image converting this to a ByteArray when being pushed as value in a ffiCall?
>
>
> comments/ideas?
>
>
> <Unclassified.st>

_,,,^..^,,,_

best, Eliot

Eliot Miranda-2

Re: FFIExternalStructureReferenceHandle and pushing it to the stack

In reply to this post by Holger Freyther

Hi Holger,

On Thu, Apr 4, 2019 at 12:36 AM Holger Freyther <[hidden email]> wrote:

In addition to my mail from yesterday to vm-dev I want to share the following example to illustrate the problem. I have taken StructA and StructNested from the FFIExternalStructureReferenceHandle class comment and put it into Unclassified.st that is attached to this mail. I am also making a call to LibC's getpid and try to push StructNested to the stack. The ffiCall is failing before getpid is called (Bad argument to external function).

The reproducer is:

StructA new theNest anyCall.

This is not answering the question I need answering :-). Can you answer the Qs I asked on vm-dev yesterday about the signature of the callback etc? Can you reply there? AdvThnaksance

StructA has a ByteArray handle of the full struct. theNest returns an instance of StructNested with a FFIExternalStructureReferenceHandle. This makes sense as well. It can't be a copy of the ByteArray (or else updates would be invisible to the outer struct) and it can't be an ExternalAddress as StructA's handle might move in the Smalltalk heap.

I am not entirely sure how this is should work but what do you think of:

1st) Extending ExternalStructure class comment to list FFIExternalStructureReferenceHandle as possible handle type? I think we would only describe the reality.

2nd) ThreadedFFIPlugin>>#ffiPushStructureContentsOf: should be aware of FFIExternalStructureReferenceHandle and know how to reach the handle. Or should there be something in the image converting this to a ByteArray when being pushed as value in a ffiCall?

comments/ideas?

_,,,^..^,,,_

best, Eliot

Holger Freyther

Re: [Pharo-users] FFIExternalStructureReferenceHandle and pushing it to the stack

> On 4. Apr 2019, at 17:20, Eliot Miranda <[hidden email]> wrote:
>
> Hi Holger,

Hey!

> The reproducer is:
>
> StructA new theNest anyCall.
>
> This is not answering the question I need answering :-). Can you answer the Qs I asked on vm-dev yesterday about the signature of the callback etc? Can you reply there? AdvThnaksance

sorry about that. I thought this is an even easier reproducer. Let me respond in the other thread.