RE: [Pharo-project] Debug-it and external calls

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

RE: [Pharo-project] Debug-it and external calls

Schwab,Wilhelm K

So the external call does not happen and the fall-back code executes.  That explains it.  It might be nice if FFI calls could be made to happen, or at least to have a more informative error message, such as "Primitive not executed by the debugger" rather than "Unable to find function address."

Bill




________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
Sent: Monday, July 19, 2010 12:20 PM
To: [hidden email]
Subject: Re: [Pharo-project] Debug-it and external calls

On Mon, 19 Jul 2010, Schwab,Wilhelm K wrote:

> I often (not always??) find that using the Debug-it command and debugging into an external call (FFI) leads to a false claim of "Unable to find function address."  Stepping over the call works; stepping into it make it look like it fails.
>
> I am curently using a 1.1 RC2 image and the 4.0.3 vm on Ubuntu.  I cannot easily move the offending code to Windows, nor do I particularly care to do so (feels good<g>), and FFI is fairly uncommon in Squeak/Pharo.
>
> Can anyone else tinker with this to see if it is real?

IIRC when you're using the debugger, primitives are not evaluated.


Levente

>
> Bill
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

RE: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls

Schwab,Wilhelm K

If the FFI call is flawed, why does it work when not stepping in the debugger?  I'm not buying it :)




-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Igor Stasenko
Sent: Monday, July 19, 2010 3:02 PM
To: Pharo Development
Subject: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls

---------- Forwarded message ----------
From: Igor Stasenko <[hidden email]>
Date: 19 July 2010 23:00
Subject: Re: [squeak-dev] Re: [Pharo-project] Debug-it and external calls
To: The general-purpose Squeak developers list <[hidden email]>


On 19 July 2010 21:42, Andreas Raab <[hidden email]> wrote:

> On 7/19/2010 9:29 AM, Igor Stasenko wrote:
>>
>> No, debugger, actually doing a primitive call, but in order to
>> intercept a primitive failure and 'step into' method, it does the
>> trick with replacing the subject method with temporary method (which
>> will call the same primitive with same arguments&  recevier).
>> Then, if primitive runs ok, it simply behaves as a 'step over', and
>> if primitive fails, then debugger itercepts it and creates a context
>> for subject method, which should handle primitive failure.
>
> Correct. Stepping over FFI calls will only fail if the FFI call
> actually fails. So whatever the problem it's somewhere in your FFI call.
>
But! The problem which i discovered not long ago, that if FFI method contains many arguments (close to max 15), then debugger is unable to invoke it, because #perform:... method works in a way, that its using temps of context, where it were invoked. I fixed it by setting a #perform method's frame to be a large one.


See http://bugs.squeak.org/view.php?id=7534

> Cheers,
>  - Andreas
>
>> On 19 July 2010 19:22, Schwab,Wilhelm K<[hidden email]>  wrote:
>>>
>>> So the external call does not happen and the fall-back code executes.
>>>  That explains it.  It might be nice if FFI calls could be made to
>>> happen, or at least to have a more informative error message, such
>>> as "Primitive not executed by the debugger" rather than "Unable to find function address."
>>>
>>> Bill
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email]
>>> [[hidden email]] On Behalf Of Levente
>>> Uzonyi [[hidden email]]
>>> Sent: Monday, July 19, 2010 12:20 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] Debug-it and external calls
>>>
>>> On Mon, 19 Jul 2010, Schwab,Wilhelm K wrote:
>>>
>>>> I often (not always??) find that using the Debug-it command and
>>>> debugging into an external call (FFI) leads to a false claim of
>>>> "Unable to find function address."  Stepping over the call works;
>>>> stepping into it make it look like it fails.
>>>>
>>>> I am curently using a 1.1 RC2 image and the 4.0.3 vm on Ubuntu.  I
>>>> cannot easily move the offending code to Windows, nor do I
>>>> particularly care to do so (feels good<g>), and FFI is fairly uncommon in Squeak/Pharo.
>>>>
>>>> Can anyone else tinker with this to see if it is real?
>>>
>>> IIRC when you're using the debugger, primitives are not evaluated.
>>>
>>>
>>> Levente
>>>
>>>>
>>>> Bill
>>>>
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>>
>>
>>
>>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

RE: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls

Eliot Miranda-2
 


On Mon, Jul 19, 2010 at 1:10 PM, Schwab,Wilhelm K <[hidden email]> wrote:

If the FFI call is flawed, why does it work when not stepping in the debugger?  I'm not buying it :)

A few people have seen problems with the part of the debugger in question.  Here's the canonical version of the method by Andreas 'ar 5/25/2000 20:41'.  It is invoked from ContextPart>>doPrimitive:method:receiver:args: and its job is to invoke just the primitive and either answer the result or a special PrimitiveFailed token.  The debugger then checks for the result being the PrimitiveFailed token and if so steps into the method.

ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments
"Hack. Attempt to execute the named primitive from the given compiled method"
| selector theMethod spec |
arguments size > 8 ifTrue:[^PrimitiveFailToken].
selector _ #(
tryNamedPrimitive 
tryNamedPrimitive: 
tryNamedPrimitive:with: 
tryNamedPrimitive:with:with: 
tryNamedPrimitive:with:with:with:
tryNamedPrimitive:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.
theMethod _ aReceiver class lookupSelector: selector.
theMethod == nil ifTrue:[^PrimitiveFailToken].
spec _ theMethod literalAt: 1.
spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1) startingAt: 1.
^aReceiver perform: selector withArguments: arguments
 
What this does is slam the named primitive to be evaluated into the relevant implementation of tryNamedPrimitive[:with:with:with:with:with:with:with:] and invoke it.  One minor problem is that there are only 9 methods so there's only support for 0 to 8 arguments.  The real problem with this is that it doesn't flush the VM's method lookup cache and so one can end up invoking a different named primitive, one that was installed into the same method by a previous invocation of tryNamedPrimitiveIn:for:withArgs:.

Michael Haupt saw the effect of this doing bytecode simulation on Cog.  For some reason it's much more likely to hit the cache flush issue in Cog tan in the standard interpreter (probably because Cog makes less use of the method cache because machine code sends have their send caches inline in the machine code).

I fixed this for Michael by using the following version 'eem 5/23/2010 14:13'
ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments
"Hack. Attempt to execute the named primitive from the given compiled method"
| selector theMethod spec |
arguments size > 8 ifTrue:[^PrimitiveFailToken].
selector := #(
tryNamedPrimitive 
tryNamedPrimitive: 
tryNamedPrimitive:with: 
tryNamedPrimitive:with:with: 
tryNamedPrimitive:with:with:with:
tryNamedPrimitive:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.
theMethod := aReceiver class lookupSelector: selector.
theMethod == nil ifTrue:[^PrimitiveFailToken].
spec := theMethod literalAt: 1.
spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1) startingAt: 1.
theMethod flushCache.
selector flushCache.
^aReceiver perform: selector withArguments: arguments

It adds "theMethod flushCache. selector flushCache" to flush the cache and at least for a couple of months things seemed to be OK.  But Michael sent me a different case a week ago that turns out to be due to exactly the same cause.  When simulated his code was trying to invoke the primDigitDivNegative primitive via digitDiv:neg: (which doesn't exist in Cog and so should simply fail) but it was succeeding, running some bogus primitive.

There is a more fundamental issue with the above.  try debugging it.  If you do you stand a good chance of the debugger overwriting what the debugger version of the method is trying to effect.  That's why if you look at the analogous machinery for num bered primitives you'll see the above approach (which was used in Smalltalk-80 V2) is discarded and a much more reliable approach, a primitive-executing primitive, is used, namely ProtoObject>>#tryPrimitive:withArgs: 'ajh 1/31/2003 22:21':
ProtoObject>>tryPrimitive: primIndex withArgs: argumentArray
"This method is a template that the Smalltalk simulator uses to 
execute primitives. See Object documentation whatIsAPrimitive."

<primitive: 118>
^ ContextPart primitiveFailToken

In the Teleplace images I've taken a similar approach, providing a primitive in Cog.  The below is slightly different because the Teleplace images debugger supports primitive error codes (and I need to fold this back into Squeak asap). ContextPart>>tryNamedPrimitiveIn:for:withArgs: eem 5/12/2009

ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments
| selector theMethod spec receiverClass |
<primitive: 218 error: ec>
ec ifNotNil:
["If ec is an integer other than -1 there was a problem with primitive 218,
 not with the external primitive itself.  -1 indicates a generic failure (where
 ec should be nil) but ec = nil means primitive 218 is not implemented.  So
 interpret -1 to mean the external primitive failed with a nil error code."
ec isInteger ifTrue:
[ec = -1
ifTrue: [ec := nil]
ifFalse: [self primitiveFailed]].
^{PrimitiveFailToken. ec}].
"Assume a nil error code implies the primitive is not implemented and fall back on the old code."
"Hack. Attempt to execute the named primitive from the given compiled method"
arguments size > 8 ifTrue:
[^{PrimitiveFailToken. nil}].
selector := #(
tryNamedPrimitive 
tryNamedPrimitive: 
tryNamedPrimitive:with: 
tryNamedPrimitive:with:with: 
tryNamedPrimitive:with:with:with:
tryNamedPrimitive:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.
receiverClass := self objectClass: aReceiver.
theMethod := receiverClass lookupSelector: selector.
theMethod == nil ifTrue:
[^{PrimitiveFailToken. nil}].
spec := theMethod literalAt: 1.
spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1) startingAt: 1.
Smalltalk unbindExternalPrimitives.
^self object: aReceiver perform: selector withArguments: arguments inClass: receiverClass

With this approach there's no argument count limit and there's no chance of executing the wrong primitive.  But it requires a primitive in the VM.  Perhaps the VM folks can take a look at porting the Cog primitive primitiveDoNamedPrimitiveWithArgs back to the base VM and then we can use something like the above with the fallback code, liberally sprinkled with cache flushes, only as a fallback.

HTH
Eliot





-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Igor Stasenko
Sent: Monday, July 19, 2010 3:02 PM
To: Pharo Development
Subject: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls

---------- Forwarded message ----------
From: Igor Stasenko <[hidden email]>
Date: 19 July 2010 23:00
Subject: Re: [squeak-dev] Re: [Pharo-project] Debug-it and external calls
To: The general-purpose Squeak developers list <[hidden email]>


On 19 July 2010 21:42, Andreas Raab <[hidden email]> wrote:
> On 7/19/2010 9:29 AM, Igor Stasenko wrote:
>>
>> No, debugger, actually doing a primitive call, but in order to
>> intercept a primitive failure and 'step into' method, it does the
>> trick with replacing the subject method with temporary method (which
>> will call the same primitive with same arguments&  recevier).
>> Then, if primitive runs ok, it simply behaves as a 'step over', and
>> if primitive fails, then debugger itercepts it and creates a context
>> for subject method, which should handle primitive failure.
>
> Correct. Stepping over FFI calls will only fail if the FFI call
> actually fails. So whatever the problem it's somewhere in your FFI call.
>
But! The problem which i discovered not long ago, that if FFI method contains many arguments (close to max 15), then debugger is unable to invoke it, because #perform:... method works in a way, that its using temps of context, where it were invoked. I fixed it by setting a #perform method's frame to be a large one.


See http://bugs.squeak.org/view.php?id=7534

> Cheers,
>  - Andreas
>
>> On 19 July 2010 19:22, Schwab,Wilhelm K<[hidden email]>  wrote:
>>>
>>> So the external call does not happen and the fall-back code executes.
>>>  That explains it.  It might be nice if FFI calls could be made to
>>> happen, or at least to have a more informative error message, such
>>> as "Primitive not executed by the debugger" rather than "Unable to find function address."
>>>
>>> Bill
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email]
>>> [[hidden email]] On Behalf Of Levente
>>> Uzonyi [[hidden email]]
>>> Sent: Monday, July 19, 2010 12:20 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] Debug-it and external calls
>>>
>>> On Mon, 19 Jul 2010, Schwab,Wilhelm K wrote:
>>>
>>>> I often (not always??) find that using the Debug-it command and
>>>> debugging into an external call (FFI) leads to a false claim of
>>>> "Unable to find function address."  Stepping over the call works;
>>>> stepping into it make it look like it fails.
>>>>
>>>> I am curently using a 1.1 RC2 image and the 4.0.3 vm on Ubuntu.  I
>>>> cannot easily move the offending code to Windows, nor do I
>>>> particularly care to do so (feels good<g>), and FFI is fairly uncommon in Squeak/Pharo.
>>>>
>>>> Can anyone else tinker with this to see if it is real?
>>>
>>> IIRC when you're using the debugger, primitives are not evaluated.
>>>
>>>
>>> Levente
>>>
>>>>
>>>> Bill
>>>>
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>>
>>
>>
>>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project