[squeak-dev] Debbuging problems, FFI

2009/8/2 Andreas Raab <[hidden email]>:

> Mariano Martinez Peck wrote:
>>
>> Javier has done an excellent job and here is his last post. Can someone
>> read what he found and see if we can fix this annoying bug?
>
> The problem is that you can't call #valueWithReceiver:args: since this would
> both execute the entire method and do it in the wrong context where instead
> you need to execute *just* the primitive (since you'll simulate the rest if
> it fails).
>
> Fortunately, external functions have two ways to invoke them; one implicitly
> as primitive in a method (i.e., prim 120) and another one explicitly (via
> invokeWithArgs:). A variant on the latter that returns
> ContextPart>>primFailToken does the trick nicely.
>
> I've updated both http://source.squeak.org/FFI and
> http://source.squeak.org/trunk with the ncessary bits.

Andreas, does your change fixes the issue? Because its not clear from
your message - is it solves the problem altogether, or it is just a
first step towards proper fix?

> Cheers,
>  - Andreas
>
>>
>> Thanks,
>>
>> Mariano
>>
>>
>> ---------- Forwarded message ----------
>> From: *Javier Pimás* <[hidden email]
>> <mailto:[hidden email]>>
>> Date: 2009/8/2
>> Subject: Re: [Pharo-project] Debbuging problems, FFI
>> To: [hidden email]
>> <mailto:[hidden email]>
>>
>>
>> I Think I found where the problem lies, it's just sitting there waiting to
>> be resolved.
>>
>> To see what was happening I debugged the debuger, which was really weird.
>> I found the answer when I broke into the step over button by adding a self
>> break in OTCmdOverDebugger>>execute. After catching it I removed the break
>> and stepped into everything to see how step over worked. The intresting part
>> is in MethodContext(ContextPart)>>step, which sends
>> interpretNextInstructionFor: self. The idea is this, instead of advancing
>> the vm program counter within the vm code, the debugger simulates the
>> execution cycle, step by step. It not so hard as it seems but also not very
>> simple. The problem comes when the debugger tryies to simulate primitive
>> calls. When the simulator detects a primitive, it issues a
>>  MethodContext(ContextPart)>>doPrimitive:method:receiver:args:, here you can
>> see the detection:
>>
>> tryPrimitiveFor: method receiver: receiver args: arguments
>>    "..."
>>    | primIndex |
>>    (primIndex := method primitive) = 0 ifTrue: [^ PrimitiveFailToken].
>>    ^ self doPrimitive: primIndex method: method receiver: receiver args:
>> arguments
>>
>> well, the problem is that doPrimitive doesn't seem to implement all
>> primitives (I recomend you to look it's code), but just a few ones. Also, as
>> far as I can see, it calls primitive 19, which i found is (by looking vm C
>> code) primitiveFail, and that primitive just sets a flag that says that
>> something failed.
>>
>> Then, if my calculations are correct, the problem is that the FFI Call
>> primitive needs to be handled by hand. I tryied adding this to
>> ContextPart>>doPrimitive:method:receiver:args:
>>
>> primitiveIndex = 120 ifTrue: "Calling an FFI method, just evaluate it and
>> return"
>>        [ meth valueWithReceiver: receiver arguments: arguments . ^self].
>>
>> just before
>>
>>    arguments size > 6 ifTrue: [^PrimitiveFailToken].
>>
>> which I think almost worked. I think the primitive was executed, but this
>> doesn't handle the execution context that is being simulated, and the stack
>> gets corrupted. Somebody with more experience in ContextPart, MethodContext,
>> etc. may give us a hand here, I think the fix is almost there.
>>
>> Bye,
>>      Javier.
>>
>> 2009/7/31 Mariano Martinez Peck <[hidden email]
>> <mailto:[hidden email]>>
>>
>>
>>
>>    2009/7/31 Javier Pimás <[hidden email]
>>    <mailto:[hidden email]>>
>>
>>        Does anybody know if the problem occurs just in Windows or in
>>        any OS? this can give a clue of the problem...
>>
>>
>>    I don't remember debugging in Windows, but in Linux I have this
>>    problem for sure.
>>
>>
>>        On Thu, Jul 30, 2009 at 11:28 PM, Javier Pimás
>>        <[hidden email] <mailto:[hidden email]>>
>>        wrote:
>>
>>            Well, at least I feel better that I'm not crazy, (or not the
>>            only one...). Maybe if someone could give us a clue about
>>            it, we could spot where the problem is. Do you think it is a
>>            vm problem or something related to the smalltalk code?
>>
>>            2009/7/30 Mariano Martinez Peck <[hidden email]
>>            <mailto:[hidden email]>>
>>
>>                Yes. This is really annoying for me. I am writing a C
>>                library plugin since a year and a half and this is very
>>                frustrated :(
>>                You cannot debug when using FFI. I mean, you cannot get
>>                advantages of one of the best Smalltalk features! What I
>>                do is to write a self break before and after the call
>>                and use "proceed" but I don't like it at all.
>>
>>                Once I tried to see what the problem was, but I was to
>>                difficult to me to understand it and fix it. We need a
>>                guru here I think hahaha.
>>
>>                Best,
>>
>>                Mariano
>>
>>                2009/7/30 Javier Pimás <[hidden email]
>>                <mailto:[hidden email]>>
>>
>>                    Hi, I'm having problems with the debugger. Every
>>                    time I try to debug code that uses FFI, if I step
>>                    over a message that internally does an FFI call, I
>>                    get an externalCallFailed error and then I land in
>>                    MethodContext(ContextPart)>>runUntilErrorOrReturnFrom:
>>                    If I run the code without debugging, then I have no
>>                    problems at all.
>>
>>                    I'm using SqueakPharo v3.11.2 with
>>                    pharo0.1-10373web09.07.2.
>>
>>                    To reproduce it you can try this:
>>
>>                    ScriptLoader loadFFI.
>>                    then load latest GLMorphic from squeaksource, and
>>                    uncomment the "self break." in
>>                    GLCanvas>>displayString:from:to:at:strikeFont:kern:
>>
>>                    doit
>>                    World fullDrawOn: GLDisplayScreen testCanvas
>>                    hit debug and step over until you reach
>>                    ogl glTexImage2D: GLTexture2d with:...
>>
>>                    also you'll see wrong values for local vars, like
>>                    aPoint which will look as #(nil nil nil nil nil)
>>                    when it'll actually have a point.
>>
>>                    Is there any other way to set up a break point
>>                    instead of adding self break to the code??
>>
>>                    Thanks,
>>                           Javier.
>>
>>                    --                    Javier Pimás
>>                    Ciudad de Buenos Aires
>>
>>                    _______________________________________________
>>                    Pharo-project mailing list
>>                    [hidden email]
>>                    <mailto:[hidden email]>
>>
>>  http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>>
>>                _______________________________________________
>>                Pharo-project mailing list
>>                [hidden email]
>>                <mailto:[hidden email]>
>>
>>  http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>>
>>
>>            --            Javier Pimás
>>            Ciudad de Buenos Aires
>>
>>
>>
>>
>>        --        Javier Pimás
>>        Ciudad de Buenos Aires
>>
>>        _______________________________________________
>>        Pharo-project mailing list
>>        [hidden email]
>>        <mailto:[hidden email]>
>>        http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>>
>>    _______________________________________________
>>    Pharo-project mailing list
>>    [hidden email]
>>    <mailto:[hidden email]>
>>    http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>>
>>
>> --
>> Javier Pimás
>> Ciudad de Buenos Aires
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> <mailto:[hidden email]>
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Mariano Martinez Peck wrote:

   Andreas, does your change fixes the issue? Because its not clear from
   your message - is it solves the problem altogether, or it is just a
   first step towards proper fix?

Thanks Igor. I had the same questions as you. However, I downloaded that new version, and tried but I got same results (the behaviour of the first email). I mean, when I did step over in the api call I landed in and in MethodContext(ContextPart)>>runUntilErrorOrReturnFrom:

If that happens then you haven't updated your image. You can verify this by looking for both implementors and senders of #tryInvokeWithArguments: - there should be exactly one implementor (ExternalFunction) and one sender (ContextPart).

Cheers,
 - Andreas

Mariano Martinez Peck wrote:

Andreas: I double checked and I see ExternalFunction>>tryInvokeWithArguments:    but then I look senders, I see no senders. I am trying with FFI-Kernel-ar.11.
Should I do something more?

Yes. Like I said before, you need to update your image. One part of the fix is in the FFI package, the other part is in Kernel (ContextPart>>doPrimitive...) That's why you need to update your image. Without updating your image you will only get half of the fix and it will not work.

Cheers,
 - Andreas

Mariano Martinez Peck wrote:

Andreas: I am under a Pharo image, can you teld me which .mcz (version) and package did you change? so that I can see the changes and apply them to pharo?  Or even better, just send the .cs. The update is not the same in Pharo as in Squeak that's why I ask.

Oh, I see. I thought you said you were using Squeak. Attached is a fileOut of the relevant method from Squeak but be warned that your milage may vary greatly if there have been any other changes. Good luck.

Cheers,
 - Andreas