Fwd: [V3dot9] Fwd: FFI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: [V3dot9] Fwd: FFI

Stéphane Ducasse-3
why FFI will be in 3.9.
I think that the email is self-explanatory.

Stef
PS: lukas has some good idea how to make FFI and other compiler  
extensions modular, but we should have interest in that.

Begin forwarded message:

> From: Marcus Denker <[hidden email]>
> Date: 10 août 2006 12:54:58 HAEC
> To: "3.9 Squeak" <[hidden email]>
> Subject: [V3dot9] Fwd: FFI
>
>
>
> Begin forwarded message:
>
>> From: Marcus Denker <[hidden email]>
>> Date: 10. August 2006 12:46:11 GMT+02:00
>> To: Stephane Ducasse <[hidden email]>
>> Subject: FFI
>>
>> Hi,
>>
>> I am allways saying that we should not just do everything in the  
>> last second.
>> I am very much for having this as a strict rule, covering FFI,  
>> too. There was
>> time to do it, and we didn't. That's life, there is a version  
>> after 3.9 to do it.
>>
>> Unloading FFI is not as simple as people think it is:
>> There is a method "discardFFI", this contains the knowledge to  
>> unload it:
>>
>> self
>> at: #ExternalStructure
>> ifPresent: [:cls | (ChangeSet superclassOrder: cls  
>> withAllSubclasses asArray)
>> reverseDo: [:c | c removeFromSystem]].
>> SystemOrganization removeCategoriesMatching: 'FFI-*'.
>> self recreateSpecialObjectsArray.
>>
>> We have hidden references to FFI left over in
>>
>>  -> SpecialObjectArray
>>  -> Compiler
>>  -> the unload method itself.
>>
>> all of them are hacked to use the "Smalltalk at:  
>> #ExternalStructure" hack to not
>> leave references the system knows about, but of course, it leaved  
>> references
>> nevertheless of the kind that is un-tracable, not documented, and  
>> easily overlooked.
>>
>> What happens when we load FFI? Does is call the specialObject  
>> registration?
>> If we then clean up the image further, do we just delete the  
>> discardFFI method
>> and loose the ability to unload FFI? Will we "clean" the compiler  
>> and the specialObject
>> array at some point by accident?
>>
>> Where are those dependencies documented?
>>
>> I can do it now in the hackish way (leaving the non-tracable bits  
>> of FFI), but I wasted another
>> hour(!) for this!!!
>>
>> I want to stop. I don't care anymore. I want to finish.
>>
>>      Marcus
>>
>
> _______________________________________________
> V3dot9 mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/v3dot9


Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [V3dot9] Fwd: FFI

Andreas.Raab
That just raises my previous question again: What do I have to do to get
the FFI out of the image?

To remind you: Removing the FFI was part of my "FFI maintenance deal".
Back in 10/05 I both asked for it to be removed and filed a bug report
[1]. In 01/06 I asked *again* for it to be removed [2], to no avail.
Then I asked *again* in 07/06 and nothing happened [3]. Then I asked
*again* in 08/06 [4]. Each time the base line response was "yes, we'll
do it in 3.9" - except that now it's "too late, that's life". I don't
find that self-explanatory.

If I spend time and effort to try to get things done in an "orderly" way
I think I can at least expect a response and an explanation of what
exactly the steps in the process are and where exactly this process went
wrong.

Would somebody *please* bother telling me which steps exactly I need to
follow to get this done?

[1]http://bugs.impara.de/view.php?id=2123
[2]http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-January/099871.html
[3]http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-July/106389.html
[4]http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-August/106709.html



Stéphane Ducasse wrote:

> why FFI will be in 3.9.
> I think that the email is self-explanatory.
>
> Stef
> PS: lukas has some good idea how to make FFI and other compiler
> extensions modular, but we should have interest in that.
>
> Begin forwarded message:
>
>> From: Marcus Denker <[hidden email]>
>> Date: 10 août 2006 12:54:58 HAEC
>> To: "3.9 Squeak" <[hidden email]>
>> Subject: [V3dot9] Fwd: FFI
>>
>>
>>
>> Begin forwarded message:
>>
>>> From: Marcus Denker <[hidden email]>
>>> Date: 10. August 2006 12:46:11 GMT+02:00
>>> To: Stephane Ducasse <[hidden email]>
>>> Subject: FFI
>>>
>>> Hi,
>>>
>>> I am allways saying that we should not just do everything in the last
>>> second.
>>> I am very much for having this as a strict rule, covering FFI, too.
>>> There was
>>> time to do it, and we didn't. That's life, there is a version after
>>> 3.9 to do it.
>>>
>>> Unloading FFI is not as simple as people think it is:
>>> There is a method "discardFFI", this contains the knowledge to unload
>>> it:
>>>
>>> self
>>>         at: #ExternalStructure
>>>         ifPresent: [:cls | (ChangeSet superclassOrder: cls
>>> withAllSubclasses asArray)
>>>                 reverseDo: [:c | c removeFromSystem]].
>>>     SystemOrganization removeCategoriesMatching: 'FFI-*'.
>>>     self recreateSpecialObjectsArray.
>>>
>>> We have hidden references to FFI left over in
>>>
>>>  -> SpecialObjectArray
>>>  -> Compiler
>>>  -> the unload method itself.
>>>
>>> all of them are hacked to use the "Smalltalk at: #ExternalStructure"
>>> hack to not
>>> leave references the system knows about, but of course, it leaved
>>> references
>>> nevertheless of the kind that is un-tracable, not documented, and
>>> easily overlooked.
>>>
>>> What happens when we load FFI? Does is call the specialObject
>>> registration?
>>> If we then clean up the image further, do we just delete the
>>> discardFFI method
>>> and loose the ability to unload FFI? Will we "clean" the compiler and
>>> the specialObject
>>> array at some point by accident?
>>>
>>> Where are those dependencies documented?
>>>
>>> I can do it now in the hackish way (leaving the non-tracable bits of
>>> FFI), but I wasted another
>>> hour(!) for this!!!
>>>
>>> I want to stop. I don't care anymore. I want to finish.
>>>
>>>      Marcus
>>>
>>
>> _______________________________________________
>> V3dot9 mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/mailman/listinfo/v3dot9
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [V3dot9] Fwd: FFI

Andreas.Raab
In reply to this post by Stéphane Ducasse-3
I just got reminded that I hadn't responded to the actual points Marcus
was making, so here we go:

> We have hidden references to FFI left over in
>
>  -> SpecialObjectArray
>  -> Compiler
>  -> the unload method itself.
>
> all of them are hacked to use the "Smalltalk at: #ExternalStructure"
> hack to not leave references the system knows about, but of course,
> it leaved references nevertheless of the kind that is un-tracable,
> not documented, and easily overlooked.

If you think that this is a blocking criteria for unloading the FFI, you
should point that out and we should have a discussion about how to deal
with situations like those. Saying that any such occurrence means the
package cannot possibly be removed is certainly not a good answer.
Nowadays, and assuming some of the upcoming VM changes, all of these can
be solved in pretty decent ways:

a) With the latest VM changes, these classes can be stored directly in
the FFI plugin and don't need to be splObjs. This is on my TODO list as
soon as we have VMs that support it.

b) The Compiler could have class variable where an FFI extension can be
stored (which means that another implementation of the FFI could hook
into the compiler as well)

c) The method #discardFFI should be removed alltogether.

Of the above a) and b) may go a bit too far for 3.9 but at least I'd
like to have a discussion about available options. Personally, I don't
think that these references are really *that* problematic, but I'm open
for contrary arguments.

> What happens when we load FFI? Does is call the specialObject
> registration?

While I don't see how this question relates to unloading the FFI, the
obvious answer is: Yes, it does. If it doesn't, you file a bug report
and I'll fix it.

> If we then clean up the image further, do we just delete the
> discardFFI method and loose the ability to unload FFI?

Again, I fail to see the relevance of that question for the problem at
hand, but again the obvious answer is: No you don't. If you do, you file
a bug report and I'll fix it.

> Will we "clean" the compiler and the specialObject
> array at some point by accident?

By accident? How exactly do you imagine that to happen? The only way I
could see that happen is by someone making pretty explicit and
deliberate changes. I have never seen code "just vanishing".

> Where are those dependencies documented?

Where would you like them to be documented? If that is a criteria for
removal, just let me know and I'll fix it.

> I can do it now in the hackish way (leaving the non-tracable bits of
> FFI), but I wasted another hour(!) for this!!!
>
> I want to stop. I don't care anymore. I want to finish.

And why in the lord's name did you waste an hour instead of spending
five minutes writing me an email saying "hey, I've got this and that
problem, got any ideas"? It strikes me as the obvious thing to do, in
particular because your complaining to Stef of all people won't change a
single thing. If you want a resolution you should talk to the maintainer.

That would be me.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: [V3dot9] Fwd: FFI

stéphane ducasse-2
Hi andreas

May be we should have said that before.
Now this is just that we are **********exhauted************ by squeak.
So I pushed marcus to have a look and he tried because this is a cool  
guy and he saw all that and was
sick because he wants to stop. So now you know the background.

> If you think that this is a blocking criteria for unloading the  
> FFI, you should point that out and we should have a discussion  
> about how to deal with situations like those. Saying that any such  
> occurrence means the package cannot possibly be removed is  
> certainly not a good answer.

see background above. Sorry I can try to do something if you gave me  
the stuff done and some time
since I do not have time the next three or four days.


> Nowadays, and assuming some of the upcoming VM changes, all of  
> these can be solved in pretty decent ways:
>
> a) With the latest VM changes, these classes can be stored directly  
> in the FFI plugin and don't need to be splObjs. This is on my TODO  
> list as soon as we have VMs that support it.
>
> b) The Compiler could have class variable where an FFI extension  
> can be stored (which means that another implementation of the FFI  
> could hook into the compiler as well)
>
> c) The method #discardFFI should be removed alltogether.
>
> Of the above a) and b) may go a bit too far for 3.9 but at least  
> I'd like to have a discussion about available options. Personally,  
> I don't think that these references are really *that* problematic,  
> but I'm open for contrary arguments.

I do not know so I followed marcus when he has conservative arguments :)


>
>> What happens when we load FFI? Does is call the specialObject  
>> registration?
>
> While I don't see how this question relates to unloading the FFI,  
> the obvious answer is: Yes, it does. If it doesn't, you file a bug  
> report and I'll fix it.

Ok

We got a really good discussion with lukas on how to use pragmas so  
that we could have FFI function
using pragmas in a nice way and we could get the compiler nicely  
extended in a modular way.
I would love to see you talking with him about that and that we could  
integrate these ideas in 3.10.
Lukas?


>> If we then clean up the image further, do we just delete the  
>> discardFFI method and loose the ability to unload FFI?
>
> Again, I fail to see the relevance of that question for the problem  
> at hand, but again the obvious answer is: No you don't. If you do,  
> you file a bug report and I'll fix it.

Ok

>> Will we "clean" the compiler and the specialObject
>> array at some point by accident?
>
> By accident? How exactly do you imagine that to happen? The only  
> way I could see that happen is by someone making pretty explicit  
> and deliberate changes. I have never seen code "just vanishing".

The point of marcus was that if something is not explicit then  
someone could change the code and we would lose some information
and break the fact that FFI could be used or something like that.

>> Where are those dependencies documented?
>
> Where would you like them to be documented? If that is a criteria  
> for removal, just let me know and I'll fix it.
>
>> I can do it now in the hackish way (leaving the non-tracable bits  
>> of FFI), but I wasted another hour(!) for this!!!
>>
>> I want to stop. I don't care anymore. I want to finish.
>
> And why in the lord's name did you waste an hour instead of  
> spending five minutes writing me an email saying "hey, I've got  
> this and that problem, got any ideas"?

Indeed this is a good question. May be the communication between us  
is broken and we need to drink more
beers together. I imagine that this is true squeakers need to meet  
more often.

> It strikes me as the obvious thing to do, in particular because  
> your complaining to Stef of all people won't change a single thing.  
> If you want a resolution you should talk to the maintainer.

Yes but marcus was telling me that because he is ****TIRED/
EXHAUSTED**** and I was still pushing.
>
> That would be me.

Andreas here is what I suggest:
        give me all the information plus code and I will do it

Marcus do you have the script to create the new source?
I can take some time this week to solve that problem as soon as  
andreas gives to me the info I need.
And we release 3.9

>
> Cheers,
>   - Andreas
>


Reply | Threaded
Open this post in threaded view
|

Re: [V3dot9] Fwd: FFI

Andreas.Raab
stéphane ducasse wrote:
> Andreas here is what I suggest:
>     give me all the information plus code and I will do it

Simple:
a) Unload the FFI package using Monticello
b) Evaluate "Smalltalk recreateSpecialObjectsArray"
c) [Optionally] remove the method #discardFFI

This leaves no obsolete references behind in 3.9-7051 and causes no
tests to fail that have previously passed. It will leave the two places
with extra references (special objects and parser) but fixing those is
not realistic at this point.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: [V3dot9] Fwd: FFI

Marcus Denker
In reply to this post by stéphane ducasse-2
>
> Andreas here is what I suggest:
> give me all the information plus code and I will do it
>

We are in *beta*. Let's stop. If it's FFI, why not Speech? Why not
5 others? Why not then those 20 enhancements here and those
normally not that important fixes there?

We once decided that beta means to only fix show-stopping bugs.
FFI is not a bug.

      Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [V3dot9] Fwd: FFI

Marcus Denker

On 12.08.2006, at 10:24, Marcus Denker wrote:

>>
>> Andreas here is what I suggest:
>> give me all the information plus code and I will do it
>>
>
> We are in *beta*. Let's stop. If it's FFI, why not Speech? Why not
> 5 others? Why not then those 20 enhancements here and those
> normally not that important fixes there?
>
> We once decided that beta means to only fix show-stopping bugs.
> FFI is not a bug.

Whatever... I think I am just completely burned out. So just do it...
But I stop.

     Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [V3dot9] Fwd: FFI

stéphane ducasse-2
I understand, I will do it.
Relax you did an excellent and amazing amount of work so far. We  
should celebrate that next time
I'm at Berne.

Stef

On 12 août 06, at 11:05, Marcus Denker wrote:

>
> On 12.08.2006, at 10:24, Marcus Denker wrote:
>
>>>
>>> Andreas here is what I suggest:
>>> give me all the information plus code and I will do it
>>>
>>
>> We are in *beta*. Let's stop. If it's FFI, why not Speech? Why not
>> 5 others? Why not then those 20 enhancements here and those
>> normally not that important fixes there?
>>
>> We once decided that beta means to only fix show-stopping bugs.
>> FFI is not a bug.
>
>
> Whatever... I think I am just completely burned out. So just do it...
> But I stop.
>
>     Marcus


Reply | Threaded
Open this post in threaded view
|

Re: [V3dot9] Fwd: FFI

Klaus D. Witzel
On Sat, 12 Aug 2006 11:20:03 +0200, stéphane ducasse wrote:

> I understand, I will do it.
> Relax you did an excellent and amazing amount of work so far. We should  
> celebrate that next time
> I'm at Berne.

This is the minimum that I'd expect if I'd have done that job. So all I  
can do now is to buy him another beer (besides your's :) next time we sit  
together in Bern(e), which will be in August, IIRC.

/Klaus

> Stef