Issue with traits

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue with traits

Igor Stasenko
Create a trait:

Trait named: #ABBA
        uses: {}
        category: 'ABBA'.

- add a method #foo to ABBA trait

Then create 2 classes which using it:

Object subclass: #ABBAB
        uses: ABBA
        instanceVariableNames: ''
        classVariableNames: ''
        poolDictionaries: ''
        category: 'ABBA'.

Object subclass: #ABBAC
        uses: ABBA
        instanceVariableNames: ''
        classVariableNames: ''
        poolDictionaries: ''
        category: 'ABBA'.


now what i found interesting:

(ABBAB methodDict at: #foo) == (ABBAC methodDict at: #foo)

but as side effect, we got this one:

(ABBAB methodDict at: #foo) methodClass  ==> ABBAC

which i find a bit disturbing :(

I'm not sure if given issue could have a major impact somewhere...
But i would just #clone the trait method before installing it..
Yes, it takes more space, but it makes each of such methods unrelated.

And i strongly assume that these methods should be unrelated, and
moreover, recompiled for each class separately, because
global var in one class could be a class var in another one, so a
trait method, like:

foo
  ^ Bar

will behave depending on class environment.


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Issue with traits

Andreas.Raab
Igor Stasenko wrote:
> now what i found interesting:
>
> (ABBAB methodDict at: #foo) == (ABBAC methodDict at: #foo)
>
> but as side effect, we got this one:
>
> (ABBAB methodDict at: #foo) methodClass  ==> ABBAC
>
> which i find a bit disturbing :(

Yes, seems wrong. VERY wrong. Try this:

Object subclass: #BaseA

BaseA>>foo
   ^42

Object subclass: #BaseB

BaseB>>foo
   ^13

And then create subclasses of BaseA and BaseB which use your trait and
have the trait implementation use "^super foo". One of two things should
happen:

a) They both return the same (incorrect) value (which would be expected
if methodClass is the same). In this case it's just plain broken.

b) They both return different values, in which case there is code that
handles super sends specifically and the sharing is simply an
unnecessary optimization.

I hope it's the latter case which would mean you only need to throw out
the test for super sends and do whatever that code was doing to begin
with ;-)

Cheers,
   - Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Re: Issue with traits

Igor Stasenko
2009/10/21 Andreas Raab <[hidden email]>:

> Igor Stasenko wrote:
>>
>> now what i found interesting:
>>
>> (ABBAB methodDict at: #foo) == (ABBAC methodDict at: #foo)
>>
>> but as side effect, we got this one:
>>
>> (ABBAB methodDict at: #foo) methodClass  ==> ABBAC
>>
>> which i find a bit disturbing :(
>
> Yes, seems wrong. VERY wrong. Try this:
>
> Object subclass: #BaseA
>
> BaseA>>foo
>  ^42
>
> Object subclass: #BaseB
>
> BaseB>>foo
>  ^13
>
> And then create subclasses of BaseA and BaseB which use your trait and have
> the trait implementation use "^super foo". One of two things should happen:
>
> a) They both return the same (incorrect) value (which would be expected if
> methodClass is the same). In this case it's just plain broken.
>
> b) They both return different values, in which case there is code that
> handles super sends specifically and the sharing is simply an unnecessary
> optimization.
>
> I hope it's the latter case which would mean you only need to throw out the
> test for super sends and do whatever that code was doing to begin with ;-)
>

Ouch... so VM behavior depends on correct methodClass slot. Thanks for
reminding!
I remember that methodClass plays important role (and its important to
have method and its
methodClass being in sync with class where the method installed), but
forgot the details :)

That could lead to even worse results, if you use super sends, and then
in base class , accessing the instance variables in base method. It
could corrupt the memory & crash the VM.

I hope this is already fixed, because the image where i checked it
is quite old one - 3.10.2

> Cheers,
>  - Andreas


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Re: Issue with traits

Andreas.Raab
Igor Stasenko wrote:

> Ouch... so VM behavior depends on correct methodClass slot. Thanks for
> reminding!
> I remember that methodClass plays important role (and its important to
> have method and its
> methodClass being in sync with class where the method installed), but
> forgot the details :)
>
> That could lead to even worse results, if you use super sends, and then
> in base class , accessing the instance variables in base method. It
> could corrupt the memory & crash the VM.

Yes, that's why I returned a literal (so the experiment wouldn't crash
you ;-)

> I hope this is already fixed, because the image where i checked it
> is quite old one - 3.10.2

I doubt it. Traits are just another abandoned experiment in Squeak.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Re: Issue with traits

Nicolas Cellier
2009/10/21 Andreas Raab <[hidden email]>:

> Igor Stasenko wrote:
>>
>> Ouch... so VM behavior depends on correct methodClass slot. Thanks for
>> reminding!
>> I remember that methodClass plays important role (and its important to
>> have method and its
>> methodClass being in sync with class where the method installed), but
>> forgot the details :)
>>
>> That could lead to even worse results, if you use super sends, and then
>> in base class , accessing the instance variables in base method. It
>> could corrupt the memory & crash the VM.
>
> Yes, that's why I returned a literal (so the experiment wouldn't crash you
> ;-)
>
>> I hope this is already fixed, because the image where i checked it
>> is quite old one - 3.10.2
>
> I doubt it. Traits are just another abandoned experiment in Squeak.
>

Just tried in Pharo, there is a copy.

(ABBAB methodDict at: #foo) == (ABBAC methodDict at: #foo). -> false

(ABBAB methodDict at: #foo) methodClass. -> ABBAB

At least Traits are maintained there, and pharo changes should be picked.

Nicolas

> Cheers,
>  - Andreas
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Re: Issue with traits

Adrian Lienhard
In reply to this post by Andreas.Raab
I've fixed this issue in Pharo a couple of months ago. CompiledMethods  
are not shared anymore, so methodClass returns the expected class. At  
the time I implemented traits (2004?), there were no method properties  
and hence sharing compiled methods worked -- of course except for  
super sends in which case the method was always recompiled (so the  
answer to Andreas' question below is b)).

Cheers,
Adrian


On Oct 21, 2009, at 08:42 , Andreas Raab wrote:

> Igor Stasenko wrote:
>> now what i found interesting:
>> (ABBAB methodDict at: #foo) == (ABBAC methodDict at: #foo)
>> but as side effect, we got this one:
>> (ABBAB methodDict at: #foo) methodClass  ==> ABBAC
>> which i find a bit disturbing :(
>
> Yes, seems wrong. VERY wrong. Try this:
>
> Object subclass: #BaseA
>
> BaseA>>foo
>  ^42
>
> Object subclass: #BaseB
>
> BaseB>>foo
>  ^13
>
> And then create subclasses of BaseA and BaseB which use your trait  
> and have the trait implementation use "^super foo". One of two  
> things should happen:
>
> a) They both return the same (incorrect) value (which would be  
> expected if methodClass is the same). In this case it's just plain  
> broken.
>
> b) They both return different values, in which case there is code  
> that handles super sends specifically and the sharing is simply an  
> unnecessary optimization.
>
> I hope it's the latter case which would mean you only need to throw  
> out the test for super sends and do whatever that code was doing to  
> begin with ;-)
>
> Cheers,
>  - Andreas
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Issue with traits

Andreas.Raab
In reply to this post by Nicolas Cellier
Nicolas Cellier wrote:
> Just tried in Pharo, there is a copy.
>
> (ABBAB methodDict at: #foo) == (ABBAC methodDict at: #foo). -> false
>
> (ABBAB methodDict at: #foo) methodClass. -> ABBAB
>
> At least Traits are maintained there, and pharo changes should be picked.

If you feel comfortable doing this, please do. The reason I'm calling it
an abandoned experiment is that unless I'm mistaken, there is nobody
left in the Squeak community who could fix any issues arising here. That
means that although we might be able to mechanically pick some fixes
from Pharo, there is no ownership of the code itself which means that if
there is ever a real issue we'll be left without help. Which is a risk
that I'm generally not comfortable in taking and less so if the kernel
of the system is involved. We need to own that code if we want to
develop it further.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] Re: Issue with traits

Marcus Denker-3
In reply to this post by Adrian Lienhard

On 21.10.2009, at 09:41, Adrian Lienhard wrote:

> I've fixed this issue in Pharo a couple of months ago. CompiledMethods
> are not shared anymore, so methodClass returns the expected class. At
> the time I implemented traits (2004?), there were no method properties
> and hence sharing compiled methods worked -- of course except for
> super sends in which case the method was always recompiled (so the
> answer to Andreas' question below is b)).
>

We did a small paper for this years ESUG that describes the changes.
It will be online at esug.org soon with all the other papers.

If somebody wants a copy, I can send it.

        Marcus

--
Marcus Denker  --  [hidden email]
http://www.2denker.de

2denker UG (haftungsbeschränkt)
Sitz Köln
Amtsgericht - Registergericht - Köln, HRB 65065
Geschäftsführer: Christian Denker und Dr. Marcus Denker





Reply | Threaded
Open this post in threaded view
|

Fwd: [Pharo-project] Re: Issue with traits

cedreek


---------- Forwarded message ----------
From: Marcus Denker <[hidden email]>
Date: 2009/10/21
Subject: Re: [Pharo-project] [squeak-dev] Re: Issue with traits
To: [hidden email]
Cc: The general-purpose Squeak developers list <[hidden email]>



On 21.10.2009, at 09:41, Adrian Lienhard wrote:

> I've fixed this issue in Pharo a couple of months ago. CompiledMethods
> are not shared anymore, so methodClass returns the expected class. At
> the time I implemented traits (2004?), there were no method properties
> and hence sharing compiled methods worked -- of course except for
> super sends in which case the method was always recompiled (so the
> answer to Andreas' question below is b)).
>

We did a small paper for this years ESUG that describes the changes.
It will be online at esug.org soon with all the other papers.

If somebody wants a copy, I can send it.

       Marcus

--
Marcus Denker  --  [hidden email]
http://www.2denker.de

2denker UG (haftungsbeschränkt)
Sitz Köln
Amtsgericht - Registergericht - Köln, HRB 65065
Geschäftsführer: Christian Denker und Dr. Marcus Denker





_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project



--
Cédrick


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] Re: Issue with traits

KenDickey
In reply to this post by Igor Stasenko
Why not share the code?  Just use a "cell" to hold the class and have
methodClass accessor do the indirection.

If you have a bit in the method header, the VM could use it as a 0/1 index
into the code to call to do (or not do) the extra indirection.  Just think of
the a trait method as a closure which is parameterized on the class.

Am I missing something deep here?

-KenD

> Message: 7
> Date: Wed, 21 Oct 2009 09:41:08 +0200
> From: Adrian Lienhard <[hidden email]>
> Subject: Re: [Pharo-project] [squeak-dev] Re: Issue with traits
> To: The general-purpose Squeak developers list
>         <[hidden email]>,        Pharo Development
>         <[hidden email]>
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>
> I've fixed this issue in Pharo a couple of months ago. CompiledMethods  
> are not shared anymore, so methodClass returns the expected class. At  
> the time I implemented traits (2004?), there were no method properties  
> and hence sharing compiled methods worked -- of course except for  
> super sends in which case the method was always recompiled (so the  
> answer to Andreas' question below is b)).
>
> Cheers,
> Adrian

-KenD
Reply | Threaded
Open this post in threaded view
|

Re: Re: Issue with traits

Andreas.Raab
Ken.Dickey wrote:
> Why not share the code?  Just use a "cell" to hold the class and have
> methodClass accessor do the indirection.
>
> If you have a bit in the method header, the VM could use it as a 0/1 index
> into the code to call to do (or not do) the extra indirection.  Just think of
> the a trait method as a closure which is parameterized on the class.
>
> Am I missing something deep here?

Mostly that it's generally hard to change the VM and asking for extra
header bits is generally frowned upon because they are in short supply.
So in short, yes what you are proposing would be one possible solution
but by and large it will be *much* simpler to just not share trait
methods (which is a fix that can be implemented without VM mods).

Cheers,
   - Andreas

>
> -KenD
>
>> Message: 7
>> Date: Wed, 21 Oct 2009 09:41:08 +0200
>> From: Adrian Lienhard <adi-cTM//Nd5/[hidden email]>
>> Subject: Re: [Pharo-project] [squeak-dev] Re: Issue with traits
>> To: The general-purpose Squeak developers list
>>         <[hidden email]>,        Pharo Development
>>         <pharo-project-bM+ny+RY8h+a+bCvCPl5/[hidden email]>
>> Message-ID: <3C7E5C51-21A8-483A-81A0-40854430A43B-cTM//Nd5/[hidden email]>
>> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>>
>> I've fixed this issue in Pharo a couple of months ago. CompiledMethods  
>> are not shared anymore, so methodClass returns the expected class. At  
>> the time I implemented traits (2004?), there were no method properties  
>> and hence sharing compiled methods worked -- of course except for  
>> super sends in which case the method was always recompiled (so the  
>> answer to Andreas' question below is b)).
>>
>> Cheers,
>> Adrian


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] Re: Issue with traits

Marcus Denker-3
In reply to this post by KenDickey

On 21.10.2009, at 22:28, Ken.Dickey wrote:

> Why not share the code?  Just use a "cell" to hold the class and have
> methodClass accessor do the indirection.
>
But based on which info should this indirection be made?

The idea with a simple backpointer was done because it's already there  
in case of super-sends.
So you would need to do something special there again (the last  
literal *has to be* a class
for those methds).

> If you have a bit in the method header, the VM could use it as a 0/1  
> index
> into the code to call to do (or not do) the extra indirection.

If there is something that is a scarce resource, than it's bits in the  
method header. And
I don't get it: there is one header per object, not one per reference.  
Thus
the bit could not be diffrerent per *use* of the compiledMethod.

>  Just think of
> the a trait method as a closure which is parameterized on the class.
>
> Am I missing something deep here?
>

You overlook complexity ;-)


> -KenD
>
>> Message: 7
>> Date: Wed, 21 Oct 2009 09:41:08 +0200
>> From: Adrian Lienhard <[hidden email]>
>> Subject: Re: [Pharo-project] [squeak-dev] Re: Issue with traits
>> To: The general-purpose Squeak developers list
>>         <[hidden email]>,        Pharo  
>> Development
>>         <[hidden email]>
>> Message-ID: <[hidden email]>
>> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>>
>> I've fixed this issue in Pharo a couple of months ago.  
>> CompiledMethods
>> are not shared anymore, so methodClass returns the expected class. At
>> the time I implemented traits (2004?), there were no method  
>> properties
>> and hence sharing compiled methods worked -- of course except for
>> super sends in which case the method was always recompiled (so the
>> answer to Andreas' question below is b)).
>>
>> Cheers,
>> Adrian
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] Re: Issue with traits

Eliot Miranda-2
In reply to this post by KenDickey


On Wed, Oct 21, 2009 at 1:28 PM, Ken.Dickey <[hidden email]> wrote:
Why not share the code?  Just use a "cell" to hold the class and have
methodClass accessor do the indirection.

The methodClass is used to identify the owning class for the purposes of super sends.  IIUC, your approach would have each class reference a method with a pair, containing a backpointer to the class and a forward pointer to the method, right?

There are several objections to this, which are a matter of taste (yer makes yer choice and yer pays yer price)

- there is space overhead.  Every compiled method requires an additional two slot object.  That's about 800k in a 20 Meg image.  I just went through the effort of eliminating MethodProperties for most compiled methods in my closure compiler.

- without a JIT there is time overhead.  The pair has to be part of the VM state for the VM to access the class through it.  Either the VM has to maintain two objects (the closure and the method) on each send/return or it has to maintain the closure and pay an extra indirection on every access to the method's literals.

- with a JIT there is the complexity of having the JIT manage copying the method on a per-class basis so it can inline the closure's class reference (for methods containing super sends).  Again possible, and even to be preferred if one is approaching adaptive optimization in a particular way.  ut it is not an approach I've taken in Cog (BrouHaHa did this, VW did not.  I like to keep things simple and not do it. become and changeClass are complicated by doing it).

But there is another approach which is taken by Newspeak.  There's a paper on it and its a little complicated; too complicated for me to go into the gory details here.  But the idea is essentially to have the VM cope with two separate hierarchies, the mixin hierarchy that exists to hold code, and the class hierarchy (actually a forrest) created at runtime to run code.  Essentially at run-time a module containing mixins is instantiated to create a class hierarchy.

An instance is an instance of a class, as ever.  But a class has a slot referencing the mixin from which it was created.  The method lives in the mixin, and the method's methodClass points to the mixin.  A super send involves walking the receiver's class hierarchy to find the class whose mixin slot matches the method's methodClass/methodMixin.  The super send can then proceed from that class, which may be a superclass of the receiver's class.

Newspeak also requires special support for lexically-scoped implicit receiver sends, which again involve walking the two hierarchies to orient a send correctly.  Luckily all of Newspeak's special sends are amenable to inline cacheing, so only an interpreter pays a significant cost.  However, I think you can see that supporting this is non-trvial.  One has to reconfigure the class system to implement proper mixins, and provide new bytecode support, or at least new semantics for super sends.

I would simply duplicate trait methods containing super sends and start using Newspeak when stable and fast implementations become available.


If you have a bit in the method header, the VM could use it as a 0/1 index
into the code to call to do (or not do) the extra indirection.  Just think of
the a trait method as a closure which is parameterized on the class.

This has similar objections.  Having the VM in two states depending on the bit is expensive (in the limit one ends up wth two interpreters) and the bit still needs testing at least on each frame-building send.  Again a JIT can change the equation.  But this is non-trivial work for questionable benefit (sharing those methods which contain super sends and are provided through traits)

Am I missing something deep here?

No.  As ever, its a small matter of programing.  As Travis Griggs is fond of saying, 10 small words: if it is to be, it is up to me.


-KenD

best
Eliot
 

> Message: 7
> Date: Wed, 21 Oct 2009 09:41:08 +0200
> From: Adrian Lienhard <[hidden email]>
> Subject: Re: [Pharo-project] [squeak-dev] Re: Issue with traits
> To: The general-purpose Squeak developers list
>         <[hidden email]>,        Pharo Development
>         <[hidden email]>
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>
> I've fixed this issue in Pharo a couple of months ago. CompiledMethods  
> are not shared anymore, so methodClass returns the expected class. At  
> the time I implemented traits (2004?), there were no method properties  
> and hence sharing compiled methods worked -- of course except for  
> super sends in which case the method was always recompiled (so the  
> answer to Andreas' question below is b)).
>
> Cheers,
> Adrian




Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] Re: Issue with traits

Marcus Denker-3
In reply to this post by Marcus Denker-3

On 21.10.2009, at 11:13, Marcus Denker wrote:

>
> On 21.10.2009, at 09:41, Adrian Lienhard wrote:
>
>> I've fixed this issue in Pharo a couple of months ago.  
>> CompiledMethods
>> are not shared anymore, so methodClass returns the expected class. At
>> the time I implemented traits (2004?), there were no method  
>> properties
>> and hence sharing compiled methods worked -- of course except for
>> super sends in which case the method was always recompiled (so the
>> answer to Andreas' question below is b)).
>>
>
> We did a small paper for this years ESUG that describes the changes.
> It will be online at esug.org soon with all the other papers.
>

Here is the pdf:

http://marcusdenker.de/publications/Duca09bTraitsAPI.pdf

        Marcus



--
Marcus Denker  --  [hidden email]
http://www.2denker.de

Reply | Threaded
Open this post in threaded view
|

Re: Re: Issue with traits

Andreas.Raab
Marcus Denker wrote:
>> We did a small paper for this years ESUG that describes the changes.
>> It will be online at esug.org soon with all the other papers.
>
> Here is the pdf:
>
> http://marcusdenker.de/publications/Duca09bTraitsAPI.pdf

Thanks though it seems there isn't much detail on the sharing issue; it
appears that all that's being said is the following (for those of you
who haven't read the paper):

        5.2 Non Sharing
        The new implementation copies the method instances and
        their source code on a per use basis (for each class or trait).
        This way a compiled method has a unique selector and a
        unique class to which it belongs. The source is also not
        shared anymore since with aliases the selector of a method
        can differ from its original (in the old implementation, the
        source was patched before shown to the user). [...]

Cheers,
   - Andreas