How to copy a CompiledMethod?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

How to copy a CompiledMethod?

Gabriel Hernán Barbuto
Hi

I am trying to make a complete duplicate of a CompiledMethod. I
started with CompiledMethod>>#copyWithTrailerBytes: but I think that
this way still leaves some things to copy. It duplicates all the
literals, but I think that it doesn't duplicate a Pragma if for
example, the method has one.

Is there a proper way to completely make deep copy of a
CompiledMethod? I mean, without writing my own. If there is a way to
do it, I don't want to duplicate it.

Thanks in advance.
Gabriel

Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Igor Stasenko
On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]> wrote:
> Hi
>
> I am trying to make a complete duplicate of a CompiledMethod. I
> started with CompiledMethod>>#copyWithTrailerBytes: but I think that
> this way still leaves some things to copy. It duplicates all the
> literals, but I think that it doesn't duplicate a Pragma if for
> example, the method has one.
>
AFAIR, pragmas are stored as one of literals.
So, it actually should also make a copy of pragmas.

> Is there a proper way to completely make deep copy of a
> CompiledMethod? I mean, without writing my own. If there is a way to
> do it, I don't want to duplicate it.
>
if you tell more, why #copy is not enough for you.

> Thanks in advance.
> Gabriel
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Gabriel Hernán Barbuto
Hi Igor

Thanks for your response. I am quite a new to all this low level
stuff. So please bear with me if I am missing something obvious.

I need to make a deep copy of a method dictionary. I was using copy
but CompiledMethod does not define copy, and so it inherits it from
Object. Object's copy does a shallowCopy followed by a postCopy. None
of these methods is defined by CompiledMethod. So if I use that I will
only get a shallow copy of the CompiledMethod instead of a deep copy.

For example, if I try the following code snippet:

| source copy |
source := Object methodDict at: #at:.
copy := source copy.
Transcript show: 'Equals: '; show: source = copy; cr.
Transcript show: 'Same: '; show: source == copy; cr.
Transcript show: 'Literal at: '; show: source numLiterals - 1; show: ' '.
Transcript show: (source literalAt: source numLiterals - 1); cr.
Transcript show: 'Same AdditionalMethodState: ';
        show: (source literalAt: source numLiterals - 1) == (copy literalAt:
copy numLiterals -1); cr.

I get the following results on Transcript:

Equals: true
Same: false
Literal at: 8 an AdditionalMethodState
Same AdditionalMethodState: true

Here the literal in position 8, an AdditionalMethodState is not being
duplicated. Of course, I selected number 8 because that is the one I
am interested in, but any literal besides symbols will do.

If I replace copy with veryDeepCopy in the snippet above I get the
following results:

Equals: true
Same: true
Literal at: 8 an AdditionalMethodState
Same AdditionalMethodState: true

which are odd since I would not expect them to be the same.

With respect to Pragmas. Please, inspect Object>>#at: and you will see
that there is no pragma in the literals. The pragma is referenced by
the AdditionalMethodState literal.

So I am still looking for the correct, if there is one, way to
completely duplicate a CompiledMethod. Maybe I should take a look at
veryDeepCopy and see why it gives the results it gives and not what I
expect.

Any ideas are welcome.

Best regards
Gabriel

 The problem was that I was still getting references from outside and
I was wondering why.

On Thu, Nov 25, 2010 at 10:14 PM, Igor Stasenko <[hidden email]> wrote:

> On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]> wrote:
>> Hi
>>
>> I am trying to make a complete duplicate of a CompiledMethod. I
>> started with CompiledMethod>>#copyWithTrailerBytes: but I think that
>> this way still leaves some things to copy. It duplicates all the
>> literals, but I think that it doesn't duplicate a Pragma if for
>> example, the method has one.
>>
> AFAIR, pragmas are stored as one of literals.
> So, it actually should also make a copy of pragmas.
>
>> Is there a proper way to completely make deep copy of a
>> CompiledMethod? I mean, without writing my own. If there is a way to
>> do it, I don't want to duplicate it.
>>
> if you tell more, why #copy is not enough for you.
>
>> Thanks in advance.
>> Gabriel
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Stéphane Ducasse
In reply to this post by Igor Stasenko
May be you should try to define a postCopy method on compiledMethod.
Now I do not know if the system relies on assumption that copy = shallowcopy for compiledMethods.

Did you check the senders of verydeepcopy?

Stef
On Nov 25, 2010, at 11:57 PM, Gabriel Hernán Barbuto wrote:

> Hi Igor
>
> Thanks for your response. I am quite a new to all this low level
> stuff. So please bear with me if I am missing something obvious.
>
> I need to make a deep copy of a method dictionary. I was using copy
> but CompiledMethod does not define copy, and so it inherits it from
> Object. Object's copy does a shallowCopy followed by a postCopy. None
> of these methods is defined by CompiledMethod. So if I use that I will
> only get a shallow copy of the CompiledMethod instead of a deep copy.
>
> For example, if I try the following code snippet:
>
> | source copy |
> source := Object methodDict at: #at:.
> copy := source copy.
> Transcript show: 'Equals: '; show: source = copy; cr.
> Transcript show: 'Same: '; show: source == copy; cr.
> Transcript show: 'Literal at: '; show: source numLiterals - 1; show: ' '.
> Transcript show: (source literalAt: source numLiterals - 1); cr.
> Transcript show: 'Same AdditionalMethodState: ';
> show: (source literalAt: source numLiterals - 1) == (copy literalAt:
> copy numLiterals -1); cr.
>
> I get the following results on Transcript:
>
> Equals: true
> Same: false
> Literal at: 8 an AdditionalMethodState
> Same AdditionalMethodState: true
>
> Here the literal in position 8, an AdditionalMethodState is not being
> duplicated. Of course, I selected number 8 because that is the one I
> am interested in, but any literal besides symbols will do.
>
> If I replace copy with veryDeepCopy in the snippet above I get the
> following results:
>
> Equals: true
> Same: true
> Literal at: 8 an AdditionalMethodState
> Same AdditionalMethodState: true
>
> which are odd since I would not expect them to be the same.
>
> With respect to Pragmas. Please, inspect Object>>#at: and you will see
> that there is no pragma in the literals. The pragma is referenced by
> the AdditionalMethodState literal.
>
> So I am still looking for the correct, if there is one, way to
> completely duplicate a CompiledMethod. Maybe I should take a look at
> veryDeepCopy and see why it gives the results it gives and not what I
> expect.
>
> Any ideas are welcome.
>
> Best regards
> Gabriel
>
> The problem was that I was still getting references from outside and
> I was wondering why.
>
> On Thu, Nov 25, 2010 at 10:14 PM, Igor Stasenko <[hidden email]> wrote:
>> On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]> wrote:
>>> Hi
>>>
>>> I am trying to make a complete duplicate of a CompiledMethod. I
>>> started with CompiledMethod>>#copyWithTrailerBytes: but I think that
>>> this way still leaves some things to copy. It duplicates all the
>>> literals, but I think that it doesn't duplicate a Pragma if for
>>> example, the method has one.
>>>
>> AFAIR, pragmas are stored as one of literals.
>> So, it actually should also make a copy of pragmas.
>>
>>> Is there a proper way to completely make deep copy of a
>>> CompiledMethod? I mean, without writing my own. If there is a way to
>>> do it, I don't want to duplicate it.
>>>
>> if you tell more, why #copy is not enough for you.
>>
>>> Thanks in advance.
>>> Gabriel
>>>
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Gabriel Hernán Barbuto
No. I did something even more simpler. I looked at CompiledMethod's
veryDeepCopy and that explains the behavior. The comment states:

        "Return self.  I am always shared.  Do not record me.  Only use this
for blocks.  Normally methodDictionaries should not be copied this
way."

So, I wonder in which way should method dictionaries be copied.

Bests
Gabriel



On Fri, Nov 26, 2010 at 10:09 AM, Stéphane Ducasse
<[hidden email]> wrote:

> May be you should try to define a postCopy method on compiledMethod.
> Now I do not know if the system relies on assumption that copy = shallowcopy for compiledMethods.
>
> Did you check the senders of verydeepcopy?
>
> Stef
> On Nov 25, 2010, at 11:57 PM, Gabriel Hernán Barbuto wrote:
>
>> Hi Igor
>>
>> Thanks for your response. I am quite a new to all this low level
>> stuff. So please bear with me if I am missing something obvious.
>>
>> I need to make a deep copy of a method dictionary. I was using copy
>> but CompiledMethod does not define copy, and so it inherits it from
>> Object. Object's copy does a shallowCopy followed by a postCopy. None
>> of these methods is defined by CompiledMethod. So if I use that I will
>> only get a shallow copy of the CompiledMethod instead of a deep copy.
>>
>> For example, if I try the following code snippet:
>>
>> | source copy |
>> source := Object methodDict at: #at:.
>> copy := source copy.
>> Transcript show: 'Equals: '; show: source = copy; cr.
>> Transcript show: 'Same: '; show: source == copy; cr.
>> Transcript show: 'Literal at: '; show: source numLiterals - 1; show: ' '.
>> Transcript show: (source literalAt: source numLiterals - 1); cr.
>> Transcript show: 'Same AdditionalMethodState: ';
>>       show: (source literalAt: source numLiterals - 1) == (copy literalAt:
>> copy numLiterals -1); cr.
>>
>> I get the following results on Transcript:
>>
>> Equals: true
>> Same: false
>> Literal at: 8 an AdditionalMethodState
>> Same AdditionalMethodState: true
>>
>> Here the literal in position 8, an AdditionalMethodState is not being
>> duplicated. Of course, I selected number 8 because that is the one I
>> am interested in, but any literal besides symbols will do.
>>
>> If I replace copy with veryDeepCopy in the snippet above I get the
>> following results:
>>
>> Equals: true
>> Same: true
>> Literal at: 8 an AdditionalMethodState
>> Same AdditionalMethodState: true
>>
>> which are odd since I would not expect them to be the same.
>>
>> With respect to Pragmas. Please, inspect Object>>#at: and you will see
>> that there is no pragma in the literals. The pragma is referenced by
>> the AdditionalMethodState literal.
>>
>> So I am still looking for the correct, if there is one, way to
>> completely duplicate a CompiledMethod. Maybe I should take a look at
>> veryDeepCopy and see why it gives the results it gives and not what I
>> expect.
>>
>> Any ideas are welcome.
>>
>> Best regards
>> Gabriel
>>
>> The problem was that I was still getting references from outside and
>> I was wondering why.
>>
>> On Thu, Nov 25, 2010 at 10:14 PM, Igor Stasenko <[hidden email]> wrote:
>>> On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]> wrote:
>>>> Hi
>>>>
>>>> I am trying to make a complete duplicate of a CompiledMethod. I
>>>> started with CompiledMethod>>#copyWithTrailerBytes: but I think that
>>>> this way still leaves some things to copy. It duplicates all the
>>>> literals, but I think that it doesn't duplicate a Pragma if for
>>>> example, the method has one.
>>>>
>>> AFAIR, pragmas are stored as one of literals.
>>> So, it actually should also make a copy of pragmas.
>>>
>>>> Is there a proper way to completely make deep copy of a
>>>> CompiledMethod? I mean, without writing my own. If there is a way to
>>>> do it, I don't want to duplicate it.
>>>>
>>> if you tell more, why #copy is not enough for you.
>>>
>>>> Thanks in advance.
>>>> Gabriel
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>>
>>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Eliot Miranda-2
In reply to this post by Igor Stasenko


On Thu, Nov 25, 2010 at 1:14 PM, Igor Stasenko <[hidden email]> wrote:
On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]> wrote:
> Hi
>
> I am trying to make a complete duplicate of a CompiledMethod. I
> started with CompiledMethod>>#copyWithTrailerBytes: but I think that
> this way still leaves some things to copy. It duplicates all the
> literals, but I think that it doesn't duplicate a Pragma if for
> example, the method has one.
>
AFAIR, pragmas are stored as one of literals.
So, it actually should also make a copy of pragmas.

Yes. pragmas are effectively referenced from literals.  If a method needs ti the penultimate literal (which otherwise is the method;s selector) holds an AdditionalMethodState which refers back to its method.  This needs to be copied if one is copying a method, since the backpointer needs to refer to the owning method, not the method form which the copy was made.


> Is there a proper way to completely make deep copy of a
> CompiledMethod? I mean, without writing my own. If there is a way to
> do it, I don't want to duplicate it.
>
if you tell more, why #copy is not enough for you.

If literals are read-only (and Array/ByteArray/String/Float literals are conceptually read-only but not actually) you don't need to make a deep copy.  You only need to copy the AdditionalMethodState, if present, and the class association, if you want to install the method somewhere else.

best
Eliot

> Thanks in advance.
> Gabriel
>
>



--
Best regards,
Igor Stasenko AKA sig.


Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Gabriel Hernán Barbuto
Eliot

Thanks for your response.

On Fri, Nov 26, 2010 at 5:49 PM, Eliot Miranda <[hidden email]> wrote:

>
>
> On Thu, Nov 25, 2010 at 1:14 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]>
>> wrote:
>> > Hi
>> >
>> > I am trying to make a complete duplicate of a CompiledMethod. I
>> > started with CompiledMethod>>#copyWithTrailerBytes: but I think that
>> > this way still leaves some things to copy. It duplicates all the
>> > literals, but I think that it doesn't duplicate a Pragma if for
>> > example, the method has one.
>> >
>> AFAIR, pragmas are stored as one of literals.
>> So, it actually should also make a copy of pragmas.
>
> Yes. pragmas are effectively referenced from literals.  If a method needs ti
> the penultimate literal (which otherwise is the method;s selector) holds an
> AdditionalMethodState which refers back to its method.  This needs to be
> copied if one is copying a method, since the backpointer needs to refer to
> the owning method, not the method form which the copy was made.
>>
>> > Is there a proper way to completely make deep copy of a
>> > CompiledMethod? I mean, without writing my own. If there is a way to
>> > do it, I don't want to duplicate it.
>> >
>> if you tell more, why #copy is not enough for you.
>
> If literals are read-only (and Array/ByteArray/String/Float literals are
> conceptually read-only but not actually) you don't need to make a deep copy.
>  You only need to copy the AdditionalMethodState, if present, and the class
> association, if you want to install the method somewhere else.

Yes that was exactly what I did. I handle both Association and
AdditionalMethodState in a special way and just send copy to all the
other literals. I was not sure what I could encounter in the literals
so I opted to send copy to them. Maybe they will just return
themselves or make an unnecessary duplicate.

I have also taken care of copying the Pragmas referenced from any
AdditionalMethodState. Igor mentioned that there could be Pragmas
directly on the literals but I am not sure about that. I will have to
get all the CompiledMethods and check if anyone has a Pragma among its
literals, but that is not enough, maybe my image doesn't have any.

I understand that all the literals are conceptually read only, and
that copying anything besides the Associations, AdditionalMethodStates
and Pragmas can waste memory and CPU cycles, but I am not interested
in speed nor memory consumption right now. But I will take a note, in
case some day I need to improve the copy.

After copying all the Associations, AdditionalMethodStates and Pragmas
that I've found I fix the references of both AdditionalMethodStates
and Pragmas to the new method in order to break all references to
objects in the current image that I don't want to show up in the new
image.

Thanks again.

Best
Gabriel

> best
> Eliot
>>
>> > Thanks in advance.
>> > Gabriel
>> >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Stéphane Ducasse
In reply to this post by Eliot Miranda-2
gabriel

veryDeepCopy should be replaced by copy and postCopy.
So could you publish a version of postCopy with a couple of tests?
Because this would be good to make progress from this side.

Stef


On Nov 26, 2010, at 6:14 PM, Gabriel Hernán Barbuto wrote:

> Eliot
>
> Thanks for your response.
>
> On Fri, Nov 26, 2010 at 5:49 PM, Eliot Miranda <[hidden email]> wrote:
>>
>>
>> On Thu, Nov 25, 2010 at 1:14 PM, Igor Stasenko <[hidden email]> wrote:
>>>
>>> On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]>
>>> wrote:
>>>> Hi
>>>>
>>>> I am trying to make a complete duplicate of a CompiledMethod. I
>>>> started with CompiledMethod>>#copyWithTrailerBytes: but I think that
>>>> this way still leaves some things to copy. It duplicates all the
>>>> literals, but I think that it doesn't duplicate a Pragma if for
>>>> example, the method has one.
>>>>
>>> AFAIR, pragmas are stored as one of literals.
>>> So, it actually should also make a copy of pragmas.
>>
>> Yes. pragmas are effectively referenced from literals.  If a method needs ti
>> the penultimate literal (which otherwise is the method;s selector) holds an
>> AdditionalMethodState which refers back to its method.  This needs to be
>> copied if one is copying a method, since the backpointer needs to refer to
>> the owning method, not the method form which the copy was made.
>>>
>>>> Is there a proper way to completely make deep copy of a
>>>> CompiledMethod? I mean, without writing my own. If there is a way to
>>>> do it, I don't want to duplicate it.
>>>>
>>> if you tell more, why #copy is not enough for you.
>>
>> If literals are read-only (and Array/ByteArray/String/Float literals are
>> conceptually read-only but not actually) you don't need to make a deep copy.
>>  You only need to copy the AdditionalMethodState, if present, and the class
>> association, if you want to install the method somewhere else.
>
> Yes that was exactly what I did. I handle both Association and
> AdditionalMethodState in a special way and just send copy to all the
> other literals. I was not sure what I could encounter in the literals
> so I opted to send copy to them. Maybe they will just return
> themselves or make an unnecessary duplicate.
>
> I have also taken care of copying the Pragmas referenced from any
> AdditionalMethodState. Igor mentioned that there could be Pragmas
> directly on the literals but I am not sure about that. I will have to
> get all the CompiledMethods and check if anyone has a Pragma among its
> literals, but that is not enough, maybe my image doesn't have any.
>
> I understand that all the literals are conceptually read only, and
> that copying anything besides the Associations, AdditionalMethodStates
> and Pragmas can waste memory and CPU cycles, but I am not interested
> in speed nor memory consumption right now. But I will take a note, in
> case some day I need to improve the copy.
>
> After copying all the Associations, AdditionalMethodStates and Pragmas
> that I've found I fix the references of both AdditionalMethodStates
> and Pragmas to the new method in order to break all references to
> objects in the current image that I don't want to show up in the new
> image.
>
> Thanks again.
>
> Best
> Gabriel
>
>> best
>> Eliot
>>>
>>>> Thanks in advance.
>>>> Gabriel
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to copy a CompiledMethod?

Mariano Martinez Peck
In reply to this post by Gabriel Hernán Barbuto
Gabriel, I am also interested  in being able to completly copy a CompiledMethod. Do you have in mind to open a ticket and integrate it to Pharo?

thanks

mariano

On Fri, Nov 26, 2010 at 6:14 PM, Gabriel Hernán Barbuto <[hidden email]> wrote:
Eliot

Thanks for your response.

On Fri, Nov 26, 2010 at 5:49 PM, Eliot Miranda <[hidden email]> wrote:
>
>
> On Thu, Nov 25, 2010 at 1:14 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> On 25 November 2010 20:44, Gabriel Hernán Barbuto <[hidden email]>
>> wrote:
>> > Hi
>> >
>> > I am trying to make a complete duplicate of a CompiledMethod. I
>> > started with CompiledMethod>>#copyWithTrailerBytes: but I think that
>> > this way still leaves some things to copy. It duplicates all the
>> > literals, but I think that it doesn't duplicate a Pragma if for
>> > example, the method has one.
>> >
>> AFAIR, pragmas are stored as one of literals.
>> So, it actually should also make a copy of pragmas.
>
> Yes. pragmas are effectively referenced from literals.  If a method needs ti
> the penultimate literal (which otherwise is the method;s selector) holds an
> AdditionalMethodState which refers back to its method.  This needs to be
> copied if one is copying a method, since the backpointer needs to refer to
> the owning method, not the method form which the copy was made.
>>
>> > Is there a proper way to completely make deep copy of a
>> > CompiledMethod? I mean, without writing my own. If there is a way to
>> > do it, I don't want to duplicate it.
>> >
>> if you tell more, why #copy is not enough for you.
>
> If literals are read-only (and Array/ByteArray/String/Float literals are
> conceptually read-only but not actually) you don't need to make a deep copy.
>  You only need to copy the AdditionalMethodState, if present, and the class
> association, if you want to install the method somewhere else.

Yes that was exactly what I did. I handle both Association and
AdditionalMethodState in a special way and just send copy to all the
other literals. I was not sure what I could encounter in the literals
so I opted to send copy to them. Maybe they will just return
themselves or make an unnecessary duplicate.

I have also taken care of copying the Pragmas referenced from any
AdditionalMethodState. Igor mentioned that there could be Pragmas
directly on the literals but I am not sure about that. I will have to
get all the CompiledMethods and check if anyone has a Pragma among its
literals, but that is not enough, maybe my image doesn't have any.

I understand that all the literals are conceptually read only, and
that copying anything besides the Associations, AdditionalMethodStates
and Pragmas can waste memory and CPU cycles, but I am not interested
in speed nor memory consumption right now. But I will take a note, in
case some day I need to improve the copy.

After copying all the Associations, AdditionalMethodStates and Pragmas
that I've found I fix the references of both AdditionalMethodStates
and Pragmas to the new method in order to break all references to
objects in the current image that I don't want to show up in the new
image.

Thanks again.

Best
Gabriel

> best
> Eliot
>>
>> > Thanks in advance.
>> > Gabriel
>> >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>