Questions related to immutability in Cog

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Questions related to immutability in Cog

Clément Béra
 
Hello everyone,

I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:

- Does it actually work and is it used ?
- Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
- How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

Thanks ! 
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Ryan Macnak
 
Hi Clement,

Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.

Ryan

On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
 
Hello everyone,

I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:

- Does it actually work and is it used ?
- Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
- How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

Thanks ! 


Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Eliot Miranda-2
In reply to this post by Clément Béra

Hi Clément,

> On Nov 10, 2015, at 6:17 AM, Clément Bera <[hidden email]> wrote:
>
> Hello everyone,
>
> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:

Ryan's answers are good, but I'll give a little more detail.
>
> - Does it actually work and is it used ?

The scheme is identical to the VisualWorks one, and the VW version has been in use for over ten years, so the scheme definitely works.  But it's not yet implemented in Cog.

> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.

When implemented, the VM  checks the immutability bit before each write, which means before writing to an inst var or global variable when executing an inst var write bytecode or global var write bytecode, or when writing to an object in a primitive.  So the bit protects an object from any write.

The bit can also be checked before a become.  In VW the bit protects two-way become but not one-way become IIRC.

If the bit is set when a write bytecode is executed the VM sends attemptToWrite: aValue atIndex: index to the immutable object.  If the bit is set when a write in a primitive is attempted the primitive fails with a suitable error code (#'no modification' ?).

> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

It's not implemented yet.  The code to perform the checks and the send back need to be implemented.  It is not a lot of work.  Perhaps two weeks.

>
> Thanks !

_,,,^..^,,,_ (phone)
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Clément Béra
 
Thanks for your answers.

The implementation implies:
- adding support for the callback attemptToWrite: aValue atIndex: index which is partially done 
- checking for immutability at each write (at:put:, bytecoded inst var access, instVarAt:put:, become: , ... ). both in the interpreter and jitted code
- I guess adding 2 primitives to mark the object as immutable and mark it back as mutable.

I believe it's not a lot of work. If no one has done that before May (a phd student was interested at some point, he may do it) I could try to supervise an intern on this task. When he finishes the implementation he could work in actually using the immutability in the image.



2015-11-11 4:46 GMT-03:00 Eliot Miranda <[hidden email]>:

Hi Clément,

> On Nov 10, 2015, at 6:17 AM, Clément Bera <[hidden email]> wrote:
>
> Hello everyone,
>
> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:

Ryan's answers are good, but I'll give a little more detail.
>
> - Does it actually work and is it used ?

The scheme is identical to the VisualWorks one, and the VW version has been in use for over ten years, so the scheme definitely works.  But it's not yet implemented in Cog.

> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.

When implemented, the VM  checks the immutability bit before each write, which means before writing to an inst var or global variable when executing an inst var write bytecode or global var write bytecode, or when writing to an object in a primitive.  So the bit protects an object from any write.

The bit can also be checked before a become.  In VW the bit protects two-way become but not one-way become IIRC.

If the bit is set when a write bytecode is executed the VM sends attemptToWrite: aValue atIndex: index to the immutable object.  If the bit is set when a write in a primitive is attempted the primitive fails with a suitable error code (#'no modification' ?).

> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

It's not implemented yet.  The code to perform the checks and the send back need to be implemented.  It is not a lot of work.  Perhaps two weeks.

>
> Thanks !

_,,,^..^,,,_ (phone)

Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

stephane ducasse-2
In reply to this post by Eliot Miranda-2


>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>
> When implemented, the VM  checks the immutability bit before each write, which means before writing to an inst var or global variable when executing an inst var write bytecode or global var write bytecode, or when writing to an object in a primitive.  So the bit protects an object from any write.
>
> The bit can also be checked before a become.  In VW the bit protects two-way become but not one-way become IIRC.


eliot why the become?
is it when the become tackles instance variables of an object and that such object should be immutable?


> If the bit is set when a write bytecode is executed the VM sends attemptToWrite: aValue atIndex: index to the immutable object.  If the bit is set when a write in a primitive is attempted the primitive fails with a suitable error code (#'no modification' ?).

Like that we can use it for write barrier.

>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

I think that sebastian could have a look at that with clement.

Syef
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

stephane ducasse-2
In reply to this post by Clément Béra
 

On 11 Nov 2015, at 11:03, Clément Bera <[hidden email]> wrote:

Thanks for your answers.

The implementation implies:
- adding support for the callback attemptToWrite: aValue atIndex: index which is partially done 
- checking for immutability at each write (at:put:, bytecoded inst var access, instVarAt:put:, become: , ... ). both in the interpreter and jitted code
- I guess adding 2 primitives to mark the object as immutable and mark it back as mutable.

I believe it's not a lot of work. If no one has done that before May (a phd student was interested at some point, he may do it) I could try to supervise an intern on this task. When he finishes the implementation he could work in actually using the immutability in the image.

yes clement this would be good to see if seb can do it. 

Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Tudor Girba-2

Hi,

> On Nov 11, 2015, at 11:31 AM, stephane ducasse <[hidden email]> wrote:
>
>
> On 11 Nov 2015, at 11:03, Clément Bera <[hidden email]> wrote:
>
>> Thanks for your answers.
>>
>> The implementation implies:
>> - adding support for the callback attemptToWrite: aValue atIndex: index which is partially done
>> - checking for immutability at each write (at:put:, bytecoded inst var access, instVarAt:put:, become: , ... ). both in the interpreter and jitted code
>> - I guess adding 2 primitives to mark the object as immutable and mark it back as mutable.
>>
>> I believe it's not a lot of work. If no one has done that before May (a phd student was interested at some point, he may do it) I could try to supervise an intern on this task. When he finishes the implementation he could work in actually using the immutability in the image.
>
> yes clement this would be good to see if seb can do it.

The way I see it, this would be a significant contribution.

Cheers,
Doru


>

--
www.tudorgirba.com

"We cannot reach the flow of things unless we let go."



Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Tobias Pape
In reply to this post by Ryan Macnak

Hi all,

I just want to add my 2ct to this.
I'd rather like to habe the immutability bit be one-shot,
non-resettable, ie, an immutable object stays that way its entire lifetime.
I know that conflicts with the idea of the lightweight read barrier, but
I think this would be worthwhile for value classes :)

Can we find a compromise here?

Best regards
        -Tobias

On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:

> Hi Clement,
>
> Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.
>
> Ryan
>
> On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
>  
> Hello everyone,
>
> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:
>
> - Does it actually work and is it used ?
> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>
> Thanks !


Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

stephane ducasse-2

Hi tobias

I definitively would like to see the impact of having value classes.
This is a topic I mention to seb krinsky for his PhD.
I would like to know the impact of having value classes on containers.

Now with a proper Mop and API why we could not have both
temporarily and permanent immutable objects.

Stef

On 11 Nov 2015, at 11:50, Tobias Pape <[hidden email]> wrote:

>
> Hi all,
>
> I just want to add my 2ct to this.
> I'd rather like to habe the immutability bit be one-shot,
> non-resettable, ie, an immutable object stays that way its entire lifetime.
> I know that conflicts with the idea of the lightweight read barrier, but
> I think this would be worthwhile for value classes :)
>
> Can we find a compromise here?
>
> Best regards
> -Tobias
>
> On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:
>
>> Hi Clement,
>>
>> Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.
>>
>> Ryan
>>
>> On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
>>
>> Hello everyone,
>>
>> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:
>>
>> - Does it actually work and is it used ?
>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>>
>> Thanks !
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Tobias Pape


On 11.11.2015, at 13:03, stephane ducasse <[hidden email]> wrote:

>
> Hi tobias
>
> I definitively would like to see the impact of having value classes.
> This is a topic I mention to seb krinsky for his PhD.
> I would like to know the impact of having value classes on containers.
>
> Now with a proper Mop and API why we could not have both
> temporarily and permanent immutable objects.

https://media1.giphy.com/media/zbzNUbpFnlw8E/200.gif

>
> Stef
>
> On 11 Nov 2015, at 11:50, Tobias Pape <[hidden email]> wrote:
>
>>
>> Hi all,
>>
>> I just want to add my 2ct to this.
>> I'd rather like to habe the immutability bit be one-shot,
>> non-resettable, ie, an immutable object stays that way its entire lifetime.
>> I know that conflicts with the idea of the lightweight read barrier, but
>> I think this would be worthwhile for value classes :)
>>
>> Can we find a compromise here?
>>
>> Best regards
>> -Tobias
>>
>> On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:
>>
>>> Hi Clement,
>>>
>>> Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.
>>>
>>> Ryan
>>>
>>> On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
>>>
>>> Hello everyone,
>>>
>>> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:
>>>
>>> - Does it actually work and is it used ?
>>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>>>
>>> Thanks !


Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Eliot Miranda-2
In reply to this post by stephane ducasse-2

Hi Stephane,

> On Nov 11, 2015, at 2:30 AM, stephane ducasse <[hidden email]> wrote:
>
>
>
>>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>>
>> When implemented, the VM  checks the immutability bit before each write, which means before writing to an inst var or global variable when executing an inst var write bytecode or global var write bytecode, or when writing to an object in a primitive.  So the bit protects an object from any write.
>>
>> The bit can also be checked before a become.  In VW the bit protects two-way become but not one-way become IIRC.
>
>
> eliot why the become?
> is it when the become tackles instance variables of an object and that such object should be immutable?

It's more that one use of the scheme is to make sure literals are immutable and so one shouldn't just be able to become them by mistake.  Remember one can always turn off the bit so if you really want to you can become a literal.  So checking in two-way become makes sense to me.

>> If the bit is set when a write bytecode is executed the VM sends attemptToWrite: aValue atIndex: index to the immutable object.  If the bit is set when a write in a primitive is attempted the primitive fails with a suitable error code (#'no modification' ?).
>
> Like that we can use it for write barrier.

Right.  That's the most compelling use case.  For example the Gemstone system uses this to trap writes to objects.

>
>>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>
> I think that sebastian could have a look at that with clement.
>
> Syef
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Eliot Miranda-2
In reply to this post by Clément Béra
 
Hi Clément,

On Nov 11, 2015, at 2:03 AM, Clément Bera <[hidden email]> wrote:

Thanks for your answers.

The implementation implies:
- adding support for the callback attemptToWrite: aValue atIndex: index which is partially done 
- checking for immutability at each write (at:put:, bytecoded inst var access, instVarAt:put:, become: , ... ). both in the interpreter and jitted code
- I guess adding 2 primitives to mark the object as immutable and mark it back as mutable.

Right.  Note that the newspeak interpreter VM has many primitives already rewritten but some may be obsolete.


I believe it's not a lot of work. If no one has done that before May (a phd student was interested at some point, he may do it) I could try to supervise an intern on this task. When he finishes the implementation he could work in actually using the immutability in the image.

That would be gr8!




2015-11-11 4:46 GMT-03:00 Eliot Miranda <[hidden email]>:

Hi Clément,

> On Nov 10, 2015, at 6:17 AM, Clément Bera <[hidden email]> wrote:
>
> Hello everyone,
>
> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:

Ryan's answers are good, but I'll give a little more detail.
>
> - Does it actually work and is it used ?

The scheme is identical to the VisualWorks one, and the VW version has been in use for over ten years, so the scheme definitely works.  But it's not yet implemented in Cog.

> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.

When implemented, the VM  checks the immutability bit before each write, which means before writing to an inst var or global variable when executing an inst var write bytecode or global var write bytecode, or when writing to an object in a primitive.  So the bit protects an object from any write.

The bit can also be checked before a become.  In VW the bit protects two-way become but not one-way become IIRC.

If the bit is set when a write bytecode is executed the VM sends attemptToWrite: aValue atIndex: index to the immutable object.  If the bit is set when a write in a primitive is attempted the primitive fails with a suitable error code (#'no modification' ?).

> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

It's not implemented yet.  The code to perform the checks and the send back need to be implemented.  It is not a lot of work.  Perhaps two weeks.

>
> Thanks !

_,,,^..^,,,_ (phone)

Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Eliot Miranda-2
In reply to this post by Tobias Pape

Hi Tobias,

> On Nov 11, 2015, at 2:50 AM, Tobias Pape <[hidden email]> wrote:
>
>
> Hi all,
>
> I just want to add my 2ct to this.
> I'd rather like to habe the immutability bit be one-shot,
> non-resettable, ie, an immutable object stays that way its entire lifetime.
> I know that conflicts with the idea of the lightweight read barrier, but
> I think this would be worthwhile for value classes :)
>
> Can we find a compromise here?

The VM is agnostic about how the upper levels of the system uses it.  There's nothing to stop you overriding the method that sets the bit, so I think we can have both.  Just remember that the lightweight read barrier is /really/ useful for persistence schemes and that's v important to typical industrial applications.

>
> Best regards
>    -Tobias
>
>> On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:
>>
>> Hi Clement,
>>
>> Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.
>>
>> Ryan
>>
>> On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
>>
>> Hello everyone,
>>
>> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:
>>
>> - Does it actually work and is it used ?
>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>>
>> Thanks !
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Tobias Pape


On 11.11.2015, at 16:21, Eliot Miranda <[hidden email]> wrote:

>
> Hi Tobias,
>
>> On Nov 11, 2015, at 2:50 AM, Tobias Pape <[hidden email]> wrote:
>>
>>
>> Hi all,
>>
>> I just want to add my 2ct to this.
>> I'd rather like to habe the immutability bit be one-shot,
>> non-resettable, ie, an immutable object stays that way its entire lifetime.
>> I know that conflicts with the idea of the lightweight read barrier, but
>> I think this would be worthwhile for value classes :)
>>
>> Can we find a compromise here?
>
> The VM is agnostic about how the upper levels of the system uses it.  There's nothing to stop you overriding the method that sets the bit, so I think we can have both.  Just remember that the lightweight read barrier is /really/ useful for persistence schemes and that's v important to typical industrial applications.
>

I understand it is important, so maybe we should look at how make a read barrier a proper citizen in our town?


(Ramblings:
My feeling now is that the on/off-immutability is primarily useful for that case (which is fine), but I can imagine cases where immutable immutability is more favourable, like Accounting: when I only can create but not _modify_ any object, no changes are lost (as I, for example, can only create newer versions of an object but not actually modify them).
Also, when immutable objects are actually immutable, I can rely on that fact; nobody can change them under the hood.
I as a VM developer then can speculate on that fact.

Well, maybe I also want a stronger concept here (viz. Value classes), and be able to map identity to value equality. But I'd need immutable immutability, nonetheless…)

So, I'd be more happy if we'd name the temporary immutability "write-locked" or so, because a lock can be lifted, actual immutability is perpetual…
I can live with a write-lock bit :D


Best regards
        -Tobias

(my fingers always want to write immutablity… does anyone know that word?)


>>
>> Best regards
>>   -Tobias
>>
>>> On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:
>>>
>>> Hi Clement,
>>>
>>> Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.
>>>
>>> Ryan
>>>
>>> On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
>>>
>>> Hello everyone,
>>>
>>> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:
>>>
>>> - Does it actually work and is it used ?
>>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>>>
>>> Thanks !


cbc
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

cbc
 
 immutablity - and immutable bitblit?

Depending on  how paranoid you want to be with the accounting, should you also have an ungarbagecollector bit as well, so that you can't forget the entry and re-enter it later?

Just curious,
cbc

On Wed, Nov 11, 2015 at 7:34 AM, Tobias Pape <[hidden email]> wrote:


On 11.11.2015, at 16:21, Eliot Miranda <[hidden email]> wrote:

>
> Hi Tobias,
>
>> On Nov 11, 2015, at 2:50 AM, Tobias Pape <[hidden email]> wrote:
>>
>>
>> Hi all,
>>
>> I just want to add my 2ct to this.
>> I'd rather like to habe the immutability bit be one-shot,
>> non-resettable, ie, an immutable object stays that way its entire lifetime.
>> I know that conflicts with the idea of the lightweight read barrier, but
>> I think this would be worthwhile for value classes :)
>>
>> Can we find a compromise here?
>
> The VM is agnostic about how the upper levels of the system uses it.  There's nothing to stop you overriding the method that sets the bit, so I think we can have both.  Just remember that the lightweight read barrier is /really/ useful for persistence schemes and that's v important to typical industrial applications.
>

I understand it is important, so maybe we should look at how make a read barrier a proper citizen in our town?


(Ramblings:
My feeling now is that the on/off-immutability is primarily useful for that case (which is fine), but I can imagine cases where immutable immutability is more favourable, like Accounting: when I only can create but not _modify_ any object, no changes are lost (as I, for example, can only create newer versions of an object but not actually modify them).
Also, when immutable objects are actually immutable, I can rely on that fact; nobody can change them under the hood.
I as a VM developer then can speculate on that fact.

Well, maybe I also want a stronger concept here (viz. Value classes), and be able to map identity to value equality. But I'd need immutable immutability, nonetheless…)

So, I'd be more happy if we'd name the temporary immutability "write-locked" or so, because a lock can be lifted, actual immutability is perpetual…
I can live with a write-lock bit :D


Best regards
        -Tobias

(my fingers always want to write immutablity… does anyone know that word?)


>>
>> Best regards
>>   -Tobias
>>
>>> On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:
>>>
>>> Hi Clement,
>>>
>>> Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.
>>>
>>> Ryan
>>>
>>> On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:
>>>
>>> Hello everyone,
>>>
>>> I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:
>>>
>>> - Does it actually work and is it used ?
>>> - Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
>>> - How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?
>>>
>>> Thanks !



Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

stephane ducasse-2
In reply to this post by Tobias Pape
 


The VM is agnostic about how the upper levels of the system uses it.  There's nothing to stop you overriding the method that sets the bit, so I think we can have both.  Just remember that the lightweight read barrier is /really/ useful for persistence schemes and that's v important to typical industrial applications.


I understand it is important, so maybe we should look at how make a read barrier a proper citizen in our town?


(Ramblings: 
My feeling now is that the on/off-immutability is primarily useful for that case (which is fine), but I can imagine cases where immutable immutability is more favourable, like Accounting: when I only can create but not _modify_ any object, no changes are lost (as I, for example, can only create newer versions of an object but not actually modify them).
Also, when immutable objects are actually immutable, I can rely on that fact; nobody can change them under the hood. 
I as a VM developer then can speculate on that fact. 

Well, maybe I also want a stronger concept here (viz. Value classes), and be able to map identity to value equality. But I'd need immutable immutability, nonetheless…)

I would really like to see what you can do with that. 
But I have the impression that we will not have two bits to express both :)

Stef


So, I'd be more happy if we'd name the temporary immutability "write-locked" or so, because a lock can be lifted, actual immutability is perpetual…
I can live with a write-lock bit :D


Best regards
-Tobias

(my fingers always want to write immutablity… does anyone know that word?)



Best regards
 -Tobias

On 10.11.2015, at 17:35, Ryan Macnak <[hidden email]> wrote:

Hi Clement,

Per object immutability was implemented in the old, pre-Cog Newspeak VM (NewspeakInterpreter). It has not yet been implemented current Cog Newspeak VM (Stack/CoInterpreter + NewspeakVM=true), but the necessary bit is reserved in the object header. It was used as part of an orthogonal synchronization scheme to detect changed objects: mark synchronized objects as immutable, trap on change attempt, add to dirty list and mark as mutable again. A marked object is only shallowly immutable, and its instance variables may contain mutable objects.

Ryan

On Tue, Nov 10, 2015 at 6:17 AM, Clément Bera <[hidden email]> wrote:

Hello everyone,

I tried to understand how immutability is implemented in the CogVM (NewSpeak flavor). I have some questions:

- Does it actually work and is it used ?
- Does an object being immutable means that you can't edit its variable fields only or that you can't either edit its instance variables ? Because I don't see any code checking for instance variable stores in the Newspeak interpreter.
- How does the immutability check works in machine code ? In the Newspeak interpreter there is an immutability check in #at:put: . In the at:put: generated by the JIT (for example CogObjectRepresentationFor32BitSpur) I don't see any immutability check. Did I miss some clever bit trick ? Does the Newspeak spur VMs use the #at:put: machine code version of the primitive ?

Thanks !

Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Colin Putney-3
In reply to this post by Tobias Pape
 


On Wed, Nov 11, 2015 at 7:34 AM, Tobias Pape <[hidden email]> wrote:
 
So, I'd be more happy if we'd name the temporary immutability "write-locked" or so, because a lock can be lifted, actual immutability is perpetual…
I can live with a write-lock bit :D

Yeah, I'd like to see this too. I think there are 3 interesting cases here:


Write Barrier

This what VW (mistakenly) calls "immutability". As Eliot described up thread, each object has a bit can be set to trap writes. When the bit is set, any primitive that modifies the object fails, and any bytecode that modifies the object instead sends #attemptToWrite:atIndex:. It lets writes be intercepted by image-level code, which is useful for external persistence mechanisms, eg Gemstone.

An alternate implementation might be to just set a bit whenever a write operation occurs, and then have primitives for testing and clearing the bit explicitly. Persistence mechanisms tend to just want to know when an object has changed, and don't really need to intercept the change before it happens, so this would be sufficient. 


Read Barrier

Similar to the write barrier, but for reads. Again, I could imagine two ways of doing it. One would be to trap reads by sending something like #attemptToRead:atIndex: and then having the read complete with whatever that method returns. Another would be to set a bit on every read, and have primitives for testing and clearing it.

Persistence code typically creates stub objects that "fault" when they receive a message, pull in data from outside the image, then become the appropriate object. Having a read barrier of the #attemptToRead:atIndex: sort would let the object be created with the correct class, and then populated with state when it's actually needed. 

The other implementation with a bit that keeps track of which objects have been read would be useful for other cases. One example I ran into is in Altitude. Managing state is the essence of web applications, and getting good performance and scalability requires caching of state at several different levels of the system. One thing I was trying to do was generate ETag headers based on which model objects were read during the rendering of a particular resource, so that the ETag values would change iff the state of the model objects that affected the rendered data changed. 

One might do the same in the case of an IDE, for example. The visual state of a given browser window depends on the state of a certain set of objects in the image - classes, methods, organizers etc. By knowing *which* objects contributed to a given view, we can know that we have to update the view when, and only when, those objects change. 


Immutability

True immutability, not the VW version. I'd like to see this implemented as a property of classes. Immutable classes create instances via a new primitive that instantiates and initializes the object in one step. After that, the object cannot be modified. Having that invariant would be useful both in the VM and in the image. 

In the VM, we'd be free to take liberties with object identity. As Tobias mentioned we could have "value" object that get copied rather than passed as references whenever that makes sense. We might inline value objects within other objects to avoid chasing pointers. We might reorganize arrays of value objects into arrays of their individual fields where that would be useful for performance. The GC might merge all references to identical value objects during heap compaction.

In the image, we could share value objects between processes without worrying about synchronization. We could copy them over the network to make remote messaging more efficient. Persistence frameworks could read and write clusters of value objects along with their owning entities, rather than faulting them in one at a time. 

Even for more prosaic applications, immutable objects are pretty useful. I find I often create pseudo-immutable classes, where the immutability isn't really enforced, but it happens that the only method that modifies an object's state is the initializer. Having that enforced by the VM would be create, and would let the image-level tools be smart about it as well. A browser might warn you if you accept a method that does a write on an immutable class.


Anyway, I'm not trying to set the agenda for VM development, but could we please avoid Cincom's naming mistake and keep open the possibility of true immutability?

-Colin
Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

stephane ducasse-2
 
Thanks Colin. 
Yes I would like the same as you ;)

Stef

On 11 Nov 2015, at 18:57, Colin Putney <[hidden email]> wrote:



On Wed, Nov 11, 2015 at 7:34 AM, Tobias Pape <[hidden email]> wrote:
 
So, I'd be more happy if we'd name the temporary immutability "write-locked" or so, because a lock can be lifted, actual immutability is perpetual…
I can live with a write-lock bit :D

Yeah, I'd like to see this too. I think there are 3 interesting cases here:


Write Barrier

This what VW (mistakenly) calls "immutability". As Eliot described up thread, each object has a bit can be set to trap writes. When the bit is set, any primitive that modifies the object fails, and any bytecode that modifies the object instead sends #attemptToWrite:atIndex:. It lets writes be intercepted by image-level code, which is useful for external persistence mechanisms, eg Gemstone.

An alternate implementation might be to just set a bit whenever a write operation occurs, and then have primitives for testing and clearing the bit explicitly. Persistence mechanisms tend to just want to know when an object has changed, and don't really need to intercept the change before it happens, so this would be sufficient. 


Read Barrier

Similar to the write barrier, but for reads. Again, I could imagine two ways of doing it. One would be to trap reads by sending something like #attemptToRead:atIndex: and then having the read complete with whatever that method returns. Another would be to set a bit on every read, and have primitives for testing and clearing it.

Persistence code typically creates stub objects that "fault" when they receive a message, pull in data from outside the image, then become the appropriate object. Having a read barrier of the #attemptToRead:atIndex: sort would let the object be created with the correct class, and then populated with state when it's actually needed. 

The other implementation with a bit that keeps track of which objects have been read would be useful for other cases. One example I ran into is in Altitude. Managing state is the essence of web applications, and getting good performance and scalability requires caching of state at several different levels of the system. One thing I was trying to do was generate ETag headers based on which model objects were read during the rendering of a particular resource, so that the ETag values would change iff the state of the model objects that affected the rendered data changed. 

One might do the same in the case of an IDE, for example. The visual state of a given browser window depends on the state of a certain set of objects in the image - classes, methods, organizers etc. By knowing *which* objects contributed to a given view, we can know that we have to update the view when, and only when, those objects change. 


Immutability

True immutability, not the VW version. I'd like to see this implemented as a property of classes. Immutable classes create instances via a new primitive that instantiates and initializes the object in one step. After that, the object cannot be modified. Having that invariant would be useful both in the VM and in the image. 

In the VM, we'd be free to take liberties with object identity. As Tobias mentioned we could have "value" object that get copied rather than passed as references whenever that makes sense. We might inline value objects within other objects to avoid chasing pointers. We might reorganize arrays of value objects into arrays of their individual fields where that would be useful for performance. The GC might merge all references to identical value objects during heap compaction.

In the image, we could share value objects between processes without worrying about synchronization. We could copy them over the network to make remote messaging more efficient. Persistence frameworks could read and write clusters of value objects along with their owning entities, rather than faulting them in one at a time. 

Even for more prosaic applications, immutable objects are pretty useful. I find I often create pseudo-immutable classes, where the immutability isn't really enforced, but it happens that the only method that modifies an object's state is the initializer. Having that enforced by the VM would be create, and would let the image-level tools be smart about it as well. A browser might warn you if you accept a method that does a write on an immutable class.


Anyway, I'm not trying to set the agenda for VM development, but could we please avoid Cincom's naming mistake and keep open the possibility of true immutability?

-Colin

Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

timrowledge
In reply to this post by cbc


> On 11-11-2015, at 7:47 AM, Chris Cunningham <[hidden email]> wrote:
>
>  immutablity - an immutable bitblit?

exactly what I was thinking :-)

A couple of other immutability thoughts;
copying an imm. (I refuse to type it out every time) object - the copy primitive would need to clear the imm. bit unless we declare that a copy is also imm. In which case only ST code copying could be used to make non-imm. copies.
we’re SOL anytime any external prim or call is passed the guts of an imm. object.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
A hacker does for love what others would not do for money.


Reply | Threaded
Open this post in threaded view
|

Re: Questions related to immutability in Cog

Clément Béra
 
Hi all,

I am willing to have this done because I need it for the sista production version (I need the literals not to be able to be become).

So my questions are:
- Is Sebastian or someone going to do this ? Else I will try to find a student for next summer or do it myself.
- We agreed that what we build is a write barrier and not immutability. Hence, how should I call the methods marking the object as non writable ? #markAsNonWritable, #markAsWritable, #isWritable ? The call back selector for instance variables will remain #attemptToWrite: value atIndex: index.

If I do it or supervise someone to do it, the first version will be a VM compilation option because there will be performance loss with this feature due to inst var stores and at:put: being slowed (we expect 5% overhead).

The scheme works for all objects but contexts. I think for contexts I will just forbid to mark them as non writable for now. In the future we could divorce and discard the frame, mark the context as non writeable, and returns across stack page would handle differently returns to non writable contexts (for example we could ask the image to interpret them). I guess it could make sense in some cases.
Best,

Clement

 

2015-11-11 15:30 GMT-03:00 tim Rowledge <[hidden email]>:


> On 11-11-2015, at 7:47 AM, Chris Cunningham <[hidden email]> wrote:
>
>  immutablity - an immutable bitblit?

exactly what I was thinking :-)

A couple of other immutability thoughts;
copying an imm. (I refuse to type it out every time) object - the copy primitive would need to clear the imm. bit unless we declare that a copy is also imm. In which case only ST code copying could be used to make non-imm. copies.
we’re SOL anytime any external prim or call is passed the guts of an imm. object.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
A hacker does for love what others would not do for money.



12