NewCompiler load on PharoCore

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Eliot Miranda-2


On Wed, Jan 13, 2010 at 12:45 PM, Igor Stasenko <[hidden email]> wrote:
2010/1/13 Martin McClure <[hidden email]>:
> Eliot Miranda wrote:
>> A *much* better way to implement this is to support immutability in the
>> VM (I know, I know, but all the code is available in the Newspeak VM),
>> mark objects one wants to mark as dirty as immutable, catch the
>> NoModificationError exception, and proceed after having made the object
>> mutable and marked it dirty.  No creating hidden classes, no trying to
>> get change class to work for compact classes, etc.  Just a simple
>> VM-implemented write barrier that can also be used for a host of other
>> things (object-database mapping, debugging, immutable literals etc).
>
> Since object dirtying is at the core of the product I work with, and
> I've worked extensively with both methods of implementing write barriers...
>
> I very strongly agree with Eliot. *So* much nicer a way to do this.
>

It could be more efficient, in respect that you don't need to create
shadow classes.
But triggering exception leads to stack unwinding to find a handler,
which inevitably leads to deoptimizing all contexts..

Uh, not so.  In VW and Cog examining a context does _not_ deoptimize a context.  A context will be "deoptimized" (actually converted to a vanilla heap context) only if you write to other than its stack contents or sender.  i.e. a context's frame is only discarded if 
- one assigns to any of its instance variables other than sender
- the stack zone runs out of room for new frames and a stack page must be vacated, causing all frames on that page to be converted to stack contexts

So exception handling does *not* usually involve converting contexts.  It would be very slow if it did.

  
and if stack depth is high
(between point of writing attempt and hook, where magma will handle
exception), this will be much slower
than WriteBarrier implementation, described by Cris,
which checking the value in-place, without the need of touching
exception machinery.

I disagree strongly.  Martin's experience with Gemstone (and Gemstone's experience) covers over twenty years and the VM-supported immutability implementations (first in VisualAge IIRC) of Gemstone are far more performant and less problematic than the code-rewriting implementations similar to Magma.  Martin really knows what he's talking about.


Just 2 cents.

> Regards,
>
> -Martin
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Igor Stasenko
2010/1/13 Eliot Miranda <[hidden email]>:

>
>
> On Wed, Jan 13, 2010 at 12:45 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> 2010/1/13 Martin McClure <[hidden email]>:
>> > Eliot Miranda wrote:
>> >> A *much* better way to implement this is to support immutability in the
>> >> VM (I know, I know, but all the code is available in the Newspeak VM),
>> >> mark objects one wants to mark as dirty as immutable, catch the
>> >> NoModificationError exception, and proceed after having made the object
>> >> mutable and marked it dirty.  No creating hidden classes, no trying to
>> >> get change class to work for compact classes, etc.  Just a simple
>> >> VM-implemented write barrier that can also be used for a host of other
>> >> things (object-database mapping, debugging, immutable literals etc).
>> >
>> > Since object dirtying is at the core of the product I work with, and
>> > I've worked extensively with both methods of implementing write
>> > barriers...
>> >
>> > I very strongly agree with Eliot. *So* much nicer a way to do this.
>> >
>>
>> It could be more efficient, in respect that you don't need to create
>> shadow classes.
>> But triggering exception leads to stack unwinding to find a handler,
>> which inevitably leads to deoptimizing all contexts..
>
> Uh, not so.  In VW and Cog examining a context does _not_ deoptimize a
> context.  A context will be "deoptimized" (actually converted to a vanilla
> heap context) only if you write to other than its stack contents or sender.
>  i.e. a context's frame is only discarded if
> - one assigns to any of its instance variables other than sender
> - the stack zone runs out of room for new frames and a stack page must be
> vacated, causing all frames on that page to be converted to stack contexts
> So exception handling does *not* usually involve converting contexts.  It
> would be very slow if it did.
>
Ah, cool.. So, even when debugging (and hence accessing context state)
does not leads automatically to deoptimization?

>>
>> and if stack depth is high
>> (between point of writing attempt and hook, where magma will handle
>> exception), this will be much slower
>> than WriteBarrier implementation, described by Cris,
>> which checking the value in-place, without the need of touching
>> exception machinery.
>
> I disagree strongly.  Martin's experience with Gemstone (and Gemstone's
> experience) covers over twenty years and the VM-supported immutability
> implementations (first in VisualAge IIRC) of Gemstone are far more
> performant and less problematic than the code-rewriting implementations
> similar to Magma.  Martin really knows what he's talking about.

I'm not saying anything against immutability, but capturing the object
state change
seem will be less efficient.
By reading Cris description i imagine that WriteBarrier rewriting a code like:

SomeClass>>foo: newValue
  foo := newValue

to:

SomeClass*>>foo: newValue
  | old |
  old := foo.
  foo := newValue.
  old == foo ifFalse: [ Magma markAsDirty: self ]

now compare performance of evaluating:

[[[[[[ self foo: 5 ]]]]]]

with:

[[[[[[ self foo: 5 ]]]]]] on: ImmutableException do: [:ex |  Magma
markAsDirty: ex receiver. ex receiver beMutable. ex resume ]

where [[[[[]]]]]] is a call stack, which can be deeeeep.

>>
>> Just 2 cents.
>>
>> > Regards,
>> >
>> > -Martin
>> >
>> >
>> > _______________________________________________
>> > Pharo-project mailing list
>> > [hidden email]
>> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Igor Stasenko
2010/1/13 Igor Stasenko <[hidden email]>:

> 2010/1/13 Eliot Miranda <[hidden email]>:
>>
>>
>> On Wed, Jan 13, 2010 at 12:45 PM, Igor Stasenko <[hidden email]> wrote:
>>>
>>> 2010/1/13 Martin McClure <[hidden email]>:
>>> > Eliot Miranda wrote:
>>> >> A *much* better way to implement this is to support immutability in the
>>> >> VM (I know, I know, but all the code is available in the Newspeak VM),
>>> >> mark objects one wants to mark as dirty as immutable, catch the
>>> >> NoModificationError exception, and proceed after having made the object
>>> >> mutable and marked it dirty.  No creating hidden classes, no trying to
>>> >> get change class to work for compact classes, etc.  Just a simple
>>> >> VM-implemented write barrier that can also be used for a host of other
>>> >> things (object-database mapping, debugging, immutable literals etc).
>>> >
>>> > Since object dirtying is at the core of the product I work with, and
>>> > I've worked extensively with both methods of implementing write
>>> > barriers...
>>> >
>>> > I very strongly agree with Eliot. *So* much nicer a way to do this.
>>> >
>>>
>>> It could be more efficient, in respect that you don't need to create
>>> shadow classes.
>>> But triggering exception leads to stack unwinding to find a handler,
>>> which inevitably leads to deoptimizing all contexts..
>>
>> Uh, not so.  In VW and Cog examining a context does _not_ deoptimize a
>> context.  A context will be "deoptimized" (actually converted to a vanilla
>> heap context) only if you write to other than its stack contents or sender.
>>  i.e. a context's frame is only discarded if
>> - one assigns to any of its instance variables other than sender
>> - the stack zone runs out of room for new frames and a stack page must be
>> vacated, causing all frames on that page to be converted to stack contexts
>> So exception handling does *not* usually involve converting contexts.  It
>> would be very slow if it did.
>>
> Ah, cool.. So, even when debugging (and hence accessing context state)
> does not leads automatically to deoptimization?
>
>>>
>>> and if stack depth is high
>>> (between point of writing attempt and hook, where magma will handle
>>> exception), this will be much slower
>>> than WriteBarrier implementation, described by Cris,
>>> which checking the value in-place, without the need of touching
>>> exception machinery.
>>
>> I disagree strongly.  Martin's experience with Gemstone (and Gemstone's
>> experience) covers over twenty years and the VM-supported immutability
>> implementations (first in VisualAge IIRC) of Gemstone are far more
>> performant and less problematic than the code-rewriting implementations
>> similar to Magma.  Martin really knows what he's talking about.
>
> I'm not saying anything against immutability, but capturing the object
> state change
> seem will be less efficient.
> By reading Cris description i imagine that WriteBarrier rewriting a code like:
>
> SomeClass>>foo: newValue
>  foo := newValue
>
> to:
>
> SomeClass*>>foo: newValue
>  | old |
>  old := foo.
>  foo := newValue.
>  old == foo ifFalse: [ Magma markAsDirty: self ]
>
> now compare performance of evaluating:
>
> [[[[[[ self foo: 5 ]]]]]]
>
> with:
>
> [[[[[[ self foo: 5 ]]]]]] on: ImmutableException do: [:ex |  Magma
> markAsDirty: ex receiver. ex receiver beMutable. ex resume ]
>
> where [[[[[]]]]]] is a call stack, which can be deeeeep.
>

A stack unwinding, actually could be avoided, if we suppose that VM
signaling the exception , by sending
SomeExceptionSpecialObject>>#signal: immutableObject

Then Magma could hook there and override that method, to something like:
SomeExceptionSpecialObject>>#signal: immutableObject
   (Magma watchingOverThisObject: immutableObject) ifFalse: [ ^ self
new signal: immutableObject ].
   Magma markAsDirty: immutableObject.
   immutableObject beMutable.
   "retry code here"


--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Eliot Miranda-2
In reply to this post by Igor Stasenko


On Wed, Jan 13, 2010 at 1:29 PM, Igor Stasenko <[hidden email]> wrote:
2010/1/13 Eliot Miranda <[hidden email]>:
>
>
> On Wed, Jan 13, 2010 at 12:45 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> 2010/1/13 Martin McClure <[hidden email]>:
>> > Eliot Miranda wrote:
>> >> A *much* better way to implement this is to support immutability in the
>> >> VM (I know, I know, but all the code is available in the Newspeak VM),
>> >> mark objects one wants to mark as dirty as immutable, catch the
>> >> NoModificationError exception, and proceed after having made the object
>> >> mutable and marked it dirty.  No creating hidden classes, no trying to
>> >> get change class to work for compact classes, etc.  Just a simple
>> >> VM-implemented write barrier that can also be used for a host of other
>> >> things (object-database mapping, debugging, immutable literals etc).
>> >
>> > Since object dirtying is at the core of the product I work with, and
>> > I've worked extensively with both methods of implementing write
>> > barriers...
>> >
>> > I very strongly agree with Eliot. *So* much nicer a way to do this.
>> >
>>
>> It could be more efficient, in respect that you don't need to create
>> shadow classes.
>> But triggering exception leads to stack unwinding to find a handler,
>> which inevitably leads to deoptimizing all contexts..
>
> Uh, not so.  In VW and Cog examining a context does _not_ deoptimize a
> context.  A context will be "deoptimized" (actually converted to a vanilla
> heap context) only if you write to other than its stack contents or sender.
>  i.e. a context's frame is only discarded if
> - one assigns to any of its instance variables other than sender
> - the stack zone runs out of room for new frames and a stack page must be
> vacated, causing all frames on that page to be converted to stack contexts
> So exception handling does *not* usually involve converting contexts.  It
> would be very slow if it did.
>
Ah, cool.. So, even when debugging (and hence accessing context state)
does not leads automatically to deoptimization?

>>
>> and if stack depth is high
>> (between point of writing attempt and hook, where magma will handle
>> exception), this will be much slower
>> than WriteBarrier implementation, described by Cris,
>> which checking the value in-place, without the need of touching
>> exception machinery.
>
> I disagree strongly.  Martin's experience with Gemstone (and Gemstone's
> experience) covers over twenty years and the VM-supported immutability
> implementations (first in VisualAge IIRC) of Gemstone are far more
> performant and less problematic than the code-rewriting implementations
> similar to Magma.  Martin really knows what he's talking about.

I'm not saying anything against immutability, but capturing the object
state change
seem will be less efficient.
By reading Cris description i imagine that WriteBarrier rewriting a code like:

SomeClass>>foo: newValue
 foo := newValue

to:

SomeClass*>>foo: newValue
 | old |
 old := foo.
 foo := newValue.
 old == foo ifFalse: [ Magma markAsDirty: self ]

now compare performance of evaluating:

[[[[[[ self foo: 5 ]]]]]]

with:

[[[[[[ self foo: 5 ]]]]]] on: ImmutableException do: [:ex |  Magma
markAsDirty: ex receiver. ex receiver beMutable. ex resume ]

where [[[[[]]]]]] is a call stack, which can be deeeeep.

1. Yes, but there are lots of other ways to go about it.  One scheme (which I think Gemstone uses as well the immutability stuff for Newspeak) is to have a policy object control what happens when a NoModificationError occurs.  Its function is to decide what to do for the immutable object.  It can do things like maintain a mapping from objects (or blocks evaluated on objects) to handlers.  Only if there is no handler found for an object is the exception actually delivered.

Another scheme is to allow objects to override the basic immutability machinery that by default delivers a NoModificationException.  Immutability violations come into the image in two ways.  If an attempt is made to assign an inst var the Vm sends attemptToAssign: aValue withIndex: instVarIndex to the immutable object.  If an attempt is made to assign to an object in a primitive the primitive fails with a #'no modification error' error code.  In Object there is a default:

noModificationError: selector arguments: arguments
^NoModificationError
signal: self
message: (Message selector: selector arguments: arguments)


attemptToAssign: aValue withIndex: index
"The VM sends this message to objects when attempts are made to assign to inst vars of immutable objects"
^self noModificationError: #instVarAt:put: arguments: {index. aValue}

at: index put: aValue 
<primitive: 61> "primitiveAtPut" 
index isInteger ifTrue:
[self class isVariable
ifTrue: [(index >= 1 and: [index <= self size])
ifTrue: [self isImmutable ifTrue:
[^ self noModificationError: #at:put: arguments: {index. aValue}].
self errorImproperStore]
ifFalse: [self errorSubscriptBounds: index]]
ifFalse: [self errorNotIndexable]].
index isNumber ifTrue:
[^ self at: index asInteger put: aValue].
self errorNonIntegerIndex

So a specific class can implement noModificationError:arguments: to short cut even delivering the exception imn the first place.


2. Yes, but it doesn't cost as much as you think.  If one is using immutability to mark objects dirty then the exception only gets delivered once for that object.  Once the object is dirty it can remain mutable until a batch of objects are updated, etc.


Martin could you say something about the dispatch schemes Gemstone uses and what the performance issues have been in practice?  (TIA)


best
Eliot


>>
>> Just 2 cents.
>>
>> > Regards,
>> >
>> > -Martin
>> >
>> >
>> > _______________________________________________
>> > Pharo-project mailing list
>> > [hidden email]
>> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Eliot Miranda-2
In reply to this post by Igor Stasenko


On Wed, Jan 13, 2010 at 1:41 PM, Igor Stasenko <[hidden email]> wrote:
2010/1/13 Igor Stasenko <[hidden email]>:
> 2010/1/13 Eliot Miranda <[hidden email]>:
>>
>>
>> On Wed, Jan 13, 2010 at 12:45 PM, Igor Stasenko <[hidden email]> wrote:
>>>
>>> 2010/1/13 Martin McClure <[hidden email]>:
>>> > Eliot Miranda wrote:
>>> >> A *much* better way to implement this is to support immutability in the
>>> >> VM (I know, I know, but all the code is available in the Newspeak VM),
>>> >> mark objects one wants to mark as dirty as immutable, catch the
>>> >> NoModificationError exception, and proceed after having made the object
>>> >> mutable and marked it dirty.  No creating hidden classes, no trying to
>>> >> get change class to work for compact classes, etc.  Just a simple
>>> >> VM-implemented write barrier that can also be used for a host of other
>>> >> things (object-database mapping, debugging, immutable literals etc).
>>> >
>>> > Since object dirtying is at the core of the product I work with, and
>>> > I've worked extensively with both methods of implementing write
>>> > barriers...
>>> >
>>> > I very strongly agree with Eliot. *So* much nicer a way to do this.
>>> >
>>>
>>> It could be more efficient, in respect that you don't need to create
>>> shadow classes.
>>> But triggering exception leads to stack unwinding to find a handler,
>>> which inevitably leads to deoptimizing all contexts..
>>
>> Uh, not so.  In VW and Cog examining a context does _not_ deoptimize a
>> context.  A context will be "deoptimized" (actually converted to a vanilla
>> heap context) only if you write to other than its stack contents or sender.
>>  i.e. a context's frame is only discarded if
>> - one assigns to any of its instance variables other than sender
>> - the stack zone runs out of room for new frames and a stack page must be
>> vacated, causing all frames on that page to be converted to stack contexts
>> So exception handling does *not* usually involve converting contexts.  It
>> would be very slow if it did.
>>
> Ah, cool.. So, even when debugging (and hence accessing context state)
> does not leads automatically to deoptimization?
>
>>>
>>> and if stack depth is high
>>> (between point of writing attempt and hook, where magma will handle
>>> exception), this will be much slower
>>> than WriteBarrier implementation, described by Cris,
>>> which checking the value in-place, without the need of touching
>>> exception machinery.
>>
>> I disagree strongly.  Martin's experience with Gemstone (and Gemstone's
>> experience) covers over twenty years and the VM-supported immutability
>> implementations (first in VisualAge IIRC) of Gemstone are far more
>> performant and less problematic than the code-rewriting implementations
>> similar to Magma.  Martin really knows what he's talking about.
>
> I'm not saying anything against immutability, but capturing the object
> state change
> seem will be less efficient.
> By reading Cris description i imagine that WriteBarrier rewriting a code like:
>
> SomeClass>>foo: newValue
>  foo := newValue
>
> to:
>
> SomeClass*>>foo: newValue
>  | old |
>  old := foo.
>  foo := newValue.
>  old == foo ifFalse: [ Magma markAsDirty: self ]
>
> now compare performance of evaluating:
>
> [[[[[[ self foo: 5 ]]]]]]
>
> with:
>
> [[[[[[ self foo: 5 ]]]]]] on: ImmutableException do: [:ex |  Magma
> markAsDirty: ex receiver. ex receiver beMutable. ex resume ]
>
> where [[[[[]]]]]] is a call stack, which can be deeeeep.
>

A stack unwinding, actually could be avoided, if we suppose that VM
signaling the exception , by sending
SomeExceptionSpecialObject>>#signal: immutableObject

Then Magma could hook there and override that method, to something like:
SomeExceptionSpecialObject>>#signal: immutableObject
  (Magma watchingOverThisObject: immutableObject) ifFalse: [ ^ self
new signal: immutableObject ].
  Magma markAsDirty: immutableObject.
  immutableObject beMutable.
  "retry code here"

Right.  Something of this order.  Again lighter-weight than delivering an exception and much lighter weight than constructing a special class.  But see my previous message for how to write it nicely, or play with Gemstone and VW non-commercial to see how it is done in a production system.



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Igor Stasenko
In reply to this post by Eliot Miranda-2
2010/1/13 Eliot Miranda <[hidden email]>:
>
>
> 1.

[ Agreed, and snipped.  ]

>
> 2. Yes, but it doesn't cost as much as you think.  If one is using
> immutability to mark objects dirty then the exception only gets delivered
> once for that object.  Once the object is dirty it can remain mutable until
> a batch of objects are updated, etc.
>

Right. But we should not fool ourselves and actually play nicely even
for the worst case here, when application, for some stupid reason,
tries to modify every object it can put its hands on ;)

Also, one subtle detail about following:

 old := foo.
 foo := newValue.
 old == foo ifFalse: [ Magma markAsDirty: self ]

that if  old == foo, object will not be marked dirty and write will be
performed.
But in case of NoModificationError, object will be marked dirty,
because you don't know
what actual value were attempted to be written, and what sits on that place now.

Maybe VM should play nice and just ignore cases when write attempt
does not changing anything , because old slot value
is same as new one.


> Martin could you say something about the dispatch schemes Gemstone uses and
> what the performance issues have been in practice?  (TIA)
>
> best
> Eliot
>>




--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Martin McClure-2
In reply to this post by Igor Stasenko
Igor Stasenko wrote:

>>
>> [[[[[[ self foo: 5 ]]]]]] on: ImmutableException do: [:ex |  Magma
>> markAsDirty: ex receiver. ex receiver beMutable. ex resume ]
>>
>> where [[[[[]]]]]] is a call stack, which can be deeeeep.
>>
>
> A stack unwinding, actually could be avoided, if we suppose that VM
> signaling the exception , by sending
> SomeExceptionSpecialObject>>#signal: immutableObject
>
> Then Magma could hook there and override that method, to something like:
> SomeExceptionSpecialObject>>#signal: immutableObject
>    (Magma watchingOverThisObject: immutableObject) ifFalse: [ ^ self
> new signal: immutableObject ].
>    Magma markAsDirty: immutableObject.
>    immutableObject beMutable.
>    "retry code here"

This is actually very close to what we do currently. We intercept before
the exception is signaled.

The generic handler like Eliot describes is what we'd like to move to,
as our current scheme would conflict with anyone else who was doing
something similar.

----

Eliot Miranda wrote:

 >
 > Martin could you say something about the dispatch schemes Gemstone uses
 > and what the performance issues have been in practice?  (TIA)
 >

The maintainability of the code has been much better than schemes that
modify the application's bytecodes, which is what we did previously,
similar to what Chris describes.

Performance is better also, though I don't have figures. Imagine that
you need to track changes to an Array. You either have to make that one
Array in reality a different class than all the untracked Arrays, in
which case somewhere along the line someone's going to catch you at that
trick and some code will break, or you have to instrument basicAt:put:
for every Array in the system. As Eliot points out, with VM support you
only pay a penalty for writes to clean tracked objects. Untracked
objects and already-dirtied objects pay no penalty. Since these are the
majority of assignments, it's a clear win.

And I believe that Eliot's implementation of the VM support for
immutability was clever enough that it carried exactly zero penalty for
writes to mutable objects, by combining the immutability check into the
already-necessary range check.

----

Igor Stasenko wrote:
 >
 > Also, one subtle detail about following:
 >
 >  old := foo.
 >  foo := newValue.
 >  old == foo ifFalse: [ Magma markAsDirty: self ]
 >
 > that if  old == foo, object will not be marked dirty and write will be
 > performed.
 > But in case of NoModificationError, object will be marked dirty,
 > because you don't know
 > what actual value were attempted to be written, and what sits on that
place now.
 >

Actually, the VW VM gives you the value that failed to be assigned (and
you need that anyway to re-do the assignment after the mark dirty) so we
use that to ignore identical assignment at the Smalltalk level. I
probably wouldn't put that in the VM code, it's not a common enough case
to warrant the complexity.

Regards,

-Martin

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Chris Muller-3
In reply to this post by Eliot Miranda-2
Compilation of the subclass is only done once per class (and cached).
It takes under one second per class and could be done at application
start-up anyway.  So I'm dubious that any kind of Exception Handling
approach is going to be "far more performant" that simply calling
"self modified:" as the existing WriteBarrier implementation does.

As for "less-problematic", that might have been true with GemStone.
Personally, I would worry about forgetting to set the immutable bit on
my objects.  If that were the basis by which changed-detection occurs,
forgetting to set immutable could result in object changes not being
detected, and therefore not committed.  Silently - you wouldn't know
you lost work until much later.  *That* sounds problematic!

For Magma, the existing WriteBarrier approach really hasn't been
problematic at all, even under lots of heavy use (though we run with
it turned off in our own production systems, don't need it).  It
operates transparently to the Magma user as a "set and forget".  I
concur, however, that it *is* a more complex solution than simply
handling the NoModify exception approach.

Introducing the notion of a VM-level immutable bit to objects may be a
nice feature, but "feels" less-dynamic, since it reflects a similarity
with static typing.  I have "set up" objects for use and, introduces
the notion that every single mutative operation, in all places, must
(or at least should) now have to handle the NoModify exception.

 - Chris

2010/1/13 Eliot Miranda <[hidden email]>:

>
>
> On Wed, Jan 13, 2010 at 12:45 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> 2010/1/13 Martin McClure <[hidden email]>:
>> > Eliot Miranda wrote:
>> >> A *much* better way to implement this is to support immutability in the
>> >> VM (I know, I know, but all the code is available in the Newspeak VM),
>> >> mark objects one wants to mark as dirty as immutable, catch the
>> >> NoModificationError exception, and proceed after having made the object
>> >> mutable and marked it dirty.  No creating hidden classes, no trying to
>> >> get change class to work for compact classes, etc.  Just a simple
>> >> VM-implemented write barrier that can also be used for a host of other
>> >> things (object-database mapping, debugging, immutable literals etc).
>> >
>> > Since object dirtying is at the core of the product I work with, and
>> > I've worked extensively with both methods of implementing write
>> > barriers...
>> >
>> > I very strongly agree with Eliot. *So* much nicer a way to do this.
>> >
>>
>> It could be more efficient, in respect that you don't need to create
>> shadow classes.
>> But triggering exception leads to stack unwinding to find a handler,
>> which inevitably leads to deoptimizing all contexts..
>
> Uh, not so.  In VW and Cog examining a context does _not_ deoptimize a
> context.  A context will be "deoptimized" (actually converted to a vanilla
> heap context) only if you write to other than its stack contents or sender.
>  i.e. a context's frame is only discarded if
> - one assigns to any of its instance variables other than sender
> - the stack zone runs out of room for new frames and a stack page must be
> vacated, causing all frames on that page to be converted to stack contexts
> So exception handling does *not* usually involve converting contexts.  It
> would be very slow if it did.
>
>>
>> and if stack depth is high
>> (between point of writing attempt and hook, where magma will handle
>> exception), this will be much slower
>> than WriteBarrier implementation, described by Cris,
>> which checking the value in-place, without the need of touching
>> exception machinery.
>
> I disagree strongly.  Martin's experience with Gemstone (and Gemstone's
> experience) covers over twenty years and the VM-supported immutability
> implementations (first in VisualAge IIRC) of Gemstone are far more
> performant and less problematic than the code-rewriting implementations
> similar to Magma.  Martin really knows what he's talking about.
>>
>> Just 2 cents.
>>
>> > Regards,
>> >
>> > -Martin
>> >
>> >
>> > _______________________________________________
>> > Pharo-project mailing list
>> > [hidden email]
>> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Stéphane Ducasse
In reply to this post by Eliot Miranda-2
Elliot

it would be realllllllly cool to have this in the squeak-vm what should be done to get it?

Stef
On Jan 13, 2010, at 7:41 PM, Eliot Miranda wrote:

> A *much* better way to implement this is to support immutability in the VM (I know, I know, but all the code is available in the Newspeak VM), mark objects one wants to mark as dirty as immutable, catch the NoModificationError exception, and proceed after having made the object mutable and marked it dirty.  No creating hidden classes, no trying to get change class to work for compact classes, etc.  Just a simple VM-implemented write barrier that can also be used for a host of other things (object-database mapping, debugging, immutable literals etc).
>
> On Wed, Jan 13, 2010 at 9:55 AM, Chris Muller <[hidden email]> wrote:
> A great description of what WriteBarrier does and how it works can be
> found in the WriteBarrier class-comment, itself.
>
> In a nutshell, when an object is added to a WriteBarrier, a subclass
> is dynamically compiled for it, but not added to the Smalltalk
> dictionary.  The name of the class is the same as its superclass but
> with a '*' appended to it.
>
> After creating the subclass, WriteBarrier looks at all "definitions"
> of each instance variable, and adds methods to override the methods in
> its superclass.  The generated override checks the pre-state of the
> instVar, calls super, then compares the post-state of the instVar to
> the pre-state.
>
> If it changed, it signals so that the application (Magma) can mark it dirty.
>
> So, it allows Magma to operate with a smaller readSet, which speeds up
> the part of the commit process relating to detection of changed
> objects for building the CommitPackage which is shipped off to the
> server..
>
>  - Chris
>
> 2010/1/13 Miguel Enrique Cobá Martinez <[hidden email]>:
> > El mié, 13-01-2010 a las 18:08 +0200, Igor Stasenko escribió:
> >> Can someone clarify, why WriteBarrier depends on compiler (new
> >> compiler or old compiler - not really important).
> >> I just wonder what compiler functionality its depends on.
> >
> > WriteBarrier uses the class BytecodeGenerator that is part of the
> > package NewCompiler (that is currently on rewrite for the closure
> > images). Why it need it and how it use it, I really don't know. I have
> > only a vague idea of what WriteBarrier does, and as Mariano said, I use
> > Magma but never used or activated the WriteBarrier functionality of it.
> >
> > Cheers
> >>
> >> 2010/1/13 Mariano Martinez Peck <[hidden email]>:
> >> >
> >> >
> >> > On Wed, Jan 13, 2010 at 5:02 PM, Chris Muller <[hidden email]> wrote:
> >> >>
> >> >> 2010/1/12 Mariano Martinez Peck <[hidden email]>:
> >> >> > Magma depends on NewCompiler ???
> >> >>
> >> >> No, WriteBarrier depends on NewCompiler.
> >> >>
> >> >> Magma can, at the explicit option of the user, turn on the
> >> >> WriteBarrier option if the WriteBarrier package is present.
> >> >> WriteBarrier used to be self-contained, later it became dependent on
> >> >> NewCompiler.
> >> >>
> >> >> Unless the WriteBarrier option is switched on by the user, its
> >> >> presence or absence in the image (as well as NewCompiler) has no
> >> >> effect on Magma.
> >> >>
> >> >
> >> > Thanks for the clarification Chris. It was strange for me that  as I used
> >> > Magma a couple of times and I never installed WriteBarrier neither
> >> > NewCompiler.
> >> >
> >> > Cheers
> >> >
> >> > Mariano
> >> >
> >> >
> >> >>
> >> >>  - Chris
> >> >>
> >> >> _______________________________________________
> >> >> Pharo-project mailing list
> >> >> [hidden email]
> >> >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >> >
> >> >
> >> > _______________________________________________
> >> > Pharo-project mailing list
> >> > [hidden email]
> >> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >> >
> >>
> >>
> >>
> >> --
> >> Best regards,
> >> Igor Stasenko AKA sig.
> >>
> >> _______________________________________________
> >> Pharo-project mailing list
> >> [hidden email]
> >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >
> > --
> > Miguel Cobá
> > http://miguel.leugim.com.mx
> >
> >
> > _______________________________________________
> > Pharo-project mailing list
> > [hidden email]
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Eliot Miranda-2


On Thu, Jan 14, 2010 at 12:45 AM, Stéphane Ducasse <[hidden email]> wrote:
Elliot

it would be realllllllly cool to have this in the squeak-vm what should be done to get it?

I'll upload the Newspeak VMMaker package to a suitable repository.  I could also try and do an extract and merge.
Alternatively someone who is interested could do the same starting from the Newspeak release.
 

Stef
On Jan 13, 2010, at 7:41 PM, Eliot Miranda wrote:

> A *much* better way to implement this is to support immutability in the VM (I know, I know, but all the code is available in the Newspeak VM), mark objects one wants to mark as dirty as immutable, catch the NoModificationError exception, and proceed after having made the object mutable and marked it dirty.  No creating hidden classes, no trying to get change class to work for compact classes, etc.  Just a simple VM-implemented write barrier that can also be used for a host of other things (object-database mapping, debugging, immutable literals etc).
>
> On Wed, Jan 13, 2010 at 9:55 AM, Chris Muller <[hidden email]> wrote:
> A great description of what WriteBarrier does and how it works can be
> found in the WriteBarrier class-comment, itself.
>
> In a nutshell, when an object is added to a WriteBarrier, a subclass
> is dynamically compiled for it, but not added to the Smalltalk
> dictionary.  The name of the class is the same as its superclass but
> with a '*' appended to it.
>
> After creating the subclass, WriteBarrier looks at all "definitions"
> of each instance variable, and adds methods to override the methods in
> its superclass.  The generated override checks the pre-state of the
> instVar, calls super, then compares the post-state of the instVar to
> the pre-state.
>
> If it changed, it signals so that the application (Magma) can mark it dirty.
>
> So, it allows Magma to operate with a smaller readSet, which speeds up
> the part of the commit process relating to detection of changed
> objects for building the CommitPackage which is shipped off to the
> server..
>
>  - Chris
>
> 2010/1/13 Miguel Enrique Cobá Martinez <[hidden email]>:
> > El mié, 13-01-2010 a las 18:08 +0200, Igor Stasenko escribió:
> >> Can someone clarify, why WriteBarrier depends on compiler (new
> >> compiler or old compiler - not really important).
> >> I just wonder what compiler functionality its depends on.
> >
> > WriteBarrier uses the class BytecodeGenerator that is part of the
> > package NewCompiler (that is currently on rewrite for the closure
> > images). Why it need it and how it use it, I really don't know. I have
> > only a vague idea of what WriteBarrier does, and as Mariano said, I use
> > Magma but never used or activated the WriteBarrier functionality of it.
> >
> > Cheers
> >>
> >> 2010/1/13 Mariano Martinez Peck <[hidden email]>:
> >> >
> >> >
> >> > On Wed, Jan 13, 2010 at 5:02 PM, Chris Muller <[hidden email]> wrote:
> >> >>
> >> >> 2010/1/12 Mariano Martinez Peck <[hidden email]>:
> >> >> > Magma depends on NewCompiler ???
> >> >>
> >> >> No, WriteBarrier depends on NewCompiler.
> >> >>
> >> >> Magma can, at the explicit option of the user, turn on the
> >> >> WriteBarrier option if the WriteBarrier package is present.
> >> >> WriteBarrier used to be self-contained, later it became dependent on
> >> >> NewCompiler.
> >> >>
> >> >> Unless the WriteBarrier option is switched on by the user, its
> >> >> presence or absence in the image (as well as NewCompiler) has no
> >> >> effect on Magma.
> >> >>
> >> >
> >> > Thanks for the clarification Chris. It was strange for me that  as I used
> >> > Magma a couple of times and I never installed WriteBarrier neither
> >> > NewCompiler.
> >> >
> >> > Cheers
> >> >
> >> > Mariano
> >> >
> >> >
> >> >>
> >> >>  - Chris
> >> >>
> >> >> _______________________________________________
> >> >> Pharo-project mailing list
> >> >> [hidden email]
> >> >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >> >
> >> >
> >> > _______________________________________________
> >> > Pharo-project mailing list
> >> > [hidden email]
> >> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >> >
> >>
> >>
> >>
> >> --
> >> Best regards,
> >> Igor Stasenko AKA sig.
> >>
> >> _______________________________________________
> >> Pharo-project mailing list
> >> [hidden email]
> >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >
> > --
> > Miguel Cobá
> > http://miguel.leugim.com.mx
> >
> >
> > _______________________________________________
> > Pharo-project mailing list
> > [hidden email]
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Stéphane Ducasse
> it would be realllllllly cool to have this in the squeak-vm what should be done to get it?
>
> I'll upload the Newspeak VMMaker package to a suitable repository.  I could also try and do an extract and merge.
> Alternatively someone who is interested could do the same starting from the Newspeak release.
>  
this is not that I'm not interested but I never played with VMMaker and now I should not play (only project proposal and integrating fixes during recess).

Stef


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Marcus Denker-4
In reply to this post by Eliot Miranda-2

On Jan 14, 2010, at 7:56 PM, Eliot Miranda wrote:

>
>
> On Thu, Jan 14, 2010 at 12:45 AM, Stéphane Ducasse <[hidden email]> wrote:
> Elliot
>
> it would be realllllllly cool to have this in the squeak-vm what should be done to get it?
>
> I'll upload the Newspeak VMMaker package to a suitable repository.  I could also try and do an extract and merge.
> Alternatively someone who is interested could do the same starting from the Newspeak release.
>  
>

I would be interested to have a look at some point... maybe not the next weeks, but it would really be nice to have
that. So I am sure I will find time at some point.

        Marcus

--
Marcus Denker  -- http://www.marcusdenker.de
INRIA Lille -- Nord Europe. Team RMoD.


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: NewCompiler load on PharoCore

Stéphane Ducasse
jean-baptite should help after the tools paper.

Stef

On Jan 17, 2010, at 6:31 PM, Marcus Denker wrote:

>
> On Jan 14, 2010, at 7:56 PM, Eliot Miranda wrote:
>
>>
>>
>> On Thu, Jan 14, 2010 at 12:45 AM, Stéphane Ducasse <[hidden email]> wrote:
>> Elliot
>>
>> it would be realllllllly cool to have this in the squeak-vm what should be done to get it?
>>
>> I'll upload the Newspeak VMMaker package to a suitable repository.  I could also try and do an extract and merge.
>> Alternatively someone who is interested could do the same starting from the Newspeak release.
>>
>>
>
> I would be interested to have a look at some point... maybe not the next weeks, but it would really be nice to have
> that. So I am sure I will find time at some point.
>
> Marcus
>
> --
> Marcus Denker  -- http://www.marcusdenker.de
> INRIA Lille -- Nord Europe. Team RMoD.
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
12