The Inbox: Kernel-dtl.1310.mcz

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

The Inbox: Kernel-dtl.1310.mcz

commits-2
David T. Lewis uploaded a new version of Kernel to project The Inbox:
http://source.squeak.org/inbox/Kernel-dtl.1310.mcz

==================== Summary ====================

Name: Kernel-dtl.1310
Author: dtl
Time: 6 March 2020, 7:29:11.316779 pm
UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
Ancestors: Kernel-mt.1309

Change the Squeak default bytecode set to Sista in trunk.

Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.

Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.

Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.

=============== Diff against Kernel-mt.1309 ===============

Item was added:
+ ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
+ multipleBytecodeSetsActive: aBoolean
+ "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
+ in addition to the traditional V3 bytecode set, are now in use is this image.
+ The VM may use this information to update the image format number when
+ saving the image to the file system."
+
+ <primitive: 'primitiveMultipleBytecodeSetsActive'>
+ !

Item was added:
+ ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
+ useSista: useSistaEncoder
+ "Switch to or from the Sista bytecode encoder, and recompile the system
+ using that encoder. Assumes that Compiler recompileAll is working for the
+ existing system. Assumes that the currently available primary and secondary
+ bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
+ This is a convenience method that must be updated as the available encoders
+ are changed."
+
+ "CompiledCode useSista: true"
+ "CompiledCode useSista: false"
+
+ | standardEncoder sistaEncoder activeEncoder |
+ standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
+ sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
+ activeEncoder := self preferredBytecodeSetEncoderClass.
+ useSistaEncoder
+ ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
+ self preferredBytecodeSetEncoderClass: sistaEncoder.
+ activeEncoder ~= sistaEncoder
+ ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
+ self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
+ ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
+ self preferredBytecodeSetEncoderClass: standardEncoder.
+ activeEncoder ~= standardEncoder
+ ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
+ self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
+
+ !

Item was changed:
+ (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
+
+ CompiledCode useSista: true.
+ '!
- (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
- "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
-  rehash all hashed collections that contain hashed large integers."
- HashedCollection allSubclassesDo:
- [:c| | f |
- f := (c includesBehavior: Set)
- ifTrue: [[:i| i]]
- ifFalse: [[:i| i keys]].
- c allInstancesDo:
- [:h|
- ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
- [h rehash]]]'!


Reply | Threaded
Open this post in threaded view
|

Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

David T. Lewis
The Sista bytecode set enables important reasearch and development work.
Eliot and Clemont can explain better than me (apologies to Clemont Bera
for using my seven-bit character set on your name).

Squeak 5.3 is released and we are starting the new release cycle, so it
is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
activates Sista in the package postscipt.

Dave

On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:

> David T. Lewis uploaded a new version of Kernel to project The Inbox:
> http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
>
> ==================== Summary ====================
>
> Name: Kernel-dtl.1310
> Author: dtl
> Time: 6 March 2020, 7:29:11.316779 pm
> UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
> Ancestors: Kernel-mt.1309
>
> Change the Squeak default bytecode set to Sista in trunk.
>
> Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.
>
> Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.
>
> Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.
>
> =============== Diff against Kernel-mt.1309 ===============
>
> Item was added:
> + ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
> + multipleBytecodeSetsActive: aBoolean
> + "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
> + in addition to the traditional V3 bytecode set, are now in use is this image.
> + The VM may use this information to update the image format number when
> + saving the image to the file system."
> +
> + <primitive: 'primitiveMultipleBytecodeSetsActive'>
> + !
>
> Item was added:
> + ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
> + useSista: useSistaEncoder
> + "Switch to or from the Sista bytecode encoder, and recompile the system
> + using that encoder. Assumes that Compiler recompileAll is working for the
> + existing system. Assumes that the currently available primary and secondary
> + bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
> + This is a convenience method that must be updated as the available encoders
> + are changed."
> +
> + "CompiledCode useSista: true"
> + "CompiledCode useSista: false"
> +
> + | standardEncoder sistaEncoder activeEncoder |
> + standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
> + sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
> + activeEncoder := self preferredBytecodeSetEncoderClass.
> + useSistaEncoder
> + ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
> + self preferredBytecodeSetEncoderClass: sistaEncoder.
> + activeEncoder ~= sistaEncoder
> + ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> + self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
> + ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
> + self preferredBytecodeSetEncoderClass: standardEncoder.
> + activeEncoder ~= standardEncoder
> + ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> + self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
> +
> + !
>
> Item was changed:
> + (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
> +
> + CompiledCode useSista: true.
> + '!
> - (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
> - "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
> -  rehash all hashed collections that contain hashed large integers."
> - HashedCollection allSubclassesDo:
> - [:c| | f |
> - f := (c includesBehavior: Set)
> - ifTrue: [[:i| i]]
> - ifFalse: [[:i| i keys]].
> - c allInstancesDo:
> - [:h|
> - ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
> - [h rehash]]]'!
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

David T. Lewis
I will leave this in the inbox for a couple of days, and if there are
no objections I will move it to trunk.

Dave

On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:

> The Sista bytecode set enables important reasearch and development work.
> Eliot and Clemont can explain better than me (apologies to Clemont Bera
> for using my seven-bit character set on your name).
>
> Squeak 5.3 is released and we are starting the new release cycle, so it
> is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> activates Sista in the package postscipt.
>
> Dave
>
> On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> >
> > ==================== Summary ====================
> >
> > Name: Kernel-dtl.1310
> > Author: dtl
> > Time: 6 March 2020, 7:29:11.316779 pm
> > UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
> > Ancestors: Kernel-mt.1309
> >
> > Change the Squeak default bytecode set to Sista in trunk.
> >
> > Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.
> >
> > Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.
> >
> > Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.
> >
> > =============== Diff against Kernel-mt.1309 ===============
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
> > + multipleBytecodeSetsActive: aBoolean
> > + "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
> > + in addition to the traditional V3 bytecode set, are now in use is this image.
> > + The VM may use this information to update the image format number when
> > + saving the image to the file system."
> > +
> > + <primitive: 'primitiveMultipleBytecodeSetsActive'>
> > + !
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
> > + useSista: useSistaEncoder
> > + "Switch to or from the Sista bytecode encoder, and recompile the system
> > + using that encoder. Assumes that Compiler recompileAll is working for the
> > + existing system. Assumes that the currently available primary and secondary
> > + bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
> > + This is a convenience method that must be updated as the available encoders
> > + are changed."
> > +
> > + "CompiledCode useSista: true"
> > + "CompiledCode useSista: false"
> > +
> > + | standardEncoder sistaEncoder activeEncoder |
> > + standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
> > + sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
> > + activeEncoder := self preferredBytecodeSetEncoderClass.
> > + useSistaEncoder
> > + ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
> > + self preferredBytecodeSetEncoderClass: sistaEncoder.
> > + activeEncoder ~= sistaEncoder
> > + ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > + self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
> > + ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
> > + self preferredBytecodeSetEncoderClass: standardEncoder.
> > + activeEncoder ~= standardEncoder
> > + ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > + self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
> > +
> > + !
> >
> > Item was changed:
> > + (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
> > +
> > + CompiledCode useSista: true.
> > + '!
> > - (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
> > - "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
> > -  rehash all hashed collections that contain hashed large integers."
> > - HashedCollection allSubclassesDo:
> > - [:c| | f |
> > - f := (c includesBehavior: Set)
> > - ifTrue: [[:i| i]]
> > - ifFalse: [[:i| i keys]].
> > - c allInstancesDo:
> > - [:h|
> > - ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
> > - [h rehash]]]'!
> >
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Christoph Thiede

Hi Dave, hi all,


this sounds like a really interesting changeover. I have heard and read some rumors about Sista in the past, but I could not find any brief but complete summary of all the changes this will make to our existing Smalltalk system. So it would be great if you could point us to some kind of changelog.


In what way will this affect your typical Squeak experience? What about performance? What about debugging/simulating code? As someone who did not yet deal with any VM side effects, I would be interested to hear about the practical effects you can see when playing around with Context & Compiler stuff. I read about FullBlockClosure which appears to be kind of detached of its defining method. How will this change accessing thisContext home etc. from a FullBlockClosure, for example? I read Sista stores an optimization cache between several runs. Where will this information be saved, at the image side or at the VM side?

Lots of questions, maybe someone can answer a few of them :-)


Best,

Christoph


Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
Gesendet: Samstag, 7. März 2020 18:29:40
An: [hidden email]
Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
 
I will leave this in the inbox for a couple of days, and if there are
no objections I will move it to trunk.

Dave

On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> The Sista bytecode set enables important reasearch and development work.
> Eliot and Clemont can explain better than me (apologies to Clemont Bera
> for using my seven-bit character set on your name).
>
> Squeak 5.3 is released and we are starting the new release cycle, so it
> is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> activates Sista in the package postscipt.
>
> Dave
>
> On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> >
> > ==================== Summary ====================
> >
> > Name: Kernel-dtl.1310
> > Author: dtl
> > Time: 6 March 2020, 7:29:11.316779 pm
> > UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
> > Ancestors: Kernel-mt.1309
> >
> > Change the Squeak default bytecode set to Sista in trunk.
> >
> > Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.
> >
> > Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.
> >
> > Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.
> >
> > =============== Diff against Kernel-mt.1309 ===============
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
> > + multipleBytecodeSetsActive: aBoolean
> > +    "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
> > +    in addition to the traditional V3 bytecode set, are now in use is this image.
> > +    The VM may use this information to update the image format number when
> > +    saving the image to the file system."
> > +
> > +    <primitive: 'primitiveMultipleBytecodeSetsActive'>
> > + !
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
> > + useSista: useSistaEncoder
> > +    "Switch to or from the Sista bytecode encoder, and recompile the system
> > +    using that encoder. Assumes that Compiler recompileAll is working for the
> > +    existing system. Assumes that the currently available primary and secondary
> > +    bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
> > +    This is a convenience method that must be updated as the available encoders
> > +    are changed."
> > +
> > +    "CompiledCode useSista: true"
> > +    "CompiledCode useSista: false"
> > +
> > +    | standardEncoder sistaEncoder activeEncoder |
> > +    standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
> > +    sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
> > +    activeEncoder := self preferredBytecodeSetEncoderClass.
> > +    useSistaEncoder
> > +            ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: sistaEncoder.
> > +                    activeEncoder ~= sistaEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
> > +            ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: standardEncoder.
> > +                    activeEncoder ~= standardEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
> > +
> > + !
> >
> > Item was changed:
> > + (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
> > +
> > + CompiledCode useSista: true.
> > + '!
> > - (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
> > - "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
> > -  rehash all hashed collections that contain hashed large integers."
> > - HashedCollection allSubclassesDo:
> > -    [:c| | f |
> > -    f := (c includesBehavior: Set)
> > -                    ifTrue: [[:i| i]]
> > -                    ifFalse: [[:i| i keys]].
> > -    c allInstancesDo:
> > -            [:h|
> > -             ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
> > -                    [h rehash]]]'!
> >
> >
>



Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

David T. Lewis
Hi Christoph,

On Sat, Mar 07, 2020 at 06:11:56PM +0000, Thiede, Christoph wrote:
> Hi Dave, hi all,
>
>
> this sounds like a really interesting changeover. I have heard and
> read some rumors about Sista in the past, but I could not find any
> brief but complete summary of all the changes this will make to our
 existing Smalltalk system. So it would be great if you could point
> us to some kind of changelog.
>

You can think of the bytecode set as the machine level instructions
that are executed by the virtual machine, analogous to the hardware
instructions that are executed by a CPU. Eliot has arranged for Squeak
(and Pharo and others) to be able to change over to an alternate
instruction set, sort of like changing the CPU on your computer without
turning the power off.

The motivation for the new bytcodes (called Sista) is to enable certain
kinds of performance optimizations directly in the image. Think in
terms of some of the performance optimizations in the Cog/Spur VMs,
but now potentially being able to do it directly in the image.

Others will have to give you the details, but that's the basic idea.

>
> In what way will this affect your typical Squeak experience? What
> about performance? What about debugging/simulating code? As someone
> who did not yet deal with any VM side effects, I would be interested
> to hear about the practical effects you can see when playing around
> with Context & Compiler stuff. I read about FullBlockClosure which
> appears to be kind of detached of its defining method. How will this
> change accessing thisContext home etc. from a FullBlockClosure, for
> example? I read Sista stores an optimization cache between several
> runs. Where will this information be saved, at the image side or
> at the VM side?
>

I cannot answer much of this, other than to say that initially you
should expect to see no difference in the user experience. Various
people have been using Sista for some time without problems, so it
will initially be a transparent change, with the more interesting
things happening in the future :-)

Dave


> Lots of questions, maybe someone can answer a few of them :-)
>
>
> Best,
>
> Christoph
>
> ________________________________
> Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
> Gesendet: Samstag, 7. M?rz 2020 18:29:40
> An: [hidden email]
> Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
>
> I will leave this in the inbox for a couple of days, and if there are
> no objections I will move it to trunk.
>
> Dave
>
> On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> > The Sista bytecode set enables important reasearch and development work.
> > Eliot and Clemont can explain better than me (apologies to Clemont Bera
> > for using my seven-bit character set on your name).
> >
> > Squeak 5.3 is released and we are starting the new release cycle, so it
> > is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> > activates Sista in the package postscipt.
> >
> > Dave
> >
> > On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> > >


Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Nicolas Cellier
Hi all,
I agree for moving to trunk asap.
It must happen in early phases so that we test eagerly.
There's a few test failing with wrong expectations now, but no catastrophic consequences.

Le sam. 7 mars 2020 à 19:47, David T. Lewis <[hidden email]> a écrit :
Hi Christoph,

On Sat, Mar 07, 2020 at 06:11:56PM +0000, Thiede, Christoph wrote:
> Hi Dave, hi all,
>
>
> this sounds like a really interesting changeover. I have heard and
> read some rumors about Sista in the past, but I could not find any
> brief but complete summary of all the changes this will make to our
 existing Smalltalk system. So it would be great if you could point
> us to some kind of changelog.
>

You can think of the bytecode set as the machine level instructions
that are executed by the virtual machine, analogous to the hardware
instructions that are executed by a CPU. Eliot has arranged for Squeak
(and Pharo and others) to be able to change over to an alternate
instruction set, sort of like changing the CPU on your computer without
turning the power off.

The motivation for the new bytcodes (called Sista) is to enable certain
kinds of performance optimizations directly in the image. Think in
terms of some of the performance optimizations in the Cog/Spur VMs,
but now potentially being able to do it directly in the image.

Others will have to give you the details, but that's the basic idea.

>
> In what way will this affect your typical Squeak experience? What
> about performance? What about debugging/simulating code? As someone
> who did not yet deal with any VM side effects, I would be interested
> to hear about the practical effects you can see when playing around
> with Context & Compiler stuff. I read about FullBlockClosure which
> appears to be kind of detached of its defining method. How will this
> change accessing thisContext home etc. from a FullBlockClosure, for
> example? I read Sista stores an optimization cache between several
> runs. Where will this information be saved, at the image side or
> at the VM side?
>

I cannot answer much of this, other than to say that initially you
should expect to see no difference in the user experience. Various
people have been using Sista for some time without problems, so it
will initially be a transparent change, with the more interesting
things happening in the future :-)

Dave


> Lots of questions, maybe someone can answer a few of them :-)
>
>
> Best,
>
> Christoph
>
> ________________________________
> Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
> Gesendet: Samstag, 7. M?rz 2020 18:29:40
> An: [hidden email]
> Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
>
> I will leave this in the inbox for a couple of days, and if there are
> no objections I will move it to trunk.
>
> Dave
>
> On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> > The Sista bytecode set enables important reasearch and development work.
> > Eliot and Clemont can explain better than me (apologies to Clemont Bera
> > for using my seven-bit character set on your name).
> >
> > Squeak 5.3 is released and we are starting the new release cycle, so it
> > is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> > activates Sista in the package postscipt.
> >
> > Dave
> >
> > On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> > >




Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Nicolas Cellier
The adaptive optimization (inlining etc...) done at image side is called Scorch.
Clément has prototyped it in Pharo at http://smalltalkhub.com/#!/~ClementBera/Scorch
It's not advertised as ready for consumption yet.

Le sam. 7 mars 2020 à 20:45, Nicolas Cellier <[hidden email]> a écrit :
Hi all,
I agree for moving to trunk asap.
It must happen in early phases so that we test eagerly.
There's a few test failing with wrong expectations now, but no catastrophic consequences.

Le sam. 7 mars 2020 à 19:47, David T. Lewis <[hidden email]> a écrit :
Hi Christoph,

On Sat, Mar 07, 2020 at 06:11:56PM +0000, Thiede, Christoph wrote:
> Hi Dave, hi all,
>
>
> this sounds like a really interesting changeover. I have heard and
> read some rumors about Sista in the past, but I could not find any
> brief but complete summary of all the changes this will make to our
 existing Smalltalk system. So it would be great if you could point
> us to some kind of changelog.
>

You can think of the bytecode set as the machine level instructions
that are executed by the virtual machine, analogous to the hardware
instructions that are executed by a CPU. Eliot has arranged for Squeak
(and Pharo and others) to be able to change over to an alternate
instruction set, sort of like changing the CPU on your computer without
turning the power off.

The motivation for the new bytcodes (called Sista) is to enable certain
kinds of performance optimizations directly in the image. Think in
terms of some of the performance optimizations in the Cog/Spur VMs,
but now potentially being able to do it directly in the image.

Others will have to give you the details, but that's the basic idea.

>
> In what way will this affect your typical Squeak experience? What
> about performance? What about debugging/simulating code? As someone
> who did not yet deal with any VM side effects, I would be interested
> to hear about the practical effects you can see when playing around
> with Context & Compiler stuff. I read about FullBlockClosure which
> appears to be kind of detached of its defining method. How will this
> change accessing thisContext home etc. from a FullBlockClosure, for
> example? I read Sista stores an optimization cache between several
> runs. Where will this information be saved, at the image side or
> at the VM side?
>

I cannot answer much of this, other than to say that initially you
should expect to see no difference in the user experience. Various
people have been using Sista for some time without problems, so it
will initially be a transparent change, with the more interesting
things happening in the future :-)

Dave


> Lots of questions, maybe someone can answer a few of them :-)
>
>
> Best,
>
> Christoph
>
> ________________________________
> Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
> Gesendet: Samstag, 7. M?rz 2020 18:29:40
> An: [hidden email]
> Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
>
> I will leave this in the inbox for a couple of days, and if there are
> no objections I will move it to trunk.
>
> Dave
>
> On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> > The Sista bytecode set enables important reasearch and development work.
> > Eliot and Clemont can explain better than me (apologies to Clemont Bera
> > for using my seven-bit character set on your name).
> >
> > Squeak 5.3 is released and we are starting the new release cycle, so it
> > is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> > activates Sista in the package postscipt.
> >
> > Dave
> >
> > On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> > >




Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Nicolas Cellier
So I accidentally published a small fix in trunk without thinking that the ancestor was in inbox...
Consequently, all is in trunk now with an advance of half a couple of days.

Le sam. 7 mars 2020 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
The adaptive optimization (inlining etc...) done at image side is called Scorch.
Clément has prototyped it in Pharo at http://smalltalkhub.com/#!/~ClementBera/Scorch
It's not advertised as ready for consumption yet.

Le sam. 7 mars 2020 à 20:45, Nicolas Cellier <[hidden email]> a écrit :
Hi all,
I agree for moving to trunk asap.
It must happen in early phases so that we test eagerly.
There's a few test failing with wrong expectations now, but no catastrophic consequences.

Le sam. 7 mars 2020 à 19:47, David T. Lewis <[hidden email]> a écrit :
Hi Christoph,

On Sat, Mar 07, 2020 at 06:11:56PM +0000, Thiede, Christoph wrote:
> Hi Dave, hi all,
>
>
> this sounds like a really interesting changeover. I have heard and
> read some rumors about Sista in the past, but I could not find any
> brief but complete summary of all the changes this will make to our
 existing Smalltalk system. So it would be great if you could point
> us to some kind of changelog.
>

You can think of the bytecode set as the machine level instructions
that are executed by the virtual machine, analogous to the hardware
instructions that are executed by a CPU. Eliot has arranged for Squeak
(and Pharo and others) to be able to change over to an alternate
instruction set, sort of like changing the CPU on your computer without
turning the power off.

The motivation for the new bytcodes (called Sista) is to enable certain
kinds of performance optimizations directly in the image. Think in
terms of some of the performance optimizations in the Cog/Spur VMs,
but now potentially being able to do it directly in the image.

Others will have to give you the details, but that's the basic idea.

>
> In what way will this affect your typical Squeak experience? What
> about performance? What about debugging/simulating code? As someone
> who did not yet deal with any VM side effects, I would be interested
> to hear about the practical effects you can see when playing around
> with Context & Compiler stuff. I read about FullBlockClosure which
> appears to be kind of detached of its defining method. How will this
> change accessing thisContext home etc. from a FullBlockClosure, for
> example? I read Sista stores an optimization cache between several
> runs. Where will this information be saved, at the image side or
> at the VM side?
>

I cannot answer much of this, other than to say that initially you
should expect to see no difference in the user experience. Various
people have been using Sista for some time without problems, so it
will initially be a transparent change, with the more interesting
things happening in the future :-)

Dave


> Lots of questions, maybe someone can answer a few of them :-)
>
>
> Best,
>
> Christoph
>
> ________________________________
> Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
> Gesendet: Samstag, 7. M?rz 2020 18:29:40
> An: [hidden email]
> Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
>
> I will leave this in the inbox for a couple of days, and if there are
> no objections I will move it to trunk.
>
> Dave
>
> On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> > The Sista bytecode set enables important reasearch and development work.
> > Eliot and Clemont can explain better than me (apologies to Clemont Bera
> > for using my seven-bit character set on your name).
> >
> > Squeak 5.3 is released and we are starting the new release cycle, so it
> > is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> > activates Sista in the package postscipt.
> >
> > Dave
> >
> > On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> > >




Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Nicolas Cellier
Before i stop, here is what I found:
There are still many (old) BlockClosures active in the image (thru Services, Pluggable*Somthing, etc...)
Debugger will (sometimes?) fail to decompile those BlockClosures if encountered on the stack (example below).
Not a show stopper, but we should take some corrective action.

--------------

Requestor(Object)>>doesNotUnderstand: #browseMcMethodRevisions
Receiver: a Requestor
Arguments and temporary variables:
aMessage: browseMcMethodRevisions
exception: MessageNotUnderstood: Requestor>>browseMcMethodRevisions
resumeValue: nil
Receiver's instance variables:
caption: 'Enter text'
answer: a WriteStream

[] in MCHttpRepository class(MCRepository class)>>browseMethodRevisionsService
Receiver: MCHttpRepository
Arguments and temporary variables:
<<error during printing>
Receiver's instance variables:
superclass: MCFileBasedRepository
methodDict: a MethodDictionary(#allFileNames->(MCHttpRepository>>#allFileNames ...etc...
format: 65547
instanceVariables: #('location' 'user' 'password' 'readerCache' 'indexed' 'webC...etc...
organization: ('overriding')
('private' displayProgress:during: flushCache httpGet:arguments:...etc...
subclasses: nil
name: #MCHttpRepository
classPool: a Dictionary(#UseSharedWebClientInstance->false )
sharedPools: nil
environment: Smalltalk
category: #'Monticello-Repositories'

BlockClosure>>valueWithRequestor:
Receiver: [closure] in MCHttpRepository class(MCRepository class)>>browseMethodRevisionsService
Arguments and temporary variables:
aRequestor: a Requestor
Receiver's instance variables:
outerContext: MCHttpRepository class(MCRepository class)>>browseMethodRevisionsService...etc...
startpc: 82
numArgs: 1

ServiceAction>>execute
Receiver: a ServiceAction browseMcMethodRevisions
Arguments and temporary variables:

Receiver's instance variables:
condition: [closure] in MCHttpRepository class(MCRepository class)>>browseMethodRevisionsService...etc...
action: [closure] in MCHttpRepository class(MCRepository class)>>browseMethodRevisionsService...etc...
requestor: a BrowserRequestor
label: 'browse revisions'
shortLabel: 'mc'
description: 'Browse revisions of this method from the first-listed HTTP reposi...etc...
id: #browseMcMethodRevisions
provider: nil
enabled: true

Le dim. 8 mars 2020 à 21:34, Nicolas Cellier <[hidden email]> a écrit :
So I accidentally published a small fix in trunk without thinking that the ancestor was in inbox...
Consequently, all is in trunk now with an advance of half a couple of days.

Le sam. 7 mars 2020 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
The adaptive optimization (inlining etc...) done at image side is called Scorch.
Clément has prototyped it in Pharo at http://smalltalkhub.com/#!/~ClementBera/Scorch
It's not advertised as ready for consumption yet.

Le sam. 7 mars 2020 à 20:45, Nicolas Cellier <[hidden email]> a écrit :
Hi all,
I agree for moving to trunk asap.
It must happen in early phases so that we test eagerly.
There's a few test failing with wrong expectations now, but no catastrophic consequences.

Le sam. 7 mars 2020 à 19:47, David T. Lewis <[hidden email]> a écrit :
Hi Christoph,

On Sat, Mar 07, 2020 at 06:11:56PM +0000, Thiede, Christoph wrote:
> Hi Dave, hi all,
>
>
> this sounds like a really interesting changeover. I have heard and
> read some rumors about Sista in the past, but I could not find any
> brief but complete summary of all the changes this will make to our
 existing Smalltalk system. So it would be great if you could point
> us to some kind of changelog.
>

You can think of the bytecode set as the machine level instructions
that are executed by the virtual machine, analogous to the hardware
instructions that are executed by a CPU. Eliot has arranged for Squeak
(and Pharo and others) to be able to change over to an alternate
instruction set, sort of like changing the CPU on your computer without
turning the power off.

The motivation for the new bytcodes (called Sista) is to enable certain
kinds of performance optimizations directly in the image. Think in
terms of some of the performance optimizations in the Cog/Spur VMs,
but now potentially being able to do it directly in the image.

Others will have to give you the details, but that's the basic idea.

>
> In what way will this affect your typical Squeak experience? What
> about performance? What about debugging/simulating code? As someone
> who did not yet deal with any VM side effects, I would be interested
> to hear about the practical effects you can see when playing around
> with Context & Compiler stuff. I read about FullBlockClosure which
> appears to be kind of detached of its defining method. How will this
> change accessing thisContext home etc. from a FullBlockClosure, for
> example? I read Sista stores an optimization cache between several
> runs. Where will this information be saved, at the image side or
> at the VM side?
>

I cannot answer much of this, other than to say that initially you
should expect to see no difference in the user experience. Various
people have been using Sista for some time without problems, so it
will initially be a transparent change, with the more interesting
things happening in the future :-)

Dave


> Lots of questions, maybe someone can answer a few of them :-)
>
>
> Best,
>
> Christoph
>
> ________________________________
> Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
> Gesendet: Samstag, 7. M?rz 2020 18:29:40
> An: [hidden email]
> Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
>
> I will leave this in the inbox for a couple of days, and if there are
> no objections I will move it to trunk.
>
> Dave
>
> On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> > The Sista bytecode set enables important reasearch and development work.
> > Eliot and Clemont can explain better than me (apologies to Clemont Bera
> > for using my seven-bit character set on your name).
> >
> > Squeak 5.3 is released and we are starting the new release cycle, so it
> > is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> > activates Sista in the package postscipt.
> >
> > Dave
> >
> > On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> > >




Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Eliot Miranda-2
In reply to this post by Christoph Thiede
Hi Christoph, Hi All,

On Sat, Mar 7, 2020 at 10:12 AM Thiede, Christoph <[hidden email]> wrote:

Hi Dave, hi all,


this sounds like a really interesting changeover. I have heard and read some rumors about Sista in the past, but I could not find any brief but complete summary of all the changes this will make to our existing Smalltalk system. So it would be great if you could point us to some kind of changelog.


The class comment for EncoderForSistaV1 gives a good introduction.  Further information is available in the papers and presentations mentioned on squeak.org/research  But the key property is that, as a bytecode set, it does not affect semantics.  It is an instruction set, not a new virtual machine.
  1. Clément Béra, Eliot Miranda, Tim Felgentreff, Marcus Denker, and Stéphane Ducasse. “Sista: Saving Optimized Code in Snapshots for Fast Start-Up.” In Proceedings of the 14th International Conference on Managed Languages and Runtimes, 1–11. ManLang 2017. New York, NY, USA: ACM, 2017.http://doi.acm.org/10.1145/3132190.3132201.
  2. Clément Béra, Eliot Miranda, Marcus Denker, and Stéphane Ducasse. “Practical Validation of Bytecode to Bytecode JIT Compiler Dynamic Deoptimization.” Journal of Object Technology 15, no. 2 (2016): 1:1–26. https://doi.org/10.5381/jot.2016.15.2.a1.

In what way will this affect your typical Squeak experience?

You will be able to compile larger methods.
 

What about performance?

In an interpreter one may be able to find examples that are faster, but by very small percentages.  Sista as a bytecode set on its own does not affect performance.  However, once used by an adaptive optimizer (Clément's Scorch) performance should be in the 3x to 4x faster range.  Of course, there is zero support and zero funding for this work.  Clément is now at Google.  I am attempting to self-fund this work for my PhD.  I feel extremely let down by the entire community.  This is excellent work but there is *zero*controbution from outside Clément and myself.  Inria continue to crow about the PhartoVM (intellectual theft) but contribute nothing materially to improving the VM.  The Potsdam folks continue to work on Graal, leaving this more interesting and more concentrative, all in Smalltalk work alone.

I feel very negative about the various communities' inability to support, and especially, join in this work.  I find the position of Inria downright unethical; intellectual theft, deeply selfish.  I find the lack of participation by the Potsdam folks very dispiriting.

What about debugging/simulating code?


No difference.  It is a bytecode set.
 

As someone who did not yet deal with any VM side effects, I would be interested to hear about the practical effects you can see when playing around with Context & Compiler stuff.


Again there should be no effects.  There may be bugs lurking in the bytecode set specific code (see the class side of BytecodeEncoder, EncoderForv3, EncoderForV3PlusClosures and EncoderForSistaV1).  The main source of bugs for the SistaV1 bytecode set is in its use of prefix bytecodes to extend the ranges of operands (see the class comment).  The scheme is more complex than a non-prefix bytecode set, but I've been using the set for a year now and have fixed a few bugs over that time, the last known bug having been fixed in September of last year.

I read about FullBlockClosure which appears to be kind of detached of its defining method. How will this change accessing thisContext home etc. from a FullBlockClosure, for example?


It does not change semantics at all.  It only changes where the bytecodes and literals for a block reside, not what their execution effects.
 

I read Sista stores an optimization cache between several runs. Where will this information be saved, at the image side or at the VM side?

Scorch, the optimizer that exploits the unsafe primitives in the Sista bytecode set, optimizes by inlining unoptimized byte coded methods into byte coded methods which can contain unsafe primitives.  Execution is faster due to the elimination of overheads (such as block creation) through inlining and through the use of unsafe primitives, proved safe by the Scorch optimizer, to perform arithmetic, object access, and instantiation.  The optimized methods are stored in normal method dictionaries, shadowing their unoptimized source methods. So they persist in method dictionaries.

Lots of questions, maybe someone can answer a few of them :-)

I have a question.
Why is no one in the community joining me and Clément in trying to finish this work? 

Best,

Christoph


Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
Gesendet: Samstag, 7. März 2020 18:29:40
An: [hidden email]
Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
 
I will leave this in the inbox for a couple of days, and if there are
no objections I will move it to trunk.

Dave

On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> The Sista bytecode set enables important reasearch and development work.
> Eliot and Clemont can explain better than me (apologies to Clemont Bera
> for using my seven-bit character set on your name).
>
> Squeak 5.3 is released and we are starting the new release cycle, so it
> is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> activates Sista in the package postscipt.
>
> Dave
>
> On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> >
> > ==================== Summary ====================
> >
> > Name: Kernel-dtl.1310
> > Author: dtl
> > Time: 6 March 2020, 7:29:11.316779 pm
> > UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
> > Ancestors: Kernel-mt.1309
> >
> > Change the Squeak default bytecode set to Sista in trunk.
> >
> > Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.
> >
> > Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.
> >
> > Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.
> >
> > =============== Diff against Kernel-mt.1309 ===============
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
> > + multipleBytecodeSetsActive: aBoolean
> > +    "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
> > +    in addition to the traditional V3 bytecode set, are now in use is this image.
> > +    The VM may use this information to update the image format number when
> > +    saving the image to the file system."
> > +
> > +    <primitive: 'primitiveMultipleBytecodeSetsActive'>
> > + !
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
> > + useSista: useSistaEncoder
> > +    "Switch to or from the Sista bytecode encoder, and recompile the system
> > +    using that encoder. Assumes that Compiler recompileAll is working for the
> > +    existing system. Assumes that the currently available primary and secondary
> > +    bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
> > +    This is a convenience method that must be updated as the available encoders
> > +    are changed."
> > +
> > +    "CompiledCode useSista: true"
> > +    "CompiledCode useSista: false"
> > +
> > +    | standardEncoder sistaEncoder activeEncoder |
> > +    standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
> > +    sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
> > +    activeEncoder := self preferredBytecodeSetEncoderClass.
> > +    useSistaEncoder
> > +            ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: sistaEncoder.
> > +                    activeEncoder ~= sistaEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
> > +            ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: standardEncoder.
> > +                    activeEncoder ~= standardEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
> > +
> > + !
> >
> > Item was changed:
> > + (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
> > +
> > + CompiledCode useSista: true.
> > + '!
> > - (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
> > - "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
> > -  rehash all hashed collections that contain hashed large integers."
> > - HashedCollection allSubclassesDo:
> > -    [:c| | f |
> > -    f := (c includesBehavior: Set)
> > -                    ifTrue: [[:i| i]]
> > -                    ifFalse: [[:i| i keys]].
> > -    c allInstancesDo:
> > -            [:h|
> > -             ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
> > -                    [h rehash]]]'!
> >
> >
>




--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Christoph Thiede

Hi Eliot,


(better answer late than never :-))

Thanks a lot for the detailed answer! It's really interesting to hear about these changes, and it would be fantastic to benefit from Scorch in Squeak someday.


It does not change semantics at all.


Actually, I found a small change indeed:


[] method now returns a CompiledBlock rather than a CompiledMethod. This makes sense, but the term #method is not very accurate any longer, is it? ;) See also Tools-ct.958. Should we maybe rename this thing?


Why is no one in the community joining me and Clément in trying to finish this work?


I can only speak for myself: I'm still doing my basic studies and did not yet have an opportunity to learn the fundaments of VM programming at all. I love to contribute to the inbox as you can effect a lot by making small steps. Compared to this, acquiring the conditions for co-developing at a VM structure appears to consume quite more time than I could spend besides studying ... In a few years, when I have learned more about this stuff, I would be happy to address projects like Scorch. But probably you will have completed your thesis until then :-)


Best,

Christoph



Von: Squeak-dev <[hidden email]> im Auftrag von Eliot Miranda <[hidden email]>
Gesendet: Montag, 9. März 2020 17:24:18
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
 
Hi Christoph, Hi All,

On Sat, Mar 7, 2020 at 10:12 AM Thiede, Christoph <[hidden email]> wrote:

Hi Dave, hi all,


this sounds like a really interesting changeover. I have heard and read some rumors about Sista in the past, but I could not find any brief but complete summary of all the changes this will make to our existing Smalltalk system. So it would be great if you could point us to some kind of changelog.


The class comment for EncoderForSistaV1 gives a good introduction.  Further information is available in the papers and presentations mentioned on squeak.org/research  But the key property is that, as a bytecode set, it does not affect semantics.  It is an instruction set, not a new virtual machine.
  1. Clément Béra, Eliot Miranda, Tim Felgentreff, Marcus Denker, and Stéphane Ducasse. “Sista: Saving Optimized Code in Snapshots for Fast Start-Up.” In Proceedings of the 14th International Conference on Managed Languages and Runtimes, 1–11. ManLang 2017. New York, NY, USA: ACM, 2017.http://doi.acm.org/10.1145/3132190.3132201.
  2. Clément Béra, Eliot Miranda, Marcus Denker, and Stéphane Ducasse. “Practical Validation of Bytecode to Bytecode JIT Compiler Dynamic Deoptimization.” Journal of Object Technology 15, no. 2 (2016): 1:1–26. https://doi.org/10.5381/jot.2016.15.2.a1.

In what way will this affect your typical Squeak experience?

You will be able to compile larger methods.
 

What about performance?

In an interpreter one may be able to find examples that are faster, but by very small percentages.  Sista as a bytecode set on its own does not affect performance.  However, once used by an adaptive optimizer (Clément's Scorch) performance should be in the 3x to 4x faster range.  Of course, there is zero support and zero funding for this work.  Clément is now at Google.  I am attempting to self-fund this work for my PhD.  I feel extremely let down by the entire community.  This is excellent work but there is *zero*controbution from outside Clément and myself.  Inria continue to crow about the PhartoVM (intellectual theft) but contribute nothing materially to improving the VM.  The Potsdam folks continue to work on Graal, leaving this more interesting and more concentrative, all in Smalltalk work alone.

I feel very negative about the various communities' inability to support, and especially, join in this work.  I find the position of Inria downright unethical; intellectual theft, deeply selfish.  I find the lack of participation by the Potsdam folks very dispiriting.

What about debugging/simulating code?


No difference.  It is a bytecode set.
 

As someone who did not yet deal with any VM side effects, I would be interested to hear about the practical effects you can see when playing around with Context & Compiler stuff.


Again there should be no effects.  There may be bugs lurking in the bytecode set specific code (see the class side of BytecodeEncoder, EncoderForv3, EncoderForV3PlusClosures and EncoderForSistaV1).  The main source of bugs for the SistaV1 bytecode set is in its use of prefix bytecodes to extend the ranges of operands (see the class comment).  The scheme is more complex than a non-prefix bytecode set, but I've been using the set for a year now and have fixed a few bugs over that time, the last known bug having been fixed in September of last year.

I read about FullBlockClosure which appears to be kind of detached of its defining method. How will this change accessing thisContext home etc. from a FullBlockClosure, for example?


It does not change semantics at all.  It only changes where the bytecodes and literals for a block reside, not what their execution effects.
 

I read Sista stores an optimization cache between several runs. Where will this information be saved, at the image side or at the VM side?

Scorch, the optimizer that exploits the unsafe primitives in the Sista bytecode set, optimizes by inlining unoptimized byte coded methods into byte coded methods which can contain unsafe primitives.  Execution is faster due to the elimination of overheads (such as block creation) through inlining and through the use of unsafe primitives, proved safe by the Scorch optimizer, to perform arithmetic, object access, and instantiation.  The optimized methods are stored in normal method dictionaries, shadowing their unoptimized source methods. So they persist in method dictionaries.

Lots of questions, maybe someone can answer a few of them :-)

I have a question.
Why is no one in the community joining me and Clément in trying to finish this work? 

Best,

Christoph


Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
Gesendet: Samstag, 7. März 2020 18:29:40
An: [hidden email]
Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)
 
I will leave this in the inbox for a couple of days, and if there are
no objections I will move it to trunk.

Dave

On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> The Sista bytecode set enables important reasearch and development work.
> Eliot and Clemont can explain better than me (apologies to Clemont Bera
> for using my seven-bit character set on your name).
>
> Squeak 5.3 is released and we are starting the new release cycle, so it
> is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> activates Sista in the package postscipt.
>
> Dave
>
> On Sat, Mar 07, 2020 at 12:29:13AM +0000, [hidden email] wrote:
> > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> >
> > ==================== Summary ====================
> >
> > Name: Kernel-dtl.1310
> > Author: dtl
> > Time: 6 March 2020, 7:29:11.316779 pm
> > UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
> > Ancestors: Kernel-mt.1309
> >
> > Change the Squeak default bytecode set to Sista in trunk.
> >
> > Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.
> >
> > Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.
> >
> > Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.
> >
> > =============== Diff against Kernel-mt.1309 ===============
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
> > + multipleBytecodeSetsActive: aBoolean
> > +    "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
> > +    in addition to the traditional V3 bytecode set, are now in use is this image.
> > +    The VM may use this information to update the image format number when
> > +    saving the image to the file system."
> > +
> > +    <primitive: 'primitiveMultipleBytecodeSetsActive'>
> > + !
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
> > + useSista: useSistaEncoder
> > +    "Switch to or from the Sista bytecode encoder, and recompile the system
> > +    using that encoder. Assumes that Compiler recompileAll is working for the
> > +    existing system. Assumes that the currently available primary and secondary
> > +    bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
> > +    This is a convenience method that must be updated as the available encoders
> > +    are changed."
> > +
> > +    "CompiledCode useSista: true"
> > +    "CompiledCode useSista: false"
> > +
> > +    | standardEncoder sistaEncoder activeEncoder |
> > +    standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
> > +    sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
> > +    activeEncoder := self preferredBytecodeSetEncoderClass.
> > +    useSistaEncoder
> > +            ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: sistaEncoder.
> > +                    activeEncoder ~= sistaEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
> > +            ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: standardEncoder.
> > +                    activeEncoder ~= standardEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
> > +
> > + !
> >
> > Item was changed:
> > + (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
> > +
> > + CompiledCode useSista: true.
> > + '!
> > - (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
> > - "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
> > -  rehash all hashed collections that contain hashed large integers."
> > - HashedCollection allSubclassesDo:
> > -    [:c| | f |
> > -    f := (c includesBehavior: Set)
> > -                    ifTrue: [[:i| i]]
> > -                    ifFalse: [[:i| i keys]].
> > -    c allInstancesDo:
> > -            [:h|
> > -             ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
> > -                    [h rehash]]]'!
> >
> >
>




--
_,,,^..^,,,_
best, Eliot