[IMPORTANT] Starting migration to Spur VM
Locked
 
|
We will start migration to Spur today.
To complete it, we will require some time, specially to adapt the CI and check everything is ok. Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). - No more GC stops (noticeable when running large systems) - We will be able to scale our systems up to 2G memory consumption without loosing performance. But, this will have some drawbacks in the first times: 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: - there are some stuff not possible to compatibilise, notably: - Structures now need to inherit from FFIExternalStructure - Arrays now are now shadowed - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. 2.1) ASMJIT will be removed from system and put in a separated packages. NOTE: There will be a blog post explaining FFI-NB architecture during the week. 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). |
|
|
> On 14 Dec 2015, at 11:24, Esteban Lorenzano <[hidden email]> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > ah, I forget it! 6) Some changes to the VM accumulated during the time of transition (some months now). So, some changes introduced there will not be present in new VM. I will merge this changes in the next weeks. cheers! Esteban |
Re: [IMPORTANT] Starting migration to Spur VM
|
|
Thanks!
> On 14 dic 2015, at 11:32 a.m., Esteban Lorenzano <[hidden email]> wrote: > > >> On 14 Dec 2015, at 11:24, Esteban Lorenzano <[hidden email]> wrote: >> >> We will start migration to Spur today. >> To complete it, we will require some time, specially to adapt the CI and check everything is ok. >> >> Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: >> - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). >> - No more GC stops (noticeable when running large systems) >> - We will be able to scale our systems up to 2G memory consumption without loosing performance. >> >> But, this will have some drawbacks in the first times: >> >> 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). >> - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. >> >> 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: >> - there are some stuff not possible to compatibilise, notably: >> - Structures now need to inherit from FFIExternalStructure >> - Arrays now are now shadowed >> - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. >> - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. >> 2.1) ASMJIT will be removed from system and put in a separated packages. >> NOTE: There will be a blog post explaining FFI-NB architecture during the week. >> >> 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) >> >> 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. >> >> 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). >> > > ah, I forget it! > > 6) Some changes to the VM accumulated during the time of transition (some months now). So, some changes introduced there will not be present in new VM. I will merge this changes in the next weeks. > > cheers! > Esteban |
Re: [IMPORTANT] Starting migration to Spur VM
|
30% - 100% wow thats quite a speed boost, well done guys! And 2GB images are more than welcomed , especially for me that I have to deal with 3d data. On Mon, Dec 14, 2015 at 1:01 PM Guillermo Polito <[hidden email]> wrote: Thanks! |
|
|
not yet… but that is the path :)
|
Re: [IMPORTANT] Starting migration to Spur VM
|
|
In reply to this post by EstebanLM
Thank you Eliot & Esteban for making this happen!
Exciting times :) Cheers, Doru > On Dec 14, 2015, at 11:24 AM, Esteban Lorenzano <[hidden email]> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > > -- www.tudorgirba.com www.feenk.com "Presenting is storytelling." |
Re: [IMPORTANT] Starting migration to Spur VM
|
|
2015-12-14 10:14 GMT-03:00 Tudor Girba <[hidden email]>:
> Thank you Eliot & Esteban for making this happen! +1 I look forward to see this in action! :) |
Re: [IMPORTANT] Starting migration to Spur VM
|
|
Exciting times indeed! Many thanks to all who work on this/prepare this.
But we should also be aware: 1. many people already use Pharo 5 as it is very usable and stable I think this deserves a better naming/closing before the integration than just a simple update number. Would it make sense to close down the Pharo 5 line and start with the Spur thing as Pharo 6 as this would make it more easily distinguishable? Especially since Pharo 5 is very stable, already really usuable and with Spur we do not follow our mantra of small steps. This would really avoid confusion afterwards within and outside of the community. 2. the VM change might affect PharoLauncher images as well. It would be good to still have a PharoLauncher that can launch old non-spur (up to now) and new spur-based images Thanks T. |
|
|
In reply to this post by EstebanLM
Excellent work, thanks a lot !! > On Dec 14, 2015, at 07:24, Esteban Lorenzano <[hidden email]> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > > > ---> Save our in-boxes! http://emailcharter.org <--- Johan Fabry - http://pleiad.cl/~jfabry PLEIAD and RyCh labs - Computer Science Department (DCC) - University of Chile |
Re: [Pharo-users] [IMPORTANT] Starting migration to Spur VM
|
|
Pharo 5 your mission should you choose to accept it is to integrate with Spur this image will self destruct in 5 4 3 2 1 On Mon, Dec 14, 2015 at 4:15 PM Johan Fabry <[hidden email]> wrote:
|
Re: [IMPORTANT] Starting migration to Spur VM
|
|
In reply to this post by Torsten Bergmann
Torsten Bergmann <[hidden email]> writes:
> Would it make sense to close down the Pharo 5 line and start with the > Spur thing as Pharo 6 as this would make it more easily distinguishable? > Especially since Pharo 5 is very stable, already really usuable and > with Spur we do not follow our mantra of small steps. > > This would really avoid confusion afterwards within and outside of the > community. this makes a lot of sense in my opinion and I vote +1. But releasing a new Pharo currently is a lot of work, both technical and non-technical (announces, changelog, ...). -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill |
Re: [Pharo-users] [IMPORTANT] Starting migration to Spur VM
|
|
In reply to this post by Tudor Girba-2
> On 14 Dec 2015, at 14:14, Tudor Girba <[hidden email]> wrote: > > Thank you Eliot & Esteban for making this happen! Yes! > Exciting times :) Yes! > Cheers, > Doru > > > >> On Dec 14, 2015, at 11:24 AM, Esteban Lorenzano <[hidden email]> wrote: >> >> We will start migration to Spur today. >> To complete it, we will require some time, specially to adapt the CI and check everything is ok. >> >> Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: >> - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). >> - No more GC stops (noticeable when running large systems) >> - We will be able to scale our systems up to 2G memory consumption without loosing performance. >> >> But, this will have some drawbacks in the first times: >> >> 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). >> - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. >> >> 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: >> - there are some stuff not possible to compatibilise, notably: >> - Structures now need to inherit from FFIExternalStructure >> - Arrays now are now shadowed >> - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. >> - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. >> 2.1) ASMJIT will be removed from system and put in a separated packages. >> NOTE: There will be a blog post explaining FFI-NB architecture during the week. >> >> 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) >> >> 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. >> >> 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). >> >> > > -- > www.tudorgirba.com > www.feenk.com > > "Presenting is storytelling." > > |
Re: [IMPORTANT] Starting migration to Spur VM
|
|
In reply to this post by EstebanLM
Cool, important and challenging!
Thierry Le 14/12/2015 11:32, Esteban Lorenzano a écrit : > >> On 14 Dec 2015, at 11:24, Esteban Lorenzano <[hidden email]> wrote: >> >> We will start migration to Spur today. >> To complete it, we will require some time, specially to adapt the CI and check everything is ok. >> >> Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: >> - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). >> - No more GC stops (noticeable when running large systems) >> - We will be able to scale our systems up to 2G memory consumption without loosing performance. >> >> But, this will have some drawbacks in the first times: >> >> 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). >> - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. >> >> 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: >> - there are some stuff not possible to compatibilise, notably: >> - Structures now need to inherit from FFIExternalStructure >> - Arrays now are now shadowed >> - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. >> - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. >> 2.1) ASMJIT will be removed from system and put in a separated packages. >> NOTE: There will be a blog post explaining FFI-NB architecture during the week. >> >> 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) >> >> 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. >> >> 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). >> > > ah, I forget it! > > 6) Some changes to the VM accumulated during the time of transition (some months now). So, some changes introduced there will not be present in new VM. I will merge this changes in the next weeks. > > cheers! > Esteban > |
|
|
In reply to this post by EstebanLM
Sorry for the stupid question, but how should I adapt my CI job? I saw that the last build failed with no tests being run, so I guess this was due to the VM being changed. > On Dec 14, 2015, at 07:24, Esteban Lorenzano <[hidden email]> wrote: > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). ---> Save our in-boxes! http://emailcharter.org <--- Johan Fabry - http://pleiad.cl/~jfabry PLEIAD and RyCh labs - Computer Science Department (DCC) - University of Chile |
Re: [IMPORTANT] Starting migration to Spur VM
|
|
In reply to this post by EstebanLM
2015-12-14 11:24 GMT+01:00 Esteban Lorenzano <[hidden email]>: We will start migration to Spur today. Some more information about lost instvars for some classes Character https://pharo.fogbugz.com/f/cases/17393/prepare-integration-of-session-manager#BugEvent.149851 Athens 17357 Lost instance variables (AthensCairoPatternSurfacePaint) Is it right that they are gone (new immediate character class and new FFI base for Athens) or what happened? |
|
|
Could this be the cause of Johan crashes since Spur?
Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
|
|
|
this what?
|
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |