We will start migration to Spur today.
To complete it, we will require some time, specially to adapt the CI and check everything is ok. Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). - No more GC stops (noticeable when running large systems) - We will be able to scale our systems up to 2G memory consumption without loosing performance. But, this will have some drawbacks in the first times: 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: - there are some stuff not possible to compatibilise, notably: - Structures now need to inherit from FFIExternalStructure - Arrays now are now shadowed - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. 2.1) ASMJIT will be removed from system and put in a separated packages. NOTE: There will be a blog post explaining FFI-NB architecture during the week. 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). |
> On 14 Dec 2015, at 11:24, Esteban Lorenzano <[hidden email]> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > ah, I forget it! 6) Some changes to the VM accumulated during the time of transition (some months now). So, some changes introduced there will not be present in new VM. I will merge this changes in the next weeks. cheers! Esteban |
Thanks!
> On 14 dic 2015, at 11:32 a.m., Esteban Lorenzano <[hidden email]> wrote: > > >> On 14 Dec 2015, at 11:24, Esteban Lorenzano <[hidden email]> wrote: >> >> We will start migration to Spur today. >> To complete it, we will require some time, specially to adapt the CI and check everything is ok. >> >> Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: >> - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). >> - No more GC stops (noticeable when running large systems) >> - We will be able to scale our systems up to 2G memory consumption without loosing performance. >> >> But, this will have some drawbacks in the first times: >> >> 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). >> - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. >> >> 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: >> - there are some stuff not possible to compatibilise, notably: >> - Structures now need to inherit from FFIExternalStructure >> - Arrays now are now shadowed >> - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. >> - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. >> 2.1) ASMJIT will be removed from system and put in a separated packages. >> NOTE: There will be a blog post explaining FFI-NB architecture during the week. >> >> 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) >> >> 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. >> >> 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). >> > > ah, I forget it! > > 6) Some changes to the VM accumulated during the time of transition (some months now). So, some changes introduced there will not be present in new VM. I will merge this changes in the next weeks. > > cheers! > Esteban |
30% - 100% wow thats quite a speed boost, well done guys! And 2GB images are more than welcomed , especially for me that I have to deal with 3d data. On Mon, Dec 14, 2015 at 1:01 PM Guillermo Polito <[hidden email]> wrote: Thanks! |
not yet… but that is the path :)
|
even more good news , thank you :) On Mon, Dec 14, 2015 at 1:15 PM Esteban Lorenzano <[hidden email]> wrote:
|
In reply to this post by EstebanLM
This is great news. Thanks Eliot and Esteban. From the 2GB limit I
take this to be 32-bit Spur? Is it feasible to jump the image build numbers to make that first image requiring a new VM align on a 100s build numbers. I see we are currently 50496. 50500 is nice and distinctive, but even even 50600 would be fine if the last few 400s are consumed in the meantime. On Mon, Dec 14, 2015 at 6:24 PM, Esteban Lorenzano <[hidden email]> wrote: > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). Maybe PharoLauncher can be a special hardcoded rule to have two template headings... * Pharo 5.0 Spur (beta); and * Pharo 5.0 Pre-Spur (beta) Enhanced multi-VM support perhaps based off the template headings would be nice. cheers -ben |
> On 14 Dec 2015, at 12:21, Ben Coman <[hidden email]> wrote: > > This is great news. Thanks Eliot and Esteban. From the 2GB limit I > take this to be 32-bit Spur? yes of course. We are already working on 64bits spur but is not ready yet (it will be some point of 2016) > > Is it feasible to jump the image build numbers to make that first > image requiring a new VM align on a 100s build numbers. I see we are > currently 50496. 50500 is nice and distinctive, but even even 50600 > would be fine if the last few 400s are consumed in the meantime. mmm… that I don’t know… correlation of version numbers is usually important for us (for statistics purposes)… also changing number generation is more annoying than you would think (that’s why we don’t have 4.1, 4.2, etc.). but well… everything is possible, we’ll see :) Esteban > > > On Mon, Dec 14, 2015 at 6:24 PM, Esteban Lorenzano <[hidden email]> wrote: >> We will start migration to Spur today. >> To complete it, we will require some time, specially to adapt the CI and check everything is ok. >> >> Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: >> - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). >> - No more GC stops (noticeable when running large systems) >> - We will be able to scale our systems up to 2G memory consumption without loosing performance. >> >> But, this will have some drawbacks in the first times: >> >> 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). >> - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. >> >> 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: >> - there are some stuff not possible to compatibilise, notably: >> - Structures now need to inherit from FFIExternalStructure >> - Arrays now are now shadowed >> - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. >> - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. >> 2.1) ASMJIT will be removed from system and put in a separated packages. >> NOTE: There will be a blog post explaining FFI-NB architecture during the week. >> >> 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) >> >> 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. >> >> 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > > Maybe PharoLauncher can be a special hardcoded rule to have two > template headings... > * Pharo 5.0 Spur (beta); and > * Pharo 5.0 Pre-Spur (beta) > Enhanced multi-VM support perhaps based off the template headings would be nice. > > cheers -ben > |
On Mon, Dec 14, 2015 at 7:27 PM, Esteban Lorenzano <[hidden email]> wrote:
> >> On 14 Dec 2015, at 12:21, Ben Coman <[hidden email]> wrote: >> >> This is great news. Thanks Eliot and Esteban. From the 2GB limit I >> take this to be 32-bit Spur? > > yes of course. > We are already working on 64bits spur but is not ready yet (it will be some point of 2016) > >> >> Is it feasible to jump the image build numbers to make that first >> image requiring a new VM align on a 100s build numbers. I see we are >> currently 50496. 50500 is nice and distinctive, but even even 50600 >> would be fine if the last few 400s are consumed in the meantime. > > mmm… that I don’t know… correlation of version numbers is usually important for us (for statistics purposes)… also changing number generation is more annoying than you would think (that’s why we don’t have 4.1, 4.2, etc.). > but well… everything is possible, we’ll see :) Or do we just append "-spur" to the build number - but there will be no new builds without the "-spur" tag, and continue this up to Pharo 5 Release. I am just thinking of making it easy to distinguish the required VM when I want to open old builds to bisect which build instorduced a bug. cheer -ben |
again, harder than it seems (because releasing a version is stupidly-hard)… I’m thinking on move all 50 pre-spur into a 49 directory (4.9), then all is correctly separated in file server. This is the easiest I can do it… I think :) Esteban
|
On Mon, Dec 14, 2015 at 8:08 PM, Esteban Lorenzano <[hidden email]> wrote:
> again, harder than it seems (because releasing a version is stupidly-hard)… > I’m thinking on move all 50 pre-spur into a 49 directory (4.9), then all is > correctly separated in file server. > This is the easiest I can do it… I think :) That sounds fine, but maybe the directory can be "50preSpur" (if possible) ? cheers -ben |
> On 14 Dec 2015, at 13:15, Ben Coman <[hidden email]> wrote: > > On Mon, Dec 14, 2015 at 8:08 PM, Esteban Lorenzano <[hidden email]> wrote: >> again, harder than it seems (because releasing a version is stupidly-hard)… >> I’m thinking on move all 50 pre-spur into a 49 directory (4.9), then all is >> correctly separated in file server. >> This is the easiest I can do it… I think :) > > That sounds fine, but maybe the directory can be "50preSpur" (if possible) ? yeah, it could. > cheers -ben > |
In reply to this post by EstebanLM
Thank you Eliot & Esteban for making this happen!
Exciting times :) Cheers, Doru > On Dec 14, 2015, at 11:24 AM, Esteban Lorenzano <[hidden email]> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > > -- www.tudorgirba.com www.feenk.com "Presenting is storytelling." |
In reply to this post by EstebanLM
Excellent work, thanks a lot !! > On Dec 14, 2015, at 07:24, Esteban Lorenzano <[hidden email]> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > > > ---> Save our in-boxes! http://emailcharter.org <--- Johan Fabry - http://pleiad.cl/~jfabry PLEIAD and RyCh labs - Computer Science Department (DCC) - University of Chile |
Pharo 5 your mission should you choose to accept it is to integrate with Spur this image will self destruct in 5 4 3 2 1 On Mon, Dec 14, 2015 at 4:15 PM Johan Fabry <[hidden email]> wrote:
|
In reply to this post by Ben Coman
Ben Coman <[hidden email]> writes:
>> 5) You will need to adapt your Pharo 5.0 related apps and CI >> processes (to take care about new VM). Some programs will stop work >> at all (for example, I think Pharo-launcher will need to be adapted). > > Maybe PharoLauncher can be a special hardcoded rule to have two > template headings... > * Pharo 5.0 Spur (beta); and > * Pharo 5.0 Pre-Spur (beta) > Enhanced multi-VM support perhaps based off the template headings would be nice. what happens when executing the VM on an incompatible image (new VM on old image, old VM on new image)? If the launcher can detect the problem, it can also try to open the selected image with a different VM. Launcher2 will be more flexible, but I need someone to help. -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill |
Excerpts from Damien Cassou's message of 2015-12-14 16:25:15 +0100:
> Launcher2 will be more flexible, but I need someone to help. what kind of help do you need? greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin Bähr working in china http://societyserver.org/mbaehr/ |
Martin Bähr <[hidden email]> writes:
> Excerpts from Damien Cassou's message of 2015-12-14 16:25:15 +0100: >> Launcher2 will be more flexible, but I need someone to help. > > what kind of help do you need? - I need people to help me think about the new GUI design and interactions (I have some mockups but they need some love) - I need people to extract pieces of current launcher into external packages (I already created a Jenkins Pharo project https://github.com/DamienCassou/pharo-jenkins) - I need someone to leverage Cocoon to specify all launcher settings users might want to tweak (proxy configuration, visible jenkins servers, location of images and cache...) http://www.smalltalkhub.com/#!/PharoExtras/Cocoon - I need documentation about Brick - I need people to design a CLI interface to the launcher - ... -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill |
In reply to this post by Tudor Girba-2
> On 14 Dec 2015, at 14:14, Tudor Girba <[hidden email]> wrote: > > Thank you Eliot & Esteban for making this happen! Yes! > Exciting times :) Yes! > Cheers, > Doru > > > >> On Dec 14, 2015, at 11:24 AM, Esteban Lorenzano <[hidden email]> wrote: >> >> We will start migration to Spur today. >> To complete it, we will require some time, specially to adapt the CI and check everything is ok. >> >> Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: >> - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). >> - No more GC stops (noticeable when running large systems) >> - We will be able to scale our systems up to 2G memory consumption without loosing performance. >> >> But, this will have some drawbacks in the first times: >> >> 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). >> - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. >> >> 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: >> - there are some stuff not possible to compatibilise, notably: >> - Structures now need to inherit from FFIExternalStructure >> - Arrays now are now shadowed >> - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. >> - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. >> 2.1) ASMJIT will be removed from system and put in a separated packages. >> NOTE: There will be a blog post explaining FFI-NB architecture during the week. >> >> 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) >> >> 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. >> >> 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). >> >> > > -- > www.tudorgirba.com > www.feenk.com > > "Presenting is storytelling." > > |
In reply to this post by EstebanLM
Sorry for the stupid question, but how should I adapt my CI job? I saw that the last build failed with no tests being run, so I guess this was due to the VM being changed. > On Dec 14, 2015, at 07:24, Esteban Lorenzano <[hidden email]> wrote: > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). ---> Save our in-boxes! http://emailcharter.org <--- Johan Fabry - http://pleiad.cl/~jfabry PLEIAD and RyCh labs - Computer Science Department (DCC) - University of Chile |
Free forum by Nabble | Edit this page |