Ever growing image
Locked
123
123
 
|
Hi,
Does the current 64bits VM suffers of the ever growing image syndrom? Thanks Hilaire -- Dr. Geo http://drgeo.eu |
|
> On 1 Mar 2018, at 15:55, Hilaire <[hidden email]> wrote: > > Hi, > > Does the current 64bits VM suffers of the ever growing image syndrom? no. but the “easy leaking” syndrom, yes ;) Esteban |
|
|
Le 01/03/2018 à 16:03, Esteban Lorenzano a écrit :
> but the “easy leaking” syndrom, yes;) ah? -- Dr. Geo http://drgeo.eu |
|
|
We have a bug in the GC but this one is siply crashing the image not
growing it. I think that what esteban is mentioning is that your code or some parts of Pharo may be leaking memory. We got some experience with GT for example. Stef On Thu, Mar 1, 2018 at 4:05 PM, Hilaire <[hidden email]> wrote: > Le 01/03/2018 à 16:03, Esteban Lorenzano a écrit : >> >> but the “easy leaking” syndrom, yes;) > > ah? > > > -- > Dr. Geo > http://drgeo.eu > > > |
|
|
ok, thanks for the clarification. It is not really a concern to built
and shrink DrGeo image. Hilaire Le 01/03/2018 à 16:26, Stephane Ducasse a écrit : > We have a bug in the GC but this one is siply crashing the image not > growing it. > I think that what esteban is mentioning is that your code or some > parts of Pharo may be > leaking memory. We got some experience with GT for example. -- Dr. Geo http://drgeo.eu |
|
|
In reply to this post by HilaireFernandes
I use 64bits on Mac and I suffer from this. Today I started a new image because the old one was 1,2 GB in size and starting to behave sluggish.
Norbert > Am 01.03.2018 um 15:55 schrieb Hilaire <[hidden email]>: > > Hi, > > Does the current 64bits VM suffers of the ever growing image syndrom? > > Thanks > > Hilaire > > -- > Dr. Geo > http://drgeo.eu > > > |
|
|
Hi norbert this is "good" to know.
On Thu, Mar 1, 2018 at 5:42 PM, Norbert Hartl <[hidden email]> wrote: > I use 64bits on Mac and I suffer from this. Today I started a new image because the old one was 1,2 GB in size and starting to behave sluggish. > > Norbert > > >> Am 01.03.2018 um 15:55 schrieb Hilaire <[hidden email]>: >> >> Hi, >> >> Does the current 64bits VM suffers of the ever growing image syndrom? >> >> Thanks >> >> Hilaire >> >> -- >> Dr. Geo >> http://drgeo.eu >> >> >> > > |
|
|
In reply to this post by NorbertHartl
It will be great if somebody has images with this problem, and it can be shared without trouble, to share it. So we can investigate if the problem is image or vm related. On 1 Mar 2018 17:43, "Norbert Hartl" <[hidden email]> wrote: I use 64bits on Mac and I suffer from this. Today I started a new image because the old one was 1,2 GB in size and starting to behave sluggish. |
|
|
Sure, here is a DrGeo image of an unreasonnable size, you can
investigage on: https://www.dropbox.com/s/wc18e21p371z28f/DrGeo.app-18.01a.zip?dl=0 It is a based on a P7 64bits, don't use the shipped VMs, there are not matching. Thanks Hilaire Le 02/03/2018 à 09:13, [hidden email] a écrit : > It will be great if somebody has images with this problem, and it can > be shared without trouble, to share it. So we can investigate if the > problem is image or vm related. -- Dr. Geo http://drgeo.eu |
|
|
Thank you On 2 Mar 2018 09:23, "Hilaire" <[hidden email]> wrote: Sure, here is a DrGeo image of an unreasonnable size, you can investigage on: |
|
|
Thanks Pablo! and Hilaire!
On Fri, Mar 2, 2018 at 9:30 AM, [hidden email] <[hidden email]> wrote: > Thank you > > On 2 Mar 2018 09:23, "Hilaire" <[hidden email]> wrote: >> >> Sure, here is a DrGeo image of an unreasonnable size, you can investigage >> on: >> >> https://www.dropbox.com/s/wc18e21p371z28f/DrGeo.app-18.01a.zip?dl=0 >> >> >> It is a based on a P7 64bits, don't use the shipped VMs, there are not >> matching. >> >> Thanks >> >> Hilaire >> >> Le 02/03/2018 à 09:13, [hidden email] a écrit : >>> >>> It will be great if somebody has images with this problem, and it can be >>> shared without trouble, to share it. So we can investigate if the problem is >>> image or vm related. >> >> >> -- >> Dr. Geo >> http://drgeo.eu >> >> >> > |
|
|
Hi Hilaire, is it possible that is the wrong image? This is a 32bits image, and it is around 52 MB, quite similar to the Pharo base image. Thanks. On Fri, Mar 2, 2018 at 11:27 AM, Stephane Ducasse <[hidden email]> wrote: Thanks Pablo! and Hilaire! Pablo Tesone.
[hidden email] |
|
|
Ah, you are right it is a 32bits image.
My bad, Dr.Geo does not build with 64bits image Pharo7.0-64bit-52a28a8 ConfigurationOfDrGeo class methodsFor: 'metacelUnmatched ' in string literal. -> May be should try a more recent P7 64 bits image Note: 52 MB is an overlarge image size for me (down 50% will be nice) Hilaire Le 02/03/2018 à 16:41, [hidden email] a écrit : > Hi Hilaire, is it possible that is the wrong image? > This is a 32bits image, and it is around 52 MB, quite similar to the > Pharo base image. > -- Dr. Geo http://drgeo.eu |
|
|
we should again do some analysis where space is going… the Pharo7 download is now 38MB (the image, decompressed). There are things to look for: 1) wasted space due to not thinking about memory and just “having it”. Of course some of it are things like duplications (we now have 2 browsers, 2 inspectors, 2 debuggers… time to clean up a bit…). Then every test we add takes space, every “help topic”, every feature... But of course sometimes being able to invest memory into a better system can be important, too. Not all growth in memory is negativ if you can afford it. I always like to play the game to think “what would they have done in 1978 if they had this amount of memory in even a $5 machine?" Of course one can go very quickly in the wrong direction, but nevertheless: sometimes it is really worth to think about “spending memory” to buy abstraction. (especially a there are counter measures… we under-utilise both compression and disk storage) 2) wasted space that is just not needed. e.g. there is issue That we have some huge strings to init unicode. Do we need them? 3) Memory leaks, bit they are more for the case when the system is running for a while. Marcus |
|
|
Le 02/03/2018 à 22:52, Marcus Denker a écrit :
> > we should again do some analysis where space is going… the Pharo7 download > is now 38MB (the image, decompressed). Situation improved :) Remember P3 was just above 20 MB > > There are things to look for: > > 1) wasted space due to not thinking about memory and just “having it”. > > Of course some of it are things like duplications (we now have 2 > browsers, 2 inspectors, > 2 debuggers… time to clean up a bit…). > > Then every test we add takes space, every “help topic”, every feature... It looks like this topic comes and goes since 20 years: new features added, image growth then/or it becomes hard to unload ; on Squeak then now with Pharo at a much higher rate. We all know about the right approach (GNU Smalltalk, CUIS Smalltalk), but it is not the road chosen with Pharo and it is becoming extremely uninteresting for me to code on DrGeo now. So far I fell ashame to ship end user application with a 50MB DrGeo image, knowing only 10% has purpose related to DrGeo domain. When situation improves, motivation may come back. For this same reason and other, I turned to Python for a planed programming course to mid high school students. > > But of course sometimes being able to invest memory into a better > system can be important, too. > Not all growth in memory is negativ if you can afford it. > > I always like to play the game to think “what would they have done in > 1978 if they had this amount > of memory in even a $5 machine?" Of course one can go very quickly in > the wrong direction, but Very likely not that much. May be the wonders came because of the hight constraints. > nevertheless: sometimes it is really worth to think about “spending > memory” to buy abstraction. > (especially a there are counter measures… we under-utilise both > compression and disk storage) Helps could (should?) be file based, and maybe other stuff I don't know. How the image is considered in the Pharo team? A kind of sacred place, where stuff are added following a few guidelines (you will not duplicate features but replace/imprive/whatever -- aka funny shortcut mess, you will make your code unloadable, etc...) or just like a forum where teams add feature according to projects or vision. Sorry to discuss these topics again, really don't want but still need to share a few thoughts. > > 2) wasted space that is just not needed. e.g. there is issue > https://pharo.fogbugz.com/f/cases/21172/ > That we have some huge strings to init unicode. Do we need them? > > 3) Memory leaks, bit they are more for the case when the system is > running for a while. > -- Dr. Geo http://drgeo.eu |
Re: Ever growing image
|
|
On a side note, being a long-time VW & VAST user, I've always felt the Image Stripper & Runtime Packager did a fairly good job at removing crap from the resulting image. I don't recall ready anything substantial comparing the Pharo way (bootstrapping from a minimal image and adding stuff) to the VW & VAST way (removing unreferenced stuff) but this would definitely be an interesting topic to read about. ----------------- Benoît St-Jean Yahoo! Messenger: bstjean Twitter: @BenLeChialeux Pinterest: benoitstjean Instagram: Chef_Benito IRC: lamneth Blogue: endormitoire.wordpress.com "A standpoint is an intellectual horizon of radius zero". (A. Einstein)
On Saturday, March 3, 2018, 9:22:06 AM EST, Hilaire <[hidden email]> wrote:
Le 02/03/2018 à 22:52, Marcus Denker a écrit : > > we should again do some analysis where space is going… the Pharo7 download > is now 38MB (the image, decompressed). Situation improved :) Remember P3 was just above 20 MB |
|
|
In reply to this post by HilaireFernandes
Hi Hilaire,
> On 3 Mar 2018, at 15:21, Hilaire <[hidden email]> wrote: > > Le 02/03/2018 à 22:52, Marcus Denker a écrit : >> >> we should again do some analysis where space is going… the Pharo7 download >> is now 38MB (the image, decompressed). > > Situation improved :) Remember P3 was just above 20 MB > >> >> There are things to look for: >> >> 1) wasted space due to not thinking about memory and just “having it”. >> >> Of course some of it are things like duplications (we now have 2 browsers, 2 inspectors, >> 2 debuggers… time to clean up a bit…). >> >> Then every test we add takes space, every “help topic”, every feature... > > It looks like this topic comes and goes since 20 years: new features added, image growth then/or it becomes hard to unload ; on Squeak then now with Pharo at a much higher rate. > > We all know about the right approach (GNU Smalltalk, CUIS Smalltalk), but it is not the road chosen with Pharo and it is becoming extremely uninteresting for me to code on DrGeo now. So far I fell ashame to ship end user application with a 50MB DrGeo image, knowing only 10% has purpose related to DrGeo domain. When situation improves, motivation may come back. For this same reason and other, I turned to Python for a planed programming course to mid high school students. I think that’s unfair. First, 38m is a really small application nowadays. I think for the amount of features included, 38m is a very small size (just look at any other application around… and of course scripting does not count by itself, it has to be added with all support files it come). Second, “the right approach” is always a matter of opinion. Of course, we believe *we* took the right path :) Third, yes, we added new functionality that could or could not be there (for example, I believe Athens should be a loadable project, not part of the image, but since everybody want to use it and even we were planning to move our tools to use, we included it). And yes, we have duplicated tools that we need to clean. But creating an environment that allow the development of any kind of application will always mean we increase the size of the release artefact. Finally fourth, we invested a LOT of effort on bootstrapping so you can do your image of the size you want. So yes, we deliver an image of 38m, and also a minimal image of 18m which you can use as base to create your production images. And if that’s not enough, you can always do *your own*, since the scripts for doing it are available by just cloning the sources. > >> >> But of course sometimes being able to invest memory into a better system can be important, too. >> Not all growth in memory is negativ if you can afford it. >> >> I always like to play the game to think “what would they have done in 1978 if they had this amount >> of memory in even a $5 machine?" Of course one can go very quickly in the wrong direction, but > > Very likely not that much. May be the wonders came because of the hight constraints. > > >> nevertheless: sometimes it is really worth to think about “spending memory” to buy abstraction. >> (especially a there are counter measures… we under-utilise both compression and disk storage) > Helps could (should?) be file based, and maybe other stuff I don't know. many things could be outside the image, yes. But then we need to distribute them. And that adds complexity. > How the image is considered in the Pharo team? A kind of sacred place, where stuff are added following a few guidelines (you will not duplicate features but replace/imprive/whatever -- aka funny shortcut mess, you will make your code unloadable, etc...) or just like a forum where teams add feature according to projects or vision. We have a close control of what goes inside the image. And we take care things are well documented and with coherence of their components. Of course, we have made mistakes (maybe a lot), but in general things are improving. Esteban > > Sorry to discuss these topics again, really don't want but still need to share a few thoughts. > >> >> 2) wasted space that is just not needed. e.g. there is issue >> https://pharo.fogbugz.com/f/cases/21172/ >> That we have some huge strings to init unicode. Do we need them? >> >> 3) Memory leaks, bit they are more for the case when the system is running for a while. >> > > -- > Dr. Geo > http://drgeo.eu > > > |
|
|
In reply to this post by Pharo Smalltalk Users mailing list
No idea if any “substantial" thing has been written (there are some phds around that may talk about), but main difference is very simple to explain: first approach (which was the same for Pharo before) is non-deterministic while the second is. Then, for purpose of repeatability of process (and testing, etc.) bootstrapping is better. You would not know the amount of effort made to make this possible, it required at least 2 phd works and a considerable amount of engineering effort. In our community, Pavel is the person that has the bigger experience on shrinking process images. I followed his work for years and I know the fragility of the process. Basically each new addition/removal was breaking it and needing dependency tracking, etc. Now is the opposite: you just add what you want, and you are sure the resulting image is a) healthy and b) identical when running. Esteban
|
Re: Ever growing image
|
|
Estaban, I understand the difference between the 2 approaches my point was just to mention that I guess there are some advantages to shrink the image (as compared to bootstrapping from a minimal image) since all major vendors used that approach (and are still using it). It's just that I was curious to know if anyone knows any paper/document/study comparing the 2 approaches or explaining in detail both of them. I googled for a few hours but couldn't find anything... ----------------- Benoît St-Jean Yahoo! Messenger: bstjean Twitter: @BenLeChialeux Pinterest: benoitstjean Instagram: Chef_Benito IRC: lamneth Blogue: endormitoire.wordpress.com "A standpoint is an intellectual horizon of radius zero". (A. Einstein)
On Sunday, March 4, 2018, 5:13:30 AM EST, Esteban Lorenzano <[hidden email]> wrote:
No idea if any “substantial" thing has been written (there are some phds around that may talk about), but main difference is very simple to explain: first approach (which was the same for Pharo before) is non-deterministic while the second is. Then, for purpose of repeatability of process (and testing, etc.) bootstrapping is better. You would not know the amount of effort made to make this possible, it required at least 2 phd works and a considerable amount of engineering effort. In our community, Pavel is the person that has the bigger experience on shrinking process images. I followed his work for years and I know the fragility of the process. Basically each new addition/removal was breaking it and needing dependency tracking, etc. Now is the opposite: you just add what you want, and you are sure the resulting image is a) healthy and b) identical when running. Esteban
|
|
|
I think that vendors did not go the bootstrap way because it is a lot
more difficult to implement than a bunch of scripts to unload packages. On Sun, Mar 4, 2018 at 11:53 AM, Benoit St-Jean via Pharo-users <[hidden email]> wrote: > > > ---------- Forwarded message ---------- > From: Benoit St-Jean <[hidden email]> > To: Any question about pharo is welcome <[hidden email]>, Esteban Lorenzano <[hidden email]> > Cc: Hilaire <[hidden email]> > Bcc: > Date: Sun, 4 Mar 2018 10:53:04 +0000 (UTC) > Subject: Re: [Pharo-users] Ever growing image > Estaban, > > I understand the difference between the 2 approaches my point was just to mention that I guess there are some advantages to shrink the image (as compared to bootstrapping from a minimal image) since all major vendors used that approach (and are still using it). It's just that I was curious to know if anyone knows any paper/document/study comparing the 2 approaches or explaining in detail both of them. I googled for a few hours but couldn't find anything... > > > ----------------- > Benoît St-Jean > Yahoo! Messenger: bstjean > Twitter: @BenLeChialeux > Pinterest: benoitstjean > Instagram: Chef_Benito > IRC: lamneth > Blogue: endormitoire.wordpress.com > "A standpoint is an intellectual horizon of radius zero". (A. Einstein) > > > On Sunday, March 4, 2018, 5:13:30 AM EST, Esteban Lorenzano <[hidden email]> wrote: > > > > > On 3 Mar 2018, at 16:35, Benoit St-Jean via Pharo-users <[hidden email]> wrote: > > > From: Benoit St-Jean <[hidden email]> > Subject: Re: [Pharo-users] Ever growing image > Date: 3 March 2018 at 16:35:19 CET > To: [hidden email], Hilaire <[hidden email]> > Reply-To: Benoit St-Jean <[hidden email]> > > > On a side note, being a long-time VW & VAST user, I've always felt the Image Stripper & Runtime Packager did a fairly good job at removing crap from the resulting image. > > I don't recall ready anything substantial comparing the Pharo way (bootstrapping from a minimal image and adding stuff) to the VW & VAST way (removing unreferenced stuff) but this would definitely be an interesting topic to read about. > > > > No idea if any “substantial" thing has been written (there are some phds around that may talk about), but main difference is very simple to explain: first approach (which was the same for Pharo before) is non-deterministic while the second is. Then, for purpose of repeatability of process (and testing, etc.) bootstrapping is better. > You would not know the amount of effort made to make this possible, it required at least 2 phd works and a considerable amount of engineering effort. > In our community, Pavel is the person that has the bigger experience on shrinking process images. I followed his work for years and I know the fragility of the process. Basically each new addition/removal was breaking it and needing dependency tracking, etc. Now is the opposite: you just add what you want, and you are sure the resulting image is a) healthy and b) identical when running. > > Esteban > > > ----------------- > Benoît St-Jean > Yahoo! Messenger: bstjean > Twitter: @BenLeChialeux > Pinterest: benoitstjean > Instagram: Chef_Benito > IRC: lamneth > Blogue: endormitoire.wordpress.com > "A standpoint is an intellectual horizon of radius zero". (A. Einstein) > > > On Saturday, March 3, 2018, 9:22:06 AM EST, Hilaire <[hidden email]> wrote: > > > Le 02/03/2018 à 22:52, Marcus Denker a écrit : >> >> we should again do some analysis where space is going… the Pharo7 download >> is now 38MB (the image, decompressed). > > Situation improved :) Remember P3 was just above 20 MB > > > > |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |