Re: [Pharo-project] out of memory - cog on windows
Locked
Re: [Pharo-project] out of memory - cog on windows
 
|
On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: > Hi, > > I am back with some more input on the matter. We played with it, and found that basically any image that goes beyond something like 200MB limit (or maybe it's the number of objects), will not open on Windows. > > I would need some help on this. Has nobody else hit this wall, or is it that I am doing something wrong? > > One scenario that we used to reproduce the problem looks like this: > 1. Take a Moose image: > http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip > 2. Run: > 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] > This will basically create some 850000 objects and will get the image to some 400+MB > 3. Save and quit the image > 4. Reopen it > > A ready-made image with the result of this can be found here: > http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip > > This works well on Mac, but on Windows when you reopen the image, you get the out of memory message. > > Cheers, > Doru > > I using latest VMs built by cmake on windows, and everytime i run this image, it opens a 'Space is low' dialog and then image not reacting on anything. VM not crashing however and responds to windows events.. But because UI process is broken an image simply not reacts to anything. I think that the problem is that this dialog are shown at early image startup time, before everything is properly initialized, and because of that UI stalls somewhere. Next things which i tried is to increase a virtual memory limit in sqWin32Alloc.h #ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif first i raised it to 768 Mbytes.. same behavior. then i raised it to 1Gb and again same behavior.. So, either this constant is overridden somewhere or it actually doesn't affecting the low-space detection mechanism as i would expect. Any suggestions? I will continue looking , what happens on VM side, but in addition to that, i think we should do something on image side as well: - a low space watcher should not pop up before passing image startup, because if preempted process is UI process (and in 99.99% cases it is), then it means that low space watcher blocks UI process and as a consequence, your image stops handling events. Also, i thinking about different approach for signaling a low space condition. Instead of signaling a low space semaphore, what VM could do is to fail an allocation primitive and set the error code to "low memory warning" then a primitive failure code could actually throw an exception, which then could be handled as usual... so you could write a code, like: [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] ifTrue: [ err pass ] ] Because by preempting a currently active process, which "possibly" causing a memory problems is not a solution, as you can see, if it happens during startup phase, then you it just stucks and image hangs. But if we could use exceptions, we could just ignore this warning (or do something else) during image startup, instead of blocking UI process , showing a useless popup message and letting image hang like that. -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] out of memory - cog on windows
|
|
Hi, Is there a possibility to have any sort of answer on this topic? It looks to me like this bug is critical given that it prevents us to work with images larger than some 200M. Cheers, Tudor On 16 May 2011, at 11:06, Tudor Girba wrote: > Hi Igor, > > Thanks a lot for looking into this. For us (around Moose), this is a critical issue because we play with our models in memory and regularly have images of several hundred megabytes. While most of us work on Mac without a problem, some other people work on Windows and this is a bottleneck. > > Unfortunately, I do not have enough know-how to dive into the VM, but I would be gladly help in any way I can. > > Cheers, > Doru > > > On 16 May 2011, at 10:50, Igor Stasenko wrote: > >> On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: >>> Hi, >>> >>> I am back with some more input on the matter. We played with it, and found that basically any image that goes beyond something like 200MB limit (or maybe it's the number of objects), will not open on Windows. >>> >>> I would need some help on this. Has nobody else hit this wall, or is it that I am doing something wrong? >>> >>> One scenario that we used to reproduce the problem looks like this: >>> 1. Take a Moose image: >>> http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip >>> 2. Run: >>> 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] >>> This will basically create some 850000 objects and will get the image to some 400+MB >>> 3. Save and quit the image >>> 4. Reopen it >>> >>> A ready-made image with the result of this can be found here: >>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>> >>> This works well on Mac, but on Windows when you reopen the image, you get the out of memory message. >>> >>> Cheers, >>> Doru >>> >>> >> >> I using latest VMs built by cmake on windows, and everytime i run this image, >> it opens a 'Space is low' dialog and then image not reacting on anything. >> VM not crashing however and responds to windows events.. But because >> UI process is broken >> an image simply not reacts to anything. >> >> >> I think that the problem is that this dialog are shown at early image >> startup time, >> before everything is properly initialized, and because of that UI >> stalls somewhere. >> >> >> Next things which i tried is to increase a virtual memory limit in >> sqWin32Alloc.h >> >> #ifndef MAX_VIRTUAL_MEMORY >> #define MAX_VIRTUAL_MEMORY 512*1024*1024 >> #endif >> >> first i raised it to 768 Mbytes.. same behavior. >> then i raised it to 1Gb and again same behavior.. >> >> So, either this constant is overridden somewhere or it actually >> doesn't affecting the low-space detection mechanism as i would expect. >> Any suggestions? >> >> >> I will continue looking , what happens on VM side, >> but in addition to that, i think we should do something on image side as well: >> - a low space watcher should not pop up before passing image startup, >> because if preempted process is UI process (and in 99.99% cases it is), >> then it means that low space watcher blocks UI process and as a consequence, >> your image stops handling events. >> >> >> Also, i thinking about different approach for signaling a low space condition. >> Instead of signaling a low space semaphore, what VM could do is to >> fail an allocation primitive >> and set the error code to "low memory warning" >> then a primitive failure code could actually throw an exception, which >> then could be handled as usual... >> so you could write a code, like: >> >> [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | >> self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] >> ifTrue: [ err pass ] >> ] >> >> Because by preempting a currently active process, which "possibly" >> causing a memory problems is not a solution, >> as you can see, if it happens during startup phase, then you it just >> stucks and image hangs. >> But if we could use exceptions, we could just ignore this warning (or >> do something else) during image startup, >> instead of blocking UI process , showing a useless popup message and >> letting image hang like that. >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > -- > www.tudorgirba.com > > "If you interrupt the barber while he is cutting your hair, > you will end up with a messy haircut." > -- www.tudorgirba.com "Obvious things are difficult to teach." |
Re: [Pharo-project] out of memory - cog on windows
|
|
On Wed, 18 May 2011, Tudor Girba wrote: > > Hi, > > Is there a possibility to have any sort of answer on this topic? > > It looks to me like this bug is critical given that it prevents us to work with images larger than some 200M. IMHO it's a problem on the image side. Change the behavior of #lowSpaceWatcher to fix the problem. Levente > > Cheers, > Tudor > > > > On 16 May 2011, at 11:06, Tudor Girba wrote: > >> Hi Igor, >> >> Thanks a lot for looking into this. For us (around Moose), this is a critical issue because we play with our models in memory and regularly have images of several hundred megabytes. While most of us work on Mac without a problem, some other people work on Windows and this is a bottleneck. >> >> Unfortunately, I do not have enough know-how to dive into the VM, but I would be gladly help in any way I can. >> >> Cheers, >> Doru >> >> >> On 16 May 2011, at 10:50, Igor Stasenko wrote: >> >>> On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: >>>> Hi, >>>> >>>> I am back with some more input on the matter. We played with it, and found that basically any image that goes beyond something like 200MB limit (or maybe it's the number of objects), will not open on Windows. >>>> >>>> I would need some help on this. Has nobody else hit this wall, or is it that I am doing something wrong? >>>> >>>> One scenario that we used to reproduce the problem looks like this: >>>> 1. Take a Moose image: >>>> http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip >>>> 2. Run: >>>> 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] >>>> This will basically create some 850000 objects and will get the image to some 400+MB >>>> 3. Save and quit the image >>>> 4. Reopen it >>>> >>>> A ready-made image with the result of this can be found here: >>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>> >>>> This works well on Mac, but on Windows when you reopen the image, you get the out of memory message. >>>> >>>> Cheers, >>>> Doru >>>> >>>> >>> >>> I using latest VMs built by cmake on windows, and everytime i run this image, >>> it opens a 'Space is low' dialog and then image not reacting on anything. >>> VM not crashing however and responds to windows events.. But because >>> UI process is broken >>> an image simply not reacts to anything. >>> >>> >>> I think that the problem is that this dialog are shown at early image >>> startup time, >>> before everything is properly initialized, and because of that UI >>> stalls somewhere. >>> >>> >>> Next things which i tried is to increase a virtual memory limit in >>> sqWin32Alloc.h >>> >>> #ifndef MAX_VIRTUAL_MEMORY >>> #define MAX_VIRTUAL_MEMORY 512*1024*1024 >>> #endif >>> >>> first i raised it to 768 Mbytes.. same behavior. >>> then i raised it to 1Gb and again same behavior.. >>> >>> So, either this constant is overridden somewhere or it actually >>> doesn't affecting the low-space detection mechanism as i would expect. >>> Any suggestions? >>> >>> >>> I will continue looking , what happens on VM side, >>> but in addition to that, i think we should do something on image side as well: >>> - a low space watcher should not pop up before passing image startup, >>> because if preempted process is UI process (and in 99.99% cases it is), >>> then it means that low space watcher blocks UI process and as a consequence, >>> your image stops handling events. >>> >>> >>> Also, i thinking about different approach for signaling a low space condition. >>> Instead of signaling a low space semaphore, what VM could do is to >>> fail an allocation primitive >>> and set the error code to "low memory warning" >>> then a primitive failure code could actually throw an exception, which >>> then could be handled as usual... >>> so you could write a code, like: >>> >>> [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | >>> self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] >>> ifTrue: [ err pass ] >>> ] >>> >>> Because by preempting a currently active process, which "possibly" >>> causing a memory problems is not a solution, >>> as you can see, if it happens during startup phase, then you it just >>> stucks and image hangs. >>> But if we could use exceptions, we could just ignore this warning (or >>> do something else) during image startup, >>> instead of blocking UI process , showing a useless popup message and >>> letting image hang like that. >>> >>> -- >>> Best regards, >>> Igor Stasenko AKA sig. >> >> -- >> www.tudorgirba.com >> >> "If you interrupt the barber while he is cutting your hair, >> you will end up with a messy haircut." >> > > -- > www.tudorgirba.com > > "Obvious things are difficult to teach." > > > > |
Re: [Pharo-project] out of memory - cog on windows
|
|
Hi, On 18 May 2011, at 16:30, Levente Uzonyi wrote: > On Wed, 18 May 2011, Tudor Girba wrote: > >> >> Hi, >> >> Is there a possibility to have any sort of answer on this topic? >> >> It looks to me like this bug is critical given that it prevents us to work with images larger than some 200M. > > IMHO it's a problem on the image side. Change the behavior of #lowSpaceWatcher to fix the problem. How would you do it? :) And if it is an image problem, why does it only appear on Windows? Cheers, Doru > > Levente > >> >> Cheers, >> Tudor >> >> >> >> On 16 May 2011, at 11:06, Tudor Girba wrote: >> >>> Hi Igor, >>> >>> Thanks a lot for looking into this. For us (around Moose), this is a critical issue because we play with our models in memory and regularly have images of several hundred megabytes. While most of us work on Mac without a problem, some other people work on Windows and this is a bottleneck. >>> >>> Unfortunately, I do not have enough know-how to dive into the VM, but I would be gladly help in any way I can. >>> >>> Cheers, >>> Doru >>> >>> >>> On 16 May 2011, at 10:50, Igor Stasenko wrote: >>> >>>> On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: >>>>> Hi, >>>>> >>>>> I am back with some more input on the matter. We played with it, and found that basically any image that goes beyond something like 200MB limit (or maybe it's the number of objects), will not open on Windows. >>>>> >>>>> I would need some help on this. Has nobody else hit this wall, or is it that I am doing something wrong? >>>>> >>>>> One scenario that we used to reproduce the problem looks like this: >>>>> 1. Take a Moose image: >>>>> http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip >>>>> 2. Run: >>>>> 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] >>>>> This will basically create some 850000 objects and will get the image to some 400+MB >>>>> 3. Save and quit the image >>>>> 4. Reopen it >>>>> >>>>> A ready-made image with the result of this can be found here: >>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>> >>>>> This works well on Mac, but on Windows when you reopen the image, you get the out of memory message. >>>>> >>>>> Cheers, >>>>> Doru >>>>> >>>>> >>>> >>>> I using latest VMs built by cmake on windows, and everytime i run this image, >>>> it opens a 'Space is low' dialog and then image not reacting on anything. >>>> VM not crashing however and responds to windows events.. But because >>>> UI process is broken >>>> an image simply not reacts to anything. >>>> >>>> >>>> I think that the problem is that this dialog are shown at early image >>>> startup time, >>>> before everything is properly initialized, and because of that UI >>>> stalls somewhere. >>>> >>>> >>>> Next things which i tried is to increase a virtual memory limit in >>>> sqWin32Alloc.h >>>> >>>> #ifndef MAX_VIRTUAL_MEMORY >>>> #define MAX_VIRTUAL_MEMORY 512*1024*1024 >>>> #endif >>>> >>>> first i raised it to 768 Mbytes.. same behavior. >>>> then i raised it to 1Gb and again same behavior.. >>>> >>>> So, either this constant is overridden somewhere or it actually >>>> doesn't affecting the low-space detection mechanism as i would expect. >>>> Any suggestions? >>>> >>>> >>>> I will continue looking , what happens on VM side, >>>> but in addition to that, i think we should do something on image side as well: >>>> - a low space watcher should not pop up before passing image startup, >>>> because if preempted process is UI process (and in 99.99% cases it is), >>>> then it means that low space watcher blocks UI process and as a consequence, >>>> your image stops handling events. >>>> >>>> >>>> Also, i thinking about different approach for signaling a low space condition. >>>> Instead of signaling a low space semaphore, what VM could do is to >>>> fail an allocation primitive >>>> and set the error code to "low memory warning" >>>> then a primitive failure code could actually throw an exception, which >>>> then could be handled as usual... >>>> so you could write a code, like: >>>> >>>> [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | >>>> self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] >>>> ifTrue: [ err pass ] >>>> ] >>>> >>>> Because by preempting a currently active process, which "possibly" >>>> causing a memory problems is not a solution, >>>> as you can see, if it happens during startup phase, then you it just >>>> stucks and image hangs. >>>> But if we could use exceptions, we could just ignore this warning (or >>>> do something else) during image startup, >>>> instead of blocking UI process , showing a useless popup message and >>>> letting image hang like that. >>>> >>>> -- >>>> Best regards, >>>> Igor Stasenko AKA sig. >>> >>> -- >>> www.tudorgirba.com >>> >>> "If you interrupt the barber while he is cutting your hair, >>> you will end up with a messy haircut." >>> >> >> -- >> www.tudorgirba.com >> >> "Obvious things are difficult to teach." >> >> >> >> -- www.tudorgirba.com "Next time you see your life passing by, say 'hi' and get to know her." |
Re: [Pharo-project] out of memory - cog on windows
|
|
On Wed, 18 May 2011, Tudor Girba wrote: > > Hi, > > On 18 May 2011, at 16:30, Levente Uzonyi wrote: > >> On Wed, 18 May 2011, Tudor Girba wrote: >> >>> >>> Hi, >>> >>> Is there a possibility to have any sort of answer on this topic? >>> >>> It looks to me like this bug is critical given that it prevents us to work with images larger than some 200M. >> >> IMHO it's a problem on the image side. Change the behavior of #lowSpaceWatcher to fix the problem. > > How would you do it? :) According to Igor's mail, the UI process is suspended by the low space watcher, which prevents you from seeing the cause of the problem. So, for example you could wake up the UI process if that was suspended by the VM and/or log some info about the situation. This won't save your image, but since it works on non-windows VMs, that's not a problem. > > And if it is an image problem, why does it only appear on Windows? Are you using the same VMs (interpreter/cog, version number matches, etc)? If you didn't try so far, then open the image with the latest interpreter VM on windows, it has some extra checks for allocations which is not implemented in Cog yet. Levente > > > Cheers, > Doru > > > >> >> Levente >> >>> >>> Cheers, >>> Tudor >>> >>> >>> >>> On 16 May 2011, at 11:06, Tudor Girba wrote: >>> >>>> Hi Igor, >>>> >>>> Thanks a lot for looking into this. For us (around Moose), this is a critical issue because we play with our models in memory and regularly have images of several hundred megabytes. While most of us work on Mac without a problem, some other people work on Windows and this is a bottleneck. >>>> >>>> Unfortunately, I do not have enough know-how to dive into the VM, but I would be gladly help in any way I can. >>>> >>>> Cheers, >>>> Doru >>>> >>>> >>>> On 16 May 2011, at 10:50, Igor Stasenko wrote: >>>> >>>>> On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: >>>>>> Hi, >>>>>> >>>>>> I am back with some more input on the matter. We played with it, and found that basically any image that goes beyond something like 200MB limit (or maybe it's the number of objects), will not open on Windows. >>>>>> >>>>>> I would need some help on this. Has nobody else hit this wall, or is it that I am doing something wrong? >>>>>> >>>>>> One scenario that we used to reproduce the problem looks like this: >>>>>> 1. Take a Moose image: >>>>>> http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip >>>>>> 2. Run: >>>>>> 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] >>>>>> This will basically create some 850000 objects and will get the image to some 400+MB >>>>>> 3. Save and quit the image >>>>>> 4. Reopen it >>>>>> >>>>>> A ready-made image with the result of this can be found here: >>>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>>> >>>>>> This works well on Mac, but on Windows when you reopen the image, you get the out of memory message. >>>>>> >>>>>> Cheers, >>>>>> Doru >>>>>> >>>>>> >>>>> >>>>> I using latest VMs built by cmake on windows, and everytime i run this image, >>>>> it opens a 'Space is low' dialog and then image not reacting on anything. >>>>> VM not crashing however and responds to windows events.. But because >>>>> UI process is broken >>>>> an image simply not reacts to anything. >>>>> >>>>> >>>>> I think that the problem is that this dialog are shown at early image >>>>> startup time, >>>>> before everything is properly initialized, and because of that UI >>>>> stalls somewhere. >>>>> >>>>> >>>>> Next things which i tried is to increase a virtual memory limit in >>>>> sqWin32Alloc.h >>>>> >>>>> #ifndef MAX_VIRTUAL_MEMORY >>>>> #define MAX_VIRTUAL_MEMORY 512*1024*1024 >>>>> #endif >>>>> >>>>> first i raised it to 768 Mbytes.. same behavior. >>>>> then i raised it to 1Gb and again same behavior.. >>>>> >>>>> So, either this constant is overridden somewhere or it actually >>>>> doesn't affecting the low-space detection mechanism as i would expect. >>>>> Any suggestions? >>>>> >>>>> >>>>> I will continue looking , what happens on VM side, >>>>> but in addition to that, i think we should do something on image side as well: >>>>> - a low space watcher should not pop up before passing image startup, >>>>> because if preempted process is UI process (and in 99.99% cases it is), >>>>> then it means that low space watcher blocks UI process and as a consequence, >>>>> your image stops handling events. >>>>> >>>>> >>>>> Also, i thinking about different approach for signaling a low space condition. >>>>> Instead of signaling a low space semaphore, what VM could do is to >>>>> fail an allocation primitive >>>>> and set the error code to "low memory warning" >>>>> then a primitive failure code could actually throw an exception, which >>>>> then could be handled as usual... >>>>> so you could write a code, like: >>>>> >>>>> [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | >>>>> self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] >>>>> ifTrue: [ err pass ] >>>>> ] >>>>> >>>>> Because by preempting a currently active process, which "possibly" >>>>> causing a memory problems is not a solution, >>>>> as you can see, if it happens during startup phase, then you it just >>>>> stucks and image hangs. >>>>> But if we could use exceptions, we could just ignore this warning (or >>>>> do something else) during image startup, >>>>> instead of blocking UI process , showing a useless popup message and >>>>> letting image hang like that. >>>>> >>>>> -- >>>>> Best regards, >>>>> Igor Stasenko AKA sig. >>>> >>>> -- >>>> www.tudorgirba.com >>>> >>>> "If you interrupt the barber while he is cutting your hair, >>>> you will end up with a messy haircut." >>>> >>> >>> -- >>> www.tudorgirba.com >>> >>> "Obvious things are difficult to teach." >>> >>> >>> >>> > > -- > www.tudorgirba.com > > "Next time you see your life passing by, say 'hi' and get to know her." > > > > |
Re: [Pharo-project] out of memory - cog on windows
|
|
On 18 May 2011 16:51, Levente Uzonyi <[hidden email]> wrote: > > On Wed, 18 May 2011, Tudor Girba wrote: > >> >> Hi, >> >> On 18 May 2011, at 16:30, Levente Uzonyi wrote: >> >>> On Wed, 18 May 2011, Tudor Girba wrote: >>> >>>> >>>> Hi, >>>> >>>> Is there a possibility to have any sort of answer on this topic? >>>> >>>> It looks to me like this bug is critical given that it prevents us to >>>> work with images larger than some 200M. >>> >>> IMHO it's a problem on the image side. Change the behavior of >>> #lowSpaceWatcher to fix the problem. >> >> How would you do it? :) > > According to Igor's mail, the UI process is suspended by the low space > watcher, which prevents you from seeing the cause of the problem. So, for > example you could wake up the UI process if that was suspended by the VM > and/or log some info about the situation. This won't save your image, but > since it works on non-windows VMs, that's not a problem. > Yes. I don't want to change the low-space signaling locgic in VM, since it works not only just for Windows but for other systems. It could be possible that Windows-specific code either reports wrong memory limit (which triggers low-space logic sooner than expected, even when you set the hardcoded virtual memory allocation to be much higher), or actually there is not enough address space on windows, and requesting to allocate contiguous 1Gb address space fails, and actually VM forced to use smaller address space for object memory. But then i would expect another failure, because a reasonably big image (>512Mb) could even not fit into memory during image startup, because there's not enough address space. >> >> And if it is an image problem, why does it only appear on Windows? > > Are you using the same VMs (interpreter/cog, version number matches, etc)? > If you didn't try so far, then open the image with the latest interpreter VM > on windows, it has some extra checks for allocations which is not > implemented in Cog yet. > > > Levente > >> >> >> Cheers, >> Doru >> >> >> >>> >>> Levente >>> >>>> >>>> Cheers, >>>> Tudor >>>> >>>> >>>> >>>> On 16 May 2011, at 11:06, Tudor Girba wrote: >>>> >>>>> Hi Igor, >>>>> >>>>> Thanks a lot for looking into this. For us (around Moose), this is a >>>>> critical issue because we play with our models in memory and regularly have >>>>> images of several hundred megabytes. While most of us work on Mac without a >>>>> problem, some other people work on Windows and this is a bottleneck. >>>>> >>>>> Unfortunately, I do not have enough know-how to dive into the VM, but I >>>>> would be gladly help in any way I can. >>>>> >>>>> Cheers, >>>>> Doru >>>>> >>>>> >>>>> On 16 May 2011, at 10:50, Igor Stasenko wrote: >>>>> >>>>>> On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am back with some more input on the matter. We played with it, and >>>>>>> found that basically any image that goes beyond something like 200MB limit >>>>>>> (or maybe it's the number of objects), will not open on Windows. >>>>>>> >>>>>>> I would need some help on this. Has nobody else hit this wall, or is >>>>>>> it that I am doing something wrong? >>>>>>> >>>>>>> One scenario that we used to reproduce the problem looks like this: >>>>>>> 1. Take a Moose image: >>>>>>> >>>>>>> http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip >>>>>>> 2. Run: >>>>>>> 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] >>>>>>> This will basically create some 850000 objects and will get the image >>>>>>> to some 400+MB >>>>>>> 3. Save and quit the image >>>>>>> 4. Reopen it >>>>>>> >>>>>>> A ready-made image with the result of this can be found here: >>>>>>> >>>>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>>>> >>>>>>> This works well on Mac, but on Windows when you reopen the image, you >>>>>>> get the out of memory message. >>>>>>> >>>>>>> Cheers, >>>>>>> Doru >>>>>>> >>>>>>> >>>>>> >>>>>> I using latest VMs built by cmake on windows, and everytime i run this >>>>>> image, >>>>>> it opens a 'Space is low' dialog and then image not reacting on >>>>>> anything. >>>>>> VM not crashing however and responds to windows events.. But because >>>>>> UI process is broken >>>>>> an image simply not reacts to anything. >>>>>> >>>>>> >>>>>> I think that the problem is that this dialog are shown at early image >>>>>> startup time, >>>>>> before everything is properly initialized, and because of that UI >>>>>> stalls somewhere. >>>>>> >>>>>> >>>>>> Next things which i tried is to increase a virtual memory limit in >>>>>> sqWin32Alloc.h >>>>>> >>>>>> #ifndef MAX_VIRTUAL_MEMORY >>>>>> #define MAX_VIRTUAL_MEMORY 512*1024*1024 >>>>>> #endif >>>>>> >>>>>> first i raised it to 768 Mbytes.. same behavior. >>>>>> then i raised it to 1Gb and again same behavior.. >>>>>> >>>>>> So, either this constant is overridden somewhere or it actually >>>>>> doesn't affecting the low-space detection mechanism as i would expect. >>>>>> Any suggestions? >>>>>> >>>>>> >>>>>> I will continue looking , what happens on VM side, >>>>>> but in addition to that, i think we should do something on image side >>>>>> as well: >>>>>> - a low space watcher should not pop up before passing image startup, >>>>>> because if preempted process is UI process (and in 99.99% cases it >>>>>> is), >>>>>> then it means that low space watcher blocks UI process and as a >>>>>> consequence, >>>>>> your image stops handling events. >>>>>> >>>>>> >>>>>> Also, i thinking about different approach for signaling a low space >>>>>> condition. >>>>>> Instead of signaling a low space semaphore, what VM could do is to >>>>>> fail an allocation primitive >>>>>> and set the error code to "low memory warning" >>>>>> then a primitive failure code could actually throw an exception, which >>>>>> then could be handled as usual... >>>>>> so you could write a code, like: >>>>>> >>>>>> [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | >>>>>> self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] >>>>>> ifTrue: [ err pass ] >>>>>> ] >>>>>> >>>>>> Because by preempting a currently active process, which "possibly" >>>>>> causing a memory problems is not a solution, >>>>>> as you can see, if it happens during startup phase, then you it just >>>>>> stucks and image hangs. >>>>>> But if we could use exceptions, we could just ignore this warning (or >>>>>> do something else) during image startup, >>>>>> instead of blocking UI process , showing a useless popup message and >>>>>> letting image hang like that. >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Igor Stasenko AKA sig. >>>>> >>>>> -- >>>>> www.tudorgirba.com >>>>> >>>>> "If you interrupt the barber while he is cutting your hair, >>>>> you will end up with a messy haircut." >>>>> >>>> >>>> -- >>>> www.tudorgirba.com >>>> >>>> "Obvious things are difficult to teach." >>>> >>>> >>>> >>>> >> >> -- >> www.tudorgirba.com >> >> "Next time you see your life passing by, say 'hi' and get to know her." >> >> >> >> > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] out of memory - cog on windows
|
|
In reply to this post by Levente Uzonyi-2
Hi, On 18 May 2011, at 16:51, Levente Uzonyi wrote: > On Wed, 18 May 2011, Tudor Girba wrote: > >> >> Hi, >> >> On 18 May 2011, at 16:30, Levente Uzonyi wrote: >> >>> On Wed, 18 May 2011, Tudor Girba wrote: >>> >>>> >>>> Hi, >>>> >>>> Is there a possibility to have any sort of answer on this topic? >>>> >>>> It looks to me like this bug is critical given that it prevents us to work with images larger than some 200M. >>> >>> IMHO it's a problem on the image side. Change the behavior of #lowSpaceWatcher to fix the problem. >> >> How would you do it? :) > > According to Igor's mail, the UI process is suspended by the low space watcher, which prevents you from seeing the cause of the problem. So, for example you could wake up the UI process if that was suspended by the VM and/or log some info about the situation. This won't save your image, but since it works on non-windows VMs, that's not a problem. Thanks for the reply. Unfortunately, I do not know how to do that. Furthermore, it's not about one image. These images are produced automatically several times per day, and people want to be able to download them and play with the data inside. And again, having a 200M image cannot be so unusual. I tried in multiple situations to simply generate a large enough image and it won't open. >> And if it is an image problem, why does it only appear on Windows? > > Are you using the same VMs (interpreter/cog, version number matches, etc)? If you didn't try so far, then open the image with the latest interpreter VM on windows, it has some extra checks for allocations which is not implemented in Cog yet. I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. Cheers, Doru > > Levente > >> >> >> Cheers, >> Doru >> >> >> >>> >>> Levente >>> >>>> >>>> Cheers, >>>> Tudor >>>> >>>> >>>> >>>> On 16 May 2011, at 11:06, Tudor Girba wrote: >>>> >>>>> Hi Igor, >>>>> >>>>> Thanks a lot for looking into this. For us (around Moose), this is a critical issue because we play with our models in memory and regularly have images of several hundred megabytes. While most of us work on Mac without a problem, some other people work on Windows and this is a bottleneck. >>>>> >>>>> Unfortunately, I do not have enough know-how to dive into the VM, but I would be gladly help in any way I can. >>>>> >>>>> Cheers, >>>>> Doru >>>>> >>>>> >>>>> On 16 May 2011, at 10:50, Igor Stasenko wrote: >>>>> >>>>>> On 10 May 2011 09:34, Tudor Girba <[hidden email]> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I am back with some more input on the matter. We played with it, and found that basically any image that goes beyond something like 200MB limit (or maybe it's the number of objects), will not open on Windows. >>>>>>> >>>>>>> I would need some help on this. Has nobody else hit this wall, or is it that I am doing something wrong? >>>>>>> >>>>>>> One scenario that we used to reproduce the problem looks like this: >>>>>>> 1. Take a Moose image: >>>>>>> http://hudson.moosetechnology.org/job/moose-latest-dev/lastSuccessfulBuild/artifact/moose/*zip*/moose.zip >>>>>>> 2. Run: >>>>>>> 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] >>>>>>> This will basically create some 850000 objects and will get the image to some 400+MB >>>>>>> 3. Save and quit the image >>>>>>> 4. Reopen it >>>>>>> >>>>>>> A ready-made image with the result of this can be found here: >>>>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>>>> >>>>>>> This works well on Mac, but on Windows when you reopen the image, you get the out of memory message. >>>>>>> >>>>>>> Cheers, >>>>>>> Doru >>>>>>> >>>>>>> >>>>>> >>>>>> I using latest VMs built by cmake on windows, and everytime i run this image, >>>>>> it opens a 'Space is low' dialog and then image not reacting on anything. >>>>>> VM not crashing however and responds to windows events.. But because >>>>>> UI process is broken >>>>>> an image simply not reacts to anything. >>>>>> >>>>>> >>>>>> I think that the problem is that this dialog are shown at early image >>>>>> startup time, >>>>>> before everything is properly initialized, and because of that UI >>>>>> stalls somewhere. >>>>>> >>>>>> >>>>>> Next things which i tried is to increase a virtual memory limit in >>>>>> sqWin32Alloc.h >>>>>> >>>>>> #ifndef MAX_VIRTUAL_MEMORY >>>>>> #define MAX_VIRTUAL_MEMORY 512*1024*1024 >>>>>> #endif >>>>>> >>>>>> first i raised it to 768 Mbytes.. same behavior. >>>>>> then i raised it to 1Gb and again same behavior.. >>>>>> >>>>>> So, either this constant is overridden somewhere or it actually >>>>>> doesn't affecting the low-space detection mechanism as i would expect. >>>>>> Any suggestions? >>>>>> >>>>>> >>>>>> I will continue looking , what happens on VM side, >>>>>> but in addition to that, i think we should do something on image side as well: >>>>>> - a low space watcher should not pop up before passing image startup, >>>>>> because if preempted process is UI process (and in 99.99% cases it is), >>>>>> then it means that low space watcher blocks UI process and as a consequence, >>>>>> your image stops handling events. >>>>>> >>>>>> >>>>>> Also, i thinking about different approach for signaling a low space condition. >>>>>> Instead of signaling a low space semaphore, what VM could do is to >>>>>> fail an allocation primitive >>>>>> and set the error code to "low memory warning" >>>>>> then a primitive failure code could actually throw an exception, which >>>>>> then could be handled as usual... >>>>>> so you could write a code, like: >>>>>> >>>>>> [ self allocateSomethingReallyHuge ] on: LowMemoryWarning do: [:err | >>>>>> self shouldWeReallyCare ifFalse: [ self tellVMThatItsOk. err resume ] >>>>>> ifTrue: [ err pass ] >>>>>> ] >>>>>> >>>>>> Because by preempting a currently active process, which "possibly" >>>>>> causing a memory problems is not a solution, >>>>>> as you can see, if it happens during startup phase, then you it just >>>>>> stucks and image hangs. >>>>>> But if we could use exceptions, we could just ignore this warning (or >>>>>> do something else) during image startup, >>>>>> instead of blocking UI process , showing a useless popup message and >>>>>> letting image hang like that. >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Igor Stasenko AKA sig. >>>>> >>>>> -- >>>>> www.tudorgirba.com >>>>> >>>>> "If you interrupt the barber while he is cutting your hair, >>>>> you will end up with a messy haircut." >>>>> >>>> >>>> -- >>>> www.tudorgirba.com >>>> >>>> "Obvious things are difficult to teach." >>>> >>>> >>>> >>>> >> >> -- >> www.tudorgirba.com >> >> "Next time you see your life passing by, say 'hi' and get to know her." >> >> >> >> -- www.tudorgirba.com "Every thing should have the right to be different." |
Re: [Pharo-project] out of memory - cog on windows
|
|
On 18 May 2011 17:08, Tudor Girba <[hidden email]> wrote: > > Hi, > > On 18 May 2011, at 16:51, Levente Uzonyi wrote: > >> On Wed, 18 May 2011, Tudor Girba wrote: >> >>> >>> Hi, >>> >>> On 18 May 2011, at 16:30, Levente Uzonyi wrote: >>> >>>> On Wed, 18 May 2011, Tudor Girba wrote: >>>> >>>>> >>>>> Hi, >>>>> >>>>> Is there a possibility to have any sort of answer on this topic? >>>>> >>>>> It looks to me like this bug is critical given that it prevents us to work with images larger than some 200M. >>>> >>>> IMHO it's a problem on the image side. Change the behavior of #lowSpaceWatcher to fix the problem. >>> >>> How would you do it? :) >> >> According to Igor's mail, the UI process is suspended by the low space watcher, which prevents you from seeing the cause of the problem. So, for example you could wake up the UI process if that was suspended by the VM and/or log some info about the situation. This won't save your image, but since it works on non-windows VMs, that's not a problem. > > Thanks for the reply. Unfortunately, I do not know how to do that. > > Furthermore, it's not about one image. These images are produced automatically several times per day, and people want to be able to download them and play with the data inside. > > And again, having a 200M image cannot be so unusual. I tried in multiple situations to simply generate a large enough image and it won't open. > >>> And if it is an image problem, why does it only appear on Windows? >> >> Are you using the same VMs (interpreter/cog, version number matches, etc)? If you didn't try so far, then open the image with the latest interpreter VM on windows, it has some extra checks for allocations which is not implemented in Cog yet. > > I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. > @Eliot, can you comment or recommend something concerning this issue? > Cheers, > Doru > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] out of memory - cog on windows
|
|
In reply to this post by Tudor Girba
On Wed, 18 May 2011, Tudor Girba wrote: snip > > I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip Levente snip |
Re: [Pharo-project] out of memory - cog on windows
|
|
In reply to this post by Igor Stasenko
Hi, On 18 May 2011, at 18:07, Levente Uzonyi wrote: > On Wed, 18 May 2011, Tudor Girba wrote: > > snip > >> >> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. > > You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip I tried with this VM, but it crashes with the attached error. In any case, if someone wants to reproduce it, here is an image to play with: http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip Cheers, Doru > > Levente > > snip -- www.tudorgirba.com "Being happy is a matter of choice." |
crash.dmp (1K)
Re: [Pharo-project] out of memory - cog on windows
|
|
On 20 May 2011 15:25, Tudor Girba <[hidden email]> wrote: > Here an excert from crash.dmp: Memory Information (upon launch): Physical Memory Size: 4194303 kbytes Physical Memory Free: 2507424 kbytes Page File Size: 4194303 kbytes Page File Free: 4194303 kbytes Virtual Memory Size: 2097024 kbytes Virtual Memory Free: 2030280 kbytes Memory Load: 59 percent this is strange. Of course, with sizes like above , it cannot fit into memory. > Hi, > > On 18 May 2011, at 18:07, Levente Uzonyi wrote: > >> On Wed, 18 May 2011, Tudor Girba wrote: >> >> snip >> >>> >>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >> >> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip > > I tried with this VM, but it crashes with the attached error. > > In any case, if someone wants to reproduce it, here is an image to play with: > http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip > > Cheers, > Doru > > > > > >> >> Levente >> >> snip > > -- > www.tudorgirba.com > > "Being happy is a matter of choice." > > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] out of memory - cog on windows
|
|
On 20 May 2011 16:57, Igor Stasenko <[hidden email]> wrote: > On 20 May 2011 15:25, Tudor Girba <[hidden email]> wrote: >> > > Here an excert from crash.dmp: > > Memory Information (upon launch): > Â Â Â Â Physical Memory Size: 4194303 kbytes > Â Â Â Â Physical Memory Free: 2507424 kbytes > Â Â Â Â Page File Size: 4194303 kbytes > Â Â Â Â Page File Free: 4194303 kbytes > Â Â Â Â Virtual Memory Size: 2097024 kbytes > Â Â Â Â Virtual Memory Free: 2030280 kbytes > Â Â Â Â Memory Load: 59 percent > > this is strange. > Of course, with sizes like above , it cannot fit into memory. > i thought i was 'bytes' 2507424 kbytes = 2507424 * 1024 bytes, so its more than 2G > >> Hi, >> >> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >> >>> On Wed, 18 May 2011, Tudor Girba wrote: >>> >>> snip >>> >>>> >>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>> >>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >> >> I tried with this VM, but it crashes with the attached error. >> >> In any case, if someone wants to reproduce it, here is an image to play with: >> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >> >> Cheers, >> Doru >> >> >> >> >> >>> >>> Levente >>> >>> snip >> >> -- >> www.tudorgirba.com >> >> "Being happy is a matter of choice." >> >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] out of memory - cog on windows
|
|
In reply to this post by Igor Stasenko
On 5/20/2011 16:57, Igor Stasenko wrote: > Here an excert from crash.dmp: > > Memory Information (upon launch): > Physical Memory Size: 4194303 kbytes > Physical Memory Free: 2507424 kbytes > Page File Size: 4194303 kbytes > Page File Free: 4194303 kbytes > Virtual Memory Size: 2097024 kbytes > Virtual Memory Free: 2030280 kbytes > Memory Load: 59 percent > > this is strange. > Of course, with sizes like above , it cannot fit into memory. Nothing strange about it. Standard 4GB Ram configuration with about 2GB available. Cheers, - Andreas > >> Hi, >> >> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >> >>> On Wed, 18 May 2011, Tudor Girba wrote: >>> >>> snip >>> >>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >> I tried with this VM, but it crashes with the attached error. >> >> In any case, if someone wants to reproduce it, here is an image to play with: >> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >> >> Cheers, >> Doru >> >> >> >> >> >>> Levente >>> >>> snip >> -- >> www.tudorgirba.com >> >> "Being happy is a matter of choice." >> >> > > |
Re: [Pharo-project] out of memory - cog on windows
|
|
Hi, I am not sure I understand where we are on this issue. I will say it again that I find it to be critical to be able to work with images that are not small :). Would it be possible to find some resolution on it? Cheers, Doru On 20 May 2011, at 17:08, Andreas Raab wrote: > On 5/20/2011 16:57, Igor Stasenko wrote: >> Here an excert from crash.dmp: >> >> Memory Information (upon launch): >> Physical Memory Size: 4194303 kbytes >> Physical Memory Free: 2507424 kbytes >> Page File Size: 4194303 kbytes >> Page File Free: 4194303 kbytes >> Virtual Memory Size: 2097024 kbytes >> Virtual Memory Free: 2030280 kbytes >> Memory Load: 59 percent >> >> this is strange. >> Of course, with sizes like above , it cannot fit into memory. > > Nothing strange about it. Standard 4GB Ram configuration with about 2GB available. > > Cheers, > - Andreas > > >> >>> Hi, >>> >>> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >>> >>>> On Wed, 18 May 2011, Tudor Girba wrote: >>>> >>>> snip >>>> >>>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >>> I tried with this VM, but it crashes with the attached error. >>> >>> In any case, if someone wants to reproduce it, here is an image to play with: >>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>> >>> Cheers, >>> Doru >>> >>> >>> >>> >>> >>>> Levente >>>> >>>> snip >>> -- >>> www.tudorgirba.com >>> >>> "Being happy is a matter of choice." >>> >>> >> >> -- www.tudorgirba.com "Every thing has its own flow." |
Re: [Pharo-project] out of memory - cog on windows
|
|
On 5/20/2011 17:19, Tudor Girba wrote: > I am not sure I understand where we are on this issue. I will say it again that I find it to be critical to be able to work with images that are not small :). We are nowhere on this issue. I should add that with OpenQwaq / Teleplace we've been running large images (300-500MB) all the time on Windows with no problems. And I am not at all convinced that this is even a VM issue; I find it more likely that there might be some platform dependency that causes some code to go wrong and actually eat all available memory. My comment below was just pointing out that your OS memory configuration looks perfectly normal. What I would do in your situation is to run the code that created this image directly under Windows, then save and try to start it up again. If you get an error when you run the code, you will be able to tell whether it's a memory problem or not. If the image saves and starts fine again on Windows, you can be pretty sure that it's not a VM problem but some platform dependency. And then you might try to move the image to a Mac to see if that teaches you anything. Cheers, - Andreas > Would it be possible to find some resolution on it? > > Cheers, > Doru > > > On 20 May 2011, at 17:08, Andreas Raab wrote: > >> On 5/20/2011 16:57, Igor Stasenko wrote: >>> Here an excert from crash.dmp: >>> >>> Memory Information (upon launch): >>> Physical Memory Size: 4194303 kbytes >>> Physical Memory Free: 2507424 kbytes >>> Page File Size: 4194303 kbytes >>> Page File Free: 4194303 kbytes >>> Virtual Memory Size: 2097024 kbytes >>> Virtual Memory Free: 2030280 kbytes >>> Memory Load: 59 percent >>> >>> this is strange. >>> Of course, with sizes like above , it cannot fit into memory. >> Nothing strange about it. Standard 4GB Ram configuration with about 2GB available. >> >> Cheers, >> - Andreas >> >> >>>> Hi, >>>> >>>> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >>>> >>>>> On Wed, 18 May 2011, Tudor Girba wrote: >>>>> >>>>> snip >>>>> >>>>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>>>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >>>> I tried with this VM, but it crashes with the attached error. >>>> >>>> In any case, if someone wants to reproduce it, here is an image to play with: >>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>> >>>> Cheers, >>>> Doru >>>> >>>> >>>> >>>> >>>> >>>>> Levente >>>>> >>>>> snip >>>> -- >>>> www.tudorgirba.com >>>> >>>> "Being happy is a matter of choice." >>>> >>>> >>> > -- > www.tudorgirba.com > > "Every thing has its own flow." > > > > > |
Re: [Pharo-project] out of memory - cog on windows
|
|
Hi Andreas, On 20 May 2011, at 17:44, Andreas Raab wrote: > On 5/20/2011 17:19, Tudor Girba wrote: >> I am not sure I understand where we are on this issue. I will say it again that I find it to be critical to be able to work with images that are not small :). > > We are nowhere on this issue. I should add that with OpenQwaq / Teleplace we've been running large images (300-500MB) all the time on Windows with no problems. I also thought that this cannot be that obvious of a problem, because otherwise people would have stumbled over it before. > And I am not at all convinced that this is even a VM issue; I find it more likely that there might be some platform dependency that causes some code to go wrong and actually eat all available memory. My comment below was just pointing out that your OS memory configuration looks perfectly normal. I understood that. I just wanted to pick the discussion up and see what can be done. > What I would do in your situation is to run the code that created this image directly under Windows, then save and try to start it up again. If you get an error when you run the code, you will be able to tell whether it's a memory problem or not. If the image saves and starts fine again on Windows, you can be pretty sure that it's not a VM problem but some platform dependency. And then you might try to move the image to a Mac to see if that teaches you anything. I tried the followings: - Build the image on Windows 7, save, and open on Windows 7 ==> out of memory message - Build the image on Windows 7, save, and open on OS X 10.6 ==> fine - Build the image on OS X 10.6, save, and open on Windows 7 ==> out of memory message To me, it looks like a Windows-related problem. A problem is that I do not have a Windows machine on which I can test this too much, but when I tried, I could reproduce it repeatedly: 1. Take a Moose image: http://ci.moosetechnology.org/job/moose-latest-dev/375/artifact/moose/*zip*/moose.zip 2. Run in a Workspace (it takes a while): 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] This will basically create some 850000 objects and will get the image to some 400+MB 3. Save and quit the image 4. Reopen it Cheers, Doru > Cheers, > - Andreas > >> Would it be possible to find some resolution on it? >> >> Cheers, >> Doru >> >> >> On 20 May 2011, at 17:08, Andreas Raab wrote: >> >>> On 5/20/2011 16:57, Igor Stasenko wrote: >>>> Here an excert from crash.dmp: >>>> >>>> Memory Information (upon launch): >>>> Physical Memory Size: 4194303 kbytes >>>> Physical Memory Free: 2507424 kbytes >>>> Page File Size: 4194303 kbytes >>>> Page File Free: 4194303 kbytes >>>> Virtual Memory Size: 2097024 kbytes >>>> Virtual Memory Free: 2030280 kbytes >>>> Memory Load: 59 percent >>>> >>>> this is strange. >>>> Of course, with sizes like above , it cannot fit into memory. >>> Nothing strange about it. Standard 4GB Ram configuration with about 2GB available. >>> >>> Cheers, >>> - Andreas >>> >>> >>>>> Hi, >>>>> >>>>> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >>>>> >>>>>> On Wed, 18 May 2011, Tudor Girba wrote: >>>>>> >>>>>> snip >>>>>> >>>>>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>>>>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >>>>> I tried with this VM, but it crashes with the attached error. >>>>> >>>>> In any case, if someone wants to reproduce it, here is an image to play with: >>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>> >>>>> Cheers, >>>>> Doru >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Levente >>>>>> >>>>>> snip >>>>> -- >>>>> www.tudorgirba.com >>>>> >>>>> "Being happy is a matter of choice." >>>>> >>>>> >>>> >> -- >> www.tudorgirba.com >> >> "Every thing has its own flow." >> >> >> >> >> -- www.tudorgirba.com "If you interrupt the barber while he is cutting your hair, you will end up with a messy haircut." |
Re: [Pharo-project] out of memory - cog on windows
|
|
Thanks doru. Igor tried to do some experiences creating large images on mac and opening them on windows last friday. So he will certainly have a look at it again Stef On May 21, 2011, at 7:13 PM, Tudor Girba wrote: > > Hi Andreas, > > On 20 May 2011, at 17:44, Andreas Raab wrote: > >> On 5/20/2011 17:19, Tudor Girba wrote: >>> I am not sure I understand where we are on this issue. I will say it again that I find it to be critical to be able to work with images that are not small :). >> >> We are nowhere on this issue. I should add that with OpenQwaq / Teleplace we've been running large images (300-500MB) all the time on Windows with no problems. > > I also thought that this cannot be that obvious of a problem, because otherwise people would have stumbled over it before. > >> And I am not at all convinced that this is even a VM issue; I find it more likely that there might be some platform dependency that causes some code to go wrong and actually eat all available memory. My comment below was just pointing out that your OS memory configuration looks perfectly normal. > > I understood that. I just wanted to pick the discussion up and see what can be done. > >> What I would do in your situation is to run the code that created this image directly under Windows, then save and try to start it up again. If you get an error when you run the code, you will be able to tell whether it's a memory problem or not. If the image saves and starts fine again on Windows, you can be pretty sure that it's not a VM problem but some platform dependency. And then you might try to move the image to a Mac to see if that teaches you anything. > > I tried the followings: > - Build the image on Windows 7, save, and open on Windows 7 ==> out of memory message > - Build the image on Windows 7, save, and open on OS X 10.6 ==> fine > - Build the image on OS X 10.6, save, and open on Windows 7 ==> out of memory message > > To me, it looks like a Windows-related problem. > > A problem is that I do not have a Windows machine on which I can test this too much, but when I tried, I could reproduce it repeatedly: > > 1. Take a Moose image: > http://ci.moosetechnology.org/job/moose-latest-dev/375/artifact/moose/*zip*/moose.zip > 2. Run in a Workspace (it takes a while): > 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] > This will basically create some 850000 objects and will get the image to some 400+MB > 3. Save and quit the image > 4. Reopen it > > Cheers, > Doru > > >> Cheers, >> - Andreas >> >>> Would it be possible to find some resolution on it? >>> >>> Cheers, >>> Doru >>> >>> >>> On 20 May 2011, at 17:08, Andreas Raab wrote: >>> >>>> On 5/20/2011 16:57, Igor Stasenko wrote: >>>>> Here an excert from crash.dmp: >>>>> >>>>> Memory Information (upon launch): >>>>> Physical Memory Size: 4194303 kbytes >>>>> Physical Memory Free: 2507424 kbytes >>>>> Page File Size: 4194303 kbytes >>>>> Page File Free: 4194303 kbytes >>>>> Virtual Memory Size: 2097024 kbytes >>>>> Virtual Memory Free: 2030280 kbytes >>>>> Memory Load: 59 percent >>>>> >>>>> this is strange. >>>>> Of course, with sizes like above , it cannot fit into memory. >>>> Nothing strange about it. Standard 4GB Ram configuration with about 2GB available. >>>> >>>> Cheers, >>>> - Andreas >>>> >>>> >>>>>> Hi, >>>>>> >>>>>> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >>>>>> >>>>>>> On Wed, 18 May 2011, Tudor Girba wrote: >>>>>>> >>>>>>> snip >>>>>>> >>>>>>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>>>>>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >>>>>> I tried with this VM, but it crashes with the attached error. >>>>>> >>>>>> In any case, if someone wants to reproduce it, here is an image to play with: >>>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>>> >>>>>> Cheers, >>>>>> Doru >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Levente >>>>>>> >>>>>>> snip >>>>>> -- >>>>>> www.tudorgirba.com >>>>>> >>>>>> "Being happy is a matter of choice." >>>>>> >>>>>> >>>>> >>> -- >>> www.tudorgirba.com >>> >>> "Every thing has its own flow." >>> >>> >>> >>> >>> > > -- > www.tudorgirba.com > > "If you interrupt the barber while he is cutting your hair, > you will end up with a messy haircut." > |
Re: [Pharo-project] out of memory - cog on windows
|
|
In reply to this post by Tudor Girba-2
Hi again, Just an update on this issue. It looks like it is an issue with the memory allocation in the Cog VM on Windows. Igor did a good job to fix the issue and now large images can be opened again. The fixes are in his branch. Igor, could you provide more technical details? I think it is an issue that should be taken seriously and integrated in the main branch. Cheers, Doru On 21 May 2011, at 19:13, Tudor Girba wrote: > Hi Andreas, > > On 20 May 2011, at 17:44, Andreas Raab wrote: > >> On 5/20/2011 17:19, Tudor Girba wrote: >>> I am not sure I understand where we are on this issue. I will say it again that I find it to be critical to be able to work with images that are not small :). >> >> We are nowhere on this issue. I should add that with OpenQwaq / Teleplace we've been running large images (300-500MB) all the time on Windows with no problems. > > I also thought that this cannot be that obvious of a problem, because otherwise people would have stumbled over it before. > >> And I am not at all convinced that this is even a VM issue; I find it more likely that there might be some platform dependency that causes some code to go wrong and actually eat all available memory. My comment below was just pointing out that your OS memory configuration looks perfectly normal. > > I understood that. I just wanted to pick the discussion up and see what can be done. > >> What I would do in your situation is to run the code that created this image directly under Windows, then save and try to start it up again. If you get an error when you run the code, you will be able to tell whether it's a memory problem or not. If the image saves and starts fine again on Windows, you can be pretty sure that it's not a VM problem but some platform dependency. And then you might try to move the image to a Mac to see if that teaches you anything. > > I tried the followings: > - Build the image on Windows 7, save, and open on Windows 7 ==> out of memory message > - Build the image on Windows 7, save, and open on OS X 10.6 ==> fine > - Build the image on OS X 10.6, save, and open on Windows 7 ==> out of memory message > > To me, it looks like a Windows-related problem. > > A problem is that I do not have a Windows machine on which I can test this too much, but when I tried, I could reproduce it repeatedly: > > 1. Take a Moose image: > http://ci.moosetechnology.org/job/moose-latest-dev/375/artifact/moose/*zip*/moose.zip > 2. Run in a Workspace (it takes a while): > 1 to: 3 do: [ :i | MooseScripts createModelForConfigurationOfMoose ] > This will basically create some 850000 objects and will get the image to some 400+MB > 3. Save and quit the image > 4. Reopen it > > Cheers, > Doru > > >> Cheers, >> - Andreas >> >>> Would it be possible to find some resolution on it? >>> >>> Cheers, >>> Doru >>> >>> >>> On 20 May 2011, at 17:08, Andreas Raab wrote: >>> >>>> On 5/20/2011 16:57, Igor Stasenko wrote: >>>>> Here an excert from crash.dmp: >>>>> >>>>> Memory Information (upon launch): >>>>> Physical Memory Size: 4194303 kbytes >>>>> Physical Memory Free: 2507424 kbytes >>>>> Page File Size: 4194303 kbytes >>>>> Page File Free: 4194303 kbytes >>>>> Virtual Memory Size: 2097024 kbytes >>>>> Virtual Memory Free: 2030280 kbytes >>>>> Memory Load: 59 percent >>>>> >>>>> this is strange. >>>>> Of course, with sizes like above , it cannot fit into memory. >>>> Nothing strange about it. Standard 4GB Ram configuration with about 2GB available. >>>> >>>> Cheers, >>>> - Andreas >>>> >>>> >>>>>> Hi, >>>>>> >>>>>> On 18 May 2011, at 18:07, Levente Uzonyi wrote: >>>>>> >>>>>>> On Wed, 18 May 2011, Tudor Girba wrote: >>>>>>> >>>>>>> snip >>>>>>> >>>>>>>> I tried with several versions of Cog including the latest ones (on both platforms) and I got the same result. I did not try with an interpreter VM because I only have Cog images around (based on Pharo 1.2) and they are not compatible. >>>>>>> You can use this interpreter vm to open cog images: http://leves.web.elte.hu/squeak/SqueakVM-Win32-4.4.9-2358-non-official-bin.zip >>>>>> I tried with this VM, but it crashes with the attached error. >>>>>> >>>>>> In any case, if someone wants to reproduce it, here is an image to play with: >>>>>> http://dl.dropbox.com/u/18323746/Tmp/moose-case-study-windows-problem.zip >>>>>> >>>>>> Cheers, >>>>>> Doru >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Levente >>>>>>> >>>>>>> snip >>>>>> -- >>>>>> www.tudorgirba.com >>>>>> >>>>>> "Being happy is a matter of choice." >>>>>> >>>>>> >>>>> >>> -- >>> www.tudorgirba.com >>> >>> "Every thing has its own flow." >>> >>> >>> >>> >>> > > -- > www.tudorgirba.com > > "If you interrupt the barber while he is cutting your hair, > you will end up with a messy haircut." > -- www.tudorgirba.com "Every thing should have the right to be different." |
Re: [Pharo-project] out of memory - cog on windows
|
|
On 27 June 2011 20:08, Tudor Girba <[hidden email]> wrote: > > Hi again, > > Just an update on this issue. > > It looks like it is an issue with the memory allocation in the Cog VM on Windows. Igor did a good job to fix the issue and now large images can be opened again. The fixes are in his branch. > Except that i didn't uploaded new version yet. Will do that tomorrow. > Igor, could you provide more technical details? > yes i added the issue to cog tracker: http://code.google.com/p/cog/issues/detail?id=46 > I think it is an issue that should be taken seriously and integrated in the main branch. > > Cheers, > Doru > > > On 21 May 2011, at 19:13, Tudor Girba wrote: > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] out of memory - cog on windows
|
|
Thanks, Igor. Doru On 27 Jun 2011, at 23:07, Igor Stasenko wrote: > > On 27 June 2011 20:08, Tudor Girba <[hidden email]> wrote: >> >> Hi again, >> >> Just an update on this issue. >> >> It looks like it is an issue with the memory allocation in the Cog VM on Windows. Igor did a good job to fix the issue and now large images can be opened again. The fixes are in his branch. >> > Except that i didn't uploaded new version yet. Will do that tomorrow. > >> Igor, could you provide more technical details? >> > > yes i added the issue to cog tracker: > > http://code.google.com/p/cog/issues/detail?id=46 > > >> I think it is an issue that should be taken seriously and integrated in the main branch. >> >> Cheers, >> Doru >> >> >> On 21 May 2011, at 19:13, Tudor Girba wrote: >> > > -- > Best regards, > Igor Stasenko AKA sig. -- www.tudorgirba.com "Every now and then stop and ask yourself if the war you're fighting is the right one." |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |