Saving the image in the background
Locked
Saving the image in the background
 
|
Hi,
I would like to know if the following code is safe: [SmalltalkImage current snapshot: true andQuit: false] forkAt: Processor userBackgroundPriority Is there any problem if the main process modifies something in the same time ? Thank you |
RE: Saving the image in the background
|
|
> From: Damien Cassou
> I would like to know if the following code is safe: > > [SmalltalkImage current snapshot: true andQuit: false] > forkAt: Processor userBackgroundPriority > > Is there any problem if the main process modifies something > in the same time ? In general it is not safe. Code before the snapshot puts a number of objects into a state suitable for recovery after the image restarts, and code after the snapshot restores them. You also are forking at userBackgroundPriority, which means that the system's tidy-up before a snapshot and recovery after a snapshot could be pre-empted by your own processes, so you might get errors in the main process from a freshly-started image that has only partly recovered when restarted. Your specific case may not trigger any of the problems - follow the code through the various hooks to see what is affected. - Peter |
Re: Saving the image in the background
|
|
Peter Crowther wrote:
>> From: Damien Cassou >> I would like to know if the following code is safe: >> >> [SmalltalkImage current snapshot: true andQuit: false] >> forkAt: Processor userBackgroundPriority >> >> Is there any problem if the main process modifies something >> in the same time ? > > In general it is not safe. Code before the snapshot puts a number of > objects into a state suitable for recovery after the image restarts, and > code after the snapshot restores them. You also are forking at > userBackgroundPriority, which means that the system's tidy-up before a > snapshot and recovery after a snapshot could be pre-empted by your own > processes, so you might get errors in the main process from a > freshly-started image that has only partly recovered when restarted. > > Your specific case may not trigger any of the problems - follow the code > through the various hooks to see what is affected. Is it better if I do not fork at all ? |
Re: Saving the image in the background
|
|
In reply to this post by Damien Cassou-3
2006/7/30, Damien Cassou <[hidden email]>:
> Hi, > > I would like to know if the following code is safe: > > [SmalltalkImage current snapshot: true andQuit: false] > forkAt: Processor userBackgroundPriority > > Is there any problem if the main process modifies something in the same > time ? Doesn't thay stall the image anyway? IIRC it makes a full GC with blocks. If you have OSProcess you can fork the image (search the mailinglist, Avi once asked about this). I don't know if this now works on Mac OS. Also pay attention that if you have two forked images at the same time, that they don't write into the same image file (if you want to use it for Pier). Otherwise you might end up with a corrupted image file. Philippe |
Re: Saving the image in the background
|
|
On Sun, Jul 30, 2006 at 03:04:58PM +0200, Philippe Marschall wrote:
> > If you have OSProcess you can fork the image (search the mailinglist, > Avi once asked about this). I don't know if this now works on Mac OS. > Also pay attention that if you have two forked images at the same > time, that they don't write into the same image file (if you want to > use it for Pier). Otherwise you might end up with a corrupted image > file. This is done by evaluating "UnixProcess saveImageInBackgroundNicely". The image will be saved by a copy of your Squeak image running in a separate OS process at low priority, with little or no impact on your main Squeak process. It should also work on OS X if you are using the unix VM for OS X that supplies the OSPP plugin (I would appreciate if someone can confirm this). It might also work on the regular Mac VM if you compile your own OSPP plugin, but I'm not sure of this. Dave |
Re: Saving the image in the background
|
|
In reply to this post by Damien Cassou-3
On 30-Jul-06, at 6:02 AM, Damien Cassou wrote: > Peter Crowther wrote: >>> From: Damien Cassou >>> I would like to know if the following code is safe: >>> >>> [SmalltalkImage current snapshot: true andQuit: false] >>> forkAt: Processor userBackgroundPriority >>> >>> Is there any problem if the main process modifies something in >>> the same time ? >> In general it is not safe. Code before the snapshot puts a number of >> objects into a state suitable for recovery after the image >> restarts, and >> code after the snapshot restores them. You also are forking at >> userBackgroundPriority, which means that the system's tidy-up >> before a >> snapshot and recovery after a snapshot could be pre-empted by your >> own >> processes, so you might get errors in the main process from a >> freshly-started image that has only partly recovered when restarted. >> Your specific case may not trigger any of the problems - follow >> the code >> through the various hooks to see what is affected. > > Is it better if I do not fork at all ? I strongly urge you to not try saving an image in a background process, except just possibly in a very high priority one. As Peter mentions any other priority can be pre-empted and so all that careful go-to-sleep code (take a look at the fairly scary list of jobs to be done before the actual snapshot) could be completely messed up. That could lead to both the active image and the saved snapshot being unusable, which is unlikely to please you! Many years ago somebody at a large US automotive manufacturer put a snapshot primitive call into a background process. The resulting debugging work at PPS cost my team 5-6 man-months of effort and a lot of aggravation. Bad idea. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim "Oh bother" said Pooh, as he reached for the reset button |
Re: Saving the image in the background
|
|
Tim,
let me understand the difference between the "regular" process priority when a save/+-/quit is done from a menu and when it is done at userBackgroundPriority. In both cases the shutDown list is executed and sensors and processes (and other facilities) are made aware that they must become quiet now because it must be possible for them to come to live on another platform. If there where any problem(s) at userBackgroundPriority, wouldn't they be the same at the UIProcess's userSchedulingPriority? Thank you in advance. /Klaus P.S. I also cannot understand the original issue, the snapshot primitive does not run faster or slower in background: garbage collection (part of snapshot) and snapshot itself are atomic. On Sun, 30 Jul 2006 18:09:15 +0200, tim Rowledge <[hidden email]> wrote: > On 30-Jul-06, at 6:02 AM, Damien Cassou wrote: >> Peter Crowther wrote: >>>> From: Damien Cassou >>>> I would like to know if the following code is safe: >>>> >>>> [SmalltalkImage current snapshot: true andQuit: false] >>>> forkAt: Processor userBackgroundPriority >>>> >>>> Is there any problem if the main process modifies something in the >>>> same time ? >>> In general it is not safe. Code before the snapshot puts a number of >>> objects into a state suitable for recovery after the image restarts, >>> and >>> code after the snapshot restores them. You also are forking at >>> userBackgroundPriority, which means that the system's tidy-up before a >>> snapshot and recovery after a snapshot could be pre-empted by your own >>> processes, so you might get errors in the main process from a >>> freshly-started image that has only partly recovered when restarted. >>> Your specific case may not trigger any of the problems - follow the >>> code >>> through the various hooks to see what is affected. >> >> Is it better if I do not fork at all ? > > I strongly urge you to not try saving an image in a background process, > except just possibly in a very high priority one. As Peter mentions any > other priority can be pre-empted and so all that careful go-to-sleep > code (take a look at the fairly scary list of jobs to be done before the > actual snapshot) could be completely messed up. That could lead to both > the active image and the saved snapshot being unusable, which is > unlikely to please you! > > Many years ago somebody at a large US automotive manufacturer put a > snapshot primitive call into a background process. The resulting > debugging work at PPS cost my team 5-6 man-months of effort and a lot of > aggravation. Bad idea. > > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > "Oh bother" said Pooh, as he reached for the reset button > > > > |
|
|
In reply to this post by David T. Lewis
With the mac carbon 3 .8.12 series of VMs you can call any unix DLL
you have handy. Thus if you have a Unix OSX version of the OSPP you can just use that, no need to compile a *.bundle. On 30-Jul-06, at 8:25 AM, David T. Lewis wrote: > On Sun, Jul 30, 2006 at 03:04:58PM +0200, Philippe Marschall wrote: >> >> If you have OSProcess you can fork the image (search the mailinglist, >> Avi once asked about this). I don't know if this now works on Mac OS. >> Also pay attention that if you have two forked images at the same >> time, that they don't write into the same image file (if you want to >> use it for Pier). Otherwise you might end up with a corrupted image >> file. > > This is done by evaluating "UnixProcess saveImageInBackgroundNicely". > The image will be saved by a copy of your Squeak image running in a > separate OS process at low priority, with little or no impact on your > main Squeak process. > > It should also work on OS X if you are using the unix VM for OS X that > supplies the OSPP plugin (I would appreciate if someone can confirm > this). It might also work on the regular Mac VM if you compile your > own OSPP plugin, but I'm not sure of this. > > Dave > > -- ======================================================================== === John M. McIntosh <[hidden email]> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== === |
|
|
In reply to this post by timrowledge
On 30-Jul-06, at 9:09 AM, tim Rowledge wrote: > I strongly urge you to not try saving an image in a background > process, except just possibly in a very high priority one. As Peter > mentions any other priority can be pre-empted and so all that > careful go-to-sleep code (take a look at the fairly scary list of > jobs to be done before the actual snapshot) could be completely > messed up. That could lead to both the active image and the saved > snapshot being unusable, which is unlikely to please you! > > Many years ago somebody at a large US automotive manufacturer put a > snapshot primitive call into a background process. The resulting > debugging work at PPS cost my team 5-6 man-months of effort and a > lot of aggravation. Bad idea. Tim, I think the code suggestion from many months ago forks a background unix process that does the snapshot. It of course is an identical copy of the parent at the time of the fork maintained by the magic of virtual memory. The original parent process just keeps running and memory updates don't alter what the child process has. On a mulitple CPU that tosses the repsonsiblity of saving to another process at the cost of memory. -- ======================================================================== === John M. McIntosh <[hidden email]> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== === |
Re: Saving the image in the background
|
|
In reply to this post by Damien Cassou-3
Damien dijo:
> I would like to know if the following code is safe: > > [SmalltalkImage current snapshot: true andQuit: false] > forkAt: Processor userBackgroundPriority > > Is there any problem if the main process modifies something in the same > time ? > > The problems have already explained, and I've been following this thread with great interest, because a "similar" problem we have is to find a way to save the image doing everything from Squeak (suppose you want, for example, to "save" the image over the net, using FTP, implementing the FTP protocol in Squeak: the image will be changing while you do the networking stuff) The solution we came up to, and until somebody finds a better way (please do!), is to make a copy of the image in a String or ByteArray (from a primitive). This will effectively duplicate the size of the image, but so far, is the only solution we found. This trick may apply to your case, depending on what you really want to put in background, but as Klaus said, if you want to put the garbage collection in background, you cannot do it by forking a Squeak process, as this is a natively atomic operation, and the same goes for storing the bits in a file. This two can only be forked natively. Any better ideas? please?! :-) gera |
Re: Saving the image in the background
|
|
In reply to this post by Klaus D. Witzel
On 30-Jul-06, at 10:28 AM, Klaus D. Witzel wrote: > Tim, > > let me understand the difference between the "regular" process > priority when a save/+-/quit is done from a menu and when it is > done at userBackgroundPriority. In both cases the shutDown list is > executed and sensors and processes (and other facilities) are made > aware that they must become quiet now because it must be possible > for them to come to live on another platform. Correct. > > If there where any problem(s) at userBackgroundPriority, wouldn't > they be the same at the UIProcess's userSchedulingPriority? Same problem but higher likelihood of them happening, I think. You might consider some critical section protection to avoid interruptions. The problem at PPS I mentioned happened because the customer had a background process that ran the snapshot primitive rather than the entire snapshot sequence. Thus the snapshot file on disc had a lot of objects in indeterminate state. Not a happy thing for a payroll processing program. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: CAO: Compare Apples to Oranges |
Re: Saving the image in the background
|
|
In reply to this post by Damien Cassou-3
Damien Cassou wrote:
> I would like to know if the following code is safe: > > [SmalltalkImage current snapshot: true andQuit: false] > forkAt: Processor userBackgroundPriority > > Is there any problem if the main process modifies something in the same > time ? I'm sorry not having been clear enough. I just wanted to save the image when a user modify the model. To avoid a long delay for him, I wanted to use a fork. But it seems this was not a good idea. My solution is to remove the fork and let the user wait some seconds :-) Thank you for all your explanations |
Re: Saving the image in the background
|
|
On Jul 31, 2006, at 11:01 AM, Damien Cassou wrote: > Damien Cassou wrote: >> I would like to know if the following code is safe: >> [SmalltalkImage current snapshot: true andQuit: false] >> forkAt: Processor userBackgroundPriority >> Is there any problem if the main process modifies something in the >> same time ? > > I'm sorry not having been clear enough. I just wanted to save the > image when a user modify the model. To avoid a long delay for him, > I wanted to use a fork. > > But it seems this was not a good idea. > > My solution is to remove the fork and let the user wait some > seconds :-) Forking a separate Smalltalk process is a bad idea, but forking the whole unix process works just fine. You just need to modify the snapshot prim to do this. There was a long thread about this earlier, here's a post with some code: http://lists.squeakfoundation.org/pipermail/squeak-dev/2005-October/ 095896.html I used a VM with that prim for months without any ill effects. Avi |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |