faster snapshot support in VM?
Locked
faster snapshot support in VM?
 
|
In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient: primFork "return true if we are in the forked image, false in the original" <primitive: XXX> self error: 'fork failed' snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot]. Apart being dependent on another feature of the OS, do you know any reason why this would not work? PS: sorry if double posting, I clicked on wrong vm-dev address first time |
Re: faster snapshot support in VM?
|
|
Nicolas Cellier wrote: > Apart being dependent on another feature of the OS, do you know any > reason why this would not work? Well, there is the small matter that fork() doesn't exist on Windows. And no equivalent functionality either. Cygwin might emulate this but I have severe doubts as to whether this would work reliably in non-trivial cases (i.e., threads using sockets etc) and I can almost guarantee that it's going to be very, very slow if it worked at all. Cheers, - Andreas |
Re: faster snapshot support in VM?
|
|
2009/7/17 Andreas Raab <[hidden email]>: > > Nicolas Cellier wrote: >> >> Apart being dependent on another feature of the OS, do you know any >> reason why this would not work? > > Well, there is the small matter that fork() doesn't exist on Windows. And no > equivalent functionality either. Cygwin might emulate this but I have severe > doubts as to whether this would work reliably in non-trivial cases (i.e., > threads using sockets etc) and I can almost guarantee that it's going to be > very, very slow if it worked at all. > Yeah, and instead of this, i'd rather look for a ways to optimize cold/warm startup code. Many classes ignoring this difference and doing all things from scratch. > Cheers, > - Andreas > -- Best regards, Igor Stasenko AKA sig. |
Re: faster snapshot support in VM?
|
|
In reply to this post by Nicolas Cellier
On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote: > > In recent pharo thread > [Pharo-project] Save and quit vs. Save - a clue to performance??? > it is said that saveAndQuit is amazingly fast, while save is damned slow... > ... because it goes through all image startup. > Cloning the vm/image could be more efficient: > > primFork > "return true if we are in the forked image, false in the original" > > <primitive: XXX> > self error: 'fork failed' > > snapshot > self primFork ifTrue: [ > self aboutToSnapshot. > self primSaveAndQuit. > self returnFromSnapshot]. > The primFork that you describe has been part of OSProcess for a long time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the unix flavor of OSProcessPlugin, so it's probably already in whatever unix VM you are running today. For what you describe, you would probably want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit: So you can just try it and see if it gives the performance you want. > Apart being dependent on another feature of the OS, do you know any > reason why this would not work? It definitely does work, but it is very OS dependent, and that's an extremely good reason for not doing it as a general solution. I don't understand why save should be "amazingly slow". What's going on in the image startup that would cause this be become an issue? It sounds to me like somebody is not checking the resuming flag in a #startup: method, hence running a bunch of initialization code that is not required after an image save. Dave |
Re: faster snapshot support in VM?
|
|
I'll wager a guess on the "amazingly slow" startup: Freetype. Cheers, - Andreas David T. Lewis wrote: > > On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote: >> >> In recent pharo thread >> [Pharo-project] Save and quit vs. Save - a clue to performance??? >> it is said that saveAndQuit is amazingly fast, while save is damned slow... >> ... because it goes through all image startup. >> Cloning the vm/image could be more efficient: >> >> primFork >> "return true if we are in the forked image, false in the original" >> >> <primitive: XXX> >> self error: 'fork failed' >> >> snapshot >> self primFork ifTrue: [ >> self aboutToSnapshot. >> self primSaveAndQuit. >> self returnFromSnapshot]. >> > > The primFork that you describe has been part of OSProcess for a long > time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the > unix flavor of OSProcessPlugin, so it's probably already in whatever > unix VM you are running today. For what you describe, you would probably > want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit: > > So you can just try it and see if it gives the performance you want. > >> Apart being dependent on another feature of the OS, do you know any >> reason why this would not work? > > It definitely does work, but it is very OS dependent, and that's an > extremely good reason for not doing it as a general solution. > > I don't understand why save should be "amazingly slow". What's going > on in the image startup that would cause this be become an issue? It > sounds to me like somebody is not checking the resuming flag in a > #startup: method, hence running a bunch of initialization code that > is not required after an image save. > > Dave > |
Re: faster snapshot support in VM?
|
|
In reply to this post by David T. Lewis
OK, thanks to all, OS dependency is a good reason, we definitely prefer the more portable Smalltalk way :) Still, cloning an image with a HydraVM would be a portable option :) (Efficient if pages can be optimized for copy-on-write which is again an OS-dependent feature I presume...). Maybe I'll give a try in Unix just for fun, or I'll have a deeper look on what's going on exactly. Cheers Nicolas 2009/7/17 David T. Lewis <[hidden email]>: > > On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote: >> >> In recent pharo thread >> [Pharo-project] Save and quit vs. Save - a clue to performance??? >> it is said that saveAndQuit is amazingly fast, while save is damned slow... >> ... because it goes through all image startup. >> Cloning the vm/image could be more efficient: >> >> primFork >> "return true if we are in the forked image, false in the original" >> >> <primitive: XXX> >> self error: 'fork failed' >> >> snapshot >> self primFork ifTrue: [ >> self aboutToSnapshot. >> self primSaveAndQuit. >> self returnFromSnapshot]. >> > > The primFork that you describe has been part of OSProcess for a long > time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the > unix flavor of OSProcessPlugin, so it's probably already in whatever > unix VM you are running today. For what you describe, you would probably > want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit: > > So you can just try it and see if it gives the performance you want. > >> Apart being dependent on another feature of the OS, do you know any >> reason why this would not work? > > It definitely does work, but it is very OS dependent, and that's an > extremely good reason for not doing it as a general solution. > > I don't understand why save should be "amazingly slow". What's going > on in the image startup that would cause this be become an issue? It > sounds to me like somebody is not checking the resuming flag in a > #startup: method, hence running a bunch of initialization code that > is not required after an image save. > > Dave > > |
Re: faster snapshot support in VM?
|
|
2009/7/17 Nicolas Cellier <[hidden email]>: > > OK, thanks to all, > OS dependency is a good reason, we definitely prefer the more portable > Smalltalk way :) > Still, cloning an image with a HydraVM would be a portable option :) > (Efficient if pages can be optimized for copy-on-write which is again > an OS-dependent feature I presume...). > > Maybe I'll give a try in Unix just for fun, or I'll have a deeper look > on what's going on exactly. > Nicolas, if your image is a complete sandbox, which using no any external resources , then a proposed approach makes sense. But in most cases it is not, and i don't see how external resources, allocated by different primitives/plugins can be safely 'cloned'/forked. So, i think, we should focus on improving snapshot/startup code on image side, not in VM. > Cheers > > Nicolas > > 2009/7/17 David T. Lewis <[hidden email]>: >> >> On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote: >>> >>> In recent pharo thread >>> [Pharo-project] Save and quit vs. Save - a clue to performance??? >>> it is said that saveAndQuit is amazingly fast, while save is damned slow... >>> ... because it goes through all image startup. >>> Cloning the vm/image could be more efficient: >>> >>> primFork >>> "return true if we are in the forked image, false in the original" >>> >>> <primitive: XXX> >>> self error: 'fork failed' >>> >>> snapshot >>> self primFork ifTrue: [ >>> self aboutToSnapshot. >>> self primSaveAndQuit. >>> self returnFromSnapshot]. >>> >> >> The primFork that you describe has been part of OSProcess for a long >> time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the >> unix flavor of OSProcessPlugin, so it's probably already in whatever >> unix VM you are running today. For what you describe, you would probably >> want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit: >> >> So you can just try it and see if it gives the performance you want. >> >>> Apart being dependent on another feature of the OS, do you know any >>> reason why this would not work? >> >> It definitely does work, but it is very OS dependent, and that's an >> extremely good reason for not doing it as a general solution. >> >> I don't understand why save should be "amazingly slow". What's going >> on in the image startup that would cause this be become an issue? It >> sounds to me like somebody is not checking the resuming flag in a >> #startup: method, hence running a bunch of initialization code that >> is not required after an image save. >> >> Dave >> >> > -- Best regards, Igor Stasenko AKA sig. |
Re: faster snapshot support in VM?
|
|
Igor, agree, cloning an image is functionnally the same as returning from snapshot... The difference here is that we haven't performed the cleanup before snapshot :( So my scheme was more or less to release all external pointers in the clone before snapshoting, without performing any OS action (like flushing stream buffers etc...). But I should check what fork() really do exactly about allocated ressources, because it's a long time since I last use this kind of magic... The only interest of this approach was to lazily rely on already optimized OS features, but you already convinced me this was not portable, so beat it :) cheers Nicolas 2009/7/17 Igor Stasenko <[hidden email]>: > > 2009/7/17 Nicolas Cellier <[hidden email]>: >> >> OK, thanks to all, >> OS dependency is a good reason, we definitely prefer the more portable >> Smalltalk way :) >> Still, cloning an image with a HydraVM would be a portable option :) >> (Efficient if pages can be optimized for copy-on-write which is again >> an OS-dependent feature I presume...). >> >> Maybe I'll give a try in Unix just for fun, or I'll have a deeper look >> on what's going on exactly. >> > > Nicolas, if your image is a complete sandbox, which using no any > external resources , then a proposed > approach makes sense. But in most cases it is not, and i don't see how > external resources, allocated by different primitives/plugins can be > safely 'cloned'/forked. > So, i think, we should focus on improving snapshot/startup code on > image side, not in VM. > >> Cheers >> >> Nicolas >> >> 2009/7/17 David T. Lewis <[hidden email]>: >>> >>> On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote: >>>> >>>> In recent pharo thread >>>> [Pharo-project] Save and quit vs. Save - a clue to performance??? >>>> it is said that saveAndQuit is amazingly fast, while save is damned slow... >>>> ... because it goes through all image startup. >>>> Cloning the vm/image could be more efficient: >>>> >>>> primFork >>>> "return true if we are in the forked image, false in the original" >>>> >>>> <primitive: XXX> >>>> self error: 'fork failed' >>>> >>>> snapshot >>>> self primFork ifTrue: [ >>>> self aboutToSnapshot. >>>> self primSaveAndQuit. >>>> self returnFromSnapshot]. >>>> >>> >>> The primFork that you describe has been part of OSProcess for a long >>> time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the >>> unix flavor of OSProcessPlugin, so it's probably already in whatever >>> unix VM you are running today. For what you describe, you would probably >>> want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit: >>> >>> So you can just try it and see if it gives the performance you want. >>> >>>> Apart being dependent on another feature of the OS, do you know any >>>> reason why this would not work? >>> >>> It definitely does work, but it is very OS dependent, and that's an >>> extremely good reason for not doing it as a general solution. >>> >>> I don't understand why save should be "amazingly slow". What's going >>> on in the image startup that would cause this be become an issue? It >>> sounds to me like somebody is not checking the resuming flag in a >>> #startup: method, hence running a bunch of initialization code that >>> is not required after an image save. >>> >>> Dave >>> >>> >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |