Snapshot to memory then to file
Locked
 
|
Dear all,
As a big fan of image based persistency thanks to very reliable VisualWorks I need bigger and bigger images, with a negative consequence that snapshot takes more and more time. To speed it up, a two-phase snapshot, to memory first then to a file can be a solution. As Squeak already has. Is there any plan to provide such snapshot in VW? Would it be difficult to achieve? Best regards Janko -- Janko Mivšek Aida/Web Smalltalk Web Application Server http://www.aidaweb.si _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
Save to a RAM disk then copy to hard disk? E.g. Dataram RAMDisk
http://memory.dataram.com/products-and-services/software/ramdisk is free up to 4GB and was the fastest free tool in a survey (IIRC): http://www.raymond.cc/blog/archives/2009/12/08/12-ram-disk-software-benchmarked-for-fastest-read-and-write-speed/ Steve > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Janko Mivšek > Sent: 1. joulukuuta 2011 18:40 > To: 'VWNC' > Subject: [vwnc] Snapshot to memory then to file > > Dear all, > > As a big fan of image based persistency thanks to very reliable > VisualWorks I need bigger and bigger images, with a negative > consequence > that snapshot takes more and more time. > > To speed it up, a two-phase snapshot, to memory first then to a file > can > be a solution. As Squeak already has. > > Is there any plan to provide such snapshot in VW? Would it be > difficult > to achieve? > > Best regards > Janko > > -- > Janko Mivšek > Aida/Web > Smalltalk Web Application Server > http://www.aidaweb.si > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
How big are your images? and how long do your image saves take?
But more importantly -- how long is "too long" when it comes to saving? Saving a snapshot of a semi large image (150mb) is just about instant for me when running on modern hardware and using the latest version of VW. On Thu, Dec 1, 2011 at 9:21 AM, Steven Kelly <[hidden email]> wrote: Save to a RAM disk then copy to hard disk? E.g. Dataram RAMDisk _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
In reply to this post by Janko Mivšek
On a 32 bit system, how does Squeak snapshot an image > 2 gb to memory
and then to disk? On 12/1/2011 8:39 AM, Janko Mivšek wrote: > Dear all, > > As a big fan of image based persistency thanks to very reliable > VisualWorks I need bigger and bigger images, with a negative consequence > that snapshot takes more and more time. > > To speed it up, a two-phase snapshot, to memory first then to a file can > be a solution. As Squeak already has. > > Is there any plan to provide such snapshot in VW? Would it be difficult > to achieve? > > Best regards > Janko > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
In reply to this post by Janko Mivšek
We need some sort of ASL for the environment here...
Which OS? Which VM version? Image size? Where is it being saved, local disk, network?... If Windows and saving to local disk, is the disk reasonably defragmented? After you save the file, how many file fragments does the image have? 1? 10? 10000? If not the latest version of the VM, did you try the latest 7.9 VM to see if it makes any difference? How many fixed objects in the image? On 12/1/2011 8:39 AM, Janko Mivšek wrote: > Dear all, > > As a big fan of image based persistency thanks to very reliable > VisualWorks I need bigger and bigger images, with a negative consequence > that snapshot takes more and more time. > > To speed it up, a two-phase snapshot, to memory first then to a file can > be a solution. As Squeak already has. > > Is there any plan to provide such snapshot in VW? Would it be difficult > to achieve? > > Best regards > Janko > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
In reply to this post by Andres Valloud-6
The most obvious answer would be to see if available RAM is sufficient compared to the size of the running VW system before attempting to save to RAM. Dave Stevenson[hidden email] From: Andres Valloud <[hidden email]> To: "[hidden email]" <[hidden email]> Sent: Thu, December 1, 2011 12:59:08 PM Subject: Re: [vwnc] Snapshot to memory then to file On a 32 bit system, how does Squeak snapshot an image > 2 gb to memory and then to disk? On 12/1/2011 8:39 AM, Janko Mivšek wrote: > Dear all, > > As a big fan of image based persistency thanks to very reliable > VisualWorks I need bigger and bigger images, with a negative consequence > that snapshot takes more and more time. > > To speed it up, a two-phase snapshot, to memory first then to a file can > be a solution. As Squeak already has. > > Is there any plan to provide such snapshot in VW? Would it be difficult > to achieve? > > Best regards > Janko > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
You can’t trust the snapshot until it’s been written out to disk from memory, so what’s the benefit here? -Boris From: [hidden email] [mailto:[hidden email]] On Behalf Of Dave Stevenson The most obvious answer would be to see if available RAM is sufficient compared to the size of the running VW system before attempting to save to RAM. Dave Stevenson From: Andres Valloud <[hidden email]> _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
Oh yeah... "the VM crashed right after the snapshot primitive spawned a
thread to write the memory snapshot... your image file is now 10240 bytes, sorry about that!" :)... On 12/1/2011 11:05 AM, Boris Popov, DeepCove Labs wrote: > You can’t trust the snapshot until it’s been written out to disk from > memory, so what’s the benefit here? > > -Boris > > *From:*[hidden email] [mailto:[hidden email]] *On > Behalf Of *Dave Stevenson > *Sent:* Thursday, December 01, 2011 2:02 PM > *To:* Andres Valloud; [hidden email] > *Subject:* Re: [vwnc] Snapshot to memory then to file > > The most obvious answer would be to see if available RAM is sufficient > compared to the size of the running VW system before attempting to save > to RAM. > > Dave Stevenson > [hidden email] <mailto:[hidden email]> > > ------------------------------------------------------------------------ > > *From:*Andres Valloud <[hidden email] <mailto:[hidden email]>> > *To:* "[hidden email] <mailto:[hidden email]>" <[hidden email] > <mailto:[hidden email]>> > *Sent:* Thu, December 1, 2011 12:59:08 PM > *Subject:* Re: [vwnc] Snapshot to memory then to file > > On a 32 bit system, how does Squeak snapshot an image > 2 gb to memory > and then to disk? > > On 12/1/2011 8:39 AM, Janko Mivšek wrote: >> Dear all, >> >> As a big fan of image based persistency thanks to very reliable >> VisualWorks I need bigger and bigger images, with a negative consequence >> that snapshot takes more and more time. >> >> To speed it up, a two-phase snapshot, to memory first then to a file can >> be a solution. As Squeak already has. >> >> Is there any plan to provide such snapshot in VW? Would it be difficult >> to achieve? >> >> Best regards >> Janko >> > _______________________________________________ > vwnc mailing list > [hidden email] <mailto:[hidden email]> > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
In reply to this post by Jon Paynter-2
2011/12/2 Janko Mivšek <[hidden email]>
S, Jon Paynter piše: Based on what you posted, its definitely time for upgrades. Id go with a Disk upgrade first. With the prices of SSDs falling fast, you can get a good one that will go a long ways to solve the problem. Also for the time difference from VW7.2 to VW.78, I would recommend upgrading production to 7.8. Let us know what happens when your done. Jon. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Snapshot to memory then to file
|
|
2011/12/2 Janko Mivšek <[hidden email]> S, Jon Paynter piše: You can warm up the image file by reading it, but I don't think you want to. I don't know how Linux behaves, but if you care about your image as a backup you don't want to write the new file onto the same sectors as the old file: Any problem halfway through and you end up with neither old nor new.
Your image can't answer requests while it's saving, so you want to minimize that time. The easiest way is to save to flash/SSD, or RAM disk if you have the memory. If you're saving to a RAM disk, you then spawn a background process to copy the image to the hard disk, compressing it if you like. In any case save the new image to a different name from the old one (e.g. by renaming the old file first), and only delete the old file after the new one has been fully written.
Steve _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
«
Return to VisualWorks
|
1 view|%1 views
| Free forum by Nabble | Edit this page |