[Glass] Purge tranlogs?
Locked
[Glass] Purge tranlogs?
 
|
Hi,
Is that possible? They are taking some space in my laptop... I periodically make some backups from GemTools. Thanks, Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
Am 13.12.2013 um 21:53 schrieb Mariano Martinez Peck <[hidden email]>: You might find that script useful Norbert _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Purge tranlogs?
|
|
On Sat, Dec 14, 2013 at 11:49 AM, Norbert Hartl <[hidden email]> wrote:
Thanks Norbert, That was pretty easy to use and clear. I could release my last 15GB of my HDD ;)
I wonder if such a script could be included in /opt/gemstone/product/seaside/bin ? I wonder the side effects of removing the obsolete translogs. I guess not that much but I am not 100% clear of what that means exactly. Need to read more about it.
Thanks! Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
Mariano, As GLASS (soon to be GsFoundation) moves forward, I will be putting these kinds of scripts (or references to projects like Norberts) in a directory that is managed with git ... we don't release GemStone often enough to be able to manage the version of such scripts so a separate project is called for ... also different installations needs support for different things that depend upon volume and size of the system ... Dale From: "Mariano Martinez Peck" <[hidden email]> _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Purge tranlogs?
|
|
On Wed, Dec 18, 2013 at 5:14 PM, Dale K. Henrichs <[hidden email]> wrote:
Indeed, that makes total sense!!! So cool, I would appreciate this script as being part of such a git ;)
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
Am 19.12.2013 um 03:07 schrieb Mariano Martinez Peck <[hidden email]>: The script is in git… I would be happy if the whole (or most) of the stone-creator could be added to gemstone :) My old sysadmin sole does not want to start things in their installation directory. Stone-creator isolates a stone installation from the gemstone installation directory. This way you can have as many stones as you like to. I also have to mangle Dales scripts on any new release because I have start scripts that aren’t meant to run forever like Dales scripts. I’ll configure GC via cron where I think it is supposed to be. This is not blaming. The ability to install the package, start and go is great. But if you are heading for modularity there are some difficulties and some weakness like that netldi still gets its environment from the gemstone install dir and for gemtools you need to adjust those scripts if you need to change the temp object space settings and such. I’m happy to help (if I find the time nick nack) arranging a few things. Just some food for thought, Norbert
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
In reply to this post by Mariano Martinez Peck
Am 18.12.2013 um 19:28 schrieb Mariano Martinez Peck <[hidden email]>: I talked to James in Edinburgh about this and the conclusion is that there is a chance this goes wrong but the odds are very low. If you want to be save I think you need to do an explicit snapshot before invoking the script. James can tell it how it is exactly. Norbert _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
The basic idea is that to do a full restore to the current point you need all the tranlogs that go with whatever backup you have to start the restore. A typical model is to do daily backups and keep two backups and the tranlogs starting from the earlier backup. Beyond that I tend to keep a weekly backup for a month, monthly backups for a quarter, quarterly backups for a year, and then yearly backups. You can’t restore to the current point with these (since you wouldn’t have the tranlogs), but it is occasionally interesting to go back and see something. See the System Admin Guide (http://downloads.gemtalksystems.com/docs/GemStone64/3.1.x/GS64-SysAdminGuide-3.1.pdf) and http://www.youtube.com/watch?v=RTRQhUQqCcc. James _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
Am 20.12.2013 um 22:40 schrieb James Foster <[hidden email]>: Now I remember. That was the point making the script point to the backup instead of the extent. My script is #!/bin/sh for stone in $STONE_LIST; do date >> /opt/application/$stone/log/cleanup-tranlogs.log /opt/stone-creator/bin/delete-old-tranlogs.sh -g /opt/gemstone/product -d /opt/application/$stone/data -f /opt/application/$stone/backup/backup.dbf.gz -r >> /opt/application/$stone/log/cleanup-tranlogs.log 2>&1 done Norbert _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
In reply to this post by NorbertHartl
The issue here is what is your backup strategy ...the definition of an unneeded tranlog is defined by the last backup that you are keeping ... how many backups are your keeping and where are you keeping your backups ... it's a very good idea to do these things...for ss3 i copy tranlogs to a separate host every two hours and delete tranlogs that are 2 days old ... so I have some redundancy, but I am keeping my backups and tranlogs on a different machine .... there are relatively easy formulas, but there are a number of different solutions and there are tradeoffs as well ... My preference is for folks who are using GLASS to describe the different schemes that they are using and through the community develop the sets of scripts that work for different groups of communities ... I don't want to dictate how you set things up in production, but I am willing to dictate how you should set things to get started with GemStone:) Dale From: "Norbert Hartl" <[hidden email]> _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
May I assume you start a new tranlog every two hours before copying them to a new machine. I personally wouldn’t transfer a file that is still being written even if there is no random access regarding file position. Furthermore what probably isn’t clear by now. A fullbackup as the name says contains everything you need. Having the tranlogs just closes the gap between backup time and now, right? So I’m a little bit lazy. I have arranged schedules via cron. 30 minutes to midnight I start the cleanup handler for my business model. This removes old and obsolete stuff (producing dead objects). 5 minutes later I start an MFC and let that settle for 15 minutes because it is just marking and after that the gc thread kicks in removing the objects for real (removing dead objects). Then I start a full backup of the stone and compress it. A little while later the backup process comes visiting, collects the backups, encrypts and transfers them to a backup machine of the provider. That’s all. Imperfect but fine for me :) Norbert Am 20.12.2013 um 23:40 schrieb Dale K. Henrichs <[hidden email]>:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
From: "Norbert Hartl" <[hidden email]>No I don't start a new tranlog, so I'm willing to potentially lose some data at the cost of simplicity ... I use rsync to do the copy and it repairs the tranlogs at the next boundary ... once you make a full backup you only need to keep around the tranlogs that were "open at the time of the backup" plus all tranlogs created after the backup. By "open at the time of the backup", I am referring to the fact that to properly restore from a backup you may need tranlog records that predate the start of the backup and those records may multiple tranlog files earlier. There is a system call (that I don't recall right now) that will tell you the oldest tranlog needed for recovery ... all of the tranlogs previous to that one are not needed. If you want to be complete, then you are probably going to want to keep multiple backups and tranlogs going back in time ... and while we are on the subject it is worth validating that your freshly taken backup is not corrupted and that is done by using copydbf to copy the file ... copydbf will fail if the file has not been correctly written ... Other than differences in a couple of details, this is what I've done for ss3 ... Dale
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
Am 21.12.2013 um 02:36 schrieb Dale K. Henrichs <[hidden email]>:
I do SystemRepository startNewLog. SystemRepository fullBackupCompressedTo: '/opt/application/$1/backup/backup.dbf.gz' So the tranlog being open while backing up is the new one. So I should good to go only having the new tranlog?
Norbert
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
|
|
From: "Norbert Hartl" <[hidden email]>I'm pretty sure that recovering from tranlogs is pretty bullet proof ... our ACID guarantee is that what is written to disk is good. Commit is complete after tranlog record makes it to disk, so partial records should be tolerated if recovery needed ... copying every two hours means I'm willing to tolerate some lost records in the case where a machine and disks fry, so partial records are not critical (to me)... I guess this could be tested by taking copies of a tranlog that successfully restored from backup and copy the tranlog shaving off a byte or two at the end ... Potentially more than one log file could be needed. The expression `SystemRepository oldestLogFileIdForRecovery` gives you the correct answer.
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
«
Return to GLASS
|
1 view|%1 views
| Free forum by Nabble | Edit this page |