issue with swazoo in gemstone
Locked
issue with swazoo in gemstone
 
|
all of a sudden today two of our production machines and the dev
environment are not displaying seaside pages as they should has anyone else experienced any wierdness today or no why this is happening? |
Re: issue with swazoo in gemstone
|
|
Clayton Cottingham wrote:
> all of a sudden today two of our production machines and the dev > environment are not displaying seaside pages as they should > > > has anyone else experienced any wierdness today or no why this is happening? > Clayton, Have you recently updated GLASS (or any other part of the product)? There's always room for bugs:) Are there error messages in the any of the gem log files? Dale |
Re: issue with swazoo in gemstone
|
|
no updates, no bug fixes, no feature creep!
it seems to be something time related... at about 2:30 edt our one server went down and 8:15 bst our other server went down...thats only 45min apart given time diff and by 'went down' i mean that since that time all pages are rendered with no output aka 0 bytes our dev vm has the same issue as ive said before but... ive managed to turn back the clock on dev vm and it will render pages as normal if i set date to 08232010 thats all i got for now....... i am hoping to re -re produce this error and have some log info that will point to the issue On 10-08-31 3:24 PM, Dale Henrichs wrote: > Clayton Cottingham wrote: >> all of a sudden today two of our production machines and the dev >> environment are not displaying seaside pages as they should >> >> >> has anyone else experienced any wierdness today or no why this is >> happening? >> > > Clayton, > > Have you recently updated GLASS (or any other part of the product)? > There's always room for bugs:) > > Are there error messages in the any of the gem log files? > > Dale > |
Re: issue with swazoo in gemstone
|
|
In reply to this post by Dale Henrichs
OK i re produced this and include the errors a set my dev-vm back to aug 23 and was able to render and log into our admin page i then stopped the gemstone services and mved the logs out of the way , set the date to aug 30 11:45pm rebooted and tested ...started tailing all logs, the admin page still up.... at minutes before midnight... at 12:02 i checked the page was still up instincively i re check , yup its down now tailed logs below... now im not sure if im understanding this but it seems that on the last day of the month some sort of garbage collection maintenance tasks kick in ? ==> /opt/gemstone/log/seaside_1476admingcgem.log <== Starting doSweepWsUnion at 08/31/2010 12:04:26 AM EDT Starting values: WSU size=56 PD size=20248 MarkSweepArgsSType::(): buffer size= 320 pages GarRemoveNotDead: removing live from PD. live size: 56 Finished first GarRemoveNotDead at 08/31/2010 12:04:26 AM EDT. Removed 0 objects from possibleDead [Info]: WsUnion during first sweep size = 56 MarkSweepArgsSType::(): buffer size= 320 pages GarRemoveNotDead: removing live from PD. live size: 56 [Info]: Finished second GarRemoveNotDead at 08/31/2010 12:04:26 AM EDT. Removed 0 objs from possibleDead [Info]: SweepWsUnion ending possibleDead size = 20248 Finished sweeping possible dead at 08/31/2010 12:04:26 AM EDT. possible dead size=20248 notDead size=0 ==> /opt/gemstone/log/seaside_1466reclaimgcgem0.log <== 08/31/2010 12:04:27 AM EDT 1 reclaims 88 pagesProcessed 88 pagesReclaimed 8 allValidPages 2 singleObjPages 20196 processedObjs 3831 liveObjs 60 shadowObjs 16305 deadObjs 229.5 avgObjsPerPage ==> /opt/gemstone/log/seaside_1485symbolgem.log <== commit after 15 secs, newSyms 1 requests 1 sleeps 293 totalSpins 294 commit after 15 secs, newSyms 1 requests 1 sleeps 17 totalSpins 18 ==> /opt/gemstone/log/maintenance_gem.log <== 2010-08-30T20:51:18-07:00 Unregistering... Begin AutoTransaction: true Finish expiration loop: true ...Expired: 0 sessions. Before commit: true 2010-08-30T20:51:20-07:00 Starting markForCollect. start MFC mark/sweep phase: 08/30/2010 11:51:19 PM EDT WARNING, unable to resolve either garMarkSweepPageBufSize or mfcGcPageBufSize, using default 320 MarkSweepArgsSType::(): buffer size= 320 pages start MFC process dead phase: 08/31/2010 12:04:23 AM EDT MarkSweepArgsSType::(): buffer size= 320 pages GarRemoveNotDead: removing live from PD. live size: 1328 end MFC: 08/31/2010 12:04:24 AM EDT AbortingError 3020: Successful completion of markForCollection. <6824879> live objects found. <20248> possible dead objects, occupying approximately <1822320> bytes, may be reclaimed. 2010-08-30T21:04:26-07:00 ...finished markForCollect. 2010-08-30T21:05:26-07:00 Unregistering... Begin AutoTransaction: true Finish expiration loop: true ...Expired: 0 sessions. Before commit: true <<<SNIP>>> 2010-08-30T21:27:27-07:00 Unregistering... Begin AutoTransaction: true Finish expiration l On 10-08-31 3:24 PM, Dale Henrichs wrote: > Clayton Cottingham wrote: >> all of a sudden today two of our production machines and the dev >> environment are not displaying seaside pages as they should >> >> >> has anyone else experienced any wierdness today or no why this is >> happening? >> > > Clayton, > > Have you recently updated GLASS (or any other part of the product)? > There's always room for bugs:) > > Are there error messages in the any of the gem log files? > > Dale > |
Re: issue with swazoo in gemstone
|
|
Clayton,
I'd like to see the logs for the seaside vms ... in case there is an error message being dumped to the logs ... If there is no information in the log files, then you should try the experiment again: 1. Before midnight, after ensuring that vms are serving pages, sample the Smalltalk stack for each of the seaside vms using 'kill -USR1'. The smalltalk stack will be dumped to the seaside gem log. Sample the stack for each vm several times 2. Once the vms have stopped serving pages, use 'kill -USR1' to sample the smalltalk stack several times again for each of the vms ... Hopefully this will give you an idea of where to begin looking for the problem... Dale Clayton Cottingham wrote: > OK > i re produced this and include the errors > > a set my dev-vm back to aug 23 and was able to render and log into our > admin page > i then stopped the gemstone services and mved the logs out of the way , > set the date to aug 30 11:45pm > > rebooted and tested ...started tailing all logs, the admin page still > up.... at minutes before midnight... > > at 12:02 i checked the page was still up > instincively i re check , yup its down now > > tailed logs below... > > now im not sure if im understanding this but it seems that on the last > day of the month some sort of garbage collection maintenance tasks kick in ? > > > > > ==> /opt/gemstone/log/seaside_1476admingcgem.log <== > Starting doSweepWsUnion at 08/31/2010 12:04:26 AM EDT > Starting values: WSU size=56 PD size=20248 > MarkSweepArgsSType::(): buffer size= 320 pages > GarRemoveNotDead: removing live from PD. live size: 56 > Finished first GarRemoveNotDead at 08/31/2010 12:04:26 AM EDT. Removed > 0 objects from possibleDead > [Info]: WsUnion during first sweep size = 56 > MarkSweepArgsSType::(): buffer size= 320 pages > GarRemoveNotDead: removing live from PD. live size: 56 > [Info]: Finished second GarRemoveNotDead at 08/31/2010 12:04:26 AM EDT. > Removed 0 objs from possibleDead > [Info]: SweepWsUnion ending possibleDead size = 20248 > Finished sweeping possible dead at 08/31/2010 12:04:26 AM EDT. > possible dead size=20248 notDead size=0 > > ==> /opt/gemstone/log/seaside_1466reclaimgcgem0.log <== > 08/31/2010 12:04:27 AM EDT > 1 reclaims 88 pagesProcessed 88 pagesReclaimed 8 allValidPages 2 > singleObjPages > 20196 processedObjs 3831 liveObjs 60 shadowObjs 16305 deadObjs > 229.5 avgObjsPerPage > > > ==> /opt/gemstone/log/seaside_1485symbolgem.log <== > commit after 15 secs, newSyms 1 requests 1 sleeps 293 totalSpins 294 > commit after 15 secs, newSyms 1 requests 1 sleeps 17 totalSpins 18 > > ==> /opt/gemstone/log/maintenance_gem.log <== > 2010-08-30T20:51:18-07:00 > Unregistering... > Begin AutoTransaction: true > Finish expiration loop: true > ...Expired: 0 sessions. > Before commit: true > 2010-08-30T20:51:20-07:00 > Starting markForCollect. > start MFC mark/sweep phase: 08/30/2010 11:51:19 PM EDT > WARNING, unable to resolve either garMarkSweepPageBufSize or > mfcGcPageBufSize, using default 320 > MarkSweepArgsSType::(): buffer size= 320 pages > start MFC process dead phase: 08/31/2010 12:04:23 AM EDT > MarkSweepArgsSType::(): buffer size= 320 pages > GarRemoveNotDead: removing live from PD. live size: 1328 > end MFC: 08/31/2010 12:04:24 AM EDT > AbortingError 3020: Successful completion of markForCollection. > <6824879> live objects found. > <20248> possible dead objects, occupying approximately <1822320> > bytes, may be reclaimed. > 2010-08-30T21:04:26-07:00 > ...finished markForCollect. > 2010-08-30T21:05:26-07:00 > Unregistering... > Begin AutoTransaction: true > Finish expiration loop: true > ...Expired: 0 sessions. > Before commit: true > > <<<SNIP>>> > > 2010-08-30T21:27:27-07:00 > Unregistering... > Begin AutoTransaction: true > Finish expiration l > > > > > > On 10-08-31 3:24 PM, Dale Henrichs wrote: >> Clayton Cottingham wrote: >>> all of a sudden today two of our production machines and the dev >>> environment are not displaying seaside pages as they should >>> >>> >>> has anyone else experienced any wierdness today or no why this is >>> happening? >>> >> Clayton, >> >> Have you recently updated GLASS (or any other part of the product)? >> There's always room for bugs:) >> >> Are there error messages in the any of the gem log files? >> >> Dale >> |
«
Return to GLASS
|
1 view|%1 views
| Free forum by Nabble | Edit this page |