Nightly Cleanup Recipe (Was Proper object removal)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Nightly Cleanup Recipe (Was Proper object removal)

Rob Rothwell
On Thu, Jun 5, 2008 at 9:44 AM, Janko Mivšek <[hidden email]> wrote:
Yes, WebGrid (kind of smart web table) caches presented objects in session presentation state (instances of your Apps, that is subclasses of WebApplication) until you clean up that state. Because Aida web apps don't have a session timeout by default, this cleanup is done explicitly, usually every night.

I think I really need some advice on a "tried and true" recipe for getting the state cleaned up.  No matter what I do, the garbageCollect does not seem to work (instances of my objects still remain) UNTIL I load a page referencing the objects again after having first doing some not-quite-reproducible combination of:

SwazooAida stop.
(AIDASite named: 'aidademo') urlResolver removeObject: m.
SwazooServer singleton removeSite: (AIDASite named: 'aidademo').
WebSessionManager allInstancesDo: [:each | each removeAllSessions].
WebSession allInstances do: [:each | each initAppsForObjects].
Undeclared removeUnreferencedKeys.
Symbol compactSymbolTable.
Smalltalk garbageCollect.
SwazooAida demoStart.
(AIDASite named: 'aidademo') urlResolver defaultURL: '/listmanager.html' forObject: m .
Smalltalk garbageCollect.

But let we move this discussion back to Aida mailing list. Others can follow or join it here: http://www.aidaweb.si/community.html

Sorry...I wasn't trying to take the conversation elsewhere...I just thought I was doing something "generally" wrong...!
 

Rob
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Janko Mivšek


Rob Rothwell wrote:

> On Thu, Jun 5, 2008 at 9:44 AM, Janko Mivšek <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Yes, WebGrid (kind of smart web table) caches presented objects in
>     session presentation state (instances of your Apps, that is
>     subclasses of WebApplication) until you clean up that state. Because
>     Aida web apps don't have a session timeout by default, this cleanup
>     is done explicitly, usually every night.
>
>
> I think I really need some advice on a "tried and true" recipe for
> getting the state cleaned up.  

Below cleanup is quite drastic I should say :) I think enough will be
just to:

1. remove object from urlResolver immediatelly after not needed anymore:

   (AIDASite named: 'aidademo') urlResolver removeObject: m

2. Overnight do :

   WebSessionManager allInstancesDo: [:each | each removeGuestSessions]
   WebSession allInstances do: [:each | each initAppsForObjects]
   Smalltalk garbageCollect

That way I'm sure you will keep object under control, at least from Aida
standpoint. You can put overnight cleanup routine into override of:

SwazooServer>>watchdogOther

   self isNightlyBackupTime ifTrue:
     [Transcript cr; show: 'closing all HTTP connections...'.
      HTTPConnection allInstances do: [:each | each close].
        "to GC all streams and buffers"
     Transcript cr; show: 'removing guest sessions...'.
     WebSessionManager allInstances do: [:each |
       each removeGuestSessions].
     Transcript cr; show: 'removing apps for objects in sessions...'.
     WebSession allInstances do: [:each | each initAppsForObjects].
     Transcript cr; show: 'backup...'.
     "put your backup call here"
     (Delay forSeconds: self watchdogPeriod) wait.
         "otherwise isNightlyBackupTime'll be again true!

SwazooServer>>isNightlyBackupTime
   "backup should be done at 4:30:00"
   "SwazooServer singleton isNightlyBackupTime"
   | backupTime |
   backupTime := SpTimestamp
      fromDate: Date today
      andTime: (Time readFrom: '4:30:00' readStream).
   ^(SpTimestamp now asSeconds > backupTime asSeconds) &
       (SpTimestamp now asSeconds < (backupTime asSeconds +
           self watchdogPeriod))


This is what I'm using in currently for cleaning my Squeak image.

Janko

> No matter what I do, the garbageCollect
> does not seem to work (instances of my objects still remain) UNTIL I
> load a page referencing the objects again after having first doing some
> not-quite-reproducible combination of:
>
> SwazooAida stop.
> (AIDASite named: 'aidademo') urlResolver removeObject: m.
> SwazooServer singleton removeSite: (AIDASite named: 'aidademo').
> WebSessionManager allInstancesDo: [:each | each removeAllSessions].
> WebSession allInstances do: [:each | each initAppsForObjects].
> Undeclared removeUnreferencedKeys.
> Symbol compactSymbolTable.
> Smalltalk garbageCollect.
> SwazooAida demoStart.
> (AIDASite named: 'aidademo') urlResolver defaultURL: '/listmanager.html'
> forObject: m .
> Smalltalk garbageCollect.
>
>     But let we move this discussion back to Aida mailing list. Others
>     can follow or join it here: http://www.aidaweb.si/community.html
>
>
> Sorry...I wasn't trying to take the conversation elsewhere...I just
> thought I was doing something "generally" wrong...!
>  
>
> Rob
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Aida mailing list
> [hidden email]
> http://lists.aidaweb.si/mailman/listinfo/aida

--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Rob Rothwell
On Thu, Jun 5, 2008 at 12:02 PM, Janko Mivšek <[hidden email]> wrote:


Rob Rothwell wrote:

On Thu, Jun 5, 2008 at 9:44 AM, Janko Mivšek <[hidden email] <mailto:[hidden email]>> wrote:

   Yes, WebGrid (kind of smart web table) caches presented objects in
   session presentation state (instances of your Apps, that is
   subclasses of WebApplication) until you clean up that state. Because
   Aida web apps don't have a session timeout by default, this cleanup
   is done explicitly, usually every night.


I think I really need some advice on a "tried and true" recipe for getting the state cleaned up.  

Below cleanup is quite drastic I should say :) I think enough will be just to:

1. remove object from urlResolver immediatelly after not needed anymore:


 (AIDASite named: 'aidademo') urlResolver removeObject: m

2. Overnight do :

 WebSessionManager allInstancesDo: [:each | each removeGuestSessions]

 WebSession allInstances do: [:each | each initAppsForObjects]
 Smalltalk garbageCollect

This is what I took away from a previous email from you, but "when" should the object disappear?  I am still using the example I sent yesterday, have created one item, done the above, and it just won't go away!  I still see WebGridColumn>>defaultViewBlock, and aWebCheckBox still pointing to my item.

I am using the Aida5.6-np.49 version of Aida so I could play with the Scriptaculous stuff...

That way I'm sure you will keep object under control, at least from Aida standpoint. You can put overnight cleanup routine into override of:

SwazooServer>>watchdogOther

 self isNightlyBackupTime ifTrue:
   [Transcript cr; show: 'closing all HTTP connections...'.
    HTTPConnection allInstances do: [:each | each close].
      "to GC all streams and buffers"
   Transcript cr; show: 'removing guest sessions...'.
   WebSessionManager allInstances do: [:each |
     each removeGuestSessions].
   Transcript cr; show: 'removing apps for objects in sessions...'.

   WebSession allInstances do: [:each | each initAppsForObjects].
   Transcript cr; show: 'backup...'.
   "put your backup call here"
   (Delay forSeconds: self watchdogPeriod) wait.
       "otherwise isNightlyBackupTime'll be again true!

SwazooServer>>isNightlyBackupTime
 "backup should be done at 4:30:00"
 "SwazooServer singleton isNightlyBackupTime"
 | backupTime |
 backupTime := SpTimestamp
    fromDate: Date today
    andTime: (Time readFrom: '4:30:00' readStream).
 ^(SpTimestamp now asSeconds > backupTime asSeconds) &
     (SpTimestamp now asSeconds < (backupTime asSeconds +
         self watchdogPeriod))


This is what I'm using in currently for cleaning my Squeak image.

Thanks for the watchdog example, though.  When I do get this figured out, it will be great to automate it.

Rob

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Rob Rothwell
On Thu, Jun 5, 2008 at 1:36 PM, Rob Rothwell <[hidden email]> wrote:
   Yes, WebGrid (kind of smart web table) caches presented objects in
   session presentation state (instances of your Apps, that is
   subclasses of WebApplication) until you clean up that state. Because
   Aida web apps don't have a session timeout by default, this cleanup
   is done explicitly, usually every night.

After much experimentation, I am of the opinion that a the checkbox collection and the compiled WebGridColumn>>defaultViewBlock hold on to their items quite tenaciously and survive normal attempts at cleaning them up!

For now I am trying to figure out how to ask any of the WebGrid(s) I have created to release their objects. 

For example just telling the associated WebCheckBox>>object := nil and then doing a garbageCollect frees up one pointer to my object.

The other one exists (ListItem) as shown below in the defaultViewBlock for the grid after running suggested procedure, garbage collecting, saving the image, etc...

WebGridColumn>>defaultViewBlock
root: WebGridColumn>>defaultViewBlock
    sender: nil
    pc: nil
    stackp: 1
    method: a CompiledMethod (548)
    receiverMap: nil
    receiver: aWebGridColumn id: 1
    1: a ListItem

Perhaps no one has created and released as many grids as I am to have noticed this before?  Either that, or I am still doing something wrong.

Rob


_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Rob Rothwell
On Fri, Jun 6, 2008 at 5:23 PM, Rob Rothwell <[hidden email]> wrote:
After much experimentation, I am of the opinion that a the checkbox collection and the compiled WebGridColumn>>defaultViewBlock hold on to their items quite tenaciously and survive normal attempts at cleaning them up!

Apologies...a little more exploring revealed that WebSession>>lastApp was holding on to my application and keeping anything from being released.  This explains a lot, except I don't know why WebSessionManager allInstances do: [:each | each removeAllSessions] does not actually mean that I don't have instances of WebSession hanging out...

Anyway..can I safely clear lastApp to enable full garbage collection?  Even when I use your code:

    HTTPConnection allInstances do: [:each | each close].
      "to GC all streams and buffers"
   Transcript cr; show: 'removing guest sessions...'.
   WebSessionManager allInstances do: [:each |
     each removeAllSessions].

To close connections, etc... WebSession(s) still remain(?) and retain their lastApp, which holds on to it's domain object, etc...

So if I have an image whose sole purpose is to do one thing, the lastApp will be the ONLY application, and hence nothing will ever get garbage collected!  Which, of course, was why my image kept growing.

Now if I just go to the demo app site so it becomes the last app, and then run the clean up code, everything goes away (except the WebSessions).

Sorry for so many questions on this one, but it was quite mysterious to me.  I'm still a little confused, I guess, and am probably doing something not quite right, but at least I think I can keep it under control now.

Or, is it a bigger problem that I can't seem to clear the WebSessions?  That seems like it could be a big issue at some point!

Thanks again,

Rob



_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Janko Mivšek
Rob Rothwell wrote:

> Apologies...a little more exploring revealed that WebSession>>lastApp
> was holding on to my application and keeping anything from being
> released.  This explains a lot, except I don't know why
> WebSessionManager allInstances do: [:each | each removeAllSessions] does
> not actually mean that I don't have instances of WebSession hanging out...
>
> Anyway..can I safely clear lastApp to enable full garbage collection?  

Yes, if you do initAppsForObjects, then you should aslo nil lastApp in
all sessions. So, the final cleanup routine for Squeak would be:

    HTTPConnection allInstances do: [:each | each close]
    WebSessionManager allInstancesDo: [:each | each removeGuestSessions]
    WebSession allInstances do: [:each |
        each initAppsForObjects; nilLastApp]
    Smalltalk garbageCollect

Sorry that I forgot on that.

Janko

> Even when I use your code:
>
>     HTTPConnection allInstances do: [:each | each close].
>       "to GC all streams and buffers"
>    Transcript cr; show: 'removing guest sessions...'.
>    WebSessionManager allInstances do: [:each |
>      each removeAllSessions].
>
> To close connections, etc... WebSession(s) still remain(?) and retain
> their lastApp, which holds on to it's domain object, etc...
>
> So if I have an image whose sole purpose is to do one thing, the lastApp
> will be the ONLY application, and hence nothing will ever get garbage
> collected!  Which, of course, was why my image kept growing.
>
> Now if I just go to the demo app site so it becomes the last app, and
> then run the clean up code, everything goes away (except the WebSessions).
>
> Sorry for so many questions on this one, but it was quite mysterious to
> me.  I'm still a little confused, I guess, and am probably doing
> something not quite right, but at least I think I can keep it under
> control now.
>
> Or, is it a bigger problem that I can't seem to clear the WebSessions?  
> That seems like it could be a big issue at some point!
>
> Thanks again,
>
> Rob
>
>

--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Rob Rothwell
On Sat, Jun 7, 2008 at 7:29 AM, Janko Mivšek <[hidden email]> wrote:
Yes, if you do initAppsForObjects, then you should aslo nil lastApp in all sessions. So, the final cleanup routine for Squeak would be:


  HTTPConnection allInstances do: [:each | each close]
  WebSessionManager allInstancesDo: [:each | each removeGuestSessions]
  WebSession allInstances do: [:each |
       each initAppsForObjects; nilLastApp]
  Smalltalk garbageCollect

Sorry that I forgot on that.

I think the hardest thing for me about Smalltalk is carefully reading through the categories and methods others have written and actually FINDING something...like "nilLastApp."  Just what I needed, but it's almost like I don't yet "think" to look for it or something.

Thanks!

Rob


_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Janko Mivšek
Rob Rothwell wrote:

> I think the hardest thing for me about Smalltalk is carefully reading
> through the categories and methods others have written and actually
> FINDING something...like "nilLastApp."  Just what I needed, but it's
> almost like I don't yet "think" to look for it or something.

Yes, in Smalltalk the best way to learn is just to wander around all
those methods from time to time. You'll learn a lot that way. And when
you'll really need something, you'll remember that you see somewhere
some method, useful for your case. Now you just need to find it again...

Janko

--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida
Reply | Threaded
Open this post in threaded view
|

Re: Nightly Cleanup Recipe (Was Proper object removal)

Rob Rothwell
In reply to this post by Janko Mivšek
On Sat, Jun 7, 2008 at 7:29 AM, Janko Mivšek <[hidden email]> wrote:
Yes, if you do initAppsForObjects, then you should aslo nil lastApp in all sessions. So, the final cleanup routine for Squeak would be:


  HTTPConnection allInstances do: [:each | each close]
  WebSessionManager allInstancesDo: [:each | each removeGuestSessions]
  WebSession allInstances do: [:each |
       each initAppsForObjects; nilLastApp]
  Smalltalk garbageCollect

Another place objects can "hide out" is in the URLResolver, which you can take care of pro-actively of course with

(AIDASite named: 'mysite') urlResolver removeObject: myObject as soon as you are done with the object, but if you are developing and have not done this correctly, it appears that adding:

(AIDASite named: 'mysite') urlResolver initialize before the garbageCollect will clean things up...

Rob

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida