Files | #startUp and #shutDown in FileDirectory

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Files | #startUp and #shutDown in FileDirectory

marcel.taeumel
Hi, there. :-)

Why do we reset file-related stuff (e.g. source file handles, default directory class)  on every snapshot? Shouldn't there be a check for "resuming" (or "quitting") in FileDirectory class >> #startUp(:) (and #shutDown(:))?

...or am I missing something? Tim (tpr)! You started it in 2003. Please, explain. :-D

Maybe ... we don't ever want to store dangling source-file handles in the image?

Best,
Marcel


Reply | Threaded
Open this post in threaded view
|

Re: Files | #startUp and #shutDown in FileDirectory

David T. Lewis
On Wed, May 27, 2020 at 08:58:48AM +0200, Marcel Taeumel wrote:
> Hi, there. :-)
>
> Why do we reset file-related stuff (e.g. source file handles, default directory class)?? on every snapshot? Shouldn't there be a check for "resuming" (or "quitting") in FileDirectory class >> #startUp(:) (and #shutDown(:))?
>
> ...or am I missing something? Tim (tpr)! You started it in 2003. Please, explain.??:-D
>
> Maybe ... we don't ever want to store dangling source-file handles in the image?
>

Yes I think that is the reason. The image snapshot should contain little or
no platform-specific stuff, and there is no point in allowing it to contain
obsolete file handles that would need to be detected by primitive fallback
code when the snapshot is next loaded.

After all, the image that you snapshot today may next be opened by someone
with a Risc OS computer :-)

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Files | #startUp and #shutDown in FileDirectory

marcel.taeumel
Hi Dave!

Thanks. :-)

Considering a bigger but similar, maybe FFI-based, mechanism, do you think it makes sense to clean up such things on a fresh startup only instead of every snapshot? Think of flushing the entire GPU-state you are currently managing just to retain some other in-image object. Seems overkill.

What's the underlying best practice? :-)

Best,
Marcel

Am 28.05.2020 15:26:09 schrieb David T. Lewis <[hidden email]>:

On Wed, May 27, 2020 at 08:58:48AM +0200, Marcel Taeumel wrote:
> Hi, there. :-)
>
> Why do we reset file-related stuff (e.g. source file handles, default directory class)?? on every snapshot? Shouldn't there be a check for "resuming" (or "quitting") in FileDirectory class >> #startUp(:) (and #shutDown(:))?
>
> ...or am I missing something? Tim (tpr)! You started it in 2003. Please, explain.??:-D
>
> Maybe ... we don't ever want to store dangling source-file handles in the image?
>

Yes I think that is the reason. The image snapshot should contain little or
no platform-specific stuff, and there is no point in allowing it to contain
obsolete file handles that would need to be detected by primitive fallback
code when the snapshot is next loaded.

After all, the image that you snapshot today may next be opened by someone
with a Risc OS computer :-)

Dave



Reply | Threaded
Open this post in threaded view
|

Re: Files | #startUp and #shutDown in FileDirectory

David T. Lewis
Hi Marcel,

I don't really know the answer, but I would suggest to think of it in
terms of what things you want to store in the actual snapshot file.
If something like the GPU-state is safe to store, in the sense that it
will cause no harm for someone trying to load the same image file, then
maybe it is just a tradeoff between space wasted in the snapshot versus
the time it takes to recover state in your running image after doing
a snapshot.

In the case of the source files, it is simple because reopening the
source files it easy and has no performance implications. But for
something complex such as GPU-state, you might want to adopt a different
policy.

In the back of my mind I am also thinking of image snapshot objects
that can be transmitted over the network with no disk-based files,
e.g. http://www.squeaksource.com/ImageSnapshot. That would suggest
that when in doubt it is best to err on the side of cleaning up the
garbage before doing a snapshot.

Dave

On Thu, May 28, 2020 at 03:42:11PM +0200, Marcel Taeumel wrote:

> Hi Dave!
>
> Thanks. :-)
>
> Considering a bigger but similar, maybe FFI-based, mechanism, do you think it makes sense to clean up such things on a fresh startup only instead of every snapshot? Think of flushing the entire GPU-state you are currently managing just to retain some other in-image object. Seems overkill.
>
> What's the underlying best practice? :-)
>
> Best,
> Marcel
> Am 28.05.2020 15:26:09 schrieb David T. Lewis <[hidden email]>:
> On Wed, May 27, 2020 at 08:58:48AM +0200, Marcel Taeumel wrote:
> > Hi, there. :-)
> >
> > Why do we reset file-related stuff (e.g. source file handles, default directory class)?? on every snapshot? Shouldn't there be a check for "resuming" (or "quitting") in FileDirectory class >> #startUp(:) (and #shutDown(:))?
> >
> > ...or am I missing something? Tim (tpr)! You started it in 2003. Please, explain.??:-D
> >
> > Maybe ... we don't ever want to store dangling source-file handles in the image?
> >
>
> Yes I think that is the reason. The image snapshot should contain little or
> no platform-specific stuff, and there is no point in allowing it to contain
> obsolete file handles that would need to be detected by primitive fallback
> code when the snapshot is next loaded.
>
> After all, the image that you snapshot today may next be opened by someone
> with a Risc OS computer :-)
>
> Dave
>

>


Reply | Threaded
Open this post in threaded view
|

Re: Files | #startUp and #shutDown in FileDirectory

K K Subbu
In reply to this post by David T. Lewis
On 28/05/20 6:56 pm, David T. Lewis wrote:
> Yes I think that is the reason. The image snapshot should contain
> little or no platform-specific stuff,

I guess you meant process-specific state. After a snapshot, an image may
be resumed by a VM process on the same platform, but we shouldn't be
holding on per-process states like namespace handles, file handles,
sockets, security context, gpu states etc.

I always wondered why Squeak choose a single entry point in classes for
coldStart and warmResume. Expecting classes to resolve these conditions
means that every class has to test the resuming flag again to branch out
to different code paths.

Embedded system programmers would do something like:

coldStart: hwOptions "pass hardware detected/selected here"
       "initialize virtualized h/w state (e.g. display, input)"
       "load session options (e.g. saved prefs)"
       self warmResume: sessionOptions

warmResume: sessionOptions "pass session specific configs here"

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: Files | #startUp and #shutDown in FileDirectory

timrowledge
In reply to this post by marcel.taeumel


> On 2020-05-26, at 11:58 PM, Marcel Taeumel <[hidden email]> wrote:
> ...or am I missing something? Tim (tpr)! You started it in 2003. Please, explain. :-D
>
> Maybe ... we don't ever want to store dangling source-file handles in the image?

Wow, that's another ancient thing. I think yes, the desire to not have dangling potential-crap pointers would have driven that. You certainly don't want the saved image to get started up with with raw addresses to OS stuff. I can't find any emails mentioning it.

I *suspect* that back then we probably didn't have the extra bit of info to say if the image was being saved & quit, saved & resumed, started up, etc. If that was the case then it would sensible to shut down everything and restart it on returning to normal running.

If one has a suitable way to carry the information about what is happening then you might choose to make the startup-from-snaphsot code do the work of killing old pointers and (re)opening things. A save&resume could be allowed to keep things to save some time.  We could consider making a save&quit do a cleanup as a courtesy to the next time the saved image is started. It might be helpful to Dave's osprocess forking etc to do cleanup before the fork?

I haven't looked at this code in a long time, so maybe we are already near to doing this.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: YII: Yield to Irresistable Impulse



Reply | Threaded
Open this post in threaded view
|

Re: Files | #startUp and #shutDown in FileDirectory

Eliot Miranda-2
In reply to this post by marcel.taeumel
Hi Marcel,

On Thu, May 28, 2020 at 6:42 AM Marcel Taeumel <[hidden email]> wrote:
Hi Dave!

Thanks. :-)

Considering a bigger but similar, maybe FFI-based, mechanism, do you think it makes sense to clean up such things on a fresh startup only instead of every snapshot? Think of flushing the entire GPU-state you are currently managing just to retain some other in-image object. Seems overkill.

It's not just overkill, it is wrong.  This is a very important issue.  Here's an example:

shutDown: quitting

quitting ifTrue: [
ScorePlayer allSubInstancesDo: [:ea | [ea stopMIDIPlaying] on: Error do: [] ] ].

This in fact should be done on the startUp: side.

If there is an alternative implementation it would be something like this:

ScorePlayer class>>shutDown

Smalltalk for: self addVeryEarlyStartupAction: [ScorePlayer allSubInstancesDo: [:ea | [ea stopMIDIPlaying] on: Error do: [] ] ]

where the early startup actions would be executed before the rest of the startup actions.

But better is, I think, to start-up by

- first suspending any and all runnable processes except the current process, the finalization process, and maybe the low space process, to stop any actions left from the snapshot occurring until after start-up is complete
- run the startUp: actions, having moved things like ScorePlayer class>>shutDown's body to ScorePlayer class>>startUp:
- carefully consider the ordering of start-up actions (having a pragma based scheme <dependsOn: #ClassName> would allow automatic sorting of the list in a rational way)
- resume any processes suspended before start-up




What's the underlying best practice? :-)

Best,
Marcel

Am 28.05.2020 15:26:09 schrieb David T. Lewis <[hidden email]>:

On Wed, May 27, 2020 at 08:58:48AM +0200, Marcel Taeumel wrote:
> Hi, there. :-)
>
> Why do we reset file-related stuff (e.g. source file handles, default directory class)?? on every snapshot? Shouldn't there be a check for "resuming" (or "quitting") in FileDirectory class >> #startUp(:) (and #shutDown(:))?
>
> ...or am I missing something? Tim (tpr)! You started it in 2003. Please, explain.??:-D
>
> Maybe ... we don't ever want to store dangling source-file handles in the image?
>

Yes I think that is the reason. The image snapshot should contain little or
no platform-specific stuff, and there is no point in allowing it to contain
obsolete file handles that would need to be detected by primitive fallback
code when the snapshot is next loaded.

After all, the image that you snapshot today may next be opened by someone
with a Risc OS computer :-)

Dave




--
_,,,^..^,,,_
best, Eliot