Auto strip leaves development class references

I'm stumped on how to get an image strip in D5 to be clean, without any
development classes remaining, when run automatically. The exact same
strip, when done manually comes up clean:
        "All development classes were successfully stripped from the image"
But the automatic one has: [1]
        "Development classes remaining in image
        ClassBrowserAbstract has 0 instance(s)
        SmalltalkSystem has 1 instance(s)
        SmalltalkToolShell has 0 instance(s)
        SmalltalkWorkspaceDocument has 0 instance(s)
        SystemBrowserShell has 0 instance(s)
        ViewComposer has 0 instance(s)"

Testing using the script:
------
"
SessionManager inputState queueDeferredAction: (MessageSend receiver:
(Package manager packageNamed: 'ST Application Linked') imageStripper
selector: #stripAndSaveWithProgress).
"
        Smalltalk at: #DoingAutomaticStrip put: true.
        (SmalltalkWorkspaceDocument filename:
'Z:\working\devel\Dolphin5\StripLinkedImage.st') workspace
                selectionRange: (4 to: 182)
                " ; evaluateIt. "
!
------

It's a bit funky in that it opens a workspace on itself, and selects the
commented out code at the top. But I wanted to minimize the manual
interaction needed (and the differences between doing it manually and
automatically).

This file is filed in by MyDevelopmentSessionManager>>main (due to a
command line argument). For the manual case, after the workspace appears
I can type a single ctrl-E and the strip occurs. For the automatic case,
the script (with the #evaluateIt uncommented) does the same strip, but
ends up with development classes remaining.

Any ideas?

I've tried so many combinations of forks, delays and postToInputQueues
that I can't see straight any more. And having it take about 6 minutes
for each attempt makes it rather painful.

thanks,
-Bill

---------------
[1] Details for extant references:

ClassBrowserAbstract 0 0 0 #(SystemBrowserShell)

SmalltalkSystem 1 0 0 #(a SmalltalkSystem)

SmalltalkToolShell 0 0 0 #(ClassBrowserAbstract ViewComposer)

SmalltalkWorkspaceDocument 0 0 0 #(a WeakIdentityDictionary(a WeakSet()
-> an EventsCollection(#elementsExpired: -> an EventMessageSequence)
SystemBrowserShell -> an EventsCollection(#viewOpened: -> an
EventMessageSequence) a SchedulingLinkedSessionManager -> an
EventsCollection(#sessionStopped -> an EventMessageSequence
#imageSaveStarting -> an EventMessageSequence #sessionStarted -> an
EventMessageSequence) SmalltalkWorkspaceDocument -> an
EventsCollection(#viewOpened: -> an EventMessageSequence)) a
SmalltalkSystem)

SystemBrowserShell 0 0 0 #(a WeakIdentityDictionary(a WeakSet() -> an
EventsCollection(#elementsExpired: -> an EventMessageSequence)
SystemBrowserShell -> an EventsCollection(#viewOpened: -> an
EventMessageSequence) a SchedulingLinkedSessionManager -> an
EventsCollection(#sessionStopped -> an EventMessageSequence
#imageSaveStarting -> an EventMessageSequence #sessionStarted -> an
EventMessageSequence) SmalltalkWorkspaceDocument -> an
EventsCollection(#viewOpened: -> an EventMessageSequence)) a
SmalltalkSystem)

ViewComposer 0 0 0 #(a SmalltalkSystem)

--
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA

Bill,

> SessionManager inputState queueDeferredAction: (MessageSend receiver:
> (Package manager packageNamed: 'ST Application Linked') imageStripper
> selector: #stripAndSaveWithProgress).

FWIW, I use exactly that technique in my QuickDeploy tool and, so far, it has
always seemed to work perfectly.   The only difference I can think from what
you describe is that there's no workspace involved for me -- just a simple GUI
front-end telling an object to enqueue a MessageSend.  It is possible that the
workspace itself, or something associated with it such as the compiled doIt, is
holding on to unwanted references ?

    -- chris

Chris Uppal wrote:
> FWIW, I use exactly that technique in my QuickDeploy tool and, so far, it has
> always seemed to work perfectly.   The only difference I can think from what
> you describe is that there's no workspace involved for me -- just a simple GUI
> front-end telling an object to enqueue a MessageSend.  It is possible that the
> workspace itself, or something associated with it such as the compiled doIt, is
> holding on to unwanted references ?

It's not the workspace, since that works when fired off interactively.
In fact the workspace wasn't there at all when doing the normal
automatic way. As I said, it's just there to make the manual way (which
does work) need the minimal amount of interaction (a single keystroke),
and make the manual and automatic ways as close as possible to each
other (in hopes of discovering what the crucial difference is).

Note that the "automatic" that I'm talking about here is all the way
from the DOS command line (the strip is just part of a bigger batch
process).

There may be some sort of lazy initialization that occurs after the
image starts that makes the difference. Though wrapping the body of the
script in a background fork with a delay, to allow the main UI process
to quiesce and give the idle process a chance to run, before queuing the
strip hasn't helped. I'm baffled.

--
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA

Bill,

> There may be some sort of lazy initialization that occurs after the
> image starts that makes the difference. Though wrapping the body of the
> script in a background fork with a delay, to allow the main UI process
> to quiesce and give the idle process a chance to run, before queuing the
> strip hasn't helped. I'm baffled.

Have you tried some explicit #collectGarbage and #administerLastRites sends?

Have a good one,

Bill


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Bill Schwab wrote:
> Have you tried some explicit #collectGarbage and #administerLastRites
> sends?

I hadn't bothered after seeing how the ImageStripper was doing those at
every turn. Didn't see how a few additional ones before everything
really started would be any help. But for completeness, I just gave it a
try. I threw in 10, both in the main process and in the background
process (after the main had idled), before queuing the strip. As I
suspected, no difference. :-(


--
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA

Bill,

> > Have you tried some explicit #collectGarbage and #administerLastRites
> > sends?
>
> I hadn't bothered after seeing how the ImageStripper was doing those at
> every turn. Didn't see how a few additional ones before everything
> really started would be any help. But for completeness, I just gave it a
> try. I threw in 10, both in the main process and in the background
> process (after the main had idled), before queuing the strip. As I
> suspected, no difference. :-(

Can you create a reasonably simple reproducible case, perhaps using one of the
sample deployables in Dolphin ?

    -- chris

Chris Uppal wrote:
> Can you create a reasonably simple reproducible case, perhaps using one of the
> sample deployables in Dolphin ?

My first reaction was no, I can't. But nothing like trying, and proving
myself wrong. ;-)

What's been described previously has been in Pro 5.1.4, but after
isolating it, I found the same problem manifests in Pro 6.02.

I found that how the image is saved initially (to serve as the base for
all future automatic building) affects whether all the development
classes can be cleanly removed or not. I haven't figured out why. Maybe
someone else can.

Starting with a fresh 6.02 image (looks cleaner if tool tips have been
turned off, but I found that it actually doesn't matter) --

methods that leave some development classes:
1a)
     - File In "MyDevelopmentSessionManager.st"
     - Save Image As "Test"
     - Exit - don't save image
1b)
     - Save Image As "Test"
     - File In "MyDevelopmentSessionManager.st"
     - Save Image
     - Exit - don't save image

methods that strip cleanly:
2a)
     - Save Image As "Test"
     - File In "MyDevelopmentSessionManager.st"
     - Exit and Save image
2b)
     - File In "MyDevelopmentSessionManager.st"
     - Save Image As "Test"
     - Exit and Save image

For each case, run the testStrip.bat batch file to start with Test.img
and create the Hello World.exe.

The key seems to be doing the save that occurs as part of exiting, and
not a stand-alone save.

When I use the Executable Browser to look at the .xml manifest, I see
that in case (1) that, for example ClassBrowserShell, is still there,
whereas in case (2) it's not.

[BTW, in D5 the .htm manifest had a heading "Development classes
remaining in image" followed by a list of them. Is there something
similar it D6? Or does one have to scroll through the classes in the
executable browser and try to pick out those that shouldn't be there?]

Would be interesting to understand what's really going on here. Why the
type of save matters. And why initiating the strip manually works.

-Bill


----- testStrip.bat -----
start /wait "title" "C:\Program Files\Object Arts\Dolphin Smalltalk
6.0\Dolphin.exe" -nosplash Test.img -fileIn=StripTestImage.st
----------

----- StripTestImage.st -----
SessionManager inputState queueDeferredAction: (MessageSend
        receiver:(Package manager packageNamed: 'Hello World') imageStripper
        selector: #stripAndSaveWithProgress)!
----------

----- MyDevelopmentSessionManager.st -----
DevelopmentSessionManager subclass: #MyDevelopmentSessionManager
        instanceVariableNames: ''
        classVariableNames: ''
        poolDictionaries: ''
        classInstanceVariableNames: ''!

!MyDevelopmentSessionManager methodsFor!

main
        | prefix |
        super main.
        prefix := '-fileIn='.
        self argv do:
                [:each |
                ((each leftString: prefix size) sameAs: prefix)
                        ifTrue: [SourceManager default fileIn: (each rightString: each size -
prefix size)]]! !

MyDevelopmentSessionManager installNew!
----------

--
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA

Bill

> I found that how the image is saved initially (to serve as the base for
> all future automatic building) affects whether all the development
> classes can be cleanly removed or not. I haven't figured out why. Maybe
> someone else can.

I can tell you half the answer, but only half, and probably not the most
interesting half at that :-(

The half I can say is that when you save the image explicitly, that is
implemented by some Command object being routed to the current SmalltalkSystem.
At the instant the image is saved, that Command is still active, and so is
still the value of the 'Current' classvar of Command. It contains a reference
to its #target -- which is the current SmalltalkSystem.  On the other hand,
when you save the image as a side effect of exiting Dolphin, the quit command
is posted to the input queue, and so will not be acted on until after the
originating Command object has done it's stuff, and in winding up has reset
Command.Current to nil.

Now what I don't understand is why Command.Current is not reset as the
snapshotted (and snapshotting) 'main' process's stack is unwound as the
deployment image is started up.

I believe that your SessionManager's #main (and so your deployment script) will
be run from the Process which performed the snapshot, as if invoked directly
from the SessionManager>>snapshot method.  So (if I'm right) the Command object
which initiated the save will still be in its #value method while your #main
runs -- and so that method's #ensure: block will not have been executed yet.
Thus Command.Current will not yet have been nilled-out.  So far so good.  But
since you use a deferred action, and especially since you mention adding delays
in various places, I don't see why your #main hasn't returned by the time the
queued action is executed.  Possibly it's just bad luck.  It might be worth
putting a delay into the executed action itself, as an experiment to see if
that allows #main to tidy up (putting delays in #main would just prolong the
time before you return to Command>>value).

Anyway, whatever the reason why Command.Current hasn't been reset; you can
confirm that that is the problem by defining something like
    Command class>> discardCurrent
        Current := nil.
and then invoking that as the first step of your automatic deployment script.
For me (in D5) that seems to make the problem go away.

Alternatively, and less hackishly, it seems simpler always to build deployment
images using an implicit save on exit ;-)

    -- chris

Chris, thanks for your thoughts! Now as part of the "team" effort, I
think that I can fill in the blanks.

Chris Uppal wrote:
Explanation to follow...

> I believe that your SessionManager's #main (and so your deployment script) will
> be run from the Process which performed the snapshot, as if invoked directly
> from the SessionManager>>snapshot method.  

BTW, Doesn't look that way. SessionManager>>main is called from
#mainLoopStarted which is done as a deferred action by
SessionManager>>forkMain.

> So (if I'm right) the Command object
> which initiated the save will still be in its #value method while your #main
> runs -- and so that method's #ensure: block will not have been executed yet.
> Thus Command.Current will not yet have been nilled-out.  So far so good.  But
> since you use a deferred action, and especially since you mention adding delays
> in various places, I don't see why your #main hasn't returned by the time the
> queued action is executed.

Check out the end of SessionManager>>onStartup:. It ends with "Processor
activeProcess kill" to end the "snapshotting" process. Looking at the
comment in #kill we see "Terminate the receiver with extreme prejudice.
Any outstanding unwind blocks will not be run."

So that's why Command.Current isn't cleared by #ensure: block. This also
explains why my earlier description of firing off the strip manually
worked -- the command was cleared after the evaluateIt was evoked from
the keyboard, but not when done automatically from the script since it
did #evaluateIt directly on the workspace, bypassing the command routing.

> Alternatively, and less hackishly, it seems simpler always to build deployment
> images using an implicit save on exit ;-)

That's what I did, once I figured out that would work. Nice to have
another option now. Maybe OA could put in a tweak so that the save image
/ restart would be a bit cleaner.

--
Bill Dargel            [hidden email]
Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA

Bill,

> Now as part of the "team" effort, I
> think that I can fill in the blanks.

That all sounds correct.  Good stuff !

(I am clearly in need of a refresher course on the startup sequence ;-)

    -- chris