[vwnc] Image building under Linux

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[vwnc] Image building under Linux

cdavidshaffer
I've been trying to automate building my VW7.6 images under Linux but
I'm having a problem.  If my build script closes a window such as:

Workbook allInstances do: [:each | each closeAndUnschedule].
[ObjectMemory saveAs: 'myimage' thenQuit: true] fork.

If I load a script like the above using "visual visualnc.im -filein
script.st" then resulting image is unstable.  When it is running
head-full I get errors like:

X11 Error:
Display:    0x0810e5a8
Serial:    0x0000008a
Major:    2
Minor:    0
XID:    0x0260006c
BadWindow (invalid Window parameter)

and the VM segfaults on image save.  Is there a proper way to do this?

On a more general note, are there alternatives to Randy's CruiseControl
or is this really the only tool for automating image builds in VW?

David

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

david.long
I would trying the build from an ultra clean image. Start with a new image, and don't open or run anything at all. Just do the build and let it make the runtime image. Old objects can wreak havoc if they get into an image just before it is built into a runtime.

Keep me posted on how it goes -

David

On Thu, 2008-07-31 at 13:01 -0400, C. David Shaffer wrote:
I've been trying to automate building my VW7.6 images under Linux but 
I'm having a problem.  If my build script closes a window such as:

Workbook allInstances do: [:each | each closeAndUnschedule].
[ObjectMemory saveAs: 'myimage' thenQuit: true] fork.

If I load a script like the above using "visual visualnc.im -filein 
script.st" then resulting image is unstable.  When it is running 
head-full I get errors like:

X11 Error:
Display:    0x0810e5a8
Serial:    0x0000008a
Major:    2
Minor:    0
XID:    0x0260006c
BadWindow (invalid Window parameter)

and the VM segfaults on image save.  Is there a proper way to do this? 

On a more general note, are there alternatives to Randy's CruiseControl 
or is this really the only tool for automating image builds in VW?

David

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Your business, your way.
http://sageteagroup.com

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Terry Raymond
In reply to this post by cdavidshaffer
David

I cannot say for sure what the problem is, but only the
window manager process should close a window. You should
do something like;

Workbook allInstances do:
        [:each | [each closeAndUnschedule] uiEventFor: each builder window].
(Delay forSeconds: 5) wait. "You need to wait for all the windows to close"
[ObjectMemory saveAs: 'myimage' thenQuit: true] fork.

Terry
 
===========================================================
Terry Raymond
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of C. David Shaffer
> Sent: Thursday, July 31, 2008 1:02 PM
> To: [hidden email]
> Subject: [vwnc] Image building under Linux
>
> I've been trying to automate building my VW7.6 images under Linux but
> I'm having a problem.  If my build script closes a window such as:
>
> Workbook allInstances do: [:each | each closeAndUnschedule].
> [ObjectMemory saveAs: 'myimage' thenQuit: true] fork.
>
> If I load a script like the above using "visual visualnc.im -filein
> script.st" then resulting image is unstable.  When it is running
> head-full I get errors like:
>
> X11 Error:
> Display:    0x0810e5a8
> Serial:    0x0000008a
> Major:    2
> Minor:    0
> XID:    0x0260006c
> BadWindow (invalid Window parameter)
>
> and the VM segfaults on image save.  Is there a proper way to do this?
>
> On a more general note, are there alternatives to Randy's CruiseControl
> or is this really the only tool for automating image builds in VW?
>
> David
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Randy Coulman
In reply to this post by cdavidshaffer

On Thu, Jul 31, 2008 at 10:01 AM, C. David Shaffer <[hidden email]> wrote:
I've been trying to automate building my VW7.6 images under Linux but
I'm having a problem.  If my build script closes a window such as:

Workbook allInstances do: [:each | each closeAndUnschedule].
[ObjectMemory saveAs: 'myimage' thenQuit: true] fork.

If I load a script like the above using "visual visualnc.im -filein
script.st" then resulting image is unstable.  When it is running
head-full I get errors like:

X11 Error:
Display:    0x0810e5a8
Serial:    0x0000008a
Major:    2
Minor:    0
XID:    0x0260006c
BadWindow (invalid Window parameter)

and the VM segfaults on image save.  Is there a proper way to do this?

I don't have access to my build scripts right now, but I believe I throw a garbage collection operation in there before the last save.  That might help you here.


On a more general note, are there alternatives to Randy's CruiseControl
or is this really the only tool for automating image builds in VW?

You could do pretty much any kind of homegrown scripting here with cron jobs or whatever.  CruiseControl is really just a wrapper around the various parts you have to set up yourself.  But, if you need something that watches for changes somewhere, launches your build for you, and reports the results, then CruiseControl is a decent option.

Randy
--
Randy Coulman
[hidden email]

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Alan Knight-2
In reply to this post by Terry Raymond
I suspect the critical bit is the delay and forking the save. If you're running the script via command-line options like -doit, then that's running as part of the image start-up logic. You want to make sure that that startup logic has entirely finished before you save your image, and you don't really want all of that stuff on your stack (even just for the space it likely takes). And with things like closing windows that run asynchronously, you want to make sure they've entirely finished - you don't want your image saved with a window half-way through closing.

We've thought about trying to provide something more convenient when saving, but it's rather tricky. You know that the minute you define something that says "do this at the end of the startup process" that you will immediately need to make something run afterwards. The forking is what we do internally in image builds.

At 01:25 PM 7/31/2008, Terry Raymond wrote:
David

I cannot say for sure what the problem is, but only the
window manager process should close a window. You should
do something like;

Workbook allInstances do:
        [:each | [each closeAndUnschedule] uiEventFor: each builder window].
(Delay forSeconds: 5) wait. "You need to wait for all the windows to close"
[ObjectMemory saveAs: 'myimage' thenQuit: true] fork.

Terry
 
===========================================================
Terry Raymond
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
< http://www.craftedsmalltalk.com>
===========================================================

> -----Original Message-----
> From: [hidden email] [[hidden email]] On Behalf Of C. David Shaffer
> Sent: Thursday, July 31, 2008 1:02 PM
> To: [hidden email]
> Subject: [vwnc] Image building under Linux
>
> I've been trying to automate building my VW7.6 images under Linux but
> I'm having a problem.  If my build script closes a window such as:
>
> Workbook allInstances do: [:each | each closeAndUnschedule].
> [ObjectMemory saveAs: 'myimage' thenQuit: true] fork.
>
> If I load a script like the above using "visual visualnc.im -filein
> script.st" then resulting image is unstable.  When it is running
> head-full I get errors like:
>
> X11 Error:
> Display:    0x0810e5a8
> Serial:    0x0000008a
> Major:    2
> Minor:    0
> XID:    0x0260006c
> BadWindow (invalid Window parameter)
>
> and the VM segfaults on image save.  Is there a proper way to do this?
>
> On a more general note, are there alternatives to Randy's CruiseControl
> or is this really the only tool for automating image builds in VW?
>
> David
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

--
Alan Knight [|], Engineering Manager, Cincom Smalltalk

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

cdavidshaffer
In reply to this post by cdavidshaffer
Oops, I need to learn to press "reply all" :-(

Terry Raymond wrote:

> David
>
> I cannot say for sure what the problem is, but only the
> window manager process should close a window. You should
> do something like;
>
> Workbook allInstances do:
> [:each | [each closeAndUnschedule] uiEventFor: each builder window].
> (Delay forSeconds: 5) wait. "You need to wait for all the windows to close"
> [ObjectMemory saveAs: 'myimage' thenQuit: true] fork.
>
> Terry
>  
That fixed it!  Thanks Terry.

David



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

cdavidshaffer
In reply to this post by cdavidshaffer
... :-(

Alan Knight wrote:

> I suspect the critical bit is the delay and forking the save. If
> you're running the script via command-line options like -doit, then
> that's running as part of the image start-up logic. You want to make
> sure that that startup logic has entirely finished before you save
> your image, and you don't really want all of that stuff on your stack
> (even just for the space it likely takes). And with things like
> closing windows that run asynchronously, you want to make sure they've
> entirely finished - you don't want your image saved with a window
> half-way through closing.
>
> We've thought about trying to provide something more convenient when
> saving, but it's rather tricky. You know that the minute you define
> something that says "do this at the end of the startup process" that
> you will immediately need to make something run afterwards. The
> forking is what we do internally in image builds.
>
Does this imply that I also need a delay inside my fork block?  That is,
the delay that Terry used would not permit the rest of the startup code
to run assuming it is all in the same thread as is executing my
-filein.  Putting a delay in the "forked save" would allow that startup
logic to finish.


David



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Alan Knight-2
At 01:44 PM 7/31/2008, C. David Shaffer wrote:
Alan Knight wrote:
I suspect the critical bit is the delay and forking the save. If you're running the script via command-line options like -doit, then that's running as part of the image start-up logic. You want to make sure that that startup logic has entirely finished before you save your image, and you don't really want all of that stuff on your stack (even just for the space it likely takes). And with things like closing windows that run asynchronously, you want to make sure they've entirely finished - you don't want your image saved with a window half-way through closing.

We've thought about trying to provide something more convenient when saving, but it's rather tricky. You know that the minute you define something that says "do this at the end of the startup process" that you will immediately need to make something run afterwards. The forking is what we do internally in image builds.
Does this imply that I also need a delay inside my fork block?  That is, the delay that Terry used would not permit the rest of the startup code to run assuming it is all in the same thread as is executing my -filein.  Putting a delay in the "forked save" would allow that startup logic to finish.


Yes. In fact, that's the only delay you really need. I guess I didn't look at the proposed code closely enough. But you just need to make sure that other stuff gets a chance to finish before you do the image save.

--
Alan Knight [|], Engineering Manager, Cincom Smalltalk

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Stefan Schmiedl
In reply to this post by cdavidshaffer
On Thu, 31 Jul 2008 13:01:57 -0400
"C. David Shaffer" <[hidden email]> wrote:

> I've been trying to automate building my VW7.6 images under Linux but
> I'm having a problem.  If my build script closes a window such as:
>
> Workbook allInstances do: [:each | each closeAndUnschedule].
> [ObjectMemory saveAs: 'myimage' thenQuit: true] fork.

This is what I have as final chunk the end of my load script:

!
[ Transcript clear.
Delay forSeconds: 3.
ObjectMemory globalGarbageCollect;
  globalCompactingGC.
Snapshot new saveAs: 'store' thenQuit: true.
] fork

> If I load a script like the above using "visual visualnc.im -filein
> script.st" then resulting image is unstable.  When it is running
> head-full I get errors like:
>
> X11 Error:
> Display:    0x0810e5a8
> Serial:    0x0000008a
> Major:    2
> Minor:    0
> XID:    0x0260006c
> BadWindow (invalid Window parameter)

I see those errors very rarely and have never had any problems
saving an image. I guess they might have something to do with
visible transient windows like tooltips or menus or so during
snapshotting.

s.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

cdavidshaffer
Stefan Schmiedl wrote:

>
> This is what I have as final chunk the end of my load script:
>
> !
> [ Transcript clear.
> Delay forSeconds: 3.
> ObjectMemory globalGarbageCollect;
>   globalCompactingGC.
> Snapshot new saveAs: 'store' thenQuit: true.
> ] fork
>  
Thanks.  I've added this to my script.
>  
>
> I see those errors very rarely and have never had any problems
> saving an image. I guess they might have something to do with
> visible transient windows like tooltips or menus or so during
> snapshotting.
>
>  
Oddly enough for me this is 100% reproducible using the simple example
that I gave.  Terry's suggestion of closing from the UI process fixes
the problem.  Adding the delays alone doesn't fix the problem (although
I understand the need for the delays).  I don't claim to understand the
X errors but I can chant the mantra "User interface interactions need to
be generated in (the correct) UI process".

David

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

cdavidshaffer
In reply to this post by Randy Coulman
Randy Coulman wrote:
> [snip]
>
> You could do pretty much any kind of homegrown scripting here with
> cron jobs or whatever.  CruiseControl is really just a wrapper around
> the various parts you have to set up yourself.  But, if you need
> something that watches for changes somewhere, launches your build for
> you, and reports the results, then CruiseControl is a decent option.
>
Ah, I see.  Thanks for the clarification.  I'm not allergic to other
tools but my application is so simple right now that the facilities
provided by CrusieControl are more trouble than they're worth.  I'm the
only developer and I just "cron" a nightly build and look at the output
in the morning.  If all goes well things will be more complicated some
day :-) then maybe I'll have a look.

David

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Terry Raymond
In reply to this post by Alan Knight-2

Generally speaking, I do not like to use Delay. I would prefer to change

the state of something or generate an event to show that everything is

done and I can now exit, or whatever.

 

Terry

===========================================================
Terry Raymond
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================


From: [hidden email] [mailto:[hidden email]] On Behalf Of Alan Knight
Sent: Thursday, July 31, 2008 1:46 PM
To: C. David Shaffer; [hidden email]
Subject: Re: [vwnc] Image building under Linux

 

At 01:44 PM 7/31/2008, C. David Shaffer wrote:

Alan Knight wrote:

I suspect the critical bit is the delay and forking the save. If you're running the script via command-line options like -doit, then that's running as part of the image start-up logic. You want to make sure that that startup logic has entirely finished before you save your image, and you don't really want all of that stuff on your stack (even just for the space it likely takes). And with things like closing windows that run asynchronously, you want to make sure they've entirely finished - you don't want your image saved with a window half-way through closing.

We've thought about trying to provide something more convenient when saving, but it's rather tricky. You know that the minute you define something that says "do this at the end of the startup process" that you will immediately need to make something run afterwards. The forking is what we do internally in image builds.

Does this imply that I also need a delay inside my fork block?  That is, the delay that Terry used would not permit the rest of the startup code to run assuming it is all in the same thread as is executing my -filein.  Putting a delay in the "forked save" would allow that startup logic to finish.



Yes. In fact, that's the only delay you really need. I guess I didn't look at the proposed code closely enough. But you just need to make sure that other stuff gets a chance to finish before you do the image save.


--
Alan Knight [|], Engineering Manager, Cincom Smalltalk

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Alan Knight-2
Generally speaking, I agree with you, but in this case it's rather tricky to do so without leaving other stacks around that you don't really want. A polling loop to see if some other known process has finished yet doesn't really seem like an improvement.

At 02:11 PM 7/31/2008, Terry Raymond wrote:
Generally speaking, I do not like to use Delay. I would prefer to change
the state of something or generate an event to show that everything is
done and I can now exit, or whatever.
 

Terry

===========================================================
Terry Raymond
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
< http://www.craftedsmalltalk.com>
===========================================================

From: [hidden email] [[hidden email]] On Behalf Of Alan Knight
Sent: Thursday, July 31, 2008 1:46 PM
To: C. David Shaffer; [hidden email]
Subject: Re: [vwnc] Image building under Linux
 
At 01:44 PM 7/31/2008, C. David Shaffer wrote:

Alan Knight wrote:

I suspect the critical bit is the delay and forking the save. If you're running the script via command-line options like -doit, then that's running as part of the image start-up logic. You want to make sure that that startup logic has entirely finished before you save your image, and you don't really want all of that stuff on your stack (even just for the space it likely takes). And with things like closing windows that run asynchronously, you want to make sure they've entirely finished - you don't want your image saved with a window half-way through closing.

We've thought about trying to provide something more convenient when saving, but it's rather tricky. You know that the minute you define something that says "do this at the end of the startup process" that you will immediately need to make something run afterwards. The forking is what we do internally in image builds.
Does this imply that I also need a delay inside my fork block?  That is, the delay that Terry used would not permit the rest of the startup code to run assuming it is all in the same thread as is executing my -filein.  Putting a delay in the "forked save" would allow that startup logic to finish.


Yes. In fact, that's the only delay you really need. I guess I didn't look at the proposed code closely enough. But you just need to make sure that other stuff gets a chance to finish before you do the image save.

--
Alan Knight [|], Engineering Manager, Cincom Smalltalk
[hidden email]
[hidden email]
http://www.cincom.com/smalltalk
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

--
Alan Knight [|], Engineering Manager, Cincom Smalltalk

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Reinout Heeck
In reply to this post by Alan Knight-2

On Jul 31, 2008, at 7:46 PM, Alan Knight wrote:

> At 01:44 PM 7/31/2008, C. David Shaffer wrote:
>> Alan Knight wrote:
>>> I suspect the critical bit is the delay and forking the save. If  
>>> you're running the script via command-line options like -doit,  
>>> then that's running as part of the image start-up logic. You want  
>>> to make sure that that startup logic has entirely finished before  
>>> you save your image, and you don't really want all of that stuff  
>>> on your stack (even just for the space it likely takes). And with  
>>> things like closing windows that run asynchronously, you want to  
>>> make sure they've entirely finished - you don't want your image  
>>> saved with a window half-way through closing.
>>>
>>> We've thought about trying to provide something more convenient  
>>> when saving, but it's rather tricky. You know that the minute you  
>>> define something that says "do this at the end of the startup  
>>> process" that you will immediately need to make something run  
>>> afterwards. The forking is what we do internally in image builds.
>> Does this imply that I also need a delay inside my fork block?  
>> That is, the delay that Terry used would not permit the rest of the  
>> startup code to run assuming it is all in the same thread as is  
>> executing my -filein.  Putting a delay in the "forked save" would  
>> allow that startup logic to finish.
>
>
> Yes. In fact, that's the only delay you really need. I guess I  
> didn't look at the proposed code closely enough. But you just need  
> to make sure that other stuff gets a chance to finish before you do  
> the image save.


At Soops we found this solution to be too brittle, so we wrapped the  
delay in a loop. On every iteration we walk the stack of the startup  
process to determine whether it has proceeded far enough, only if that  
is the case do we exit the loop and fire off the saving of the image.




HTH,

Reinout
-------
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Image building under Linux

Alan Knight-2
At 05:12 AM 8/1/2008, Reinout Heeck wrote:

On Jul 31, 2008, at 7:46 PM, Alan Knight wrote:

> At 01:44 PM 7/31/2008, C. David Shaffer wrote:
>> Alan Knight wrote:
>>> I suspect the critical bit is the delay and forking the save. If 
>>> you're running the script via command-line options like -doit, 
>>> then that's running as part of the image start-up logic. You want 
>>> to make sure that that startup logic has entirely finished before 
>>> you save your image, and you don't really want all of that stuff 
>>> on your stack (even just for the space it likely takes). And with 
>>> things like closing windows that run asynchronously, you want to 
>>> make sure they've entirely finished - you don't want your image 
>>> saved with a window half-way through closing.
>>>
>>> We've thought about trying to provide something more convenient 
>>> when saving, but it's rather tricky. You know that the minute you 
>>> define something that says "do this at the end of the startup 
>>> process" that you will immediately need to make something run 
>>> afterwards. The forking is what we do internally in image builds.
>> Does this imply that I also need a delay inside my fork block?  
>> That is, the delay that Terry used would not permit the rest of the 
>> startup code to run assuming it is all in the same thread as is 
>> executing my -filein.  Putting a delay in the "forked save" would 
>> allow that startup logic to finish.
>
>
> Yes. In fact, that's the only delay you really need. I guess I 
> didn't look at the proposed code closely enough. But you just need 
> to make sure that other stuff gets a chance to finish before you do 
> the image save.


At Soops we found this solution to be too brittle, so we wrapped the 
delay in a loop. On every iteration we walk the stack of the startup 
process to determine whether it has proceeded far enough, only if that 
is the case do we exit the loop and fire off the saving of the image.

Yes, I can see that being the case if your application is doing a lot of startup work, especially if it gets done in the UI process. For our purposes, a few seconds delay has been adequate.

Somebody had the suggestion of putting Subsystem allSubclasses - self as the prerequisites for a subsystem that you want to run last.  You'd probably still want to fork so that the startup process has a chance to finish, but that might actually work. Of course if two people try to do this, it will probably hang the image on startup.


--
Alan Knight [|], Engineering Manager, Cincom Smalltalk

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc