Automatically building Application Images via Jenkins/CLI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Automatically building Application Images via Jenkins/CLI

David Gregory
You will find attached the system we use for automatically building smalltalk images for our applications from Jenkins as well as manually from the command line (if our developers want an ad-hoc image for testing).  Unfortunately google groups prevents posting windows batch scripts so they are posted with a .txt extension added to get them accepted.

I'm posting this to help others that may also want to automatically build their applications images and thus to get community co-operation in improving this system so we can all benefit.  What I have here is by no means perfect.  I'll list the known issues towards the end of the post.

Our organization had an issue in that our applications were only being reliably built by one of our developers using a magic master image setup 'just right'.  We wanted to fix this so that image creation was repeatable and automated and thus anyone could do it and do it quickly.  Attached is the result of this effort.  I'll note that the scripts to create the image that are attached are only part of the solution.  This system relies on you having config maps containing everything you need to create your application.  If you don't have these already, I cannot really help too much.  It took us a bit of effort to get our config maps containing all the relevant dependencies, but that is a per-requisite of this solution.  Basically, you need to be able to build your applications manually from a clean 'new' image to then automate the process.

The attached solution assumes that Windows is the primary build environment and that UNIX is the XD environment.  In our case we have 3 windows images and 8 UNIX images to build for each major release and build all of them from the windows version of VA Smalltalk.  If Linux is your primary build environment, you would only have to adapt the batch scripts to shell scripts and change the UNIX.txt file (which defines the XD image properties) to be a different platform.

This build system also assumes VAST 9.1 (as that is what we use right now - we will be upgrading to 9.2 on the next release cycle for our application) although the only thing that makes that assumption is the paths used in the batch files.  I believe it will work in any version where the PostStartUp capability in image startup scripts exists (the doco at https://www.instantiations.com/docs/91/sg/wwhelp/wwhimpl/common/html/wwhelp.htm#href=stug515.html&single=true says this is from 8.6.2).  You just have to change the paths to nodialog.exe and newimage\abt.icx to match your installation.

Pre-requisities
Our solution assumes you have:
  1. A config map that has all the features needed by your application along with all the packaging instructions classes for your application.  If you need to build XD images, you need this config map to include all the XD packaging support.  In our case this map also includes some of the VAST Goodies and 3rd party applications we use although that is a personal choice to cut down the time it takes to create the applications images.
  2. If you need to build an XD image, a config map containing the features needed by the XD image for your application
  3. A config map for your application.  The attached solution assumes that you name the versions of this config map with a version number prefixed by a capital V somewhere in the version name.
High Level Flow
The solution involves to basic steps:
  1. Build an image that is used as the basis image from which the application image is built.  If you build images manually, this takes the place of the Master image you always load to create a release image.  In our case, we have a single 'build' image from which we build the 11 applications images.  The 'build' image is the one that the config map containing all the features if put into.  We only re-build this image if we change the config map used by this image - which for us is only really at major release boundaries.
  2. Using this build image, then load the application config map and run the packager for the application
Note that the way things are written right now, the packaging instructions for the application must exist in the 'build' image.  If you try this out and it cannot find the packaging instruction class, they I suspect you have it in the application config map rather than the 'build' config map.

Components
There are very few components to our solution.  You will need to supply your own abt.ini with the reference to your library and any parameters you need.  Otherwise in the attached zip file you will find
  • abt.cnf: The initial image startup script used to create the build image.  This takes the following command line options
    • -user=yourusername that will own the image.  Must be a user in the library.
    • -map=MainConfigMap that points to the features you need in the main build image
    • Optionally -xdmap=XDConfigMap if you need an XD image listing the config map for the features to put into the XD Image
  • UNIX.txt: A text file used by the XD image creator that defines the properties of the XD image.  Note that while this file has a place to define InstalledFeatures, this list is ignored by the XD subsystem.  Instead, you will find a bit of code in abt.cnf that does 'xdImage installFeatures:' that will load features not in your config map
  • build.cnf: The image startup script used by the builder image to create your final application images.  This takes the following command line options
    • -map=ApplicationConfigMap which is your application config map
    • -pkg=PkgInstructionName which is the name of the packaging instruction to use
    • Optionally -ver=<version> which is the version number of the application config map to load.  If no version is supplied, the latest edition of the config map (versioned or not) is built.  Else the latest versioned config map which has the string V<version> in it is built.
  • create_builder.cmd: A sample windows batch file to make it quick to create new builder images.  You will have to edit this for your situation.
  • build_image.cmd: A windows batch file to use the build image to create application images.  You might have to edit this for your situation.
We use a 3 segment version number for our config maps.  V<major>.<minor>.<patch>.  If you use a different system you will need to change the edition detect: logic in build.cnf

The build.cnf looks at the type of packaging instruction supplied on the command line.  If it is an XD packaging instruction, the config map and version is loaded into the development image and XD image and the XD packager is invoked.

Running
To create a build.icx image used to build application images you can put the attached files into a directory and edit create_builder script or run

COPY /Y "C:\Program Files (x86)\Instantiations\VA Smalltalk\9.1x86\newimage\abt.icx"
"C:\Program Files (x86)\Instantiations\VA Smalltalk\9.1x86\nodialog.exe" -iabt.icx -ini:abt.ini -loutput.txt -user=yourusername "-map=MainConfigMap" "-xdmap=XDConfigMap"
TYPE output.txt

To create an application image, you can then use this build.icx and run build_image.cmd which basically does

"C:\Program Files (x86)\Instantiations\VA Smalltalk\9.1x86\nodialog.exe" -ibuild.icx -ini:abt.ini -loutput.txt "-map=ApplicationConfigMap" "-pkg=PackagingInstructionName"-ver=X.X
TYPE output.txt

Outputs
You will (hopefully) get all the outputs configured in your packaging instruction.  In addition, we generate an HTML snippet containing the description of the config map edition being built.  We use this as part of our release note generator for our releases.

Jenkins tips and tricks
Jenkins cannot consume the -lCON output generated by abt,exe/nodialog.exe.  It assumes that the output it will grab is generated directly by the command that is being executed.  To get around this, I tell abt.exe to write to a file (output.txt) and once the operation is completed I output this file to jenkins.  This means that the console output in Jenkins is not updated while an image is being built - only at the end.

The second tip I can give is that it is not possible (well I couldn't find a way) to run the packager in a purely headless mode.  This means it requires a desktop to work.  Jenkins only runs a slave build on a desktop if the agent is run from a desktop - and not as a windows service.  While the build will still run if your run it as a build job on a windows service jenkins slave, you will not be able to resolve any issues with the build job (as you will have no UI).

We have one build job to create the build.icx which is then archived as an artifact by Jenkins.  The application image build jobs then Copy this artifact as part of the application image build jobs.  The output images are then fed to the build jobs we use to package our total application (eg with the VA smalltalk runtime DLLs and all our support images and stuff).

Issues
  1. I've not found a way to guarantee that the process 'exits' on failure.  Any walkback inducing error will launch the interactive debugger.  I've tried to intercept most prompts and errors such that Jenkins is told of the error, but definitely not all.  If nodialog.exe does not 'exit' then Jenkins waits forever for it to finish.  The only way to 'fix' this is to login to the desktop running the jenkins slave and answer whatever prompt is being displayed in the VA Smalltalk UI.
  2. I've not found a way to intercept the XD transcript so that it appears in the jenkins job output.  There appears to be lots of code inside the XD system that assumes the XD transcript is a CwShell style window.  So right now, you cannot see progress of the XD portion of the build.  If anyone has a tip or trick to fix this, I'd be interested in it.
  3. We've not yet added in the running of the SUnit tests as part of this.  Right now we run this sort of testing prior to versioning the config map for release.  We'll be looking at adding a build job stage to run the tests in the near future.  This first 'cut'
Feel free to post questions about this solution and I'll try to help, but if you have improvements, I'd be most interested.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/641e2778-2b57-453a-b1ae-4f8d84093faa%40googlegroups.com.

abt.cnf (5K) Download Attachment
build.cnf (8K) Download Attachment
UNIX.txt (267 bytes) Download Attachment
build_image.bat.txt (343 bytes) Download Attachment
create_builder.bat.txt (390 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Automatically building Application Images via Jenkins/CLI

jtuchel
David,


three thumbs up!!!

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/f082b525-6b67-4185-8605-12d67af881a1%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Automatically building Application Images via Jenkins/CLI

Seth Berman
What a great description!
Thanks for taking the time to present this David...many of us are going to find this very useful.

- Seth

On Monday, January 13, 2020 at 8:29:17 AM UTC-5, Joachim Tuchel wrote:
David,


three thumbs up!!!

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/a0f075de-dd9d-4800-a286-dcc728cda465%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Automatically building Application Images via Jenkins/CLI

Mariano Martinez Peck-2
In reply to this post by David Gregory
Greetings David,

What an impressive post! Thanks a lot not only for sharing your solution but also for writing that detailed documentation. 

Honestly, I have no questions. Everything was clear to me. Of course, I may have some when I try it for real. In 9.2 we are shipping a Seaside XD example (with packaging instructions, map with everything, etc). It should be simple enough I think to try all your code with this Seaside example. The only thing I would have to update is the map organization a bit and the versioning of the maps. What do you think about putting this into github (with open source license) whenever I can start with it? That way we can have your solution plus a working example.

I did read the "Issues" part and I will pay special attention to them as soon as I can try this out. I will obviously let you know of everything I found. 

Here at Instantiations we thought really well of this post and we think this can help a lot of folks in the software community. That's why I just tweet about it: https://twitter.com/MartinezPeck/status/1217074661604896768 (previous ACK from David)

Best regards, 

Mariano


On Mon, Jan 13, 2020 at 3:24 AM David Gregory <[hidden email]> wrote:
You will find attached the system we use for automatically building smalltalk images for our applications from Jenkins as well as manually from the command line (if our developers want an ad-hoc image for testing).  Unfortunately google groups prevents posting windows batch scripts so they are posted with a .txt extension added to get them accepted.

I'm posting this to help others that may also want to automatically build their applications images and thus to get community co-operation in improving this system so we can all benefit.  What I have here is by no means perfect.  I'll list the known issues towards the end of the post.

Our organization had an issue in that our applications were only being reliably built by one of our developers using a magic master image setup 'just right'.  We wanted to fix this so that image creation was repeatable and automated and thus anyone could do it and do it quickly.  Attached is the result of this effort.  I'll note that the scripts to create the image that are attached are only part of the solution.  This system relies on you having config maps containing everything you need to create your application.  If you don't have these already, I cannot really help too much.  It took us a bit of effort to get our config maps containing all the relevant dependencies, but that is a per-requisite of this solution.  Basically, you need to be able to build your applications manually from a clean 'new' image to then automate the process.

The attached solution assumes that Windows is the primary build environment and that UNIX is the XD environment.  In our case we have 3 windows images and 8 UNIX images to build for each major release and build all of them from the windows version of VA Smalltalk.  If Linux is your primary build environment, you would only have to adapt the batch scripts to shell scripts and change the UNIX.txt file (which defines the XD image properties) to be a different platform.

This build system also assumes VAST 9.1 (as that is what we use right now - we will be upgrading to 9.2 on the next release cycle for our application) although the only thing that makes that assumption is the paths used in the batch files.  I believe it will work in any version where the PostStartUp capability in image startup scripts exists (the doco at https://www.instantiations.com/docs/91/sg/wwhelp/wwhimpl/common/html/wwhelp.htm#href=stug515.html&single=true says this is from 8.6.2).  You just have to change the paths to nodialog.exe and newimage\abt.icx to match your installation.

Pre-requisities
Our solution assumes you have:
  1. A config map that has all the features needed by your application along with all the packaging instructions classes for your application.  If you need to build XD images, you need this config map to include all the XD packaging support.  In our case this map also includes some of the VAST Goodies and 3rd party applications we use although that is a personal choice to cut down the time it takes to create the applications images.
  2. If you need to build an XD image, a config map containing the features needed by the XD image for your application
  3. A config map for your application.  The attached solution assumes that you name the versions of this config map with a version number prefixed by a capital V somewhere in the version name.
High Level Flow
The solution involves to basic steps:
  1. Build an image that is used as the basis image from which the application image is built.  If you build images manually, this takes the place of the Master image you always load to create a release image.  In our case, we have a single 'build' image from which we build the 11 applications images.  The 'build' image is the one that the config map containing all the features if put into.  We only re-build this image if we change the config map used by this image - which for us is only really at major release boundaries.
  2. Using this build image, then load the application config map and run the packager for the application
Note that the way things are written right now, the packaging instructions for the application must exist in the 'build' image.  If you try this out and it cannot find the packaging instruction class, they I suspect you have it in the application config map rather than the 'build' config map.

Components
There are very few components to our solution.  You will need to supply your own abt.ini with the reference to your library and any parameters you need.  Otherwise in the attached zip file you will find
  • abt.cnf: The initial image startup script used to create the build image.  This takes the following command line options
    • -user=yourusername that will own the image.  Must be a user in the library.
    • -map=MainConfigMap that points to the features you need in the main build image
    • Optionally -xdmap=XDConfigMap if you need an XD image listing the config map for the features to put into the XD Image
  • UNIX.txt: A text file used by the XD image creator that defines the properties of the XD image.  Note that while this file has a place to define InstalledFeatures, this list is ignored by the XD subsystem.  Instead, you will find a bit of code in abt.cnf that does 'xdImage installFeatures:' that will load features not in your config map
  • build.cnf: The image startup script used by the builder image to create your final application images.  This takes the following command line options
    • -map=ApplicationConfigMap which is your application config map
    • -pkg=PkgInstructionName which is the name of the packaging instruction to use
    • Optionally -ver=<version> which is the version number of the application config map to load.  If no version is supplied, the latest edition of the config map (versioned or not) is built.  Else the latest versioned config map which has the string V<version> in it is built.
  • create_builder.cmd: A sample windows batch file to make it quick to create new builder images.  You will have to edit this for your situation.
  • build_image.cmd: A windows batch file to use the build image to create application images.  You might have to edit this for your situation.
We use a 3 segment version number for our config maps.  V<major>.<minor>.<patch>.  If you use a different system you will need to change the edition detect: logic in build.cnf

The build.cnf looks at the type of packaging instruction supplied on the command line.  If it is an XD packaging instruction, the config map and version is loaded into the development image and XD image and the XD packager is invoked.

Running
To create a build.icx image used to build application images you can put the attached files into a directory and edit create_builder script or run

COPY /Y "C:\Program Files (x86)\Instantiations\VA Smalltalk\9.1x86\newimage\abt.icx"
"C:\Program Files (x86)\Instantiations\VA Smalltalk\9.1x86\nodialog.exe" -iabt.icx -ini:abt.ini -loutput.txt -user=yourusername "-map=MainConfigMap" "-xdmap=XDConfigMap"
TYPE output.txt

To create an application image, you can then use this build.icx and run build_image.cmd which basically does

"C:\Program Files (x86)\Instantiations\VA Smalltalk\9.1x86\nodialog.exe" -ibuild.icx -ini:abt.ini -loutput.txt "-map=ApplicationConfigMap" "-pkg=PackagingInstructionName"-ver=X.X
TYPE output.txt

Outputs
You will (hopefully) get all the outputs configured in your packaging instruction.  In addition, we generate an HTML snippet containing the description of the config map edition being built.  We use this as part of our release note generator for our releases.

Jenkins tips and tricks
Jenkins cannot consume the -lCON output generated by abt,exe/nodialog.exe.  It assumes that the output it will grab is generated directly by the command that is being executed.  To get around this, I tell abt.exe to write to a file (output.txt) and once the operation is completed I output this file to jenkins.  This means that the console output in Jenkins is not updated while an image is being built - only at the end.

The second tip I can give is that it is not possible (well I couldn't find a way) to run the packager in a purely headless mode.  This means it requires a desktop to work.  Jenkins only runs a slave build on a desktop if the agent is run from a desktop - and not as a windows service.  While the build will still run if your run it as a build job on a windows service jenkins slave, you will not be able to resolve any issues with the build job (as you will have no UI).

We have one build job to create the build.icx which is then archived as an artifact by Jenkins.  The application image build jobs then Copy this artifact as part of the application image build jobs.  The output images are then fed to the build jobs we use to package our total application (eg with the VA smalltalk runtime DLLs and all our support images and stuff).

Issues
  1. I've not found a way to guarantee that the process 'exits' on failure.  Any walkback inducing error will launch the interactive debugger.  I've tried to intercept most prompts and errors such that Jenkins is told of the error, but definitely not all.  If nodialog.exe does not 'exit' then Jenkins waits forever for it to finish.  The only way to 'fix' this is to login to the desktop running the jenkins slave and answer whatever prompt is being displayed in the VA Smalltalk UI.
  2. I've not found a way to intercept the XD transcript so that it appears in the jenkins job output.  There appears to be lots of code inside the XD system that assumes the XD transcript is a CwShell style window.  So right now, you cannot see progress of the XD portion of the build.  If anyone has a tip or trick to fix this, I'd be interested in it.
  3. We've not yet added in the running of the SUnit tests as part of this.  Right now we run this sort of testing prior to versioning the config map for release.  We'll be looking at adding a build job stage to run the tests in the near future.  This first 'cut'
Feel free to post questions about this solution and I'll try to help, but if you have improvements, I'd be most interested.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/641e2778-2b57-453a-b1ae-4f8d84093faa%40googlegroups.com.


--
Mariano Martinez Peck
Software Engineer, Instantiations Inc.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/CAOUkibFjUn7VcjkvU73FKmQdFy6XAcTV0J%2BT0e9LVxQsaacURw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Automatically building Application Images via Jenkins/CLI

bonndias-3
In reply to this post by David Gregory
Cool!!!

Wasn't aware of the changes to abt.cnf in VA 8.62. Your code gives me the chance to rework our own solution, which relies on a build image doing the packaging job quite similar to yours on startup. Additionally our packaging process adds a line to the description field of the top ConfigMap.

Looks like this:
<<<< 
    $Packaging Entry - Do not Change this line$
    AutoPackagingNr: 7425    TimeStamp: 2020-01-14-10.57.19.311000    User: #XYZ
    AutoPackagingNr: 7424    TimeStamp: 2020-01-13-12.47.46.668000    User: #XYZ
>>>>


Thanks a lot

Joa

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/454228f8-2ec8-4639-ba7d-278b2d8eb685%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Automatically building Application Images via Jenkins/CLI

Dominik Jung
Hello David,

thank you very much for sharing your findings and even the code of your solution. We had the same solution with the same issues for our automatic build process, with a magic master image setup "just right". With your example, I was able to automate the creation of that master image, and even more important, setup the XD images for automatic packaging of UNIX images. You really helped us a lot!

Greetings from Germany,
Dominik

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/1e95d37e-f95d-4fb9-9bea-54378b10db2c%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Automatically building Application Images via Jenkins/CLI

David Gregory
Dominik,

Very glad it was useful.  We've also found that the automation is now allowing us to do 'nightly' builds of the application images (technically we are only doing it weekly as we don't have the team size to consume a new release each day).  Jenkins builds the open application config map into an image so we can find packaging problems faster and start functional testing earlier.

If you have improvements or found ways to get things like the XD transcript output captured, please post back your updates or code snippets.

David

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/cebb737b-81dd-447b-a901-4ade1600539e%40googlegroups.com.