Product Maintenance & Support

I wonder if there is a generally accepted method for supporting deployed
(and possibly somewhat outdated) product releases with patches/updates
etc. while at the same time the product development is going on.

I'm a little worried about letting Store do all the versioning
continously in a single repository (i.e. set all package versions and
blessings to a meaningful "milestone" for each public release). The
reason is my products aren't mapped to bundles 1:1, but instead products
are different combinations of bundles and packages, depending on their
selected features.

I'm also worried about losing control over numerous version branches. I
guess it could be problematic to restore the exact state of a product
released months ago in order to work on bug fixes and patches that were
discovered later in the life cicle of a product. It simply doesn't fee
safe enough (my standard requirement for "safety" is: It must not do any
harm, even if I was drunk and it was 3 o'clock  AM ;-)

Hence, I'm planning to create a separate "archive" repository for each
product, exactly keeping the source of the released product. Patches and
updates could be developed in that image (or copied there from the
current development image) and published incrementally to the old
repository, while the current bleeding edge development is safely taking
place elsewhere.

Is that a valid approach, or did I miss something that could be even
safer and simpler?

Andre

Having all releases and patches of your product in one single Store
database has clear advantages, so I would recommend following this
path instead. You will be able to compare versions, and easily move
changes between releases. One way of doing this is having a bundle
version all your components when you make a new build.

I have made a small add-on to Store which lets you use bundles as a
configuration tool. More information can be found here:
http://www.cincomsmalltalk.com/userblogs/runarj/blogView?showComments=true&entry=3321184256

I think a good practice to follow is to depend solely on Store for
code management. Do not depend on a magic image with needed code
preloaded, but make a script that can load your entire project from a
base Cincom image:
http://www.cincomsmalltalk.com/userblogs/runarj/blogView?showComments=true&entry=3319375571
If you do this, all you need to backup is your (simple) script and the
Store database schema.

Runar Jordahl

Hello Andre,

> I'm a little worried about letting Store do all the versioning
> continously in a single repository (i.e. set all package versions and
> blessings to a meaningful "milestone" for each public release). The
> reason is my products aren't mapped to bundles 1:1, but instead products
> are different combinations of bundles and packages, depending on their
> selected features.

In the project where I'm currently working, we have three different
products with an overlapping code base. The project shares many
components with several other teams, and all of them use some packages
and bundles from a set of reusable components. We keep all this in one
central Store repository.

This works because we make sure that every end product has exactly one
top-level bundle which enables one-click loading of everything needed
into a common base image, which is essentially a virgin visual.im with
very few additions. The additions include tools which evaluate certain
commands from the command line, thus enabling automated build and test
runs controlled by batch files (including runtime image creation).

Once you have one-click-loading with appropriately configured bundles,
you can be sure that you can reproduce an image whenever you need it. I
would recommend to store configuration files etc. in Store as well (even
though tool support is still missing). I've seen more than one project
where it was not possible to do a one-click-load. People used to "know"
which Envy configuration maps or Store bundles had to be loaded into
which image (e.g. "always use the latest version names matching the
first three letters of the release name, with blessing level of at least
xyz"). Sooner or later, something went wrong because they accidentally
selected a wrong version during their manual build processes. The lesson
learned was: automate whatever can be automated, and make sure that
there is exactly one thing (in the case of Store: one bundle version)
which completely describes what's in the product.

BTW, make sure that *all* the code is in Store, never load anything from
parcels. Which parcels are loaded (especially the parcels from the
VisualWorks distribution) is influenced by environment variables etc.
Never rely on this, you'll end up loading a wrong parcel sooner or
later. Always publish all the code into Store.

I also think that it is a very bad idea to use a "current development
image" for longer than a few days. We rebuild images from scratch on a
daily base, just to make sure that a) the code is really loadable from
Store, b) everybody is working on the latest code base, c) we don't
accidentally write code which works only if a certain test case has a
failure leaving something behind in an incomplete tearDown method (e.g.
in a shared variable). We have branches for maintaining older versions
besides the current development stream, and using a single Store
repository makes it easy to integrate bug fixes from a maintenance
stream into the main stream.

>
> Is that a valid approach, or did I miss something that could be even
> safer and simpler?

I wouldn't do it. I am afraid that it makes it even more complicated to
stay in control. How would you ensure that a bug fix in a common
component really makes it into every archive repository for every
product which must be patched? How do you make sure that bug fixes from
a maintenance stream make it into the main stream? I'd rather rely on
merging a patch into all products using a single repository, which
amounts to fixing the broken package(s) and updating the prerequisite
versions in the top-level bundles. It's also a good idea to write your
code in a way which makes it possible to load the code of all your
products into a single image, and ideally be able to actually run every
product's code in this image. This makes it easier to align the
configurations. You still have to test the products using single-product
images, of course.

To make a long story short:
> my products aren't mapped to bundles 1:1
IMO, that's the problem - add a one-click-loadable bundle for each of
your products, and you are done.

I hope this helps,
Joachim Geidel

Runar Jordahl wrote:
> Having all releases and patches of your product in one single Store
> database has clear advantages, so I would recommend following this
> path instead. You will be able to compare versions, and easily move
> changes between releases. One way of doing this is having a bundle
> version all your components when you make a new build.

Is that possible with overlapping bundles? I mean, can I do the following:

ProductBundle 1 = [ Bundle A = [ Package X, Package Y], Package D ]
ProductBundle 2 = [ Bundle B = [ Package D, Bundle A, Package Z ],
Package Y ]

Or simpler: Can a pundle be a member of many bundles at the same time
while each parent bundle manages and ensures a different version
(mainetance stream) of it? That'd be cool, but I'm still afraid this
would overstress my concentration - too many cross-linked trees to keep
track of.

I must admit that I didn't yet fully grasp all the sophisticated things
possible with diverging development streams in Store. That's obviously
rocket science. I also can't see how prerequisites can be versioned in
Store without manually setting them. That'd be a hard time checking all
prerequisite versions by hand.

> I think a good practice to follow is to depend solely on Store for
> code management. Do not depend on a magic image with needed code
> preloaded, but make a script that can load your entire project from a
> base Cincom image:

Agreed. I already use special "packaging" bundles for product deployment
that load all required sources. Anyway, most dependencies are resolved
by prerequisites which can not be versioned (as far as I understood).

Andre

Joachim Geidel wrote:

This works because we make sure that every end product has exactly one
top-level bundle which enables one-click loading of everything needed
into a common base image, which is essentially a virgin visual.im with
very few additions. The additions include tools which evaluate certain
commands from the command line, thus enabling automated build and test
runs controlled by batch files (including runtime image creation).
  

That's quite similar to what I'm doing at the moment. The only thing I did not yet dare is to rely on the Store versioning when it comes to maintaining/patching older bundles/packages in a separate version stream (mainly because of the prerequisites not being versioned automatically).

Once you have one-click-loading with appropriately configured bundles,
you can be sure that you can reproduce an image whenever you need it. 

Won't this load the most current prerequisites by default? What if the prerequiste packages have changed since then?

BTW, make sure that *all* the code is in Store, never load anything from
parcels. Which parcels are loaded (especially the parcels from the
VisualWorks distribution) is influenced by environment variables etc.
Never rely on this, you'll end up loading a wrong parcel sooner or
later. Always publish all the code into Store.
  

Ok, I didn't yet publish base packages like networking and such.

I also think that it is a very bad idea to use a "current development
image" for longer than a few days. 

I'm the only developer so far, so this is not a problem for me.

We have branches for maintaining older versions
besides the current development stream, and using a single Store
repository makes it easy to integrate bug fixes from a maintenance
stream into the main stream.
  

I see I have to try this out safely in a test respository.

I'd rather rely on
merging a patch into all products using a single repository, which
amounts to fixing the broken package(s) and updating the prerequisite
versions in the top-level bundles. 

It's also a good idea to write your
code in a way which makes it possible to load the code of all your
products into a single image, and ideally be able to actually run every
product's code in this image. This makes it easier to align the
configurations. You still have to test the products using single-product
images, of course.
  

That's what I'm doing. Most products are derivates of a common code base and everything is in the development image, so heavy refactorings correctly affect all existing code.

To make a long story short:
  

my products aren't mapped to bundles 1:1
    

IMO, that's the problem - add a one-click-loadable bundle for each of
your products, and you are done.

  

There's still the prerequisites problem. I'd really like Store to save the exact versions of all prerequisites on every publication automatically (like bundles do for their packages).


Thanks Runar and Joachim, I think I'll have plenty to try out.

Andre

Andre Schnoor schrieb am 07.05.2006 18:27:
>> Once you have one-click-loading with appropriately configured bundles,
>> you can be sure that you can reproduce an image whenever you need it.
>
> Won't this load the most current prerequisites by default? What if the
> prerequiste packages have changed since then?

Have a look at the Properties tab for Bundles, select Deployment
Prerequisites or Development Prerequisites. There is an input field
below the list of prerequisites. When you select one of the
prerequisites, you can enter the version name which is to be loaded. (We
have modified this to show a combo box of the published versions of the
selected prerequisite, making it easier to enter a valid version name.)
The prerequisite configuration, including versions, is versioned with
the bundle. Whenever you load a bundle which has version information for
all of its prerequisites (and their prerequisites, recursively), it will
load exactly what you want.

During development, it is a bit cumbersome to have versioned
prerequisites, as you start versioning your bundles every time a package
 in the bundle or a package in a prerequisite of a prerequisite of... a
bundle is changed. That's a weakness of Store. Therefore, we are using a
tool which lets us load the most recently published versions of packages
in a bundle, even if there is no newer bundle version. This avoids
having to version the bundles too often. Runar Jordahl's tool is very
similar to ours.

>> I also think that it is a very bad idea to use a "current development
>> image" for longer than a few days.
>
> I'm the only developer so far, so this is not a problem for me.

It will be a problem when the first magically uninitialized shared
variable bites you. ;-)

>> We have branches for maintaining older versions
>> besides the current development stream, and using a single Store
>> repository makes it easy to integrate bug fixes from a maintenance
>> stream into the main stream.
>>  
> I see I have to try this out safely in a test respository.

The Merge Tool works quite well to support this kind of work. BTW, we
have a refactoring browser tool which shows a graphical version graph of
a selected pundle, class, or method (based on HotDraw), and allows
loading, merging, reconciling, and versioning from context menus. Really
cool stuff which would be very useful in the VisualWorks base, but as
the "we" in this text is not my own company, I can't just publish it
into the Public Repository... :-(

> There's still the prerequisites problem. I'd really like Store to save
> the exact versions of all prerequisites on every publication
> automatically (like bundles do for their packages).

In our browser there's a button which updates all the prerequisite
versions from what's loaded in the image. Also, the prerequisite list
prints loaded versions in bold and versions which differ from the loaded
ones non-bold. But again, that's not the base image...

Joachim

> Is that possible with overlapping bundles? I mean, can I do the following:
>
> ProductBundle 1 = [ Bundle A = [ Package X, Package Y], Package D ]
> ProductBundle 2 = [ Bundle B = [ Package D, Bundle A, Package Z ],
> Package Y ]

In short, the answer is "no". My tool uses the version of the parent
bundle to determine which version of a package should be loaded. If
for example "Bundle A" is version 1.3.131, this specifies that the
version of the containing packages to look for is the most recent with
version 1.3.*. This might not line up with the version you are on at
"Bundle B".

I will think through my tool again. Maybe instead of using the *bundle
version* to determine the version of the packages to load, it should
use the referred *package version*. It will see if a newer version of
the package is published for the same stream. Consider this example:

Bundle A version 1.3.131
        Package A 2.0.10

Now, if a 2.0.11 (or 2.0.12 etc.) version of "Package A" exists, it
will be loaded instead of 2.0.10. But it will not load a 2.1.0, or
3.0.0 versions, as these are not in the same stream.

You can read how my tool currently works at
http://www.cincomsmalltalk.com/publicRepository/Epigent%20Store%20Configuration%20Support.html

Follow my blog for updates to the tool.

Runar Jordahl