[9.2] Features over Conf Maps - how to work in the future?

Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].

You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?

So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:

In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.

Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.

Joachim

Little addition:

what I mean with dev code and runtime code is nicely demonstrated with z.ST: Server Smalltalk (SST) - Mail (IMAP/SMTP).

It will load for example:

Loading the configuration map Server Smalltalk - Examples V 8.0.1  [134]...
Loaded Server Smalltalk - Examples V 8.0.1  [134].

To me it seems we will have to analyse the features and manually extract all the required maps from that feature which we also want for our runtime image, in order to provide our own "runtime MAIL stuff" map. Even in cases where this is really just "leave the samples out". I do accept this is feasible for things where we do "complicated" stuff, but I am not sure this should be required for obvious things like examples or feaures with Browser extensions and such...

Joachim

Am Dienstag, 17. Dezember 2019 09:45:03 UTC+1 schrieb Joachim Tuchel:

Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].

You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?

So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:

In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.

Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.

Joachim

Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].

You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?

So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:

In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.

Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.

Joachim

Hi Joachim and Everyone,

I have been too busy lately with personal things to check out version 9.2 so I don't know if there are changes that make this situation worse or it is a tightening up of existing checks.  That said, I would like to add a circumstance to Joachim's.  From time to time I have to apply a fix to base code and getting that fix included when loading from config maps requires changing one and sometimes many config maps.  This can rapidly become problematic.  I think it would be helpful to have an option that allows the loading of the "latest" version of all applications.  I have not thought this through, so it may not be doable or may lead to other problems.  Any thoughts?

Lou


On Tuesday, December 17, 2019 at 3:45:03 AM UTC-5, Joachim Tuchel wrote:

Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].

You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?

So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:

In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.

Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.

Joachim

Little addition:

what I mean with dev code and runtime code is nicely demonstrated with z.ST: Server Smalltalk (SST) - Mail (IMAP/SMTP).

It will load for example:

Loading the configuration map Server Smalltalk - Examples V 8.0.1  [134]...
Loaded Server Smalltalk - Examples V 8.0.1  [134].

To me it seems we will have to analyse the features and manually extract all the required maps from that feature which we also want for our runtime image, in order to provide our own "runtime MAIL stuff" map. Even in cases where this is really just "leave the samples out". I do accept this is feasible for things where we do "complicated" stuff, but I am not sure this should be required for obvious things like examples or feaures with Browser extensions and such...

Joachim

Am Dienstag, 17. Dezember 2019 09:45:03 UTC+1 schrieb Joachim Tuchel:

Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].

You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?

So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:

In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.

Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.

Joachim

Hi Joachim,

I’ll start with our definition of a Feature …  a set of code defined by a configuration map relying on one or more applications and/or configuration maps which must load automatically and cleanly into an IBMST or a full VA Smalltalk image.  This includes loading the appropriate version of a Feature based on the version of VA Smalltalk being used.  As some others have noted, many people , including me, work out of a single manager that we load all code into.  If you have 8.6.3, 9.0, 9.1, and 9.2 code all in your manager, the Feature system should load the appropriate version of the Feature that matches the version of VA Smalltalk that is being used.  

This leads to the first important reason for using Features … making it as easy and as error-free as possible to load cohesive sets of code into the image without having to know what what specific config map would load it, as well as which version of that map to use for the current image.   It is not always reasonable, or even possible, to just load the latest version of a feature map (z*)  into an image in cases where multiple versions of VA Smalltalk are supported by a single manager.

The second important reason has to with the processes involved with building the product.  As Richard noted “…one of the greatest strengths of ENVY is the ability to precisely specify the prerequisites chain of code. Admittedly, it is also a pain to maintain.”.  As you are probably aware, we create new versions of all Feature (z*. maps) every release.  Since any particular build could be a release, we actually create them every time we build the product, generally once a week.  We take advantage of the Feature system to automatically build them from the latest code available, avoiding the error-prone and tedious process of chasing down all prerequisites to a particular change by hand.  I guarantee we would make a ton or mistakes if we did it by hand. 

I think it is safe to say that the Feature system will continue to be used for these purposes. For those that are not aware, it is acceptable to use the underlying z*. maps as required maps to your own application maps to make it easier to load your own config maps as well as to tie a particular version of your code to a particular version of a feature.  This is a way to load a Feature without having to do so using the  Load/Unload features menu item.

"I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?"

I find this idea interesting.  A little background … the ‘normal’ config maps  referenced by any of our Feature  (z*) maps  are something we call Feature Support Maps.  These maps look like normal config maps, and actually, are normal config maps, but there is one particular requirement from a Feature perspective which must be adhered to … they can not define any required maps or they break the build and Feature systems.  So while they look like something you could load ‘with required maps’ successfully, chances are,  you can’t.  I don’t think renaming all of these Feature Support Maps is a reasonable solution .   We would have to really understand the ramifications of doing so.  The tracking of the history  of config maps would be effected and potentially customers relying on their current names are a couple of problem areas off the top.  Much thought would have to go into making a change such as this.

I can see the usefulness of identifying the Feature Support Maps as I have fallen victim to trying to load one of the maps myself. Off the top, perhaps using a special decoration next to the map name could be helpful? 

Hopefully you will find this information enlightening.

Bob

On Tuesday, December 17, 2019 at 3:45:03 AM UTC-5, Joachim Tuchel wrote:

Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].

You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?

So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:

In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.

Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.

Joachim