[9.2] Features over Conf Maps - how to work in the future?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[9.2] Features over Conf Maps - how to work in the future?

jtuchel
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/b973cd07-7cea-4095-931c-94262386e14b%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

jtuchel
Little addition:

what I mean with dev code and runtime code is nicely demonstrated with z.ST: Server Smalltalk (SST) - Mail (IMAP/SMTP).

It will load for example:

Loading the configuration map Server Smalltalk - Examples V 8.0.1  [134]...
Loaded Server Smalltalk - Examples V 8.0.1  [134].


(Not sure I should be worried by the version numver of that map in 9.2, but will see later today)

To me it seems we will have to analyse the features and manually extract all the required maps from that feature which we also want for our runtime image, in order to provide our own "runtime MAIL stuff" map. Even in cases where this is really just "leave the samples out". I do accept this is feasible for things where we do "complicated" stuff, but I am not sure this should be required for obvious things like examples or feaures with Browser extensions and such...

Joachim




Am Dienstag, 17. Dezember 2019 09:45:03 UTC+1 schrieb Joachim Tuchel:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/53a6293b-ded8-4cdc-acdd-bb25f185a3ab%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

gcotelli
I don't know exactly this case, but usually for features when you install the feature in an XD Image it will only load the runtime support so I'll assume there's some kind of configuration map in the zz. ones having only the runtime support. We usually end up creating our own runtime map if there's no official one provided to use as required map in our own configurations.

On Tuesday, December 17, 2019 at 6:08:15 AM UTC-3, Joachim Tuchel wrote:
Little addition:

what I mean with dev code and runtime code is nicely demonstrated with z.ST: Server Smalltalk (SST) - Mail (IMAP/SMTP).

It will load for example:

Loading the configuration map Server Smalltalk - Examples V 8.0.1  [134]...
Loaded Server Smalltalk - Examples V 8.0.1  [134].



To me it seems we will have to analyse the features and manually extract all the required maps from that feature which we also want for our runtime image, in order to provide our own "runtime MAIL stuff" map. Even in cases where this is really just "leave the samples out". I do accept this is feasible for things where we do "complicated" stuff, but I am not sure this should be required for obvious things like examples or feaures with Browser extensions and such...

Joachim




Am Dienstag, 17. Dezember 2019 09:45:03 UTC+1 schrieb Joachim Tuchel:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/0ef6e014-59ab-49d8-83e7-e190944270ff%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

Louis LaBrunda
In reply to this post by jtuchel
Hi Joachim and Everyone,

I have been too busy lately with personal things to check out version 9.2 so I don't know if there are changes that make this situation worse or it is a tightening up of existing checks.  That said, I would like to add a circumstance to Joachim's.  From time to time I have to apply a fix to base code and getting that fix included when loading from config maps requires changing one and sometimes many config maps.  This can rapidly become problematic.  I think it would be helpful to have an option that allows the loading of the "latest" version of all applications.  I have not thought this through, so it may not be doable or may lead to other problems.  Any thoughts?

Lou


On Tuesday, December 17, 2019 at 3:45:03 AM UTC-5, Joachim Tuchel wrote:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/665ef418-66cf-4ff7-9397-b45bf8bc8cf8%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

Richard Sargent
Administrator
In reply to this post by jtuchel
I hear you and share your feelings.

On Tuesday, December 17, 2019 at 12:45:03 AM UTC-8, Joachim Tuchel wrote:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/77a685b5-ef3d-4562-9b53-f3efc960035f%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

Richard Sargent
Administrator
In reply to this post by Louis LaBrunda
On Wednesday, December 18, 2019 at 4:59:21 AM UTC-8, Louis LaBrunda wrote:
Hi Joachim and Everyone,

I have been too busy lately with personal things to check out version 9.2 so I don't know if there are changes that make this situation worse or it is a tightening up of existing checks.  That said, I would like to add a circumstance to Joachim's.  From time to time I have to apply a fix to base code and getting that fix included when loading from config maps requires changing one and sometimes many config maps.  This can rapidly become problematic.  I think it would be helpful to have an option that allows the loading of the "latest" version of all applications.  I have not thought this through, so it may not be doable or may lead to other problems.  Any thoughts?

Something like this would be relatively easy to do. You could easily programmatically load the root configuration map object and traverse all its required maps to identify the maps to load and their load order, and then load the latest version of each. See EmConfigurationMap. The load with required maps code is there (or starts there) and one can easily clone it.

Having said that, it is potentially dangerous to do this. (In most cases, it would probably be just fine.) If you are like me, you may have an "everything.dat" repository with all VA versions present. I don't know that Instantiations would ever produce a new release of an older version, but that would interfere with the safety of your plan. (e.g. if they provide support and updates for more than just the current version, as we do at GemTalk Systems[*])


- - -
[*] We currently are supporting version 3.4.x while 3.5.x is current and 3.6.0 is in development. Under the right circumstances, we would backport a fix even to 3.3.x and potentially 3.2.x. Those would be extremely exceptional cases, of course.


Lou


On Tuesday, December 17, 2019 at 3:45:03 AM UTC-5, Joachim Tuchel wrote:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/477d00b1-dec0-46c7-b7ef-e6328486a606%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

Richard Sargent
Administrator
In reply to this post by jtuchel
On Tuesday, December 17, 2019 at 1:08:15 AM UTC-8, Joachim Tuchel wrote:
Little addition:

what I mean with dev code and runtime code is nicely demonstrated with z.ST: Server Smalltalk (SST) - Mail (IMAP/SMTP).

It will load for example:

Loading the configuration map Server Smalltalk - Examples V 8.0.1  [134]...
Loaded Server Smalltalk - Examples V 8.0.1  [134].



To me it seems we will have to analyse the features and manually extract all the required maps from that feature which we also want for our runtime image, in order to provide our own "runtime MAIL stuff" map. Even in cases where this is really just "leave the samples out". I do accept this is feasible for things where we do "complicated" stuff, but I am not sure this should be required for obvious things like examples or feaures with Browser extensions and such...

In my opinion, one of the greatest strengths of ENVY is the ability to precisely specify the prerequisites chain of code. Admittedly, it is also a pain to maintain.

So, I have mixed feelings about the abandonment of configuration map prerequisites. In general, I am emphatic about the precision and reliability that arises from keeping and maintaining them.

I think it is probable that tooling to update the existing prerequisite relationships when Instantiations builds new versions can be built and would reduce or even eliminate this burden. After all, the prerequisite chain says not only something is required but that a specific version is certified to work. I've only given this brief thought, but I can elaborate on what it might take to build such a tool, if anyone is interested.


Joachim




Am Dienstag, 17. Dezember 2019 09:45:03 UTC+1 schrieb Joachim Tuchel:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/438178c5-96a7-4513-a8d3-12654fae5060%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

Bob Brodd
In reply to this post by jtuchel
Hi Joachim,

I’ll start with our definition of a Feature …  a set of code defined by a configuration map relying on one or more applications and/or configuration maps which must load automatically and cleanly into an IBMST or a full VA Smalltalk image.  This includes loading the appropriate version of a Feature based on the version of VA Smalltalk being used.  As some others have noted, many people , including me, work out of a single manager that we load all code into.  If you have 8.6.3, 9.0, 9.1, and 9.2 code all in your manager, the Feature system should load the appropriate version of the Feature that matches the version of VA Smalltalk that is being used.  


This leads to the first important reason for using Features … making it as easy and as error-free as possible to load cohesive sets of code into the image without having to know what what specific config map would load it, as well as which version of that map to use for the current image.   It is not always reasonable, or even possible, to just load the latest version of a feature map (z*)  into an image in cases where multiple versions of VA Smalltalk are supported by a single manager.


The second important reason has to with the processes involved with building the product.  As Richard noted “…one of the greatest strengths of ENVY is the ability to precisely specify the prerequisites chain of code. Admittedly, it is also a pain to maintain.”.  As you are probably aware, we create new versions of all Feature (z*. maps) every release.  Since any particular build could be a release, we actually create them every time we build the product, generally once a week.  We take advantage of the Feature system to automatically build them from the latest code available, avoiding the error-prone and tedious process of chasing down all prerequisites to a particular change by hand.  I guarantee we would make a ton or mistakes if we did it by hand. 


I think it is safe to say that the Feature system will continue to be used for these purposes. For those that are not aware, it is acceptable to use the underlying z*. maps as required maps to your own application maps to make it easier to load your own config maps as well as to tie a particular version of your code to a particular version of a feature.  This is a way to load a Feature without having to do so using the  Load/Unload features menu item.



"I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?"

I find this idea interesting.  A little background … the ‘normal’ config maps  referenced by any of our Feature  (z*) maps  are something we call Feature Support Maps.  These maps look like normal config maps, and actually, are normal config maps, but there is one particular requirement from a Feature perspective which must be adhered to … they can not define any required maps or they break the build and Feature systems.  So while they look like something you could load ‘with required maps’ successfully, chances are,  you can’t.  I don’t think renaming all of these Feature Support Maps is a reasonable solution .   We would have to really understand the ramifications of doing so.  The tracking of the history  of config maps would be effected and potentially customers relying on their current names are a couple of problem areas off the top.  Much thought would have to go into making a change such as this.


I can see the usefulness of identifying the Feature Support Maps as I have fallen victim to trying to load one of the maps myself. Off the top, perhaps using a special decoration next to the map name could be helpful? 


Hopefully you will find this information enlightening.


Bob




On Tuesday, December 17, 2019 at 3:45:03 AM UTC-5, Joachim Tuchel wrote:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/5e72d2af-ef71-4903-8aba-ce86cbf236fa%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

jtuchel
Hi Bob,

Thanks for your clarifications. I am still not sure, however, if I understand correctly.

I'll try to phrase my questions a bit more concrete to keep this discussion going:

  •  Is there any difference between using the Load/Unload Feature menu item and loading a z.ST or z.VA map?
  • If there is a difference, would it be desirable for me to use the features instead of maps for my own code? How would I do that? Where is the definition of a feature
  • How can I load a feature in an automated way, like for setting up a daily fresh image every morning?
  • How do I define a set of features that are needed in my develeopment image?
  • How do I use the features to define the set of classes needed for, say an XD packaging image?
  • If I wanted to build a config map that simply loads the "appropriate" version of Refactoring Browser, Glorp, Whatever for the currently used VA Smalltalk version. What exactly would I put into teh req. maps? My understanding is that I must define a concrete version to laod anyways (and that is a good thing, imo)...
  • I still think it is irritating and may cause frustraton that you cannot load the map "Refactoring Browser" any more (there are more examples) but have to load z.ST: Refactoring Browser instead
There are at least three scenarios in which loading code has to be accomplished:

  • a developer wants to prepare their development image to work on stuff
  • a team wants a fast, one-click (or even automated) procedure to make the preparation of a fresh dev image for several project phases
    • I want a maintenance image for the current production verson (Maybe based on Version X of VAST)
    • I want an image to work on new features (maybe based on Version X+1 of VAST)
  • the build system (e.g. triggerd by a Jenkins job) prepares a fresh image for packaging (unattended, fully automated), especially challenging in an XD packaging scenario

AFAIU, Features do not fit well into the automated scenarios. Or I am missing something very substantial about loading and defining dependencies between features?

Joachim




Am Freitag, 20. Dezember 2019 22:32:11 UTC+1 schrieb Bob Brodd:
Hi Joachim,

I’ll start with our definition of a Feature …  a set of code defined by a configuration map relying on one or more applications and/or configuration maps which must load automatically and cleanly into an IBMST or a full VA Smalltalk image.  This includes loading the appropriate version of a Feature based on the version of VA Smalltalk being used.  As some others have noted, many people , including me, work out of a single manager that we load all code into.  If you have 8.6.3, 9.0, 9.1, and 9.2 code all in your manager, the Feature system should load the appropriate version of the Feature that matches the version of VA Smalltalk that is being used.  


This leads to the first important reason for using Features … making it as easy and as error-free as possible to load cohesive sets of code into the image without having to know what what specific config map would load it, as well as which version of that map to use for the current image.   It is not always reasonable, or even possible, to just load the latest version of a feature map (z*)  into an image in cases where multiple versions of VA Smalltalk are supported by a single manager.


The second important reason has to with the processes involved with building the product.  As Richard noted “…one of the greatest strengths of ENVY is the ability to precisely specify the prerequisites chain of code. Admittedly, it is also a pain to maintain.”.  As you are probably aware, we create new versions of all Feature (z*. maps) every release.  Since any particular build could be a release, we actually create them every time we build the product, generally once a week.  We take advantage of the Feature system to automatically build them from the latest code available, avoiding the error-prone and tedious process of chasing down all prerequisites to a particular change by hand.  I guarantee we would make a ton or mistakes if we did it by hand. 


I think it is safe to say that the Feature system will continue to be used for these purposes. For those that are not aware, it is acceptable to use the underlying z*. maps as required maps to your own application maps to make it easier to load your own config maps as well as to tie a particular version of your code to a particular version of a feature.  This is a way to load a Feature without having to do so using the  Load/Unload features menu item.



"I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?"

I find this idea interesting.  A little background … the ‘normal’ config maps  referenced by any of our Feature  (z*) maps  are something we call Feature Support Maps.  These maps look like normal config maps, and actually, are normal config maps, but there is one particular requirement from a Feature perspective which must be adhered to … they can not define any required maps or they break the build and Feature systems.  So while they look like something you could load ‘with required maps’ successfully, chances are,  you can’t.  I don’t think renaming all of these Feature Support Maps is a reasonable solution .   We would have to really understand the ramifications of doing so.  The tracking of the history  of config maps would be effected and potentially customers relying on their current names are a couple of problem areas off the top.  Much thought would have to go into making a change such as this.


I can see the usefulness of identifying the Feature Support Maps as I have fallen victim to trying to load one of the maps myself. Off the top, perhaps using a special decoration next to the map name could be helpful? 


Hopefully you will find this information enlightening.


Bob




On Tuesday, December 17, 2019 at 3:45:03 AM UTC-5, Joachim Tuchel wrote:
Dear VAST team and users,

over the last few releases of VA Smalltalk, I realized Instantiations seems to prefer defining Features over Conf Maps. Even pre-existing features got "broken" by this. The latest version of a map couldn't be loaded on its own any more.

This leads to situations like this:

Loading the configuration map Server Smalltalk - Mail Support V 9.2.0  [453]...
Error: 346   Cannot complete the load because the prerequisite SstTcpCommunications of SstMailCommunications V 9.2.0  [453] must also be loaded.
Failed trying to load Server Smalltalk - Mail Support V 9.2.0  [453].


You cannot easily solve this, because this map has no required maps. If you want to load this, you really have to use the Load/Unload features menu item.

I am trying not to judge this, I'd like to understand the idea behind this change first. Since it is a change at first, it feels like it's in my way when it comes to loading stuff, be it manually or by defining my project's config maps structure. But this is probably completely wrong and Instantiations has really good reasons to switch to z.Something maps as "Bundles of Maps that make up features".

I wonder, however, if it's a good idea to have the "low-level, not for direct loading" maps with a nice name and prefix the ones that you want to load with z. or zz. ....?


So I'd first like to understand the reasoning of behind these changes and then I'd like to know how we are considered to build our map structures in the future.

An area that I wonder about most is this:
In the past we had runtime and dev maps. That's especially a thing when you intend to package headless for runtime.
Will we have runtime and dev features? What will be the naming conventions? Do we postpone the removal of dev code to the packaging step?

This might become a lengthy discussion and I apaologize for it. But I think the new 9.2 release is a good point in time to rethink the structures, processes and code management concepts for the future, and to me it seems this is an important thing to think about.


Joachim











--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/cd266cbd-f135-40ea-8986-5b9999b09503%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [9.2] Features over Conf Maps - how to work in the future?

Bob Brodd

See my >> comments throughout: 

  •  Is there any difference between using the Load/Unload Feature menu item and loading a z.ST or z.VA map?
    >> Sort of … the Feature Loader looks for the version of the associated z. or zz. config map to load based on what version of VA Smalltalk you are running; however, there is a one-to-one relationship with a Feature and a z. map. 
  • If there is a difference, would it be desirable for me to use the features instead of maps for my own code? How would I do that? Where is the definition of a feature
    >>Probably not.  Loading a Feature is a bit less deterministic than loading a version of a config map . 
    >>Features are defined by their z.* map.  There is some additional information in the Comments for the z. maps that help with loading the proper version based on what version of VA Smalltalk is running as well as what the current platform  is (windows, unix, etc.). Not all Features are suitable for all platforms.
  • How can I load a feature in an automated way, like for setting up a daily fresh image every morning?
    >>There is no automated way provided with the product to do this.  How do I define a set of features that are needed in my development image?
    >>I would use a config map defining all of the z. feature maps you need as required maps of the map.
  • How do I use the features to define the set of classes needed for, say an XD packaging image?
    >>XD Features are defined by the zz.* maps.
  • How do I load a feature like SMTP support - without the examples or support tools that are not needed in a runtime image? 
    • I only know how to do that by hand-picking underlying maps by hand. Am I missing something or is this the way to do it?
      >> You are not missing anything.  Keep in mind that unless you include the examples in your required app lineup, they will not be included in your packaged image.
  • If I wanted to build a config map that simply loads the "appropriate" version of Refactoring Browser, Glorp, Whatever for the currently used VA Smalltalk version. What exactly would I put into the req. maps? My understanding is that I must define a concrete version to laod anyways (and that is a good thing, imo)...
    >>You can not load “appropriate” version of Features  using the required maps, they must be concrete versions.  Yes, and using concrete versions mean you have a repeatable system.
  • I still think it is irritating and may cause frustration that you cannot load the map "Refactoring Browser" any more (there are more examples) but have to load z.ST: Refactoring Browser instead
    >>I understand, but it is baked into the Feature system, so not likely to change.  I am still considering ways to mark the Feature Support Maps in a way that does not effect any customers (as renaming might do) but still makes them more apparent, such as using decorations next to the name .


AFAIU, Features do not fit well into the automated scenarios. Or I am missing something very substantial about loading and defining dependencies between features?
>>No, you are not missing anything.  That said, it is a ‘simple’ matter of programming to create such a utility, at least once you dive into the Feature code .  


P.S. This whole complex of questions is closely related to migrating from one VAST version to another. These are topics to take special care of whenever you move your code to a new VAST release. It can be very time consuming and error-prone and is hard to explain and document, so you need to be an "expert" to dare doing it. I'd say that this is the biggest part of migrating code, because you need to re-load images, re-package and re-test over and over again. The migration guide covers the code changes and steps needed to adapt to code changes in VAST very nicely and accurate, but this area at hand is not covered (and I have no idea how I would document it, to be honest)

>>Migration can be a tedious task, particularly if your system includes changes to the base code.  I have performed many migrations of large code bases over the years and found it more of an art than science.  I typically allotted about 1 week of time to migrate a large application with lots of base fixes and enhancements.  Without  modifications to the base , migrations are pretty easy.



--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/455f4e92-4b35-42df-9423-f8b7b4b6da67%40googlegroups.com.