Doubt about IcuLibraryVersion

On 11/22/17 10:30 AM, Mariano Martinez Peck via Glass wrote:

Hi guys,

I am in the process of upgrading to 3.4.0. Actually, I am almost done. However, I have a doubt about IcuLibraryVersion. I did read the manual and it says:

(Globals at: #IcuLibraryVersion) is automatica lly set during upgrade to v3.3.1 or later. Applications upgraded from 3.3.x to 3.4 wi ll use ICU v54.1 or 51.2, depending on the previous upgrade. 

In my case I have some stones migrated with this same sequence:

(Globals at: #'StringConfiguration')  Unicode16

(Globals at: #IcuLibraryVersion)  '51.2'

(Globals at: #DbfHistory) 

'GemStone/S64 v3.1.0.6 kernel classes filein completed at 17/04/2014 16:28:18

upgrade to GemStone/S 64 Bit 3.2.9 completed at 27/10/2015 16:32:12

upgrade to GemStone/S 64 Bit 3.3.3 completed at 30/01/2017 17:34:50

upgrade to GemStone/S 64 Bit 3.4.0 completed at 22/11/2017 09:55:31

'

And another this way:

(Globals at: #'StringConfiguration')  Unicode16

(Globals at: #IcuLibraryVersion)  '54.1'

(Globals at: #DbfHistory) 

 'GemStone/S64 v3.3.3 kernel classes filein completed at 30/11/2016 14:47:52

'

I have read all the possible paths from the manual and my conclusion is that I would have never needed a manual re-sort / re-hash (assume I did use unicode as keys in dictionaries etc). I never found a problem, but an extra confirmation from your side would be nice.

I believe that you have been using the GsDevKit_home upgradeStone script for each of the upgrades, and there are some automatic post upgrade operations being performed (see the /home/utils/upgrade/postUpgrade tODE script) --- sorted collections and CharacterCollection-based indexes are preformed when called for --- the scripts only perform the operations if needed.

I don't think that hashing is an issue across different ICU versions (so using unicode strings as keys are not troublesome) the big thing that changes across ICU versions are the sort keys themselves ... unless you are using specific strings for which the sort order has changed (pretty rare --- and can be determined by reading the ICU release notes) even sorted collections are not likely to change.

The "big issue" is that for the Basic indexes the ICU 'sort key' is inserted into the index structures and it is the sort key that is almost "guaranteed to change" between ICU releases ...So one of the reasons that we don't automatically update the ICU library on upgrade is  to avoid having to force customers to rebuild indexes because of changes to to the icu 'sort key" ...

The new indexing structures introduced in 3.4.0 no longer embed the ICU 'sort key' in the indexing structures, so "required index rebuilds" may not be necessary ... I have to be mealy mouthed and say "may" because it is always possible that the sort order of two strings may change between ICU releases and the only way to know for certain is to read the ICU release notes ... or be paranoid and rebuild "everything"...

So staying with a given ICU release for the "life of a stone including upgrades" is a good way to avoid having to worry about whether or not you will be affected by a change in ICU sort order.

Finally, what would happen with future releases? Let's say I have now stones using either 51.2 or 54.1 ... would those be converted automatically to 58.2 (or whatever stable version at the moment of the future upgrade) in a future upgrade?  Or it would be better to upgrade NOW to the latest stable ICU ?

I don't think we will automatically convert the ICU version on upgrade (unless absolutely necessary) ... fresh repositories will always use the latest ICU version and upgraded repositories will use the ICU version they were created with ... it is then up to you whether or not you switch to a new version of ICU and up to you to decide which structures (if any) you need to rebuild ...

As to stability while there are continual bugfixes with each ICU release, the primary reason for new releases is that new character sets are always being added which changes the sort keys for everyone; as well as bugfixes in the sorting algorithms for existing character sets ...

Therefore, I think the best course of action is to stick with whatever version of ICU a stone is using and only consider upgrading ICU version when you encounter an issue that directly affects your application --- either in terms of a bug in the ICU code itself or in terms of need to take advantage of new character sets/sort orders ...