Changes To Come

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
April 1, 2011 10:16 AM

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

B

            B1

                        B11

                                   P11, P12, P13             (packages of B11)

                        B12     ...

                        B13     ...

            B2

                        B21     ...

                        B22     ...

If there is a new (more recent) version in the StoreDB (with same blessing level) B1´of B1

(and new versions, indicated by " ´ " of    B11, P11, P13 and very new Versions B14 and P14)

but there is not a more recent version of B :

B1´

                        B11’

                                   P11’ P13’ P14            (packages of B11´)

                        B13     ...

                        B14     ...

What do we have after loading via the above code, with mainBundleName := B?

B                                                                      

            B1’…                                                                (**)

            B2 …

Or the following?:

B

            B1’

            B2

P12                                          no more contained item in any bundle, but loaded

B12     …                                no more contained item in any bundle, but loaded

Or are P14, B14 event not loaded?

(**) is what we are loading now, every day – due to our own implementation of this behavior. If this will be gotten by such easy utilities as in your code above, it would be very helpful for us.

Regards, Ulli

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

--
Alan Knight [|], Engineering Manager, Cincom Smalltalk
[hidden email]
[hidden email]
http://www.cincomsmalltalk.com

---

[hidden email]
April 4, 2011 9:41 AM

Thanks, Alan. It is very interesting and can be very helpful for us.

May be, we could waste much of our (own implemented) utilities for publishing and loading. And we are very interested in to waste it, cause it's not our job to do.


Am 01.04.2011 16:16, schrieb Alan Knight:

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

Does it really do what we need? For example:
First (in our image) we have a bundle structure like

B

            B1

                        B11

                                   P11, P12, P13             (packages of B11)

                        B12     ...

                        B13     ...

            B2

                        B21     ...

                        B22     ...

If there is a new (more recent) version in the StoreDB (with same blessing level) B1´of B1

(and new versions, indicated by " ´ " of    B11, P11, P13 and very new Versions B14 and P14)

but there is not a more recent version of B :

B1´

                        B11’

                                   P11’ P13’ P14            (packages of B11´)

                        B13     ...

                        B14     ...

What do we have after loading via the above code, with mainBundleName := B?

B                                                                      

            B1’…                                                                (**)

            B2 …

Or the following?:

B

            B1’

            B2

P12                                          no more contained item in any bundle, but loaded

B12     …                                no more contained item in any bundle, but loaded

Or are P14, B14 event not loaded?

(**) is what we are loading now, every day – due to our own implementation of this behavior. If this will be gotten by such easy utilities as in your code above, it would be very helpful for us.

Regards, Ulli

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
April 1, 2011 10:16 AM

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

If I follow your example correctly, I think that your question about added packages or sub-bundles is the first point I list as a possible improvement. If we update to the latest bundle version, then recursively update each of its sub-components, then new packages and sub-bundles should be found and loaded.

If a package or sub-bundle is removed and we load the parent bundle, then I believe Store behaviour at the moment is that the removed version will be left in the image, but orphaned. I think that's always been the behaviour, but now that you bring it up, I don't think it's the right thing to do. Historically, I think this was probably because you didn't want to delete a package that was just being moved to another bundle or another place in the same bundle. But with the analysis loader I don't think that would be a problem now. We've discussed this, and created AR 62806 for it. Thanks for pointing that out.

---

[hidden email]
April 4, 2011 9:41 AM

Thanks, Alan. It is very interesting and can be very helpful for us.

May be, we could waste much of our (own implemented) utilities for publishing and loading. And we are very interested in to waste it, cause it's not our job to do.


Am 01.04.2011 16:16, schrieb Alan Knight:

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

Does it really do what we need? For example:
First (in our image) we have a bundle structure like

B

            B1

                        B11

                                   P11, P12, P13             (packages of B11)

                        B12     ...

                        B13     ...

            B2

                        B21     ...

                        B22     ...

If there is a new (more recent) version in the StoreDB (with same blessing level) B1´of B1

(and new versions, indicated by " ´ " of    B11, P11, P13 and very new Versions B14 and P14)

but there is not a more recent version of B :

B1´

                        B11’

                                   P11’ P13’ P14            (packages of B11´)

                        B13     ...

                        B14     ...

What do we have after loading via the above code, with mainBundleName := B?

B                                                                      

            B1’…                                                                (**)

            B2 …

Or the following?:

B

            B1’

            B2

P12                                          no more contained item in any bundle, but loaded

B12     …                                no more contained item in any bundle, but loaded

Or are P14, B14 event not loaded?

(**) is what we are loading now, every day – due to our own implementation of this behavior. If this will be gotten by such easy utilities as in your code above, it would be very helpful for us.

Regards, Ulli

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
April 1, 2011 10:16 AM

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

--
Alan Knight [|], Engineering Manager, Cincom Smalltalk
[hidden email]
[hidden email]
http://www.cincomsmalltalk.com

---

[hidden email]
April 5, 2011 10:56 AM

In this context I want to remember to one issue in our meeting in Barcelona: giving a utility for simply producing a patch. I think you've had similar ideas.
We are making patches for our customers approximately twice each weak. So we can guaranty, that each of our customers can be immediately updated  to the recent state of our (released) code.

A Patch is a parcel containing all differences between the state in the image at the deployment date and the state of most recent versions of all our packages ("our" is characterized by package name conventions). The utility first compares all our packages in the image with the most recent in StoreDB and notice the ChangeSets. Then it loads all most recent packages and moves all changed/added methods noticed in the ChangeSets to the Patch. The removed methods in the noticed ChangeSets will be used to create a method in the Patch returning a dictionary (class->method selector) for later removals. This dictionary is used in the preLoadBlock of the Patch parcel to remove all methods that has to be removed when starting our application. Of course, all this is based on a certain fixed blessing level.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
April 4, 2011 7:37 PM

If I follow your example correctly, I think that your question about added packages or sub-bundles is the first point I list as a possible improvement. If we update to the latest bundle version, then recursively update each of its sub-components, then new packages and sub-bundles should be found and loaded.

If a package or sub-bundle is removed and we load the parent bundle, then I believe Store behaviour at the moment is that the removed version will be left in the image, but orphaned. I think that's always been the behaviour, but now that you bring it up, I don't think it's the right thing to do. Historically, I think this was probably because you didn't want to delete a package that was just being moved to another bundle or another place in the same bundle. But with the analysis loader I don't think that would be a problem now. We've discussed this, and created AR 62806 for it. Thanks for pointing that out.

---

[hidden email]
April 4, 2011 9:41 AM

Thanks, Alan. It is very interesting and can be very helpful for us.

May be, we could waste much of our (own implemented) utilities for publishing and loading. And we are very interested in to waste it, cause it's not our job to do.


Am 01.04.2011 16:16, schrieb Alan Knight:

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

Does it really do what we need? For example:
First (in our image) we have a bundle structure like

B

            B1

                        B11

                                   P11, P12, P13             (packages of B11)

                        B12     ...

                        B13     ...

            B2

                        B21     ...

                        B22     ...

If there is a new (more recent) version in the StoreDB (with same blessing level) B1´of B1

(and new versions, indicated by " ´ " of    B11, P11, P13 and very new Versions B14 and P14)

but there is not a more recent version of B :

B1´

                        B11’

                                   P11’ P13’ P14            (packages of B11´)

                        B13     ...

                        B14     ...

What do we have after loading via the above code, with mainBundleName := B?

B                                                                      

            B1’…                                                                (**)

            B2 …

Or the following?:

B

            B1’

            B2

P12                                          no more contained item in any bundle, but loaded

B12     …                                no more contained item in any bundle, but loaded

Or are P14, B14 event not loaded?

(**) is what we are loading now, every day – due to our own implementation of this behavior. If this will be gotten by such easy utilities as in your code above, it would be very helpful for us.

Regards, Ulli

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt

---

[hidden email]
March 30, 2011 4:24 AM

Samuel S. Shuster wrote:

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What are the old and the new objects exactly?

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

Can you provide, please, an exact design documentation? Or is there already one, anywhere? A recipe for migration will be helpful - what should be replaced by what; which behavior is now realized an other way then before; how to do that the new way...

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

Obviously, this could only help the first time after deploying of 7.9...

To explain my interest:
We have implemented our own load mechanism. We load every time the most recent pundles of a certain blessing level (http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Streamlined+Loading). If somebody published a pundle, all prior Version of this blessing level are automatically set to be "integrated". This mechanism is also the base of producing a patch. A patch for our customers contains the difference (class definitions, changed/removed methods...) between the deployment image and the most recent pundles of a certain blessing level.

If you are the maintainer or owner of 

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Regards, Ulli

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
March 25, 2011 3:51 PM

Dear Friends,

In the work we are beginning to do for our Next release, currently targeted as 7.9 (Not the current release which is soon to be ready: 7.8), we are removing the Old Store Database Objects from the StoreBase Bundle/Parcel.

What does that mean for you?

Depends...

If you are using Store out of the box, no thing n

If you have extended Store, and use the non-Glorp objects, then you can begin to migrate to the new Glorp Store objects.

In 7.9 the old code and objects will be available in a a Parcel which will be named OldStoreDatabase.

If you are the maintainer or owner of

StarBrowser2
PostgreSQLMonitor.pst
MooseSuiteLoaderFromSCGStore.pst

You might want to update these to use the new Glorp Store objects.

And So It Goes
Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, Store Project
Smalltalk Enables Success -- What Are YOU Using?





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

---

[hidden email]
April 1, 2011 10:16 AM

As Sames said, most of the class side APIs are also supported on the newer objects, which also correspond roughly one to one with the database tables in Store, but they have significantly more behaviour than the old Store Record classes. While limited, the "Store Workbook" shows a number of examples of how to work with those objects. In terms of loading the latest versions of things, here's a simple workspace script that loads the latest versions at or above a certain blessing level for everything in a bundle. It could fairly easily be generalized to get the list of things it was loading from somewhere else.

mainBundleName := 'Base VisualWorks'.
blessingLevel := Store.Policies blessingPolicy blessingNumber: 'Development'.
bundle := Store.Registry bundleNamed: mainBundleName.
allPundles := (Array with: bundle), bundle allContainedItems.
allPundles do: [:each |
    session := StoreLoginFactory currentStoreSession.
    query := Query readOneOf: each storeForGlorpPundleClass
        where: [:eachDBPundle | eachDBPundle name = each name & (eachDBPundle blessingLevel >= blessingLevel)].
    query orderBy: [:x | x timestamp descending].
    version := session execute: query.
    version loadSource].
 
This gets all of the sub-components of the bundle (at the image level, using the package and bundleModel objects, same as in previous versions), in order, bundles before their contained packages. Then it goes through each item in the list and loads it. So it would load the Base VisualWorks latest version at or above the 'Development' blessing level, then load the latest version of each sub-component. There are, of course, a good number of things that could be improved on that.

First, it gets the structure all at once. It would probably be better if, after loading a bundle, it then looked at the sub-components of that bundle. That way, if there were changes in bundle structure from what's loaded, it would see those, and automatically load the latest version of the new things.

Second, the querying is not particularly efficient. It queries each package or bundle individually for the latest version. We could fairly easily condense that to querying for all of them, or at least all of them within a bundle, at once. It also doesn't pay any attention to whether the version that it's loading is already loaded or not.

It also loads each item completely individually. If we were being clever, we could group all of the different loads into a single atomic load, so if there was a problem updating, or the update involved complex code changes that were likely to break the system while only partially applied, they'd work better, and if they didn't work, the image would just exist in its previous state, and wouldn't be broken. But that would be a bit more complicated, because it would probably have to go away from the strategy of loading the bundle and then updating each of the contained items. We'd probably want to figure out exactly what we were loading, and then load it, and it might get a bit tricky if both the main bundle and individual items needed to be updated.

One obvious generalization would be to apply this to everything in the image that was reconciled to the database. rather than to a single bundle. A rather crude version would be

bundles := Store.Registry allBundles.
packages := Store.Registry allPackages.
topLevelBundles := bundles reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
topLevelPackages := packages reject: [:each | bundles anySatisfy: [:eachBundle | eachBundle allContainedItems includes: each]].
reconciled := topLevelBundles, topLevelPackages reject: [:each | each dbTrace isNil].
    and then proceed to operate on the reconciled list as above

    

---

[hidden email]
March 30, 2011 5:41 AM

We also would be interested in such documentation. Most important
"store extension": we dropped bundles in favor of lineups/configs
("TIO Store extensions" in public repository).

greetings

Mathieu van Echtelt