VW 7.9.1 and Store Question

We've seen very slow results for browsing method version history - right click on a method, pick Store>>Browse Versions.  In tracing this, we found our way to this method:

Store.Glorp.StoreMethodInPackage>>allVersionsWIthNameLinClass:in:

which executes a query, then filters out duplicates in Smalltalk.  For a method with one version, but with many (1903) package publishes, we got 1903 results from the query, which Smalltalk then filtered down to the one result.  The first time we executed this, it took minutes (we use a PostGres 8.4 repository).  Subsequent requests seem to be cached, although we aren't sure how long Postgres will keep that query cached.

Is anyone else seeing slow responses from this use of Store (7.9.1)?  We see this whether we use an older repository that was upgraded using the various #upgrade... methods, or in a new one created from 7.9.1 with our code replicated over.  

For comparison purposes, 7.6 finds the versions (same repository) nearly instantly.


James Robertson
http://www.jarober.com
[hidden email]




_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

James,

A (possible) solution would be a reworking of that code.

I would suggest adding, to the "alsoFetch" lines, is one that reads:

        query alsoFetch: [:each | each definition primaryKey].

If the problem is that #primaryKey has to go out to the database for each item it fetched during the #removeDuplicatesFrom: call, this would force that value to come across as part of the original query... The goal is to have everything come across once in one query, which is always faster than faulting in values.

If that doesn't work, a more radical approach might be needed.

> We've seen very slow results for browsing method version history - right click on a method, pick Store>>Browse Versions.  In tracing this, we found our way to this method:
>
> Store.Glorp.StoreMethodInPackage>>allVersionsWIthNameLinClass:in:
>
> which executes a query, then filters out duplicates in Smalltalk.  For a method with one version, but with many (1903) package publishes, we got 1903 results from the query, which Smalltalk then filtered down to the one result.  The first time we executed this, it took minutes (we use a PostGres 8.4 repository).  Subsequent requests seem to be cached, although we aren't sure how long Postgres will keep that query cached.


                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

James, et al.,

This question/issue has been spinning around in my brain for the last two weeks, so I thought why not try to make a solution anyone could use.

I've published a new package to the Public Repository named FastMethodVersions, which adds a new menu option off of the Store submenu of the Methods menu, named "Fast Method Versions".

With the quick part out of the way, now the details:

All of the existing Browse Versions utilities use StoreXXXXInPackage objects, and then whittles down the result to the "oldest" version of each. So, it is getting all 1903 StoreMethodInPackage objects, then throwing out all but the oldest package version for each unique method.

Why does it do this? Well, perhaps because I was trying to be too helpful at the expense of being concise. Or maybe it was just a bad decision on my part.

The upside of the existing utility is that you see the package version which was the first to publish each unique method both in the list item, and when you select an item, on the status line.

Also, if the same method is in different packages, you can see the different package names/versions with the existing tool.

My "Fast" version no longer shows this information, which is the tradeoff.

In effect this "Fast" version is more like the 7.6 version, which did not use Glorp objects.

If you're interested, upon request I'd be glad to make a version of this for Classes, NameSpaces and also Shared Variables, which would also exhibit the same problem.

I can even imaging a slightly less lean version where one could do fetches of just the oldest package and cobble up a fake StoreXXXInPackage that could indeed show the package information... However I was loathe to do so because that would require one additional DB fetch for each unique method, which felt wrong.

Cincom is welcome to take this work and incorporate it any way they wish.

Enjoy!

> We've seen very slow results for browsing method version history - right click on a method, pick Store>>Browse Versions.  In tracing this, we found our way to this method:
>
> Store.Glorp.StoreMethodInPackage>>allVersionsWIthNameLinClass:in:
>
> which executes a query, then filters out duplicates in Smalltalk.  For a method with one version, but with many (1903) package publishes, we got 1903 results from the query, which Smalltalk then filtered down to the one result.  The first time we executed this, it took minutes (we use a PostGres 8.4 repository).  Subsequent requests seem to be cached, although we aren't sure how long Postgres will keep that query cached.
>
> Is anyone else seeing slow responses from this use of Store (7.9.1)?  We see this whether we use an older repository that was upgraded using the various #upgrade... methods, or in a new one created from 7.9.1 with our code replicated over.  
>
> For comparison purposes, 7.6 finds the versions (same repository) nearly instantly.

                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

I'll give it a whirl in our system and see how it goes.  Thanks

On Jan 31, 2013, at 4:07 PM, Samuel S. Shuster wrote:

James Robertson
http://www.jarober.com
[hidden email]




_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc