Changes file access speed degradation

I've got this 3.7 image that has been through quite a lot.  It is the  
one where I hit the wall on changes file size.  Eventually getting  
changes condensed, I applied the patch pointed to by Tim for slower  
growing changes files.

Since this time, I am finding all operations involving accessing  
source code, browsing, compiling, generating MC snapshots, are now  
taking progressively longer periods of time.  On average, it takes  
3-5 seconds for a bit of source code to pop into the browser and MC  
has become pretty much unusable.

A MC snapshot operation message tally is attached as you can see,  
most time is spent in "primitives".

VM version I'm using is Carbon 3.8.12beta-4u although unix vm 3.9-9  
seems to exhibit the same problems.

So, what could it be?  How can I restore performance?  Rebuilding the  
image will take a couple days I'd guess with all the packages  
shoehorned into it with various compatibility tweaks I've made along  
the way.

-Todd Blanchard

- 1704830 tallies, 27602874 msec.

**Tree**
100.0% {27602874ms} MCWorkingCopy>>newVersionWithName:message:
   99.9% {27575271ms} MCPackage>>snapshot
     99.9% {27575271ms} MethodReference>>asMethodDefinition
       99.9% {27575271ms} MCMethodDefinition class>>forMethodReference:
         99.9% {27575271ms} MethodReference>>timeStamp
           99.9% {27575271ms} CompiledMethod>>timeStamp
             99.8% {27547668ms} StandardFileStream>>close
               99.8% {27547668ms} StandardFileStream>>unregister
                 99.8% {27547668ms} StandardFileStream  
class>>unregister:
                   99.8% {27547668ms} WeakRegistry>>remove:ifAbsent:
                     99.8% {27547668ms} WeakKeyDictionary(Dictionary)
 >>removeKey:ifAbsent:
                       99.8% {27547668ms}  
WeakKeyDictionary>>fixCollisionsFrom:
                         99.8% {27547668ms} WeakKeyDictionary>>rehash
                           99.7% {27520065ms}  
WeakKeyDictionary>>scanForNil:
                             99.6% {27492463ms} primitives
**Leaves**
99.7% {27520065ms} WeakKeyDictionary>>scanForNil:

**Memory**
        old +11,980,828 bytes
        young -1,937,972 bytes
        used +10,042,856 bytes

On Dec 23, 2006, at 4:10 PM, Todd Blanchard wrote:

> Since this time, I am finding all operations involving accessing  
> source code, browsing, compiling, generating MC snapshots, are now  
> taking progressively longer periods of time.  On average, it takes  
> 3-5 seconds for a bit of source code to pop into the browser and MC  
> has become pretty much unusable.

This is a problem with MC having zillions of weak key items in the  
weak key dictionary exceeding some nice
round magic number so searching for things becomes quite linear. I  
seen some magic doits to clean up the
keys, but I don't have them handy, will look, but perhaps someone  
else can remember ?

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

Try this via a note from Bert.

(CompiledMethod classPool at: #MethodProperties) in: [:props | [props  
size * 3 < props capacity] whileFalse: [props grow]]


On Dec 23, 2006, at 4:10 PM, Todd Blanchard wrote:

> MethodReference

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

I got 'key not found' on the #MethodProperties - so I did the grow to  
WeakKeyDictionary allInstances.  Hey, there's only 5 of them.

Worked like a charm.  Much more responsive again.  Thanks!

-Todd Blanchard

On Dec 23, 2006, at 6:26 PM, John M McIntosh wrote: