vm crash in updatePointersinRangeFromto via primitiveArrayBecome (mac vm 3.8.7b5)

Last night I had an vm crash as I was working on TK4, this is an  
unusual event so I'll bring it to everyone's attention.  I recall I  
clicked on the squeak desktop and everything
went away...  This afternoon I'm still working with the image in  
question without issues. Lately we added some changes by Andreas for  
correct weak array handling, some changes to how become: works, and my  
work in VM GC statistical data, so I cann't say which is at fault, if  
any...

In looking I found that my 3.8.7b5 VM had crashed in  
updatePointersinRangeFromto() at  21777 via calling from  
primitiveArrayBecome.

21776 fieldOop = longAt(oop + fieldOffset);
21777 if (((fieldOop & 1) == 0) && (((longAt(fieldOop)) & MarkBit) !=  
0)) {
Which resolves to this assembler, and we croak on the (lwz) load word  
zero high order bits at + 268 attempting to get the word at 0x3618bca2  
(illegal)

                + 256  lwzx     r2,r10,r11 3:1 Stall=2, Loop start[2]  
interp.c:21776 + 260  andi.    r0,r2,0x0001 1:1 interp.c:21777 +  
264  bne      $+280 <updatePointersInRangeFromto + 544> 1:1  
interp.c:21777 + 268  lwz      
r0,0(r2) 3:1 Stall=2 interp.c:21777 + 272  cmpwi    
cr6,r0,0 1:1 interp.c:21777 + 276  bge+     cr6,$+268  
<updatePointersInRangeFromto + 544> 1:1 interp.c:21777

Host Name:      ap0.smalltalkconsulting.net
Date/Time:      2005-04-26 21:34:29 -0700
OS Version:     10.3.9 (Build 7W98)
Report Version: 2

Command: Squeak VM Opt
Path:    /Users/johnmci/Documents/Squeak3.8.0/build/Squeak  
3.8.7Beta5.app/Contents/MacOS/Squeak VM Opt
Version: Squeak VM 3.8.7b5 (3.8.7b5)
PID:     12999
Thread:  2

Exception:  EXC_BAD_ACCESS (0x0001)
Codes:      KERN_INVALID_ADDRESS (0x0001) at 0x3618bca2

Thread 0:
0   libSystem.B.dylib               0x90007878 mach_msg_trap + 0x8
1   libSystem.B.dylib               0x900073f8 mach_msg + 0x38
2   com.apple.CoreFoundation       0x901c16e0 __CFRunLoopRun + 0x350
3   com.apple.CoreFoundation       0x901c5e6c CFRunLoopRunSpecific +  
0x148
4   com.apple.HIToolbox             0x92885f60  
RunCurrentEventLoopInMode + 0xac
5   com.apple.HIToolbox             0x9288c6c8 ReceiveNextEventCommon +  
0x17c
6   com.apple.HIToolbox             0x928ae1a0 AcquireNextEventInMode +  
0x48
7   com.apple.HIToolbox             0x928ab670 RunApplicationEventLoop  
+ 0x88
8   org.squeak.Squeak               0x0006dab0 main + 0x390
9   org.squeak.Squeak               0x00009118 _start + 0x188  
(crt.c:267)
10  dyld                           0x8fe1a278 _dyld_start + 0x64

Thread 1:
0   libSystem.B.dylib               0x90018e18  
semaphore_timedwait_signal_trap + 0x8
1   libSystem.B.dylib               0x9000e9d4 _pthread_cond_wait +  
0x268
2   ...ple.CoreServices.CarbonCore 0x902da570 TSWaitOnSemaphoreCommon  
+ 0xb0
3   ...ple.CoreServices.CarbonCore 0x902ee4f0 TimerThread + 0x3c
4   libSystem.B.dylib               0x90024910 _pthread_body + 0x28

Thread 2 Crashed:
0   org.squeak.Squeak               0x000c3f9c  
updatePointersInRangeFromto + 0x10c
1   org.squeak.Squeak               0x000c19fc becomewithtwoWaycopyHash  
+ 0xfc
2   org.squeak.Squeak               0x000c1b00 primitiveArrayBecome +  
0x30
3   org.squeak.Squeak               0x000ca840 interpret + 0x1af0
4   libSystem.B.dylib               0x90024910 _pthread_body + 0x28

PPC Thread State:
   srr0: 0x000c3f9c srr1: 0x0000d030                vrsave: 0x00000000
     cr: 0x24000244  xer: 0x00000004   lr: 0x000c19fc  ctr: 0x0000004e
     r0: 0x00000000   r1: 0xf0203d90   r2: 0x3618bca2   r3: 0x0000005c
     r4: 0x07348b6c   r5: 0x00000160   r6: 0x4d616369   r7: 0x00107fc0
     r8: 0x3a2f2f2f   r9: 0x00000059  r10: 0x071b8734  r11: 0x00000138
    r12: 0x00000008  r13: 0x00000000  r14: 0x00000000  r15: 0x00000000
    r16: 0x00000000  r17: 0x00000000  r18: 0x00000000  r19: 0x00000000
    r20: 0x00000000  r21: 0x054c962c  r22: 0x07348530  r23: 0x07348510
    r24: 0x00107fc0  r25: 0x000f1b18  r26: 0x00000001  r27: 0x00000001
    r28: 0x00107fc0  r29: 0x00000004  r30: 0x07348b60  r31: 0x07348b58

Binary Images Description:
     0x1000 -    0xebfff org.squeak.Squeak Squeak VM 3.8.7b5  
(3.8.7b5) /Users/johnmci/Documents/Squeak3.8.0/build/Squeak  
3.8.7Beta5.app/Contents/MacOS/Squeak VM Opt


Changes leading up to this VM are:


3.8.7b5 Fix logic to allow you to start VM from command line, changed
                                sqMacFileLogic.c makeHFSFromPosixPath added to convert posix cmd  
line path names to HFS
                                sqMacMain.c add logic to call cmd line parser
                                sqMacUnixCommandLineInterface.c new file logic to parse cmd line
                                sqMacUnixCommandLineInterface.h
                                sqMacWindow.c minor changes to correct compiler errors

3.8.7b4/b3 Limited release for testing

3.8.7b2 sqUIEvents.c changed keyUp/keyDown to supply mac virtual  
keycode versus unicode, added new parm to keyChar to supply UTF-32  
Unicode.
                                As per Andreas request for Tweak.

3.8.7b1 sqUIEvents.c changed keyUp/keyDown logic to match Windows  
pattern, key repeat becomes keyDown/keyDown/keyUp

3.8.6b6 sqMacMIDI.c add #def so I can compile under code warrior os-9  
again
                                sqMacEncoding.c, sqMacEncoding.h  Minor changes so I can compile  
user code warrior os-9 again
                                sqMacUnixInterfaceSound.c  Fix problem with empty buffer being full  
of junk under certain conditions, that causes a sharp sound  bite.

3.8.6b5 Change to JMM & AR GC instrumentation patch, collect a few  
less statistics. Also change carbon VM to use Unix/Quartz sound driver
                                which interfaces to Core Audio stuff.

3.8.6b4 Ensure VMMaker 3.8b2 - the VM code classes and tools for  
building a working vm for Squeak version  > 3.8-6461. is incorporated.
                                Add Andreas's GC improvement changes for
                                * primitiveIsYoung: This answers the question whether an object  
currently lives in young or in old space.

                                * primitiveIsRoot: Answers the question whether any given object is  
currently a root for young space.

                                * primitiveRootTable: Answers a snapshot of the current root table.  
Useful to examine the roots table if the analysis requires complex  
other operations during which the root table might be modified itself.  
Note that since this primitive can cause GC there is a small chance  
that it will give an inaccurate answer.

                                * primitiveRootTableAt: Answers a single element of the root table  
(by one-based index). This primitive can be used to quickly scan the  
root table for certain objects.

                                * primitiveSetGCSemaphore: Indicates a semaphore (index) to be  
signaled whenever a garbage collection occurs. I can see at least two  
uses uses for the GC semaphore: running cleanup actions (for example  
after full GCs occured) and dynamic parameter tuning for the GC  
algorithm itself.

                                Add John's changes for

                                Smalltalk setGCBiasToGrowGCLimit: 16*1024*1024. "Set growth limit  
before full GC to 16MB"

                                Smalltalk setGCBiasToGrow: 1.
                                Set bias to grow upto GCLimit, this can avoid a problem where we  
attempt to avoid growing but results in thousands of incremental GC  
events as
                                we approach a knee in a curve of space used versus the  
growth/compaction decision.


3.8.6b3 Add in logic for become: to fix issues with changes to  
become: logic.

3.8.6b2 sqMacWindow.c  reduce number of lock/unlock pixel events and  
flush no more than 91 fps, with flush check every 1/50 or 1/60 and 1/5  
of a second. this
                                shaves another 50% off making drawing pixels 1.3% of the time  
originally taken.

3.8.6b1 sqMacWindow.c, sqUIEvents.c, changed logic to flush screen  
every 125 fps, this makes pixel level drawing complete in 2% of the  
time originally taken.
                       

3.8.5b1 Fix issue with gnuify. The offical gnu header file used a  
different #ifdef for the foo register than an altered gnu header file I  
was using for
                                many years. This resulted in slower performance, which is fixed in  
3.8.5
3.8.4b3 Support for GC statistical data and change to how memory is  
allocated and tenured.
                                        Additional statistical information from the GC routines
                                        Additional primitives to set GC semaphore, and to force a tenure

                                Change to make forward logic to look for "foo become: foo" and  
cause a prim failure.
                                Earlier version of 3.8.4 where distributed to Ffenstri testers.
               
                                sqMacHostWindow Change calls to move and size window to use  
GUILocker callbacks to avoid VM crash.
                                sqMacWIndow.c Alter 1/2/4 bit support to pass in bit size of  
target window, then copy is more efficient.


3.8.3b3 Support for Host menus and windows. Also added back 1bit  
(B/W) and 2bit & 4bit color as requested by Dan Ingalls for os-x

                                Earlier versions of 3.8.0/1/2/3 were distributed to the TK4 team,  
not the public.
                               
                                sq.h  Added events, EventTypeMenu, EventTypeWindow to handle menu  
and window events generated by the Host OS
                                        and placed on the Squeak Event Queue to be read by EventSensor
                               
                                sqGnu.h Switch to the current file used by Ian for  Unix, versus  
older modified copy.
                                sqMacExternalPrims.c Cleanup of warning message
                                sqMacImageIO.c When you set the image name, we set the title on  
window 1.
                                sqMacMain.c Do not create the main window until we actually use  
it.
                                sqMacNSPlugin.c Always refer to window 1, not sure what it means  
to have other windows in the plugin yet...
                                sqMacUIEvents.c OS-9
                                                                        Pass cmd-key menu activity up
                                                                        Pass update event bounding box window events up
                                                                        Pass window activate events up
                                                                        Pass menu interaction activity up
                                                                        Pass window events, resize,zoom,close,move,inconize up
                               
                                                                        OS-X
                                                                        Same as above
                                                                        getUIToLock needs to ReleaseEvent on the passed in event for  
GC reasons
                                sqMacUIMenuBar.c disable edit menu at start, if later we find it  
enabled, don't touch, otherwise handle os-9 DA issues.
                                sqmacWindow.c sqShowWindow now gets a windowIndex parm.
                                                                        ioShowDisplayOnWindow is added, with logic to build main  
window at first draw time,
                                                                        also fix some boundary checks, and provide support (again) for  
1/2/4 bit color under os-x ioHasDisplayDepth() will return true  
for 1/2/4/8/16/32 bit color


--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

In message <[hidden email]>
          John M McIntosh <[hidden email]> wrote:

> question without issues. Lately we added some changes by Andreas for  
> correct weak array handling, some changes to how become: works, and my  
> work in VM GC statistical data, so I cann't say which is at fault, if  
> any...
>
I'm currently hip-deep in similar excrement after merging in the new lowspace-
process handling and the gc instrumentation/weak pointer stuff. It _looks_ as
if something is getting twisted in the general gc area since the free block
size ends up being set to 4. As in '4' not 4k or 4mb, just 4. That quite
unsurprisingly upsets the sufficientSpaceToAllocate: code and we get an Earth-
shattering kaboom. Once my head has stopped spinning I'll try the lowspace
changes without the gc changes.

All to try to make it possible to debug some stupid problem in TK4 tree
walking....


tim
--
Tim Rowledge, [hidden email], http://sumeru.stanford.edu/tim
The hardness of the butter is proportional to the softness of the bread.