Re: Difficult to debug VM crash with full blocks and Sista V1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Nicola Mingotti

I can help you a bit only on this point:
"- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.

Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
macOS> apropos dummynet
dummynet(4) ....

So, yes, it is there.

HTH

bye
Nicola




 




On 9/13/19 8:15 PM, Eliot Miranda wrote:
Hi All,

    there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.

This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.

This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.

- is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?

- is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?

- is there a way of introducing network delays in Mac OS that might help me induce the bug?

- can anyone think of any other strategies I might take to try and reproduce this?

I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?

Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).

(lldb) thr b

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)

  * frame #0: 0x000000010de5700a

    frame #1: 0x000000010dd7b174

    frame #2: 0x000000010dd45f1c

    frame #3: 0x000000010dd44534

    frame #4: 0x000000010dd44c60

(lldb) x/10i 0x000000010de5700a


(lldb) call printStackCallStackOf($rbp)

    0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap

    0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap

    0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class

    0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class

    0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class


(lldb) x/10i 0x000000010dd7b174

    0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx

    0x10dd7b178: 48 89 ec     movq   %rbp, %rsp

    0x10dd7b17b: 5d           popq   %rbp

    0x10dd7b17c: c2 10 00     retq   $0x10

    0x10dd7b17f: cc           int3

    0x10dd7b180: cc           int3

    0x10dd7b181: cc           int3

    0x10dd7b182: cc           int3

    0x10dd7b183: cc           int3

    0x10dd7b184: cc           int3

(lldb) print whereIs(0x000000010dd7b174)

(char *) $0 = 0x00000001000f83ff " is in generated methods"

(lldb) call printCogMethodFor((void *)0x000000010dd7b174)

       0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:

(lldb) print whereIs(0x000000010de5700a)

(char *) $1 = 0x00000001000f83ff " is in generated methods"

(lldb) call printCogMethodFor((void *)0x000000010de5700a)

       0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20


this method ends up being the fitted version of Delay class>> startTimerEventLoop
_,,,^..^,,,_
best, Eliot


    



Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Tobias Pape

> On 14.09.2019, at 06:03, Nicola Mingotti <[hidden email]> wrote:
>
>
> I can help you a bit only on this point:
> "- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yea it is called "network link conditioner.prefpane" :D


>
> Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.
>
> Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
> macOS> apropos dummynet
> dummynet(4) ....
>
> So, yes, it is there.
>
> HTH
>
> bye
> Nicola
>
>
>
>
>  
>
>
>
>
> On 9/13/19 8:15 PM, Eliot Miranda wrote:
>> Hi All,
>>
>>     there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.
>>
>> This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.
>>
>> This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.
>>
>> - is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?
>>
>> - is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?
>>
>> - is there a way of introducing network delays in Mac OS that might help me induce the bug?
>>
>> - can anyone think of any other strategies I might take to try and reproduce this?
>>
>> I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?
>>
>> Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).
>>
>> (lldb) thr b
>> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>>   * frame #0: 0x000000010de5700a
>>     frame #1: 0x000000010dd7b174
>>     frame #2: 0x000000010dd45f1c
>>     frame #3: 0x000000010dd44534
>>     frame #4: 0x000000010dd44c60
>> (lldb) x/10i 0x000000010de5700a
>>
>> (lldb) call printStackCallStackOf($rbp)
>>     0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class
>>
>> (lldb) x/10i 0x000000010dd7b174
>>     0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx
>>     0x10dd7b178: 48 89 ec     movq   %rbp, %rsp
>>     0x10dd7b17b: 5d           popq   %rbp
>>     0x10dd7b17c: c2 10 00     retq   $0x10
>>     0x10dd7b17f: cc           int3
>>     0x10dd7b180: cc           int3
>>     0x10dd7b181: cc           int3
>>     0x10dd7b182: cc           int3
>>     0x10dd7b183: cc           int3
>>     0x10dd7b184: cc           int3
>> (lldb) print whereIs(0x000000010dd7b174)
>> (char *) $0 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010dd7b174)
>>        0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:
>> (lldb) print whereIs(0x000000010de5700a)
>> (char *) $1 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010de5700a)
>>        0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20
>>
>> this method ends up being the fitted version of Delay class>> startTimerEventLoop
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Nicolas Cellier
No idea if this is related or not,
but i got regular crash on macos own compiled x64 artefact by just running:
    SocketTest suite run.

I first though about inquiring some UB, but this happens in debug version, so it's probably something else.
I did not try to simulate (but can we simulate socket tests?).
This does not happen in windows 64 nor linux 64 (wsl) where I can run all the tests.
Note that on wsl i had to evaluate (Compiler recompileAll) in trunk6-64.image, otherwise a bunch of tests are failing... Mysterious.
I also had 2 ByteSymbol differing from their interned version.

Le sam. 14 sept. 2019 à 09:33, Tobias Pape <[hidden email]> a écrit :

> On 14.09.2019, at 06:03, Nicola Mingotti <[hidden email]> wrote:
>
>
> I can help you a bit only on this point:
> "- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yea it is called "network link conditioner.prefpane" :D


>
> Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.
>
> Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
> macOS> apropos dummynet
> dummynet(4) ....
>
> So, yes, it is there.
>
> HTH
>
> bye
> Nicola
>
>
>
>

>
>
>
>
> On 9/13/19 8:15 PM, Eliot Miranda wrote:
>> Hi All,
>>
>>     there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.
>>
>> This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.
>>
>> This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.
>>
>> - is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?
>>
>> - is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?
>>
>> - is there a way of introducing network delays in Mac OS that might help me induce the bug?
>>
>> - can anyone think of any other strategies I might take to try and reproduce this?
>>
>> I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?
>>
>> Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).
>>
>> (lldb) thr b
>> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>>   * frame #0: 0x000000010de5700a
>>     frame #1: 0x000000010dd7b174
>>     frame #2: 0x000000010dd45f1c
>>     frame #3: 0x000000010dd44534
>>     frame #4: 0x000000010dd44c60
>> (lldb) x/10i 0x000000010de5700a
>>
>> (lldb) call printStackCallStackOf($rbp)
>>     0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class
>>
>> (lldb) x/10i 0x000000010dd7b174
>>     0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx
>>     0x10dd7b178: 48 89 ec     movq   %rbp, %rsp
>>     0x10dd7b17b: 5d           popq   %rbp
>>     0x10dd7b17c: c2 10 00     retq   $0x10
>>     0x10dd7b17f: cc           int3
>>     0x10dd7b180: cc           int3
>>     0x10dd7b181: cc           int3
>>     0x10dd7b182: cc           int3
>>     0x10dd7b183: cc           int3
>>     0x10dd7b184: cc           int3
>> (lldb) print whereIs(0x000000010dd7b174)
>> (char *) $0 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010dd7b174)
>>        0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:
>> (lldb) print whereIs(0x000000010de5700a)
>> (char *) $1 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010de5700a)
>>        0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20
>>
>> this method ends up being the fitted version of Delay class>> startTimerEventLoop
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>





Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Nicolas Cellier
Forgot to say: it happens in spur only, not stack.

Le lun. 16 sept. 2019 à 22:52, Nicolas Cellier <[hidden email]> a écrit :
No idea if this is related or not,
but i got regular crash on macos own compiled x64 artefact by just running:
    SocketTest suite run.

I first though about inquiring some UB, but this happens in debug version, so it's probably something else.
I did not try to simulate (but can we simulate socket tests?).
This does not happen in windows 64 nor linux 64 (wsl) where I can run all the tests.
Note that on wsl i had to evaluate (Compiler recompileAll) in trunk6-64.image, otherwise a bunch of tests are failing... Mysterious.
I also had 2 ByteSymbol differing from their interned version.

Le sam. 14 sept. 2019 à 09:33, Tobias Pape <[hidden email]> a écrit :

> On 14.09.2019, at 06:03, Nicola Mingotti <[hidden email]> wrote:
>
>
> I can help you a bit only on this point:
> "- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yea it is called "network link conditioner.prefpane" :D


>
> Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.
>
> Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
> macOS> apropos dummynet
> dummynet(4) ....
>
> So, yes, it is there.
>
> HTH
>
> bye
> Nicola
>
>
>
>

>
>
>
>
> On 9/13/19 8:15 PM, Eliot Miranda wrote:
>> Hi All,
>>
>>     there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.
>>
>> This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.
>>
>> This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.
>>
>> - is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?
>>
>> - is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?
>>
>> - is there a way of introducing network delays in Mac OS that might help me induce the bug?
>>
>> - can anyone think of any other strategies I might take to try and reproduce this?
>>
>> I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?
>>
>> Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).
>>
>> (lldb) thr b
>> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>>   * frame #0: 0x000000010de5700a
>>     frame #1: 0x000000010dd7b174
>>     frame #2: 0x000000010dd45f1c
>>     frame #3: 0x000000010dd44534
>>     frame #4: 0x000000010dd44c60
>> (lldb) x/10i 0x000000010de5700a
>>
>> (lldb) call printStackCallStackOf($rbp)
>>     0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class
>>
>> (lldb) x/10i 0x000000010dd7b174
>>     0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx
>>     0x10dd7b178: 48 89 ec     movq   %rbp, %rsp
>>     0x10dd7b17b: 5d           popq   %rbp
>>     0x10dd7b17c: c2 10 00     retq   $0x10
>>     0x10dd7b17f: cc           int3
>>     0x10dd7b180: cc           int3
>>     0x10dd7b181: cc           int3
>>     0x10dd7b182: cc           int3
>>     0x10dd7b183: cc           int3
>>     0x10dd7b184: cc           int3
>> (lldb) print whereIs(0x000000010dd7b174)
>> (char *) $0 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010dd7b174)
>>        0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:
>> (lldb) print whereIs(0x000000010de5700a)
>> (char *) $1 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010de5700a)
>>        0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20
>>
>> this method ends up being the fitted version of Delay class>> startTimerEventLoop
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>





Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Nicolas Cellier
Attached a bunch of crash dumps, some of them with same stack as your last example...
You definitely want my internet connection (or not...)!

Le lun. 16 sept. 2019 à 23:05, Nicolas Cellier <[hidden email]> a écrit :
Forgot to say: it happens in spur only, not stack.

Le lun. 16 sept. 2019 à 22:52, Nicolas Cellier <[hidden email]> a écrit :
No idea if this is related or not,
but i got regular crash on macos own compiled x64 artefact by just running:
    SocketTest suite run.

I first though about inquiring some UB, but this happens in debug version, so it's probably something else.
I did not try to simulate (but can we simulate socket tests?).
This does not happen in windows 64 nor linux 64 (wsl) where I can run all the tests.
Note that on wsl i had to evaluate (Compiler recompileAll) in trunk6-64.image, otherwise a bunch of tests are failing... Mysterious.
I also had 2 ByteSymbol differing from their interned version.

Le sam. 14 sept. 2019 à 09:33, Tobias Pape <[hidden email]> a écrit :

> On 14.09.2019, at 06:03, Nicola Mingotti <[hidden email]> wrote:
>
>
> I can help you a bit only on this point:
> "- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yea it is called "network link conditioner.prefpane" :D


>
> Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.
>
> Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
> macOS> apropos dummynet
> dummynet(4) ....
>
> So, yes, it is there.
>
> HTH
>
> bye
> Nicola
>
>
>
>

>
>
>
>
> On 9/13/19 8:15 PM, Eliot Miranda wrote:
>> Hi All,
>>
>>     there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.
>>
>> This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.
>>
>> This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.
>>
>> - is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?
>>
>> - is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?
>>
>> - is there a way of introducing network delays in Mac OS that might help me induce the bug?
>>
>> - can anyone think of any other strategies I might take to try and reproduce this?
>>
>> I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?
>>
>> Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).
>>
>> (lldb) thr b
>> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>>   * frame #0: 0x000000010de5700a
>>     frame #1: 0x000000010dd7b174
>>     frame #2: 0x000000010dd45f1c
>>     frame #3: 0x000000010dd44534
>>     frame #4: 0x000000010dd44c60
>> (lldb) x/10i 0x000000010de5700a
>>
>> (lldb) call printStackCallStackOf($rbp)
>>     0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class
>>
>> (lldb) x/10i 0x000000010dd7b174
>>     0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx
>>     0x10dd7b178: 48 89 ec     movq   %rbp, %rsp
>>     0x10dd7b17b: 5d           popq   %rbp
>>     0x10dd7b17c: c2 10 00     retq   $0x10
>>     0x10dd7b17f: cc           int3
>>     0x10dd7b180: cc           int3
>>     0x10dd7b181: cc           int3
>>     0x10dd7b182: cc           int3
>>     0x10dd7b183: cc           int3
>>     0x10dd7b184: cc           int3
>> (lldb) print whereIs(0x000000010dd7b174)
>> (char *) $0 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010dd7b174)
>>        0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:
>> (lldb) print whereIs(0x000000010de5700a)
>> (char *) $1 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010de5700a)
>>        0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20
>>
>> this method ends up being the fitted version of Delay class>> startTimerEventLoop
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>






crash.dmp (333K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

state of FFI?

Squeak - Dev mailing list
In reply to this post by Nicolas Cellier
Looks like some ppl might be trying to apply the paddles to the
Cobalt/Croquet fork...

That was based on a FFI system to make OpenGL calls. Does anyone know
the state of FFI? It does not seem to be in the 5.2 image I pulled from
squeak.org. Has it been supersceded or was it a special feature of the
Croquet distribution to begin with?



--
Clowns feed off of funny money;
Funny money comes from the FED
so NO FED -> NO CLOWNS!!!

Powers are not rights.


Reply | Threaded
Open this post in threaded view
|

Re: state of FFI?

Chris Muller-3
Hi Alan,

The FFI package with tests is available in Squeak 5.2 via the SqueakMap browser.

 - Chris

On Mon, Sep 16, 2019 at 5:49 PM Alan Grimes via Squeak-dev <[hidden email]> wrote:
Looks like some ppl might be trying to apply the paddles to the
Cobalt/Croquet fork...

That was based on a FFI system to make OpenGL calls. Does anyone know
the state of FFI? It does not seem to be in the 5.2 image I pulled from
squeak.org. Has it been supersceded or was it a special feature of the
Croquet distribution to begin with?



--
Clowns feed off of funny money;
Funny money comes from the FED
so NO FED -> NO CLOWNS!!!

Powers are not rights.




Reply | Threaded
Open this post in threaded view
|

Re: state of FFI?

timrowledge


> On 2019-09-16, at 3:52 PM, Chris Muller <[hidden email]> wrote:
>
> Hi Alan,
>
> The FFI package with tests is available in Squeak 5.2 via the SqueakMap browser.

And let's not forget the recently discussed
`Installer new merge: #ffi`
approach!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
My computer NEVER cras



Reply | Threaded
Open this post in threaded view
|

Re: state of FFI?

marcel.taeumel
Or that checkbox in the "PreferenceWizardMorph open":


Best,
Marcel

Am 17.09.2019 00:57:29 schrieb tim Rowledge <[hidden email]>:



> On 2019-09-16, at 3:52 PM, Chris Muller wrote:
>
> Hi Alan,
>
> The FFI package with tests is available in Squeak 5.2 via the SqueakMap browser.

And let's not forget the recently discussed
`Installer new merge: #ffi`
approach!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
My computer NEVER cras





Reply | Threaded
Open this post in threaded view
|

Re: state of FFI?

Hannes Hirzel
What does this checkbox in the "PreferenceWizardMorph open" do?
I'd like to put that information on the wiki.

--Hannes



On 9/17/19, Marcel Taeumel <[hidden email]> wrote:

> Or that checkbox in the "PreferenceWizardMorph open":
>
>
> Best,
> Marcel
> Am 17.09.2019 00:57:29 schrieb tim Rowledge <[hidden email]>:
>
>
>> On 2019-09-16, at 3:52 PM, Chris Muller wrote:
>>
>> Hi Alan,
>>
>> The FFI package with tests is available in Squeak 5.2 via the SqueakMap
>> browser.
>
> And let's not forget the recently discussed
> `Installer new merge: #ffi`
> approach!
>
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> My computer NEVER cras
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: state of FFI?

marcel.taeumel
I mean that you can also install FFI support through the preference wizard. :-) It should be straightforward. It does this:

Metacello new
configuration: 'FFI';
load.

Best,
Marcel

Am 17.09.2019 11:30:01 schrieb H. Hirzel <[hidden email]>:

What does this checkbox in the "PreferenceWizardMorph open" do?
I'd like to put that information on the wiki.

--Hannes



On 9/17/19, Marcel Taeumel wrote:
> Or that checkbox in the "PreferenceWizardMorph open":
>
>
> Best,
> Marcel
> Am 17.09.2019 00:57:29 schrieb tim Rowledge :
>
>
>> On 2019-09-16, at 3:52 PM, Chris Muller wrote:
>>
>> Hi Alan,
>>
>> The FFI package with tests is available in Squeak 5.2 via the SqueakMap
>> browser.
>
> And let's not forget the recently discussed
> `Installer new merge: #ffi`
> approach!
>
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> My computer NEVER cras
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: state of FFI?

Hannes Hirzel
Thank you

https://wiki.squeak.org/squeak/2426

Could you also please put this on the manual testing list (different platforms).

--Hannes

On 9/17/19, Marcel Taeumel <[hidden email]> wrote:

> I mean that you can also install FFI support through the preference wizard.
> :-) It should be straightforward. It does this:
>
> Metacello new
> configuration: 'FFI';
> load.
>
> Best,
> Marcel
> Am 17.09.2019 11:30:01 schrieb H. Hirzel <[hidden email]>:
> What does this checkbox in the "PreferenceWizardMorph open" do?
> I'd like to put that information on the wiki.
>
> --Hannes
>
>
>
> On 9/17/19, Marcel Taeumel wrote:
>> Or that checkbox in the "PreferenceWizardMorph open":
>>
>>
>> Best,
>> Marcel
>> Am 17.09.2019 00:57:29 schrieb tim Rowledge :
>>
>>
>>> On 2019-09-16, at 3:52 PM, Chris Muller wrote:
>>>
>>> Hi Alan,
>>>
>>> The FFI package with tests is available in Squeak 5.2 via the SqueakMap
>>> browser.
>>
>> And let's not forget the recently discussed
>> `Installer new merge: #ffi`
>> approach!
>>
>>
>> tim
>> --
>> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
>> My computer NEVER cras
>>
>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Nicolas Cellier
In reply to this post by Nicolas Cellier
A typical debug session: (maybe I shall remove -fsanitize=undefined, it's verbose and mostly benign...)
Note the assertion failures before 1st crash, but none before 2nd & 3rd crash...
The crash happens invariably in first few runs of SocketTest suite... But it takes varying disguise...

Last login: Tue Sep 17 20:18:40 on console
MBP-de-Nicolas:~ nicolas$ cd Smalltalk/OpenSmalltalk/opensmalltalk-vm/image
MBP-de-Nicolas:image nicolas$ lldb ../build.macos64x64/squeak.cog.spur/SqueakDebug.app/Contents/MacOS/Squeak
(lldb) target create "../build.macos64x64/squeak.cog.spur/SqueakDebug.app/Contents/MacOS/Squeak"
Current executable set to '../build.macos64x64/squeak.cog.spur/SqueakDebug.app/Contents/MacOS/Squeak' (x86_64).
(lldb) run trunk6-64.image
Process 701 launched: '../build.macos64x64/squeak.cog.spur/SqueakDebug.app/Contents/MacOS/Squeak' (x86_64)
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.
(lldb) cont
Process 701 resuming
../../platforms/Cross/vm/sqMemoryAccess.h:141:59: runtime error: load of misaligned address 0x000108c01ca1 for type 'sqInt' (aka 'long'), which requires 8 byte alignment
0x000108c01ca1: note: pointer points here
 75 68 49  b9 0d f0 ea 05 00 00 00  00 90 e9 7d e1 0a 00 49  b9 51 a5 ad 0b 00 00 00  00 90 3d 16 1f
              ^
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.
(lldb)  
Process 701 resuming
../../platforms/Cross/vm/sqMemoryAccess.h:139:66: runtime error: load of misaligned address 0x000108c01cab for type 'int', which requires 4 byte alignment
0x000108c01cab: note: pointer points here
 00  00 90 e9 7d e1 0a 00 49  b9 51 a5 ad 0b 00 00 00  00 90 3d 16 1f be ba 0f  84 77 e1 0a 00 49 b9
              ^
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.
(lldb)  
Process 701 resuming
../../platforms/Cross/vm/sqMemoryAccess.h:142:71: runtime error: store to misaligned address 0x000108c01cb1 for type 'sqInt' (aka 'long'), which requires 8 byte alignment
0x000108c01cb1: note: pointer points here
 0a 00 49  b9 51 a5 ad 0b 00 00 00  00 90 3d 16 1f be ba 0f  84 77 e1 0a 00 49 b9 52  a5 ad 0b 00 00
              ^
Process 701 stopped
* thread #11, name = 'com.apple.coreaudio.AQClient', stop reason = Invalid bool load
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.
(lldb) cont
Process 701 resuming
<unknown>: runtime error: load of value 110, which is not a valid value for type 'BOOL' (aka 'signed char')
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.
(lldb) cont
Process 701 resuming
../../platforms/iOS/vm/Common/Classes/sqSqueakFileDirectoryInterface.m:379:78: runtime error: load of misaligned address 0x000100528a37 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0x000100528a37: note: pointer points here
 67 69 6e 00 53  54 69 6d 00 46 41 53 54  00 21 28 69 73 4f 6f 70  46 6f 72 77 61 72 64 65  64 28 61
             ^
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.

Launched SocketTest suite run at image side... Note: i add to do it 3 or 4 times before crash

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
  * frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
    frame #1: 0x0000000100905f6c libclang_rt.ubsan_osx_dynamic.dylib`__ubsan::Diag::~Diag() + 140
    frame #2: 0x0000000100907a30 libclang_rt.ubsan_osx_dynamic.dylib`handleTypeMismatchImpl(__ubsan::TypeMismatchData*, unsigned long, __ubsan::ReportOptions) + 1216
    frame #3: 0x0000000100907564 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_handle_type_mismatch_v1 + 68
    frame #4: 0x00000001004aebeb Squeak`nameToAddr(hostName="localhost") at sqUnixSocket.c:282
    frame #5: 0x00000001004bbe5d Squeak`sqResolverStartNameLookup(hostName="localhost", nameSize=9) at sqUnixSocket.c:1553
    frame #6: 0x00000001004aa7b1 Squeak`primitiveResolverStartNameLookup at SocketPlugin.c:738
    frame #7: 0x000000010006a34d Squeak`primitiveExternalCall at gcc3x-cointerp.c:76406
    frame #8: 0x000000010005c97f Squeak`interpretMethodFromMachineCode at gcc3x-cointerp.c:17729
    frame #9: 0x00000001000664da Squeak`ceSendsupertonumArgs(selector=4454304704, superNormalBar=0, rcvr=4455972808, numArgs=1) at gcc3x-cointerp.c:15448
    frame #10: 0x0000000108c0014d
    frame #11: 0x0000000100001831 Squeak`interpret at gcc3x-cointerp.c:2767
    frame #12: 0x000000010027efb7 Squeak`-[sqSqueakMainApplication runSqueak](self=0x0000000102026740, _cmd="runSqueak") at sqSqueakMainApplication.m:201
    frame #13: 0x00007fff44b077b8 Foundation`__NSFirePerformWithOrder + 360
    frame #14: 0x00007fff4297cf57 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
    frame #15: 0x00007fff4297ce7f CoreFoundation`__CFRunLoopDoObservers + 527
    frame #16: 0x00007fff4295f3f8 CoreFoundation`__CFRunLoopRun + 1240
    frame #17: 0x00007fff4295ec93 CoreFoundation`CFRunLoopRunSpecific + 483
    frame #18: 0x00007fff41c49d96 HIToolbox`RunCurrentEventLoopInMode + 286
    frame #19: 0x00007fff41c49a0f HIToolbox`ReceiveNextEventCommon + 366
    frame #20: 0x00007fff41c49884 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 64
    frame #21: 0x00007fff3fef9a73 AppKit`_DPSNextEvent + 2085
    frame #22: 0x00007fff4068fe34 AppKit`-[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044
    frame #23: 0x00007fff3feee885 AppKit`-[NSApplication run] + 764
    frame #24: 0x00007fff3febda72 AppKit`NSApplicationMain + 804
    frame #25: 0x0000000100275ce6 Squeak`main(argc=2, argv=0x00007ffeefbff9f8, envp=0x00007ffeefbffa10) at main.m:74
    frame #26: 0x00007fff6a88c015 libdyld.dylib`start + 1
    frame #27: 0x00007fff6a88c015 libdyld.dylib`start + 1
(lldb) cont
Process 701 resuming
../../platforms/unix/plugins/SocketPlugin/sqUnixSocket.c:282:12: runtime error: load of misaligned address 0x00010179bafa for type 'char *', which requires 8 byte alignment
0x00010179bafa: note: pointer points here
 00 00  00 00 0a bb 79 01 01 00  00 00 00 00 00 00 00 00  00 00 7f 00 00 01 00 00  ba 9b 17 10 00 00
              ^
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
    frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report:
->  0x10090b230 <+0>: pushq  %rbp
    0x10090b231 <+1>: movq   %rsp, %rbp
    0x10090b234 <+4>: popq   %rbp
    0x10090b235 <+5>: retq  
Target 0: (Squeak) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
  * frame #0: 0x000000010090b230 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_on_report
    frame #1: 0x0000000100905f6c libclang_rt.ubsan_osx_dynamic.dylib`__ubsan::Diag::~Diag() + 140
    frame #2: 0x0000000100907a30 libclang_rt.ubsan_osx_dynamic.dylib`handleTypeMismatchImpl(__ubsan::TypeMismatchData*, unsigned long, __ubsan::ReportOptions) + 1216
    frame #3: 0x0000000100907564 libclang_rt.ubsan_osx_dynamic.dylib`__ubsan_handle_type_mismatch_v1 + 68
    frame #4: 0x00000001004aec2b Squeak`nameToAddr(hostName="localhost") at sqUnixSocket.c:282
    frame #5: 0x00000001004bbe5d Squeak`sqResolverStartNameLookup(hostName="localhost", nameSize=9) at sqUnixSocket.c:1553
    frame #6: 0x00000001004aa7b1 Squeak`primitiveResolverStartNameLookup at SocketPlugin.c:738
    frame #7: 0x000000010006a34d Squeak`primitiveExternalCall at gcc3x-cointerp.c:76406
    frame #8: 0x000000010005c97f Squeak`interpretMethodFromMachineCode at gcc3x-cointerp.c:17729
    frame #9: 0x00000001000664da Squeak`ceSendsupertonumArgs(selector=4454304704, superNormalBar=0, rcvr=4455972808, numArgs=1) at gcc3x-cointerp.c:15448
    frame #10: 0x0000000108c0014d
    frame #11: 0x0000000100001831 Squeak`interpret at gcc3x-cointerp.c:2767
    frame #12: 0x000000010027efb7 Squeak`-[sqSqueakMainApplication runSqueak](self=0x0000000102026740, _cmd="runSqueak") at sqSqueakMainApplication.m:201
    frame #13: 0x00007fff44b077b8 Foundation`__NSFirePerformWithOrder + 360
    frame #14: 0x00007fff4297cf57 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
    frame #15: 0x00007fff4297ce7f CoreFoundation`__CFRunLoopDoObservers + 527
    frame #16: 0x00007fff4295f3f8 CoreFoundation`__CFRunLoopRun + 1240
    frame #17: 0x00007fff4295ec93 CoreFoundation`CFRunLoopRunSpecific + 483
    frame #18: 0x00007fff41c49d96 HIToolbox`RunCurrentEventLoopInMode + 286
    frame #19: 0x00007fff41c49a0f HIToolbox`ReceiveNextEventCommon + 366
    frame #20: 0x00007fff41c49884 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 64
    frame #21: 0x00007fff3fef9a73 AppKit`_DPSNextEvent + 2085
    frame #22: 0x00007fff4068fe34 AppKit`-[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044
    frame #23: 0x00007fff3feee885 AppKit`-[NSApplication run] + 764
    frame #24: 0x00007fff3febda72 AppKit`NSApplicationMain + 804
    frame #25: 0x0000000100275ce6 Squeak`main(argc=2, argv=0x00007ffeefbff9f8, envp=0x00007ffeefbffa10) at main.m:74
    frame #26: 0x00007fff6a88c015 libdyld.dylib`start + 1
    frame #27: 0x00007fff6a88c015 libdyld.dylib`start + 1
(lldb) cont
Process 701 resuming
../../platforms/unix/plugins/SocketPlugin/sqUnixSocket.c:282:12: runtime error: load of misaligned address 0x00010179bb0a for type 'long', which requires 8 byte alignment
0x00010179bb0a: note: pointer points here
 00 00  00 00 7f 00 00 01 00 00  ba 9b 17 10 00 00 00 60  9a 80 17 10 00 00 00 30  5a 54 55 4d 00 00
              ^

kosherYoungReferrers() 11597

allMethodsHaveCorrectHeader() 4543

((gcMode & (GCModeFull + GCModeNewSpace)) == 0) || (kosherYoungReferrers()) 4545

kosherYoungReferrers() 11597

allMethodsHaveCorrectHeader() 4543

((gcMode & (GCModeFull + GCModeNewSpace)) == 0) || (kosherYoungReferrers()) 4545

kosherYoungReferrers() 11597

allMethodsHaveCorrectHeader() 4543

((gcMode & (GCModeFull + GCModeNewSpace)) == 0) || (kosherYoungReferrers()) 4545

kosherYoungReferrers() 11597

allMethodsHaveCorrectHeader() 4543

((gcMode & (GCModeFull + GCModeNewSpace)) == 0) || (kosherYoungReferrers()) 4545
Process 701 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x0000000108c7413c
Target 0: (Squeak) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x0000000108c7413c
    frame #1: 0x0000000108c70a39
    frame #2: 0x0000000108c80375
    frame #3: 0x0000000108c89b1d
    frame #4: 0x0000000108c759d6
    frame #5: 0x0000000108c0e25d
    frame #6: 0x0000000108c90505
    frame #7: 0x0000000108c7585f
    frame #8: 0x0000000108c88a04
    frame #9: 0x0000000108c88906
    frame #10: 0x0000000108c9197d
(lldb) call printStackCallStackOf($rbp)
    0x7ffeefbc75f8 M MouseMoveEvent(MouseEvent)>asMouseLeave 0x1092cbb90: a(n) MouseMoveEvent
    0x7ffeefbc7640 M MouseOverHandler>processMouseOver: 0x10c2b2020: a(n) MouseOverHandler
    0x7ffeefbc7680 M HandMorph>handleEvent: 0x109c84f50: a(n) HandMorph
    0x7ffeefbc76d8 M HandMorph>processEvents 0x109c84f50: a(n) HandMorph
    0x7ffeefbc7710 M [] in WorldState>doOneCycleNowFor: 0x109bd3590: a(n) WorldState
    0x7ffeefbc7758 M Array(SequenceableCollection)>do: 0x10960c6c0: a(n) Array
    0x7ffeefbc7790 M WorldState>handsDo: 0x109bd3590: a(n) WorldState
    0x7ffeefbc77d0 M WorldState>doOneCycleNowFor: 0x109bd3590: a(n) WorldState
    0x7ffeefbc7808 M WorldState>doOneCycleFor: 0x109bd3590: a(n) WorldState
    0x7ffeefbc7840 M PasteUpMorph>doOneCycle 0x1099ed988: a(n) PasteUpMorph
    0x7ffeefbc7870 M [] in MorphicProject>spawnNewProcess 0x109da9f40: a(n) MorphicProject
(sqInt) $7 = 0
(lldb) disassemble -s 0x0000000108c70a39
    0x108c70a39: pushq  %rdx
    0x108c70a3a: movq   -0x30(%rbp), %rdx
    0x108c70a3e: popq   %r15
    0x108c70a40: movq   %r15, 0x10(%rdx)
    0x108c70a44: testb  $0x7, %r15b
    0x108c70a48: jne    0x108c70a6e
    0x108c70a4a: movabsq $0x1095e78e0, %rax        ; imm = 0x1095E78E0
    0x108c70a54: nop    
    0x108c70a55: cmpq   %rax, %rdx
(lldb) print whereIs(0x0000000108c70a39)
(char *) $8 = 0x0000000100546be5 " is in generated methods"
(lldb) call printCogMethodFor((void *)0x0000000108c70a39)
       0x108c706e8 <->        0x108c711d8: method:        0x10b39c2c0 selector:        0x10983ac58 processMouseOver:
(lldb) print whereIs(0x0000000108c7413c)
(char *) $9 = 0x0000000100546be5 " is in generated methods"
(lldb) call printCogMethodFor((void *)0x0000000108c7413c)
       0x108c73e80 <->        0x108c77e8a: Open PIC  selector:                0x0
(lldb) 

----
next session, crashed at first [SocketTest suite run] bench.
...snip the UBsan...

../../platforms/unix/plugins/SocketPlugin/sqUnixSocket.c:282:12: runtime error: load of misaligned address 0x0001021d18ca for type 'long', which requires 8 byte alignment
0x0001021d18ca: note: pointer points here
 00 00  00 00 7f 00 00 01 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  69 01 00 00 00 00
              ^
Process 824 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xd70)
    frame #0: 0x0000000108c84c88
->  0x108c84c88: addb   %al, (%rax)
    0x108c84c8a: addb   %al, (%rax)
    0x108c84c8c: addb   %al, (%rax)
    0x108c84c8e: addb   %al, (%rax)
Target 0: (Squeak) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xd70)
  * frame #0: 0x0000000108c84c88
    frame #1: 0x0000000108cb5ec7
    frame #2: 0x0000000108cb5549
    frame #3: 0x0000000108cb5c34
    frame #4: 0x0000000108cb5de8
    frame #5: 0x0000000108cb7e77
    frame #6: 0x0000000108cb81ae
    frame #7: 0x0000000108cb8110
    frame #8: 0x0000000108cb51e1
    frame #9: 0x0000000108bfc07e
    frame #10: 0x0000000108c82729
    frame #11: 0x0000000108cb5178
    frame #12: 0x0000000108bfc07e
    frame #13: 0x0000000108c829b1
    frame #14: 0x0000000108cb50a0
    frame #15: 0x0000000108bfc07e
    frame #16: 0x0000000108c82c39
    frame #17: 0x0000000108cb4f86
    frame #18: 0x0000000108c7fd55
    frame #19: 0x0000000108cbdda4
    frame #20: 0x0000000108c87d18
    frame #21: 0x0000000108c8c415
    frame #22: 0x0000000108c87636
    frame #23: 0x0000000108bfb4c5
    frame #24: 0x0000000108c8c045
    frame #25: 0x0000000108c874bf
    frame #26: 0x0000000108c8b0f4
    frame #27: 0x0000000108c8aff6
    frame #28: 0x0000000108c922fd
(lldb) call printStackCallStackOf($rbp)
    0x7ffeefbcd0d0 M TransformMorph(Morph)>transformedFrom: 0x10beed508: a(n) TransformMorph
    0x7ffeefbcd128 M [] in MorphicEventDispatcher>doBubblingForFocusEvent:with: 0x109234328: a(n) MorphicEventDispatcher
    0x7ffeefbcd160 M TransformMorph(Morph)>withAllOwnersDo: 0x10beed508: a(n) TransformMorph
    0x7ffeefbcd198 M TextMorphForEditView(Morph)>allOwnersDo: 0x10beed2f8: a(n) TextMorphForEditView
    0x7ffeefbcd1e0 M MorphicEventDispatcher>doBubblingForFocusEvent:with: 0x109234328: a(n) MorphicEventDispatcher
    0x7ffeefbcd228 M MorphicEventDispatcher>dispatchFocusEvent:with: 0x109234328: a(n) MorphicEventDispatcher
    0x7ffeefbcd268 M TextMorphForEditView(Morph)>processFocusEvent:using: 0x10beed2f8: a(n) TextMorphForEditView
    0x7ffeefbcd2a8 M TextMorphForEditView(Morph)>processFocusEvent: 0x10beed2f8: a(n) TextMorphForEditView
    0x7ffeefbcd2f8 M [] in HandMorph>sendFocusEvent:to:clear: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd338 M BlockClosure>ensure: 0x109234210: a(n) BlockClosure
    0x7ffeefbcd378 M KeyboardEvent(MorphicEvent)>becomeActiveDuring: 0x109233a58: a(n) KeyboardEvent
    0x7ffeefbcd3c8 M [] in HandMorph>sendFocusEvent:to:clear: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd408 M BlockClosure>ensure: 0x109234040: a(n) BlockClosure
    0x7ffeefbcd448 M HandMorph>becomeActiveDuring: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd498 M [] in HandMorph>sendFocusEvent:to:clear: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd4d8 M BlockClosure>ensure: 0x109233e70: a(n) BlockClosure
    0x7ffeefbcd518 M PasteUpMorph>becomeActiveDuring: 0x1099dc988: a(n) PasteUpMorph
    0x7ffeefbcd560 M HandMorph>sendFocusEvent:to:clear: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd5b8 M HandMorph>sendEvent:focus:clear: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd600 M HandMorph>sendKeyboardEvent: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd640 M HandMorph>handleEvent: 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd698 M HandMorph>processEvents 0x109c73f50: a(n) HandMorph
    0x7ffeefbcd6d0 M [] in WorldState>doOneCycleNowFor: 0x109bc2590: a(n) WorldState
    0x7ffeefbcd718 M Array(SequenceableCollection)>do: 0x1095fb6c0: a(n) Array
    0x7ffeefbcd750 M WorldState>handsDo: 0x109bc2590: a(n) WorldState
    0x7ffeefbcd790 M WorldState>doOneCycleNowFor: 0x109bc2590: a(n) WorldState
    0x7ffeefbcd7c8 M WorldState>doOneCycleFor: 0x109bc2590: a(n) WorldState
    0x7ffeefbcd800 M PasteUpMorph>doOneCycle 0x1099dc988: a(n) PasteUpMorph
    0x7ffeefbcd830 M [] in MorphicProject>spawnNewProcess 0x109d98f40: a(n) MorphicProject
(sqInt) $6 = 0
(lldb) print whereIs(0x0000000108cb5ec7)
(char *) $7 = 0x0000000100546be5 " is in generated methods"
(lldb) call printCogMethodFor((void *)0x0000000108cb5ec7)
not a method
(lldb) print whereIs(0x0000000108cb5549)
(char *) $8 = 0x0000000100546be5 " is in generated methods"
(lldb) call printCogMethodFor((void *)0x0000000108cb5549)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108cb5c34)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108cb5de8)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108cb7e77)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108cb81ae)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108cb8110)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108cb51e1)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108bfc07e)
       0x108bfbff0 <->        0x108bfc0e0: method:        0x10b7e95f8 prim 198 selector:        0x109811660 ensure:
(lldb) call printCogMethodFor((void *)0x0000000108c84c88)
not a method
(lldb) disass -s 0x0000000108cb5ec7
    0x108cb5ec7: movq   %rdx, %rdi
    0x108cb5eca: popq   %rdx
    0x108cb5ecb: movq   $0xd62, %rcx              ; imm = 0xD62
    0x108cb5ed2: callq  0x108cbae28
    0x108cb5ed7: pushq  %rdx
    0x108cb5ed8: movq   -0x30(%rbp), %rdx
    0x108cb5edc: popq   %r15
    0x108cb5ede: movq   %r15, 0x18(%rdx)
    0x108cb5ee2: testb  $0x7, %r15b

----

next session, crashed at third [SocketTest suite run] bench

Process 907 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000108ce5364
->  0x108ce5364: addb   %al, (%rax)
    0x108ce5366: addb   %al, (%rax)
    0x108ce5368: addb   %al, (%rax)
    0x108ce536a: addb   %al, (%rax)
Target 0: (Squeak) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000108ce5364
    frame #1: 0x0000000108ce5539
    frame #2: 0x0000000108ce4faf
    frame #3: 0x0000000108ce0105
    frame #4: 0x0000000108cd41ee
    frame #5: 0x0000000108c0ebb6
    frame #6: 0x0000000108cd4bb5
    frame #7: 0x0000000108c0dfa6
    frame #8: 0x0000000108cd49ad
    frame #9: 0x0000000108cd413c
(lldb) call printStackCallStackOf($rbp)
    0x7ffeefbe85e8 M Socket>(nil) 0x109001e10: a(n) Socket
    0x7ffeefbe8630 M Socket>sendSomeData: 0x109001e10: a(n) Socket
    0x7ffeefbe8678 M SocketTest>testSendTimeout 0x108fac4d0: a(n) SocketTest
    0x7ffeefbe86a8 M SocketTest(TestCase)>performTest 0x108fac4d0: a(n) SocketTest
    0x7ffeefbe86d8 M [] in SocketTest(TestCase)>runCase 0x108fac4d0: a(n) SocketTest
    0x7ffeefbe8710 M BlockClosure>on:do: 0x108ffed20: a(n) BlockClosure
    0x7ffeefbe8760 M [] in SocketTest(TestCase)>timeout:after: 0x108fac4d0: a(n) SocketTest
    0x7ffeefbe87a0 M BlockClosure>ensure: 0x108fff630: a(n) BlockClosure
    0x7ffeefbe87f0 M SocketTest(TestCase)>timeout:after: 0x108fac4d0: a(n) SocketTest
    0x7ffeefbe8830 M [] in SocketTest(TestCase)>runCase 0x108fac4d0: a(n) SocketTest
    0x7ffeefbd41d8 M BlockClosure>ensure: 0x108ffcd98: a(n) BlockClosure
    0x7ffeefbd4210 M SocketTest(TestCase)>runCase 0x108fac4d0: a(n) SocketTest
    0x7ffeefbd4248 M [] in TestResult>runCase: 0x108fac668: a(n) TestResult
    0x7ffeefbd4280 M Time class>microsecondsToRun: 0x10998a4e0: a(n) Time class
    0x7ffeefbd42b8 M Time class>millisecondsToRun: 0x10998a4e0: a(n) Time class
    0x7ffeefbd42f0 M BlockClosure>timeToRun 0x108ffccb8: a(n) BlockClosure
    0x7ffeefbd4328 M BlockClosure>timeToRunWithoutGC 0x108ffccb8: a(n) BlockClosure
    0x7ffeefbd4368 M [] in TestResult>runCase: 0x108fac668: a(n) TestResult
    0x7ffeefbd43a0 M BlockClosure>on:do: 0x108ffca88: a(n) BlockClosure
    0x7ffeefbd43f0 M [] in BlockClosure>on:do:on:do: 0x108ffca88: a(n) BlockClosure
    0x7ffeefbd4428 M BlockClosure>on:do: 0x108ffcbd0: a(n) BlockClosure
    0x7ffeefbd4468 M BlockClosure>on:do:on:do: 0x108ffca88: a(n) BlockClosure
    0x7ffeefbd44c0 M TestResult>runCase: 0x108fac668: a(n) TestResult
    0x7ffeefbd44f8 M SocketTest(TestCase)>run: 0x108fac4d0: a(n) SocketTest
    0x7ffeefbd4538 M [] in TestSuite>run: 0x108fa36d0: a(n) TestSuite
    0x7ffeefbd4578 M OrderedCollection>do: 0x108fac2d8: a(n) OrderedCollection
    0x7ffeefbd45b0 M TestSuite>run: 0x108fa36d0: a(n) TestSuite
    0x7ffeefbd45f0 M [] in TestSuite>run 0x108fa36d0: a(n) TestSuite
    0x7ffeefbd4630 M BlockClosure>ensure: 0x108faed40: a(n) BlockClosure
    0x7ffeefbd4670 M TestSuite>run 0x108fa36d0: a(n) TestSuite
    0x7ffeefbd46a0 M [] in UndefinedObject>DoIt 0x1095e78e0: a(n) UndefinedObject
    0x7ffeefbd4700 M BlockClosure>benchFor: 0x108fa2dd0: a(n) BlockClosure
    0x7ffeefbd4748 I BlockClosure>bench 0x108fa2dd0: a(n) BlockClosure
    0x7ffeefbd4778 M UndefinedObject>DoIt 0x1095e78e0: a(n) UndefinedObject
    0x7ffeefbd47d0 I Compiler>evaluateCue:ifFail: 0x108de4cc0: a(n) Compiler
    0x7ffeefbd4818 M Compiler>evaluateCue:ifFail:logged: 0x108de4cc0: a(n) Compiler
    0x7ffeefbf1248 I Compiler>evaluate:in:to:environment:notifying:ifFail:logged: 0x108de4cc0: a(n) Compiler
    0x7ffeefbf12c0 M [] in SmalltalkEditor(TextEditor)>evaluateSelectionAndDo: 0x10c0445c0: a(n) SmalltalkEditor
    0x7ffeefbf12f8 M BlockClosure>on:do: 0x108de6a28: a(n) BlockClosure
    0x7ffeefbf1350 M SmalltalkEditor(TextEditor)>evaluateSelectionAndDo: 0x10c0445c0: a(n) SmalltalkEditor
    0x7ffeefbf1388 M SmalltalkEditor(TextEditor)>printIt 0x10c0445c0: a(n) SmalltalkEditor
    0x7ffeefbf13b8 M SmalltalkEditor(TextEditor)>printIt: 0x10c0445c0: a(n) SmalltalkEditor
    0x7ffeefbf1410 I SmalltalkEditor(TextEditor)>dispatchOnKeyboardEvent: 0x10c0445c0: a(n) SmalltalkEditor
    0x7ffeefbf1458 I SmalltalkEditor(TextEditor)>keyStroke: 0x10c0445c0: a(n) SmalltalkEditor
    0x7ffeefbf1498 M [] in TextMorphForEditView(TextMorph)>keyStroke: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf14e8 M TextMorphForEditView(TextMorph)>handleInteraction:fromEvent: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf1528 M TextMorphForEditView>handleInteraction:fromEvent: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf1578 M [] in TextMorphForEditView(TextMorph)>keyStroke: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf15b8 I StandardToolSet class>codeCompletionAround:textMorph:keyStroke: 0x1099a1398: a(n) StandardToolSet class
    0x7ffeefbf1600 M ToolSet class>codeCompletionAround:textMorph:keyStroke: 0x109999d08: a(n) ToolSet class
    0x7ffeefbf1648 M TextMorphForEditView(TextMorph)>keyStroke: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf1698 I TextMorphForEditView>keyStroke: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf16e8 I TextMorphForEditView(Morph)>handleKeystroke: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf1728 M TextMorphForEditView(TextMorph)>handleKeystroke: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf1760 M KeyboardEvent>sentTo: 0x108de6c08: a(n) KeyboardEvent
    0x7ffeefbf17a0 M TextMorphForEditView(Morph)>handleEvent: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf17d8 M TextMorphForEditView(Morph)>handleFocusEvent: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbf1820 M MorphicEventDispatcher>doHandlingForFocusEvent:with: 0x108de4170: a(n) MorphicEventDispatcher
    0x7ffeefbd1228 M MorphicEventDispatcher>dispatchFocusEvent:with: 0x108de4170: a(n) MorphicEventDispatcher
    0x7ffeefbd1268 M TextMorphForEditView(Morph)>processFocusEvent:using: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbd12a8 M TextMorphForEditView(Morph)>processFocusEvent: 0x10befe2f8: a(n) TextMorphForEditView
    0x7ffeefbd12f8 M [] in HandMorph>sendFocusEvent:to:clear: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd1338 M BlockClosure>ensure: 0x108de4250: a(n) BlockClosure
    0x7ffeefbd1378 M KeyboardEvent(MorphicEvent)>becomeActiveDuring: 0x108de3110: a(n) KeyboardEvent
    0x7ffeefbd13c8 M [] in HandMorph>sendFocusEvent:to:clear: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd1408 M BlockClosure>ensure: 0x108de4420: a(n) BlockClosure
    0x7ffeefbd1448 M HandMorph>becomeActiveDuring: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd1498 M [] in HandMorph>sendFocusEvent:to:clear: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd14d8 M BlockClosure>ensure: 0x108de45f0: a(n) BlockClosure
    0x7ffeefbd1518 M PasteUpMorph>becomeActiveDuring: 0x1099ed988: a(n) PasteUpMorph
    0x7ffeefbd1560 M HandMorph>sendFocusEvent:to:clear: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd15b8 M HandMorph>sendEvent:focus:clear: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd1600 M HandMorph>sendKeyboardEvent: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd1640 M HandMorph>handleEvent: 0x109c84f50: a(n) HandMorph
    0x7ffeefbd1698 M HandMorph>processEvents 0x109c84f50: a(n) HandMorph
    0x7ffeefbd16d0 M [] in WorldState>doOneCycleNowFor: 0x109bd3590: a(n) WorldState
    0x7ffeefbd1718 M Array(SequenceableCollection)>do: 0x10960c6c0: a(n) Array
    0x7ffeefbd1750 M WorldState>handsDo: 0x109bd3590: a(n) WorldState
    0x7ffeefbd1790 M WorldState>doOneCycleNowFor: 0x109bd3590: a(n) WorldState
    0x7ffeefbd17c8 M WorldState>doOneCycleFor: 0x109bd3590: a(n) WorldState
    0x7ffeefbd1800 M PasteUpMorph>doOneCycle 0x1099ed988: a(n) PasteUpMorph
    0x7ffeefbd1830 M [] in MorphicProject>spawnNewProcess 0x109da9f40: a(n) MorphicProject
(sqInt) $7 = 0
(lldb) call printCogMethodFor((void *)0x0000000108ce5539)
not a method
(lldb) call printCogMethodFor((void *)0x0000000108ce4faf)
       0x108ce4e70 <->        0x108ce52b0: method:        0x10bf04548 selector:        0x1095e78e0 (nil: testSendTimeout)
(lldb) disass -s 0x0000000108ce4faf
    0x108ce4faf: addb   %al, (%rax)
    0x108ce4fb1: addb   %al, (%rax)
    0x108ce4fb3: addb   %al, (%rax)
    0x108ce4fb5: addb   %al, (%rax)
    0x108ce4fb7: addb   %al, (%rax)
    0x108ce4fb9: addb   %al, (%rax)
    0x108ce4fbb: addb   %al, (%rax)
    0x108ce4fbd: addb   %al, 0x2(%rdx)
    0x108ce4fc3: addb   %al, (%rax)
    0x108ce4fc5: addb   %ch, (%rax)
    0x108ce4fc7: pushq  %rcx
    0x108ce4fc8: cmpsl  %es:(%rdi), (%rsi)
    0x108ce4fc9: pushfq
(lldb) call printCogMethodFor((void *)0x0000000108ce0105)
       0x108ce0068 <->        0x108ce0118: method:        0x10b43fe40 selector:        0x109863ea0 performTest
(lldb) disass -s 0x0000000108ce5539
    0x108ce5539: movq   %rbp, %rsp
    0x108ce553c: popq   %rbp
    0x108ce553d: retq   $0x10
    0x108ce5540: int3  
    0x108ce5541: int3  
    0x108ce5542: int3  
    0x108ce5543: addb   %bh, %al
    0x108ce5545: cld    
    0x108ce5546: movb   $0x78, %dh
    0x108ce5548: addb   %al, (%rax)
    0x108ce554a: addb   %al, (%rax)
    0x108ce554c: addb   %al, (%rax)
    0x108ce554e: addb   %al, (%rax)
    0x108ce5550: addl   %esp, (%rax,%rcx,2)
    0x108ce5554: fadds  (%rax)
    0x108ce5556: addb   %al, (%rax)



Le lun. 16 sept. 2019 à 23:48, Nicolas Cellier <[hidden email]> a écrit :
Attached a bunch of crash dumps, some of them with same stack as your last example...
You definitely want my internet connection (or not...)!

Le lun. 16 sept. 2019 à 23:05, Nicolas Cellier <[hidden email]> a écrit :
Forgot to say: it happens in spur only, not stack.

Le lun. 16 sept. 2019 à 22:52, Nicolas Cellier <[hidden email]> a écrit :
No idea if this is related or not,
but i got regular crash on macos own compiled x64 artefact by just running:
    SocketTest suite run.

I first though about inquiring some UB, but this happens in debug version, so it's probably something else.
I did not try to simulate (but can we simulate socket tests?).
This does not happen in windows 64 nor linux 64 (wsl) where I can run all the tests.
Note that on wsl i had to evaluate (Compiler recompileAll) in trunk6-64.image, otherwise a bunch of tests are failing... Mysterious.
I also had 2 ByteSymbol differing from their interned version.

Le sam. 14 sept. 2019 à 09:33, Tobias Pape <[hidden email]> a écrit :

> On 14.09.2019, at 06:03, Nicola Mingotti <[hidden email]> wrote:
>
>
> I can help you a bit only on this point:
> "- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yea it is called "network link conditioner.prefpane" :D


>
> Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.
>
> Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
> macOS> apropos dummynet
> dummynet(4) ....
>
> So, yes, it is there.
>
> HTH
>
> bye
> Nicola
>
>
>
>

>
>
>
>
> On 9/13/19 8:15 PM, Eliot Miranda wrote:
>> Hi All,
>>
>>     there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.
>>
>> This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.
>>
>> This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.
>>
>> - is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?
>>
>> - is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?
>>
>> - is there a way of introducing network delays in Mac OS that might help me induce the bug?
>>
>> - can anyone think of any other strategies I might take to try and reproduce this?
>>
>> I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?
>>
>> Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).
>>
>> (lldb) thr b
>> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>>   * frame #0: 0x000000010de5700a
>>     frame #1: 0x000000010dd7b174
>>     frame #2: 0x000000010dd45f1c
>>     frame #3: 0x000000010dd44534
>>     frame #4: 0x000000010dd44c60
>> (lldb) x/10i 0x000000010de5700a
>>
>> (lldb) call printStackCallStackOf($rbp)
>>     0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class
>>
>> (lldb) x/10i 0x000000010dd7b174
>>     0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx
>>     0x10dd7b178: 48 89 ec     movq   %rbp, %rsp
>>     0x10dd7b17b: 5d           popq   %rbp
>>     0x10dd7b17c: c2 10 00     retq   $0x10
>>     0x10dd7b17f: cc           int3
>>     0x10dd7b180: cc           int3
>>     0x10dd7b181: cc           int3
>>     0x10dd7b182: cc           int3
>>     0x10dd7b183: cc           int3
>>     0x10dd7b184: cc           int3
>> (lldb) print whereIs(0x000000010dd7b174)
>> (char *) $0 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010dd7b174)
>>        0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:
>> (lldb) print whereIs(0x000000010de5700a)
>> (char *) $1 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010de5700a)
>>        0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20
>>
>> this method ends up being the fitted version of Delay class>> startTimerEventLoop
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>





Reply | Threaded
Open this post in threaded view
|

Re: Difficult to debug VM crash with full blocks and Sista V1

Eliot Miranda-2
In reply to this post by Nicolas Cellier
Hi Nicolas,

On Mon, Sep 16, 2019 at 1:52 PM Nicolas Cellier <[hidden email]> wrote:
No idea if this is related or not,
but i got regular crash on macos own compiled x64 artefact by just running:
    SocketTest suite run.

When I try in my 64-bit Spur image with full blocks/SistaV1 it does not crash :-(.  What is the exact image configuration you're testing?  Can you perhaps zip the image and changes and mail them to me?
 

I first though about inquiring some UB, but this happens in debug version, so it's probably something else.
I did not try to simulate (but can we simulate socket tests?).
This does not happen in windows 64 nor linux 64 (wsl) where I can run all the tests.
Note that on wsl i had to evaluate (Compiler recompileAll) in trunk6-64.image, otherwise a bunch of tests are failing... Mysterious.
I also had 2 ByteSymbol differing from their interned version.

Le sam. 14 sept. 2019 à 09:33, Tobias Pape <[hidden email]> a écrit :

> On 14.09.2019, at 06:03, Nicola Mingotti <[hidden email]> wrote:
>
>
> I can help you a bit only on this point:
> "- is there a way of introducing network delays in Mac OS that might help me induce the bug?"

Yea it is called "network link conditioner.prefpane" :D


>
> Yes, in theory it is possible. Some time ago I red the documentation of 'dummynet' in FreeBSD for the firewall 'ipfw', it seemed to be very interesting but I never had occasion to use it.
>
> Now, Apple Unix is in large part taken from FreeBSD => I check if they took also dummynet:
> macOS> apropos dummynet
> dummynet(4) ....
>
> So, yes, it is there.
>
> HTH
>
> bye
> Nicola
>
>
>
>

>
>
>
>
> On 9/13/19 8:15 PM, Eliot Miranda wrote:
>> Hi All,
>>
>>     there is a VM bug in 64-bit Spur with the Sista V1 bytecode set and full blocks.  The symptom is that when waiting for a remote Monticello repository to update and/or deliver a package version the system crashes in JITTED code after what appears to be some kind of wait.
>>
>> This is a reliably occurring bug b ut maddeningly difficult to reproduce.  The bug reliably occurs when interacting with a remote rep[ository (e.g. http://source.squeak.org/VMMaker) when the server is "cold", and hence makes the image wait.  Every time I have tried to repeat the failing sequence the crash has not occurre3d, I think because the server is now "hot" and serves up the version quickly.  Today I even tried shutting down my machine for over an hour and rebooting.  But I could not get the crash to occur even though it seems to me that every time I try it the first time in the4 day it does crash.
>>
>> This is an important bug to fix.  If it cannot be fixed then full blocks and Sista V1 are not ready for use in the upcoming Squeak release.  I am looking for help in debugging this.
>>
>> - is anyone else uising the 64-bit VM with full blocks and Sista V1 who sees hard VM crashes?  If so, under what circumstances?
>>
>> - is it possible to flush caches in the http://source.squeak.org/VMMaker server, or could people tolerate me rebooting the server?
>>
>> - is there a way of introducing network delays in Mac OS that might help me induce the bug?
>>
>> - can anyone think of any other strategies I might take to try and reproduce this?
>>
>> I may have to try and reproduce e the bug in the simulator to have a chance of identifying the bug.  Does anyone have a good enough mental model of the Monticello server interaction and have energy to help me figure this one out?
>>
>> Here is some information from the last crash I did see in the debugger (alas it is incomplete; there are a number of additional pieces of info I could have collected).
>>
>> (lldb) thr b
>> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>>   * frame #0: 0x000000010de5700a
>>     frame #1: 0x000000010dd7b174
>>     frame #2: 0x000000010dd45f1c
>>     frame #3: 0x000000010dd44534
>>     frame #4: 0x000000010dd44c60
>> (lldb) x/10i 0x000000010de5700a
>>
>> (lldb) call printStackCallStackOf($rbp)
>>     0x7ffeefbdfc30 M Heap>upHeap: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfc68 M Heap>add: 0x11273ca90: a(n) Heap
>>     0x7ffeefbdfca0 M Delay class>scheduleDelay:from: 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfcf0 M Delay class>handleTimerEvent 0x1123ebfb8: a(n) Delay class
>>     0x7ffeefbdfd20 M Delay class>runTimerEventLoop 0x1123ebfb8: a(n) Delay class
>>
>> (lldb) x/10i 0x000000010dd7b174
>>     0x10dd7b174: 48 8b 55 10  movq   0x10(%rbp), %rdx
>>     0x10dd7b178: 48 89 ec     movq   %rbp, %rsp
>>     0x10dd7b17b: 5d           popq   %rbp
>>     0x10dd7b17c: c2 10 00     retq   $0x10
>>     0x10dd7b17f: cc           int3
>>     0x10dd7b180: cc           int3
>>     0x10dd7b181: cc           int3
>>     0x10dd7b182: cc           int3
>>     0x10dd7b183: cc           int3
>>     0x10dd7b184: cc           int3
>> (lldb) print whereIs(0x000000010dd7b174)
>> (char *) $0 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010dd7b174)
>>        0x10dd7afc0 <->        0x10dd7b198: method:        0x112f23c10 selector:        0x112232c20 add:
>> (lldb) print whereIs(0x000000010de5700a)
>> (char *) $1 = 0x00000001000f83ff " is in generated methods"
>> (lldb) call printCogMethodFor((void *)0x000000010de5700a)
>>        0x10de56ba0 <->        0x10de57078: method:        0x1126ec218 prim 23856 selector:     0x7ffeefbf3d20
>>
>> this method ends up being the fitted version of Delay class>> startTimerEventLoop
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>






--
_,,,^..^,,,_
best, Eliot