regression: TestObjectsAsMethods crash in 4.19.5 (not in 4.16.7)

>  
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
> As can be seen in the screenshot at:
>
> http://docs.openindiana.org/handbook/community/squeak/index.html
>
> I'm deselecting the Tests-ObjectsAsMethods test (1 test),
> because it causes (reproducible) SIGSEGV on Solaris cc/OpenIndiana gcc.
>
> I think the segmentation fault is new in recent 4.19, I think it didn't happen
> a while ago in 4.16.
>
> I can test this as follows: when I install an older version
>
> squeak -version
> 4.16.7-3775
>
> then I go into test runner: Tests-ObjectsAsMethods and select
> TestObjectsAsMethods that works in 4.16.7
>
> Test Runner
>
> ...
> TestObjectsAsMethods
>
> 3 run, 3 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes
>
> But it stopped working in 4.19.x which is from I believe from:
>
> ContextInterpreter VMMaker-dtl.422 uuid: e72b95a0-204e-45a1-a4e4-3ac3c9e7a51a
>
> the interp.c file is automatically generated from VMMaker-dtl.422.
>
> It's reproducible in the sense that if I deselect all tests, and just select
> that one single test, I can repeatedly and reproducible SIGSEGV the VM.
>
> When I run the VM under a debugger:
>
> dbx: warning: Bad transition in runtime linker interface. CONSISTENT->CONSISTENT
> t@1 (l@1) signal SEGV (no mapping at the fault address) in interpret at line 9120 in file "interp.c"
>  9120   foo->freeContexts = longAt((newContext + (BASE_HEADER_SIZE)) + (0 << (SHIFT_FOR_WORD)));
>
> (dbx) where
> current thread: t@1
> =>[1] interpret(), line 9120 in "interp.c"
>   [2] main(argc = 1, argv = 0xfeffe250, envp = 0xfeffe258), line 1484 in "sqUnixMain.c"
>
> The above is from Solaris with cc/dbx but the same thing appears to happen
> for me on OpenIndiana with gcc/gdb.
>
> Unfortunately because the code of interp.c is automatically generated,
> it looks complicated to me and I don't see what's wrong with those "
> freeContext" code.
>
> The crash appears to be in:
>
>                                 /* begin internalActivateNewMethod */
>                                 methodHeader = longAt((foo->newMethod + (BASE_HEADER_SIZE)) + (HeaderIndex << (SHIFT_FOR_WORD)));
>                                 needsLarge = methodHeader & LargeContextBit;
>                                 if ((needsLarge == 0) && (foo->freeContexts != NilContext)) {
>                                         newContext = foo->freeContexts;
>                                         /* begin setFreeContextsAfter: */
>                                         foo->freeContexts = longAt((newContext + (BASE_HEADER_SIZE)) + (0 << (SHIFT_FOR_WORD)));
>                                 } else {
>                                         /* begin externalizeIPandSP */
>
> Has anyone seen this ?
>
> Also what is the test
> TestObjectsAsMethods
> actually doing please ? what is it testing ?
>
>
> Regards,
> David Stes
>
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQEcBAEBCAAGBQJgMnV5AAoJEAwpOKXMq1MafEgH/3EWQxaSaVm2g4r/2p99Wc21
> P+U+ijqKpVTDfJ1smwV/GsgF0V8ZrZky0k7BzRDAyq3Gi/HGVm0e2bqOAKa1fo2Y
> MUS9JHOW4Lys+9qWgT0aLiWypjYlzThtYS0/Lfh013tsF1bBv2eppTceUyq/Zitv
> 6J0IFvDOspMN/zHwBw/ux3H6uR049boZ3mvk23sp3KIHDc2Yw2kF4TAXBwjZXmVO
> UFlIAC4EAahrtNZyLZSIBDbsXOl+wJGmQTsOIBG81pfSFpP6RBrIARcu6enZC3Wc
> bwsvWYADs49SKgVq3NBovfyzkZBIW30V82xlVKpOnp6A4FnOYXxQiVm9sNaOVXc=
> =34TJ
> -----END PGP SIGNATURE-----

>  
> Thanks.
>
> Confirmed on Linux amd64, so it is not Solaris/OpenIndiana issue.
>
> Dave
>
> On Sun, Feb 21, 2021 at 04:02:38PM +0100, [hidden email] wrote:
> >  
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA256
> >
> >
> > As can be seen in the screenshot at:
> >
> > http://docs.openindiana.org/handbook/community/squeak/index.html
> >
> > I'm deselecting the Tests-ObjectsAsMethods test (1 test),
> > because it causes (reproducible) SIGSEGV on Solaris cc/OpenIndiana gcc.
> >
> > I think the segmentation fault is new in recent 4.19, I think it didn't happen
> > a while ago in 4.16.
> >
> > I can test this as follows: when I install an older version
> >
> > squeak -version
> > 4.16.7-3775
> >
> > then I go into test runner: Tests-ObjectsAsMethods and select
> > TestObjectsAsMethods that works in 4.16.7
> >
> > Test Runner
> >
> > ...
> > TestObjectsAsMethods
> >
> > 3 run, 3 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes
> >
> > But it stopped working in 4.19.x which is from I believe from:
> >
> > ContextInterpreter VMMaker-dtl.422 uuid: e72b95a0-204e-45a1-a4e4-3ac3c9e7a51a
> >
> > the interp.c file is automatically generated from VMMaker-dtl.422.
> >
> > It's reproducible in the sense that if I deselect all tests, and just select
> > that one single test, I can repeatedly and reproducible SIGSEGV the VM.
> >
> > When I run the VM under a debugger:
> >
> > dbx: warning: Bad transition in runtime linker interface. CONSISTENT->CONSISTENT
> > t@1 (l@1) signal SEGV (no mapping at the fault address) in interpret at line 9120 in file "interp.c"
> >  9120   foo->freeContexts = longAt((newContext + (BASE_HEADER_SIZE)) + (0 << (SHIFT_FOR_WORD)));
> >
> > (dbx) where
> > current thread: t@1
> > =>[1] interpret(), line 9120 in "interp.c"
> >   [2] main(argc = 1, argv = 0xfeffe250, envp = 0xfeffe258), line 1484 in "sqUnixMain.c"
> >
> > The above is from Solaris with cc/dbx but the same thing appears to happen
> > for me on OpenIndiana with gcc/gdb.
> >
> > Unfortunately because the code of interp.c is automatically generated,
> > it looks complicated to me and I don't see what's wrong with those "
> > freeContext" code.
> >
> > The crash appears to be in:
> >
> >                                 /* begin internalActivateNewMethod */
> >                                 methodHeader = longAt((foo->newMethod + (BASE_HEADER_SIZE)) + (HeaderIndex << (SHIFT_FOR_WORD)));
> >                                 needsLarge = methodHeader & LargeContextBit;
> >                                 if ((needsLarge == 0) && (foo->freeContexts != NilContext)) {
> >                                         newContext = foo->freeContexts;
> >                                         /* begin setFreeContextsAfter: */
> >                                         foo->freeContexts = longAt((newContext + (BASE_HEADER_SIZE)) + (0 << (SHIFT_FOR_WORD)));
> >                                 } else {
> >                                         /* begin externalizeIPandSP */
> >
> > Has anyone seen this ?
> >
> > Also what is the test
> > TestObjectsAsMethods
> > actually doing please ? what is it testing ?
> >
> >
> > Regards,
> > David Stes
> >
> >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2
> >
> > iQEcBAEBCAAGBQJgMnV5AAoJEAwpOKXMq1MafEgH/3EWQxaSaVm2g4r/2p99Wc21
> > P+U+ijqKpVTDfJ1smwV/GsgF0V8ZrZky0k7BzRDAyq3Gi/HGVm0e2bqOAKa1fo2Y
> > MUS9JHOW4Lys+9qWgT0aLiWypjYlzThtYS0/Lfh013tsF1bBv2eppTceUyq/Zitv
> > 6J0IFvDOspMN/zHwBw/ux3H6uR049boZ3mvk23sp3KIHDc2Yw2kF4TAXBwjZXmVO
> > UFlIAC4EAahrtNZyLZSIBDbsXOl+wJGmQTsOIBG81pfSFpP6RBrIARcu6enZC3Wc
> > bwsvWYADs49SKgVq3NBovfyzkZBIW30V82xlVKpOnp6A4FnOYXxQiVm9sNaOVXc=
> > =34TJ
> > -----END PGP SIGNATURE-----

>  
> > Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> > replacing obsolete primitiveInvokeObjectAsMethod.
>
> Any news on this Tests-ObjectsAsMethods issue ?
>
> The purpose of those tests are not clear to me;
>
> Perhaps the change for the "primitives" which was made, can be reverted ?
>
> Possibly that creates a new problem for primitiveArrayBecomeOneWayNoCopyHash.
>
> David Stes
>

>  
> On Mon, Mar 1, 2021 at 16:05 David T. Lewis <[hidden email]> wrote:
>
> >
> > On Mon, Mar 01, 2021 at 06:20:58PM +0100, [hidden email] wrote:
> > >
> > > > Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> > > > replacing obsolete primitiveInvokeObjectAsMethod.
> > >
> > > Any news on this Tests-ObjectsAsMethods issue ?
> > >
> > > The purpose of those tests are not clear to me;
> > >
> > > Perhaps the change for the "primitives" which was made, can be reverted ?
> > >
> > > Possibly that creates a new problem for
> > primitiveArrayBecomeOneWayNoCopyHash.
> > >
> > > David Stes
> > >
> >
> > You can consider this a bug in the most recent interpreter VMs. I had
> > failed to notice that there is a hard coded reference to that primitive
> > number within the VM itself, and this reference is now resulting in
> > invocation of the wrong function.
> >
> > A trivial fix is to rearrange some primtive number assignments, but the
> > correct thing to do is adopt the logic that Eliot uses in oscog so that
> > the function is not invoked as a "primitive" at all.
> >
> > I'll try to follow up on this next week.
> >
> > Dave
>
>
>
> This test crashes SqueakJS, too. Could be the same primitive reassignment
> confusion.
>
> Can you describe this ???hard coded reference???? Under what circumstances is a
> primitive being called without being an actual primitive?
>

>  
>
> Ah that makes sense. Yes, in SqueakJS it is used directly too:
>
> https://github.com/codefrau/SqueakJS/blob/02d88e6cd5592029071c1939d85181b79a8f0d7a/vm.interpreter.js#L934
>
>
> ... and the easiest fix would indeed be to assign an unused primitive
> number, because that would require no changes to the rest of the
> execution mechanism. Putting it in the lookup phase, and caching it, seems
> quite elegant. Otherwise we would have to find another place to check that
> the method dict entry is actually a method, and invoke the run:with:in:
> mechanism if not.
>

>
> Also I'll try to test SqueakJS which is something I have no experience with,
> at all (JS = Javascript).
>
> As far as I understand, SqueakJS is a Javascript runtime for Squeak.
>
> I'll try to test that as well ...
>
> I'm learning about all the different runtimes ...
>
>   - squeak-5 (OpenSmalltalk) and squeak-5 Cog Spur (COG VM)
>   - squeak-4 classical VM
>   - squeakJS javascript
>

>
>
> On Sat, Mar 20, 2021 at 08:14:30PM +0100, [hidden email] wrote:
> >
> > Also I'll try to test SqueakJS which is something I have no experience with,
> > at all (JS = Javascript).
> >
> > As far as I understand, SqueakJS is a Javascript runtime for Squeak.
> >
> > I'll try to test that as well ...
> >
> > I'm learning about all the different runtimes ...
> >
> >   - squeak-5 (OpenSmalltalk) and squeak-5 Cog Spur (COG VM)
> >   - squeak-4 classical VM
> >   - squeakJS javascript
> >
>
> I certainly encourage you to get acquainted with SqueakJS. It is not the
> fastest VM for Squeak, but is it certainly the most portable with respect
> to supporting various Squeak images, and it demonstrates that the virtual
> machine that runs the Squeak image does not necessarily need to be tied
> to any specific operating system or implementation language.
>
> Dave
>

>
> On Sat, Mar 20, 2021 at 11:17:23AM -0700, tim Rowledge wrote:
> >
> > > On 2021-03-20, at 9:01 AM, David T. Lewis <[hidden email]> wrote:
> > >
> > > That protects it against accidental invocation from the image as more numbered
> > > primitives get added (yuk). I'll give that a try and see how it looks.
> >
> > I really wish for no numbered primitives at all. I thought I had worked out how once upon a time but not quite.
> >
>
> just FYI, I had to use a numbered primitive for highBit because to my
> knowledge, named primitives cannot be jitted yet...
> And the main interest for this primitive is to be jitted.
> https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/418
>