Need help compiling the JIT

Hello,

I'm working on a side-project in GNU Smalltalk. I use the latest version from Git, and when I tried enabling JIT via ./configure --enable-jit option, I got the following error during compilation:

    In file included from ../lightning/lightning.h:47,
                     from xlat.c:57:
    ../lightning/core.h:155:41: error: invalid type argument of unary ‘*’ (have ‘int’)
      155 | #define jit_patch_short_at(jump_pc,v)  (*_PSI((jump_pc) - sizeof(int)) = _jit_SI((jit_insn *)(v) - (jump_pc)))
          |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../lightning/core.h:156:89: note: in expansion of macro ‘jit_patch_short_at’
      156 | #define jit_patch_at(jump_pc,v) (_jitl.long_jumps ? jit_patch_long_at((jump_pc)-3, v) : jit_patch_short_at(jump_pc, v))
          |                                                                                         ^~~~~~~~~~~~~~~~~~
    ../lightning/core-common.h:168:32: note: in expansion of macro ‘jit_patch_at’
      168 | #define jit_patch(pv)          jit_patch_at ((pv), (_jit.x.pc))
          |                                ^~~~~~~~~~~~
    xlat.c:3269:3: note: in expansion of macro ‘jit_patch’
     3269 |   jit_patch (x);
          |   ^~~~~~~~~
 
I'm on 64bit Linux, Fedora 32. I couldn't find any information about building the JIT, or its status, ie. if it's even expected to compile, or is it a broken/unfinished part of the GST. Any help would be very appreciated :-)


Best regards,
Piotr Klibert

On 2021-06-03 20:26:51Z, Piotr Klibert wrote:
> Hello,
>
> I'm working on a side-project in GNU Smalltalk. I use the latest version from Git, and when I tried enabling JIT via ./configure --enable-jit option, I got the following error during compilation:

> I'm on 64bit Linux, Fedora 32. I couldn't find any information about building the JIT, or its status, ie. if it's even expected to compile, or is it a broken/unfinished part of the GST. Any help would be very appreciated :-)

I don't think the JIT is finished. JIT is hard to get right; for example, Erlang only get JIT this year, at version 24, at the mature age of 35 :)

Derek

> I don't think the JIT is finished.

Thank you! It's not a problem for me, right now at least, it's just that while I'm reasonably confident in being able to understand the byte-code compiler and interpreter, the JITs are still a bit of an unknown land to me, so I wasn't able to immediately tell how complete this one is :-)

> JIT is hard to get right; for example,
> Erlang only get JIT this year, at version 24, at the mature
> age of 35 :)

Yes, but: Erlang had an AOT compiler in HiPE for the last 15 years (although it was found not to improve things much, due to the inability of optimizing calls across modules and there being too little sequential code in most Erlang systems). On the other hand, it had at least four attempts at making a JIT in its history, which also shows that it's indeed tricky to get one right ;-)

On another note: I found a bug that affected VisualGST browse senders/implementors functionality, located here: https://github.com/gnu-smalltalk/smalltalk/blob/master/kernel/ProcEnv.st#L77-L88

    valueIfAbsent: aBlock [
        "Return the value of this variable in the current process."
        <category: 'accessing'>
        ^Processor activeProcess environment at: self key ifAbsent: [ nil ]
    ]

    value [
        "Return the value of this variable in the current process."
        <category: 'accessing'>
        ^Processor activeProcess environment at: self key ifAbsent: [ nil ]
    ]

The #valueIfAbsent: doesn't use its argument block at all, probably due to copy&paste from #value.

What should I do to make my discovery (I'd say "a fix", but its a bit too trivial to be called that ;-)) useful to others? I also made some improvements to smalltalk-mode.el, and will probably continue to do so as I work with GST.


Best regards,
Piotr Klibert

On 2021-06-04 09:11:58Z, Piotr Klibert wrote:

> What should I do to make my discovery (I'd say "a fix", but its a bit too trivial to be called that ;-)) useful to others?

The author of VisualGST and maintainer of gnu-smalltalk is in this list.
> I also made some improvements to smalltalk-mode.el, and will probably continue to do so as I work with GST.

up to date smalltalk-mode is maintained in [ELPA](https://elpa.gnu.org/packages/smalltalk-mode.html), so one can just `M-x package-install smalltalk-mode` in emacs. There are lots of changes. I am the designated maintainer of smalltalk-mode. If you want to contribute, please do the following:

```
# this will grab the whole elpa
git clone https://git.savannah.gnu.org/git/emacs/elpa.git
# this will show only smalltalk-mode
git checkout externals/smalltalk-mode
# make a local branch, make your changes, test it, etc.
```  

Then you can publish your repo on github or something, send me a mail, so I can pull and push upstream.

Or you can just send me a patch.

Derek

Thank you for all the info! I thought I have the ELPA version, but apparently, I downloaded just `smalltalk-mode.el` some time ago (like, in 2014...) and I've been using that version all along. I'll update and see which of my changes still apply (if any).

Final questions:

1. Should I re-post the #valueIfAbsent: bug properly in a new thread?

2. I encountered some problems compiling GST - in the code generated from 'genbc-impl.y' apparently. Which build tools versions I should include in the bug report in this case, and also in general?


Best regards,
Piotr Klibert


On Fri, Jun 4, 2021, at 16:04, Derek Zhou wrote:

----- Op 4 jun 2021 om 16:50 schreef Piotr Klibert [hidden email]:

> 2. I encountered some problems compiling GST - in the code generated from
> 'genbc-impl.y' apparently. Which build tools versions I should include in the
> bug report in this case, and also in general?


This depends on the problem.

Can you give an indication please what compile problem you encountered ?

I can confirm that the build tools and PATH settings are important;
for example to succesfully build I have to use  NM='/usr/gnu/bin/nm'
and the build fails with weird messages in the case of the auto-detected /usr/bin/nm on my system.

This requires some puzzling and investigation ...

I would also like to draw your attention to an extremely simple but very important patch,
posted here by Tino Calancha :

Date:   Thu Mar 25 15:46:18 2021 +0100

     * main.c (long_options): Option "file" requires an argument

diff --git a/main.c b/main.c
index f28aad3e..a4a9128b 100644
--- a/main.c
+++ b/main.c
@@ -132,7 +132,7 @@ static const struct option long_options[] = {
    {"core-dump", 0, 0, 'c'},
    {"declaration-trace", 0, 0, 'D'},
    {"execution-trace", 0, 0, 'E'},
-  {"file", 0, 0, 'f'},
+  {"file", 1, 0, 'f'},


Hopefully that *important* patch gets added to GNU smalltalk 3.3 or 3.2.6 !

Perhaps you can try to contact Steve Byrne ([hidden email]) and Paolo Bonzini ([hidden email])
or Holger Hans Peter Freyther  <[hidden email]>

Holger Freyther and Paolo Bonzini sometimes help on this list, and are around.

Regards,
David Stes

>
> Can you give an indication please what compile problem you encountered ?
>

My previous post was light on details because it was written from memory, sorry about that. I recompiled GST during the weekend and this time I recorded the details. The problem was about a redefinition of three variables, which were repeated across `genbc-impl.c` and `genbc-decl.c`. Assuming that the files will be compiled together anyway, I added `extern` to the declaractions in `genbc-impl.c`, which made the compilation pass. The exact error message (formatted for brevity) looked like this:

    /usr/bin/ld: genbc-impl.o:.../libgst/genbc-impl.c:913:
    multiple definition of `yynerrs'; genbc-decl.o:.../libgst/genbc-decl.c:1016: first defined here

    /usr/bin/ld: genbc-impl.o:.../libgst/genbc-impl.c:908:
    multiple definition of `yychar'; genbc-decl.o:.../libgst/genbc-decl.c:1011: first defined here

    /usr/bin/ld: genbc-impl.o:.../libgst/genbc-impl.c:911:
    multiple definition of `yylval'; genbc-decl.o:.../libgst/genbc-decl.c:1014: first defined here

    collect2: error: ld returned 1 exit status

My problem is that I don't know anything about the tools used to generate the .c files, so while I fixed it for myself, I have no idea how to make it so the declarations are not repeated next time. I suspect that the fix is trivial to anyone who knows the tooling used, so I thought it will be more efficient to report it here instead of trying to fix this on my own :-)

I didn't encounter the problem with `-f` option; not sure why, but I started with `gst file.st -a arg1 ...` command syntax and it worked/works ok. I had a bigger problem with locating `SystemDictionary>>arguments` (ie. where the cmdline args are stored), as I was a bit confused by `SystemDictionary>>arguments:do:`, but there's a question on SO that helped me.


Best regards,
Piotr Klibert


On Mon, Jun 7, 2021, at 20:55, [hidden email] wrote: