compiling the stack VM with LLVM at -Ofast

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

compiling the stack VM with LLVM at -Ofast

johnmci
 
Due to attending the Vancouver Island Camp Smalltalk 2014, I spent a few hours (ok many) being motivated, and trying to compile a 2009 era stack VM with LLVM optimization fully on. 

I can say that part is mostly completed, but I need some help understanding how to bring the VM source up to date (without GCC) so that I can see how best to tackle a few remaining issues. Mostly focusing around something odd in the LargeInteger Plugin. 


--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

David T. Lewis
 
On Wed, Oct 08, 2014 at 09:39:02PM -0700, John McIntosh wrote:
>
> Due to attending the Vancouver Island Camp Smalltalk 2014, I spent a few
> hours (ok many) being motivated, and trying to compile a 2009 era stack VM
> with LLVM optimization fully on.
>
> I can say that part is mostly completed, but I need some help understanding
> how to bring the VM source up to date (without GCC) so that I can see how
> best to tackle a few remaining issues. Mostly focusing around something odd
> in the LargeInteger Plugin.
Hi John,

Great to hear from you!

Are you working with the trunk SVN code base, or with the oscog branch? I
am away and cannot check right now, but I expect that both the SVN support
code and the VMMaker Smalltalk should currently be the same (or very
nearly so) for this and other plugins. I do want to try to keep it that
way if we can.

By way of terminology, we are using the term “interpreter VM” to refer to
the trunk SVN and VMMaker that produces the traditional Squeak VM, and
“stack interpreter”, “cog”, “oscog” refer to Eliot’s branch. Most of the
plugins (including VM plugins) are reasonably in sync in the two branches,
but VM itself is not. In 2009 you would have been working in trunk branch.
Tim has recently been doing work with the stack interpreter in the oscog
branches of SVN and VMMaker, and I am assuming that this is what you are
looking at now.

I want to mention that Javier Diaz-Reinoso has recently stepped forward
(at my persistent request) to do a recompile of your 4.2.5beta2U VM, with
the VMMaker sources updated to the latest. This is to enable the VM to run
recent closure images saved from a Cog VM, as well as older images such as
Etoys.

Javier announced this on squeak dev:
 http://lists.squeakfoundation.org/pipermail/squeak-dev/2014-September/179894.html

And I put a copy of his rebuilt VM is at:
http://www.squeakvm.org/~lewis/MacOS-test-Javier/

We have not gotten any feedback on this, but if you or anyone else can
confirm that Javier’s build is ok, it would be very helpful for folks with
older Macs, as well as for Etoys.

Just in case you are working with the trunk sources, as opposed to the
oscog branch, and if you are trying to bring your generated sources up to
date within an xcode project, then you may be trying to do the same thing
that Javier did. I’m not sure, but you might find exactly what you are
looking for in his zip file at
http://www.squeakvm.org/~lewis/MacOS-test-Javier/SqueakVM.zip
Also, the latest generated sources from VMMaker are now being kept along
with the platforms sources:

  svn co http://squeakvm.org/svn/squeak/trunk/src
  svn co http://squeakvm.org/svn/squeak/trunk/platforms

The trunk/src directory contains the most recently generated VMMaker
sources. I’m generating them from a unix platform and I think there are
few Mac plugins missing (Javier just copied them in from your earlier
distribution I think), but these sources should be useable with
trunk/platforms for any plantform.

I suspect that I have just answered the wrong question several times over
here, so please correct me and I’ll try to help if I can.

Dave



Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

Eliot Miranda-2
In reply to this post by johnmci
 
Hi John,

On Wed, Oct 8, 2014 at 9:39 PM, John McIntosh <[hidden email]> wrote:
 
Due to attending the Vancouver Island Camp Smalltalk 2014, I spent a few hours (ok many) being motivated, and trying to compile a 2009 era stack VM with LLVM optimization fully on. 

Why??  Why are you not trying to compile a 2014 version, e.g. in http://www.squeakvm.org/svn/squeak/branches/Cog/stacksrc?  That's over five years old.
 

I can say that part is mostly completed, but I need some help understanding how to bring the VM source up to date (without GCC) so that I can see how best to tackle a few remaining issues. Mostly focusing around something odd in the LargeInteger Plugin. 

The up-to-date code is in svn.

HTH
--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

johnmci
 


On Thu, Oct 9, 2014 at 9:28 AM, Eliot Miranda <[hidden email]> wrote:
 
Hi John,

On Wed, Oct 8, 2014 at 9:39 PM, John McIntosh <[hidden email]> wrote:
 
Due to attending the Vancouver Island Camp Smalltalk 2014, I spent a few hours (ok many) being motivated, and trying to compile a 2009 era stack VM with LLVM optimization fully on. 

Why??  Why are you not trying to compile a 2014 version, e.g. in http://www.squeakvm.org/svn/squeak/branches/Cog/stacksrc?  That's over five years old.

Goush, I was at Camp Smalltalk, cutoff from civilization, and only had a 8 year old mac book pro running 10.7.5 and Xcode 4.6 with a source tree for my iOS Fractions Calculator. So it was a comfortable viewpoint to have in order to figure out where llvm was busted. 
 
 
 

I can say that part is mostly completed, but I need some help understanding how to bring the VM source up to date (without GCC) so that I can see how best to tackle a few remaining issues. Mostly focusing around something odd in the LargeInteger Plugin. 

The up-to-date code is in svn.

HTH
--
best,
Eliot




--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

Ken G. Brown
 
Whatdyamean, we were civilized! :)

Ken,
from my iPhone

On Oct 9, 2014, at 10:49, John McIntosh <[hidden email]> wrote:



On Thu, Oct 9, 2014 at 9:28 AM, Eliot Miranda <[hidden email]> wrote:
 
Hi John,

On Wed, Oct 8, 2014 at 9:39 PM, John McIntosh <[hidden email]> wrote:
 
Due to attending the Vancouver Island Camp Smalltalk 2014, I spent a few hours (ok many) being motivated, and trying to compile a 2009 era stack VM with LLVM optimization fully on. 

Why??  Why are you not trying to compile a 2014 version, e.g. in http://www.squeakvm.org/svn/squeak/branches/Cog/stacksrc?  That's over five years old.

Goush, I was at Camp Smalltalk, cutoff from civilization, and only had a 8 year old mac book pro running 10.7.5 and Xcode 4.6 with a source tree for my iOS Fractions Calculator. So it was a comfortable viewpoint to have in order to figure out where llvm was busted. 
 
 
 

I can say that part is mostly completed, but I need some help understanding how to bring the VM source up to date (without GCC) so that I can see how best to tackle a few remaining issues. Mostly focusing around something odd in the LargeInteger Plugin. 

The up-to-date code is in svn.

HTH
--
best,
Eliot




--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

timrowledge


On 09-10-2014, at 11:04 AM, Ken G. Brown <[hidden email]> wrote:

> Whatdyamean, we were civilized! :)

We were better than civilised, we had coffee, cookies, muffins, pizza, fruit, tables, a projector and chocolate!

Those of you that missed the 2014 Camp Smalltalk *really* missed the fun.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLBM: Ruin Logic Board Multiple


Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

johnmci
 
Yes wonderful time, but was reflecting on camp when I was a child where camp was out somewhere in the wilderness, cutoff from civilization and giving you time to reflect on life and what you should do with this llvm bug. 


On Thu, Oct 9, 2014 at 11:15 AM, tim Rowledge <[hidden email]> wrote:


On 09-10-2014, at 11:04 AM, Ken G. Brown <[hidden email]> wrote:

> Whatdyamean, we were civilized! :)

We were better than civilised, we had coffee, cookies, muffins, pizza, fruit, tables, a projector and chocolate!

Those of you that missed the 2014 Camp Smalltalk *really* missed the fun.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLBM: Ruin Logic Board Multiple





--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

Eliot Miranda-2
 
Hi John,

On Thu, Oct 9, 2014 at 11:18 AM, John McIntosh <[hidden email]> wrote:
 
Yes wonderful time, but was reflecting on camp when I was a child where camp was out somewhere in the wilderness, cutoff from civilization and giving you time to reflect on life and what you should do with this llvm bug. 

What's the bug?  The llvm folks claim their compiler passes the test suites and is correct (but I'm not siding with them).  So I'm very curious...
 


On Thu, Oct 9, 2014 at 11:15 AM, tim Rowledge <[hidden email]> wrote:


On 09-10-2014, at 11:04 AM, Ken G. Brown <[hidden email]> wrote:

> Whatdyamean, we were civilized! :)

We were better than civilised, we had coffee, cookies, muffins, pizza, fruit, tables, a projector and chocolate!

Those of you that missed the 2014 Camp Smalltalk *really* missed the fun.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLBM: Ruin Logic Board Multiple





--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================




--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofasty

Holger Freyther
 
On Thu, Oct 09, 2014 at 11:20:44AM -0700, Eliot Miranda wrote:

> What's the bug?  The llvm folks claim their compiler passes the test suites
> and is correct (but I'm not siding with them).  So I'm very curious...

Well, a start would be to address the compiler warnings and compiler
errors (e.g. warnings about integer/long shifts, returning voide from
a method that should return int)

I had already raised some of these points when building your cog
branch on FreeBSD with clang. In terms of system tools using the undefined
behavior sanitizer and address sanitizer might reveal some further
issues.

cheers
        holger
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofast

johnmci
In reply to this post by Eliot Miranda-2
 
If you supply function prototypes for the squeak memory access routines you can get beyond the initial VM crash to the point of running many many byte codes. You may call that a bug, but I'm sure the LLVM folks would say you are an idiot not to have function prototypes. This enables the image to run  upto the point when it tries to print the timestamp to the change log, our first use of a LargeInteger 

At this point when we calculate the number of seconds in the large positive integer by rem the seconds for the day by 60 we end up with a large positive integer of 5 bytes versus 4, but contains for example 40 . This then is fed to the positive32BitValueOf which informs you that you are evil as lengthOf is not 4, and the resulting prim failure causes the higher up logic to spin up some exception handling recovery that exhausts memory. 

For fun if you set 

    foo->successFlag = 1;

as the first step in positive32BitValueOf

So the chain of large number calculations can run and the badly sized integer math can somehow complete why the image will start, and play nice until I ask for factorial 100. 

At this point I was going to upgrade everything so I can see if something was fixed in the plugin.

Also post-camp I migrated to regular work machine so using LLVM 6.x and xCode 6.x 



On Thu, Oct 9, 2014 at 11:20 AM, Eliot Miranda <[hidden email]> wrote:
 
Hi John,

On Thu, Oct 9, 2014 at 11:18 AM, John McIntosh <[hidden email]> wrote:
 
Yes wonderful time, but was reflecting on camp when I was a child where camp was out somewhere in the wilderness, cutoff from civilization and giving you time to reflect on life and what you should do with this llvm bug. 

What's the bug?  The llvm folks claim their compiler passes the test suites and is correct (but I'm not siding with them).  So I'm very curious...
 


On Thu, Oct 9, 2014 at 11:15 AM, tim Rowledge <[hidden email]> wrote:


On 09-10-2014, at 11:04 AM, Ken G. Brown <[hidden email]> wrote:

> Whatdyamean, we were civilized! :)

We were better than civilised, we had coffee, cookies, muffins, pizza, fruit, tables, a projector and chocolate!

Those of you that missed the 2014 Camp Smalltalk *really* missed the fun.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLBM: Ruin Logic Board Multiple





--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================




--
best,
Eliot




--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofasty

johnmci
In reply to this post by Holger Freyther
 
Ya, void is void not int  (a really old change set attached)

On Thu, Oct 9, 2014 at 11:35 AM, Holger Hans Peter Freyther <[hidden email]> wrote:

On Thu, Oct 09, 2014 at 11:20:44AM -0700, Eliot Miranda wrote:

> What's the bug?  The llvm folks claim their compiler passes the test suites
> and is correct (but I'm not siding with them).  So I'm very curious...

Well, a start would be to address the compiler warnings and compiler
errors (e.g. warnings about integer/long shifts, returning voide from
a method that should return int)

I had already raised some of these points when building your cog
branch on FreeBSD with clang. In terms of system tools using the undefined
behavior sanitizer and address sanitizer might reveal some further
issues.

cheers
        holger



--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================

voidIsVoidNotInt.10-2.cs (117K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofasty

Eliot Miranda-2
In reply to this post by Holger Freyther

Hi Hans,

On Oct 9, 2014, at 11:35 AM, Holger Hans Peter Freyther <[hidden email]> wrote:

>
> On Thu, Oct 09, 2014 at 11:20:44AM -0700, Eliot Miranda wrote:
>
>> What's the bug?  The llvm folks claim their compiler passes the test suites
>> and is correct (but I'm not siding with them).  So I'm very curious...
>
> Well, a start would be to address the compiler warnings and compiler
> errors (e.g. warnings about integer/long shifts, returning voide from
> a method that should return int)

The shift warnings are not really fixable without going to a #define for something evaluated at compile time (sizeof(sqInt)) so they can be safely ignored, and fixed with something ugly.  Really the C compiler should recognize that the shift warning is in unreachable code.

But other warnings can and should be fixed.  I have fixed all other warnings in the core VM (cointerp.c, interp.c, cogit.c) except an annoying printf warning in cogit.c that IMO shouldn't be fixed because the "fix" adds unnecessary code.  But that leaves the plugins.  Any fixed you have therein are gratefully received.

>
> I had already raised some of these points when building your cog
> branch on FreeBSD with clang. In terms of system tools using the undefined
> behavior sanitizer and address sanitizer might reveal some further
> issues.
>
> cheers
>    holger


Eliot (phone)
Reply | Threaded
Open this post in threaded view
|

Re: compiling the stack VM with LLVM at -Ofasty

David T. Lewis
In reply to this post by johnmci
 
On Thu, Oct 09, 2014 at 12:36:16PM -0700, John McIntosh wrote:
>  
> Ya, void is void not int  (a really old change set attached)

John, I checked this change set and I'm fairly sure that all of your
changes from voidIsVoidNotInt.cs have been incorporated in VMMaker.
I did not check the oscog branch for the stack interpreter, but given
the time stamps they are probably in both branches.

I have not heard any reports of a stack interpreter compiled in 64-bit
mode, so it is likely that there may be pointer/int issues that have not
been addressed in oscog. Eliot has done work to address method return
types (void versus int and so forth), so the oscog branch may be in
better shape in that regard.

Dave


>
> On Thu, Oct 9, 2014 at 11:35 AM, Holger Hans Peter Freyther <
> [hidden email]> wrote:
>
> >
> > On Thu, Oct 09, 2014 at 11:20:44AM -0700, Eliot Miranda wrote:
> >
> > > What's the bug?  The llvm folks claim their compiler passes the test
> > suites
> > > and is correct (but I'm not siding with them).  So I'm very curious...
> >
> > Well, a start would be to address the compiler warnings and compiler
> > errors (e.g. warnings about integer/long shifts, returning voide from
> > a method that should return int)
> >
> > I had already raised some of these points when building your cog
> > branch on FreeBSD with clang. In terms of system tools using the undefined
> > behavior sanitizer and address sanitizer might reveal some further
> > issues.
> >
> > cheers
> >         holger
> >
>
>
>
> --
> ===========================================================================
> John M. McIntosh <[hidden email]>
> https://www.linkedin.com/in/smalltalk
> ===========================================================================