Compiling Flow.so under linux

I have been unable to deduce the steps required to build Flow.so
under linux. There are guides [1] [2], but ran into problems
following each one, such as configure barfing on an empty build
directory, where to put the source, whether VMMaker is needed,
and whether generated or raw source is required to build
Flow.so. So, here is what I am doing. It does not work:

1.  Download the 3.9-8 VM source code:
    http://squeakvm.org/unix/release/Squeak-3.9-8.src.tar.gz
2.  Unzip it to /home/tapple/squeak/vm
3.  Download the 2a5 Flow source code:
    http://netjam.org/flow2a5.tar.gz
4.  Unzip it to
    /home/tapple/squeak/vm/platforms/Cross/plugins/FlowPlugin
5.  Download and run squeak-dev:
    http://damien.cassou.free.fr/squeak-dev/sq3.9-7067dev07.08.1.zip
6.  Install VMMaker-3.8b6.1 from Universes (Category: System)
7.  Load "spoonProcessor Changes.st" from the components
    directory of
    http://www.movedigital.com/go/netjam/63004/spoon.2a12.linux.zip
8.  Open VMMaker Tool and configure:
    8.1. Interpreter Class Name: Interpreter
    8.2. Path to Platform Code: /home/tapple/squeak/vm/platforms
    8.3. Platform Name: unix
    8.4. Path to Generated Sources: /home/tapple/squeak/vm/src
    8.5. Internal Plugins: None
    8.6. External Plugins: Flow
9.  Generate Entire. This generates, among other things, a
    single file containing the generated Flow source:
    /home/tapple/squeak/vm/src/plugins/Flow/Flow.c
10. In shell:
    10.1. cd /home/tapple/squeak/vm/src
    10.2. ../platforms/unix/config/configure
    10.3. make
11. Lots of undefined refs to halt in gnu-interp.c
12. Try just compiling plugins: make plugins
13. Lots of syntax errors in the generated Flow.c file

I have a feeling that the generated Flow.c file is the problem,
since the source is distributed as a set of files, but only one
is generated

[1]: http://wiki.squeak.org/squeak/2105
[2]: http://squeakvm.org/unix/devel.html

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/
Help improve Squeak Documentation: http://wiki.squeak.org/squeak/808
_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

On Sun, Aug 26, 2007 at 03:18:39PM -0700, Matthew Fulmer wrote:
I am having several simultaneous uncertainties about the build
process that are not addressed in any walk through I found:

- are 'spoonProcessor Changes.st' and 'flow2a5.tar.gz' meant to
  be used together? 'spoonProcessor Changes.st' adds a flow
  plugin to VMMaker, and 'flow2a5.tar.gz' is the source code.
  Both seem necessary to me, but I have no idea how to tell if
  both are working together or not
- How to you generate the Makefile to build Flow.so? is that the
  job of VMMaker, or of configure?
- Should configure be run before or after running VMMaker?
- should there be a generated Flow.c file, or should the source
  just be copied from platforms/Cross/FlowPlugin?
- is platforms/Cross/FlowPlugin even the right place to extract
  flow2a5.tar.gz?

Thanks; sorry I have so many questions

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/
Help improve Squeak Documentation: http://wiki.squeak.org/squeak/808
_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

After some help from vatic and Craig on IRC, I am able to build
something broken

On Sun, Aug 26, 2007 at 03:18:39PM -0700, Matthew Fulmer wrote:
4.1.  Download the Flow build files:
      http://netjam.org/flow2aPrimitiveSources.zip
4.2.  Extract
      flow2aPrimitiveSources.zip/flow/platformSpecific/linux/Makefile.in
      to /home/tapple/squeak/vm/platforms/unix/plugins/FlowPlugin

> 5.  Download and run squeak-dev:
>     http://damien.cassou.free.fr/squeak-dev/sq3.9-7067dev07.08.1.zip
> 6.  Install VMMaker-3.8b6.1 from Universes (Category: System)

Don't do step 7:

> 7.  Load "spoonProcessor Changes.st" from the components
>     directory of
>     http://www.movedigital.com/go/netjam/63004/spoon.2a12.linux.zip

This step should not be done, and seems to be the cause of the
spurious Flow.c I found last time (see step 9 below). It seems
VMMaker is generating bad source for the Flow simulator support

> 8.  Open VMMaker Tool and configure:
>     8.1. Interpreter Class Name: Interpreter
>     8.2. Path to Platform Code: /home/tapple/squeak/vm/platforms
>     8.3. Platform Name: unix
>     8.4. Path to Generated Sources: /home/tapple/squeak/vm/src
>     8.5. Internal Plugins: None
>     8.6. External Plugins: Flow

8.7 should be: External Plugins: None

> 9.  Generate Entire. This generates, among other things, a
>     single file containing the generated Flow source:
>     /home/tapple/squeak/vm/src/plugins/Flow/Flow.c

Flow.c is no longer generated, as it shouldn't be

9.1.  add B3DAccelleratorPlugin UnixOSProcessPlugin Flow to the
      list in plugins.ext

> 10. In shell:
>     10.1. cd /home/tapple/squeak/vm/src
>     10.2. ../platforms/unix/config/configure
>     10.3. make

I get, in the source directory, an executable named squeak and a
Flow.la in Flow. I try running the squeak vm with the spoon
image, and I get the following error:

Recursive not understood error encountered


Segmentation fault

I try moving Flow and B3DAcceleratorPlugin to plugins.int. I
also notice that I have no plugin named UnixOSProcessPlugin in
either my platforms/unix/plugins or platforms/Cross/plugins
directories. I look at the size of the compiled UnixOSProcess.la and it
seems empty (only 758 bytes). So I delete it from the list of
compiled plugins. No matter whether I use UnixProcessorPlugin,
or whether plugins are internal or external, I still get the
same error whenever I try running the compiled squeak
executable.

Also, no .so files are ever compiled, only .la and .a files (for
external and internal plugins, respectively)

I may yet figure this out. Thanks for the help

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/
Help improve Squeak Documentation: http://wiki.squeak.org/squeak/808
_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

OK, OK, I got out my Linux box and had a go at this for a while. Used
the new 2a5 sources, but I think they're still in need of work to get
Spoon running under Linux. But here are the steps. Sorry I can't pin
down some of this exactly:

On Thu, 06 Sep 2007 00:43:00 -0700, Matthew Fulmer wrote:

>> 1.  Download the 3.9-8 VM source code:
>>     http://squeakvm.org/unix/release/Squeak-3.9-8.src.tar.gz

>> 5.  Download and run squeak-dev:
>>     http://damien.cassou.free.fr/squeak-dev/sq3.9-7067dev07.08.1.zip

So now you have a working "vanilla" Squeak.

. . .

Get the entire VM SVN:

svn co http://squeakvm.org/svn/squeak/trunk squeak-svn

>>     http://netjam.org/flow2a5.tar.gz
>> 4.  Unzip it to
>>     ~/platforms/Cross/plugins/FlowPlugin/

OK, but be sure that your folders across "platforms" are named either
Flow or FlowPlugin, ie named consistently. Craig named them "Flow", so
beware!

> 4.1.  Download the Flow build files:
>       http://netjam.org/flow2aPrimitiveSources.zip
> 4.2.  Extract
>       flow2aPrimitiveSources.zip/flow/platformSpecific/linux/Makefile.in
>       to ~/platforms/unix/plugins/FlowPlugin/

Extract
flow2aPrimitiveSources.zip/flow/platformSpecific/linux/plugins.ext
        to ~/platforms/unix/src/

I should say I also removed config.h from
~/platforms/Cross/plugins/FlowPlugin/ as it looked like a MacOS
include...

So now you have your sources set up.

. . .

I'm not convinced the following is necessary just to build Flow.so.
(You might want it for other reasons though):

>> 6.  Install VMMaker-3.8b6.1 from Universes (Category: System)

>> 8.  Open VMMaker Tool and configure:
>>     8.1. Interpreter Class Name: Interpreter
>>     8.2. Path to Platform Code: ~/platforms
>>     8.3. Platform Name: unix
>>     8.4. Path to Generated Sources: ~/platform/unix/src
>>     8.5. Internal Plugins: None
>>     8.6. External Plugins: Flow

. . .

All that VMMaker is being used for is to generate the plugins.ext file
which doesn't have to say anything more than:

EXTERNAL_PLUGINS = FlowPlugin

...which you have already courtesy Craig. If this file doesn't say
that, the make process will still chug foward, stumble and generate a
Flow.la because ./configure found it, but the Makefiles weren't built
correctly. I found that fiddling with VMMakerTool merely erased what
was in plugins.ext. (Not sure why)

So the final part is to switch to ~/platforms/unix/ and run make. This
will go through and run many ./configures, build the requisite
Makefiles, and should leave Flow.so in ~/platforms/unix/bld/FlowPlugin/

But as I said, I believe there are still problems with it, but not the
same as on my previous tries...

:-)

I'll follow this email with with my typescript of the make process. It
has a bunch of warnings that I don't know right now whether they are
serious or not...

Best, vatic


_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

Hi Matthew, hi list,

I've finally got around to trying to build a spoon VM of my own, and
I've met with some success (!).

* I've managed to build a VM that can load and run "memory.image"
* I get a page at http://localhost:8090/3
* I've build a separate Flow plugin that works with a stock VM well
   enough to allow the naiad components to load and partially run

I now run into the problem where

Smalltalk
     at: #ExampleWormholeServer
     put: (WormholeServer atPort: 9999)
   succeeds,

ExampleWormholeServer connectToPort: 7777 atHostNamed: 'localhost'
   succeeds,

but

OtherBrowser openOnWormhole: ExampleWormholeServer clients any
   fails, with an error

MessageNotUnderstood: Other>>module

The next message will contain the steps I took to build a spoonable VM.

Regards,
   Tony

_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

Below are my notes from building a VM capable of running memory.image.
Note that I ran VMMaker three times - the first two times are not
needed! I was just proceeding in small steps to do things like look at
diffs etc etc.

Things that puzzle me:

  - the calls to halt ("halt(3)" appears a lot in interp.c)
  - the call to cleanUpContexts
  - the error generating LargeIntegersPlugin
  - the nasty code in the generated LargeIntegersPlugin

Regards,
   Tony

== COMPILATION NOTES ==

Get http://squeakvm.org/unix/release/Squeak-3.9-8.src.tar.gz
Get http://netjam.org/flow2a5.tar.gz
Get
http://www.netjam.org/flow/releases/current/bits/flow2aPrimitiveSources.zip
Get http://damien.cassou.free.fr/squeak-dev/sq3.9-7067dev07.09.1.zip

Unzip sq3.9-7067dev07.09.1.zip somewhere convenient.
  - Untar Squeak-3.9-8.src.tar.gz into the resulting
sq3.9-7067dev07.09.1 directory.
  - Unzip spoon.2a12.linux.zip as well
  - Unzip flow2a5.tar.gz, and rename "flow/" to "flow2a5/"
  - Unzip flow2aPrimitiveSources.zip, and rename "flow/" to "flow2a/"

Edit flow2a/platformSpecific/linux/Makefile.in, commenting out the "cp
Flow.so $(plgdir)" line.

Open the image in the sq3.9-7067dev07.09.1 directory.

Open Package Universe Browser
  - Choose "--System--", "VMMaker", "3.8b6.1"
  - Middle-click "3.8b6.1", choose "select this version"
  - Click "install selection"

Much code downloading and installation happens.

Open VMMaker
  - click "Find Path" at the end of the "Path to platforms code" line
  - Choose your current working directory's subdirectory "Squeak-3.9-8".
  - Click OK
  - Click "Find platform" on the "Platform name" line.
  - Choose "unix" from the menu that appears.
  - Click "Core+Internal" on the "Generate" line
  - When it's done, save and quit the image

Rename away the generated source directory:
   $ mv src32 src32-without-patches

Reopen the Squeak-Dev image.
File in "./spoon/components/spoonProcessor Changes.st"
Click "Core+Internal" again.
Save and quit the image again.

Rename the generated source directory appropriately:
   $ mv src32 src32-with-patches

If you like, examine the core changes to the interpreter:
   $ diff -w -ru3 src32-without-patches/ src32-with-patches/

Regenerate the VM one more time, with as many plugins as possible
external, and the rest internal.
  - open the image
  - middle click "Plugins not built" in the VMMaker
  - choose "make all external"
  - drag "LargeIntegersSimulator" out of the "External Plugins" panel
    into the "Plugins not built" panel
  - click "Entire"
  - You get an error during LargeIntegersPlugin! Click "proceed".
Save and quit the image.

Edit src32/vm/interp.c
  - add the line "#define halt(x)" to the top
  - search for the line "cleanUpContexts();" and comment it out

Now apply the changes to the VM sources and build the VM.
   $ rm -rf Squeak-3.9-8/platforms/unix/src
   $ rm -rf Squeak-3.9-8/platforms/unix/src32
   $ rm -rf Squeak-3.9-8/platforms/unix/src64
   $ mv src32 Squeak-3.9-8/platforms/unix/src
   $ mv Squeak-3.9-8/platforms/unix/src/plugins/Flow/
Flow-plugin-as-generated
   $ cp -r flow2a5 Squeak-3.9-8/platforms/unix/plugins/Flow
   $ cp flow2a/platformSpecific/linux/Makefile.in
Squeak-3.9-8/platforms/unix/plugins/Flow/
   $ mkdir Squeak-3.9-8/build
   $ (cd Squeak-3.9-8/platforms/unix/config/; make)
   $ (cd Squeak-3.9-8/build/; ../platforms/unix/config/configure
--prefix=/opt/spoon)

Fix LargeIntegers.c line 1370 - comment out the line
        halt(,("missing primitive: ", primitiveName));

   $ (cd Squeak-3.9-8/build/; make)
   $ (cd Squeak-3.9-8/build/; make install)

Now /opt/spoon/bin/squeak memory.image should work.

_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

Hi Tony--

 > I now run into the problem where
 >
 > Smalltalk
 >     at: #ExampleWormholeServer
 >     put: (WormholeServer atPort: 9999)
 >   succeeds,
 >
 > ExampleWormholeServer connectToPort: 7777 atHostNamed: 'localhost'
 >   succeeds,
 >
 > but
 >
 > OtherBrowser openOnWormhole: ExampleWormholeServer clients any
 >   fails, with an error
 >
 > MessageNotUnderstood: Other>>module

      All messages to instances of Other are supposed to be forwarded by
the virtual machine. Are you running with a VM that has remote messaging
support (i.e., the Spoon VM you built)?


      thanks,

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

Hi Tony--

 > Things that puzzle me:
 >
 > - the calls to halt ("halt(3)" appears a lot in interp.c)

      Yeah, looks like I left in a "3 halt" by mistake in
Interpreter>>primitiveFail, which of course gets sent by a lot of other
places. There's also a halt I was using during simulation in
LargeIntegersPlugin>>noteMissingPrimitive:, which you noticed after
translation to C (it should be removed before translation).

 >  - the call to cleanUpContexts [in
 > Interpreter>>primitiveSnapshotWithoutStaleMethods]

      That's a vestigial thing from 3.2 that I didn't catch, that line
should just be removed.

 >  - the error generating LargeIntegersPlugin
 >  - the nasty code in the generated LargeIntegersPlugin

      That happened because of the halt in LargeIntegersPlugin mentioned
above, I assume.

      Thanks for going through the building process, and for taking notes!


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

Hi,

Craig Latta wrote:
>      All messages to instances of Other are supposed to be forwarded by
> the virtual machine. Are you running with a VM that has remote messaging
> support (i.e., the Spoon VM you built)?

Ah, ok. I was actually running memory.image with my spoon VM, but the
control image with a stock-VM-plus-Flow.

I did try my spoon VM with the control image (Squeak-dev 3.9 + naiad),
but it loaded, gave a white screen with a smallish black rectangle in
the top left, and SEGVd. I guess my VM isn't yet quite correct. I'll try
finding out where it crashed, I suppose.

I'm glad to be making progress - it's quite exciting, getting closer to
being able to experiment with spoon!

Regards,
   Tony

_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

Hi,

Craig Latta wrote:
>      Yeah, looks like I left in a "3 halt" by mistake in
> Interpreter>>primitiveFail, which of course gets sent by a lot of other
> places. There's also a halt I was using during simulation in
> LargeIntegersPlugin>>noteMissingPrimitive:, which you noticed after
> translation to C (it should be removed before translation).
> [...]
>  >  - the call to cleanUpContexts [in
>      That's a vestigial thing from 3.2 that I didn't catch, that line
> should just be removed.

Good to know. Thanks.

>  >  - the error generating LargeIntegersPlugin
>  >  - the nasty code in the generated LargeIntegersPlugin
>      That happened because of the halt in LargeIntegersPlugin mentioned
> above, I assume.

It's possible - it didn't immediately seem to be related. I will try
generating the plugin in an image that doesn't have the spoon changes,
to see if it's an existing problem.

Regards,
   Tony

_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon

I'm just about to walk out the door to work, but before I go I'll send
the stack trace I get running my spoon VM on my squeak-dev+naiad image,
just in case it provides a valuable clue. The VM dies in the GC. I'll
investigate further over the weekend, with luck.

(The location of the SEGV, gnu-interp.c:11796, corresponds to the first
occurrence of the line "header2 = longAt(oop1);" in the function
markAndTrace() in interp.c.)

Regards,
   Tony

~/Squeak/spoon/building-from-scratch/sq3.9-7067dev07.09.1$ gdb /opt/spoon/bin/squeak
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
Using host libthread_db library "/lib/i686/cmov/libthread_db.so.1".
(gdb) run sq3.9-7067dev07.09.1.image
Starting program: /opt/spoon/bin/squeak sq3.9-7067dev07.09.1.image
Failed to read a valid object file image from memory.
[Thread debugging using libthread_db enabled]
[New Thread -1210169680 (LWP 18218)]
Error while reading shared library symbols:
Cannot find new threads: generic error

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210169680 (LWP 18218)]
markAndTrace (oop=2022687948) at gnu-interp.c:11796
11796                           header2 = longAt(oop1);
(gdb) bt
#0  markAndTrace (oop=2022687948) at gnu-interp.c:11796
#1  0x0805eddf in markAndTraceInterpreterOops () at gnu-interp.c:12250
#2  0x0805eedd in markPhase () at gnu-interp.c:12289
#3  0x0805f5d4 in fullGC () at gnu-interp.c:5101
#4  0x0805fa65 in sufficientSpaceAfterGC (minFree=2504) at gnu-interp.c:23434
#5  0x0806312c in primitiveNew () at gnu-interp.c:17703
#6  0x08054207 in dispatchFunctionPointer (aFunctionPointer=0x80630f0) at gnu-interp.c:4130
#7  0x08066cee in interpret () at gnu-interp.c:8104
#8  0x08052b0a in main (argc=Cannot access memory at address 0x1
) at /home/tonyg/Squeak/spoon/building-from-scratch/sq3.9-7067dev07.09.1/Squeak-3.9-8/platforms/unix/vm/sqUnixMain.c:1388
(gdb)
_______________________________________________
Spoon mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/spoon