Building a 64-bit VM for FreeBSD

 
Hi everyone,

I'm trying to wrap my head around how to use VMMaker to generate source for
a VM.

I took a stock 3.9 (Squeak3.9-final-7067, to be precise) image on my Windows
machine, installed VMMaker from SqueakMaps, and fired up the VMMakerTool. I
checked out http://squeakvm.org/svn/squeak/trunk/platforms into a src
directory, and pointed VMMakerTool at it. I told it "build all plugins
internally" (because I want to Make It Work first).

That generated some files. (In the unix subdirectory: interp.c, interp.h,
sqNamedPrims.h. In unix/vm/intplugins/Foo/, a Foo.c, where Foo is each of
the plugin names.)

Then:

cd ~/temp
mkdir src
cd src
svn co http://squeakvm.org/svn/squeak/trunk/platforms

I copied the various produced files on my Windows manually over to the new
repository. Then:

mkdir bld
cd bld
../platforms/unix/config/configure
make

and everything started compiling. Everything went fine, until we started
compiling sqUnixSecurity.c, when I got this:

gcc -g -O2 -DLSB_FIRST=1  -DHAVE_CONFIG_H  -DSQUEAK_BUILTIN_PLUGIN  -I/usr/h
ome/frank/temp/src/bld -I/usr/home/frank/temp/src/platforms/unix/vm -I/usr/h
ome/frank/temp/src/platforms/Cross/vm -I/usr/home/frank/temp/src/platforms/u
nix/src/vm -I/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin
   -c -o sqUnixSecurity.o
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c
In file included from
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:16:
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:4: error: conflicting types for 'ioDisableImageWrite'
/usr/home/frank/temp/src/platforms/Cross/vm/sq.h:345: error: previous
declaration of 'ioDisableImageWrite' was here
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:5: error: conflicting types for 'ioCanWriteImage'
/usr/home/frank/temp/src/platforms/Cross/vm/sq.h:344: error: previous
declaration of 'ioCanWriteImage' was here
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:28: error: conflicting types for 'ioCanRenameImage'
/usr/home/frank/temp/src/platforms/Cross/vm/sq.h:343: error: previous
declaration of 'ioCanRenameImage' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:67: error: conflicting types for 'ioCanCreatePathOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:20: error: previous declaration of 'ioCanCreatePathOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:74: error: conflicting types for 'ioCanListPathOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:24: error: previous declaration of 'ioCanListPathOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:81: error: conflicting types for 'ioCanDeletePathOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:22: error: previous declaration of 'ioCanDeletePathOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:91: error: conflicting types for 'ioCanOpenFileOfSizeWritable'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:26: error: previous declaration of 'ioCanOpenFileOfSizeWritable' was
here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:98: error: conflicting types for 'ioCanOpenAsyncFileOfSizeWritable'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:25: error: previous declaration of 'ioCanOpenAsyncFileOfSizeWritable'
was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:104: error: conflicting types for 'ioCanDeleteFileOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:21: error: previous declaration of 'ioCanDeleteFileOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:110: error: conflicting types for 'ioCanRenameFileOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:27: error: previous declaration of 'ioCanRenameFileOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:117: error: conflicting types for 'ioCanGetFileTypeOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:23: error: previous declaration of 'ioCanGetFileTypeOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:123: error: conflicting types for 'ioCanSetFileTypeOfSize'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:29: error: previous declaration of 'ioCanSetFileTypeOfSize' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:132: error: conflicting types for 'ioDisableFileAccess'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:30: error: previous declaration of 'ioDisableFileAccess' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:139: error: conflicting types for 'ioHasFileAccess'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:31: error: previous declaration of 'ioHasFileAccess' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:174: error: conflicting types for 'ioCanCreateSocketOfType'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:16: error: previous declaration of 'ioCanCreateSocketOfType' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:180: error: conflicting types for 'ioCanConnectToPort'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:17: error: previous declaration of 'ioCanConnectToPort' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:186: error: conflicting types for 'ioCanListenOnPort'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:13: error: previous declaration of 'ioCanListenOnPort' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:192: error: conflicting types for 'ioDisableSocketAccess'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:18: error: previous declaration of 'ioDisableSocketAccess' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:199: error: conflicting types for 'ioHasSocketAccess'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:19: error: previous declaration of 'ioHasSocketAccess' was here
/usr/home/frank/temp/src/platforms/unix/plugins/SecurityPlugin/sqUnixSecurit
y.c:224: error: conflicting types for 'ioInitSecurity'
/usr/home/frank/temp/src/platforms/Cross/plugins/SecurityPlugin/SecurityPlug
in.h:12: error: previous declaration of 'ioInitSecurity' was here
*** Error code 1

Stop in /usr/home/frank/temp/src/bld/SecurityPlugin.
*** Error code 1

Stop in /usr/home/frank/temp/src/bld.

I'm running gcc (GCC) 4.2.1 20070719  [FreeBSD].

How have I messed up, and do I need to give any further info?

frank

 
On Wed, Feb 04, 2009 at 11:31:12AM +0200, Frank Shearar wrote:
>  
> Hi everyone,
>
> I'm trying to wrap my head around how to use VMMaker to generate source for
> a VM.

The Unix build instructions are included in the platforms sources:
  http://squeakvm.org/cgi-bin/viewcvs.cgi/trunk/platforms/unix/doc/

An overview of 64-bit Squeak VM and image is here:
  http://squeakvm.org/squeak64/faq.html

A slightly out of date recipe for building from source on a 64-bit unix/linux
platform is here:
  http://lists.squeakfoundation.org/pipermail/vm-dev/2007-December/001703.html

You can save some steps by loading VMMaker from squeaksource.com, which
contains many of the updates in the recipe (especially steps 2 and 3):
  http://squeaksource.com/VMMaker/VMMaker-dtl.113.mcz

I usually organize the build directories thus:
  ./platforms
  ./src32
  ./build32

Then do a symlink:
  ln -s ./src32 src

Let VMMaker put the generated sources in ./src32, and do the build in ./build32.
The symlink to ./src is used by configure and make to find the sources.

HTH,
Dave

 
"David T. Lewis" <[hidden email]> writes:


> On Wed, Feb 04, 2009 at 11:31:12AM +0200, Frank Shearar wrote:
> >
> > Hi everyone,
> >
> > I'm trying to wrap my head around how to use VMMaker to generate source
for
> > a VM.
>
> The Unix build instructions are included in the platforms sources:
>   http://squeakvm.org/cgi-bin/viewcvs.cgi/trunk/platforms/unix/doc/
>
<snip>
> Let VMMaker put the generated sources in ./src32, and do the build in
./build32.
> The symlink to ./src is used by configure and make to find the sources.

Thanks for the response, David.

I still don't understand the process. (I promise, I did read the FAQ and the
64-bit Unix Squeak pages!) OK, so there's a Subversion repository for the
Squeak VM. And there's VMMaker. Now if I run VMMaker (having first created a
./platform/unix), I get a small number of files. For instance, in the vm
directory there's interp.c, interp.h and sqNamedPrims.h. These files aren't
in the svn repository.

Should VMMaker have its "Path to generated sources" point to the checked-out
svn repository instead, so the VMMaker-generated files alter the repository?

frank

 

On 11.02.2009, at 12:24, Frank Shearar wrote:

You also need to select the plugins you want to build.

> For instance, in the vm
> directory there's interp.c, interp.h and sqNamedPrims.h. These files  
> aren't
> in the svn repository.

http://squeakvm.org/cgi-bin/viewcvs.cgi/trunk/platforms/unix/src/vm/

> Should VMMaker have its "Path to generated sources" point to the  
> checked-out
> svn repository instead, so the VMMaker-generated files alter the  
> repository?


No, you delete the pre-generated src directory and replace it with  
your own generated sources.

- Bert -

 
"Bert Freudenberg" <[hidden email]> wrote:

Sorry, I should've mentioned that I'd done that. For the moment I've just
gone the "all internal" route (for all the available ports, at least).

> > For instance, in the vm
> > directory there's interp.c, interp.h and sqNamedPrims.h. These files
> > aren't
> > in the svn repository.
>
> http://squeakvm.org/cgi-bin/viewcvs.cgi/trunk/platforms/unix/src/vm/

Ah, I NOW see there's a unix/src/vm/ - I was looking in unix/vm/.

> > Should VMMaker have its "Path to generated sources" point to the
> > checked-out
> > svn repository instead, so the VMMaker-generated files alter the
> > repository?
>
>
> No, you delete the pre-generated src directory and replace it with
> your own generated sources.

OK. Does it matter that I'm running VMMaker on a Windows machine and then
copying the resulting src/ over to a FreeBSD machine? (I'm thinking in
particular of line endings.)

I've tried various things, and had various errors preventing compilation.

I thought then I'd try compile the 3.10-4 straight as-is. This should
generate a 32 bit VM, yes? At least, it looks that way judging from the
enormous number of warnings complaining about size differences in casts.
Anyway, after a lot of sweating and consulting my local gcc guru, the
following incantation resulted in a fully-compiling 32 bit VM:

  mkdir bld
  cd bld
  ../platforms/unix/config/configure CPPFLAGS=-I/usr/local/include
LDFLAGS="-L/usr/local/lib" --without-npsqueak
  make
  sudo make install

I'll try that out shortly with a Seaside image.

frank

 
"Frank Shearar" <[hidden email]> writes:
<snip>
>
> I'll try that out shortly with a Seaside image.

I forgot to add: thank you very much, Bert and David, for putting up with my
ridiculously newbie questions.

frank

 
On 11.02.2009, at 19:32, Frank Shearar wrote:
> OK. Does it matter that I'm running VMMaker on a Windows machine and  
> then
> copying the resulting src/ over to a FreeBSD machine? (I'm thinking in
> particular of line endings.)

No. What I am wondering though is why you regenerate the sources at  
all. Did you change anything?

> I thought then I'd try compile the 3.10-4 straight as-is.

Right. What's the problem with that?

- Bert -

 
"Bert Freudenberg" <[hidden email]> wrote:

> On 11.02.2009, at 19:32, Frank Shearar wrote:
> > OK. Does it matter that I'm running VMMaker on a Windows machine and
> > then
> > copying the resulting src/ over to a FreeBSD machine? (I'm thinking in
> > particular of line endings.)
>
> No. What I am wondering though is why you regenerate the sources at
> all. Did you change anything?

No; I'd tried simply checking out the squeakvm repository and compiling
that. That failed (see below), and so I thought that perhaps I'd _have_to_
have a 64 bit VM, that being the machine's architecture. (Also, the
lang/squeak port is explicitly marked as i386 only.)

> > I thought then I'd try compile the 3.10-4 straight as-is.
>
> Right. What's the problem with that?

Here's what I did:

cd ~/temp
mkdir bld
svn co http://squeakvm.org/svn/squeak/trunk/platforms

That checked out revision 1967.

cd ../bld
script configure.txt ../platforms/unix/config/configure

I've attached the resulting files.

frank

 
On Wed, Feb 11, 2009 at 08:32:57PM +0200, Frank Shearar wrote:
>  
> OK. Does it matter that I'm running VMMaker on a Windows machine and then
> copying the resulting src/ over to a FreeBSD machine? (I'm thinking in
> particular of line endings.)

Yes it is possible that this would cause problems. On Linux, I just
use the /usr/local/bin/svn command line tool. You should be able to
find that for your FreeBSD system also.

I am attaching two files to show the complete build process on a
64-bit Linux system. I expect that it should work exactly the same
on your 64-bit FreeBSD.

The typescript.gz file is a complete copy of a terminal session
(captured using /usr/bin/script) in which I download the Subversion
sources, generate the VM source from Squeak, and then do a configure,
make, and make install.

The world.png file is a screen shot of the Squeak image with VMMaker
that I used to generate the sources.

Note that I am using a different directory structure than Bert
suggested (I put the generated sources in a directory separate
from platforms, and use symlinks). This is a matter of personal
preference and either approach should work. Either way, it is
important for the configure script to be able to find the right
../src directory, otherwise you will get errors.

The entire build process will only take a few minutes once you
get the hang of it :)

Dave
 

 

On 12.02.2009, at 06:12, David T. Lewis wrote:

> I am attaching two files to show the complete build process on a
> 64-bit Linux system. I expect that it should work exactly the same
> on your 64-bit FreeBSD.


If the goal is just to make a usable VM on a 64-bit Linux system, is  
the VMMaker step necessary? I thought not, but if the sources fail to  
compile out-of-the-box then we should try to fix that.

- Bert -

 
On Thu, Feb 12, 2009 at 10:08:53AM +0100, Bert Freudenberg wrote:
I guess that the platforms/unix/src sources should work in principle,
although I would not be surprised if there are problems when compiling
on a 64-bit system. The platforms/unix/src source probably contain
source for plugins that have not been updated for 64-bit.

Personally, I'm just in the habit of using VMMaker and I tend to
think of the platforms/unix/src files as something that is there
mainly for reference.

Dave

 

On 12.02.2009, at 13:53, David T. Lewis wrote:


Well it's also there for convenience, so a standard "configure; make;  
make install" works.

- Bert -

 
"Bert Freudenberg" <[hidden email]> wrote:

I'd hoped it would be this simple. I'd be happy to find out that no, it's
just me being silly, and not a bug in the codebase. I _didn't_ find it as
simple as configure; make; make install. It took me pasting snippets of gcc
output to a C fundi to come up with CPPFLAGS and LDFLAGS and such to get a
VM compiled.

I'm happy, if it's not simply a malconfigured system on my part, to
periodically run a smoke-test on my machine.

frank

 

On 13.02.2009, at 20:21, Frank Shearar wrote:

If it does not work it *is* a bug.

> I _didn't_ find it as
> simple as configure; make; make install. It took me pasting snippets  
> of gcc
> output to a C fundi to come up with CPPFLAGS and LDFLAGS and such to  
> get a
> VM compiled.
>
> I'm happy, if it's not simply a malconfigured system on my part, to
> periodically run a smoke-test on my machine.


The problem is that Dave has been pretty much working alone in making  
the 64 bit VM work, as all the other "VM guys" are not using a 64 bit  
host. Interest is ramping up, but not volunteers. Thanks for  
volunteering to test, now we just need volunteer coders :)

- Bert -