A Bounty for CMake-ifying stack/Cog vm build process

It's absolutely none of my business what tools others choose to use, but I will say this:

Having more of this process in Smalltalk and less in other (arcane) languages increases the confidence that, should we lose someone's services, others could step up to fill the gap.

Cheers,
Bob

On 11/7/13 8:28 AM, Igor Stasenko wrote:

[hidden email]" type="cite">

On 7 November 2013 08:58, Tobias Pape <[hidden email]> wrote:

Hey Tim and all,

On 07.11.2013, at 05:00, Ron Teitelbaum <[hidden email]> wrote:

> Hey Tim,
>
> Göran and Eliot have been discussing this.  I just talked to Eliot about it
> yesterday.  He said he was happy to support cmake but didn't want to have to
> do the work himself.  I'm hoping that the work Göran is doing on this now
> can be turned over to Eliot when it is finished.  I think there is agreement
> that combining the excellent work of pharo build with Eliot's new vm work is
> a very good thing to do.  The work to merge the build environments is
> already being done at 3D ICC.  We are working on this for the Mac also but
> have not discussed it with Ian yet.
>
> Ron Teitelbaum
>
>> -----Original Message-----
>> From: [hidden email] [mailto:[hidden email]
>> [hidden email]] On Behalf Of tim Rowledge
>> Sent: Wednesday, November 06, 2013 9:04 PM
>> To: The general-purpose Squeak developers list
>> Subject: Re: [squeak-dev] A Bounty for CMake-ifying stack/Cog vm build
> process
>>
>>
>> Wow, no interest? Is it because make is painful or because nobody needs to
>> make any more money?

I actually have interest but I am currently reluctant,
for time-capturing reasons in my personal and work life.

However, Here is what I would do:

• Start from Ians CMake work, and try to generalize it for the
  non-Linux platforms
• Windows would be kind-of straight-forward,
• For OS X, I would really want to have the Cocoa-based VM stuff
  (aka Squeak VM 5.4.7.x) integrated and proper Xcode files
  generated, especially to be able to have iOS VMs built automatically.
   Here, some information from John McIntosh would be helpfull, especially
  a simple walk-through.
   I have some experience with that[1]


Despite all admiration I have for the way the PharoVM is built, there is one
thing that bothers me:
• You first somehow generate the VM sources (which is fine, and the automated way is amazing)
• THEN, the a Script generates CMake-Files
• CMake then generates Makefiles
• and then finally a vm is compiled.
This is one generating step too much, for my taste.
Personally, I'd prefer CMake to just pick up the C-files generated by VMMaker,
so that adding a new platform does not need to change the VMMaker…

The main big issue with 'platform-neutral' static source files, that they tend to

grow with tons of #ifdef-s over time, reducing readability and creating a mess.

My point is that one or another way you have to deal with platform differences,

and the way how i prefer to do it is using smalltalk code,

but not .m4 files, or .cmake files, or writing a medium novel long configuration files

using strange (better to say weird) scripting DSL.

I can express all build process logic in smalltalk, which does not lessens the amount
of work to deal with all platform nuances and dependencies etc, but at least it allows me to organize and shape it
into some manageable form, which is easy to change & learn.

 

 And another difference is, that i prefer working with browser & classes than
with dozens of directories and files lying here and there.

But sure thing, it is a question of taste. If you prefer to maintain this:
-------------------------
MACRO (INTERNAL_PLUGIN plugin)
  SET (plugin_sources "")
  IF (DEFINED ${plugin}_sources)
    SET (plugin_sources ${${plugin}_sources})
  ELSE (DEFINED ${plugin}_sources)
    FOREACH (dir ${src}/vm/intplugins ${cross}/plugins ${unix}/plugins)
      SET (tmp "")
      AUX_SOURCE_DIRECTORY (${dir}/${plugin} tmp)
      STRING_APPEND (plugin_sources "${tmp}")
    ENDFOREACH (dir)
  ENDIF (DEFINED ${plugin}_sources)
  IF (DEFINED ${plugin}_extra_sources)
    STRING_APPEND (plugin_sources "${${plugin}_extra_sources}")
  ENDIF (DEFINED ${plugin}_extra_sources)
  FILE (WRITE ${bld}/${plugin}/CMakeLists.in "")
  FOREACH (dir ${unix}/plugins ${unix})
    FILE_APPEND (${bld}/${plugin}/CMakeLists.in ${dir}/${plugin}/build.cmake)
  ENDFOREACH (dir)
  FILE_APPEND (${bld}/${plugin}/CMakeLists.in ${config}/PluginInternal.cmake)
  CONFIGURE_FILE (${bld}/${plugin}/CMakeLists.in ${bld}/${plugin}/CMakeLists.txt @ONLY)
  ADD_SUBDIRECTORY (${bld}/${plugin} ${bld}/${plugin})
ENDMACRO (INTERNAL_PLUGIN)
--------------------------

And/or this:
--------------------------
AC_INIT([config.h.in])

SVNREV=`grep '\$Rev: ' ${srcdir}/../../../platforms/Cross/vm/sqSCCSVersion.h \
        | sed 's/.*$Rev: \([[0-9]][[0-9]]*\).*/\1/'`
AC_VM_VERSION(4,0,${SVNREV},4,2,0)

topdir=`cd ${srcdir}/../../..; pwd`
cfgdir=`cd ${srcdir}; pwd`

AC_ARG_WITH(src,
[  --with-src=dir          generated src directory [default=src]],
[ vmmsrc="${with_src}"],
[ vmmsrc="src"])

vmmdir="${topdir}/${vmmsrc}"

if test ! -d "${topdir}/${vmmsrc}"; then
  if test -d "${topdir}/platforms/unix/${vmmsrc}"; then
    vmmdir="${topdir}/platforms/unix/${vmmsrc}"
    AC_MSG_RESULT([using built-in src directory])
  fi
fi

AC_MSG_RESULT(${vmmdir})

blddir=`pwd`

AC_ARG_WITH(vmmcfg,
[  --with-vmmcfg=dir        vm configuration directory containing plugins.int and plugins.ext [default=.]],
[ vmmcfg="${with-vmmcfg}"],
[ vmmcfg="${blddir}"])

# Compiling a Cogit VM or not?  If so, need a cogit$o, cointerp, etc.

AC_ARG_ENABLE(cogit,
[  --enable-cogit            compile a cogit VM [default=yes]],
cogit="no", cogit="yes")
AC_SUBST(cogit)

#AC_ARG_ENABLE(jit,
#[  --enable-jit            enable J4 support [default=no]],
#JIT="yes", JIT="no")
#
#test $JIT = "yes" && J_CFLAGS="-DJ_ENABLED"
#AC_SUBST(J_CFLAGS)

# Check the generated src dir looks sane

AC_CHECK_VMM_DIR

AC_SUBST(topdir)
AC_SUBST(cfgdir)
AC_SUBST(vmmdir)
AC_SUBST(vmmcfg)
AC_SUBST(blddir)

SQ_VERSION=${SQ_MAJOR}.${SQ_MINOR}-${SQ_UPDATE}

AC_DEFINE_UNQUOTED(SQ_VERSION, "${SQ_VERSION}")

AC_SUBST(SQ_MAJOR)
AC_SUBST(SQ_MINOR)
AC_SUBST(SQ_UPDATE)
AC_SUBST(SQ_VERSION)

VM_VERSION=${VM_MAJOR}.${VM_MINOR}-${VM_RELEASE}

AC_DEFINE_UNQUOTED(VM_VERSION, "${VM_VERSION}")

AC_SUBST(VM_MAJOR)
AC_SUBST(VM_MINOR)
AC_SUBST(VM_RELEASE)
AC_SUBST(VM_VERSION)

# libdir contains ${exec_prefix}, so we have to default and expand early
test "x$prefix" = xNONE && prefix=$ac_default_prefix
test "x$exec_prefix" = xNONE && exec_prefix='${prefix}'
imgdir=`eval echo ${libdir}/squeak`
expanded_relative_imgdir=`eval echo lib/squeak/${VM_VERSION}`
plgdir='${imgdir}/'`eval echo ${VM_VERSION}`

AC_SUBST(imgdir)
AC_SUBST(expanded_relative_imgdir)
AC_SUBST(plgdir)

AC_DEFINE(OS_TYPE, "unix")

AC_CANONICAL_HOST

host_cpu=`echo $host | sed 's,-.*,,'`
host=`echo $host | sed 's,-unknown,,'`

AC_SUBST(host)
AC_SUBST(host_cpu)
AC_SUBST(host_vendor)
AC_SUBST(host_os)
----------------------------

instead of plain smalltalk code, it is up to you.

But that's about 'too much' for my taste.

But if the community is positive to just move to the Pharo way of
building VMs, I'd be fine with that, since that would unify a lot
of infrastructure :)

Best
        -Tobias

[1] https://github.com/krono/self/blob/cmake/vm/cmake/mac_osx.cmake

--
Best regards,
Igor Stasenko.

Hi Timothy,

On Thu, Nov 7, 2013 at 7:02 AM, gettimothy <[hidden email]> wrote:

Is there any chance Cog could be made to run on AMD64 natively without 32 bit libs?

Yes, that's a goal of my current work.  Spur has a common object representation between 32- and 64-bits.  Getting to a native Stack VM on AMD64 should be straight-forward.  Getting to a Cog VM on AMD64 also requires writing a code generator for AMD64.  That's actually a fun project.  It involves configuring a Bochs plugin for an AMD64/IA32 simulator, and then writing a CogIA64Compiler subclass of CogAbstractInstruction.

If so, I would like to help out with whoever takes up this task. I would need direction, however as I have been do app development for too many years and my sys skills are atrophied.

How strong are your low-level skills?  You'll need to be quite familiar with machine code, basic machine organization, etc.  If you're not then you could instead have a go at getting the Stack VM working in 64-bits, and that would involve getting the 64-bit VM simulator working in VMMaker.

It will be great to have another collaborator!

t.

---- On Thu, 07 Nov 2013 06:01:00 -0800 Bob Arning<[hidden email]> wrote ----

It's absolutely none of my business what tools others choose to use, but I will say this:

Having more of this process in Smalltalk and less in other (arcane) languages increases the confidence that, should we lose someone's services, others could step up to fill the gap.

Cheers,
Bob

On 11/7/13 8:28 AM, Igor Stasenko wrote:

[hidden email]" type="cite">

On 7 November 2013 08:58, Tobias Pape <[hidden email]> wrote:

Hey Tim and all,

On 07.11.2013, at 05:00, Ron Teitelbaum <[hidden email]> wrote:

> Hey Tim,
>
> Göran and Eliot have been discussing this.  I just talked to Eliot about it
> yesterday.  He said he was happy to support cmake but didn't want to have to
> do the work himself.  I'm hoping that the work Göran is doing on this now
> can be turned over to Eliot when it is finished.  I think there is agreement
> that combining the excellent work of pharo build with Eliot's new vm work is
> a very good thing to do.  The work to merge the build environments is
> already being done at 3D ICC.  We are working on this for the Mac also but
> have not discussed it with Ian yet.
>
> Ron Teitelbaum
>
>> -----Original Message-----
>> From: [hidden email] [mailto:[hidden email]
>> [hidden email]] On Behalf Of tim Rowledge
>> Sent: Wednesday, November 06, 2013 9:04 PM
>> To: The general-purpose Squeak developers list
>> Subject: Re: [squeak-dev] A Bounty for CMake-ifying stack/Cog vm build
> process
>>
>>
>> Wow, no interest? Is it because make is painful or because nobody needs to
>> make any more money?

I actually have interest but I am currently reluctant,
for time-capturing reasons in my personal and work life.

However, Here is what I would do:

• Start from Ians CMake work, and try to generalize it for the
  non-Linux platforms
• Windows would be kind-of straight-forward,
• For OS X, I would really want to have the Cocoa-based VM stuff
  (aka Squeak VM 5.4.7.x) integrated and proper Xcode files
  generated, especially to be able to have iOS VMs built automatically.
   Here, some information from John McIntosh would be helpfull, especially
  a simple walk-through.
   I have some experience with that[1]


Despite all admiration I have for the way the PharoVM is built, there is one
thing that bothers me:
• You first somehow generate the VM sources (which is fine, and the automated way is amazing)
• THEN, the a Script generates CMake-Files
• CMake then generates Makefiles
• and then finally a vm is compiled.
This is one generating step too much, for my taste.
Personally, I'd prefer CMake to just pick up the C-files generated by VMMaker,
so that adding a new platform does not need to change the VMMaker…

The main big issue with 'platform-neutral' static source files, that they tend to

grow with tons of #ifdef-s over time, reducing readability and creating a mess.

My point is that one or another way you have to deal with platform differences,

and the way how i prefer to do it is using smalltalk code,

but not .m4 files, or .cmake files, or writing a medium novel long configuration files

using strange (better to say weird) scripting DSL.

I can express all build process logic in smalltalk, which does not lessens the amount
of work to deal with all platform nuances and dependencies etc, but at least it allows me to organize and shape it
into some manageable form, which is easy to change & learn.

 

 And another difference is, that i prefer working with browser & classes than
with dozens of directories and files lying here and there.

But sure thing, it is a question of taste. If you prefer to maintain this:
-------------------------
MACRO (INTERNAL_PLUGIN plugin)
  SET (plugin_sources "")
  IF (DEFINED ${plugin}_sources)
    SET (plugin_sources ${${plugin}_sources})
  ELSE (DEFINED ${plugin}_sources)
    FOREACH (dir ${src}/vm/intplugins ${cross}/plugins ${unix}/plugins)
      SET (tmp "")
      AUX_SOURCE_DIRECTORY (${dir}/${plugin} tmp)
      STRING_APPEND (plugin_sources "${tmp}")
    ENDFOREACH (dir)
  ENDIF (DEFINED ${plugin}_sources)
  IF (DEFINED ${plugin}_extra_sources)
    STRING_APPEND (plugin_sources "${${plugin}_extra_sources}")
  ENDIF (DEFINED ${plugin}_extra_sources)
  FILE (WRITE ${bld}/${plugin}/CMakeLists.in "")
  FOREACH (dir ${unix}/plugins ${unix})
    FILE_APPEND (${bld}/${plugin}/CMakeLists.in ${dir}/${plugin}/build.cmake)
  ENDFOREACH (dir)
  FILE_APPEND (${bld}/${plugin}/CMakeLists.in ${config}/PluginInternal.cmake)
  CONFIGURE_FILE (${bld}/${plugin}/CMakeLists.in ${bld}/${plugin}/CMakeLists.txt @ONLY)
  ADD_SUBDIRECTORY (${bld}/${plugin} ${bld}/${plugin})
ENDMACRO (INTERNAL_PLUGIN)
--------------------------

And/or this:
--------------------------
AC_INIT([config.h.in])

SVNREV=`grep '\$Rev: ' ${srcdir}/../../../platforms/Cross/vm/sqSCCSVersion.h \
        | sed 's/.*$Rev: \([[0-9]][[0-9]]*\).*/\1/'`
AC_VM_VERSION(4,0,${SVNREV},4,2,0)

topdir=`cd ${srcdir}/../../..; pwd`
cfgdir=`cd ${srcdir}; pwd`

AC_ARG_WITH(src,
[  --with-src=dir          generated src directory [default=src]],
[ vmmsrc="${with_src}"],
[ vmmsrc="src"])

vmmdir="${topdir}/${vmmsrc}"

if test ! -d "${topdir}/${vmmsrc}"; then
  if test -d "${topdir}/platforms/unix/${vmmsrc}"; then
    vmmdir="${topdir}/platforms/unix/${vmmsrc}"
    AC_MSG_RESULT([using built-in src directory])
  fi
fi

AC_MSG_RESULT(${vmmdir})

blddir=`pwd`

AC_ARG_WITH(vmmcfg,
[  --with-vmmcfg=dir        vm configuration directory containing plugins.int and plugins.ext [default=.]],
[ vmmcfg="${with-vmmcfg}"],
[ vmmcfg="${blddir}"])

# Compiling a Cogit VM or not?  If so, need a cogit$o, cointerp, etc.

AC_ARG_ENABLE(cogit,
[  --enable-cogit            compile a cogit VM [default=yes]],
cogit="no", cogit="yes")
AC_SUBST(cogit)

#AC_ARG_ENABLE(jit,
#[  --enable-jit            enable J4 support [default=no]],
#JIT="yes", JIT="no")
#
#test $JIT = "yes" && J_CFLAGS="-DJ_ENABLED"
#AC_SUBST(J_CFLAGS)

# Check the generated src dir looks sane

AC_CHECK_VMM_DIR

AC_SUBST(topdir)
AC_SUBST(cfgdir)
AC_SUBST(vmmdir)
AC_SUBST(vmmcfg)
AC_SUBST(blddir)

SQ_VERSION=${SQ_MAJOR}.${SQ_MINOR}-${SQ_UPDATE}

AC_DEFINE_UNQUOTED(SQ_VERSION, "${SQ_VERSION}")

AC_SUBST(SQ_MAJOR)
AC_SUBST(SQ_MINOR)
AC_SUBST(SQ_UPDATE)
AC_SUBST(SQ_VERSION)

VM_VERSION=${VM_MAJOR}.${VM_MINOR}-${VM_RELEASE}

AC_DEFINE_UNQUOTED(VM_VERSION, "${VM_VERSION}")

AC_SUBST(VM_MAJOR)
AC_SUBST(VM_MINOR)
AC_SUBST(VM_RELEASE)
AC_SUBST(VM_VERSION)

# libdir contains ${exec_prefix}, so we have to default and expand early
test "x$prefix" = xNONE && prefix=$ac_default_prefix
test "x$exec_prefix" = xNONE && exec_prefix='${prefix}'
imgdir=`eval echo ${libdir}/squeak`
expanded_relative_imgdir=`eval echo lib/squeak/${VM_VERSION}`
plgdir='${imgdir}/'`eval echo ${VM_VERSION}`

AC_SUBST(imgdir)
AC_SUBST(expanded_relative_imgdir)
AC_SUBST(plgdir)

AC_DEFINE(OS_TYPE, "unix")

AC_CANONICAL_HOST

host_cpu=`echo $host | sed 's,-.*,,'`
host=`echo $host | sed 's,-unknown,,'`

AC_SUBST(host)
AC_SUBST(host_cpu)
AC_SUBST(host_vendor)
AC_SUBST(host_os)
----------------------------

instead of plain smalltalk code, it is up to you.

But that's about 'too much' for my taste.

But if the community is positive to just move to the Pharo way of
building VMs, I'd be fine with that, since that would unify a lot
of infrastructure :)

Best
        -Tobias

[1] https://github.com/krono/self/blob/cmake/vm/cmake/mac_osx.cmake

--
Best regards,
Igor Stasenko.

--
best,

Eliot