Quantcast

Teleplace Cog VMs are now available

classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Squeak 4.1 on Cog (Re: Teleplace Cog VMs are now available)

laza
With "same image, same system" I was just referring to my setup (old notebook).

2010/6/21 Göran Krampe <[hidden email]>
On 06/21/2010 10:49 AM, Alexander Lazarević wrote:
Just for the heck of it:

Squeak = Squeak-4.0.3.2202-linux_i386 on Ubuntu 10.4
CogVM = wine and win32 CogVM on Ubuntu 10.4 same image, same system

[Smalltalk browseAllCallsOn: #do:] timeToRun.

 Squeak: 1984
 CogVM:   877
[SNIP]

Interesting, you don't seem to get as good numbers as Andreas did? Do you guys have different CPUs too?

regards, Göran





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Richard Durr-2
In reply to this post by Paolo Bonzini-2
Yay, I did not think this would happen anymore!
Great Work and many thanks!

On Mon, Jun 21, 2010 at 11:41 AM, Paolo Bonzini <[hidden email]> wrote:

> On 06/20/2010 10:11 PM, Eliot Miranda wrote:
>>
>> Hi All,
>>
>> it gives me great pleasure to announce that the Teleplace Cog VMs are
>> now available.  Huge thanks to all at Teleplace who have given me the
>> opportunity to build Cog and release it as open source
>
> Great news, thanks Eliot for your work!
>
> Paolo
>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Levente Uzonyi-2
In reply to this post by Eliot Miranda-2
On Sun, 20 Jun 2010, Eliot Miranda wrote:

> Hi All,
>
>
> it gives me great pleasure to announce that the Teleplace Cog VMs are now
> available.  Huge thanks to all at Teleplace who have given me the

This is fantastic, thanks.

<snip>

> attempt to port it, e.g. to ARM, PowerPC and x86-64.  The Cogit (excuse the
> pun) is so named because it is both an interpreter and a JIT, choosing not
> to generate machine code for large methods, interpreting them instead, the
> default policy being not to JIT methods with more than 60 literals.

I think it would be better to use an adaptive policy later. Something
like: all methods will be inlined if they are evaluated enough times. Say
a method with 10 literals will be inlined if it's evaluated 100 times. A
method with 100 literals will need 10000 runs, etc. (note: i just made up
the numbers).


Levente

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Jecel Assumpcao Jr.
In reply to this post by Eliot Miranda-2
Eliot Miranda wrote on Sun, 20 Jun 2010 13:11:38 -0700

> it gives me great pleasure to announce that the Teleplace Cog VMs are now available.
>  Huge thanks to all at Teleplace who have given me the opportunity to build Cog and
> release it as open source, been willing guinea pigs braving its bugs, and providing
> indispensable participation in getting Cog to its current state.  Huge thanks are also
> due to the original Back To The Future team whose VMMaker Cog extends to write
> the VM, and to Peter Deutsch from whom I've taken many ideas.

Congratulations and thanks to Eliot and all involved!

One interesting use for this would be improving the performance of Etoys
on the old OLPC XO-1 machines currently in use by over one million
children around the world. The "few SSE2 instructions" mentioned in
http://wiki.laptop.org/go/Geode_instruction_set might be a problem and
loading projects into an image with a different format would certainly
be a problem (though Andreas has suggested a way to deal with that). If
these obstacles can be overcome, however, it would avoid splitting the
OLPC community into early adopters and owners of shiny new XO-1.5,
XO-1.75 and XO-3 devices.

-- Jecel


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Squeak 4.1 on Cog (Re: Teleplace Cog VMs are now available)

Levente Uzonyi-2
In reply to this post by Andreas.Raab
On Sun, 20 Jun 2010, Andreas Raab wrote:

> Folks -
>
> I've prepared a version of Squeak 4.1 for those of you who are not into
> recompiling VMs yourself. You can download it from:
>
> http://squeakvm.org/win32/experimental/CogSqueak4.1.zip
>
> The package contains a freshly compiled Cog VM for Windows and an updated
> Squeak4.1 image. Some things to note:
>
> * Once you save the image with Cog, you can only run it with Cog. So load
> your benchmarks into the image *before* you launch it with Cog (i.e., to
> ensure you're comparing apples with apples you should use the same image)
>
> * There are some changes that are required in the image. Most importantly,
> some LargeInt primitives need to be removed:
> #(= bitAnd: bitOr: bitShift: bitXor: ~=) do:[:sel|
> LargePositiveInteger removeSelector: sel.
> ].
>
> This will be necessary for any images you want to run with Cog. There are
> also some other changes but they are not critical (i.e., don't crash the VM).

Thanks. I tried some basic things with this image and crashed the VM a few
times. For example updating from trunk crashes while the image is
recompiling classes. An updated trunk image gives much better results. A
few things are different or were not adopted to the VM:
- Juan's BitBlt fixes seem to be missing
- primitive 71 has an overflow bug (probably the same older Squeak VMs
had). Allocating an array with 1073741824 elements gives an empty array,
with 1073741825 elements results in an array with one slot.
- TestObjectsAsMethods crashes the VM
- Some VM parameters (Smalltalk vmParameters) are not integers but nil.
This causes problems with MessageTally.

And some good news: evaluating [Morph methodDict valuesDo: 1] works as it
should (http://bugs.squeak.org/view.php?id=7453 ).


Levente

>
> Enjoy,
>  - Andreas
>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Performance (Re: Teleplace Cog VMs are now available)

hilaire
In reply to this post by Andreas.Raab
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Performance (Re: Teleplace Cog VMs are now available)

hilaire
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Juan Vuletich-4
In reply to this post by Eliot Miranda-2
Hi Eliot,

Eliot Miranda wrote:

>
> Hi All,
>
>
> it gives me great pleasure to announce that the Teleplace Cog VMs are
> now available.  Huge thanks to all at Teleplace who have given me the
> opportunity to build Cog and release it as open source, been willing
> guinea pigs braving its bugs, and providing indispensable
> participation in getting Cog to its current state.  Huge thanks are
> also due to the original Back To The Future team whose VMMaker Cog
> extends to write the VM, and to Peter Deutsch from whom I've taken
> many ideas.
>
...


This is wonderful news! Thank you, Andreas, and all the people at
Teleplace who made this possible!

I tried to run Cuis on the Windows VM Andreas did, and I'm having
trouble. I did as follows:

1) Start Cuis 2.4 with the Closure VM
2) Install the attached changeset (made after your post and the tweaks
3) Exit saving the image
4) Start it with the Cog VM

When starting, it does an invalid memory access, and terminates without
any error message or log. I guess I'd need to setup a VM building and
debugging environment, etc, and learn quite a bit to fix this myself. I
guess it would take you minutes to spot this, compared with the days I'd
need to spend... I'd very much appreciate your help.

Thank you.

Cheers,
Juan Vuletich



CogCuis.2.zip (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

A New Horse (was [squeak-dev] Teleplace Cog VMs are now available)

Casey Ransberger-2
In reply to this post by Eliot Miranda-2
Excellence! Many thanks to everyone at Teleplace who made this possible, and special thanks to Eliot Miranda: I believe that your creation will be a great and lasting asset to this community. Thanks for putting up with all of our incessant inquiry:) 

I think it's especially cool to have an open source VM of this class written in Slang. I know that I personally am fascinated with e.g. inline caching, and I'm sure I'm not alone. A fast VM, and one that is well situated to become a reference implementation, is about the greatest gift a guy like me could hope for, and it's not even Christmas. 

I look forward to learning everything that I can from Cog!

"It's like they just gave you a new horse." --attrib. Dan Ingalls



On Jun 20, 2010, at 1:11 PM, Eliot Miranda <[hidden email]> wrote:

Hi All,


it gives me great pleasure to announce that the Teleplace Cog VMs are now available.  Huge thanks to all at Teleplace who have given me the opportunity to build Cog and release it as open source, been willing guinea pigs braving its bugs, and providing indispensable participation in getting Cog to its current state.  Huge thanks are also due to the original Back To The Future team whose VMMaker Cog extends to write the VM, and to Peter Deutsch from whom I've taken many ideas.


This release contains two VMs.  The Stack VM, is a cross-platform interpreter that uses context-to-stack mapping to achieve modest performance gains.  The Cog VM is a just-in-time compiler that currently supports only x86 that builds upon the Stack VM to achieve substantial performance improvements.  The release is in the form of a Monticello package containing the VMMaker source and a tarball containing the platform sources, the generated sources and a Squeak 4.1 image containing the VMMaker sources.  Download both at

http://ftp.squeak.org/Cog/VMMaker-oscog.11.mcz

http://ftp.squeak.org/Cog/OpenSourceCog.tar.gz


Cog VMs:


The Cog VMs are Squeak/Croquet VMs that run closure Squeak/Croquet/Pharo/Cuis images. The VMs support existing plugin source but will require plugins to be recompiled as the VM_PROXY_MAJOR plugin api has been extended.


This release contains two distinct VMs, the StackInterpreter and the Cogit.  The StackInterpreter is a fully-portable plug-in replacement for the current closure Squeak VMs and images.  The Stack VM uses context-to-stack mapping and a somewhat improved garbage collector to achieve modest but useful performance gains in the 10% to 15% range.  The StackInterpreter is intended to supersede the Squeak VM on platforms where the Cogit cannot be used.  The Cogit extends the StackInterpreter with a just-in-time compiler that uses aggressive inline caching techniques to deliver substantial performance gains in the 3x to 15x range, depending on benchmark.  The Cogit currently supports only x86 and the floating-point primitives and parts of the platform support code depend on SSE2.  I hope members of the community will attempt to port it, e.g. to ARM, PowerPC and x86-64.  The Cogit (excuse the pun) is so named because it is both an interpreter and a JIT, choosing not to generate machine code for large methods, interpreting them instead, the default policy being not to JIT methods with more than 60 literals.


The Cog VM requires a few minor image changes all in image/NecessaryImageChangesForCogToWork.1.cs.  The JIT's machine-code SmallInteger primitives insist on a SmallInteger receiver so the primitives in LargePositiveInteger = ~= bitAnd: bitOr: butShift: and bitXor: cannot be used and these methods must be deleted.  The Cogit inlines the address of the Character instance table, Smalltalk specialObjectsArray at: 25, into the machine-code at: primitive for faster ByteString>>at: and so the table cannot be rebuilt in SmalltalkImage>>recreateSpecialObjectsArray.  The new version preserves the existing table.  Both VMs maintain floats in platform order to ease implementation of machine code floating-point primitives, and hence internally are in little-endian order instead of big-endian in current Squeak images.  While the VMs convert float order automatically on load they do require special accessing primitives Float>>basicAt: & Float>>basicAt:put: that undo the reversal and answer Float contents in big-endian order so that e.g. Float>>hash is unchanged.  The methods assume these primitives can fail, allowing the code to be used on current Squeak VMs.


The image/VMMaker-Squeak4.1.image is a Squeak 4.1 image, runnable with the current Squeak VMs, that contains these changes, and can hence also be run with a Cog VM.  But beware, once an image has been saved on Cog it cannot be run by an existing Squeak VM, because existing VMs cannot undo the Float order change.



Platform Subsystem:


Most of the platform subsystem is unchanged but there are some important changes that need description.  The biggest change is the heartbeat and the clock in platforms/unix/vm/sqUnixHeartbeat.c and platforms/win32/vm/sqWin32Heartbeat.c.  The Cog VMs avoid the slow and variable interruptCheckCounter, folding the event check into the stack overflow check on frame build.  The heartbeat, typically 500Hz or 1KHz, changes the stackLimit to a value that will always fail.  On the next frame building send the VM will enter stack overflow handling that, as a side effect, will also check for events.  This is more efficient than the update of interruptCheckCounter and much more regular.  If one is running code that executes long-running primitives (e.g. large integer arithmetic) the counter approach will result in too low an interrupt check frequency, and conversely if one is running normal code the interrupt check frequency can be very high.


The heartbeat also maintains a 64-bit microsecond clock, UTC microseconds from 1901, from which the backward-compatible millisecond and second clocks are derived.  Primitives exist to answer UTC microseconds and local microseconds.  Updating the clock in the heartbeat results in a 1 or 2 millisecond resolution but avoids the cost of accessing the OS time on every prim tie which we've found important for performance at Teleplace.  The 64-bit microsecond clocks provide a unified time basis and eliminate wrapping (for the next 54,000 years at least).  I hope community images will move to these clocks.  It's worked well in VisualWorks.


Another significant change is in the external semaphore table support code.  This is now lock-free at the cost of having to specify a maximum number of external semaphores at start-up (default 256).  The support code for the lock-free data structures are processor-specific and is currently implemented only for x86 and gcc-compatible compilers; see platforms/Cross/vm/{sqAtomicOps.h,sqMemoryFence.h}.


There is also improved crash reporting code that prints a primitive log and a C backtrace in addition to the Smalltalk backtrace.  See platforms/Mac OS/vm/sqMacMain.c, platforms/unix/vm/sqUnixMain.c, platforms/win32/vm/sqWin32Intel.c & platforms/win32/vm/sqWin32Backtrace.c.


Finally there is support for the QVMProfiler, a pc-sampling profiler for profiling at the VM level.  See platforms/unix/vm/sqUnixVMProfile.c and platforms/win32/vm/sqWin32VMProfile.c.  The profiler itself is in the VMMaker image described below in Qwaq-VMProfiling.


There are also changes to do with Teleplace-specific extensions to the HostWindowPlugin but these are not essential to Cog.



VMMaker and Slang:


The image/VMMaker-Squeak4.1.image Squeak 4.1 image contains the complete Cog VMMaker with necessary support code for simulation. This image was used to generate the sources in the src and stacksrc directories.


Cog's VMMaker is substantially revised and extended from the current VMMaker.  It supports multiple classes, not just Interpreter and superclasses, because both context-to-stack mapping and the Cogit are too complex to write monolithically.  Classes can specify ancilliaryClasses and ancilliaryStructClasses, such as CoInterpreterStackPage, CogMethod and CogAbstractInstruction.  The Monticello package version is included in the header of all generated files and constitutes the version stamp for generated code.  Code is generated in sorted order so that minor changes in the Smalltalk source produce correspondingly minor changes in the generated code.  The gnuification step is built-in to VMMaker.  No effort has been made to maintain 64-bit compatibility.  Apologies, this was unaffordable.


The VMMaker generates a single source tree used by all platforms.  Instead of deciding at generation time whether to use the Interpreter struct the generated code depends on the SQ_USE_GLOBAL_STRUCT define which can be overridden in platform makefiles.  All plugins live in src/plugins and platform makefiles along with plugins.int and plugins.ext files in the build subdirectories decide which plugins are built as external or internal.  The VM Generation Workspace from Workspace.text workspace contains dots to generate the sources.  We no longer use the VMMakerTool since there should be nothing platform-specific in the generated sources (if we add ports to other ISAs all their source can be included and selected as required by the platform makefiles).


Since the Cogit generates x86 machine code simulation is much more complex.  There is a support plugin, platforms/Cross/plugins/BochsIA32Plugin that depends on a large simulation of the x86 family implemented in C++ (see  processors/IA32/bochs) and on Alien.  I use the simulator frequently (but note that I haven't had time to build a working version for Squeak 4.1).  I have tested Cog simulation in this image, running on the image/VMMaker-Squeak4.1.image itself.  The VM Simulation Workspace in the VMMaker image contains an example doit that starts the simulator. Be patient, even on a fast machine unhibernating the Squeak display background image takes nearly a minute.  Native fonts do not (yet) simulate correctly, but the system runs.  But note that I have only attempted to build and run the simulator on Mac OS X.  I expect Bochs can be built on linux and win32 but I have not tried.  By the way, I've not described how to run the Bochs simulator on the current Squeak VM.  That's because the plugin depends on the heartbeat to break out of simulation occasionally via a new interpreterProxy entry point setInterruptCheckChain.  As this isn't supported by the current Squeak VMs the plugin would require modification.  So to simulate first build either of the Cog VMs and then run the simulation with it.


There are a number of unpublished changes to the base other than those in NecessaryImageChangesForCogToWork.1.cs.  This is partly laziness on my part, partly avoiding publishing things in advance of Cog.  These changes are better motivated once Cog is in use.  There are changes to the "translated primitives" (see implementors of translatedPrimitives) which replace messages with method tags for generation directives.  The Cog VMMaker uses Object>>perform:with:with:with:with: & Object>>perform:with:with:with:with:with: during simulation, and Collection>>#fold: & SquenceableCollection>>#copyUpThrough: during generation.  Object>>inline: and Object var:declareC:, which are mispackaged in Kernel in Squeak 4.1 are obsolete (method tags being used instead) and have been removed. I have changed Integer>>hex and Integer>>hex8 back to their original semantics as of 3.8.  Backward compatibility is important and one can easily add new selectors if one wants different functionality.  VMMaker was here first ;)



Tarball:


The top-level directories in the tarball are


src

the tree for the Cog generated sources including all plugins

stacksrc/vm

the directory containing the Stack VM source (plugins can be taken from above)

platforms

the usual svn platform tree but including Cog specific changes such as the heartbeat

processors

the tree containing simulation support code, i.e. the bochs C++ x86 simulation library, along with a potential ARM, PowerPC & MIPS simulator, Skeye.


image

the Cog-prepared Squeak 4.1 VMMaker image

scripts

some svn scripts to revert unchanged plugins that haven't really changed


cygwinbuild

the win32 build directory

winbuild

the old win32 build directory for minnow gcc 2.95.  Not entirely obsolete as the cygwin build as yet fails to generate a functional FFIPlugin

macbuild

the CoreVM.xcodeproj and support build projects for Mac OS X 10.5 or better

unixbuild

the build directory for linux



Building Cog:


Each build directory above contains a HowToBuild file that describes building in more detail.  The build directories only contain Cogit makefiles.  f you want to build a Stack VM you're on your own but this is very close to the existing Squeak VM build.



Status:

The Cogit VM has been our sole VM at Teleplace for nearly a year.  We do occasionally find bugs and there are almost certainly areas of functionality that we have not touched (for example I know that co-routining does not yet work).  If you find a bug please try and create a reproducible test case and let me know.  I can't promise to take a look or fix it but I am motivated to do so and will try my best as time allows.  Better still if you find and fix bugs be sure to let me know.


License (MIT):

All contributions from Teleplace in this release are

Copyright (c) 2010 Teleplace, Inc.


Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:


The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.


THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN

THE SOFTWARE.


Eliot Miranda

June 2010





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Chris Muller-3
In reply to this post by Eliot Miranda-2
Oh my gosh, I have been thinking about this moment for years, and your
note took me by surprise.  This is incredible!  I will try it asap.

Eliot, congratulations!  You, Teleplaces and Andreas have my sincere
gratitude, thank you.

Thank you!
  Chris



On Sun, Jun 20, 2010 at 3:11 PM, Eliot Miranda <[hidden email]> wrote:

> Hi All,
>
> it gives me great pleasure to announce that the Teleplace Cog VMs are now
> available.  Huge thanks to all at Teleplace who have given me the
> opportunity to build Cog and release it as open source, been willing guinea
> pigs braving its bugs, and providing indispensable participation in getting
> Cog to its current state.  Huge thanks are also due to the original Back To
> The Future team whose VMMaker Cog extends to write the VM, and to Peter
> Deutsch from whom I've taken many ideas.
>
> This release contains two VMs.  The Stack VM, is a cross-platform
> interpreter that uses context-to-stack mapping to achieve modest performance
> gains.  The Cog VM is a just-in-time compiler that currently supports only
> x86 that builds upon the Stack VM to achieve substantial performance
> improvements.  The release is in the form of a Monticello package containing
> the VMMaker source and a tarball containing the platform sources, the
> generated sources and a Squeak 4.1 image containing the VMMaker sources.
> Download both at
>
> http://ftp.squeak.org/Cog/VMMaker-oscog.11.mcz
>
> http://ftp.squeak.org/Cog/OpenSourceCog.tar.gz
>
> Cog VMs:
>
> The Cog VMs are Squeak/Croquet VMs that run closure
> Squeak/Croquet/Pharo/Cuis images. The VMs support existing plugin source but
> will require plugins to be recompiled as the VM_PROXY_MAJOR plugin api has
> been extended.
>
> This release contains two distinct VMs, the StackInterpreter and the Cogit.
> The StackInterpreter is a fully-portable plug-in replacement for the current
> closure Squeak VMs and images.  The Stack VM uses context-to-stack mapping
> and a somewhat improved garbage collector to achieve modest but useful
> performance gains in the 10% to 15% range.  The StackInterpreter is intended
> to supersede the Squeak VM on platforms where the Cogit cannot be used.  The
> Cogit extends the StackInterpreter with a just-in-time compiler that uses
> aggressive inline caching techniques to deliver substantial performance
> gains in the 3x to 15x range, depending on benchmark.  The Cogit currently
> supports only x86 and the floating-point primitives and parts of the
> platform support code depend on SSE2.  I hope members of the community will
> attempt to port it, e.g. to ARM, PowerPC and x86-64.  The Cogit (excuse the
> pun) is so named because it is both an interpreter and a JIT, choosing not
> to generate machine code for large methods, interpreting them instead, the
> default policy being not to JIT methods with more than 60 literals.
>
> The Cog VM requires a few minor image changes all in
> image/NecessaryImageChangesForCogToWork.1.cs.  The JIT's machine-code
> SmallInteger primitives insist on a SmallInteger receiver so the primitives
> in LargePositiveInteger = ~= bitAnd: bitOr: butShift: and bitXor: cannot be
> used and these methods must be deleted.  The Cogit inlines the address of
> the Character instance table, Smalltalk specialObjectsArray at: 25, into the
> machine-code at: primitive for faster ByteString>>at: and so the table
> cannot be rebuilt in SmalltalkImage>>recreateSpecialObjectsArray.  The new
> version preserves the existing table.  Both VMs maintain floats in platform
> order to ease implementation of machine code floating-point primitives, and
> hence internally are in little-endian order instead of big-endian in current
> Squeak images.  While the VMs convert float order automatically on load they
> do require special accessing primitives Float>>basicAt: &
> Float>>basicAt:put: that undo the reversal and answer Float contents in
> big-endian order so that e.g. Float>>hash is unchanged.  The methods assume
> these primitives can fail, allowing the code to be used on current Squeak
> VMs.
>
> The image/VMMaker-Squeak4.1.image is a Squeak 4.1 image, runnable with the
> current Squeak VMs, that contains these changes, and can hence also be run
> with a Cog VM.  But beware, once an image has been saved on Cog it cannot be
> run by an existing Squeak VM, because existing VMs cannot undo the Float
> order change.
>
> Platform Subsystem:
>
> Most of the platform subsystem is unchanged but there are some important
> changes that need description.  The biggest change is the heartbeat and the
> clock in platforms/unix/vm/sqUnixHeartbeat.c and
> platforms/win32/vm/sqWin32Heartbeat.c.  The Cog VMs avoid the slow and
> variable interruptCheckCounter, folding the event check into the stack
> overflow check on frame build.  The heartbeat, typically 500Hz or 1KHz,
> changes the stackLimit to a value that will always fail.  On the next frame
> building send the VM will enter stack overflow handling that, as a side
> effect, will also check for events.  This is more efficient than the update
> of interruptCheckCounter and much more regular.  If one is running code that
> executes long-running primitives (e.g. large integer arithmetic) the counter
> approach will result in too low an interrupt check frequency, and conversely
> if one is running normal code the interrupt check frequency can be very
> high.
>
> The heartbeat also maintains a 64-bit microsecond clock, UTC microseconds
> from 1901, from which the backward-compatible millisecond and second clocks
> are derived.  Primitives exist to answer UTC microseconds and local
> microseconds.  Updating the clock in the heartbeat results in a 1 or 2
> millisecond resolution but avoids the cost of accessing the OS time on every
> prim tie which we've found important for performance at Teleplace.  The
> 64-bit microsecond clocks provide a unified time basis and eliminate
> wrapping (for the next 54,000 years at least).  I hope community images will
> move to these clocks.  It's worked well in VisualWorks.
>
> Another significant change is in the external semaphore table support code.
> This is now lock-free at the cost of having to specify a maximum number of
> external semaphores at start-up (default 256).  The support code for the
> lock-free data structures are processor-specific and is currently
> implemented only for x86 and gcc-compatible compilers; see
> platforms/Cross/vm/{sqAtomicOps.h,sqMemoryFence.h}.
>
> There is also improved crash reporting code that prints a primitive log and
> a C backtrace in addition to the Smalltalk backtrace.  See platforms/Mac
> OS/vm/sqMacMain.c, platforms/unix/vm/sqUnixMain.c,
> platforms/win32/vm/sqWin32Intel.c & platforms/win32/vm/sqWin32Backtrace.c.
>
> Finally there is support for the QVMProfiler, a pc-sampling profiler for
> profiling at the VM level.  See platforms/unix/vm/sqUnixVMProfile.c and
> platforms/win32/vm/sqWin32VMProfile.c.  The profiler itself is in the
> VMMaker image described below in Qwaq-VMProfiling.
>
> There are also changes to do with Teleplace-specific extensions to the
> HostWindowPlugin but these are not essential to Cog.
>
> VMMaker and Slang:
>
> The image/VMMaker-Squeak4.1.image Squeak 4.1 image contains the complete Cog
> VMMaker with necessary support code for simulation. This image was used to
> generate the sources in the src and stacksrc directories.
>
> Cog's VMMaker is substantially revised and extended from the current
> VMMaker.  It supports multiple classes, not just Interpreter and
> superclasses, because both context-to-stack mapping and the Cogit are too
> complex to write monolithically.  Classes can specify ancilliaryClasses and
> ancilliaryStructClasses, such as CoInterpreterStackPage, CogMethod and
> CogAbstractInstruction.  The Monticello package version is included in the
> header of all generated files and constitutes the version stamp for
> generated code.  Code is generated in sorted order so that minor changes in
> the Smalltalk source produce correspondingly minor changes in the generated
> code.  The gnuification step is built-in to VMMaker.  No effort has been
> made to maintain 64-bit compatibility.  Apologies, this was unaffordable.
>
> The VMMaker generates a single source tree used by all platforms.  Instead
> of deciding at generation time whether to use the Interpreter struct the
> generated code depends on the SQ_USE_GLOBAL_STRUCT define which can be
> overridden in platform makefiles.  All plugins live in src/plugins and
> platform makefiles along with plugins.int and plugins.ext files in the build
> subdirectories decide which plugins are built as external or internal.  The
> VM Generation Workspace from Workspace.text workspace contains dots to
> generate the sources.  We no longer use the VMMakerTool since there should
> be nothing platform-specific in the generated sources (if we add ports to
> other ISAs all their source can be included and selected as required by the
> platform makefiles).
>
> Since the Cogit generates x86 machine code simulation is much more complex.
> There is a support plugin, platforms/Cross/plugins/BochsIA32Plugin that
> depends on a large simulation of the x86 family implemented in C++ (see
> processors/IA32/bochs) and on Alien.  I use the simulator frequently (but
> note that I haven't had time to build a working version for Squeak 4.1).  I
> have tested Cog simulation in this image, running on the
> image/VMMaker-Squeak4.1.image itself.  The VM Simulation Workspace in the
> VMMaker image contains an example doit that starts the simulator. Be
> patient, even on a fast machine unhibernating the Squeak display background
> image takes nearly a minute.  Native fonts do not (yet) simulate correctly,
> but the system runs.  But note that I have only attempted to build and run
> the simulator on Mac OS X.  I expect Bochs can be built on linux and win32
> but I have not tried.  By the way, I've not described how to run the Bochs
> simulator on the current Squeak VM.  That's because the plugin depends on
> the heartbeat to break out of simulation occasionally via a new
> interpreterProxy entry point setInterruptCheckChain.  As this isn't
> supported by the current Squeak VMs the plugin would require modification.
> So to simulate first build either of the Cog VMs and then run the simulation
> with it.
>
> There are a number of unpublished changes to the base other than those in
> NecessaryImageChangesForCogToWork.1.cs.  This is partly laziness on my part,
> partly avoiding publishing things in advance of Cog.  These changes are
> better motivated once Cog is in use.  There are changes to the "translated
> primitives" (see implementors of translatedPrimitives) which replace
> messages with method tags for generation directives.  The Cog VMMaker uses
> Object>>perform:with:with:with:with: &
> Object>>perform:with:with:with:with:with: during simulation, and
> Collection>>#fold: & SquenceableCollection>>#copyUpThrough: during
> generation.  Object>>inline: and Object var:declareC:, which are mispackaged
> in Kernel in Squeak 4.1 are obsolete (method tags being used instead) and
> have been removed. I have changed Integer>>hex and Integer>>hex8 back to
> their original semantics as of 3.8.  Backward compatibility is important and
> one can easily add new selectors if one wants different functionality.
> VMMaker was here first ;)
>
> Tarball:
>
> The top-level directories in the tarball are
>
> src
>
> the tree for the Cog generated sources including all plugins
>
> stacksrc/vm
>
> the directory containing the Stack VM source (plugins can be taken from
> above)
>
> platforms
>
> the usual svn platform tree but including Cog specific changes such as the
> heartbeat
>
> processors
>
> the tree containing simulation support code, i.e. the bochs C++ x86
> simulation library, along with a potential ARM, PowerPC & MIPS simulator,
> Skeye.
>
> image
>
> the Cog-prepared Squeak 4.1 VMMaker image
>
> scripts
>
> some svn scripts to revert unchanged plugins that haven't really changed
>
> cygwinbuild
>
> the win32 build directory
>
> winbuild
>
> the old win32 build directory for minnow gcc 2.95.  Not entirely obsolete as
> the cygwin build as yet fails to generate a functional FFIPlugin
>
> macbuild
>
> the CoreVM.xcodeproj and support build projects for Mac OS X 10.5 or better
>
> unixbuild
>
> the build directory for linux
>
> Building Cog:
>
> Each build directory above contains a HowToBuild file that describes
> building in more detail.  The build directories only contain Cogit
> makefiles.  f you want to build a Stack VM you're on your own but this is
> very close to the existing Squeak VM build.
>
> Status:
>
> The Cogit VM has been our sole VM at Teleplace for nearly a year.  We do
> occasionally find bugs and there are almost certainly areas of functionality
> that we have not touched (for example I know that co-routining does not yet
> work).  If you find a bug please try and create a reproducible test case and
> let me know.  I can't promise to take a look or fix it but I am motivated to
> do so and will try my best as time allows.  Better still if you find and fix
> bugs be sure to let me know.
>
> License (MIT):
>
> All contributions from Teleplace in this release are
>
> Copyright (c) 2010 Teleplace, Inc.
>
> Permission is hereby granted, free of charge, to any person obtaining a copy
>
> of this software and associated documentation files (the "Software"), to
> deal
>
> in the Software without restriction, including without limitation the rights
>
> to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>
> copies of the Software, and to permit persons to whom the Software is
>
> furnished to do so, subject to the following conditions:
>
> The above copyright notice and this permission notice shall be included in
>
> all copies or substantial portions of the Software.
>
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>
> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>
> FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>
> AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>
> LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> FROM,
>
> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>
> THE SOFTWARE.
>
> Eliot Miranda
>
> June 2010
>
>
>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Squeak 4.1 on Cog (Re: Teleplace Cog VMs are now available)

Andreas.Raab
In reply to this post by laza
On 6/21/2010 1:49 AM, Alexander Lazarević wrote:
> Just for the heck of it:
>
> Squeak = Squeak-4.0.3.2202-linux_i386 on Ubuntu 10.4
> CogVM = wine and win32 CogVM on Ubuntu 10.4 same image, same system
>
> [Smalltalk browseAllCallsOn: #do:] timeToRun.
>
>   Squeak: 1984
>   CogVM:   877

That's ... odd. Perhaps some Wine thingie? We get *excellent*
performance improvements on our RHEL servers in fact more so than on the
client since the client is so much more I/O bound with graphics etc.

> PS: Running all tests with the win32 CogVM using wine will produce a
> fatal vm crash at some point

Yup. TestObjectsAsMethods will die. I've just posted a fix for the trunk.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Andreas.Raab
In reply to this post by Juan Vuletich-4
On 6/21/2010 11:29 AM, Juan Vuletich wrote:
> When starting, it does an invalid memory access, and terminates without
> any error message or log. I guess I'd need to setup a VM building and
> debugging environment, etc, and learn quite a bit to fix this myself. I
> guess it would take you minutes to spot this, compared with the days I'd
> need to spend... I'd very much appreciate your help.

I've seen that before :-) Cog requires LargeNegativeInteger to be
compact at 5 (previously PseudoContext). After installing and running
the attached code in addition to the other changes you're good to go.

Enjoy!
   - Andreas



Integer class-initialize.st (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

espin
In reply to this post by Eliot Miranda-2
Eliot, Andreas, Teleplace people,
many thanks for this inspiring present.

Bye
Enrico

On Sun, Jun 20, 2010 at 22:11, Eliot Miranda <[hidden email]> wrote:

Hi All,


it gives me great pleasure to announce that the Teleplace Cog VMs are now available.  Huge thanks to all at Teleplace who have given me the opportunity to build Cog and release it as open source, been willing guinea pigs braving its bugs, and providing indispensable participation in getting Cog to its current state.  Huge thanks are also due to the original Back To The Future team whose VMMaker Cog extends to write the VM, and to Peter Deutsch from whom I've taken many ideas.


[...] 

Eliot Miranda

June 2010








--
Enrico Spinielli
"Do Androids dream of electric sheep?"— Philip K. Dick
"Hear and forget; see and remember;do and understand."—Mitchel Resnick


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Frank Shearar
Yes: thank you Eliot, Andreas, and Teleplace.

And with this awesome gift, there's an obligation: now we must go forth
and build compelling applications on top of our spanking new engine!

frank

On 2010/06/22 09:49, Enrico Spinielli wrote:

> Eliot, Andreas, Teleplace people,
> many thanks for this inspiring present.
>
> Bye
> Enrico
>
> On Sun, Jun 20, 2010 at 22:11, Eliot Miranda <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi All,
>
>
>     it gives me great pleasure to announce that the Teleplace Cog VMs
>     are now available.  Huge thanks to all at Teleplace who have given
>     me the opportunity to build Cog and release it as open source, been
>     willing guinea pigs braving its bugs, and providing indispensable
>     participation in getting Cog to its current state.  Huge thanks are
>     also due to the original Back To The Future team whose VMMaker Cog
>     extends to write the VM, and to Peter Deutsch from whom I've taken
>     many ideas.
>
>
>     [...]
>
>     Eliot Miranda
>
>     June 2010
>
>
>
>
>
>
>
>
> --
> Enrico Spinielli
> "Do Androids dream of electric sheep?"— Philip K. Dick
> "Hear and forget; see and remember;do and understand."—Mitchel Resnick
>
>
>
>


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Igor Stasenko
A long anticipated event! :)


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Juan Vuletich-4
In reply to this post by Andreas.Raab
Hi Andreas,

Andreas Raab wrote:

> On 6/21/2010 11:29 AM, Juan Vuletich wrote:
>> When starting, it does an invalid memory access, and terminates without
>> any error message or log. I guess I'd need to setup a VM building and
>> debugging environment, etc, and learn quite a bit to fix this myself. I
>> guess it would take you minutes to spot this, compared with the days I'd
>> need to spend... I'd very much appreciate your help.
>
> I've seen that before :-) Cog requires LargeNegativeInteger to be
> compact at 5 (previously PseudoContext). After installing and running
> the attached code in addition to the other changes you're good to go.
>
> Enjoy!
>   - Andreas

Yes. I yesterday downloaded XCode and built the Mac VM, only to find
that it throws a clear error (LargeNegativeInteger is not compact at 5,
can't continue) to the console. I told this to Eliot, but forgot to say
it in the lists. I apologize. Anyway, for some reason, the win vm you
posted, instead of giving the error and exiting cleanly, it does an
invalid access and is terminated. Maybe you want to check that.

BTW, most console messages (i.e. stdout) will be missed in these days of
graphical shells. An OS dialog would be better.

Thank Eliot, you and Teleplace for this great gift! This is a major
milestone in the history of Squeak.

Cheers,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Denis Kudriashov
In reply to this post by Eliot Miranda-2
Hello.

Cog performance is incredible. My application improved 3 times speedup. And I now does not need special caching logic in my program.

But I found very strange behavior in xml dom parsing of big text with many russian text (by XML-Parser XMLDOMParser class).

russionXml := String streamContents: [:s | s nextPutAll: '<root>'.
    1000 timesRepeat: [
        s nextPutAll: '<item>'.
        s nextPutAll: 'фывапролд'.
        s nextPutAll: '</item>'].
    s nextPutAll: '</root>'
    ].

[XMLDOMParser parseDocumentFrom: russionXml] timeToRun .

In pharo 1.1 I have

567 ms with old VM.
6673 ms with Cog VM.

But all other stuff that I tested work faster with Cog (at 3-4 times faster).





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Igor Stasenko
2010/6/22 Denis Kudriashov <[hidden email]>:

> Hello.
>
> Cog performance is incredible. My application improved 3 times speedup. And
> I now does not need special caching logic in my program.
>
> But I found very strange behavior in xml dom parsing of big text with many
> russian text (by XML-Parser XMLDOMParser class).
>
> russionXml := String streamContents: [:s | s nextPutAll: '<root>'.
>     1000 timesRepeat: [
>         s nextPutAll: '<item>'.
>         s nextPutAll: 'фывапролд'.
>         s nextPutAll: '</item>'].
>     s nextPutAll: '</root>'
>     ].
>
> [XMLDOMParser parseDocumentFrom: russionXml] timeToRun .
>
> In pharo 1.1 I have
>
> 567 ms with old VM.
> 6673 ms with Cog VM.
>
> But all other stuff that I tested work faster with Cog (at 3-4 times
> faster).
>

Looks like the code using different paths.
I suspect that in one case, some operations are done primitively,
while in another one - interpreted.

>
>
>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Levente Uzonyi-2
In reply to this post by Denis Kudriashov
On Wed, 23 Jun 2010, Denis Kudriashov wrote:

> Hello.
>
> Cog performance is incredible. My application improved 3 times speedup. And
> I now does not need special caching logic in my program.
>
> But I found very strange behavior in xml dom parsing of big text with many
> russian text (by XML-Parser XMLDOMParser class).
>
> russionXml := String streamContents: [:s | s nextPutAll: '<root>'.
>    1000 timesRepeat: [
>        s nextPutAll: '<item>'.
>        s nextPutAll: '?????????'.
>        s nextPutAll: '</item>'].
>    s nextPutAll: '</root>'
>    ].
>
> [XMLDOMParser parseDocumentFrom: russionXml] timeToRun .

I thought it's time to make use of the profiler, so I tried the same
tests
in a Squeak trunk image (4.2alpha-10280). To reduce the noise I ran the
test 5 times:

russionXml := String streamContents: [:s | s nextPutAll: '<root>'.
     1000 timesRepeat: [
         s nextPutAll: '<item>'.
         s nextPutAll: '.........'.
         s nextPutAll: '</item>'].
     s nextPutAll: '</root>'
     ].
((1 to: 5) collect: [ :run |
  [ XMLDOMParser parseDocumentFrom: russionXml readStream ] timeToRun ]) average asFloat

My results are:
SqueakVM: 73.4
CogVM: 17.6

So CogVM is more than 4x faster than SqueakVM for this test.


Levente

>
> In pharo 1.1 I have
>
> 567 ms with old VM.
> 6673 ms with Cog VM.
>
> But all other stuff that I tested work faster with Cog (at 3-4 times
> faster).
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Teleplace Cog VMs are now available

Denis Kudriashov
In reply to this post by Igor Stasenko
It's strange. I loaded last version of XMLParser and all work very well (at 3 times faster)

2010/6/25 Igor Stasenko <[hidden email]>
2010/6/22 Denis Kudriashov <[hidden email]>:
> Hello.
>
> Cog performance is incredible. My application improved 3 times speedup. And
> I now does not need special caching logic in my program.
>
> But I found very strange behavior in xml dom parsing of big text with many
> russian text (by XML-Parser XMLDOMParser class).
>
> russionXml := String streamContents: [:s | s nextPutAll: '<root>'.
>     1000 timesRepeat: [
>         s nextPutAll: '<item>'.
>         s nextPutAll: 'фывапролд'.
>         s nextPutAll: '</item>'].
>     s nextPutAll: '</root>'
>     ].
>
> [XMLDOMParser parseDocumentFrom: russionXml] timeToRun .
>
> In pharo 1.1 I have
>
> 567 ms with old VM.
> 6673 ms with Cog VM.
>
> But all other stuff that I tested work faster with Cog (at 3-4 times
> faster).
>

Looks like the code using different paths.
I suspect that in one case, some operations are done primitively,
while in another one - interpreted.

>
>
>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.




123
Loading...