Spur performance report...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Spur performance report...

Eliot Miranda-2
 
Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

EstebanLM
 
wow! impressive. 

can’t wait to integrate that :)

Esteban

On 23 Apr 2014, at 00:05, Eliot Miranda <[hidden email]> wrote:

Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Nicolas Cellier
In reply to this post by Eliot Miranda-2
 
2014-04-23 0:05 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot


So this is loading .mcz from package cache, uncompressing, compiling, installing the packages.
I presume this qualifies as a macro benchmark...
I didn't follow the full flow of changes, how difficult is it right now to boostrap a spur.image?

Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Eliot Miranda-2
 
Hi Nicolas,

On Tue, Apr 22, 2014 at 3:21 PM, Nicolas Cellier <[hidden email]> wrote:
 
2014-04-23 0:05 GMT+02:00 Eliot Miranda <[hidden email]>:
 
Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot


So this is loading .mcz from package cache, uncompressing, compiling, installing the packages.

Exactly.
 
I presume this qualifies as a macro benchmark...

yes :-)
 
I didn't follow the full flow of changes, how difficult is it right now to bootstrap a spur.image?

Right now the transformation of the images is straight-forward, e.g.:

SpurBootstrap bootstrapImage: 'Squeak-4.5-All-in-One.app/Contents/Resources/Squeak4.5-13680'
 
But fixing the Monticello packages is a little more involved.  One has to collect the relevant packages in a directory (Kernel, Collections & System) and then do e.g.
(SpurBootstrapMonticelloPackagePatcher new
from: 'image/package-cache'
to: 'image/spur-package-cache')
patch
and then load the packages into the Spur image.

When I get my ass in gear this step should be unnecessary since http://source.squeak.org/spur is ready and waiting.  So the Spur trunk for Squeak V5.0 will parallel Squeak 4.6 and hopefully supplant it soon enough.

The main hold up right now is the combination of the facts that the current compaction algorithm doesn't work well, and image start-up always adds 50% of image size as free space, so the image balloons on each save.  I'm finally getting back to working on the compaction algorithm after a very satisfying three weeks working with Clément Béra on Sista.  So I hope to have movement on releasing Spur for the 4.6/5.0 development cycle soon.
--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] [Vm-dev] Spur performance report...

David T. Lewis
 
On Tue, Apr 22, 2014 at 03:44:46PM -0700, Eliot Miranda wrote:

> Hi Nicolas,
>
> On Tue, Apr 22, 2014 at 3:21 PM, Nicolas Cellier <
> [hidden email]> wrote:
>
> >
> > 2014-04-23 0:05 GMT+02:00 Eliot Miranda <[hidden email]>:
> >
> >>
> >> Hi All,
> >>
> >>     I should write a blog post on this, but I can't wait...
> >>
> >> In recent days I've written a script to build a Cog VMMaker image from
> >> Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.
> >>  This has allowed me to run the current Cog VM against Spur side-to-side.
> >>
> >> To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using
> >> the current Cog VM takes about 2 and a half minutes:
> >>
> >> McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st
> >> real    2m30.671s
> >> user    2m15.683s
> >> sys     0m5.283s
> >>
> >> To build the equivalent image using Spur takes about 1 and a half minutes:
> >> McStalker.image$ time spurcfvm CogVMMaker-spur.image
> >> BuildSqueak45Image.st
> >> real    1m34.943s
> >> user    1m23.666s
> >> sys     0m6.810s
> >>
> >> Comparing:
> >>
> >> 94.943 - 150.671 / 150.671 * 100 -36.99
> >> 83.666 - 135.681 / 135.681 * 100 -38.34
> >> 150.671 / 94.943 1.59
> >> 135.681 / 83.666 1.62
> >>
> >> that's about a -37% speedup, or 1.6x faster.
> >> --
> >> best,
> >> Eliot
> >>
> >>
> > So this is loading .mcz from package cache, uncompressing, compiling,
> > installing the packages.
> >
>
> Exactly.
>
>
> > I presume this qualifies as a macro benchmark...
> >
>
> yes :-)
>

That's quite impressive. I suspect that if you run this under a time profiler
that you'll see a lot of time going into I/O even though it is using a local
MC cache.  That would mean that this macro benchmark is quite conservative,
and the actual computational speedup may be considerably better than 1.6x.

Bravo!

Dave

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] [Vm-dev] Spur performance report...

Eliot Miranda-2
 
Hi David,


On Tue, Apr 22, 2014 at 4:21 PM, David T. Lewis <[hidden email]> wrote:

On Tue, Apr 22, 2014 at 03:44:46PM -0700, Eliot Miranda wrote:
> Hi Nicolas,
>
> On Tue, Apr 22, 2014 at 3:21 PM, Nicolas Cellier <
> [hidden email]> wrote:
>
> >
> > 2014-04-23 0:05 GMT+02:00 Eliot Miranda <[hidden email]>:
> >
> >>
> >> Hi All,
> >>
> >>     I should write a blog post on this, but I can't wait...
> >>
> >> In recent days I've written a script to build a Cog VMMaker image from
> >> Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.
> >>  This has allowed me to run the current Cog VM against Spur side-to-side.
> >>
> >> To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using
> >> the current Cog VM takes about 2 and a half minutes:
> >>
> >> McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st
> >> real    2m30.671s
> >> user    2m15.683s
> >> sys     0m5.283s
> >>
> >> To build the equivalent image using Spur takes about 1 and a half minutes:
> >> McStalker.image$ time spurcfvm CogVMMaker-spur.image
> >> BuildSqueak45Image.st
> >> real    1m34.943s
> >> user    1m23.666s
> >> sys     0m6.810s
> >>
> >> Comparing:
> >>
> >> 94.943 - 150.671 / 150.671 * 100 -36.99
> >> 83.666 - 135.681 / 135.681 * 100 -38.34
> >> 150.671 / 94.943 1.59
> >> 135.681 / 83.666 1.62
> >>
> >> that's about a -37% speedup, or 1.6x faster.
> >> --
> >> best,
> >> Eliot
> >>
> >>
> > So this is loading .mcz from package cache, uncompressing, compiling,
> > installing the packages.
> >
>
> Exactly.
>
>
> > I presume this qualifies as a macro benchmark...
> >
>
> yes :-)
>

That's quite impressive. I suspect that if you run this under a time profiler
that you'll see a lot of time going into I/O even though it is using a local
MC cache.  That would mean that this macro benchmark is quite conservative,
and the actual computational speedup may be considerably better than 1.6x.

I think that the i/o time is included in the "sys" sub-total, which will include reading the image files and the mczs.  The rest is, as they say, up to us.  So the difference in performance between including and excluding I/O is -36.99 vs -38.34, or 1.59 vs 1.62.
 
Bravo!

Dave

--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Tudor Girba-2
In reply to this post by EstebanLM
 
This is exciting!

Doru


On Wed, Apr 23, 2014 at 12:16 AM, Esteban Lorenzano <[hidden email]> wrote:
 
wow! impressive. 

can’t wait to integrate that :)

Esteban

On 23 Apr 2014, at 00:05, Eliot Miranda <[hidden email]> wrote:

Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot





--

"Every thing has its own flow"
Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

philippeback
 
Thanks for pushing the envelope. 

For my little brain, this thing is mind bending.


I am curious about these features:

Fast become and Pinned objects ?

Thanks for all the good work!!

Phil


On Wed, Apr 23, 2014 at 8:07 AM, Tudor Girba <[hidden email]> wrote:
 
This is exciting!

Doru


On Wed, Apr 23, 2014 at 12:16 AM, Esteban Lorenzano <[hidden email]> wrote:
 
wow! impressive. 

can’t wait to integrate that :)

Esteban

On 23 Apr 2014, at 00:05, Eliot Miranda <[hidden email]> wrote:

Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot





--

"Every thing has its own flow"


Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Henrik Sperre Johansen
 

On 23 Apr 2014, at 9:06 , [hidden email] wrote:



I am curious about these features:

Fast become and Pinned objects ?

Thanks for all the good work!!

Phil



Fast become is explained in 
Basically, it no longer scans the memory space on become to change all refs, but instead makes the old object header a forwarding pointer to the become’d object, and deals with updating references to such a forwarding pointer when they are encountered.

Pinned objects are nice when you pass in certain object types as arguments to a non-blocking FFI call, with the GC running in parallel (to the FFI) you would previously risk the data pointed to in the FFI call moving while the call was still executing, leading to all kinds of nasty things. 
In other words, you can now ensure correct behavior just by flipping a bit in the argument objects header instead of copying to/from a separately managed heap, which is much faster.

Cheers,
Henry

signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Blake McBride
In reply to this post by Eliot Miranda-2
 
Let's see the timing with the two done in reverse order.  OS cache could account for some of that difference.

Blake McBride



On Tue, Apr 22, 2014 at 5:05 PM, Eliot Miranda <[hidden email]> wrote:
 
Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Colin Putney-3
In reply to this post by Eliot Miranda-2
 


On 22 Apr, 2014 at 5:05:12 PM, Eliot Miranda ([hidden email]) wrote:
> >
> In recent days I've written a script to build a Cog VMMaker image
> from Squeak 4.5. See http://www.squeakvm.org/svn/squeak/branches/Cog/image.
> This has allowed me to run the current Cog VM against Spur side-to-side.
>

Nice. How much of the code that gets loaded is updating existing packages, rather than completely new code? Updating existing code usually involves lots of #become: and generates garbage...


--
Colin Putney


Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Eliot Miranda-2
 



On Wed, Apr 23, 2014 at 6:09 AM, Colin Putney <[hidden email]> wrote:



On 22 Apr, 2014 at 5:05:12 PM, Eliot Miranda ([hidden email]) wrote:
> >
> In recent days I've written a script to build a Cog VMMaker image
> from Squeak 4.5. See http://www.squeakvm.org/svn/squeak/branches/Cog/image.
> This has allowed me to run the current Cog VM against Spur side-to-side.
>

Nice. How much of the code that gets loaded is updating existing packages, rather than completely new code? Updating existing code usually involves lots of #become: and generates garbage...

It's all new code.  Here's the timing for just updating, but I'm not sure this is apples-to-apples because Kernel System and Collections differ between the two packages, and because there's a lot of network delay in this comparison:

McStalker.image$  time spurcfvm CogVMMaker-spur.image UpdateSqueakTrunkImage.st
real    0m45.886s
user    0m12.751s
sys     0m0.690s
McStalker.image$ time oscfvm CogVMMaker.image UpdateSqueakTrunkImage.st 
real    0m35.929s
user    0m6.235s
sys     0m0.603s
--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Spur performance report...

Eliot Miranda-2
In reply to this post by Blake McBride
 



On Wed, Apr 23, 2014 at 5:00 AM, Blake McBride <[hidden email]> wrote:
 
Let's see the timing with the two done in reverse order.  OS cache could account for some of that difference.
 
McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st
real    2m31.703s
user    2m10.281s
sys     0m4.020s
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st exactFitCompact fits: 771 misfits: 354821
real    1m35.447s
user    1m15.879s
sys     0m6.047s


Looks much the same.  The Spur image is larger by 15% which could account for some of the sys difference.

Blake McBride



On Tue, Apr 22, 2014 at 5:05 PM, Eliot Miranda <[hidden email]> wrote:
 
Hi All,

    I should write a blog post on this, but I can't wait...

In recent days I've written a script to build a Cog VMMaker image from Squeak 4.5.  See http://www.squeakvm.org/svn/squeak/branches/Cog/image.  This has allowed me to run the current Cog VM against Spur side-to-side.

To build a Cog VMMaker image on my 2.2GHz Intel Core i7 MacBook Pro using the current Cog VM takes about 2 and a half minutes:

McStalker.image$ time oscfvm CogVMMaker.image BuildSqueak45Image.st 
real    2m30.671s
user    2m15.683s
sys     0m5.283s

To build the equivalent image using Spur takes about 1 and a half minutes:
McStalker.image$ time spurcfvm CogVMMaker-spur.image BuildSqueak45Image.st 
real    1m34.943s
user    1m23.666s
sys     0m6.810s

Comparing:

94.943 - 150.671 / 150.671 * 100 -36.99
83.666 - 135.681 / 135.681 * 100 -38.34
150.671 / 94.943 1.59
135.681 / 83.666 1.62

that's about a -37% speedup, or 1.6x faster.
--
best,
Eliot






--
best,
Eliot