Smalltalk › Squeak › Squeak - Dev

To a terabyte and... beyond

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Dan Ingalls

To a terabyte and... beyond

Hi Guys ;-)

Anyone have experience with running super huge Squeaks? I have the
chance to use an absurdly big machine. I'd appreciate advice
(offline unless you think it would be generally useful) regarding the
relative stability of, say, linux and windoze builds of 64-bit
Squeaks known (or thought) to run with over a terabyte of objects,
and also where there may be some pre-built 64-bit images (sorry if
this is in an obvious place). Finally (since I've mainly used a Mac
for Squeak) I guess I'll have to know how to tell it how big to be,
or would it grow gracefully if I just said "(String new:
1000000000000) size" in a normal image?

Thanks

- Dan

Bert Freudenberg

Re: To a terabyte and... beyond

On 29.10.2010, at 06:25, Dan Ingalls wrote:

> Hi Guys ;-)
>
> Anyone have experience with running super huge Squeaks? I have the chance to use an absurdly big machine. I'd appreciate advice (offline unless you think it would be generally useful) regarding the relative stability of, say, linux and windoze builds of 64-bit Squeaks known (or thought) to run with over a terabyte of objects, and also where there may be some pre-built 64-bit images (sorry if this is in an obvious place). Finally (since I've mainly used a Mac for Squeak) I guess I'll have to know how to tell it how big to be, or would it grow gracefully if I just said "(String new: 1000000000000) size" in a normal image?
>
> Thanks
>
> - Dan

Dave Lewis has been working on 64 bit stuff, he surely will respond (and please keep responses on list, this is rather interesting). A recent image is here:

http://squeakvm.org/squeak64/sq64-dtl.zip

I think there isn't even much experience with a Gigabyte-sized image ...

- Bert -

David T. Lewis

Re: To a terabyte and... beyond

On Fri, Oct 29, 2010 at 09:48:11AM +0200, Bert Freudenberg wrote:

> On 29.10.2010, at 06:25, Dan Ingalls wrote:
>
> > Hi Guys ;-)
> >
> > Anyone have experience with running super huge Squeaks? I have the chance to use an absurdly big machine. I'd appreciate advice (offline unless you think it would be generally useful) regarding the relative stability of, say, linux and windoze builds of 64-bit Squeaks known (or thought) to run with over a terabyte of objects, and also where there may be some pre-built 64-bit images (sorry if this is in an obvious place). Finally (since I've mainly used a Mac for Squeak) I guess I'll have to know how to tell it how big to be, or would it grow gracefully if I just said "(String new: 1000000000000) size" in a normal image?
> >
> > Thanks
> >
> > - Dan
>
> Dave Lewis has been working on 64 bit stuff, he surely will respond (and please keep responses on list, this is rather interesting). A recent image is here:
>
> http://squeakvm.org/squeak64/sq64-dtl.zip
>
> I think there isn't even much experience with a Gigabyte-sized image ...

Hi Dan,

The image that Bert mentions is an updated version of your original
64-bit image, suitable for running on a current Squeak VM. I have
not tried tracing the latest Squeak trunk image (sounds like a good
project for this weekend though :)

The quickest way to get a VM running is to start with an up to
date Squeak image, load latest VMMaker from www.squeaksource.com/VMMaker
and platform sources from Subversion. The "64-bit VM" button is
gone from the VMMaker tool, all that is needed to build the 64 bit
version is to set a macro either in ./build/config.h or in
./src/vm/interp.h:

#define SQ_VI_BYTES_PER_WORD 4

I use Linux for running this stuff, so I can't speak to Windows or
OS X. On a unix platform, if you evaluate "(String new: 1000000000000) size"
in sq64-dtl.image, you will get a low space notifier. I remember adding
some checks to the VM to prevent address overflows for allocations
that are "unreasonably large), and I think this notifier reflects
hitting that limit (sorry I can't recall the details, I'll check later).
It would be fun to see how big "unreasonably large" could be though :)

A gigabyte is no problem though. The following succeeds on a
64-bit image:

(String new: 1000000000) size ==> 1000000000

For allocation requests that are within the current VM limits,
what you can expect to see on a unix box is that your VM will
cheerfully keep allocating memory from the OS until your system
starts swapping pages to disk, at which point everything on the
system slows to a crawl. It will be interesting to see what happens
on a really big box with lots of memory.

Dave

johnmci

Re: To a terabyte and... beyond

Well I think there are two interesting factors if not more here.

(a) when you ask for the 1gb I believe the logic zeros the memory
chunk. Without specialized virual memory manager aware code this could
take a long Long time. Specialized code could ask the VM to provide
zero pages from the operating systems queue of zero pages it has on
hand.

(b) I'm not sure where this will end up, old space or new space. A big
hurdle here is that on the compaction it's going to move GB of memory
just possibly a few kilobytes in distance.

It is of course viable to adjust the gc prams to avoid a full gc cycle
at all cost, which is what i do for wikiserver on the iPhone

Frankly to avoid the two points above squeak needs the notion of large
space or fixed space. I thought Eliot mentioned he was
thinking/planning that?

Oh got funding? I have hours looking for a person to pay for...

Lastly the vm logic isn't setup right to handle the remember table,
allocate a 1gb array and the vm might just iterate over the 1 gb
elements looking for a root.

These three item will bring things to a crawl in my opinion

On 10/29/10, David T. Lewis <[hidden email]> wrote:

> On Fri, Oct 29, 2010 at 09:48:11AM +0200, Bert Freudenberg wrote:
>> On 29.10.2010, at 06:25, Dan Ingalls wrote:
>>
>> > Hi Guys ;-)
>> >
>> > Anyone have experience with running super huge Squeaks? I have the
>> > chance to use an absurdly big machine. I'd appreciate advice (offline
>> > unless you think it would be generally useful) regarding the relative
>> > stability of, say, linux and windoze builds of 64-bit Squeaks known (or
>> > thought) to run with over a terabyte of objects, and also where there
>> > may be some pre-built 64-bit images (sorry if this is in an obvious
>> > place). Finally (since I've mainly used a Mac for Squeak) I guess I'll
>> > have to know how to tell it how big to be, or would it grow gracefully
>> > if I just said "(String new: 1000000000000) size" in a normal image?
>> >
>> > Thanks
>> >
>> > - Dan
>>
>> Dave Lewis has been working on 64 bit stuff, he surely will respond (and
>> please keep responses on list, this is rather interesting). A recent image
>> is here:
>>
>> http://squeakvm.org/squeak64/sq64-dtl.zip
>>
>> I think there isn't even much experience with a Gigabyte-sized image ...
>
> Hi Dan,
>
> The image that Bert mentions is an updated version of your original
> 64-bit image, suitable for running on a current Squeak VM. I have
> not tried tracing the latest Squeak trunk image (sounds like a good
> project for this weekend though :)
>
> The quickest way to get a VM running is to start with an up to
> date Squeak image, load latest VMMaker from www.squeaksource.com/VMMaker
> and platform sources from Subversion. The "64-bit VM" button is
> gone from the VMMaker tool, all that is needed to build the 64 bit
> version is to set a macro either in ./build/config.h or in
> ./src/vm/interp.h:
>
> #define SQ_VI_BYTES_PER_WORD 4
>
> I use Linux for running this stuff, so I can't speak to Windows or
> OS X. On a unix platform, if you evaluate "(String new: 1000000000000) size"
> in sq64-dtl.image, you will get a low space notifier. I remember adding
> some checks to the VM to prevent address overflows for allocations
> that are "unreasonably large), and I think this notifier reflects
> hitting that limit (sorry I can't recall the details, I'll check later).
> It would be fun to see how big "unreasonably large" could be though :)
>
> A gigabyte is no problem though. The following succeeds on a
> 64-bit image:
>
> (String new: 1000000000) size ==> 1000000000
>
> For allocation requests that are within the current VM limits,
> what you can expect to see on a unix box is that your VM will
> cheerfully keep allocating memory from the OS until your system
> starts swapping pages to disk, at which point everything on the
> system slows to a crawl. It will be interesting to see what happens
> on a really big box with lots of memory.
>
> Dave
>
>
>

--
===========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================

David T. Lewis

Re: To a terabyte and... beyond

In reply to this post by David T. Lewis

(cc squeak-dev per Bert's suggestion)

Hi Dan,

Indeed it is "#define SQ_VI_BYTES_PER_WORD 8". This macro
is the one used in sqMemoryAccess.h, so BYTES_PER_WORD and
BASE_HEADER_SIZE are set from SQ_VI_BYTES_PER_WORD. It turned
out that it is possible to rearrange some class and instance
variables in ObjectMemory and Intepreter such that the generated
C code is the same for 32 and 64 bit object memory, controlled
at compile time by the SQ_VI_BYTES_PER_WORD macro. I'm afraid
it makes the Smalltalk source less readable, but it's nice
not to need to generate different sets of C sources for src32
and src64.

I should mention that recent versions of VMMaker can no longer
be loaded into a Squeak 3.8 image, primarily due to the use of
pragmas, e.g. "<inline: true>" rather than "self inline: true",
which in turn means that at the moment one cannot genererate a
current VM from the Squeak 3.8 based 64-bit image. I will have
a bit of time to play with this on Tuesday, so I'll see if I can
find a way to either load current VMMaker into the 3.8 image,
or to trace an up-to-date Squeak 4.1 image to 64 bits. I'll let
you know before Wednesday if I have made any progress.

Dave

On Fri, Oct 29, 2010 at 10:40:39PM -0700, Dan Ingalls wrote:

> Hi David -
>
> Thanks for the pointers. I'm assuming you meant BYTES_PER_WORD 8, right?
> Do let me know if you gather more infor in the next few days. I think I'm not going to have a chance at this until after next Wed.
>
> All the best
> - Dan
> ---------------------------------------
> >On Fri, Oct 29, 2010 at 09:48:11AM +0200, Bert Freudenberg wrote:
> >> On 29.10.2010, at 06:25, Dan Ingalls wrote:
> >>
> >> > Hi Guys ;-)
> >> >
> >> > Anyone have experience with running super huge Squeaks? I have the chance to use an absurdly big machine. I'd appreciate advice (offline unless you think it would be generally useful) regarding the relative stability of, say, linux and windoze builds of 64-bit Squeaks known (or thought) to run with over a terabyte of objects, and also where there may be some pre-built 64-bit images (sorry if this is in an obvious place). Finally (since I've mainly used a Mac for Squeak) I guess I'll have to know how to tell it how big to be, or would it grow gracefully if I just said "(String new: 1000000000000) size" in a normal image?
> >> >
> >> > Thanks
> >> >
> >> > - Dan
> >>
> >> Dave Lewis has been working on 64 bit stuff, he surely will respond (and please keep responses on list, this is rather interesting). A recent image is here:
> >>
> >> http://squeakvm.org/squeak64/sq64-dtl.zip
> >>
> >> I think there isn't even much experience with a Gigabyte-sized image ...
> >
> >Hi Dan,
> >
> >The image that Bert mentions is an updated version of your original
> >64-bit image, suitable for running on a current Squeak VM. I have
> >not tried tracing the latest Squeak trunk image (sounds like a good
> >project for this weekend though :)
> >
> >The quickest way to get a VM running is to start with an up to
> >date Squeak image, load latest VMMaker from www.squeaksource.com/VMMaker
> >and platform sources from Subversion. The "64-bit VM" button is
> >gone from the VMMaker tool, all that is needed to build the 64 bit
> >version is to set a macro either in ./build/config.h or in
> >./src/vm/interp.h:
> >
> > #define SQ_VI_BYTES_PER_WORD 4
> >
> >I use Linux for running this stuff, so I can't speak to Windows or
> >OS X. On a unix platform, if you evaluate "(String new: 1000000000000) size"
> >in sq64-dtl.image, you will get a low space notifier. I remember adding
> >some checks to the VM to prevent address overflows for allocations
> >that are "unreasonably large), and I think this notifier reflects
> >hitting that limit (sorry I can't recall the details, I'll check later).
> >It would be fun to see how big "unreasonably large" could be though :)
> >
> >A gigabyte is no problem though. The following succeeds on a
> >64-bit image:
> >
> > (String new: 1000000000) size ==> 1000000000
> >
> >For allocation requests that are within the current VM limits,
> >what you can expect to see on a unix box is that your VM will
> >cheerfully keep allocating memory from the OS until your system
> >starts swapping pages to disk, at which point everything on the
> >system slows to a crawl. It will be interesting to see what happens
> >on a really big box with lots of memory.
> >
> >Dave

David T. Lewis

Re: To a terabyte and... beyond

In reply to this post by Dan Ingalls

On Thu, Oct 28, 2010 at 09:25:18PM -0700, Dan Ingalls wrote:

> Hi Guys ;-)
>
> Anyone have experience with running super huge Squeaks? I have the
> chance to use an absurdly big machine. I'd appreciate advice
> (offline unless you think it would be generally useful) regarding the
> relative stability of, say, linux and windoze builds of 64-bit
> Squeaks known (or thought) to run with over a terabyte of objects,
> and also where there may be some pre-built 64-bit images (sorry if
> this is in an obvious place). Finally (since I've mainly used a Mac
> for Squeak) I guess I'll have to know how to tell it how big to be,
> or would it grow gracefully if I just said "(String new:
> 1000000000000) size" in a normal image?

Hi Dan,

I've made a few updates today to VMMaker and the unix platform code that
should make it easier to experiment with large object memories. These
consist of a few type declaration fixes and updates to permit larger
expansion of object memory and allocation of large objects when word
size is 8 and host platform is 64-bit.

I put a copy of a working Squeak 3.8 based 64-bit image, along with VMMaker
and platform sources, at http://squeakvm.org/squeak64.

The image is at http://squeakvm.org/squeak64/sq64-20101119-dtl.zip. The
image along with a copy of complete platform sources and generated C code
from VMMaker is at http://squeakvm.org/squeak64/sq64-20101119-dtl.zip.

The VMMaker in this image is derived from http://www.squeaksource.com/VMMaker/VMMaker-dtl.200.mcz.
I manually removed various things that are not compatible with Squeak 3.8
(pragma declarations and various things related to the blockclosure updates)
in order to make it loadable in Squeak 3.8. There may be some errors but
I think this is functionally equivalent to VMMaker-dtl.200.mcz. I am not
yet able to trace a current Squeak trunk image (the SystemTrace rcan be
found at http://www.squeaksource.com/SystemTracing), which would surely
be a very good thing to do at this point.

The PC that I used has 8MB of real memory, and is running Linux. On this
box, I am able to create an image of about 7MB with no trouble, and to
save an apparently valid snapshot to disk (although I cannot reload
anything bigger than a few GB from disk on this system).

The garbage collector seems to still work under these conditions, although
I'm not really stressing it since I'm only allocating a small number of
very large objects. John pointed out a few likely problem areas here; I'm
sure there will be quite a few.

Objects (e.g. a String) of several GB can be allocated and seem to function
correctly, but do not display properly in an inspector.

Hopefully this will allow you to do some experimenting on a box with
very large memory. A terabyte seems like a bit of a stretch for a software
system that started out on 16 bit hardware, but we're only a few orders of
magnitude away so who is to say it can't be done?

Attached is a screen shot of the 64-bit image that you and Ian originally
did, now bloated out to about 7GB in size after allocating some gigabyte-sized
strings.

Please keep us posted if you break any interesting boundaries :)

Dave

world.png (81K) Download Attachment

ccrraaiigg

Re: To a terabyte and... beyond

In reply to this post by Dan Ingalls

Hi Dan!

Why? :)

-C

--
Craig Latta
www.netjam.org
+ 31 020 894 6247
+ 1 415 287 3547