Big Array of unique objects crashes the system.

Hello,

I come along to create an Array with many unique Strings. On low sizes
such 1 Megabyte, it works. But if I increase the size to 10 or 100
Megabyte, Dolphin crashes without an error message. The process simply
terminates and Dolphin writes an error file.

Here is the workspace code:

| bar |
bar := Array new: 10000000. "10000 Kilobyte | 10 Megabyte"
1 to: bar size do: [:count | bar at: count put: 't' copy].

If I left off the copy message, it works. At my first attempt, I
accessed the Array through a Stream. The system crashed also.

My system:
CPU: AMD AthlonXP 1800+
Memory: 768 MByte
OS: Microsoft Windows XP SP2
Dolphin: Dolphin Smalltalk X6 professional 6.02

What's going on here?
Is this a problem only on my machine?

Nice greets,
Marc Michael

Marc,

> What's going on here?
> Is this a problem only on my machine?

WARNING: the following all applies to Dolphin 5.  It appears to apply to D6
too, but use with great caution.  And make backups first !

I suspect that it's a combination of two things.  One is that Dolphin has a
hard (but adjustable) limit on how many objects can exist at once, that limit
is set by the size of the "object table".  The other is that when that limit is
breached, Dolphin just crashes (I'm sure it isn't /meant/ to die so abruptly,
but that's what it does).

The size is configured by an entry in the image file itself.  You can read the
limit configured into any image file with code like:
    f := FileStream
                    open: (SessionManager current imageFileName)
                    mode: #open
                    text: false.
    f position: 16r20.
    otSize := f nextSDWORD.
    f close.
    otSize.

Which, in a default D6 image, returns 8388608 (16r800000) -- less than the
number of Strings you want to create.

To change the limit, you can use code like:
    f := FileStream
              open: .. some image file ...
              mode: #open
              text: false.
    f position: 16r20.
    f nextSDWORDPut: 16r1000000.
    f close.
which -- if I haven't introduced any typos -- should set the limit to 16777216
(16r1000000).  The new limit doesn't take effect until you next start Dolphin
on that image.

BTW, according to Blair,

    "This will increase the amount of virtual memory that Dolphin
    reserves for the object headers, but not the amount committed.
    It will not significantly increase the memory consumption"

I believe (but have never attempted to check) that the limit wired into the
image from which you deploy is copied into the resulting application.

    -- chris

Chris Uppal wrote:

> I believe (but have never attempted to check) that the limit wired into the
> image from which you deploy is copied into the resulting application.

This is the part, which didn't work. From a Workspace, I am able to
create big numbers of unique objects, but if I want to deploy, Dolphin
crashes. It seems that there is another point where this limit is set.

cu,
Yogi Marc Michael

Marc,

> > I believe (but have never attempted to check) that the limit wired into
> > the image from which you deploy is copied into the resulting
> > application.
>
> This is the part, which didn't work. From a Workspace, I am able to
> create big numbers of unique objects, but if I want to deploy, Dolphin
> crashes. It seems that there is another point where this limit is set.

[I'm assuming that you mean that the change fixes the problem for Dolphin's
IDE, but not for deployed apps.]

Hmm, that's a bit odd.  It's quite possible that something's happening that I
don't know about, but I /think/ it should work :-(

Anyway, you can check whether the modified OT limit is being copied into your
deployed application by modifying ImageStripper>>keepImageFile to answer true.
With that change, deploying MyApp.exe should leave a MyApp.tmp file in the same
directory as the .exe. That is the image file itself (which is bound into
the .exe).  Then use the snippet of code from my earlier post to see if the
value has been copied across from your deployment image[*] into MyApp.tmp.   I
tried a quick test, and it /seems/ to work, but I haven't modified an
application to see if the OT size itself has changed correspondingly.

([*] It's normal, and highly recommended, to use a clean image for deployment.
So it's easy to forget to modify the deployment image's OT size too.  Perhaps
OT size should be a settable option in Lagoon.)

It's also worth running DebugMon from SysInternals while you start/run your
application, to see if the Dolphin VM is trying to tell you anything.

Other than that, I'm afraid I've run out of ideas...

    -- chris