Strange crash - Memory Allocation?

I am seeing a strange crash that results in in either a Windows error
that forces Dolphin to close or, occasionally, a dead Dolphin that needs
to be closed with the TaskManager.  It looks like it is related to
memory allocation but I'm not really sure - it may even be specific to
my machine?.

Dolphin 5.1.3 on Windows XP with 512MB

I've added a test class that shows the problem  (file it in via a
workspace and then "Demo new run") but, please, ignore the coding :-)  I
can usually get up to around 500000 before it crashes....

--
Ian

====================== copy from here ...

"Filed out from Dolphin Smalltalk XP"!

Object subclass: #Demo
 instanceVariableNames: ''
 classVariableNames: ''
 poolDictionaries: ''
 classInstanceVariableNames: ''!
Demo guid: (GUID fromString: '{ADD48A8A-68DF-4AD2-AE0D-11E260DBCD38}')!
Demo comment: ''!
!Demo categoriesForClass!Unclassified! !
!Demo methodsFor!

pairsIn: anArray
 | pairs |
 pairs := OrderedCollection new.
 1
  to: anArray size - 1
  do:
   [:index1 |
   index1 + 1
    to: anArray size
    do: [:index2 | pairs add: (anArray at: index1) -> (anArray at:
index2)]].
 ^pairs!

run
 "
Demo new run
"

 | test |
 test := Array new: 1000000.
 1 to: test size do: [:i | test at: i put: (Array with: i) , (1 to: 5)
asArray].
 test collect:
   [:each |
   (each at: 1) \\ 1000 = 0
    ifTrue:
     [Transcript
      print: each;
      cr].
   self pairsIn: each]! !
!Demo categoriesFor: #pairsIn:!public! !
!Demo categoriesFor: #run!public! !

====================== ... to here

Ian Bartholomew wrote:

> Dolphin 5.1.3 on Windows XP with 512MB

D 5.1.2 on Win XP 512 MB, pagefile limited to 770MB.

> I've added a test class that shows the problem  [...]  I
> can usually get up to around 500000 before it crashes....

Tried it once, Dolphin crashed silently leaving a dump file. I don't think it
ever reached the limits set by the pagefile size (it didn't seem to be
thrashing before it died).

The first bit of the dump file is at the end of this post, I haven't bothered
with the full stack trace since it's just falling over in creating an
Association.

BTW. It seems that crashdump looks in HKLM/Software/Object Arts/Dolphin
Smalltalk 5.0/CrashDump/<default> for the pathname to write the dumpfile to.
That's five point *zero* not five point *one* -- took quite a long time to
find/notice what was wrong...

I tried it a second time (I really have to find a more exiting hobby) and this
time Windows came up with it's popup window, so I let it start the debugger.
The 'C' (as it were) stack is probably not very helpful, but FWIW:

> kernel32.dll!_RaiseException@16()  + 0x50
  DolphinVM005.dll!1000b584()
  DolphinVM005.dll!1000b33d()
  DolphinVM005.dll!1000b1ef()
  msvcrt.dll!__except_handler3()  + 0x4b
  ntdll.dll!77fb172e()
  ntdll.dll!77fb1700()
  ntdll.dll!77f9790d()
  user32.dll!77d6ccd4()
  msvcrt.dll!__global_unwind2()  + 0x18
  ntdll.dll!77f75dba()

(Incidentally I get quite a few hits on my website from people looking for
"images of dolphins" and the like, I suppose this thread will now show up on
the radar of people googling for "dolphin dies thrashing" ;-)

    -- chris

======= crash dump follows =======

12:25:11, 07/12/2003: Dolphin.exe caused an unhandled Win32 Exception C00000FD
at 100070C1 in module 10000000 (C:\Program Files\Common Files\Object
Arts\Dolphin Smalltalk 5.1\DolphinVM005.dll)

*----> Exception Parameters <----*
00000000
00412E34 ?

*----> CPU Context for thread 0x654 <----*
EAX = 16FCDE70 EBX = 09A10D80 ECX = 0000005C
ESI = 00000054 EDI = C0000005 EIP = 100070C1
ESP = 00413100 EBP = 00413130 EFL = 00010202
CS = 001B SS = 0023 DS = 0023
ES = 0023 FS = 003B GS = 0000

*----> VM Context <----*
Process: {01BA0004:size 16f words, suspended frame 01BA05AD, priority 5,
callbacks 1
last failure 0:nil, FPE mask 3, thread nil}
Active Method: Object>>->
IP: 072A53E8 (8)
SP: 01BA05CC
BP: 01BA05AC (346)
ActiveFrame: {01BA05B0: cf 01BA0581, sp 00000000, bp 01BA05AC, ip 0,
SmallInteger(Object)>>->}
 receiver: 2
 arg[0]: 3


New Method: Association class>>key:value:
Message Selector: #key:value:

... etc...

Chris,

Thanks for confirming that it is not just my machine.  FWIW, the
following workspace script exhibits the same behaviour - and is a lot
easier to work with :-)

a := Array new: 1000000.
1 to: a size do: [:i |  | pairs |
 pairs := OrderedCollection new.
 1 to: 6 do: [:i1 |
  i1 + 1 to: 6 do: [:i2 |
   pairs add: i1 -> i2]].
 a at: i put: pairs]

It always seems to crash close to the point where physical memory runs
out and swapping starts - but why would that affect Dolphin so badly?

> BTW. It seems that crashdump looks in HKLM/Software/Object
> Arts/Dolphin Smalltalk 5.0/CrashDump/<default> for the pathname to
> write the dumpfile to. That's five point *zero* not five point *one*
> -- took quite a long time to find/notice what was wrong...

It's so long since I used it I couldn't remember how to turn it on at
all.  Perhaps it might be an idea to have this as a user option so it
could be toggled on/off from within Dolphin?

--
Ian

Ian,

I tried that too, this time it resulted in Dolphin creating a dump file, and
then (apparently) going into an infinite loop halfway through writting the
stack trace.

> It always seems to crash close to the point where physical memory runs
> out and swapping starts - but why would that affect Dolphin so badly?

Yes, that's what I saw too -- Dolphin itself was "only" using around 290M but
the system as a whole would have started paging quite soon.

I've just tried it three times on a 512MB W2K box, and got a:
    The exception unknown software exception (oxa0000203) occurred in the
application at location 0x7c4ea4e1.
error popup each time.  I suspect that is a different effect, though, since the
machine *had* extended itself to use all the available paging space (also
limited to 760MB) before the error occurred.  (And it really did have to do
some paging too -- that machine also runs SuSE under VMware which is set up to
grab half the physical memory for its own use)

Dolphin 5.0, on the same W2K box (I don't have 5.0 on the XP box) gets to the
same point and then apparently goes into an infinite loop.  Two times our of
two.


> It's so long since I used it [crash dump] I couldn't remember how
> to turn it on at all.  Perhaps it might be an idea to have this as a
> user option so it could be toggled on/off from within Dolphin?

Or, at minimum, it could be used as an example of the registry access package,
which would provide *some* documentation at least.

    -- chris

On Sat, 6 Dec 2003 21:46:59 -0000,
Ian Bartholomew <[hidden email]> wrote:
I managed to hang the application much sooner with
[Demo new run] forkAt: 1
with a ProcessMonitor open.

After about 6 hours I'm still waiting for the
Update Speed ... submenu to appear, although CPU load
has dropped from 100% to zero sometime in between.

s.

I wrote:

> It's so long since I used it I couldn't remember how to turn it on at
> all.  Perhaps it might be an idea to have this as a user option so it
> could be toggled on/off from within Dolphin?

Once again I had the "it seems like a good idea" feeling so at

http://www.idb.me.uk/files/crashdump.zip

is a small (<1K) add-on that, when filed in, adds a "crashDump" option
to the User Preferences for "Development System".

nb
- It's a bit non-standard as it doesn't have an inst var to store the
state (it just gets it from the registry)
- As Chris pointed out earlier, you have to use the DS 5.0 registry key
as the root.  That will need to be changed when (if) OA update the key
used.

Now I shan't _ever_ forget how to turn crash dump on/off :-)
--
Ian

"Ian Bartholomew" <[hidden email]> wrote in message
news:gnGAb.18144$[hidden email]...
> Chris,
>
> Thanks for confirming that it is not just my machine.  FWIW, the
> following workspace script exhibits the same behaviour - and is a lot
> easier to work with :-)
> ...

Start up an instance of Dbgview.exe (from Sysinternals) and try this:

a := Array new: 1000000.
1 to: a size
 do:
  [:i |
  | pairs count |
  pairs := OrderedCollection new.
  1 to: 6 do: [:i1 | i1 + 1 to: 6 do: [:i2 | pairs add: i1 -> i2]].
  a at: i put: pairs.
  (i rem: 1000) = 0
   ifTrue:
    [| count |
    count := MemoryManager current objectCount.
    Trace
     print: count;
     nextPutAll: ' objects';
     cr]]

You should find it goes bang somewhere around 8 million objects, and the
truth is that your script is simply exceeding the default maximum number of
objects. The VM reserves an area of virtual memory sufficient to hold the
headers for a certain number of objects, by default around 8M, which it then
commits from on demand. Ultimately this reserve can be exhausted if one
tries hard enough, and the VM cannot continue.

Of course just crashing is a rather inelegant way to handle exceeding a
limit. I was surprised by this since I seem to remember this situation
causing a more informative message box to appear, ... immediately before
crashing :-).  I will look into that.

The following page on the Wiki explains how to increase the maximum number
of objects.

http://www.object-arts.co.uk/wiki/html/Dolphin/ImplementationLimits.htm

>
> It always seems to crash close to the point where physical memory runs
> out and swapping starts - but why would that affect Dolphin so badly?

I would think that is just coincidental, because of the amount of memory you
happen to have in your machine.

It is turned on by default, and writes to the .ERRORS file associated with
the image. If you run the test above you should see a trace entry telling
you where it was written. If the file cannot be opened, then the entire dump
goes to the trace device.

Regards

Blair

Blair,

> You should find it goes bang somewhere around 8 million objects, and
> the truth is that your script is simply exceeding the default maximum
> number of objects.

Right, that makes sense.  Thanks for the explanation.

--
Ian

Use the Reply-To address to contact me.
Mail sent to the From address is ignored.

Blair,

> You should find it goes bang somewhere around 8 million objects, and
> the truth is that your script is simply exceeding the default maximum
> number of objects.

Right, that makes sense.  Thanks for the explanation.

--
Ian

Blair McGlashan wrote:

> the
> truth is that your script is simply exceeding the default maximum number
> of objects.

Aha!  Thanks.

And yes it is failing at the same point under W2K, although the actual failure
mode is different.

    -- chris

"Blair McGlashan" <[hidden email]> wrote in message
news:br70ct$p9t$[hidden email]...
...
> You should find it goes bang somewhere around 8 million objects, and the
> truth is that your script is simply exceeding the default maximum number
of
> objects. The VM reserves an area of virtual memory sufficient to hold the
> headers for a certain number of objects, by default around 8M, which it
then
> commits from on demand. Ultimately this reserve can be exhausted if one
> tries hard enough, and the VM cannot continue.
...

I wonder if anyone has hit this object pointer limit in a real world Dolphin
application yet?  I would be interested to hear stories.

I used to program in an environment called Actor, it was an old 16 bit
Smalltalk clone with a Pascal-ish syntax (ie: message(receiver, arg1,
arg1);).  The part that made it interesting was that it had an object
pointer limit of 32 thousand objects.  That limit drove us crazy towards the
end.  Try doing OOP when you can't afford to create any objects! :(  Those
were not fun times.  Fortunately that application has now be ported to
Dolphin Smalltalk and I have the freedom to use objects once again.  I don't
anticipate hitting the 8 million object limit any time soon, but it is
REALLY nice to know I can increase it if I ever need to.

Chris

Christopher J. Demers wrote:

> I wonder if anyone has hit this object pointer limit in a real world
> Dolphin application yet?  I would be interested to hear stories.

I haven't, not quite, not yet...

Something I'm working on currently (with incomplete functionality and working
on cut-down dataset) adds around 1M objects to my image, when it's
completed and is working on a full-sized dataset I guess I'll be seeing counts
in the 2-3M range.  So "plenty" of headroom left there ;-)

A thing I tried to do some years back would almost certainly have exceeded the
limit but a flaw in the Database connection (since fixed) made loading the data
too slow, so I couldn't scale the app up to the point where object counts were
an issue.

    -- chris

Chris,

We looked at Actor back then too. From what I remember we never had
enough memory to do anything worthwhile, let alone have a chance to run
out of object pointers. It wasn't Actor's fault though; back then we
were limited to 640K and once you took out the 350K (was it?) cost of
Windows 2.0 there wasn't much left for Actor applications to play with.

Was Actor still going with Windows 3.0 and later? Perhaps that's how
you managed to do something useful with it. It's a shame we couldn't
manage to do more, I quite liked Actor and it could have saved me years
of C++ unpleasantness (especially with the original Cfront) and we
might have got into Smalltalk just a bit earlier.

--
Andy Bower
Dolphin Support
www.object-arts.com

On 11 Dec 2003 17:04:35 -0500, Andy Bower wrote:

I should remember it as I, too, messed with it.  I can't remember - wasn't
the biggest problem that you couldn't persist objects easily?  
--

Larry
email is rapp at lmr dot com

"Andy Bower" <[hidden email]> wrote in message
news:3fd8e9f3$[hidden email]...
I did not start using Actor until version 3.2p (which was apparently
released in 1991) on Windows 3.11 and then Windows 95, my boss had been
using it before.  By that time I think they had worked out the memory
limits, and doubled the object pointers from 16 thousand to 32 thousand.  I
recall using memory settings of 700 static, and 950 dynamic when I would
deploy our application.  I don't recall the memory limit ever being a
problem for us, we always hit the OP limit first.  They released a version
4.0, but I never used it because it consumed too many object pointers
itself.  Symantec purchased Actor from the Whitewater group and let it die.
I have lost two tools to them over the years.

There is a small (probably < 30 people) mailing list of people interested in
Actor, there is an archive available here:
 http://listserv.surfnet.nl/scripts/wa.exe?A0=actor-l&D=0&F=&H=0&O=T&S=&T=1
.  There has also been some interest in doing a 32 bit rewrite
http://biphome.spray.se/mikael_aronsson/index_ac.html called ReActor.

I am glad that my company used Actor, because it made Smalltalk the most
logical next step.  Actor was very powerful within its object pointer limit,
and was probably ahead of its time.

Chris

Blair McGlashan wrote:

> The following page on the Wiki explains how to increase the maximum number
> of objects.
>
> http://www.object-arts.co.uk/wiki/html/Dolphin/ImplementationLimits.htm
>
>

So what would be the maximum permissible number for patching the header?
(2^30)-1? (Not counting of course the fact, that Windows might not have
enough memory to satisfy the allocation of the object table)

Not that I'm running into a problem, but just out of curiosity ;)

Bernhard

"Andy Bower" <[hidden email]> wrote in message
news:3fd8e9f3$[hidden email]...
Hi,
we where using Actor in 1991 for projects in a german car manufacturing
company - In that time Actor was distributed in germany by Intellis -
Logitek oHG my firm was doing support - In fact we tried to purchase Actor
but Symantec got it and after that it was almost dead.

As far as I know there is a newer effort of porting Actor to Win32 - Actor
is written in Forth. There is a Win32 version called ReActor (
http://bigphome.spray.se/mikael_aronsson/index_ac.html )

with best regards, Frank Lesser, www.lesser-software.com

"Bernhard Kohlhaas" <[hidden email]> wrote in message
news:[hidden email]...
>
>
> Blair McGlashan wrote:
>
> > The following page on the Wiki explains how to increase the maximum
number
Curious enough to try it and see?

The object headers are 12 bytes in size, and you can address only 2Gb of the
4Gb of memory, so the maximum theoretical amount would be 16rAAAAAAA. Of
course you can't reserve that much virtual memory, as there are other
demands on the address space, nevertheless it is a start point. So using
that figure and the expression on the above wiki page one can set about
doing a binary chop to find the actual maximum reserve.

On the particular setup I tried it on it came out at 16r63C0000, or over 104
million objects. The system will actually start up with this number of
objects reserved regardless of whether you really have the physical memory
to commit it. There isn't much cost to reserve the space and Dolphin will
start up as normal, however reserving 1Gb of address space will reduce that
available for other allocations (such as Dolphin's own threads), so this
wouldn't be useful. Also if the average object size in the test image of
about 47 bytes (which includes all the overhead) were maintained that number
of objects would require more than 4.5Gb of memory.

So given the 2Gb memory address range available to applications, in practice
I suspect one could not exceed 32 million objects. Actually the relatively
small number of objects available brings home how much we (the computer
industry) need to move to 64-bits.

Regards

Blair

Blair,

thanks so much for the very detailed explanation.

I think I'm going to be fine for a little longer with 32 million objects. ;)

So will we see a version of Dolphin for 64-bit Windows in the near
or not-so-near future?

Bernhard

Blair McGlashan wrote:
[...]