New Dolphin 5.1.4 Compiler available

We have now fixed the issue relating to block temps appearing out of scope
that was being caused by a bug in the Dolphin compiler (#1461; "Compiler
generated temps maps yield incorrect scope for block arguments"). The new
compiler DLL can be downloaded from:

http://object-arts.com/Lib/Update/Dolphin/5.1/DolphinCR005.zip

The new DolphinCR005.dll contained in the zip should be extracted into your
Dolphin common files installation directory (C:\Program Files\Common
Files\Object Arts\Dolphin Smalltalk 5.1) to overwrite your old compiler
(though I recommend making a copy of the old one first). Obviously you will
have to do this when Dolphin itself is not running as otherwise you will not
be able to overwrite the file.

You can test it is working correctly by running this simple expression
(courtesy of Maxim Fridental):

    [#(1 2 3) do: [:each | each = 2 ifTrue: [Error signal: 'Error']]] on:
Error do: [:ex | self halt]

Go into the debugger when the breakpoint is hit, and if you can see the 'ex'
argument, then the new compiler is being used. If not then the old compiler
is still being picked up for some reason. If you find this to be the case
you may have to try unregistering and then re-registering DolphinCR005.DLL
using regsvr32.exe. Failing that, try placing the new compiler in the same
directory as Dolphin.exe (C:\Program Files\Object Arts\Dolphin Smalltalk
5.1).

Rather embarrassingly this bug was caused by the absence of two ampersand
characters from the C++ source code of the compiler, meaning that the temp
map was not being updated as the generated code is optimised. :-$

Regards

Blair

Blair,

> Go into the debugger when the breakpoint is hit, and if you can see the
'ex'
> argument, then the new compiler is being used. If not then the old
compiler
> is still being picked up for some reason. If you find this to be the case
> you may have to try unregistering and then re-registering DolphinCR005.DLL
> using regsvr32.exe. Failing that, try placing the new compiler in the same
> directory as Dolphin.exe (C:\Program Files\Object Arts\Dolphin Smalltalk
> 5.1).

THANKS!!!!   The debugger is whole again :)  This is well-timed too, because
I will soon be giving the debugger quite a workout (and vice versa).

A couple of questions: will the windows installer try to undo the fix?  I do
not personally use its shortcuts, with the exception of the fresh
installation on occaision.

Will it be necessary to compile methods for the change to be felt throughout
the system?  Of course, recompiling is a likely side effect of debugging, so
it would be largely self-repairing.


> Rather embarrassingly this bug was caused by the absence of two ampersand
> characters from the C++ source code of the compiler, meaning that the temp
> map was not being updated as the generated code is optimised. :-$

That's the problem with computers, they do what we tell them, not what we
want them to do :)

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

"Bill Schwab" <[hidden email]> wrote in message
news:c28edn$um2$[hidden email]...
> Blair,
>
> > Go into the debugger when the breakpoint is hit, and if you can see the
> 'ex'
> > argument, then the new compiler is being used. If not then the old
> compiler
> > is still being picked up for some reason. If you find this to be the
case
> > you may have to try unregistering and then re-registering
DolphinCR005.DLL
> > using regsvr32.exe. Failing that, try placing the new compiler in the
same
> > directory as Dolphin.exe (C:\Program Files\Object Arts\Dolphin Smalltalk
> > 5.1).
>
> THANKS!!!!   The debugger is whole again :)  This is well-timed too,
because
> I will soon be giving the debugger quite a workout (and vice versa).
>
> A couple of questions: will the windows installer try to undo the fix?  I
do
> not personally use its shortcuts, with the exception of the fresh
> installation on occaision.

It appears not, though this may depend on the particular version of MSI that
you have installed. In my testing I actually had some trouble persuading it
to restore the original compiler DLL when I wanted it to - I had to delete
the new DLL before running the "repair".

>
> Will it be necessary to compile methods for the change to be felt
throughout
> the system?  Of course, recompiling is a likely side effect of debugging,
so
> it would be largely self-repairing.
>

No. This only affects debug information, which is built on-the-fly and not
stored in the compiled methods themselves. There is a cache of this
information, however, which can be cleared thus:

    CompiledCode initializeInfoCache

There is now a self-contained chunk file patch for this on our web site
which will install the new compiler and clear the cache. All you need do is
file it in and then restart Dolphin:

http://object-arts.com/Lib/Update/Dolphin/5.1/1461.st

Regards

Blair

Blair,

>[...] debug information, which is built on-the-fly and not
> stored in the compiled methods themselves. There is a cache of this
> information, however, which can be cleared thus:
>
>     CompiledCode initializeInfoCache

That reminds me of something that I've been meaning to ask about for some time.

In one of my things (actually JNIPort, which I've mentioned before) I generate
large numbers of temporary classes, with references to some of my "system
objects" embedded in the temporary methods' literal frames (as if by the
##(xxx) mechanism, although I implement it differently).  JNIPort can't persist
over image save/restart so it shuts itself down automatically as Dolphin starts
up.  That all works fine, but I was finding that even after shutting down, I
still had *LOTS* of references to objects that were left over from the dead
JNIPort session.  I eventually traced it to the debug cache which was keeping
methods I'd stepped through alive, and hence keeping the entire JNIPort system
from being GCed.  My current workaround is to do a
    CompiledCode initializeInfoCache
as part of my cleanup-on-image-start code, which seems to work OK.

After all that, my question is, shouldn't Dolphin be doing this itself anyway ?
Or maybe the cache should use weak references ?

It's not a big deal, I can always just leave my own call to
#initializeInfoCache (unless it's likely to be harming something I haven't
thought of ?), but I thought I'd ask anyway.

    -- chris

Blair,

I don't know if it has anything to do with the new compiler, but after
installing it, the automatic selection when single-stepping through code
using the debugger sometimes isn't on the correct word. I have only seen
this in a very long method automatically generated by SmaCC (about 130 lines
of code) and unfortunately I hadn't tried this code before installing the
new compiler, so it might have been the same before.

Best regards,

Mikael Svane

"Mikael Svane" <[hidden email]> wrote in message
news:c2dhe3$1rik3r$[hidden email]...
> Blair,
>
> I don't know if it has anything to do with the new compiler, but after
> installing it, the automatic selection when single-stepping through code
> using the debugger sometimes isn't on the correct word. I have only seen
> this in a very long method automatically generated by SmaCC (about 130
lines
> of code) and unfortunately I hadn't tried this code before installing the
> new compiler, so it might have been the same before.

Normally I would not be confident that changing a C++ module did not have
some unexpected side effect, but in this case the modification really could
not possibly have had any effect on the text maps, and therefore this must
be a pre-existing issue. If you can e-mail me a reproduceable case then I
will record it as a new issue.

Regards

Blair

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]...
> Blair,
>
> >[...] debug information, which is built on-the-fly and not
> > stored in the compiled methods themselves. There is a cache of this
> > information, however, which can be cleared thus:
> >
> >     CompiledCode initializeInfoCache
>
> That reminds me of something that I've been meaning to ask about for some
time.
> ...
>...the debug cache which was keeping
> methods I'd stepped through alive, and hence keeping the entire JNIPort
system
> from being GCed.  My current workaround is to do a
>     CompiledCode initializeInfoCache
> as part of my cleanup-on-image-start code, which seems to work OK.
>
> After all that, my question is, shouldn't Dolphin be doing this itself
anyway ?
> Or maybe the cache should use weak references ?

The cache does use weak references, at least it weakly references the
methods with which the debug information is associated. However I don't
think it would do any harm at all to clear the cache on session startup, or
even at other times such as image save. The caching of the information is
only necessary to maintain reasonable performance of the debugger when it is
stepping through code, so its content can be freely discarded.

Regards

Blair

Blair,

After examining the problem a bit more, it turns out that it is caused by a
bug in SmaCC. The code generated in order to check for whitespaces is not
correct, which means that the code in a particular method cannot be
accepted. However, there seems to exist a working compiled method, since
everything works ok when the method is evaluated. I have contacted
Refactory, Inc. about this but not yet received an answer. Manually
correcting the errors in the code and then accepting the changes makes the
problem wich appeared in the debugger go away.

Best regards,
Mikael Svane


"Blair McGlashan" <blair@no spam object-arts.com> skrev i meddelandet
news:[hidden email]... the
> > new compiler, so it might have been the same before.
>
> Normally I would not be confident that changing a C++ module did not have
> some unexpected side effect, but in this case the modification really
could
> not possibly have had any effect on the text maps, and therefore this must
> be a pre-existing issue. If you can e-mail me a reproduceable case then I
> will record it as a new issue.
>
> Regards
>
> Blair
>
>

"Mikael Svane" <[hidden email]> wrote in message
news:c2ip0j$1tt9nm$[hidden email]...
> Blair,
>
> After examining the problem a bit more, it turns out that it is caused by
a
> bug in SmaCC. The code generated in order to check for whitespaces is not
> correct, which means that the code in a particular method cannot be
> accepted. However, there seems to exist a working compiled method, since
> everything works ok when the method is evaluated. I have contacted
> Refactory, Inc. about this but not yet received an answer. Manually
> correcting the errors in the code and then accepting the changes makes the
> problem wich appeared in the debugger go away.

I think you may find that the problem with the generated message is that it
contains control characters in the source code text, e.g. $<backspace>.
These will compile correctly when passed into the Compiler, but are
presumably interpreted as control characters (rather than text) by the
RichEdit control when displayed. This would mean that what you see is not
what you've got :-), and if you try to Accept the method to recompile it,
then it will fail. It's possible that the compiler should be quoting the
control characters in some way in the RTF it generates, although strictly
speaking I'm not sure that unprintable characters are actually valid
character literals in Smalltalk, so it should probably reject them with a
compilation error. In Dolphin 6 which uses Scintilla instead of the RichEdit
control for source panes (and no RTF is involved), there is no problem
displaying unprintable character literals, since Scintilla is able to
display them as special markers, e.g. it displays $BS (on a filled
background) for $<backspace>.

Regards

Blair

Blair

> The cache does use weak references, at least it weakly references the
> methods with which the debug information is associated.

Ah, I see.  I think :-(

The keys are weak, but the values are strong, and they contain an indirect
reference to the debug version of the method, which in turn has the same
set of literals as the real method.  Hence the network of objects is kept
alive.

Thanks for the clarification.  I think I'll change the way I'm doing things so
that I routinely clear the cache as part of my shutdown code -- seems a bit
cleaner than only doing it on image startup.

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]...
That might appear to be the case, but as far as I can remember (and tell
from a quick investigation) the debug information is retrieved in a separate
compilation pass, so the method held in the "value" is not that in the
"key". Certainly this would appear to be confirmed by running this
expression:

    keys := CompiledCode classPool at: 'InfoCacheKeys'.
    values := CompiledCode classPool at: 'InfoCacheValues'.
    (1 to: keys size) select: [:each | (keys at: each) notNil and: [(keys
at: each) == (values at: each) method]]

Regards

Blair

Blair,

Thanks for checking.  Yes, the value's method is a different object all right,
but it is still a version of the "same" method, and so it needs to have
references to the same objects in its literal frame (I think).

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]... right,
> but it is still a version of the "same" method, and so it needs to have
> references to the same objects in its literal frame (I think).

Yes, that would be true, but equally it could not be preventing the entry in
the info cache from disappearing. If there are no references to the actual
compiled method that was debugged (that is the debug version of a method
that was created by the debugger on the first 'step' inside it) then it
should eventually get collected. Since a debug method is "unbound", in that
it does not occur in any class' method dictionary, it should become eligible
for collection as soon as it is no longer in use by the debugged process. It
might take an arbitrary number of collection cycles for the method to
disappear, however, if it is in a long chain of weak references, but that
wouldn't just be the result of the debug info cache.

Actually I suspect that the extra method held in the value is not needed,
and could be discarded. If this were done, or better still the compiler did
not build it in the first place, then there would be little danger of the
debug info cache extending the lifetime of application objects. I'd need to
check that the debugger doesn't use it, however.

Regards

Blair

Blair,

> If there are no references to the
> actual compiled method that was debugged (that is the debug version of a
> method that was created by the debugger on the first 'step' inside it)
> then it should eventually get collected.

Hmm, yes.  Odd that.

I notice that if I debug something (in a clean image) then the cache doesn't
seem to clear itself, even after several GC's.  Presumably something is holding
on to references somewhere.  The cache doesn't seem to grow without limit,
though, so I guess that there are Blocks somewhere or other that are holding
references to the debug methods until they are re-used for something else (it's
a difficult thing to investigate ;-).

Anyway, unless this is ringing alarm bells for you, I am just going to assume
that this is an anomaly of the current Block implementation that should be
fixed by the new version in D6, and that until then the brute force that I'm
already using is as good a fix as any.

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]... doesn't
> seem to clear itself, even after several GC's.  Presumably something is
holding
> on to references somewhere.

I'd be surprised if you find any debug methods in there when things have
quiesced:

    (CompiledCode classPool at: 'InfoCacheKeys') select: [:each | each
notNil and: [each isDebugMethod]]

Normal methods also get inserted, and these will remain while they are bound
into a class' method dictionary. You may wonder why normal methods are in
there, well the answer (in case you didn't already know) is because until
you actually attempt to step through a method the "normal" method is left in
the call stack. The debugger still needs to be able to display the source
position and in-scope temps for frames that have not been converted to debug
frames.

>...The cache doesn't seem to grow without limit,...

The cache is of a fixed size, approx 100 (nearest prime) slots I think. New
entries simply overwrite old ones that hash to the same slot.

> ...though, so I guess that there are Blocks somewhere or other that are
holding
> references to the debug methods until they are re-used for something else
(it's
> a difficult thing to investigate ;-).
>

The ReferenceFinder is your friend :-)

> Anyway, unless this is ringing alarm bells for you, I am just going to
assume
> that this is an anomaly of the current Block implementation that should be
> fixed by the new version in D6, and that until then the brute force that
I'm
> already using is as good a fix as any.

Fair enough, that is often the cause.

Regards

Blair

Blair,

> Normal methods also get inserted, and these will remain while they are
> bound into a class' method dictionary. You may wonder why normal methods
> are in there, well the answer [...]

I didn't realise that.  Interesting.  Thank you.


> > ...though, so I guess that there are Blocks somewhere or other that are
> > holding references to the debug methods until they are re-used for
> > something else (it's a difficult thing to investigate ;-).
> >
>
> The ReferenceFinder is your friend :-)

Albeit, something of a fair-weather friend...

It was this exact issue that first persuaded me that the ReferenceFinder
couldn't be relied on -- it couldn't find anything at all that was keeping the
network of JNIPort objects alive (I don't know why, always meant to
investigate, but...).  It's still a valuable tool when it works, though.

    -- chris

Chris,

> > The ReferenceFinder is your friend :-)
>
> Albeit, something of a fair-weather friend...
>
> It was this exact issue that first persuaded me that the ReferenceFinder
> couldn't be relied on -- it couldn't find anything at all that was keeping
the
> network of JNIPort objects alive (I don't know why, always meant to
> investigate, but...).  It's still a valuable tool when it works, though.

I don't know what you mean by "works".  You might want to try the following
to simplify launching it w/o clutter from inspectors.

Have a good one,

Bill

---------------------------------------------------

Object subclass: #ReferenceFinderLatch
 instanceVariableNames: 'subject'
 classVariableNames: 'Current'
 poolDictionaries: ''
 classInstanceVariableNames: ''!
ReferenceFinderLatch guid: (GUID fromString:
'{1CD025C4-4A21-4747-910D-71FB676AA459}')!
ReferenceFinderLatch comment: 'This documentation represents too little too
late, but I _think_ the idea is as follows:

<pre>
    ReferenceFinderLatch subject:SUnitBrowser allInstances first.
    ReferenceFinderLatch find.
</pre>

Note that you will want to inspect the results of #find.  The idea is to
skip having to find the object of interest in SomeClass allInstances.'!
!ReferenceFinderLatch categoriesForClass!Kernel-Objects! !




!ReferenceFinderLatch methodsFor!

find
 "Ask ReferenceFinder to locate the receiver's subject"

 [
  ^self findAllPathsFrom:Smalltalk
 ] ensure:[
  self class clear.
 ]
! !
!ReferenceFinderLatch categoriesFor: #find!public! !



!ReferenceFinderLatch methodsFor!

findAllPathsFrom:anObject
 "Ask ReferenceFinder to locate the receiver's subject"

  | paths latchPath |

 paths := ReferenceFinder findAllPathsTo:subject from:anObject.

 latchPath := paths detect:[ :each | ( each at:each size - 1 ) == self ].
 ^paths remove:latchPath; yourself
! !
!ReferenceFinderLatch categoriesFor: #findAllPathsFrom:!public! !


!ReferenceFinderLatch methodsFor!

subject
 ^subject! !
!ReferenceFinderLatch categoriesFor: #subject!accessing!private! !



!ReferenceFinderLatch methodsFor!

subject:aSubject
 subject := aSubject.! !
!ReferenceFinderLatch categoriesFor: #subject:!initializing!private! !


!ReferenceFinderLatch class methodsFor!

clear
 Current := nil
! !
!ReferenceFinderLatch class categoriesFor: #clear!public! !

!ReferenceFinderLatch class methodsFor!

find
 ^Current find
! !
!ReferenceFinderLatch class categoriesFor: #find!public! !


!ReferenceFinderLatch class methodsFor!

subject
 "Answer the currently latched subject to be found"

 ^Current subject
! !
!ReferenceFinderLatch class categoriesFor: #subject!instance
creation!public! !

!ReferenceFinderLatch class methodsFor!

subject:aSubject
 "Answer a new latch for finding paths to aSubject"

 ^( Current := self new
  subject:aSubject;
  yourself )
! !
!ReferenceFinderLatch class categoriesFor: #subject:!instance
creation!public! !


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Bill,

> I don't know what you mean by "works".

I asked it to find the paths to one of the objects that I *knew* was hanging
around in the image (but "shouldn't" have been) and it whirred for a while and
then told me that there were none.

I didn't believe it.

And after rather more detective work than I really like, managed to narrow it
down to the compiled method cache.  I have no idea why (or even how) it was
managing to miss that, but apparently it was...


> You might want to try the
> following to simplify launching it w/o clutter from inspectors.

Thanks, Bill.  Added to my toolset.

    -- chris