Some segfault crashes and freezes when trying to re-run saved pharo 7 images

Hello,

I've had a few issues with Pharo 7 lately. After downloading pharo 7
images, doing some work in them and saving them, the images either
instantly crashe or freeze when I try to re-run them.
If that helps, I was working with and on the calypso browser.

Attached are the crash dump files from some of these faulty images.

vm version: 201708271955
image version: 6521
OS: Ubuntu 16.04 LTS

Cheers,
Thomas

Thanks thomas we should find the problem.

Stef

On Fri, Oct 20, 2017 at 3:57 PM, Thomas Dupriez
<[hidden email]> wrote:

> On 21. Oct 2017, at 20:30, Stephane Ducasse <[hidden email]> wrote:
>


Hey!


> Thanks thomas we should find the problem.
>

my CI started to suffer from this too (and I have one where it is broken and now one where I can reproduce it). I started the macOS VM with --trace[1] and have the attached the trace.

What is note-worthy:

1.) FT2Handle class>>#startUp: isn't called. Which means FreeTypeFace>>#beNull has not been called yet!

 (GlobalIdentifier, Delay, DelaySpinScheduler, ProcessorScheduler seem to have went through >>#startUp:)


2.) No one calls >>#fileContentsExternalMemory:


3.) We touch old memory and crash!


FreeTypeGlyphRenderer class>current
FreeTypeSubPixelAntiAliasedGlyphRenderer>subGlyphOf:colorValue:mono:subpixelPosition:font:
FreeTypeSubPixelAntiAliasedGlyphRenderer>renderStretchedGlyph:depth:subpixelPosition:font:
...
FreeTypeFont>face
FreeTypeFace>validate
SmalltalkImage>session
SessionManager class>default
SessionManager>currentSession
SmalltalkImage>session
FreeTypeFace>create
FreeTypeFace(FT2Face)>newFaceFromExternalMemory:index:



I don't know what the actions should be?

* Don't touch anything with fonts in the start-up path?
* Run FT2Handle class >> #startUp earlier?
* Pass the ByteArray to FreeTypeFace and let it pin it to memory? I think FreeTypeExternalMemory could go away with Spur?
* Reset FreeTypeFace instVar fileContentsExternalMemory in >>#beNull?
* Throw away all FreeTypeFace when we have a new Session?

I think implementing:

        FreeTypeExternalMemory class >> #bytes: aByteArray
                ^(aByteArray copy)
                        pin;
                        yourself

could solve most of it? (We can argue about the copy...)


holger



[1] Well it is broken as VMOPTIONOBJ adds the "-" to the front but all code that is using numbers do not handle the extra digit in sqSqueakOSXApplication.m. Maybe somebody beats me with a PR for it. ;)

> On 24. Mar 2018, at 16:42, Holger Freyther <[hidden email]> wrote:
>
>
>
> 1.) FT2Handle class>>#startUp: isn't called. Which means FreeTypeFace>>#beNull has not been called yet!
>



> I think implementing:
>
> FreeTypeExternalMemory class >> #bytes: aByteArray
> ^(aByteArray copy)
> pin;
> yourself
>
> could solve most of it? (We can argue about the copy...)

It doesn't because most of FT2Plugin expects the "handle" it invented. But if we look at it...

FreeTypeFace>validate
SmalltalkImage>session
SessionManager class>default
SessionManager>currentSession
SmalltalkImage>session
FreeTypeFace>create
FreeTypeFace(FT2Face)>newFaceFromExternalMemory:index:
FreeTypeExternalMemory>validate
FreeTypeExternalMemory(FT2Handle)>isValid

FreeTypeFace>>#validate already does:

        (session == Smalltalk session
                ...


And this is why FreeTypeFace>>#create is being called and that will call validate on the FreeTypeExternalMemory instance...


FreeTypeExternalMemory>validate just checks if the handle isValid but remember 1st from my previous mail. We have not _yet_ cleared the FT2Handle SubInstances.. So the memory is all good.


So back to the ideas.

a.) Make sure FT2Handler class>>#startUp runs a lot earlier
b.) Find out which UI thing uses freetype earlier (but FreeType can be used in non GUI apps. E.g. to draw text to an image...)
c.) Keep the session in FreeTypeExternalMemory as well and use it in >>#validate.. (not relying on the startUp order)
d.) Re-write FreeType with Alien and just use the Plugin to conveniently link/load to freetype..


holger

Hi Holger,


I wonder if there is sense in trying to come up with a general memory handle object that includes a session identifier, so that attempts to free stake memory always fail.

> holger

> On 24. Mar 2018, at 17:03, Holger Freyther <[hidden email]> wrote:
>
>
>
> FreeTypeExternalMemory>validate just checks if the handle isValid but remember 1st from my previous mail. We have not _yet_ cleared the FT2Handle SubInstances.. So the memory is all good.

The snippet below is my work-around. I think the proper fix is to get FT2Handle to invalidate handles a lot earlier (Gui->System startup?).


Author useAuthor: 'Hacks' during: [
 FreeTypeExternalMemory addInstVarNamed: 'session'.
 FreeTypeExternalMemory compile: 'validate
        (session == Smalltalk session
                and: [ self isValid ]) ifTrue: [ ^self ].
        bytes ifNil: [ ^ self  ].
        session := Smalltalk session.
        self primCopyToExternalMemory: bytes.
        self isValid ifTrue: [ self class register: self ]'
].
Smalltalk snapshot: true andQuit: true.

> On 24. Mar 2018, at 19:22, Eliot Miranda <[hidden email]> wrote:
>

Dear Eliot,


>> d.) Re-write FreeType with Alien and just use the Plugin to conveniently link/load to freetype..
>
> I wonder if there is sense in trying to come up with a general memory handle object that includes a session identifier, so that attempts to free stake memory always fail.

I think it would help with the FT2Handle (to invalidate it on a new session) but it will not solve other image resume/FFI issues. Let's assume we have multiple calls into a C library (get a handle, call 1st method, call 2nd method, have a loop with backwards jumps). Something like:


   | memory |
   memory := GetSomeMemorySomewhere.

   1 to: 5 do: [
     self doSomeCStuffWithMemory: memory pointerAndCheckStillValid.
     ... more C stuff
   ].

We could be interrupted at any point and when the execution is resumed the handles might be invalid. In Python terminology something like a context manager could help:


# Image saving is delayed/inhibited until after the callout chain
CCallOut with: [
   1 to: 5 do: [
     self doSomeCStuffWithMemory: memory pointerAndCheckStillValid.
     ... more C stuff
   ].
]

What do you think?

        holger

> On 26. Mar 2018, at 00:00, Eliot Miranda <[hidden email]> wrote:
>
> Hi Holger,

Hey!


> Is the intent of CCallOut with: aBlock to collect and defer deallocations until the block completes?  I think it's nice but complex and wonder how general it is.  But it seems like it would impact a lot of code and require a lot of effort changing existing code bases.  My handle scheme is only intended to fix the issue of images crashing on startup. The problems with crashing on startup being a) one loses one's work and b) the issue is hard to debug.  That, for me, motivates something like the simple fix I p[roposed.  I'm not standing in the way of something more beautiful, but I do believe that one shouldn't make the perfect the enemy of the good.

By all means, let's have a smart pointer! It will be beneficial for freetype and many other places. Nobody likes crashing images (for a mistake made in a previous run).


My intent with the CCallOut is inhibit image saving until the code is outside a sequence of (interruptible, hence the backward jump int the example) C calls working with one or more pieces of manually managed memory. But maybe it is best to call it by what it does instead of finding a name of where it is used.

I can't come up with a better example right now. But I think a Smartpointer wouldn't be of much help with a sequence of strtok calls.

        word := CStringAPI strtok: cStringPtr safeBytes separator: '\r'.
        [word isNil] whileFalse: [
                word = 'Foo'...
                word = 'Bla'...
                word := CStringAPI strtok: nil separator: '\r'
 ------------- Image Save happening right here and word is not nil -----
        ].

I hope this is more clear.


holger