Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

Mariano Martinez Peck
 
So...after debugging the VM, it looks it is crashing in DeflatePlugin >> encodeMatch: length distance: dist

encodeMatch: length distance: dist
    "Encode the given match of length length starting at dist bytes ahead"
    | literal distance |
    <inline: true>
    zipLiterals at: zipLiteralCount put: length - DeflateMinMatch.
    zipDistances at: zipLiteralCount put: dist.
    literal := (zipMatchLengthCodes at: length - DeflateMinMatch).
    zipLiteralFreq at: literal put: (zipLiteralFreq at: literal) + 1.
    dist < 257
        ifTrue:[distance := zipDistanceCodes at: dist - 1]
        ifFalse:[distance := zipDistanceCodes at: 256 + (dist - 1 bitShift: -7)].
    zipDistanceFreq at: distance put: (zipDistanceFreq at: distance) + 1.
    zipLiteralCount := zipLiteralCount + 1.
    zipMatchCount := zipMatchCount + 1.
    ^zipLiteralCount = zipLiteralSize "We *must* flush"
        or:[(zipLiteralCount bitAnd: 16rFFF) = 0 "Only check every N kbytes"
            and:[self shouldFlush]]

When it does:     zipDistanceFreq at: distance put: (zipDistanceFreq at: distance) + 1.
it gives a EXC_BAD_ACCESS.
distance value is 1444965993

The sender is #deflateBlock: lastIndex chainLength: chainLength goodMatch: goodMatch
and sender #primitiveDeflateBlock

If I do a call printAllStacks(), I get this part:

(gdb) call printAllStacks()
Process 0x1f70c924 priority 40
0xbff65968 M GZipWriteStream(DeflateStream)>deflateBlock 555574892: a(n) GZipWriteStream
0xbff6598c M GZipWriteStream(DeflateStream)>nextPutAll: 555574892: a(n) GZipWriteStream
0xbff659a8 M GZipWriteStream(DeflateStream)>next:putAll:startingAt: 555574892: a(n) GZipWriteStream
0xbff659cc M FLBufferedWriteStream>flushBuffer 556072120: a(n) FLBufferedWriteStream
0xbff659e4 M FLBufferedWriteStream>nextBytesPutAll: 556072120: a(n) FLBufferedWriteStream
0xbff65a04 M FLBufferedWriteStream>nextStringPut: 556072120: a(n) FLBufferedWriteStream
0xbff65a24 M [] in FLVariablesMapping>serializeMappingOn: 557179132: a(n) FLVariablesMapping
0xbff65a44 M OrderedCollection>do: 557179148: a(n) OrderedCollection
0xbff65a64 M FLVariablesMapping>serializeMappingOn: 557179132: a(n) FLVariablesMapping
0xbff65a80 M FLFixedObjectCluster(FLPointerObjectCluster)>serializeInstancesStepWith: 556081188: a(n) FLFixedObjectCluster
0xbff65a9c M FLSerialization>clusterInstancesStepOf: 556076884: a(n) FLSerialization
0xbff65ab8 M [] in FLSerialization>instancesStep 556076884: a(n) FLSerialization


And from the gdb I get:

#0    0x000b3721 in deflateBlockchainLengthgoodMatch at ZipPlugin.c:303
#1    0x000b4a62 in primitiveDeflateBlock at ZipPlugin.c:719
#2    0x1f400858 in ??
#3    0x00067820 in threadSchedulingLoop at gcc3x-cointerpmt.c:45853
#4    0x0003d79b in initialEnterSmalltalkExecutive at gcc3x-cointerpmt.c:18372
#5    0x0003e45f in initStackPagesAndInterpret at gcc3x-cointerpmt.c:18839
#6    0x00022ae8 in interpret at gcc3x-cointerpmt.c:2076
#7    0x0006df20 in -[sqSqueakMainApplication runSqueak] at sqSqueakMainApplication.m:174
#8    0x92ee186c in __NSFirePerformWithOrder
#9    0x967fadd2 in __CFRunLoopDoObservers
#10    0x967b6ced in __CFRunLoopRun
#11    0x967b63c4 in CFRunLoopRunSpecific
#12    0x967b61f1 in CFRunLoopRunInMode
#13    0x98262e04 in RunCurrentEventLoopInMode
#14    0x98262af5 in ReceiveNextEventCommon
#15    0x98262a3e in BlockUntilNextEventMatchingListInMode
#16    0x93159595 in _DPSNextEvent
#17    0x93158dd6 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
#18    0x9311b1f3 in -[NSApplication run]
#19    0x93113289 in NSApplicationMain
#20    0x0006bec9 in main at main.m:52


Any pointer is appreaciated.

THanks



On Fri, Dec 2, 2011 at 11:57 PM, Diogenes Moreira <[hidden email]> wrote:
en que SO estas corriendo..primitiva ??
por otro lado que tenes en el PharoDebug.Log

http://about.me/diogenes.moreira



2011/12/2 Mariano Martinez Peck <[hidden email]>:
> Hi guys. We are doing some experiments with Fuel and GZip streams... Now I
> have a beatiful test like this:
>
> testMethodContextThisContext
>
>     | materializedContext context |
>     context := thisContext copy.
>     materializedContext := self resultOfSerializeAndMaterialize: context.
>
> and it crashes like crazy. From the VM crash stack, I can see:
>
>
> Smalltalk stack dump:
> 0xbff66ba4 M GZipWriteStream(DeflateStream)>deflateBlock 447551436: a(n)
> GZipWriteStream
> 0xbff66bc8 M GZipWriteStream(DeflateStream)>nextPutAll: 447551436: a(n)
> GZipWriteStream
> 0xbff66be4 M GZipWriteStream(DeflateStream)>next:putAll:startingAt:
> 447551436: a(n) GZipWriteStream
> 0xbff66c08 M FLBufferedWriteStream>flushBuffer 448048664: a(n)
> FLBufferedWriteStream
> 0xbff66c20 M FLBufferedWriteStream>nextBytesPutAll: 448048664: a(n)
> FLBufferedWriteStream
> 0xbff66c40 M FLBufferedWriteStream>nextStringPut: 448048664: a(n)
> FLBufferedWriteStream
> 0xbff66c5c M FLByteStringCluster>serialize:on: 448054196: a(n)
> FLByteStringCluster
> 0xbff66c7c M FLByteStringCluster(FLPrimitiveCluster)>serializeInstance:with:
> 448054196: a(n) FLByteStringCluster
> 0xbff66ca0 M [] in
> FLByteStringCluster(FLIteratingCluster)>serializeInstances: 448054196: a(n)
> FLByteStringCluster
> 0xbff66cc8 M IdentitySet(Set)>do: 451711680: a(n) IdentitySet
> 0xbff66ce4 M FLByteStringCluster(FLIteratingCluster)>serializeInstances:
> 448054196: a(n) FLByteStringCluster
> 0xbff66d00 M
> FLByteStringCluster(FLIteratingCluster)>serializeInstancesStepWith:
> 448054196: a(n) FLByteStringCluster
> 0xbff66d1c M FLSerialization>clusterInstancesStepOf: 448053428: a(n)
> FLSerialization
> 0xbff66d38 M [] in FLSerialization>instancesStep 448053428: a(n)
> FLSerialization
> 0xbff66d58 M OrderedCollection>do: 449989736: a(n) OrderedCollection
> 0xbff66d74 M FLSerialization>instancesStep 448053428: a(n) FLSerialization
> 0xbff66d8c M FLSerialization>run 448053428: a(n) FLSerialization
> 0xbff66db0 I [] in FLSerializer>defaultSerialization 447550464: a(n)
> FLSerializer
> 0xbff66dd8 M [] in FLSerializer>serialize:on: 447550464: a(n) FLSerializer
> 0xbff66df8 M BlockClosure>ensure: 448052884: a(n) BlockClosure
> 0xbff66e18 M FLSerializer>serialize:on: 447550464: a(n) FLSerializer
> 0xbff66e3c M [] in
> FLMethodContextSerializationTest(FLSerializationTest)>serialize: 447235380:
> a(n) FLMethodContextSerializationTest
> 0xbff66e60 M [] in GZipWriteStream class(PositionableStream class)>on:do:
> 419845144: a(n) GZipWriteStream class
> 0xbff66e80 M BlockClosure>ensure: 448048616: a(n) BlockClosure
> 0xbff66ea0 M GZipWriteStream class(PositionableStream class)>on:do:
> 419845144: a(n) GZipWriteStream class
> 0xbff66ec4 M [] in FLGZipStrategy>writeStreamDo: 447550896: a(n)
> FLGZipStrategy
> 0xbff66ee0 M FLByteArrayStreamStrategy>writeStreamDo: 447550884: a(n)
> FLByteArrayStreamStrategy
> 0xbff66efc M FLGZipStrategy>writeStreamDo: 447550896: a(n) FLGZipStrategy
> 0xbff66f18 M
> FLMethodContextSerializationTest(FLSerializationTest)>serialize: 447235380:
> a(n) FLMethodContextSerializationTest
> 0xbff66f34 M
> FLMethodContextSerializationTest(FLSerializationTest)>resultOfSerializeAndMaterialize:
> 447235380: a(n) FLMethodContextSerializationTest
> 0xbff66f58 M FLMethodContextSerializationTest>testMethodContextThisContext
> 447235380: a(n) FLMethodContextSerializationTest
>
>
> any idea of where I should start to look at?
>
> Thanks
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>



--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

David T. Lewis
 
Hi Mariano,

This looks like a nasty bug in the plugin, but I cannot reproduce it.
I have Fuel loaded in my Squeak image running on Linux, and I ran
FLMethodContextSerializationTest>>testMethodContextThisContext without
a crash. I tried it on both Cog and a standard VM, and did not get
a crash in either case. The test fails (probably for some Squeak/Pharo issue)
but it gets through the serialize/materialize part of the test without
any problem.

Am I perhaps running a different version of Fuel that does not
exercise the ZipPlugin in the same way?

Dave


On Fri, Dec 09, 2011 at 07:31:56PM +0100, Mariano Martinez Peck wrote:

>  
> So...after debugging the VM, it looks it is crashing in DeflatePlugin >>
> encodeMatch: length distance: dist
>
> encodeMatch: length distance: dist
>     "Encode the given match of length length starting at dist bytes ahead"
>     | literal distance |
>     <inline: true>
>     zipLiterals at: zipLiteralCount put: length - DeflateMinMatch.
>     zipDistances at: zipLiteralCount put: dist.
>     literal := (zipMatchLengthCodes at: length - DeflateMinMatch).
>     zipLiteralFreq at: literal put: (zipLiteralFreq at: literal) + 1.
>     dist < 257
>         ifTrue:[distance := zipDistanceCodes at: dist - 1]
>         ifFalse:[distance := zipDistanceCodes at: 256 + (dist - 1 bitShift:
> -7)].
>     zipDistanceFreq at: distance put: (zipDistanceFreq at: distance) + 1.
>     zipLiteralCount := zipLiteralCount + 1.
>     zipMatchCount := zipMatchCount + 1.
>     ^zipLiteralCount = zipLiteralSize "We *must* flush"
>         or:[(zipLiteralCount bitAnd: 16rFFF) = 0 "Only check every N kbytes"
>             and:[self shouldFlush]]
>
> When it does:     zipDistanceFreq at: distance put: (zipDistanceFreq at:
> distance) + 1.
> it gives a EXC_BAD_ACCESS.
> distance value is 1444965993
>
> The sender is #deflateBlock: lastIndex chainLength: chainLength goodMatch:
> goodMatch
> and sender #primitiveDeflateBlock
>
> If I do a call printAllStacks(), I get this part:
>
> (gdb) call printAllStacks()
> Process 0x1f70c924 priority 40
> 0xbff65968 M GZipWriteStream(DeflateStream)>deflateBlock 555574892: a(n)
> GZipWriteStream
> 0xbff6598c M GZipWriteStream(DeflateStream)>nextPutAll: 555574892: a(n)
> GZipWriteStream
> 0xbff659a8 M GZipWriteStream(DeflateStream)>next:putAll:startingAt:
> 555574892: a(n) GZipWriteStream
> 0xbff659cc M FLBufferedWriteStream>flushBuffer 556072120: a(n)
> FLBufferedWriteStream
> 0xbff659e4 M FLBufferedWriteStream>nextBytesPutAll: 556072120: a(n)
> FLBufferedWriteStream
> 0xbff65a04 M FLBufferedWriteStream>nextStringPut: 556072120: a(n)
> FLBufferedWriteStream
> 0xbff65a24 M [] in FLVariablesMapping>serializeMappingOn: 557179132: a(n)
> FLVariablesMapping
> 0xbff65a44 M OrderedCollection>do: 557179148: a(n) OrderedCollection
> 0xbff65a64 M FLVariablesMapping>serializeMappingOn: 557179132: a(n)
> FLVariablesMapping
> 0xbff65a80 M
> FLFixedObjectCluster(FLPointerObjectCluster)>serializeInstancesStepWith:
> 556081188: a(n) FLFixedObjectCluster
> 0xbff65a9c M FLSerialization>clusterInstancesStepOf: 556076884: a(n)
> FLSerialization
> 0xbff65ab8 M [] in FLSerialization>instancesStep 556076884: a(n)
> FLSerialization
>
>
> And from the gdb I get:
>
> #0    0x000b3721 in deflateBlockchainLengthgoodMatch at ZipPlugin.c:303
> #1    0x000b4a62 in primitiveDeflateBlock at ZipPlugin.c:719
> #2    0x1f400858 in ??
> #3    0x00067820 in threadSchedulingLoop at gcc3x-cointerpmt.c:45853
> #4    0x0003d79b in initialEnterSmalltalkExecutive at
> gcc3x-cointerpmt.c:18372
> #5    0x0003e45f in initStackPagesAndInterpret at gcc3x-cointerpmt.c:18839
> #6    0x00022ae8 in interpret at gcc3x-cointerpmt.c:2076
> #7    0x0006df20 in -[sqSqueakMainApplication runSqueak] at
> sqSqueakMainApplication.m:174
> #8    0x92ee186c in __NSFirePerformWithOrder
> #9    0x967fadd2 in __CFRunLoopDoObservers
> #10    0x967b6ced in __CFRunLoopRun
> #11    0x967b63c4 in CFRunLoopRunSpecific
> #12    0x967b61f1 in CFRunLoopRunInMode
> #13    0x98262e04 in RunCurrentEventLoopInMode
> #14    0x98262af5 in ReceiveNextEventCommon
> #15    0x98262a3e in BlockUntilNextEventMatchingListInMode
> #16    0x93159595 in _DPSNextEvent
> #17    0x93158dd6 in -[NSApplication
> nextEventMatchingMask:untilDate:inMode:dequeue:]
> #18    0x9311b1f3 in -[NSApplication run]
> #19    0x93113289 in NSApplicationMain
> #20    0x0006bec9 in main at main.m:52
>
>
> Any pointer is appreaciated.
>
> THanks
>
>
>
> On Fri, Dec 2, 2011 at 11:57 PM, Diogenes Moreira <
> [hidden email]> wrote:
>
> > en que SO estas corriendo..primitiva ??
> > por otro lado que tenes en el PharoDebug.Log
> >
> > http://about.me/diogenes.moreira
> >
> >
> >
> > 2011/12/2 Mariano Martinez Peck <[hidden email]>:
> > > Hi guys. We are doing some experiments with Fuel and GZip streams... Now
> > I
> > > have a beatiful test like this:
> > >
> > > testMethodContextThisContext
> > >
> > >     | materializedContext context |
> > >     context := thisContext copy.
> > >     materializedContext := self resultOfSerializeAndMaterialize: context.
> > >
> > > and it crashes like crazy. From the VM crash stack, I can see:
> > >
> > >
> > > Smalltalk stack dump:
> > > 0xbff66ba4 M GZipWriteStream(DeflateStream)>deflateBlock 447551436: a(n)
> > > GZipWriteStream
> > > 0xbff66bc8 M GZipWriteStream(DeflateStream)>nextPutAll: 447551436: a(n)
> > > GZipWriteStream
> > > 0xbff66be4 M GZipWriteStream(DeflateStream)>next:putAll:startingAt:
> > > 447551436: a(n) GZipWriteStream
> > > 0xbff66c08 M FLBufferedWriteStream>flushBuffer 448048664: a(n)
> > > FLBufferedWriteStream
> > > 0xbff66c20 M FLBufferedWriteStream>nextBytesPutAll: 448048664: a(n)
> > > FLBufferedWriteStream
> > > 0xbff66c40 M FLBufferedWriteStream>nextStringPut: 448048664: a(n)
> > > FLBufferedWriteStream
> > > 0xbff66c5c M FLByteStringCluster>serialize:on: 448054196: a(n)
> > > FLByteStringCluster
> > > 0xbff66c7c M
> > FLByteStringCluster(FLPrimitiveCluster)>serializeInstance:with:
> > > 448054196: a(n) FLByteStringCluster
> > > 0xbff66ca0 M [] in
> > > FLByteStringCluster(FLIteratingCluster)>serializeInstances: 448054196:
> > a(n)
> > > FLByteStringCluster
> > > 0xbff66cc8 M IdentitySet(Set)>do: 451711680: a(n) IdentitySet
> > > 0xbff66ce4 M FLByteStringCluster(FLIteratingCluster)>serializeInstances:
> > > 448054196: a(n) FLByteStringCluster
> > > 0xbff66d00 M
> > > FLByteStringCluster(FLIteratingCluster)>serializeInstancesStepWith:
> > > 448054196: a(n) FLByteStringCluster
> > > 0xbff66d1c M FLSerialization>clusterInstancesStepOf: 448053428: a(n)
> > > FLSerialization
> > > 0xbff66d38 M [] in FLSerialization>instancesStep 448053428: a(n)
> > > FLSerialization
> > > 0xbff66d58 M OrderedCollection>do: 449989736: a(n) OrderedCollection
> > > 0xbff66d74 M FLSerialization>instancesStep 448053428: a(n)
> > FLSerialization
> > > 0xbff66d8c M FLSerialization>run 448053428: a(n) FLSerialization
> > > 0xbff66db0 I [] in FLSerializer>defaultSerialization 447550464: a(n)
> > > FLSerializer
> > > 0xbff66dd8 M [] in FLSerializer>serialize:on: 447550464: a(n)
> > FLSerializer
> > > 0xbff66df8 M BlockClosure>ensure: 448052884: a(n) BlockClosure
> > > 0xbff66e18 M FLSerializer>serialize:on: 447550464: a(n) FLSerializer
> > > 0xbff66e3c M [] in
> > > FLMethodContextSerializationTest(FLSerializationTest)>serialize:
> > 447235380:
> > > a(n) FLMethodContextSerializationTest
> > > 0xbff66e60 M [] in GZipWriteStream class(PositionableStream class)>on:do:
> > > 419845144: a(n) GZipWriteStream class
> > > 0xbff66e80 M BlockClosure>ensure: 448048616: a(n) BlockClosure
> > > 0xbff66ea0 M GZipWriteStream class(PositionableStream class)>on:do:
> > > 419845144: a(n) GZipWriteStream class
> > > 0xbff66ec4 M [] in FLGZipStrategy>writeStreamDo: 447550896: a(n)
> > > FLGZipStrategy
> > > 0xbff66ee0 M FLByteArrayStreamStrategy>writeStreamDo: 447550884: a(n)
> > > FLByteArrayStreamStrategy
> > > 0xbff66efc M FLGZipStrategy>writeStreamDo: 447550896: a(n) FLGZipStrategy
> > > 0xbff66f18 M
> > > FLMethodContextSerializationTest(FLSerializationTest)>serialize:
> > 447235380:
> > > a(n) FLMethodContextSerializationTest
> > > 0xbff66f34 M
> > >
> > FLMethodContextSerializationTest(FLSerializationTest)>resultOfSerializeAndMaterialize:
> > > 447235380: a(n) FLMethodContextSerializationTest
> > > 0xbff66f58 M
> > FLMethodContextSerializationTest>testMethodContextThisContext
> > > 447235380: a(n) FLMethodContextSerializationTest
> > >
> > >
> > > any idea of where I should start to look at?
> > >
> > > Thanks
> > >
> > >
> > > --
> > > Mariano
> > > http://marianopeck.wordpress.com
> > >
> >
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

Mariano Martinez Peck
 


On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis <[hidden email]> wrote:

Hi Mariano,

This looks like a nasty bug in the plugin, but I cannot reproduce it.
I have Fuel loaded in my Squeak image running on Linux, and I ran
FLMethodContextSerializationTest>>testMethodContextThisContext without
a crash. I tried it on both Cog and a standard VM, and did not get
a crash in either case. The test fails (probably for some Squeak/Pharo issue)
but it gets through the serialize/materialize part of the test without
any problem.

Am I perhaps running a different version of Fuel that does not
exercise the ZipPlugin in the same way?


Yes, exactly. Sorry for not saying that before. If you want to reproduce it, you must also install the package FuelCompression and then change FLSerializationTest >> #setUp  to send the message #useGzipInMemoryStream  rather than #useStandardFileStream. Now if you run #testMethodContextThisContext it would do it using a GZip stream.

Anyway, I have also sent an image to Eliot since he asked it.

Here it is the image: https://gforge.inria.fr/frs/download.php/29895/Pharo-CrashGzipThisContext.image.zip
I found a couple of interesting things:

1) it only fails the test for thisContext (testMethodContextThisContext)
, since the rest are ok.
2) I tried with StackVM as well as previous CogVMs and it also crash. However, there is a VM which it doesn't crash: Squeak5.8b4

With the attached image, you have opened the test to run (testMethodContextThisContext). In my case I can reproduce it always.
 
Let me know if you could reproduce it.


@John:  indeed, it is very much likely there is a problem with the size, because it only crashes with #testMethodContextThisContext (and few others) where the graph we serialize is really big (in this case because we serialize thisContext which means we seralize the whole stack of senders and senders). And for the rest of the tests (which are smaller) it usually does not crash.  I will try to let you know the exact size.


Thanks in advance,



 
Dave


On Fri, Dec 09, 2011 at 07:31:56PM +0100, Mariano Martinez Peck wrote:
>
> So...after debugging the VM, it looks it is crashing in DeflatePlugin >>
> encodeMatch: length distance: dist
>
> encodeMatch: length distance: dist
>     "Encode the given match of length length starting at dist bytes ahead"
>     | literal distance |
>     <inline: true>
>     zipLiterals at: zipLiteralCount put: length - DeflateMinMatch.
>     zipDistances at: zipLiteralCount put: dist.
>     literal := (zipMatchLengthCodes at: length - DeflateMinMatch).
>     zipLiteralFreq at: literal put: (zipLiteralFreq at: literal) + 1.
>     dist < 257
>         ifTrue:[distance := zipDistanceCodes at: dist - 1]
>         ifFalse:[distance := zipDistanceCodes at: 256 + (dist - 1 bitShift:
> -7)].
>     zipDistanceFreq at: distance put: (zipDistanceFreq at: distance) + 1.
>     zipLiteralCount := zipLiteralCount + 1.
>     zipMatchCount := zipMatchCount + 1.
>     ^zipLiteralCount = zipLiteralSize "We *must* flush"
>         or:[(zipLiteralCount bitAnd: 16rFFF) = 0 "Only check every N kbytes"
>             and:[self shouldFlush]]
>
> When it does:     zipDistanceFreq at: distance put: (zipDistanceFreq at:
> distance) + 1.
> it gives a EXC_BAD_ACCESS.
> distance value is 1444965993
>
> The sender is #deflateBlock: lastIndex chainLength: chainLength goodMatch:
> goodMatch
> and sender #primitiveDeflateBlock
>
> If I do a call printAllStacks(), I get this part:
>
> (gdb) call printAllStacks()
> Process 0x1f70c924 priority 40
> 0xbff65968 M GZipWriteStream(DeflateStream)>deflateBlock 555574892: a(n)
> GZipWriteStream
> 0xbff6598c M GZipWriteStream(DeflateStream)>nextPutAll: 555574892: a(n)
> GZipWriteStream
> 0xbff659a8 M GZipWriteStream(DeflateStream)>next:putAll:startingAt:
> 555574892: a(n) GZipWriteStream
> 0xbff659cc M FLBufferedWriteStream>flushBuffer 556072120: a(n)
> FLBufferedWriteStream
> 0xbff659e4 M FLBufferedWriteStream>nextBytesPutAll: 556072120: a(n)
> FLBufferedWriteStream
> 0xbff65a04 M FLBufferedWriteStream>nextStringPut: 556072120: a(n)
> FLBufferedWriteStream
> 0xbff65a24 M [] in FLVariablesMapping>serializeMappingOn: 557179132: a(n)
> FLVariablesMapping
> 0xbff65a44 M OrderedCollection>do: 557179148: a(n) OrderedCollection
> 0xbff65a64 M FLVariablesMapping>serializeMappingOn: 557179132: a(n)
> FLVariablesMapping
> 0xbff65a80 M
> FLFixedObjectCluster(FLPointerObjectCluster)>serializeInstancesStepWith:
> 556081188: a(n) FLFixedObjectCluster
> 0xbff65a9c M FLSerialization>clusterInstancesStepOf: 556076884: a(n)
> FLSerialization
> 0xbff65ab8 M [] in FLSerialization>instancesStep 556076884: a(n)
> FLSerialization
>
>
> And from the gdb I get:
>
> #0    0x000b3721 in deflateBlockchainLengthgoodMatch at ZipPlugin.c:303
> #1    0x000b4a62 in primitiveDeflateBlock at ZipPlugin.c:719
> #2    0x1f400858 in ??
> #3    0x00067820 in threadSchedulingLoop at gcc3x-cointerpmt.c:45853
> #4    0x0003d79b in initialEnterSmalltalkExecutive at
> gcc3x-cointerpmt.c:18372
> #5    0x0003e45f in initStackPagesAndInterpret at gcc3x-cointerpmt.c:18839
> #6    0x00022ae8 in interpret at gcc3x-cointerpmt.c:2076
> #7    0x0006df20 in -[sqSqueakMainApplication runSqueak] at
> sqSqueakMainApplication.m:174
> #8    0x92ee186c in __NSFirePerformWithOrder
> #9    0x967fadd2 in __CFRunLoopDoObservers
> #10    0x967b6ced in __CFRunLoopRun
> #11    0x967b63c4 in CFRunLoopRunSpecific
> #12    0x967b61f1 in CFRunLoopRunInMode
> #13    0x98262e04 in RunCurrentEventLoopInMode
> #14    0x98262af5 in ReceiveNextEventCommon
> #15    0x98262a3e in BlockUntilNextEventMatchingListInMode
> #16    0x93159595 in _DPSNextEvent
> #17    0x93158dd6 in -[NSApplication
> nextEventMatchingMask:untilDate:inMode:dequeue:]
> #18    0x9311b1f3 in -[NSApplication run]
> #19    0x93113289 in NSApplicationMain
> #20    0x0006bec9 in main at main.m:52
>
>
> Any pointer is appreaciated.
>
> THanks
>
>
>
> On Fri, Dec 2, 2011 at 11:57 PM, Diogenes Moreira <
> [hidden email]> wrote:
>
> > en que SO estas corriendo..primitiva ??
> > por otro lado que tenes en el PharoDebug.Log
> >
> > http://about.me/diogenes.moreira
> >
> >
> >
> > 2011/12/2 Mariano Martinez Peck <[hidden email]>:
> > > Hi guys. We are doing some experiments with Fuel and GZip streams... Now
> > I
> > > have a beatiful test like this:
> > >
> > > testMethodContextThisContext
> > >
> > >     | materializedContext context |
> > >     context := thisContext copy.
> > >     materializedContext := self resultOfSerializeAndMaterialize: context.
> > >
> > > and it crashes like crazy. From the VM crash stack, I can see:
> > >
> > >
> > > Smalltalk stack dump:
> > > 0xbff66ba4 M GZipWriteStream(DeflateStream)>deflateBlock 447551436: a(n)
> > > GZipWriteStream
> > > 0xbff66bc8 M GZipWriteStream(DeflateStream)>nextPutAll: 447551436: a(n)
> > > GZipWriteStream
> > > 0xbff66be4 M GZipWriteStream(DeflateStream)>next:putAll:startingAt:
> > > 447551436: a(n) GZipWriteStream
> > > 0xbff66c08 M FLBufferedWriteStream>flushBuffer 448048664: a(n)
> > > FLBufferedWriteStream
> > > 0xbff66c20 M FLBufferedWriteStream>nextBytesPutAll: 448048664: a(n)
> > > FLBufferedWriteStream
> > > 0xbff66c40 M FLBufferedWriteStream>nextStringPut: 448048664: a(n)
> > > FLBufferedWriteStream
> > > 0xbff66c5c M FLByteStringCluster>serialize:on: 448054196: a(n)
> > > FLByteStringCluster
> > > 0xbff66c7c M
> > FLByteStringCluster(FLPrimitiveCluster)>serializeInstance:with:
> > > 448054196: a(n) FLByteStringCluster
> > > 0xbff66ca0 M [] in
> > > FLByteStringCluster(FLIteratingCluster)>serializeInstances: 448054196:
> > a(n)
> > > FLByteStringCluster
> > > 0xbff66cc8 M IdentitySet(Set)>do: 451711680: a(n) IdentitySet
> > > 0xbff66ce4 M FLByteStringCluster(FLIteratingCluster)>serializeInstances:
> > > 448054196: a(n) FLByteStringCluster
> > > 0xbff66d00 M
> > > FLByteStringCluster(FLIteratingCluster)>serializeInstancesStepWith:
> > > 448054196: a(n) FLByteStringCluster
> > > 0xbff66d1c M FLSerialization>clusterInstancesStepOf: 448053428: a(n)
> > > FLSerialization
> > > 0xbff66d38 M [] in FLSerialization>instancesStep 448053428: a(n)
> > > FLSerialization
> > > 0xbff66d58 M OrderedCollection>do: 449989736: a(n) OrderedCollection
> > > 0xbff66d74 M FLSerialization>instancesStep 448053428: a(n)
> > FLSerialization
> > > 0xbff66d8c M FLSerialization>run 448053428: a(n) FLSerialization
> > > 0xbff66db0 I [] in FLSerializer>defaultSerialization 447550464: a(n)
> > > FLSerializer
> > > 0xbff66dd8 M [] in FLSerializer>serialize:on: 447550464: a(n)
> > FLSerializer
> > > 0xbff66df8 M BlockClosure>ensure: 448052884: a(n) BlockClosure
> > > 0xbff66e18 M FLSerializer>serialize:on: 447550464: a(n) FLSerializer
> > > 0xbff66e3c M [] in
> > > FLMethodContextSerializationTest(FLSerializationTest)>serialize:
> > 447235380:
> > > a(n) FLMethodContextSerializationTest
> > > 0xbff66e60 M [] in GZipWriteStream class(PositionableStream class)>on:do:
> > > 419845144: a(n) GZipWriteStream class
> > > 0xbff66e80 M BlockClosure>ensure: 448048616: a(n) BlockClosure
> > > 0xbff66ea0 M GZipWriteStream class(PositionableStream class)>on:do:
> > > 419845144: a(n) GZipWriteStream class
> > > 0xbff66ec4 M [] in FLGZipStrategy>writeStreamDo: 447550896: a(n)
> > > FLGZipStrategy
> > > 0xbff66ee0 M FLByteArrayStreamStrategy>writeStreamDo: 447550884: a(n)
> > > FLByteArrayStreamStrategy
> > > 0xbff66efc M FLGZipStrategy>writeStreamDo: 447550896: a(n) FLGZipStrategy
> > > 0xbff66f18 M
> > > FLMethodContextSerializationTest(FLSerializationTest)>serialize:
> > 447235380:
> > > a(n) FLMethodContextSerializationTest
> > > 0xbff66f34 M
> > >
> > FLMethodContextSerializationTest(FLSerializationTest)>resultOfSerializeAndMaterialize:
> > > 447235380: a(n) FLMethodContextSerializationTest
> > > 0xbff66f58 M
> > FLMethodContextSerializationTest>testMethodContextThisContext
> > > 447235380: a(n) FLMethodContextSerializationTest
> > >
> > >
> > > any idea of where I should start to look at?
> > >
> > > Thanks
> > >
> > >
> > > --
> > > Mariano
> > > http://marianopeck.wordpress.com
> > >
> >
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com




--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

David T. Lewis
 
On Sat, Dec 10, 2011 at 03:58:29PM +0100, Mariano Martinez Peck wrote:

>  
> On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis <[hidden email]> wrote:
>
> > This looks like a nasty bug in the plugin, but I cannot reproduce it.
> > I have Fuel loaded in my Squeak image running on Linux, and I ran
> > FLMethodContextSerializationTest>>testMethodContextThisContext without
> > a crash. I tried it on both Cog and a standard VM, and did not get
> > a crash in either case. The test fails (probably for some Squeak/Pharo
> > issue)
> > but it gets through the serialize/materialize part of the test without
> > any problem.
> >
> > Am I perhaps running a different version of Fuel that does not
> > exercise the ZipPlugin in the same way?
> >
> >
> Yes, exactly. Sorry for not saying that before. If you want to reproduce
> it, you must also install the package FuelCompression and then change
> FLSerializationTest >> #setUp  to send the message #useGzipInMemoryStream
> rather than #useStandardFileStream. Now if you run
> #testMethodContextThisContext it would do it using a GZip stream.

I loaded this in my Squeak image on Linux and can now confirm the
crash for both Cog and standard interpreter VM.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

Mariano Martinez Peck
 


On Sat, Dec 10, 2011 at 5:34 PM, David T. Lewis <[hidden email]> wrote:

On Sat, Dec 10, 2011 at 03:58:29PM +0100, Mariano Martinez Peck wrote:
>
> On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis <[hidden email]> wrote:
>
> > This looks like a nasty bug in the plugin, but I cannot reproduce it.
> > I have Fuel loaded in my Squeak image running on Linux, and I ran
> > FLMethodContextSerializationTest>>testMethodContextThisContext without
> > a crash. I tried it on both Cog and a standard VM, and did not get
> > a crash in either case. The test fails (probably for some Squeak/Pharo
> > issue)
> > but it gets through the serialize/materialize part of the test without
> > any problem.
> >
> > Am I perhaps running a different version of Fuel that does not
> > exercise the ZipPlugin in the same way?
> >
> >
> Yes, exactly. Sorry for not saying that before. If you want to reproduce
> it, you must also install the package FuelCompression and then change
> FLSerializationTest >> #setUp  to send the message #useGzipInMemoryStream
> rather than #useStandardFileStream. Now if you run
> #testMethodContextThisContext it would do it using a GZip stream.

I loaded this in my Squeak image on Linux and can now confirm the
crash for both Cog and standard interpreter VM.


Thanks Dave for the confirmation. We haven't use GZip before so I hope we are not doing something wrong ourselfs.



--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

David T. Lewis
 
On Sat, Dec 10, 2011 at 05:38:24PM +0100, Mariano Martinez Peck wrote:
>  
> On Sat, Dec 10, 2011 at 5:34 PM, David T. Lewis <[hidden email]> wrote:
> >
> > I loaded this in my Squeak image on Linux and can now confirm the
> > crash for both Cog and standard interpreter VM.
> >
> Thanks Dave for the confirmation. We haven't use GZip before so I hope we
> are not doing something wrong ourselfs.

We do not want the VM to crash when someone does something wrong, so
I think there must be a problem with the plugin in any case.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

David T. Lewis
In reply to this post by Mariano Martinez Peck
 
On Sat, Dec 10, 2011 at 05:38:24PM +0100, Mariano Martinez Peck wrote:

>  
> On Sat, Dec 10, 2011 at 5:34 PM, David T. Lewis <[hidden email]> wrote:
>
> >
> > On Sat, Dec 10, 2011 at 03:58:29PM +0100, Mariano Martinez Peck wrote:
> > >
> > > On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis <[hidden email]>
> > wrote:
> > >
> > > > This looks like a nasty bug in the plugin, but I cannot reproduce it.
> > > > I have Fuel loaded in my Squeak image running on Linux, and I ran
> > > > FLMethodContextSerializationTest>>testMethodContextThisContext without
> > > > a crash. I tried it on both Cog and a standard VM, and did not get
> > > > a crash in either case. The test fails (probably for some Squeak/Pharo
> > > > issue)
> > > > but it gets through the serialize/materialize part of the test without
> > > > any problem.
> > > >
> > > > Am I perhaps running a different version of Fuel that does not
> > > > exercise the ZipPlugin in the same way?
> > > >
> > > >
> > > Yes, exactly. Sorry for not saying that before. If you want to reproduce
> > > it, you must also install the package FuelCompression and then change
> > > FLSerializationTest >> #setUp  to send the message #useGzipInMemoryStream
> > > rather than #useStandardFileStream. Now if you run
> > > #testMethodContextThisContext it would do it using a GZip stream.
> >
> > I loaded this in my Squeak image on Linux and can now confirm the
> > crash for both Cog and standard interpreter VM.
> >
> >
> Thanks Dave for the confirmation. We haven't use GZip before so I hope we
> are not doing something wrong ourselfs.
>

This is starting to look like an interesting bug. There certainly is
some kind of problem in the serialization phase of the Fuel test, but
of course it is hard to debug that when the VM is crashing.

Good news: I figured out how to make it stop crashing, which should
make it possible for you to go back to debugging your Fuel test.

Bad news: The way to make the VM stop crashing is to compile it with
optimization turned off. That means that perhaps the plugin itself
is not directly at fault, but maybe it is something induced by gcc
optimization. Or possibly the reverse, it may be a bug in the plugin
that is exposed by the optimizer.

I found this by accident because I built a VM with debugging on and
optimization off so that I would be able to catch the crash in a
debugger. Wonder of wonders, it stopped crashing. Specifically, it
worked with 'CFLAGS=-g' and it crashes with 'CFLAGS=-g -O1' or with
'CFLAGS=-g -O2'.

The issue is specific to the plugin itself. If I compile the VM with
optimization on, and the plugin with optimization off, then there
is no crash.

I am using:
Linux version 2.6.34.7-0.7-desktop (geeko@buildhost) (gcc version 4.5.0 20100604 [gcc-4_5-branch revision 160292] (SUSE Linux) ) #1 SMP PREEMPT 2010-12-13 11:13:53 +0100

This is with a standard interpreter VM compiled with gcc 4.5.0. I did
not try reproducing this with Cog, but since the previously reported
symptoms were basically the same for both Cog and the interpreter VM,
I expect that turning off optimization for the ZipPlugin will give the
same result for Cog.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

Mariano Martinez Peck
 


On Sun, Dec 11, 2011 at 6:46 AM, David T. Lewis <[hidden email]> wrote:

On Sat, Dec 10, 2011 at 05:38:24PM +0100, Mariano Martinez Peck wrote:
>
> On Sat, Dec 10, 2011 at 5:34 PM, David T. Lewis <[hidden email]> wrote:
>
> >
> > On Sat, Dec 10, 2011 at 03:58:29PM +0100, Mariano Martinez Peck wrote:
> > >
> > > On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis <[hidden email]>
> > wrote:
> > >
> > > > This looks like a nasty bug in the plugin, but I cannot reproduce it.
> > > > I have Fuel loaded in my Squeak image running on Linux, and I ran
> > > > FLMethodContextSerializationTest>>testMethodContextThisContext without
> > > > a crash. I tried it on both Cog and a standard VM, and did not get
> > > > a crash in either case. The test fails (probably for some Squeak/Pharo
> > > > issue)
> > > > but it gets through the serialize/materialize part of the test without
> > > > any problem.
> > > >
> > > > Am I perhaps running a different version of Fuel that does not
> > > > exercise the ZipPlugin in the same way?
> > > >
> > > >
> > > Yes, exactly. Sorry for not saying that before. If you want to reproduce
> > > it, you must also install the package FuelCompression and then change
> > > FLSerializationTest >> #setUp  to send the message #useGzipInMemoryStream
> > > rather than #useStandardFileStream. Now if you run
> > > #testMethodContextThisContext it would do it using a GZip stream.
> >
> > I loaded this in my Squeak image on Linux and can now confirm the
> > crash for both Cog and standard interpreter VM.
> >
> >
> Thanks Dave for the confirmation. We haven't use GZip before so I hope we
> are not doing something wrong ourselfs.
>

This is starting to look like an interesting bug. There certainly is
some kind of problem in the serialization phase of the Fuel test, but
of course it is hard to debug that when the VM is crashing.

Yes! What should happen with that test is that it should fail because it doesn't find an instVar called 'testSelector'. If that happens, then it is perfect ;)
this is because of something we have recently changed and we need to fix it.
 

Good news: I figured out how to make it stop crashing, which should
make it possible for you to go back to debugging your Fuel test.

Bad news: The way to make the VM stop crashing is to compile it with
optimization turned off. That means that perhaps the plugin itself
is not directly at fault, but maybe it is something induced by gcc
optimization. Or possibly the reverse, it may be a bug in the plugin
that is exposed by the optimizer.

I found this by accident because I built a VM with debugging on and
optimization off so that I would be able to catch the crash in a
debugger. Wonder of wonders, it stopped crashing. Specifically, it
worked with 'CFLAGS=-g' and it crashes with 'CFLAGS=-g -O1' or with
'CFLAGS=-g -O2'.

The issue is specific to the plugin itself. If I compile the VM with
optimization on, and the plugin with optimization off, then there
is no crash.


Thanks Dave for digging in the issue. I tried to do the same as you did with the CMakeVMMaker but I it cashed anyway, so I think I am doing something wrong here.
Igor, could you take a look?  I have implemented:

configureDeflatePlugin: maker

    maker addDefinitions: '-g -O0'.
   

Which generates the following CMakeLists.txt:

# This is automatically generated file using CogMTCocoaIOSConfig on 11 December 2011 1:38:27 pm
cmake_minimum_required(VERSION 2.6.2)
include(CMakeDetermineSystem)
project("ZipPlugin")
include(../directories.cmake)
set(CMAKE_CONFIGURATION_TYPES Release)
message("Adding internal plugin: ZipPlugin")
set(pluginName "ZipPlugin")
set(pluginSrc "${srcPluginsDir}/ZipPlugin")
set(pluginCross "${crossDir}/plugins/ZipPlugin")
set(pluginPlatform "${targetPlatform}/plugins/ZipPlugin")
add_definitions(-DSQUEAK_BUILTIN_PLUGIN)
list(APPEND sources  "${pluginSrc}/ZipPlugin.c")
include_directories(${pluginSrc} ${pluginCross} ${targetPlatform}/plugins/${pluginName})
add_definitions(-arch i386 -mmacosx-version-min=10.4 -DHAVE_UUID_GENERATE -g3 -O0 -msse3 -funroll-loops -fasm-blocks -finline-functions -mfpmath=sse -march=pentium-m -falign-functions=16 -fno-gcse -fno-cse-follow-jumps -std=gnu99 -DBUILD_FOR_OSX -DUSE_INLINE_MEMORY_ACCESSORS -DLSB_FIRST -DHAVE_SYS_TIME_H -DHAVE_NANOSLEEP -DDEBUGVM=1 -DCOGMTVM=1 -DUSE_GLOBAL_STRUCT=0 -DBASE_HEADER_SIZE=4 -DCOGVM)
add_definitions(-g -O0)
add_library(ZipPlugin STATIC ${sources})
set(CMAKE_OSX_ARCHITECTURES i386)
set_target_properties(ZipPlugin PROPERTIES COMPILE_FLAGS "-include \"/Users/mariano/Pharo/VM/git/cogVMBlessedSSH/blessed/build/CogMTVM_Prefix.pch\"")
set_target_properties(ZipPlugin PROPERTIES XCODE_ATTRIBUTE_GCC_PREFIX_HEADER "${buildDir}/CogMTVM_Prefix.pch")
set_property(TARGET ZipPlugin PROPERTY LINK_FLAGS "${linkFlags}")


Thanks
 
I am using:
Linux version 2.6.34.7-0.7-desktop (geeko@buildhost) (gcc version 4.5.0 20100604 [gcc-4_5-branch revision 160292] (SUSE Linux) ) #1 SMP PREEMPT 2010-12-13 11:13:53 +0100

This is with a standard interpreter VM compiled with gcc 4.5.0. I did
not try reproducing this with Cog, but since the previously reported
symptoms were basically the same for both Cog and the interpreter VM,
I expect that turning off optimization for the ZipPlugin will give the
same result for Cog.

Dave





--
Mariano
http://marianopeck.wordpress.com

Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-project] GZipWriteStream crashing my VM when serializing ?

David T. Lewis
 
On Sun, Dec 11, 2011 at 02:10:25PM +0100, Mariano Martinez Peck wrote:

>  
> On Sun, Dec 11, 2011 at 6:46 AM, David T. Lewis <[hidden email]> wrote:
> >
> > This is starting to look like an interesting bug. There certainly is
> > some kind of problem in the serialization phase of the Fuel test, but
> > of course it is hard to debug that when the VM is crashing.
> >
>
> Yes! What should happen with that test is that it should fail because it
> doesn't find an instVar called 'testSelector'. If that happens, then it is
> perfect ;)
> this is because of something we have recently changed and we need to fix
> it.
For what it's worth, I when I run the test with a non-optimized plugin (so
it does not crash), I am able to get to a debugger that suggests some problem
related to array bounds (see attached). No surprise there, but it suggests
that the test is probably feeding some invalid parameters to a primitive
that crashes it in one case but not the other. This is just speculation of
course, but perhaps this is the reason that no one has encountered the bug
up to this time.

> >
> > Good news: I figured out how to make it stop crashing, which should
> > make it possible for you to go back to debugging your Fuel test.
> >
> > Bad news: The way to make the VM stop crashing is to compile it with
> > optimization turned off. That means that perhaps the plugin itself
> > is not directly at fault, but maybe it is something induced by gcc
> > optimization. Or possibly the reverse, it may be a bug in the plugin
> > that is exposed by the optimizer.
> >
> > I found this by accident because I built a VM with debugging on and
> > optimization off so that I would be able to catch the crash in a
> > debugger. Wonder of wonders, it stopped crashing. Specifically, it
> > worked with 'CFLAGS=-g' and it crashes with 'CFLAGS=-g -O1' or with
> > 'CFLAGS=-g -O2'.
> >
> > The issue is specific to the plugin itself. If I compile the VM with
> > optimization on, and the plugin with optimization off, then there
> > is no crash.
> >
> >
> Thanks Dave for digging in the issue. I tried to do the same as you did
> with the CMakeVMMaker but I it cashed anyway, so I think I am doing
> something wrong here.
> Igor, could you take a look?  I have implemented:
>
> configureDeflatePlugin: maker
>
>     maker addDefinitions: '-g -O0'.
>
I was using Ian's CMake build (the standard interpreter VM one for unix),
and I hacked around it by editing this file:
  ZipPlugin/CMakeFiles/ZipPlugin.dir/flags.make

You can probably do something similar with Igor's CMake build as a
workaround for your testing. Just ignore the "do not edit this file"
warning ;)

Dave


error.png (113K) Download Attachment