Smalltalk › Frameworks & Tools › Magma

Serialization sizeWarningThreshold reached

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

2 messages Options

fvozzi

Serialization sizeWarningThreshold reached

Hi Chris, hi all,

I'm testing my app with many objects by bulk loading it. When I do it for 1250 simple object transactions it works fine but when I start with the first of ten files with 42000 simple transactions by each I got this error so I "Proceed" and I got it again, and again. Finally a got Out of memory because Croquet.exe is about 500 mb of ram.

All is working fine, process the file, creating all objects and adding to my application but the problem is when I do commit: to it.

With this information could you guess what i'm doing wrong?

Thanks in advance,

p.s.: in my test i would like bulk load 42000 customers with 10 simple transactions by each.

MaObjectSerializationWarning: Serialization sizeWarningThreshold reached. Proceed to double it.

Debug Proceed Full Stack

Stack Trace

thisContext
MaVariableObjectBuffer(MaObjectBuffer)>>ensureSpaceFor:using:
self
a MaVariableObjectBuffer oid : 0 classId : 0 objectInstSize : -3 commitNumber : MaBufferPositio...etc...
storageObject
a Dictionary(size 47600)
bpMapClass

MagmaBufferPositionMap
totalSize
nil
requiredSpace

nil
thisContext
MaObjectSerializer>>bufferFor:storageObject:startingAt:
self

a MaObjectSerializer
anObject
a Dictionary(size 47600)
storageObject

a Dictionary(size 47600)
anInteger
9691714
buffer

nil
thisContext
MaObjectSerializer>>append:
self

a MaObjectSerializer
anObject
a Dictionary(size 47600)
storageObject

nil
thisContext
[] in MaObjectSerializer>>appendGraph:do:
self

a MaObjectSerializer
oneArgBlock
an OrderedCollection(a MaIdentitySet(size 94307) a Dictionary(size 47600))

path
a MaIdentitySet(size 94307)
parent

50581
index
nil
thisContext

MaIdentitySet(ProtoObject)>>maValueGraphNode:index:using:with:path:with:
self
a MaIdentitySet(size 94307)

anObject
a Dictionary(size 47600)
anInteger

50581
aObjectTraversalStrategy
a MaObjectGraphTraversalStrategy
aBlock

[closure] in MaObjectSerializer>>appendGraph:do:
path
an OrderedCollection(a MaIdentitySet(size 94307) a Dictionary(size 47600))

alreadyVisitedSet
a MaIdentitySet(size 171342)
stepInto

nil
obj
nil

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

hernanmd

Re: Serialization sizeWarningThreshold reached

Hi Facundo,

You may try smaller commits? A script like the following could work to
commit every 100 additions to a MagmaCollection:

>>addToCollectionAt: aDirectoryName collection: aCollection

| myMagmaSession commitIntervalCount magmaCol |

myMagmaSession := self sessionClass openLocal: aDirectoryName.
myMagmaSession connectAs: self authorName.
magmaCol := myMagmaSession root first.

commitIntervalCount := 100.
1 to: aCollection size by: commitIntervalCount do: [: startIndex |
| stopIndex |
( stopIndex := startIndex + commitIntervalCount - 1 ) <= aCollection size
ifTrue: [
myMagmaSession begin.
( aCollection copyFrom: startIndex to: stopIndex ) do: [: obj |
magmaCol add: obj ].
myMagmaSession commit ].
]

You may pre-set the sizeWarningThreshold to a bigger value too.

Cheers,

2011/3/26 Facundo Vozzi <[hidden email]>:

> Hi Chris, hi all,
>
> I'm testing my app with many objects by bulk loading it. When I do it for
> 1250 simple object transactions it works fine but when I start with the
> first of ten files with 42000 simple transactions by each I got this error
> so I "Proceed" and I got it again, and again. Finally a got Out of memory
> because Croquet.exe is about 500 mb of ram.
> All is working fine, process the file, creating all objects and adding to my
> application but the problem is when I do commit: to it.
> With this information could you guess what i'm doing wrong?
> Thanks in advance,
> FV
> p.s.: in my test i would like bulk load 42000 customers with 10 simple
> transactions by each.
>
> MaObjectSerializationWarning: Serialization sizeWarningThreshold reached.
> Proceed to double it.
>
> Debug Proceed Full Stack
>
> Stack Trace
>
> thisContextMaVariableObjectBuffer(MaObjectBuffer)>>ensureSpaceFor:using:
> selfa MaVariableObjectBuffer oid : 0 classId : 0 objectInstSize : -3
> commitNumber : MaBufferPositio...etc...storageObjecta Dictionary(size
> 47600)bpMapClass MagmaBufferPositionMaptotalSizenilrequiredSpace nil
> thisContextMaObjectSerializer>>bufferFor:storageObject:startingAt:self a
> MaObjectSerializer anObjecta Dictionary(size 47600)storageObject a
> Dictionary(size 47600)anInteger9691714buffer nil
> thisContextMaObjectSerializer>>append:self a MaObjectSerializer anObjecta
> Dictionary(size 47600)storageObject nil
> thisContext[] in MaObjectSerializer>>appendGraph:do:self a
> MaObjectSerializer oneArgBlockan OrderedCollection(a MaIdentitySet(size
> 94307) a Dictionary(size 47600)) patha MaIdentitySet(size 94307)parent
> 50581indexnil
> thisContext
> MaIdentitySet(ProtoObject)>>maValueGraphNode:index:using:with:path:with:selfa
> MaIdentitySet(size 94307) anObjecta Dictionary(size 47600)anInteger
> 50581aObjectTraversalStrategya MaObjectGraphTraversalStrategyaBlock
> [closure] in MaObjectSerializer>>appendGraph:do:pathan OrderedCollection(a
> MaIdentitySet(size 94307) a Dictionary(size 47600)) alreadyVisitedSeta
> MaIdentitySet(size 171342)stepInto nilobjnil
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
>

--
Hernán Morales
Information Technology Manager,
Institute of Veterinary Genetics.
National Scientific and Technical Research Council (CONICET).
La Plata (1900), Buenos Aires, Argentina.
Telephone: +54 (0221) 421-1799.
Internal: 422
Fax: 425-7980 or 421-1799.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma