Smalltalk › Cincom › VisualWorks

interpreting a primative failure inside of gcMalloc:

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

6 messages Options

Joerg Beekmann, DeepCove Labs (YVR)

interpreting a primative failure inside of gcMalloc:

We integrated LibTIFF and have been using it as part of a server process that generates X937 files of cheques converted to electronic form. The server was running perfectly for over a week and then be began failing when trying to allocate memory using gcMalloc (see the stack below). This code runes in a loop and is called about 4000 times per file generated. Restarting the image fixed the problem.

Looking at the primitive code it seems primMalloc:pointerKind: can fail in two ways:

PrimErrorBadArgument

PrimErrorAllocationFailed

The stack does not seem to indicate which of these situations occurred. We are guessing it is PrimErrorAllocationFailed because restarting the image fixed the problem.

Does anyone see anything wrong here? Is gcMalloc the wrong thing to do? We could use newInFixedSpace instead or we could use malloc: and call free ourselves is either of these likely to be better?

Active Process

Context Stack:

[1] CTypedefType(Object)>>error:

[2] CTypedefType(Object)>>primitiveFailed

[3] CTypedefType(CType)>>primMalloc:pointerKind:

[4] CTypedefType(CType)>>malloc:

[5] CTypedefType(CType)>>gcMalloc:

[6] LibTIFF.Interface>>writeStrips:into:

[7] LibTIFF.Interface>>save:tif:tags:

…….

Initial Context Stack Arguments:

[1] CTypedefType(Object)>>error:

Receiver: (id=1803) tdata_t

Arg1: (id=13065) an UserMessage('a primitive has failed', #errPrimitiveFailed, #dialogs)

[2] CTypedefType(Object)>>primitiveFailed

Receiver: (id=1803) tdata_t

[3] CTypedefType(CType)>>primMalloc:pointerKind:

Receiver: (id=1803) tdata_t

Arg1: (id=497696) 497696

Arg2: (id=13) 13

[4] CTypedefType(CType)>>malloc:

Receiver: (id=1803) tdata_t

Arg1: (id=124424) 124424

[5] CTypedefType(CType)>>gcMalloc:

Receiver: (id=1803) tdata_t

Arg1: (id=124424) 124424

[6] LibTIFF.Interface>>writeStrips:into:

Receiver: (id=4227) a LibTIFF.Interface

Arg1: (id=12145) Depth1Image(extent: 1642@604 depth: 1)

Arg2: (id=3493) a CCompositePointer {011FD4F8} (TIFF * )

[7] LibTIFF.Interface>>save:tif:tags:

Receiver: (id=4227) a LibTIFF.Interface

Arg1: (id=12145) Depth1Image(extent: 1642@604 depth: 1)

Arg2: (id=3493) a CCompositePointer {011FD4F8} (TIFF * )

Arg3: (id=9933) Dictionary (#TIFFTAG_ROWSPERSTRIP->604 #TIFFTAG_ORIENTATION->#ORIENTATION_TOPLEFT #TIFFTAG_IMAGELENGTH->604 #TIFFTAG_PHOTOMETRIC->0 #TIFFTAG_YRESOLUTION->200 #TIFFTAG_FILLORDER->#FILLORDER_MSB2LSB #TIFFTAG_XRESOLUTION->200 #TIFFTAG_COMPRESSION->#COMPRESSION_CCITTFAX4 #TIFFTAG_RESOLUTIONUNIT->#RESUNIT_INCH #TIFFTAG_IMAGEWIDTH->1642 #TIFFTAG_PLANARCONFIG->#PLANARCONFIG_CONTIG #TIFFTAG_SAMPLESPERPIXEL->1 #TIFFTAG_BITSPERSAMPLE->1 )

-----

Joerg Beekmann
DeepCove Labs
4th floor 595 Howe Street
Vancouver, BC, V6C 2T5
voice +1.604.689.0322
fax +1.604.689.0311
[hidden email]

CONFIDENTIALITY NOTICE
This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.

Martin McClure

Re: interpreting a primative failure inside of gcMalloc:

Joerg Beekmann wrote:
> We integrated LibTIFF and have been using it as part of a server process
> that generates X937 files of cheques converted to electronic form. The
> server was running perfectly for over a week and then be began failing
> when trying to allocate memory using gcMalloc (see the stack below).
> This code runes in a loop and is called about 4000 times per file
> generated. Restarting the image fixed the problem.

You've probably already done this, but just in case....

Have you verified that the pointer objects are being garbage collected?
If something is holding on to the pointers, the external memory will
never be freed.

Regards,

-Martin

Joerg Beekmann, DeepCove Labs (YVR)

RE: interpreting a primative failure inside of gcMalloc:

In reply to this post by Joerg Beekmann, DeepCove Labs (YVR)

Joerg Beekmann wrote:
> We integrated LibTIFF and have been using it as part of a server
process
> that generates X937 files of cheques converted to electronic form. The

> server was running perfectly for over a week and then be began failing

> when trying to allocate memory using gcMalloc (see the stack below).
> This code runes in a loop and is called about 4000 times per file
> generated. Restarting the image fixed the problem.

[hidden email] wrote:
>Have you verified that the pointer objects are being garbage collected?

>If something is holding on to the pointers, the external memory will
>never be freed.

Well we are sure that we are not keeping a reference to the memory. We
also know that running a test case that processes 250,000 images does
not show any perceivable image growth after a forced global GC.

Reinout Heeck

Re: interpreting a primative failure inside of gcMalloc:

In reply to this post by Joerg Beekmann, DeepCove Labs (YVR)

Joerg Beekmann wrote:
>
> PrimErrorAllocationFailed
>
>
> Does anyone see anything wrong here? Is gcMalloc the wrong thing to do?
> We could use newInFixedSpace instead or we could use malloc: and call
> free ourselves is either of these likely to be better?

Consider memory fragmentation, perhaps you can get around it by sequencing
#malloc:/#free in some (application dependent) way.

Reinout
-------

Boris Popov, DeepCove Labs (SNN)

RE: interpreting a primative failure inside of gcMalloc:

In reply to this post by Joerg Beekmann, DeepCove Labs (YVR)

I guess it'd be nice to see why the primitive has failed in the first place,
otherwise we're shooting in the dark as far as possible solutions go, aren't
we?

I did try changing our use of gcMalloc to use malloc+free combo, which seems
to have made no difference as far as our test environment is concerned, but
like Joerg has said, we weren't able to reproduce the failure in the test
environment with gcMalloc either. We've only seen it fail after about a week
of production use, but we may just go ahead and deploy malloc+free version
there as well.

Cheers!

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.

> -----Original Message-----
> From: Reinout Heeck [mailto:[hidden email]]
> Sent: Tuesday, February 14, 2006 12:49 PM
> To: [hidden email]
> Subject: Re: interpreting a primative failure inside of gcMalloc:
>
> Joerg Beekmann wrote:
> >
> > PrimErrorAllocationFailed
> >
> >
> > Does anyone see anything wrong here? Is gcMalloc the wrong
> thing to do?
> > We could use newInFixedSpace instead or we could use
> malloc: and call
> > free ourselves is either of these likely to be better?
>
> Consider memory fragmentation, perhaps you can get around it
> by sequencing #malloc:/#free in some (application dependent) way.
>
>
>
> Reinout
> -------
>
>

smime.p7s (4K) Download Attachment

Joerg Beekmann, DeepCove Labs (YVR)

RE: interpreting a primative failure inside of gcMalloc:

In reply to this post by Joerg Beekmann, DeepCove Labs (YVR)

-----Original Message-----
From: Mark Pirogovsky [mailto:[hidden email]]
Sent: Tuesday, February 14, 2006 1:39 PM
To: Joerg Beekmann
Subject: Re: interpreting a primative failure inside of gcMalloc:

I think it also possible that he problem is outside the VisualWorks
Could it be the case that the LibTIFF has some kind of memory leaking
problem itself ?

Humm.. like it may have a memory leak that is fragmenting the heap with
small allocations... I can't see it being anything else since the
failure in the VW memory allocation primitive. Hard to see how a bug in
libTIFF could affect it.

One thing to check would be to try to unload the DLL from the VW process

as soon as you start seeing those problems.

So this would release all memory allocated by the DLL? I don't know
enough about DLL's do they keep their own list of allocated memory?.

Joerg

My 2c.

--Mark

and then reload it again

Joerg Beekmann wrote:

>
>
> We integrated LibTIFF and have been using it as part of a server
process
> that generates X937 files of cheques converted to electronic form. The

> server was running perfectly for over a week and then be began failing

> when trying to allocate memory using gcMalloc (see the stack below).
> This code runes in a loop and is called about 4000 times per file
> generated. Restarting the image fixed the problem.
>
>
>
> Looking at the primitive code it seems primMalloc:pointerKind: can
fail
> in two ways:
>
> PrimErrorBadArgument
>
> PrimErrorAllocationFailed
>
>
>
> The stack does not seem to indicate which of these situations
occurred.
> We are guessing it is PrimErrorAllocationFailed because restarting the

> image fixed the problem.
>
>
>
> Does anyone see anything wrong here? Is gcMalloc the wrong thing to
do?

> We could use newInFixedSpace instead or we could use malloc: and call
> free ourselves is either of these likely to be better?
>
>
>
>
>
> Active Process
>
> Context Stack:
>
> [1] CTypedefType(Object)>>error:
>
> [2] CTypedefType(Object)>>primitiveFailed
>
> [3] CTypedefType(CType)>>primMalloc:pointerKind:
>
> [4] CTypedefType(CType)>>malloc:
>
> [5] CTypedefType(CType)>>gcMalloc:
>
> [6] LibTIFF.Interface>>writeStrips:into:
>
> [7] LibTIFF.Interface>>save:tif:tags:
>
> .......
>
> Initial Context Stack Arguments:
>
> [1] CTypedefType(Object)>>error:
>
> Receiver: (id=1803) tdata_t
>
> Arg1: (id=13065) an UserMessage('a primitive has failed',
> #errPrimitiveFailed, #dialogs)
>
> [2] CTypedefType(Object)>>primitiveFailed
>
> Receiver: (id=1803) tdata_t
>
> [3] CTypedefType(CType)>>primMalloc:pointerKind:
>
> Receiver: (id=1803) tdata_t
>
> Arg1: (id=497696) 497696
>
> Arg2: (id=13) 13
>
> [4] CTypedefType(CType)>>malloc:
>
> Receiver: (id=1803) tdata_t
>
> Arg1: (id=124424) 124424
>
> [5] CTypedefType(CType)>>gcMalloc:
>
> Receiver: (id=1803) tdata_t
>
> Arg1: (id=124424) 124424
>
> [6] LibTIFF.Interface>>writeStrips:into:
>
> Receiver: (id=4227) a LibTIFF.Interface
>
> Arg1: (id=12145) Depth1Image(extent: 1642@604 depth: 1)
>
> Arg2: (id=3493) a CCompositePointer {011FD4F8} (TIFF * )
>
> [7] LibTIFF.Interface>>save:tif:tags:
>
> Receiver: (id=4227) a LibTIFF.Interface
>
> Arg1: (id=12145) Depth1Image(extent: 1642@604 depth: 1)
>
> Arg2: (id=3493) a CCompositePointer {011FD4F8} (TIFF * )
>
> Arg3: (id=9933) Dictionary (#TIFFTAG_ROWSPERSTRIP->604
> #TIFFTAG_ORIENTATION->#ORIENTATION_TOPLEFT #TIFFTAG_IMAGELENGTH->604
> #TIFFTAG_PHOTOMETRIC->0 #TIFFTAG_YRESOLUTION->200
> #TIFFTAG_FILLORDER->#FILLORDER_MSB2LSB #TIFFTAG_XRESOLUTION->200
> #TIFFTAG_COMPRESSION->#COMPRESSION_CCITTFAX4
> #TIFFTAG_RESOLUTIONUNIT->#RESUNIT_INCH #TIFFTAG_IMAGEWIDTH->1642
> #TIFFTAG_PLANARCONFIG->#PLANARCONFIG_CONTIG

#TIFFTAG_SAMPLESPERPIXEL->1

> #TIFFTAG_BITSPERSAMPLE->1 )
>
>
>
>
>
>
>
>
>
>
>
> -----
>
> Joerg Beekmann
> DeepCove Labs
> 4th floor 595 Howe Street
> Vancouver , BC , V6C 2T5
> voice +1.604.689.0322
> fax +1.604.689.0311
> [hidden email] <mailto:[hidden email]>
>
>
>
>
> CONFIDENTIALITY NOTICE
> This email is intended only for the persons named in the message
> header. Unless otherwise indicated, it contains information that is
> private and confidential. If you have received it in error, please
> notify the sender and delete the entire message including any
> attachments.
>
>
>
> Thank you.
>
>
>