Smalltalk › Squeak › Squeak VM

Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Mariano Martinez Peck

Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

Hi folks. Sorry for the cross post this time, but I guess it is worth.

In Pharo we were trying to get a way to know the amount of memory (bytes) that an objects is occupying in RAM memory.

For such purpose, Adrian posposed the following method:

Object>>sizeInMemory
"Returns the number of bytes used by this object in memory (including its header)"

| headerSize instanceSize |
headerSize := (self class indexIfCompact > 0 ifTrue: [ 4 ] ifFalse: [ 8 ]).
instanceSize := (self class isVariable
ifFalse: [ self class instSize * Smalltalk wordSize ]
ifTrue: [ (self basicSize * (self class isBytes
ifTrue: [ 1 ] ifFalse: [ Smalltalk wordSize ])) ]).
^ headerSize + instanceSize

Do you think this is correct for all cases? Is there a way to know this but from the VM side also ? how ?
I really need it from the VM side :( but I have no idea how to do it.

Thanks

Mariano

---------- Forwarded message ----------
From: Adrian Lienhard <[hidden email]>
Date: Tue, Apr 27, 2010 at 1:19 PM
Subject: Re: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?
To: [hidden email]

Its a simple method in Object:

Object>>sizeInMemory
"Returns the number of bytes used by this object in memory (including its header)"

| headerSize instanceSize |
headerSize := (self class indexIfCompact > 0 ifTrue: [ 4 ] ifFalse: [ 8 ]).
instanceSize := (self class isVariable
ifFalse: [ self class instSize * Smalltalk wordSize ]
ifTrue: [ (self basicSize * (self class isBytes
ifTrue: [ 1 ] ifFalse: [ Smalltalk wordSize ])) ]).
^ headerSize + instanceSize

Please also note the other mail I sent to this thread.

Adrian

On Apr 27, 2010, at 13:16 , Mariano Martinez Peck wrote:

> On Tue, Apr 27, 2010 at 12:45 PM, Adrian Lienhard <[hidden email]> wrote:
>
>> I once sent some code to the mailing list (search for thread named "Size of
>> objects").
>
>
> Thanks Adrian...I couldn't find it. Can you forward it to me please? or
> just send me the code...
>
>
>> We should add this to the image. I think I named it #sizeInMemory.
>>
>>
> There are only changes to the image side ? or the vm also ?
>
> Thanks
>
> Mariano
>
>
>
>> Adrian
>>
>> On Apr 27, 2010, at 12:03 , Mariano Martinez Peck wrote:
>>
>>> Hi. I don't know if "memory occupation" is the better name. I just want
>> to
>>> know the amount of memory that an object is occupying in RAM. I mean, the
>>> amount of bytes.
>>>
>>> Is this possible ? if true, how ? I would like to do it from both
>> sides:
>>> image and VM.
>>>
>>> I checked both but I didn't find anything.
>>>
>>> Thanks in advance
>>>
>>> Mariano
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Andreas.Raab

Re: Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

On 4/28/2010 11:00 AM, Mariano Martinez Peck wrote:

> Hi folks. Sorry for the cross post this time, but I guess it is worth.
>
> In Pharo we were trying to get a way to know the amount of memory
> (bytes) that an objects is occupying in RAM memory.
>
> For such purpose, Adrian posposed the following method:
>
> Object>>sizeInMemory
> "Returns the number of bytes used by this object in memory (including
> its header)"
>
> | headerSize instanceSize |
> headerSize := (self class indexIfCompact > 0 ifTrue: [ 4 ]
> ifFalse: [ 8 ]).
> instanceSize := (self class isVariable
> ifFalse: [ self class instSize * Smalltalk wordSize ]
> ifTrue: [ (self basicSize * (self class isBytes
> ifTrue: [ 1 ] ifFalse: [ Smalltalk wordSize ])) ]).
> ^ headerSize + instanceSize
>
>
>
> Do you think this is correct for all cases?

It's close but not entirely correct. The computation needs to include
the extra header word for "large" instances (> 255 bytes). Squeak has
three possible header sizes:

1 header word - used by compact classes
2 header words - used by regular classes
3 header words - used by "large" instances

BTW, SpaceTally>>spaceForInstancesOf:withInstanceCount: has the correct
computation (except for 64 bit :-)

> Is there a way to know this
> but from the VM side also ? how ?
> I really need it from the VM side :( but I have no idea how to do it.

It's easy to do in the VM, just use this:

primitiveByteSize
"Answers the number of bytes the receiver occupies"

| nbytes|
self export: true.
oop := self popStack.
nbytes := (self sizeBitsOf: oop) + (self extraHeaderBytes: oop).
self pushInteger: nbytes.

Cheers,
- Andreas

stephane ducasse-2

Re: [squeak-dev] Re: [Vm-dev] Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

>
>
> It's close but not entirely correct. The computation needs to include the extra header word for "large" instances (> 255 bytes). Squeak has three possible header sizes:
>
> 1 header word - used by compact classes
> 2 header words - used by regular classes
> 3 header words - used by "large" instances

mariano you should have a look at the object engine chapter written by tim (available on my web page)
it explains the different headers.

David T. Lewis

Re: Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

In reply to this post by Andreas.Raab

On Wed, Apr 28, 2010 at 11:26:54AM -0700, Andreas Raab wrote:
>
> BTW, SpaceTally>>spaceForInstancesOf:withInstanceCount: has the correct
> computation (except for 64 bit :-)

Somebody should do something about that one of these days, so I put it
on Mantis: http://bugs.squeak.org/view.php?id=7518

Dave

Mariano Martinez Peck

Re: [squeak-dev] Re: [Vm-dev] Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

In reply to this post by Andreas.Raab

It's close but not entirely correct. The computation needs to include the extra header word for "large" instances (> 255 bytes). Squeak has three possible header sizes:

1 header word - used by compact classes
2 header words - used by regular classes
3 header words - used by "large" instances

BTW, SpaceTally>>spaceForInstancesOf:withInstanceCount: has the correct computation (except for 64 bit :-)

Ok, perfect. Thanks :)

Is there a way to know this
but from the VM side also ? how ?
I really need it from the VM side :( but I have no idea how to do it.

It's easy to do in the VM, just use this:

primitiveByteSize
"Answers the number of bytes the receiver occupies"

| nbytes|
self export: true.
oop := self popStack.
nbytes := (self sizeBitsOf: oop) + (self extraHeaderBytes: oop).
self pushInteger: nbytes.

I tried this and seems to work ok. I don't really need a primitive but an internal method that is called by a primitive. So, I did something like this:

internalByteSize: anOop
       "Answers the number of bytes the receiver occupies"
       | nbytes|
       self inline: true.
       nbytes := (self sizeBitsOf: anOop) + (self extraHeaderBytes: anOop).
       ^ nbytes.

and then I call it like this for example:

oop := self firstAccessibleObject.
    [oop = nil] whileFalse: [
        (self isIntegerObject: oop)
            ifFalse: [
                size := self internalByteSize: oop.
           ......]
oop := self accessibleObjectAfter: oop.
    ].

but internalByteSize ALWAYS return me the same value...do you know what can be wrong ?

Thanks in advance

Mariano

Mariano Martinez Peck

Re: [squeak-dev] Re: [Vm-dev] Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

Ok, the problem was that I was doing:

usedMemory := usedMemory + self internalByteSize: oop.

that didn't work and always answered me the same number.

Now, I had to do this:

usedMemory := usedMemory + (self internalByteSize: oop).

I don't understand why.

Thanks

Mariano

On Thu, Apr 29, 2010 at 3:22 PM, Mariano Martinez Peck <[hidden email]> wrote:

It's close but not entirely correct. The computation needs to include the extra header word for "large" instances (> 255 bytes). Squeak has three possible header sizes:

1 header word - used by compact classes
2 header words - used by regular classes
3 header words - used by "large" instances

BTW, SpaceTally>>spaceForInstancesOf:withInstanceCount: has the correct computation (except for 64 bit :-)

Ok, perfect. Thanks :)

Is there a way to know this
but from the VM side also ? how ?
I really need it from the VM side :( but I have no idea how to do it.

It's easy to do in the VM, just use this:

primitiveByteSize
"Answers the number of bytes the receiver occupies"

| nbytes|
self export: true.
oop := self popStack.
nbytes := (self sizeBitsOf: oop) + (self extraHeaderBytes: oop).
self pushInteger: nbytes.

I tried this and seems to work ok. I don't really need a primitive but an internal method that is called by a primitive. So, I did something like this:

internalByteSize: anOop

       "Answers the number of bytes the receiver occupies"
       | nbytes|
       self inline: true.
       nbytes := (self sizeBitsOf: anOop) + (self extraHeaderBytes: anOop).
       ^ nbytes.

and then I call it like this for example:

oop := self firstAccessibleObject.
    [oop = nil] whileFalse: [
        (self isIntegerObject: oop)
            ifFalse: [
                size := self internalByteSize: oop.
           ......]
oop := self accessibleObjectAfter: oop.
    ].

but internalByteSize ALWAYS return me the same value...do you know what can be wrong ?

Thanks in advance

Mariano

Andreas.Raab

Re: [squeak-dev] Re: [Vm-dev] Fwd: [Pharo-project] is it possible to know the memory occupation (bytes) of an object?

In reply to this post by Mariano Martinez Peck

On 4/29/2010 6:22 AM, Mariano Martinez Peck wrote:

> and then I call it like this for example:
>
> oop := self firstAccessibleObject.
> [oop = nil] whileFalse: [
> (self isIntegerObject: oop)
> ifFalse: [
> size := self internalByteSize: oop.
> ......]
> oop := self accessibleObjectAfter: oop.
> ].

Do you realize that this simple computes the used memory which is
information that's directly accessible via the vm parameters? I don't
recall which one but you might want to look at these to see if they
already have what you need.

Cheers,
- Andreas

Mariano Martinez Peck

Re: [Pharo-project] [squeak-dev] Re: [Vm-dev] Fwd: is it possible to know the memory occupation (bytes) of an object?

On Thu, Apr 29, 2010 at 6:00 PM, Andreas Raab <[hidden email]> wrote:

On 4/29/2010 6:22 AM, Mariano Martinez Peck wrote:

and then I call it like this for example:

oop := self firstAccessibleObject.
[oop = nil] whileFalse: [
(self isIntegerObject: oop)
ifFalse: [
size := self internalByteSize: oop.
......]
oop := self accessibleObjectAfter: oop.
].

Do you realize that this simple computes the used memory which is information that's directly accessible via the vm parameters? I don't recall which one but you might want to look at these to see if they already have what you need.

Thanks Andreas. I guess I should have explained my situation. Basically, what I was trying to do (and I did at the end) is to use the last free bit of the object header to use it to detect used and unused objects. I modified the VM so that when an object receives a message, it enables such bit. I also have primitives to mark and unmark all objects.

I am not sure what I will do then with the unused objects, but for the moment, I just wanted to get numbers. Statistics. So, for example I wanted to know how many objects were with the bit on and how many with the bit off. Also the amount of memory those objects represents.

The method is:

primitiveGetStadistics
    | oop usedCount unusedCount results usedMemory unusedMemory usedCountOop unusedCountOop usedMemoryOop unusedMemoryOop |

    usedCount := unusedCount := 0.
    usedMemory := unusedMemory := 0.

    self print: 'Start to check objects'; cr.
    oop := self firstAccessibleObject.
    [oop = nil] whileFalse: [
        (self isIntegerObject: oop)
            ifFalse: [
                (self internalIsUsed: oop)
                    ifTrue: [
                        usedCount := usedCount +1.
                        usedMemory := usedMemory + (self internalByteSize: oop). ]
                    ifFalse: [
                        unusedCount := unusedCount +1.
                        unusedMemory := unusedMemory + (self internalByteSize: oop). ].
            ].
        oop := self accessibleObjectAfter: oop.
    ].
    self print: 'Finish to check objects'; cr.

    self print: 'Push stadistics'; cr.
    self pushRemappableOop:
        (self instantiateClass: (self classArray) indexableSize: 4).
    self pushRemappableOop:
        (self positive64BitIntegerFor: usedCount).
    self pushRemappableOop:
        (self positive64BitIntegerFor: usedMemory).
    self pushRemappableOop:
        (self positive64BitIntegerFor: unusedCount).
    self pushRemappableOop:
        (self positive64BitIntegerFor: unusedMemory).

    self print: 'Pop stadistics'; cr.
    unusedMemoryOop := self popRemappableOop.
    unusedCountOop := self popRemappableOop.
    usedMemoryOop := self popRemappableOop.
    usedCountOop := self popRemappableOop.
    results := self popRemappableOop.

    self print: 'Write stadistics in array'; cr.
    self storePointer: 0 ofObject: results withValue: usedCountOop.
    self storePointer: 1 ofObject: results withValue: unusedCountOop.
    self storePointer: 2 ofObject: results withValue: usedMemoryOop.
    self storePointer: 3 ofObject: results withValue: unusedMemoryOop.

    self print: 'Push result array in stadistics'; cr.
    self pop: 1 thenPush: results.



And as you may guess,
internalIsUsed: oop
    self inline: true.
    ^((self baseHeader: oop) bitAnd: UsedObjectBit) ~= 0

So then, from the image side, I can get statistics at certain point, and get something like the attached screenshot.

Thanks.

Mariano

Cheers,
- Andreas

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Picture 6.png (23K) Download Attachment