[vw 7.1] total memory occupied by an object at run time

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[vw 7.1] total memory occupied by an object at run time

mani kartha

Hi,

Is it possible to know the total memory occupied by an object at run-time?

By total memory, I mean all the memory that is occupied by itself & the objects contained inside that object and so on, recursively.


I tried with ObjectMemory > numOopsNumBytesIn:

but seems it is not giving the value I expect.


I think so because,

I want to know the total memory occupancy of a collection of objects (instances of MyProxy class).  The subject of the proxy can be one from a hierarchy of objects, containing different amount of information .But MyProxy objects always show the size as 20 bytes as per ObjectMemory > numOopsNumBytesIn:


Do you think I need to be more clear, if not , have any suggestions/references?


Thanks in advance


Mani


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vw 7.1] total memory occupied by an object at run time

Ralph Johnson
"Total memory" is not well defined.   In general, "contained" is not well defined.   Sometimes an instance variables points to an object that is contained, and sometimes it isn't, and it is hard to tell that automatically.   For example, suppose an instance variable points to a class.   We usually don't think of a class as being contained by another object.   But suppose an instance variable contained a set of classes.  The instance of Set would probably be contained, but the classes themselves wouldn't be.

Suppose we have a payroll application with Employees, Timecards and Paychecks.   Do you want to consider the Timecards and Paychecks as being contained in the Employees?   If you say "yes", then suppose we had Projects and we knew that an Employee worked on a variety of projects and a Timecard had to list the projects that were worked on.   A Project kept track of all the work that was done on it, i.e. it could tell you the Timecards that mentioned it.   Certainly a Project isn't contained in an Employee.  So, if a Timecard is contained in an Employee then it can't be contained in a Project, yet both might have lists of Timecards.   How do we know when a collection is "contained" and when it isn't?

This is actually a very important concept.   You need to know what an object contains when you write it to disk.  You need to know what it contains to copy it, or to send it over a wire.  It is helpful to know object boundaries if you are writing parallel programs and need to lock objects.  I've seen Smalltalk frameworks that try to define this, but I've never used one of them and don't know how well they work.  I'm not up-to-date with VisualWorks, but the older versions never had a framework like this.  They were pretty much like all other object-oriented languages, leaving it up to the skill of the programmer.

Eric Evan's book "Domain Driven Design" does a great job of covering this concept in his patterns "Aggregate" and "Entity".  The book uses Java, but Eric is an old-time Smalltalk programmer, and it shows.

Ralph Johnson

On Thu, Jun 11, 2015 at 10:55 AM, mani kartha <[hidden email]> wrote:

Hi,

Is it possible to know the total memory occupied by an object at run-time?

By total memory, I mean all the memory that is occupied by itself & the objects contained inside that object and so on, recursively.


I tried with ObjectMemory > numOopsNumBytesIn:

but seems it is not giving the value I expect.


I think so because,

I want to know the total memory occupancy of a collection of objects (instances of MyProxy class).  The subject of the proxy can be one from a hierarchy of objects, containing different amount of information .But MyProxy objects always show the size as 20 bytes as per ObjectMemory > numOopsNumBytesIn:


Do you think I need to be more clear, if not , have any suggestions/references?


Thanks in advance


Mani


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Fw: [vw 7.1] total memory occupied by an object at run time

David Siegel-6


On Friday, June 12, 2015 2:23 AM, David Siegel <[hidden email]> wrote:


Hi Ralph!

I developed the "Aggregate" pattern that Eric later wrote up, and I wrote the framework that supported it, in VisualWorks and Gemstone Smalltalk.

As you realize, the notion of "ownership" is not well-defined in Smalltalk. In the general case, an object can be part of a general graph that includes the entire image.

If you use the "Aggregate" pattern, the "ownership" and, hence, the size, of an aggregate is well-defined. This doesn't directly help the original poster, since Mani's image isn't organized by Aggregate, but perhaps Mani might be able to adapt the general concept.

The "Aggregate" pattern worked well in practice, in part because we run-time checked Aggregate integrity.

The pattern works well for a large range of applications, though you'll occasionally run into cases that don't fit well. In your example, Employees, Projects and Timecards should be aggregates, and you haven't provided enough information to decide whether Paychecks should be aggregates or part of Employees, as you suggested.

Back to Mani's problem:

The aggregate rules of "ownership" are relatively intuitive and if your objects are well structured, you might be able to use those rules to decide the closure of your object.

Aggregates divide objects into 4 categories:

Entities
Collections
Primitives
Everything Else

Some Entities are Aggregate roots. For example, in Ralph's example, every Employee is the root of an aggregate. (Aside: It's not necessary to assign aggregate status by class; I could do it by instance, but it's simpler to just say which classes are Aggregates, so that's what we'll do for this discussion).

An aggregate owns everything it can reach, until it comes up against another Aggregate or an 'Everything Else' (which, would include, for example, a Class object).

The crucial rule that makes the pattern work is: an Aggregate can never point into another Aggregate. Aggregates are encapsulation boundaries. By convention, "Everything Else" (everything that isn't an Entity, collection or Primitive) is treated as the root of another Aggregate, and is not owned by the current Aggregate.

Given this rule, the "ownership" of an Aggregate is completely well-defined.

In more detail:

Primitives never point to anything else. Primitives are objects like Strings and numbers. They're owned by the aggregate that contains them.

Collections have members. A member of a collection can be any of our 4 categories: an Entity, a primitive, a collection or an "Everything Else". The collection owns its member primitives and member collections. The collection never owns an "Everything Else". If the member is an Entity, the Entity is another Aggregate, or part of the current Aggregate. The member must never be an Entity that is part of another Aggregate (as I mentioned above, we checked this at run-time).

Entities follow the same rules as collections. An Entity owns its instance variables that are primitives or collections. It never owns an "Everything Else". If an Entity's instance variable references another Entity, it owns it, unless the Entity is another Aggregate. As with collections, an Entity's instance variable must never reference an Entity that is part of another Aggregate.

Given that these rules are followed, it's straightforward to start at an Aggregate and collect its transitive closure -- the objects it owns directly, and the objects those objects own, and so on, and so on, until you reach the edge of the Aggregate.

So, Mani, if your application follows this pattern rigorously (sadly, it probably doesn't), then the "ownership" of objects is well defined.

You might be able to break your domain classes down approximately into Aggregates and contained Entities. If you can do this, then you can write a utility that starts at an aggregate instance and collects all the objects it "contains". Of course, how well this works depends on how well your application actually follows the Aggregate rules.

In our case, we wanted to be absolutely certain that these rules were followed, and had a framework that ensured it, so we had a rigorous definition for the size of an Aggregate. Trying to apply these rules post facto, to an application that wasn't built to follow these rules will not work as well.

Good luck,
-dms



On Thursday, June 11, 2015 7:04 PM, Ralph Johnson <[hidden email]> wrote:


"Total memory" is not well defined.   In general, "contained" is not well defined.   Sometimes an instance variables points to an object that is contained, and sometimes it isn't, and it is hard to tell that automatically.   For example, suppose an instance variable points to a class.   We usually don't think of a class as being contained by another object.   But suppose an instance variable contained a set of classes.  The instance of Set would probably be contained, but the classes themselves wouldn't be.

Suppose we have a payroll application with Employees, Timecards and Paychecks.   Do you want to consider the Timecards and Paychecks as being contained in the Employees?   If you say "yes", then suppose we had Projects and we knew that an Employee worked on a variety of projects and a Timecard had to list the projects that were worked on.   A Project kept track of all the work that was done on it, i.e. it could tell you the Timecards that mentioned it.   Certainly a Project isn't contained in an Employee.  So, if a Timecard is contained in an Employee then it can't be contained in a Project, yet both might have lists of Timecards.   How do we know when a collection is "contained" and when it isn't?

This is actually a very important concept.   You need to know what an object contains when you write it to disk.  You need to know what it contains to copy it, or to send it over a wire.  It is helpful to know object boundaries if you are writing parallel programs and need to lock objects.  I've seen Smalltalk frameworks that try to define this, but I've never used one of them and don't know how well they work.  I'm not up-to-date with VisualWorks, but the older versions never had a framework like this.  They were pretty much like all other object-oriented languages, leaving it up to the skill of the programmer.

Eric Evan's book "Domain Driven Design" does a great job of covering this concept in his patterns "Aggregate" and "Entity".  The book uses Java, but Eric is an old-time Smalltalk programmer, and it shows.

Ralph Johnson

On Thu, Jun 11, 2015 at 10:55 AM, mani kartha <[hidden email]> wrote:
Hi,
Is it possible to know the total memory occupied by an object at run-time?
By total memory, I mean all the memory that is occupied by itself & the objects contained inside that object and so on, recursively.

I tried with ObjectMemory > numOopsNumBytesIn:
but seems it is not giving the value I expect.

I think so because,
I want to know the total memory occupancy of a collection of objects (instances of MyProxy class).  The subject of the proxy can be one from a hierarchy of objects, containing different amount of information .But MyProxy objects always show the size as 20 bytes as per ObjectMemory > numOopsNumBytesIn:

Do you think I need to be more clear, if not , have any suggestions/references?

Thanks in advance

Mani

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vw 7.1] total memory occupied by an object at run time

David Siegel-6
In reply to this post by mani kartha
I guess I wasn't entirely clear. And, I've corrected the addressing.

Yes, an Entity is always the root of an Aggregate. An Aggregate is just a distinguished Entity. The original framework identified specific Entity classes, like Invoice, as Aggregate roots, but it could be done on an instance basis.

Entities, even non-Aggregate Entities, have unique ids, although the scope of a non-Aggregate Entity's id is the containing Aggregate. An Aggregate does not need to be a tree; it can have an internal graph structure.

In your example, InvoiceLine would either be implemented by a collection, like an array, or a non-Aggregate Entity. Either way, it would be owned by its parent Aggregate Invoice. And, either way, within the model, it has a locally unique id within its Aggregate. Within an ordinary o-o language, of course, it would have the identity semantics of the host language, like a Smalltalk oid.

An object-oriented language, or database, could be developed that supported the notion of global and local id natively. It would have the advantage of much faster garbage collection, since the global address space should be much smaller, and even large Aggregates are tiny compared to the global address space.

The point of defining "everything else" as a category in the Aggregate model was just to have an easy way to delineate the boundary of an Aggregate -- another Aggregate or "everything else". That way, it's clear that if Invoice 3456 refers to the class Invoice, that's an external reference, and Invoice 3456 doesn't contain the class Invoice.



On Friday, June 12, 2015 7:10 AM, Ralph Johnson <[hidden email]> wrote:


You sent that only to me, not to the list, so Mani won't get it.

Shouldn't an Entity always be the root of an Aggregate and an Aggregate always owns an "anything else"?  An Entity is supposed to have a unique ID.   Consider an invoice.   The invoice is made up of "lines".  An invoice has a unique ID, but a line doesn't, unless you consider "line 14 of invoice 3456" an ID.   The invoice is a typical Entity since you store it all at once, it is a unit of locking, etc.   A line of an invoice is owned by the invoice.   If you say that an Aggregate owns "everything else" then there is no problem.   Otherwise you have to label the InvoiceLine as a Entity, or make it out of collections when you would prefer to make a class out of it, or something like that.

On Fri, Jun 12, 2015 at 1:23 AM, David Siegel <[hidden email]> wrote:
Hi Ralph!

I developed the "Aggregate" pattern that Eric later wrote up, and I wrote the framework that supported it, in VisualWorks and Gemstone Smalltalk.

As you realize, the notion of "ownership" is not well-defined in Smalltalk. In the general case, an object can be part of a general graph that includes the entire image.

If you use the "Aggregate" pattern, the "ownership" and, hence, the size, of an aggregate is well-defined. This doesn't directly help the original poster, since Mani's image isn't organized by Aggregate, but perhaps Mani might be able to adapt the general concept.

The "Aggregate" pattern worked well in practice, in part because we run-time checked Aggregate integrity.

The pattern works well for a large range of applications, though you'll occasionally run into cases that don't fit well. In your example, Employees, Projects and Timecards should be aggregates, and you haven't provided enough information to decide whether Paychecks should be aggregates or part of Employees, as you suggested.

Back to Mani's problem:

The aggregate rules of "ownership" are relatively intuitive and if your objects are well structured, you might be able to use those rules to decide the closure of your object.

Aggregates divide objects into 4 categories:

Entities
Collections
Primitives
Everything Else

Some Entities are Aggregate roots. For example, in Ralph's example, every Employee is the root of an aggregate. (Aside: It's not necessary to assign aggregate status by class; I could do it by instance, but it's simpler to just say which classes are Aggregates, so that's what we'll do for this discussion).

An aggregate owns everything it can reach, until it comes up against another Aggregate or an 'Everything Else' (which, would include, for example, a Class object).

The crucial rule that makes the pattern work is: an Aggregate can never point into another Aggregate. Aggregates are encapsulation boundaries. By convention, "Everything Else" (everything that isn't an Entity, collection or Primitive) is treated as the root of another Aggregate, and is not owned by the current Aggregate.

Given this rule, the "ownership" of an Aggregate is completely well-defined.

In more detail:

Primitives never point to anything else. Primitives are objects like Strings and numbers. They're owned by the aggregate that contains them.

Collections have members. A member of a collection can be any of our 4 categories: an Entity, a primitive, a collection or an "Everything Else". The collection owns its member primitives and member collections. The collection never owns an "Everything Else". If the member is an Entity, the Entity is another Aggregate, or part of the current Aggregate. The member must never be an Entity that is part of another Aggregate (as I mentioned above, we checked this at run-time).

Entities follow the same rules as collections. An Entity owns its instance variables that are primitives or collections. It never owns an "Everything Else". If an Entity's instance variable references another Entity, it owns it, unless the Entity is another Aggregate. As with collections, an Entity's instance variable must never reference an Entity that is part of another Aggregate.

Given that these rules are followed, it's straightforward to start at an Aggregate and collect its transitive closure -- the objects it owns directly, and the objects those objects own, and so on, and so on, until you reach the edge of the Aggregate.

So, Mani, if your application follows this pattern rigorously (sadly, it probably doesn't), then the "ownership" of objects is well defined.

You might be able to break your domain classes down approximately into Aggregates and contained Entities. If you can do this, then you can write a utility that starts at an aggregate instance and collects all the objects it "contains". Of course, how well this works depends on how well your application actually follows the Aggregate rules.

In our case, we wanted to be absolutely certain that these rules were followed, and had a framework that ensured it, so we had a rigorous definition for the size of an Aggregate. Trying to apply these rules post facto, to an application that wasn't built to follow these rules will not work as well.

Good luck,
-dms



On Thursday, June 11, 2015 7:04 PM, Ralph Johnson <[hidden email]> wrote:


"Total memory" is not well defined.   In general, "contained" is not well defined.   Sometimes an instance variables points to an object that is contained, and sometimes it isn't, and it is hard to tell that automatically.   For example, suppose an instance variable points to a class.   We usually don't think of a class as being contained by another object.   But suppose an instance variable contained a set of classes.  The instance of Set would probably be contained, but the classes themselves wouldn't be.

Suppose we have a payroll application with Employees, Timecards and Paychecks.   Do you want to consider the Timecards and Paychecks as being contained in the Employees?   If you say "yes", then suppose we had Projects and we knew that an Employee worked on a variety of projects and a Timecard had to list the projects that were worked on.   A Project kept track of all the work that was done on it, i.e. it could tell you the Timecards that mentioned it.   Certainly a Project isn't contained in an Employee.  So, if a Timecard is contained in an Employee then it can't be contained in a Project, yet both might have lists of Timecards.   How do we know when a collection is "contained" and when it isn't?

This is actually a very important concept.   You need to know what an object contains when you write it to disk.  You need to know what it contains to copy it, or to send it over a wire.  It is helpful to know object boundaries if you are writing parallel programs and need to lock objects.  I've seen Smalltalk frameworks that try to define this, but I've never used one of them and don't know how well they work.  I'm not up-to-date with VisualWorks, but the older versions never had a framework like this.  They were pretty much like all other object-oriented languages, leaving it up to the skill of the programmer.

Eric Evan's book "Domain Driven Design" does a great job of covering this concept in his patterns "Aggregate" and "Entity".  The book uses Java, but Eric is an old-time Smalltalk programmer, and it shows.

Ralph Johnson

On Thu, Jun 11, 2015 at 10:55 AM, mani kartha <[hidden email]> wrote:
Hi,
Is it possible to know the total memory occupied by an object at run-time?
By total memory, I mean all the memory that is occupied by itself & the objects contained inside that object and so on, recursively.

I tried with ObjectMemory > numOopsNumBytesIn:
but seems it is not giving the value I expect.

I think so because,
I want to know the total memory occupancy of a collection of objects (instances of MyProxy class).  The subject of the proxy can be one from a hierarchy of objects, containing different amount of information .But MyProxy objects always show the size as 20 bytes as per ObjectMemory > numOopsNumBytesIn:

Do you think I need to be more clear, if not , have any suggestions/references?

Thanks in advance

Mani

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc






_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vw 7.1] total memory occupied by an object at run time

Anthony Lander-3
In reply to this post by mani kartha
You can always BOSS out the object and look at the file size. It won't be exact, but it will give you a good idea.

Hope this helps,

  -Anthony

On Thu, Jun 11, 2015 at 11:55 AM, mani kartha <[hidden email]> wrote:

Hi,

Is it possible to know the total memory occupied by an object at run-time?

By total memory, I mean all the memory that is occupied by itself & the objects contained inside that object and so on, recursively.


I tried with ObjectMemory > numOopsNumBytesIn:

but seems it is not giving the value I expect.


I think so because,

I want to know the total memory occupancy of a collection of objects (instances of MyProxy class).  The subject of the proxy can be one from a hierarchy of objects, containing different amount of information .But MyProxy objects always show the size as 20 bytes as per ObjectMemory > numOopsNumBytesIn:


Do you think I need to be more clear, if not , have any suggestions/references?


Thanks in advance


Mani


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc