Re: Doru would like to participate to the effort

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Doru would like to participate to the effort

Nicolas Anquetil

OK

thanks Cedric.
I will look into that when I have time which means probably only monday.


Still we have a real issue here: how to get the model of a big (10K classes) system?

compilers don't have this problem because they use incremental compilation (e.g. one java file at a time).
But in our case, we want the entire famix model in memory so what can we do?
- any tricks we could use to improve memory usage?
- may be the complete model solution is not the right one?
- or may be it is the right one and there is no good solution?

nicolas


De: "Cedric Dumoulin" <[hidden email]>
À: "Nicolas Anquetil" <[hidden email]>
Cc: "Stéphane Ducasse" <[hidden email]>, "Laval Jannik" <[hidden email]>, "Tudor Girba" <[hidden email]>
Envoyé: Vendredi 1 Juillet 2011 12:39:05
Objet: Re: Doru would like to participate to the effort


  Hi,

  About the 'uml' directory:

  This directory contains diagrams' code generated by GMF. GMF generate numerous classes.
  It should be possible to parse only a subset of 'uml', for example the oep.uml.diagram.clazz and all the required plugins.
  I join a class diagram showing all of the required plugins, from 'uml', of diagram.clazz. On its left, plugins from 'uml', on its right, plugin from 'core'. Some required plugins from 'core' are missing in the diagram.
  I suppose that the dependencies are the same for each diagram (oep.uml.diagram.[diagramnam]).

  Steph, which mailing list are you talking about ?
 
  Cedric

)
Nicolas Anquetil a écrit :
Welcome,

state of affairs:
- we have implemented Eclipse plugins in Moose
- we have an exporter to generate the information in MSE
- wednesday I implemented a new option in verveineJ that allows to not export "local references/entities"

On my computer I have source code in 7 directories:
core, developer, ocl, others, profile-tool, sysml, uml

I can parse 'core' (1383 java files), 5 others would probably not raise any problem: 'developer'=155 files, 'ocl'=6 files, 'others'=115 files, 'profile-tool'=20 files and 'sysml'=780 files.

The difficulty is with 'uml': 6269 java files.
I cannot even parse it alone ... despite incremental (separate) parsing and the 'no-locals' option, I still cannot generate data for this 'uml' directory, let alone the entire Papyrus (JDT explodes the memory).

so, I am welcoming any idea on how to solve this problem ...

nicolas

PS: the 'no-locals' option needs to be better tested. It works on a small project (~= 10 classes), but we should watch it carefully on real projects.
(Removing locals is not as simple as it may appear)


----- Mail original -----
  
De: "Stéphane Ducasse" [hidden email]
À: "Cedric Dumoulin" [hidden email]
Cc: "Nicolas Anquetil" [hidden email], "Laval Jannik" [hidden email], "Tudor Girba"
[hidden email]
Envoyé: Jeudi 30 Juin 2011 23:51:01
Objet: Doru would like to participate to the effort
Hi guys

I'm at doru place and he would like to help for the papyrus case.
What would be good is also to have the discussion via the mailing-list
so that people can follow.

Stef
    
  


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doru would like to participate to the effort

Nicolas Anquetil
update:

turn out I had some time to look at it.

Well the outofmemory exceptino is a very special one and I failed to notice that:
java.lang.OutOfMemoryError: PermGen space

In summary (e.g. see http://www.freshblurbs.com/explaining-java-lang-outofmemoryerror-permgen-space), this is a special part of the memory  that got busted.
It happens because Fame tries to save memory by reusing strings. To do so it puts the strings in this special part of the memory (PermGen)

Actually Fame's code already gives a possible solution:
        // TODO which is the better strategy?
        // http://www.codeinstructions.com/2008/09/instance-pools-with-weakhashmap.html
        // http://kohlerm.blogspot.com/2009/01/is-javalangstringintern-really-evil.html
        // http://www.codeinstructions.com/2009/01/busting-javalangstringintern-myths.html

so there is some hope.
I will look into it monday ...

nicolas



De: "Nicolas Anquetil" <[hidden email]>
À: "Moose-related development" <[hidden email]>
Envoyé: Vendredi 1 Juillet 2011 13:00:28
Objet: [Moose-dev] Re: Doru would like to participate to the effort


OK

thanks Cedric.
I will look into that when I have time which means probably only monday.


Still we have a real issue here: how to get the model of a big (10K classes) system?

compilers don't have this problem because they use incremental compilation (e.g. one java file at a time).
But in our case, we want the entire famix model in memory so what can we do?
- any tricks we could use to improve memory usage?
- may be the complete model solution is not the right one?
- or may be it is the right one and there is no good solution?

nicolas


De: "Cedric Dumoulin" <[hidden email]>
À: "Nicolas Anquetil" <[hidden email]>
Cc: "Stéphane Ducasse" <[hidden email]>, "Laval Jannik" <[hidden email]>, "Tudor Girba" <[hidden email]>
Envoyé: Vendredi 1 Juillet 2011 12:39:05
Objet: Re: Doru would like to participate to the effort


  Hi,

  About the 'uml' directory:

  This directory contains diagrams' code generated by GMF. GMF generate numerous classes.
  It should be possible to parse only a subset of 'uml', for example the oep.uml.diagram.clazz and all the required plugins.
  I join a class diagram showing all of the required plugins, from 'uml', of diagram.clazz. On its left, plugins from 'uml', on its right, plugin from 'core'. Some required plugins from 'core' are missing in the diagram.
  I suppose that the dependencies are the same for each diagram (oep.uml.diagram.[diagramnam]).

  Steph, which mailing list are you talking about ?
 
  Cedric

)
Nicolas Anquetil a écrit :
Welcome,

state of affairs:
- we have implemented Eclipse plugins in Moose
- we have an exporter to generate the information in MSE
- wednesday I implemented a new option in verveineJ that allows to not export "local references/entities"

On my computer I have source code in 7 directories:
core, developer, ocl, others, profile-tool, sysml, uml

I can parse 'core' (1383 java files), 5 others would probably not raise any problem: 'developer'=155 files, 'ocl'=6 files, 'others'=115 files, 'profile-tool'=20 files and 'sysml'=780 files.

The difficulty is with 'uml': 6269 java files.
I cannot even parse it alone ... despite incremental (separate) parsing and the 'no-locals' option, I still cannot generate data for this 'uml' directory, let alone the entire Papyrus (JDT explodes the memory).

so, I am welcoming any idea on how to solve this problem ...

nicolas

PS: the 'no-locals' option needs to be better tested. It works on a small project (~= 10 classes), but we should watch it carefully on real projects.
(Removing locals is not as simple as it may appear)


----- Mail original -----
  
De: "Stéphane Ducasse" [hidden email]
À: "Cedric Dumoulin" [hidden email]
Cc: "Nicolas Anquetil" [hidden email], "Laval Jannik" [hidden email], "Tudor Girba"
[hidden email]
Envoyé: Jeudi 30 Juin 2011 23:51:01
Objet: Doru would like to participate to the effort
Hi guys

I'm at doru place and he would like to help for the papyrus case.
What would be good is also to have the discussion via the mailing-list
so that people can follow.

Stef
    
  


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doru would like to participate to the effort

Tudor Girba-2
Hi Nicolas,

Sounds very interesting. Please keep us posted.

Cheers,
Doru


On 1 Jul 2011, at 16:26, Nicolas Anquetil wrote:

> update:
>
> turn out I had some time to look at it.
>
> Well the outofmemory exceptino is a very special one and I failed to notice that:
> java.lang.OutOfMemoryError: PermGen space
>
> In summary (e.g. see http://www.freshblurbs.com/explaining-java-lang-outofmemoryerror-permgen-space), this is a special part of the memory  that got busted.
> It happens because Fame tries to save memory by reusing strings. To do so it puts the strings in this special part of the memory (PermGen)
>
> Actually Fame's code already gives a possible solution:
>         // TODO which is the better strategy?
>         // http://www.codeinstructions.com/2008/09/instance-pools-with-weakhashmap.html
>         // http://kohlerm.blogspot.com/2009/01/is-javalangstringintern-really-evil.html
>         // http://www.codeinstructions.com/2009/01/busting-javalangstringintern-myths.html
>
> so there is some hope.
> I will look into it monday ...
>
> nicolas
>
>
> De: "Nicolas Anquetil" <[hidden email]>
> À: "Moose-related development" <[hidden email]>
> Envoyé: Vendredi 1 Juillet 2011 13:00:28
> Objet: [Moose-dev] Re: Doru would like to participate to the effort
>
>
> OK
>
> thanks Cedric.
> I will look into that when I have time which means probably only monday.
>
>
> Still we have a real issue here: how to get the model of a big (10K classes) system?
>
> compilers don't have this problem because they use incremental compilation (e.g. one java file at a time).
> But in our case, we want the entire famix model in memory so what can we do?
> - any tricks we could use to improve memory usage?
> - may be the complete model solution is not the right one?
> - or may be it is the right one and there is no good solution?
>
> nicolas
>
> De: "Cedric Dumoulin" <[hidden email]>
> À: "Nicolas Anquetil" <[hidden email]>
> Cc: "Stéphane Ducasse" <[hidden email]>, "Laval Jannik" <[hidden email]>, "Tudor Girba" <[hidden email]>
> Envoyé: Vendredi 1 Juillet 2011 12:39:05
> Objet: Re: Doru would like to participate to the effort
>
>
>   Hi,
>
>   About the 'uml' directory:
>
>   This directory contains diagrams' code generated by GMF. GMF generate numerous classes.
>   It should be possible to parse only a subset of 'uml', for example the oep.uml.diagram.clazz and all the required plugins.
>   I join a class diagram showing all of the required plugins, from 'uml', of diagram.clazz. On its left, plugins from 'uml', on its right, plugin from 'core'. Some required plugins from 'core' are missing in the diagram.
>   I suppose that the dependencies are the same for each diagram (oep.uml.diagram.[diagramnam]).
>
>   Steph, which mailing list are you talking about ?
>  
>   Cedric
>
> <moz-screenshot-13.png>)
> Nicolas Anquetil a écrit :
> Welcome,
>
> state of affairs:
> - we have implemented Eclipse plugins in Moose
> - we have an exporter to generate the information in MSE
> - wednesday I implemented a new option in verveineJ that allows to not export "local references/entities"
>
> On my computer I have source code in 7 directories:
> core, developer, ocl, others, profile-tool, sysml, uml
>
> I can parse 'core' (1383 java files), 5 others would probably not raise any problem: 'developer'=155 files, 'ocl'=6 files, 'others'=115 files, 'profile-tool'=20 files and 'sysml'=780 files.
>
> The difficulty is with 'uml': 6269 java files.
> I cannot even parse it alone ... despite incremental (separate) parsing and the 'no-locals' option, I still cannot generate data for this 'uml' directory, let alone the entire Papyrus (JDT explodes the memory).
>
> so, I am welcoming any idea on how to solve this problem ...
>
> nicolas
>
> PS: the 'no-locals' option needs to be better tested. It works on a small project (~= 10 classes), but we should watch it carefully on real projects.
> (Removing locals is not as simple as it may appear)
>
>
> ----- Mail original -----
>  
>
> De: "Stéphane Ducasse" <[hidden email]>
>
> À: "Cedric Dumoulin"
> <[hidden email]>
>
> Cc: "Nicolas Anquetil"
> <[hidden email]>, "Laval Jannik" <[hidden email]>
> , "Tudor Girba"
>
> <[hidden email]>
>
> Envoyé: Jeudi 30 Juin 2011 23:51:01
> Objet: Doru would like to participate to the effort
> Hi guys
>
> I'm at doru place and he would like to help for the papyrus case.
> What would be good is also to have the discussion via the mailing-list
> so that people can follow.
>
> Stef
>    
>
>  
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
www.tudorgirba.com

"Be rather willing to give than demanding to get."




_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev