verveinej out of memory

> De: "Tudor Girba" <[hidden email]>
> À: "Moose-dev Moose Dev" <[hidden email]>
> Envoyé: Vendredi 22 Avril 2011 14:51:32
> Objet: [Moose-dev] verveinej out of memory
> Hi,
>
> I am testing VerveineJ and I am getting "Exception in thread "main"
> java.lang.OutOfMemoryError: Java heap space".
>
> To reproduce:
>
> 1. download
> https://github.com/Silverpeas/Silverpeas-Core/zipball/master
>
> 2. run
> ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
>
>
> Nicolas, is there anything to do to get this working?
>
> Cheers,
> Doru
>
> --
> www.tudorgirba.com
>
> "What we can governs what we wish."
>
>
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

> Yes, this is a feature
> The "--" serves as a separator between the arguments you want to pass to the JVM (before --) and the arguments you want to pass to the parser (after --)
>
> For example to use a specific class path for the JVM (maybe to use a different FAMIX implementation that the one provided) and to specify a class path for verveineJ where to look for things used in the source code, you can do:
>
> verveinej.sh -cp <class-path-for-the-JVM> -- -cp <class-path-for-verveinej> <source directory(ies)>
>
> nicolas
>
> De: "Matthias Junker" <[hidden email]>
> À: "Moose-related development" <[hidden email]>
> Envoyé: Jeudi 28 Avril 2011 23:07:53
> Objet: [Moose-dev] Re: verveinej out of memory
>
> Hey,
> i just realized that executing the script without having the "--" in the command ignores the JOPTs argument. If you execute it like this:
>
> ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
>
> the "-Xmx2000m" is ignored.
>
> if you execute it like this it works:
>
> ./verveinej.sh -Xmx2000m -- PATH_TO_SILVERPEAS
>
> Cheers
> Matt
>
> On Tue, Apr 26, 2011 at 8:28 AM, Nicolas Anquetil <[hidden email]> wrote:
> just when I was going to say that it is working and I corrected the bug you had when parsing ArgoUML :-(
> (it takes 1 hour to parse it all :-( Something I will have to look into without taking too much memory)
>
> Memory is indeed a problem currently and unfortunately there is not much I can do, since a lot of it is used by JDT's parser.
> There are few possible solutions:
>
> - there is a limit to the max. memory the JVM can use, but on my computer iti is at least 2500m. You specified only 2000m, so there is a possibility that the extra 500m will be enough...
>
> - incremental parsing: VerveineJ generates an output.mse file. When it starts, it check if such a file already exist. If so, it loads it in memory (silently, this is not a good idea) and new entities will be added to the existing ones. so you can call it several time with various parts of the project to parse. If the classpath is correctly specified, it should be able to do the name resolution correctly. (I am not sure how much this can improve things ...)
>
> - I could start to implement options reducing the level of detail of the output: --noFileAnchor, --noInvocation, --noLocalVariable, ... ? If it is an option in this case, it would help.
>
> - finally, if all else fails, there is always Markus' theory that they will always build newer, faster, bigger, computers. In 18 months, we will have twice as much memory to run verveineJ ...
>
> nicolas
>
> PS: I will try your test case ASAP (but it might be difficult this week)
>
> ----- Mail original -----
> > De: "Tudor Girba" <[hidden email]>
> > À: "Moose-dev Moose Dev" <[hidden email]>
> > Envoyé: Vendredi 22 Avril 2011 14:51:32
> > Objet: [Moose-dev] verveinej out of memory
> > Hi,
> >
> > I am testing VerveineJ and I am getting "Exception in thread "main"
> > java.lang.OutOfMemoryError: Java heap space".
> >
> > To reproduce:
> >
> > 1. download
> > https://github.com/Silverpeas/Silverpeas-Core/zipball/master
> >
> > 2. run
> > ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
> >
> >
> > Nicolas, is there anything to do to get this working?
> >
> > Cheers,
> > Doru
> >
> > --
> > www.tudorgirba.com
> >
> > "What we can governs what we wish."
> >
> >
> >
> >
> > _______________________________________________
> > Moose-dev mailing list
> > [hidden email]
> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

> De: "Tudor Girba" <[hidden email]>
> À: "Moose-related development" <[hidden email]>
> Envoyé: Jeudi 28 Avril 2011 23:21:38
> Objet: [Moose-dev] Re: verveinej out of memory
> I just tried with the latest version of VerveineJ and not I do not get
> Out of Memory anymore, but I get the following exception:
>
> VerveineJ processing file:
> /Users/girba/Desktop/Silverpeas-Silverpeas-Core-8b2eafd/lib-core/src/test/java/com/silverpeas/scheduler/SchedulerTest.java
> Exception in thread "main" java.lang.ClassCastException:
> fr.inria.verveine.core.gen.famix.Type cannot be cast to
> fr.inria.verveine.core.gen.famix.Class
> at fr.inria.verveine.extractor.java.VerveineVisitor.visit(Unknown
> Source)
> at
> org.eclipse.jdt.core.dom.ClassInstanceCreation.accept0(ClassInstanceCreation.java:323)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
> at
> org.eclipse.jdt.core.dom.MethodInvocation.accept0(MethodInvocation.java:245)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2528)
> at
> org.eclipse.jdt.core.dom.ExpressionStatement.accept0(ExpressionStatement.java:144)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
> at org.eclipse.jdt.core.dom.Block.accept0(Block.java:136)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2528)
> at
> org.eclipse.jdt.core.dom.MethodDeclaration.accept0(MethodDeclaration.java:504)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
> at
> org.eclipse.jdt.core.dom.TypeDeclaration.accept0(TypeDeclaration.java:484)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
> at
> org.eclipse.jdt.core.dom.CompilationUnit.accept0(CompilationUnit.java:219)
> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
> at fr.inria.verveine.extractor.java.FamixRequestor.acceptAST(Unknown
> Source)
> at
> org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1016)
> at
> org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:628)
> at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:982)
> at fr.inria.verveine.extractor.java.VerveineJParser.parse(Unknown
> Source)
> at fr.inria.verveine.extractor.java.VerveineJParser.main(Unknown
> Source)
>
> Cheers,
> Doru
>
>
> On 28 Apr 2011, at 23:15, Nicolas Anquetil wrote:
>
> > Yes, this is a feature
> > The "--" serves as a separator between the arguments you want to
> > pass to the JVM (before --) and the arguments you want to pass to
> > the parser (after --)
> >
> > For example to use a specific class path for the JVM (maybe to use a
> > different FAMIX implementation that the one provided) and to specify
> > a class path for verveineJ where to look for things used in the
> > source code, you can do:
> >
> > verveinej.sh -cp <class-path-for-the-JVM> -- -cp
> > <class-path-for-verveinej> <source directory(ies)>
> >
> > nicolas
> >
> > De: "Matthias Junker" <[hidden email]>
> > À: "Moose-related development" <[hidden email]>
> > Envoyé: Jeudi 28 Avril 2011 23:07:53
> > Objet: [Moose-dev] Re: verveinej out of memory
> >
> > Hey,
> > i just realized that executing the script without having the "--" in
> > the command ignores the JOPTs argument. If you execute it like this:
> >
> > ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
> >
> > the "-Xmx2000m" is ignored.
> >
> > if you execute it like this it works:
> >
> > ./verveinej.sh -Xmx2000m -- PATH_TO_SILVERPEAS
> >
> > Cheers
> > Matt
> >
> > On Tue, Apr 26, 2011 at 8:28 AM, Nicolas Anquetil
> > <[hidden email]> wrote:
> > just when I was going to say that it is working and I corrected the
> > bug you had when parsing ArgoUML :-(
> > (it takes 1 hour to parse it all :-( Something I will have to look
> > into without taking too much memory)
> >
> > Memory is indeed a problem currently and unfortunately there is not
> > much I can do, since a lot of it is used by JDT's parser.
> > There are few possible solutions:
> >
> > - there is a limit to the max. memory the JVM can use, but on my
> > computer iti is at least 2500m. You specified only 2000m, so there
> > is a possibility that the extra 500m will be enough...
> >
> > - incremental parsing: VerveineJ generates an output.mse file. When
> > it starts, it check if such a file already exist. If so, it loads it
> > in memory (silently, this is not a good idea) and new entities will
> > be added to the existing ones. so you can call it several time with
> > various parts of the project to parse. If the classpath is correctly
> > specified, it should be able to do the name resolution correctly. (I
> > am not sure how much this can improve things ...)
> >
> > - I could start to implement options reducing the level of detail of
> > the output: --noFileAnchor, --noInvocation, --noLocalVariable, ... ?
> > If it is an option in this case, it would help.
> >
> > - finally, if all else fails, there is always Markus' theory that
> > they will always build newer, faster, bigger, computers. In 18
> > months, we will have twice as much memory to run verveineJ ...
> >
> > nicolas
> >
> > PS: I will try your test case ASAP (but it might be difficult this
> > week)
> >
> > ----- Mail original -----
> > > De: "Tudor Girba" <[hidden email]>
> > > À: "Moose-dev Moose Dev" <[hidden email]>
> > > Envoyé: Vendredi 22 Avril 2011 14:51:32
> > > Objet: [Moose-dev] verveinej out of memory
> > > Hi,
> > >
> > > I am testing VerveineJ and I am getting "Exception in thread
> > > "main"
> > > java.lang.OutOfMemoryError: Java heap space".
> > >
> > > To reproduce:
> > >
> > > 1. download
> > > https://github.com/Silverpeas/Silverpeas-Core/zipball/master
> > >
> > > 2. run
> > > ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
> > >
> > >
> > > Nicolas, is there anything to do to get this working?
> > >
> > > Cheers,
> > > Doru
> > >
> > > --
> > > www.tudorgirba.com
> > >
> > > "What we can governs what we wish."
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Moose-dev mailing list
> > > [hidden email]
> > > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
> >
> > _______________________________________________
> > Moose-dev mailing list
> > [hidden email]
> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
> >
> >
> > _______________________________________________
> > Moose-dev mailing list
> > [hidden email]
> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
> >
> > _______________________________________________
> > Moose-dev mailing list
> > [hidden email]
> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
> --
> www.tudorgirba.com
>
> "Next time you see your life passing by, say 'hi' and get to know
> her."
>
>
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

>
> latest changes seem to have corrected this ...
>
> nicolas
>
> ----- Mail original -----
>> De: "Tudor Girba" <[hidden email]>
>> À: "Moose-related development" <[hidden email]>
>> Envoyé: Jeudi 28 Avril 2011 23:21:38
>> Objet: [Moose-dev] Re: verveinej out of memory
>> I just tried with the latest version of VerveineJ and not I do not get
>> Out of Memory anymore, but I get the following exception:
>>
>> VerveineJ processing file:
>> /Users/girba/Desktop/Silverpeas-Silverpeas-Core-8b2eafd/lib-core/src/test/java/com/silverpeas/scheduler/SchedulerTest.java
>> Exception in thread "main" java.lang.ClassCastException:
>> fr.inria.verveine.core.gen.famix.Type cannot be cast to
>> fr.inria.verveine.core.gen.famix.Class
>> at fr.inria.verveine.extractor.java.VerveineVisitor.visit(Unknown
>> Source)
>> at
>> org.eclipse.jdt.core.dom.ClassInstanceCreation.accept0(ClassInstanceCreation.java:323)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
>> at
>> org.eclipse.jdt.core.dom.MethodInvocation.accept0(MethodInvocation.java:245)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2528)
>> at
>> org.eclipse.jdt.core.dom.ExpressionStatement.accept0(ExpressionStatement.java:144)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
>> at org.eclipse.jdt.core.dom.Block.accept0(Block.java:136)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2528)
>> at
>> org.eclipse.jdt.core.dom.MethodDeclaration.accept0(MethodDeclaration.java:504)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
>> at
>> org.eclipse.jdt.core.dom.TypeDeclaration.accept0(TypeDeclaration.java:484)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2551)
>> at
>> org.eclipse.jdt.core.dom.CompilationUnit.accept0(CompilationUnit.java:219)
>> at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2480)
>> at fr.inria.verveine.extractor.java.FamixRequestor.acceptAST(Unknown
>> Source)
>> at
>> org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1016)
>> at
>> org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:628)
>> at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:982)
>> at fr.inria.verveine.extractor.java.VerveineJParser.parse(Unknown
>> Source)
>> at fr.inria.verveine.extractor.java.VerveineJParser.main(Unknown
>> Source)
>>
>> Cheers,
>> Doru
>>
>>
>> On 28 Apr 2011, at 23:15, Nicolas Anquetil wrote:
>>
>>> Yes, this is a feature
>>> The "--" serves as a separator between the arguments you want to
>>> pass to the JVM (before --) and the arguments you want to pass to
>>> the parser (after --)
>>>
>>> For example to use a specific class path for the JVM (maybe to use a
>>> different FAMIX implementation that the one provided) and to specify
>>> a class path for verveineJ where to look for things used in the
>>> source code, you can do:
>>>
>>> verveinej.sh -cp <class-path-for-the-JVM> -- -cp
>>> <class-path-for-verveinej> <source directory(ies)>
>>>
>>> nicolas
>>>
>>> De: "Matthias Junker" <[hidden email]>
>>> À: "Moose-related development" <[hidden email]>
>>> Envoyé: Jeudi 28 Avril 2011 23:07:53
>>> Objet: [Moose-dev] Re: verveinej out of memory
>>>
>>> Hey,
>>> i just realized that executing the script without having the "--" in
>>> the command ignores the JOPTs argument. If you execute it like this:
>>>
>>> ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
>>>
>>> the "-Xmx2000m" is ignored.
>>>
>>> if you execute it like this it works:
>>>
>>> ./verveinej.sh -Xmx2000m -- PATH_TO_SILVERPEAS
>>>
>>> Cheers
>>> Matt
>>>
>>> On Tue, Apr 26, 2011 at 8:28 AM, Nicolas Anquetil
>>> <[hidden email]> wrote:
>>> just when I was going to say that it is working and I corrected the
>>> bug you had when parsing ArgoUML :-(
>>> (it takes 1 hour to parse it all :-( Something I will have to look
>>> into without taking too much memory)
>>>
>>> Memory is indeed a problem currently and unfortunately there is not
>>> much I can do, since a lot of it is used by JDT's parser.
>>> There are few possible solutions:
>>>
>>> - there is a limit to the max. memory the JVM can use, but on my
>>> computer iti is at least 2500m. You specified only 2000m, so there
>>> is a possibility that the extra 500m will be enough...
>>>
>>> - incremental parsing: VerveineJ generates an output.mse file. When
>>> it starts, it check if such a file already exist. If so, it loads it
>>> in memory (silently, this is not a good idea) and new entities will
>>> be added to the existing ones. so you can call it several time with
>>> various parts of the project to parse. If the classpath is correctly
>>> specified, it should be able to do the name resolution correctly. (I
>>> am not sure how much this can improve things ...)
>>>
>>> - I could start to implement options reducing the level of detail of
>>> the output: --noFileAnchor, --noInvocation, --noLocalVariable, ... ?
>>> If it is an option in this case, it would help.
>>>
>>> - finally, if all else fails, there is always Markus' theory that
>>> they will always build newer, faster, bigger, computers. In 18
>>> months, we will have twice as much memory to run verveineJ ...
>>>
>>> nicolas
>>>
>>> PS: I will try your test case ASAP (but it might be difficult this
>>> week)
>>>
>>> ----- Mail original -----
>>>> De: "Tudor Girba" <[hidden email]>
>>>> À: "Moose-dev Moose Dev" <[hidden email]>
>>>> Envoyé: Vendredi 22 Avril 2011 14:51:32
>>>> Objet: [Moose-dev] verveinej out of memory
>>>> Hi,
>>>>
>>>> I am testing VerveineJ and I am getting "Exception in thread
>>>> "main"
>>>> java.lang.OutOfMemoryError: Java heap space".
>>>>
>>>> To reproduce:
>>>>
>>>> 1. download
>>>> https://github.com/Silverpeas/Silverpeas-Core/zipball/master
>>>>
>>>> 2. run
>>>> ./verveinej.sh -Xmx2000m PATH_TO_SILVERPEAS
>>>>
>>>>
>>>> Nicolas, is there anything to do to get this working?
>>>>
>>>> Cheers,
>>>> Doru
>>>>
>>>> --
>>>> www.tudorgirba.com
>>>>
>>>> "What we can governs what we wish."
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moose-dev mailing list
>>>> [hidden email]
>>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>>
>>> _______________________________________________
>>> Moose-dev mailing list
>>> [hidden email]
>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>>
>>>
>>> _______________________________________________
>>> Moose-dev mailing list
>>> [hidden email]
>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>>
>>> _______________________________________________
>>> Moose-dev mailing list
>>> [hidden email]
>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>
>> --
>> www.tudorgirba.com
>>
>> "Next time you see your life passing by, say 'hi' and get to know
>> her."
>>
>>
>>
>>
>> _______________________________________________
>> Moose-dev mailing list
>> [hidden email]
>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev