Smalltalk › Frameworks & Tools › Moose

Development of FAMIX AST meta-model extension

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

13 messages Options

Uko2

Development of FAMIX AST meta-model extension

Hi everyone! I'm extending FAMIX meta-model to add support for AST models. The project can be found at: http://smalltalkhub.com/#!/~YuriyTymchuk/Famix-AST

I'll be thankful for any suggestions an support :)

At the moment I've developed a hierarchy of the AST meta-model itself and you can load it from MSE files, as an example you can use this one: https://raw.github.com/Uko/metamodels-support-files/master/mse/ast.mse Please note that only middleOf:And: method has it's AST defined. To explore AST model you should select a method node and see statements as it's subnode.
My next step is automatic creation of the AST model from existing source of the method.

Here is a diagram of a current Famix-AST class hierarchy:

View this message in context: Development of FAMIX AST meta-model extension
Sent from the moose-dev mailing list archive at Nabble.com.

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Tudor Girba-2

Re: Development of FAMIX AST meta-model extension

Hi Yuriy,

Excellent topic :).

I think the first thing in this project would be to set the goal. So, what is the goal of your meta-model?

I did not look at the code yet, but by looking at the diagram, it seems to me that the meta-model is quite specific to Smalltalk. If you only want to keep it for Smalltalk, you can use the existing AST that is provided by the refactoring browser.

So, do you have the intention of handling other languages as well? If yes, then it would be good to have some basic targets. For example, here is an idea that I used in another project: Provide a nesting model for conditionals, and loops, and link Invocation and Access to the AST nodes. In this way we can queries for certain invocations that happen within an if or a for statement.

May I ask in what context are you interested in this?

Cheers,
Doru

On 28 Nov 2012, at 23:15, Yuriy Tymchuk <[hidden email]> wrote:

> Hi everyone! I'm extending FAMIX meta-model to add support for AST models. The project can be found at: http://smalltalkhub.com/#!/~YuriyTymchuk/Famix-AST
>
> I'll be thankful for any suggestions an support :)
>
> At the moment I've developed a hierarchy of the AST meta-model itself and you can load it from MSE files, as an example you can use this one: https://raw.github.com/Uko/metamodels-support-files/master/mse/ast.mse Please note that only middleOf:And: method has it's AST defined. To explore AST model you should select a method node and see statements as it's subnode.
> My next step is automatic creation of the AST model from existing source of the method.
>
> Here is a diagram of a current Famix-AST class hierarchy:
> View this message in context: Development of FAMIX AST meta-model extension
> Sent from the moose-dev mailing list archive at Nabble.com.
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
www.tudorgirba.com

"What is more important: To be happy, or to make happy?"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Uko2

Re: Development of FAMIX AST meta-model extension

Hi, Tudor,

I've completely forgot to write that this is my master's project and I'm
doing it under a supervision of Nicolas Anquetil and Anne Etien

I want to extend the AST meta-model for other languages (java should be the
next one) and I'll add associations between already existing nodes like
Invocation and my AST nodes. I've started from Smalltalk because it seams to
have an easier structure. Also I think that it will be really interesting to
implement some convertors that can transform model of one language into the
other one. In the future this can be developed into powerful tool to convert
a source between the programing languages. Also we can make some "general
imperative languages meta-model" and convert into it models of another
languages to improve the ability to analyze the model itself. For example it
should be easier to work with the "if" statement nodes when with
ifTrue:IfFalse: message expressions.
On the other hand AST meta-models can be used to automatically analyze code
and find problems like
int i = 0;
if (i != 0) …
but right now I'm thinking of creating a fully functional meta-models and
then probably about conversions.

Regards.
Yuriy

--
View this message in context: http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025676.html
Sent from the moose-dev mailing list archive at Nabble.com.

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

abergel

Re: Development of FAMIX AST meta-model extension

In reply to this post by Uko2

This is an excellent topic!
Go go go!

Alexandre

On Nov 28, 2012, at 11:15 PM, Yuriy Tymchuk <[hidden email]> wrote:

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

SergeStinckwich

Re: Development of FAMIX AST meta-model extension

In reply to this post by Uko2

Hi all,

BTW, I talk recently with Doru and another colleague about this topic ;-)

We were interested to have a group of students working on this, but apparently

you already start something and we don't want to duplicate the effort.

How can we help here ?

As already said by Alex, go go go !

Regards,

On Wed, Nov 28, 2012 at 11:15 PM, Yuriy Tymchuk <[hidden email]> wrote:

Hi everyone! I'm extending FAMIX meta-model to add support for AST models. The project can be found at: http://smalltalkhub.com/#!/~YuriyTymchuk/Famix-AST

I'll be thankful for any suggestions an support :)

At the moment I've developed a hierarchy of the AST meta-model itself and you can load it from MSE files, as an example you can use this one: https://raw.github.com/Uko/metamodels-support-files/master/mse/ast.mse Please note that only middleOf:And: method has it's AST defined. To explore AST model you should select a method node and see statements as it's subnode.
My next step is automatic creation of the AST model from existing source of the method.

Here is a diagram of a current Famix-AST class hierarchy:
View this message in context: Development of FAMIX AST meta-model extension
Sent from the moose-dev mailing list archive at Nabble.com.

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://doesnotunderstand.org/

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Nicolas Anquetil

Re: Development of FAMIX AST meta-model extension

Hi all,

thanks for the support.

The ultimate idea is to have something as complete and detailed as
possible that could represent any language.

Possible applications could be:
- rule checking (as offered by smalllint, findbug, ...)
- automatic correction of rules (again as smalllint)
- computing metrics (e.g. Cyclomatic Complexity) directly on the model
- a generic symbol resolution tool (linking an identifier definition to
all its uses)
- manipulating the source code through the metamodel (refactoring)
- generating source code
- ... (the sky is the limit)

As already said by Yuriy, he started to specify the metamodel for ST and
will now start to work on the extractor from the ST AST.
Next step will probably be to consider another language (probably Java)

If anybody wants to step in, he is welcome.
More immediate possibilities are looking at modeling another language
AST, looking at how to integrate this smoothly with moose (typically one
will not be able to generate this for an entire system, so how to
generate the AST-model on the fly, for the right entities and conect it
to the current model).

nicolas

On 29/11/12 14:15, Serge Stinckwich wrote:

> Hi all,
>
> BTW, I talk recently with Doru and another colleague about this topic ;-)
> We were interested to have a group of students working on this, but
> apparently
> you already start something and we don't want to duplicate the effort.
>
> How can we help here ?
>
> As already said by Alex, go go go !
> Regards,
>

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Uko2

Re: Development of FAMIX AST meta-model extension

Can anyone give me a hint on where the parsed Smalltalk tokens are converted
into FAMIX nodes, so I can use the ideas from there? I know that it should
be all possible with some Visitor that will traverse the parsed tree, but
it's always useful to know already existing solutions :)

--
View this message in context: http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025705.html
Sent from the moose-dev mailing list archive at Nabble.com.
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Tudor Girba-2

Re: Development of FAMIX AST meta-model extension

Hi,

You should look in the Moose-SmalltalkImporter package at the following classes:

- SmalltalkImporter: responsible for creating the Famix model out of Smalltalk code.

- SmalltalkMethodVisitor: a helper class that visits the AST of each method and creates the corresponding entities.

If you want to get an idea of how to AST looks like for a specific method, you can use the GTInspector. For example, in a Moose image:

- execute: (MooseModel>>#setRootFolder) parseTree explore

- choose the tree tab

See the result in the attached screenshot.

Cheers,

Doru

On 1 Dec 2012, at 14:22, Yuriy Tymchuk <[hidden email]> wrote:

Can anyone give me a hint on where the parsed Smalltalk tokens are converted
into FAMIX nodes, so I can use the ideas from there? I know that it should
be all possible with some Visitor that will traverse the parsed tree, but
it's always useful to know already existing solutions :)

--
View this message in context: http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025705.html
Sent from the moose-dev mailing list archive at Nabble.com.
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
www.tudorgirba.com

"Every thing should have the right to be different."

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Uko2

Re: Development of FAMIX AST meta-model extension

Hi.

I'm successfully building a FamixAST from the method source, but at the
moment my entities are only referenced by each other and not added to a
moose model. I'm having a trouble finding a right way to add all the new
entities to the moose model (all I can think of is adding them upon creation
as I can get a model from a FamixMethod instance for example). Can you help
me a bit with this?

--
View this message in context: http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025807.html
Sent from the moose-dev mailing list archive at Nabble.com.
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Tudor Girba-2

Re: Development of FAMIX AST meta-model extension

Hi,

I do not quite understand where the problem is.

To build the FamixAST, you should use some Importer, and this Importer should have the model being passed to it. Is it not like that?

Btw, where is the code?

Cheers,
Doru

On 8 Dec 2012, at 00:20, Yuriy Tymchuk <[hidden email]> wrote:

> Hi.
>
> I'm successfully building a FamixAST from the method source, but at the
> moment my entities are only referenced by each other and not added to a
> moose model. I'm having a trouble finding a right way to add all the new
> entities to the moose model (all I can think of is adding them upon creation
> as I can get a model from a FamixMethod instance for example). Can you help
> me a bit with this?
>
>
>
> --
> View this message in context: http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025807.html
> Sent from the moose-dev mailing list archive at Nabble.com.
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
www.tudorgirba.com

"The coherence of a trip is given by the clearness of the goal."

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Uko2

Re: Development of FAMIX AST meta-model extension

Hi,

Here is a Monticello repository:
http://smalltalkhub.com/#!/~YuriyTymchuk/Famix-AST/

I'm using menu item to run the AST generation on FAMIXMethod. Basically it
gets the source with `sourceText` method and parses in with
PPSmalltalkParser. The hardest part is linking an AST entities with existing
ones. I'm not using any king of importer right now, all the job is done by
the visitor.

I think that maybe it's worth to try adding the AST generation to the
general Smalltalk importer. There all the processed entities are available.

Regards,
Yuriy

P.S. excuse me please for a crappy code. A part of it was used for some kind
of debugging and a part is just like that because I don't know yet all the
easier ways smalltalk provides. I'm fixing it gradually.

--
View this message in context: http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025816.html
Sent from the moose-dev mailing list archive at Nabble.com.
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Stéphane Ducasse

Re: Development of FAMIX AST meta-model extension

In reply to this post by Uko2

Excellent! :)
Yuriy I will have a look at your AST model.

I do not know if you did that but
what would be nice is to think about the SMalltalk AST model but adding Package, Class, Global nodes.
The AST model used by the default smalltalk compiler just focuses on methods and it would be nice to have
something more complete: like compilation Unit in other languages. This is something that I always wanted for Smalltalk.

Stef

On Nov 28, 2012, at 11:15 PM, Yuriy Tymchuk wrote:

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Uko2

Re: Development of FAMIX AST meta-model extension

Hi.

I don't know if I understand correctly, but FAMIX already has a Class and Package nodes in it. Anyway I plan to extend the AST metamodel to support Java, so I think that I'll encounter details that I've not yet thought about.

P.S. I've started to modify existing code and saved changes to the same repository where AST metamodel is. I'm not quite sure if it's ok because the changes are stored on my repository and all the previous versions on the other one. I have a hard times finding some good Monticello workflows, so any suggestion is appreciated

Yuriy

12 груд. 2012 о 10:44 Stéphane Ducasse [via moose-dev] <[hidden email]> написав(ла):

Excellent! :)
Yuriy I will have a look at your AST model.

I do not know if you did that but
what would be nice is to think about the SMalltalk AST model but adding Package, Class, Global nodes.
The AST model used by the default smalltalk compiler just focuses on methods and it would be nice to have
something more complete: like compilation Unit in other languages. This is something that I always wanted for Smalltalk.

Stef

On Nov 28, 2012, at 11:15 PM, Yuriy Tymchuk wrote:

> Hi everyone! I'm extending FAMIX meta-model to add support for AST models. The project can be found at: http://smalltalkhub.com/#!/~YuriyTymchuk/Famix-AST
>
> I'll be thankful for any suggestions an support :)
>
> At the moment I've developed a hierarchy of the AST meta-model itself and you can load it from MSE files, as an example you can use this one: https://raw.github.com/Uko/metamodels-support-files/master/mse/ast.mse Please note that only middleOf:And: method has it's AST defined. To explore AST model you should select a method node and see statements as it's subnode.
> My next step is automatic creation of the AST model from existing source of the method.
>
> Here is a diagram of a current Famix-AST class hierarchy:
> View this message in context: Development of FAMIX AST meta-model extension
> Sent from the moose-dev mailing list archive at Nabble.com.
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

If you reply to this email, your message will be added to the discussion below:
http://moose-dev.97923.n3.nabble.com/Development-of-FAMIX-AST-meta-model-extension-tp4025674p4025872.html

To unsubscribe from Development of FAMIX AST meta-model extension, click here.
NAML

View this message in context: Re: Development of FAMIX AST meta-model extension
Sent from the moose-dev mailing list archive at Nabble.com.

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev