(no subject)

I get a #dnu on RBParser>>#initializeParserWith:type:, called by
OR2Command>>requestMethodFor:

Do I need to load/fix something myself, or is this a bug?

--
Henrik Jegbjerg Hansen


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

OR2 depends on the old Refactoring Engine from 1999. I recently ported
some newer code of John Brant that avoids unnecessary reformatting. It
looks like O2 hasn't been updated yet. The refactoring tools should
work well in the default OmniBrowser.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Lukas Renggli <[hidden email]> writes:

> OR2 depends on the old Refactoring Engine from 1999. I recently ported
> some newer code of John Brant that avoids unnecessary reformatting. It
> looks like O2 hasn't been updated yet. The refactoring tools should
> work well in the default OmniBrowser.
>
> Lukas

Is formatting when refactoring removed altogether?  If so, that is great
news!

However, I get the same error with OBSystemBrowser.  (I assume this is
the "default OmniBrowser")?


--
Henrik Jegbjerg Hansen

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

I still have issues with not preserving code formatting in the latest
pharo-dev. Particularly, method renaming cause code to be reformatted.

> Lukas Renggli <[hidden email]> writes:
>
>> OR2 depends on the old Refactoring Engine from 1999. I recently ported
>> some newer code of John Brant that avoids unnecessary reformatting. It
>> looks like O2 hasn't been updated yet. The refactoring tools should
>> work well in the default OmniBrowser.
>>
>> Lukas

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

The problem is that O2 overrides methods in the system and therefor
breaks also the default system browser. I am afraid that you need to
start from a core image and only load the default browser to get a
working configuration
(http://code.google.com/p/pharo/wiki/ImageBuildScripts).

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> I still have issues with not preserving code formatting in the latest
> pharo-dev. Particularly, method renaming cause code to be reformatted.

The formatting cannot be preserved in all cases. For example, if the
renamed method has many arguments or is sent with complex arguments
(block, nested message sends, ...) the code is still reformatted. For
the cases where an automatic formatting is not avoidable, the class
RBConfigurableFormatter can be configured to produce something that is
more to your likings.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On Nov 22, 2009, at 13:28 , Lukas Renggli wrote:

Do we have a ticket for this already?

Adrian

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Thanks for the explanation. But I have just a test method without any
arguments which isn't referenced from anywhere. And this very method
loses its formatting after renaming. I've tried to rename some other
methods without parameters but the result is the same.

2009/11/22 Lukas Renggli <[hidden email]>:
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Thanks for the explanation. But I have just a test method without any
> arguments which isn't referenced from anywhere. And this very method
> loses its formatting after renaming. I've tried to rename some other
> methods without parameters but the result is the same.

Are you sure you have the latest code, e.g. AST-Core-lr.27,
Refactoring-Core-lr.77 and OB-Refactory-lr.170?

Lukas



--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Yes.

2009/11/22 Lukas Renggli <[hidden email]>:
>> Thanks for the explanation. But I have just a test method without any
>> arguments which isn't referenced from anywhere. And this very method
>> loses its formatting after renaming. I've tried to rename some other
>> methods without parameters but the result is the same.
>
> Are you sure you have the latest code, e.g. AST-Core-lr.27,
> Refactoring-Core-lr.77 and OB-Refactory-lr.170?
>
> Lukas

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Lukas,

What about the handling of comments?  The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless.  If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad.

Bill


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Lukas Renggli
Sent: Sunday, November 22, 2009 7:33 AM
To: [hidden email]
Subject: Re: [Pharo-project] (no subject)

> I still have issues with not preserving code formatting in the latest
> pharo-dev. Particularly, method renaming cause code to be reformatted.

The formatting cannot be preserved in all cases. For example, if the renamed method has many arguments or is sent with complex arguments (block, nested message sends, ...) the code is still reformatted. For the cases where an automatic formatting is not avoidable, the class RBConfigurableFormatter can be configured to produce something that is more to your likings.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

at one point gwenael changed the parser to handle comments but apparently the code could not be loaded and
it was never integrated. I imagine that now it is too divergent.

Stef

On Nov 22, 2009, at 2:44 PM, Schwab,Wilhelm K wrote:


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> What about the handling of comments?  The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless.

The new RB has a better representation of where comments are put.
Since I personally almost never put comments within the method body, I
cannot comment if the formatting of comments has improved.

For tools the Smalltalk syntax is a nightmare. A parser can only use
heuristics to assign a comment to the right statement. Writing a
formatter is even more difficult, because about everything can be
nested in everything else.

> If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad.

That's what RB does (and always did) when you enabled the preference
#promptOnRefactoring (which is the default in OB-Refactory).

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> at one point gwenael changed the parser to handle comments but apparently the code could not be loaded and
> it was never integrated. I imagine that now it is too divergent.

The code never worked for me. It broke most parts of the Refactoring Engine.

The new code is a fresh port from John's repository. There existing
port was a total mess, partly broken, partly incomplete, very
inefficient and polluted with code from other projects.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Without looking at RB code, this is how I would handle it:
1) Reify Smalltalk tokens, and create a SpaceToken (can include
various separators character and comments)
  the tokens just include a copy of corresponding source text.
2) Scanner creates a linked list of tokens including SpaceTokens
3) Parser parses the stream of tokens and build an AST that retain
pointers to the links (firstLink and lastLink)
  The advantage of storing links rather than position is that we can
add/remove links easily without recomputing positions
4) refactoring operates on AST and replaces links content,
  eventually if it add or remove links, then some special care of
intersticial SpaceToken has to happen.
5) printing refactored source code is printing the tokens verbatim
(that is just nextPutAll: their contents)

Nicolas

2009/11/22 Lukas Renggli <[hidden email]>:
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Lukas,

Interesting news about #promptOnRefactoring. However, you are the first person involved with the RB to ever mention it to me.  Previously, I was simply told that commenting methods is a waste of time and I would be happier and healthier if I would change my misguided ways.  Obviously, I take a different view of it, and all the more so having recently ported a lot of aging code to Pharo; comments saved my skin yet again.  If the RB can be made to respect them, I will eagerly use it.

Bill



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Lukas Renggli
Sent: Sunday, November 22, 2009 9:05 AM
To: [hidden email]
Subject: Re: [Pharo-project] (no subject)

> What about the handling of comments?  The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless.

The new RB has a better representation of where comments are put.
Since I personally almost never put comments within the method body, I cannot comment if the formatting of comments has improved.

For tools the Smalltalk syntax is a nightmare. A parser can only use heuristics to assign a comment to the right statement. Writing a formatter is even more difficult, because about everything can be nested in everything else.

> If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad.

That's what RB does (and always did) when you enabled the preference #promptOnRefactoring (which is the default in OB-Refactory).

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

If anyone builds this, I can easily clone one of my images and give it torture test with comments littering method bodies.  If it can keep the "data collector" code in tact, it works.  

Bill



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Nicolas Cellier
Sent: Sunday, November 22, 2009 9:56 AM
To: [hidden email]
Subject: Re: [Pharo-project] (no subject)

Without looking at RB code, this is how I would handle it:
1) Reify Smalltalk tokens, and create a SpaceToken (can include various separators character and comments)
  the tokens just include a copy of corresponding source text.
2) Scanner creates a linked list of tokens including SpaceTokens
3) Parser parses the stream of tokens and build an AST that retain pointers to the links (firstLink and lastLink)
  The advantage of storing links rather than position is that we can add/remove links easily without recomputing positions
4) refactoring operates on AST and replaces links content,
  eventually if it add or remove links, then some special care of intersticial SpaceToken has to happen.
5) printing refactored source code is printing the tokens verbatim (that is just nextPutAll: their contents)

Nicolas

2009/11/22 Lukas Renggli <[hidden email]>:
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

2009/11/22 Nicolas Cellier <[hidden email]>:
> Without looking at RB code, this is how I would handle it:
> 1) Reify Smalltalk tokens, and create a SpaceToken (can include
> various separators character and comments)
>  the tokens just include a copy of corresponding source text.

As far as I know this is what Gwenael did.

> 2) Scanner creates a linked list of tokens including SpaceTokens

This approach works very well for instance variable or class renames,
because a single identifier token is replace with a different one.

Most refactorings however replace complex node trees being generated
from code, coming from a rewrite expression or another method. This
typically involves several nested replacements, inserting/removing
parenthesis, inserting/removing statement separators,
wrapping/unwrapping into blocks, changing indention, etc. Furthermore
the code needs to decide if the space/comments between two adjacent
nodes belong to the node before or after the tokens.

> 3) Parser parses the stream of tokens and build an AST that retain
> pointers to the links (firstLink and lastLink)
>  The advantage of storing links rather than position is that we can
> add/remove links easily without recomputing positions

Yes, but at the same time the engine has to ensure that all the
transformations are performed correctly. As soon as you have to keep
the two data-structures (the token-list and the AST) in sync, this
becomes highly non-trivial for anything more complex than a simple
rename.

> 4) refactoring operates on AST and replaces links content,
>  eventually if it add or remove links, then some special care of
> intersticial SpaceToken has to happen.
> 5) printing refactored source code is printing the tokens verbatim
> (that is just nextPutAll: their contents)

The current code does a much more conservative approach, that is very
simple and ensures correctness at each step. Whenever a node is
replaced, at the same time change-objects on the original source
string are created. The new source code is then validated against the
transformed AST nodes to ensure no bugs were introduced. If there is a
problem, pretty printing kicks in.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

You might want to try and pretty print the complete image and see if
you loose any comments or if comments are moved at the wrong place.

From within a green browser press "refactor > pretty print" and get a
coffee (it takes a long time).

Note that you might want to disable pretty-printing in the refactoring
changes. The diff looks better with pretty printing enabled, but then
this is not what you get afterwards.

Lukas

2009/11/22 Schwab,Wilhelm K <[hidden email]>:


--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Interesting news about #promptOnRefactoring. However, you are the first person involved with the RB to ever mention it to me.  Previously, I was simply told that commenting methods is a waste of time and I would be happier and healthier if I would change my misguided ways.

Don Roberts and John Brant improved the refactoring engine a lot in
2002. I assume that there was some pressure from the VisualWorks and
Dolphin folks. Unfortunately Squeak and Pharo people were stuck with
the refactoring code from the last century.

> Obviously, I take a different view of it, and all the more so having recently ported a lot of aging code to Pharo; comments saved my skin yet again.  If the RB can be made to respect them, I will eagerly use it.

I think the situation is much better now, but you cannot expect magic.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project