Smalltalk › Pharo › Pharo Smalltalk Developers

Multiple syntax support in Pharo

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

10 messages Options

Clément Béra

Multiple syntax support in Pharo

Hello Pharoers,

I'd like to do something a bit crazy and I think it's possible in Pharo so I'm asking for advises here.

I would like to have packages / hierarchy of classes in Pharo that have methods written with a different syntax.

I know that syntax coloring and most refactoring tools work on AST. So is it possible, for a given class, to override #parser or #parserClass class side in a similar way that you can override #compiler and #compilerClass, and from then on, the class methods will use this parser instead of RBParser to be parse its methods source code to the AST in the compilation chain and for syntax coloring (hence with full compatibility with the class browser) ?

I am talking about another syntax that would be parsed to the same AST for now, though as the compiler can be changed too, I guess a complete separate compilation chain and AST could work fine if the AST has common APIs with RB for the refactoring browser and syntax coloring.

What do you think ?

Jan Vrany

Re: Multiple syntax support in Pharo

Hi Clement,

I have done exactly this a long time ago. I went even further,
allowing different methods being written in different languages
(so one could have a class with one method in Smalltalk, other in
Java, third in JavaScript, fourth in Ruby or even LISP)

Now, fiddling with parser/compiler was actually the easiest part.
The hard (though not "difficult") part was to make it working so
one can commit and load back the package smoothly, senders /
implementors work as expected, debugger is not confused, inspector
still works and uses the right compiler for the object, changes
are written properly and could be actually loaded back. Myriads of
little details...

And yes, as soon as "the other language" semantics differ from
Smalltalk, you expose yourself to a number of interoperability issues,
depending how much different it is. Some of these problems are easy,
some of them tricky. Most of them tricky :-) So better stick to
Smalltalk semantics...

And what is the worst :-) even if you manage to make it up and running,
then only to find out that "ordinary" programmers don't like it.
They want only one language to rule them all. How many times I have
heard "I don't like programming in this or that, I want program
everything in Smalltalk (substitute for your favorite)". So, be
prepared.

If you'd still give it a go, I'd be happy to discuss details.

My 2 cents.

Jan

On Tue, 2015-08-04 at 10:08 +0200, Clément Bera wrote:

> Hello Pharoers,
>
> I'd like to do something a bit crazy and I think it's possible in
> Pharo so I'm asking for advises here.
>
> I would like to have packages / hierarchy of classes in Pharo that
> have methods written with a different syntax.
>
> I know that syntax coloring and most refactoring tools work on AST.
> So is it possible, for a given class, to override #parser or
> #parserClass class side in a similar way that you can override
> #compiler and #compilerClass, and from then on, the class methods
> will use this parser instead of RBParser to be parse its methods
> source code to the AST in the compilation chain and for syntax
> coloring (hence with full compatibility with the class browser) ?
>
> I am talking about another syntax that would be parsed to the same
> AST for now, though as the compiler can be changed too, I guess a
> complete separate compilation chain and AST could work fine if the
> AST has common APIs with RB for the refactoring browser and syntax
> coloring.
>
> What do you think ?

Thierry Goubier

Re: Multiple syntax support in Pharo

In reply to this post by Clément Béra

Le 04/08/2015 10:08, Clément Bera a écrit :

> Hello Pharoers,
>
> I'd like to do something a bit crazy and I think it's possible in Pharo
> so I'm asking for advises here.
>
> I would like to have packages / hierarchy of classes in Pharo that have
> methods written with a different syntax.
>
> I know that syntax coloring and most refactoring tools work on AST. So
> is it possible, for a given class, to override #parser or #parserClass
> class side in a similar way that you can override #compiler and
> #compilerClass, and from then on, the class methods will use this parser
> instead of RBParser to be parse its methods source code to the AST in
> the compilation chain and for syntax coloring (hence with full
> compatibility with the class browser) ?

It depends on how the system browser is implemented, but it sounds doable.

I would be able to do that fairly easily on AltBrowser, since commands
(all menus and shortcuts, including refactorings) are delegated to the
node you are using (i.e. the class, the method, the command) in the
browser. Syntax coloring is sort of keyed to the node (i.e. a comment
node doesn't do syntax coloring the way a method node does).

> I am talking about another syntax that would be parsed to the same AST
> for now, though as the compiler can be changed too, I guess a complete
> separate compilation chain and AST could work fine if the AST has common
> APIs with RB for the refactoring browser and syntax coloring.

No, that wouldn't work for refactoring. I believe RB apply refactorings
by source code changes, and so it would introduce the old syntax in the
middle of the new.

If you use a SmaCC-based parser for the new syntax, then you would get
along the refactoring engine for that syntax and AST.

Writing a syntax coloring tool from a Parser is extremely easy,
independently of the way this parser is implemented. It can be
auto-generated; I did some experiments with Usman on that, and they were
very interesting, in particular about the speed of the resulting syntax
coloring tool.

> What do you think ?

It is a good way to "mold" the browser, in a way. Doru, what do you think?

Thierry

Stephan Eggermont-3

Re: Multiple syntax support in Pharo

In reply to this post by Clément Béra

On 04-08-15 10:08, Clément Bera wrote:
> Hello Pharoers,
>
> I'd like to do something a bit crazy and I think it's possible in Pharo
> so I'm asking for advises here.
>
> I would like to have packages / hierarchy of classes in Pharo that have
> methods written with a different syntax.

I strongly support that.
The PostgresV3 integration in Squeak uses that, and I want that for
ATDD support (Features, Scenarios, Goals, Stakeholders)

Stephan

EstebanLM

Re: Multiple syntax support in Pharo

In reply to this post by Thierry Goubier

Hi,

AFAIK, this was Lukas phd thesis: Helvetia.

http://scg.unibe.ch/research/helvetia

cheers,

Esteban

On 04 Aug 2015, at 10:48, Thierry Goubier <[hidden email]> wrote:

Le 04/08/2015 10:08, Clément Bera a écrit :
Hello Pharoers,

I'd like to do something a bit crazy and I think it's possible in Pharo
so I'm asking for advises here.

I would like to have packages / hierarchy of classes in Pharo that have
methods written with a different syntax.

I know that syntax coloring and most refactoring tools work on AST. So
is it possible, for a given class, to override #parser or #parserClass
class side in a similar way that you can override #compiler and
#compilerClass, and from then on, the class methods will use this parser
instead of RBParser to be parse its methods source code to the AST in
the compilation chain and for syntax coloring (hence with full
compatibility with the class browser) ?

It depends on how the system browser is implemented, but it sounds doable.

I would be able to do that fairly easily on AltBrowser, since commands (all menus and shortcuts, including refactorings) are delegated to the node you are using (i.e. the class, the method, the command) in the browser. Syntax coloring is sort of keyed to the node (i.e. a comment node doesn't do syntax coloring the way a method node does).

I am talking about another syntax that would be parsed to the same AST
for now, though as the compiler can be changed too, I guess a complete
separate compilation chain and AST could work fine if the AST has common
APIs with RB for the refactoring browser and syntax coloring.

No, that wouldn't work for refactoring. I believe RB apply refactorings by source code changes, and so it would introduce the old syntax in the middle of the new.

If you use a SmaCC-based parser for the new syntax, then you would get along the refactoring engine for that syntax and AST.

Writing a syntax coloring tool from a Parser is extremely easy, independently of the way this parser is implemented. It can be auto-generated; I did some experiments with Usman on that, and they were very interesting, in particular about the speed of the resulting syntax coloring tool.

What do you think ?

It is a good way to "mold" the browser, in a way. Doru, what do you think?

Thierry

Tudor Girba-2

Re: Multiple syntax support in Pharo

Indeed, Helvetia did this and little more. For example:

- it provided a way to combine multiple syntaxes within the same method

- it provided complete debugging with stepping and proper highlighting

The only reason this was not productized more thoroughly at the time is that we did not have Opal. But, as we have it now, it would be really cool to have Helvetia resurrected. Especially now that, as Thierry said, we also start to have moldable tools so the tooling part becomes easier, too.

@Clement: would you like to look at Helvetia and explore how feasible it is to resurrect it?

Cheers,

Doru

On Tue, Aug 4, 2015 at 11:09 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

AFAIK, this was Lukas phd thesis: Helvetia.

http://scg.unibe.ch/research/helvetia

cheers,
Esteban

On 04 Aug 2015, at 10:48, Thierry Goubier <[hidden email]> wrote:

Le 04/08/2015 10:08, Clément Bera a écrit :
Hello Pharoers,

I'd like to do something a bit crazy and I think it's possible in Pharo
so I'm asking for advises here.

I would like to have packages / hierarchy of classes in Pharo that have
methods written with a different syntax.

I know that syntax coloring and most refactoring tools work on AST. So
is it possible, for a given class, to override #parser or #parserClass
class side in a similar way that you can override #compiler and
#compilerClass, and from then on, the class methods will use this parser
instead of RBParser to be parse its methods source code to the AST in
the compilation chain and for syntax coloring (hence with full
compatibility with the class browser) ?

It depends on how the system browser is implemented, but it sounds doable.

I would be able to do that fairly easily on AltBrowser, since commands (all menus and shortcuts, including refactorings) are delegated to the node you are using (i.e. the class, the method, the command) in the browser. Syntax coloring is sort of keyed to the node (i.e. a comment node doesn't do syntax coloring the way a method node does).

I am talking about another syntax that would be parsed to the same AST
for now, though as the compiler can be changed too, I guess a complete
separate compilation chain and AST could work fine if the AST has common
APIs with RB for the refactoring browser and syntax coloring.

No, that wouldn't work for refactoring. I believe RB apply refactorings by source code changes, and so it would introduce the old syntax in the middle of the new.

If you use a SmaCC-based parser for the new syntax, then you would get along the refactoring engine for that syntax and AST.

Writing a syntax coloring tool from a Parser is extremely easy, independently of the way this parser is implemented. It can be auto-generated; I did some experiments with Usman on that, and they were very interesting, in particular about the speed of the resulting syntax coloring tool.

What do you think ?

It is a good way to "mold" the browser, in a way. Doru, what do you think?

Thierry

www.tudorgirba.com

"Every thing has its own flow"

SergeStinckwich

Re: Multiple syntax support in Pharo

I start to work to resurrect Helvetia some

months ago by starting with QuasiQuote:

http://smalltalkhub.com/#!/~SergeStinckwich/Helvetia/

but I was distracted by other tasks.

I'm still interested to made progress here.

I will have a look when I came back from my hollidays.

Sent from my iPhone

On 4 août 2015, at 17:01, Tudor Girba <[hidden email]> wrote:

Indeed, Helvetia did this and little more. For example:
- it provided a way to combine multiple syntaxes within the same method
- it provided complete debugging with stepping and proper highlighting

The only reason this was not productized more thoroughly at the time is that we did not have Opal. But, as we have it now, it would be really cool to have Helvetia resurrected. Especially now that, as Thierry said, we also start to have moldable tools so the tooling part becomes easier, too.

@Clement: would you like to look at Helvetia and explore how feasible it is to resurrect it?

Cheers,
Doru

On Tue, Aug 4, 2015 at 11:09 AM, Esteban Lorenzano <[hidden email]> wrote:
Hi,

AFAIK, this was Lukas phd thesis: Helvetia.

http://scg.unibe.ch/research/helvetia

cheers,
Esteban

On 04 Aug 2015, at 10:48, Thierry Goubier <[hidden email]> wrote:

Le 04/08/2015 10:08, Clément Bera a écrit :
Hello Pharoers,

I'd like to do something a bit crazy and I think it's possible in Pharo
so I'm asking for advises here.

I would like to have packages / hierarchy of classes in Pharo that have
methods written with a different syntax.

I know that syntax coloring and most refactoring tools work on AST. So
is it possible, for a given class, to override #parser or #parserClass
class side in a similar way that you can override #compiler and
#compilerClass, and from then on, the class methods will use this parser
instead of RBParser to be parse its methods source code to the AST in
the compilation chain and for syntax coloring (hence with full
compatibility with the class browser) ?

It depends on how the system browser is implemented, but it sounds doable.

I would be able to do that fairly easily on AltBrowser, since commands (all menus and shortcuts, including refactorings) are delegated to the node you are using (i.e. the class, the method, the command) in the browser. Syntax coloring is sort of keyed to the node (i.e. a comment node doesn't do syntax coloring the way a method node does).

I am talking about another syntax that would be parsed to the same AST
for now, though as the compiler can be changed too, I guess a complete
separate compilation chain and AST could work fine if the AST has common
APIs with RB for the refactoring browser and syntax coloring.

No, that wouldn't work for refactoring. I believe RB apply refactorings by source code changes, and so it would introduce the old syntax in the middle of the new.

If you use a SmaCC-based parser for the new syntax, then you would get along the refactoring engine for that syntax and AST.

Writing a syntax coloring tool from a Parser is extremely easy, independently of the way this parser is implemented. It can be auto-generated; I did some experiments with Usman on that, and they were very interesting, in particular about the speed of the resulting syntax coloring tool.

What do you think ?

It is a good way to "mold" the browser, in a way. Doru, what do you think?

Thierry

--
www.tudorgirba.com

"Every thing has its own flow"

philippeback

Re: Multiple syntax support in Pharo

In reply to this post by Stephan Eggermont-3

Not to mention the ability to edit JavaScript methods when writing Seaside apps with some syntax highlighting and no string escaping.

Same story for CSS, images etc. (I think we miss a basic picture editor in Pharo).

Phil

On Tue, Aug 4, 2015 at 10:58 AM, Stephan Eggermont <[hidden email]> wrote:

On 04-08-15 10:08, Clément Bera wrote:

Hello Pharoers,

I'd like to do something a bit crazy and I think it's possible in Pharo
so I'm asking for advises here.

I would like to have packages / hierarchy of classes in Pharo that have
methods written with a different syntax.

I strongly support that.
The PostgresV3 integration in Squeak uses that, and I want that for
ATDD support (Features, Scenarios, Goals, Stakeholders)

Stephan

Alain Rastoul-2

Re: Multiple syntax support in Pharo

In reply to this post by Clément Béra

Hi Clement

I think your idea is very interesting under several perspectives, and
not crazy at all .

Do you have a particular language or project in mind or is it a general
question ?

Apart of what has been said by others (agree with everything - and I
didn't know Helvetia, it looks great),
it could be a great opening for Pharo, very appealing to lot of developers.
IMHO it could be extremely interesting to have Javascript or Groovy
language support integrated,
Those languages are often used as base scripting systems in various
tools plus they have lot of users.
(about tools, JS in browsers, nodejs and mongo, Groovy in Gradle, JMeter
and ElasticSearch,
firsts that came in my mind).
Plus Pivotal stopped Groovy development funding few month ago, however,
most of them use java interop,
may be a problem?
That may be one of the biggest problem : not the language and the
compiler but the libraries,
base classes it relies upon and it's semantic.
And as Jan said, targeting for a particular system means a compatibility
level ,
and "myriad of little details ..." that are very importants

About semantics -and just for fun - let's talk about ruby
and javascript
https://www.destroyallsoftware.com/talks/wat
:)

Regards,

Alain

Alain Rastoul-2

Re: Multiple syntax support in Pharo

addendum:
Some times ago, I played a bit with a cool Prolog implementation in Squeak

IIRC, it had some features you are looking for: IDE integration,
compiler, runtime and semantics.
I'm not sure it is this one, it was several years ago, but googling a
bit I found
http://map.squeak.org/package/cb6c7afe-d517-470f-8857-d764ef321725
http://www.zogotounga.net/comp/squeak/prolog.htm
owner MarkusDenker, ... mmm I think I've already seen that name ? ... ;)
may be worth have a look too.

And of course there is Newspeak (way cool too), I think Eliot is
maintaining the vm (?) and may be you could have some support from him?

Regards,

Alain