Smalltalk › Pharo › Pharo Smalltalk Developers

Some more effort to make Slang and VMMaker work on Pharo [for review]

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

11 messages Options

Guillermo Polito

Some more effort to make Slang and VMMaker work on Pharo [for review]

Hi all,

Just to keep everyone informed, last week I’ve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot.

The summary of what I did is:

- we can generate, compile and run a stack vm

- we can generate, compile and run a cog vm

- for this, I’ve done a couple of patches to VMMaker

- the ones specific to Pharo has been pushed

- the ones that touch cross dialect code are still in review (I think)

To do this work

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...

- I’ve written several unit tests to ensure that future migrations are easier to do

- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

Cheers,

Guille

Inicio del mensaje reenviado:

De: Guillermo Polito <[hidden email]>
Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]
Fecha: 9 de mayo de 2019, 17:48:03 CEST
Para: Open Smalltalk Virtual Machine Development Discussion <[hidden email]>

I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST.
I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable...
Maybe we can do an iteration afterwards.

Cheers,
Guille

On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <[hidden email]> wrote:
Hi all,

On Wed, May 8, 2019 at 2:17 AM David T. Lewis <[hidden email]> wrote:

On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:

> In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of
> asString. asString is a horrible hack. printString says what we mean.

Sure. I've submitted another version to the inbox:

Name: VMMaker.oscog-GAP.2534
Author: GAP
Time: 9 May 2019, 5:12:22.055089 pm
UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1
Ancestors: VMMaker.oscog-eem.2532

Convert numeric constants to strings for concatenation using #printString.
Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.

> ==== Patch 1 ====
> > Name: VMMaker.oscog-GAP.2533
> > Author: GAP
> > Time: 7 May 2019, 10:57:16.034354 am
> > UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee
> > Ancestors: VMMaker.oscog-eem.2532
> >
> > Convert numeric constants to strings for concatenation.
> > Required for Pharo compatibility, where the arguments of concatenation are
> > not automatically coerced to strings.
> >
>
> I trust you on these. I'm not really in a position to test this out right
> now. I suggest you push the VMMakerCompatibilityForPharo6 packages to
> VMMaker as soon as you'd like. You can write VMMaker right?
>

+1

Guille, if you want commit access to the repo, you can make an account
on source.squeak.org and I'll add you the the VMMaker project. Or if
it's easier, just say which packages you are ready to move and either
I or Eliot (or any of the others with commit access) will move it to
VMMaker for you.

I note that VMMaker.oscog-GAP.2533 package is part of the main
VMMaker.oscog package. If you need to have that tested on Squeak I
can probably do that for you within the next couple of days if no
one else gets to it first.

Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo.
So, I'd say I've tried it myself already.

For the Pharo specific package, I've already uploaded it to the VMMaker repository.
I had already rights to do it :).

Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11
Author: GuillermoPolito
Time: 9 May 2019, 5:15:50.00473 pm
UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a
Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10

With these changes I can already compile and run Cog and almost all tests pass :)

I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs.
Would you like that I push them inside the VMMaker package or make a separate package?

Thanks!
Guille

--

Guille Polito
Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

Nicolas Cellier

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

Thanks Guile,

This is super important to ease contributions from Pharo teams.

Le mar. 14 mai 2019 à 16:30, Guillermo Polito <[hidden email]> a écrit :

Hi all,

Just to keep everyone informed, last week I’ve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot.
The summary of what I did is:

- we can generate, compile and run a stack vm
- we can generate, compile and run a cog vm

- for this, I’ve done a couple of patches to VMMaker
- the ones specific to Pharo has been pushed
- the ones that touch cross dialect code are still in review (I think)

To do this work

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
- I’ve written several unit tests to ensure that future migrations are easier to do
- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

Cheers,
Guille

Inicio del mensaje reenviado:

De: Guillermo Polito <[hidden email]>
Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]
Fecha: 9 de mayo de 2019, 17:48:03 CEST
Para: Open Smalltalk Virtual Machine Development Discussion <[hidden email]>

I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST.
I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable...
Maybe we can do an iteration afterwards.

Cheers,
Guille

On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <[hidden email]> wrote:
Hi all,

On Wed, May 8, 2019 at 2:17 AM David T. Lewis <[hidden email]> wrote:

On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:

> In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of
> asString. asString is a horrible hack. printString says what we mean.

Sure. I've submitted another version to the inbox:

Name: VMMaker.oscog-GAP.2534
Author: GAP
Time: 9 May 2019, 5:12:22.055089 pm
UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1
Ancestors: VMMaker.oscog-eem.2532

Convert numeric constants to strings for concatenation using #printString.
Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.

> ==== Patch 1 ====
> > Name: VMMaker.oscog-GAP.2533
> > Author: GAP
> > Time: 7 May 2019, 10:57:16.034354 am
> > UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee
> > Ancestors: VMMaker.oscog-eem.2532
> >
> > Convert numeric constants to strings for concatenation.
> > Required for Pharo compatibility, where the arguments of concatenation are
> > not automatically coerced to strings.
> >
>
> I trust you on these. I'm not really in a position to test this out right
> now. I suggest you push the VMMakerCompatibilityForPharo6 packages to
> VMMaker as soon as you'd like. You can write VMMaker right?
>

+1

Guille, if you want commit access to the repo, you can make an account
on source.squeak.org and I'll add you the the VMMaker project. Or if
it's easier, just say which packages you are ready to move and either
I or Eliot (or any of the others with commit access) will move it to
VMMaker for you.

I note that VMMaker.oscog-GAP.2533 package is part of the main
VMMaker.oscog package. If you need to have that tested on Squeak I
can probably do that for you within the next couple of days if no
one else gets to it first.

Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo.
So, I'd say I've tried it myself already.

For the Pharo specific package, I've already uploaded it to the VMMaker repository.
I had already rights to do it :).

Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11
Author: GuillermoPolito
Time: 9 May 2019, 5:15:50.00473 pm
UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a
Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10

With these changes I can already compile and run Cog and almost all tests pass :)

I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs.
Would you like that I push them inside the VMMaker package or make a separate package?

Thanks!
Guille

--

Guille Polito
Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

Guillermo Polito

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by Guillermo Polito

Ah yes, and I’ve done this so far on Pharo6.1, platform where Eliot was working on initially.

I don’t think there is much to do to move it to Pharo7, but still I’ve not explored it.

El 14 may 2019, a las 16:29, Guillermo Polito <[hidden email]> escribió:

Hi all,

Just to keep everyone informed, last week I’ve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot.
The summary of what I did is:

- we can generate, compile and run a stack vm
- we can generate, compile and run a cog vm

- for this, I’ve done a couple of patches to VMMaker
- the ones specific to Pharo has been pushed
- the ones that touch cross dialect code are still in review (I think)

To do this work

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
- I’ve written several unit tests to ensure that future migrations are easier to do
- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

Cheers,
Guille

Inicio del mensaje reenviado:

De: Guillermo Polito <[hidden email]>
Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]
Fecha: 9 de mayo de 2019, 17:48:03 CEST
Para: Open Smalltalk Virtual Machine Development Discussion <[hidden email]>

I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST.
I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable...
Maybe we can do an iteration afterwards.

Cheers,
Guille

On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <[hidden email]> wrote:
Hi all,

On Wed, May 8, 2019 at 2:17 AM David T. Lewis <[hidden email]> wrote:

On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:

> In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of
> asString. asString is a horrible hack. printString says what we mean.

Sure. I've submitted another version to the inbox:

Name: VMMaker.oscog-GAP.2534
Author: GAP
Time: 9 May 2019, 5:12:22.055089 pm
UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1
Ancestors: VMMaker.oscog-eem.2532

Convert numeric constants to strings for concatenation using #printString.
Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.

> ==== Patch 1 ====
> > Name: VMMaker.oscog-GAP.2533
> > Author: GAP
> > Time: 7 May 2019, 10:57:16.034354 am
> > UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee
> > Ancestors: VMMaker.oscog-eem.2532
> >
> > Convert numeric constants to strings for concatenation.
> > Required for Pharo compatibility, where the arguments of concatenation are
> > not automatically coerced to strings.
> >
>
> I trust you on these. I'm not really in a position to test this out right
> now. I suggest you push the VMMakerCompatibilityForPharo6 packages to
> VMMaker as soon as you'd like. You can write VMMaker right?
>

+1

Guille, if you want commit access to the repo, you can make an account
on source.squeak.org and I'll add you the the VMMaker project. Or if
it's easier, just say which packages you are ready to move and either
I or Eliot (or any of the others with commit access) will move it to
VMMaker for you.

I note that VMMaker.oscog-GAP.2533 package is part of the main
VMMaker.oscog package. If you need to have that tested on Squeak I
can probably do that for you within the next couple of days if no
one else gets to it first.

Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo.
So, I'd say I've tried it myself already.

For the Pharo specific package, I've already uploaded it to the VMMaker repository.
I had already rights to do it :).

Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11
Author: GuillermoPolito
Time: 9 May 2019, 5:15:50.00473 pm
UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a
Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10

With these changes I can already compile and run Cog and almost all tests pass :)

<image.png>

I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs.
Would you like that I push them inside the VMMaker package or make a separate package?

Thanks!
Guille

--

Guille Polito
Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

Ben Coman

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

This is really great, thx Guille. I'd love to use Pharo environment when I play with the VM and look forward to trying it.

Is it feasible to add VMMaker to the Catalog and make it a one click install?

Thanks also Elliot for the initial work.

cheers -ben

On Tue, 14 May 2019 at 23:05, Guillermo Polito <[hidden email]> wrote:

Ah yes, and I’ve done this so far on Pharo6.1, platform where Eliot was working on initially.
I don’t think there is much to do to move it to Pharo7, but still I’ve not explored it.

El 14 may 2019, a las 16:29, Guillermo Polito <[hidden email]> escribió:

Hi all,

Just to keep everyone informed, last week I’ve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot.
The summary of what I did is:

- we can generate, compile and run a stack vm
- we can generate, compile and run a cog vm

- for this, I’ve done a couple of patches to VMMaker
- the ones specific to Pharo has been pushed
- the ones that touch cross dialect code are still in review (I think)

To do this work

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
- I’ve written several unit tests to ensure that future migrations are easier to do
- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

Cheers,
Guille

Inicio del mensaje reenviado:

De: Guillermo Polito <[hidden email]>
Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]
Fecha: 9 de mayo de 2019, 17:48:03 CEST
Para: Open Smalltalk Virtual Machine Development Discussion <[hidden email]>

I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST.
I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable...
Maybe we can do an iteration afterwards.

Cheers,
Guille

On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <[hidden email]> wrote:
Hi all,

On Wed, May 8, 2019 at 2:17 AM David T. Lewis <[hidden email]> wrote:

On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:

> In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of
> asString. asString is a horrible hack. printString says what we mean.

Sure. I've submitted another version to the inbox:

Name: VMMaker.oscog-GAP.2534
Author: GAP
Time: 9 May 2019, 5:12:22.055089 pm
UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1
Ancestors: VMMaker.oscog-eem.2532

Convert numeric constants to strings for concatenation using #printString.
Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.

> ==== Patch 1 ====
> > Name: VMMaker.oscog-GAP.2533
> > Author: GAP
> > Time: 7 May 2019, 10:57:16.034354 am
> > UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee
> > Ancestors: VMMaker.oscog-eem.2532
> >
> > Convert numeric constants to strings for concatenation.
> > Required for Pharo compatibility, where the arguments of concatenation are
> > not automatically coerced to strings.
> >
>
> I trust you on these. I'm not really in a position to test this out right
> now. I suggest you push the VMMakerCompatibilityForPharo6 packages to
> VMMaker as soon as you'd like. You can write VMMaker right?
>

+1

Guille, if you want commit access to the repo, you can make an account
on source.squeak.org and I'll add you the the VMMaker project. Or if
it's easier, just say which packages you are ready to move and either
I or Eliot (or any of the others with commit access) will move it to
VMMaker for you.

I note that VMMaker.oscog-GAP.2533 package is part of the main
VMMaker.oscog package. If you need to have that tested on Squeak I
can probably do that for you within the next couple of days if no
one else gets to it first.

Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo.
So, I'd say I've tried it myself already.

For the Pharo specific package, I've already uploaded it to the VMMaker repository.
I had already rights to do it :).

Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11
Author: GuillermoPolito
Time: 9 May 2019, 5:15:50.00473 pm
UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a
Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10

With these changes I can already compile and run Cog and almost all tests pass :)

<image.png>

I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs.
Would you like that I push them inside the VMMaker package or make a separate package?

Thanks!
Guille

--

Guille Polito
Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

ducasse

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by Guillermo Polito

This is super nice and this was a couple of days well invested. I thought that it would be a lot more challenging

so you prove it by doing it fast and this is super cool. In addition this is excellent that you added tests while doing it.

I really want to have a tutorial on extending the VM with Pharo.

Stef

Hi all,

Just to keep everyone informed, last week I’ve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot.
The summary of what I did is:

- we can generate, compile and run a stack vm
- we can generate, compile and run a cog vm

- for this, I’ve done a couple of patches to VMMaker
- the ones specific to Pharo has been pushed
- the ones that touch cross dialect code are still in review (I think)

To do this work

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
- I’ve written several unit tests to ensure that future migrations are easier to do
- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

Cheers,
Guille

Inicio del mensaje reenviado:

De: Guillermo Polito <[hidden email]>
Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]
Fecha: 9 de mayo de 2019, 17:48:03 CEST
Para: Open Smalltalk Virtual Machine Development Discussion <[hidden email]>

I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST.
I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable...
Maybe we can do an iteration afterwards.

Cheers,
Guille

On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <[hidden email]> wrote:
Hi all,

On Wed, May 8, 2019 at 2:17 AM David T. Lewis <[hidden email]> wrote:

On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:

> In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of
> asString. asString is a horrible hack. printString says what we mean.

Sure. I've submitted another version to the inbox:

Name: VMMaker.oscog-GAP.2534
Author: GAP
Time: 9 May 2019, 5:12:22.055089 pm
UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1
Ancestors: VMMaker.oscog-eem.2532

Convert numeric constants to strings for concatenation using #printString.
Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.

> ==== Patch 1 ====
> > Name: VMMaker.oscog-GAP.2533
> > Author: GAP
> > Time: 7 May 2019, 10:57:16.034354 am
> > UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee
> > Ancestors: VMMaker.oscog-eem.2532
> >
> > Convert numeric constants to strings for concatenation.
> > Required for Pharo compatibility, where the arguments of concatenation are
> > not automatically coerced to strings.
> >
>
> I trust you on these. I'm not really in a position to test this out right
> now. I suggest you push the VMMakerCompatibilityForPharo6 packages to
> VMMaker as soon as you'd like. You can write VMMaker right?
>

+1

Guille, if you want commit access to the repo, you can make an account
on source.squeak.org and I'll add you the the VMMaker project. Or if
it's easier, just say which packages you are ready to move and either
I or Eliot (or any of the others with commit access) will move it to
VMMaker for you.

I note that VMMaker.oscog-GAP.2533 package is part of the main
VMMaker.oscog package. If you need to have that tested on Squeak I
can probably do that for you within the next couple of days if no
one else gets to it first.

Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo.
So, I'd say I've tried it myself already.

For the Pharo specific package, I've already uploaded it to the VMMaker repository.
I had already rights to do it :).

Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11
Author: GuillermoPolito
Time: 9 May 2019, 5:15:50.00473 pm
UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a
Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10

With these changes I can already compile and run Cog and almost all tests pass :)

<image.png>

I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs.
Would you like that I push them inside the VMMaker package or make a separate package?

Thanks!
Guille

--

Guille Polito
Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

Eliot Miranda-2

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by Guillermo Polito

Hi Guille,

On Tue, May 14, 2019 at 7:30 AM Guillermo Polito <[hidden email]> wrote:

Hi all,

Just to keep everyone informed, last week I’ve spent a couple of days making slang work on Pharo again, based on the initial effort of Eliot.
The summary of what I did is:

- we can generate, compile and run a stack vm
- we can generate, compile and run a cog vm

Great news!

- for this, I’ve done a couple of patches to VMMaker
- the ones specific to Pharo has been pushed
- the ones that touch cross dialect code are still in review (I think)

To do this work

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...

Can you show a little of th differences in output between Squeak compiler derived sources and RB parser derived sources? I'm not going to demand changes or fixes or that they be exactly the same. But I am curious to know what kind of changes there are.

- I’ve written several unit tests to ensure that future migrations are easier to do
- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

This is great1 Thank you. Perhaps we can move some of the source generation tests into the core. It would be good for there to be tests that apply to the Squeak compiler side too. I never took the time to run tests (I look at the generated source every time I make a change Slang), and writing a really good set of tests always seemed like a lot of work. But I might be tempted to add to the ones you've written.

Cheers,
Guille

Inicio del mensaje reenviado:

De: Guillermo Polito <[hidden email]>
Asunto: Re: [Vm-dev] Some more effort to make Slang and VMMaker work on Pharo [for review]
Fecha: 9 de mayo de 2019, 17:48:03 CEST
Para: Open Smalltalk Virtual Machine Development Discussion <[hidden email]>

I've decided for the moment to put the tests in the PharoCompatibility package, since they are too close of Pharo's AST.
I've tried to make them work in squeak but I've found too many incompatibilities in the AST and some in the collections that would make the tests too verbose to make them reusable...
Maybe we can do an iteration afterwards.

Cheers,
Guille

On Thu, May 9, 2019 at 5:21 PM Guillermo Polito <[hidden email]> wrote:
Hi all,

On Wed, May 8, 2019 at 2:17 AM David T. Lewis <[hidden email]> wrote:

On Tue, May 07, 2019 at 03:36:32PM -0700, Eliot Miranda wrote:

> In VMMaker.oscog-GAP.2533 I would prefer to see printString instead of
> asString. asString is a horrible hack. printString says what we mean.

Sure. I've submitted another version to the inbox:

Name: VMMaker.oscog-GAP.2534
Author: GAP
Time: 9 May 2019, 5:12:22.055089 pm
UUID: cd6aae16-6f20-4ddb-9328-64b19b413cb1
Ancestors: VMMaker.oscog-eem.2532

Convert numeric constants to strings for concatenation using #printString.
Required for Pharo compatibility, where the arguments of concatenation are not automatically coerced to strings.

> ==== Patch 1 ====
> > Name: VMMaker.oscog-GAP.2533
> > Author: GAP
> > Time: 7 May 2019, 10:57:16.034354 am
> > UUID: 3209319f-60f3-4586-8d03-cfac3c9fddee
> > Ancestors: VMMaker.oscog-eem.2532
> >
> > Convert numeric constants to strings for concatenation.
> > Required for Pharo compatibility, where the arguments of concatenation are
> > not automatically coerced to strings.
> >
>
> I trust you on these. I'm not really in a position to test this out right
> now. I suggest you push the VMMakerCompatibilityForPharo6 packages to
> VMMaker as soon as you'd like. You can write VMMaker right?
>

+1

Guille, if you want commit access to the repo, you can make an account
on source.squeak.org and I'll add you the the VMMaker project. Or if
it's easier, just say which packages you are ready to move and either
I or Eliot (or any of the others with commit access) will move it to
VMMaker for you.

I note that VMMaker.oscog-GAP.2533 package is part of the main
VMMaker.oscog package. If you need to have that tested on Squeak I
can probably do that for you within the next couple of days if no
one else gets to it first.

Well, the change I did in that package was fairly small, and I'm using squeak to generate also some parts of the VM to compare to what I generate from Pharo.
So, I'd say I've tried it myself already.

For the Pharo specific package, I've already uploaded it to the VMMaker repository.
I had already rights to do it :).

Name: VMMakerCompatibilityForPharo6-GuillermoPolito.11
Author: GuillermoPolito
Time: 9 May 2019, 5:15:50.00473 pm
UUID: e1a685a9-d744-0d00-b3c1-ad600f1a7e9a
Ancestors: VMMakerCompatibilityForPharo6-GuillermoPolito.10

With these changes I can already compile and run Cog and almost all tests pass :)

I have also a couple dozen tests about AST translation and C generation, mainly about those parts that are covered differently between Pharo's and Squeak's ASTs.
Would you like that I push them inside the VMMaker package or make a separate package?

Thanks!
Guille

--

Guille Polito
Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

_,,,^..^,,,_

best, Eliot

K K Subbu

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by Guillermo Polito

On 14/05/19 7:59 PM, Guillermo Polito wrote:
>
> To do this work
>
> - I had to review the AST-to-AST transformation, checking the output,
> comparing it to what squeak does and so on...

Guillermo,

This is a superb effort! Thanks. Are you planning to write a paper on
this part of your work? A blog? It can open new vistas for research
students in Pharo.

Regards .. Subbu

ducasse

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

Hi Subbu

>> To do this work
>> - I had to review the AST-to-AST transformation, checking the output,
>> comparing it to what squeak does and so on...
>
> Guillermo,
>
> This is a superb effort! Thanks. Are you planning to write a paper on this part of your work? A blog? It can open new vistas for research students in Pharo.

We will see if we can produce a tutorial in how to extend the VM. Igor did it in 2011 and we will redo it. Life is a cycle.

Then we would love to see if we can have a VMMaker repackaged (it should not be difficult) with less code so that we can do lectures on the Pharo VM
and exercises for students with it. It is not really pedagogical to tell them to jump over many not used classes.
And more important since we want to attract smart guys they will not take us seriously if we present a system that is monolithic
- we got several times students asking why VMMaker could not be cut into different packages and well they are right.

What I was thinking is that we should have soon a travis to build it automatically with Pharo.
If you are interested to help please join the effort.
What guille showed us is that it was not difficult to do it in fact.

Stef

>
> Regards .. Subbu
>

Guillermo Polito

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by Eliot Miranda-2

Hi Eliot,

El 15 may 2019, a las 3:20, Eliot Miranda <[hidden email]> escribió:

Hi Guille,

On Tue, May 14, 2019 at 7:30 AM Guillermo Polito <[hidden email]> wrote:

- I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...

Can you show a little of th differences in output between Squeak compiler derived sources and RB parser derived sources? I'm not going to demand changes or fixes or that they be exactly the same. But I am curious to know what kind of changes there are.

From what I have now, I can see three main kind of differences (I actually compared with source code I generated myself from squeak and the existing sources in the git repo).

I did not work on them because the VM seemed to work, so I left them for later:

1) some comments are placed in different places in the output source code. This is entirely cosmetic, but it brings unnecessary noise to C source diffs.

2) labels in the gnu’ed code are numbered differently. I did not spot any “bug” in that yet, I did not dig enough into the gnuification to see why the difference, and I’m not entirely sure that this difference is caused by that either :)…

Anyhow, this is mostly annoying because it also brings lots of noise to diffs.

3) some for loops inline the condition while they probably shouldn't. For example:

0 to: (self numSlotsOf: array1) - 1 do: [ … ]

gets in my output translated to

for (i = 0; i < (numSlotsOf(array1)); i += 1) { ...

while in the version generated from squeak

for (i = 0, iLimiT = ((numSlotsOf(array1)) - 1); i <= iLimiT; i += 1) { …

This seems the most “dangerous" difference I have so far specially if the limit condition has some side effect.

We will probably agree in that this third point should be fixed ^^.

- I’ve written several unit tests to ensure that future migrations are easier to do
- I’ve introduced several compatibility classes/methods related to PackageInfo, Time and so on…

This is great1 Thank you. Perhaps we can move some of the source generation tests into the core. It would be good for there to be tests that apply to the Squeak compiler side too. I never took the time to run tests (I look at the generated source every time I make a change Slang), and writing a really good set of tests always seemed like a lot of work. But I might be tempted to add to the ones you've written.

Maybe in the next weeks I can try to massage the tests to make them less dependent on the source (wether squeak's or pharo’s) AST.

The thing is that Pharo’s AST is closer to the source code, while Squeak’s has already some pre-cooked code transformations (like #ifNil: & co), and my tests assume a specific AST structure to traverse the tree and compare…

Cheers,

Guille

Guillermo Polito

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by ducasse

Hi Subbu,

I do not believe this deserves a paper :P. At most a blogpost. What I can do quickly here is to draft the “methodology” I followed by instinct, but that can be reproduced by anyone.

1) generate sources from pharo
2) try to compile (and run)
3) in case of failure, make a diff with the latest version of the same file in the git repository of opensmalltalk vm
4) spot an interesting difference (most common cases were related to loops and conditions, missing or extra inlinings) => get ONE method with the problem
4.1) Check 1: verify that the generating that method from squeak does **not** have the problem
4.2) Check 2: verify that the generating that method from pharo **does have** the problem
For these checks I used some of the already given VMMaker scripts
5) generate a small artificial but representative case for testing. For example:

methodWithIfNil
self something
ifNil: [ 1 ]
ifNotNil: [ 2 ]

Then I have two level of tests: AST-to-AST transformation, and generation tests.

testIfNilIfNotNilBecomesIfTrueIfFalse

| translation thisAST |
thisAST := (self class >> #methodWithIfNil) ast.
translation := thisAST asTranslationMethodOfClass: TMethod.

self assert: translation statements first selector equals: #ifTrue:ifFalse:

testSimpleIfNilAssignment

| translation thisAST codeGenerator result |
thisAST := (self class >> #methodWithIfNil) ast.
translation := thisAST asTranslationMethodOfClass: TMethod.
codeGenerator := CCodeGeneratorGlobalStructure new.
codeGenerator generateDeadCode: false.
codeGenerator addMethod: translation.
codeGenerator doInlining: true.

result := String streamContents: [ :stream |
translation parseTree statements first emitCCodeOn: stream level: 0 generator: codeGenerator.
].

self assert: result equals: 'if ((something()) == null) {
}
else {
}’

6) Fix it

* Most of the “bugs” were related to making the different #asTranslatorNodeIn: methods in the RB AST compliant to what it was doing in squeak.
* Sometimes I ended up doing parallel stepping in Pharo and Squeak to spot where the translation diverged
* Something important that I’ve learnt in the process is that Squeak’s AST contain already some transformations/expansions like

what you see/write: self x ifNil: [ … ]
what you actually have: self x == nil ifTrue: [ … ]

BUT, the most confusing point is that even if the AST internally is structured as in the version in the right, the printString of the AST in the inspector showed the left version.

Nothing new under the sun ^^.

Guille

> El 15 may 2019, a las 9:34, ducasse <[hidden email]> escribió:
>
> Hi Subbu
>
>>> To do this work
>>> - I had to review the AST-to-AST transformation, checking the output,
>>> comparing it to what squeak does and so on...
>>
>> Guillermo,
>>
>> This is a superb effort! Thanks. Are you planning to write a paper on this part of your work? A blog? It can open new vistas for research students in Pharo.
>
> We will see if we can produce a tutorial in how to extend the VM. Igor did it in 2011 and we will redo it. Life is a cycle.
>
> Then we would love to see if we can have a VMMaker repackaged (it should not be difficult) with less code so that we can do lectures on the Pharo VM
> and exercises for students with it. It is not really pedagogical to tell them to jump over many not used classes.
> And more important since we want to attract smart guys they will not take us seriously if we present a system that is monolithic
> - we got several times students asking why VMMaker could not be cut into different packages and well they are right.
>
> What I was thinking is that we should have soon a travis to build it automatically with Pharo.
> If you are interested to help please join the effort.
> What guille showed us is that it was not difficult to do it in fact.
>
> Stef
>
>>
>> Regards .. Subbu
>>
>
>

David T. Lewis

Re: Some more effort to make Slang and VMMaker work on Pharo [for review]

In reply to this post by Guillermo Polito

Guille,

This is excellent work. Thank you also for your careful explanations.

Dave

On Wed, May 15, 2019 at 10:33:38AM +0200, Guillermo Polito wrote:

> Hi Eliot,
>
> > El 15 may 2019, a las 3:20, Eliot Miranda <[hidden email]> escribi??:
> >
> > Hi Guille,
> >
> > On Tue, May 14, 2019 at 7:30 AM Guillermo Polito <[hidden email] <mailto:[hidden email]>> wrote:
> >
> > - I had to review the AST-to-AST transformation, checking the output, comparing it to what squeak does and so on...
> >
> > Can you show a little of th differences in output between Squeak compiler derived sources and RB parser derived sources? I'm not going to demand changes or fixes or that they be exactly the same. But I am curious to know what kind of changes there are.
>
> From what I have now, I can see three main kind of differences (I actually compared with source code I generated myself from squeak and the existing sources in the git repo).
> I did not work on them because the VM seemed to work, so I left them for later:
>
> 1) some comments are placed in different places in the output source code. This is entirely cosmetic, but it brings unnecessary noise to C source diffs.
>
> 2) labels in the gnu???ed code are numbered differently. I did not spot any ???bug??? in that yet, I did not dig enough into the gnuification to see why the difference, and I???m not entirely sure that this difference is caused by that either :)???
> Anyhow, this is mostly annoying because it also brings lots of noise to diffs.
>
> 3) some for loops inline the condition while they probably shouldn't. For example:
>
> 0 to: (self numSlotsOf: array1) - 1 do: [ ??? ]
>
> gets in my output translated to
>
> for (i = 0; i < (numSlotsOf(array1)); i += 1) { ...
>
> while in the version generated from squeak
>
> for (i = 0, iLimiT = ((numSlotsOf(array1)) - 1); i <= iLimiT; i += 1) { ???
>
> This seems the most ???dangerous" difference I have so far specially if the limit condition has some side effect.
> We will probably agree in that this third point should be fixed ^^.
>
> >
> > - I???ve written several unit tests to ensure that future migrations are easier to do
> > - I???ve introduced several compatibility classes/methods related to PackageInfo, Time and so on???
> >
> > This is great1 Thank you. Perhaps we can move some of the source generation tests into the core. It would be good for there to be tests that apply to the Squeak compiler side too. I never took the time to run tests (I look at the generated source every time I make a change Slang), and writing a really good set of tests always seemed like a lot of work. But I might be tempted to add to the ones you've written.
>
> Maybe in the next weeks I can try to massage the tests to make them less dependent on the source (wether squeak's or pharo???s) AST.
> The thing is that Pharo???s AST is closer to the source code, while Squeak???s has already some pre-cooked code transformations (like #ifNil: & co), and my tests assume a specific AST structure to traverse the tree and compare???
>
> Cheers,
> Guille