VMMaker in Pharo [Was: Squeak and Tonel]

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

VMMaker in Pharo [Was: Squeak and Tonel]

Tudor Girba-2
 
Hi Eliot,

You raise a few technical concerns related to VMMaker that seem addressable:

1. Rendering technology: Both in Pharo and in GT, there exist in-image canvases, Athens and Sparta respectively, that are backend independent. This allows us to have image-based drawing logic, like BitBlt or even better, the drawing capabilities that Juan Vuletich built in Morphic 3, transparently. In fact, we even started to look at the code from Morphic 3 and we think it’s not much effort to make it work behind Sparta. This would provide a complete solution that would not rely on any external libraries.

2. Git history. We used to have the author and timestamp in the metadata of Tonel but because it generated unwanted conflicts it was decided to remove it. I think this is a good decision and one based on the experience of using it. But, why do you think the algorithm of retrieving Git could not be made to work in Squeak? Is there a specific technical reason, or is it more that you do not see the effort being spent in this direction? I am asking to understand what could a possible solution be.


But, let’s take a step back. George and Andrei did move the VMMaker code, including all its history, into Git using (and enhancing) https://github.com/peteruhnak/git-migration.

The code is here:
https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker

Our goal was to make the code loadable and runnable in Pharo through Git. We have a reproducible process to produce this history, and this means that we can also synchronize all changes from the Monticello VMMaker repository and produce a history-preserving Git repository.

The current code comes with a BaselineOfVMMaker that loads in Pharo, and the code generation seems to work to a good extent without much effort. There is still work that needs to be done especially on the simulation part, but we think most of it can be addressed through a Pharo-specific package that provides compatibility logic.

I see two benefits for doing this:
1. It allows people in Pharo to play with the VM within the Pharo context and consequently contribute.
2. It allows us to build an environment dedicated for VM development without affecting the current contribution process. We can judge later whether this is useful.

Cheers,
Doru


> On Feb 17, 2019, at 9:27 PM, Eliot Miranda <[hidden email]> wrote:
>
> Hi All, Hi Torsten, Hi Jakob,
>
> On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <[hidden email]> wrote:
> Hi,
>
> as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
>
> In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
> with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
>
> Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
> repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
> to Tonel format [3].
>
> So I wonder about adoption of Tonel in Squeak land.
>
> Any status/comments?
>
> Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker.  The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
>
> This project, though no fault of feenk's did not go well.  A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog.  But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places.  In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model".  This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated).  As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced.  Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
>
> I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
>
> So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
>
> I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether.  The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
>
> Here's the kind of crap people like Ducasse throw at me:
>
> "Eliot
>
> At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy
> because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have
> to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world
> of the “open” smalltalk VM. "
>
> I am so over this crap.
> _,,,^..^,,,_
> best, Eliot

--
www.feenk.com

“Software has no shape. Actually, it has no one shape. It has many."

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Tudor Girba-2
 
Hi Eliot,

Please, let me clarify the current direction: I am not trying to get you to Pharo at this point. I only want to reach a point where the VMMaker code runs in Pharo. This will allow us to bootstrap an environment that can later be evaluated.

Having the sources in Git is secondary at this point. With the current infrastructure, we get those sources in Git while preserving the history from the Squeaksource repo automatically. The code will be loadable from either Git or Squeaksource, but the commit repository is still the Squeaksource one. So, nothing would change from the  workflow of the current committers. What we gain with the Git repository is a test ground and an easier ability to load the code in Pharo, all at no extra cost and without affecting your flow.

Let me emphasize this again: the Git support should not be the main issue at this point. Now we would only need to get the code to run in Pharo. For this, we would need to introduce a few support methods. One key method that Andrei and I would like to propose, and that we used before in other porting projects, is:
Object>>vmMakerForSqueak: aBlockToBeExecutedOnlyInSqueak forPharo: aBlockToBeExecutedOnlyInPharo

Andrei will propose a commit in VMMaker Inbox this week.

What do you say?

Cheers,
Doru


> On Feb 18, 2019, at 7:55 PM, Eliot Miranda <[hidden email]> wrote:
>
> Doru,
>
> On Sun, Feb 17, 2019 at 10:33 PM Tudor Girba <[hidden email]> wrote:
> Hi Eliot,
>
> You raise a few technical concerns related to VMMaker that seem addressable:
>
> 1. Rendering technology: Both in Pharo and in GT, there exist in-image canvases, Athens and Sparta respectively, that are backend independent. This allows us to have image-based drawing logic, like BitBlt or even better, the drawing capabilities that Juan Vuletich built in Morphic 3, transparently. In fact, we even started to look at the code from Morphic 3 and we think it’s not much effort to make it work behind Sparta. This would provide a complete solution that would not rely on any external libraries.
>
> 2. Git history. We used to have the author and timestamp in the metadata of Tonel but because it generated unwanted conflicts it was decided to remove it. I think this is a good decision and one based on the experience of using it. But, why do you think the algorithm of retrieving Git could not be made to work in Squeak? Is there a specific technical reason, or is it more that you do not see the effort being spent in this direction? I am asking to understand what could a possible solution be.
>
>
> But, let’s take a step back. George and Andrei did move the VMMaker code, including all its history, into Git using (and enhancing) https://github.com/peteruhnak/git-migration.
>
> The code is here:
> https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
>
> Our goal was to make the code loadable and runnable in Pharo through Git. We have a reproducible process to produce this history, and this means that we can also synchronize all changes from the Monticello VMMaker repository and produce a history-preserving Git repository.
>
> The current code comes with a BaselineOfVMMaker that loads in Pharo, and the code generation seems to work to a good extent without much effort. There is still work that needs to be done especially on the simulation part, but we think most of it can be addressed through a Pharo-specific package that provides compatibility logic.
>
> I see two benefits for doing this:
> 1. It allows people in Pharo to play with the VM within the Pharo context and consequently contribute.
> 2. It allows us to build an environment dedicated for VM development without affecting the current contribution process. We can judge later whether this is useful.
>
> All this is great work.  But I cannot use it until I can also function in Squeak.  I know you want me to move to Pharo, but I am not convinced that it is a safe place for me to land.  I have voiced my concerns before (stability, moving away from support I need (e.g. Pharo dropping read./write file streams is the latest example).  Let me simply restate that "t6he devil you know is better than the devil you don't", and that I know I am productive with VMMaker in Squeak, but I *do not* know that I can be productive in Pharo (in fact, I have lots of experience of the contrary).  So I will not move to Pharo any time soon and so, until there is a solution to git/tonel/squeak integration I cannot join the unified repository, must as I would like to.
>
> I'm sorry for the frustration this causes other.  I am radical in some things, but deeply conservative in others.  And my ability to be productive is obviously foundational.
>
>
> Cheers,
> Doru
>
>
> > On Feb 17, 2019, at 9:27 PM, Eliot Miranda <[hidden email]> wrote:
> >
> > Hi All, Hi Torsten, Hi Jakob,
> >
> > On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <[hidden email]> wrote:
> > Hi,
> >
> > as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
> >
> > In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
> > with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
> >
> > Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
> > repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
> > to Tonel format [3].
> >
> > So I wonder about adoption of Tonel in Squeak land.
> >
> > Any status/comments?
> >
> > Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker.  The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
> >
> > This project, though no fault of feenk's did not go well.  A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog.  But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places.  In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model".  This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated).  As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced.  Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
> >
> > I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
> >
> > So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
> >
> > I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether.  The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
> >
> > Here's the kind of crap people like Ducasse throw at me:
> >
> > "Eliot
> >
> > At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy
> > because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have
> > to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world
> > of the “open” smalltalk VM. "
> >
> > I am so over this crap.
> > _,,,^..^,,,_
> > best, Eliot
>
> --
> www.feenk.com
>
> “Software has no shape. Actually, it has no one shape. It has many."
>
>
>
> --
> _,,,^..^,,,_
> best, Eliot

--
www.feenk.com

"Sometimes the best solution is not the best solution."

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Tudor Girba-2
In reply to this post by Tudor Girba-2
 
Hi Nicolas,


> On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:
>
> Hi Tudor,
> Retrieving author and timestamp from git history is doable.
> The question is whether it will be still possible to contribute thru MC or not.
> If it is metadataless MC, there's no point, it's unusable.
> I think that the minimal metadata we have to maintain is the last MC ancestor. This can be done by convention thru a .last_mc_ancestor file. Of course, there will be conflicts, but easy to (automatically) solve.
> That could also lie in commit message, but is somehow a bit more fragile... it would mean scanning thru ancestry for getting such information.
> By symmetry, merged agit sha should be in mc commit, either a specific file in mcz, or commit message.

That sounds intriguing and interesting at the same time. However, I do not have a qualified enough opinion on this matter.


> Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> https://github.com/hpi-swa/Squot
> There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> IMO, a broken history has not much interest.

We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker


> The question is whether it is possible to come to acceptable compromise?

As I wrote in the reply to Eliot, I believe the main point at this moment is not Git, but getting the VMMaker code to run in Pharo. I think we should start there, and see if we can produce an alternative environment that provides value.

Cheers,
Doru


> Le lun. 18 févr. 2019 à 07:33, Tudor Girba <[hidden email]> a écrit :
>  
> Hi Eliot,
>
> You raise a few technical concerns related to VMMaker that seem addressable:
>
> 1. Rendering technology: Both in Pharo and in GT, there exist in-image canvases, Athens and Sparta respectively, that are backend independent. This allows us to have image-based drawing logic, like BitBlt or even better, the drawing capabilities that Juan Vuletich built in Morphic 3, transparently. In fact, we even started to look at the code from Morphic 3 and we think it’s not much effort to make it work behind Sparta. This would provide a complete solution that would not rely on any external libraries.
>
> 2. Git history. We used to have the author and timestamp in the metadata of Tonel but because it generated unwanted conflicts it was decided to remove it. I think this is a good decision and one based on the experience of using it. But, why do you think the algorithm of retrieving Git could not be made to work in Squeak? Is there a specific technical reason, or is it more that you do not see the effort being spent in this direction? I am asking to understand what could a possible solution be.
>
>
> But, let’s take a step back. George and Andrei did move the VMMaker code, including all its history, into Git using (and enhancing) https://github.com/peteruhnak/git-migration.
>
> The code is here:
> https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
>
> Our goal was to make the code loadable and runnable in Pharo through Git. We have a reproducible process to produce this history, and this means that we can also synchronize all changes from the Monticello VMMaker repository and produce a history-preserving Git repository.
>
> The current code comes with a BaselineOfVMMaker that loads in Pharo, and the code generation seems to work to a good extent without much effort. There is still work that needs to be done especially on the simulation part, but we think most of it can be addressed through a Pharo-specific package that provides compatibility logic.
>
> I see two benefits for doing this:
> 1. It allows people in Pharo to play with the VM within the Pharo context and consequently contribute.
> 2. It allows us to build an environment dedicated for VM development without affecting the current contribution process. We can judge later whether this is useful.
>
> Cheers,
> Doru
>
>
> > On Feb 17, 2019, at 9:27 PM, Eliot Miranda <[hidden email]> wrote:
> >
> > Hi All, Hi Torsten, Hi Jakob,
> >
> > On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <[hidden email]> wrote:
> > Hi,
> >
> > as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
> >
> > In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
> > with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
> >
> > Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
> > repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
> > to Tonel format [3].
> >
> > So I wonder about adoption of Tonel in Squeak land.
> >
> > Any status/comments?
> >
> > Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker.  The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
> >
> > This project, though no fault of feenk's did not go well.  A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog.  But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places.  In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model".  This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated).  As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced.  Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
> >
> > I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
> >
> > So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
> >
> > I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether.  The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
> >
> > Here's the kind of crap people like Ducasse throw at me:
> >
> > "Eliot
> >
> > At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy
> > because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have
> > to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world
> > of the “open” smalltalk VM. "
> >
> > I am so over this crap.
> > _,,,^..^,,,_
> > best, Eliot
>
> --
> www.feenk.com
>
> “Software has no shape. Actually, it has no one shape. It has many."
>

--
www.feenk.com

“How do you feenk today?"

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Tudor Girba-2
In reply to this post by Tudor Girba-2
 
Hi,

I have seen that code, and the issue I wanted to address is to have the decision about which dialect a piece of code is ran on in one single place. This should make it easier to find statically.

Indeed, this will not necessarily be guaranteed to work when moving from one version of Pharo to another. But, that problem can be addressed by tracing releases in a reproducible fashion. And in the VMMaker case, the scenario is about a development environment not a production system. So, it should be reasonable to point developers to a working configuration that pairs the Pharo/Squeak version and the VMMaker version.

What do you think?

Cheers,
Doru


> On Feb 20, 2019, at 4:39 AM, Eliot Miranda <[hidden email]> wrote:
>
> Hi Doru,
>
> On Mon, Feb 18, 2019 at 8:14 PM Tudor Girba <[hidden email]> wrote:
> Hi Eliot,
>
> Please, let me clarify the current direction: I am not trying to get you to Pharo at this point. I only want to reach a point where the VMMaker code runs in Pharo. This will allow us to bootstrap an environment that can later be evaluated.
>
> Having the sources in Git is secondary at this point. With the current infrastructure, we get those sources in Git while preserving the history from the Squeaksource repo automatically. The code will be loadable from either Git or Squeaksource, but the commit repository is still the Squeaksource one. So, nothing would change from the  workflow of the current committers. What we gain with the Git repository is a test ground and an easier ability to load the code in Pharo, all at no extra cost and without affecting your flow.
>
> Let me emphasize this again: the Git support should not be the main issue at this point. Now we would only need to get the code to run in Pharo. For this, we would need to introduce a few support methods. One key method that Andrei and I would like to propose, and that we used before in other porting projects, is:
> Object>>vmMakerForSqueak: aBlockToBeExecutedOnlyInSqueak forPharo: aBlockToBeExecutedOnlyInPharo
>
> I am doing things on a case-by-case basis.  Currently I use classNamed:, e.g.
>
> (Smalltalk classNamed: #FileSystem)
> ifNotNil: [:fileSystem| fileSystem disk delimiter asInteger]
> ifNil: [FileDirectory pathNameDelimiter asInteger]
>
> (Smalltalk classNamed: #CurrentReadOnlySourceFiles)
> ifNil: syncAction
> ifNotNil: [:crosf| crosf cacheDuring: syncAction]
>
> (Smalltalk classNamed: #Project)
> ifNotNil: [:project| Project uiProcess] "Squeak"
> ifNil: [UIManager default uiProcess] "Pharo"
>
>  ((Smalltalk classNamed: #FillInTheBlankMorph)
> ifNotNil: "Squeak"
> [:fITBM|
> fITBM
> request: 'Input please!'
> initialAnswer: ''
> centerAt: ActiveHand cursorPoint
> inWorld: ActiveWorld
> onCancelReturn: nil
> acceptOnCR: true]
> ifNil: "Pharo; onCancelReturn: nil is the default here"
> [UIManager default
> request: 'Input please!'
> initialAnswer: '']).
>
> this at least allows Squeak to migrate to FileSystem without having to rewrite.  Using the above I was already able to get most of VMMaker working in Pharo6.  Things broke for Pharo7, which indicates that the vmMakerForSqueak:forPharo: approach has vulnerabilities.
>
>
> Andrei will propose a commit in VMMaker Inbox this week.
>
> What do you say?
>
> Cheers,
> Doru
>
>
> > On Feb 18, 2019, at 7:55 PM, Eliot Miranda <[hidden email]> wrote:
> >
> > Doru,
> >
> > On Sun, Feb 17, 2019 at 10:33 PM Tudor Girba <[hidden email]> wrote:
> > Hi Eliot,
> >
> > You raise a few technical concerns related to VMMaker that seem addressable:
> >
> > 1. Rendering technology: Both in Pharo and in GT, there exist in-image canvases, Athens and Sparta respectively, that are backend independent. This allows us to have image-based drawing logic, like BitBlt or even better, the drawing capabilities that Juan Vuletich built in Morphic 3, transparently. In fact, we even started to look at the code from Morphic 3 and we think it’s not much effort to make it work behind Sparta. This would provide a complete solution that would not rely on any external libraries.
> >
> > 2. Git history. We used to have the author and timestamp in the metadata of Tonel but because it generated unwanted conflicts it was decided to remove it. I think this is a good decision and one based on the experience of using it. But, why do you think the algorithm of retrieving Git could not be made to work in Squeak? Is there a specific technical reason, or is it more that you do not see the effort being spent in this direction? I am asking to understand what could a possible solution be.
> >
> >
> > But, let’s take a step back. George and Andrei did move the VMMaker code, including all its history, into Git using (and enhancing) https://github.com/peteruhnak/git-migration.
> >
> > The code is here:
> > https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
> >
> > Our goal was to make the code loadable and runnable in Pharo through Git. We have a reproducible process to produce this history, and this means that we can also synchronize all changes from the Monticello VMMaker repository and produce a history-preserving Git repository.
> >
> > The current code comes with a BaselineOfVMMaker that loads in Pharo, and the code generation seems to work to a good extent without much effort. There is still work that needs to be done especially on the simulation part, but we think most of it can be addressed through a Pharo-specific package that provides compatibility logic.
> >
> > I see two benefits for doing this:
> > 1. It allows people in Pharo to play with the VM within the Pharo context and consequently contribute.
> > 2. It allows us to build an environment dedicated for VM development without affecting the current contribution process. We can judge later whether this is useful.
> >
> > All this is great work.  But I cannot use it until I can also function in Squeak.  I know you want me to move to Pharo, but I am not convinced that it is a safe place for me to land.  I have voiced my concerns before (stability, moving away from support I need (e.g. Pharo dropping read./write file streams is the latest example).  Let me simply restate that "t6he devil you know is better than the devil you don't", and that I know I am productive with VMMaker in Squeak, but I *do not* know that I can be productive in Pharo (in fact, I have lots of experience of the contrary).  So I will not move to Pharo any time soon and so, until there is a solution to git/tonel/squeak integration I cannot join the unified repository, must as I would like to.
> >
> > I'm sorry for the frustration this causes other.  I am radical in some things, but deeply conservative in others.  And my ability to be productive is obviously foundational.
> >
> >
> > Cheers,
> > Doru
> >
> >
> > > On Feb 17, 2019, at 9:27 PM, Eliot Miranda <[hidden email]> wrote:
> > >
> > > Hi All, Hi Torsten, Hi Jakob,
> > >
> > > On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <[hidden email]> wrote:
> > > Hi,
> > >
> > > as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
> > >
> > > In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
> > > with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
> > >
> > > Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
> > > repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
> > > to Tonel format [3].
> > >
> > > So I wonder about adoption of Tonel in Squeak land.
> > >
> > > Any status/comments?
> > >
> > > Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker.  The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
> > >
> > > This project, though no fault of feenk's did not go well.  A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog.  But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places.  In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model".  This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated).  As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced.  Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
> > >
> > > I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
> > >
> > > So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
> > >
> > > I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether.  The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
> > >
> > > Here's the kind of crap people like Ducasse throw at me:
> > >
> > > "Eliot
> > >
> > > At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy
> > > because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have
> > > to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world
> > > of the “open” smalltalk VM. "
> > >
> > > I am so over this crap.
> > > _,,,^..^,,,_
> > > best, Eliot
> >
> > --
> > www.feenk.com
> >
> > “Software has no shape. Actually, it has no one shape. It has many."
> >
> >
> >
> > --
> > _,,,^..^,,,_
> > best, Eliot
>
> --
> www.feenk.com
>
> "Sometimes the best solution is not the best solution."
>
>
>
> --
> _,,,^..^,,,_
> best, Eliot

--
www.feenk.com

"Beauty is where we see it."





Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

David T. Lewis
 
On Wed, Feb 20, 2019 at 09:23:55AM +0100, Tudor Girba wrote:

>  
> Hi,
>
> I have seen that code, and the issue I wanted to address is to have the
> decision about which dialect a piece of code is ran on in one single place.
> This should make it easier to find statically.
>
> Indeed, this will not necessarily be guaranteed to work when moving from
> one version of Pharo to another. But, that problem can be addressed by
> tracing releases in a reproducible fashion. And in the VMMaker case, the
> scenario is about a development environment not a production system. So,
> it should be reasonable to point developers to a working configuration
> that pairs the Pharo/Squeak version and the VMMaker version.
>
> What do you think?
>
> Cheers,
> Doru
>

Hi Doru and Eliiot,

The approach that I have been using to address this in OSProcess is
to put the compatibility methods in one place, in the 'compatibility'
category in class OSProcess. This allows support for the different
file system interfaces in Cuis, Pharo and Squeak.

For example, one of the compatibility methods is:

OSProcess class>>directoryExists: path
        "Answer true if a directory of the given name exists. The given name may
        be either a full path name or a local directory within this directory."

        ^ self useFileMan
                ifTrue: [((Smalltalk at: #DirectoryEntry) perform: #withPathName: with: path) exists]
                ifFalse: [self useFileSystem
                        ifTrue: [ (path perform: #asFileReference) exists ]
                        ifFalse: [ (Smalltalk at: #FileDirectory) default directoryExists: path ]]


And the tests for Cuis/Pharo/Squeak are in these two methods:

OSProcess class>>useFileSystem
        "If true use FileSystem, otherwise use traditional FileDirectory. See senders
        for methods with file system dependencies."

        ^ Smalltalk hasClassNamed: #FileReference

OSProcess class>>useFileMan
        "If true use FileMan for directory and file access. See senders for methods with file
        system dependencies."

        ^ Smalltalk hasClassNamed: #FileIOAccessor


This is ugly but it does work, and I been able to maintain OSProcess and
CommandShell for the Squeak diaspora over a number of years in this manner.
It also means that I do not need to worry if Squeak moves to FileMan(*) or
FileSystem at some point in the future.

I don't think that VMMaker should have a hard dependency on OSProcess,
but as a practical matter you probably have OSProcess loaded in the VM
development image anyway, so you could try using these methods directly
and see how ugly things get.

HTH,
Dave

* Personally I like the FileMan approach in Cuis best, but that is
another topic entirely ;-)


Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Eliot Miranda-2
In reply to this post by Tudor Girba-2
 


On Wed, Feb 20, 2019 at 12:23 AM Tudor Girba <[hidden email]> wrote:
Hi,

I have seen that code, and the issue I wanted to address is to have the decision about which dialect a piece of code is ran on in one single place. This should make it easier to find statically.

Indeed, this will not necessarily be guaranteed to work when moving from one version of Pharo to another. But, that problem can be addressed by tracing releases in a reproducible fashion. And in the VMMaker case, the scenario is about a development environment not a production system. So, it should be reasonable to point developers to a working configuration that pairs the Pharo/Squeak version and the VMMaker version.

What do you think?

I think the solution I've used so far is fine for now.  There's no need to invent another convention.  If one wants to identify statically, beyond a comment saying :Squeak5", "Pharo6" (which can still be used in a static query), via a message then invent a tag scheme.  But for now I don't see that any effort is justified.


As far as versioning goes, this goes to the heart of the stability issue for Pharo.  Pharo needs to decide to support certain external packages and not pull the rug from them all the time.  Since VMMaker is indeed a development environment it makes sense for it to function in versions close to the tip ("trunk" in Squeak).  In Pharo that should translate into the last release and the current development, but as we've seen there are stability issues with that.


Cheers,
Doru


> On Feb 20, 2019, at 4:39 AM, Eliot Miranda <[hidden email]> wrote:
>
> Hi Doru,
>
> On Mon, Feb 18, 2019 at 8:14 PM Tudor Girba <[hidden email]> wrote:
> Hi Eliot,
>
> Please, let me clarify the current direction: I am not trying to get you to Pharo at this point. I only want to reach a point where the VMMaker code runs in Pharo. This will allow us to bootstrap an environment that can later be evaluated.
>
> Having the sources in Git is secondary at this point. With the current infrastructure, we get those sources in Git while preserving the history from the Squeaksource repo automatically. The code will be loadable from either Git or Squeaksource, but the commit repository is still the Squeaksource one. So, nothing would change from the  workflow of the current committers. What we gain with the Git repository is a test ground and an easier ability to load the code in Pharo, all at no extra cost and without affecting your flow.
>
> Let me emphasize this again: the Git support should not be the main issue at this point. Now we would only need to get the code to run in Pharo. For this, we would need to introduce a few support methods. One key method that Andrei and I would like to propose, and that we used before in other porting projects, is:
> Object>>vmMakerForSqueak: aBlockToBeExecutedOnlyInSqueak forPharo: aBlockToBeExecutedOnlyInPharo
>
> I am doing things on a case-by-case basis.  Currently I use classNamed:, e.g.
>
> (Smalltalk classNamed: #FileSystem)
>       ifNotNil: [:fileSystem| fileSystem disk delimiter asInteger]
>       ifNil: [FileDirectory pathNameDelimiter asInteger]
>
> (Smalltalk classNamed: #CurrentReadOnlySourceFiles)
>       ifNil: syncAction
>       ifNotNil: [:crosf| crosf cacheDuring: syncAction]
>
> (Smalltalk classNamed: #Project)
>       ifNotNil: [:project| Project uiProcess] "Squeak"
>       ifNil: [UIManager default uiProcess] "Pharo"
>
>  ((Smalltalk classNamed: #FillInTheBlankMorph)
>       ifNotNil: "Squeak"
>               [:fITBM|
>                fITBM
>                       request: 'Input please!'
>                       initialAnswer: ''
>                       centerAt: ActiveHand cursorPoint
>                       inWorld: ActiveWorld
>                       onCancelReturn: nil
>                       acceptOnCR: true]
>       ifNil: "Pharo; onCancelReturn: nil is the default here"
>               [UIManager default
>                       request: 'Input please!'
>                       initialAnswer: '']).
>
> this at least allows Squeak to migrate to FileSystem without having to rewrite.  Using the above I was already able to get most of VMMaker working in Pharo6.  Things broke for Pharo7, which indicates that the vmMakerForSqueak:forPharo: approach has vulnerabilities.
>
>
> Andrei will propose a commit in VMMaker Inbox this week.
>
> What do you say?
>
> Cheers,
> Doru
>
>
> > On Feb 18, 2019, at 7:55 PM, Eliot Miranda <[hidden email]> wrote:
> >
> > Doru,
> >
> > On Sun, Feb 17, 2019 at 10:33 PM Tudor Girba <[hidden email]> wrote:
> > Hi Eliot,
> >
> > You raise a few technical concerns related to VMMaker that seem addressable:
> >
> > 1. Rendering technology: Both in Pharo and in GT, there exist in-image canvases, Athens and Sparta respectively, that are backend independent. This allows us to have image-based drawing logic, like BitBlt or even better, the drawing capabilities that Juan Vuletich built in Morphic 3, transparently. In fact, we even started to look at the code from Morphic 3 and we think it’s not much effort to make it work behind Sparta. This would provide a complete solution that would not rely on any external libraries.
> >
> > 2. Git history. We used to have the author and timestamp in the metadata of Tonel but because it generated unwanted conflicts it was decided to remove it. I think this is a good decision and one based on the experience of using it. But, why do you think the algorithm of retrieving Git could not be made to work in Squeak? Is there a specific technical reason, or is it more that you do not see the effort being spent in this direction? I am asking to understand what could a possible solution be.
> >
> >
> > But, let’s take a step back. George and Andrei did move the VMMaker code, including all its history, into Git using (and enhancing) https://github.com/peteruhnak/git-migration.
> >
> > The code is here:
> > https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
> >
> > Our goal was to make the code loadable and runnable in Pharo through Git. We have a reproducible process to produce this history, and this means that we can also synchronize all changes from the Monticello VMMaker repository and produce a history-preserving Git repository.
> >
> > The current code comes with a BaselineOfVMMaker that loads in Pharo, and the code generation seems to work to a good extent without much effort. There is still work that needs to be done especially on the simulation part, but we think most of it can be addressed through a Pharo-specific package that provides compatibility logic.
> >
> > I see two benefits for doing this:
> > 1. It allows people in Pharo to play with the VM within the Pharo context and consequently contribute.
> > 2. It allows us to build an environment dedicated for VM development without affecting the current contribution process. We can judge later whether this is useful.
> >
> > All this is great work.  But I cannot use it until I can also function in Squeak.  I know you want me to move to Pharo, but I am not convinced that it is a safe place for me to land.  I have voiced my concerns before (stability, moving away from support I need (e.g. Pharo dropping read./write file streams is the latest example).  Let me simply restate that "t6he devil you know is better than the devil you don't", and that I know I am productive with VMMaker in Squeak, but I *do not* know that I can be productive in Pharo (in fact, I have lots of experience of the contrary).  So I will not move to Pharo any time soon and so, until there is a solution to git/tonel/squeak integration I cannot join the unified repository, must as I would like to.
> >
> > I'm sorry for the frustration this causes other.  I am radical in some things, but deeply conservative in others.  And my ability to be productive is obviously foundational.
> >
> >
> > Cheers,
> > Doru
> >
> >
> > > On Feb 17, 2019, at 9:27 PM, Eliot Miranda <[hidden email]> wrote:
> > >
> > > Hi All, Hi Torsten, Hi Jakob,
> > >
> > > On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <[hidden email]> wrote:
> > > Hi,
> > >
> > > as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
> > >
> > > In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
> > > with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
> > >
> > > Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
> > > repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
> > > to Tonel format [3].
> > >
> > > So I wonder about adoption of Tonel in Squeak land.
> > >
> > > Any status/comments?
> > >
> > > Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker.  The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
> > >
> > > This project, though no fault of feenk's did not go well.  A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog.  But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places.  In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model".  This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated).  As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced.  Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
> > >
> > > I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
> > >
> > > So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
> > >
> > > I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether.  The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
> > >
> > > Here's the kind of crap people like Ducasse throw at me:
> > >
> > > "Eliot
> > >
> > > At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy
> > > because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have
> > > to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world
> > > of the “open” smalltalk VM. "
> > >
> > > I am so over this crap.
> > > _,,,^..^,,,_
> > > best, Eliot
> >
> > --
> > www.feenk.com
> >
> > “Software has no shape. Actually, it has no one shape. It has many."
> >
> >
> >
> > --
> > _,,,^..^,,,_
> > best, Eliot
>
> --
> www.feenk.com
>
> "Sometimes the best solution is not the best solution."
>
>
>
> --
> _,,,^..^,,,_
> best, Eliot

--
www.feenk.com

"Beauty is where we see it."







--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Nicolas Cellier
In reply to this post by Tudor Girba-2
 
Hi Doru,

Le mar. 19 févr. 2019 à 10:39, Tudor Girba <[hidden email]> a écrit :
 
Hi Nicolas,


> On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:
>
> Hi Tudor,
> Retrieving author and timestamp from git history is doable.
> The question is whether it will be still possible to contribute thru MC or not.
> If it is metadataless MC, there's no point, it's unusable.
> I think that the minimal metadata we have to maintain is the last MC ancestor. This can be done by convention thru a .last_mc_ancestor file. Of course, there will be conflicts, but easy to (automatically) solve.
> That could also lie in commit message, but is somehow a bit more fragile... it would mean scanning thru ancestry for getting such information.
> By symmetry, merged agit sha should be in mc commit, either a specific file in mcz, or commit message.

That sounds intriguing and interesting at the same time. However, I do not have a qualified enough opinion on this matter.


> Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> https://github.com/hpi-swa/Squot
> There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> IMO, a broken history has not much interest.

We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker


Ah OK, I'll try to have a look.


> The question is whether it is possible to come to acceptable compromise?

As I wrote in the reply to Eliot, I believe the main point at this moment is not Git, but getting the VMMaker code to run in Pharo. I think we should start there, and see if we can produce an alternative environment that provides value.

Cheers,
Doru

Maybe some well trained Pharoer should pair with Eliot then, otherwise it's like wasting Eliot's time on non fundamental VM subjects.


> Le lun. 18 févr. 2019 à 07:33, Tudor Girba <[hidden email]> a écrit :

> Hi Eliot,
>
> You raise a few technical concerns related to VMMaker that seem addressable:
>
> 1. Rendering technology: Both in Pharo and in GT, there exist in-image canvases, Athens and Sparta respectively, that are backend independent. This allows us to have image-based drawing logic, like BitBlt or even better, the drawing capabilities that Juan Vuletich built in Morphic 3, transparently. In fact, we even started to look at the code from Morphic 3 and we think it’s not much effort to make it work behind Sparta. This would provide a complete solution that would not rely on any external libraries.
>
> 2. Git history. We used to have the author and timestamp in the metadata of Tonel but because it generated unwanted conflicts it was decided to remove it. I think this is a good decision and one based on the experience of using it. But, why do you think the algorithm of retrieving Git could not be made to work in Squeak? Is there a specific technical reason, or is it more that you do not see the effort being spent in this direction? I am asking to understand what could a possible solution be.
>
>
> But, let’s take a step back. George and Andrei did move the VMMaker code, including all its history, into Git using (and enhancing) https://github.com/peteruhnak/git-migration.
>
> The code is here:
> https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
>
> Our goal was to make the code loadable and runnable in Pharo through Git. We have a reproducible process to produce this history, and this means that we can also synchronize all changes from the Monticello VMMaker repository and produce a history-preserving Git repository.
>
> The current code comes with a BaselineOfVMMaker that loads in Pharo, and the code generation seems to work to a good extent without much effort. There is still work that needs to be done especially on the simulation part, but we think most of it can be addressed through a Pharo-specific package that provides compatibility logic.
>
> I see two benefits for doing this:
> 1. It allows people in Pharo to play with the VM within the Pharo context and consequently contribute.
> 2. It allows us to build an environment dedicated for VM development without affecting the current contribution process. We can judge later whether this is useful.
>
> Cheers,
> Doru
>
>
> > On Feb 17, 2019, at 9:27 PM, Eliot Miranda <[hidden email]> wrote:
> >
> > Hi All, Hi Torsten, Hi Jakob,
> >
> > On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <[hidden email]> wrote:
> > Hi,
> >
> > as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
> >
> > In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
> > with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
> >
> > Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
> > repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
> > to Tonel format [3].
> >
> > So I wonder about adoption of Tonel in Squeak land.
> >
> > Any status/comments?
> >
> > Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker.  The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
> >
> > This project, though no fault of feenk's did not go well.  A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog.  But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places.  In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model".  This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated).  As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced.  Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
> >
> > I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
> >
> > So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
> >
> > I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether.  The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
> >
> > Here's the kind of crap people like Ducasse throw at me:
> >
> > "Eliot
> >
> > At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy
> > because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have
> > to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world
> > of the “open” smalltalk VM. "
> >
> > I am so over this crap.
> > _,,,^..^,,,_
> > best, Eliot
>
> --
> www.feenk.com
>
> “Software has no shape. Actually, it has no one shape. It has many."
>

--
www.feenk.com

“How do you feenk today?"

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Nicolas Cellier
 


Le dim. 24 févr. 2019 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
Hi Doru,

Le mar. 19 févr. 2019 à 10:39, Tudor Girba <[hidden email]> a écrit :
 
Hi Nicolas,


> On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:

> Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> https://github.com/hpi-swa/Squot
> There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> IMO, a broken history has not much interest.

We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker


Ah OK, I'll try to have a look.


Hi Doru,
as follow up, I don't see the progress in package history.
If I look for example at history of ThreadedFFIPlugin, then I see this:


Normally I should see only one diff, like this:


This is because true parent is VMMaker.oscog-eem.2108
But git parent is VMMaker.oscog-eem.2109
So it seems like I am undoing all the changes from Eliot... Thats is obscuring the intentions of the commit.

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Jakob Reschke
 
I had a quick look at the git-migration code. From what I saw, it binds all packages together in a linear history. Though the versions are topologically sorted by the Monticello ancestry, this approach cannot preserve any branches, or parallel changes, as Nicolas noted. Thus, it produces wrong diffs. Though, it is perfectly adequate for most one-person projects or projects with a Subversion-like commit policy ("you must always merge newer versions before you save"). I guess that is the majority of Monticello projects.

For the converter in Squot, I chose to write one strand of commits for each package, ending in a branch. The history on each strand maps the package's Monticello ancestry accurately (except that the diffs for missing versions are still missing). But it is not appealing to work with each package on a different branch, of course. Unless you have Monticello configurations, that's all the model provides, is it? You can still get an idea of the inter-package history by viewing the log sorted by author timestamps (--author-date-order), but you cannot really bisect it, for example.

What I did in my repositories so far is import the packages on separate branches and then create an n-way merge (with the command line tools) that pulls all the packages together from the point on when I established the Git repository.

Binding the packages together in a common history needs some kind of heuristic that guesses the past configurations that were in use. For example, timestamps and authors could be used. Let's say contributors abc and xyz worked on some project in parallel and created versions in multiple packages. Then you could assume that PkgX-abc.8 and PkgY-abc.42 with similar timestamps belong to the same branch, while PkgX-xyz.8 (with the same parent as PkgX-abc.8) and PkgZ-xyz.5 with similar timestamps belong to a different branch. There will most certainly be scenarios where the heuristic breaks. For example, when the change in PkgZ-xyz.5 is also relevant for abc's work and she "invisibly" adopts it on her "branch" in Monticello. Or when abc works on her branch and then must quickly fix a bug on whatever "master" in Monticello is, but modifies a different package there (so you cannot see the "switch" in the ancestry).

So the one-nonlinear-branch-per-package approach is unwieldy, but at least it is does not produce incorrect results until someone comes up with a better solution. As written above, the git-migration approach is fine for one-person or branch-less projects.

Am So., 3. März 2019 um 09:23 Uhr schrieb Nicolas Cellier <[hidden email]>:
 


Le dim. 24 févr. 2019 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
Hi Doru,

Le mar. 19 févr. 2019 à 10:39, Tudor Girba <[hidden email]> a écrit :
 
Hi Nicolas,


> On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:

> Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> https://github.com/hpi-swa/Squot
> There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> IMO, a broken history has not much interest.

We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker


Ah OK, I'll try to have a look.


Hi Doru,
as follow up, I don't see the progress in package history.
If I look for example at history of ThreadedFFIPlugin, then I see this:


Normally I should see only one diff, like this:


This is because true parent is VMMaker.oscog-eem.2108
But git parent is VMMaker.oscog-eem.2109
So it seems like I am undoing all the changes from Eliot... Thats is obscuring the intentions of the commit.

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Tudor Girba-2
 
Hi,

Thanks for this summary!

Do you have experience of using Squot on a project of the size of VMMaker? Would you like to try to see how it works? And does it work for Tonel, or only for FileTree?

In any case, I also believe that it is not unreasonable to merge things in one linear history (or even to discard parallel Monticello branches). Here is my reasoning: the way the whole history in Monticello is, to a good extent, via the author timestamp. This essentially provides one change. So, for all practical purposes, preserving the main linear history is good enough.

Cheers,
Doru


> On Mar 4, 2019, at 8:42 AM, Jakob Reschke <[hidden email]> wrote:
>
> I had a quick look at the git-migration code. From what I saw, it binds all packages together in a linear history. Though the versions are topologically sorted by the Monticello ancestry, this approach cannot preserve any branches, or parallel changes, as Nicolas noted. Thus, it produces wrong diffs. Though, it is perfectly adequate for most one-person projects or projects with a Subversion-like commit policy ("you must always merge newer versions before you save"). I guess that is the majority of Monticello projects.
>
> For the converter in Squot, I chose to write one strand of commits for each package, ending in a branch. The history on each strand maps the package's Monticello ancestry accurately (except that the diffs for missing versions are still missing). But it is not appealing to work with each package on a different branch, of course. Unless you have Monticello configurations, that's all the model provides, is it? You can still get an idea of the inter-package history by viewing the log sorted by author timestamps (--author-date-order), but you cannot really bisect it, for example.
>
> What I did in my repositories so far is import the packages on separate branches and then create an n-way merge (with the command line tools) that pulls all the packages together from the point on when I established the Git repository.
>
> Binding the packages together in a common history needs some kind of heuristic that guesses the past configurations that were in use. For example, timestamps and authors could be used. Let's say contributors abc and xyz worked on some project in parallel and created versions in multiple packages. Then you could assume that PkgX-abc.8 and PkgY-abc.42 with similar timestamps belong to the same branch, while PkgX-xyz.8 (with the same parent as PkgX-abc.8) and PkgZ-xyz.5 with similar timestamps belong to a different branch. There will most certainly be scenarios where the heuristic breaks. For example, when the change in PkgZ-xyz.5 is also relevant for abc's work and she "invisibly" adopts it on her "branch" in Monticello. Or when abc works on her branch and then must quickly fix a bug on whatever "master" in Monticello is, but modifies a different package there (so you cannot see the "switch" in the ancestry).
>
> So the one-nonlinear-branch-per-package approach is unwieldy, but at least it is does not produce incorrect results until someone comes up with a better solution. As written above, the git-migration approach is fine for one-person or branch-less projects.
>
> Am So., 3. März 2019 um 09:23 Uhr schrieb Nicolas Cellier <[hidden email]>:
>  
>
>
> Le dim. 24 févr. 2019 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
> Hi Doru,
>
> Le mar. 19 févr. 2019 à 10:39, Tudor Girba <[hidden email]> a écrit :
>  
> Hi Nicolas,
>
>
> > On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:
>
> > Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> > https://github.com/hpi-swa/Squot
> > There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> > IMO, a broken history has not much interest.
>
> We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
> https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
>
>
> Ah OK, I'll try to have a look.
>
>
> Hi Doru,
> as follow up, I don't see the progress in package history.
> If I look for example at history of ThreadedFFIPlugin, then I see this:
>
> https://github.com/feenkcom/opensmalltalk-vm/commit/2ec3b0f6da1a5c7801f7ce8973213ba00b10d7fd#diff-36ccd126e0dd4f5ba2a399c962426080
>
> Normally I should see only one diff, like this:
>
> http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2109.diff
>
> This is because true parent is VMMaker.oscog-eem.2108
> But git parent is VMMaker.oscog-eem.2109
> So it seems like I am undoing all the changes from Eliot... Thats is obscuring the intentions of the commit.
>

--
www.feenk.com

"What we can governs what we wish."





Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Nicolas Cellier
 
Hi Tudor,
It depends good enough for what?
If i'm the author of a change that happens to be a mistake, then i want to be able to exert my responsibility.
But i must first understand the context in which i took sone decisions, and a broken history won't help.
Also linear history is breaking authorship, so maybe i won't even blame myself... example:

Vm.branch1-eem.10 change method a
Vm.branch2-nice.10 change method b
Vm.merge.eem.11 => i'm the author of b changes

But

Vm.branch1-eem.10 change method a
Vm.branch2-nice.10 change method b
Vm.branch1-eem.11 change method c
Vm.merge.eem.12 => Eliot is attributed the authorship of b changes

That often happens with my own changes because Eliot is continuously developing. So from my POV, it's not good enough... just unusable.

Le lun. 4 mars 2019 à 15:23, Tudor Girba <[hidden email]> a écrit :
 
Hi,

Thanks for this summary!

Do you have experience of using Squot on a project of the size of VMMaker? Would you like to try to see how it works? And does it work for Tonel, or only for FileTree?

In any case, I also believe that it is not unreasonable to merge things in one linear history (or even to discard parallel Monticello branches). Here is my reasoning: the way the whole history in Monticello is, to a good extent, via the author timestamp. This essentially provides one change. So, for all practical purposes, preserving the main linear history is good enough.

Cheers,
Doru


> On Mar 4, 2019, at 8:42 AM, Jakob Reschke <[hidden email]> wrote:
>
> I had a quick look at the git-migration code. From what I saw, it binds all packages together in a linear history. Though the versions are topologically sorted by the Monticello ancestry, this approach cannot preserve any branches, or parallel changes, as Nicolas noted. Thus, it produces wrong diffs. Though, it is perfectly adequate for most one-person projects or projects with a Subversion-like commit policy ("you must always merge newer versions before you save"). I guess that is the majority of Monticello projects.
>
> For the converter in Squot, I chose to write one strand of commits for each package, ending in a branch. The history on each strand maps the package's Monticello ancestry accurately (except that the diffs for missing versions are still missing). But it is not appealing to work with each package on a different branch, of course. Unless you have Monticello configurations, that's all the model provides, is it? You can still get an idea of the inter-package history by viewing the log sorted by author timestamps (--author-date-order), but you cannot really bisect it, for example.
>
> What I did in my repositories so far is import the packages on separate branches and then create an n-way merge (with the command line tools) that pulls all the packages together from the point on when I established the Git repository.
>
> Binding the packages together in a common history needs some kind of heuristic that guesses the past configurations that were in use. For example, timestamps and authors could be used. Let's say contributors abc and xyz worked on some project in parallel and created versions in multiple packages. Then you could assume that PkgX-abc.8 and PkgY-abc.42 with similar timestamps belong to the same branch, while PkgX-xyz.8 (with the same parent as PkgX-abc.8) and PkgZ-xyz.5 with similar timestamps belong to a different branch. There will most certainly be scenarios where the heuristic breaks. For example, when the change in PkgZ-xyz.5 is also relevant for abc's work and she "invisibly" adopts it on her "branch" in Monticello. Or when abc works on her branch and then must quickly fix a bug on whatever "master" in Monticello is, but modifies a different package there (so you cannot see the "switch" in the ancestry).
>
> So the one-nonlinear-branch-per-package approach is unwieldy, but at least it is does not produce incorrect results until someone comes up with a better solution. As written above, the git-migration approach is fine for one-person or branch-less projects.
>
> Am So., 3. März 2019 um 09:23 Uhr schrieb Nicolas Cellier <[hidden email]>:

>
>
> Le dim. 24 févr. 2019 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
> Hi Doru,
>
> Le mar. 19 févr. 2019 à 10:39, Tudor Girba <[hidden email]> a écrit :

> Hi Nicolas,
>
>
> > On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:
>
> > Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> > https://github.com/hpi-swa/Squot
> > There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> > IMO, a broken history has not much interest.
>
> We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
> https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
>
>
> Ah OK, I'll try to have a look.
>
>
> Hi Doru,
> as follow up, I don't see the progress in package history.
> If I look for example at history of ThreadedFFIPlugin, then I see this:
>
> https://github.com/feenkcom/opensmalltalk-vm/commit/2ec3b0f6da1a5c7801f7ce8973213ba00b10d7fd#diff-36ccd126e0dd4f5ba2a399c962426080
>
> Normally I should see only one diff, like this:
>
> http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2109.diff
>
> This is because true parent is VMMaker.oscog-eem.2108
> But git parent is VMMaker.oscog-eem.2109
> So it seems like I am undoing all the changes from Eliot... Thats is obscuring the intentions of the commit.
>

--
www.feenk.com

"What we can governs what we wish."





Reply | Threaded
Open this post in threaded view
|

Re: VMMaker in Pharo [Was: Squeak and Tonel]

Jakob Reschke
In reply to this post by Tudor Girba-2
 
Hi,

Am Mo., 4. März 2019 um 15:24 Uhr schrieb Tudor Girba <[hidden email]>:
 
Do you have experience of using Squot on a project of the size of VMMaker? Would you like to try to see how it works? And does it work for Tonel, or only for FileTree?

At the moment, it is only FileTree, but the Tonel adapter is in the works. No, I have not used Squot on a project as large as VMMaker. Squot itself is probably medium-sized and is the largest project I use it on. I have also not used the Monticello converter on a larger history than that of FileSystem-Git and FileSystem for Squeak (their numbers were less than a tenth of those on VMMaker). The converter is probably slower than git-migration because it uses neither git-fast-import, nor libgit2. Since the history conversion is a seldom task, I did not bother to make it as quick as possible. I might try to convert the VMMaker ancestry to see how it goes and how long it takes. But since I have no personal projects in VMMaker, I would only evaluate how the conversion went and nothing more.
 

In any case, I also believe that it is not unreasonable to merge things in one linear history (or even to discard parallel Monticello branches). Here is my reasoning: the way the whole history in Monticello is, to a good extent, via the author timestamp. This essentially provides one change. So, for all practical purposes, preserving the main linear history is good enough.

It is good enough to continue working on the present state of a project, sure. But it will incite angriness if someone inspects the history (someone will do it) and runs into problems like Nicolas did. Once you have chosen to lose something (accurate ancestry) during the conversion, you won't be able to get it back so easily in the future. Moreover, you cannot easily rewrite the Git history again after you have shared it (in case you might change your mind and want to fix up the ancestry later). That's why I would be very cautious and advise against being satisfied with just "good enough". The Squot converter is not perfect in that regard either, as it currently loses the Monticello version names and UUIDs. Maybe I should better put these at the bottom of the commit messages.

Missing versions are a nuisance, too. But unless you know where to find them...

Best,
Jakob
 

> On Mar 4, 2019, at 8:42 AM, Jakob Reschke <[hidden email]> wrote:
>
> I had a quick look at the git-migration code. From what I saw, it binds all packages together in a linear history. Though the versions are topologically sorted by the Monticello ancestry, this approach cannot preserve any branches, or parallel changes, as Nicolas noted. Thus, it produces wrong diffs. Though, it is perfectly adequate for most one-person projects or projects with a Subversion-like commit policy ("you must always merge newer versions before you save"). I guess that is the majority of Monticello projects.
>
> For the converter in Squot, I chose to write one strand of commits for each package, ending in a branch. The history on each strand maps the package's Monticello ancestry accurately (except that the diffs for missing versions are still missing). But it is not appealing to work with each package on a different branch, of course. Unless you have Monticello configurations, that's all the model provides, is it? You can still get an idea of the inter-package history by viewing the log sorted by author timestamps (--author-date-order), but you cannot really bisect it, for example.
>
> What I did in my repositories so far is import the packages on separate branches and then create an n-way merge (with the command line tools) that pulls all the packages together from the point on when I established the Git repository.
>
> Binding the packages together in a common history needs some kind of heuristic that guesses the past configurations that were in use. For example, timestamps and authors could be used. Let's say contributors abc and xyz worked on some project in parallel and created versions in multiple packages. Then you could assume that PkgX-abc.8 and PkgY-abc.42 with similar timestamps belong to the same branch, while PkgX-xyz.8 (with the same parent as PkgX-abc.8) and PkgZ-xyz.5 with similar timestamps belong to a different branch. There will most certainly be scenarios where the heuristic breaks. For example, when the change in PkgZ-xyz.5 is also relevant for abc's work and she "invisibly" adopts it on her "branch" in Monticello. Or when abc works on her branch and then must quickly fix a bug on whatever "master" in Monticello is, but modifies a different package there (so you cannot see the "switch" in the ancestry).
>
> So the one-nonlinear-branch-per-package approach is unwieldy, but at least it is does not produce incorrect results until someone comes up with a better solution. As written above, the git-migration approach is fine for one-person or branch-less projects.
>
> Am So., 3. März 2019 um 09:23 Uhr schrieb Nicolas Cellier <[hidden email]>:

>
>
> Le dim. 24 févr. 2019 à 21:19, Nicolas Cellier <[hidden email]> a écrit :
> Hi Doru,
>
> Le mar. 19 févr. 2019 à 10:39, Tudor Girba <[hidden email]> a écrit :

> Hi Nicolas,
>
>
> > On Feb 18, 2019, at 3:12 PM, Nicolas Cellier <[hidden email]> wrote:
>
> > Also note that vmmaker history as proposed in opensmalltalk pr was broken (no respect of branch topology). Maybe retry with Squot?
> > https://github.com/hpi-swa/Squot
> > There is still a problem of synching commits of other packages (cog, various plugins ...), and if nobody is working on it, we are condemned to a mediocre solution.
> > IMO, a broken history has not much interest.
>
> We know it was broken, and thank you for pointing it out. We went through multiple iterations, and we think the latest one should be better. Please take a look and let us know if you still find issues:
> https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest5/vmmaker
>
>
> Ah OK, I'll try to have a look.
>
>
> Hi Doru,
> as follow up, I don't see the progress in package history.
> If I look for example at history of ThreadedFFIPlugin, then I see this:
>
> https://github.com/feenkcom/opensmalltalk-vm/commit/2ec3b0f6da1a5c7801f7ce8973213ba00b10d7fd#diff-36ccd126e0dd4f5ba2a399c962426080
>
> Normally I should see only one diff, like this:
>
> http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2109.diff
>
> This is because true parent is VMMaker.oscog-eem.2108
> But git parent is VMMaker.oscog-eem.2109
> So it seems like I am undoing all the changes from Eliot... Thats is obscuring the intentions of the commit.
>

--
www.feenk.com

"What we can governs what we wish."