Squeak and Tonel

Hi,

as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].

In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues
with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).

Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel
repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System)
to Tonel format [3].

So I wonder about adoption of Tonel in Squeak land.

Any status/comments?

Thx
T.

[1] https://github.com/pharo-vcs/tonel
[2] https://github.com/GemTalk/Rowan
[3] https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-tools-for-pharo6-1/

On Fri, Feb 15, 2019 at 8:58 AM Jakob Reschke <[hidden email]> wrote:

With these modest requirements, David, please try the port and open issues for anything that bugs you about it.

Am Fr., 15. Feb. 2019, 02:14 hat David T. Lewis <[hidden email]> geschrieben:

Hi Torsten,

Thank you very much for asking about this, and thanks to Nicolas for the
explanations.

As a package maintainer (OSProcess and others), I want to be able to
provide continued support for Pharo users, and it would probably be helpful
if I could read and write to Tonel as an external format. I am also interested
in being able to browse and read the projects (many of them quite excellent)
that are being developed in Pharo from my Squeak image.

I am a big fan of git, but I do not care about integrating it with the
image (Squeak/Pharo) because I prefer using the in-image tools in Squeak,
and when I do use git, I prefer to use git tools directly.

I believe Jakob's Git integration (see below) can also be controlled from within a workspace.

 

I am not well informed about this topic, but overall I would say that being
able to read and write Tonel format from Squeak would be very helpful, and
I would probably not care very much about version control integration.

Thanks again for asking,

Dave


On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
> I think that we should at least support the input/output of tonel packages
> for inter-operability.
> Since the git repositories are metadataless, we can't do much more without
> a lot of work.
>
> Since tonel has no support for timestamps, it looses authorship (it is
> replaced by committer-ship).
> If we don't want to loose authorship, then we need to port whole history to
> git with known initials <-> git(hub) account database.
> This is currently working for single package with complex graph (to
> filetree format, but could work for Tonel too).
> If you want to commit several MC packages in same git repository, then it's
> currently impossible to support complex history.
> There is generally no information that enables inter-MC-package
> synchronization (unless some sort of MCConfigurationMap is used, which at
> least gives a few sync point in history...).

Agreed. Too bad Tonel has become the default format in Pharo already. If we want to support it, we also have to live with it's flaws...

 

>
> If we want to perform essential operations like commit/pull/merge or access
> essential information as history/revisions/diffs then we have to
> - either reproduce the very hard work that began in Pharo: program a git
> client inside the image,
> - or loose inside image tools.
>
> Squeak MC tools are much more productive than Pharo MC tools:
> - they scale well even for massive changes (i did check with auto generated
> Smallapack interface)
> - they support cherry-picking: you see and control what you commit
> - they are quasi bugfree
> - the magma backend provides a few additional features (revisions...)
> Pharo team has produced huge effort for the github switch, but it does not
> go without pain. Despite the importance of being visible on github, and
> profit by state of the art web collaborating tools, i don't see Squeak able
> to take this major turn in near future.

Squeak's MC tools are great and I also thought Squeak can't support Git any time soon. It turns out Jakob has already done most of the heavy lifting as he worked on a Smalltalk implementation of Git. It even comes with a Sourcetree-like UI which supports the most important operations already. In case you haven't seen it, you can install it via "Do" > "Installer installGitInfrastructure" (Squeak-5.2 or later). You can then find it's UI under "Apps" > "Git Browser". Please file bugs at [1]. Unfortunately, the bus factor of this integration is still quite low, but I'd like to see more people working on it.

Fabio

[1] https://github.com/hpi-swa/Squot/issues

 

>
> If we want to have mix development with packages maintained both in a MC
> repository and a git metadataless repository,
> then we have the problem of finding a common ancestor.
> I think that this could be doable with some conventions (like storing the
> .last_known_mc_ancestor in a file)
> Of course, it's re-introduction of metadata, but the minimal possible, and
> leads to easy to resolve conflicts (that can be automated)...
>
> The last grief I have with Tonel is that it is not syntax agnostic.
> Since we have a language where we can define compilerClass/parserClass,
> that's unfortunate (The application i developed in the 90's did use such
> important feature, it's not just theoretical)
> It could have been language agnostic with very little effort (remember the
> discussion on Pharo-dev, just using an arbitrary number of [[[   ]]] as
> method body separators would suffice)
> For most packages, that will not be a problem though.
>
>
> Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann <[hidden email]> a ??crit :
>
> > Hi,
> >
> > as some of you might already know "Tonel" is a file-per-class format for
> > monticello repositories [1].
> >
> > In Pharo land many projects are already converted to it - as regular "file
> > tree" solution faces issues
> > with git handling due to very long file names / deep file hierarchy (often
> > leading to trouble on Windows).
> >
> > Dales's Rowan (a new project/package manager for Smalltalk) supports
> > FileTree and also Tonel
> > repositories [2]. There is also a tool to convert from STORE (VisualWorks
> > Source-Code Versioning System)
> > to Tonel format [3].
> >
> > So I wonder about adoption of Tonel in Squeak land.
> >
> > Any status/comments?
> >
> > Thx
> > T.
> >
> > [1] https://github.com/pharo-vcs/tonel
> > [2] https://github.com/GemTalk/Rowan
> > [3]
> > https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-tools-for-pharo6-1/
> >
> >
> >
> >

>

BTW Eliot, I just discovered by accident emails you sent me last week there are full on insults . I will not forward them in this mailing-list because it does not help. 

This is not the first that you do that to me. The first time I went over them and supported

Clément to work with you. Now I will not consider your mails anymore. 

Sorry but you cannot talk publicly about collaboration and privately insult people that way. 

Now if you believe that I do pharo about ego then you do not understand anything :)

And I will not try to convince you otherwise. This is my last email sent to you. 

Stef

On 17 Feb 2019, at 21:59, ducasse <[hidden email]> wrote:

Hi eliot

What is common in your thread is that we always look like the super emotional, or bad guys. 

You never ask yourselves why Esteban left the VM mailing-list. 

And I do not see why you cannot fork Tonel to produce your own version. 

Git is about distributed projects. People do that all the time. You can take Tonel and hack it to death

without having your hacks pushed back into Pharo and this is perfectly ok for us. 

So Esteban has the right to say no and you have the right to hack your own version. 

If you want to have some support for porting Scorch/Sista to github and loading you should ask 

but asking it more nicely. 

About the simulation of the UI, if you do not talk to us we cannot pay attention.

I do not see why we cannot keep a package with the image level UI for the simulator. 

Now again you are talking about collaboration but you do not talk to us and you do not 

listen to us so do not expect that people are willing to spend their free time to help.

I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else”, 

This is fun to ask Pharo not to grow up to use modern technology to manage source code under the 

premisses that Squeak cannot or will not. 

This is so funny how you state it. I will not comment more than that.

People around are adults and they will be able to judge by themselves. 

and considering leaving VMMaker altogether. 

If you leave VMMaker, let us know because we will port it to Github and make it work in Pharo. 

Esteban did it several times in the past. At least this will have the benefit to clarify the situation. 

I think that this is good that you tell us that you do not want to cooperate with us. 

This will have at least the impact to kick our ass and pay attention to us. 

The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.

Here's the kind of crap people like Ducasse throw at me:

"Eliot 

At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy

because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have 

to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world 

of the “open” smalltalk VM. "

I am so over this crap.

This is not a crap. I can restate what I said. Because of responsibility of a research team and creating a future for students. You know I’m not alone, I have quite some responsibility towards PhD students

of my group and yes I cannot make them fail by construction (or produce unadequate results)

just by imposing them to use a system with far too many constraints. 

I will not ask them to work on the openSmalltalk vm directly because this is not their responsibilities

to have to jump over newspeak ifdef and others.  A job of a PhD is to be able to brainstorm and create new ideas. Look at Stefan Marr (He is working on SOM or Truffle). 

Now you can think that I’m an asshole, arrogant, or I do not know pick what you want.

***I do not care***. I do not have Diva syndrome I have responsibilities towards people. 

I was discussing with some truffle experts and they told me that this is can be complex. I believe it. 

I would like to avoid pypy for obvious reasons. So what we will probably do for their PhD is to see if we can use a light version of opensmalltalk. I do not want to ask them to jump over many things that are totally useless for them all the time. 

PS: personnally I do not get why VMMaker would be the only project on earth that cannot be managed using git and Pharo. 

But for us the future is Git and we will continue to build on this infrastructure. 

Stef

_,,,^..^,,,_

best, Eliot

Hi eliot

What is common in your thread is that we always look like the super emotional, or bad guys. 

You never ask yourselves why Esteban left the VM mailing-list. 

And I do not see why you cannot fork Tonel to produce your own version. 

Git is about distributed projects. People do that all the time. You can take Tonel and hack it to death

without having your hacks pushed back into Pharo and this is perfectly ok for us. 

So Esteban has the right to say no and you have the right to hack your own version. 

If you want to have some support for porting Scorch/Sista to github and loading you should ask 

but asking it more nicely. 

About the simulation of the UI, if you do not talk to us we cannot pay attention.

I do not see why we cannot keep a package with the image level UI for the simulator. 

Now again you are talking about collaboration but you do not talk to us and you do not 

listen to us so do not expect that people are willing to spend their free time to help.

I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress.  I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else”,

This is fun to ask Pharo not to grow up to use modern technology to manage source code under the 

premisses that Squeak cannot or will not. 

This is so funny how you state it. I will not comment more than that.

People around are adults and they will be able to judge by themselves. 

and considering leaving VMMaker altogether. 

If you leave VMMaker, let us know because we will port it to Github and make it work in Pharo. 

Esteban did it several times in the past. At least this will have the benefit to clarify the situation. 

I think that this is good that you tell us that you do not want to cooperate with us. 

This will have at least the impact to kick our ass and pay attention to us. 

The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.

Here's the kind of crap people like Ducasse throw at me:

"Eliot 

At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy

because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have 

to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world 

of the “open” smalltalk VM. "

I am so over this crap.

This is not a crap. I can restate what I said. Because of responsibility of a research team and creating a future for students. You know I’m not alone, I have quite some responsibility towards PhD students

of my group and yes I cannot make them fail by construction (or produce unadequate results)

just by imposing them to use a system with far too many constraints. 

I will not ask them to work on the openSmalltalk vm directly because this is not their responsibilities

to have to jump over newspeak ifdef and others.  A job of a PhD is to be able to brainstorm and create new ideas. Look at Stefan Marr (He is working on SOM or Truffle). 

Now you can think that I’m an asshole, arrogant, or I do not know pick what you want.

***I do not care***. I do not have Diva syndrome I have responsibilities towards people. 

I was discussing with some truffle experts and they told me that this is can be complex. I believe it. 

I would like to avoid pypy for obvious reasons. So what we will probably do for their PhD is to see if we can use a light version of opensmalltalk. I do not want to ask them to jump over many things that are totally useless for them all the time. 

PS: personnally I do not get why VMMaker would be the only project on earth that cannot be managed using git and Pharo. 

But for us the future is Git and we will continue to build on this infrastructure. 

Stef

_,,,^..^,,,_

best, Eliot

I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg.  I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent.  Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak.  I asked Esteban and was met by a flat no.  Not "OK, but this is an option only you will use", but a flat "not under any circumstance".

So here I am joining in this thread reluctantly.  I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects.  Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch.  The port *does not work*.  Metacello fails to load Scorch.  Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me.  So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.