Re: Squeak, Tonel, and the Tonel standard
Posted by
Martin McClure-2 on
Aug 27, 2020; 1:39am
URL: https://forum.world.st/Git-and-Tonel-and-Magritte-tp5120850p5120928.html
Good question. Why a thing like Tonel,
instead of just Smalltalk?
The short answer is that Smalltalk doesn't have syntax for
anything larger than a method.
Filetree creates one file per method, so each file can be just
Smalltalk. Filetree doesn't work well for large bodies of code --
operating systems tend to allocate a minimum of 4K to each file.
Since most methods are *much* shorter than 4K, most of the space
is unused. If you have a million methods (and some of my customers
*do* have a million methods) the space requirements are
prohibitive. The answer is to put more than one method into most
files.
When you start to pack more than one method into the same file,
you need some kind of syntax beyond Smalltalk to tell you where
each method begins and ends, and other meta-information like class
definitions and whether a method is a class method or an instance
method and what category it belongs in is also needed. Smalltalk
doesn't have syntax for that.
Tonel tries to keep the methods themselves as much pure Smalltalk
and as readable as possible. The extra syntax is needed for the
things outside the methods.
Note that Tonel is *not* chunk format. Chunk format dates back to
Smalltalk-80, and is not part of Tonel. No '!!' in Tonel files.
:-)
Chunk format did exist for a similar reason, though, that some
kind of syntax beyond Smalltalk is needed if you want to put more
than one method in a file. Chunk format was imperative for the
things outside of methods (things like class definitions are
essentially do-its). This has led to serious awkwardness over the
years, so Tonel is entirely declarative.
Hope that helps,
-Martin
On 8/26/20 5:04 PM, Kjell Godo wrote:
i realize that you want to keep the ball rolling
and all
and not waste the previous efforts
but why make a special Tonel syntax ?
why not just use Smalltalk and String literals
Number literals etc
then it will be easy to change
and isn’t this whole chunk format
that shows up in Package Files
an artifact of the limited hardware of yor
or i can’t tell why they used it instead of just
using Smalltalk
i can’t see why you wouldn’t just use Smalltalk
instead of these ! ! bang things
and all are there some C style hard limits
that can run you afoul ?
no there are not
but this is not helping to keep the ball rolling
no
Hi Jakob,
Thanks for your interest. I'll send you the draft under
separate cover.
Putting it on GitHub is probably the best option -- I'll try
to get that
set up within the next couple of days, and announce it on
the list when
I do.
Regards,
-Martin
On 8/26/20 1:04 PM, Jakob Reschke wrote:
> Hello Martin,
>
> While I probably won't be able to provide comprehensive
feedback or
> even read the whole text in the near future, I would
certainly be
> interested to have a look.
>
> What is the licensing status of the draft and future
proposed
> standard? If it is in some plain text/markup format, is
it possible to
> upload the draft to a public GitHub repository? Or
something similar
> that allows for commenting on changes. On GitHub one
can only comment
> on diffs, but the initial commit will add all the
lines, so one could
> still comment on all of the text. Also people could
provide suggested
> changes via pull requests.
>
> Kind regards,
> Jakob
>
> Am Mi., 26. Aug. 2020 um 21:20 Uhr schrieb Martin
McClure
> <[hidden email]>:
>> Hi all,
>>
>> Dale let me know this morning that Tonel was being
discussed on
>> Squeak-dev. I beg your pardon for arriving late.
>>
>> As Dale said, I'm the minder of the Tonel 1.0
standard, which is
>> currently in draft, with initial feedback from some
of the folks who
>> have implemented or are interested in implementing
Tonel for their
>> Smalltalk dialect.
>>
>> I'm interested in getting feedback from *all*
dialects who are
>> interested in using Tonel. I hope that the Squeak
community will
>> participate.
>>
>> Tonel started as a collaboration between Pharo
(primarily Esteban
>> Lorenzano) and GemTalk (primarily me), but the
intent has always been to
>> have a format that can be used in common between
*all* dialects of
>> Smalltalk. The fundamental shape of Tonel is
probably not going to
>> change at this point, but feedback so far is likely
to result in some
>> changes to the draft standard.
>>
>> Note that Tonel as currently implemented by Pharo
is a bit different
>> from standard Tonel. Pharo has agreed to make
several changes to the
>> format to standardize it. I'm not sure what the
status of those changes
>> within the Pharo project. The GemStone
implementation is closer to the
>> standard, though not fully there yet.
>>
>> I encourage the Squeak community to consider
whether coding to the
>> standard early on might be a desirable direction.
As Dale explained, the
>> standard does allow for dialect-specific
properties, so adding
>> timestamps probably doesn't require any changes to
the spec.
>>
>> I'd like it If a few folks from the community would
volunteer to give
>> the spec a careful read-through and comment. It's a
bit of a read (and
>> was quite a bit of effort to write!) so I won't
post the draft on the
>> list, but it's available by request, and I need to
figure out a place to
>> make it readily available.
>>
>> Regards,
>> -Martin
>>
>>
>> On 8/26/20 9:41 AM, Dale Henrichs wrote:
>>> Eliot,
>>>
>>> When you repost, could you make sure that you
include a description of
>>> where the time stamps are stashed ... I'm
assuming that you are adding
>>> it to the method properties (where the category
is stashed) as I think
>>> that this is the right place, but it's always
best not to guess:)
>>>
>>> The current crop of Tonel readers/writers do
not necessarily do a good
>>> job of preserving, foreign data (IIRC, the
convention is to add a
>>> platform prefix to the property name, which is
the same convention
>>> used for class properties ... at GemStone we
add a leading `gs_` to
>>> our property names), so until we can get all of
the different Tonel
>>> readers/writers to preserve foreign properties
(package, class and
>>> method) it will continue to be dicey business
for preserving foreign
>>> properties in cross platform projects ...
>>>
>>> Monticello does not explicitly preserve foreign
properties as the
>>> definitions get created from the native
objects, so it takes some
>>> additional work to arrange to preserve foreign
properties for
>>> packages, classes, and methods in the objects
themselves. So it is
>>> worth considering what you will do with
preserving foreign properties
>>> from other platforms.
>>>
>>> FWIW, I intend to support the preservation of
foreign properties in
>>> Rowan (read that as "I haven't done that yet":)
>>>
>>> Finally, Martin McClure has started working on
a spec for the next
>>> generation Tonel format ... to add a few
missing pieces and tweak the
>>> format to make it possible to continue to
evolve the Tonel format into
>>> the future in a somewhat sane way. If there are
folks here in the
>>> Squeak community who would like to review,
comment and participate in
>>> the creation of the next generation format,
send me (or Martin) mail
>>> and we'll get you added to the mailing list.
>>>
>>> Dale
>>>
>>> On 8/26/20 7:15 AM, Eliot Miranda wrote:
>>>> Jakob,
>>>>
>>>>> On Aug 26, 2020, at 2:45 AM, Jakob
Reschke <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Hi Eliot,
>>>>>
>>>>>> Am Mi., 26. Aug. 2020 um 11:20 Uhr
schrieb Eliot Miranda
>>>>>> <[hidden email]>:
>>>>>>
>>>>>> If we go with Tonel then we must
change it to support method
>>>>>> timestamps. I have a change set
that does this (it is a trivial
>>>>>> change). And my changes are
controlled by a class variable so that
>>>>>> the same code can produce Pharo
format or a slightly modified
>>>>>> format that includes method
timestamps.
>>>>>>
>>>>>> What I don’t understand is why
Esteban Lorenzano refuses to accept
>>>>>> my changes and allow Tonel to be
used either with or without method
>>>>>> timestamps.
>>>>>>
>>>>> Method timestamps produce merge
conflicts inevitably.
>>>> a) only when they change, and they change
only when methods change
>>>> b) inadvertent method timestamp changes can
be undone automatically
>>>> c) Squeak uses method timestamps; we have
lots of tools that use
>>>> them. Dropping them just so that we can use
Tonel is an example of
>>>> the tail wagging the dog. The Tonel
interface must instead be made to
>>>> function with method timestamps
>>>>
>>>> If there is a decision not to support
method timestamps then I will
>>>> not support the work. This is a make or
break issue for me.
>>>>
>>>>> Can you re-post
>>>>> your changeset here?
>>>> I’ll post it later today (reading email in
bed right now).
>>>>
>>>>> https://github.com/squeak-smalltalk/squeak-tonel/issues
>>>> OK
>>>>
>>>>> Then we could
>>>>> discuss whether to include it at least
in the Squeak version. As we
>>>>> have heard from Mariano some time ago,
VA Smalltalk also puts its
>>>>> dialect-specific metadata in the Tonel
format, so Squeak would not be
>>>>> the first to do so.
>>>> Good.
>>>>
>>>>> Kind regards,
>>>>> Jakob
>>