[squeak-dev] Making a better Compiler for all

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Making a better Compiler for all

Igor Stasenko
Hello folks,

during my visit at ESUG i had a conversation with Stephane, about
different things, one of them is to make a better and cleaner compiler
implementation.

Current compiler code contains a lot of hacks and to be fair, is not a
best example of good smalltalk code.
What i'm proposing, is to establish a comittee for designing a better
smalltalk compiler , which will replace old one on all existing Squeak
forks.
I think comittee should collect & discuss all feature requests, which
new compiler should have, and then we can implement it and include in
every and all forks.
It is also, IMO, would be a good to make compiler modular - put it as
separate package and maintain it as separate fork-independent entity.
This will ensure that all existing forks in future will use the same
compiler as the rest. Which will make people's life much easier :)

I hope with support of atomic loading (btw a new compiler is a good
test-bed for atomic loading) we should be able to load it at will :)

--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Making a better Compiler for all

David Zmick
OMeta?

On Sun, Aug 31, 2008 at 10:19 AM, Igor Stasenko <[hidden email]> wrote:
Hello folks,

during my visit at ESUG i had a conversation with Stephane, about
different things, one of them is to make a better and cleaner compiler
implementation.

Current compiler code contains a lot of hacks and to be fair, is not a
best example of good smalltalk code.
What i'm proposing, is to establish a comittee for designing a better
smalltalk compiler , which will replace old one on all existing Squeak
forks.
I think comittee should collect & discuss all feature requests, which
new compiler should have, and then we can implement it and include in
every and all forks.
It is also, IMO, would be a good to make compiler modular - put it as
separate package and maintain it as separate fork-independent entity.
This will ensure that all existing forks in future will use the same
compiler as the rest. Which will make people's life much easier :)

I hope with support of atomic loading (btw a new compiler is a good
test-bed for atomic loading) we should be able to load it at will :)

--
Best regards,
Igor Stasenko AKA sig.




--
David Zmick
/dz0004455\
http://dz0004455.googlepages.com
http://dz0004455.blogspot.com


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Making a better Compiler for all

Ralph Johnson
In reply to this post by Igor Stasenko
On Sun, Aug 31, 2008 at 10:19 AM, Igor Stasenko <[hidden email]> wrote:
> Hello folks,
>
> during my visit at ESUG i had a conversation with Stephane, about
> different things, one of them is to make a better and cleaner compiler
> implementation.

I thought this was the purpose of the NewCompiler project.  Is that
project not good enough?  Or is this a request to make it standard?

-Ralph

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Making a better Compiler for all

Klaus D. Witzel
On Mon, 01 Sep 2008 03:59:44 +0200, Ralph Johnson wrote:

> On Sun, Aug 31, 2008 at 10:19 AM, Igor Stasenko wrote:
>> Hello folks,
>>
>> during my visit at ESUG i had a conversation with Stephane, about
>> different things, one of them is to make a better and cleaner compiler
>> implementation.
>
> I thought this was the purpose of the NewCompiler project.

No, you cannot remove Squeak's compiler and use only NewCompiler, if that  
would have been your question.

/Klaus

>  Is that
> project not good enough?  Or is this a request to make it standard?
>
> -Ralph
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Igor Stasenko
2008/9/1 Klaus D. Witzel <[hidden email]>:

> On Mon, 01 Sep 2008 03:59:44 +0200, Ralph Johnson wrote:
>
>> On Sun, Aug 31, 2008 at 10:19 AM, Igor Stasenko wrote:
>>>
>>> Hello folks,
>>>
>>> during my visit at ESUG i had a conversation with Stephane, about
>>> different things, one of them is to make a better and cleaner compiler
>>> implementation.
>>
>> I thought this was the purpose of the NewCompiler project.
>
> No, you cannot remove Squeak's compiler and use only NewCompiler, if that
> would have been your question.
>
> /Klaus
>
>>  Is that
>> project not good enough?  Or is this a request to make it standard?
>>

Yes, the main question about it, why it didn't replaced old compiler?

There are some bad things in current compiler, which makes it really
hard to play with:
try implement a visitor pattern with its AST - you will go into
trouble very soon, because some nodes properties is not fully exposed,
or accessible in a cryptic way. That's why VMMaker, for instance,
using an extensions/patching to be able to reach all interesting
properties of parse nodes.
A bytecode generation is spead among AST nodes, and interesting, that
you can't insert some kind of analyzer or pre/postprocessor without
patching methods, because there is no separation in responsibility who
emits the bytecodes.A MethodNode returns you a hot baked
CompiledMethod instance. It would be good to make a bytecode
generation placed in separate class , which using a visitor pattern to
generate a bytecode.
There are also an agressive rewrite rules, sitting at early processing
stage which replacing original selectors like #ifTrue:{ifFalse:} or
#to:do: , so you can't see an original parsed AST from source code,
but getting already twisted preoptimized tree , which not reflects an
original source 1:1.

>> -Ralph
>>

--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Ralph Johnson
In reply to this post by Klaus D. Witzel
On Mon, Sep 1, 2008 at 2:13 AM, Klaus D. Witzel <[hidden email]> wrote:

> No, you cannot remove Squeak's compiler and use only NewCompiler, if that
> would have been your question.

That was not really my question.

I thought that the purpose of the NewCompiler project was to replace
the old compiler.  Am I wrong?

If so, is this just another way of telling people to support the
NewCompiler project, or is it saying that we need a NewNewCompiler
project.

If not what is wrong with NewCompiler, and why can't it become the
standard Squeak compiler some day?

If you are saying we should all get behind NewCompiler, then we can
ask what needs to be done to finish it, but first I am asking whether
that is what you mean.

I've hacked the Squeak compiler a little and I think that everybody
who has done that will agree that it should be replaced.  As far as I
can see, the NewCompiler people have a good design, and it should be
possible use it in place of the old compiler, if not now, then with a
little more work.

-Ralph

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Brent Pinkney-2
> On Mon, Sep 1, 2008 at 2:13 AM, Klaus D. Witzel <[hidden email]> wrote:
>
> > No, you cannot remove Squeak's compiler and use only NewCompiler, if that
> > would have been your question.

May I urge that all efforts include Eliot's Cog work. He is making changes to the compiler to support closures and the Cog VM.

--
Brent

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Making a better Compiler for all

Klaus D. Witzel
In reply to this post by Ralph Johnson
On Mon, 01 Sep 2008 15:09:31 +0200, Ralph Johnson wrote:

> On Mon, Sep 1, 2008 at 2:13 AM, Klaus D. Witzel wrote:
>
>> No, you cannot remove Squeak's compiler and use only NewCompiler, if  
>> that
>> would have been your question.
>
> That was not really my question.

NP, and sorry for disturbing your cycles.

> I thought that the purpose of the NewCompiler project was to replace
> the old compiler.  Am I wrong?

This would be interesting to find out, what "to replace" plans where  
there, also from a "managable" POV. I'm afraid I don't know any historical  
records which detail such plan, except the "can replace" wording mentioned  
in

-  
http://wiki.squeak.org/squeak/search?search=newcompiler&casesensitive=false&and=true

Anybody have more/other pointers?

> If so, is this just another way of telling people to support the
> NewCompiler project, or is it saying that we need a NewNewCompiler
> project.

No, I'm in favor of Igor's suggestion: make a new compiler. And ask people  
what they want in it, possible answers (but no limited to)

o faster compilation (faster than what? *)
o don't aim to be a framework for *everything*
o just be reliable and do the job
o decompilation from CompiledMethod guaranteed (!)
o very good syntax error messages/selections
o very good spelling/correction support
o no more "private" public hacks, because:
o extensibility/dialects only by subclassing (or so, Traits anybody?)
o better/more optimization of compiled methods?
o backward compatability at the bytecode level?
o evaluation restricted to what properly does #storeOn:?
o clear rules for name-spaceing and modularizing things?
o my favorite: less knowledge about how other classes work ;)

*) there are some 123 classes in NewCompiler & friends (not counting some  
roots like SmaCCParser) , versus some 30 classes in old compiler & friends.

> If not what is wrong with NewCompiler, and why can't it become the
> standard Squeak compiler some day?

Because too much packages depend on what/how old compiler makes/does  
things? (my POV depends on my use of things, YMMV).

For a prominent example, VMMaker cannot make much use of NewCompiler  
without rewriting the most vital parts of it almost from scratch (IMHO).  
But VMMaker is needed for getting the next VM out.

> If you are saying we should all get behind NewCompiler, then we can
> ask what needs to be done to finish it, but first I am asking whether
> that is what you mean.

I mean, that Squeak community cannot have both: complains about too many  
hacks being/"unmaintainable" in the old compiler and, be rather silent  
when NewCompiler happens to be mentioned.

That would be a huge waste of time and effort and a very dark & sticky &  
disgusting shadow over things IMHO.

The compiler is too important for letting business go as usual (e.g. let's  
wait what happens and if nothing happens then do nothing; no offense, we  
all have our full plate). That's how I think about it.

> I've hacked the Squeak compiler a little and I think that everybody
> who has done that will agree that it should be replaced.

Yes, replaced, made obsolete, so that nothing can depend on it any longer.  
Then it can be put on the downloadable package list, for people who want  
it and/or want to maintain it. Good move, that is.

> As far as I
> can see, the NewCompiler people have a good design, and it should be
> possible use it in place of the old compiler, if not now, then with a
> little more work.

Then why is it not used in a promising project like COG, right from the  
beginning? This rather rhetorical question has nothing against Eliot, I  
understand the answer he blogged.

/Klaus

> -Ralph
>
>



Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Making a better Compiler for all

Klaus D. Witzel
In reply to this post by Brent Pinkney-2
On Mon, 01 Sep 2008 18:06:00 +0200, Brent Pinkney wrote:

>> On Mon, Sep 1, 2008 at 2:13 AM, Klaus D. Witzel wrote:
>>
>> > No, you cannot remove Squeak's compiler and use only NewCompiler, if  
>> that
>> > would have been your question.
>
> May I urge that all efforts include Eliot's Cog work. He is making  
> changes to the compiler to support closures and the Cog VM.

Yes, sure. A good compiler supports *any* underlying instruction set and  
architecture (within a predetermined, plan-able framework), even the  
historical ones if someone wants to afford that.

I'm used to that in my shop; I think that everybody is.

>



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Eliot Miranda-2
In reply to this post by Igor Stasenko
Please have a look at my closure compiler.  This puts bytecode generation into a separate class (a subclass of Encoder).
You can find the code on the download page of my blog.

Marcus Denker and I are in agreement that we will merge my Closure compiler into Markus' NewCompiler "some time soon".

The thought of a committee fills me with horror :)  I don't think we need anything so baroque :)

Best
Eliot

On Mon, Sep 1, 2008 at 3:43 AM, Igor Stasenko <[hidden email]> wrote:
2008/9/1 Klaus D. Witzel <[hidden email]>:
> On Mon, 01 Sep 2008 03:59:44 +0200, Ralph Johnson wrote:
>
>> On Sun, Aug 31, 2008 at 10:19 AM, Igor Stasenko wrote:
>>>
>>> Hello folks,
>>>
>>> during my visit at ESUG i had a conversation with Stephane, about
>>> different things, one of them is to make a better and cleaner compiler
>>> implementation.
>>
>> I thought this was the purpose of the NewCompiler project.
>
> No, you cannot remove Squeak's compiler and use only NewCompiler, if that
> would have been your question.
>
> /Klaus
>
>>  Is that
>> project not good enough?  Or is this a request to make it standard?
>>

Yes, the main question about it, why it didn't replaced old compiler?

There are some bad things in current compiler, which makes it really
hard to play with:
try implement a visitor pattern with its AST - you will go into
trouble very soon, because some nodes properties is not fully exposed,
or accessible in a cryptic way. That's why VMMaker, for instance,
using an extensions/patching to be able to reach all interesting
properties of parse nodes.
A bytecode generation is spead among AST nodes, and interesting, that
you can't insert some kind of analyzer or pre/postprocessor without
patching methods, because there is no separation in responsibility who
emits the bytecodes.A MethodNode returns you a hot baked
CompiledMethod instance. It would be good to make a bytecode
generation placed in separate class , which using a visitor pattern to
generate a bytecode.
There are also an agressive rewrite rules, sitting at early processing
stage which replacing original selectors like #ifTrue:{ifFalse:} or
#to:do: , so you can't see an original parsed AST from source code,
but getting already twisted preoptimized tree , which not reflects an
original source 1:1.

>> -Ralph
>>

--
Best regards,
Igor Stasenko AKA sig.




Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Edgar J. De Cleene



El 9/1/08 5:39 PM, "Eliot Miranda" <[hidden email]> escribió:

> The thought of a committee fills me with horror :)  I don't think we need
> anything so baroque :)
>
> Best
> Eliot


Peron said:
If you wish nothing happens, nominate a comitee



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Igor Stasenko
In reply to this post by Eliot Miranda-2
2008/9/1 Eliot Miranda <[hidden email]>:
> Please have a look at my closure compiler.  This puts bytecode generation
> into a separate class (a subclass of Encoder).

This is one of the things we really need to be decoupled.

> You can find the code on the download page of my blog.
> Marcus Denker and I are in agreement that we will merge my Closure compiler
> into Markus' NewCompiler "some time soon".
> The thought of a committee fills me with horror :)  I don't think we need
> anything so baroque :)

Well, i only hope, that you will not need to say to people words like:
'this stuff not working because you need my compiler to use it'.
Or words like: 'this stuff works only in mine fork'.
Maybe it is not hurts someone losing a potential contributors and
splitting community over and over. But i think having something in
common, like compiler is for best of all.
Maybe i'm not right - it is always easier to manage a small group of
contributors which never asking anyone's opinion how to do things, and
in result they deliver things like Morphic and Etoys (sorry :) which
nearly impossible to decouple from the rest of system, because
obviously no one thought about things like running squeak w/o Morphic
before.

And me, as a Squeak board member, as well as others were elected on
the wave of making Squeak having better modularity. The lack of
modularity in Squeak is one of the bigger issues in system, which
prevents it to be easily managed and adopted for different solutions.
One of the critical steps towards modularity is to make better,
modular compiler. Of course, we can wait a little and start using
NewSpeak, so there is nothing to do or worry about, isnt? ;)

> Best
> Eliot
>


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Igor Stasenko
In reply to this post by Edgar J. De Cleene
2008/9/2 Edgar J. De Cleene <[hidden email]>:

>
>
>
> El 9/1/08 5:39 PM, "Eliot Miranda" <[hidden email]> escribió:
>
>> The thought of a committee fills me with horror :)  I don't think we need
>> anything so baroque :)
>>
>> Best
>> Eliot
>
>
> Peron said:
> If you wish nothing happens, nominate a comitee
>
You both right. My intent was only to get people's attention on this
problem and discuss different points of view and their needs.
Then implementors (like me) making things need to be done. Thats all.
If no-one sharing the thought that we need a better compiler at all,
then i can simply take away my words, and we will stay where we still
sitting. :)



--
Best regards,
Igor Stasenko AKA sig.


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Alejandro F. Reimondo
In reply to this post by Edgar J. De Cleene
Dear Edgar,

>Peron said:
>If you wish nothing happens, nominate a comitee

The problem is that it is imposible today that "nothing" happen...
 and most problems are visible after time... as seen in the
 country of Peronismo.

> The thought of a committee fills me with horror :)  I don't think we need
> anything so baroque :)

Eliot: you must also firm a license agreement ...

Ale.



----- Original Message -----
From: "Edgar J. De Cleene" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Monday, September 01, 2008 6:02 PM
Subject: Re: [squeak-dev] Re: Making a better Compiler for all





El 9/1/08 5:39 PM, "Eliot Miranda" <[hidden email]> escribió:

> The thought of a committee fills me with horror :)  I don't think we need
> anything so baroque :)
>
> Best
> Eliot


Peron said:
If you wish nothing happens, nominate a comitee






Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Alejandro F. Reimondo
In reply to this post by Igor Stasenko
Hi,

> Maybe i'm not right - it is always easier to manage a small group of
> contributors which never asking anyone's opinion how to do things, and
> in result they deliver things like Morphic and Etoys (sorry :) which
> nearly impossible to decouple from the rest of system, because
> obviously no one thought about things like running squeak w/o Morphic
> before.

I think it show that they can´t doit in other way, and it is good to
 work the best way for the "author/worker".
The problem is what is missing that make people think
 they can´t decouple (anyPartOfTheSystem) by themselves.
imho it reveals a failure in education efforts -download aFish
 instead of try fishing- and is a problem of this times (too easy/fast
 to fileIn and doIt).

A compiler framework is described as something we want,
 but we need more than two instances/compilers to start to
 think we have a framework (it can´t be invented).
A way to make a framework emerge is to have more than
 one person/group working on alternatives and provide a set
 of open topDown requirements in form of tutorials/exemplars
 of use of Smalltalk compilation solutions.
A swiki page with guides on how people want to
 use aNewSmalltalkCompiler may be used to guide people
 interested in write a concrete support.

best,
Ale.


----- Original Message -----
From: "Igor Stasenko" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Monday, September 01, 2008 6:08 PM
Subject: Re: [squeak-dev] Re: Making a better Compiler for all


> 2008/9/1 Eliot Miranda <[hidden email]>:
>> Please have a look at my closure compiler.  This puts bytecode generation
>> into a separate class (a subclass of Encoder).
>
> This is one of the things we really need to be decoupled.
>
>> You can find the code on the download page of my blog.
>> Marcus Denker and I are in agreement that we will merge my Closure
>> compiler
>> into Markus' NewCompiler "some time soon".
>> The thought of a committee fills me with horror :)  I don't think we need
>> anything so baroque :)
>
> Well, i only hope, that you will not need to say to people words like:
> 'this stuff not working because you need my compiler to use it'.
> Or words like: 'this stuff works only in mine fork'.
> Maybe it is not hurts someone losing a potential contributors and
> splitting community over and over. But i think having something in
> common, like compiler is for best of all.
> Maybe i'm not right - it is always easier to manage a small group of
> contributors which never asking anyone's opinion how to do things, and
> in result they deliver things like Morphic and Etoys (sorry :) which
> nearly impossible to decouple from the rest of system, because
> obviously no one thought about things like running squeak w/o Morphic
> before.
>
> And me, as a Squeak board member, as well as others were elected on
> the wave of making Squeak having better modularity. The lack of
> modularity in Squeak is one of the bigger issues in system, which
> prevents it to be easily managed and adopted for different solutions.
> One of the critical steps towards modularity is to make better,
> modular compiler. Of course, we can wait a little and start using
> NewSpeak, so there is nothing to do or worry about, isnt? ;)
>
>> Best
>> Eliot
>>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Kjell Godo
In reply to this post by Igor Stasenko
Where is this new compiler project?  Where is NewCompiler?  I would like to see it.
Does anybody know where that book about the Squeak Compiler went to?
 
the rest down below is all nonsense and I wouldn't read it if I were you.
 
i knew this was going to cost me.
 
What is atomic loading?  Does it mean no dependencies or dependencies are handled?
It seems to me that there needs to be some kind of intellegent dependencies manager that works a lot better and is a lot smarter than what has been put out there so far.
 
How can I learn about how a good Squeak compiler works?  Without years and millions of dead hours?
 
Modularity is very good.  I think that all of Squeak should be very self explaining.  This can be done if you put your explanations of what is going on after the body of the method.  Colored source is good too.  See Dolphin.  But without reformating.
 
I am making picoLARC on sourceforge.net.  Each lisp/smalltalk expression gets compiled by an instance of a Compiler Class.  Each expression( let if define etc ) has its own KEGLambdaLispCompiler subClass with one standard method and zero or more helper methods in it.  Each Compiler outputs an instance of a subClass of the Eval Class.  An Eval can be evaluated at runtime by >>evalWithActivationRec: or it could generate byte codes which do the same thing via some method like
EvalSubClass>>generateByteCodesOn:usingCodeWalker: where the CodeWalker could tie Evals together or do optimizations?  Is this not a good design?  I know I like the part about one Compiler Class for each expression and one corresponding Eval Class.  But I haven't done any byte code generation yet so I don't know about that part.  One Compiler per Eval is not strict.  The ApplicationCompiler can output several different related kinds of Evals for the different function calls and message sends.
 
What is this visitor pattern?  I don't like the idea of putting byte code generation into a single Class.  But I feel like maybe I don't know what I'm talking about.  To modify the byte code generation for an expression you would subClass the Eval Class and modify the
>>generateByteCodeOn: aCodeStream.  The initial implementor would try to seperate out the parts that might be modified by someone into seperate methods that get called by
>>generateByteCodeOn: so these helper methods would generally be overridden and not
>>generateByteCodeOn: unless that method was really simple.  So the initial implementor has to think about reuse and the places where modification might occure.  So you would have a lot of simple
>>generateByteCodeOn: methods instead of one big complex one.
 
There are all different ways of calling a function or method or query etc in picoLARC and these are all subClasses of KEGLambdaLispApplicationEvalV6p1 and it seems to work fine.
 
But overriding >>generateByteCodesOn: is not good enough is it?  The Compiler Classes can't have hard coded Eval instance creations either right?  The Compiler Class has to be subClassed also and the
>>meaningOf:inEnviron: needs to have a
( self createEval ) expression in it that could be subClassed and overridden.  And then that subClass has to be easily inserted into the expression dispatch table that pairs up expressions with expression Compilers.  So when that table gets made there should be a
( tableModifier modify: table ) which could stick the < expression Compiler > pairs in that are needed.
 
I think that is all that would be required to modify the compilation of an expression.
I will have to make these changes to picoLARC so it will be more modifiable.
 
I think the Compiler should be very modular.  For picoLARC one Class per expression and one Class per Eval seems to work good.  Stuffing lots of seperate things into a single Class and doing a procedural functional thing and not an OOP thing does not seem good to me.
 
I think that the Compiler should be very clean and a best practices example with a long comment at the bottom of each method telling all about what it does.  Writing it out and referencing other related methods helps to think about what is really going on and then a better design without hacks comes out.  I don't think hacking should be encouraged at all.  Hacking just makes a mess.
 
And then this practice of not making any Package comments has got to stop.  I think that people who do that should be admonished in some way.  I think that the Package comment for the Compiler should contain the design document for it that tells all about how it is designed.  If it needs to be long then it should be long.  It should include: How to understand the Compiler.  There should be a sequence of test cases that start simple and show how it all works.
 
And that should go for the VM too.  This idea that the VM can be opaque and only recognizable to a few is not good.
 
These should be works of art and not hacked up piles of rubbish to be hidden away into obscurity.
 
There is this idea that one should only care about what something does.  And the insides of it are a random black box that you tweek and pray on.  But I think that the insides should be shown to the world.  They should be displayed on a backdrop of velvet.  Especially the Compiler and VM and VM maker.  And then the whole Windowing thing should be modularized so you can have multiple different Windowing systems.
 
And what about having multiple VMs?  It would be cool if picoLARC could be inside of Squeak in that way.  It would be cool if one VM was generalized so that it could support different dialects and languages.  And another was specific and fast.  And you could make various kinds of VMs and images and output them onto disk without a lot of trouble.  It would come with gcc and all that junk all set up so it would just work.  If you already had gcc you could tell it not to download it.
 
picoLARC has simple name spaces called Nodules where you can have Nodules inside of Nodules and Nodules can multiply inherit variables from others.  Maybe such a thing could be used in Squeak?  Then you could have multiple VMs.  And VMs inside of VMs.
 
I think that Dolphin Smalltalk could be held up as an example of pretty.
 
I hope picoLARC will be another one.
 
I think that Squeak is pretty in a somewhat cancerous sort of way.
The cancer is all the hacking.  That goes on.
The vision is great but the hacking and undocumenting gum up all those big ideas.
 
Sure it's quick but it rots away quickly too.
 
Undocumented features.  In Smalltalk this is less of a problem but in like Lisp say you make this great feature but then don't document it.  You might as well have not even made it.

On Mon, Sep 1, 2008 at 2:12 PM, Igor Stasenko <[hidden email]> wrote:
2008/9/2 Edgar J. De Cleene <[hidden email]>:
>
>
>
> El 9/1/08 5:39 PM, "Eliot Miranda" <[hidden email]> escribió:
>
>> The thought of a committee fills me with horror :)  I don't think we need
>> anything so baroque :)
>>
>> Best
>> Eliot
>
>
> Peron said:
> If you wish nothing happens, nominate a comitee
>

You both right. My intent was only to get people's attention on this
problem and discuss different points of view and their needs.
Then implementors (like me) making things need to be done. Thats all.
If no-one sharing the thought that we need a better compiler at all,
then i can simply take away my words, and we will stay where we still
sitting. :)



--
Best regards,
Igor Stasenko AKA sig.






Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Igor Stasenko
2008/9/2 Kjell Godo <[hidden email]>:

> Where is this new compiler project?  Where is NewCompiler?  I would like to
> see it.
> Does anybody know where that book about the Squeak Compiler went to?
>
> the rest down below is all nonsense and I wouldn't read it if I were you.
>
> i knew this was going to cost me.
>
> What is atomic loading?  Does it mean no dependencies or dependencies are
> handled?
> It seems to me that there needs to be some kind of intellegent dependencies
> manager that works a lot better and is a lot smarter than what has been put
> out there so far.
>

The atomic loading is not about handling dependencies (they are
present and adressed as well, of course), but about installing a
number of changes in system replacing old behavior in a single short
operation, which can guarantee a safety from old/new behaviour
conflicts.

> How can I learn about how a good Squeak compiler works?  Without years and
> millions of dead hours?
>

Sure, you need some experience in compiling things, especially
smalltalk. Or , at least , if you even don't have such experience, but
using Parser/Compiler for own purposes, your experience is valuable as
well, since you can highlight different problems or propose better
interface or new functionality.

> Modularity is very good.  I think that all of Squeak should be very self
> explaining.  This can be done if you put your explanations of what is going
> on after the body of the method.  Colored source is good too.  See Dolphin.
> But without reformating.
>
> I am making picoLARC on sourceforge.net.  Each lisp/smalltalk expression
> gets compiled by an instance of a Compiler Class.  Each expression( let if
> define etc ) has its own KEGLambdaLispCompiler subClass with one
> standard method and zero or more helper methods in it.  Each Compiler
> outputs an instance of a subClass of the Eval Class.  An Eval can be
> evaluated at runtime by >>evalWithActivationRec: or it could generate byte
> codes which do the same thing via some method like
> EvalSubClass>>generateByteCodesOn:usingCodeWalker: where the CodeWalker
> could tie Evals together or do optimizations?  Is this not a good design?  I
> know I like the part about one Compiler Class for each expression and one
> corresponding Eval Class.  But I haven't done any byte code generation yet
> so I don't know about that part.  One Compiler per Eval is not strict.  The
> ApplicationCompiler can output several different related kinds of Evals for
> the different function calls and message sends.
>
> What is this visitor pattern?

http://en.wikipedia.org/wiki/Visitor_pattern

>  I don't like the idea of putting byte code
> generation into a single Class.  But I feel like maybe I don't know what I'm
> talking about.  To modify the byte code generation for an expression you
> would subClass the Eval Class and modify the
>>>generateByteCodeOn: aCodeStream.  The initial implementor would try to
>>> seperate out the parts that might be modified by someone into seperate
>>> methods that get called by
>>>generateByteCodeOn: so these helper methods would generally be overridden
>>> and not
>>>generateByteCodeOn: unless that method was really simple.  So the initial
>>> implementor has to think about reuse and the places where modification might
>>> occure.  So you would have a lot of simple
>>>generateByteCodeOn: methods instead of one big complex one.
>
> There are all different ways of calling a function or method or query etc in
> picoLARC and these are all subClasses of KEGLambdaLispApplicationEvalV6p1
> and it seems to work fine.
>
> But overriding >>generateByteCodesOn: is not good enough is it?  The
> Compiler Classes can't have hard coded Eval instance creations either
> right?  The Compiler Class has to be subClassed also and the

The problem in Squeak compiler that you will need to override much
more classes than just Compiler to emit different bytecode, for
instance.

>>>meaningOf:inEnviron: needs to have a
> ( self createEval ) expression in it that could be subClassed and
> overridden.  And then that subClass has to be easily inserted into the
> expression dispatch table that pairs up expressions with expression
> Compilers.  So when that table gets made there should be a
> ( tableModifier modify: table ) which could stick the < expression Compiler
>> pairs in that are needed.
>
> I think that is all that would be required to modify the compilation of an
> expression.
> I will have to make these changes to picoLARC so it will be more modifiable.
>
> I think the Compiler should be very modular.  For picoLARC one Class per
> expression and one Class per Eval seems to work good.  Stuffing lots of
> seperate things into a single Class and doing a procedural functional thing
> and not an OOP thing does not seem good to me.
>
> I think that the Compiler should be very clean and a best practices example
> with a long comment at the bottom of each method telling all about what it
> does.  Writing it out and referencing other related methods helps to think
> about what is really going on and then a better design without hacks comes
> out.  I don't think hacking should be encouraged at all.  Hacking just makes
> a mess.
>
+1

The design should allow replacing critical parts of compiler by
subclassing without the need in modifying original classes.
A so-called extensions , or monkey patching is very bad practice which
in straightly opposite direction from modularity.

I thinking, maybe at some point, to prevent monkey patching, a
deployed classes can disallow installing or modifying their methods.


> And then this practice of not making any Package comments has got to stop.
> I think that people who do that should be admonished in some way.  I think
> that the Package comment for the Compiler should contain the design document
> for it that tells all about how it is designed.  If it needs to be long then
> it should be long.  It should include: How to understand the Compiler.
> There should be a sequence of test cases that start simple and show how it
> all works.
>
> And that should go for the VM too.  This idea that the VM can be opaque and
> only recognizable to a few is not good.
>

VM tend to be complex. And complexity comes from inability of our
hardware/OS work in a ways how we need/want it.

> These should be works of art and not hacked up piles of rubbish to be hidden
> away into obscurity.
>
> There is this idea that one should only care about what something does.  And
> the insides of it are a random black box that you tweek and pray on.  But I
> think that the insides should be shown to the world.  They should
> be displayed on a backdrop of velvet.  Especially the Compiler and VM and VM
> maker.  And then the whole Windowing thing should be modularized so you can
> have multiple different Windowing systems.
>
> And what about having multiple VMs?  It would be cool if picoLARC could be
> inside of Squeak in that way.  It would be cool if one VM was generalized so
> that it could support different dialects and languages.  And another was
> specific and fast.  And you could make various kinds of VMs and images and
> output them onto disk without a lot of trouble.  It would come with gcc and
> all that junk all set up so it would just work.  If you already had gcc you
> could tell it not to download it.
>

What is gcc? And why it required to make VM? ;)

> picoLARC has simple name spaces called Nodules where you can have Nodules
> inside of Nodules and Nodules can multiply inherit variables from others.
> Maybe such a thing could be used in Squeak?  Then you could have multiple
> VMs.  And VMs inside of VMs.
>
> I think that Dolphin Smalltalk could be held up as an example of pretty.
>

Maybe, if you know how to deal with license & copyrights when taking
their source and blindly putting it to Squeak :)

> I hope picoLARC will be another one.
>
> I think that Squeak is pretty in a somewhat cancerous sort of way.
> The cancer is all the hacking.  That goes on.
> The vision is great but the hacking and undocumenting gum up all those big
> ideas.
>
> Sure it's quick but it rots away quickly too.
>
> Undocumented features.  In Smalltalk this is less of a problem but in like
> Lisp say you make this great feature but then don't document it.  You might
> as well have not even made it.
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

David Zmick
So, here is an idea, start the VM from scratch, and, redo the entire project to allow what we want in Squeak, and the compiler.  I know that is a really crazy idea, but I think it could be possible.  I have been thinking about a couple of very unlikely, but, possible, maybe, VM ideas, but, what do you guys think about that?

On Mon, Sep 1, 2008 at 7:56 PM, Igor Stasenko <[hidden email]> wrote:
2008/9/2 Kjell Godo <[hidden email]>:
> Where is this new compiler project?  Where is NewCompiler?  I would like to
> see it.
> Does anybody know where that book about the Squeak Compiler went to?
>
> the rest down below is all nonsense and I wouldn't read it if I were you.
>
> i knew this was going to cost me.
>
> What is atomic loading?  Does it mean no dependencies or dependencies are
> handled?
> It seems to me that there needs to be some kind of intellegent dependencies
> manager that works a lot better and is a lot smarter than what has been put
> out there so far.
>

The atomic loading is not about handling dependencies (they are
present and adressed as well, of course), but about installing a
number of changes in system replacing old behavior in a single short
operation, which can guarantee a safety from old/new behaviour
conflicts.

> How can I learn about how a good Squeak compiler works?  Without years and
> millions of dead hours?
>

Sure, you need some experience in compiling things, especially
smalltalk. Or , at least , if you even don't have such experience, but
using Parser/Compiler for own purposes, your experience is valuable as
well, since you can highlight different problems or propose better
interface or new functionality.

> Modularity is very good.  I think that all of Squeak should be very self
> explaining.  This can be done if you put your explanations of what is going
> on after the body of the method.  Colored source is good too.  See Dolphin.
> But without reformating.
>
> I am making picoLARC on sourceforge.net.  Each lisp/smalltalk expression
> gets compiled by an instance of a Compiler Class.  Each expression( let if
> define etc ) has its own KEGLambdaLispCompiler subClass with one
> standard method and zero or more helper methods in it.  Each Compiler
> outputs an instance of a subClass of the Eval Class.  An Eval can be
> evaluated at runtime by >>evalWithActivationRec: or it could generate byte
> codes which do the same thing via some method like
> EvalSubClass>>generateByteCodesOn:usingCodeWalker: where the CodeWalker
> could tie Evals together or do optimizations?  Is this not a good design?  I
> know I like the part about one Compiler Class for each expression and one
> corresponding Eval Class.  But I haven't done any byte code generation yet
> so I don't know about that part.  One Compiler per Eval is not strict.  The
> ApplicationCompiler can output several different related kinds of Evals for
> the different function calls and message sends.
>
> What is this visitor pattern?

http://en.wikipedia.org/wiki/Visitor_pattern

>  I don't like the idea of putting byte code
> generation into a single Class.  But I feel like maybe I don't know what I'm
> talking about.  To modify the byte code generation for an expression you
> would subClass the Eval Class and modify the
>>>generateByteCodeOn: aCodeStream.  The initial implementor would try to
>>> seperate out the parts that might be modified by someone into seperate
>>> methods that get called by
>>>generateByteCodeOn: so these helper methods would generally be overridden
>>> and not
>>>generateByteCodeOn: unless that method was really simple.  So the initial
>>> implementor has to think about reuse and the places where modification might
>>> occure.  So you would have a lot of simple
>>>generateByteCodeOn: methods instead of one big complex one.
>
> There are all different ways of calling a function or method or query etc in
> picoLARC and these are all subClasses of KEGLambdaLispApplicationEvalV6p1
> and it seems to work fine.
>
> But overriding >>generateByteCodesOn: is not good enough is it?  The
> Compiler Classes can't have hard coded Eval instance creations either
> right?  The Compiler Class has to be subClassed also and the

The problem in Squeak compiler that you will need to override much
more classes than just Compiler to emit different bytecode, for
instance.

>>>meaningOf:inEnviron: needs to have a
> ( self createEval ) expression in it that could be subClassed and
> overridden.  And then that subClass has to be easily inserted into the
> expression dispatch table that pairs up expressions with expression
> Compilers.  So when that table gets made there should be a
> ( tableModifier modify: table ) which could stick the < expression Compiler
>> pairs in that are needed.
>
> I think that is all that would be required to modify the compilation of an
> expression.
> I will have to make these changes to picoLARC so it will be more modifiable.
>
> I think the Compiler should be very modular.  For picoLARC one Class per
> expression and one Class per Eval seems to work good.  Stuffing lots of
> seperate things into a single Class and doing a procedural functional thing
> and not an OOP thing does not seem good to me.
>
> I think that the Compiler should be very clean and a best practices example
> with a long comment at the bottom of each method telling all about what it
> does.  Writing it out and referencing other related methods helps to think
> about what is really going on and then a better design without hacks comes
> out.  I don't think hacking should be encouraged at all.  Hacking just makes
> a mess.
>
+1

The design should allow replacing critical parts of compiler by
subclassing without the need in modifying original classes.
A so-called extensions , or monkey patching is very bad practice which
in straightly opposite direction from modularity.

I thinking, maybe at some point, to prevent monkey patching, a
deployed classes can disallow installing or modifying their methods.


> And then this practice of not making any Package comments has got to stop.
> I think that people who do that should be admonished in some way.  I think
> that the Package comment for the Compiler should contain the design document
> for it that tells all about how it is designed.  If it needs to be long then
> it should be long.  It should include: How to understand the Compiler.
> There should be a sequence of test cases that start simple and show how it
> all works.
>
> And that should go for the VM too.  This idea that the VM can be opaque and
> only recognizable to a few is not good.
>

VM tend to be complex. And complexity comes from inability of our
hardware/OS work in a ways how we need/want it.

> These should be works of art and not hacked up piles of rubbish to be hidden
> away into obscurity.
>
> There is this idea that one should only care about what something does.  And
> the insides of it are a random black box that you tweek and pray on.  But I
> think that the insides should be shown to the world.  They should
> be displayed on a backdrop of velvet.  Especially the Compiler and VM and VM
> maker.  And then the whole Windowing thing should be modularized so you can
> have multiple different Windowing systems.
>
> And what about having multiple VMs?  It would be cool if picoLARC could be
> inside of Squeak in that way.  It would be cool if one VM was generalized so
> that it could support different dialects and languages.  And another was
> specific and fast.  And you could make various kinds of VMs and images and
> output them onto disk without a lot of trouble.  It would come with gcc and
> all that junk all set up so it would just work.  If you already had gcc you
> could tell it not to download it.
>

What is gcc? And why it required to make VM? ;)

> picoLARC has simple name spaces called Nodules where you can have Nodules
> inside of Nodules and Nodules can multiply inherit variables from others.
> Maybe such a thing could be used in Squeak?  Then you could have multiple
> VMs.  And VMs inside of VMs.
>
> I think that Dolphin Smalltalk could be held up as an example of pretty.
>

Maybe, if you know how to deal with license & copyrights when taking
their source and blindly putting it to Squeak :)

> I hope picoLARC will be another one.
>
> I think that Squeak is pretty in a somewhat cancerous sort of way.
> The cancer is all the hacking.  That goes on.
> The vision is great but the hacking and undocumenting gum up all those big
> ideas.
>
> Sure it's quick but it rots away quickly too.
>
> Undocumented features.  In Smalltalk this is less of a problem but in like
> Lisp say you make this great feature but then don't document it.  You might
> as well have not even made it.
>



--
Best regards,
Igor Stasenko AKA sig.




--
David Zmick
/dz0004455\
http://dz0004455.googlepages.com
http://dz0004455.blogspot.com


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Karl Ramberg
In reply to this post by Kjell Godo
Kjell Godo wrote:
> Where is this new compiler project?  Where is NewCompiler?  I would
> like to see it.
> Does anybody know where that book about the Squeak Compiler went to?
Isn't it based on the Blue Book and the bluebook "missing chapters" ?
http://stephane.ducasse.free.fr/FreeBooks.html

karl

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Making a better Compiler for all

Kjell Godo


On Tue, Sep 2, 2008 at 12:38 PM, Karl Ramberg <[hidden email]> wrote:
Kjell Godo wrote:
Where is this new compiler project?  Where is NewCompiler?  I would like to see it.
Does anybody know where that book about the Squeak Compiler went to?
Isn't it based on the Blue Book and the bluebook "missing chapters" ?
http://stephane.ducasse.free.fr/FreeBooks.html

karl

 
There was an online book called something like:
Guided Tour Of The Squeak Compiler
or something like that.  I saw the link to it and tried to
click on it but it was not there.


12