questions about internals

I've been studying the internals just so I can get more familiar and perhaps lend a hand in there sometimes. Here's my sketchy understanding and some questions and comments.

- Every Smalltalk object is an instance of ProtoObject, even Integers. Also Classes and Metaclasses. 

- ProtoObject has these inst vars:

  - cls, another ProtoObject, the ST class of the ST object.

  - superclass, another ProtoObject, the ST superclass of the ST object.

  - javaValue, a concrete Java object representing the ST object, e.g. a BigInteger for ST Integers, a String for ST String (literals). For "ordinary" ST objects, this is null.

  - name, a String which appears to be used just for ST classes and metaclasses, not ordinary ST objects.

  - packages, ?

  - instanceVariables, a Map<String, ProtoObject> containing the list of ST inst var names. Not sure what the value side of the Map is used for.

Looks like this is used just for ST classes and metaclasses, not ordinary ST objects.

  - methods, a Map<String, ProtoMethod> containing the methods for this class. Again, it appears to be used just for ST classes and metaclasses, not ordinary ST objects.

  - variables, a Map<String, ProtoObject> containing the ST instance variables for an ST object. 

- Every Smalltalk method is an instance of ProtoMethod (which holds generated jvm bytecodes? and where do java Methods come into play?).

- Message sending is done somewhat like an ordinary Smalltalk would do it -- look up a selector in the receiver's class's method dictionary (and superclass dictionaries), and invoke the method. 

- ThisContext represents a stack frame.

- ProtoBlock represents a ST Block and is a subclass of ProtoMethod.

Is that basically correct? 

My biggest question is why Smalltalk classes and metaclasses are not implemented as separate subclasses of ProtoObject, more cleanly separating the instance variables in ProtoObject. It seems odd to me that everything is an instance of ProtoObject, and that ProtoObject has several instance variables that appear to be only used by classes and metaclasses, not ordinary objects. Similarly, I don't see why ProtoMethod is a subclass of ProtoObject -- does a ProtoMethod actually use/inherit anything from ProtoObject? 

A minor question: Why ProtoObject? Why not SmalltalkObject?

I expected to see a java class hierarchy something like this:

SmalltalkObject (SmalltalkClass class, Map<String, SmalltalkObject> variables)

  SmalltalkClass (String name, List<String> instanceVariables, SmalltalkClass superclass, Map<String, ProtoMethod> methods)

    SmalltalkMetaclass (SmalltalkClass thisClass)

      

Or even more finely broken down to match the Smalltalk Behavior hierarchy? 

Some other random comments and questions:

- I see several places where System.out.println statements are commented out. This kind of thing is typically done using a logging framework such as log4j. Is there a reason why you did not use one? It's not too late!

- As I mentioned elsewhere, none of the primitive methods in Primitives.java are commented. I would expect at least a very brief description or perhaps just the Smalltalk class/method that it goes with.

- I'm surprised there are not more unit tests for all the primitives. 

- I'm surprised there are not unit tests for the Smalltalk compiler and bytecode generator. How were these things developed? 

Sorry this sounds so critical, I'm trying to be constructive and helpful. I'd like to see this effort succeed, but I'm concerned.

Thanks,

Lee

Answer / comments inline.

Thanks for taking the time to look into the internals so deeply.

On Mon, Jan 2, 2012 at 3:56 AM, Lee Breisacher <[hidden email]> wrote:

I've been studying the internals just so I can get more familiar and perhaps lend a hand in there sometimes. Here's my sketchy understanding and some questions and comments.

- Every Smalltalk object is an instance of ProtoObject, even Integers. Also Classes and Metaclasses. 

true.
 

- ProtoObject has these inst vars:

  - cls, another ProtoObject, the ST class of the ST object.

  - superclass, another ProtoObject, the ST superclass of the ST object.

  - javaValue, a concrete Java object representing the ST object, e.g. a BigInteger for ST Integers, a String for ST String (literals). For "ordinary" ST objects, this is null.

true.

  - name, a String which appears to be used just for ST classes and metaclasses, not ordinary ST objects.

true

  - packages, ?

Packages is the map of packages imported into this object
 

  - instanceVariables, a Map<String, ProtoObject> containing the list of ST inst var names. Not sure what the value side of the Map is used for.

Looks like this is used just for ST classes and metaclasses, not ordinary ST objects.

True - used to indicate the variables to create when making an instance of this class.

  - methods, a Map<String, ProtoMethod> containing the methods for this class. Again, it appears to be used just for ST classes and metaclasses, not ordinary ST objects.

true

  - variables, a Map<String, ProtoObject> containing the ST instance variables for an ST object. 

true

- Every Smalltalk method is an instance of ProtoMethod (which holds generated jvm bytecodes? and where do java Methods come into play?).

True - Which Java methods are you referring to here, those defined in external java classes?

- Message sending is done somewhat like an ordinary Smalltalk would do it -- look up a selector in the receiver's class's method dictionary (and superclass dictionaries), and invoke the method. 

true

- ThisContext represents a stack frame.

true

- ProtoBlock represents a ST Block and is a subclass of ProtoMethod.

true

Is that basically correct? 

Well understood.

My biggest question is why Smalltalk classes and metaclasses are not implemented as separate subclasses of ProtoObject, more cleanly separating the instance variables in ProtoObject. It seems odd to me that everything is an instance of ProtoObject, and that ProtoObject has several instance variables that appear to be only used by classes and metaclasses, not ordinary objects. Similarly, I don't see why ProtoMethod is a subclass of ProtoObject -- does a ProtoMethod actually use/inherit anything from ProtoObject? 

I have considered the approach of splitting things more. I did that initially and it caused some problems, then I put them together again.
Essentially as long as SomeObject supports a message send, the rest should be un-required.

A minor question: Why ProtoObject? Why not SmalltalkObject?

Tomatoe / Tomarto - ProtoObject is less to type and is a little analogous to ProtoObject in Pharo.Also - It isnt a SmalltalkObject, its a Prototype.

I expected to see a java class hierarchy something like this:

SmalltalkObject (SmalltalkClass class, Map<String, SmalltalkObject> variables)

  SmalltalkClass (String name, List<String> instanceVariables, SmalltalkClass superclass, Map<String, ProtoMethod> methods)

    SmalltalkMetaclass (SmalltalkClass thisClass)

      

Or even more finely broken down to match the Smalltalk Behavior hierarchy? 

It appears easier to match the hierarchy but that road leads to pain, since you then have to map what the compiler/interpreter is doing
and the underlying class hierarchy. The most important thing is that everything is an Object, not subclasses of Object.

Some other random comments and questions:

- I see several places where System.out.println statements are commented out. This kind of thing is typically done using a logging framework such as log4j. Is there a reason why you did not use one? It's not too late!

Sure, which one should we use? This presents some issue, it also means we get further away from a Smalltalk solution - should the logging in java also be the logging in smalltalk, how should they interract?

- As I mentioned elsewhere, none of the primitive methods in Primitives.java are commented. I would expect at least a very brief description or perhaps just the Smalltalk class/method that it goes with.

I agree. 

- I'm surprised there are not more unit tests for all the primitives. 

I agree. Time is my enemy at the moment and he is very elusive.
 

- I'm surprised there are not unit tests for the Smalltalk compiler and bytecode generator. How were these things developed? 

How do you suggest unit tests for the compiler and bytecode generator are done?
I did do this in previous versions, and it got cumbersome. Again, time got me here.

Sorry this sounds so critical, I'm trying to be constructive and helpful. I'd like to see this effort succeed, but I'm concerned.

Thanks,

Lee

All good.

It isnt a SmalltalkObject, its a Prototype 

Not sure what you mean by that. Do you have some special meaning behind the term Prototype? Like js objects? It seems to me that (an instance of) a Redline ProtoObject *is* (an instance of) a Smalltalk object.  

 

- Every Smalltalk method is an instance of ProtoMethod (which holds generated jvm bytecodes? and where do java Methods come into play?).

True - Which Java methods are you referring to here, those defined in external java classes?

No no. I'm just wondering how ProtoMethods are related to java methods. Don't the generated bytecodes need to be "attached" to a java method somehow in order to be executed? 

Sure, which one should we use? This presents some issue, it also means we get further away from a Smalltalk solution  

I like log4j, but mostly just because I've used it so much. I think the built-in jdk logging is probably fine too. I don't see how this "gets further away from smalltalk" -- I'm just talking about implementation logging to assist with debugging. Since the impl is written in java, it makes sense to do java logging.

- should the logging in java also be the logging in smalltalk, how should they interract? 

Not what I was thinking. Seems to me that Smalltalk logging can/should be separate, but now that you mention it, I suppose it would be nice if there was Smalltalk logging that was just a wrapper around some standard java logging. 

- As I mentioned elsewhere, none of the primitive methods in Primitives.java are commented. I would expect at least a very brief description or perhaps just the Smalltalk class/method that it goes with.

I agree.  

I suppose I could volunteer to do some of this, and unit tests. I know it's not your highest priority, but I feel it's pretty important. 

- I'm surprised there are not more unit tests for all the primitives. 

I agree. Time is my enemy at the moment and he is very elusive.

At the risk of sounding like a nagging mother, I claim that unit tests save you time in the long (or even medium) run. 

How do you suggest unit tests for the compiler and bytecode generator are done?

I did do this in previous versions, and it got cumbersome. 

I haven't thought about it at all, but there must be a way. I'll give it some thought...

keep in mind the amount of chicken and egg...

Yeah, good point. Thinking about metaclasses is always a bit mind-bending. But you seem to have a good handle on it via the Bootstrap class. 

How do you send a message to an object to make an instance, when that instance is required before you can send a message?

I understand chicken/egg issues, but I don't understand this example. You send a message to a *class* object to create an instance of the class. What's the issue?

Lee

I bunched up my responses as inline seems like a good idea until I got your additional response :)

re: Prototype.
It is a prototype in that you build it up and then 'new' it to become a real instance.
Personally I'm not that happy with ProtoObject and it could be made so much cleaner and simpler.
The benefit of hindsight.

re: Smalltalk methods and java methods
Smalltalk methods become java methods when the compiler creates a subclass of the class ProtoMethod and
overrides the appropriate applyTo() method to execute the bytecode contained within your smalltalk method.
Each smalltalk method becomes a class whose sole method is applyTo()

re: Logging
I did have a version of Redline that uses logging, SL4J as it gave a lot of flexibility. However, the output was rather verbose.
I guess we can put it back at some point.

re: Primitives
I agree it would be much better if we had a description of what each primitive did.
Will look at adding these and not accepting pull requests without them.

re: unit testing
While the code may not show it, im a BIG fan of unit testing.
One version of Redline was entirely written using TDD, including a hand written compiler.
It is a good way to go and it was always my intention to drive things out this way.
Not sure exactly why this got dropped or forgotten or sidetracked.
I'm also not sure how to get back on track with all the test coverage.

re: unit testing ANTLR
I have seen some unit tests around ANTLR and grammars and I think as a minimum
it would be nice to have this covered too.

BIG QUESTION: Would people be willing to wait for a V1.0 while we added coverage?