Smalltalk › Squeak › Squeak - Dev

Questions about Environments

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

3 messages Options

Levente Uzonyi-2

Questions about Environments

Hi,

I started rewriting Environments to use a different kind of dictionary,
which won't get into an inconsistent state when a binding is modified.
The rewrite is almost ready, but I still don't understand a few things:

1) Some methods use the instance variable 'references' to look up
bindings, others use 'declarations'. After the rewrtie some tests don't
pass, because the bindings don't get added to 'references', but they are
present in 'declarations', so using the latter will make the tests pass.
According to the class comment:

declarations <IdentityDictionary>
Bindings for globals that have been declared inside me

references <IdentityDictionary>
Bindings for globals that are used by methods compiled inside me.

Which suggests that neither 'declarations', nor 'references' are good
enough for lookup in case of #associationAt:, #associationAt:ifAbsent: or
#associationOrUndeclaredAt:, #at:, #at:if*:, etc.
There either need to be a dictionary which contains all the bindings
visible from the Environment, or the class comment is wrong and
'declarations' is exactly that. In the latter case 'references' seems to
be superfluous to me.

2) The only way for a binding to get into 'references' is to get into
'imports' first and get copied to 'references' in #bindingOf:ifAbsent:.
But how does a newly compiled class's binding get into 'imports'?
And why does that happen? The class comments suggests that only referenced
(used by some methods) classes' bindings get into 'references', but new
classes are not referenced by any method.

3) There are multiple classes being used for bindings including:
ClassBinding, Global, Alias and Association (all bindings created in
Undeclared are Associations, since it's a simple IdentityDictionary).
The type of these bindings can change. Sometimes a class is created from a
global (Global->Class) or a class gets declared (Association->Class), etc
In these cases the class of the bindings should follow these changes. But
AFAIK that's not happening now, and those would require a #becomeForward:
send, which is rather slow. We could address these if the type of the
binding would be stored in an instance variable, but that's a bit less
flexible. Any ideas what to do with these?

Cheers,
Levente

Colin Putney-3

Re: Questions about Environments

On Sun, Jun 9, 2013 at 9:16 AM, Levente Uzonyi <[hidden email]> wrote:

Hi,

I started rewriting Environments to use a different kind of dictionary, which won't get into an inconsistent state when a binding is modified. The rewrite is almost ready, but I still don't understand a few things:

Interesting. By modified, you mean changing the key? What's the motivation for the rewrite?

1) Some methods use the instance variable 'references' to look up bindings, others use 'declarations'. After the rewrtie some tests don't pass, because the bindings don't get added to 'references', but they are present in 'declarations', so using the latter will make the tests pass.
According to the class comment:

declarations <IdentityDictionary>
Bindings for globals that have been declared inside me

references <IdentityDictionary>
Bindings for globals that are used by methods compiled inside me.

Which suggests that neither 'declarations', nor 'references' are good enough for lookup in case of #associationAt:, #associationAt:ifAbsent: or #associationOrUndeclaredAt:, #at:, #at:if*:, etc.

This is probably the most difficult part of the transition. Until now, you could make the assumption that all bindings are visible everywhere. So clever code could rifle through the SystemDictionary and adapt it's behaviour to what it finds, or even make modifications. But now code that operates at that level has to make the distinction between declarations and references. This is why I'd like to create separate protocols for this sort of thing. Sending #bindingOf: clearly asks for the reference—the binding that a method compiled in the environment would use. We could use #declarationOf: or some such to mean the binding of a class declared in the environment. Using #associationAt: bypasses any abstraction messes directly with the implementation, which is now changing.

There either need to be a dictionary which contains all the bindings visible from the Environment, or the class comment is wrong and 'declarations' is exactly that. In the latter case 'references' seems to be superfluous to me.

Both are needed. Let's imagine the case where we create an empty environment and then file-in a package containing classes and methods. When a class is built and installed in to the environment, a new binding is created and added to declarations. (So "declarations" contains all the classes that have been declared inside the environment. Ok, syntactically it's not really a declaration, this is the most descriptive name I could find… certainly better than "contents".)

When a method is compiled, we first look in "references" for the required bindings. If we find one, great, we use that. But since this is a brand new environment, references is empty. So we have to import the binding. The "imports" variable contains a list of rules for where we should look for the binding. One of the rules will look in "declarations" and find the binding there. Now that we have the binding, it gets added to references so that we'll get consistent bindings for all the methods compiled in the environment, and returned to the compiler to be added the method's literal frame.

Make sense? We need to keep separate the bindings that are *created* in the environment from the bindings that are *used* in the environment.

2) The only way for a binding to get into 'references' is to get into 'imports' first and get copied to 'references' in #bindingOf:ifAbsent:. But how does a newly compiled class's binding get into 'imports'?

Imports doesn't contain bindings. It contains rules about where to go looking for bindings.

And why does that happen? The class comments suggests that only referenced (used by some methods) classes' bindings get into 'references', but new classes are not referenced by any method.

It happens lazily. When compiled method refers to a class, the binding gets copied into references. Until that happens, the binding is only in declarations.

3) There are multiple classes being used for bindings including: ClassBinding, Global, Alias and Association (all bindings created in Undeclared are Associations, since it's a simple IdentityDictionary).
The type of these bindings can change. Sometimes a class is created from a global (Global->Class) or a class gets declared (Association->Class), etc
In these cases the class of the bindings should follow these changes. But AFAIK that's not happening now, and those would require a #becomeForward: send, which is rather slow. We could address these if the type of the binding would be stored in an instance variable, but that's a bit less flexible. Any ideas what to do with these?

I don't think speed is an issue here—it doesn't happen very often, right? I had thought that we no longer need a full #becomeForward: when converting bindings, and #asBinding: is sufficient. I could be wrong though. What problems are you seeing?

Colin

Levente Uzonyi-2

Re: Questions about Environments

On Sun, 9 Jun 2013, Colin Putney wrote:

>
>
>
> On Sun, Jun 9, 2013 at 9:16 AM, Levente Uzonyi <[hidden email]> wrote:
> Hi,
>
> I started rewriting Environments to use a different kind of dictionary, which won't get into an inconsistent state when a binding is modified. The rewrite is almost ready, but I still don't
> understand a few things:
>
>
> Interesting. By modified, you mean changing the key? What's the motivation for the rewrite?

Exactly. The motivation is that originally the bindings were stored in a
single dictionary (either SystemDictionary default, or Undeclared), but
now they are stored in multiple dictionaries, so changing the key in one
of them will break the rest.

>
> 1) Some methods use the instance variable 'references' to look up bindings, others use 'declarations'. After the rewrtie some tests don't pass, because the bindings don't get added to
> 'references', but they are present in 'declarations', so using the latter will make the tests pass.
> According to the class comment:
>
> declarations <IdentityDictionary>
> Bindings for globals that have been declared inside me
>
> references <IdentityDictionary>
> Bindings for globals that are used by methods compiled inside me.
>
> Which suggests that neither 'declarations', nor 'references' are good enough for lookup in case of #associationAt:, #associationAt:ifAbsent: or #associationOrUndeclaredAt:, #at:, #at:if*:, etc.
>
>
> This is probably the most difficult part of the transition. Until now, you could make the assumption that all bindings are visible everywhere. So clever code could rifle through the SystemDictionary and adapt
> it's behaviour to what it finds, or even make modifications. But now code that operates at that level has to make the distinction between declarations and references. This is why I'd like to create separate
> protocols for this sort of thing. Sending #bindingOf: clearly asks for the reference—the binding that a method compiled in the environment would use. We could use #declarationOf: or some such to mean the
> binding of a class declared in the environment. Using #associationAt: bypasses any abstraction messes directly with the implementation, which is now changing.

So all of these methods should use #bindingOf:ifAbsent: in order to return
the binding we are really looking for, right?

>
> There either need to be a dictionary which contains all the bindings visible from the Environment, or the class comment is wrong and 'declarations' is exactly that. In the latter case 'references'
> seems to be superfluous to me.
>
>
> Both are needed. Let's imagine the case where we create an empty environment and then file-in a package containing classes and methods. When a class is built and installed in to the environment, a new binding
> is created and added to declarations. (So "declarations" contains all the classes that have been declared inside the environment. Ok, syntactically it's not really a declaration, this is the most descriptive
> name I could find… certainly better than "contents".)
>
> When a method is compiled, we first look in "references" for the required bindings. If we find one, great, we use that. But since this is a brand new environment, references is empty. So we have to import the
> binding. The "imports" variable contains a list of rules for where we should look for the binding. One of the rules will look in "declarations" and find the binding there. Now that we have the binding, it
> gets added to references so that we'll get consistent bindings for all the methods compiled in the environment, and returned to the compiler to be added the method's literal frame.
>
> Make sense? We need to keep separate the bindings that are *created* in the environment from the bindings that are *used* in the environment.
>
> 2) The only way for a binding to get into 'references' is to get into 'imports' first and get copied to 'references' in #bindingOf:ifAbsent:. But how does a newly compiled class's binding get into
> 'imports'?
>
>
> Imports doesn't contain bindings. It contains rules about where to go looking for bindings.
>
> And why does that happen? The class comments suggests that only referenced (used by some methods) classes' bindings get into 'references', but new classes are not referenced by any method.
>
>
> It happens lazily. When compiled method refers to a class, the binding gets copied into references. Until that happens, the binding is only in declarations.

So basically 'references' is a cache, which avoids doing the lookups from
'imports' multiple times, right?
This means that 'references' should never be accessed directly, but
through #bindingOf:ifAbsent:, and 'declarations' should only be used if we
want to do something with the locally (in this Environment) declared
bindings.

What's the advantage of the lazy intialization here?

>
> 3) There are multiple classes being used for bindings including: ClassBinding, Global, Alias and Association (all bindings created in Undeclared are Associations, since it's a simple
> IdentityDictionary).
> The type of these bindings can change. Sometimes a class is created from a global (Global->Class) or a class gets declared (Association->Class), etc
> In these cases the class of the bindings should follow these changes. But AFAIK that's not happening now, and those would require a #becomeForward: send, which is rather slow. We could address
> these if the type of the binding would be stored in an instance variable, but that's a bit less flexible. Any ideas what to do with these?
>
>
> I don't think speed is an issue here—it doesn't happen very often, right? I had thought that we no longer need a full #becomeForward: when converting bindings, and #asBinding: is sufficient. I could be wrong
> though. What problems are you seeing?

The main problem is that the class of the binding is not updated when
needed:
Let's say we compile a method which refers to a class, which is not in the
image. This can happen when the method is created by some code instead of
the tools:

Object compile: 'foo ^Bar' classified: 'foo'.

The binding is created in Undeclared, which is an IdentityDictionary, so
its class will be Association. At this point we couldn't decide which
binding class to use, so it's fine.

Then create the class Bar:

Object subclass: #Bar
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: ''
category: 'Foo'.

Let's check if the method works:

self foo. "==> Bar"

It works. Let's see the class of the binding:

self environment bindingOf: #Bar. "#Bar->Bar"

It's still an Association instead of a ClassBinding.
I don't see how #asBinding: could help in this situation. It either
returns 'self' or another object. Using 'self' obviously won't solve the
problem. Another object is wrong, because the existing method will still
use the original binding (the Association).

About the speed: I think the only reason why bindings (Associations) are
being used from the CompiledMethods instead of direct class references is
speed.

Levente

>
> Colin
>
>