Creating new context classes for Exupery

I've just started work on compiling blocks but unfortunately there
isn't a spare slot in BlockContexts to hold a pointer to the natively
compiled code so I'm thinking about creating context classes for
Exupery.

The new contexts will be removed when Exupery is shutdown or Exupery's
code cache is initialised because they will hold pointers to compiled
code which isn't saved with the image.

However they will need a compact class number so they can be checked
for. This is important because the GC needs to check if an object is a
context so it can deal with the stack. If I use a compact class then
the extra check will just be an extra integer comparison.

I'll probably swap the position of the caller and the instruction
pointer for Exupery's contexts.  This way an integer check of either
position will indicate if it's an Exupery compiled context or an
interpreted context. Exupery's instruction pointers are pointers to
compiled code tagged as integers. The caller or sender slot should
always be a pointer object not a SmallInteger while both Exupery
instruction pointers and interpreted instruction pointers are always
SmallIntegers.

Would checking the instruction pointers size be more sensible? In
practise it would be safe but there's a risk that the code cache might
be allocated at memory location 0 on some platforms.

I'll need to use my own contexts for natively compiled methods as well
once Exupery starts inlining methods. This is so that image level code
will be able to hide the inlining or de-inline if necessary (say when
single stepping in the debugger).

Any objections?

Bryce

Hi Bryce,

 > Any objections?

Depends on what you're asking for. Reading your message, I am not sure
whether you are saying "here are some changes that I need in the VM to
work with Exupery" or whether you are saying "here are some design
decisions that I need to make at a point in the near future, please let
me know what you think". The point being that your message is unclear
whether you are planning to implement entirely *new* context classes
(which would interoperate somehow with what's there already) or whether
you are looking at making the existing VM deal with a *change* to the
existing contexts.

Depending on which my answers would be vastly different: If the first
(you're writing this from scratch) I think what you're proposing sounds
generally good and I don't see any big issues off hand. If the latter,
well that would be a different story, and we would need to look very
hard at a variety of issues, including backward compatibility,
interoperability issues (like the work on closures that is still
pending), the general cost-benefit ratio etc. At least if you mean this
to be for inclusion in "production releases" instead of some special
purpose Exupery VMs (which would be fine too, and probably the best way
of developing it).

Please clarify what kind of feedback you'd like to get here.

Cheers,
   - Andreas


Bryce Kampjes wrote:

Hi Andreas,
Thanks for the response.

I'm asking two separate questions: The first is to make sure that what
I'm planning will be acceptable for a production release of Squeak
when ready. The second is to get feedback about some design decisions
I'll be making in the next few weeks.

I'm thinking about creating two new contexts. Only the new block
context will be created soon. The new method context is only needed
for inlining which is still a long way off. They will both need to be
compact classes.

I need the new contexts first to quickly check whether they are
interpreted contexts or compiled contexts and second as a place to put
a native machine code program counter. Every context change (send,
return, or block entry) needs to check if it's a compiled or
interpreted context to execute it appropriately. I currently do this
by checking whether the unused slot in a message context is an
integer, it's always an integer for compiled contexts and always a nil
(pointer object) for interpreted contexts. The interpreter uses this
trick to detect if it's a block or message context. There's no unused
slots in block contexts so I'm thinking about creating my own context.

Having my own context objects will provide a place to put image level
support code. Compiled contexts can not be single-stepped so may need
to be deconverted into interpreted contexts while debugging. More
drastic deconversions will be needed after inlining.

There shouldn't be any backwards compatibility issues because the
contexts will have to be removed when the image is saved. They contain
references to natively compiled code which isn't saved with the image.
If they are not cleared out when necessary then the image will crash
if they are entered.

There may be things that I haven't thought about. Dealing with the
interpreter is fairly easy however there is a large amount of
primitive code that could access contexts internals.  I'm assuming
that if I leave the stack pointer and the stack at the same locations
then primitives should just work with Exupery's new contexts. Even
with new context objects, I'll still need to be compatible with any
VM code that may be called.

While I'd like Exupery to eventually be incorporated in the main
production releases, it is being developed with it's own custom
VMs. I'd like to make sure that my decisions now do not cause
avoidable problems later.

Bryce

P.S. I'm unable to respond to email until Monday.