Integrations in Pharo for next generation VMs: FullBlockClosure, SistaV1 bytecode set and Read-only objects

Clement

Many thanks for your dedication. Your work open many nice paths for the future.

Hi everyone,

With the help of multiple people (Marcus Denker, Eliot Miranda and some more), I introduced over the past year multiple changes in the Pharo image to support the next generation VMs. It's still a work in progress, but once issue 19083 will be introduced the bulk of the changes will be in the Pharo image.

These changes have no impact on developers only reading/writing source code and analysing/working with the ASTs, but may have impact with people working with Opal's IR, the bytecode or directly with the BlockClosure and CompiledMethod instances (People building frameworks like Reflectivity, using OpalCompiler extensions or people working on object serializers). 

Three main changes were introduced:

- Secondary bytecode set support

- Read-only objects

- FullBlockClosure

The secondary bytecode set support allows the image to run on a VM supporting 2 bytecode sets. Each compiled method has a flag precising which set it uses. This change allows the image to convert all the methods from one bytecode set to another one without any bootstrap (only in-image do-its). The SistaV1 bytecode set was introduced as the second bytecode set. This bytecode set removes many encoding limitations of the existing bytecode set, simplifies bytecode decoding and features new instructions (such as full block closure instructions). 

Read-only objects allows one to mark any object as read-only. Any attempt to mutate a read-only object (primitive operations, instance variable store) fails, and the programmer can handle the failures with Smalltalk code. This feature is present with less than 1% overhead in all of our benchmark suite. In the near future, we plan to have all literals read-only to allow more aggressive compiler optimizations and avoid literal mutation inconsistency bugs. This feature can also be used to build efficient object modification trackers and could be used to improve the performance of part of the Reflectivity framework. It may break your code if you mutate literals (Normally you don't).

FullBlockClosures are a new implementation of BlockClosures. They allow efficient implementation of Blocks, with the Copying block and Clean block optimizations present in other Smalltalks. They simplify part of the VM, making Blocks more similar to methods. They also remove many dependencies between BlockClosure, compiled method and contexts, which will allow us to implement more aggressive optimizations in the VM in the future. With FullBlockClosures, the closure's bytecode is present in a separate object (an instance of compiled block) instead of being inlined in the enclosing method. This is still confusing some tools. The debugging support of FullBlockClosure seems however to be complete.

With the issue 19083 integrated and the latest VM compiled with different settings than the current Pharo VM (MULTIPLEBYTECODESETS true bytecodeTableInitializer initializeBytecodeTableForSqueakV3PlusClosuresAndSistaV1 -DIMMUTABILITY 1), all these features seems to be working in the Pharo image. The goal is to get all of these features used by default before the Pharo 6 feature freeze in November. Alternatively I will need to wait many more months (or Pharo will get unstable and Esteban will scream at me 'No do not integrate that').

If you worry how any of these changes can impact your frameworks, please answer this thread so we can analyse the potential problems.

Regards,