Smalltalk › Squeak › Squeak - Dev

[ANN] SimulationStudio and sandboxed execution for Squeak

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 17 messages Options
Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

[ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

Hi all! :-)


I'm very excited to finally share my new project with you today, which I call SimulationStudio and have made available on GitHub under [1]. Its primary function is to provide an inheritable SimulationContext subclass of Context that makes it easy to simulate entire call trees while customizing the execution or installing custom hooks for various tracing and measurement purposes. Another accomplishment is the Sandbox which allows you to run arbitrary Smalltalk code in an isolated environment, separating any side effects that occur during the simulation from the rest of the image.


Sandbox

The original idea has arisen already one and a half years ago when we were discussing the necessary restrictions of the current MethodFinder design, which uses a huge allow-list of selectors that can be safely performed on unknown objects without the risk of dangerous side effects [2]. It was Eliot who proposed to apply simulation to this problem in order to run the unknown code step-by-step and prevent any side effects. I liked this idea very much and recently was finally able to build a working Sandbox prototype. However, instead of only aborting execution or creating a snapshot when a side effect is reached, I implemented an indirection layer for all side effects so that while the side effects are not actually applied to the entire image, they will be transparently available from the perspective of the code under simulation.


Here is a simple application example:


World color: Color transparent.

World color isTransparent.


If we execute these statements regularly, in the PasteUpMorph, the instance variable color will be set to transparent, then it will be accessed again and sent the message #isTransparent which will eventually be answered true.

However, in this example, your global image state would also be affected, and your wonderful desktop decorations would have been erased irrecoverably. If you only want to find out what the code above returns, you can now run it in an isolated sandbox without actually impairing your image:


Sandbox evaluate: [

    World color: Color transparent.

    World color isTransparent].


The script will answer true, again, but this time the world's actual color won't have changed. And here is how it works: The sandbox uses a SandboxContext under the hood, which is a subclass of Context that is only meant to be used for simulation purposes. This SandboxContext overrides a number of simulation operations, including #object:instVarAt:put: for writing and #object:instVarAt: for reading instance variables. Now, when an attempt is made to produce a side effect by assigning a new value to the color instvar in PasteUpMorph, the SandboxContext notices this attempt and adds the object (World) to the sandbox space, which is just a big WeakIdentityKeyDictionary that maps real-world objects to sandbox-manipulated objects. The actual manipulation then is only applied to the sandbox-manipulated copy of the original object, so the rest of the image does not get affected. Analogously, #object:instVarAt: checks whether the requested object is already part of the sandbox space, and if this is true, it redirects the instvar request to the sandboxed version of the original object. Of course, these indirections are not only implemented for instance variable accesses but also many other bytecode operations such as variable object access (#at:[put:]), identity exchanging/forwarding, instance adoption, and many other VM primitives and modules. By this means, the sandbox space is extended dynamically on demand and all side effects caused by the sandboxed code remain locally in the sandbox environment without affecting the rest of the image.


Sandboxes are thread-safe (i.e. global-state free) and self-transparent, i.e. you can even do the following:

Sandbox evaluate: [Sandbox evaluate: [6 * 7]].


SimulationContext

A central idea while implementing the SandboxContext was to inherit from the regular Context class in order to reuse the existing simulation logic. However, subclassing from Context is not straightforward due to the special role it plays in the connection to the VM Executor (shortly, Context instances are only created on demand for optimization purposes). Eliot wrote a really worthwhile description of this mechanism in [3] recently. Some context primitives and the stack functionality which is used by most bytecode operations won't work in a plain subclass of Context. For this reason, I have created the SimulationContext subclass of Context which overrides these functionalities by providing purely image-based implementations. Another responsibility of SimulationContext is to preserve the class of context instances while simulating a call stack - that is, new stack frames that are created when simulating a message send are automatically converted to instances of the current SimulationContext class, too.


With this base, it is fairly easy to create subclasses of SimulationContext to customize simulation at will, without needing to deal with any further VM magic. So does SandboxContext, but I also created another subclass just for proof of concept, SimulationMsrContext, which integrates with the MessageSendRecorder of the HPI group [4, 5] but provides a complete trace of all message sends. At the same time, SimulationMsrContext eliminates the need for installing and uninstalling global MethodWrappers in the image (which is an exclusive operation and can be slow at some times) and thus makes it even possible to transparently trace the simulation itself. Note that this only a rough experiment with some hick-ups and the concepts do not match each other perfectly. From a long-term perspective, something like a BytecodeBrowser might provide an even more detailed trace of the execution.


While SimulationContexts cannot be executed by the VM any longer, I wrote a small debugging adapter, making it possible to debug all kinds of SimulationContext in a regular debugger (see the debugging extensions on Context class). You can also debug the implementation of the SimulationContext from there by loading the BytecodeDebugger changeset [6].


Applications

I think SimulationStudio opens or at least facilitates a bunch of possibilities around the exciting simulation machinery of Squeak. I already have identified some real use cases for the sandbox:


  • An alternative MethodFinder implementation that is not restricted to an allow-list of selectors as mentioned above. A prototype for this is available in the repository on the class-side of Sandbox.
Example:
Sandbox
    findSelectorsFor: {1. 2. 3. 4}
    arguments: {[:x | x asWords] ascending}
    thatAnswer: {4. 1. 3. 2}.  "==> #(quickSort: sort: sorted:)"
The implementation itself is noticeably simple:

| selectors |
selectors := receiver class allSelectors select: [:sel | sel numArgs = arguments size].
^ Generator on: [:generator |
  selectors do: [:sel |
        | match sandbox |
        sandbox := Sandbox new.
        sandbox stepLimit: 100000.
        match := sandbox
            evaluate: [
                | result |
                result := (receiver perform: sel withArguments: arguments).
                result = expectedResult]
            ifFailed: [false].
        match ifTrue: [generator nextPut: sel] ] ]

As you can see, it would be also a small change only to find methods based on side effects instead on its return value.
  • Type-guessing support for Autocompletion: Autocompletion by Leon Matthes [7] is a code completion implementation for Squeak based on eCompletion [8] that, inter alia, provides type guessing based on previously entered literals that are recognized as the receiver, instance variables, global bindings, or quick return selectors. By combining this idea with the possibility of running arbitrary code in a sandbox, the quality of type guessing and thus completion suggestions can be improved significantly for long expressions in a scripting environment. My long-term vision in this regard is to provide a UI that is equal or superior to the code completion into Chrome Dev Console. :-)
I have available an okayish patch (< 20 lines!) for this in my image and can share you a changeset on demand, but it still requires some fine polish before I will officially release it.
  • I already have many other ideas around dynamic code analysis (for instance, fine-granular execution tracing, logging of side effects, bytecode-specific coverage measurements, revertable debugger stepping, ...) and will definitively have some more fun in this field in the near future. If you are interested, you can watch the repository on GitHub. :-)


Limitations and challenges

Well, first of all ... performance. :-) Some recent measurements that I have run have shown that the simulator reaches about 0.1 percent (sic) of the speed of the VM executor, depending on the domain and the distribution of bytecode operations. The layers added on top of the simulator by SimulationStudio, moreover the fact that I did not yet run any systematic attempts on optimization, slow down the execution even more by further ~50%. While it's still "fast enough for our neurons" in most scenarios I came up with so far, computation-intensive operations such as rendering 4K images can indeed require a certain amount of patience. However, from a conceptual perspective, I comfort myself by saying that this is "only" an optimization problem.


Another challenge is the handling of primitive operations during the sandbox execution. At the moment, I have disabled most primitives that I did not need and tried to fall back on image implementations whenever possible (see SandboxContext >> #doPrimitive:method:receiver:args: and #doNamedPrimitiveIn:for:withArgs:). However, for some areas, mostly external I/O plugins, this is not possible, and some customized solutions will be required to provide these functionalities while running in an isolated sandbox context.


Last but not least, it's hard to preserve a consistent sandbox space while making modifications in the real image. Just take the code example from above and wonder what should happen when the color of the world is changed half-way during the execution. Should the sandboxed copy of the world be affected, too? It is not possible to track changes to the real image space without severe performance drawbacks. This is a synchronization issue for which I don't know a good solution, yet.


I want you! :-)

This is an ongoing project and I'm looking very much forward to your feedback, questions, and ideas on this matter! If you are looking for a challenge, you can try to find a gap in the sandbox isolation. I have spent many hours sealing it in all conscience, I did not check everything systematically so I would not be very surprised (but delighted!) if you manage to find any vulnerability. Ideally, any kind of contribution will be welcome too, of course - just create your issues and pull requests in the repository at [1]! :-)


Carpe Squeak!


Best,

Christoph


PS: Sorry for the long text. I thank everyone who made it this far. :-)




Carpe Squeak!
K K Subbu
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

K K Subbu

On 16/03/21 4:51 pm, Thiede, Christoph wrote:

> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu

Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu

Carpe Squeak!
Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...


Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu



Carpe Squeak!
codefrau
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

codefrau
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu
Yoshiki Ohshima-3
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Yoshiki Ohshima-3
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki
Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki



Carpe Squeak!
Yoshiki Ohshima-3
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Yoshiki Ohshima-3
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki
Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki



Carpe Squeak!
Yoshiki Ohshima-3
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Yoshiki Ohshima-3
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki
Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 01:02:26
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki



Carpe Squeak!
K K Subbu
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

K K Subbu
In reply to this post by Christoph Thiede
On 16/03/21 7:40 pm, Thiede, Christoph wrote:
> image. By the way, is there already some research about unioning these
> two worlds in order to reprogram the VM from within the image that is
> running in it? I would love this.

Not to my knowledge. The closest is Eliot's Cog VM simulator.
ObjectMemory requires lot of work to simulate variations in bitsize,
byteorder, endianness etc.

> What are white-outs? I could not find any reference on this.
It is a term used in systems that do fork->edit->merge-commit. It marks
deleted objects like the way dirty bit marks modified ones.
copy-on-write (dirty) propagates down the fork chain while white-outs
propagate up the chain during merge-commit.

For instance, when a directory is cached in RAM and one of its entry is
deleted, the entry is marked as white-out until the cache is synced.

When an object copied over (loaned) from parent sandbox becomes garbage
in the child sandbox it must not be finalized or we won't be able to
commit deletes to the parent. It is marked 'white-out' so that it is no
longer accessible in the child. When the changes are committed to the
parent, the white-out is propagated into the parent for possible
finalization. If the object is strictly local to child white-outs may be
dropped and the object is subject to the usual GC.

> For some I/O primitives, however, virtualization is rather easy. For
> example, when writing on the global Display, you can simply copy this
> object like any other object into your sandbox space/world. However,
> other primitives, such as from the FilePlugin, operate on state is
> managed outside of the image ...

This is a reasonable limitation of sandbox but is not a show-stopper for
merge-commit feature. Now that memory is so affordable, speculative
simulation can be a valuable feature even with this limitation.

Regards .. Subbu

marcel.taeumel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

marcel.taeumel
In reply to this post by Christoph Thiede
Hi Christoph.

> Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Frank-170908 is based on Squeak 4.4 alpha. The VM from the Squeak 4.4 release works fine under Windows 10: http://files.squeak.org/4.4/

Best,
Marcel

Am 17.03.2021 01:36:08 schrieb Thiede, Christoph <[hidden email]>:

Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 01:02:26
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki



Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

Hi Marcel,


it's not working for me:


---------------------------
Squeak!
---------------------------
Could not open the Squeak image file 'C:\Users\Christoph\OneDrive\Downloads\Squeak-4.4-All-in-One\Squeak-4.4-All-in-One.app\Contents\Resources\Squeak4.4-12327.image'

There are several ways to open an image file. You can:
  1. Double-click on the desired image file.
  2. Drop the image file onto the application.
Aborting...

---------------------------
OK   
---------------------------

How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only. Dropping another image file on the executable only gives me a load error from within the 4.4 image. The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Mittwoch, 17. März 2021 09:13:04
An: squeak-dev
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi Christoph.

> Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Frank-170908 is based on Squeak 4.4 alpha. The VM from the Squeak 4.4 release works fine under Windows 10: http://files.squeak.org/4.4/

Best,
Marcel

Am 17.03.2021 01:36:08 schrieb Thiede, Christoph <[hidden email]>:

Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 01:02:26
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki



Carpe Squeak!
marcel.taeumel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

marcel.taeumel
Hi Christoph.

> How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only.

Change the .ini file :-) Remove the hard-coded path in there.

> The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.

I suppose you need the other DLLs and resources, too? What I tried was to copy the VM files over to the contents from the ZIP file. That worked. :-)

Best,
Marcel

Am 17.03.2021 14:06:05 schrieb Thiede, Christoph <[hidden email]>:

Hi Marcel,


it's not working for me:


---------------------------
Squeak!
---------------------------
Could not open the Squeak image file 'C:\Users\Christoph\OneDrive\Downloads\Squeak-4.4-All-in-One\Squeak-4.4-All-in-One.app\Contents\Resources\Squeak4.4-12327.image'

There are several ways to open an image file. You can:
  1. Double-click on the desired image file.
  2. Drop the image file onto the application.
Aborting...

---------------------------
OK   
---------------------------

How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only. Dropping another image file on the executable only gives me a load error from within the 4.4 image. The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Mittwoch, 17. März 2021 09:13:04
An: squeak-dev
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi Christoph.

> Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Frank-170908 is based on Squeak 4.4 alpha. The VM from the Squeak 4.4 release works fine under Windows 10: http://files.squeak.org/4.4/

Best,
Marcel

Am 17.03.2021 01:36:08 schrieb Thiede, Christoph <[hidden email]>:

Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 01:02:26
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki



Christoph Thiede
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Christoph Thiede

Hi Marcel,


ah, that was missing! Thank you. :-)


Some first impressions of the Worlds implementation for Squeak: If I understand correctly, it only works with objects that explicitly support the worlds mechanism. So for example:

z := #(1).
world := WWorld thisWorld sprout.
{world eval: [
z at: 1 put: 2.
z].
z}  "==> #(#(2) #(2))

Sandbox would answer:

z := #(1).
{Sandbox evaluate: [
z at: 1 put: 2.
z].
z}  "==>  #(#(2) #(1))"

For Worlds, I apparently would need to use a specialized WArray instead. This has the advantage of being much faster than Sandbox but also the disadvantage of not fitting for general-purpose Smalltalk expressions. Still, a very interesting approach!

Best,
Christoph
Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Mittwoch, 17. März 2021 14:12:12
An: squeak-dev
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi Christoph.

> How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only.

Change the .ini file :-) Remove the hard-coded path in there.

> The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.

I suppose you need the other DLLs and resources, too? What I tried was to copy the VM files over to the contents from the ZIP file. That worked. :-)

Best,
Marcel

Am 17.03.2021 14:06:05 schrieb Thiede, Christoph <[hidden email]>:

Hi Marcel,


it's not working for me:


---------------------------
Squeak!
---------------------------
Could not open the Squeak image file 'C:\Users\Christoph\OneDrive\Downloads\Squeak-4.4-All-in-One\Squeak-4.4-All-in-One.app\Contents\Resources\Squeak4.4-12327.image'

There are several ways to open an image file. You can:
  1. Double-click on the desired image file.
  2. Drop the image file onto the application.
Aborting...

---------------------------
OK   
---------------------------

How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only. Dropping another image file on the executable only gives me a load error from within the 4.4 image. The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Mittwoch, 17. März 2021 09:13:04
An: squeak-dev
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi Christoph.

> Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Frank-170908 is based on Squeak 4.4 alpha. The VM from the Squeak 4.4 release works fine under Windows 10: http://files.squeak.org/4.4/

Best,
Marcel

Am 17.03.2021 01:36:08 schrieb Thiede, Christoph <[hidden email]>:

Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 01:02:26
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki



Carpe Squeak!
Yoshiki Ohshima-3
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [ANN] SimulationStudio and sandboxed execution for Squeak

Yoshiki Ohshima-3
It is great to hear that image is working there.

Yes, in the current implementation, things have to use the custom compiler to hook the action. It is, though, conceivable to make Object be Worlds-aware and all objects would behave that way...  (including the data structure that supports the Worlds mechanism, or not!), except all primitives that simply write data into objects.

The JS implementation is a bit different and it compiles a given program into a Worlds-aware form.

Aran Lunzer found that the simple approach has some unexpected behavior, and he and I wrote a report on it and how to mitigate it. It might be of your interest:



On Wed, Mar 17, 2021 at 6:29 AM Thiede, Christoph <[hidden email]> wrote:

Hi Marcel,


ah, that was missing! Thank you. :-)


Some first impressions of the Worlds implementation for Squeak: If I understand correctly, it only works with objects that explicitly support the worlds mechanism. So for example:

z := #(1).
world := WWorld thisWorld sprout.
{world eval: [
z at: 1 put: 2.
z].
z}  "==> #(#(2) #(2))

Sandbox would answer:

z := #(1).
{Sandbox evaluate: [
z at: 1 put: 2.
z].
z}  "==>  #(#(2) #(1))"

For Worlds, I apparently would need to use a specialized WArray instead. This has the advantage of being much faster than Sandbox but also the disadvantage of not fitting for general-purpose Smalltalk expressions. Still, a very interesting approach!

Best,
Christoph
Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Mittwoch, 17. März 2021 14:12:12
An: squeak-dev
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi Christoph.

> How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only.

Change the .ini file :-) Remove the hard-coded path in there.

> The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.

I suppose you need the other DLLs and resources, too? What I tried was to copy the VM files over to the contents from the ZIP file. That worked. :-)

Best,
Marcel

Am 17.03.2021 14:06:05 schrieb Thiede, Christoph <[hidden email]>:

Hi Marcel,


it's not working for me:


---------------------------
Squeak!
---------------------------
Could not open the Squeak image file 'C:\Users\Christoph\OneDrive\Downloads\Squeak-4.4-All-in-One\Squeak-4.4-All-in-One.app\Contents\Resources\Squeak4.4-12327.image'

There are several ways to open an image file. You can:
  1. Double-click on the desired image file.
  2. Drop the image file onto the application.
Aborting...

---------------------------
OK   
---------------------------

How am I supposed to tell the VM which image file to load? It always wants to open the Squeak4.4-12327.image file only. Dropping another image file on the executable only gives me a load error from within the 4.4 image. The message above is a result of replacing the default image and changes file in the AllInOne bundle with frank.image and frank.changes.


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Mittwoch, 17. März 2021 09:13:04
An: squeak-dev
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi Christoph.

> Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Frank-170908 is based on Squeak 4.4 alpha. The VM from the Squeak 4.4 release works fine under Windows 10: http://files.squeak.org/4.4/

Best,
Marcel

Am 17.03.2021 01:36:08 schrieb Thiede, Christoph <[hidden email]>:

Hm, the 4.1 4.2 4.3 VMs refuse to open this image. I also tried to load Grease-yo.69.mcz into a fresh 4.2 image but it hangs up as well.

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 01:02:26
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
It was created on Squeak4.2, Squeak4.4, etc.  I think this one has stuff preloaded:

And you need to find a VM something along the line of:

 'Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.56] Croquet Cog 4.0.0'

Let me (us) know if you have more questions!

On Tue, Mar 16, 2021 at 4:39 PM Thiede, Christoph <[hidden email]> wrote:

My image hangs when loading Greases ... Can you recommend an older version of Squeak which is officially supported? :-)


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Mittwoch, 17. März 2021 00:20:53
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
W2Compiler, right? It took sometime to dig it up but it is in the Grease package in the directory.  The main things is to compile AssingmentNode into a message call, and to avoid conflict such accessors were prefiexed with "wiv666" ("world instance variable 666") The majority of job is done by the W2Parser.

On Tue, Mar 16, 2021 at 3:55 PM Thiede, Christoph <[hidden email]> wrote:

Thanks! Unfortunately, it's referring to a WCompiler which apparently is not part of the mcz file ...


Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Yoshiki Ohshima <[hidden email]>
Gesendet: Dienstag, 16. März 2021 23:48:52
An: Vanessa Freudenberg
Cc: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 
Hi!

We still maintain the packages from VPRI.  Looking at the directory, we have quite a few versions of Worlds.  So there is a chance that it does something.


On Tue, Mar 16, 2021 at 1:59 PM Vanessa Freudenberg <[hidden email]> wrote:
On Tue, Mar 16, 2021 at 9:52 AM Thiede, Christoph <[hidden email]> wrote:

One more question: Do you know where I can find the Smalltalk implementation of worlds? Is there any at all? The abstract in ResearchGate mentions this but is not addressed in the paper ...

Well luckily one of the authors is still on this list ... 
 
Looks really interesting and useful!

Vanessa

Best,

Christoph

Von: Thiede, Christoph
Gesendet: Dienstag, 16. März 2021 15:10:37
An: [hidden email]
Betreff: AW: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

Hi Subbu, thanks a lot for your thoughts! :-)


> I am not sure if you have read Warth's research at VPRI


No, I have not. Looks very interesting, I will read this!

The only literature I had found on this (I have to admit that I did not spend much time on research) was this one: https://wiki.squeak.org/squeak/uploads/2074/sandbox.pdf It mentions a number of problems but does not come up with a simulation solution, so it would exclusively lock your image, I guess.

> I expect efficiency to improve if VM (e.g. object memory) exposes primitives for object spaces with copy-on-write and white-outs (for objects finalized in child space but not in parent). A sandbox could carve out an object space to hold modified/deleted objects and then either commit to parent or dispose it off on close. ObjectMemory already has support for multiple spaces (e.g. old and young). It needs to be exposed, with lots of care, to ST-side code.

Yeah, integrating the sandbox partially into the VM would make things faster, of course ... I haven't made any attempts in this direction for two reasons: First because I, honestly, still did not find the time for getting started with VMMaker, and second because while I know that the VM is written in a Smalltalk dialect as well, it hurts me a bit that it lacks the liveness of manipulating everything from within the running image. By the way, is there already some research about unioning these two worlds in order to reprogram the VM from within the image that is running in it? I would love this.

What are white-outs? I could not find any reference on this.

> Handling primitives, particularly those that involve physical i/o, is a difficult problem. This is typically solved by using virtualization and double buffering (e.g. display or input). It is ok to work with non-volatile state variables for the first cut and then introduce virtualization for transactional access.

Actually, my current Sandbox approach does not even intend to "commit" changes back to the rest of the image (rather to work like a Docker container in small). Thus my goal would rather be to virtualize all physical operations but keeping the effects back from the real physical devices. For filesystem accesses, for example, I thought about redirecting all write operations to a copy of the files in a special folder ... Not sure how far this approach would work well, though.

For some I/O primitives, however, virtualization is rather easy. For example, when writing on the global Display, you can simply copy this object like any other object into your sandbox space/world. However, other primitives, such as from the FilePlugin, operate on state is managed outside of the image ...

Best,
Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von K K Subbu <[hidden email]>
Gesendet: Dienstag, 16. März 2021 14:21:16
An: [hidden email]
Betreff: Re: [squeak-dev] [ANN] SimulationStudio and sandboxed execution for Squeak
 

On 16/03/21 4:51 pm, Thiede, Christoph wrote:
> Hi all! :-)
>
>
> I'm very excited to finally share my new project with you today, which I
> call *SimulationStudio* and have made available on GitHub under [1]. Its
> primary function is to provide an inheritable
> *SimulationContext* subclass of Context that makes it easy to simulate
> entire call trees while customizing the execution or installing custom
> hooks for various tracing and measurement purposes. Another
> accomplishment is the *Sandbox* which allows you to run arbitrary
> Smalltalk code in an isolated environment, separating any side effects
> that occur during the simulation from the rest of the image.

This is an amazing development!

I am not sure if you have read Warth's research at VPRI:

World: Controlling the Scope of Side Effects by A Warth et. all,

 
https://www.researchgate.net/publication/221496497_Worlds_Controlling_the_Scope_of_Side_Effects

Worlds is like your Sandbox. It also had a commit method to propagate
state changes from the child to the parent. This is useful when a method
modifies multiple state variables subject to strong invariants. A method
may open a World, make changes and then verify invariants are preserved
before committing the changes and closing the World.

> *Limitations and challenges*
>
> Well, first of all ... *performance.* :-) Some recent measurements that
> I have run have shown that the simulator reaches about 0.1 percent (sic)
> of the speed of the VM executor, depending on the domain and the
> distribution of bytecode operations.

I expect efficiency to improve if VM (e.g. object memory) exposes
primitives for object spaces with copy-on-write and white-outs (for
objects finalized in child space but not in parent). A sandbox could
carve out an object space to hold modified/deleted objects and then
either commit to parent or dispose it off on close. ObjectMemory already
has support for multiple spaces (e.g. old and young). It needs to be
exposed, with lots of care, to ST-side code.

> Another challenge is the *handling of primitive operations* during the
> sandbox execution.

Handling primitives, particularly those that involve physical i/o, is a
difficult problem. This is typically solved by using virtualization and
double buffering (e.g. display or input). It is ok to work with
non-volatile state variables for the first cut and then introduce
virtualization for transactional access.

Regards .. Subbu




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki




--
-- Yoshiki
Free forum by Nabble Edit this page