yet another smallest snapshot, and a visualization

Hi--

        I've reduced the size of the smallest snapshot[1] a bit, to 1,337 bytes
(actually 1,108 bytes with padding added for humor value[2] :). You can
also see a visualization of it[3].


-C

[1] http://netjam.org/spoon/smallest.image
[2] google "1337"
[3] http://netjam.org/spoon/smallest.gif

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Am 19.01.2006 um 20:52 schrieb Craig Latta:

Way, way, cool. Do we have a tool to explore an image? This thing is  
so small you could have a complete object diagram of it, with all the  
pointers ...


- Bert -

Hi Bert--

 > Do we have a tool to explore an image? This thing is so small you
 > could have a complete object diagram of it, with all the pointers...

        That'd be fun. :)  At the moment I've just got my viz tool, where I
click on a byte of the drawing and a simulator prints out a description
of the corresponding object to the Transcript (e.g., [1]).


-C

[1] http://netjam.org/spoon/viz/browser

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

On 19-Jan-06, at 4:28 PM, Craig Latta wrote:

>
> Hi Bert--
>
> > Do we have a tool to explore an image? This thing is so small you
> > could have a complete object diagram of it, with all the pointers...
>
> That'd be fun. :)  At the moment I've just got my viz tool, where  
> I click on a byte of the drawing and a simulator prints out a  
> description of the corresponding object to the Transcript (e.g., [1]).
Hook it all up to Trygve's BabyUML tools!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
New: It comes in different colors from the previous version.

That was pretty funny.  I had no idea about 1337.  

(Must be getting old)

|20n

Hey Craig, I am amazed and mystified by this work you
are doing.  

Can you help myself a layperson understand this
better?  To do 3+4, you have to send a message (#+) to
an instance of SmallInteger (ok, maybe just the
immediate object?).  Still, you have two
CompiledMethods in there, so you have CompiledMethod
class (but obviously not all of its superclasses).
Classes have a lot of information, how can all this
fit into 18 objects?

My crude unerstanding of Spoon is that many other
objects are dynamically faulted into this tiny image
at run-time (to do the 3+4), it this right?  So are
you able to measure how many objects are faulted in at
run-time?

I just want to learn, thanks,
  Chris

Hi Chris--

        The short answer is that you can leave out a lot of things when you
know you're going to terminate within a few instructions. :)

 > My crude unerstanding of Spoon is that many other objects are
 > dynamically faulted into this tiny image at run-time (to do the 3+4),
 > is this right?

        This snapshot is not one that can grow into something more useful. It
was just an experiment to see how small a snapshot I could make, with
the new system tracer I wrote. I'm still working on a small growable
system (currently 162k uncompressed), which does operate as you say (in
addition to performing some other tricks).

        When you run the 1337 snapshot, the virtual machine gets the object
address ("oop") for the special-objects array from the snapshot header
(first 60 bytes). Indirectly from that array (through the Processor and
its active process) it finds the active context and continues execution.
The instructions executed come from a method used by the context. The
instructions are: push (3), push (4), add, send (#quitPrimitive). The
send invokes >>quitPrimitive, another method. That method terminates the
virtual machine via a primitive. The virtual machine needs to look up
 >>quitPrimitive in the method dictionary of the class of Smalltalk, a
literal association in the literal frame of the initial context.

        In the above, we've touched the following objects from the snapshot
file, in the following order:

1. the special-objects array
2. the global Processor association
3. the Processor
4. the active Process
5. the active Process's suspended context
6. the now-active context's method
7. the symbol #quitPrimitive from the context's literal frame
8. the global Smalltalk association
9. the system dictionary (Smalltalk)
10. class SystemDictionary
11. SystemDictionary's method dictionary
12. SystemDictionary's method dictionary's association array
13. nil (touched during method lookup)
14. that array's association for key #quitPrimitive
15. the value for key #quitPrimitive (the second method)

        The method dictionary of the first method is in there also, as is true
and false, making 18 objects. (The numbers 3 and 4 are immediates in the
first method's literal frame.) I could kill those last three, but I
declared victory when I got the thing under 1,337 bytes. :)

        I did all this using a system tracer implemented as a specialization of
the virtual machine simulator. It's a simulator which keeps track of
every object it touches as it runs. At any point, you can stop it and
investigate the touched objects (read descriptions, follow pointers,
etc.). Their cached oops are properly remapped during garbage
collections. I used this information to guide a final garbage collection
with extremely aggressive mark phase (basically, only mark the touched
objects, and don't trace them), then just wrote a snapshot normally.

        So...

 > To do 3+4, you have to send a message (#+)...

        No, it has its own instruction ("bytecode").

 > ...to an instance of SmallInteger (ok, maybe just the immediate
 > object?).

        Right, small integers are immediate.

 > Still, you have two CompiledMethods in there, so you have
 > CompiledMethod class (but obviously not all of its superclasses).
 > Classes have a lot of information, how can all this fit into 18
 > objects?

        As you can see (hopefully) from the above, the only class we need is
the one for the method for the single message-send we're doing
(>>quitPrimitive). And the only one of its fields we need is the one for
its method dictionary.

 > So are you able to measure how many objects are faulted [into the
 > growable sytsem] at run-time?

        Yes. :)

 > I just want to learn, thanks...

        Thanks for asking! It was good to write this down.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Hi Craig -

>     When you run the 1337 snapshot, the virtual machine gets the object
> address ("oop") for the special-objects array from the snapshot header
> (first 60 bytes). Indirectly from that array (through the Processor and
> its active process) it finds the active context and continues execution.

This is interesting to me. Do you think it is feasable to take out all
of the process management from the VM and put it back in, say, via a
plugin? There are probably a few issues wrt. timers etc. but what I'd be
curious about is whether such a VM could be used as a step towards a
native multi-threaded version (e.g., run multiple versions of that VM to
achieve concurrency)?

Cheers,
   - Andreas

On 20-Jan-06, at 5:53 PM, Andreas Raab wrote:
I think so. No need for the special objects array or Processor  
instance if the saved context oop is either explicitly saved in the  
image header or simply put somewhere by convention (like nil true and  
false used to be) which would be trivial with Craig's version of the  
tracer. You could start a copy of a VM for each process and terminate  
it when the process concludes. This VM would need no process handling  
stuff such as wakeHighestPriority but might need some semaphore stuff?
Debugging might be fun. Coordinating threads would have the potential  
for getting to be as awful as java. On the other hand it wold be an  
interesting way of making completely separated cells of objects each  
running their own process and having to communicate only via proper  
interfaces instead of messing in a global object soup. All sorts of  
fun possibilities if one has the tim and funding

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Machine-independent:  Does not run on any existing machine.

Hi Andreas--

 > > ...the virtual machine gets the object address for the
 > > special-objects array from the snapshot header... Indirectly from
 > > that array (through the Processor and its active process) it finds
 > > the active context and continues execution.
 >
 > This is interesting to me. Do you think it is feasable to take out all
 > of the process management from the VM and put it back in, say, via a
 > plugin?

        Yes indeed!

        As Tim mentioned, we could change the snapshot header format to get rid
of the scheduler and several of the other "special" objects from the
initial snapshot (I wanted the 1337 snapshot to be runnable by VMs
currently out there).

 > There are probably a few issues wrt. timers etc. but what I'd be
 > curious about is whether such a VM could be used as a step towards a
 > native multi-threaded version (e.g., run multiple versions of that VM
 > to achieve concurrency)?

        Sure. In fact, concurrent operation (but across many distinct machines)
is one of the things I plan to do with Spoon.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Craig Latta puso en su mail :

> I did all this using a system tracer implemented as a specialization of
> the virtual machine simulator. It's a simulator which keeps track of
> every object it touches as it runs. At any point, you can stop it and
> investigate the touched objects (read descriptions, follow pointers,
> etc.). Their cached oops are properly remapped during garbage
> collections. I used this information to guide a final garbage collection
> with extremely aggressive mark phase (basically, only mark the touched
> objects, and don't trace them), then just wrote a snapshot normally.

This kind of simulator is available for normal Squeak ?
Sounds as a dream have tool.

Congratulations again!

Edgar



       

       
               
___________________________________________________________
1GB gratis, Antivirus y Antispam
Correo Yahoo!, el mejor correo web del mundo
http://correo.yahoo.com.ar 

On 21.01.2006 09:02, Craig Latta wrote:
>...

>  > There are probably a few issues wrt. timers etc. but what I'd be
>  > curious about is whether such a VM could be used as a step towards a
>  > native multi-threaded version (e.g., run multiple versions of that VM
>  > to achieve concurrency)?
>

> Sure. In fact, concurrent operation (but across many distinct machines)
> is one of the things I plan to do with Spoon.

I like this idea!

Multiple image/VM combinations could be seen as VMObjects understanding
messages from other ones (communication between them seen as message
passing). And a cluster of them could be aggregated to a higher level
VMClusterObject running at a cluster controlling machine...

If there would be direct references to external objects (something like
squeak://somehost/vmProcessId/objectPointer), which had to be accessed
via message passing to another VM, the problem of VM spanning GC of
external objects would arise. But by encapsulating object spaces into
VMObjects (not allowing direct access to external objects inside them)
this problem could be avoided: resulting objects of messages to them
could just be newly created objects in the receiver VMObject (under
control of its GC).


Regards,
Stephan

PS: I think I should study the whitepapers about Croquet...

>
>
> -C
>

--
Stephan Rudlof ([hidden email])
   "Genius doesn't work on an assembly line basis.
    You can't simply say, 'Today I will be brilliant.'"
    -- Kirk, "The Ultimate Computer", stardate 4731.3

Hi Stephan--

 > If there would be direct references to external objects (something
 > like squeak://somehost/vmProcessId/objectPointer)...

        (See class Other in Spoon, what I use for proxies.)

 > ...which had to be accessed via message passing to another VM, the
 > problem of VM spanning GC of external objects would arise. But by
 > encapsulating object spaces into VMObjects (not allowing direct access
 > to external objects inside them) this problem could be avoided:
 > resulting objects of messages to them could just be newly created
 > objects in the receiver VMObject (under control of its GC).

        I'm just planning to make instances of Other play nice with finalization.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Hi Edgar--

 > This kind of simulator is available for normal Squeak?

        It's usable in normal Squeak, yes; it doesn't depend on any of the
Spoon virtual machine changes. But I just wrote it and haven't gotten
around to releasing it. :)

 > Congratulations again!

        Thanks!


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

On Jan 21, 2006, at 12:28 AM, tim Rowledge wrote:

Indeed. The end result would be message-passing concurrency, rather  
than shared-state concurrency. Each VM instance would be a Vat/Island/
Task and you'd have distant object references for sending messages  
between them. VM instances could also be on a separate thread, in a  
separate process or on a separate machine, depending on what kind of  
performance characteristics you were looking for, pretty much  
transparently. Beautiful.

On 21-Jan-06, at 10:56 AM, Colin Putney wrote:
That's about it. It is not a million miles from stuff some of us were  
fantasizing about a while ago (1988 or thereabouts) in UK as part of  
an Esprit project. As with so many Esprit projects it went nowhere as  
the money was siphoned off to the mafia and other corporate slush  
funds and then the people scattered to the four winds. Maybe we could  
actually do it this time round? Anyone interested in funding the work  
properly?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IG: Insert Garbage

On Sat, 21 Jan 2006 11:23:19 -0800, tim Rowledge <[hidden email]> wrote:

> That's about it. It is not a million miles from stuff some of us were  
> fantasizing about a while ago (1988 or thereabouts) in UK as part of an  
> Esprit project. As with so many Esprit projects it went nowhere as the  
> money was siphoned off to the mafia and other corporate slush funds and  
> then the people scattered to the four winds. Maybe we could actually do  
> it this time round? Anyone interested in funding the work properly?

What does it mean: "Fund the work properly"?

I could provide leg breaking services....

I was thinking about something similar today.

My fantasy is to have a smalltalk system that's friendly to professional
software developers and it occures to me that the ability to run a program
in is own memory space making it free of the possibility of being reverse
engineered, might be important to some developers.  But you'd still want to
be able to communicate with the smalltalk desktop, etc, thus the message
passing.

And speaking of concurency, another thing that I think would make squeak
more attractive to developers would be to make the user interface run in a
separate, preemtive thread or threads.

The current design where any program can bring the user interface to a dead
halt effectively crashing the desktop is unacceptably anacronistic.  For
instance, if you start Squeakmap on a system with a slow modem internet
connection, the whole desktop is locked out for minutes.  If an error
occures in the wrong part of the negotiation, Squeak is permanently locked
up and the only thing you can do is kill the process.

Joshua Scholar

----- Original Message -----
From: "Colin Putney" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Saturday, January 21, 2006 10:56 AM
Subject: Re: yet another smallest snapshot, and a visualization

I *really* need to work on my Occlumency skills ;-)

Cheers,
   - Andreas

Colin Putney wrote:
> Indeed. The end result would be message-passing concurrency, rather  
> than shared-state concurrency. Each VM instance would be a Vat/Island/
> Task and you'd have distant object references for sending messages  
> between them. VM instances could also be on a separate thread, in a  
> separate process or on a separate machine, depending on what kind of  
> performance characteristics you were looking for, pretty much  
> transparently. Beautiful.

On Jan 21, 2006, at 5:38 PM, Andreas Raab wrote:

> I *really* need to work on my Occlumency skills ;-)

Well, publishing Islands was a pretty big tip-off. :-)

I've been mulling this over from time to time as well. It occurred to  
me that one of the big drawbacks to message passing concurrency is  
that Islands are much heavier than Processes. That would be even more  
true if they were based on native threads or OS processes.

But, since we're talking about substantially modifying the VM, there  
might be a way to avoid bloat. If we allowed class pointers to be  
distant as well, we could have a system that created relatively small  
Islands, starting with a single object. Islands would grow as new  
objects were created inside them, message arguments were passed by  
value, or CompiledMethods, Classes, literals and the like were pulled  
into the Island spoon-style.

This kind of a system would require rethinking the way the  
development environment works, system changes are managed etc, since  
there isn't really a clearly-delimited "image" anymore. It would be  
interesting to see what kind of system emerged from that.

Colin