HydraTools and minimal images

On Fri, 15 Feb 2008 13:56:37 +0100, Igor Stasenko wrote:

...[snipped a lotta rock'n roll]...
> For this reasons, i'd like to operate with interpreters using handles
> instead of indexes.

Here you seem to demonstrate an enormous lack of inconsistency :( In a  
previous posting you advertise use of an index

-  
http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-February/125150.html

which suddenly no longer shall be used?

> This, in particular, can guarantee that if you save your image which
> contains references to handles, they will render invalid during next
> squeak session.

Who wants that, what would that be good for?

> While indexes is not.

?

/Klaus

On 15/02/2008, Klaus D. Witzel <[hidden email]> wrote:
It's just doits. If you look at regular code in classes i never using
#at: patterns, only handles.

>  -
>
> http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-February/125150.html
>
>
> which suddenly no longer shall be used?
>

you can use them in doits, i'm just wanted to say, that in objects it
better to keep handles, not indexes.

>
>  > This, in particular, can guarantee that if you save your image which
>  > contains references to handles, they will render invalid during next
>  > squeak session.
>
>
> Who wants that, what would that be good for?

Don't ask me who may wants that. I don't know :)
If you keeping an indexes within objects, then after restart, your
code will continue working not suspecting that there is no image with
given index or even worse , there is, but its serving for different
purposes.


--
Best regards,
Igor Stasenko AKA sig.

On Fri, 15 Feb 2008 16:56:45 +0100, Igor Stasenko wrote:
[oops, s/of incons/of cons/]
>>  previous posting you advertise use of an index
>>
>
> It's just doits.

How do you mean that, is (sms := HydraSMS doitAt: 2), the statement you  
seem to be concerned about, not doit?

> If you look at regular code in classes i never using
> #at: patterns, only handles.

No, I looked at HydraVM>>#doit:at:, subtracted one colon and therefore got  
HydraSMS>>#doitAt: ... :)

>>  -
>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-February/125150.html
>>
>> which suddenly no longer shall be used?
>>
>
> you can use them in doits,

But I did ! (sms := HydraSMS doitAt: 2)

> i'm just wanted to say, that in objects it
> better to keep handles, not indexes.

I cannot see any reason for this. The collection is created and populated  
by+after startup.

BTW: can you run a snapshot of a Hydra-infected .image (which may contain  
your handles) with a non-Hydra interpreter .exe?

>>
>>  > This, in particular, can guarantee that if you save your image which
>>  > contains references to handles, they will render invalid during next
>>  > squeak session.
>>
>>
>> Who wants that, what would that be good for?
>
> Don't ask me who may wants that.

Okay. Forget I asked.

> I don't know :)
> If you keeping an indexes within objects, then after restart, your
> code will continue working not suspecting that there is no image with
> given index or even worse , there is, but its serving for different
> purposes.
>
>

On 15/02/2008, Klaus D. Witzel <[hidden email]> wrote:
Primitives accept only handles. So you should use
#pritimiveInterpreterInstanceAt
or HydraVM interpreters at: to obtain a handle.

>
>  > i'm just wanted to say, that in objects it
>  > better to keep handles, not indexes.
>
>
> I cannot see any reason for this. The collection is created and populated
>  by+after startup.
>
HydraVM>>#interpreters returning a freshly populated collection each
time you calling it.
Actually this method was for convenience to iterate through all
interpreter instances.


>  BTW: can you run a snapshot of a Hydra-infected .image (which may contain
>  your handles) with a non-Hydra interpreter .exe?
>
Yes, i can. A handles not anything special but just integers at language side.
In same way, as SocketHandle is just bytearray in ST, but pointer to
structure for VM.


Okay, a little more reasoning, and i will change primitives to use
indexes instead of handles.
The only reason why its better is only for speed reasons: all
primitives should check if handle is valid, so it you have 100 images,
it can take to iterate through list of 100 intems to verify that
handle is valid.
But for security reasons, i'm still thinking that handles are better.

--
Best regards,
Igor Stasenko AKA sig.

On 15/02/2008, Keith Hodges <[hidden email]> wrote: You right, this is better, and should look like that.
Yes, it would be better to have a HydraInterpreter class, which is a
wrapping around handle.
Don't blame me for bad design, i was mainly preoccupied with VM , not ST side :)

>  Keith
>
>
>


--
Best regards,
Igor Stasenko AKA sig.

Okay, i think i have a solution which will suit both of us.
By adding a HydraInterpreter class, instances of which will be used in
primitives.
Class should have two fields: index and handle.
Primitives then checking:
- if given argument is of class HydraInterpreter,
then reads index and handle.
then checks if by given index VM having same interpreter handle, as
specified in HydraInterpreter instance.
If not, then it tries to search its list of interpreters to find given
handle, and if found, updates index automatically.
If handle not found, then primitive fails.

--
Best regards,
Igor Stasenko AKA sig.

On Fri, 15 Feb 2008 23:57:58 +0100, Igor Stasenko wrote:
...
>>  > you can use them in doits,
>>
>>
>> But I did ! (sms := HydraSMS doitAt: 2)
>>
>
> Primitives accept only handles. So you should use
> #pritimiveInterpreterInstanceAt
> or HydraVM interpreters at: to obtain a handle.

I don't care; I'll be using the same as your #doit:at: is using, whatever  
object that is :)

No objection.

> Actually this method was for convenience to iterate through all
> interpreter instances.

No, not for convenience. Do you think that Hydra is usable for  
*whatsoever* without the developer being able to enumerate all existing  
interpreter instances?

Form your discussion with Keith you have instances of HydraInterpreter.  
Now look at this

  | notWantedByIgor |
  notWantedByIgor := HydraInterpreter allInstances.
  notWantedByIgor at: 2

>
>>  BTW: can you run a snapshot of a Hydra-infected .image (which may  
>> contain
>>  your handles) with a non-Hydra interpreter .exe?
>>
> Yes, i can. A handles not anything special but just integers at language  
> side.
> In same way, as SocketHandle is just bytearray in ST, but pointer to
> structure for VM.

Good to know, can save lots of headache at debug time :)

>
> Okay, a little more reasoning, and i will change primitives to use
> indexes instead of handles.

Not necessary, not for me. It is perhaps possible that you interpret too  
much into my #doitAt:, I thought I'd index your collection as is done by  
your #doit:at:, in the way you provide it now (unless you no longer  
support your own code for #doit:at: ;-)

> The only reason why its better is only for speed reasons: all
> primitives should check if handle is valid, so it you have 100 images,

You cannot do anything with Hydra when you *not* check that your handle is  
valid (anything other than potentially crash the system or so).

> it can take to iterate through list of 100 intems to verify that
> handle is valid.

There's always some price to pay, when you want to maintain control over  
your asynchronous computations. Another possibility would be to base  
absence(dead), presence(alive), other state(busy,idle) of your interpreter  
instances on events. Have you considered using events for that, would that  
be easy to implement in Hydra?

> But for security reasons, i'm still thinking that handles are better.

Can you implement your #doit:at: using only your "better" handles, without  
enumerating the interpreter instances?

>

On 13-Feb-08, at 8:32 AM, David Mitchell wrote:

> I'm very interested in OB-Web
>
> Tried loading it yesterday through Monticello, but couldn't figure get
> it running.
>
> I'm going to keep trying, but pointers would be useful.


Yeah, it's a little tricky still. Here's the procedure:

Start with a stock 3.7 image. It may work on other versions of Squeak,  
but I haven't tried it. You can download the image here:
http://ftp.squeak.org/3.7/Squeak3.7-5989-basic.zip

Then download FastSocketStream and install it:

http://map.squeak.org/accountbyid/9bdedc18-1525-44a6-9b79-db5d4a87f6f8/files/FastSocketStream-gk.4.cs.gz

Load your favorite version of Monticello, and add the OmniBrowser  
repository:

http://source.wiresong.ca/ob

Then load the following packages:

DynamicBindings-gk.1
KomServices-avi.4
KomHttpServer-ac.9
JSON-cwp.19
OmniBrowser-dc.405
OB-Morphic-lr.45
OB-Web-cwp.59
OB-Standard-cwp.322

In a workspace, execute "OBKomHttpService startOn: 8000."

You'll need to use Safari 3 or Firefox 2. Make sure pop-up blocking is  
turned off, then visit:

http://localhost:8000/ and click on the link.

Context menus work, but keyboard shortcuts only work in the text pane.  
Auto-correction when compiling methods works, but only if you load the  
attached ChangeSet, which fixes bugs in Parser.





If you want to hack on it, you'll want to load some tools for  
modifying the Javascript and CSS via FTP. From the OmniBrowser  
repository, load:

FTP-cwp.27
OB-Dev-FTP-cwp.3

Then you can execute "FTPServer startForOBDev" and connect an FTP  
client to the following URL:

ftp://localhost:10021/

You'll also want to load the tests from the OmniBrowser repository:

OB-Fake-lr.9
OB-Tests-Core-cwp.65
OB-Tests-Morphic-cwp.16
OB-Tests-Web-cwp.27
OB-Tests-Standard-dr.86

Whew! Clearly this has to be cleaned up for OB-Web to be more than a  
toy. Anyway have fun.

Colin

On Sat, 16 Feb 2008 00:15:43 +0100, Igor Stasenko wrote:

> Okay, i think i have a solution which will suit both of us.

For the records: in the previous I just tried to use *your* existing index  
into *your* existing collection :)

> By adding a HydraInterpreter class, instances of which will be used in
> primitives.

That sounds good.

> Class should have two fields: index and handle.
> Primitives then checking:
> - if given argument is of class HydraInterpreter,
> then reads index and handle.

How would you map one of these indexes to its corresponding instance of  
HydraInterpreter, how would your #doit:at: do that mapping?

> then checks if by given index VM having same interpreter handle, as
> specified in HydraInterpreter instance.
> If not, then it tries to search its list of interpreters to find given
> handle, and if found, updates index automatically.
> If handle not found, then primitive fails.
>

Looks like a lot of work (if only for your index); did you compare against  
simple (HydraInterpreter allInstances) ? This (or something like this)  
needs only be done after a new HydraInterpreter was  
created(copied)/started, and, depending on needs, after one of the  
HydraInterpreter instances gone away.

And Smalltalk's collection hierarchy has subclasses which are accessible  
by keys (indexes) which don't need to be stored anywhere (the diagram on  
page 146 of the Blue Book).

/Klaus

On 16/02/2008, Klaus D. Witzel <[hidden email]> wrote:
>
> There's always some price to pay, when you want to maintain control over
> your asynchronous computations. Another possibility would be to base
> absence(dead), presence(alive), other state(busy,idle) of your interpreter
> instances on events. Have you considered using events for that, would that
> be easy to implement in Hydra?
>

Well, i can add some code at the end of image startup phase
broadcasting to others that new image is ready for work. As well as,
at shutdown, a broadcast that it's going to vanish.
This can be done using current set of features (channels), no need to
put this code into VM.
In same way, for a busy/idle state: you may put code in image to
declare by yourself, when it is busy or idle, and respond with
appropriate message on request, if needed.
Channels are implemented using events, e.g. when you putting something
in channel, in reality its an event posted to corresponding event
queue.

index and handle is the result of previous calls to primitives which
should return a HydraInterpreter instance from now.
..erm.. do you mean by HydraInterpreter  as an interpreter at VM side,
or as an object?
Objects can contain a random values, nothing bad will happen if you
pass it to primitive.

A primitive #interpreterInstanceAt: will be refactored to return a
freshly created HydraInterpreter instance. The rest of code have to
deal with interpreters using HydraInterpreter, not handles or indexes.
So, in #doit: at: , first you converting an index to HydraInterpreter
instance (by invoking primitive), and then you're free to do anything
you want with it.

> And Smalltalk's collection hierarchy has subclasses which are accessible
> by keys (indexes) which don't need to be stored anywhere (the diagram on
> page 146 of the Blue Book).
>
> /Klaus
>


--
Best regards,
Igor Stasenko AKA sig.

On Sat, 16 Feb 2008 13:56:54 +0100, Igor Stasenko wrote:

Ahh, now we are talking events :)

> This can be done using current set of features (channels), no need to
> put this code into VM.

Exactly, I thought about the same.

> In same way, for a busy/idle state: you may put code in image to
> declare by yourself, when it is busy or idle, and respond with
> appropriate message on request, if needed.

For dead/alive: an interval timed object which periodically (3-4 secs)  
sends "I'm alive" ;-)

> Channels are implemented using events, e.g. when you putting something
> in channel, in reality its an event posted to corresponding event
> queue.

Sure.

Still unclear: your user (a developer) has an index, say an element from  
(1 to: n), for which you want to return her/him a HydraInterpreter  
instance from now on?

Cannot understand "as an interpreter at VM side": didn't you write that  
everything is an object on the Smalltalk side (even your handles, which  
seem to represent the magic)? To be sure, I didn't think about anything  
"behind" your handle.

And by HydraInterpreter instance I meant the one from your discussion with  
Keith. Perhaps I associate your handle with an instance of  
HydraInterpreter, whereas you treat them as different objects. No problem,  
just put a "represents" symbol between them.

BTW: isn't the HydraInterpreter name misleading (and causing confusion in  
the above). How about using another name, without "interpreter" in it,  
because your "as an interpreter at VM side" is something else.

> Objects can contain a random values, nothing bad will happen if you
> pass it to primitive.

Thanks to the implementor of the primitive and to the writer of the  
corresponding test cases ;-)

>
> A primitive #interpreterInstanceAt: will be refactored to return a
> freshly created HydraInterpreter instance.

Ah, this is where you map elements of (1 to: n)? But didn't you write you  
are concerned that is slow, because you always have to enumerate your 100  
instances (example)?

And, did you consider to implement #= and #hash which are now necessary  
for all your freshly created HydraInterpreter instances?

> The rest of code have to
> deal with interpreters using HydraInterpreter, not handles or indexes.

Except when asking for (aHydraInterpreter index), because otherwise nobody  
except you would know?

> So, in #doit: at: , first you converting an index to HydraInterpreter
> instance (by invoking primitive),

No, I don't invoke primitives; I send messages to your Hydra  
classes+instances, whatever they are or will be :)

> and then you're free to do anything you want with it.

No doubt.

>> And Smalltalk's collection hierarchy has subclasses which are accessible
>> by keys (indexes) which don't need to be stored anywhere (the diagram on
>> page 146 of the Blue Book).
>>
>> /Klaus
>>
>
>

On 16/02/2008, Klaus D. Witzel <[hidden email]> wrote:
Right.
Hmm, i don't know.
We have HydraVM , which runs multiple interpreters and
HydraInterpreter "represents" one of them.
If you having better naming suggestions, just say it.
Yes, i did :)
By now, lets stop tempting peoples patience by discussing usage of
#at: patterns :)
I don't think this can be crucial for the system at the end :)


--
Best regards,
Igor Stasenko AKA sig.

On Sat, 16 Feb 2008 18:09:27 +0100, Igor Stasenko wrote:

> On 16/02/2008, Klaus D. Witzel wrote:
>> On Sat, 16 Feb 2008 13:56:54 +0100, Igor Stasenko wrote:
...
>> > index and handle is the result of previous calls to primitives which
>> > should return a HydraInterpreter instance from now.
>>
>> Still unclear: your user (a developer) has an index, say an element from
>> (1 to: n), for which you want to return her/him a HydraInterpreter
>> instance from now on?
>>
>
> Right.

</phew> and no further comment on index dilemma :)

...
>> > ..erm.. do you mean by HydraInterpreter  as an interpreter at VM side,
>> > or as an object?
...
>> BTW: isn't the HydraInterpreter name misleading (and causing confusion  
>> in
>> the above). How about using another name, without "interpreter" in it,
>> because your "as an interpreter at VM side" is something else.
>>
> Hmm, i don't know.
> We have HydraVM , which runs multiple interpreters and
> HydraInterpreter "represents" one of them.
> If you having better naming suggestions, just say it.

It doesn't interpret anything (it is as dead as a handle is dead), so I'd  
say HydraImageReference (like MethodReference etc).

/Klaus

P.S. any time frame for the next step(s) ?

On 16/02/2008, Klaus D. Witzel <[hidden email]> wrote:
Forgot to notice, that this would require registering HydraInterpreter
in special objects array, to be able primitives to deal with it.
If this is acceptable, then i will do the changes in VM and
primitives. If not, then i'll keep primitives untouched, but refactor
classes to use HydraInterpreter instances instead of handles.
Can't make a decision, if making HydraInterpreter a special object
worth doing or not. Meanwhile i'll start implementing a HydraStream.

Image is quite dead too - it's just a bunch of bytes sitting in memory.
See, when you do:

Smalltalk garbageCollect
Smalltalk version

it seems more logical than:

SmalltalkImage garbageCollect
SmalltalkImage version

so, i'd say that if you wanna do something with image/memory you
should ask interpreter to do it for you, not image.

In that way:
interpreter doit: 'Foo bar'
seem more logical than
image doit: 'Foo bar'

also consider that under the hood, it's actually an interpreter
instance, which in own turn refers to it's image and make it running,
but in addition it having own state and behavior not quite relevant to
image (object memory).


> /Klaus
>
> P.S. any time frame for the next step(s) ?
>

--
Best regards,
Igor Stasenko AKA sig.

On Mon, 18 Feb 2008 17:45:10 +0100, Igor Stasenko wrote:

I see that I still cannot see what you aim for. Can't you let the user  
(developer) make her/his decisions and, when she/he implements it as an  
xyzCollection with cool speedy abcIndex, just let them do so?

We discussed the "a new Hydra .image started", "and now one went away"  
events (also the heartbeat event). If someone wants to maintain a  
collection with this happenings, they should be free to do so.

> If this is acceptable, then i will do the changes in VM and
> primitives. If not, then i'll keep primitives untouched, but refactor
> classes to use HydraInterpreter instances instead of handles.

Sounds good.

> Can't make a decision, if making HydraInterpreter a special object
> worth doing or not.

If your handle is representable by 31bit quantity then it could be the  
first instance variable, like in CompiledMethod the header field. Would  
let sophisticated user add more instance variables (as opposed to, say  
MethodDictionary which is hardcoded).

But that can wait, no doubts.

> Meanwhile i'll start implementing a HydraStream.

And the reliable, bidirectional communication, will that be part of your  
stream thing or come later? I just want to know when I can start my  
attempt with HydraSMS, no real hurry (other things must be tested+working  
before that).

That depends from what the garbage disappears. Smalltalk is generally free  
of garbage and shame on those who claim the contrary ;-) But your running  
image has garbage, and that was created by you ! (you can't send your  
.image with garbage to someone else, no you can't ;-)

BTW someone (whether they saw the mess or had other reasons) already moved  
#version to SystemVersion anyways.

Fell free to see it so, also when usually (all the users of something) ~~  
(the creator of something) and the former have their own ideas of what is  
what. No need to forsee *everything*, I mean by that.

>
>
>> /Klaus
>>
>> P.S. any time frame for the next step(s) ?
>>
>

Just to remind that (Smalltalk version) is needed for compatibility
reasons with other dialects. Ah historical decisions...

Nicolas