Simulating the Cog V3 VM

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

re: Simulating the Cog V3 VM

Laura Perez Cerrato
 
I thought I'd share how things are coming along since the other day,
since you've all been so welcoming :) First of all, thanks for that!
As a newbie VM developer you've been of so much help.

After loading the fix Elliot uploaded earlier, everything seems to be
working as expected. A debbuger pops up whenever a redraw is issued
(from my understanding), displaying the message "Error: inconsistent
values: 2047 vs 0", which is triggered by calling BalloonArray>>at:.
However, I've noticed that there's a comment in that message stating
"Debug only..." on top of the error block, so I guess everything's
working fine. Proceeding performs the redraw accordingly. So, besides
this minor annoyance, the simulator seems to be working ok.

I've also tried generating the source for JPEGReadWriter2 only using
the snippet provided in one of the workspaces and compiling the
resulting sources; which also seems to be working accordingly.

I've noticed that building the Spur VMMaker image by means of the
scripts provided now fails when trying to automatically load all the
necessary packages to have a working VM simulation environment: when
initializing the last version of OSProcess, a debugger pops up
alerting that primitiveChangeClassTo: failed. Proceeding seems to
continue the package loading without further issues. On first starting
the image, the same debugger pops up. Proceeding then again seems to
have no effect in the things I've tried so far.

I'll now work on trying to debug my modifications to the plugin from
the VM simulator.

I also thought I'd share a bit more about the project I'm working on.
I'm working with Juan at Satellogic on satellite imagery processing.
As we mostly work with 8-bit deep images and currently the plugin only
supports writing 32-bit deep images to disk, we found adding support
for such a feature would come handy in time.

My apologies if my messages seem odd; I'm not used to participating in
mailing lists. I hope I can contribute something more valuable in the
future :)
-Laura Perez Cerrato


On 12 May 2016 at 12:23, Laura Perez Cerrato
<[hidden email]> wrote:

> Craig,
> Thanks for the offering! If you could point me in the right direction
> in order to take this approach I'd greatly appreciate it. Is there any
> documentation available on the subject?
>
> -Laura Perez Cerrato
>
>
> On 12 May 2016 at 10:23, Craig Latta <[hidden email]> wrote:
>>
>>
>> Hi--
>>
>>      Eliot writes:
>>
>>> The system is designed for one to be able to Instantiate a plugin and
>>> use it in objects in the current image by using an InterpreterProxy
>>> instance to interface the plugin code with ordinary Smalltalk
>>> objects.  But I've never done this so I can't help.  Perhaps
>>> someone with experience of dong this can provide help.
>>
>>      I've done this, for network access used by remote messaging between
>> two simulators on different machines (or a simulator and a normal
>> system). I originally took the LargeIntegerPlugin simulator support as
>> my example, but have worked with this technique extensively since then.
>> Laura, I'd be happy to help if you'd like.
>>
>>
>>      thanks,
>>
>> -C
>>
>> --
>> Craig Latta
>> Black Page Digital
>> Amsterdam
>> [hidden email]
>> +31   6 2757 7177 (SMS ok)
>> + 1 415  287 3547 (no SMS)
>>
Reply | Threaded
Open this post in threaded view
|

re: Simulating the Cog V3 VM

David T. Lewis
 
Thank you for these updates! It's very helpful to know what you are
working on.

Dave

>
> I thought I'd share how things are coming along since the other day,
> since you've all been so welcoming :) First of all, thanks for that!
> As a newbie VM developer you've been of so much help.
>
> After loading the fix Elliot uploaded earlier, everything seems to be
> working as expected. A debbuger pops up whenever a redraw is issued
> (from my understanding), displaying the message "Error: inconsistent
> values: 2047 vs 0", which is triggered by calling BalloonArray>>at:.
> However, I've noticed that there's a comment in that message stating
> "Debug only..." on top of the error block, so I guess everything's
> working fine. Proceeding performs the redraw accordingly. So, besides
> this minor annoyance, the simulator seems to be working ok.
>
> I've also tried generating the source for JPEGReadWriter2 only using
> the snippet provided in one of the workspaces and compiling the
> resulting sources; which also seems to be working accordingly.
>
> I've noticed that building the Spur VMMaker image by means of the
> scripts provided now fails when trying to automatically load all the
> necessary packages to have a working VM simulation environment: when
> initializing the last version of OSProcess, a debugger pops up
> alerting that primitiveChangeClassTo: failed. Proceeding seems to
> continue the package loading without further issues. On first starting
> the image, the same debugger pops up. Proceeding then again seems to
> have no effect in the things I've tried so far.
>
> I'll now work on trying to debug my modifications to the plugin from
> the VM simulator.
>
> I also thought I'd share a bit more about the project I'm working on.
> I'm working with Juan at Satellogic on satellite imagery processing.
> As we mostly work with 8-bit deep images and currently the plugin only
> supports writing 32-bit deep images to disk, we found adding support
> for such a feature would come handy in time.
>
> My apologies if my messages seem odd; I'm not used to participating in
> mailing lists. I hope I can contribute something more valuable in the
> future :)
> -Laura Perez Cerrato
>
>
> On 12 May 2016 at 12:23, Laura Perez Cerrato
> <[hidden email]> wrote:
>> Craig,
>> Thanks for the offering! If you could point me in the right direction
>> in order to take this approach I'd greatly appreciate it. Is there any
>> documentation available on the subject?
>>
>> -Laura Perez Cerrato
>>
>>
>> On 12 May 2016 at 10:23, Craig Latta <[hidden email]> wrote:
>>>
>>>
>>> Hi--
>>>
>>>      Eliot writes:
>>>
>>>> The system is designed for one to be able to Instantiate a plugin and
>>>> use it in objects in the current image by using an InterpreterProxy
>>>> instance to interface the plugin code with ordinary Smalltalk
>>>> objects.  But I've never done this so I can't help.  Perhaps
>>>> someone with experience of dong this can provide help.
>>>
>>>      I've done this, for network access used by remote messaging
>>> between
>>> two simulators on different machines (or a simulator and a normal
>>> system). I originally took the LargeIntegerPlugin simulator support as
>>> my example, but have worked with this technique extensively since then.
>>> Laura, I'd be happy to help if you'd like.
>>>
>>>
>>>      thanks,
>>>
>>> -C
>>>
>>> --
>>> Craig Latta
>>> Black Page Digital
>>> Amsterdam
>>> [hidden email]
>>> +31   6 2757 7177 (SMS ok)
>>> + 1 415  287 3547 (no SMS)
>>>
>


Reply | Threaded
Open this post in threaded view
|

re: Simulating the Cog V3 VM

Eliot Miranda-2
In reply to this post by Laura Perez Cerrato
 
Hi Laura, Hi Balloon Plugin Experts,

On Mon, May 16, 2016 at 10:15 AM, Laura Perez Cerrato <[hidden email]> wrote:

I thought I'd share how things are coming along since the other day,
since you've all been so welcoming :) First of all, thanks for that!
As a newbie VM developer you've been of so much help.

After loading the fix Elliot uploaded earlier, everything seems to be
working as expected. A debbuger pops up whenever a redraw is issued
(from my understanding), displaying the message "Error: inconsistent
values: 2047 vs 0", which is triggered by calling BalloonArray>>at:.
However, I've noticed that there's a comment in that message stating
"Debug only..." on top of the error block, so I guess everything's
working fine. Proceeding performs the redraw accordingly. So, besides
this minor annoyance, the simulator seems to be working ok.

Yes, I don't understand the Balloon simulation code enough to be able to fix this.  It has been a persistent annoyance.  I encourage anyone who does understand the code to take a look.  As I understand it there is an Array of floats in the plugin whose values should mirror an array of floating-point values in the simulated heap, and that inconsistencies arise because of bugs in the simulation code.  But as I say I don't understand the code well enough to fix it.  If you do have expertise here we can try and help you reproduce an example by simulating a specific image.


I've also tried generating the source for JPEGReadWriter2 only using
the snippet provided in one of the workspaces and compiling the
resulting sources; which also seems to be working accordingly.

I've noticed that building the Spur VMMaker image by means of the
scripts provided now fails when trying to automatically load all the
necessary packages to have a working VM simulation environment: when
initializing the last version of OSProcess, a debugger pops up
alerting that primitiveChangeClassTo: failed. Proceeding seems to
continue the package loading without further issues. On first starting
the image, the same debugger pops up. Proceeding then again seems to
have no effect in the things I've tried so far.

Ah, this is a V3 issue.  I'm sure that the same code won't fail under Spur, which has a more general change-class primitive.  Can you post the stack of the failure?  Perhaps there's a different way to implement it under V3.

I'll now work on trying to debug my modifications to the plugin from
the VM simulator.

I also thought I'd share a bit more about the project I'm working on.
I'm working with Juan at Satellogic on satellite imagery processing.
As we mostly work with 8-bit deep images and currently the plugin only
supports writing 32-bit deep images to disk, we found adding support
for such a feature would come handy in time.

Cool!

My apologies if my messages seem odd; I'm not used to participating in
mailing lists. I hope I can contribute something more valuable in the
future :)

Don't downplay your contribution.  Simply being involved and using the simulator for development is a big step that few developers attempt.  You're very welcome and your contribution is much appreciated.  Thank you.
 
-Laura Perez Cerrato


On 12 May 2016 at 12:23, Laura Perez Cerrato
<[hidden email]> wrote:
> Craig,
> Thanks for the offering! If you could point me in the right direction
> in order to take this approach I'd greatly appreciate it. Is there any
> documentation available on the subject?
>
> -Laura Perez Cerrato
>
>
> On 12 May 2016 at 10:23, Craig Latta <[hidden email]> wrote:
>>
>>
>> Hi--
>>
>>      Eliot writes:
>>
>>> The system is designed for one to be able to Instantiate a plugin and
>>> use it in objects in the current image by using an InterpreterProxy
>>> instance to interface the plugin code with ordinary Smalltalk
>>> objects.  But I've never done this so I can't help.  Perhaps
>>> someone with experience of dong this can provide help.
>>
>>      I've done this, for network access used by remote messaging between
>> two simulators on different machines (or a simulator and a normal
>> system). I originally took the LargeIntegerPlugin simulator support as
>> my example, but have worked with this technique extensively since then.
>> Laura, I'd be happy to help if you'd like.
>>
>>
>>      thanks,
>>
>> -C
>>
>> --
>> Craig Latta
>> Black Page Digital
>> Amsterdam
>> [hidden email]
>> <a href="tel:%2B31%20%20%206%202757%207177" value="+31627577177">+31 6 2757 7177 (SMS ok)
>> <a href="tel:%2B%201%20415%20%20287%203547" value="+14152873547">+ 1 415 287 3547 (no SMS)
>>



--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Simulating the Cog V3 VM

Juan Vuletich-3
In reply to this post by David T. Lewis
 
On 5/11/2016 9:05 PM, David T. Lewis wrote:

>
> On Wed, May 11, 2016 at 04:46:32PM -0700, tim Rowledge wrote:
>>
>>> On 11-05-2016, at 4:17 PM, Levente Uzonyi<[hidden email]>  wrote:
>>>
>>> Spur support has been added to SystemTracer a while ago.
>>> http://www.squeaksource.com/SystemTracing.html
>> Excellent; looks like other-Tim got to it and seems to have made something working. So, why isn???t that the solution to migrating Cuis?
>>
> Because it is not just a matter of converting the existing object memory,
> there are also changes in the image.
>
> Regarding SystemTracer, this is a nice light-weight approach to doing
> the conversion, but Eliot's bootstrap process that uses the simulation
> machinery in VMMaker has a lot of advantages too.

I understand. But in any case, have you, Levente, David or anyone else
produced a spur image that can start, using SystemTracer?

If so, using what subject image? And what host image?

Thanks!
Juan Vuletich

> In addition to Eliot's documentation of the image bootstrap process, you
> can also get a rough idea of the image-side differences by looking at
> the four packages in this repository from your up to date trunk Spur
> image:
>
>    MCHttpRepository
> location: 'http://www.squeaksource.com/TrunkUpdateStreamV3'
> user: ''
> password: ''
>
> This overstates the differences needed for bootstrapping a V3 image
> to Spur, but it does give a sense of where the differences lie.
>
> Dave
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Simulating the Cog V3 VM

timrowledge


> On 17-05-2016, at 6:01 AM, Juan Vuletich <[hidden email]> wrote:
>
> On 5/11/2016 9:05 PM, David T. Lewis wrote:
>>
>> On Wed, May 11, 2016 at 04:46:32PM -0700, tim Rowledge wrote:
>>>
>>>> On 11-05-2016, at 4:17 PM, Levente Uzonyi<[hidden email]>  wrote:
>>>>
>>>> Spur support has been added to SystemTracer a while ago.
>>>> http://www.squeaksource.com/SystemTracing.html
>>> Excellent; looks like other-Tim got to it and seems to have made something working. So, why isn???t that the solution to migrating Cuis?
>>>
>> Because it is not just a matter of converting the existing object memory,
>> there are also changes in the image.
>>
>> Regarding SystemTracer, this is a nice light-weight approach to doing
>> the conversion, but Eliot's bootstrap process that uses the simulation
>> machinery in VMMaker has a lot of advantages too.
>
> I understand. But in any case, have you, Levente, David or anyone else produced a spur image that can start, using SystemTracer?

The submit comments in the history of the SystemTracer2 package claim it has been used to trace a working Spur image. I imagine it would have been a contemporary v3 trunk image loaded with SystemTracer2.

Unless some very big changes have been made in Cuis that completely break the concept of a system tracer I have trouble imagining how it could be unable to produce a proper spur version. I don’t doubt there might be some interesting details to solve.
In the past I know the tracer has been used to convert bytecode sets, object formats, word sizes,  and method formats, as well as to create application specific images etc. I know from personal experience that it can be made to do a lot of interesting things.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
"Bother" said Pooh, as he flunked the the sobriety test.


Reply | Threaded
Open this post in threaded view
|

Re: Simulating the Cog V3 VM

Eliot Miranda-2

Hi Tim, Hi All,


> On May 17, 2016, at 10:22 AM, tim Rowledge <[hidden email]> wrote:
>
>
>
>> On 17-05-2016, at 6:01 AM, Juan Vuletich <[hidden email]> wrote:
>>
>> On 5/11/2016 9:05 PM, David T. Lewis wrote:
>>>
>>>> On Wed, May 11, 2016 at 04:46:32PM -0700, tim Rowledge wrote:
>>>>
>>>>> On 11-05-2016, at 4:17 PM, Levente Uzonyi<[hidden email]>  wrote:
>>>>>
>>>>> Spur support has been added to SystemTracer a while ago.
>>>>> http://www.squeaksource.com/SystemTracing.html
>>>> Excellent; looks like other-Tim got to it and seems to have made something working. So, why isn???t that the solution to migrating Cuis?
>>> Because it is not just a matter of converting the existing object memory,
>>> there are also changes in the image.
>>>
>>> Regarding SystemTracer, this is a nice light-weight approach to doing
>>> the conversion, but Eliot's bootstrap process that uses the simulation
>>> machinery in VMMaker has a lot of advantages too.
>>
>> I understand. But in any case, have you, Levente, David or anyone else produced a spur image that can start, using SystemTracer?
>
> The submit comments in the history of the SystemTracer2 package claim it has been used to trace a working Spur image. I imagine it would have been a contemporary v3 trunk image loaded with SystemTracer2.

Being able to output an image in Spur format is different to outputting a Spur image that includes all the modified methods in object allocation and object enumeration.  I doubt that the system tracer correctly bootstraps a sour image if run from within v3.  I do expect it is able to trace a working spur image from within spur.  Am I wrong to think it isn't smart enough to bootstrap a spur from v3?

>
> Unless some very big changes have been made in Cuis that completely break the concept of a system tracer I have trouble imagining how it could be unable to produce a proper spur version. I don’t doubt there might be some interesting details to solve.
> In the past I know the tracer has been used to convert bytecode sets, object formats, word sizes,  and method formats, as well as to create application specific images etc. I know from personal experience that it can be made to do a lot of interesting things.
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> "Bother" said Pooh, as he flunked the the sobriety test.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Simulating the Cog V3 VM

timrowledge


> On 17-05-2016, at 10:42 AM, Eliot Miranda <[hidden email]> wrote:
> {snip}
>
> Being able to output an image in Spur format is different to outputting a Spur image that includes all the modified methods in object allocation and object enumeration.  I doubt that the system tracer correctly bootstraps a sour image if run from within v3.  I do expect it is able to trace a working spur image from within spur.  Am I wrong to think it isn't smart enough to bootstrap a spur from v3?
>

Yup. Though it *is* harder; you have to substitute those methods as part of the writing out. In general we’d compile the new methods under fake names (assuming it’s actually possible to create appropriate methods, which I guess may be a problem under some circumstances) and substitute them for the old versions, and of course, leave the old versions out altogether. Another possible approach is to rewrite those methods to work for either case, sometimes a doable thing.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: HAL: Murder Operator


Reply | Threaded
Open this post in threaded view
|

Re: Simulating the Cog V3 VM

Eliot Miranda-2
 
Tim,

On Tue, May 17, 2016 at 11:09 AM, tim Rowledge <[hidden email]> wrote:


> On 17-05-2016, at 10:42 AM, Eliot Miranda <[hidden email]> wrote:
> {snip}
>
> Being able to output an image in Spur format is different to outputting a Spur image that includes all the modified methods in object allocation and object enumeration.  I doubt that the system tracer correctly bootstraps a sour image if run from within v3.  I do expect it is able to trace a working spur image from within spur.  Am I wrong to think it isn't smart enough to bootstrap a spur from v3?
>

Yup. Though it *is* harder; you have to substitute those methods as part of the writing out. In general we’d compile the new methods under fake names (assuming it’s actually possible to create appropriate methods, which I guess may be a problem under some circumstances) and substitute them for the old versions, and of course, leave the old versions out altogether. Another possible approach is to rewrite those methods to work for either case, sometimes a doable thing.

That's not what I mean.  Yes, of course it's possible to bootstrap via tracer.  What I'm asking is is has it been done?

Last time I had energy for the Cuis Spur port we got as far as doing the Float => Float,BoxedFloat64,SmallFloat64 putsch (see http://www.squeakvm.org/svn/squeak/branches/Cog/image/old/MorphFloat.st) which failed because of missing sport in the Cuis method protocol IIRC.
 
tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: HAL: Murder Operator

_,,,^..^,,,_
best, Eliot
12