New Cog VMs available

> On Nov 9, 2014, at 20:34, Ken.Dickey <[hidden email]> wrote:
>
> On Sun, 9 Nov 2014 17:10:26 -0800
> Douglas McPherson <[hidden email]> wrote:
>
>> Pre-built Squeak stack and stack/spur VMs for ARMv6 (i.e. Raspberry
>> Pi) and ARMv7 (i.e. BeagleBone Black, Parallella, numerous
>> Chromebooks, etc) are now available at the same location as the other
>> Cog and Spur VMs.
>> ( http://www.mirandabanda.org/files/Cog/VM/VM.r3126/
>> <http://www.mirandabanda.org/files/Cog/VM/VM.r3126/> ).
>
> Doug,
>
> Thanks much!  I can run CUIS Smalltalk fine on a Samsung ARMv7
> Chromebook using the v7 stklinuxhtARM VM.
>

> Quoting Douglas McPherson <[hidden email]>:
>
>> Hi Ken,
>>
>>> On Nov 11, 2014, at 16:28, Ken Dickey <[hidden email]> wrote:
>>>
>>>
>>> On Sun, 9 Nov 2014 20:56:47 -0800
>>> Douglas McPherson <[hidden email]> wrote:
>>>
>>> KenD>>
>>>>> The stkspurlinuxhtARM VM, however, complains:
>>>>> This interpreter (vers. 6521) cannot read image file (vers. 6505).
>>>>>
>>>>> I presume some conversion is required to get the image into SPUR format?
>>>> Yes, the Spur VMs require a new image format. There are recent Squeak trunk images in the spur format on Eliot’s site. I’ll let Eliot and others comment about how a Cuis image might be converted.
>>>
>>> I tried out the Squeak SPUR image from the Cog site and everything Squeak seems fine.
>>>
>>>
>>> There is a timing bug with Cuis redraw that shows up on both the stklinuxhtARM vm and the one you sent me earlier.
>>>
>>> Basically, I am using DisplayObject>>slideFrom:to:nSteps:delay: to do some animation.  When multiple timer events fire during an animation it sometimes appears that the x,y redraw origin is used from the currently drawing morph's position, not 0,0.  I.e. part of the redisplay is offset.  [Display redraw via menu corrects the display].
>>>
>>> This _only_ happens on the Samsung ARM Chromebook, never on Intel CPUs.
>>
>> Thanks for reporting this. I’ll try to look into it. There are definitely ARM-specific changes to support certain atomic operations, so there may be something not quite right with these.
>>
>> Can you suggest a method for reproducing this in Squeak? I repeatedly sent morphs to the trash (with appropriate preference set to enable the animation) but didn’t see an issue. I suspect I’m not driving it hard enough.
>>
>>> My life is really, really busy right now, so I have never had time to dig into this.  I can redraw the display, so a workaround exists.
>>>
>>> [One possible resource.  The Apache Portable Runtime, http://apr.apache.org/ , implements compareAndSwap and various other atomic operations across various OSs.]
>>
>> Thanks for the reference. It may be useful. I also found a library called Mintomic (http://preshing.com/20130505/introducing-mintomic-a-small-portable-lock-free-api/) which may help.
>>
>>> I am happy to do quick tests, but must apologize for lack of time to dig into this.
>>
>> No problems, and thanks again for reporting it.
>>
>>
>>> FYI,
>>> -KenD

> Hi Juan,
>
>
> On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)"
> <[hidden email]> wrote:
>
>> Hi Folks,
>>
>> The Squeak process scheduler usually preempts a process only by  
>> another one of higher priority. And when the scheduler is ready to  
>> go back at the lower priority, it will always resume the same  
>> process that was last suspended. The only way for two or more  
>> processes of the same priority to share the processor is when the  
>> running process calls #wait or #yield.
>
> No.  According to the blue book preempting sends the preempted process
> to the back if its run queue so preempting forces a yield.  Obviously
> this is wrong, but that's the way it was defined.  The ObjectWorks &
> VisualWorks VMs changed this to work as you describe.

> When I implemented Cog I wanted to fix this but not break old code so
> I made it optional.  The behavior is controlled by a flag in the
> image.  You can see the flags via Smalltalk vmParameterAt: 48.  Bit 2
> (value 4) is set if preempting puts the preempted process at the head
> of its list, and hence does not yield.
>
> Here's the part of the comment from vmParameterAt:
>
>     48 various properties of the Cog VM as an integer encoding an
> array of bit flags.
>         Bit 0: implies the image's Process class has threadId as its
> 3rd inst var (zero relative)
>         Bit 1: on Cog VMs asks the VM to set the flag bit in  
> interpreted methods
>         Bit 2: if set, preempting a process puts it to the head of its
> run queue, not the back,
>                 i.e. preempting a process by a higher one will not
> cause the process to yield
>                 to others at the same priority.
>
>
>> This means that using shared state from several processes at the  
>> same priority (without using any locking mechanism, such as  
>> Semaphores, Monitors or critical sections) is safe, as long as the  
>> processes do #wait or #yield only when that shared state is in a  
>> consistent state.
>
> Right.
>
>>
>> It looks like this ARM VM is somehow not following this convention.  
>> If this was the case, the problems Ken sees are to be expected.
>
> Can you check the bit is set in the image?
>
>
> Thanks for raising this.  It looks like the Spur images have the bit
> cleared.  Ouch.  Ah, it's not a problem of the bootstrap, its at least
> a problem in trunk.  The bit is not set in trunk.  I need to a) write
> some tests to test the bit and b) make sure its set in the relevant
> places.

>>
>> HTH,
>> Juan Vuletich
>
> Eliot (phone)
>
>> Quoting Douglas McPherson <[hidden email]>:
>>
>>> Hi Ken,
>>>
>>>> On Nov 11, 2014, at 16:28, Ken Dickey <[hidden email]> wrote:
>>>>
>>>>
>>>> On Sun, 9 Nov 2014 20:56:47 -0800
>>>> Douglas McPherson <[hidden email]> wrote:
>>>>
>>>> KenD>>
>>>>>> The stkspurlinuxhtARM VM, however, complains:
>>>>>> This interpreter (vers. 6521) cannot read image file (vers. 6505).
>>>>>>
>>>>>> I presume some conversion is required to get the image into SPUR format?
>>>>> Yes, the Spur VMs require a new image format. There are recent  
>>>>> Squeak trunk images in the spur format on Eliot’s site. I’ll let  
>>>>> Eliot and others comment about how a Cuis image might be  
>>>>> converted.
>>>>
>>>> I tried out the Squeak SPUR image from the Cog site and  
>>>> everything Squeak seems fine.
>>>>
>>>>
>>>> There is a timing bug with Cuis redraw that shows up on both the  
>>>> stklinuxhtARM vm and the one you sent me earlier.
>>>>
>>>> Basically, I am using DisplayObject>>slideFrom:to:nSteps:delay:  
>>>> to do some animation.  When multiple timer events fire during an  
>>>> animation it sometimes appears that the x,y redraw origin is used  
>>>> from the currently drawing morph's position, not 0,0.  I.e. part  
>>>> of the redisplay is offset.  [Display redraw via menu corrects  
>>>> the display].
>>>>
>>>> This _only_ happens on the Samsung ARM Chromebook, never on Intel CPUs.
>>>
>>> Thanks for reporting this. I’ll try to look into it. There are  
>>> definitely ARM-specific changes to support certain atomic  
>>> operations, so there may be something not quite right with these.
>>>
>>> Can you suggest a method for reproducing this in Squeak? I  
>>> repeatedly sent morphs to the trash (with appropriate preference  
>>> set to enable the animation) but didn’t see an issue. I suspect  
>>> I’m not driving it hard enough.
>>>
>>>> My life is really, really busy right now, so I have never had  
>>>> time to dig into this.  I can redraw the display, so a workaround  
>>>> exists.
>>>>
>>>> [One possible resource.  The Apache Portable Runtime,  
>>>> http://apr.apache.org/ , implements compareAndSwap and various  
>>>> other atomic operations across various OSs.]
>>>
>>> Thanks for the reference. It may be useful. I also found a library  
>>> called Mintomic  
>>> (http://preshing.com/20130505/introducing-mintomic-a-small-portable-lock-free-api/) which may  
>>> help.
>>>
>>>> I am happy to do quick tests, but must apologize for lack of time  
>>>> to dig into this.
>>>
>>> No problems, and thanks again for reporting it.
>>>
>>>
>>>> FYI,
>>>> -KenD

>
> Hi Juan,
>
>
> On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)"
> <[hidden email]> wrote:
>
>> Hi Folks,
>>
>> The Squeak process scheduler usually preempts a process only by another one of higher priority. And when the scheduler is ready to go back at the lower priority, it will always resume the same process that was last suspended. The only way for two or more processes of the same priority to share the processor is when the running process calls #wait or #yield.
>
> No.  According to the blue book preempting sends the preempted process
> to the back if its run queue so preempting forces a yield.  Obviously
> this is wrong, but that's the way it was defined.  The ObjectWorks &
> VisualWorks VMs changed this to work as you describe.

> On Fri, Dec 5, 2014 at 11:39 AM, Eliot Miranda  
> <[hidden email]> wrote:
>>
>> Hi Juan,
>>
>>
>> On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)"
>> <[hidden email]> wrote:
>>
>>> Hi Folks,
>>>
>>> The Squeak process scheduler usually preempts a process only by  
>>> another one of higher priority. And when the scheduler is ready to  
>>> go back at the lower priority, it will always resume the same  
>>> process that was last suspended. The only way for two or more  
>>> processes of the same priority to share the processor is when the  
>>> running process calls #wait or #yield.
>>
>> No.  According to the blue book preempting sends the preempted process
>> to the back if its run queue so preempting forces a yield.  Obviously
>> this is wrong, but that's the way it was defined.  The ObjectWorks &
>> VisualWorks VMs changed this to work as you describe.
>
> Why is that wrong?  By sending it to the back, one gets task-switching
> that acts like real "multitasking" as one would expect in a real
> multitasking system, without the developer having to put in explicit
> sends to #yield.  Keeping it at the front "enables" multi-processing
> code to be developed incorrectly so that, one day, when it might ever
> get to a real multitasking environment, it won't work and it'll be
> harder to debug since by then its old and probably more complex.
> #yield is not a good synchronization tool..

>
>
> Quoting Chris Muller <[hidden email]>:
>
>> On Fri, Dec 5, 2014 at 11:39 AM, Eliot Miranda
>> <[hidden email]> wrote:
>>>
>>> Hi Juan,
>>>
>>>
>>> On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)"
>>> <[hidden email]> wrote:
>>>
>>>> Hi Folks,
>>>>
>>>> The Squeak process scheduler usually preempts a process only by
>>>> another one of higher priority. And when the scheduler is ready to
>>>> go back at the lower priority, it will always resume the same
>>>> process that was last suspended. The only way for two or more
>>>> processes of the same priority to share the processor is when the
>>>> running process calls #wait or #yield.
>>>
>>> No.  According to the blue book preempting sends the preempted process
>>> to the back if its run queue so preempting forces a yield.  Obviously
>>> this is wrong, but that's the way it was defined.  The ObjectWorks &
>>> VisualWorks VMs changed this to work as you describe.
>>
>> Why is that wrong?  By sending it to the back, one gets task-switching
>> that acts like real "multitasking" as one would expect in a real
>> multitasking system, without the developer having to put in explicit
>> sends to #yield.  Keeping it at the front "enables" multi-processing
>> code to be developed incorrectly so that, one day, when it might ever
>> get to a real multitasking environment, it won't work and it'll be
>> harder to debug since by then its old and probably more complex.
>> #yield is not a good synchronization tool..
>
> Well, yes. But for that to work we'd need a highest priority process
> that keeps sleeping for, lets say, 1 mSec, and does nothing. That way
> all processes would be suspended and shuffled all the time.

> J. Vuletich (mail lists) wrote:
>>
>>
>> Quoting Chris Muller <[hidden email]>:
>>
>>> On Fri, Dec 5, 2014 at 11:39 AM, Eliot Miranda
>>> <[hidden email]> wrote:
>>>>
>>>> Hi Juan,
>>>>
>>>>
>>>> On Dec 5, 2014, at 4:56 AM, "J. Vuletich (mail lists)"
>>>> <[hidden email]> wrote:
>>>>
>>>>> Hi Folks,
>>>>>
>>>>> The Squeak process scheduler usually preempts a process only by
>>>>> another one of higher priority. And when the scheduler is ready to
>>>>> go back at the lower priority, it will always resume the same
>>>>> process that was last suspended. The only way for two or more
>>>>> processes of the same priority to share the processor is when the
>>>>> running process calls #wait or #yield.
>>>>
>>>> No.  According to the blue book preempting sends the preempted process
>>>> to the back if its run queue so preempting forces a yield.  Obviously
>>>> this is wrong, but that's the way it was defined.  The ObjectWorks &
>>>> VisualWorks VMs changed this to work as you describe.
>>>
>>> Why is that wrong?  By sending it to the back, one gets task-switching
>>> that acts like real "multitasking" as one would expect in a real
>>> multitasking system, without the developer having to put in explicit
>>> sends to #yield.  Keeping it at the front "enables" multi-processing
>>> code to be developed incorrectly so that, one day, when it might ever
>>> get to a real multitasking environment, it won't work and it'll be
>>> harder to debug since by then its old and probably more complex.
>>> #yield is not a good synchronization tool..
>>
>> Well, yes. But for that to work we'd need a highest priority process
>> that keeps sleeping for, lets say, 1 mSec, and does nothing. That way
>> all processes would be suspended and shuffled all the time.
>
> A few times I've seen the idea raised of moving process scheduling into
> the Image. This sounds interesting.  

> Now actually I don't see any reason that new class
> needs to be tied only to Delays.  It might also provide process
> scheduling.  Where the VM currently decides which process to run,
> conceivably the /timingSemepahore/ could be signaled to wake up and
> decide which process to run next, then call back into the VM to dispatch
> the selected process.
>
> Having process scheduling in the VM would conceivably facilitate
> development of domain specific process scheduling - e.g. different for
> interactive UI, web servers, batch processing, robotics, control
> systems, real-time media.
>
> Also this would make a great environment for Computer Science courses to
> teach process scheduling algorithms.
>
> The following looks interesting, although I don't follow all of it...
> https://computeradventures.wordpress.com/tag/squeak-smalltalk-vm-scheduling/ 
>
>
>
>
>> The way it is right now, this shuffling won't happen often, and
>> ("bad") code that assumes the behavior I described would work "most of
>> the time", breaking randomly, and more often on slower hardware (such
>> as the PI)...
>>
>> Is this worth doing? This "bad" code would fail all the time, and it
>> would be spotted and fixed. But we would break (a rather illusory)
>> back compatibility...
>
> Less of an issue for Pharo. Maybe this is something to consider for
> Pharo 5 ?
>
> cheers -ben
>
>> For instance, the code Ken is using for animating the game cards
>> (moving morphs around in a separate process) would break for sure. But
>> fixing it would also fix the issues he sees on the PI. It is not hard
>> to imagine that the same would happen to the sound problems Tim found
>> on Scratch on the PI...
>

> Ben Coman wrote:
>
> > For Pharo** I recently factored the
> >delay scheduling event loop code out from Delay class side to its own
> >DelayScheduler.  
>
>
> ** I forgot to say, once its been proven in Pharo, I'd be happy to do
> the same for Squeak - if there is interest.  Among other things it gets
> rid of the Delay clock rollover every six days.  I don't think there has
> been much divergence in this area and it would be good to keep in sync
> if possible.
> cheers -ben