Bug in PolygonMorph>>#filledForm

> Hmmm... the form was still filled in Squeak 5.1 (Update 16549),
> VM 201701311639 (32-bit). Not so in Squeak 5.2.

Yes, but that same 5.1 image with a current VM has the problem...

Stef

Interesting case to debug...

In 64bits VM, a problem first appears in first smear:distance: operation (direction 1@0 to the right) at skew=16...

It seems that we copy bits too far to the right (hence we overwrite the bottom rows)

Fortunately, the artefacts outside polygon bounds disappear at final bitAnd operation, but that's scary!

It could be:

- case of overlap detection failure (#checkSourceOverlap), but the code did not change

- bad initialisation in #sourceSkewAndPointerInit (the one I corrected in 2019)

 https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/426

- bug in #copyLoop (which I also corrected in 2019)

https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/441

Weird kind of integer overflow? (I've made most of the int unsigned, so this might not be detected by UB sanitizer).

Or could it be that the extra buffer overrun that I removed did have a side effect of filling more gaps?

It would be interesting to observe what happens at same #smear:distance: step with older VM.

Le ven. 21 févr. 2020 à 20:39, Stéphane Rollandin <[hidden email]> a écrit :

> Hmmm... the form was still filled in Squeak 5.1 (Update 16549),
> VM 201701311639 (32-bit). Not so in Squeak 5.2.

Yes, but that same 5.1 image with a current VM has the problem...

Stef

I do not observe the artifacts at same smear step (nor any other) with

VM [CoInterpreterPrimitives VMMaker.oscog-eem.2266] 5.0.201708312323

That's just 222 commits to bissect...

Le ven. 21 févr. 2020 à 23:00, Nicolas Cellier <[hidden email]> a écrit :

There's already a problem with end 2018 VM

VM [CoInterpreterPrimitives VMMaker.oscog-eem.2488] 5.0.201812040127

Le ven. 21 févr. 2020 à 22:48, Nicolas Cellier <[hidden email]> a écrit :

Interesting case to debug...

In 64bits VM, a problem first appears in first smear:distance: operation (direction 1@0 to the right) at skew=16...

It seems that we copy bits too far to the right (hence we overwrite the bottom rows)

Fortunately, the artefacts outside polygon bounds disappear at final bitAnd operation, but that's scary!

It could be:

- case of overlap detection failure (#checkSourceOverlap), but the code did not change

- bad initialisation in #sourceSkewAndPointerInit (the one I corrected in 2019)

 https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/426

- bug in #copyLoop (which I also corrected in 2019)

https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/441

Weird kind of integer overflow? (I've made most of the int unsigned, so this might not be detected by UB sanitizer).

Or could it be that the extra buffer overrun that I removed did have a side effect of filling more gaps?

It would be interesting to observe what happens at same #smear:distance: step with older VM.

Le ven. 21 févr. 2020 à 20:39, Stéphane Rollandin <[hidden email]> a écrit :

> Hmmm... the form was still filled in Squeak 5.1 (Update 16549),
> VM 201701311639 (32-bit). Not so in Squeak 5.2.

Yes, but that same 5.1 image with a current VM has the problem...

Stef

> On 2020-02-23, at 11:21 AM, Eliot Miranda <[hidden email]> wrote:
>
> [snip]

>
> Dan Amelang's Gezira is much more interesting. 

Yup. We (as in 'the board' in this case) tried contacting him some time ago to ask about any progress, plans etc. My email archive suggests that was around sept '17 and nothing came of it that I can discover.

The very last thing I can find is
"

A discussion on the open source nature of the Nile/Gezira/Frank on tinlizzie.org started today on http://mailman.mit.edu/mailman/listinfo/fonclist

Specifically, I asked Yoshiki:

> The Squeak board asked me to ask Alan Kay, Yoshiiki Oshima and other VPRI reseachers if this code is published as open source. When we have reconstructed the ‘final’ version of Frank and all its projects we would like to ask permission to republish it in a running state.


http://mailman.mit.edu/mailman/private/fonclist/2017-November/000048.html

"
... from Nov 2017.

I have no access to that mailing list archive unfortunately. Does anyone out there? Did it explain anything we would care about?


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLB: Ruin Logic Board

> On 2020-02-23, at 11:21 AM, Eliot Miranda <[hidden email]> wrote:
>
> [snip]

>
> Dan Amelang's Gezira is much more interesting. 

Yup. We (as in 'the board' in this case) tried contacting him some time ago to ask about any progress, plans etc. My email archive suggests that was around sept '17 and nothing came of it that I can discover.

The very last thing I can find is
"

A discussion on the open source nature of the Nile/Gezira/Frank on tinlizzie.org started today on http://mailman.mit.edu/mailman/listinfo/fonclist

Specifically, I asked Yoshiki:

> The Squeak board asked me to ask Alan Kay, Yoshiiki Oshima and other VPRI reseachers if this code is published as open source. When we have reconstructed the ‘final’ version of Frank and all its projects we would like to ask permission to republish it in a running state.


http://mailman.mit.edu/mailman/private/fonclist/2017-November/000048.html

"
... from Nov 2017.

I have no access to that mailing list archive unfortunately. Does anyone out there? Did it explain anything we would care about?


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLB: Ruin Logic Board

On Sun, Feb 23, 2020 at 11:17 AM tim Rowledge <[hidden email]> wrote:

> On 2020-02-23, at 4:42 AM, Fabio Niephaus <[hidden email]> wrote:
>
> On Sun, Feb 23, 2020 at 1:48 AM tim Rowledge <[hidden email]> wrote:
> >
> >
> > Well, that's interesting. The bug is different on a Raspberry Pi (I was hoping non-existent) because the bitblt code is quite different as a result of the Pi Foundation paying to get much faster ARM assembler blt code written.
> > Same sort of pattern but not identical. Sigh. This will be 'fun' to investigate.
>
> Hi Tim,
> Seems very similar to this:
> <image.png>
>
> This is what it looked like on GraalSqueak without the fix. Note that GraalSqueak's BitBlt is written in Java, so it cannot read past forms etc. Maybe this is also the case on ARM for some reason?

Yes, that is interesting. It certainly looks identical - so at least the bug should be the same to fix!

Having to think about the ARM specialised code reminds me - we really, really ought to try to make a way to JIT the blit. I did an ARM version 30+ years ago but it was all rather simpler back then; no cache to fart around with (not just no cache instructions with strange permissions stuff, actually no cache at all) and simple monochrome Forms.

We have the key components to hand. It seems like an idea that could be widely useful.

I dare say it is trivial with the framework we have in the JIT (but then it applies only to the JIT VM, not the stack interpreter).

 

It would surely make a fine thesis.

Dan Amelang's Gezira is much more interesting.  We should productize that.  We have to accept that we badly need a pixel independent imaging model that can be compiled to and executed by the GPU. The world has moved on and Squeak is limited by its graphics infrastructure.

_,,,^..^,,,_

best, Eliot

A little detective work showed I could still subscribe to the Fonc list at http://mailman.mit.edu/mailman/listinfo/fonclist, giving access to what archives there are.

Last email I show from http://mailman.vpri.org/mailman/listinfo/fonc_mailman.vpri.org, is 2017.07.31

Ken G. Brown

On Feb 23, 2020, at 12:35, tim Rowledge <[hidden email]> wrote:

On 2020-02-23, at 11:21 AM, Eliot Miranda <[hidden email]> wrote:

[snip]

Dan Amelang's Gezira is much more interesting.  

Yup. We (as in 'the board' in this case) tried contacting him some time ago to ask about any progress, plans etc. My email archive suggests that was around sept '17 and nothing came of it that I can discover.

The very last thing I can find is
"

A discussion on the open source nature of the Nile/Gezira/Frank on tinlizzie.org started today on http://mailman.mit.edu/mailman/listinfo/fonclist

Specifically, I asked Yoshiki:

The Squeak board asked me to ask Alan Kay, Yoshiiki Oshima and other VPRI reseachers if this code is published as open source. When we have reconstructed the ‘final’ version of Frank and all its projects we would like to ask permission to republish it in a running state.

http://mailman.mit.edu/mailman/private/fonclist/2017-November/000048.html

"
... from Nov 2017.

I have no access to that mailing list archive unfortunately. Does anyone out there? Did it explain anything we would care about?


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RLB: Ruin Logic Board

Thank you all for keeping up these efforts! :-)

I just added this issue to our automated test suite. I was just a little bit confused that the bitmap (bytes) created by #filledForm does look to irregular even though -- visually -- distinguishing just between black and transparent.

Best,

Marcel

Am 23.02.2020 22:21:34 schrieb Nicolas Cellier <[hidden email]>:

Hi all,

just to be clear, I underlined the mistakes for the purpose of explaining what I did change and why, not for bashing anyone.

It's fairly easy to correct a posteriori once we have a failing test case exposing the problem (thanks Stephane for that!).

It was less obvious to analyze and review the change made by Eliot at the time they were introduced.

These changes were SUPER important because they made us progress toward complete elimination of buffer overrun (fingers crossed...).

I'd say Eliot initiate this move and made 80% of the work, so kudos!

And thanks to all those reporting and isolating reproducible test cases!

Tim, I took a look at arm code, but there are too many macros to my taste and I can hardly follow the code just by reading (I would need live debugging session, which means messing with QEMU etc...).

The hints are:

- it's case of H-OVERLAP, so we copy bits from right to left

- we're fetching one extra word from the source (the equivalent of preload) though we shall not

Not so many operations perform a Form under (bitOr) operation, so it's fairly straight forward to put a conditional breakpoint in the debugger and just observe what's going on... Good luck :)

Le dim. 23 févr. 2020 à 20:22, Eliot Miranda <[hidden email]> a écrit :

On Sun, Feb 23, 2020 at 11:17 AM tim Rowledge <[hidden email]> wrote:

> On 2020-02-23, at 4:42 AM, Fabio Niephaus <[hidden email]> wrote:
>
> On Sun, Feb 23, 2020 at 1:48 AM tim Rowledge <[hidden email]> wrote:
> >
> >
> > Well, that's interesting. The bug is different on a Raspberry Pi (I was hoping non-existent) because the bitblt code is quite different as a result of the Pi Foundation paying to get much faster ARM assembler blt code written.
> > Same sort of pattern but not identical. Sigh. This will be 'fun' to investigate.
>
> Hi Tim,
> Seems very similar to this:
> <image.png>
>
> This is what it looked like on GraalSqueak without the fix. Note that GraalSqueak's BitBlt is written in Java, so it cannot read past forms etc. Maybe this is also the case on ARM for some reason?

Yes, that is interesting. It certainly looks identical - so at least the bug should be the same to fix!

Having to think about the ARM specialised code reminds me - we really, really ought to try to make a way to JIT the blit. I did an ARM version 30+ years ago but it was all rather simpler back then; no cache to fart around with (not just no cache instructions with strange permissions stuff, actually no cache at all) and simple monochrome Forms.

We have the key components to hand. It seems like an idea that could be widely useful.

I dare say it is trivial with the framework we have in the JIT (but then it applies only to the JIT VM, not the stack interpreter).

 

It would surely make a fine thesis.

Dan Amelang's Gezira is much more interesting.  We should productize that.  We have to accept that we badly need a pixel independent imaging model that can be compiled to and executed by the GPU. The world has moved on and Squeak is limited by its graphics infrastructure.

_,,,^..^,,,_

best, Eliot