[OpenSmalltalk/opensmalltalk-vm] cd8567: Fix the signature of b3dDrwBufferFunction

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[OpenSmalltalk/opensmalltalk-vm] cd8567: Fix the signature of b3dDrwBufferFunction

Nicolas Cellier-4
 
  Branch: refs/heads/fix_issue468_Squeak3D_64bits_crash
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: cd8567dc22901aea78d47adbb700aef5500dec07
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/cd8567dc22901aea78d47adbb700aef5500dec07
  Author: Nicolas Cellier <[hidden email]>
  Date:   2020-01-10 (Fri, 10 Jan 2020)

  Changed paths:
    M platforms/Cross/plugins/Squeak3D/b3d.h

  Log Message:
  -----------
  Fix the signature of b3dDrwBufferFunction

We instantiate this function pointer with a function taking `sqInt` rather than `int`.
In 32 bits VM, it's OK, but on 64bits VM, sqInt is 64bits wide.
This prototype mismatch did prevent sign extension of negative int values.
Thus an offset of -1 in `yValue` becomes an offset of 2^32-1 in the callee,
which then cause a buffer overrun (and most often a SEGV)


  Commit: d6ca6c105cf29c89147070e039c49d831db42a81
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/d6ca6c105cf29c89147070e039c49d831db42a81
  Author: Nicolas Cellier <[hidden email]>
  Date:   2020-01-11 (Sat, 11 Jan 2020)

  Changed paths:
    M platforms/Cross/plugins/Squeak3D/b3dMain.c

  Log Message:
  -----------
  Protect against an integer overflow in b3dComputeIntersection


  Commit: 43cc2445f46f9bffb9e445bdeceb8c9e54fec44e
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/43cc2445f46f9bffb9e445bdeceb8c9e54fec44e
  Author: Nicolas Cellier <[hidden email]>
  Date:   2020-01-11 (Sat, 11 Jan 2020)

  Changed paths:
    M platforms/Cross/plugins/Squeak3D/b3dInit.c

  Log Message:
  -----------
  clamp initial scaledX/Y in int range


  Commit: f3ef66d13463dd2fc363779d0888798343e81522
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/f3ef66d13463dd2fc363779d0888798343e81522
  Author: Nicolas Cellier <[hidden email]>
  Date:   2020-01-11 (Sat, 11 Jan 2020)

  Changed paths:
    M platforms/Cross/plugins/Squeak3D/b3dMain.c

  Log Message:
  -----------
  Avoid blitting at negative yValue offset


Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/cd8567dc2290%5E...f3ef66d13463