Vector Graphics (was: Pi performance fun)

On 26-05-2013, at 3:08 PM, Colin Putney <[hidden email]> wrote:

Seems like something worth talking about on today's Hangout,..


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
He who hesitates is probably right.

On 27-05-2013, at 10:04 AM, Colin Putney <[hidden email]> wrote:

I'm there now


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RBR: Remove Bits Randomly

On 28-05-2013, at 12:14 AM, karl ramberg <[hidden email]> wrote:

> Did you get to discuss this ?

A little but nowhere near enough.

Igor has been doing work on Athens, which certainly has promise. There is a demo video at
http://www.pharocasts.com/2012/06/athens-and-native-boost-at-work.html

Clearly what I'm most directly interested in is ways to get the most out of the GPU on a Pi, which is massively faster than the ARM cpu at moving pixels around. I'd also rather like a solution that works on RISC OS as well, which means we ought to try for something which can drive various backends and doesn't totally rely on some unix library. Good practice would be for plain bitblts to still be usable as one of the back ends.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
I asked Mom if I was a gifted child ... she said they certainly wouldn't have paid for me.

Hi Everyone,

Regarding vector graphics in squeak I believe that VPRI has been
experimenting with Nile and Gezira as a method of getting compact
vector graphics into their "Frank" system.

Bert released packages that can be used to generate a VM with support
for Nile and Gezira, and I have use these to compile one.  I was able
to play around and experiment with nicely anti-aliased graphics while
developing in the live style that I enjoy so much.

I also grabbed a Pharo VM and played with Athens a bit this morning.

So my gut would say that there are two choices before us: Adopt the
work that Pharo is doing with Athens, or go with the VPRI Gezira work.
 Below is a list of (IMO) pros and cons for each approach.  I would be
curious what other people think.

Pros for Athens:
- Relies on Cairo which is developed by a large outside group
- Cairo appears to be relatively high performance
- PDF support allows for easily printable image/document creation
- Has dedicated development from the Pharo community

Cons for Athens:
- Cairo is pretty complex if we want to ever patch or fix anything
- Relies on NativeBoost enabled VMs which are not the VMs that Eliot
provides and I believe these need pharo specific image-side code at
this point

Pro for Gezira:
- Very small and compact logic, does have ramp up time but would be
much easier to modify than ~100,000 lines of cairo code
- Used by VPRI for "Frank" so it is being used in a system of complexity
- Seemed to have a pretty decent API from what I could understand

Cons for Gezira:
- At this point the product of one man (Dan Amelang) or possibly one
research group (VPRI)
- Unknown performance relative to other libraries like Cairo.  Maybe
Bert can chime in here.
- Unknown support for PDF and other image types.
- Diverges from Pharo, so could split the community unnecessarily.

Anyway, I think that getting high quality vector graphics in Squeak
would be a fantastic boon, as Squeak is such a graphical environment
and live coding with Morphs is great right up until you see chunky
aliased lines on your canvas.

Thanks for your time,
Jeff

> Good practice would be for plain bitblts to still be usable as one of the back ends.

Definitely. It's fun to have low-level access to graphics.

Stef

On 2013-05-28, at 21:05, tim Rowledge <[hidden email]> wrote:


Access to fast blitting is a very different requirement from supporting general vector graphics. It may end up being implemented using a vector graphics library behind the scenes, but the user-visible API would be completely different.

- Bert -

On 28 May 2013 21:19, Jeff Gonis <[hidden email]> wrote: Some clarification:

There's also balloon backend for Athens (yes, as an attempt to use
existing VM functionality
it is of course not nearly as fast as cairo, and lack functionality.. but ..)

Cairo is just one of the backends
it uses for rendering. Give it more time, i would implement Quartz
backend on mac
as well as opengl backend.
The point is, that Athens provides own API for applications, so that
gives you guarantee
that your application will continue working, no matter what is used
underneath (cairo or something else).



--
Best regards,
Igor Stasenko.

On 2013-05-28, at 21:19, Jeff Gonis <[hidden email]> wrote:


Good analysis.

I don't think Gezira is ready yet outside of a research system. Cairo is much more mature and more widely supported.

The best solution IMHO would be Athens with a plugin-based backend (to make it work without NativeBoost, which as a form of FFI has no business being in the core system).

Note that there are two orthogonal problems: quality, and speed. For quality the vector graphics would help, for speed, not really. What we need for speed is hardware-based compositing.

The two can work well together. Each top-level morph (e.g. window) would render into a "surface", and all surfaces would be composited onto the screen. This gets rid of damage redraw, and allows extremely fast moving of windows. Optionally, submorphs can be backed by such surfaces too. It would be for the individual morph to decide whether to use bitblt or vector graphics to draw into its surface.

This is basically how any modern UI framework works, and the only way to make it really fast. There are just too many pixels even for a fast CPU.

The price is added complexity, and less flexibility. There won't be easy/fast access to the whole display contents anymore. Developers will need to think in layers, not pixels. An Etoys car cannot easily bounce off a window in its way anymore.

That said, it's still worth doing.

- Bert -

On 28-05-2013, at 2:15 PM, Bert Freudenberg <[hidden email]> wrote:
>
> Note that there are two orthogonal problems: quality, and speed. For quality the vector graphics would help, for speed, not really. What we need for speed is hardware-based compositing.

Yes; that's what I'm hoping for to make the Pi more useful.

>
> The two can work well together. Each top-level morph (e.g. window) would render into a "surface", and all surfaces would be composited onto the screen. This gets rid of damage redraw, and allows extremely fast moving of windows. Optionally, submorphs can be backed by such surfaces too. It would be for the individual morph to decide whether to use bitblt or vector graphics to draw into its surface.

I'm not sure you'd *need*to make each top level morph be a separate surface, though it would be nice to be able to do that and have multiple host windows. Where have I seen multiple host window support before… hmm….
Surely there is no issue with having many morphs within a single surface, using paths and fills etc to do the drawing? Such libraries must be able to handle complex drawings or they'd be of very little use.

>
> This is basically how any modern UI framework works, and the only way to make it really fast. There are just too many pixels even for a fast CPU.

Exactly what lead me to ask originally.

>
> The price is added complexity, and less flexibility. There won't be easy/fast access to the whole display contents anymore. Developers will need to think in layers, not pixels. An Etoys car cannot easily bounce off a window in its way anymore.

As above, surely that isn't any more true than now; if there is a Squeak window then we know what is drawn in it and can find 'impacts' just as now. Actually, don't Cairo-like libraries have 'A touching B' tests anyway? If there are several Squeak windows then surely we should be able to navigate an eToys car across all of them?

How much pain is there for using simple bitmap 'sprites' or 'icons' or whatever term you like? To the best of my (very limited in this area) knowledge, games still use a lot of bitmaps and so I'd be amazed if it wasn't practical to display them, move them, etc.

>
> That said, it's still worth doing.

I certainly think so. I hope we can get something done soon.

It seems to me there are two approaches that can be worked upon almost independently-
a) the low level connection to drawing libraries
b) the high level usage of the connection.

If Athens can drive the Balloon/B2DPlugin (and if that code still works as it should)  then we ought to be able to start using it very soon.Then when whichever low-level system is needed becomes available we would at least have a start on getting better. If I'm really lucky I might even be able to drive the RISC OS graphics system directly!

To see what I'm hoping to improve, take a look at the Scratch 1.4 system. There's a lot there that could be done with vector graphics rather than bitmaps. I would have thought that the script stacks in particular could be done much more nicely.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
"Oh bother" said Pooh, as he reached for the reset button *again*

On 2013-05-29, at 01:07, tim Rowledge <[hidden email]> wrote:

No, that's not what I meant. You want each top-level morph (and possibly others) to be backed by a texture in graphics memory. Those can be composited quickly by the GPU. Bits in main memory are not fast enough, CPU-GPU transfer is too slow.

> Surely there is no issue with having many morphs within a single surface, using paths and fills etc to do the drawing? Such libraries must be able to handle complex drawings or they'd be of very little use.

The only issue is speed. You want to avoid redrawing.

>> This is basically how any modern UI framework works, and the only way to make it really fast. There are just too many pixels even for a fast CPU.
>
> Exactly what lead me to ask originally.

I know, but things get muddled up if people don't know the underlying technical details.

>> The price is added complexity, and less flexibility. There won't be easy/fast access to the whole display contents anymore. Developers will need to think in layers, not pixels. An Etoys car cannot easily bounce off a window in its way anymore.
>
> As above, surely that isn't any more true than now; if there is a Squeak window then we know what is drawn in it and can find 'impacts' just as now.

Not if the final Squeak window image exists only on the graphics board with no quick way to access those pixels. The beauty of Etoys' collision tests is that it simply looks at colors on the screen. It can bounce off *anything*.

> Actually, don't Cairo-like libraries have 'A touching B' tests anyway?

Nope.

> If there are several Squeak windows then surely we should be able to navigate an eToys car across all of them?

We should, I'm just saying it's harder.

> How much pain is there for using simple bitmap 'sprites' or 'icons' or whatever term you like? To the best of my (very limited in this area) knowledge, games still use a lot of bitmaps and so I'd be amazed if it wasn't practical to display them, move them, etc.

They upload them to GPU memory (a.k.a. textures) and then just refer to them by id for drawing.


I'm pretty sure this won't buy you any speed.

- Bert -