Fast Drawing of Graphics

Support for flicker-free, fast processing of graphical objects is
gradually becoming more important when making modern desktop
applications. Look at how Vista and OS X use such graphics. Another
interesting example is BumpTop
(http://honeybrown.ca/Pubs/BumpTop.html). Customers expect a slick,
colourful, and responsive GUI.

In VisualWorks, graphics is handled at the Smalltalk level and in the
Virtual Machine (VM). In the VM, functions for making basic elements
like lines and boxes are implemented in C code. A Pixmap represents a
bitmap and VM primitives are used to draw on it. I have never measured
the speed, but my impression is that the performance is OK.

Some graphics however, is processed at the Smalltalk level. The
subclasses of Image are then typically used. At this level performance
appears to be not that good. I have previously posted a description on
how to display transparent PNG images at:
http://www.cincomsmalltalk.com/userblogs/runarj/blogView?showComments=true&entry=3328177749
My code does all processing in Smalltalk, which is too slow if you
want to display large images, or show animation.

I might have misunderstood, but there seems be a plan to move more
graphic code up to the Smalltalk layer. This is at least the
impression I get when hearing about "Peaches". I fear that basic
graphic operations will become too slow if everything is moved into
Smalltalk. In fact, I would think doing the opposite -- moving more
functions down to the VM -- is required in order to speed up graphics.

Basically, to do graphics operations we have three levels where the
work can be carried out:
- Smalltalk, very portable and easy to maintain.
- VM implements operations in C code. Should be portable, but harder
to maintain.
- VM delegates operations to the GPU
(http://en.wikipedia.org/wiki/Graphics_processing_unit). Probably not
very portable.

Operations on images like basic shape drawing, transformation,
zooming, shading, and transparency operations should be done by the
VM. In addition, shouldn't the VM delegate some of these operations to
the GPU to improve the speed even more?

Runar Jordahl

I have a question.

In my experience doing "high end" graphics requires hooking up to specific  
high end graphics frameworks on the respective operating systems e.g.  
DirectX on Windows. This we can do via DLLCC and perhaps that new  
C++ wrapping framwork whose name escapes me at the moment. I think that 2D  
ops we do fine enough i.e. GF/ST , HotDraw.

So notwithstanding what we must do to support native widgets (and whether  
that is really of that great value) , should we not just face the fact  
that what we need to do is develop a wrap for DirectX,and perhaps enhance  
the wrap for OpenGL for those that care about being cross-platform?

The moment we talk VM, we are talking about Eliot and he has not even  
started with AOP stuff :)

On the other hand something like wrapping DirectX is something that the  
community could take on.

my 3 cents

----------------------------

Charles A. Monteiro
http://wiki.nycsmalltalk.org
http://www.monteirosfusion.com
http://monteirofusion.blogspot.com



On Thu, 22 Jun 2006 09:33:57 -0400, Runar Jordahl  
<[hidden email]> wrote:



--
Charles A. Monteiro
http://wiki.nycsmalltalk.org
http://www.monteirosfusion.com
http://monteirofusion.blogspot.com

Runar Jordahl wrote:
> Support for flicker-free, fast processing of graphical objects is
> gradually becoming more important when making modern desktop
> applications. Look at how Vista and OS X use such graphics. Another
> interesting example is BumpTop
> (http://honeybrown.ca/Pubs/BumpTop.html). Customers expect a slick,
> colourful, and responsive GUI.

Well, that's for sure a lot of fun, but I doubt this kind of eye candy
is really needed.

I'd happy already if VW could support basic and simple things that were
"standard" 5 years ago already: An alpha channel for icons and images,
so they seamlessly merge into the window background.

Another thing is: VW was never used for "modern desktop applications" at
all (yet). This is probably not a viable market for Cincom. I am
currently  having a hard time to change this. When presenting my
products to potential partners, the one thing that is mentioned every
time is: "What a breathtaking technology, but .... ".

I tried to compensate for the limited possibilities by using a flat,
"minimalistic" design with high-contrast colors. Anyway, there's still
this touch of "scientific boredom" that doesn't appeal much to consumers.

Andre

I guess that most of what I have built in VW has been on a "fat" desktop  
app context, in my experience you just need to blow the customer away with  
functionality and the promise of quick change, they quickly stop dropping  
any UI like comments i.e. for business apps. So I guess I don't know what  
you mean by "not a viable market" , we must be talking about different  
types of apps?

--
Charles A. Monteiro
http://wiki.nycsmalltalk.org
http://www.monteirosfusion.com
http://monteirofusion.blogspot.com

On Thu, 22 Jun 2006 11:52:26 -0400, Andre Schnoor <[hidden email]>  
wrote:



--
Charles A. Monteiro
http://wiki.nycsmalltalk.org
http://www.monteirosfusion.com
http://monteirofusion.blogspot.com

On Jun 22, 2006, at 6:33, Runar Jordahl wrote:

The current VM/VI split is very 4GL-ish. High level descriptor at the  
VI level, magic that makes it happen buried in the VM. The VM does  
some very interesting things from platform to platform to try and  
make the VI model work. The architecture is very 1980's X-drawable-
ish. As you noted, the X drawable thing has grown long in the tooth:  
witness Aqua/Quartz, Aero/Glass, as well as all of the X work that's  
been going on (XFT, xRender, Cairo, X extension ad nauseum).

The problem with the current arrangement is that it's a black box.  
Welded shut. You can use DLLCC to do your own thing, which more and  
more of us have, but then there's a serious disconnect between the  
stuff VW is managing and the stuff you do. Try for example to control  
the cursor. Or to use DLLCC to place a child window for khtml  
rendering inside of a VW managed window. The other problem with the  
current arrangement is that the VM often has to dive down past the OS  
provided services to come up with an interface/representation that is  
savvy with it's dated idea about graphics, and then bring it back up  
to the level where it's at. Aqua is a good example of this.

As I understand/hope the vision to be... it is to make the interface  
between the VM and VI more apparent. To not have the VM be so smart  
for you. But to simply expose a wider interface to you. One can very  
easily imagine a much larger version of the kind of arrangement you  
see with the Filename class heirarchy. Where you have basic  
operations at the top, but then you can dive down into the platform  
specific subclasses and see how it accomplishes some of the different  
things from platform to platform, as well as add platform specific  
features (symlinks come to mind). I don't think the notion is to  
necessarily do more of the computation in Smalltalk, but by exposing  
the interface at a higher level, allow the VI to have a much more  
efficient interface to the underlying OS services. For example, not  
recreate an XGC on nearly every drawing primitive, but allow it to  
move up and down the tree and be preserved, the same way the  
Smalltalk on does. Or to use the platforms services to render bitmaps  
of varying color palettes rather than the arcane method of  
convertForGraphicsDevice: which I described in my blog response to  
yours. One of my most anticipated operations will be to move the  
XFlush/XSync calls up out of the VM into Smalltalk managed loops  
using thapi or something. Currently... every message send and  
backwards jump "checks with X" to see if it should note any user  
activity.

--
Travis Griggs
Objologist
"It's [a spec] _the_ single worst way to write software, because it  
by definition means that the software was written to match theory,  
not reality" - Linus Torvalds



-----------------------------------------
DISCLAIMER: This email is bound by the terms and conditions
described at
http://www.key.net/disclaimer.htm

Runar,

>I might have misunderstood, but there seems be a plan to move more
>graphic code up to the Smalltalk layer. This is at least the
>impression I get when hearing about "Peaches". I fear that basic
>graphic operations will become too slow if everything is moved into
>Smalltalk. In fact, I would think doing the opposite -- moving more
>functions down to the VM -- is required in order to speed up graphics.

Actually, if done right, it will not be slower, and in fact faster in almost
every area.

The issue is abstraction levels. Currently, the VM has to take the Image's
abstraction and translate it into a native behavior and/or format. If instead,
the Image were to make native calls and use native formats for each platform,
the overall speed would be faster.

For instance, if there were a more complete native graphics context model in the
image, such things as reloading ALL the values (which currently happens on EACH
graphics context call in the VM) would not be needed, because the GC could know
what (if anything) has actually changed.


                                And So It Goes
                                     Sames
______________________________________________________________________

Samuel S. Shuster [|]
VisualWorks Engineering, GUI Project
Smalltalk Enables Success -- What Are YOU Using?