Seriously. We've been sitting around twiddling thumbs about better graphics for decades now.
We had Balloon2D & 3D. We had 'Pooh graphics' to do vector forms. We have some excellent stuff being done by the Cuis crew. We have some amazing ideas from Nile/Gezira. We have a number of potential 3rd party graphics libraries we might adopt. We have an advanced JIT that could be used to make on-demand graphics routines either on their own or working with pretty much any of the above. We probably have other options I've not even heard of. Maybe we should actually do something? Can we at least talk about it? tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Strange OpCodes: SVE: Skip on Vernal Equinox |
Tim Rowledge wrote on Sun, 28 Jun 2020 17:53:21 -0700
> Seriously. We've been sitting around twiddling thumbs about better graphics for decades now. > > We had Balloon2D & 3D. > We had 'Pooh graphics' to do vector forms. > We have some excellent stuff being done by the Cuis crew. > We have some amazing ideas from Nile/Gezira. > We have a number of potential 3rd party graphics libraries we might adopt. > We have an advanced JIT that could be used to make on-demand graphics routines either on their own or working with pretty much any of the above. > We probably have other options I've not even heard of. You didn't mention the original BitBlt and WarpBlit, which is understandable if your focus is on vector graphics. > Maybe we should actually do something? Can we at least talk about it? Though the survey I did for the Board pointed to OpenGL ES as our best option (at the time, things have changed since), it is not something I personally like. I want something that is not a black box so I can single step from the highest level to individual pixels when debugging issues. But I also want it to take advantage of all available resources (including GPUs, extra processors) by translating debugged code into something else. I just don't want to debug in that something else, which would be the case for OpenGL ES shaders. Nile/Gezira + JIT would be my vote, but it has been planned for a long time and, as far as I know, is not happening right now. -- Jecel |
Hi everyone, I am of particularly low expertise in these matters, but in the grand tradition of online discourse I thought I'd chime in anyway.
Not sure if everyone keeps up with the guys on the other side of the tracks, but the Bloc library in Pharo took the approach of using Skia (originally it used Moz2D). A lot of work went into it, and a year or two ago I was able to get some interesting things up in my own experiments. This work is now no longer being done on the mainline branch, but instead as part of the Feenk GToolkit project. I'm not sure if or when the Pharo people will pick it up for incorporation. When talking about incorporating 3rd party libraries, do we mean as VM plugins or through FFI? Is there some advantage of one over the other? On an unrelated thread (in which I posted off-topic, shame) I mentioned that Vulkan provides a complete XML description of its API, which, as far as I'm able to understand these things, means the enormous pain of creating FFI bindings is somewhat ameliorated here. That said, I don't really know anything about Vulkan other than the fact it's supposed to be the "next generation" graphics api from the same producers that brought us OpenGL and OpenGL 2: The Reckoning.
Nile seems like a fascinating option and I've tried to read as much about it as I can. Unfortunately aside from the various posted talks online and the VPRI papers, there's not a lot of information about using it (unless I am missing something -- you can track a github issue about its status here). My guess is that the Gezira part, of which there is even less information, is very closely tied to the specific Frank environment in which it was used -- is that correct? Using plain Squeak how would one even incorporate Nile?
One thing about Squeak and friends that is very nice is the way it can abstract over essential "kinds" of activity in the computer. For example, the global `Processor` class manages everything I really need from my actual computer's processor: scheduling different things to run and then doing so, while providing me a no-nonsense interface into itself. The current BitBlt graphics stack is also a sweet-spot abstraction for the kinds of graphics it's working with. Once you dig down and get the hang of bits, forms, and canvases, it all comes together in a delightfully lively way. I see what Jecel is saying here and I think it's a worthy goal -- what would be the equivalent of the `Processor` global for the GPU (even in cases where the actual machine doesn't have a "real" GPU)? -- Eric |
In reply to this post by Jecel Assumpcao Jr
|
In reply to this post by darth-cheney
A setof thoughts from'offscreen'. I think a useful starting point is whatis graphics for? It has a big bearing 2/3D is an obvious distinction. However. Is GPU& shader support necessary/desirable: yes if real-time rendering for thighs like games/VR are important, but in that case there is a bigger problem jitter caused by GC, no if the main driver is application like a GUI. visualisation. Looking at the history of Smalltalk and graphics in ST the best results have been where there has been parsimony between the drover, ST the language & ST the platform: -BitBlt which was driven by the aims of the LRG and early applications. what is the vision today? -Moose/Pharao driven by data vis applications -Croquet driven by a vision of collaborative VR Trying to be all things to all users might be a nice vision but the result may be unusable by any one constituency. Other issues: what about input: morphic, tea-time, or something akin to FRP? what about other Media(sound, video) I like theidea of not having a blackboc so that you canunderstand/ the process, Balloon was a nice idea. However I found it difficult to pick up as the documentation was sparse, Other platforms to consider for inspiration: - OpenGL was not the first graphics standard, There was also PHIGS and GKS, whilst dated these had some useful ideas, e.g. PHIGS's structures and the ability to edit these. Their input handling via device abstractions was an improvement on their successors - VTK The 'Visualization Tool kit' is an Object-Oriented Approach to3 graphics that has had some success in the (scientific)visualisation community If you want to also support other media, then PREMO (Presentation Environment for Multimedia Objects) might have some ideas that are worth looking at, It is anISO/SC24 Standard that was designed with OOP in mind as the baseline for implementation( arguably premature). I'm familiar with it as I worked on it for some years. Again comes back to that question of what is the goal and use cases? The Nile work looks cool, would be nice to incorporate it or at least leave the door open. Background. An occassional Squeak user. I did once try to use squeak for vis, explorong VTK-like system in Squeak. However I backed out in part due to lack of documentation, and not the right platform community for the community I was working with. Over the last two decades I've spent time working largely in pure FP (Haskell) doing large-scale vis. I'd be interested in tracking any effort and possibly contributing. --David - - On Tue, Jun 30, 2020 at 12:15 AM Eric Gade <[hidden email]> wrote:
David Duke Professor of Computer Science School of Computing University of Leeds UK |
> Is GPU& shader support necessary/desirable:
> yes if real-time rendering for thighs like games/VR are important, but > in that case there > is a bigger problem jitter caused by GC, In the image I just uploaded two days ago and that one can get from http://www.zogotounga.net/comp/squeak/rogue.htm the main feature is a 2D real-time top-down game. I found that the GC-caused jitter was indeed a problem, but also that the main culprit was the time it takes to simply paint an image which is a surprisingly (to me) slow operation. That, plus the alpha-blending issue that was discussed on the VM list a little ago, where Juan proposed two new primitives. Stef |
In reply to this post by Jecel Assumpcao Jr
On Mon, Jun 29, 2020 at 5:01 AM Jecel Assumpcao Jr <[hidden email]> wrote: Tim Rowledge wrote on Sun, 28 Jun 2020 17:53:21 -0700 I have the same thoughts on this. Faster, smoother graphics would be really good. At the same time I like access to everything within the image. Best, Karl But I also want it to take advantage of all available resources |
> Though the survey I did for the Board pointed to OpenGL ES as our best
> option (at the time, things have changed since), it is not something I > personally like. I want something that is not a black box so I can > single step from the highest level to individual pixels when debugging > issues. > > I have the same thoughts on this. > Faster, smoother graphics would be really good. > At the same time I like access to everything within the image. +1 Stef |
In reply to this post by Karl Ramberg
On Thu, 2 Jul 2020 at 09:22, karl ramberg <[hidden email]> wrote:
If you could only choose one of these, which would you choose? "Access to everything within the image" is a really large anchor to attach to a graphics subsystem these days. We have a single-threaded Smalltalk that has deep assumptions about a single memory space and a single rectangular display area of consistent properties wired in over forty years of development. Meanwhile, modern graphics subsystems are *very* heavily parallel, use custom hardware, and you try to offload everything you can to the coprocessor because you don't want it even in your memory space. If we want a teaching and learning system that can be understood and modified by a normal programmer, we keep everything in the image and accept limited facilities and slowness. If we want something that makes use of all of the varied and wonderful capabilities of modern graphics hardware then a) what's the smallest subset we'll accept (to make it easy to port Squeak to new platforms), b) what knowledge do we require of anyone who wishes to investigate the graphics subsystem? Cheers, - Peter |
> If you could only choose one of these, which would you choose?
If new modern, faster, but opaque graphics system is implemented, then it does not have to replace the existing, fully accessible one, it can just complement it (being available via a specific Canvas subclass, and able to deliver a Form on demand). There is no need to have an opposition here. Just as we have Morphic *and* MVC, we can have BitBlt *and* a bridge to an opaque backend for people interested in that kind of performance. As for me, I am interested in both. Stef |
In reply to this post by Stéphane Rollandin
Le 30/06/2020 à 08:26, Stéphane
Rollandin a écrit :
Is GPU& shader support necessary/desirable:In the image I just uploaded two days ago and that one can get from http://www.zogotounga.net/comp/squeak/rogue.htm the main feature is a 2D real-time top-down game. Juan proposed these primitives to speed up the rendering of
VectorGraphics Widget Morph. Regarding graphics, Juan's work on Vector Graphics can be of
interest for Squeak. Hilaire -- Dr. Geo http://drgeo.eu https://pouet.chapril.org/@hilaire |
In reply to this post by Peter Crowther-2
Hi, I have tried to use the OpenGL from Squeak and found it very opaque and hard to use. Both because the Squeak side is not very well documented and because OpenGL is hard itself. I have also looked briefly at Dolphin Smalltalk which have quite fast graphics on Windows but the way you get it to work is quite elaborate and convoluted. I agree with what Stephane Rollandin wrote; keep BitBlt for simplicity and work on better way to access whatever we choose to use as a fast backend. Best, Karl On Fri, Jul 3, 2020 at 5:11 PM Peter Crowther <[hidden email]> wrote:
|
In reply to this post by Stéphane Rollandin
On Fri, Jul 03, 2020 at 05:58:58PM +0200, St??phane Rollandin wrote:
> >If you could only choose one of these, which would you choose? > > If new modern, faster, but opaque graphics system is implemented, then > it does not have to replace the existing, fully accessible one, it can > just complement it (being available via a specific Canvas subclass, and > able to deliver a Form on demand). There is no need to have an > opposition here. Just as we have Morphic *and* MVC, we can have BitBlt > *and* a bridge to an opaque backend for people interested in that kind > of performance. > > As for me, I am interested in both. > +1 Dave |
In reply to this post by HilaireFernandes
On Fri, Jul 03, 2020 at 06:15:17PM +0200, Hilaire wrote:
> > Le 30/06/2020 ?? 08:26, St??phane Rollandin a ??crit??: > >>Is GPU& shader ??support necessary/desirable: > >>yes if real-time rendering for thighs??like games/VR are important, > >>but in that case there > >>is a bigger problem jitter caused by GC, > >In the image I just uploaded two days ago and that one can get from > >http://www.zogotounga.net/comp/squeak/rogue.htm the main feature is a > >2D real-time top-down game. > > > >I found that the GC-caused jitter was indeed a problem, but also that > >the main culprit was the time it takes to simply paint an image which > >is a surprisingly (to me) slow operation. > > > >That, plus the alpha-blending issue that was discussed on the VM list > >a little ago, where Juan proposed two new primitives. > > Juan proposed these primitives to speed up the rendering of > VectorGraphics Widget Morph. I remember a discussion about that, though I don't have a link to it. It sounds like we need to follow up on this and make something happen. > > Regarding graphics, Juan's work on Vector Graphics can be of interest > for Squeak. > Definitely yes. Juan is working towards *both* high quality rendering *and* high performance. I think that there may be a perception that Morphic 3 and VectorGraphics might sacrifice performance for rendering quality, but I am not sure that is the case at all. This recent post by Juan would be of interest to any of us attempting to achieve graphics performance on modern hardware: https://lists.cuis.st/mailman/archives/cuis-dev/2020-June/001958.html I also have to add that the recent work that you (Hilaire) have been doing with DrGeo on Cuis really illustrates the potential. Your circular menu will be of interest to anyone who has been involved with Morphic, Etoys, and Scratch interfaces: https://mamot.fr/@drgeo/104310151994863180 Dave |
Le 03/07/2020 à 22:21, David T. Lewis a
écrit :
Juan proposed these primitives to speed up the rendering of VectorGraphics Widget Morph.I remember a discussion about that, though I don't have a link to it. It sounds like we need to follow up on this and make something happen. Here it is: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/505
-- Dr. Geo http://drgeo.eu https://pouet.chapril.org/@hilaire |
> On 2020-07-03, at 1:32 PM, Hilaire <[hidden email]> wrote: > > Le 03/07/2020 à 22:21, David T. Lewis a écrit : >>> Juan proposed these primitives to speed up the rendering of >>> VectorGraphics Widget Morph. > https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/505 I'm all for such improvements but I have no idea about updating the rather specialised ARM bitblt code that Ben Avison did for RPF a few years ago. That stuff is responsible for no small part of the really good UI performance we get on Pi. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim When all else fails, let a = 7. If that doesn't help, then read the manual. |
On Fri, Jul 03, 2020 at 03:08:28PM -0700, tim Rowledge wrote:
> > > > On 2020-07-03, at 1:32 PM, Hilaire <[hidden email]> wrote: > > > > Le 03/07/2020 ?? 22:21, David T. Lewis a ??crit : > >>> Juan proposed these primitives to speed up the rendering of > >>> VectorGraphics Widget Morph. > > > https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/505 > > I'm all for such improvements but I have no idea about updating the rather specialised ARM bitblt code that Ben Avison did for RPF a few years ago. That stuff is responsible for no small part of the really good UI performance we get on Pi. > Marcel added this comment to the issue on github: > Would be nice to also have the Smalltalk fall-back code for those new rules. :-) which seems important if some of the VMs may not yet implement the new rules. Dave |
What are we going to actually try to do here? Who has time/interest/skill?
Yes, Juan's suggested bitblt improvements sound good. Indeed pretty much anything he's done would be. This all has the advantage of being 'in system' too, with maybe some VM work to get the best results. Connecting to outside libraries always has costs; after all they're not exactly likely to be set up just as we'd like. An advantage of a decent open source library is that we could at least consider hacking at it to make a version that talks nicely to our bitmaps instead of of messing around with copy/convert to get the results back. More work though... Another option *might* be to use the algorithms if we can't really use the code. Still more work. We have a framework for dynamic translation in the cog vms. We could use much of it to do dynamically translated bitblts. The ARM specialised bitblt code kinda-sorta does that but we could do much better I think. Maybe that would make borrowing algorithms more sensible than linking to an outside library? I'm not a huge fan of a direct connect to openGL because of the issues like Apple not supporting it any more, and which version can a Raspberry Pi run etc. That might make sometihng like Skia more sensible since it seems to be aimed at multiple back ends and so maybe we could benefit indirectly. Of course, it leaves us vulnerable to Error 33. The Nile/Gezira stuff obviously sounds great but there is the question of actually getting it working usefully, let alone finishing it. Will that ever happen? Is what exists good enough and complete enough to be of use? There is a downloadable 'Frank' image that seems to start up ok at http://tinlizzie.org/~ohshima/Frank-170908.zip It is a Squeak 4.4 image as far as I can tell so you need an older VM of roughly that vintage - pre-Spur anyway. The small examples mostly work ok (see things like GeziraCanvas class>example* & GZKernel class>example*) though one or two take a very long time. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim State-of-the-art: What we could do with enough money. |
In reply to this post by timrowledge
Anybody want to help? Stephen * = starting with traditional MVC classes and system windows, and following that with some version of the “Wrapper” framework as still used in VisualWorks. -- Stephen Travis Pope Santa Barbara, California, USA
|
In reply to this post by timrowledge
Hi Tim-- > Seriously. We've been sitting around twiddling thumbs about better > graphics for decades now. I'd like to see a concise description of the problem. You used the word "better", so I take it you find the current graphics deficient. How? I'm interested in the topic, and have some time. At the moment, all my graphics energy goes into leveraging in SqueakJS what others have done for web browsers, which is quite a lot. thanks! -C -- Craig Latta Black Page Digital Berkeley, California, USA [hidden email] +1 415 287 3547 |
Free forum by Nabble | Edit this page |