Sounding Out the View Tree

Am 10.10.2011 um 18:38 schrieb Gregory Bourassa:

> That's why I suggested an XPath syntax.  :-) Seems no one is  
> interested in that, though.

How is an API that involves string parsing supposed to be any  
efficient? I mean, we are talking about message sends that occur in  
bursts of thousands per second (drag & drop, mouse moves, resizing,  
animation, etc).

I am all against turning Wrapper into kind of SQL or something. That's  
absolutely inappropriate. Smalltalk messages are just fine for that.

Andre


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Actually (inspired by the original post), I've built a Smalltalkish XPath  
API for SimpleXO.

"element/@attribute" "can be written as"
(NameTest name: 'element') /@ (NameTest name: 'attribute')
"or more verbose"
(ChildAxis ? (NameTest name: 'element')) / (AttributeAxis ? NameTest name:  
'attribute')

"//comment()" "translates to"
RootStep // CommentTest
"or more verbose"
RootStep / (DescendantOrSelfAxis ? AnyNodeTest) / (ChildAxis ? CommentTest)

Notice that the verbose variant nicly resembles the non-abbreviated xpaths
"child::element/attribute::attribute" and
"/descendant-or-self::node()/child::comment()", respectively.

Cheers, Steffen




Am 10.10.2011, 20:29 Uhr, schrieb andre <[hidden email]>:
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Andre,

The concern for efficiency is of course a good point.

However, we are talking about writing code, not executing it.   There
are many good examples of how a string-expressed (or symbol-expressed)
thing in source code can be parsed once only and then placed in a shared
variable to be used efficiently.   User messages for multi-lingual
support are a kind of example.

Besides,  most UI things happen more in "human" time.   I certainly am
much too slow to put much stress on a Smalltalk UI.  :-)

Regards.

Gregory

On 10/10/11 11:29, andre wrote:
-----------------------------------------
Key Technology, Inc. Disclaimer Notice - The information and attachment(s) contained in this communication are intended for the addressee only, and may be confidential and/or legally privileged. If you have received this communication in error, please contact the sender immediately, and delete this communication from any computer or network system. Any interception, review, printing, copying, re-transmission, dissemination, or other use of, or taking of any action upon this information by persons or entities other than the intended recipient is strictly prohibited by law and may subject them to criminal or civil liability. Key Technology, Inc. is not liable for the improper and/or incomplete transmission of the information contained in this communication or for any delay in its receipt.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

On Oct 10, 2011, at 11:29 AM, andre wrote:

This has turned into a very insightful discussion for me. Lots of good input, thanks to all, and by all means, no need to stop either. It's all helping me to find the salient points of what I'm after. I think. :)

Some questions and observations and clarifications that have come to mind.

1) I don't think anyone is talking about doing event dispatch with a different mechanism than we have currently. Widgets are modeled with a view tree, which is handy and efficient because it models the z-order of the widgets on the screen. Common/efficient things obviously flow down the view tree. The whole point of the different VisualPart subtypes is that the represent variety in that tree, variety of appearance and user interaction. But there's another set of widget access patterns that is less structured around the local containment structure of the parent/child relationships found there. ApplicationModel code wants to access widgets in a flatter namespace (iow, we don't think about the widget we're after is 3 wrappers down and 2 composites to the right). We'd like to avoid having to keep two containment models alive (a parent/child view tree, and a secondary dictionary of named items, which has to be kept in sync with the view tree, and may not be for a variety of reaso!
 ns). And elements themselves want to have conversations with each other, that are not "whole tree wide." View tree elements have two basic ways of communicating amongst themselves. They can use things like container container and component component etc, or they can use the upcast:/downcastEvent: mechanisms. Both of these have issues. I consider the ApplicationModel builder componentAt: access mode a hack with problems, including the various "layer over that" services added on top of it.

2) The issue of getting state properly coalesced into a single widget object (rather than spread out between BoundedWrappers and WidgetStateWrappers and the Widget itself) is an orthogonal issue to me. So some have taken issues with the examples because the issues of "layout being in the wrong place" are still exposed. I think that's a good discussion, but not entirely what I was after here with this spike/prototype/exploration. The only relation between the two, that I see, is that if one were to coalesce this information as an interface on the Widget objects themselves, then a flexible way of doing parent discovery would be nice. And I would like when doing so to avoid writing a bunch of explicit isKindOf: or having to add a lot of high level isLayoutObject methods. So this seemed like *possibly* a good tool to have to use to try and fix the Wrapper imposed diaspora of state.

3) The disparity of reactions regarding wanting something like jQuery is interesting to me. So far it seems to be that "people who have used jQuery a bit" would at least entertain using a pattern like that in conventional desktop UIs. Whereas those that seem to be more skeptical, seem to hearken to less jQuery experience. It's an interesting dilemma. Do those web-ui nuts out in javascript land truly have something cool, or are they still trying to catch up with us, and pretty soon they'll see the error of their ways. Or is it an apples-n-oranges situation?

4) Syntactic Sugar is indeed a thorny/contentious issue. It seems that just about every "sugar" ever made, other than preexisting real world binary selectors, must have been contentious at one point or another. Or it was simply crammed in and people got used to it. If you were designing Smalltalk from scratch today, and you proposed the selectors -> , @ , or << and >>, I would guess you'd get a disparate reaction.

5) I've been working on a version 2 of this. I've kept some of the stuff, and thrown some of the stuff out. It's lighter now I think. Many thanks to various posts that helped me refine what I really might want this to do.

--
Travis Griggs
Objologist
"The best way to know you have a mind is to change it" -Judge Pierre Leval





_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

> Besides, most UI things happen more in "human" time.

It's not your clicks only. It's the mouse moves, progress bars, button  
updates, enablements, focus changes, animations, drag & drop, etc.  
Even simple user interfaces receive quite a lot events. Only a  
fraction are initiated by the user.

One of my current UIs shows a deep structure that refreshes  
permanently in the 10-40 milliseconds range. It shows level meters,  
progress indicators, real-time scrolling data views, moving sliders,  
etc. This is not uncommon.

Andre

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hi Andre,

True.   I know UIs are remarkably busy behind the scenes; I have worked
with the Windows and Presentation Manager message queues in their native
forms, as well as portability libraries and X-windows events.

(By the way, that current UI of yours sounds interesting.)

However,  we were only talking about a more elegant and perhaps more
general way to specify other UI components than the component in which
the the source code is implemented.   That does not mean it has to be
less efficient at run-time than the "component" messages themselves.  
It also does not mean you _must_ use it in every piece of code.

The suggestions about jQuery and about XPath, regardless of their
expression in the source code,  would boil down something very like the
underlying Smalltalk messages provided by the component tree.   I assume
that an XPath expression would be resolved once-per-session-only into a
series of those messages...implementation of that is left to the
interested reader as an exercise.  :-)   Hint: In the first iteration,
do not solve for the case of GUI-builder tools in which all
normally-fixed things must remain dynamic.

Regards.

Gregory



On 10/11/11 13:12, andre wrote:
-----------------------------------------
Key Technology, Inc. Disclaimer Notice - The information and attachment(s) contained in this communication are intended for the addressee only, and may be confidential and/or legally privileged. If you have received this communication in error, please contact the sender immediately, and delete this communication from any computer or network system. Any interception, review, printing, copying, re-transmission, dissemination, or other use of, or taking of any action upon this information by persons or entities other than the intended recipient is strictly prohibited by law and may subject them to criminal or civil liability. Key Technology, Inc. is not liable for the improper and/or incomplete transmission of the information contained in this communication or for any delay in its receipt.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Dave,

Yes!

That dictionary you describe is substantially the same as what I meant when I said "resolved once-per-session-only into a series of those messages" -- only I realized as soon as I hit the send button that I didn't mean a series of messages, but rather resolved (using the expression from the code) into a direct reference to the component you wish to access.    That reference should remain in the referencing component's state as long as the latter persists.

There could be issues if the referenced component gets rubbed out in your UI and replaced with something else that the referencing component should now be accessing -- that's why I mentioned not trying to solve GUI-builder problems...

What you describe is pretty much what is needed under the hood, I think.   How we express that -- or access that -- in our code is another part of the picture, and was the part that I was ruminating on.

GB


On 10/11/11 15:02, Dave Stevenson wrote:

We don't have widget lookup performance issues with real time apps, but we do have some very large and complex dynamic UIs with thousands of widgets where performance of "walking the view hierarchy" could become an issue. The nice thing about #componentAt: is that it uses the namedComponents dictionary to provide a fast, hash-based lookup, rather than walking the tree to find the desired widget.

Because support for subcanvases in tabs is terrible, and parts of the view hierarchy are contained in different builders, we are forced to give up the fast lookup in some cases and resort to walking the tree to find what we want. But why can't we have the best of both worlds? When building the UI, why not register each widget not just its immediate container, but also with the top component, either by its ID (typically a symbol, but we have cases with integer widget IDs in our app) for high level widgets, or a path of IDs for widgets at lower levels? So the top component could have a dictionary with mixed keys, like:
    #button1 -> aButton
    #subCanvas1 -> aSubCanvas
    #(#subCanvas1 #button1) -> anotherButton

Walking the tree should still be possible, but we'd still be able to jump right to a widget multiple levels down the view heirarchy. I accept that there are reasons for walking the tree. But I think that since the most common use case is accessing a widget by ID, that access should be optimized, which means a single, fast, top-component lookup of any widget by id, or by id path for widgets lower down the hierarchy.

Dave Stevenson
[hidden email]

---

From: Gregory Bourassa [hidden email]
To: andre [hidden email]
Cc: Dave Stevenson [hidden email]; [hidden email] [hidden email]
Sent: Tue, October 11, 2011 3:34:40 PM
Subject: Re: [vwnc] Sounding Out the View Tree

Hi Andre,

True.  I know UIs are remarkably busy behind the scenes; I have worked with the Windows and Presentation Manager message queues in their native forms, as well as portability libraries and X-windows events.

(By the way, that current UI of yours sounds interesting.)

However,  we were only talking about a more elegant and perhaps more general way to specify other UI components than the component in which the the source code is implemented.  That does not mean it has to be less efficient at run-time than the "component" messages themselves.  It also does not mean you _must_ use it in every piece of code.

The suggestions about jQuery and about XPath, regardless of their expression in the source code,  would boil down something very like the underlying Smalltalk messages provided by the component tree.  I assume that an XPath expression would be resolved once-per-session-only into a series of those messages...implementation of that is left to the interested reader as an exercise.  :-)  Hint: In the first iteration, do not solve for the case of GUI-builder tools in which all normally-fixed things must remain dynamic.

Regards.

Gregory



On 10/11/11 13:12, andre wrote:
>> Besides, most UI things happen more in "human" time.
> It's not your clicks only. It's the mouse moves, progress bars, button
> updates, enablements, focus changes, animations, drag&  drop, etc.
> Even simple user interfaces receive quite a lot events. Only a
> fraction are initiated by the user.
>
> One of my current UIs shows a deep structure that refreshes
> permanently in the 10-40 milliseconds range. It shows level meters,
> progress indicators, real-time scrolling data views, moving sliders,
> etc. This is not uncommon.
>
> Andre
>
>

-----------------------------------------
Key Technology, Inc. Disclaimer Notice - The information and attachment(s) contained in this communication are intended for the addressee only, and may be confidential and/or legally privileged. If you have received this communication in error, please contact the sender immediately, and delete this communication from any computer or network system. Any interception, review, printing, copying, re-transmission, dissemination, or other use of, or taking of any action upon this information by persons or entities other than the intended recipient is strictly prohibited by law and may subject them to criminal or civil liability. Key Technology, Inc. is not liable for the improper and/or incomplete transmission of the information contained in this communication or for any delay in its receipt.

-----------------------------------------
Key Technology, Inc. Disclaimer Notice - The information and attachment(s) contained in this communication are intended for the addressee only, and may be confidential and/or legally privileged. If you have received this communication in error, please contact the sender immediately, and delete this communication from any computer or network system. Any interception, review, printing, copying, re-transmission, dissemination, or other use of, or taking of any action upon this information by persons or entities other than the intended recipient is strictly prohibited by law and may subject them to criminal or civil liability. Key Technology, Inc. is not liable for the improper and/or incomplete transmission of the information contained in this communication or for any delay in its receipt.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hi Travis,

Thanks for the deeper insight on this, and especially for the performance numbers.

My interest is much more in how to specify the relationships in a general and (somewhat) clear way.  Whatever is the most high-performing and reliable way to resolve that specification at run-time is OK with me.  :-)    Besides,  I think your original discussion was more around the "How do we specify it better?" question...wasn't it?   And that prompted my suggestion of an XPath-like syntax for expressing relationship as a search term.

Regards.

Gregory B.

On 10/12/11 08:35, Travis Griggs wrote:

On Oct 11, 2011, at 6:22 PM, Gregory Bourassa wrote:

Dave,

Yes!

That dictionary you describe is substantially the same as what I meant when I said "resolved once-per-session-only into a series of those messages" -- only I realized as soon as I hit the send button that I didn't mean a series of messages, but rather resolved (using the expression from the code) into a direct reference to the component you wish to access.    That reference should remain in the referencing component's state as long as the latter persists.

There could be issues if the referenced component gets rubbed out in your UI and replaced with something else that the referencing component should now be accessing -- that's why I mentioned not trying to solve GUI-builder problems...

What you describe is pretty much what is needed under the hood, I think.   How we express that -- or access that -- in our code is another part of the picture, and was the part that I was ruminating on.

There is a fundamental problem (IMO) with this in general, one I just worked hard to fix in the last release or so. You are talking about duplicating state in this scheme.

This has been done before, and it can lead to problems. Until recently, widgets that were interested in "tab chain" behavior were stored in a sequence on the KeyboardProcessor. This led to quite a few ARs, not all of which were ever going to be solvable I worried, and a lot of code all over the place. The code consisted mainly of "fix this up, get these two in sync again". And any time you forgot an edge case where it went, you had out of sync situations. The problems is/was that the view-tree is not necessarily a static collection of elements. People add and remove whole sub trees of elements on the fly. And it's not done in a "top down central" way. View trees can be put together bottom up, top down, or a mix that is all over the map. So with every add/remove, you end up needing to do a full tree scan and recompilation of any replicated state. You have to do it on every add/remove because since you'd like this to be encapsulated, you don't know when you're "done" adding/removing for a while.

So for me personally, I've very much come to appreciate the "Less is More" axiom. Which leads me to want to find widgets by their intrinsic state, rather than by look-aside caches.

I also very much believe in Make it Work. Make it Right. Make it Fast. Doing the tree scan is basically a linear search. So the worst case is scanning the whole tree to find the last element (or finding no match).

On my lowly MacBookPro (now 4 years old) and not one of our faster VM platforms, I can make a perfectly balanced (4 children per parent) view tree with > 1000 elements:

p := Panel new.

[p childTree size < 1000] whileTrue: 

[(p ? [:each | each childTree size <= 1]) asArray

do: [:each | 4 timesRepeat: [each add: Panel new]]].

The size ends up as 1365.

Having to find the #last element of p's childTree takes 400 microseconds.

How complex are normal Windows? I don't know. I can run this expression on a dev image with a bunch of the standard tools opened:

ScheduledWindow scheduledWindows collect: [:each | each childTree size ]

It produces these numbers.

(83 83 158 123 134 134 121 218 238 238 238 83)

For these numbers, we start seeing values more like 100 microseconds.

--

Travis Griggs

Objologist

Simplicity is the Ultimate Sophistication -- Leonardo da Vinci

-----------------------------------------
Key Technology, Inc. Disclaimer Notice - The information and attachment(s) contained in this communication are intended for the addressee only, and may be confidential and/or legally privileged. If you have received this communication in error, please contact the sender immediately, and delete this communication from any computer or network system. Any interception, review, printing, copying, re-transmission, dissemination, or other use of, or taking of any action upon this information by persons or entities other than the intended recipient is strictly prohibited by law and may subject them to criminal or civil liability. Key Technology, Inc. is not liable for the improper and/or incomplete transmission of the information contained in this communication or for any delay in its receipt.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Thanks Travis, for the performance numbers.

I too learned the hard way many times that redundant and duplicated  
states lead to nowhere but trouble. My preferred solution to this is  
lazy evaluation plus caching. It's a great strategy to keep a dynamic  
system free from redundancy and very efficient at the same time.

The only critical question to concentrate on is /when/ to clear a  
cache. Concerning the view tree, this answer is simple: When the tree  
has changed.

If you provide a reliable hook for this, any possible caching method  
could be easily implemented (it could be as simple as a nested  
IdentityDictionary). This could even be left to the developer and  
needs not necessarily be implemented in the base image.

Andre

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Travis, some comments on your previous post:

> And elements themselves want to have conversations with each other  
> [...]
> They can use things like container container and component component  
> etc, or they can use the upcast:/downcastEvent: mechanisms. Both of  
> these have issues.

Why not provide them with links to their neighbors at build time?  
Integrity needs to be maintained only when components are added or  
removed. That should not be too difficult.

> Do those web-ui nuts out in javascript land truly have something  
> cool, or are they still trying to catch up with us, and pretty soon  
> they'll see the error of their ways. Or is it an apples-n-oranges  
> situation?

It's apples and oranges. Web UIs are way different from a desktop GUI.  
The web stuff desparately attempts to mimick the desktop, but IMO it  
fails. There are cool web applications, no doubt, but if you look at  
the amount of code and the cacophony of other technologies and layers  
(browsers, protocols, mixed language code) it takes to make them work,  
you'd rather avoid it. And more often then not they dont even work,  
because your browser ... you know.

> 4) Syntactic Sugar is indeed a thorny/contentious issue.

The question mark is such a prominent and special character, I would  
not abuse it for this purpose. IMHO it should remain reserved for more  
fundamental things.

Andre

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Claus,

>> It's apples and oranges. Web UIs are way different from a desktop  
>> GUI.
>> The web stuff desparately attempts to mimick the desktop, but IMO it
>> fails.
>
> But where do they fail?

A web UI is always client/server dialog style. The user inputs  
something and gets some result in return, usually at slow response  
times. It's a heterogenous assembly that runs inside a browser that is  
written in something else. A desktop UI is not limited to that pattern.

I could not for the life of me imagine apps like Excel, QuarkXPress,  
Adobe InDesign, MS Project, Apple Logic Studio, Adobe Illustrator,  
OmniGraffle, Avid Media Composer, Visio, VMWare Fusion, XCode, Visual  
Studio, or whatever product in that league implemented with a web-
style UI. Desktop apps require quick response times, lightweight  
implementation, hardware access and tight OS integration.

Although there are attempts to move some of those to SAS models, it  
will never be the same thing.

>
>> There are cool web applications, no doubt, but if you look at
>> the amount of code and the cacophony of other technologies and layers
>> (browsers, protocols, mixed language code) it takes to make them  
>> work,
>> you'd rather avoid it.
>
> Nowadays, you write your web application using a framework like  
> Seaside or use GWT or IceFaces or others. So this is a non issue.

That still requires you to deal with web protocols, different  
browsers, style sheet compatibility issues, javascript hacks, HTML  
quirks, XML, broken connections, lost cookies, firewall issues, REST-
ful or not, Flash or not, audio and video streaming protocols, image  
formats, ... The framework might hide much of that from you when you  
start, but all that weird machinery is still working under the hood  
and failing quite often.

Most of the cool UIs I have to deal with as a consumer (my mobile  
phone provider, for instance) just don't work. Often I have to try  
three different browsers.

>
>> And more often then not they dont even work,
>> because your browser ... you know.
>
> Isnt that the case for all cross-platform development?

No. I am using a cross-platform C++ framework for my desktop apps and  
it works as well as Smalltalk does cross-platform wise: write once -  
run everywhere.

As I said: It's apples and oranges. Entirely different breeds. I'm not  
saying web UIs are inferior or something. They should just not be  
taken as a model for desktop UIs (which takes us back to the original  
topic).

Andre


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc