newbie question on treemodel

I experienced bad performance using the treemodel for storing data read from
a database with Artists, Albums and cd track information. I tried to build
the tree in pieces and after that setup the main tree, so:

tree1 add: tree2 asChildOf: nil.

But all I got is a view with just the words 'a treemodel' a dozen times,
....

Does anybody have any tips or hints...?

Thanx, ThPu

thpu wrote:
> I experienced bad performance using the treemodel for storing data
> read from a database with Artists, Albums and cd track information. I
> tried to build the tree in pieces and after that setup the main tree,
> so:
>
> tree1 add: tree2 asChildOf: nil.
>
> But all I got is a view with just the words 'a treemodel' a dozen
> times, ....

 From the sound of it, you are expecting one TreeModel to be able to be a
"branch" of another TreeModel; that's not a silly thing to think, but it
happens not to be true.

The way the TreeModels work is that they contain an internal data-structure
(the real "tree") and are able to use that to tell the TreeView what nodes
there are and what is a child/parent of what.  So they are not "trees"
themselves, they just hold *hold* a tree, so you can't compose them using
#add:asChildOf:.

You don't say what the performance problems you are seeing are, but if it's
something to do with too many updates as you are building the tree then the
thing to do is to compose the whole tree off-line (as it were), before
connecting it to the Presenter (by setting the #model:).  Otherwise I'm a bit
surprised that you are seeing performance problems with what is presumably a
fairly small tree -- you can't possibly have more than a few thousand CDs can
you ? -- so there may be some other problem with what you are doing.

HTH

    -- chris

Hi Chris,

Thanx, this is what I do:
(self readArtists)

do: [:artist | aTree add: (artist at: 'ArtistName') asChildOf: nil.

                 (self readAlbums: (artist at: 'ArtistID'))

                 do: [:album | aTree add: (album at: 'Title') asChildOf:
(artist at: 'ArtistName').

                                    (self readTracks: (album at: 'AlbumID'))

                                    do: [:track | aTree add: (track at:
'Title') asChildOf: (album at: 'Title').]]].

The database connection is pretty fast, there are about 300 artists and
about 500 cd titles with an avarage of 10 tracks. So I thought the
performance issue (about 1 minute) was in searching the right parent nodes
in the tree during insertion (by the way adding the list of artists only is
pretty fast). Building subtrees of albums and tracks and then connect that
subtree in total to the maintree should speed up that process a bit .... I
thought.

After building the tree the presenter is created, is this what you mean by
building the tree offline?

Thanx already, ThPu


"Chris Uppal" <[hidden email]> wrote in message
news:3e19e660$0$157$[hidden email]... data-structure
> (the real "tree") and are able to use that to tell the TreeView what nodes
> there are and what is a child/parent of what.  So they are not "trees"
> themselves, they just hold *hold* a tree, so you can't compose them using
> #add:asChildOf:.
>
> You don't say what the performance problems you are seeing are, but if
it's
> something to do with too many updates as you are building the tree then
the
> thing to do is to compose the whole tree off-line (as it were), before
> connecting it to the Presenter (by setting the #model:).  Otherwise I'm a
bit
> surprised that you are seeing performance problems with what is presumably
a
> fairly small tree -- you can't possibly have more than a few thousand CDs
can
> you ? -- so there may be some other problem with what you are doing.
>
> HTH
>
>     -- chris
>
>

thpu,

> The database connection is pretty fast, there are about 300 artists
> and about 500 cd titles with an avarage of 10 tracks. So I thought the
> performance issue (about 1 minute) was [..snipped...]

Wow! You have a very serious undiagnosed performance problem somewhere.  (Or
are you running on an abacus ? ;-)

I tried building a TreeModel with 500 "bands" (represented as Strings of two
random words from a wordlist), each with an average of 1.6 "cds" (represented
as Strings of three random words), each with an average of 10 "titles" (more
Strings of three random words).

(Incidently, I've been enjoying the resultant names -- very '70s.  It's a pity
that Extralinguistic Silversmith only ever released one album; but after
classic tracks like "Hugging tid bona" and "This'll fever Newport", it's hard
to see where they could go next...)

Populating the TreeModel, with no Presenter connected, took about 1/2 second
(on this 650Mz P3), of which about half was just generating the random strings.
Connecting the model to a TreePresenter before populating it made the process
go up to about a second.

So you are loosing about a minute somewhere that you shouldn't.

First thing to try is check the database stuff.  Comment out the
#add:asChildOf: calls (but still fetch the same data from the database in the
same order), does it still take more than a second or two to run ?  If so then
the database -- or more likely the way you are using it -- is the problem.

Next (if it isn't the DB) thing is get hold of Ian Bartholomew's "Profile
Browser" (from http://www.idb.me.uk/goodies.html).  That'll tell you quickly
where the time is going.  And much more easily than sitting scratching your
head, trying to work out what'll make a difference ;-)

> After building the tree the presenter is created, is this what you
> mean by building the tree offline?

Well, either that or create and populate the Model before telling the Presenter
about it:

    presenter := ...
    tree := self makeAndPopulateTree.
    presenter model: tree.

But, as I mentioned, there isn't much of a performance difference, so it's
probably not worth your while unless you find it makes the GUI look better.

Good luck!  Post again if you get stuck.

    -- chris


P.S.

    Horse Amnesia.
    Infra Sit.
    Passerby Vitrio.

I could go on and on...

Chris Uppal wrote:

> First thing to try is check the database stuff.  Comment out the
> #add:asChildOf: calls (but still fetch the same data from the database in
> the same order), does it still take more than a second or two to run ?
> If so then the database -- or more likely the way you are using it -- is
> the problem.

Another way of proving that might be to reduce the db calls by caching some
of the information to prevent multiple db accesses on the same field.

self readArtists do: [:artist | | artistName |
    artistName := artist at: 'ArtistName'.
    aTree add: artistName copy asChildOf: nil.
    (self readAlbums: (artist at: 'ArtistID')) do: [:album | | albumTitle |
        albumTitle := album at: 'Title'
        aTree add: albumTitle copy asChildOf: artistName.
        (self readTracks: (album at: 'AlbumID')) do: [:track |
            aTree add: (track at: 'Title') asChildOf: albumTitle]]].

If the db accessing is the problem (and it does look like it) then the above
ought to show an decrease in execution time.

thpu
I've added the extra #copy as there can be a problem with adding
identical strings in these simple trees - for example ...

a := 'aaa'.
b := 'bbb'.
c := 'ccc'.
t := TreeModel new.
t add: a asChildOf: nil.
t add: b asChildOf: nil.
t add: c asChildOf: a.
t add: c asChildOf: b.
TreePresenter showOn: t

... doesn't work as expected (and gets even worse if you use literal strings
in the code and don't evaluate them all together).  Something to be aware
of.

--
Ian Bartholomew

Chris, Ian,

Thanx for the tips, I did some statistics:
- the total elapsed time is about 48 sec avarage
- there are 946 database reads with an avarage time of 40 ms (including a
conversion of a DBresultset into an array)
- so this covers for about 38 sec of the 48 sec
- there are 8303 tree inserts and they take the rest of the time so 10 sec
(about 1 ms per insert).

So indeed it's not in the treemodel!

Thanx, again Theo

"thpu" <[hidden email]> wrote in message
news:avcmdu$61n$[hidden email]...
> I experienced bad performance using the treemodel for storing data read
from

"thpu" <[hidden email]> wrote in message
news:avfdag$qoe$[hidden email]...
> Chris, Ian,
>
> Thanx for the tips, I did some statistics:
> - the total elapsed time is about 48 sec avarage
> - there are 946 database reads with an avarage time of 40 ms (including a
> conversion of a DBresultset into an array)
> - so this covers for about 38 sec of the 48 sec

You might consider using precompiled statements if you are not already doing
so.  It has been a while, but I can remember one case where using
precompiled statements led to massive speed improvement.  It was a big
improvement, to the extent that the application was almost unusable without
precompilation but was quite quick with it.

For more information see the Dolphin docs here:
http://www.object-arts.com/Lib/EducationCentre4/htm/preparingsqlforlaterexec
ution.htm

These days I use a commercial add-on tool for Dolphin called ReStore (
http://www.solutionsoft.co.uk/restore/ ) for my database work.  It handles
relational to object mapping, and statement precompilation and caching.  It
is easy to use and quite powerful.

Chris

thpu,

> - there are 946 database reads with an avarage time of 40 ms
> (including a conversion of a DBresultset into an array)

Hmm, that's way too high.  If you mean that you are doing 946 DB *queries*,
then that's your problem.  A time of 40ms per query doesn't sound too bad, but
946 queries to build a simple tree is many more than I think you need.  I'd
expect you to be able to populate the tree with at most 3 queries -- something
like:

(This a mixture of SQL and Smalltalk -- don't expect it to run!)

tree := TreeModel withRoots: #() searchPolicy: SearchPolicy equality).
(select ARTIST_NAME from ARTISTS) do:
    [:artist | tree add: (Artist name: artist) asChildOf: nil].
(select CD_NAME, ARTIST_NAME from CDS) do:
    [:cd :artist | tree add: (CD name: cd) asChildOf: (Artist name: artist)].
(select TRACK, CD_NAME from TRACKS) do:
    [:track :cd | tree add: (Track name: track) asChildOf: (CD name: cd)].

That's assuming a DB layout with 3 tables, ARTISTS, CDS, and TRACKS.  Other
layouts would be perfectly sensible (and quite possibly better), so the details
of the SQL used to pull the data our would obviously change.  You could even
cut it down to one SQL query (perhaps using a join), but at the cost of
slightly more complexity in the Smalltalk code.  I doubt if it's worth it in
this case.

Still going on my assumption that you are doing 946 DB queries (and I apologise
if the assumption's wrong), then a word of advice: When you're using a SQL
database, you have to do things *its* way.  The natural way that you'd write OO
code talking to a bunch of objects, asking them for data as and when you need
it, doesn't work.  SQL databases expect to be asked for all the data you are
going to want upfront and in big gobs and, if you don't do it their way, then
your code is going to run a few orders of magnitude slower than it could.

BTW, Ian's message about the kind of comparisons used in TreeModel reminded me
that they default to identity comparison (they use #== to find if an object's
already in the tree).  I thought I'd make my example illustrate how to create a
TreeModel that uses equality comparison instead.  Note that TreeModels require
that each object appears only *once* (as determined by the kind of comparison
you use).  E.g. if I created a TreeModel that used case-insensitive String
comparison:
    TreeModel withRoots: #() searchPolicy: SearchPolicy caseInsensitive.
then I could not have 'Dog' and 'dog' appearing in two difference places in the
tree.

> - there are 8303 tree inserts and they take the rest of the time so
> 10 sec (about 1 ms per insert).


Hmm, that seems an order of magnitude too high too.  It's not your main
bottleneck, though, so it's probably not worth bothering with for performance
reasons unless/until you speed up the DB code (assuming you want to).  However
it *may* indicate a problem of some sort that hasn't yet shown up as an actual
bug.  As I've mentioned, Ian's Profile Browser is the diagnostic tool of choice
here.

    -- chris

Hi all,

Thanx for all the input, tried a lot of alternatives, but the one with the
major impact was:

-> preparing the SQL statements in advance and using the query exec:
construction.

The total time dropped from 48 to 8 sec.

Theo.

"thpu" <[hidden email]> wrote in message
news:avcmdu$61n$[hidden email]...
> I experienced bad performance using the treemodel for storing data read
from