Collections/Dictionaries

Please indulge my curiosity, and if possible, I would prefer short
succint answers, to guage overall reaction - no religious wars please
<G>

Which of the following categories do you fit into :-

(1) YES - I believe collections/dictionaries to be a MUST as an integral
part of the language, and the types of applications I use them for,
are...........

(2) NO - collections/dictionaries are 'no big deal' and I can adequately
overcome use of them by alternative methods; my types of application,
are....

(3) If you answer YES to # 1 - purely hypothetical,  and it is a big IF.
If Smalltalk did NOT contain collections/dictionaries, how would you
resolve the problem, or would it impact so severly on your design
concepts that you would have to stop and rethink your approach.

My own reaction is a very ADAMANT # 1 - YES !

Informative responses would be greatly appreciated.

James J. Gavan

> (1) YES - I believe collections/dictionaries to be a MUST as an integral
> part of the language, and the types of applications I use them for,
> are...........

I agree with this (with the caveat that they are part of the libraries, not
language).  I use them for most everything.

> (2) NO - collections/dictionaries are 'no big deal' and I can adequately
> overcome use of them by alternative methods; my types of application,
> are....

I agree with this too.  The "alternative methods" are to create my own
lookup tables or whatever.  I've done this often (in other languages).  Not
hard.

> (3) If you answer YES to # 1 - purely hypothetical,  and it is a big IF.
> If Smalltalk did NOT contain collections/dictionaries, how would you
> resolve the problem, or would it impact so severly on your design
> concepts that you would have to stop and rethink your approach.

I think my answer to (2) also answers this.

    -- chris

Chris Uppal wrote:

> James J. Gavan
>
> > (1) YES - I believe collections/dictionaries to be a MUST as an integral
> > part of the language, and the types of applications I use them for,
> > are...........
>
> I agree with this (with the caveat that they are part of the libraries, not
> language).  I use them for most everything.
>

Correct - bad wording - I should have said class libraries.

Well Chris that was a zinger. A 'Yes' to each one - not what I expected <G>. I
was hoping for (1) Yes - How else would you do it you jerk ! or (2) It's
possible without Smalltalk collections, if you did this...........

But to #2 when you say you've created your own lookup tables - do you mean
other OO languages or non-OO. Now if the former, that's really saying, if they
weren't available in Smalltalk then OO-Language X has a similar feature I can
use.

However, if you mean non-OO languages - which ones ? Don't you for the most
part come up against sizing, invariably fixed. OO-wise a collection is n
'objects' depending upon how many you wish to store in it or add, subtract.

I wouldn't mind some more observations from others, positive or negative. At
this stage, no clues where I'm heading with this one, but if I get some
reasonable responses, then I'll clarify later <G>

I note one of the current threads is about Sorted Collections - so obviously
there are 'pros' who favour collections.

[hidden email] (James J. Gavan) wrote (abridged):
> Which of the following categories do you fit into :-

I don't really understand the categories, or how you are making the
language/library divide.

A language which is not powerful enough to provide a succinct expression
of collections, is unlikely to be interesting for general purpose work.
Ideally we want to be able to design our own kinds of collections. If we
can do that, then it doesn't matter whether the standard collections are
an integral part of the language or are merely part of its standard
library.

  Dave Harris, Nottingham, UK | "Weave a circle round him thrice,
      [hidden email]      |   And close your eyes with holy dread,
                              |  For he on honey dew hath fed
 http://www.bhresearch.co.uk/ |   And drunk the milk of Paradise."

Dave Harris wrote:

Let me clarify a little. While I go gung-ho for your phrase, "to provide a
succinct expression of collections", I'm not too enthused about your use of
the phrase,
"is unlikely to be interesting for general purpose work" - substitute MUST
for 'interesting' - seems to me one could interpret 'interesting' almost as
'a nice to have', and that is not my viewpoint and I don't think it is
yours.

I believe we are also on the same wavelength - if a language is going to
have libraries, I'll be more specific, is going to have Standard libraries,
then it is absolutely essential that the Standard library(classses) must
contain classes and methods, as in the case of Smalltalk, to handle
strings/numerics, and additionally an hierarchical set of arrays,
characterarrays, collections and dictionaries etc.

This Standard set forms the basic building brick, from which as you
correctly observe, you then have the ability to clone your own specific
collection needs.

Now the word Standard might throw you because I'm not clear what the
situation is with regard to Smalltalk. I really can't develop this without
clues - so here goes. Language 'X' in which I have been programming in OO
for over five years is strangulated by the ANSI/ISO bureaucratic procedures,
similarly the 'X' Standards Committee (the actual doers) is also enmeshed in
the same bureaucracy. Hint - Smalltalk is some 30 years old, Language 'X" is
40 years old <G>.

Because 90% of developers in language 'X", (survey suggests some 3 million),
code for mainframes, OO is no big deal - it's something you use with those
wretched little desktops - and of course OO is a nifty tool to do all that
GUI windowing, or Linux if you use Motif. So you will appreciate that
standards wise the emphasis is focused at mainframes.

Nevertheless, in the early 90's three brave souls proposed that the language
should include OO and put in a tremendous amount of effort, reviewing other
OO languages looking for techniques etc. You can well appreciate as an
abstract task this was a daunting effort. Approximately in '96 they came up
with a proposed list of classes and methods which should form the guts of a
standard library.

Back then the Standards people didn't twig the impact of OO, any more than
they twigged what was happening when Steve Jobs got himself that infamous
visit to Xerox PARC. However, three compiler vendors saw where it was going,
and seeing a marketing opportunity introduced OO versions of the language,
each grabbing at  different points in time at a version of the intended
standard - hence no compatibility. Two have Japanese names and the third,
Clue #2 <G> operates from Newbury, Berks. (Incidentally, I visited them last
year, my first return to Blighty in 25 years).

Now the English vendor took an independent look to see what was available -
surprise, surprise they zeroed in on Smalltalk and many of my classes
parallel yours and in fact many of the method names are identical. One
Japanese firm only markets in Japan, (I don't know why), whereas the second
is big time in the US and a direct competitor to our 'English friends'. I
don't know specifics, but can only glean that their classes are a very much
downplayed subset compared to the ones I use.

Current problem - having got over specifying changes to the non-OO version
of the language and the basics to do OO invoking, now they are taking a
fresh look at that Standard library/classes. That effort I referred to
started in the 90's - wasted, now considered old hat, and let's do a
re-think - meanwhile bear in mind we have three unofficial or extensions to
the language using OO that WORK.

So if you are a 'mainframer' you might  look at all this funny jazz
associated with OO and question why does it have this plethora of classes
and methods. They haven't latched on that the mantra is REUSABILITY, having
cloned from a base set of classes. Similarly as the language has functions
for getting dates and UPPERCASE, LOWERCASE why could you possibly want
methods "uppercase" or "lowercase" - their argument is meaningful if you are
playing with 'text'.

Because of their lack of knowledge, no slight intended, they do not
appreciate that given a collection with a set of objects you can use a
method 'set-uppercase' for example, by using a callback you can iterate
against the collection with a 'do-method'.

My concern is, that due to the fact few, if any, at the Standards level,
have precise knowledge of OO and don't code in it - they are going to arrive
at the wrong half-assed(arsed) conclusions.

OK - so now you are saying to yourself, 'Ha ! Ha! We have an 'interloper' in
our NewsGroup !" My purpose in commenting here, is one OO-er seeking the
support of other OO users - not related to a specific language, but more
importantly that a language, if it is going to use OO, must contain the
basic bread and butter techniques (Standard classes) to effectively be used
for OO, subsequently allowing you to clone from the basic set...

I'm seeking scoring points from you guys(chaps ? <G>)  to make my case to
the Standards people.

Jimmy, Calgary AB

James,

> But to #2 when you say you've created your own lookup tables - do you mean
> other OO languages or non-OO. Now if the former, that's really saying, if
they
> weren't available in Smalltalk then OO-Language X has a similar feature I
can
> use.

Well, I've done that kind of thing in both.  My major languages are
Smalltalk, C, C+, and Java.  I had to create my own containers in C
more-or-less for every project I did.  I did have a small library with some
stuff that I used fairly frequently, but most often I just created something
new for the immediate needs of the problem.  Actually, for most things I
used to use linked lists.

I've done similar things in C++ and Java, but not as often (especially these
days) since their standard containers are now more-or-less usable.  I still
need to create special-purpose ones from time to time though.  Oddly the
only language where I haven't done this is Smalltalk -- and what's odd is
that it's not that the existing containers are so flexible that I've never
felt the need, but because I've found the standard collection hierarchy
frustratingly difficult to extend reliably.

> However, if you mean non-OO languages - which ones ? Don't you for the
most
> part come up against sizing, invariably fixed. OO-wise a collection is n
> 'objects' depending upon how many you wish to store in it or add,
subtract.

I've said that I used to do this in C.  The thing that makes it practical in
C is the existence of the malloc()/realloc()/free() set of memory allocation
primitives.  Without those, I don't think it would be possible (and so,
without those, I don't think that C would be useable).

I don't really think that collections *depend* on OO.  For my money, the
language features needed to make collections (whether standardised or
roll-your-own) feasible are, in *increasing* order of desirability.

OO.  Without this it is much harder to give a simple, clean, intuitive, API
for users of the collection "module".  Also polymorphism is a major plus.
I'm not convinced that the benefits of code-sharing *within* a collections
hierarchy are all that great.  The benefits of re-use come to the users of
the hierarchy, and OO (by making the API simple) makes it more likely that
the hierarchy will be reusable and reused.

Garbage collection.  You *can* create collections objects without GC, but
it's ugly and difficult to use.  See the C++ STL for a
this-is-as-good-as-it-gets example.  Given the choice between a language
with GC but no OO, and one with OO but no GC, I'd choose the GC one for most
purposes.

Dynamic memory allocation.  Without it you are hosed.  Simple as that.  (Of
course, you may be able to create a Really Big Static Array, and then write
your own storage management to act within that arena -- it depends on the
application/platform.  It also requires that the language lets you duck
underneath the type system, if any).

For me the bottom line is that if the programming environment doesn't have
things corresponding to lookup tables and extensible linear "arrays", then
I'll add them without even stopping to think.  They are essential to
programming.  OO languages make this easier to do, and easier to use, but
I'd still do it in a non-OO language.

Oh yes, I forgot to answer one of you original questions -- application
domains.  I've been very fortunate to have worked in a wide range of
different kinds of software over the years.  I suppose I've mostly avoided
the more stereotyped domains (where the style and patterns of the code are
quite similar across projects), such as data-entry or embedded systems; this
may account for the prevalence of dynamic data-structures in my
applications.

    -- chris

Chris Uppal wrote:

Chris,

Many thanks for the detailed info, and if you read my reply to Dave you'll get a
better idea of where I'm coming from.

> Well, I've done that kind of thing in both.  My major languages are
> Smalltalk, C, C+, and Java.  I had to create my own containers in C
> more-or-less for every project I did.  I did have a small library with some
> stuff that I used fairly frequently, but most often I just created something
> new for the immediate needs of the problem.  Actually, for most things I
> used to use linked lists.
>

Well that threw me - Java ! Reading the Pletzkes - I thought you Smalltalk
people didn't even admit to knowing how to spell J-A-V-A <G>

>
> I've done similar things in C++ and Java, but not as often (especially these
> days) since their standard containers are now more-or-less usable.  I still
> need to create special-purpose ones from time to time though.  Oddly the
> only language where I haven't done this is Smalltalk -- and what's odd is
> that it's not that the existing containers are so flexible that I've never
> felt the need, but because I've found the standard collection hierarchy
> frustratingly difficult to extend reliably.
>

That's a shame, has Smalltalk, (the Grand Dame of OO languages),  missed the
boat on this one. Surely you must have some review process where such issues can
be addressed for enhancements.

> I've said that I used to do this in C.  The thing that makes it practical in
> C is the existence of the malloc()/realloc()/free() set of memory allocation
> primitives.  Without those, I don't think it would be possible (and so,
> without those, I don't think that C would be useable).
>

Primarily a designer turned programmer, I avoid things like mallocing like the
plague - although I do have facilities to use them. In the real world I
appreciate it is a necessity, but fortunately in my case although used for many
clients, (Non-Destructive Testing at oil/gas plants), it is restricted to a
small sub-set of data - so I'd be surprised if I hit a collection with say more
than 600 elements.

>
> I don't really think that collections *depend* on OO.  For my money, the
> language features needed to make collections (whether standardised or
> roll-your-own) feasible are, in *increasing* order of desirability.
>

Agreed, I initially attempted the same thing in DOS, designing screens to have a
'Windows' look For the same reasons above, no problem with collection sizes.

>
> OO.  Without this it is much harder to give a simple, clean, intuitive, API
> for users of the collection "module".  Also polymorphism is a major plus.
> I'm not convinced that the benefits of code-sharing *within* a collections
> hierarchy are all that great.  The benefits of re-use come to the users of
> the hierarchy, and OO (by making the API simple) makes it more likely that
> the hierarchy will be reusable and reused.
>

Having coded in non-OO for some 15 years, then roughly five in OO - I just find
the whole concept a DELIGHT - each day brings a new twist on something new I can
do. That's not to say a true innovator can't do the same with non-OO. It's just
the simple fact that OO gives a tremendous edge, as regards reusability, and the
ability to invoke from sub-classes up through the hierachial tree.

> Garbage collection.  You *can* create collections objects without GC, but
> it's ugly and difficult to use.  See the C++ STL for a
> this-is-as-good-as-it-gets example.  Given the choice between a language
> with GC but no OO, and one with OO but no GC, I'd choose the GC one for most
> purposes.
>

Garbage Collection - ah, there's the real pig. Well we tentatively have a draft
called 'Finalizer' but I have been - and looks like I will be for some time to
come - use the methods I have from my vendor 'finalize', 'deepfinalize' and
'deepfilanalizewithdepth'.
I control this from my Business Class using three-tier design, (Uggh! I hate
that other name 'Probelm Domain'). Should I create a temporary dictionary then
from 'Business' I'll 'finalizewithdepth' when I, (the designer) know that it is
no longer required. Similarly each Business Class has an 'exit-prgram' which
invokes 'finalize-objects', leaving untouched those objects which are truly
still 'active'.

It's not foolproof - but does work. For example, I can invoke 'objectcount',
which currently shows 10, then I might create a dialog which increases it to 15,
finalize the dialog and logic says I should finish up with 10; not so - it might
show 12 ! Still it cover the major possibilities of memory leaks. Then of course
it is up to the GC do finish the whole process off.

> Dynamic memory allocation.  Without it you are hosed.  Simple as that.  (Of
> course, you may be able to create a Really Big Static Array, and then write
> your own storage management to act within that arena -- it depends on the
> application/platform.  It also requires that the language lets you duck
> underneath the type system, if any).
>

Well I've covered that one - keep my data requirements small <G>, but agree you
are hosed with huge tables.

Many thanks Chris,

Jimmy, Calgary AB

[hidden email] (James J. Gavan) wrote (abridged):
> Let me clarify a little. While I go gung-ho for your phrase, "to
> provide a succinct expression of collections", I'm not too
> enthused about your use of the phrase, "is unlikely to be
> interesting for general purpose work" - substitute MUST
> for 'interesting' - seems to me one could interpret 'interesting'
> almost as 'a nice to have', and that is not my viewpoint and I
> don't think it is yours.

By "not interesting" I mean "not worth of further scrutiny". Consider it
British understatement if it helps :-)


> Now the word Standard might throw you because I'm not clear what the
> situation is with regard to Smalltalk.

Smalltalk has an ANSI Standard which covers both a core language and a
library. (C++ is the same, incidently.)


> My purpose in commenting here, is one OO-er seeking the support of
> other OO users - not related to a specific language, but more
> importantly that a language, if it is going to use OO, must contain the
> basic bread and butter techniques (Standard classes) to effectively be
> used for OO, subsequently allowing you to clone from the basic set...

Well, OK. I think programming is easier if we can use abstractions which
are powerful, general, and well, abstract.

I don't know what I could say to convince someone else of this, though,
if they didn't already believe it, and I wasn't familiar with their
problem domain. Also I am a bit dubious about bolting Smalltalk features
onto other languages.

  Dave Harris, Nottingham, UK | "Weave a circle round him thrice,
      [hidden email]      |   And close your eyes with holy dread,
                              |  For he on honey dew hath fed
 http://www.bhresearch.co.uk/ |   And drunk the milk of Paradise."

[hidden email] (Chris Uppal) wrote (abridged):
> Actually, for most things I used to use linked lists.

I used linked list a lot in C too. I tended not to use generic containers
as such - each module would have its own list functions embedded in its
code.


> I've said that I used to do this in C.  The thing that makes it
> practical in C is the existence of the malloc()/realloc()/free() set
> of memory allocation primitives.

Ah, here I differ. I used malloc(), of course, but as my linked lists were
not very generic they rarely needed extra memory - they could be
"intrusive" and borrow memory from the items themselves.

Thus if monsters could carry treasure we would have:

    struct Treasure {
        Treasure *pNext;      
        int value;
        //...
    };

    struct Monster {
        Treasure *pPocket;
        int eyeCount;
        int legCount;
        //...
    };

    void pickup( Monster *pMonster, Treasure *pTreasure ) {
        pTreasure->pNext = pMonster->pPocket;
        pMonster->pPocket = pTreasure;
    }

Since each piece of treasure has only 1 location, it never needs to be on
more than one list. I didn't need dynamic memory allocation to deal with
this.


> I still need to create special-purpose ones from time to time
> though.  Oddly the only language where I haven't done this is
> Smalltalk -- and what's odd is that it's not that the existing
> containers are so flexible that I've never felt the need, but
> because I've found the standard collection hierarchy
> frustratingly difficult to extend reliably.

A bit of indictment there!

Usually when I extend library collections I do so by wrapping them in my
own. What bothers me in Smalltalk is that I cannot easily offer the full
collection protocol (#select:, #collect: etc) without inheriting from
Collection, and that inheritance is often inappropriate. Is this what you
mean, or were you referring to something else?

  Dave Harris, Nottingham, UK | "Weave a circle round him thrice,
      [hidden email]      |   And close your eyes with holy dread,
                              |  For he on honey dew hath fed
 http://www.bhresearch.co.uk/ |   And drunk the milk of Paradise."

James J. Gavan wrote:

> That's a shame, has Smalltalk, (the Grand Dame of OO languages),  missed
the
> boat on this one. Surely you must have some review process where such
issues can
> be addressed for enhancements.

I think the problem is mommentum.  The standard/conventional libraries are
excellent for many purposes so it'd be difficult to gather support to make a
sweeping change just to improve the remaining 5% of uses.  I'm intending to
add more detail in a reply to Dave Harris when I have more time.

> Primarily a designer turned programmer, I avoid things like mallocing like
the
> plague - although I do have facilities to use them. In the real world I
> appreciate it is a necessity, but fortunately in my case although used for
many
> clients, (Non-Destructive Testing at oil/gas plants), it is restricted to
a
> small sub-set of data - so I'd be surprised if I hit a collection with say
more
> than 600 elements.

Hmm...  You know your business best, but I'd be a bit dubious about that.
Security alerts go out about once a week in the UNIX and Windows worlds
relating to buffer-overrun attacks.  Those happen when some C programmer has
decided that it's too much work to use a flexible-sized buffer to hold data
(and deal with the extra possibility that space might not be available to
create it), so they use a fixed-size buffer.  Years later, somebody
discovers how to send a string of data which is long enough to overrun the
buffer and overwrite other bits of program data.  If it's done with skill
and knowledge then it can be possible to trick the recieving program into
executing virtually any code the attacker wants.  Nasty stuff.  It'd be
virtually impossible to create a similar attack against a system with
easy-to-use flexibly-sized buffers.  (Which might help you sell Containers
to your audience).

> Jimmy, Calgary AB

    -- chris