Knowledgescape

Hi,

I recently found this site: http://www.kscape.com/

It seems to be a VW application that implements neural network models
and genetic algorithm optimizers.

Anybody knows if it's all written with VW?  Impressive, because I was
expecting Smalltalk to be too slow for such things...

Fernando, KnowledgeScape is indeed written in VW together with GemStone.  It
is in use in many of the worlds largest copper concentrators running
advanced control strategies based on crisp and fuzzy logic, neural network
models and ga optimizers.  Using VisualWorks and Gemstone has given us a
large business advantage as we are able to rapidly adapt our software to
business needs with a small developer group.  We are currently prototyping
Particle Swarm Optimization for addition to the package this coming year.

Mike Hales
Engineering Manager
KnowledgeScape


"Fernando" <[hidden email]> wrote in message
news:[hidden email]...

Fernando,

> I recently found this site: http://www.kscape.com/
>
> It seems to be a VW application that implements neural network models
> and genetic algorithm optimizers.
>
> Anybody knows if it's all written with VW?  Impressive, because I was
> expecting Smalltalk to be too slow for such things...

There's no reason why GA's shouldn't be written in Smalltalk.

We've built a GA "lab" in Dolphin as part of the Alchemetrics work that
was done earlier this year. The success of using a GA is far more
dependent on the mapping you choose for the genotype<->phenotype and on
the suitablity of the fitness function that you use rather than the raw
speed of the underlying mechanism. So really (as with any programming)
it's the algorithms that count rather than the low level benchmark
speed of the implementation language.

Best regards,

Andy Bower
Dolphin Support
www.object-arts.com

On 6 Dec 2004 20:05:15 GMT, "Andy Bower"
<[hidden email]> wrote:


>There's no reason why GA's shouldn't be written in Smalltalk.
>
>We've built a GA "lab" in Dolphin as part of the Alchemetrics work that
>was done earlier this year. The success of using a GA is far more
>dependent on the mapping you choose for the genotype<->phenotype and on
>the suitablity of the fitness function that you use

Could you elaborate on this? O:-)

Thanks

Fernando,

> > There's no reason why GA's shouldn't be written in Smalltalk.
> >
> > We've built a GA "lab" in Dolphin as part of the Alchemetrics work
> > that was done earlier this year. The success of using a GA is far
> > more dependent on the mapping you choose for the
> > genotype<->phenotype and on the suitablity of the fitness function
> > that you use
>
> Could you elaborate on this? O:-)

Erm.. I'm not sure which bit you want me to elaborate on but here goes.

Earlier this year we did some work building some stock market analysis
tools and a number of automated trading systems using Dolphin. The
trading systems are now live and (touch wood) so far profitable. In
order to showcase this work we've published some details at:

www.alchemetrics.org

One of the things that one often needs to do when building a trading
system is to optimize its historical performance with respect to a
number of parameters. Since it usually takes a fair few seconds to
analysis the system over (say) 1000 stocks and 10 years of data then it
is generally impractical to perform a brute force search for the ideal
parameter values. That's where a Genetic Algorithm approach can come in
handy since it allows one to perform a search over a large parameter
space with relatively few time consuming "tests" having to be performed.

When implementing a GA, the chromosomes are typically bit (or byte)
strings that are paired off, crossed and mutated at each generation
using a number of simple rules. At each generation the chromosomes are
then tested to measure their strength with respect to a particular
"fitness function". In the case of a trading system one might choose
Net Profit as a measure of the fitness of a particular system (actually
this is over simplistic but it will do as an example). The fittest
chromosomes are given a higher probability to further their line in
subsequent generations.

The next thing you need is a way of mapping the "gene" bits in the
chromosomes (sometimes called the genotype) to the appropriate
parameter values in your trading system (in biological systems this
would be the phenotype). So at each test, the chromosomes are mapped to
a trading system which is then tested for profitablity. One then runs
the GA process for a number of generations in the hope that it will
eventually converge into some "optimimal" parameter values for trading
that system.

The Dolphin Gene Lab (which may well become a sample as part of Dolphin
6) can run as a framework either for our own Smalltalk GA
implementation or the one that can be found in Diddier Besset's book
"Object Oriented Implementation of Numerical Methods". Both approaches
are fairly similar. There's a screenshot of the Gene Lab here:

http://www.object-arts.com/downloads/misc/genelab.png

If you rememeber, I said that raw speed is not necessarily the most
important factor for the success of a GA, but rather the choice of
mapping and the aptness of the fitness function. This is borne out by
the problem being solved in the above screenshot. This was a somewhat
frivolous attempt to write a GA solver for Su DokU problems that appear
in the Times newspaper:

http://www.timesonline.co.uk/section/0,,18209,00.html

I tried a number of mappings and a number of fitness functions, none of
which were successful no matter how long the algorithm was left
running. The best result was to get to 15/18 of a solution (which is
not much use for a puzzle answer). It may be that the problem is not
suited to a "Simple GA" (there are more complex variants) or it may be
that my choice of mappings were inappropriate.

We also tried a GA approach to ordering the bytecode interpreter switch
inside the Dolphin 6 VM. We wanted to ensure that the code for
bytecodes that often execute sequentially was close together in memory
(to take advantage of CPU caching etc). We ran some analysis to gather
data on common bytecode execution sequences and then built a GA to
optimize and order the bytecode dispatch. The fitness function we came
up with gave a numerical indication of how successful the ordering
would be. The best fitness order that the GA came up with was around
400,000,000. This was reached after running the GA for about an hour
but didn't improve overnight. Disappointly (from the point of view of
using GA's), Blair used a heuristic approach to give a better solution
of around 1.2 billion in around 5mins!

So, using a Genetic Algorithm still can't be considered a panacea for
"growing" all types of program. They are interesting though, I hope
you'll admit.

Best regards,

Andy Bower
Dolphin Support
www.object-arts.com

"Andy Bower" <[hidden email]> wrote in message
news:[hidden email]...
> The best fitness order that the GA came up with was around
> 400,000,000. This was reached after running the GA for about an hour
> but didn't improve overnight. Disappointly (from the point of view of
> using GA's), Blair used a heuristic approach to give a better solution
> of around 1.2 billion in around 5mins!
>
> So, using a Genetic Algorithm still can't be considered a panacea for
> "growing" all types of program. They are interesting though, I hope
> you'll admit.


Andy

Wouldn't it have been possible to give the GA Blair's heuristic solution as
a starting point, and see if it could breed anything better from that?

Peter

Peter,

After the experience of architecting KnowledgeScape, coding its GA and
applying those GAs in real-world process control applications, I remain
convinced that the same "optimization" principle governs GAs, just as it
does other optimization and learning techniques.  That is -- expressing the
knowledge (fitness function of the GA) in a way that consistently represents
the solution space and leads to findng the "correct" result.  In many cases,
how you express the problem determines whether you get one or more of the
"correct" answers.

Regards,

Randy

Hi,

"Andy Bower" wrote:

> There's no reason why GA's shouldn't be written in Smalltalk.
>
> We've built a GA "lab" in Dolphin as part of the Alchemetrics work that
> was done earlier this year. The success of using a GA is far more
> dependent on the mapping you choose for the genotype<->phenotype and on
> the suitablity of the fitness function that you use rather than the raw
> speed of the underlying mechanism. So really (as with any programming)
> it's the algorithms that count rather than the low level benchmark
> speed of the implementation language.

 Are there also prototypes of various neuronet algorithm  frameworks
available as default samples in the forthcoming D6 ?  so we can use it as a
starting point to build our AI projects easily from the shoulder of
the brilliant mind.
  Fuzzy logics included better.


Best regards,

Tk Kuo

Peter,

Possibly, and it is something we have thought of but not yet tried.
Oftens with GA's, though, it is better not to have a few chromosomes in
the population that are wildly more successful than the norm. In these
situations the fittest can rapidly swamp and kill off the poorer
members leading to a lack of diversity in the gene pool.

Best regards

Andy Bower
Dolphin Support
www.object-arts.com

Kuo,

No, we aren't likly to be including any neural net stuff in D6. The
GA stuff will only be there as a "sample" anyway, since it isn't really
the main focus of the product. In my post I was really just trying to
dispel the myth that Smalltalk is not fast enough for these sorts of
applications

Best regards,


Andy Bower
Dolphin Support
www.object-arts.com

"Impressive, because I was
> expecting Smalltalk to be too slow for such things..."

Why? Smalltalk MT runs at C++ speeds. We have several clients working with
genetic algorithms in STMT.

Dave
[hidden email]