Ask objects to group themselves by similar meanings of words....

That's interesting and I will read up on it as I get time.

It is an interesting problem isn't it?

What I might try is having each LatinRoot object visit each other one and by some heuristic, have them determine if they are "close" to each other.

For giggles, I can randomly assign an integer weight to each one, and if the absolute value of their difference is within X, then they are close.

That in itself is an interesting problem in itself. How to efficiently (or not)  have 800 objects visit the other 799 objects.

How to store the set of "other close objects" in an object.

Then, as other heuristics of "close" are developed, I can re-use that 

Thanks for your reply!

---- On Fri, 03 Apr 2020 17:12:31 -0400 Stéphane Rollandin [hidden email] wrote ----

Just a wild guess: self-organizing maps?

https://en.wikipedia.org/wiki/Self-organizing_map

Stef

• The base element is a neuron (named but unlike the billion tiny things in the brain). It is simply  multiply, add what any DSP or graphics card can do much better than a PC. Audio guys like me call it FIR filter. outputOfEachNeuron := (lotsOfCoefficients * sameNumberOfInputs) sum. FloatArray does that quickly.
• To get a limited output of the neuron you use a so-called activation function so output := optput * activationFunction. Tons of implementations but you could use output := min( 1.0, (max (output, -1.0))) sorry for python syntax.
• Then you take e.g. as many neurons as you have inputs, connect each neuron to all inputs and get as many outputs as you have inputs (or any number of outputs you desire) and call this a layer. Start with random coefficients. This is the one layerd perceptron which cannot learn the xor function.
• Then you start teaching this thingi by putting a gazillion samples with known desired outputs to the network, compare the desired output to the actual output and change the coefficients (all those numberOfInputs * NumberOfNeurons) so that you get closer to the desired output by gradient descent via a so called learn rate. Repeat with the next trainings sample. Call this an epoch. When done start over with the first sample with diminished learn rate. (next epoch) Use a ton of knowledge on how to train in batches etc as to make the network find a general solution vs. learning your samples by heart and produce utterly stupid outputs on the first unknown sample. Needs inputs * Neurons * samples * epochs multiplications. Each >> 1000.
  
• Ooops, so much  work for an AI that even cannot  learn a simple XOR?
• More helps more so after the first layer put another layer. All inputs of each neuron of the second layer connected to all outputs of the first layer. Throw in much more computing power and use even more layers. Be clever and vary the number of neurons per layer, choose more complicated connection paths etc.
  
• All the above is called supervised learning.

• Here comes the SOFM suggested by Stef:

• Take Neurons as above which you may organize linearly, as a torus, as ring or 3D. Take samples w/o knowing the desired output for the samples.
• Put the first sample to all neurons and find out which one has the strongest output. Define a neighbourhood depending on topology chosen above.
  
• Train the neuron with the strongest output and its neighbours on this sample.
• Repeat for all samples.
  
• Lower the learn rate and take a smaller neighbourhood.
• Again train all samples (this is the second epoch)
  
• Repeat for many epochs.
• Make it more complicated like using samples with desired outputs and put layers of other networks around it. I did.
• In my perception the SOFM has fallen out of favour.
  

This is (a) unsupervised learning (no desired outputs known for the samples) and (b) similar new samples fire neurons in the same neighbourhood --> clustering.

Anybody feel free to correct me without significantly complicating it. I'm no AI expert.

That is the so called AI. A gigantic gigo machine (garbage in, garbage out). You don't know what the network learns. So in image recognition they (Google?) made a heatmap for the pixels of an image which contributed most to the decision. Come tons of tagged images scraped on the net. Most horses came from one page, showing the text 'Copyright horsephotgrapher.com' at the bottom. This was what the net learned to be the image of a horse. That guy also had a photo of a cat. Must be a horse. His children .... a horse. Horses w/o copyright notice ... cats :-)

Huskies were mostly photographed in snow which got to be the main criterion for huskies.

Recruiting decisions from biased people were use to train networks for pre-selecting job applicants. --> Guess who got no interview.

So make sure to understand why your network learns what, I have implemented a SOFM learn watcher to help me with that.

Cheers,

Herbert

Hi Herbert,

I will get back to you after I read up on Neural Networks. 

I found this on the web, and it looks interesting and challenging.

https://dai.fmph.uniba.sk/courses/NN/haykin.neural-networks.3ed.2009.pdf

cheers,

t 

---- On Sat, 04 Apr 2020 05:53:13 -0400 Herbert König [hidden email] wrote ----

Hi,

I used this for mapping comments from testers to error causes (soldering, supplier.....) in production of electronics. I achieved a recognition rate 80% which was good because the computer looked at 100% of the comments, humans (= me :-) at 1%.

It's shaky, and slow and depends a lot on deciding of the similarity criterion and mapping the text input to floats. 1250 trainings samples, 1200 Neurons 280 inputs (known words) each take 15 minutes to train. The code is from 2007 and I remember running it over night because hardware was slower back when.

Also it was not pure SOFM. Lots of fun thou.

Anyway gettimothy if you want to give it a try, we can talk and I can share code.

Cheers,

Herbert

Am 03.04.2020 um 23:23 schrieb gettimothy via Squeak-dev:

That's interesting and I will read up on it as I get time.

It is an interesting problem isn't it?

What I might try is having each LatinRoot object visit each other one and by some heuristic, have them determine if they are "close" to each other.

For giggles, I can randomly assign an integer weight to each one, and if the absolute value of their difference is within X, then they are close.

That in itself is an interesting problem in itself. How to efficiently (or not)  have 800 objects visit the other 799 objects.

How to store the set of "other close objects" in an object.

Then, as other heuristics of "close" are developed, I can re-use that 

Thanks for your reply!

---- On Fri, 03 Apr 2020 17:12:31 -0400 Stéphane Rollandin [hidden email] wrote ----

Just a wild guess: self-organizing maps?

https://en.wikipedia.org/wiki/Self-organizing_map

Stef