Wednesday, September 14, 2011

I Was Wrong About Pattern and Sequence Learning

Major Revision

In view of my recent decision to use my understanding of the brain to raise funds for my research, forgive me for not revealing too much about my newly formulated theory of sensory learning and memory formation. It turns out that three of my main assumptions about sensory learning were incorrect. I am revising Rebel Cortex to reflect my new understanding. Here's what I was wrong about.

Pattern Learning

Previously, I claimed that pattern learning should be done in conjunction with sequence learning. Well, I was wrong about that. Pattern learning occurs independently of sequence learning. However, there is a trick to it. It's an amazingly simple trick, once you know what it is. But it is not an easy one to figure out: kind of like a needle in the haystack sort of thing. I still maintain that Numenta's approach, the one that calls for using a hierarchy of patterns starting with small areas of the visual field, is wrong. Heck, it's not even in the ball park of being right. Again, as I've explained in the past, patterns only exist at the bottom or entry level of the memory hierarchy. The upper levels consist only of sequences and sequences of sequences.

Quick prediction: All of Jeff Hawkins' money, entrepreneurial skills and atheistic convictions will not prevent Numenta's eventual demise. Bummer.

Signal Separation

I claimed many times that sensory signals should first go through a 10-stage signal separation layer that uses fixed time scale correlations to separate signals from their streams. I was only partially right about that. There is only a need for a single, fixed-time-scale separation stage.

Sequence Learning

I wrote on several occasions that the primary fitness criterion for sequence learning was frequency. I based this on the observation that sequences are often repeated; therefore the nodes in a sequence will often have the same frequency. However, this is only partially true. It turns out that the human brain can learn to remember a sequence of events even if it occurred only once. While frequency is important to the long term retention of memorized sequences, this is not how we learn sequences. Hint: it's simpler than you think, much simpler.

However, this does not exonerate the use of Bayesian or Markovian statistics in sequence learning. While these approaches do result in useful applications, they erect an insurmountable barrier to achieving the ultimate goal of AI research: building a machine with human-like intelligence and behavior.

Coming Up

Well, that's all I'm going to say about this topic, for now. Like I said, I intend to reveal all at a future date, but not before I make some cash for my research. I'm still busy writing code for the Rebel Speech Recognition project. I may be ready to release a demo sooner than I anticipated. Stay tuned.

See Also:

Rebel Speech Update, September 11, 2011
Rebel Speech Recognition
Rebel Cortex
Invariant Visual Recognition, Patterns and Sequences
Rebel Cortex: Temporal Learning in the Tree of Knowledge

No comments: