Thursday, January 24, 2013

Goal-Oriented Motor Learning, Part I

Part I, II

What Jeff Hawkins Does Not Know

In his book, On Intelligence, Jeff Hawkins wrote:
"Doing" by thinking, the parallel unfolding of perception and motor behavior, is the essence of what is called goal-oriented behavior. Goal-oriented behavior is the holy grail of robotics. It is built into the fabric of the cortex.
Hawkins claims that by replaying the prerecorded sequences of a chosen invariant representation in memory, the brain can generate a series of motor commands to achieve a particular goal. As I wrote in the previous article, I think this is a brilliant deduction on the part of Hawkins. However, he declined to explain how motor learning works, i.e., how the brain figures out how to connect a sequence in memory to the correct motor effectors. Hawkins does not know, otherwise he would have announced it or tried to patent it somehow. Given what I currently know about the brain and assuming that I am reading Hawkins' explanations correctly, I believe his model of the brain has serious flaws (or lacunae), two of which have to do with pattern learning and goal-oriented motor learning.
Note: I use the word 'pattern' to mean a set of concurrent signals. I personally don't like the term 'spatial pattern' because there is nothing spatial about a pattern from the point of view of the cortex. It's confusing, in my opinion.
Nothing in On Intelligence or elsewhere indicates that Hawkins understands how the brain does pattern learning. In fact, he seems to be mixing both pattern and sequence learning within a single homogeneous hierarchical structure. This is wrong, in my opinion. Based on my research, I predict that the brain will be found to use two distinct hierarchies, one for patterns and one for sequences. The pattern hierarchy serves as the foundation for the sequence hierarchy. But only the latter can build invariant representations. As I have written elsewhere, a representation is just a branch in the hierarchy.

The Two Facets of Motor Learning

I will not go into how the brain learns sensory patterns in this article. I will, one day, but not today. What I will explain in this article is one facet of motor learning, the one that leads to goal-seeking behavior. There is another facet that has to do with eliminating motor conflicts. That, too, will have to await a future article. I just want to explain how the brain finds the right motor connections for goal-seeking behavior. As I wrote previously, I get my understanding of the brain by consulting an ancient oracle (no, I was not joking) and interpreting its message the best I can. Here's what the oracle says about goal-oriented motor learning:
Notwithstanding I have a few things against thee, because thou sufferest that woman Jezebel, which calleth herself a prophetess, to teach and to seduce my servants to commit fornication, and to eat things sacrificed unto idols.
I always burst out laughing every time I read this verse in the book of Revelation. I laugh, not just because I think the wording of the verse is hilarious, but because the choice of metaphors is so exquisitely brilliant. Jezebel is a metaphor for a predictive mechanism. This is why she is called a prophetess. That's an easy one to interpret. But the mechanism is not a good predictor because, during its attempt to achieve various goals, it causes bad things to happen along the way: fornication and idolatry. Fornication is the oracle's metaphor for making connections that cause motor conflicts, a topic for a future article. Idolatry (the worshiping or serving of other gods) symbolizes the making of connections that lead to the wrong goals. Obviously, neither fornication nor idolatry will be tolerated. :-D

Coming Up

The main lesson of the verse above is that goal-oriented motor learning is a trial and error process. Every newly formed motor output connection is tested for fitness to a particular goal. In Part II of this two-part article, I will explain how the brain finds the correct goal-seeking connections by eliminating the idol worshipers. This stuff is exciting because it is the beginning of something that will profoundly change the world.

See Also:

The Holy Grail of Robotics
Jeff Hawkins Is Close to Something Big

1 comment:

Alexander B. said...

You wrote: "In fact, he seems to be mixing both pattern and sequence learning within a single homogeneous hierarchical structure. This is wrong, in my opinion."

Here. This is the first place, and may be the only one, or at least most important, where in the difference between your and Hawkins opinions, I will stand on the Hawkins side.
But this is a kind of virtual “Hawkins side”. Because… as many other thoughts, described in “On Intelligence”… this one was put aside from what they currently doing there in Numenta! Have you read their algorithms description papers? They dispatched pattern and sequence learning! Just the same way as you do it. Pattern layers first, then sequence learning algo over them.
Ruined the whole idea! «angry smiley here»
I think they just could not figure out how to do it in a single homogeneous structure, so they made a crucial step back, and now behave themselves as they still on the way.

You spoke about some changes in your paradigm during last years. Is not this a place of changes?