Monday, January 28, 2013

Goal-Oriented Motor Learning, Part II

Part I, II

Abstract

In Part I, I wrote that motor learning is a trial-and-error process and that there are two kinds of errors: 1) sending conflicting motor commands and 2) pursuing the wrong goals. In this post, I explain how the brain finds the right motor connections for a given goal.

What Is a Goal?

It is impossible to understand goal-oriented motor learning without knowing what a goal is. We know that a goal is a desired future result but that does not tell us how it is represented in the brain. We need to determine what a goal is in terms of a biologically plausible neural mechanism. I came up with a more appropriate definition for our purposes, one that may come as a surprise to some of you:
A goal is a pattern.
And vice versa. What I mean is that the goal of a pattern detector (i.e., a pattern neuron) is to detect a specific pattern. Every pattern neuron wants to be satisfied. Once you fully grok that a pattern is a goal, you are 50% of the way to a full understanding of goal-oriented motor learning.

Rebel Cortex

Just to make sure we are all on the same plane, here is a short description of the Rebel Cortex memory architecture.
Notice that, even though pattern memory is a multi-level hierarchical structure, it is depicted as a single flat layer (red spheres) in the above diagram. It is because this is the way it is seen by the sequence hierarchy (yellow spheres) and the motor cortex (not shown). In other words, the pattern hierarchy behaves as if signal propagation within it were instantaneous. At this point, I can confidently predict that, in the brain, this is accomplished with the use of fast electric synapses. A yellow sphere is either a sequence of patterns or a sequence of sequences. A sequence is a branch in the sequence hierarchy, an invariant representation of some object or concept. So-called 'grandmother cells' are found at the higher levels of the hierarchy.

Not shown in the diagram are the connections between pattern neurons (red) and motor effectors. Keep in mind that only pattern neurons are connected to effectors. The sequence neurons are used for timing, prediction (planning), invariant recognition and attention. Obviously, if every pattern is a goal, they cannot all be pursued at the same time. The brain keeps things from getting out of control and causing a traffic jam by restricting attention and motor output to one branch of the hierarchy at a time. The branch is thus the attention mechanism of the brain.

The Pursuit of Goals

Motor learning is based on a cause-and-effect principle: action precedes or causes pattern detection. In other words, a motor action must precede its result by definition. This tells us that there can be no direct connection between a sensor and an effector. The only exception to this rule occurs in the cerebellum, a different type of sensorimotor mechanism. Here's another prediction: when a pattern neuron detects a pattern and fires, it can only transmit a signal to sequence memory (for recording or learning purposes), not to the motor cortex. Again, this is because an action and its perceived result cannot occur simultaneously nor can the result precede the action.

The motor cortex receives motor signals (commands) only when a sequence of patterns is replayed internally. The motor mechanism has the ability, not only to replay a given sequence, but also to switch motor output on and off during playback. This allows the brain to consider multiple scenarios before deciding which ones to use for actions. Keep in mind that a sequence of patterns is a series of goals and sub-goals.

But how does a pattern neuron find the right motor connections that will achieve its goal? As I said earlier, it is a trial-and-error process and it is disarmingly simple. And again, the most important thing is to understand that the pattern is the goal. The system expects that the firing of a motor neuron (A) that is associated with a pattern neuron (B) will cause B to detect its pattern shortly afterwards. If the firing of A consistently causes B to fire in a timely manner, the connection is retained. Otherwise, it is severed. The simplicity of it all will be somewhat unnerving (to roboticists) but this is how goal-oriented motor learning is done in the cortex.

There is more to motor learning than making goal-seeking connections, however. There are other issues to worry about such as sequence timing, motor conflicts and appetitive/aversive stimuli. These topics will have to await a future article.

Baby Talk

How can this learning system be used in practice? Allow me to illustrate with an example. Take a baby who is trying to learn how to speak. Let us say that she already built a collection of patterns and sequences in memory that represent combinations of sounds that she learned from listening to people. In order to speak, the baby's brain must learn to generate sounds that are similar to the speech sounds that she can already recognize. It does this by trying random motor connections for any given speech pattern neuron and testing to see if firing the connections causes their associated pattern neurons to fire afterwards. If firing a motor connection is not followed by the expected speech sound (let's say, it causes the eye to move instead), the connection is severed and another one is tried. This trial-and-error process is the sine-qua non of sensorimotor learning and it continues until the baby's motor learning mechanism is confident that the sounds that she generates are close enough to the sounds that she learned.

The jerky uncoordinated movements and the goo-goo-ga-ga sounds of learning babies may look or sound funny or unimportant to us but it is serious business to the baby. This is how they learn sensorimotor coordination and everything else.

Intelligent Robots on Our Doorsteps

The principles of goal-oriented motor learning are disarmingly simple but they can give rise to extremely complex and intelligent behavior. Once the word gets out, it won't be long before this technology takes the world by storm.

See Also:

The Holy Grail of Robotics

2 comments:

Curation ary said...

http://www.sciencedaily.com/releases/2013/01/130130082300.htm

— By simulating 25,000 generations of evolution within computers, Cornell University engineering and robotics researchers have discovered why biological networks tend to be organized as modules -- a finding that will lead to a deeper understanding of the evolution of complexity.

Louis Savain said...

Curation, I am not sure why you posted this. In my opinion, biological systems are modular for a very simple reason: it's good design. Evolution, like symbolic AI and the Bayesian brain, is a red herring, a false god.