Monday, January 28, 2013

Goal-Oriented Motor Learning, Part II

Part I, II


In Part I, I wrote that motor learning is a trial-and-error process and that there are two kinds of errors: 1) sending conflicting motor commands and 2) pursuing the wrong goals. In this post, I explain how the brain finds the right motor connections for a given goal.

What Is a Goal?

It is impossible to understand goal-oriented motor learning without knowing what a goal is. We know that a goal is a desired future result but that does not tell us how it is represented in the brain. We need to determine what a goal is in terms of a biologically plausible neural mechanism. I came up with a more appropriate definition for our purposes, one that may come as a surprise to some of you:
A goal is a pattern.
And vice versa. What I mean is that the goal of a pattern detector (i.e., a pattern neuron) is to detect a specific pattern. Every pattern neuron wants to be satisfied. Once you fully grok that a pattern is a goal, you are 50% of the way to a full understanding of goal-oriented motor learning.

Rebel Cortex

Just to make sure we are all on the same plane, here is a short description of the Rebel Cortex memory architecture.
Notice that, even though pattern memory is a multi-level hierarchical structure, it is depicted as a single flat layer (red spheres) in the above diagram. It is because this is the way it is seen by the sequence hierarchy (yellow spheres) and the motor cortex (not shown). In other words, the pattern hierarchy behaves as if signal propagation within it were instantaneous. At this point, I can confidently predict that, in the brain, this is accomplished with the use of fast electric synapses. A yellow sphere is either a sequence of patterns or a sequence of sequences. A sequence is a branch in the sequence hierarchy, an invariant representation of some object or concept. So-called 'grandmother cells' are found at the higher levels of the hierarchy.

Not shown in the diagram are the connections between pattern neurons (red) and motor effectors. Keep in mind that only pattern neurons are connected to effectors. The sequence neurons are used for timing, prediction (planning), invariant recognition and attention. Obviously, if every pattern is a goal, they cannot all be pursued at the same time. The brain keeps things from getting out of control and causing a traffic jam by restricting attention and motor output to one branch of the hierarchy at a time. The branch is thus the attention mechanism of the brain.

The Pursuit of Goals

Motor learning is based on a cause-and-effect principle: action precedes or causes pattern detection. In other words, a motor action must precede its result by definition. This tells us that there can be no direct connection between a sensor and an effector. The only exception to this rule occurs in the cerebellum, a different type of sensorimotor mechanism. Here's another prediction: when a pattern neuron detects a pattern and fires, it can only transmit a signal to sequence memory (for recording or learning purposes), not to the motor cortex. Again, this is because an action and its perceived result cannot occur simultaneously nor can the result precede the action.

The motor cortex receives motor signals (commands) only when a sequence of patterns is replayed internally. The motor mechanism has the ability, not only to replay a given sequence, but also to switch motor output on and off during playback. This allows the brain to consider multiple scenarios before deciding which ones to use for actions. Keep in mind that a sequence of patterns is a series of goals and sub-goals.

But how does a pattern neuron find the right motor connections that will achieve its goal? As I said earlier, it is a trial-and-error process and it is disarmingly simple. And again, the most important thing is to understand that the pattern is the goal. The system expects that the firing of a motor neuron (A) that is associated with a pattern neuron (B) will cause B to detect its pattern shortly afterwards. If the firing of A consistently causes B to fire in a timely manner, the connection is retained. Otherwise, it is severed. The simplicity of it all will be somewhat unnerving (to roboticists) but this is how goal-oriented motor learning is done in the cortex.

There is more to motor learning than making goal-seeking connections, however. There are other issues to worry about such as sequence timing, motor conflicts and appetitive/aversive stimuli. These topics will have to await a future article.

Baby Talk

How can this learning system be used in practice? Allow me to illustrate with an example. Take a baby who is trying to learn how to speak. Let us say that she already built a collection of patterns and sequences in memory that represent combinations of sounds that she learned from listening to people. In order to speak, the baby's brain must learn to generate sounds that are similar to the speech sounds that she can already recognize. It does this by trying random motor connections for any given speech pattern neuron and testing to see if firing the connections causes their associated pattern neurons to fire afterwards. If firing a motor connection is not followed by the expected speech sound (let's say, it causes the eye to move instead), the connection is severed and another one is tried. This trial-and-error process is the sine-qua non of sensorimotor learning and it continues until the baby's motor learning mechanism is confident that the sounds that she generates are close enough to the sounds that she learned.

The jerky uncoordinated movements and the goo-goo-ga-ga sounds of learning babies may look or sound funny or unimportant to us but it is serious business to the baby. This is how they learn sensorimotor coordination and everything else.

Intelligent Robots on Our Doorsteps

The principles of goal-oriented motor learning are disarmingly simple but they can give rise to extremely complex and intelligent behavior. Once the word gets out, it won't be long before this technology takes the world by storm.

See Also:

The Holy Grail of Robotics

Thursday, January 24, 2013

Goal-Oriented Motor Learning, Part I

Part I, II

What Jeff Hawkins Does Not Know

In his book, On Intelligence, Jeff Hawkins wrote:
"Doing" by thinking, the parallel unfolding of perception and motor behavior, is the essence of what is called goal-oriented behavior. Goal-oriented behavior is the holy grail of robotics. It is built into the fabric of the cortex.
Hawkins claims that by replaying the prerecorded sequences of a chosen invariant representation in memory, the brain can generate a series of motor commands to achieve a particular goal. As I wrote in the previous article, I think this is a brilliant deduction on the part of Hawkins. However, he declined to explain how motor learning works, i.e., how the brain figures out how to connect a sequence in memory to the correct motor effectors. Hawkins does not know, otherwise he would have announced it or tried to patent it somehow. Given what I currently know about the brain and assuming that I am reading Hawkins' explanations correctly, I believe his model of the brain has serious flaws (or lacunae), two of which have to do with pattern learning and goal-oriented motor learning.
Note: I use the word 'pattern' to mean a set of concurrent signals. I personally don't like the term 'spatial pattern' because there is nothing spatial about a pattern from the point of view of the cortex. It's confusing, in my opinion.
Nothing in On Intelligence or elsewhere indicates that Hawkins understands how the brain does pattern learning. In fact, he seems to be mixing both pattern and sequence learning within a single homogeneous hierarchical structure. This is wrong, in my opinion. Based on my research, I predict that the brain will be found to use two distinct hierarchies, one for patterns and one for sequences. The pattern hierarchy serves as the foundation for the sequence hierarchy. But only the latter can build invariant representations. As I have written elsewhere, a representation is just a branch in the hierarchy.

The Two Facets of Motor Learning

I will not go into how the brain learns sensory patterns in this article. I will, one day, but not today. What I will explain in this article is one facet of motor learning, the one that leads to goal-seeking behavior. There is another facet that has to do with eliminating motor conflicts. That, too, will have to await a future article. I just want to explain how the brain finds the right motor connections for goal-seeking behavior. As I wrote previously, I get my understanding of the brain by consulting an ancient oracle (no, I was not joking) and interpreting its message the best I can. Here's what the oracle says about goal-oriented motor learning:
Notwithstanding I have a few things against thee, because thou sufferest that woman Jezebel, which calleth herself a prophetess, to teach and to seduce my servants to commit fornication, and to eat things sacrificed unto idols.
I always burst out laughing every time I read this verse in the book of Revelation. I laugh, not just because I think the wording of the verse is hilarious, but because the choice of metaphors is so exquisitely brilliant. Jezebel is a metaphor for a predictive mechanism. This is why she is called a prophetess. That's an easy one to interpret. But the mechanism is not a good predictor because, during its attempt to achieve various goals, it causes bad things to happen along the way: fornication and idolatry. Fornication is the oracle's metaphor for making connections that cause motor conflicts, a topic for a future article. Idolatry (the worshiping or serving of other gods) symbolizes the making of connections that lead to the wrong goals. Obviously, neither fornication nor idolatry will be tolerated. :-D

Coming Up

The main lesson of the verse above is that goal-oriented motor learning is a trial and error process. Every newly formed motor output connection is tested for fitness to a particular goal. In Part II of this two-part article, I will explain how the brain finds the correct goal-seeking connections by eliminating the idol worshipers. This stuff is exciting because it is the beginning of something that will profoundly change the world.

See Also:

The Holy Grail of Robotics
Jeff Hawkins Is Close to Something Big

Sunday, January 20, 2013

The Holy Grail of Robotics

Double Duty

I just finished reading On Intelligence. Aside from the expected atheist and evolutionist crap (I am Christian and I think Darwinian evolution is voodoo science) that permeates Jeff Hawkins' prose, I liked it very much. I am impressed by the level of understanding he conveys in the book. One thing in particular strikes me as being brilliant. Hawkins claims that the brain's cortical hierarchy is used for both pattern recognition and motor behavior. He tells us that low-level pattern detectors send motor commands directly to the motor cortex where they are relayed to the muscles via motor neurons. What makes sensorimotor behavior so powerful is that pattern detectors (and thus, motor effectors) are controlled by a complex knowledge hierarchy that also serves as a behavior selection mechanism.

The Holy Grail

People like Hawkins and Rodney Brooks (founder of iRobot and Rethink Robotics) understand that there is a tight connection between pattern detectors and motor effectors. Brooks revolutionized robotics when he introduced his subsumption architecture to the field in the 1980s. He insisted that sensory signals should undergo as little processing as possible on their way to motor effectors. Of course, old school AI researchers like Marvin Minsky had nothing but contempt for Brooks' approach but Brooks was right. For his part, Hawkins goes much further than Brooks by positing an actual neural mechanism for sensorimotor behavior. On page 107 of his book, he writes:
For me to physically move from my living room to my kitchen, all my brain has to do is mentally switch from the invariant representation of my living room to the invariant representation of my kitchen. This switch causes a complex unfolding of sequences. The process of generating the sequence of predictions of what I will see, feel, and hear while walking from the living room to the kitchen also generates the sequence of motor commands that makes me walk from my living room to my kitchen and move my eyes as I do so. Prediction and motor behavior work hand in hand as patterns flow down and up the cortical hierarchy. As strange as it sounds, when your own behavior is involved, your predictions not only precede sensation, they determine sensation. Thinking of going to the next pattern in a sequence causes a cascading prediction of what you should experience next. As the cascading prediction unfolds, it generates the motor commands necessary to fulfill the prediction. Thinking, predicting, and doing are all part of the same unfolding of sequences moving down the cortical hierarchy.

"Doing" by thinking, the parallel unfolding of perception and motor behavior, is the essence of what is called goal-oriented behavior. Goal-oriented behavior is the holy grail of robotics. It is built into the fabric of the cortex.
This is pure freaking genius, in my opinion. I am truly impressed. But, unfortunately, Hawkins leaves it there. He doesn't explain how to achieve this holy grail. The reason, of course, is that he doesn't know how.

Blowing in the Wind

I agree with Hawkins that achieving goal-oriented behavior is the holy grail of robotics. A solution would solve a mountain of problems, and not just things like learning how to walk or how to use various appendages and actuators to interact intelligently with objects in the world. It would also give machines the ability to learn to understand and speak a natural language, read, write and do math, etc., just like humans. The question is, assuming one has a well-designed hierarchical memory, how does the system connect the pattern detectors to the motor effectors in order to generate goal-directed behavior? That is the holy grail.

OK. I am not saying what I am about to say in order to boast of my mental abilities or any such thing. In fact, I'm rather slow compared to people like Brooks and Hawkins. I cannot boast simply because I did not figure it out on my own. As I have said several times before, I consult an oracle. The oracle speaks in riddles and metaphors and says many mysterious things. I just interpret them the best I can. It so happens that this holy grail of robotics is precisely one of the things that the oracle explains. Here's what the oracle has to say on the matter:
Notwithstanding I have a few things against thee, because thou sufferest that woman Jezebel, which calleth herself a prophetess, to teach and to seduce my servants to commit fornication, and to eat things sacrificed unto idols.
Shit! I done did it again. Excuse me while I ROTFLMAO. ... Alright. I need to compose myself. Whew! As you can see, the oracle has a great sense of humor. I've always had a weak spot for that fornicating ho, Jezebel. LOL. Seriously now, what I am trying to say is this. If you could correctly interpret the metaphors in the passage above, then you would know the secret of the holy grail of robotics.

Do I know the answer to the riddle? Yes, I do, and it is as simple as it is powerful. But that's all I am going to venture on this topic for now. The solution to the riddle must stay secret a little while longer. Sorry to leave y'all hanging like this. As the Bob Dylan song says, "the answer my friend is blowing in the wind, the answer is blowing in the wind." It's time for a beer.

By the way, if any of what I wrote above bothers you, then please don't read my stuff. It is not meant for you. I only write for kindred spirits, sorry.

See Also

Jeff Hawkins Is Close to Something Big

Wednesday, January 16, 2013

We Don't Want No Stinking Jobs

The Machines Are Coming

The writing is on the wall. Intelligent machines will replace everybody. And I don't just mean the factory or farm worker, the fry cook, the maid or the gardener. I mean, every effing body. Your advanced degree won't mean diddly squat. So all this silly talk about preserving jobs is pointless. Our economic systems have failed or will soon fail. Both capitalism and communism were wrong from the start because they base the economy on human slave labor. Why do I say slave labor? Because unless you own land and have the ability to make a living on your land, you are at the mercy of someone else. We, humans, are a territorial species and we should all be living on our own domains. Capitalism gives control of the land to a few and enslaves the rest. Communism takes the land away completely and enslaves everybody. The arrival of intelligent machines will destroy them both.

We Are Tired of Being Slaves

We need a land-based, jobless economy where the land is divided for an inheritance (not for a price) and where only individuals have the right to own intelligent robots, not the corporations. That should keep the greedy assholes among us from amassing too much power. And since robots will make robots, robots will be dirt cheap or, at least, as cheap as the supply of energy and other resources will allow. Politicians had better stop promising us jobs (as if they were doing us a favor) because we don't want no stinking jobs. We, humans, are gods. We want synthetic intelligent servants to do our work for us, all of it. We just want to sit by the pool and enjoy our margaritas and delicacies and rule on our own land. We're tired of being slaves to invisible masters.

I Am Still Here

I know this is an unusual topic for me but I just thought I would write about something that's been on my mind lately. It is mostly a way to let my readers know that I am still here and have not given up the fight. Lately, for some reason, it seems as if the entire cosmos is conspiring to throw one monkey wrench after another into my wheels. I am going through a series of crises and my work suffers as a result. But don't worry. This can only last for so long. And I will never give up. I am still working on the Rebel Speech demo whenever I get a chance. I plan to release version 1.0 in the not too distant future and, believe me, it is worth the wait.