Tuesday, November 19, 2013

Did OSU Researchers Solve the Cocktail Party Problem?

Potential Speech Recognition Breakthrough?

There is incredible news coming out of Ohio State University. Speech recognition researchers claim to be able to pick out a particular speech sound from a sound track mixed with random noise and/or other speech sounds using a deep neural network. I have my doubts about the claims but, if this is true, it would mean that they have solved a major part of the perceptual learning puzzle: the ability to focus on one thing while ignoring others. If true, it would be the biggest single breakthrough in the history of artificial intelligence research, in my opinion. That being said, I will reserve judgement until I know more about the details of the research.

Wednesday, November 6, 2013

Sequences and Analogies, Part II

Part I, II

Abstract

Previously, I wrote about Professor Douglas Hofstadter's claim that analogy is at the core of human intelligence. Although I agree with Hofstadter to a certain extent, I faulted him for focusing on language and symbols instead of investigating the root of the phenomenon. In this post, I argue that there is something that is even more fundamental than analogy: the sequence. I further argue that the neuronal mechanism responsible for discovering analogies can only work while the brain is asleep.

Temporal Proportions

Analogy comes from the Greek ἀναλογία (analogia), a word that means 'proportion' or 'proportionality'. When we think of proportionality, we usually think of geometry. For example, we can say that two triangles of different sizes are analogous if their sides are proportional. I think the ancient Greeks hit the nail right on the head. In this article, I defend the hypothesis that the mechanism used by the brain to make analogies is also based on proportionality. However, in the brain, the proportions are not measured in units of length but units of time.

Proportional Sequences

One of the curious things about analogies is that we have no recollection of learning them. They seem to suddenly materialize into our consciousness out of nowhere. We can safely surmise that the process of recognizing and establishing analogies is automatic. We can further assume that memory is organized in such a way as to make it easy for a neural mechanism to examine two chunks of knowledge and determine whether or not they are analogous, i.e., proportional.

Unlike Hofstadter, I believe that the primary function of intelligence is to make predictions, not analogies. It just so happens that predictions are possible only if recorded events are stored in sequences of patterns. The chunking ability of memory that Hofstadter is so fond of is simply the result of its hierarchical structure. Each branch of the hierarchy corresponds to a chunk. In the Rebel Science model of intelligence, a chunk is a single sequence of up to seven nodes. A node can be either another chunk or a sensory pattern. Sequences do not have fixed timing. The temporal intervals between the nodes in a sequence can vary but their proportions are invariant.
In the figure above, a yellow-filled circle represents a sequence or chunk of knowledge. I hypothesize that the brain can replay the sequences during sleep and determine whether or not any two sequences are proportional. If they are, a link is established between the two to form an analogy. The reason that analogy discovery must be done during sleep is that the sequences must be played back and that would disrupt normal brain function and behavior. During waking hours, the activation or recall of one sequence triggers the recall of another if they are analogous. I believe that all types of analogies are based on sequence timing comparisons.

Note: I am hard at work incorporating all of these principles into the Rebel Science Speech program. I hope to release a demo as soon as it is ready. Hang in there.

Monday, November 4, 2013

Sequences and Analogies, Part I

Part I, II

Abstract

How does the brain solve the problem of recognizing a musical tune or an utterance irrespective of its amplitude, tempo or overall pitch? How does it determine that a pony is a type of horse, that dogs, coyotes and wolves are related, or that going on a wild goose chase is not really about running after a bird? These are important questions because they have to do with one of the most fundamental and essential aspects of intelligence: the brain's ability to draw analogies. Without it, we would have a hard time understanding each other or the world around us. In this two-part article, I will argue that the secret to making analogies lies in the timing of sequences and the ability to make predictions.

Analogies, Hofstadter and Predictions

Professor Douglas Hofstadter is the College of Arts and Sciences Distinguished Professor of Cognitive Science and Computer Science at the University of Indiana, Bloomington. Hofstadter became famous after the publication in 1979 of his book, Gödel, Escher, Bach: An Eternal Golden Braid, for which he received a Pulitzer Prize. He is one of those elitist academics who have developed a fanatical faith in materialism and naturalism. In other words, he believes that complex life somehow sprang from dirt and evolved into bats and whales all by itself and that consciousness is an emergent property of the brain. It never ceases to amaze me that some of the most brilliant and knowledgeable people on earth can be so full of crap at the same time. From my perspective, it is fitting that true AI, the kind that materialists like Hofstadter are after, will come from the one place that they least expect.

This is not to say that Hofstadter has nothing interesting to say. Far from it. Like I said, the man is brilliant, one of those highly educated intellectuals one never gets tired of listening to, whether or not one agrees with what they have to say. The reason that I bring this up has to do with analogies. Hofstadter spent the last four decades preaching to everyone who was willing to listen that the ability to detect analogies is at the core of human intelligence. He makes a pretty convincing case. Of course, analogies are not the be-all of intelligence. There would be no intelligence without the ability to learn patterns and sequences, to make predictions, to develop motor skills, to seek goals and adapt to one's environment. Hofstadter is aware of all this, I am sure, but he has chosen to focus only on aspects of intelligence that involve analogies. For example, he speculates about how thoughts in memory are organized in chunks (what I and others are calling a hierarchical structure), and how a single conscious thought can awaken another and make it available to introspection in working memory.

Naturally, one wonders why nothing spectacular has emerged from the decades Hofstadter and his students have spent experimenting with analogies. Hofstadter has only himself to blame, in my opinion. He perfectly understands that analogies are at the core of the organization of memory but, instead of digging deep to the root of the phenomenon, he spends most of his time and energy playing with pictograms, languages and words, i.e., with symbols. It is all very entertaining but it is an altogether too high a level of abstraction. Surely the ability to recognize an analogy is much more fundamental than the high level manipulation of symbols. The only way to explain how the brain makes analogies is to come up with a biologically plausible mechanism that is universally applicable to all types of analogies. By focusing on language, Hofstadter has locked himself into the same box as the symbol manipulation proponents of the last century and the Bayesian Brain fanatics of this century. That's too bad.

Next Up

Strangely, it never occurs to Hofstadter that analogies are possible, not because the brain is designed specifically to discover them, but because the brain is designed and organized to make something that is even more fundamental than analogies: predictions. This will be the topic of my next post.