Friday, August 24, 2018

Temporal Correlations and Predictions in the Cortex

Using Minicolumns for Predictions and Temporal Correlations

This is another short post. I have written previously about cortical columns. I claimed that the minicolumns that comprise cortical columns are highly predictive event detectors. That is to say, the firing of a minicolumn is predictive of the firing of a minicolumn residing in another cortical column. Minicolumns owe their predictive abilities to the fact that they detect rare temporal correlations.
Note to believers: The Book of Zechariah (Zech 3:8) uses the Hebrew word 'mo.phet' which means wonder, miracle, symbol or sign (of a future event) to describe the activation of a 7-node minicolumn. That is to say, the activation of a minicolumn (the stone with seven eyes) is a rare event that announces another event.
Temporal correlations run the gamut from highly correlated to occasionally correlated concurrent signals. The signals that arrive at the entry or bottom layer of the neocortex come from highly correlated patterns. As one climbs up the cortical hierarchy, the minicolumns detect less and less frequent correlations. It just so happens that the less frequent a correlation is, the more deterministic it also is. A fully deterministic prediction is one that has only one predictor. In other words, minicolumns at the higher levels of the cortical hierarchy are very deterministic.
A cortical column shown with five minicolumns
This is all for now.

See Also:

Fast Cortical Learning Using Spike Timing

Thursday, August 23, 2018

How Would the World React?


A Question to Readers

How would people from different countries, cultures and religions react if artificial general intelligence suddenly arrives on the world scene (say, in the form of a human-level intelligent robot) and it is clearly demonstrated that the science behind it was written down thousands of years ago in metaphorical texts of the Old and New Testaments?

In my opinion, the scientific community would, at first, try its best to discredit the claim but eventually most of them would be forced  to acknowledge the truth of it based on the evidence. Christians and believing Jews would probably be inclined to accept it as a sign from God. I have no idea how the other religions (Islam, Buddhism, Hinduism, etc.) would react. I'm hoping some of my readers may want to chime in with their own personal insights.

Wednesday, July 25, 2018

AI Hot Potato

This AI thing is a hot potato. It's getting so hot, I'm on the verge of dropping it. I don't need the hassle. In a different world, I would be all for it. This world is not moving in the right direction. True AI can only hasten its demise. The remorse would kill me long before it happens.

Saturday, July 7, 2018

No Pattern Hierarchy in the Thalamus

I Make Mistakes All the Time

I was wrong about the need for a pattern hierarchy in the thalamus. There is no 10-level binary tree. The only memory hierarchy is in the cortex and it is a hierarchy of minicolumns. I was wrong because I trusted my own flawed assumptions over the occult texts that I use in my AI research. I should know better by now. The texts are never wrong. I have no excuse.

The result of this latest revision is that pattern learning is much, much faster and easier to implement than I had assumed. There are about four times as many sensors as pattern neurons and the pattern neurons have only four inputs on average. My newest experimental program now has fewer neurons than before and its recognition accuracy and noise tolerance have improved dramatically.

The Demo

I am working on a demo application (speech recognition) that I plan to release without the learning module (sorry). What will make this app special is that it will have unprecedented (spooky might be a better word) noise immunity. However, I do not want to attract too much attention. I don't need it. So I must plan this carefully. Hang in there.

See Also

Fast Unsupervised Pattern Learning Using Spike Timing

Friday, June 29, 2018

Sparse Pattern Recognition on Cheap Hardware

Just a quick note. Pattern recognition in spiking neural networks is almost magical. Once the network is properly trained, good recognition performance requires amazingly little information and is strongly immune to noise and distortion. This is what allows us to see shapes in the clouds and recognize a huge variety of typefaces and handwriting styles. It is all due to the magic of timing. The really good news is that it is possible to get excellent results using a regular multi-core processor because one can disable a lot of random neurons in a trained spiking neural network without significantly affecting performance.

Have a good weekend.

Tuesday, June 26, 2018

I May Have Been Wrong About the Organization of Pattern Memory

The Two Olive Trees

Special Note: This is for believers only.

After going over my current interpretation of the occult texts regarding pattern learning (I retrace my steps all the time), it is quickly dawning on me that I may be wrong. Pattern memory and pattern learning may be even simpler than I thought. I had long assumed that pattern memory had to be organized hierarchically, like a tree. I had originally arrived at that conclusion because the occult text (Zechariah 4) mentions two olive trees, an olive tree being the occult symbol for a hierarchical memory structure. At the time, I assumed that it meant that there was one hierarchy for sequence memory and another for pattern memory. I subsequently concluded that the two olive trees only referred to the two complementary hierarchies (Yin and Yang) of sequence memory. Each hierarchy contains cortical columns that complement the columns in the other one.

The Fig Trees

I continued to use a hierarchical pattern memory even though I was having problems with it in my computer experiments. My implementation did learn to detect concurrent patterns and it was a very fast learner. The problem was that it created way too many pattern neurons. I held on to my assumption because the occult text (Zechariah 3) does use a tree to symbolize pattern detectors although it is a fig tree and not an olive tree. Still, I felt that something was wrong. Why would the text use an olive tree to symbolize the hierarchical organization of sequence memory as a whole and a fig tree to symbolize a single pattern detector? It did not add up.

Much Simpler Than I Thought

I am currently thinking that pattern memory may not be hierarchical at all. But I have to give it more thought. This is a major revision to my previous model. I may be wrong again because I have been wrong many times before in my research. I am writing code to test my new hypothesis. I will post an update soon whether or not the new model is successful. Stay tuned.

Monday, June 25, 2018

The Many Functions of Cortical Columns

This is a short post. I don't know when but, when the time comes, I will post a multipart article on the many functions of cortical columns. They are responsible for or participate in the following:

  • Short and long term memory.
  • Knowledge acquisition.
  • Common sense understanding of the world.
  • Predictions.
  • Language understanding.
  • Reasoning.
  • Attention and focusing.
  • Motor learning and goal oriented behavior.
  • Invariant recognition.
  • Object clustering.
  • Planning.
  • Recollection.
  • Metaphors and analogies.
In addition, cortical columns are involved in reinforcement learning (adaptation) based on appetitive and aversive inputs. As with everything else in the brain, the underlying operating principle is temporality and the organizing principle is complementarity (Yin and Yang). Hang in there.