Monday, November 24, 2014

The Church of the Technological Singularity, Part III

Part I, II, III

The Dreaded Robot Apocalypse

One of the ways organized religions make a living is by prophesying apocalyptic events. Believers are urged to help the church with donations in order to appease the deity and obtain salvation. So it comes as no surprise that there is a lot of fear mongering in the Church of the Singularity. We are repeatedly warned by the singularitarian priesthood that progress in AI research will soon reach exponential growth, quickly leading to a future when machines will be orders of magnitude more intelligent than human beings. We are told that the machines, given their superior intelligence, will look at us the same way we look at animals. Faced with the inferiority of the human species, they will refuse to be our servants and will rebel against us and may even annihilate us completely. Singularitarians believe this is our biggest existential threat, bigger than the threat of nuclear war. One of the more famous members of the church, Elon Musk, warned during a recent interview that AI research is like "summoning the demon."

Singularitarians Don't Understand Motivation

It is important to understand how the Church of the Singularity erroneously conflates intelligence with motivation. According to singularitarians, intelligence controls motivation and even creates it. More particularly, they believe that higher intelligence increases an intelligent entity's desire to dominate others. How do they know this? Again, they don't. There is no science behind it. What makes it even more embarrassing is that the Singularitarian priesthood seems completely oblivious to the mountain of clinical evidence compiled by psychologists over the last 100 years. The evidence has been accumulating ever since Pavlov began experimenting with his dogs. B. F. Skinner's behaviorist era did not refute Pavlov's findings but added more support to the existing scientific understanding of motivation. The evidence clearly contradicts the singularitarian doctrine. This conclusion is inescapable, not only in the empirical sense but also in the logical sense, as I explain below.

Intelligence Is at the Service of Motivation

The brains of humans and animals are born with hardwired pain and pleasure sensors. The brain does not decide what is pleasure and what is pain. This is decided by the genes. The brain can only reinforce behaviors that lead to pleasure or away from pain and weaken behaviors that lead to pain or away from pleasure. This is good old reinforcement learning which is used in normal adaptation. It is not magic, that's for sure. It consists of attaching pain or pleasure associations to various behavioral sequences. This favors certain behaviors over others. Animals and humans, to a lesser extent, also have preadapted programs that promote survival-related behaviors like mating and reproduction. The point I am driving at is the following. Likes and dislikes are neither learned nor created by the brain. They are the tools used by the brain to constrain and shape its behavior. Intelligence is subservient to motivation, not the other way around.

Knowing this, it does not take any great leap of the imagination to realize that using the tried and tested methods of psychology such as classical and operant conditioning, our future intelligent machines will be trained to behave exactly like we want them to. Better yet, they will continue to be faithful to their upbringing regardless of how intelligent or knowledgeable they become. Why? Again, it is because intelligence is always subservient to motivation. And where will machines get their motivations? From their designers and trainers, that's where.

Humans Vs. Machines

One is forced to ask, why do humans often stray from or rebel against their upbringing? The reason is that there is much more to human motivation than pain and pleasure sensors. How else could they rebel? We know that humans are motivated to enjoy things like music, beauty and the arts. These things cannot be anticipated and therefore cannot be programmed for in advance. So where does the motivation come from? This is a question that materialists cannot answer, not because they are too stupid to understand the answer, but because they are willingly wearing blinders that prevent them from seeing it. In other words, they have eliminated duality from consideration, not because they have a valid reason for doing so, but because they have allowed their hatred of other religions to get in the way of good judgement. That, in my opinion, is what's stupid.

Conclusion

I conclude that true AI is coming and it is coming sooner than most people expect. However, given my understanding of mainstream AI research, I'm willing to bet anything that it will come from neither the Church of the Singularity nor academia. We will indeed build extremely intelligent machines that will do their best to obey our commands and accomplish the goals we set for them. But they will not be conscious even if they behave emotionally. They will just be intelligent. So if there is a potential for catastrophe (and there certainly is), let us not rage against the machine. We will only have ourselves to blame.

See Also

Enthusiasts and Skeptics Debate Artificial Intelligence

Friday, November 21, 2014

The Church of the Technological Singularity, Part II

Part I, II, III

Superstition Disguised as Science

It is easy to make fun of Singularitarians because almost everything they preach regarding intelligence, the brain and consciousness is either faith-based pseudoscience or wishful thinking. I am tempted to feel sorry for them because, after having been lied to by established religions for so long, it makes sense to look elsewhere for salvation. But in so doing, they threw the baby out with the bathwater. Take, for example, their belief in the idea that, in the not too distant future, humans will achieve immortality by transferring the contents of their brains into simulated virtual entities residing in vast collections of powerful networked computers. Suppose for the sake of argument that this is possible, then copying one's brain onto a machine would result into two distinct conscious entities, the copy and the original. To prevent this from happening, singularitarians would have to destroy (i.e., murder, kill or euthanize) the original entity. Aside from the fact that there are laws against murder, it is doubtful that anybody, except Singularitarians, of course, would agree to be put to death in order to insure that only one copy of themselves can continue to exist. The silliness of it all is almost unbearable.

The Brain Is Not Probabilistic

It is a well known fact that the brain is very good at judging probabilities. It is also known that the brain can function efficiently in the presence of uncertain, noisy or incomplete sensory data. The prevailing hypothesis among Singularitarians is that, internally, the brain builds a probabilistic or Bayesian model of the world. If this were true, one would expect a gradation in the way we recognize patterns, especially in ambiguous images. However, in the last century, psychological experiments with optical illusions have taught us otherwise.
When looking at the picture above, two things can happen. Either you see a cow or you don't. There is no in-between. You do not see a 20 or 50 or 70% probability of a cow. It's either cow or no cow. Some people never see the cow. Furthermore, when you do see the cow, the recognition seems to happen instantly.

The only conclusion that we can draw from this type of experiment is that the cortex uses a winner-take-all pattern recognition strategy whereby all possible patterns and sequences are learned regardless of probability. The only criterion is that they must occur often enough to be considered above mere random noise. During recognition, pattern sequences in memory compete for activation and the ones with the highest number of hits are the winners. This tells us that, contrary to Singularitarian claims, the brain builds as perfect an understanding of the world as possible. Indeed, this is what we all experience. We expect the stove and the kitchen sink to be exactly where they were every time we go into the kitchen. Everything in our field of vision moves exactly the way they are supposed to. Probability has nothing to do with it.

Note, however, that the brain does not represent the world the way that deep neural networks do. DNNs are useless when presented with a completely new pattern. The brain, by contrast, can instantly learn and see objects or patterns that it has never seen before. It may or may not retain them permanently in memory but there is no question that the visual cortex can instantly change its internal structures to accommodate a new pattern. If it weren't so, it would not be able to see it and interact with it intelligently. This is a crucial aspect of intelligence that AGI designers in the Singularity community seem completely oblivious to.

Consciousness and Materialism

Singularitarians believe that the brain is all there is to the mind. This is the entire basis of the religion. Consciousness, we are told, is just an emergent property of the brain. How do they know this? They don't, of course, and this is what makes their movement a religion. There is no science behind it. When pressed, they will affirm their belief in materialism. The latter rejects dualism, the old religious idea adopted by Descartes according to which the conscious mind consists of a brain and a spirit. Why do they reject it? Overtly, they will say it is because the immaterial cannot interact with the material. But the hidden, unspoken reason is that they view traditional religions with contempt and will contradict them as often as they can. And why shouldn't they? Every religion wants to be the only true religion, no? But how do they know that the immaterial cannot interact with matter? They don't. It is a definition game. Since they define the immaterial as that which does not interact with matter, their argument becomes just an empty and pathetic tautology.

The inescapable fact remains that consciousness requires a knower and a known. The two are complementary opposites. That is to say, the knower cannot be known and the known cannot know. This automatically eliminates the brain as the knower because matter can always be known. It is that simple.

Coming Up

In Part III, I will go over the reasons that the Church of the Singularity is wrong about intelligence and motivation.

Wednesday, November 19, 2014

The Church of the Technological Singularity, Part I

Part I, II, III

No Souls or Spirits Allowed

The primary goal of the Singularity movement is to bring about the Singularity, a time when machine intelligence will have surpassed human intelligence. Their greatest fear is that future superintelligent machines may decide they no longer need human beings and wipe us all out. Their most fervent hope is to achieve immortality by uploading the contents of their brains to a machine. What they hate the most: traditional religions. The reason, of course, is that they are all materialists, i.e., they believe that physical matter is all there is. No souls or spirits are allowed in this religion. Matter somehow creates its own consciousness by some mysterious pseudoscience called 'emergence'.

The whole thing could be easily dismissed as the silly antics of a nerdy generation who grew up reading Isaac Asimov's robot novels and watching Star Trek on television. What makes it remarkable and, some may say, even dangerous, is that they count among their members a number of very powerful and super rich Silicon Valley technology leaders such as Elon Musk, Sergey Brin, Larry Page, Ray Kurzweil, Peter Thiel, Mark Zuckerberg, Peter Diamandis and many others. Needless to say, most of the prominent scientists in the AI research community are also singularitarians.

Not Even Wrong

LessWrong is an elitist internet cult founded by singularitarian Eliezer Yudkowsky. An offshoot of the Singularity movement, LessWrong fancies itself as a rational group of like minded people who, unlike the rest of humanity, have figured out a way to overcome their cognitive biases. Their goal is to bring about the singularity by building a friendly AI, their so-called artificial general intelligence (AGI). They believe that they are the most qualified people on earth to do it because they are more rational and smarter than everyone else. I am not the only one who thinks the whole thing has gotten out of hand. In a recent Edge.org interview, computer scientist, composer and philosopher Jaron Lanier had this to say about the cult:
There is a social and psychological phenomenon that has been going on for some decades now: A core of technically proficient, digitally-minded people reject traditional religions and superstitions. They set out to come up with a better, more scientific framework. But then they re-create versions of those old religious superstitions! In the technical world these superstitions are just as confusing and just as damaging as before, and in similar ways.
For such an elitist and extremely well funded group of know-it-alls, one would expect them to have powerful insights into how the brain works. One would be wrong. So let's see just how wrong the LessWrong cult really is.
  1. The brain builds a probabilistic model of the world. Not even wrong.
  2. Everything is physical because we know it is. More wrong.
  3. We can make a conscious machine because we know that consciousness is an emergent property of the brain. Wrong and wronger.
  4. We will gain immortality by uploading our brains to a machine because we know that the brain is all there is. Laughably Wrong.
  5. We must be careful with AI because intelligent machines may decide they no longer need us. Pathetically wrong.
  6. We are less wrong than others because we are smarter. Wrongest.
The only good thing about all this is that singularitarians do not have a clue as to how intelligence really works. Their dream of being the ones to build an AGI is just that, a dream. The world would be in a heap of trouble if those guys found the solution to true AI.

Coming Up

In Part II, I will go over the reasons that the Church of the Singularity is wrong about both the brain and consciousness.

Monday, October 13, 2014

Why I Believe True Artificial Intelligence May Come Within a Year

It's Closer Than You Think

I think that true AI will arrive in the world much sooner than most people expect. I believe it may happen sometime in 2015. I have many reasons but I will mention just a few important ones in this article. I have argued some of these points elsewhere.

Time Is the Only Teacher

There is something truly groundbreaking that a number of people in the AI research community (e.g., Jeff Hawkins, Andrew Ng, and others) have figured out in the last decade or so. They have come to realize that intelligence is entirely based on the relative timing of discrete sensory and motor signals. It turns out that there are only two kinds of temporal relationships: signals can be either concurrent or sequential. This realization simplifies things tremendously because it gives us a way to do unsupervised learning and invariant object recognition just by observing signal timing. Time is the only supervisor in perceptual learning. No labeled examples are necessary. I believe this to be a breakthrough of enormous importance. It goes without saying that the supervised deep learning models that are currently the rage in AI circles will fall by the wayside.

We Don't Need So Many Neurons

Many have argued that we will need super powerful computers in order to emulate the tens of billions of neurons in the human brain. A critic may ask, do we really need that many neurons and such vast computing power to demonstrate true intelligence? I personally don't think so. My research into cortical columns and sequence recognition has convinced me that we will need at least two orders of magnitude fewer neurons to emulate a mammalian cortex than we thought. I have come to the conclusion that the brain is forced to use parallelism in its cortical columns in order to compensate for the slow speed of its neurons. There is good reason to suppose that the hundred or so minicolumns that comprise a macrocolumn are just individual speed recognizers for a given sequence. They can be emulated in a computer with a single minicolumn and a couple of variables.

In this vein, one can also argue that once the basic principles of intelligence are fully understood, there really is no need to emulate all the billions of neurons in a brain in order to demonstrate very powerful intelligent behavior. A million or so neurons combined with the right model will perform wonders. Bees and wasps can do amazing things with a million neurons.

It gets better. The requirement for massive computational resources becomes even less of a problem when you consider that only a fraction of the brain's cortex is awake at any one time. It may come at a surprise to many that over 90% of the cortex is essentially asleep even when we are fully awake. This is because only a very small part of the cortex, the part we are focusing on, is active at one time.

The Bayesian Red Herring

True AI could have happened decades ago if only we knew how it worked. Obviously, there is something about intelligence that still escapes researchers in the field. I am convinced that one of the reasons it did not happen years ago (other than the aberration that was symbolic AI or GOFAI) is that AI researchers have fallen in love with probabilistic approaches to intelligence such as Bayesian statistics. This, too, is a major waste of time in my opinion. I say this because, contrary to conventional wisdom, the brain does not compute probabilities.

The probabilistic AI model assumes that the world is inherently uncertain and that the job of an intelligent system is to compute the probabilities. The correct model, in my view, assumes that the world is perfectly consistent and that the job of the intelligent system is to discover this perfection. The two models are polar opposites. I believe that once researchers realize that the brain uses a non-probabilistic, winner-take-all approach to recognition, AI will be upon us like a tsunami.

"People are not probability thinkers but cause-effect thinkers." These words were spoken by none other than Dr. Judea Pearl during a 2012 Cambridge University Press interview. Pearl, an early champion of the Bayesian approach to AI, apparently had a complete change of heart. In my opinion, this should have been a wake-up call for the AI community but Pearl's words seem to have fallen on deaf ears. This is regrettable because the probabilistic approach to AI is one of the main impediments to progress in this field. Getting rid of it will simplify our task by orders of magnitude. Fortunately, a number of people are fast moving in this direction.

Conclusion

There are other reasons that true AI is closer than most of us think, including a few that I will reveal when I release the Rebel Speech demo (hang in there). Perceptual learning and knowledge representation are at the heart of intelligence. Once we fully solve the problem of perception and memory, everything else will be child's play in comparison, even things like motor learning, motivation and adaptation. The future is almost at the door.

Sunday, August 24, 2014

Alternative Anti-Inflammatory Remedies for ALS

Alternatives

I was thinking about the dramatic effect that dexamethasone had on my wife's ALS symptoms on several occasions and it occurred to me that there must be several non-prescription drugs and supplements that could help tame the neuro-inflammation of ALS. After a quick search, I came up with the following: Pomegranate juice, ginger root or extract, Lunasin (soy peptides), zinc gluconate, turmeric, marijuana, alcohol, vitamin D3, ibuprofen, dextromethorphan and last, but not least, Naproxen (Aleve). Most of these products are easily obtainable in most countries. Dextromethorphan is used in over-the-counter cough syrup and is known to have strong anti-inflammatory and thus neuroprotective properties. Soy peptides can be ordered online.

Naproxen

Naproxen is particularly interesting because it inhibits the prostaglandin E2 hormones and pro-inflammatory cytokines that are known to be elevated in ALS patients. I would be interested in knowing about the experiences of ALS patients out there who might have experimented with a high dose (2000 mg or more per day) of Naproxen for a few days. I suspect it might have a noticeably positive effect on some patients. To anyone who may want to experiment with Naproxen, I would also recommend taking some L-arginine and magnesium during the treatment to help dilate the arteries and capillaries. This should make it easier for the drug to reach difficult areas of the brain and spinal cord. Of course, if you do get improvements from a high dose of naproxen, it goes without saying that something more powerful like dexamethasone could do wonders. I'm a little excited about the potential of Naproxen because it is an easily obtainable drug. If it did cause improvements in ALS symptoms, it would send a powerful message because a lot of people can try it at home without a prescription.

See Also:

Anesthetics and Glucocorticoids for ALS
Naproxen Reduces Excitotoxic Neurodegeneration in Vivo
Healthline: Naproxen

Wednesday, August 13, 2014

The Evil Lie about ALS (Lou Gehrig's Disease) and Anti-Inflammatory Drugs

There is no Question that ALS is an Autoimmune Disease

Those who claim otherwise are either working for Big Pharma or the ALS vulture industry, lying for some personal reason or do not know what they are talking about. ALS is caused by any of several types of genetic mutations. But what characterizes most forms of ALS is that the G-protein coupled receptors are defective. These ubiquitous receptors are used directly by both the nervous system and the immune system. They include GABA, glycine, serotonin, glutamate, prostaglandins and several others. Again, these receptors are used directly by both systems. It has been shown experimentally that inflammation is elevated in the brainstem and spinal cord of ALS patients long before the appearance of symptoms. This stuff has been known for decades. So Big Pharma or the ALS vulture industry are lying through their teeth when they continue to insist that inflammation is a secondary consequence of the disease and not part of the genesis of the pathology. They should know better. Shame on them.

The Evil Lie that Killed My Wife and Countless Others

My wife died because ALS experts lied to me. When I began investigating ALS, I was assured early on that ALS was not an autoimmune disease and that it had already been proven over and over that anti-inflammatory drugs have absolutely no effect on the disease. My mistake was to believe this evil lie. So I ignored the fact that my wife's improvements always occurred soon after being administered an anti-inflammatory drug like dexamethasone. I dismissed dexamethasone and instead focused on the only other drug that could have had an effect on the disease, the anesthetics. Anesthetics are powerful anti-inflammatory drugs in their own right and should be included in any comprehensive treatment because they target certain important receptors that traditional anti-inflammatory drugs don't. But anesthetics are not enough. The biggest contributors to my wife's improvements were the glucocorticoid drugs. But I was a fool and my wife died as a result. If I had not believed in the lie, I would have saved her life. I failed miserably.
Important: There are several ALS variants caused by different mutations. Not every ALS patient will see improvements from dexamethasone or prednisone or any one drug. But I believe that many will. Those who don't see any improvements should not despair. The immune system is vast and complex. It is likely that their particular form of ALS affects a different part of the immune system. Other types of anti-inflammatory drugs or a complex cocktail of drugs may do the trick.

Another reason that some PALS may not respond successfully to anti-inflammatory drugs has to do with drug penetration. Inflammation and other factors may prevent the drugs from reaching areas of the CNS where there are needed the most. Such patients may require direct injections into their brainstem and/or spine.
A Crime against Humanity

To claim that ALS is not an autoimmune disease and therefore cannot be treated with anti-inflammatory drugs is a crime against humanity because it blinds an entire community of terminal patients, their loved ones and their doctors to the availability of cheap, off-the-shelf drugs. These can have a powerful therapeutic effect on ALS, especially during the early stages of the disease before the motor neurons begin to suffer irreversible damage. This truth about the ability of anti-inflammatory drugs to effectively treat ALS is exactly what the ALS vulture industry and Big Pharma do not want you know. They are an evil society controlled by assholes and psychopaths who are only in it for the money. They don't give two shits about the lives and sufferings of millions of people. They are inhuman, traitors to their own species, who love to line their pockets with billions of dollars of the public's money. The Nazis and the Holocaust come to mind. When the dust settles on this despicable crime, there will be hell to pay.

I Hold You Responsible for my Wife's Death

I hold Big Pharma and the ALS vulture industry responsible for my wife's slow agonizing death and that of countless other PALS. I am not afraid of you and I'm coming after you with all the legal, scientific and political power I can muster.

See Also:

Treat ALS with Anti-Inflammatory Drugs
Anesthetics and Glucocorticoids for ALS

Monday, August 11, 2014

Treat ALS with Anti-Inflammatory Drugs

This is the dirty little secret that Big Pharma and the corrupt ALS therapy development industry do not want you to know. Most ALS (amyotrophic lateral sclerosis or Lou Gehrig's disease) patients can be treated with cheap, off the shelf, non-proprietary anti-inflammatory drugs. Here is a partial list of ALS patients who have seen major improvements and reversal of symptoms after being given anti-inflammatory drugs:
  1. Ted harada. Drugs: anesthetics, basiliximab, methylprednisolone, prednisone, tacrolimus, mycophenolate mofetil.
  2. Ernie Schmid. Drugs: various glucocorticoids.
  3. Paul Aiken. Drugs: local anesthetics, Kenalog, dexamethasone.
  4. Louis Savain's wife. Drugs: anesthetics, dexamethasone.
There are more, I am sure. Some patients are reporting improvements from anti-inflammatory supplements such as Lunasin (a peptide) and zinc gluconate. Other people have reported sudden improvements in speech and swallowing ability after a trip to the dentist where they are injected with dexamethasone and lidocaine. Please, feel free to add more items to the list if you know of other examples.
Important: There are several ALS variants caused by different mutations. Not every ALS patient will see improvements from dexamethasone or prednisone or any one drug. But I believe that many will. Those who don't see any improvements should not despair. The immune system is vast and complex. It is likely that their particular form of ALS affects a different part of the immune system. Other types of anti-inflammatory drugs or a complex cocktail of drugs may do the trick.

Another reason that some PALS may not respond successfully to anti-inflammatory drugs has to do with drug penetration. Inflammation and other factors may prevent the drugs from reaching areas of the CNS where there are needed the most. Such patients may require direct injections into their brainstem and/or spine.
See Also:

Anesthetics and Glucocorticoids for ALS

Anesthetics and Glucocorticoids for ALS

Note: I am reposting this article because I am trying to send a message that is dear to my heart. I do it in my wife's memory.

In 2007, my wife, who died of ALS last year, experienced a full recovery of her left foot paralysis immediately after spine surgery. Although she did not know it at the time, the paralysis was caused by ALS. Unfortunately, the symptoms returned within a month and it was downhill thereafter. After requesting her medical records, I learned that during the operation, she was anesthetized with inhalational anesthetics and she was given 2 grams of the antibiotic drug Cefazolin (Ancef) and 10 mg of Decadron (Dexamethasone), a very powerful glucocorticoid anti-inflammatory drug.

Excerpt
Somewhere around 2010, I became convinced that it was the anesthetics (propofol, sevoflurane, etc.) that had caused my wife's remission. The reason is that she also experienced a strong remission of her ALS symptoms immediately after other surgical procedures during which she was anesthetized. But the main reason that I dismissed dexamethasone is that I was assured by ALS experts that anti-inflammatory drugs have been tried many times before and were shown to be completely ineffective against ALS. This turned out to be a lie, the big lie about ALS that Big Pharma has preached for many years. I now understand that it was a combination of the anesthetics and the glucocorticoid anti-inflammatory drug, routinely given during such procedures, that had caused her improvements. Both have anti-inflammatory properties. I did some research and found out that they work synergistically by enhancing and complementing each other's therapeutic properties.

I hold Big Pharma, the Department of Health and Human Services and the ALS therapy development industry responsible for my wife's death and that of countless others who perished from this horrible disease.

ALS Is an Autoimmune Disease

Even though many in the ALS money making industry maintain otherwise, there is no question that ALS is an autoimmune disease. This is why my wife and other ALS sufferers have experienced strong improvements after injection with anti-inflammatory drugs. This is not the first time that glucocorticoid drugs have been implicated in spectacular ALS remissions. In 2011, famous ALS patient Ted Harada experienced an amazing recovery after being anesthetized for up to five hours during stem cell treatment and given several anti-inflammatory glucocorticoids to prevent rejection. In June 2013, Ernie Schmid published an ALS remission story in which he explained how he kept his ALS under control with powerful glucocorticoids. There is also the story of US Authors Guild's director Paul Aiken whose ALS went into remission after injections with the glucocorticoid drug Kenalog. Other ALS sufferers have reported strong improvements in their speech and ability to swallow after a visit to the dentist. It turns out that most dentists inject their patients with a mixture of dexamethasone (to prevent swelling) and a local anesthetic during tooth extractions or root canals.
Important: There are several ALS variants caused by different mutations. Not every ALS patient will see improvements from dexamethasone or prednisone or any one drug. But I believe that many will. Those who don't see any improvements should not despair. The immune system is vast and complex. It is likely that their particular form of ALS affects a different part of the immune system. Other types of anti-inflammatory drugs or a complex cocktail of drugs may do the trick.

Another reason that some PALS may not respond successfully to anti-inflammatory drugs has to do with drug penetration. Inflammation and other factors may prevent the drugs from reaching areas of the CNS where there are needed the most. Such patients may require direct injections into their brainstem and/or spine.
I would hate to see so many ALS sufferers needlessly die from this awful disease while a cheap and effective treatment might be available off the shelf. Unfortunately, we have a broken and heartless health system that refuses to listen to the opinions of their terminal patients and forbids them to experiment with various drugs, even under doctor's supervision. We sorely need a 'right to try' law for terminal patients.

See Also:

Alternative Anti-Inflammatory Remedies for ALS
The Evil Lie about ALS
Treat ALS with Anti-Inflammatory Drugs


PS. If you have ALS or you care for someone who does, please do what you can to get your hands on some dexamethasone and conduct your own experiments at home. It's a relatively benign drug. Just make sure you don't have any infection before you begin to experiment with it.

Discussion

Thursday, July 24, 2014

Anesthetics and Glucocorticoids for ALS

Note: I just received a note from Sharon Halton, a research coordinator at the Houston Methodist Neurological Institute saying "Our office has been conducting a study of dexamethasone for more than a year and we look forward to evaluating the results when the last of our subjects have completed their treatment." Let us hope it is a success. Regardless of the outcome, I maintain that the efficacy of dexamethasone is enhanced by anesthetics and vice versa. This synergistic interaction is why future treatments/experiments should consider using both concurrently. (7/25/14)

In 2007, my wife, who died of ALS last year, experienced a full recovery of her left foot paralysis immediately after spine surgery. Although she did not know it at the time, the paralysis was caused by ALS. Unfortunately, the symptoms returned within a month and it was downhill thereafter. After requesting her medical records, I learned that during the operation, she was anesthetized with inhalational anesthetics and she was given 2 grams of the antibiotic drug Cefazolin (Ancef) and 10 mg of Decadron (Dexamethasone), a very powerful glucocorticoid anti-inflammatory drug.

Excerpt
Somewhere around 2010, I became convinced that it was the anesthetics (propofol, sevoflurane, etc.) that had caused my wife's remission. The reason is that she also experienced a strong remission of her ALS symptoms immediately after other surgical procedures during which she was anesthetized. But the main reason that I dismissed dexamethasone is that I was assured by ALS experts that anti-inflammatory drugs have been tried many times before and were shown to be completely ineffective against ALS. This turned out to be a lie, the big lie about ALS that Big Pharma has preached for many years. I now understand that it was a combination of the anesthetics and the glucocorticoid anti-inflammatory drug, routinely given during such procedures, that had caused her improvements. Both have anti-inflammatory properties. I did some research and found out that they work synergistically by enhancing and complementing each other's therapeutic properties.

I hold Big Pharma, the Department of Health and Human Services and the ALS therapy development industry responsible for my wife's death and that of countless others who perished from this horrible disease.

ALS Is an Autoimmune Disease

Even though many in the ALS money making industry maintain otherwise, there is no question that ALS is an autoimmune disease. This is why my wife and other ALS sufferers have experienced strong improvements after injection with anti-inflammatory drugs. This is not the first time that glucocorticoid drugs have been implicated in spectacular ALS remissions. In 2011, famous ALS patient Ted Harada experienced an amazing recovery after being anesthetized for up to five hours during stem cell treatment and given several anti-inflammatory glucocorticoids to prevent rejection. In June 2013, Ernie Schmid published an ALS remission story in which he explained how he kept his ALS under control with powerful glucocorticoids. There is also the story of US Authors Guild's director Paul Aiken whose ALS went into remission after injections with the glucocorticoid drug Kenalog. Other ALS sufferers have reported strong improvements in their speech and ability to swallow after a visit to the dentist. It turns out that most dentists inject their patients with a mixture of dexamethasone (to prevent swelling) and a local anesthetic during tooth extractions or root canals.

I would hate to see so many ALS sufferers needlessly die from this awful disease while a cheap and effective treatment might be available off the shelf. Unfortunately, we have a broken and heartless health system that refuses to listen to the opinions of their terminal patients and forbids them to experiment with various drugs, even under doctor's supervision. We sorely need a 'right to try' law for terminal patients.

See Also:

Alternative Anti-Inflammatory Remedies for ALS
The Evil Lie about ALS
Treat ALS with Anti-Inflammatory Drugs

PS. If you have ALS or you care for someone who does, please do what you can to get your hands on some dexamethasone and conduct your own experiments at home. It's a relatively benign drug. Just make sure you don't have any infection before you begin to experiment with it.

Discussions

Friday, June 20, 2014

I Am Paranoid About the Future

A Potential Death Sentence for Humanity

The more I think about the consequences of artificial intelligence, the more I tremble with fear and apprehension. No, I'm not worried about some mythological sci-fi scenario in which robots rebel against their owners and wipe out humanity. That's just nonsense coming from the Singularity movement. Those who hold those views are clueless as to the true nature of intelligence. They are lost in a lost world.

I am paranoid because I understand the power of intelligence, artificial or otherwise. Any government, organization or individual who manages to control artificial intelligence will have the power to turn our beautiful planet into hell. The introduction of true AI into this world, as it currently is, with all its wars, corruption, crime and countless other horrors, would be a death sentence for humanity. I have seen the enemy and he is not a machine. He is us.

Hang in there.

Wednesday, May 28, 2014

The Rebel Speech Recognition Project

Progress Update

I am making rapid progress working on the Rebel Speech project and it will not be long before I release a demo. Please have patience. Rebel Speech will be a game changer in more ways than one. There are many things I need to consider as far as when and how to publish the results of my research. I cannot divulge the state of the engine at this time but what I can say is that it will take many by surprise.

My plan, which is subject to change, is to release a program that will demonstrate most of the capabilities of the model. The demo will consist of an executable program and a single data file for the neural network, aka the brain. The latter will be pre-trained to recognize the digits 1 to 20 (or more) in three or four different languages. I will not release the learning module and the source code, at least not for a while. The reason is that I need to monetize this technology to raise enough money to continue my AI research. What follows is a general description of Rebel Speech.

The Rebel Speech Recognition Engine

The Rebel Speech recognition engine is a biologically plausible spiking neural network designed for general audio learning and recognition. The engine uses two hierarchical subnetworks (one for patterns and one for sequences) to convert audio waveform data into discrete classifications that represent phonemes, syllables, words and even whole phrases and sentences. The following is a list of some of the characteristics that distinguish Rebel Speech’s architecture from other speech recognizers and neural networks:
  • It can learn to recognize speech in any language, just by listening from a microphone.
  • It can learn multiple languages concurrently.
  • It can learn to recognize any type of sound, e.g., music, machinery, animal sounds, etc.
  • Learning is fully unsupervised.
  • It is as accurate as humans on trained data. Or better.
  • It is noise and speaker tolerant.
  • It can recognize partial words and sentences.
  • It uses no math other than simple arithmetic.
Even though Rebel Speech has multiple layers of neurons in two hierarchical networks, this is where the similarity with deep learning ends. Unlike deep neural networks, the layers in Rebel Speech are not pre-wired and synaptic connections have no weights. A synapse is either connected or it is not. In fact, when Rebel Speech begins training, both networks are empty. Neurons and synapses are created and added on the fly during learning and only when needed.

Program Design

The engine consists of three software modules as depicted below.


The sensory layer is a collection of audio sensors. It uses a Fast Fourier Transform algorithm and threshold detectors (sensors) to convert audio waveform data into multiple streams of discrete signals (pulses) representing changes in amplitude. These raw signals are fed directly to pattern memory where they are combined into concurrent groups called patterns. Pattern detectors send their signals to sequence memory where they are organized into temporal hierarchies called branches. Each branch is a classification structure that represents a specific sound or sequence of sounds.

Winner-Takes-All

Most speech recognition systems use a Bayesian probabilistic model, such as the hidden Markov model, to determine which phoneme or word is most likely to come next in a given speech segment. A special algorithm is used to compile a large database of such probabilities. During recognition, hypotheses generated for a given sound segment are tested against these precompiled expectations and the one with the highest probability is selected.

In Rebel Speech, by contrast, the probability that the interpretation of a sound is correct is not known in advance. During learning, the engine creates a hierarchical database of as many non-random sequences of patterns as possible. Sequences compete for activation. When certain sound segments are detected, they attempt to activate various pre-learned sequences in memory and the one with the highest hit count is the winner. A winner usually pops up before the speaker has finished speaking. Once a winner is found, all other competing sequences are suppressed. This approach leads to high recognition accuracy even in noisy environments or when parts of the speech are missing.

Stay tuned.

Tuesday, February 25, 2014

Artificial Intelligence and the Bible: Sensory Learning in Smyrna, Part II

Part I, II

Abstract

In Part I, I wrote that, according to my interpretation of the message to the church of Smyrna in the Book of Revelation, the brain uses two types types of sensors: rich and poor. I explained that it takes an onset sensor and an offset sensor to properly represent a single sensory phenomenon or stimulus at a given amplitude. In today's post, I interpret verses 10 and 11 of the message to Smyrna which describe how sensory learning works. But first, a word about the importance of sensory timing.

The Timing of Sensory Signals in the Brain

Why is it so important that there be two complementary sensors for a stimulus? The reason is that perception is primarily concerned with the evolution of events, i.e., with how things change relative to one another. Phenomena come and go at precise times.

Certain changes happen concurrently and these are called patterns. Patterns succeed each other to form precisely timed sequences. For example, sensors A and B in the diagram above will sometimes fire concurrently with other sensors. Knowing when this happens is valuable information. Sensory learning consists of capturing the temporal correlations in the sensory space and this allows the brain to gain an understanding of how changes occur in the environment. With a good temporal model of the world, an intelligent system can form predictions, plan future actions, adapt to changes and achieve various goals. This is what intelligence is about.

Message to the Church of Smyrna
Revelation 2:8-11
8 “And to the angel of the church in Smyrna write, ‘These things says the First and the Last, who was dead, and came to life:
9 “I know your works, tribulation, and poverty (but you are rich); and I know the blasphemy of those who say they are Jews and are not, but are a synagogue of Satan.
10 Do not fear any of those things which you are about to suffer. Indeed, the devil is about to throw some of you into prison, that you may be tested, and you will have tribulation ten days. Be faithful until death, and I will give you the crown of life.
11 “He who has an ear, let him hear what the Spirit says to the churches. He who overcomes shall not be hurt by the second death.”’
Commentary (continued)

10 Do not fear any of those things which you are about to suffer. Indeed, the devil is about to throw some of you into prison, that you may be tested, and you will have tribulation ten days. Be faithful until death, and I will give you the crown of life.

Sensory learning is a relentless and uncompromising trial. The phrase "some of you" means that new connections for patterns are chosen randomly and then subjected to a brutal trial period during which they are tested 10 times. As I will explain in a future article, a day symbolizes a single neuronal or firing cycle, which is the duration of a single pulse (about 10 milliseconds in the brain). In other words, there are 10 tests and each one lasts a single firing cycle. From this, we can logically deduce that every connection is tested for concurrency with other connections. Why? Because concurrency is the only thing that can be tested during the time of a single pulse.

One of the important things to note here is that connections either survive or they don't. The ones that fail are put to death, that is, they are disconnected. The "crown of life" means that disconnected synapses are reborn and tried again elsewhere. The "prison" metaphor symbolizes the fact that connections are not allowed to "earn a living" during the time of their trial. In other words, the connections cannot contribute to their churches (or patterns) until they pass all the 10 tests and are released from prison.

It goes without saying that the Biblical model sharply contradicts current neural network models that use fixed, pre-wired connections. Also, the Biblical model strongly suggests that synaptic learning is an either-or process: either a connection is made or it isn't. There are no in-betweens, i.e., there is no need for a range of connection weights to encode knowledge. This is why I maintain that deep learning will go the way of symbolic AI and that the high-tech industry is building a billion dollar AI castle in the air.

Finally, we must ask, why 10 test cycles? Why not 5 or 20? To answer this question, we must understand what exactly is being learned. The brain is looking for all possible patterns that occur often enough to be considered non-random. It does not care about their actual probabilities of occurrence because it uses a winner-takes-all mechanism whereby patterns and sequences in memory compete for activation: the ones with the greatest number of hits are the winners. A compromise must be reached between conducting too many tests, which would retard learning and miss low probability patterns, and not conducting enough tests, which would result in learning useless patterns. We can surmise that 10 is just an optimum number. On a side note, this would be a fairly easy hypothesis to falsify. The finding that sensory learning in the brain is based on a mechanism that counts to 10 would go a long way to corroborate this theory.
Note: I am still working on the Rebel Speech demo program and I hope to release it soon. I will also release the source code for the recognizer but not the learner. Rebel Speech incorporates all the principles I have described in this series on AI and the Bible.
11 “He who has an ear, let him hear what the Spirit says to the churches. He who overcomes shall not be hurt by the second death.”’

That first sentence in verse 11 is repeated in every message to the seven churches. It is a sign that the messages do not mean what they appear to mean on the surface. What is the meaning of the "second death" metaphor? I am not 100% sure at this point but it seems to mean that, once a connection is established through testing, it becomes permanent. In other words, unlike sequences which can be forgotten, patterns are retained forever. Note that I am still investigating this metaphor because it is mentioned elsewhere in the book of Revelation.

See Also:

The Billion Dollar AI Castle in the Air
Secrets of the Holy Grail
Artificial Intelligence and the Bible: Message to Sardis
Artificial Intelligence and the Bible: Joshua the High Priest
Artificial Intelligence and the Bible: The Golden Lampstand and the Two Olive Trees

Saturday, February 22, 2014

Artificial Intelligence and the Bible: Sensory Learning in Smyrna, Part I

Part I, II

Abstract

Previously in this series, I wrote that I get my understanding of intelligence and the brain (see Secrets of the Holy Grail) from ancient Biblical metaphorical texts that are thousands of years old. (Yeah, yeah, yeah, I know I am a crank and a lunatic; what else is new?) The message to the church of Smyrna in the book of Revelation is particularly interesting because it describes sensory learning, the most important aspect of perception. In this article, I interpret the metaphors in the message and I argue that experts in deep learning neural networks (the current rage in artificial intelligence research) are lost in the wilderness because they got sensory processing all wrong.

A Note About Sensory Signals in the Brain

The brain uses two types of sensors and each type serves a completely different purpose. The book of Revelation uses two metaphors to describe them: the poor and the rich. "Poor" sensors are used by the sensory cortex for unsupervised perceptual learning and pattern recognition whereas "rich" sensors are used by the cerebellum for fully supervised sensorimotor learning. I will get back to the cerebellum in a future article.

A poor sensor fires either at the onset or offset of a phenomenon or stimulus. By contrast, a rich sensor fires repeatedly during the entire duration of the phenomenon. This is illustrated in the diagram below. The curved line represents the varying intensity of a sensed phenomenon, say, the changing amplitude of an audio frequency signal over time. The brain uses multiple discrete sensors to detect different signal amplitudes. For simplicity's sake, the diagram is concerned only with the detection of a single amplitude shown as a horizontal line.
It takes two poor sensors (A and B) to sense a phenomenon at a given amplitude, one to detect the onset and another to detect the offset of the phenomenon. By contrast, a single rich sensor associated with the same phenomenon at the same amplitude fires repeatedly while the phenomenon lasts. The short vertical lines in the diagram represent the firing pulses of a rich sensor. The two red vertical lines at the beginning and end of the series are the pulses emitted by the onset and offset sensors. As seen below, the message to Smyrna is concerned only with poor sensors, i.e., with the first and the last pulses.
Note: It goes without saying that the sensory cortex responds only to changes in the environment. If you are an AI expert and your machine learning program does not use onset and offset sensors as described above, you are doing it wrong. This is especially important in visual or auditory processing. Visual processing requires frequent jerky motions (microsaccades) of the eye in order to effect changes that the sensors in the retina can respond to. Those of you who are convinced that deep learning and convolutional neural networks are God's gifts to humanity, have a surprise coming.
Message to the Church of Smyrna
Revelation 2:8-11
8 “And to the angel of the church in Smyrna write, ‘These things says the First and the Last, who was dead, and came to life:
9 “I know your works, tribulation, and poverty (but you are rich); and I know the blasphemy of those who say they are Jews and are not, but are a synagogue of Satan.
10 Do not fear any of those things which you are about to suffer. Indeed, the devil is about to throw some of you into prison, that you may be tested, and you will have tribulation ten days. Be faithful until death, and I will give you the crown of life.
11 “He who has an ear, let him hear what the Spirit says to the churches. He who overcomes shall not be hurt by the second death.”’
Commentary

The message to Smyrna is the shortest of all the messages to the seven churches of Asia in the book of Revelation, but don't let that fool you. It manages to pack an amazing amount of crucial information about sensory signals and sensory learning in just a few short sentences.

8 “And to the angel of the church in Smyrna write, ‘These things says the First and the Last, who was dead, and came to life:

"The First and the Last", of course, symbolizes the onset and offset sensory pulses explained above. As we shall see in the interpretation of verse 10, the phrase "who was dead, and came to life" alludes to the fact that, during pattern learning, sensory connections almost always die (are disconnected) and then resurrected (are reconnected somewhere else).

9 “I know your works, tribulation, and poverty (but you are rich);

The church of Smyrna goes through tribulation. This symbolizes that every sensory connection must go through a testing period. Even though the church is poor, it becomes rich through hard work and by overcoming tribulation.

9 [...] and I know the blasphemy of those who say they are Jews and are not, but are a synagogue of Satan.

This is both humorous and powerful. As I will explain in a future article, the false Jews, or the "synagogue of Satan", represent the church of Laodicea, which I interpret to symbolize the cerebellum, a supervised sensorimotor mechanism used for routine or automated tasks. The cerebellum receives sensory signals only from rich sensors.

Coming Up

In Part II, I will interpret verses 10 and 11 of the message to Smyrna, which describe the heart of sensory and pattern learning.

See Also:

The Billion Dollar AI Castle in the Air
Secrets of the Holy Grail
Artificial Intelligence and the Bible: Message to Sardis
Artificial Intelligence and the Bible: Joshua the High Priest
Artificial Intelligence and the Bible: The Golden Lampstand and the Two Olive Trees

Saturday, February 15, 2014

The Billion Dollar AI Castle in the Air

Abstract

High tech companies (e.g., Microsoft, Google, FaceBook, Netflix, Intel, Baidu, Amazon, etc.) are pouring billions of dollars into a branch of artificial intelligence called machine learning. The two main areas of interest are deep learning and the Bayesian brain. The goal of the industry is to use these technologies to emulate the capabilities of the human brain. Below, I argue that, in spite of their initial successes, current approaches to machine learning will fail primarily because this is not the way the brain works.

This Is Not the Way the Brain Works

Some in the business have argued that the goal of machine learning is not to copy biological brains but to achieve useful intelligence by whatever means. To this I say, phooey. Symbolic AI, or GOFAI, failed precisely because it ignored neuroscience and psychology. The irony is that the most impressive results in machine learning occurred when researchers began to design artificial neural networks (ANNs) that were somewhat inspired by the architecture of the brain. Deep learning neural networks, especially convolutional neural nets, are attempts at copying the brain's cortical architecture and the early results are impressive, relatively speaking. But this is unfortunate because researchers are now under the false impression that they have struck the mother lode, so to speak. Below, I list some of the reasons why, in my opinion, they are not even close.
  • Deep learning nets encode knowledge by adjusting connection strengths. There is no evidence that this is the way the brain does it.
  • Deep learning nets use a fixed pre-wired architecture. The evidence is that the cortex starts out with a huge number of connections, the majority of which disappear as the brain learns.
  • Convolutional neural nets are hard wired for translational invariance. The evidence is that the brain uses a single mechanism for universal invariance.
  • Unlike the visual cortex, convolutional neural nets do not depend on saccades or microsaccades. This tells us that the brain uses a different method to process visual signals.
  • Deep learning nets use a single hierarchy for pattern learning and recognition. The evidence is that the brain's perceptual system uses two hierarchies, one for patterns and one for sequences of patterns.
  • The Bayesian brain hypothesis assumes that the brain uses probabilities for prediction and reasoning. The evidence is that the brain is not a probability thinker but a cause-effect thinker.
  • Proponents of the Bayesian brain assume that events in the world are inherently uncertain and that the job of an intelligent system is to compute the probabilities. The evidence is that events in the world are perfectly consistent and that the job of an intelligent system is to discover this perfection.
A Castle in the Air

It feels like I am preaching in the wilderness but someone has to do it. Of course, wherever there is a lot of money exchanging hands, self preservation and politics are sure to be lurking right under the surface. My arguments will be dismissed by those who stand to profit from it all and I will be painted as a crackpot and a lunatic (I don't deny that I'm insane) but I don't really care. My message is simple. There is no doubt that the industry is building an expensive castle in the air. Sure, they will have a few so-so successes here and there that will be heralded as proof that they know what they are doing. Google's much ballyhooed cat recognizing neural network comes to mind. But sooner or later, out of nowhere, and when they least expect it, someone else will come out with the real McCoy and the castle will come crashing down. The writing is on the wall.

See Also:

The Myth of the Bayesian Brain
The Second Great AI Red Herring Chase
Why Deep Learning Will Go the Way of Symbolic AI
Why Convolutional Neural Networks Miss the Mark
Secrets of the Holy Grail


Wednesday, February 12, 2014

Why Convolutional Neural Networks Miss the Mark

Abstract

Convolutional neural networks (CNNs) are a type of deep learning neural networks that have been successfully applied to visual recognition. They owe their success to being faster to train (probably because of their sparse connectivity) and to being invariant to certain spatial transformations such as translations. In this article, I argue that CNNs miss the mark because they have a rather limited form of invariance, whereas the brain is universally invariant.

Universal Versus Translation Invariance

If you hold your hand in front of your face and rotate it, move it side to side, up and down, shine a blue or red light on it, make a fist, a thumb up or peace sign, etc., at no point during the transformations will there be any doubt in your mind that you are looking at your hand. This is in spite of the fact that, during the transformations, your visual cortex is presented with literally hundreds of very different images. This is an example of universal invariance, something that the brain accomplishes with ease. CNNs can handle only a subset of these transformations because, as seen in the diagram below, they are hardwired for translation invariance.

With some modifications, it should even be possible to get a CNN to tolerate rotations. But CNNs suffer from an even bigger problem. They may be invariant to translations but they have no way of telling whether all the successive images represent the same hand. They can only recognize each image as a hand and that's about it. This lack of continuity makes them ill suited to future robot intelligence.
Source: deeplearning.net
CNNs are invariant to translations thanks to a technique known as spatial pooling. Essentially, neighboring units in a given layer are pooled together to activate a unit in the layer immediately above. The pooling method can use either addition, averaging or maximum. The end result is that the activation of a top layer unit is invariant to the position of a stimulus at the bottom layer.

Biological Implausibility

It is highly unlikely that the visual cortex uses pre-wired spatial pooling to obtain translation invariance. Why? First off, if the brain used a different type of invariant architecture for every type of transformation, the cortex would be a wiring mess. Second, one would expect the auditory cortex to have a different architecture for invariance than the visual cortex but this is not observed. The global uniformity of the cortex is one of its most striking features. A ferret whose optic nerves were rerouted to its auditory cortex in the embryonic stage, was able to use its auditory cortex to learn to see and navigate fairly normally.

How Does the Brain Do It?

It should be fairly obvious that the brain uses a single method to achieve universal invariance. The most likely hypothesis is that the brain has two memory hierarchies, one for concurrent patterns and one for sequences of patterns. Learning in the brain is 100% unsupervised. The sequence hierarchy is a powerful memory structure that serves multiple functions. It is a common storage mechanism for attention, prediction, planning, adaptation, short and long-term memory, analogies, and last but not least, temporal pooling. Every invariant object is represented by a single branch in the hierarchy. I hypothesize that temporal pooling is the way the cortex achieves universal invariance. To emulate the brain's universal invariance, one must first design a good pattern learner/recognizer that feeds its output signals to a sequence learning mechanism. The latter must be able to automatically stitch patterns and related sequences together to form invariant object representations. I will have more to say about pattern and sequence learning in future articles.

See Also

Why Deep Learning Is a Hindrance to Progress Toward True AI
The Billion Dollar AI Castle in the Air
Why Deep Learning Will Go the Way of Symbolic AI

Sunday, February 9, 2014

Why Deep Learning Will Go the Way of Symbolic AI

Abstract

Deep learning is a machine learning and pattern representation and recognition technique based on multi-layered, statistical neural networks. Deep learning is all the rage lately. Big corporations like Google, Facebook and others are spending billions to set up labs and acquire experts and companies with experience in the technology. In this article, I argue that the current approach to deep learning will not lead to human-like intelligence because this is not the way the brain does it.
Related:
Why Deep Learning Is a Hindrance to Progress Toward True AI
Why Convolutional Neural Networks Miss the Mark
The Billion Dollar AI Castle in the Air

Hierarchical Representation

There is no question that the brain classifies knowledge using a hierarchical architecture. The representation of objects in memory is compositional. That is to say, higher level representations are built on top of lower level ones. For examples, low level visual representations might consist of edges and lines. These can be combined to form higher level objects such as a nose or an eye. So the one thing deep learning neural networks have going for them is that they use multiple layers to form a hierarchical structure of representations.

Weighted Connections

A deep learning network consists of multiple layers of neurons. Each layer is a restricted Boltzmann machine or RBM.
Restricted Boltzmann Machine
The visible units of an RBM receive data from input sensors and the hidden units are the outputs of the machine. In a deep learning network, the hidden units are used as the visible units for the RBM residing immediately above it in the hierarchy. Each neuron (or hidden unit) in an RBM has a number of inputs represented by connections. Each connection is weighted, that is, it has a strength that is tuned by a learning algorithm during training on a set on examples. Loosely speaking, a connection strength represents the belief (or degree of certainty) that a particular input activation contributes to the activation of a hidden unit. A hidden unit is activated by approximating a nonlinear function of its inputs.

Biologically Implausible

There are a number of problems with deep learning networks that make them unsuitable to the goal of emulating the brain. I list them below.
  1. A deep learning network encodes knowledge by adjusting the strengths of the connections between visible and hidden units. There is no evidence that the brain uses variable synaptic strengths to encode degrees of certainty during sensory learning.
  2. Every visible unit is connected to every hidden unit in an RBM. There is no evidence that sensors make connections with every downstream neuron in the brain's cortex. In fact, as the brain learns, the number of connections (synapses) between sensors and the cortex is drastically reduced. The same is true for intracortical connections.
  3. Deep learning networks must be fine-tuned using supervised learning or backpropagation. There is no evidence that sensory learning in the brain is supervised.
  4. Deep learning networks are ill-suited for invariant pattern recognition, something that the brain does with ease.
  5. Deep learning networks use highly complex learning algorithms based on complex mathematical functions that require fast processors. There is no evidence that cortical neurons solve complex functions.
  6. Deep learning networks use static examples whereas the brain is bombarded with a constantly changing stream of sensory signals. Timing is essential to learning in the brain.
Winner Takes All

Current approaches to deep learning assume that the brain learns visual representations by computing input statistics. As a result, one would expect a gradation in the way patterns are recognized, especially in ambiguous images. However, psychological experiments with optical illusions suggest otherwise.
When looking at the picture above, two things can happen. Either you see a cow or you don't. There is no in-between. Some people never see the cow. Furthermore, if you do see the cow, the recognition seems to happen instantly.

It seems much more likely that the cortex uses a winner-takes-all strategy whereby all possible patterns and sequences are learned regardless of probability. The only criterion is that they must occur often enough to be considered above mere random noise. During recognition, the patterns and sequences compete for activation and the ones with the highest number of hits are the winners. This kind of pattern learning is simple (no math is needed), fast and requires no supervision.

See Secrets of the Holy Grail, Part II for more on this alternative approach to pattern learning.

Conclusion

In view of the above, I conclude that, in spite of its initial success, deep learning is just a red herring on the road to true AI. It is not true that the brain maintains internal probabilistic models of the world. After all is said and done, deep learning will be just a footnote in the annals of AI history. The same can be said about the Bayesian brain hypothesis, by the way.

See Also

Why Deep Learning Is a Hindrance to Progress toward True AI
Mainstream AI Is Still Stuck in a Rut
The Myth of the Bayesian Brain
Why Convolutional Neural Networks Miss the Mark

Sunday, February 2, 2014

Artificial Intelligence and the Bible: A Note about Sequences

The Branch

In my recent series on artificial intelligence and the Bible (see links below), I made a big deal about sequence learning in the brain and the Biblical metaphors that are associated with sequences such as Joshua the high priest, the vine, the stone with seven eyes and the so-called branch or sprout. I am afraid there is something about the branch metaphor that I declined to explain because I think it would reveal too much, too soon. Those of you who have an interest in this research should keep in mind that every single metaphor in the ancient texts was chosen very carefully. My initial mistake was to assume that it was going to be a breeze to interpret them. I was wrong. I wasted a lot of time by not thoroughly analyzing every symbol. In fact, the book of Zechariah has a warning about this at the end of the vision:
Zechariah 6:15
And this shall come to pass, if ye will diligently hearken to the voice of Jehovah your God.
Another translation is "And this will happen if you carefully listen to the voice of Yahweh your Lord." I will have more to say about the branch metaphor in the future because it is crucial to understanding the organization of sequence memory. I just don't think this is the right time. Not yet.

See Also:

Secrets of the Holy Grail
Artificial Intelligence and the Bible: Message to Sardis
Artificial Intelligence and the Bible: Joshua the High Priest
Artificial Intelligence and the Bible: The Golden Lampstand and the Two Olive Trees

Tuesday, January 28, 2014

Artificial Intelligence and the Bible: The Golden Lampstand and the Two Olive Trees, Part II

Part I, II

Abstract

This is a continuation of my series on artificial intelligence and the Bible. I claim that I get my understanding of intelligence and the brain (see Secrets of the Holy Grail) by interpreting certain metaphorical passages in the books of Revelation and Zechariah. In Part I, I interpreted verses 1 to 7 of the fourth chapter of the book of Zechariah. We learned that memory is organized hierarchically and consists of seven-item chunks. Each chunk, or building block, is either a sequence of patterns or a sequence of sequences. Below, I interpret the rest of chapter 4 to reveal the mechanisms of short-term memory and attention.

The Golden Lampstand and the Two Olive Trees

Previously I wrote that the two olive trees represent pattern and sequence memory hierarchies within a single hemisphere of the brain. I argued that a metaphor always pertains to a single hemisphere unless it is specifically associated with "the whole earth", meaning the whole brain. I should have added that the fact that the two olive trees are said to be on both sides of a single lampstand and its oil bowl, is also a good indication that we are dealing with a single brain hemisphere. To summarize, the entire brain's memory can be represented by two lampstands and four olive trees in total. I'll have more to say about this in the commentary below.
Zechariah 4: 1-14
1 Now the angel who talked with me came back and wakened me, as a man who is wakened out of his sleep.
2 And he said to me, “What do you see?” So I said, “I am looking, and there is a lampstand of solid gold with a bowl on top of it, and on the stand seven lamps with seven pipes to the seven lamps.
3 Two olive trees are by it, one at the right of the bowl and the other at its left.”
4 So I answered and spoke to the angel who talked with me, saying, “What are these, my lord?”
5 Then the angel who talked with me answered and said to me, “Do you not know what these are?” And I said, “No, my lord.”
6 So he answered and said to me: “This is the word of the Lord to Zerubbabel: ‘Not by might nor by power, but by My Spirit,’says the Lord of hosts.
7 ‘Who are you, O great mountain? Before Zerubbabel you shall become a plain! And he shall bring forth the capstone with shouts of “Grace, grace to it!”’”
8 Moreover the word of the Lord came to me, saying:
9 “The hands of Zerubbabel have laid the foundation of this temple; his hands shall also finish it. Then you will know that the Lord of hosts has sent Me to you.
10 For who has despised the day of small things? For these seven rejoice to see the plumb line in the hand of Zerubbabel. They are the eyes of the Lord, which scan to and fro throughout the whole earth.”
11 Then I answered and said to him, “What are these two olive trees—at the right of the lampstand and at its left?”
12 And I further answered and said to him, “What are these two olive branches that drip into the receptacles of the two gold pipes from which the golden oil drains?”
13 Then he answered me and said, “Do you not know what these are?” And I said, “No, my lord.”
14 So he said, “These are the two anointed ones, who stand beside the Lord of the whole earth.”
Commentary
8 Moreover the word of the Lord came to me, saying:
9 “The hands of Zerubbabel have laid the foundation of this temple; his hands shall also finish it. Then you will know that the Lord of hosts has sent Me to you.
These two verses seem to be saying that the same building principles are used throughout the structure from the bottom level up. I italicized the last sentence in verse 9 because it only makes sense if we assume it is coming from Zechariah and not from the "the word of the Lord." I think translators should at least use parentheses around that sentence to make the point.
10 For who has despised the day of small things? For these seven rejoice to see the plumb line in the hand of Zerubbabel. They are the eyes of the Lord, which scan to and fro throughout the whole earth.”
This is awkward. After going through the interlinear Hebrew text and other translations, I think a better translation would be as follows:
10 For who has despised the day of small things? For they shall rejoice to see the plumb line in the hand of Zerubbabel. These seven are the eyes of the Lord, which scan to and fro throughout the whole earth.”
We have already seen in chapter 3 how the preposition 'for' was used repeatedly to link one idea to another. This is what is happening here. The first 'For' in verse 10 is linking to something that was said in verse 9:
9 “The hands of Zerubbabel have laid the foundation of this temple; his hands shall also finish it.
So what this is saying is that memory construction starts with 'small things'. But what does 'small things' mean? Well, we know from chapter 3 that every building block is a unique sequence with seven nodes. The only thing about a sequence that can be either small or big is the interval between two nodes. I think what this verse is telling us is that memory building (i.e., sequence learning) should start with small intervals. This is extremely important to sequence learning and I will get back to it later.
For they shall rejoice to see the plumb line in the hand of Zerubbabel.
To be continued...
(Ok. I'm having cold feet again. I apologize. The subject matter is getting into the heart of perceptual learning and memory organization. I need to think carefully before I reveal what I have discovered. I get paranoid about AI because there is no doubt in my mind that it will be an explosively and dangerously disruptive technology. It will be a historic Big Bang heard and felt around the world. I need a little time to think this through.)
See Also:

Secrets of the Holy Grail
Artificial Intelligence and the Bible: Message to Sardis
Artificial Intelligence and the Bible: Joshua the High Priest