Sunday, August 10, 2008

Parallel Computing: Timing, Reliability, Safety and Security

A Computer Is Not What It Does

The COSA computing model is closely related to my ongoing research in artificial intelligence and my rejection of the GOFAI symbol manipulation approach to AI. I understood early on that it is symbol manipulation that requires intelligence and not the other way around. In other words, symbol manipulation is something that intelligence does, not what it is. Likewise, I realized that function calculation is what a computer does and not what it is. Calculating algorithmic functions is just one of the many possible behaviors that a computer can exhibit but it is not what computing is based on. This is analogous to the way human intelligence operates. We can do all sorts of things such as speaking, running, walking, and grasping but our intelligence is not based on any of those things. Some other principle is necessary.

Nature Is Parallel and Reactive

It also occurred to me that serial processing is not an inherent property of real-world objects but an emergent phenomenon that results from the way objects react to one another. A predecessor/successor sequence should thus be seen, not as an ordered list of operations to be processed by a central processor (the Turing model), but as a causal or reactive mechanism analogous to stimulus/response coupling in psychology or to sensor/effector pairing in neuroscience. I reasoned that computers and intelligent systems are types of behaving machines and they can exhibit an indefinite number of behaviors. For more on this subject, read The Hidden Nature of Computing.

Timing Is of the Essence

It did not take me long to figure out that precise timing is essential to the reliable operation of a parallel system. It turns out that deterministic timing can be easily implemented in software by ensuring that all elementary reactions have the same duration, based on a virtual system-wide clock. This is the reason that, as I explained elsewhere on this blog, I initially promoted COSA as a solution to the software reliability crisis. I still do, although I have recently changed my approach by emphasizing the inherent parallel nature of COSA programs.

Task Scheduling vs. Security

I am so convinced of the crucial importance of deterministic timing that I insist that every program in a COSA system should be synchronized to a single virtual clock. I refuse to consider any suggestion that applications or component could have their own local clocks ticking at different frequencies so as to manage scheduling. I believe it would be disastrous to system-wide reliability and security. Which brings me to a message that Tom Lieber emailed to me recently:

Consider a program which is (maliciously?) designed with a large number of components continuously slinging signals around so that the number of signals to be processed for its execution is gigantic compared to the number of signals in the other components. In a process-based model, every process can still be given its fair share of the processor time via scheduling, but in COSA it seems that there is no recourse, if the only mechanism for scheduling priority is the message queue.
In my opinion, this is precisely why traditional task (or process) scheduling is bad. It allows malicious programs to infect a system by giving them a portion of the processor’s time. Traditional multitasking operating systems provide no easy way to detect intruders because the overall processor load (number of requested instructions for a given concurrent cycle) is unknown. By contrast, a denial of service attack, such as the one described by Tom, would never work in a COSA system because the system keeps track of the load being processed during every cycle. As soon as the load grows above a pre-configured safe limit, the system can either alert an administrator or automatically shut down the offending application. In fact, every COSA application can monitor its own performance (measured in number of application cycles per second) and trigger an alarm if it drops below a preset unsafe level.

Event Completion in Mission-Critical Applications

In many industries (e.g., aviation, power generation, transportation, health care, defense, etc.), software failure is not an option. COSA will go a long way toward eliminating the sort of accidental failures that are the bane of traditional systems by automatically eliminating blind code or hidden data dependencies. There is another way in which COSA can increase the robustness of a system even in the event of a hardware failure. Safety-critical systems, such as avionics, use a plurality of sensors to guide automated decision-making. Unlike software, hardware sensors can suffer from material fatigue. Every so often, they fail. Failure can also occur due to unforeseen circumstances in the environment. For example, a light sensor may become temporarily affected by smoke, dirt or some other obstruction.

During the testing phase of a COSA program, the system can automatically discover temporal correlations. At the discretion of the programmer, temporal watchdogs can be inserted in the program that can either trigger an alarm if an event (discrete signal) fails to occur at its expected time and/or automatically generate a replacement event in order to prevent a total breakdown. In artificial intelligence, this is called pattern completion. Human and animals could not survive without it. This anticipatory capability is what allows us to follow a conversation in a noisy room or recognize a partially occluded object. I believe it can go a long way toward increasing the robustness of complex systems even in the presence of hardware failures and sensory uncertainty.


The nice thing about the COSA model is that it addresses not just the parallel programming problem but the software reliability, safety and security issues as well. These are the most important issues facing the computer industry at this time. The captains of the industry must either embrace the COSA model or suffer increased financial and societal hardships. There are no two ways about it, in my opinion. COSA is the solution and it is not rocket science. More to come.

Related Article:

How to Solve the Parallel Programming Crisis

No comments: