Friday, August 31, 2007

Tilera vs. Godzilla

Tilera Corporation’s hot new Tile64 multicore processor is a beautiful piece of engineering. It sports integrated memory and I/O controllers and Tilera’s revolutionary Imesh(TM) on-chip network that provides for super fast inter-core communication. Each core is a processor that can support a full operating system if so desired. One could conceivably have multiple instances of Windows running on some of the cores and Linux on the others. Tilera suggests having a symmetric multiprocessing version of Linux running on all the cores and connecting them all into a powerful single-chip cluster.

It’s all very impressive on the surface but, lurking right below, is a formidable monster threatening to stomp on Tilera’s pretty little baby and mercilessly crush it out of existence. It’s an evil, relentless and heartless beast that does not take no for an answer. Here are some excerpts from a recent article in EETimes:


Tilera's chief technologist, Anant Agarwal, a longtime multicore researcher at MIT, admits his company's tools are no panacea for the formidable computer science problems that underlie programming massively parallel architectures. "There is no silver bullet in multicore programming," he said.
Success for these massively parallel processors "comes down to how effective the software tools are," said Linley Gwennap, principal of consulting firm The Linley Group (Mountain View, Calif.).
"You can slap dozens or even hundreds of cores on a die. Interconnecting them is a bit difficult, but the real problem is developing a way software can use all these processors," said Gwennap. "It's hard to find any automated software that uses more than a handful of processors."

So what’s a Tilera to do? Agarwal has no answer. He’s a hardware man more than anything. He knows his Imesh and his interconnects down to the core but parallel programming is not really his forte. But who can blame the professor? Nobody else seems to know the answer either. He’s probably hoping that the monster will just fade away so he can go back to tweaking his mesh and adjusting his tiles. Somehow, I doubt it. This multicore-chomping, Imesh-hating Godzilla is out to flatten the entire parallel processing landscape into oblivion, gobbling up hundreds of millions of dollars of VC money in the process. Ahaha... And he already has a taste for silicon. Another multicore startup, Beaverton, Oregon’s Ambric, seemed to have the right idea:

"We think you need to define the right software programming model first and then develop the circuits that support it--and that's just what we did," said Mike Butts, an Ambric fellow and vice president of architecture who presented the Hot Chips paper last year.

Yes indeed! I have always maintained that the right software model must come first. You don’t even think about designing a new multicore architecture unless you have a software model in place. That's just asking for trouble. Hooray for Butts! Finally, we have a hero to the rescue. His name is Butts, he found a silver bullet and he's about to kick some ass and take some names.

Unfortunately, “a year later, the company has yet to announce any users”. Uh Oh! What happened? Well, apparently, Ambric’s choice of a programming model (a mixture of sequential Java and a few custom libraries) is no match for the Godzilla meister. Their Kestrel chips are like so many potato chips that disintegrate under the intense core-melting heat of his fierce breath. He's mad as hell and the VC gents can only run for cover. "Ahahaha…" He just laughs at the Javas, Erlangs, Haskells, concurrent Cs, Sisals and other similarly flimsy projectiles that are thrown at him by throngs of hapless engineers and confounded PhDs. "The fools!", he mumbles to himself, "the pitiless fools! When will they learn?" And a-chomping he goes.

Next: COSA, Erlang, the Beast and the Hardware Makers

Thursday, August 30, 2007

COSA, Erlang, Computer Geeks and the Bible

What I love about most computer geeks is that they are like jihadists. They are on a crusade to convert the world to their own point of view; they have their own chicken shit religions (e.g., trekkie, Jedi, etc…); they believe in all sorts of religious nonsense (e.g., time travel, wormholes, black holes, parallel universes, materialism, big bang, immortality via brain upload, conscious robots, etc…); they have an unmitigated hatred of other religions, especially Christianity, which is understandable, most of Christianity being the crap that it is (God, I love burning bridges, ahaha…); they have no social skills; they almost never get laid, they have a touch of autism; and last but not least, they are fervent worshippers of one particular programming language or another, not to mention computer brands and operating systems. They are an interesting bunch, to say the least.

The best way to piss off a computer geek is to tell them that their favorite language (e.g., Erlang), or their favorite OS (e.g., Linux), or their favorite movie (Star Trek) is crap. It’s like throwing holy water on a vampire. Now (and this is the fun part), if you tell a computer geek that his/her entire way of building and programming computers is crap, then all hell breaks loose. This is a no-no because the computer geek fancies himself or herself to be among the smartest people on earth, you see. Telling them that they don’t know what computing is really about (which is true) is like telling them that they are stupid (which they are).

So what do computer geeks do when their favorite idols are trampled on and dragged in horse manure? Well, what do all religious fanatics do when they feel insulted? They launch a jihad, a crusade. That’s what they do. They jump up and down and foam at the mouth. They declare fatwas. Look, he believes in the Bible! The infidel must die! ahahaha…

All right. That’s enough fun for the day. Let me end this humorous tirade by pointing out that one of the goals of Project COSA is to make computer geeks obsolete (LOL). Application development will be so easy that even stupid creationists will find it a breeze (ROTFL). Ok, that’s it. I can’t take it any more. Back to bashing Erlang. That's more fun.

PS. I added some more stuff to the Parallel QuickSort page.

Wednesday, August 29, 2007

Synchronous Processes: COSA vs. Erlang


The Big Switch

Both COSA and Erlang use asynchronous signaling between concurrent modules. That is to say, a message/signal sender does not have to wait for a reply from the receiver. This is good for many reasons, not the least of which being failure localization. In this article, I would like to point out a major difference between Erlang and COSA processes. I fervently hope that this will persuade the computer industry to switch from the hopelessly flawed algorithmic model of computing to a reactive and synchronous model, a model fit for the 21st century. This is absolutely crucial to the future of computing, in my opinion. It is true that getting the entire computer industry to admit that they have been doing it wrong right from the start and that they need to retrace their steps and change the way they build and program computers is not an easy task, but someone's got to do it.

Big Problem

The bad news is that, if we don’t switch soon, we are going to be in a much bigger mess than we already are. The reason is that there is an inescapable correlation between complexity and unreliability in any system that is based on the algorithmic model. This much we already know, what with Fred Brooks, “No Silver Bullet” and all that jazz. It is a much bigger problem than most of us suspect, however. As I have said many times before, unreliability places an upper limit on the complexity of our systems. We could conceivably be riding in self-driving vehicles right now but concerns over safety, liability and development costs will not allow it. As a result, we will continue to suffer over forty thousand traffic fatalities every year, in the US alone! And that’s just the tip of the unreliability iceberg. The good news is that there is an alternative.

Bad Timing

In an Erlang program (and every system that uses algorithmic threads and processes), the timing of operations in one concurrent module is independent of the timing of operations in another. This is analogous to having objects in different universes interacting with each other. Bad things are bound to happen. The problem is that the timing of events (whether internal or external to a module) becomes unpredictable. Why is this important, you ask? It is because, without rock solid temporal expectations, timing constraints cannot be enforced. Detecting violations to assumptions made about the timing of actions is the primary method of ensuring reliability in behaving systems. Measuring and comparing temporal intervals is as important to computing as measuring and comparing distances is to architecture or map making.

Good Timing

By contrast, in a COSA operating environment, all operations are elementary synchronous processes and they all execute in locked steps. That is to say, they are synchronized to a master clock and they all have equal durations, exactly one system cycle (the actual length of the cycle can vary, but everything must remain synchronized). Alegorically speaking, we might say that they all belong in the same universe and obey the same laws. Aditionally, since COSA is reactive, meaning that all objects react to their signals as they happen, there is none of the latency problems normally associated with algorithms. As a result, the timing of events is 100% deterministic. If event A is expected to always occur at the same time as event B, then this is what should happen. Otherwise, something is wrong. This makes it easy to test modifications to a complex system and make sure that they do not introduce bad side effects.

Do not underestimate the power of timing expectations to uncover hidden bugs in a complex program. Everything that takes place on a computer is precisely timed and, in a system that enforces deterministic timing, even the tiniest abnormality is bound to violate the expectations and trigger an alarm. One of the nice things about the COSA model is that the discovery and enforcement of timing constraints can be automated. As an aside, deterministic timing, combined with the inherent reactivity of COSA objects (nothing happens unless a change is detected in the object’s environment), allows us to implement other beneficial mechanisms such as the automatic detection and resolution of data dependencies. This is explained on the COSA System page.

Conclusion

In conclusion, temporal integrity and coherence (including motor conflict detection/resolution) and the automatic resolution of dependencies effectively break the correlation between complexity and unreliability that is a characteristic of the algorithmic model. This is rather counterintuitive but the COSA promise is that the robustness (fitness) of a system will be proportional to its complexity. So, the COSA motto should be, “Keep it complex, stupid”.

Tuesday, August 28, 2007

The Seven Deadly Sins of Erlang

More Erlang Related Articles

You want to know what I really think of Erlang? Erlang is crap. That’s what I think. Now don’t feel bad if you are an Erlang fanatic. And please don’t get mad. I always tell it like I see it. That’s my style. Besides, I think all programming languages are crap. Right now, I just have a specific bone to pick with the functional programming model and Erlang in particular. It is not the panacea that its supporters make it out to be. It is a false god, in my opinion. It has nothing really interesting or revolutionary to offer to a bug-haunted computing world that is suffering from a chronic productivity crisis and is currently undergoing a painful transition from sequential processing to massive parallelism. Without further ado, here are the seven deadly sins of Erlang:
  1. It has no mechanism for automatically enforcing deterministic timing at the operation level. In other words, operations in an Erlang program are neither reactive nor synchronous. This is an absolute must for rock-solid reliability. Note: Synchronous processing is not the same as synchronous messaging.
  2. It has no mechanism for automatically enforcing data and event dependencies. Bad for the same reason as above.
  3. It is implicitly sequential. Very bad when it comes to parallel programming.
  4. It is explicitly concurrent. Bad for the same reason.
  5. It is algorithmic. The algorithmic model of computing is a rampaging dinosaur at an amusement park. It is the primary cause of every ill that ails the computer industry. It is an insidious disease that came to life close to 150 years ago back when the first computer was built with gears, cogs and rotating shafts. It hides the true nature of computing. It should be put out of its misery and securely locked up in an antique museum, right next to the slide rule, the buggy whip and the typewriter.
  6. It is a computer language (based on English). Why English? Why not Mandarin, Swahili, Romansh or Latin for that matter? Programming a computer should have absolutely nothing to do with one’s mother tongue. Written code is notoriously hard to understand, even when it is your own code. Only programming nerds are enamored with cryptic code. It's a badge of honor to them. Truth is, the very fact that programming has to rely on a specific natural language is a sure sign that something is wrong with the linguistic approach. Not to mention the nasty little fact that languages are inherently sequential, which makes them ill-suited to parallel programming.
  7. Last but not least, Erlang does not have inherent support for plug-compatibility (drag and drop programming) and a searchable/browsable object repository. This is essential to effective and fast software composition and reusability. But then again, plug-compatible components make much better sense in a graphical environment. I happen to like Jeff Han's multi touch screen interface technology. I think it's probably going to play a vital role in our parallel programming future.

My question to Erlang supporters is this. If functional programming was all that it’s cracked up to be, how come Joe Armstrong (et al) felt it necessary to provide a method for failure localization? No need to answer. Joe alreeady did. He assumed that "large systems [written in a functional language or not] will probably always be delivered containing a number of errors in the software." By contrast, the COSA philosophy is that software should never fail, period.

See Also:
Parallel Programming, Math and the Curse of the Algorithm
The Age of Crappy Concurrency: Erlang, Tilera, Intel, AMD, IBM, Freescale, etc…
Half a Century of Crappy Computing
Parallel Computers and the Algorithm: Square Peg vs. Round Hole

Monday, August 27, 2007

Yes, There Is a Silver Bullet, Professor Agarwal!

In a recent article in EETimes about Tilera Corporation’s hot new 64-core processor family, I was appalled to read the following:

Tilera's chief technologist, Anant Agarwal, a longtime multicore researcher at MIT, admits his company's tools are no panacea for the formidable computer science problems that underlie programming massively parallel architectures. "There is no silver bullet in multicore programming," he said.
Yes, there is a silver bullet, Professor Agarwal!

The reason that current multicore processors are hard to program is that they were created for the wrong software model. The algorithmic model has been with us since the days of Charles Babbage and Lady Ada. It is not suited for parallel processing because it is inherently sequential. Indeed, it is responsible for most of the ills that ail the computer industry, not just the parallel programming quagmire. It is time that we switch to a non-algorithmic model, one which is implicitly parallel. Sequential ordering should be explicit, not implicit as it is now.

The multicore CPU manufacturers have only themselves to blame for the mess that they are in. You don't design a CPU and expect the software model to change to accomodate it. It should be the other way around. The software model must come first.

So yes, there is a silver bullet and Project COSA is it. The COSA model is inherently and implicitly parallel and it makes parallel programming easy. It would not take much to convert the design of current multicore CPUs to support the fine-grain parallelism of the COSA model. COSA offers many other benefits as well. In a COSA-optimized multicore processor, load balancing would be automatic. COSA programs would automatically scale to take advantage of more powerful processors. Last but not least, due to its strict timing constraints, COSA programs will be 100% reliable, guaranteed.

Sunday, August 26, 2007

Parallel QuickSort in COSA

The day before yesterday, I did a search on the web looking for a parallel quicksort algorithm in Erlang. I could not find any. I was somewhat surprised (although not entirely) given that Erlang is promoted as one of the best concurrent languages that will make it easy to take advantage of the parallelism of our new fancy multicore processors. I must admit that Erlang's creators never claimed that it was a parallel programming language, just concurrent with light weight processes. So I decided to design a parallel quicksort component based on the COSA software model. COSA is ideal for this sort of thing because it was designed for super fine-grain parallelism.

PS. Please write me a note if you know of a parallel quicksort in Erlang or some other functional/concurrent language.

Friday, August 24, 2007

Implicit vs. Explicit Parallelism in COSA and Erlang

Even though Erlang is officially defined as a concurrent programming language, parallel processes in an Erlang program must be explicitly created or spawned whereas sequential operations are implicit. That is to say, no special keywords or primitives are needed to specify that certain operations must occur in a specific order but this is not the case for parallel processes. In Part 1 of Concurrent Programming in Erlang we read:
Sequential Erlang needs the addition of a small number of primitives to turn it into a concurrent programming language. We introduce the primitives necessary to create a parallel process and for message passing between processes.
In COSA, by contrast, parallelism is implicit and sequential operations must be explicitly specified. What this means is that, if two operations must be executed in a particular order, the programmer must explicitly specify the order. This is done simply by drawing a line between the two cells that represent the operations.




The figure above shows a COSA component or module that contains a number of elementary operations or cells (blue and yellow circles). COSA modules are automatically assumed to be concurrent. There is no need for the program designer to specify this. Cells are the only processes in COSA and, unlike Erlang processes, which have arbitrary durations, every COSA cell executes its operation in exactly one system cycle. This is done in order to enforce the deterministic timing of operations, an absolute must for reliability. A small red circle represents the output of a cell. Every cell emits an output signal as soon as it is done executing. The application designer must explicitly connect the output of one cell to the input of another cell in order to specify the order of execution. The benefit of this approach is that, unlike Erlang's algorithmic processes, a COSA sequence can easily fork into two or more other sequences. Forks should be used as much as possible because they automatically take advantage of the parallel processing capabilities of the hardware, if any. This is the main difference between algorithmic and non-algorithmic sequences. Cells are explained in greater detail on the COSA system page.


The point that I am driving at is that, in a truly concurrent development environment, parallelism must be implicit and sequential ordering explicit. Not the other way around. This approach facilitates good program comprehension and it is one of the things that differentiate COSA from concurrent programming languages like Erlang. Implicit parallelism in a graphical programming environment is part of the future of parallel software construction, in my opinion. Computer languages are ill-suited to parallel programming because they are inherently sequential. Lately, I have fallen in love with Jeff Han's amazing multi-touch interface technology. It think it is perfectly suited for parallel application development.

Thursday, August 23, 2007

In COSA, Everything Is a Process. Not So in Erlang

That is the main difference between COSA and other concurrent languages like Erlang. Whereas Erlang is only partially concurrent (it uses so-called light weight processes), COSA does not cut corners. COSA is 100% concurrent. Nothing happens in COSA unless it receives a signal. Every cell in COSA is an elementary concurrent operator that waits for a signal, does something and sends an output signal to one or more other cells. Even adding A + B is a concurrent process. A COSA program consists exclusively of cells and their environment (data).

All right. I realize that a COSA application will be rather slow on current processors compared to conventional programs, but that is not COSA’s fault. It is the fault of the computer industry for having adopted a computing model that is at least as old as Charles Babbage and Lady Ada Lovelace. The algorithmic model is just plain wrong. It is a relic from the distant past. It is not fit for the 21st century. It is the cause of the software crisis. It is the reason that complex software is unreliable. It belongs in a museum, not in our computers.

Computing will not come of age until we adopt a software model that eliminates sequential programming altogether. We must recognize the errors of our ways. We must design our multicore processors to support the correct software model, not a 150 year-old dinosaur. We must stop believing that languages are a natural way to program computers. Computer languages are primitive, period. There are inherently sequential and they are not suited for parallel computing. It is about time that the computer industry wakes up from its self-imposed stupor and realizes that it has been doing it wrong right from the start. To borrow words from Nintendo of America’s CEO, Reginald Fils-Aime, COSA is about kicking ass; COSA is about taking names. But, unlike Reggie, I am tired of playing games. I want my self-driving car now!

Sorry about the outburst. I am getting frustrated and I just had to get it out of my system. :-)

Fault Tolerance: COSA vs. Erlang

When Joe Armstrong and his colleagues designed the concurrent programming language Erlang, their main intention was to provide a tool with which to create software systems that are robust and impervious to local failures. Failure localization is an absolute must for high-availability applications. However, the difference between the COSA and the Erlang philosophies regarding failure localization is telling.

Armstrong wrote in his thesis that “large systems will probably always be delivered containing a number of errors in the software, nevertheless such systems are expected to behave in a reasonable manner.” It is clear that Erlang’s creators were mainly concerned with finding a way to prevent hidden software errors from crashing mission-critical systems. The COSA philosophy, by contrast, is that software should never fail and that fault tolerance should be needed only to prevent catastrophic failure in the case of malfunctioning hardware (sensors, effectors, etc…). In this vein, mission-critical systems should incorporate redundancy as an additional preventive measure.

I really have nothing against Erlang's use of pervasive concurrence and asynchronous messaging to create fault-tolerant software systems. COSA uses the same techniques. The reason that I keep hammering at the software reliability problem is that this is a crucial issue for the software and hardware industries. Fault tolerance is not nearly enough. In some applications, safety is so critical that any software failure, even if localized, is not an option. Unless and until a solution is found, unreliability will continue to put an upper limit on the complexity of our software systems. As an example, we could conceivably be riding in self-driving vehicles right now but concerns over reliability, safety and high development costs will not allow it. As a result, over 40,000 people die every year in traffic accidents, in the U.S. alone. Something must be done!

The solution will require a radical change in both software and CPU architectures. Now is the right time to make the switch as the industry is turning a corner: computing is going parallel. We should use the opportunity to abandon the old algorithmic approach and adopt a non-algorithmic paradigm. This is the main goal of Project COSA.

Wednesday, August 22, 2007

A Multicore CPU Architecture for the 21st Century

Forget threads and algorithms! They are the work of the devil. :-) There is a much better way. There is a way to design a multicore CPU to handle fine-grain parallelism in an MIMD environment without the use of either threads or algorithms. It will be much faster and simpler than current multicore processors and its use of deterministic timing will allow the creation of rock-solid software systems. This is the promise of the COSA CPU architecture, which is based on and optimized for the COSA software model.

The COSA Software Model

A COSA program is built from a relatively small library of elementary, concurrent, synchronous objects or cells that communicate with each other using discrete signals. Below is an example of a simple While loop component that uses a few connected COSA cells.




What makes COSA different from other concurrent models is that a cell is an elementary process (an operation) that waits for a signal, does something and sends an output signal to one or more cells. There are two types of cells, sensors and effectors. Sensors are comparison operators. Unlike state machines, a COSA program operates exclusively on state changes. Each cell data structure contains a list of the cells it sends signals to.

The Cell Processor

The way it works is not rocket science. It uses a technique that is already well known to programmers of cellular automata, neural networks and simulation applications. Essentially, a COSA CPU maintains two update (processing) lists of cells in its on-board cache, one for input and the other for output processing. While one list is being processed, the other is being populated. Then the lists are swapped and the process begins anew. Each pass is considered a single cycle. Two lists are used in order to eliminate racing problems that would otherwise emerge. The CPU can handle load balancing automatically by assigning cells to different cores for processing.

There is no need for threads, context switching, algorithms or any of the old stuff. Traditional optimization techniques can be still be used, though. For example, COSA programs can be compiled into multiple synchronized lists of cells that can be pre-fetched by the CPU while it is processing the other cells in the update lists. The COSA model does not depend on the programmer to implement concurrency in the application. Concurrency in COSA is the norm. There is no getting around it.

See How to Design a Self-Balancing Multicore CPU for Fine-Grain Parallel Applications for more.

The Pot of Gold

AMD and Intel are currently fighting over who will control the multicore CPU market. There is a lot of money at stake. While these two Goliaths are locking horns and focusing their attention on soon-to-be-obsolete technology, a third party is liable to sneak behind them and steal the pot of gold. Let's face it. The CPU architecture is due for a radical change. The computer world is going parallel and the old algorithm/thread paradigm is showing its age. There is a sweet scent of revolution in the air. Who will be the leader of the next computer revolution? Sun? IBM? NVidia? Xilinx? Tilera? Some unknown player? We'll see.

The Problem with Erlang

The problem with Erlang and other functional programming languages is that their designers assume that unreliability is an essential part of complex software systems. Joe Armstrong, the main inventor of Erlang, wrote the following at the beginning of his thesis “Making Reliable Distributed Systems in the Presence of Software Errors”:

How can we program systems which behave in a reasonable manner in the presence of software errors? This is the central question that I hope to answer in this thesis. Large systems will probably always be delivered containing a number of errors in the software, nevertheless such systems are expected to behave in a reasonable manner.

I apologize to Joe for using his thesis as an example, but it seems that the entire computer industry has swallowed Fred Brooks’ “No Silver Bullet” arguments regarding the relationship between complexity and reliability, hook, line and sinker. This is very unfortunate because, as I explain on the Silver Bullet page, Brooks’ arguments are fundamentally flawed. The COSA philosophy is that unreliability is not an essential characteristic of complex software systems. As I wrote in a previous article, there are situations where safety or uptime is so critical that not even extreme reliability is good enough. In such cases, unless a program can be guaranteed 100% reliable, it must be considered defective and should not be deployed. Achieving rock-solid reliability in a complex software system is not impossible.

First and foremost (and this is what is missing in Erlang and other concurrent languages), the timing of all processes in the system must be 100% deterministic and must be based on change. Nothing should happen unless something changes. Second, the system must automatically enforce the resolution of data/event dependencies in order to eliminate blind code. The only way to achieve these two goals is to adopt a software model that is based on elementary, concurrent, non-algorithmic, synchronous, communicating processes. In COSA, these concurrent processes are called cells of which there are only two types: sensors and effectors. The main difference between a COSA cell and a concurrent process in other languages is that COSA cells are synchronous, meaning that their execution times are equal. Every COSA cell executes its elementary function in exactly one system cycle. This result in deterministic timing, a must for reliable software.

Note: There is a difference between synchronous messaging and synchronous processing. COSA and most concurrent languages use asynchronous messaging, i.e., the message sender does not have to wait for a reply before continuing it execution.

Tuesday, August 21, 2007

COSA vs. Erlang

The functional programming language Erlang is rightfully touted by its supporters as being fault-tolerant. What they mean is that, in the event of a malfunctioning process (or even many processes) in a program written in Erlang, the other processes will continue to run properly. Erlang programs tend to fail gracefully rather than catastrophically. Contrast this with a program written in a conventional language like C or C++ where even a minor defect may result in a catastrophic failure that crashes the entire program. Fault tolerance in Erlang is due to its heavy use of concurrent processes and its strict reliance on asynchronous messaging between processes. Asynchronous messaging means that the message sender does not have to wait for a response from the receiver before continuing its execution. This way, if the receiver malfunctions and crashes, the sender can continue to perform its function, unimpeded. By contrast, synchronous messaging is what normally occurs in a function (subroutine) call, in conventional programming.

COSA shares all the fault tolerance qualities of Erlang but this is where the similarities end. The COSA philosophy is that nothing should fail, period. There are software applications where safety is so critical that not even extreme reliability is good enough. In such cases, unless a program can be guaranteed 100% reliable, it must be considered defective and should not be deployed. That’s the main goal of Project COSA: 100% reliability, guaranteed.

A COSA Object or Component

Unlike Erlang and other functional languages, COSA uses a graphical approach to software construction (see figure above) because the linguistic approach is not conducive to good program comprehension in a complex parallel environment. In addition, whereas a concurrent process in Erlang can be as long (both in number of instructions and in execution time) as the programmer desires, all processes in COSA are elementary and have equal execution times. This is what is meant by COSA using synchronous processes (cells), which is not to be confused with synchronous messaging. Thus a COSA program is based on fine-grained, synchronous parallelism. The end result is that timing is rigorously deterministic. This, alone, eliminates an untold number of timing-related bugs.

Synchronous processing is an essential characteristic of COSA but there are many other equally beneficial aspects from the point of view of reliability. These are explained on the COSA website. Do take a look. There are disadvantages, of course. For example, while Erlang is available now and can run efficiently on current processors, COSA is still in the development stage and must be interpreted on current processors. The reason is that COSA is inherently non-algorithmic and current processors are designed and optimized for the algorithm. Having said that, I feel that the advantages of COSA (e.g., total reliability) far outweigh the disadvantages. Eventually, the computer industry will get tired of the heavy price its customers continue to pay for unreliable software systems. At that time, it will come to its senses and build the right CPU architecture to handle the COSA model. When that happens, all the disadvantages of COSA will disappear and we will be able to build parallel software systems of arbitrary complexity without the accompanying unreliability. It will be the golden age of automation, which is the true promise of computing. It's worth the wait, in my opinion.

Monday, August 20, 2007

Jeff Han and the Future of Parallel Programming

Forget computer languages and keyboards. I have seen the future of computer programming and this is it. The computer industry is on the verge of a new revolution. The old algorithmic software model has reached the end of its usefulness and is about to be replaced; kicking and screaming, if need be. Programming is going parallel and Jeff Han’s multi-touch screen interface technology is going to help make it happen. The more I think about it, the more I am convinced that Han’s technology is the perfect interface for the COSA programming model. COSA is about plug-compatible objects connected to other plug-compatible objects. Just drag 'em and drop 'em. What better way is there to compose, manipulate and move objects around than Han’s touch screen interface?



This COSA component that I drew years ago looks painfully primitive compared to Han's images but I can imagine a bunch of COSA cells and components being morphed into really cool 3-D objects that can be easily rotated, opened or moved around on a multi-touch screen. A complex COSA program could be navigated through as one would navigate in a first-person 3-D video game. The COSA programmer could jump inside the program and look at the cells firing, not entirely unlike what it would look like moving through a maze of firing neurons inside the human brain. Add a speech interface and eliminate the keyboard altogether. I never liked keyboards anyway. The computer will not come of age until keyboards (a relic from the eighteenth century) go the way of the slide rule. This is exciting.

Sunday, August 19, 2007

Discovering Statistical Pairs for Motor Sequencing

How does one design a mechanism to discover consecutive motor signals? Believe it or not, this is what I have been thinking about the last several days and what has been holding up my progress with Animal. It is not as easy as it looks. For examples, here are a couple of issues that I am currently struggling with:
  • A sequence pair (AB) may sometimes run backwards (BA). Does this mean that every pair must have an opposite?
  • Consider the two consecutive pairs AB and BC. Should AC be considered a consecutive pair as well? A and C are certainly consecutive but from the point of view of generating motor command signals, AC seems superfluous since AB and BC can be chained to form a long sequence. However, what if signal B does not arrive sometimes, for whatever reason? Does this mean that AC should be kept as a valid pair?

The whole thing is getting too complicated for my taste. You all know me; I am a sucker for simplicity. I am currently leaning toward this solution: Create as many pairs as statistically possible and let the chaining mechanism discover which ones are viable within a motor sequence. That is to say, let the chaining mechanism eliminate the ones that induce motor conflicts. I suspect this may overwhelm the motor learning system but I’ll let you know how it all works out.

Friday, August 17, 2007

Another Reason That Threads Are Evil: Crybabies

One often hears parallel programming pundits complain about the human brain being more at home in a sequential environment and finding it difficult to handle parallel objects. Nothing could further from the truth! We live and survive in a concurrent universe where thousands of objects around us have independent existence and behave concurrently. Many people drive a vehicle to and from work while processing an untold number of parallel stimuli from their visual, auditory and tactile sensory spaces. We never had a problem surviving and operating in a concurrent world. Why should we suddenly be handicapped when it comes to parallel computing objects?

The real reason that we find it difficult to handle so-called parallel programs is that they are not really parallel. They are just a bunch of threads, which are sequential algorithms slapped together like spaghetti in a non-deterministic and haphazard fashion. True parallel objects have nothing to do with algorithms. "Parallel algorithm" is an oxymoron. A true parallel object is one that reacts synchronously to environmental stimuli, just like the dog, the cat or Uncle Joe. A true parallel object is a behaving machine. It has sensors to detect changes in its environment and effectors to effect changes in it. Indeed, a true parallel program should consist entirely of simple sensors, effectors and its data environment. Unless and until we come to fully grok that computing is about behavior and nothing else, we will have to keep listening to a bunch of crybabies incessantly bitch about the human brain's supposed inadequacy vis-a-vis parallel programming. It's tiresome.

That’s part of what Project COSA is about: true parallelism for our parallel minds.

Thursday, August 16, 2007

Why Threads and Current Multicore CPUs Are Evil

Lately, Intel and other CPU manufacturers have been complaining that software vendors are not keeping up with the parallel processing advances inherent in multicore processors. What they are really saying is that software applications are not using as many concurrent threads as they should. Since multicore processors are designed to process multiple threads simultaneously in the on-chip cores, an application that uses few threads cannot take full advantage of the available processing power.

The problem with threads is that they are evil and they are evil for three reasons as follows:

  1. They are algorithmic.
  2. They are asynchronous.
  3. They require context switching.

Basing software construction on the algorithm is evil because it ignores the true nature of computing. It is the primary reason for the software reliability and productivity crisis. Asynchronous processing is evil because it leads to non-deterministic timing, which, in turn, leads to unreliable software. Context switching is evil because it introduces a lot of processing overhead. This becomes unmanageable in a system that uses fine-grained parallelism to handle a huge number of simple parallel objects. There is a simple way to perform synchronous parallel processing that does not require the use of threads. CPU designers should abandon the algorithmic/thread model and adopt the COSA Software Model. COSA is a non-algorithmic, signal-based, synchronous software model. It is simpler and it solves all the nasty problems that are currently plaguing the computer industry.

PS. I'm kidding about the 'evil' part but threads are bad, any way you look at them.

Thursday, August 9, 2007

Keeping it Simple: Animal’s New Brain Design

Animal’s current brain design is much more complex than it has to be. Several of the cell assemblies in the human brain are needed mainly because the brain must process a huge number of low-level and unreliable sensory signals. By contrast, Animal’s chess environment is perfectly ordered and there are no sensory uncertainties. In addition, Animal uses very high-level sensors, which, if you think about it, already perform most of the recognition work that a real brain would have to perform in order to, say, recognize a pawn or a bishop.

In view of the above, I have decided to drastically simplify the design of Animal’s brain. I am taking out the association and sequence detection layers. They are not needed, in my opinion. Sequence detection is required only for variable time-scale sequences, which do not exist in Animal’s sensory environment. As a result, Animal’s brain design will be reduced to the following:
  • Sensor Layer
  • Sensory Cortex (signal separation)
  • Memory Layer (working memory)
  • Motor and Motivation Layer
  • Effector Layer

The memory layer is strictly working memory with a seven-item capacity (seven eyes). The function of an eye is to maintain an action cell alive for a short time.

An action cell consists of two neurons, a predecessor and a successor. The motor layer creates and manages all the action cells and organizes them into behaviors or action sequences. It receives input signals directly from the separation layer (sensory cortex) and channels them immediately to the effector layer. This is the most complex cell assembly in Animal’s brain. It handles motor conflict resolution, motivation (reinforcement learning) and anticipation. I’ll let you know how it all works out.