Monday, July 7, 2008

Erlang Is Not the Solution

[What follows is a comment that I posted at Intel Research Blogs in response to a recent article by Anwar Ghuloum titled Unwelcome Advice. I was actually replying to another commenter, Falde, who is promoting Erlang for parallel programming. Since Erlang is back in the news lately (see Slashdot article), I felt it would be appropriate to let my readers know what I feel about the suitability of Erlang as a solution to the parallel programming problem.]

The Erlang “solution” to the parallel programming problem is not the solution, otherwise we would not be here discussing the problem. The functional programming model has major drawbacks, not the least of which is that most programmers are not familiar with it and have a hard time wrapping their minds around it.

Another problem is that the Erlang message passing model forces the OS to copy entire arrays onto a message channel. This is absurd because performance takes a major hit. Shared memory messaging is the way to go. Certainly, using shared memory can be very problematic in a multithreaded environment but that is not because it is wrong. It is because multithreading is wrong. Threads are inherently non-deterministic and, as a result, hard to manage. Memory (data) is the environment that parallel agents use to accomplish a task. Agents sense and react to changes in their environment: reactive parallelism is the way it should be. Memory should be shared for the same reason that we humans share our environment. Switch to a non-algorithmic, deterministic model that does not use threads and all the problems with shared memory will disapear.

Furthermore, in spite of its proponents’ insistence on the use of words like “micro-threading” and "lightweight processes", Erlang enforces coarse-grain parallelism. In my opinion, if your OS, or programming language or multicore processor does not use fine-grain parallelism, it is crap. There are lots of situations that call for fine-grain processing. I have yet to see a fine-grained parallel quicksort implemented in Erlang.

Finally, a true parallel programming paradigm should be universal. There is no reason to have one type of processor for graphics and another for general purpose computing. If that’s what you are proposing, then it is obvious that your model is wrong and in serious need of being replaced. [Interestingly, Erlang proponents have nothing interesting to say on this issue.]

In conclusion, I will reiterate what I have said before. What is needed to solve the parallel programming crisis is a non-algorithmic, synchronous, reactive software model. By the way, I am not the only one ringing the non-algorithmic/reactive bell. Check out the work of Peter Wegner and Dina Goldin at Brown university. They’ve been ringing that bell for years but nobody is listening. You people are hard of hearing. And you are hard of hearing because you are all Turing Machine worshippers. It’s time for a Kuhnian revolution in computer science.

See also:

Parallel Computing: Why the Future Is Non-Algorithmic
How to Construct 100% Bug-Free Software
How to Solve the Parallel Programming Crisis

9 comments:

Mr. Funk said...

I'm really interested in your COSA model and ideas, but the description of the system assumes too much knowledge. I've been writing software for a very long time, and I see all the same problems you do, and I'd like to understand your proposed solution better - but the COSA site delves too quickly into terminology one doesn't get exposed to outside of academia. If you had some supplementary and more approachable information to bridge the gap between the rant and the professor, people like me could better get a grasp of your ideas and help them spread.

Diogenese said...

I'm also extremely interested in reactive systems, synchronus programming, etc...

The problem I have, as an Open Source developer, is finding a system implementation that's mature enough to be usable for major programming tasks.

I've been playing with Lucid Synchrone, but that depends on OCaml. Esterel is way out of my price range.

If there is a reactive programming system/environment that's mature I'd be glad to start working with it. Right now I'm working on a Linux AMD64 Host system.

Also, what are you're opinions on the stack based processor models developed by Chuck Moore?

Any pointers?

James said...

diogenese, I asked Louis about Forth the language a couple of posts ago. You can view his response here http://rebelscience.blogspot.com/2008/06/parallel-computing-end-of-turing_21.html

I imagine any comment on the language/interpreter is equally applicable to the processor design.

Louis Savain said...

Funk,

The system and software composition pages are pretty much what I got right now. Unfortunately, no good tutorial will be available until an actual COSA computer and its development tools are designed and built. That's going to take money and time, hopefully real soon.

Diogenese said...

Perhaps a small evolutionary step in hardware design might be appropriate to get the ball rolling.

Perhaps a peripheral PCI-E Card based on the Trinary computer concept http://jeff.tk/wiki/Trinary
could get things started. Obviously it's not a full COSA environment, but it might just be enough to spark interest.

Hypothetically it could be implemented cheaply, and mass produced. Yet still be small enough as to not pose a threat to the established players.

Yes?

Louis Savain said...

Diogenese,

I personally don't like any of the synchronous languages that are out there. I think that they are way too complicated than they need to be. In addition, the fact that they are textual languages as opposed to visual languages bothers me to no end. There is nothing like a dynamic visual diagram to enhance one's understanding of signal flow and timing.

Having said that, the only synchronous language I would recommend is SIGNAL. There is an introductory tutorial at the IRISA Espresso site. For some reason, the French are fanatical about synchronous languages and Esterel, in particular, is used in all sorts of safety-critical applications in European aviation (AIRBUS, EADS, etc...)

SIGNAL/Polychrony
SIGNAL Primer [pdf]

Dave Roberts said...

Louis, you wrote:

The Erlang “solution” to the parallel programming problem is not the solution, otherwise we would not be here discussing the problem. The functional programming model has major drawbacks, not the least of which is that most programmers are not familiar with it and have a hard time wrapping their minds around it.

Another problem is that the Erlang message passing model forces the OS to copy entire arrays onto a message channel. This is absurd because performance takes a major hit. Shared memory messaging is the way to go.


So, a few things:

1. Knocking Erlang by saying that programmers are not familiar with functional programming and then espousing synchronous, reactive programming and COSA is a bit of the pot calling the kettle black, no? I like some of your COSA ideas, but I'd say that Erlang and functional programming already have a large headstart over COSA in terms of programmers wrapping their heads around it. Right or wrong, that's certainly the case.

2. Erlang doesn't force copying of messages. The semantics of the messaging primitives are written as if copies happen, but whether they actually do or not is up to the implementation. While the current Erlang virtual machine creates separate heaps for each process, there has also been research into using a single unified heap for all processes. In this case, a message send of a large array results only in a pointer copy. In short, don't confuse Erlang the abstract language specification and Erlang the current predominant implementation. It may be a flaw of the current implementation, but it's not a flaw of the language itself.

Louis Savain said...

Dave writes:

1. Knocking Erlang by saying that programmers are not familiar with functional programming and then espousing synchronous, reactive programming and COSA is a bit of the pot calling the kettle black, no? I like some of your COSA ideas, but I'd say that Erlang and functional programming already have a large headstart over COSA in terms of programmers wrapping their heads around it. Right or wrong, that's certainly the case.

Not at all. Everybody is already intuitively familiar with the COSA model whether they know it or not. If you can understand Boolean logic circuits, you're already 99% of the way there. It's not rocket science; it's a neck bone connected to collar bone sort of thing.

Erlang (and functional languages in general), on the other hand, is not intuitive. It is a nerd language, created by nerds for nerds. What is needed is an easy graphical development tool that uses snap-together software components in a way that will turn anybody into a programmer. Programming for the masses is what the market wants, not some obscure language with an obscure syntax that nerds can use to obtain bragging rights. Erlang is a far cry from being a language for the masses. It's a hideous language that only a nerd would love. But then again, I am biased since I think most programming languages are ugly.

With the advent of multi-touch screen user interfaces, it is high time that computer programming is elevated from a nerd-only profession to something that anybody and their aunt Mary can enjoy. Software development must be severed from the antiquated ideas of the last century and brought kicking and screaming into the 21st century. If you want to know why more women are not interested in computer programming, it's because they are put off by all the geeky ugliness. LOL.

2. Erlang doesn't force copying of messages. The semantics of the messaging primitives are written as if copies happen, but whether they actually do or not is up to the implementation. While the current Erlang virtual machine creates separate heaps for each process, there has also been research into using a single unified heap for all processes.

You got any references for this? If it were that simple, it boggles the mind that the original Erlang implementers did not do it from the start. I tend to think that message copying is a requirement of the model itself and cannot be circumvented.

At any rate, my biggest beef (which you chose not to address) with functional programming languages like Erlang, other than them being ugly nerd languages that do very little for productivity, is that they use a coarse-grain approach to parallelism. This is a monumental drawback, in my opinion, because the number of applications that can benefit from fine-grain parallelism is huge. Examples are multimedia, signal processing, simulations, cellular automata, neural networks, etc... Even binary search and sorting routines can benefit tremendously from fine-grain parallelism. In addition, as massive parallelism becomes the norm, it will open the gate for a flood of new ideas for powerful fine-grain parallel applications.

Finally, I am weary of giving mathematicians (they are the most avid proponents of functional programming) any say in the design of the next generation of computers (both software and hardware models) seeing that they have already shot computing in the foot, so to speak. They are the ones who got everybody into this mess in the first place.

Toth, Laszlo said...

Even as an erlang programmer I have to say your site is great and I partly agree with your opinions about lightweight processes.

La