Wednesday, August 6, 2008

Parallel Programming: Intel, AMD, Nvidia, US and the World


There is no getting around the fact that, unless powerful domain-specific tools are built on top of GPGPU-type processors like Larrabee, programmers will find it extremely hard to write software for them. The reason is that they would need to gain intimate knowledge of the hardware.

No Solution In Sight

The problem with heterogeneous processors/GPGPUs is that they incorporate two different and incompatible approaches to parallel programming (fine-grain data-parallel SIMD and coarse-grain multithreaded MIMD). Each has its advantages and disadvantages but having to deal with both makes software development a worse pain in the butt than it already is. It does not seem that either Intel, AMD, Nvidia or any of the other multicore vendors are able or willing to go beyond this intolerable incongruity by offering a universal solution that is easy to program and is equally at home in all sorts of application environments.

Don't be Too Complacent

There are many countries and organizations in the world that would not hesitate to cease a clear opportunity to dominate the computer industry in this century. Very big money is in the balance and the industry is at a crossroad. The parallel programming crisis is quickly getting to the point where something radical will have to rise to the surface in order to solve the problem. The market wants nothing less than a solution and it wants it yesterday. Intel and the others should not be so complacent or arrogant as to think that the heads of their engineering and research teams are the brightest in the world or that their vision of the future of parallel computing is perfect. There is nothing visionary or revolutionary about the design of Larrabee, regardless of the fine engineering that went into it. The parallel programming problem is still with us and the whole world is still searching for a solution. Intel may be a giant in the processor business but so was Xerox in the copier industry. Big is not synonymous with invincible. The industry seems precariously on the verge of undergoing a seismic event that will undoubtedly send the unprepared crashing into the dust. Wisdom and caution are advised.

The Answer Is Staring at Us

The computer industry is still thinking with last century's brain. They must abandon the old stuff and embrace the new stuff, which, when you think about it, is not all that new. It's just that it has never been seen as a parallel programming model by the industry before. At least, not seriously. There is a way to design and program parallel computers that is 100% threadless while offering the flexibility of a CPU and the fast fine-grain deterministic parallelism of the GPU. It is based on a method that programmers have been using for decades to emulate parallelism in such applications as neural networks, cellular automata, simulations and video games. To find out more, read How to Solve the Parallel Programming Crisis. Dismiss it at your own peril.


To give my readers an idea of the wide fascination that the parallel programming problem holds on the world at large, here is an incomplete list of countries that regularly visit my blog, not necessarily in order of frequency, although the US is the clear leader:

United States, China, India, Singapore, Vietnam, Sweden, France, Italy, Finland, Iceland, Israel, Switzerland, Egypt, Iran, Turkey, Canada, Germany, Denmark, Belgium, Spain, Portugal, Greece, Norway, Ireland, United Kingdom, Netherlands, South Korea, Japan, Thailand, Malaysia, Brazil, Peru, Chile, Mexico, Colombia, Bolivia, Romania, Hungary, Poland, Russia, Latvia, Estonia, Serbia, Ukraine, Australia, New Zealand, Argentina, Philippines, Indonesia, Paraguay, Venezuela, Czech Republic, Mongolia, Taiwan, Hong Kong, Nigeria, Kenya, South Africa, Pakistan, Austria, etc.

All right, you get the picture. Theoretically, any one of those countries has a chance to dominate the computer industry in this century. And guess what? Given what I know about intellectual property laws, it would take only a miniscule investment on the part of a country or private organization to lock in a lion share of this business for decades to come. There is big money at stake. And I mean, BIG MONEY.

Related Article:

How To Solve the Parallel Programming Crisis

No comments: