Wednesday, November 14, 2007

SC07- Day 3 - HPC in the Kitchen and Laundry Room

After spending some time on the exhibitor floor checking out the cool Second Life demo at the Sun booth, I decided to walk over to see the "HPC in the Kitchen and Laundry Room" talk.

It's interesting to note how manufacturing is becoming virtualized at all levels. I saw a piece on the NewsHour a few months ago on how companies like CATIA and Dassault build aircrafts virtually before they assemble the real thing. This strategy allows them to find all sorts of assembly bugs that they would not be able to discover otherwise (e.g. if you attach the wing to the aircraft in that manner, it is going to smash into the fuselage on the way).

The Whirlpool researcher explained how washing machine construction and assembly is virtualized. Even the packaging of the machine into the cardboard box is modeled. (Repackaging a washing machine is expensive, if the packaging is damaged.) Other parts of the assembly workflow appear to be modeled as well. The result is a significant cost savings to Whirlpool. They have one or two clusters at the company to get the job done.

I imagine the time is not far off when when this trend will play-out in the manufacturing of all kinds of equipment, if it hasn't already. I wonder what can't be virtualized? Can home construction be virtualized?

Maybe we will be able to build a virtual house in Second Life, and even live in it for a while and make adjustments and decide if we like it before we actually have it constructed by the builder.

Tuesday, November 13, 2007

SC07- Day 3 - Neil Gershenfeld Keynote on "Programming Bits and Atoms"

I arrived this morning at SC07 excited to see Professor Neil Gershenfeld's talk on "Programming Bits and Atoms." He has been featured on the Lehrer NewsHour, my favorite news program, so I figured he must be interesting.

I got out my laptop ready to take careful notes. After a somewhat lengthy introduction composed of many illustrious dignitaries as well overwrought, loud orchestral music, and a peculiar music video featuring SC07 organizers, Professor Gershenfeld arrived on stage.

What followed was either brilliant or insane; probably mostly the former, but a bit of the latter. I was unable to take any notes because mostly, I could not comprehend what he was saying. I closed my laptop and instead focused on spirit and essence of the talk.

I think the take away point is that software and hardware professional need to stop thinking about computers as boxes with screens and keyboards that you can use surf the web, etc. Instead, let's focus on all the wonderful things that Professor Gershenfeld talked about which included: molecular biology, nanotechnology, physical objects as computers, and many, many other topics.

His discussion on molecular biology was especially interesting to me as, long ago, I had a former career in a molecular biology lab. His thoughts on the ribosome as a sort of molecular Turing machine, and his attempts to mimic that mechanism in other mediums were certainly fascinating.

Apart from that, I can't say much more. My mind is still buzzing.

Monday, November 12, 2007

SC07- Day 1 & 2 - Manycore and Multicore Computing Workshop

I am attending SC07: "The International Conference on High Performance Computing, Networking, Storage, and Analysis." The conference is taking place in Reno, Nevada -- "The Biggest Little City in the World." The town appears to be a miniature version of Las Vegas with the omnipresent casinos and slot machines -- something I find a little depressing. Many of the casinos and the general infrastructure of the town seem to suffer from accelerated decrepitude. In contrast to Las Vegas, the town is surrounded by picturesque mountains, which are pleasantly snow capped from the precipitation Saturday night. If I had time and a car, I would go for a hike.

I attended the "Manycore and Multicore Computing" workshop Sunday and Monday of SC07. As an object-oriented applications developer, my main interest in this topic is to observe how multicore architectures are changing desktop and server-side computing and to learn how to best leverage these new types of machines for my own development.

The overall message is, ready or not, HPC and parallel programming have arrived in the desktop computing environment. Many of you are probably reading this post on a dual or quad core machine. What follows are a few points I took away from the workshop. Also, I use the terms multicore and manycore somewhat interchangeably since, as far as I can tell, the distinction is not really set in stone.


The chief motivation behind manufacturing multicore machines is that users always want more computing power. This is a self-evident truth, as judged by the last few decades of computing. Users not only want to run traditional applications faster (e.g. MS Powerpoint), they want to run entirely new applications not permitted by slower computers (e.g. the 3D Internet, molecular dynamics). Single processors have hit physical limits that do not allow for increased performance. If users want faster applications, programmers will have to design and code them for manycore/multicore architectures.


There are a couple of different varieties of manycore and multicore architectures. In particular, researchers distinguish between homogeneous (CPU only) versus heterogeneous hardware. Heterogeneous architectures derive their computational horsepower from CPUs and GPUs (Graphical Processing Units). GPUs are no longer just for graphics, but can also be leveraged for intensive floating-point arithmetic. GPUs have attractive power consumption characteristics as well. Moreover, my understanding is that GPUs have not hit the sorts of physical limitations in delivering performance that CPUs have.


Multicore architectures can deliver improved performance, but there are major caveats that impact hardware and the entire software stack.

At the hardware level, there are difficult issues as to how the multicore architectures will scale out to many cores. The number of cores is expected to double every two years. The main problems appear to be memory bandwidth and IO bottlenecks. Increased power demand by the hardware is another hurdle in the way of massively multicore architectures.

At the operating system level, scheduling algorithms will have to drastically change.

The impact on software developers will be major. In particular, programmers will have to build shared memory parallel algorithms and overcome the significant challenges that parallel programming entails. The shift to the multicores will be a sea change for most application developers (outside of HPC) who will now have to reason about thread-based, shared memory parallelism in addition to locks, synchronization, data races, and fine-grained parallelism, just to cite a few examples.

Abstractions & Frameworks

To deal with this new programming landscape, libraries, frameworks, and higher level programming abstractions must be developed in order to bring parallel computing to the masses. Building these abstractions, however, will take away some of the performance benefits that multicore architectures can theoretically achieve. But this disadvantage will be more than offset by the advantage of bringing parallel programming to a much wider audience.

On the topic of frameworks, one of the most interesting talks of the workshop was Charles E. Leiserson's introduction of the Cilk suite of products that enable programmers to transition to parallel programming for the multicores. It is essentially a fork-join (or “spawn-sync” in Cilk parlance) framework for C++ that allow programmers to easily express fine-grained parallelism. Under the hood, it employs a work stealing scheduler to dynamically balance workloads. Leiserson also suggested that the multicore/manycore architectures could spell the end for distributed memory parallel programming, notwithstanding significant architectural issues that must be overcome (e.g. memory bandwidth). Cilk reminded me a bit of what I heard concerning the forthcoming Java fork-join framework, which will be part of the java.util.concurrent package.

Next Generation Parallel Languages

The speakers talked a bit about X10, Fortress, and Chapel, the so-called HPCS (High-Productivity Computing System) languages in the context of multicore. X10 received most of the attention. The designers of the X10 language removed the concurrency constructs out of Java and replaced them with a new set of concurrency idioms that better assist programmers develop deadlock-free, scalable parallel code. Some of the speakers expressed skepticism concerning these languages as they are geared more towards homogeneous multicore architectures. The Habanero language builds upon X10 to deliver programmers a high level language that can harness heterogeneous architectures.

Overall, it was an interesting series of talks combining a good blend of visionary and technical speakers. By the end of the workshop on day 2 though, I was ready to return to the Siena Hotel and go for a jog on the banks of the Truckee River, which is not exactly the Seine to say the least.