Radio Eiffel

An Annotated AJAX (via JSON) Spring Controller

2009-03-31T21:31:00.001-07:00

Because I could not find an example of an annotated Spring AJAX (via JSON) controller in Google, I forged ahead and figured this one out on my own. I thought I would share this code snippet with the blogosphere. Without further ceremony here is the code example:


@Controller
public class AjaxController {
 @RequestMapping("/index")
     public void handler(@RequestParam("myParam") Object myParam, 
HttpServletResponse response) {
             JSONObject json = new JSONObject();
             json.putAll(getMyData(myParam));
             response.setContentType("application/json");
             response.setHeader("Cache-Control", "no-cache");
             response.getWriter().write(json.toJSONString());
       }
}

The URL request mapping is specified with the RequestMapping annotation. Make sure the path is accessible by properly configuring with web.xml with the correct servlet-mapping. Note that the annotated Spring controller offers tremendous flexibility with respect to the method signature. We can leverage this feature for building the AJAX request handler method. Moreover, you can specify the required URL parameters via the RequestParam annotation and bind them to the method arguments. Lastly, you will need the HttpServletResponse object for the JSON response. Notice that the method returns void because the JSON response is returned via the response object.

The JSON library I used can be found here.

Handle the JSON in the browser with your favorite JavaScript library (e.g. YUI).

Google Threw Me a Bone

2009-02-15T21:03:00.001-08:00

This past week I received a package from Google - 1600 Amphitheatre Parkway Mountain View, CA 94043. That's right, Google. I
was unaware that Google knew of the existence of the USPS, considering it a communication mechanism from the time of Neanderthals.

I opened it to reveal:

A Rubik's Cube
A mouse pad
A pen
A "My Other Computer is a Data Center" bumper sticker

All were emblazoned with the Google logo.

And a thank you note hand written with excellent penmanship. They sent me some Google schwag as a token of their appreciation for the contributions I have made to the Google Chart API -- both on the group list and the wrapper API I developed, charts4j .

There is no name to reveal who sent it.

At any rate, in the extremely unlikely possibility the person that sent this to me is reading this blog post (I think I am the only one who reads my blog, except for my wife), thanks for the schwag, especially the Rubik's cube.

What? Java Can't Sort Numbers?

2009-02-11T22:13:00.001-08:00

I was recently working on some charts4j code to sort a list of Numbers (java.lang.Numbers -- to be exact) and discovered that it could not be done simply by calling Collections.sort. I would think, if anything ought to be sortable in Java, it should be a list of Numbers.

As most Java programmers know, sorting lists in Java is easy -- sort of. "Simply" use the static Collections.sort method, which under the hood is implemented with the classic merge sort algorithm. The major caveat, of course, is that the class of objects you are sorting must implement the Comparable interface by having a properly written compareTo method. Fortunately, many classes in the core Java language already implement Comparable: String, Date, Integer. They can immediately be sorted with Collections.sort without any special effort from the programmer, but not for Numbers. I did a little research by posting a question on this topic on Stackoverflow. I received a lot of good albeit -- Sun party line -- answers. The issue is a bit more complex than one would immediately think.

The simplest explanation for why java.lang.Number does not implement Comparable is rooted in mutability concerns. For a bit of review, java.lang.Number is the abstract super-type of AtomicInteger, AtomicLong, BigDecimal, BigInteger, Byte, Double, Float, Integer, Long, Short. On that list, AtomicInteger and AtomicLong to do not implement Comparable. Digging around, I discovered that it is not a good practice to implement Comparable on mutable types because the objects can change during or after comparison rendering the result of the comparison useless. Both AtomicLong and AtomicInteger are mutable. The API designers had the forethought to not have Number implement Comparable because it would have constrained implementation of future subtypes. Indeed, AtomicLong and AtomicInteger were added in Java 1.5 long after java.lang.Number was initially implemented.

Apart from mutability, there are probably other considerations here too. A compareTo implementation in Number would have to promote all numeric values to BigDecimal because it is capable of accommodating all the Number sub-types. The implication of that promotion in terms of mathematics and performance is a bit unclear to me, but my intuition finds that solution kludgy.

A related issue to this discussion is autoboxing. In Java 1.5, Sun introduced this language feature to seamlessly go back and forth between primitive numeric types and reference types. This is useful for adding and getting numbers in and out of Collections, for example, without having to explicitly convert the primitive type to a reference type. The Java numeric primitive types are byte, char, short, int, float, and double. They all have corresponding reference types, Byte, Character, Short, etc. Those reference types all extend Number (except for Character). The rules for converting from primitive types to boxed reference types are difficult to anticipate for purposes of building APIs. (I thought Java 1.5 was supposed to be all about type safety.) For instance, Arrays.asList(1,2,3) yields a List<Integer> and not a List<Long>. Arrays.asList(1.0,2,3) produces a List<? extends Number> instead of a List<Double>. This lack of predictability is burdensome for API designers, because they essentially have to build multiple method signatures to accommodate multiple boxed numeric reference types. Overloading the method signature may not even be an option in the case of Collections since List<Integer> and List<? extends Number> have the same erasure. One possibility is to rely only on the Number type or in the case of Collections List<? extends Number> for polymorphism but the problem is that the Number type is too general, encompassing more than the Java boxed primitive types, and it does not implement Comparable.

A potential solution is to implement an intermediate RealNumber type that would extend Number and encompass all the boxed numeric primitive types (except for char). This approach would allow polymorphism on numeric primitive types via autoboxing. Moreover, it would be reasonable for this RealNumber type to implement Comparable by promoting primitive numeric types to double in the compareTo method.

Unfortunately, this is yet another example of how Java has never been great in the numerical computation arena.

The iPhone Changes the Game for Small Independent Software Vendors

2009-01-28T22:35:00.001-08:00

Over the years at the Boulder Java User Group, I have discovered many technologies that have eventually played an important role in my daily software development, YUI or YSlow, for example. Matthew McCullough's fascinating talk earlier this month on iPhone development may
eventually turn out to have been another influential moment.

For a bit of background, last year Apple finally opened up the iPhone platform to non-Apple developers. This sea change reveals a vast and relatively unexplored landscape for software developers where there are probably many good ideas ripe for the picking. In addition, iPhone's smart phone market share is large and growing rapidly potentially providing many customers for iPhone developers. The revenue model is also tantalizing; developers may sell their applications at the iTunes store -- an extremely high visibility marketplace. I know of no other high profile online store where small independent developers have relatively easy access to huge audiences. Of course, this scenario makes the rosy assumption that Apple will approve the application for sale at the iTunes store. Nevertheless, this possibility is better than the alternatives. Hopefully, we will be seeing more of these high visibility online software bazaars.

Contrast this situation to the web application arena, where it seems every good idea has been already explored, not to mention the fact that the competition is cut throat, and the revenue model usually depends on mega traffic, which is difficult if not almost impossible to attain. Overall, it remains extremely difficult to break through and get attention in the web application world.

As a consequence, Matt's BJUG talk certainly whet my appetite for iPhone development. The major caveat, of course, is developers such as myself must learn Cocoa and Objective-C, which is not a small feat. Back in 2003, I became interested in Objective-C and Cocoa development when OS X and the iPod were really becoming popular. I spent about a year tooling around during evenings and weekends learning Objective-C and Cocoa. While I was interested in this environment, nothing concrete ever materialized from my efforts and I moved on to other projects. At any rate, I am glad I did not toss those Cocoa and Objective-C books I purchased a few years ago. Indeed, I have been brushing up on my Cocoa as fast as I can.

Stackoverflow and the Future of Collaborative Media

2009-01-12T20:47:00.001-08:00

Stackoverflow is the collaborative media site founded by the illustrious duo, Joel Spolsky and Jeff Atwood. It makes a fresh and original attempt to improve the state of the online software development community, and does so mostly successfully. The site combines the best features and concepts from wikis, blogs, forums, and sites like Digg and Reddit (and perhaps a even a touch of Twitter). The result is an impressive collaborative space where developers can post questions, obtain quick answers and spark discussions.

To highlight just a few Stackoverflow features:

Use of Open ID. Account creation -- often the biggest barrier for adoption for collaborative sites -- is near frictionless by leveraging this up-and-coming authentication mechanism.

Voting. Higher quality answers and topics tend to filter upwards via a voting and point scheme.

An outstanding user interface, with plenty of rich interactivity that adds to the user experience instead of detracting from it.

High web traffic yields quick responses and usually decent quality answers and discussions.

Questions, answers, and comments (assuming you have the credentials) are editable which means mistakes are corrected over time.

Unobtrusive ads.

The engineering behind Stackoverflow must be outstanding. I have noticed very few odd quirks (apart from the occasional dirty cache) and it seems to be scaling fine. This is certainly more proof that the ASP.net stack is at least as good as anything offered in the Java arena, not to mention the fact that Jeff Atwood and friends are extraordinary developers. They designed, implemented, and had the site up and running in just a few months. I have been fantasizing about starting a knockoff site with the Java Spring stack which would theoretically be very doable as long as you had the right talent and time. (I know, easier said than done.)

Stackoverflow is not perfect, however. Question and answers can receive up and down votes, and users accumulate reputation points based on the vote count of their questions and answers. Moreover, users obtain badges as they use the system. It can become annoyingly addictive. This merit system is both an asset and a detriment. The right answers theoretically filter to the top, but Stackoverflow also biases quick answers over the best answers because the quick answer simply receives more views and points. This fact in turn incites users to provides rapid-fire answers over more thoughtful, measured responses resulting in a positive feedback loop. They may be able to partially fix these issues through some algorithmic / social engineering tweak. The reputation point system also means that users can become competitive, which leads to pettiness, incivility, and even borderline vigilantism at times. For exhibit A, note the comments on the legendary (fellow University of Colorado alum) Alan Kay's question. Possibly a better solution is to provide a relatively low ceiling on reputation points.

Searching has problems, as well, and I have even posted questions that already had answers despite a preliminary search for my question. Again, they will fix this issue eventually.

The quality of the discussion is not universally great, but generally adequate and certainly better than the alternatives. For example, this discussion on JavaFX technology partially misses the point that JavaFX is very much related and built on top of applet technology -- a fact I try to point out.

At any rate, the development community is responding favorably to Stackoverflow, and the number of users is growing sharply.

Stackoverflow is the brainchild of Joel Spolsky and Jeff Atwood (actually mostly Jeff, I suspect) of software blogging fame. This effort is another example of how successful bloggers have bootstrapped other facets of their careers by using their blogs as self-promotional springboards.

Stackoverflow's meteoric rise proves there is a real hunger for collaborative spaces that don't suck. I predict that they are going to start spinning off other "spaces," some perhaps having nothing to do with software. Why not have cinema.stackoverflow.com, gaming.stackoverflow.com, IT.stackoverflow.com? Better yet, make those spaces integrated. For instance, if someone asks a question on the IT forum that belongs in the gaming forum, there would be a mechanism to re-assign the question to a better space. The evolution of Stackoverflow into other domains will provide an invaluable service to many online communities whose forums currently stink. How many times have you looked for something on the Internet only to arrive at a forum where your question has been asked but with no answer? Or that you do not have the credentials to provide the answer and that obtaining a login is too onerous. And, of course, you are assaulted with ads along the way. I have encountered this scenario countless times, as have my colleagues. Stackoverflow's expansion into other domains will be a quantum leap in the evolution of collaborative media. An evolution that I will wholeheartedly welcome.

Incorporating Google Charts into a Spring MVC application

2008-12-15T10:16:00.001-08:00

The last blog post on interactively building charts with Clojure generated quite a bit of interest. As a continuation of the chart theme, I will describe how to incorporate professional quality charts into a Spring MVC web application by leveraging charts4j.

The Google Chart API potentially provides developers a lightweight mechanism to incorporate static and dynamic charts into a web application. Dealing with the Google Chart API directly, however, is tedious and error prone. Fortunately, charts4j, a lightweight wrapper to the Google Chart API, makes incorporating charts into your Spring web application much easier. To see examples of all the charts that can be generated with charts4j, see the chart gallery here.

See here to download the example code that this post describes. What follows below assumes some familiarity with how to build a Spring MVC web application.

To incorporate Google Charts into your Spring application, you will first need the standard Spring MVC infrastructure:

8 Spring framework jars and friends: commons-logging-api-1.1.jar, jstl.jar, spring-beans.jar, spring-context.jar, spring-core.jar, spring-web.jar, spring-webmvc.jar, standard.jar. Put these jars in the WEB-INF/lib directory.
3 Spring and servlet XML configuration files: applicationContext.xml, frontController-servlet.xml, web.xml. Put these XML files in the WEB-INF directory.

Plus, you will need to write one Spring Controller, one JSP. And most importantly, also grab the charts4j jar and put it in WEB-INF/lib.

Here is the controller code:

@Controller
public class ChartController {
    @RequestMapping("/index.html")
    public ModelAndView indexHandler() {
        //Your really great chart.
        final Plot plot = Plots.newPlot(Data.newData(0, 66.6, 33.3, 100));
        final LineChart chart = GCharts.newLineChart(plot);
        chart.setTitle("My Really Great Chart");
        //Passing the chart to the JSP for rendering.
        return new ModelAndView().addObject("url", chart.toURLString());
    }
}

Note that I am making use of the newer style annotation based Spring MVC controllers. Also, the line

final Plot plot = Plots.newPlot(Data.newData(0, 66.6, 33.3, 100));

has static, hard coded data, but the real power of this mechanism comes into play when the application dynamically generates data. For example, perhaps you would like to display up-to-date stock price or weather data from your database in your web application.

In addition, here is the JSP code:

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>
<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Your great chart</title>
</head>
<body>
<img src="${url}" alt="Chart" />
</body>
</html>

The only noteworthy lines of code in the JSP are highlighted. To display the chart in the JSP, use an HTML <img> tag with the URL generated in the controller: <img src="${url}" alt="Chart" />. Also note that I am using the JSTL EL to access the URL string.

Deploy your web application and that is it. You are done.

The strength of this solution is its small footprint and low impact on your code base. To generate nice charts in your Spring MVC application, all you need is one 160 kb charts4j jar, an
Internet connection, a bit of charting code and you are done. No need for special graphics libraries, extra servlet infrastructure, etc. There are many more chart examples with code samples here.

Finally, the Google Chart API and charts4j will satisfy many Java application charting requirements, but there are three reasons why you may not wish to choose this technology:

No Internet connection, no Google Charts. Your application must be connected to the Internet because the charts are ultimately rendered by the Google Chart API.
There is no security with these charts. Your data will be sent as clear text over the Internet, and it would be trivial for someone to intercept this information.
If you have lots of data, roughly speaking, more than 2000 data points, you should try to sample your data less frequently, and use the simple encoding scheme. If this is not a possibility, this technology may not be right for you.

This blog post has shown you how to incorporate charts4j in your Spring MVC web application, but there is no reason why you cannot also incorporate charts4j in any Java application environment, as long as the three requirements above are met. Moreover, this design pattern of generating charts in the servlet layer and rendering the chart URL in the JSP will work for any Java web application environment (e.g Struts, etc.)

Interactive Charting with Clojure

2008-12-02T22:08:00.001-08:00

I stumbled upon the Clojure programming language a few months ago when I was trolling around the Internet for topics relating to Java threading and concurrency. This Lisp and Scheme derived dynamic language runs on the JVM and has the ability to invoke Java APIs via the Clojure special form syntax. One of Clojure's main features is its strong support for concurrent programming, taking advantage of its bias toward immutability, and leveraging its own notion of software transactional memory in addition to invoking the java.util.concurrent API and the forthcoming fork-join framework.

At any rate, Clojure is dynamic and thus has an interactive shell or REPL. Because you can invoke Java APIs from the REPL, I thought it would be a fun exercise to leverage charts4j to build charts interactively. I created the charts above and below fairly easily, certainly much easier than in a Java main program because Clojure's dynamic environment provides instant feedback. You can find the Clojure script that generated these images here.

Simply load it with the following command: (load-file "charting.clj"), setting your path appropriately, of course. You will also need the charts4j jar in your Clojure classpath. Moreover, you must be connected to the Internet for this script to run, as the charts are ultimately rendered by the Google Chart API. It will play a slide show, and can be a starting point for generating your own charts interactively. It should be straightforward to generate all the charts available in charts4j by using that script as an example. One last important note about the charting.clj script; it works with the latest Clojure SVN revision as of the day of this blog post, but this language is still young, and the Clojure authors are still fiddling with the syntax. Therefore, I also provide a "charting-20080916.clj" that works with the official latest release.

If there is enough interest, I may write a Clojure library that would make this interactive charting environment a bit easier to use, for instance by predefining more chart functions. This library could even be combined into a Clojure tutorial as programmers often respond to visual cues and eye candy as they are hacking away at code.

For the Clojure experts out there, I would welcome feedback on my Clojure code.

gchartjava moves to charts4j

2008-11-18T21:08:00.001-08:00

When the Google Chart API was released almost a year ago, I saw an opportunity to start a relatively simple, useful open source project. I quickly designed and developed an early version of a Java wrapper API enabling developers to programmatically interact with the Google Chart API from Java. I knew I had to stake out my territory quickly because many developers would have the same idea. Therefore on a whim, I chose the "gchartjava" moniker -- an accurate, but completely clunky and meaningless name to anyone outside the narrow Google API community. Moreover, having "Java" in the name may violate Sun trademark. I reluctantly lived with "gchartjava" for several months until a user pointed out the trademark problems and suggested "gchart4j", a variation on the familiar "4j" suffix (e.g. log4j). That suggestion, however, still sounded too specific, so I simplified the name a bit to arrive at "charts4j" which is apparently unique in the universal namespace. The new charts4j moniker will simply and accurately describe its purpose -- a chart API for Java developers-- but will also give it a ring of familiarity with the 4j suffix.

The cost of this move will not be cheap. I will have to spend some time rebranding, and who knows what will happen to my PageRank. But in the long run, it will cast a wider net than just trying to appeal to Google API users and developers.

The new release candidate of gchartjava

2008-10-24T19:37:00.001-07:00

I have been on blogging hiatus since last winter because I was busy at work on gchartjava.

This project has grown to 20,000 lines of code and 150 classes (including many junit test classes). The end product is still relatively small--a 161Kb jar for producing great charts and graphs with 100% pure core Java. I have received great feedback from many users and developers. There have been about 850 downloads of the distribution in 4.5 months.

This latest release candidate version of gchartjava now finally exposes nearly all the features of the Google Chart API, or at least I thought so until I saw the recent announcement about some new chart types. That is one of the problems maintaining an API; you are always playing catchup with the underlying technology. At any rate, I hope to have those chart types available in gchartjava soon, but probably not in this release.

This project has been an invaluable lesson in many aspects of software development. To name just a few:

Object design
API design
Collaborating with users and developers around the world
Managing an OSS project
Swallowing your pride when a user points out you have done something silly
Not leaving javadoc documentation to the last minute
Not leaving static analysis to the last minute

I hope to be able to blog in greater detail about some of these issues in the future. There should be more time in my schedule, now that gchartjava is mostly complete.

High Performance Web Sites and Tomcat

2008-02-24T20:28:00.000-08:00

I recently attended a Boulder Java User Group meeting, at which Scott Davis pointed us to YSlow, a website performance analysis tool. It is an add-on to the Firebug plug-in. When analyzing a web site, YSlow takes into account 10-15 criteria, and gives a grade.

As an aside, this is exactly why I attend the BJUG. I often pick up information that is instantly useful in my day-to-day software development. Another gem I recently discovered at BJUG is YUI, which I now use extensively both at work and for my own personal projects. YUI is to front-end development as Spring is to middle-tier development. In particular, YUI makes the life of the developer much easier.

At any rate, I analyzed my own web site, weathermole.com, with YSlow, and was given an 'F' for performance. Here is the full analysis:

Performance Grade: F (58)

B 1. Make fewer HTTP requests
F 2. Use a CDN
F 3. Add an Expires header
F 4. Gzip components
A 5. Put CSS at the top
A 6. Put JS at the bottom
A 7. Avoid CSS expressions
n/a 8. Make JS and CSS external
D 9. Reduce DNS lookups
A 10. Minify JS
A 11. Avoid redirects
A 12. Remove duplicate scripts
F 13. Configure ETags

To improve this grade, I have been focusing on 3,4, and 13, all of which require manipulation of the Apache HTTP Server's httpd.conf file.

The problem that I am encountering is I actually use Tomcat to serve all my web content including static resources (e.g. javascript, CSS files). I only use Apache HTTP Server as a proxy server to forward web traffic to Tomcat. My attempts at manipulating the httpd.conf file yield nothing. For example, when I insert these lines into the httpd.conf file:


<ifmodule mod_expires.c>
   <FilesMatch "\.(png|css|js)$">
        ExpiresActive on
        ExpiresDefault "access plus 10 year"
    </FilesMatch>
 </ifmodule>

it does not add any expires header to my CSS or javascript files. You can see this for yourself if you have Firebug and YSlow installed.

Perhaps there is some other explanation, but I presume this is happening because I am only using Apache HTTP Server as a proxy. All header information is actually supplied by Tomcat.

A possible solution is to have the Apache HTTP Server serve all my static resources, and have Tomcat only serve content relevant to the web application. The problem with this approach is that it makes deployment a real pain in the neck. I have to split my deployment between the HTTP Server and Tomcat. Moreover, I would also have to figure out how to selectively forward requests to Tomcat depending on what is requested (i.e. static versus dynamic content). I do not know how difficult this is to implement since I have not tried, but it just feels convoluted and kludgy. It would probably require a fair amount of trial and error mucking with the httpd.conf file.

Another potential solution is to programmatically add these headers within my web application, but this approach is somewhat absurd.

So my question is, is there a simpler solution to improving weathermole's performance?

Internet Radio finally coming to the iPod and iPhone?

2008-01-30T19:47:00.000-08:00

Macworld took place recently, and I was hoping to see an announcement from Steve Jobs explaining how the iPod Touch or iPhone can now play Internet radio streams via WiFi. For some background, I have written in the past about my love hate relationship with my Roku Internet radio. In sum, the Roku is a wonderful device for playing digital music libraries (served via iTunes or Firefly Media Server) and listening to thousands of Internet radio streams. Moreover, the sound quality is excellent. But the Roku suffers from the following big problems:

1/ Reliability. My Roku has died twice in one year. Thankfully it was still under warranty so I could replace it. Other Roku owners were not so lucky, and the Roku newgroups are full of people complaining about their Rokus dying after the warranty ran out.

2/ User Interface. The UI is mediocre at best, and at times terrible. This does not matter too much when selecting my favorite Internet radio station or changing the volume. But the UI is awful for browsing large music collections via the remote control. It is a slow, awkward and painful interaction. The user experience is incomparable to the wonderful album browsing UI available via iTunes.

3/ The wireless connection between the Roku and the MP3 music server is unreliable. It dies all the time, and I must reset it by restarting iTunes or the Firefly Media Server. This could be a networking problem too, and thus not the Roku's fault, strictly speaking.

4/ The Roku cannot play Apple's proprietary music format, so I cannot listen to music purchased at the iTunes store via the Roku.

Because of these problems I have been looking for Roku alternatives, and I have been hoping to see a solution from Apple via the iPod Touch or iPhone. Of course, these devices can play music and video digital content. Until recently, however, there has been nothing to support Internet radio streams.

So this week I have been surfing around the tech news site to see what Steve Jobs had to say at Macworld. Initially, it was looking like I was going to be dissapointed again. But then I stumbled upon flytunes.

I presume the iPod/iPhone platform must have been opened up to third party developers along the way. At any rate, flytunes (or something comparable) may be part of my dream solution of having a reliable device that can play both Internet radio and music libraries with a great user interface. I would also need an auxiliary device like this. Moreover, I would like to see more storage capacity on the iPod Touch or iPhone to hold my entire (20GB+) music collection.

This solution is a lot of money so I won't be giving up the Roku anytime soon (unless, of course it dies again), but it is nice to know that this technology has finally arrived.

OS X Tip: Use the /tmp Folder as a Scratch Directory.

2007-12-31T12:28:00.000-08:00

Here is a tip for dealing with Mac desktop clutter. But first, this tip comes with the following disclaimer:

!!!WARNING. ANY FILES YOU PUT IN THE /tmp DIRECTORY WILL BE AUTOMATICALLY EXPUNGED (I.E. PERMANENTLY DELETED/ERASED) EACH TIME YOU REBOOT OR SHUTDOWN YOUR COMPUTER. DO NOT PUT IMPORTANT FILES IN THE /tmp FOLDER!!!

With that warning out of the way, to reduce the number of files that end up on your desktop, create a soft link between your Desktop directory and the OS X /tmp directory. To do that, open the Terminal application (Applications --> Utilities), and type the following 2 commands:

cd ~/Desktop
ln -s /tmp/ tmp

These commands will result in a tmp folder visible on your desktop that corresponds to the /tmp directory. Then, add this directory to your finder sidebar by opening a finder window and dragging the folder to the sidebar.

Each time you need a temporary scratch space, you can easily access this tmp directory via the finder or the "Save" or "Save as" menu item in any application. Uninteresting files will conveniently go away whenever you shutdown or reboot your computer.

Moreover, you can use this directory to save downloads coming from your web browser. This is useful for dealing with PDFs that, over time, result desktop clutter and you don't want lying around anyway. To change where your browser saves downloaded files, go to the either the "Main" tab of the Firefox preferences dialog or the "General" tab of the Safari preferences dialog.

If the file is interesting to you, be sure to move it to a permanent location outside the /tmp directory.

Google Releases Chart API and gchartjava Wrapper API

2007-12-11T21:15:00.000-08:00

In the earth science arena, programmatically displaying charts and graphs is often a user requirement. The problem is that building charting software is not easy. To have those tick marks just right and produce a graph that looks better than amateurish can consume 100% of a good programmer's time for weeks and months. Alternatively, there are charting libraries, but as the old saying goes, "it is either free or it works." Visual Numerics looks like a decent product, but it's not free. JFreeChart looks pretty good, although you have to purchase the documentation.

It was therefore with great interest that I read that Google recently released a Chart API. I have been playing around with it , and I am mostly impressed. I wish I had more control over the orientation of the text along side the axes. Maybe that feature is forthcoming. I have not quite figured out why they released it as a service instead of a library that programmers can download. But I guess that is where Google and the world is heading, where even "medium-grained" capabilities will be offered as services. On a related note, if you ever have trouble explaining to a colleague what Service Oriented Architecture is, this is a great example of SOA technology.

Back to scientific programming, it is easy to push the limits of any charting software when dealing with weather or climate data, but for straightforward data, the Google charting API could be a great solution.

The next logical step is to define wrapper APIs in Java, C#, Python, or Ruby around the Google Chart API. The reason it is useful to wrap the GChart API with Java/ Ruby/.Net/ Python is that it allows the developer to interact with the GChart API *programmatically.* Imagine this use case: a sensor monitors traffic patterns on a busy highway and displays the data in a chart on a web page. In this case, it is nice to have your business logic query the traffic data and build those charts dynamically. Subsequently, those charts are forwarded to your (web-tier) view, for example. Also, a wrapper hides the ugly details of making and chaining the URL GET parameters.

I have taken a first stab at buildng a Java wrapper for Google Charts and you can find it here.

This wrapper API is still very much in flux, but should be solidifying soon.

Oh, and kudos to Google for a job well done. They really do live up to their reputation.

SC07- Day 3 - HPC in the Kitchen and Laundry Room

2007-11-14T20:18:00.000-08:00

After spending some time on the exhibitor floor checking out the cool Second Life demo at the Sun booth, I decided to walk over to see the "HPC in the Kitchen and Laundry Room" talk.

It's interesting to note how manufacturing is becoming virtualized at all levels. I saw a piece on the NewsHour a few months ago on how companies like CATIA and Dassault build aircrafts virtually before they assemble the real thing. This strategy allows them to find all sorts of assembly bugs that they would not be able to discover otherwise (e.g. if you attach the wing to the aircraft in that manner, it is going to smash into the fuselage on the way).

The Whirlpool researcher explained how washing machine construction and assembly is virtualized. Even the packaging of the machine into the cardboard box is modeled. (Repackaging a washing machine is expensive, if the packaging is damaged.) Other parts of the assembly workflow appear to be modeled as well. The result is a significant cost savings to Whirlpool. They have one or two clusters at the company to get the job done.

I imagine the time is not far off when when this trend will play-out in the manufacturing of all kinds of equipment, if it hasn't already. I wonder what can't be virtualized? Can home construction be virtualized?

Maybe we will be able to build a virtual house in Second Life, and even live in it for a while and make adjustments and decide if we like it before we actually have it constructed by the builder.

SC07- Day 3 - Neil Gershenfeld Keynote on "Programming Bits and Atoms"

2007-11-13T15:16:00.000-08:00

I arrived this morning at SC07 excited to see Professor Neil Gershenfeld's talk on "Programming Bits and Atoms." He has been featured on the Lehrer NewsHour, my favorite news program, so I figured he must be interesting.

I got out my laptop ready to take careful notes. After a somewhat lengthy introduction composed of many illustrious dignitaries as well overwrought, loud orchestral music, and a peculiar music video featuring SC07 organizers, Professor Gershenfeld arrived on stage.

What followed was either brilliant or insane; probably mostly the former, but a bit of the latter. I was unable to take any notes because mostly, I could not comprehend what he was saying. I closed my laptop and instead focused on spirit and essence of the talk.

I think the take away point is that software and hardware professional need to stop thinking about computers as boxes with screens and keyboards that you can use surf the web, etc. Instead, let's focus on all the wonderful things that Professor Gershenfeld talked about which included: molecular biology, nanotechnology, physical objects as computers, and many, many other topics.

His discussion on molecular biology was especially interesting to me as, long ago, I had a former career in a molecular biology lab. His thoughts on the ribosome as a sort of molecular Turing machine, and his attempts to mimic that mechanism in other mediums were certainly fascinating.

Apart from that, I can't say much more. My mind is still buzzing.

SC07- Day 1 & 2 - Manycore and Multicore Computing Workshop

2007-11-12T21:57:00.000-08:00

I am attending SC07: "The International Conference on High Performance Computing, Networking, Storage, and Analysis." The conference is taking place in Reno, Nevada -- "The Biggest Little City in the World." The town appears to be a miniature version of Las Vegas with the omnipresent casinos and slot machines -- something I find a little depressing. Many of the casinos and the general infrastructure of the town seem to suffer from accelerated decrepitude. In contrast to Las Vegas, the town is surrounded by picturesque mountains, which are pleasantly snow capped from the precipitation Saturday night. If I had time and a car, I would go for a hike.

I attended the "Manycore and Multicore Computing" workshop Sunday and Monday of SC07. As an object-oriented applications developer, my main interest in this topic is to observe how multicore architectures are changing desktop and server-side computing and to learn how to best leverage these new types of machines for my own development.

The overall message is, ready or not, HPC and parallel programming have arrived in the desktop computing environment. Many of you are probably reading this post on a dual or quad core machine. What follows are a few points I took away from the workshop. Also, I use the terms multicore and manycore somewhat interchangeably since, as far as I can tell, the distinction is not really set in stone.

Motivation

The chief motivation behind manufacturing multicore machines is that users always want more computing power. This is a self-evident truth, as judged by the last few decades of computing. Users not only want to run traditional applications faster (e.g. MS Powerpoint), they want to run entirely new applications not permitted by slower computers (e.g. the 3D Internet, molecular dynamics). Single processors have hit physical limits that do not allow for increased performance. If users want faster applications, programmers will have to design and code them for manycore/multicore architectures.

Flavors

There are a couple of different varieties of manycore and multicore architectures. In particular, researchers distinguish between homogeneous (CPU only) versus heterogeneous hardware. Heterogeneous architectures derive their computational horsepower from CPUs and GPUs (Graphical Processing Units). GPUs are no longer just for graphics, but can also be leveraged for intensive floating-point arithmetic. GPUs have attractive power consumption characteristics as well. Moreover, my understanding is that GPUs have not hit the sorts of physical limitations in delivering performance that CPUs have.

Caveats

Multicore architectures can deliver improved performance, but there are major caveats that impact hardware and the entire software stack.

At the hardware level, there are difficult issues as to how the multicore architectures will scale out to many cores. The number of cores is expected to double every two years. The main problems appear to be memory bandwidth and IO bottlenecks. Increased power demand by the hardware is another hurdle in the way of massively multicore architectures.

At the operating system level, scheduling algorithms will have to drastically change.

The impact on software developers will be major. In particular, programmers will have to build shared memory parallel algorithms and overcome the significant challenges that parallel programming entails. The shift to the multicores will be a sea change for most application developers (outside of HPC) who will now have to reason about thread-based, shared memory parallelism in addition to locks, synchronization, data races, and fine-grained parallelism, just to cite a few examples.

Abstractions & Frameworks

To deal with this new programming landscape, libraries, frameworks, and higher level programming abstractions must be developed in order to bring parallel computing to the masses. Building these abstractions, however, will take away some of the performance benefits that multicore architectures can theoretically achieve. But this disadvantage will be more than offset by the advantage of bringing parallel programming to a much wider audience.

On the topic of frameworks, one of the most interesting talks of the workshop was Charles E. Leiserson's introduction of the Cilk suite of products that enable programmers to transition to parallel programming for the multicores. It is essentially a fork-join (or “spawn-sync” in Cilk parlance) framework for C++ that allow programmers to easily express fine-grained parallelism. Under the hood, it employs a work stealing scheduler to dynamically balance workloads. Leiserson also suggested that the multicore/manycore architectures could spell the end for distributed memory parallel programming, notwithstanding significant architectural issues that must be overcome (e.g. memory bandwidth). Cilk reminded me a bit of what I heard concerning the forthcoming Java fork-join framework, which will be part of the java.util.concurrent package.

Next Generation Parallel Languages

The speakers talked a bit about X10, Fortress, and Chapel, the so-called HPCS (High-Productivity Computing System) languages in the context of multicore. X10 received most of the attention. The designers of the X10 language removed the concurrency constructs out of Java and replaced them with a new set of concurrency idioms that better assist programmers develop deadlock-free, scalable parallel code. Some of the speakers expressed skepticism concerning these languages as they are geared more towards homogeneous multicore architectures. The Habanero language builds upon X10 to deliver programmers a high level language that can harness heterogeneous architectures.

Overall, it was an interesting series of talks combining a good blend of visionary and technical speakers. By the end of the workshop on day 2 though, I was ready to return to the Siena Hotel and go for a jog on the banks of the Truckee River, which is not exactly the Seine to say the least.

My Take on the Rockies Ticket Fiasco

2007-10-29T20:49:00.000-07:00

Before I mothball all my baseball thoughts until next Spring, I want to state the following completely unsubstantiated assertions and speculative conjectures about the World Series ticket fiasco:

This was a Java application, evidenced by the fact that users were hitting JSPs. Well built Java applications a very scalable, but this one was not.
They experienced a software and not hardware failure. It was a possible to access the site, but making progress was almost impossible.
8 million hits in 90 minutes is a lot, but can be handled by a well-designed scalable system.
They were probably not the victim of a malicious attack. Hackers usually demand ransoms these days. I did not hear anything about a ransom in the news. Staging a denial of service attack "just for fun" does not ring true and would require substantial resources.
There was no "online queue". When users entered the site, the window claimed that they were given a place in line, and said not to reload, or they'd lose their place in line. I was thinking about how to implement this, and in theory it can be done with session IDs, but would be difficult to implement fairly and correctly. It may not be possible at all, and I know of no online queues like this. Users were better off hitting reload constantly which is what I did in about 20 Safari web browser tabs. I was actually able to make progress but not enough to buy tickets.
Users that were closer to the servers in California with fast Internet had an advantage over users in Colorado. My wife's cousin was able to buy eight tickets without any difficulties from California, and mysteriously also easily surpassed the four ticket limit.

Next time the Rockies make it to the World Series, they ought follow the Red Sox's example and use a lottery system. It is fairer and kinder to fans.

Oh and by the way, congratulations to the Rockies. They didn't win the World Series, but they had a much better year than anyone expected.

Book Recommendation: Java Generics and Collections

2007-10-04T20:54:00.000-07:00

I recently wrote a blog entry about reluctantly accepting Java Generics. If you are planning on genericising your Java code (which is in principle inevitable for all Java programmers as Sun recommends against using raw types, and may not support them in the future), then "Java Generics and Collections" by Maurice Naftalin, and Philip Wadler is the book for you. It is the clearest, most coherent treatise I have read on the subject. Java Generics make a lot more sense after reading this book.

Declarations like

List<? extends XYZ> list = new LinkedList<XYZ>();

and notions like covariance finally make sense. I even find using wildcards useful since they give programmers some indication as to the purpose of the data structure in addition to providing type safety.

The "Get and Put Principle" in section (2.4) is a real gem. To quote Java Generics and Collections:

The Get and Put Principle: use an extends wildcard when you only get values out of a structure, use super wildcard when you only put values into a structure, and don't use a wildcard when you both get and put.

They also give some best practices on dealing with non-generic legacy libraries. This is particularly useful information, as we are probably a long way from seeing Apache project libraries moving to generics.

(I still wonder about the cost benefit ratio adding so much complexity to the language, but let's ignore that for now.)

There are other resources out there for Java Generics, but they tend to be insufficient or a bit rote like Angelika Langer's huge FAQ. I would actually recommend reading Java Generics first, and if you have questions left over, consult the FAQ.

Moreover, there is a section on the Collections Framework interface that reviews the major Java core data structures (Sets, Queues, Lists, and Maps). This type of material is really great to know. Java programmers tend to gravitate 99% to ArrayList and HashMap, but there are a lot of other fine data structures available in the core language that may be more appropriate for your requirements. Java Generics and Collections covers them all.

There are a few obnoxious typos, such as the broken Function Pattern code. This is confirmed in the errata.

Regardless, this is the best programming book I have read since "Java Concurrency in Practice."

iTunes over NFS: A Failed Experiment

2007-09-11T20:55:00.000-07:00

I recently wrote about my home network and music library issues. To review, I have a headless Mac mini that lives under my bed. It holds our 20GB music library. It serves music to the Roku and various Macs around the house (via shared iTunes libraries). It is on 24/7, so it always ready to serve music. This solution mostly works, but the problem is that interfacing with the Mac mini (via Chicken of the VNC) is a pain since it is just a headless server. For scanning CDs and syncing an iPod it is not a workable solution.

As a workaround, I set up an NFS mount, as described here, between the iTunes Music folder on the laptops and the iTunes Music directory on the Mac mini. My aim was to solve the CD scanning and iPod syncing problem by fooling iTunes on my laptop into thinking that it was dealing with a local file system when in reality it was accessing the server Music directory. For a while, this harebrained solution actually seemed to work. I could play music with no apparent lag. Syncing the iPod was slow, but tolerable. Things quickly went downhill, however. To have this solution work, I had to muck with the Music folder file permissions, which iTunes and the Mac mini did not seem to like. (Maybe the NFS mount was set up wrong with respect to permissions, but ultimately this was the least of my problems.) Strange temp and "Damaged" files starting showing up in the Music directory in large numbers. It would not function reliably, with the iTunes library dropping out and the file permissions confusing iTunes.

In retrospect my strategy was doomed to fail. I was expecting, without realizing it, to have the iTunes file system function as a transactional music server able to handle several client iTunes. iTunes is not a music server in that sense, capable of handling several clients at once. There are perhaps other workarounds, such as making sure only one iTunes client is accessing the library at once, but that is getting too kludgy. (I know iTunes can view other iTunes on the same network via iTunes shared libraries, but, again, I want to scan CDs and sync an iPod to a remote library.)

In the end, I copied all 20GBs of music over to one of our laptops. We now use that computer for syncing iPods, and unfortunately I am stuck with having to maintain two music libraries at once; one for serving music, and another for syncing the iPod and scanning CDs. It was a frustrating experience, but the last thing I did was listen to some Brian Eno and Nick Drake, which reminded me how much I like my music collection, and put me in a good mood again.

On a related note, Apple recently released a WiFi iPod. If this device could connect to Internet radio (which as far as I can tell, it can't), this would be the ultimate solution for me, coupled with something like this.

OSS, You Get What You Pay For.

2007-09-01T18:49:00.000-07:00

Like many developers, I have mixed feelings about open source software (OSS). It is a nice idea to leverage (and even, in the spirit of OSS, contribute back) to other people's work and harness their code to achieve your goals. This strategy has led to some phenomenal success stories. Several software companies that recently experienced meteoric success (such as Google and YouTube) use open source software to run their core business processes. Evidently, some flavors of OSS work in real economic terms.

This fact, however, is somewhat ironic because a lot of open source software just plain sucks. This paradox was brought to my attention recently when I was migrating from Tomcat 5.5 to 6.

For one thing, the directory structure was reorganized so I did not know where the external jars (e.g. mysql and log4j) lived. In the end, figured out where to place the jars as described here. This portion of the upgrade was not too painful. The logic to these directory structure changes is unclear to me.

Incorporating log4j was the most unpleasant experience. I don't know about you, but when I read crummy documentation with spelling and grammar mistakes (e.g. "van be very", "teh Tomcat" ) and ambiguous directions, it does not give me much confidence about the internals of the application. With that thought in mind, I downloaded the Tomcat source code. It took me less than a minute to find code that looked like this:

try {
//Code, code, code
}
catch (Throwable t) {
return (null);
}

This is a good example of how not to write Java. To see why, consult Chapter 8 from "Effective Java" (items 39-47). In addition, the Java language Specification (JLS 11.2.4) states that catching Errors (Throwable is the parent of Exception and Error) is not recommended.

Moreover, it is conventional Java wisdom to not swallow exceptions. Technically, here, the exception was not totally swallowed, but it certainly lost much of its information. The details about what exactly went wrong and where are lost forever. All the caller receives is a measly null. (What's up with those superfluous parentheses around the null?)

With respect to proper exception handling, I would add, catch the most specific exception possible and try to handle the problem in an intelligent fashion. Clean up resources, log errors, notify the user in a friendly way that something went wrong and do stuff that is appropriate to your application logic where something unexpected has happened.

If you can't recover from the exception (which is often the case), rethrow it or better yet, throw a new (possibly runtime) exception that is appropriate to the level of abstraction at that point in the code (EJ Item 43). Only throw checked exceptions if the caller can actually recover from the problem (EJ Item 41).

So why is it that Tomcat is so popular in spite of poor coding standards? Because, to my great amazement, it works! I have been using it for some time, relatively problem free. I am not pushing it to its limits. If I were, maybe I would start seeing unusual problems -- probably obfuscated by the exception handling mechanism.

The only explanation I can think of for this sad state of affairs is that with so large a userbase, many of the bugs have been eliminated through a brute force approach of trial and error programming. I find trial and error coding particularly odious, but I guess if you have enough testers out there, OSS developers can create an application that is relatively stable despite terrible coding.

But if the application is ported to a new environment or something unusual goes wrong, good luck trying to find the source of any problems. Sometimes you really do get what you pay for.

How Java Static Fields (AKA Class Variables) Break Thread Confinement and Lead to Concurrency Bugs.

2007-08-17T20:40:00.000-07:00

Java static fields are shared between all instances of a class for a given class loader. One common use of static fields is to declare constants via the static final syntax.

Unfortunately, static final fields don’t ensure that the variable will be constant or immutable. It only guarantees that the field will be shared between all instances of the class for a given class loader (i.e. static) and that, apart from special cases involving reflection and deserialization (see JLS v3 17.5.3), it will not be reinitialized (i.e. final). In particular, there are two situations to consider corresponding to the major type categories in Java; primitive types (e.g. int, boolean, etc.) and reference types (e.g. arrays, interfaces, and classes).

Once a primitive is declared static final and it is assigned a value, there is no need to worry about it anymore. It will never change (except in unusual circumstances, see JLS v3 17.5.3).

For reference types, this is a different story. If the static final reference type is a mutable class (e.g. HashMap), it cannot be reinitialized, but the instance of that class can change after construction via the HashMap put method, for example.

Listing 1

public class Foobar{
//Mutable class variable.
private static final Map mutableField = new HashMap();
public Foobar() {}
public void qwerty(){
// Code to mutate the mutableField via put method.
}
}

And therein that static mutable field lies our thread safety concerns. If there are a couple threads floating around both using instances of that same class via the qwerty method from listing 1 for example, they can step on each other via their mutable class variable. Worse, the static mutable can be sitting "far away" from the programmer’s code in an open source library and a multi-threaded application can indirectly hammer on that static mutable possibly leaving it inconsistent and broken. The static final class variable can even be private. If it is affected by the instances of the class from multi-threaded application, it can be subject to thread safety concerns.

Even worse still, there are many ways to refactor the class referenced by static field to be immutable thereby achieving thread safety, but there is no way to make the contents of an array type immutable in Java. (This is yet another reason to consider arrays a deprecated type -- see O’Reilly’s Java Generics 6.9.)

This is not an academic point. A common way of ensuring thread safety is to have objects visible only from one thread. This is known as thread confinement and is common to client-server request-based architectures (see CPJ 2.3). The problem is that static fields break thread confinement and encapsulation because the same class variable can be accessed from multiple instances of the class, and if it is mutable is subject to race conditions in a multi-threaded environment.

This discussion may seem obvious, but I do not see this fact explicitly stated in either Java Concurrency in Practice or Concurrency in Java. Don’t get me wrong. These books are masterpieces of modern software engineering literature and I have immense admiration and respect for the authors, but I did not see any concrete mention of this danger with static mutable variables. In fact, to fix these problems I would recommend reading these books. Describing solutions and workarounds would definitely go beyond the scope of this blog entry.

Once again, as detailed in Java Concurrency in Practice, it is the shared mutable field that endangers thread safety. But in the case of static fields, the shared mutable is shared via the static modifier rather than one instance of the class shared by multiple threads.

My Home Network Joys and Woes

2007-07-25T20:51:00.000-07:00

Qwest DSL + foreThought.net ISP

I have a Qwest DSL line coming into the condo. My ISP is through foreThought.net. They gave me a dedicated IP address from which I serve a few different web sites (e.g. weathermole.com). I have no complaints here. I have experienced few service interruptions, and the guys at foreThought are responsive whenever I have a question or an issue arises.

Zoom X6 Gateway

This product claims to be an "all-in-one" solution. It has a wireless-G access point, router, and 4-port switch. I have a headless Mac Mini plugged into one of the ports. The Zoom X6 has a little embedded web server inside that users can configure the modem, set up the firewall, block ports, etc. I purchased it because it was recommended by my ISP and they know how to work with Zoom modems.

I was not crazy about this product after I purchased it. I found it hard to configure, and it froze up regularly so I would have to reboot often. After installing a bunch of firmware updates throughout the months, it seems to be doing better. I only rarely have to reboot it. The folks at foreThought originally helped me set it up shortly after I purchased it. I have since made a million screen shots of the configuration so that I will hopefully never have problems with the setup. I do not know if I would recommend it, but it basically works at this point.

Headless Mac Mini

On the Mac Mini I host a few websites (e.g. weathermole.com) and serve iTunes for various devices around the house.

I think this is a clever solution not to have a keyboard and monitor, and it does not take up much room in our tiny condo; it sits under our bed. But the fact that it is headless makes it hard to deal with at times. I connect to it via ssh to control it remotely with the command line. For desktop UI applications -- basically just iTunes-- I use Chicken of the VNC, but it is somewhat slow and kludgy. Moreover, I am always nervous during a software update that it is not going to come back up, and I do not have a spare monitor/keyboard lying around to see what is going on if that ever happens. When I originally set it up, I brought it to work and plugged it into a monitor/keyboard there to do the initial configuration. As Mac users know, a lot functionality on a Mac needs to be accessed via a user interface.

Also, my wife and I both have iPods. It is really inconvenient when we have to them sync them up. Again, it sits under a bed. I have to control iTunes via Chicken of the VNC. This aspect is pretty lame. If I ever have the funds to upgrade, the next machine will not be headless.

In the meantime, I thought about setting up a cross-mount (over the wireless) and having the iTunes directory visible on the laptops, as if it were local. I tried this plan half-heartedly , but I never had any success, and in the end, I am not sure it would work. It would nicely solve the iPod syncing problem if it did.

The headless Mac Mini is a fun but imperfect solution.

UPS

I have one of the cheaper varieties that supply uninterruptible power to the Zoom gateway and Mac Mini at least for a few minutes. We have had power interruptions so it has served its purpose.

Roku Soundbridge Radio

I have a love / hate relationship with this product. It seems to have limitless potential for serving music and Internet radio. My favorite local stations are KUNC, KUVO, KGNU -- which I could, of course, listen to with good old-fashioned technology. On the Internet we listen to WeFunk and Radio Paradise. For foreign stations, I like to listen to France Culture, Radio France Internationale, Europe 1. Plus, it is a client for our gazillions of scanned CDs that reside on the Mac Mini. My wife used to be a music writer so we have a lot of CDs.

This is all great, but I find the user interface of the Roku to be a pain in the neck. It is really hard to navigate and find a CD or podcast I want to listen to via the remote control and the small screen on the Roku. For this reason, we still use a regular CD player from time to time. Ideally, it would be great to have a full fledged iTunes interface, but without the computer. This could almost be achieved through an iPod that we would carry around the house and plug into various iPod-ready radios. The iPod's interface is far superior to that of the Roku. But, iPods cannot stream Internet radio, so that solution is out.

Apple TV could possibly work, but I want a portable solution that I can carry around the house.

It is theoretically possible that an iPhone could solve all these problems, assuming that you could somehow use the iPhone as a middle tier to the various audio devices sitting around the house. Can the iPhone iTunes connect to to Internet radio? Can you purchase an iPhone without a phone plan? Maybe there is a business opportunity in here somewhere.

Still, the Roku remains one of my favorite toys in the house.

MacBooks

I also have a couple of Macbooks which are both really great and naturally hook into the wireless network.

Other networking problems...

When I am on my local wireless network, for various reasons, I would really like to see my websites through the regular web address (i.e. weathermole.com NOT 10.0.0.10, the local address). I think that I would have to muck with a configuration file somewhere on the client machine, but I do not know which one.

Sea Change Coming for Scientific High Performance Computing Languages?

2007-07-13T20:35:00.000-07:00

I work at an institution where scientists and engineers are busy predicting the weather and the Earth's climate in decades to come. These predictions are done by simulations on very specialized super-computers and high performance computing (HPC) platforms such as the IBM Blue Gene machines.

Unfortunately, there has not been a lot of modern programming language support to take advantage of these parallel processing behemoths, mainly because of their highly specialized and narrow use in the fields of climate and weather forecasting, nuclear reaction simulations, etc. The languages that are supported are typically Fortran, C, and C++.

As a younger generation software engineer, I am a fan of more modern programming languages such as Java, C#, Python, etc., so the prospect of developing software on HPC platforms has not always been my cup of tea. Those old-school languages seem to lack many features that make life easier for programmers -- e.g. virtual machines to address portability concerns, simpler builds, more developer buy-in leading to rich community support etc. Moreover, as far as Fortran is concerned, there does not seem to be much in terms language support such as numerous open source libraries or sophisticated IDEs. Plus, I prefer modern OO languages for their notions of encapsulation and polymorphism that, if done correctly, ultimately reduce complexity and let programmers achieve more with less code.

As far as the scientific HPC programming language landscape is concerned, Fortran has been the status quo for a few decades now. I suspect that there is a sea change coming, however, in the world of HPC languages. This speculation is, of course, fueled by multi-core revolution sweeping across desktop and even laptop computers. As single CPUs have hit the wall in terms of performance, the only option is to add more CPUs (cores) per processor and shove more processors into the computer. The computer that I am using right now is a laptop with 2 cores. Apple just released an 8 core (2 quad core processors) machine. This trend hints at the fact that desktops and laptops are becoming HPC platforms.

As we start seeing a preponderance of these types of architectures, programming languages and software engineers are having to adapt. In particular, languages are going to make greater use of threads and other concurrency features to take advantage of the multi-cores. All this is not really news to anyone who has cracked open a DDJ in the last three years.

What is interesting to think about is how these changes will affect the rarefied world of HPC and supercomputing. I speculate that tomorrow's weather and climate simulation super-computers are going to be the same kinds of multi-core computers that run on our desktops, but just with a bunch more processors. The simple explanation is that this will be the path of least resistance to building a big parallel processor, rather than creating custom, one-off solutions. (A corollary perhaps is that super-computers could become cheaper. We have really seen all of this before with off-the-shelf linux clusters and the Beowulf type machines.)

In other words, the architecture will be similar enough that standard desktop applications could run on a 1024 processor machine in the not too distant future. This assumes, of course, that desktop applications will have been ported to the new multi-core architectures, but this is inevitable over time as the multi-cores become prevalent. More apropos, we could see Java Virtual Machines running on big multi-processors. And with Java's built-in support for threads, and the forthcoming fork-join framework, using Java as an HPC language remains in the realm of possibility. Software developers are already running Java on big parallel machines at places like Azul Systems, but as far as I know, they are not doing scientific programming.

For various reasons, however, Java is not the best language for scientific HPC. For one thing, there is no operator over-loading in Java. That makes dealing with complex numbers (which must necessarily be represented as objects) annoying because developers have to therefore rely on method calls that can be unwieldy for arithmetic operations. There are a multitude of other reasons why Java is poorly suited to scientific programming that go beyond what I want to write about here. With the possible exception of C++, the difficult truth is that while Fortran is not a great language, until now there has been little or nothing better to address scientific HPC domain.

Fortunately, computer scientists at various research centers are busy developing new programming languages that are well suited to the multi-core world of today and also have the potential to address various scientific-computing problems. In particular, a couple have been receiving some attention: Fortress from Sun, and X10 from IBM.

What is promising is that these languages are new (especially compared to Fortran) with modern programming notions like object orientation, but also address scientific HPC needs. Here are just a few highlights of features that look interesting:

Easy parallel task decomposition - They make it straightforward to break up work for parallel execution. For loops are parallel, by default, in Fortress. X10 has a number of built-in concurrency idioms.

Science oriented - Fortress and X10 both allow operator overloading, for example. Fortress accommodates Unicode characters in identifiers allowing for Greek symbols in code. Fortress also allows static checking of physical units and dimensions and multidimensional arrays.

Portable by way of virtual machines - The project websites of both these projects claim the languages will run on the Java Virtual Machine. Not only does this have the potential to solve all sorts of portability concerns, but, speaking speculatively, they could take advantage of any jar bytecode library that can run on the JVM. No need to write a separate lexer parser for XML in X10, in theory.

Supported by the Eclipse IDE - Fortress and X10 have Eclipse plug-ins.

There are also many other improvements over previous languages which you can read about here and here. But in conclusion, scientific programmers may finally have access to several modern programming languages that may, at long last, get us past Fortran.

Java Generics, Finally Drinking the Kool-Aid.

2007-06-24T19:36:00.001-07:00

For a long time, I was unhappy with Java generics, a language feature that was introduced in Java 1.5. Java generics are supposed to give Java programmers the ability to write code with greater type safety. Unfortunately, Java generics have also introduced a lot of complexity into the language:

Here are some of my main complaints with generics:

#1 Confusing and difficult-to-understand syntax.

Here is an example:

class Box< T extends Comparable<T> & Cloneable> implements Comparable<Box<T>>, Cloneable

I would prefer not to parse the class declaration for several minutes to figure out what is going on.

#2 Java generics do not seem to be generic.

Before:

Map map = new HashMap();

The Map declaration is nice, clean, simple, and GENERIC. It will contain any key value object pair.

After:

Map<String,String> map = new HashMap<String,String>();

The Map declaration is no longer generic but rather specific and now must hold String for both key and value. That is not generic.

#3 Poor cost-to-benefit ratio.

Much complexity has been introduced into the language as a result of generics that is time consuming to learn.

Fortunately, there are resources like Angelika Langer's 425-page FAQ. Yes, it is really 425 pages. I consider this to be one of the most valuable Java resources on the web. Indeed, if her FAQ did not exist, we would all have given up by now on Java generics. (I am hoping to read the whole thing and I am about a 1/3 of the way through it.)

But on the flip-side, if it takes me 425 pages to understand something, I'd better be getting some really great features. I do not anticipate these benefits with Java generics.

I realize that much of the complexity associated with Java generics is to accommodate backwards compatibility concerns. Brian Goetz wrote a nice article that touches upon this issue and others surrounding Java generics. Maybe if the powers that be were to get rid of raw types, generics could be simplified. The Java Language Specification states that raw types are strongly discouraged and may not be supported in future versions of Java.

#4 Did I mention the syntax?

Here is an example of a Collection data structure:

Collection< ? extends Pair<String, ?>> c = new ArrayList<Pair<String,?>>();

I would prefer not to study the Collection declaration for 10 minutes to figure out what it holds.

Or here is one I really like:

Enum<E extends Enum<E>>

Huh? I feel my mind over-heating.

Actually, this is explained in Angelika's FAQ.

Experienced developers have been struggling with generics. Where does that leave junior programmers attracted to Java supposedly because it is simpler and more elegant than other OO programming languages?

The whole thing almost smells of a political dispute within the Java language designer community. Why didn't they introduce generics from the start instead of adding it piecemeal halfway through? Is the state of type theory really that much more advanced than in 1995 when Java was introduced? Indeed, this would have made things a lot easier since a large part of why Java generics are so confusing is because of backwards compatibility issues.

Despite all these misgivings, I have been moving over to generics and in some cases genericizing some of my APIs. Part of the reason I am doing this is that I am forced to. If I want to be a decent Java programmer, I have to know how to use generics. I've heard that it is even starting to crop up in job interview questions.

Also, it does seem to make code a bit more readable if the type parameters are simple.

List<Integer> list = new ArrayList<Integer>(); //Not so bad to understand.

The new Guice IOC container from Google also makes heavy use of generics. I haven't used Guice yet, but I look forward to seeing how Java generics play out in this context.

I am starting to get used to generics, but I also feel like I have a long way to go before I can completely master this challenging subject .

Welcome to RadioEiffel

2007-06-13T19:32:00.000-07:00

Welcome to RadioEiffel. I am finally joining the 21st Century and starting a blog. I intend to ruminate about software development; my profession as well as my hobby. Thanks for reading!
Oh and here is a funny picture of my baby and me.