language parallax

Sitting on Powell’s books (not the technical one) in Portland with about 30 minutes of battery time left on the laptop. I thought I might write up some initial thoughts from the conference.

First of all, I should say that I didn’t attend much of it. I was around for most of Tuesday and only made it to one talk on Wednesday. But what I did see was exactly what I was looking for. The talks as well as my personal conversations with Robert Fuhrer of IBM and Magnus Christerson and Charles Simonyi of Intentional Software brought things into focus.

I may write later about the particular functionality I saw from both groups. But here I’d like to comment on the growing focus in the community on LOP (language oriented programming) or DSL (domain-specific language) support or whatever it will eventually be called.

The algorithms and even the decomposition of these systems are still undergoing rapid development. Christerson mentioned that they had spent a couple of months recently just thinking about how the arrow keys should work in the various “projections” — hard enough even when you’re talking about text editing, but when you throw spreadsheets, trees, and all kinds of other structures into the mix, there is definitely no consensus on the right behavior.

Fuhrer mentioned that they had initially based their effort on polyglot, but had to learn a painful lesson that its structure wasn’t suited for their purposes.

I wasn’t in the community at the time, but I imagine this is what the birth of RDBMS was like. I think LOP/DSLs are objectively a lot more complex than standardizing a syntax for relational algebra. But the process is probably similar. Initially there’s a lot of confusion about which problems need to be solved. We can start with a fairly easy-to-state goal of making it easy to generate IDE’s for arbitrary languages. But by the time its over you realize that arrow-key behavior has to be thrown it. Even more deply: it changes the framework in which static analysis algorithms operate, which is made even more difficult by the fact that the algorithms themselves are still settling.

And of course there’s the matter of competing implementations. I asked Christerson for advice on which tools to start with. An initial response was Ruby. But I then mentioned the IBM project. Ultimately I think we agreed that it’s a young area. Not only are there no publically available implementations — the private implementations are far from fully baked. So it’s a matter of finding a reasonable community to attach myself to, keeping in mind that I may have to port my work at some point.

I’d say the IBM appraoch is probably the right bandwagon to jump on, but it doesn’t plan a public open release until sometime in ‘07 (hopefully the first half but there are no guarantees).

I’ll need to figure out how to get by without the right base. Can I just assume that someday I will have it? Should I start with the Ruby approach and switch to IBM’s Safari when it comes out?

It seems that there are three concrete project I could work on:

• An integrated environment for development with javascript, java, and sql that understands refactoring. Minimally it should be able to rename columns/variables in all three versions simultaneously. There are probably a couple other interesting refactorings that can benefit from full knowledge and manipulation of languages of representations. I could also scrap the javascript piece and just focus on a hibernate replacement (java <-> sql bridge).
• Unix configuration files as DSL’s. Here I could employ my experience in large-scale unix operation environments. Software deployment and hardware/network changes could benefit greatly by a system that understood the semantics of all representations involved in these changes.
• New GUI’s for visualizing what Intentional Software calls the “kaleidoscope”. An application of a refactoring can be thought of as another projection/scope instead of an actual update of the underlying code. We can draw a network of such refactoring steps, and keep all versions editable until we forceably invalidate some of them. This will probably also involve thinking through how these systems will interact with source code control (as well as the gui to manage projections).