From Python to Clojure.

In the last post I talked about factors that have caused myself and others to look around for languages to move on to from Python. I noted that a number of languages that expat pythonistas are looking at are more recent entries in the Functional Programming space. After dabbling in a number of languages over the last two years I have settled on Clojure. This post examines the tradeoffs from my perspective.

When I first came to Python it was around 2002. With a background in Java from university Python provided an appealing language to get things done. As a quick summary:

• The syntax is clean and clear, including strong support for iteration (something that was “in the future” for Java at the time). Generator comprehensions were in the future meta-package at the time I believe.
• Dynamic exploratory development with an excellent REPL is easy.
• Access to foreign code from C-land has always been good¹.
• Great built in collections and literal support for the core ones. Lists, dictionaries, tuples, sets are all out of the box.
• Simple and expressive Object Orientation.
• Great range of libraries. Some particular worthy ones include Twisted, Django, WSGI, SciPy.
• Comprehensive namespace and packaging tools.
• Cross platform.

The big obvious tradeoff was speed but, as has been noted many times by many people, this isn't a huge impediment for a large class of programs. Not so obvious is the GIL and the problems it brings to multi-threaded programs but I don't want to go into that debate here.

Another problem that comes to the surface after extended use is packaging third party library code. Tools like virtualenv and pip etc have helped a lot in recent years, but its certainly not Python's strength.

I've had the chance to work on some really cool projects with Python. I built a skinnable, scriptable media player with webservice integration for my internship, extracted a nice little parser combinator library from this site with some friends, I worked on a Django based financial application full time and a many more small projects.

Any language I move to has to feel as expressive as the experience I get with Python (sans learning curve difficulty). This brings us to Clojure.

I've been following Clojure since mid 2008, dabbling in it and writing little toy applications. The functional style has been appealing to me more and more. So what features does Clojure bring to the table for me?

• It is a lisp, so the syntax is nearly nonexistant. However, it has more literals for data and these are used in the special forms. Clojure has (to a new-lisper) much nicer special forms, the obvious example is let compared to Common Lisp².
• Namespaces and modules support is solid. The differing rules for Clojure imports and Java imports tripped me up at first, but makes sense.
• Dynamic exploratory development with the REPL. The REPL is a bit more raw than Python's but is improving dramatically.
• As mentioned above, there is a good set of literals for data. The built in data-structures are really good. Lists, vectors, maps (dictionaries), sorted maps, sets, sorted sets. This is an area Clojure really shines; not only are there all the basic collections I want, they are super fast and persistent (immutable), which sets things up well for concurrent operations.
• Functional Programming is front and center, but supports mutability and state clearly.
• Support for Java and Objects is solid. The inter-op / foreign function interface to Java-land is a core feature of Clojure.
• “Clojure has objects but isn't oriented around them.” -- Chris Houser.
• Cross Platform
• Builds on Java's code distribution tools like jar files, and Maven repos with Leiningen.
• Fast. Type hints allow you to avoid reflection in many cases, and it compiles to the JVM.

One obvious lack for Clojure is (native) libraries that are as comprehensive as some of the ones for Python. There isn't a web framework that competes with Django in terms of feature set and breadth for example. The language is very young (just over 2 years) and it is developing rapidly, but you might still find yourself building (or wrapping java) what you could have got from the toolbox in Python.

Secondly, if Object Orientation is key to how you design code, you will be out of luck. Same goes for JVM: If you have no love for the JVM platform, Clojure won't be the right choice for you.

There is a lot of overlap between the two lists. Neither list covers all the great things about either language, just the things that spring to mind about why I like each.

So what makes Clojure more appealing than other Functional (or Hybrid OO/FP) languages? One key differentiator is that as a pragmatically designed language, IO and state considerations are clearly part of the plan. Not only this, the role of state, identity and time with regard to concurrency has been considered throughout the language's and core library's design.

Unlike Python (and other contemporaries like Java), Clojure does not provide mutability or identity to all data or variables. Instead, data is immutable, and the core data-structures are designed with that in mind. Secondly explicit reference types are provided. The programmer is expected to write core logic in a pure functionally mode where possible, and then use the reference types to maintain identity where needed. This will be familiar to anybody who has seen the buzz around functional programming and concurrency. Unlike in the C family, where pointers merely address memory, each reference type provides additional semantics. These semantics allow Clojure to provide managed statefulness; this is analogous to managed memory. A lot has been written about Clojure's concurrency features, and by smarter people than me, so I won't say any more.

One consideration with the current state of Clojure is that the planned 1.2 release is bringing two new constructs: Types and Protocols. These correspond roughly to Classes and Interfaces in Java. The current release (1.1) provides a Structs feature that uses dynamic maps as types, but this feature is due to be deprecated with 1.2. This places new users in an awkward position, but is a necessary change and will only be good in the long run.

The features that made Python a great choice in 2002 are no longer just the domain of so called 'Scripting' languages. Switching to Clojure does have trade-offs but it feels like the right choice for me.

Postscript

“Show Us The Code!” I'm not going to for this post. Snippets could be manufactured in situations that show each language as 'better'. If you would like to see a small piece of Clojure written by a relative beginner, check out Caponia. This is an ultra light weight, in memory, full text index library that Matt and I designed during lunch and he implemented over the following day.

In contrast to this rather fluffy post, the next two will be looking at state and evaluation semantics in Clojure, chock full of code snippets. Update: The first half is now up.

See also

• hga on Hacker News provides a link dump of Clojure material. A great place to get started.

1. These days its excellent with the ctypes library
2. No doubt Common Lisp has good reasons for how it does things, but from the outside its opaque and frankly not as nice as Clojure. Clearly a subjective judgement, but I'm allowed to make those when I'm evaluating a language for my own use.