I’ve finished my 3.5 year stint writing Scala, and I haven’t stopped missing writing Clojure. The knowledge of Clojure continues to heighten and inform my programmer sensibilities. One thing that I appreciated about Scala is that it was as good of a medium as you might practically find to allow writing Clojure without writing Clojure. I liked to think of Scala as the canvas on which I painted my Clojure ideas. Because Scala makes itself amenable to many styles of programming at once (at least, FP and OOP), it was possible to write code by imagining what the Clojure code would look like, and then writing that in Scala syntax. Interestingly, the more I did this, and the more faithfully I did so, the more people implicitly (no pun intended!) acknowledged the code as “good Scala code”. Because, you know, most Scala programmers agree that good Scala code puts “val”s at the top of a function body, uses immutable collections exclusively, prefers functions over (object) methods, and makes functions small, stateless, and composable. More on that later. Here, I want to simply release some of the code that I wrote in Scala to fill in a few perceived gaps in Scala’s Seq abstraction, where the perception is based on what I was accustomed to using in Clojure.
For those of you with experience with Maven, you might be wondering why anyone who is using Leiningen to build a project would then want to run that build tool from Maven, which is itself another build tool. There is a reason why I even ventured down this path. I would like to share what I have found so far, in case it benefits anyone else, but I would also like to get feedback from people who know of a better way of accomplishing the same goals.
Or: A clear example of what macros can do
I started working on a library called clj-thamil that I envision as a general-purpose library for Thamil language computing (ex: mobile & web input method), but a slight excursion in that work has led me to some very deep, intriguing ideas — some of which are technical, and some of which are socio-cultural. But they all fit together in my mind — Clojure, macros, opportunity and diversity (in computing), and the non English-speaking world.
I think that the implications are things that we should all think about. But if nothing else, hopefully you can read this account and understand something about macros — the kind of power they uniquely provide and at least good one use case where they are necessary.
Back around the December – January time frame, I was trying to implement the Lambda Architecture as described by Nathan Marz. At that time, the early-release version of his upcoming Big Data book was just at chapter 5 or 6, but my goal was to tackle what seemed liked the harder part — real-time (Storm). The book chapters hadn’t yet caught up to it. A few slide decks mentioned their current implementations of a fully thought-out, end-to-end Lambda Architecture implementation that included Storm, but no reliable, easy-to-deploy code was readily available from the interwebs.
In installing Storm, it quickly seemed apparent that having Kafka running upstream of it was one way to support both real-time and batch processing of incoming data, and probably the one of least resistance. So I added installing Kafka to my to-do list.
Cutting edge technology means dealing with rough edges. I downloaded the latest versions of the relevant software components, but the integration of all of them didn’t work. As I found out, the reason was that versions of components that finally worked together with each other for me were not the most recent, but instead maybe a version or two behind.
The code that I ended up with to get Kafka and Storm working together on a toy example using the Twitter Dev Stream is on github here:
The Clojure for Beginners workshop at the Oakland Workshop Weekend a couple of weekends ago went well. Everyone who came to the workshop came into it curious about Clojure and they all stayed engaged and curious to the end! They all wanted the slides, and after a few fixes, I’ve uploaded them to my Github project page for the presentation here.
Just FYI, here are the considerations I had when creating the slides, from an instructor perspective: