Categories
clojure java programming Rust web

Choose the exemplar programming language for the use case

I think my understanding of programming languages — ex: what role do they serve in tech engineering work, what my favorite language is — is something that continues to evolve, and has done so once more recently. Here is where I started from and where I stand:

Meandering through languages

When first learning to program, I remember learning that BASIC was the beginner’s language. The Commodore 64 was how I played computer games, but it could allow you to write programs in BASIC. I only learned, though, when I signed up for an elective in elementary school. It was the successor class to an elective where all we did was play MECC games on Apple II computers (Number Munchers and Oregon Trail, especially). Over the summer, when I tried to take that knowledge to a regular PC with MS GW-BASIC, I wasn’t able to figure out how to draw to the screen, change colors, etc.

When I took a class in the following year about Programming in Logo, it was a breath of fresh air. The syntax was easy, it was seemingly all about drawing pictures, so the feedback loop was immediate, and the notion of functions and even recursion all kind of made sense. At the end of the semester, there was a chance to try out for a programming competition. There was a prompt that describes what the program should do, and some sample input and output. Since I didn’t have a Mac at home with a Logo interpreter, I decided to write it in BASIC, and I wrote a bunch of terrible code that didn’t catch corner cases. I got really frustrated — it challenged the notion that I had built up through Logo that I was capable of expressing exactly what I wanted. And that’s when I wanted to give up on programming altogether. It was too hard, too frustrating, requires unreasonable perfection, and therefore not enjoyable.

(In hindsight, I’m not sure I ever understood BASIC subroutines, even though Logo functions made perfect sense. And only many years later did I learn enough to be able to explain the level of complexity that BASIC requires of the programmer to deal with dynamic variables (and therefore, the state of memory, to some degree), and contextual control flow via line numbers and goto and gosub and return. Logo, as a functional (Lisp) language, was far simpler in its evaluation and execution model.)

I still kind of thought programming was interesting, and thankfully, I took Pascal a few years later. Turbo Pascal was capable of stylizing text with colors, made it easy to receive input, and manipulate text. That was enough to make an interactive text-based menu program that was interesting to me. And that led to learning a little of C++, and a little more C++, before learning a smattering of Java, C++, and Perl in undergrad.  C++ seemed “adult” – this is what serious programmers used to write cool things, but it seemed complicated. Java was easier and more regular, but the design patterns & OOP seemed obtuse and clunky at times. All throughout, I didn’t have a clear favorite language. Whichever was “easiest” and allowed me to do the most would be my favorite, and since “easy” is a relative term, I was always enamored with whatever I was using at the time because I was in that headspace, and it was therefore easy to continue thinking with that mindset.

At one point in grad school, I started using Java to effectively write a simple CLI tool that would read in files in a particular directory structure and use those files as inputs to kick off other Java code with the actual complex algorithmic work. After learning Python, it made me think that I was wasting my time with Java — it was a lot of effort for little payoff, but Python allowed me to write small CLI tools with much less code and a tighter feedback loop thanks fo the scripting nature and the interactive interpreter. Ruby was an improvement over Python in that regard — more regular syntax, and functional-esque constructs (map and reduce) that saved on code and made it a little more readable. It was only when I got to Python and Ruby that I really liked them a lot for how much easier they made my current task over any other language I had learned previously.

Different meta-views of programming languages

In undergrad, my Programming Languages class taught us different types of programming languages, explained how they are designed for different use cases / different types of problems, and showed us how to use them to solve toy problems in each problem domain. I still had to learn that lesson through experience later in grad school when splitting off the CLI interface to the Java tool and converting it from Java to Python.

The Programming Languages course also introduced the idea that anything expressible in one language is expressible in another, so there’s no limit to a language’s expressive power when compared to any other. That was my first level of “wisdom” — all languages are the “same”, in a way.

The next level of wisdom was that some languages really are just “better” than others. To me, Ruby was (and still is) more readable, more regular, more expressive than Python for general use cases. After learning Clojure, I realized that it took care of annoying file parsing tasks very concisely that I might have previously used Python or Ruby for, but it also handles larger scale programs well, both in representation of complex logic as well as deployment concerns. Clojure seemed strictly better than all those languages. And I continued to work in high-level application programming for a long time, for which that sentiment held. Paul Graham’s essay “Beating the Averages” that shows that different languages have different power, and that Lisp was at the top of the power continuum, was eye-opening. And the fact that Clojure can also make it into the browser as well as the backend while not constraining the backend runtime created a compelling story for web apps that really hasn’t been pulled off by any other language yet.

Another adage about engineering that I had heard throughout all this time was “use the right tool for the job”. Even outside of engineering, you can see photographers swap between different lenses for different settings. In technology, Mathematica is a nice program to get closed form solutions to relatively simple math expressions. Some languages / software environments are good for specialized purposes, like R for statistics or Matlab for matrix-based computations. If you’re trying to do logic programming, you want to use a language designed for representing facts and constraints and computing how they are satisfied.

Use the best designed programming language per category of use case

But recently, I have come back to that “right tool for the job” adage, with a slightly modified and opinionated interpretation  Of course, we’re looking at programming languages as tools because they are. They facilitate human thoughts to be represented by computers and executed. But there is something to be said about the power index that the Blub Paradox explains. It helps explain why I had no regrets switching from Python to Ruby for simple scripting / CLI use cases. And how Clojure makes tricky parsing problems trivial (ex: I had a technical interview phone screen question that was basically to write code that parses the output of the command git log. Since I was allowed to use Clojure, and the problem became trivial in a way that would have been tedious and error-prone in Python or Ruby.) And when learning Clojure, I would take Java code and port it into Clojure, only to find that the number of lines would shrink in a 2:1 or 3:1 or 5:1 ratio.

Ultimately, what I’m getting at is being more specific about what “job” means in “right tool for the job”. The examples above all fit into a category of “general purpose” programming where the algorithm / logic is important, requirements might change as you write the code and/or unpredictably in the future, and where squeezing out every bit of runtime performance isn’t the most important factor. Scripting languages and Java/Clojure rely on garbage collection, and Java’s iterations of GC have become impressive, according to others, allowing long-lived repetitive applications to not be too much worse than if it were written C/C++ (off by a few percent versus a constant factor? I don’t have a great intuition for the numbers). The point is that there are other factors that are also important to the business bottom line: developer time, maintenance costs, including the ability to elegantly handle major requirements changes without wholesale rewrites of your code. And that category of use cases is what we should define as the “job” when it comes to programming languages.

So in addition to this category that I’m calling “general purpose programming”, there are other categories. For example, as I mentioned before, Rust is great for library / low-level systems programming. Runtime performance, including control of memory allocation and reclamation, are paramount if you and/or a large ecosystem of others are to build applications using your code. Rust explicitly prioritizes runtime performance and control, and programmer safety in doing so, above all else. Ergonomics are important, but are still less important when they come into conflict with the first 2 priorities. Rust is designed well and has a clarity of design. With this in mind, it’s easy to see some parallels in the rapid adoption by C++ users of Java and the rapid adoption by C++ users of Rust. I would argue that these languages focus on 2 distinctly different categories of use cases — the general purpose, and the low-level systems programming. And this is why you really can’t compare Rust to Java, IMO, any further than that.

There is a little bit of extra complexity to Rust as a language, but it’s not a problem to me because the language is clear about making that tradeoff to allow that cost in order to get the benefits that Rust is optimized for — performance and developer safety. For example, with the notion of ownership, borrowing, and references, you have to learn about lifetimes. Lifetimes can be tricky at first, but they’re unavoidable. The payoff is that Rust forces you at compile time to do many things that experienced C/C++ developers do to make their programs as efficient as possible. 

IMO, Rust’s concept of immutability is also “memory-based” just as much as the difference between Copy and Clone traits and methods are tied up with ownership and references in a way that makes you conscious of memory layout (are the fields laid out in the stack frame or are they allocated on the heap?) Immutability requires in Rust is about allowing either 1 mutable writer or many read-only readers, and not both in the same lifetime. 

To get a sense of why I describe Rust’s immutability as “memory-based”, compare what it looks like to insert a value into a collection and then get it back out in Rust versus assoc’ing a value to a persistent collection in Clojure, where I describe the immutability as “value-based”. Clojure makes different tradeoffs, where the language itself might be simpler, but the implementation of certain things like persistent data structures are more complex, and without the benefits like runtime speed that Rust is designed to optimize.

Another way that Rust is memory-based is that the difference between using one type or another is about whether you have to allocate memory. A Path struct instance is a read-only (no-allocation) value, but once you want to join a path segment, you end up with a PathBuf type value because you have allocated. It is akin to &str versus String, or what a slice is to a sequential collection type. And maybe you care about the corresponding difference between std::iter::Iterator and std::slice::Iter.

However, you can compare Rust to C++ and say that Rust is a much better language for the use case category that it’s designed for. For Java, yes, Java is better for general programming than C++, but then you hear similar arguments trotted out by Scala users when they compare it to Java. They talk about Java being verbose, complex, and error-prone. But frankly, I think the same thing can be said about Scala when compared to Clojure. Scala’s hodgepodge of Haskell-inspired category theory-lite paradigms and constructs have led people to program themselves into a corner with Scala, get frustrated at their inflexible overly-typed code, and switch to a language and ecosystem that isn’t so rigid. Anecdotally, I’ve seen code shrink by a factor of 2:1 when going from Java to Scala, and again from 2:1 going from Scala to Clojure.

I think each use case category will end up with an exemplar language that does the “best” for that use case. Defining the use case category as well as “good”/“best” is fuzzy and fraught, but that’s ok. I think Rust will end up being the best language for low-level programming (it already is, but I think it will stay that way). I think Clojure is the best for general purpose programming. This isn’t to say that Scala’s Haskell influence is bad, but that it’s not suited for general purpose programming. That very high level of static typing that Haskell has can be helpful where you have a high-value situation in a closed environment, meaning that every bit of extra confidence you can give your code with rigidity via high-level static typing, combined with the comfort that requirements won’t really change and you don’t have to deal too much with the unpredictable messy nature of real-world input data (because you’re not interacting much with the outside world) means that Haskell could be a good choice, ex: for banking, avionics, health devices. 

But applying a programming language that isn’t “best-in-class” for a category of use cases is something that I would now be wary about, all things being equal.  I think Scala is caught in between categories — it’s not really designed for any clear use case (because it wants to show that the marriage of OOP and Haskell-style Functional Programming is possible and beneficial), and you would be better off with Clojure or Haskell for the most likely use case categories, just as C++ users might be better off with Rust, Clojure/, etc. for their most likely use case categories. For example, the idea of using Scala for frontend development, or using Rust for general-purpose web programming, are hard passes for me.

Language isn’t just syntax

And some people think that syntax is an important factor for a language. One person recently told me that they like the idea of Clojure, but instead they like F# better as a functional programming language because the syntax “isn’t weird” — it’s more familiar (in the style of C/Java/Python etc.) That again ignores the important points of Beating the Averages, so if you haven’t read it, go read it now and come back. Ok, so did you get the point about macros? The idea of using them to eliminate domain-specific boilerplate code whenever it crops up should be important enough. 

But then with macros, you have the ability to make a language library that implements logic programming, including syntactic similarities to languages dedicated to logic programming, is just a small example how a language whose syntax can be bent into the shape of other problem domains and/or programming paradigm syntaxes is strictly better than a language that can’t. It’s a short-sided argument to say that you don’t like Lisp parentheses because you would be missing the deeper point about a language that is written in the data structures of the language allowing something special and uniquely powerful. And more importantly, it should be clear how concerns like these are missing the point. And there is a huge difference in simplicity between Lisp macros and AST-based macros — the latter is just not the same. I’m sure the uniqueness of Lisp macros is what allowed core.async to be written as a library (one that works in Clojure and CLJS), and perhaps the same is true for transducers. And not that macro-writing macros should be a point of pride — but it’s an extreme example of what is possible vs. what isn’t in an AST-based macro.

And as I’ve mentioned before, the ecosystem that Rich has created of plain data and a language full of functions that operate on those plain data structures has a huge multiplier effect of power for everybody. That’s not something that a language alone can dictate, but it is phenomenal nonetheless.

Parting Thoughts

The next time I hear someone enthusiastically be a cheerleader for a language to “take over the world”, I’ll just be ready to say, “I guess your world is a lot smaller than mine.”

And maybe if WASM’s Intermediate Representation (IR)’s S-expression syntax gets real trendy, and if the same happens for Rust’s immutability, macros, and functional programming style, it’ll be fun to see who is open-minded enough for a surprising revelation and who still isn’t.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s