A recent tweet by a friend of mine noted how the public interest in the Scala programming language seems to have plateaued or waned, which matches my feeling of the latest trends and zeitgeist. This blog post will go into why I think that has happened, where Scala stands now, and what the future holds for the Scala community.
About the Author: Haoyi is a software engineer, and the author of many open-source Scala tools such as the Ammonite REPL and the Mill Build Tool. If you enjoyed the contents on this blog, you may also enjoy the Author's book Hands-on Scala Programming
This blogpost was inspired by the following tweet:
It's a shame to see friends from Typesafe/Lightbend being laid off, but as I've always said, it's hard to sell free stuff. I do wonder about #Scala's adoption now. It's not Clojure-dead, but it seemed to peak around 2016. I have zero data to back that, though.
Jamie notes that public interest in the Scala programming language seems to have waned: less interest in meetups, less interest in conferences, and so on. This more or less matches what I am seeing as well. While the Scala-specific conferences targeting enthusiasts are still going strong (or were, until COVID19 shut down the global economy) I think there is no doubt that there is much less talk about Scala in the broader engineering community.
I think that this reduction in public interest is real, but also expected given the typical "Hype Cycle" that follows any new technology:
Early on, Scala rode a wave of hype that frankly surprised even me: hype around pushing syntactic boundaries, hype around reactive architectures, hype around functional programming, hype around the Apache Spark project. Much of that hype has since died down, and there was a period of backlash and negativity both within the Scala community and outside of it. Since then, even the backlash has faded, and what is left is a reasonable, boring language steadily advancing and providing a great platform for general software engineering.
While Scala was initially created in 2004, the language only really started taking off in the early 2010s:
~2009: Twitter started picking up the language for production systems
~2010: the first tagged release of the Scalaz functional programming library
~2011: Lightbend, a company pushing Scala and related technologies, was founded
~2014: Apache Spark was written in Scala
~2015: first tagged release of the Cats functional programming library
While there was no doubt activity in the Scala community before this, these were no doubt major milestones that kickstarted interest in the language. With large-scale commercial adoption, commercial support, and a killer big-data product like Apache Spark, it is not surprising that there was a surge in public interest. A lot of these efforts went into the following domains:
A lot of early Scala interest was in how flexible the language was: it had extension methods, operator overloading, implicit constructors/conversions, and a very flexible implicit-parameter feature that could be used in all sorts of ways. This flexibility was liberating, as it opened up all sorts of domain-specific-languages and programming styles that would be unheard of in other languages.
These were exemplified by a number of projects, such as the Databinder Dispatch HTTP client, Scala-Graph data structure library, or SBT build tool:
// Making a HTTP request and processing the output in Databinder Dispatch
executer(:/(host, port) / target << reqBody >- { fromRespStr })
// Constructing a simple directed graph with labelled edges in Scala Graph
val flights = Graph(
(jfc ~+#> fra)(Flight("LH 400" ,10 o 25, 8 h 20)),
(fra ~+#> dme)(Flight("LH 1444", 7 o 50, 3 h 10))
// Append the contents of a URL to a File after filtering through grep using SBT
url("<http://databinder.net/dispatch/About>") #> "grep JSON" #>> file("About_JSON") !
// Search for uses of null in the source directory using SBT
"find src -name *.scala -exec grep null {} ;" #| "xargs test -z" #&& "echo null-free" #|| "echo null detected" !
Many of these libraries have evolved since the early days and have since moved to more boring method-based APIs. It turns out that naming operators >- ~+#> or #&&, while possible, was not always the wisest thing to do. Nevertheless, there is no doubt that pushing the boundaries of Scala's syntax using operators and domain-specific-languages was one of the big points of interest for early Scala.