17 ms·

Tour of our 250k line Clojure codebase

373 points by grzm 5y ago

mping 5y ago
This is what I imagine experienced clojure developers can squeeze out of a language like clojure. I would venture that they can train a junior programmer in a couple of weeks, and make them productive very fast. I guess they could make it work in any language but judging by the description clojure is indeed a great fit, due to the macro capabilities, flexibility and solid runtime via JVM.
- cliftonk 5y ago
  I've found I typically reach for clojure when i need to do something on the jvm and want a better java than java.
  - lukashrb 5y ago
    Interesting take since Clojure and Java are two very different languages. And unlike for example Kotlin, Clojure does not try to be a better Java than Java. But true though Clojure leverages the power of the jvm.
    - forgetfulness 5y ago
      It was a better Java in many ways. The doto macro was a relief back in the days when Java APIs insisted on being designed around stateful setters and getters rather than the builder pattern, it allowed you to operate on such unfortunately designed objects in single logical blocks and simulating it all being an expression. Proxy allowed you to instantiate anonymous inner classes implementing only the methods of an interface that you needed, the rest you could omit; in Java you have to put them all, empty, which necessitated that you use an IDE to generate them, and back then IDEs were not as nice as today. Those two alone made interacting with contemporary Java libraries so much easier. It also was convenient to go from Java collections to Clojure collections and vice versa.
      - pjmlp 5y ago
        " We were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp." -- Guy Steele
    - pjmlp 5y ago
      Basically you get a new version of a Lisp Machine, that you can sneak up in lots of places, whereas with Common Lisp it isn't so easy to do so.
    - __jem 5y ago
      Interop from Clojure -> Java is still incredibly easy, though. I often will just wrap a Java library rather than look for a pre-existing Clojure implementation because it's so easy. Of course, most Java libs don't value immutability and functional patterns, but you can still push this kind of interop to the edges of your system and keep everything else in pure Clojure.
      - StreamBright 5y ago
        Depends on the style the Java library you are trying to use is written in. I have had mixed results. Especially the Java 8 functional style had some challenges. I might have the relevant SO question somewhere.
  - forgetfulness 5y ago
    I think that's also why Clojure use peaked right before Java 8 was released; once Java became a better Java, and libraries started to be designed around more ergonomic APIs than wiring up objects that needed miles of stateful configuration code, the pressure that drove you out of Java and into Clojure began to diminish.
    - cutler 5y ago
      Hardly. Despite being Java-compatible Clojure's philosophy is diametrically opposed to that of Java/OOP so anyone who has experienced the benefits of Clojure would hardly return to Java simply because it managed to fake FP with Single Abstract Methods.
      - forgetfulness 5y ago
        Well, maybe there were not as many die-hard lispers within the Clojure community and many were looking for something that handled better than Java 7. I know two other former Clojure programmers that went to Scala, but also find Java passable enough to just use it rather than Clojure.
  - dustingetz 5y ago
    Clojure's value prop is: 1) default immutability (same simple data structures used in every library -> ecosystem composes better) 2) portable code across multiple host platforms (jvm, node, browser) 3) metaprogramming IMO, in 2007, immutability on the JVM was a competitive value prop, but in 2021+ it is nothing special. It is the combination of the three things which is a competitive value prop today. Metaprogramming in particular is a mostly unexplored frontier, because it is very hard to do well, and very easy to do badly. Default immutability is kind of a necessary starting point to do metaprogramming well.
    - davidrupp 5y ago
      > in 2007, immutability on the JVM was a competitive value prop, but in 2021+ it is nothing special Can you elaborate on this? What do you think has changed in that time to make it "nothing special"?
      - deleted 5y ago
        [deleted]
      - dustingetz 5y ago
        immutability as a library is available in basically all mainstream languages now and mainstream frameworks leverage it (react, any UI framework, spark, any data framework or database); JS vms are competitive with the JVM and the JVM might even be losing ground in the cloud; typescript is a monster and is letting people explore haskell concepts in industry applications; scala is way better in 2021 than it was in 2011; every new PL can compile to JS and supports immutability. Clojure's sweet spot is currently developing sub-languages embedded in Clojure like RPL is doing (we are doing exactly the same thing at hyperfiddle). That is still too hard to do in typescript imo. Maybe it is possible in scala 3 but it took them 10 years of pure autism to figure out how to do monadic IO in scala due to how complex scala is, so i'd expect it to take another 10 years to figure out how to do metaprogramming in a commercially viable way, I'd be happy to be proven wrong.
        davidrupp 5y ago
- trutannus 5y ago
  > would venture that they can train a junior programmer in a couple of weeks, and make them productive very fast I've got some experience with functional languages, and a good amount with functional features in OO languages. Started learning Clojure this week, and was pleasantly surprised with how quickly I could get working projects up and running. Less upfront learning curve than Elixir, which was unexpected.
- tmountain 5y ago
  Yeah, it's nice to see such thoughtful adoption of language facilities clearly oriented towards creating a successful and maintainable codebase. The Clojure team has always done a nice job expressing the rationale for specific language features, and these rationales often lean towards solving problems that system designers historically faced. Oftentimes, I think folks think of modern languages in regard to their syntax, tooling, ergonomics, etc; however, to me, the more interesting benefit in adopting a modern language is in how its inbuilt features address design problems that earlier generation languages exposed. For a real world example of what I'm talking about, you can google "clojure expression problem" and find compelling articles about how Clojure solves this with protocols. Providing a toolkit for attacking categories of problems inherently gets people focused on the fact that these problems exist in the first place when they may not even recognize them otherwise, and regardless of the choice of language, it leads to better design oriented thinking in the context of larger and more complex systems.
  - doyougnu 5y ago
    Clojure doesn't solve the expression problem and the expression problem tends to be trivial in dynamically typed languages. Strictly speaking the expression problem is only defined for _static_ type systems because its concerned with _static_ type safety and extensions without recompilation.
    - tmountain 5y ago
      It does solve it and doing so was a key design goal of protocols. It's right there in the docs: There are several motivations for protocols: Avoid the 'expression problem' by allowing independent extension of the set of types, protocols, and implementations of protocols on types, by different parties I agree that it's a concern in static type systems, but the same issue rears its head when defining methods which operate on specific types of strongly typed data, so no, it's not always trivial, nor is it a problem exclusive to statically typed languages, and if it was, there wouldn't be a need to introduce a new abstraction for which addressing the problem is a key goal.
      - deleted 5y ago
- AtNightWeCode 5y ago
  What I heard from colleagues that work with Clojure is that it is a horrible language where the default way of writing code is an imperative programming style where contexts are passed around and updated. Far from the concepts of functional programming.
  - stingraycharles 5y ago
    I’m not sure I understand what you mean. I don’t consider Clojure to be imperative at all — everything is immutable by default, and you actually have to go through considerable efforts to write things in an imperative style. When I compare Clojure to another functional language I know well, Haskell, one of the things I really feel it lacks is proper pattern matching and currying; yes there are libraries that you can use, but it’s just not idiomatic to do in Clojure. I would not, however, assert that it’s an imperative language. Could you care to elaborate on what exactly you find is lacking in Clojure?
    - davidrupp 5y ago
      It's not difficult to write imperative code in Clojure; [1] is an example. Immutability-by-default makes you work to mutate things, for sure, but that in itself doesn't make the language inherently functional. [1] https://clojuredocs.org/clojure.core/let#example-542692c7c026201cdc3269c2 https://clojuredocs.org/clojure.core/let#example-542692c7c02...
      - robthethird 5y ago
        Your link might be pointing to the wrong example. I assume you wanted to show an example of an atom.
        davidrupp 5y ago
        Nope. The sequence of statements in the (let) block is imperative. Atoms are actually an example of how the values of references are updated functionally. Clojure has facilities for both imperative and functional style.
bradleybuda 5y ago
> Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem. This is a great insight no matter what language or framework you're operating in. Laziness has its virtues, but invariants, validity checks, run-time type checks, etc. should all be performed as early (and often) as possible - it's much easier to debug issues when the are proximate to an object or structure being created or modified, then when that data type is being used much later.
- vbsteven 5y ago
  Yes, and this is where statically typed languages shine (in my opinion). I like programming in a style that makes heavy use of the type system to enforce this. For example when writing an api endpoint to create a task I would typically deserialise the json into a CreateTaskRequest. If the object is created without exceptions I can be sure it is valid. CreateTaskRequest implements the ToTask interface. The service layer takes only objects of this interface and converts into a Task object that gets persisted. The persisted Task then gets converted into a TaskResponse so only valid JSON comes back out. Lots of classes and interfaces but they are all small and with a single purpose.
  - Quekid5 5y ago
    > Yes, and this is where statically typed languages shine (in my opinion). Indeed. This feels like a just-before-the-moment-of-realization situation. The endless cycle between "more dynamic" and "more static" continues it seems. I wonder if there is any correlation between experience in the field and static vs. dynamic vs. "fail fast dynamic". (I'd say Erlang falls in the latter category and it has a pretty good track record for reliability, but so does Python. It's an imperfect axis for sure.)
  - habibur 5y ago
    > Lots of classes and interfaces but they are all small and with a single purpose. That's the side effect. You ultimately end up with more code and not less even though it saves you from type checks. There are trade offs for both.
chairhairair 5y ago
Is there more info about the tool itself that claims 100X decrease in application development cost? Quite the claim.
- yamrzou 5y ago
  It's still in stealth mode. They raised $5M in 2019: https://news.ycombinator.com/item?id=19565267 https://news.ycombinator.com/item?id=19565267
  - swyx 5y ago
    how exactly is it stealth if they have a blog and announced funding?
    - yamrzou 5y ago
      I meant that they didn't reveal the product yet.
- mberning 5y ago
  It does seem odd. Stealth product. Bold claims. 3 blog posts one of which is a funding announcement and the other two having nothing to do with the product. I am intrigued but also a bit skeptical.
  - davidrupp 5y ago
    Nathan Marz was the original creator of what became Apache Storm [1], which powered Twitter for some time. Skepticism is healthy, perhaps even warranted here, but I'm not betting against him just yet. [1] https://en.wikipedia.org/wiki/Apache_Storm https://en.wikipedia.org/wiki/Apache_Storm
    - sortalongo 5y ago
      “Powered Twitter” is an overstatement. Storm was indeed used at Twitter for select streaming use cases, but it was a bit of a mess and ended up being rewritten from the ground up for 10x improvements in latency and throughout [1]. Marz was at the company for < 2 years. Lately, Twitter has been moving data processing use cases to GCP [2]. Storm is also not very well regarded in the stream processing community due to its restrictive model, poor guarantees, and abysmal performance [3]. I have nothing against Marz, but I do think skepticism is warranted until we see what they’ve built. [1] https://blog.twitter.com/engineering/en_us/a/2015/flying-faster-with-twitter-heron.html https://blog.twitter.com/engineering/en_us/a/2015/flying-fas... [2] https://blog.twitter.com/engineering/en_us/topics/infrastructure/2018/a-new-collaboration-with-google-cloud.html https://blog.twitter.com/engineering/en_us/topics/infrastruc... [3] I worked at Twitter for 3 years, then at Google on Millwheel and Streaming Dataflow.
misiti3780 5y ago
If I want to learn Clojure, where is the best place to start? I have a lot of experience with Python/Javascript now, and spent many years in C/C++/Objective C and Java. Also have some Go.
- sremani 5y ago
  https://purelyfunctional.tv/mini-guide/the-ultimate-guide-to-learning-clojure-for-free/ https://purelyfunctional.tv/mini-guide/the-ultimate-guide-to...
  - travisjungroth 5y ago
    Eric Normand’s videos are amazing. If someone is interested in Clojure, I highly recommend his Clojure for Beginners videos (or whatever the obvious beginner tutorial is in case I messed up the name). Even if you never touch Clojure again, it’s worth it for the aha moment that the whole thing builds up to about using plain data instead of code.
- pkd 5y ago
  If you are into books, I will recommend Clojure for the Brave and True which is free to read online [1], and Living Clojure [2], in that order. If you're into interactive kata-style problems, there's 4Clojure [3]. Also join the Clojurians's Slack [4] for the community. [1] https://www.braveclojure.com/clojure-for-the-brave-and-true/ https://www.braveclojure.com/clojure-for-the-brave-and-true/ [2] https://www.oreilly.com/library/view/living-clojure/9781491909270/ https://www.oreilly.com/library/view/living-clojure/97814919... [3] https://www.4clojure.com/ https://www.4clojure.com/ [4] https://clojurians.slack.com/ https://clojurians.slack.com/
  - markc 5y ago
    And Clojurian's zulip: https://clojurians.zulipchat.com https://clojurians.zulipchat.com
- ifFxhF938 5y ago
  The "Clojure for the Brave and True" ebook is a popular and free resource for beginners: https://www.braveclojure.com/ https://www.braveclojure.com/
nickik 5y ago
I don't really like 'Component'. I seems very clunky and we had a lot of issues with it and a lot incidental complexity in our codebase (now converted to Java). It was the first real system that did these sort of things but if I start a project now, I much rather use Integrant or Clip. https://github.com/weavejester/integrant https://github.com/weavejester/integrant https://github.com/juxt/clip https://github.com/juxt/clip I haven't used Clip a lot yet but my next project is defiantly going to be with Clip. For validation I have started to use Malli (https://github.com/metosin/malli https://github.com/metosin/malli). I really liked the idea behind clojure.spec but some of the implementation was a bit clunky and a bit too 'Rich Hickey', not sure how to describe it otherwise. Schema was again the first serious attempt at a library like that for Clojure so it was really nice when it came out. Malli sort of combines what is great about both.
amackera 5y ago
Thank you for sharing this detailed rundown. One of the challenges that I faced as a new Clojure developer was understanding how all the parts fit together (and in fact, what parts I should care about in the first place). Example: is learning Component worth it? (yes) Extremely glad to see this resource published! Thank you to the Red Planet team for releasing this!
deleted 5y ago
[deleted]
geospeck 5y ago
> And doing things dynamically means we can enforce stronger constraints than possible with static type systems. Can someone please explain this to a novice like me?
- elmers-glue 5y ago
  Most static time type systems can enforce that a voting age is an Integer, say, but can't validate that any value is at least 18, for example.
  - ajuc 5y ago
    Turbo Pascal could. In practice the only benefit is very similar to checked exceptions - can't forget to check that value is in range.
  - Zababa 5y ago
    They can, this is a misconception. Create a new type, and make it so that it's only produced by a function that checks if the age is superior to 18.
    - CraigJPerry 5y ago
      That's a poor hack, it moves an invariant that could be enforced at compile time to not much more than a convention that has to be preserved by code review. E.g. a colleague implements de-serialisation for your type but adds an empty constructor to make their life easier. You might not learn there's a hole in the boat before your first bug.
      - Zababa 5y ago
        I wouldn't call it a poor hack. In a way it's a parser, which aren't really poor hacks, but can be abused. Sure, it's not perfect and I'd like it better if it was checked at compile time, but it's way better than using a simple int. Also, the moment you have a bug, tracking it is really easy: just list the functions that returns this specific type.
jb1991 5y ago
I wonder why they are using Schema instead of Spec. Schema was popular before the latter existed, I didn’t realize anyone still uses it.
- whalesalad 5y ago
  That is probably exactly why they are using it ... you don't get to 250k lines of code overnight.
  - jb1991 5y ago
    My understanding is that the company is a few years younger than Spec, but perhaps the code base is very very old.
    - synthc 5y ago
      Schema is a good library. Everyone seems to use spec today, but Schema easier to use than spec IMO. Spec is also still in alpha, with spec2 still under development.
      - creese 5y ago
        We used clojure.spec in production for years. "Alpha" just means the API could change.
- ARandomerDude 5y ago
  I still prefer Schema because I find it more much more readable.
  - synthc 5y ago
    yes i also prefer it. With Schema, the defined schema's are normal vars, which you can jump to with editor support. With spec i'm always searching for the definitions.
rst13 5y ago
This is sick
swamiji 5y ago
this is wonderful - I've been curious about continuations and program state as a language construct. Not sure I understand the redplanet - type system et
fnord77 5y ago
we have a clojure codebase that's about 100k lines. Honestly I'm kinda fed up with it. Certain 3rd party libs we've used have been abandoned. We wrote our own libs for a major framework and it is failing behind. Too many "I'm very clever" functions that are hard to understand and also have subtle bugs.
- ashes-of-sol 5y ago
  Would you mind sharing the abandoned libs/framework? When you say you have too many "I'm clever" functions, do you mean within code your team wrote, or in the ecosystem at large?
  - fnord77 5y ago
    it is a couple of the clojurewerkz libs. the "I'm very clever" defns and macros that I'm referring to were written by our team. Most of the 3rd party libs are pretty good. Except for them being not maintained and not keeping up with the latest versions of databases, etc.
- dragandj 5y ago
  I wonder if they'd be abandoned if companies using them considered these 3rd party libraries valuable enough to contribute/fund the development.
- coneill 5y ago
  Seconded. I work in a clojure codebase that were trying to get out of. There's just dead libraries everywhere and stuff that maintained by one person that gets no updates at all. That or we just end up making functional "wrappers" around Java libraries and at that point we might as well just write straight Java. Also yea everyone wants to be so damn smart having macros within macros within macros that no one knows what the original intent of the code is anymore. The repl based development I find also breeds a really bad mentality of forgoing building a deployment process and instead people just repl in and make a bunch of changes and prod rarely matches whats checked into github.
  - 5y ago
logistark 5y ago
I would like to ask if you are comfortable of Clojure protocols, because i tend to avoid them. What do you thik about it?
- xpe 5y ago
  > I would like to ask if you Ask who?
- raspasov 5y ago
  I've been using Clojure for a while (8+ years) and I don't use them often. Unless you have a specific need (say, very high performance), regular Clojure maps give you most of the benefits without any obvious downsides.
chromanoid 5y ago
> One of the coolest parts of our codebase is the new general purpose language at its foundation. Though the semantics of the language are substantially different than Clojure, it’s defined entirely within Clojure using macros to express the differing behavior. It compiles directly to bytecode using the ASM library. The rest of our system is built using both this language and vanilla Clojure, interoperating seamlessly. Actually this sounds quite horrible.
- afro88 5y ago
  Imagine starting a job as a closure dev and being told you have to learn a new language with substantially different semantics
  - TeMPOraL 5y ago
    Everyone is doing it already, to a large extent. Where "an API" ends and "a programming language" starts is a matter of opinion. It's a fuzzy boundary.
  - xpe 5y ago
    Different semantics for different folks. For me, this sounds interesting. I'll defer judgment until I learn more.
  - jwhitlark 5y ago
    I think it really comes down to how well the new semantics fit the problem domain. If it's a much better fit than the alternative, it's a net win. But just because you can define new semantics, doesn't make it a good idea. This is Nathan Marz we're talking about though; he's on the short list of people who can both accurately identify the need and create something workable to fill it. If he told me it was necessary for a particular problem domain, I (personally) would think long and hard before I disagreed.
  - bpodgursky 5y ago
    I know people who work at RPL, am 1000% confident that anyone who joins knows _exactly_ what they are getting into (and this kind of crazy Nathan Marz rabbit hole is exactly why they signed up).
Xevi 5y ago
> Our codebase consists of 250k lines of Clojure split evenly between source and test code. It’s one of the largest Clojure codebases in the world. Aren't there many Clojure projects out there, or is the language generally not used for large projects? I have a JavaScript frontend written in Vue that is 150k+ lines of code, without any tests. Which would be 25k lines of code more than they have, disregarding their tests. I'm not saying that more code is better, I just found it odd that 250k was considered one of the largest Clojure codebases in the world
- chungus 5y ago
  In general clojure can be a very terse language. A lot of mileage per line of code, so to speak.
- AlexCoventry 5y ago
  At least according to Paul Graham, one of the appealing things about Lisp is the economy of expression. Not many Clojurists would brag about how big their code base is, I think. http://www.paulgraham.com/popular.html http://www.paulgraham.com/popular.html
coltnz 5y ago
While I've used all of them, some of their listed libraries are a little dated IMO. Of course this true of almost all mature codebases. For the sake of the less experienced, I'd point them to these substitutions in particular: Compojure: Reitit (which they used in the front end too, so migration maybe in progress) would be my preference for backend routing. Component: Integrant takes Component's ideas, but prefers the flexibility of multimethods operating on plain data to the typed records approach for defining systems. Schema: Once very popular, but superseded by clojure.spec (bigger in scope) and to a lesser degree Malli for data schemas. Potemkin: Avoid, handy for some internal code organisation purposes but hostile to tooling and debugging IMO.
- jwr 5y ago
  In a lage codebase, the cost of switching is huge, and if older libraries work well, there is often no clear incentive to do so. Clojure in general being very stable and backwards-compatible makes it even more easier to just continue using older libraries. So what if the library is "dated"? If it works well, why not continue using it? (I speak from my own experience)
  - coltnz 5y ago
    Totally agree, I didn't say they shouldn't continue using them.
rebelos 5y ago
This is the first time I'm looking closely at Clojure code and it seems wholly antithetical to the practice of software engineering. Not very readable, too many ways to create magical jank, odd coding conventions, and it looks like refactoring and maintenance would be a nightmare. I'm very glad the community has had the collective sense to not adopt this widely.
- django77 5y ago
  It's weird to dismiss something so easily after seeing it for the first time. Maybe you should give it more time.
- cutler 5y ago
  By "looking closely at" do you mean you've actually grokked the language and used it to build anything?
dmcclurg 5y ago
I jumped to the comments first, and my initial thought was, “what would Nathan Marz think of all of this...and what is he investing in these days?” I was honestly skeptical, reading all of this clojure-macro criticism (constructive for the most part), but trust was restored when I saw the author. Well done, sir! Excited to see the details.
jwr 5y ago
I develop and maintain a 60k line Clojure+ClojureScript codebase by myself, so I can definitely confirm that Clojure does allow for smaller teams to maintain larger codebases :-) I also fully agree with this: > Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem. I made it a rule to perform integrity checks as early as possible: when creating, accepting or transforming data. I use spec (but schema would work just as well here), and have lots of pre/post conditions in my code. I tend to settle on "simpler" solutions. I use mount, rather than component, because it uses the namespace hierarchy and requires less code and management. I use very few macros, and try to use simpler tools rather than more complex ones. I noticed that these days roughly 30-40% of the code I write deals with integrity checks, anomalies and anomaly handling.
- heyzk 5y ago
  > I noticed that these days roughly 30-40% of the code I write deals with integrity checks, anomalies and anomaly handling. Love to hear more about this. My approach has been to convert errors into data and have behavior that deals with conveying these errors in different ways, but it's always felt too complex for the task. I've just got a lot of stuff dealing with handling errors in the different environments (jvm / browser / nodejs / rn) and across async and non-async code.
logistark 5y ago
Seeing that there is a need for type checking in Clojure. Has anyone used https://github.com/typedclojure/typedclojure https://github.com/typedclojure/typedclojure in production?
mnming 5y ago
Really a great article! It's extremely rare to find people who truly understand Clojure/Lisp willing to share to this detail. Having said that, the article also exposes some flaws of Clojure that I also found. 1. Clojure overly obsesses with expression and non-procedural style coding. While in reality many large scale Clojure repo deploy their own macro to bring procedural style back (like letlocal in the article). 2. The builtin abstraction tools are almost always too simple to be useful. But those are not big deals IMO. At the same time, I also wonder if this article mention any feature of Clojure that is truly unique to Clojure compared to other Lisp/Scheme languages. I wonder if the article will still make sense if we simply substitute all "Clojure" to "Racket" (obviously I know ecosystem is not comparable).

373 points by grzm 5y ago

This is what I imagine experienced clojure developers can squeeze out of a language like clojure. I would venture that they can train a junior programmer in a couple of weeks, and make them productive very fast. I guess they could make it work in any language but judging by the description clojure is indeed a great fit, due to the macro capabilities, flexibility and solid runtime via JVM.

cliftonk 5y ago
I've found I typically reach for clojure when i need to do something on the jvm and want a better java than java.
- lukashrb 5y ago
  Interesting take since Clojure and Java are two very different languages. And unlike for example Kotlin, Clojure does not try to be a better Java than Java. But true though Clojure leverages the power of the jvm.
  - forgetfulness 5y ago
    It was a better Java in many ways. The doto macro was a relief back in the days when Java APIs insisted on being designed around stateful setters and getters rather than the builder pattern, it allowed you to operate on such unfortunately designed objects in single logical blocks and simulating it all being an expression. Proxy allowed you to instantiate anonymous inner classes implementing only the methods of an interface that you needed, the rest you could omit; in Java you have to put them all, empty, which necessitated that you use an IDE to generate them, and back then IDEs were not as nice as today. Those two alone made interacting with contemporary Java libraries so much easier. It also was convenient to go from Java collections to Clojure collections and vice versa.
    - pjmlp 5y ago
      " We were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp." -- Guy Steele
  - pjmlp 5y ago
    Basically you get a new version of a Lisp Machine, that you can sneak up in lots of places, whereas with Common Lisp it isn't so easy to do so.
  - __jem 5y ago
    Interop from Clojure -> Java is still incredibly easy, though. I often will just wrap a Java library rather than look for a pre-existing Clojure implementation because it's so easy. Of course, most Java libs don't value immutability and functional patterns, but you can still push this kind of interop to the edges of your system and keep everything else in pure Clojure.
    - StreamBright 5y ago
      Depends on the style the Java library you are trying to use is written in. I have had mixed results. Especially the Java 8 functional style had some challenges. I might have the relevant SO question somewhere.
- forgetfulness 5y ago
  I think that's also why Clojure use peaked right before Java 8 was released; once Java became a better Java, and libraries started to be designed around more ergonomic APIs than wiring up objects that needed miles of stateful configuration code, the pressure that drove you out of Java and into Clojure began to diminish.
  - cutler 5y ago
    Hardly. Despite being Java-compatible Clojure's philosophy is diametrically opposed to that of Java/OOP so anyone who has experienced the benefits of Clojure would hardly return to Java simply because it managed to fake FP with Single Abstract Methods.
    - forgetfulness 5y ago
      Well, maybe there were not as many die-hard lispers within the Clojure community and many were looking for something that handled better than Java 7. I know two other former Clojure programmers that went to Scala, but also find Java passable enough to just use it rather than Clojure.
- dustingetz 5y ago
  Clojure's value prop is: 1) default immutability (same simple data structures used in every library -> ecosystem composes better) 2) portable code across multiple host platforms (jvm, node, browser) 3) metaprogramming IMO, in 2007, immutability on the JVM was a competitive value prop, but in 2021+ it is nothing special. It is the combination of the three things which is a competitive value prop today. Metaprogramming in particular is a mostly unexplored frontier, because it is very hard to do well, and very easy to do badly. Default immutability is kind of a necessary starting point to do metaprogramming well.
  - davidrupp 5y ago
    > in 2007, immutability on the JVM was a competitive value prop, but in 2021+ it is nothing special Can you elaborate on this? What do you think has changed in that time to make it "nothing special"?
    - deleted 5y ago
      [deleted]
    - dustingetz 5y ago
      immutability as a library is available in basically all mainstream languages now and mainstream frameworks leverage it (react, any UI framework, spark, any data framework or database); JS vms are competitive with the JVM and the JVM might even be losing ground in the cloud; typescript is a monster and is letting people explore haskell concepts in industry applications; scala is way better in 2021 than it was in 2011; every new PL can compile to JS and supports immutability. Clojure's sweet spot is currently developing sub-languages embedded in Clojure like RPL is doing (we are doing exactly the same thing at hyperfiddle). That is still too hard to do in typescript imo. Maybe it is possible in scala 3 but it took them 10 years of pure autism to figure out how to do monadic IO in scala due to how complex scala is, so i'd expect it to take another 10 years to figure out how to do metaprogramming in a commercially viable way, I'd be happy to be proven wrong.
      - davidrupp 5y ago
trutannus 5y ago
> would venture that they can train a junior programmer in a couple of weeks, and make them productive very fast I've got some experience with functional languages, and a good amount with functional features in OO languages. Started learning Clojure this week, and was pleasantly surprised with how quickly I could get working projects up and running. Less upfront learning curve than Elixir, which was unexpected.
tmountain 5y ago
Yeah, it's nice to see such thoughtful adoption of language facilities clearly oriented towards creating a successful and maintainable codebase. The Clojure team has always done a nice job expressing the rationale for specific language features, and these rationales often lean towards solving problems that system designers historically faced. Oftentimes, I think folks think of modern languages in regard to their syntax, tooling, ergonomics, etc; however, to me, the more interesting benefit in adopting a modern language is in how its inbuilt features address design problems that earlier generation languages exposed. For a real world example of what I'm talking about, you can google "clojure expression problem" and find compelling articles about how Clojure solves this with protocols. Providing a toolkit for attacking categories of problems inherently gets people focused on the fact that these problems exist in the first place when they may not even recognize them otherwise, and regardless of the choice of language, it leads to better design oriented thinking in the context of larger and more complex systems.
- doyougnu 5y ago
  Clojure doesn't solve the expression problem and the expression problem tends to be trivial in dynamically typed languages. Strictly speaking the expression problem is only defined for _static_ type systems because its concerned with _static_ type safety and extensions without recompilation.
  - tmountain 5y ago
    It does solve it and doing so was a key design goal of protocols. It's right there in the docs: There are several motivations for protocols: Avoid the 'expression problem' by allowing independent extension of the set of types, protocols, and implementations of protocols on types, by different parties I agree that it's a concern in static type systems, but the same issue rears its head when defining methods which operate on specific types of strongly typed data, so no, it's not always trivial, nor is it a problem exclusive to statically typed languages, and if it was, there wouldn't be a need to introduce a new abstraction for which addressing the problem is a key goal.
    - deleted 5y ago
AtNightWeCode 5y ago
What I heard from colleagues that work with Clojure is that it is a horrible language where the default way of writing code is an imperative programming style where contexts are passed around and updated. Far from the concepts of functional programming.
- stingraycharles 5y ago
  I’m not sure I understand what you mean. I don’t consider Clojure to be imperative at all — everything is immutable by default, and you actually have to go through considerable efforts to write things in an imperative style. When I compare Clojure to another functional language I know well, Haskell, one of the things I really feel it lacks is proper pattern matching and currying; yes there are libraries that you can use, but it’s just not idiomatic to do in Clojure. I would not, however, assert that it’s an imperative language. Could you care to elaborate on what exactly you find is lacking in Clojure?
  - davidrupp 5y ago
    It's not difficult to write imperative code in Clojure; [1] is an example. Immutability-by-default makes you work to mutate things, for sure, but that in itself doesn't make the language inherently functional. [1] https://clojuredocs.org/clojure.core/let#example-542692c7c026201cdc3269c2 https://clojuredocs.org/clojure.core/let#example-542692c7c02...
    - robthethird 5y ago
      Your link might be pointing to the wrong example. I assume you wanted to show an example of an atom.
      - davidrupp 5y ago
        Nope. The sequence of statements in the (let) block is imperative. Atoms are actually an example of how the values of references are updated functionally. Clojure has facilities for both imperative and functional style.

bradleybuda 5y ago

> Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem. This is a great insight no matter what language or framework you're operating in. Laziness has its virtues, but invariants, validity checks, run-time type checks, etc. should all be performed as early (and often) as possible - it's much easier to debug issues when the are proximate to an object or structure being created or modified, then when that data type is being used much later.

vbsteven 5y ago
Yes, and this is where statically typed languages shine (in my opinion). I like programming in a style that makes heavy use of the type system to enforce this. For example when writing an api endpoint to create a task I would typically deserialise the json into a CreateTaskRequest. If the object is created without exceptions I can be sure it is valid. CreateTaskRequest implements the ToTask interface. The service layer takes only objects of this interface and converts into a Task object that gets persisted. The persisted Task then gets converted into a TaskResponse so only valid JSON comes back out. Lots of classes and interfaces but they are all small and with a single purpose.
- Quekid5 5y ago
  > Yes, and this is where statically typed languages shine (in my opinion). Indeed. This feels like a just-before-the-moment-of-realization situation. The endless cycle between "more dynamic" and "more static" continues it seems. I wonder if there is any correlation between experience in the field and static vs. dynamic vs. "fail fast dynamic". (I'd say Erlang falls in the latter category and it has a pretty good track record for reliability, but so does Python. It's an imperfect axis for sure.)
- habibur 5y ago
  > Lots of classes and interfaces but they are all small and with a single purpose. That's the side effect. You ultimately end up with more code and not less even though it saves you from type checks. There are trade offs for both.

chairhairair 5y ago

Is there more info about the tool itself that claims 100X decrease in application development cost? Quite the claim.

yamrzou 5y ago
It's still in stealth mode. They raised $5M in 2019: https://news.ycombinator.com/item?id=19565267 https://news.ycombinator.com/item?id=19565267
- swyx 5y ago
  how exactly is it stealth if they have a blog and announced funding?
  - yamrzou 5y ago
    I meant that they didn't reveal the product yet.
mberning 5y ago
It does seem odd. Stealth product. Bold claims. 3 blog posts one of which is a funding announcement and the other two having nothing to do with the product. I am intrigued but also a bit skeptical.
- davidrupp 5y ago
  Nathan Marz was the original creator of what became Apache Storm [1], which powered Twitter for some time. Skepticism is healthy, perhaps even warranted here, but I'm not betting against him just yet. [1] https://en.wikipedia.org/wiki/Apache_Storm https://en.wikipedia.org/wiki/Apache_Storm
  - sortalongo 5y ago
    “Powered Twitter” is an overstatement. Storm was indeed used at Twitter for select streaming use cases, but it was a bit of a mess and ended up being rewritten from the ground up for 10x improvements in latency and throughout [1]. Marz was at the company for < 2 years. Lately, Twitter has been moving data processing use cases to GCP [2]. Storm is also not very well regarded in the stream processing community due to its restrictive model, poor guarantees, and abysmal performance [3]. I have nothing against Marz, but I do think skepticism is warranted until we see what they’ve built. [1] https://blog.twitter.com/engineering/en_us/a/2015/flying-faster-with-twitter-heron.html https://blog.twitter.com/engineering/en_us/a/2015/flying-fas... [2] https://blog.twitter.com/engineering/en_us/topics/infrastructure/2018/a-new-collaboration-with-google-cloud.html https://blog.twitter.com/engineering/en_us/topics/infrastruc... [3] I worked at Twitter for 3 years, then at Google on Millwheel and Streaming Dataflow.

misiti3780 5y ago

If I want to learn Clojure, where is the best place to start? I have a lot of experience with Python/Javascript now, and spent many years in C/C++/Objective C and Java. Also have some Go.

sremani 5y ago
https://purelyfunctional.tv/mini-guide/the-ultimate-guide-to-learning-clojure-for-free/ https://purelyfunctional.tv/mini-guide/the-ultimate-guide-to...
- travisjungroth 5y ago
  Eric Normand’s videos are amazing. If someone is interested in Clojure, I highly recommend his Clojure for Beginners videos (or whatever the obvious beginner tutorial is in case I messed up the name). Even if you never touch Clojure again, it’s worth it for the aha moment that the whole thing builds up to about using plain data instead of code.
pkd 5y ago
If you are into books, I will recommend Clojure for the Brave and True which is free to read online [1], and Living Clojure [2], in that order. If you're into interactive kata-style problems, there's 4Clojure [3]. Also join the Clojurians's Slack [4] for the community. [1] https://www.braveclojure.com/clojure-for-the-brave-and-true/ https://www.braveclojure.com/clojure-for-the-brave-and-true/ [2] https://www.oreilly.com/library/view/living-clojure/9781491909270/ https://www.oreilly.com/library/view/living-clojure/97814919... [3] https://www.4clojure.com/ https://www.4clojure.com/ [4] https://clojurians.slack.com/ https://clojurians.slack.com/
- markc 5y ago
  And Clojurian's zulip: https://clojurians.zulipchat.com https://clojurians.zulipchat.com
ifFxhF938 5y ago
The "Clojure for the Brave and True" ebook is a popular and free resource for beginners: https://www.braveclojure.com/ https://www.braveclojure.com/

nickik 5y ago

I don't really like 'Component'. I seems very clunky and we had a lot of issues with it and a lot incidental complexity in our codebase (now converted to Java). It was the first real system that did these sort of things but if I start a project now, I much rather use Integrant or Clip. https://github.com/weavejester/integrant https://github.com/weavejester/integrant https://github.com/juxt/clip https://github.com/juxt/clip I haven't used Clip a lot yet but my next project is defiantly going to be with Clip. For validation I have started to use Malli (https://github.com/metosin/malli https://github.com/metosin/malli). I really liked the idea behind clojure.spec but some of the implementation was a bit clunky and a bit too 'Rich Hickey', not sure how to describe it otherwise. Schema was again the first serious attempt at a library like that for Clojure so it was really nice when it came out. Malli sort of combines what is great about both.

amackera 5y ago

Thank you for sharing this detailed rundown. One of the challenges that I faced as a new Clojure developer was understanding how all the parts fit together (and in fact, what parts I should care about in the first place). Example: is learning Component worth it? (yes) Extremely glad to see this resource published! Thank you to the Red Planet team for releasing this!

deleted 5y ago

[deleted]

geospeck 5y ago

> And doing things dynamically means we can enforce stronger constraints than possible with static type systems. Can someone please explain this to a novice like me?

elmers-glue 5y ago
Most static time type systems can enforce that a voting age is an Integer, say, but can't validate that any value is at least 18, for example.
- ajuc 5y ago
  Turbo Pascal could. In practice the only benefit is very similar to checked exceptions - can't forget to check that value is in range.
- Zababa 5y ago
  They can, this is a misconception. Create a new type, and make it so that it's only produced by a function that checks if the age is superior to 18.
  - CraigJPerry 5y ago
    That's a poor hack, it moves an invariant that could be enforced at compile time to not much more than a convention that has to be preserved by code review. E.g. a colleague implements de-serialisation for your type but adds an empty constructor to make their life easier. You might not learn there's a hole in the boat before your first bug.
    - Zababa 5y ago
      I wouldn't call it a poor hack. In a way it's a parser, which aren't really poor hacks, but can be abused. Sure, it's not perfect and I'd like it better if it was checked at compile time, but it's way better than using a simple int. Also, the moment you have a bug, tracking it is really easy: just list the functions that returns this specific type.

jb1991 5y ago

I wonder why they are using Schema instead of Spec. Schema was popular before the latter existed, I didn’t realize anyone still uses it.

whalesalad 5y ago
That is probably exactly why they are using it ... you don't get to 250k lines of code overnight.
- jb1991 5y ago
  My understanding is that the company is a few years younger than Spec, but perhaps the code base is very very old.
  - synthc 5y ago
    Schema is a good library. Everyone seems to use spec today, but Schema easier to use than spec IMO. Spec is also still in alpha, with spec2 still under development.
    - creese 5y ago
      We used clojure.spec in production for years. "Alpha" just means the API could change.
ARandomerDude 5y ago
I still prefer Schema because I find it more much more readable.
- synthc 5y ago
  yes i also prefer it. With Schema, the defined schema's are normal vars, which you can jump to with editor support. With spec i'm always searching for the definitions.

rst13 5y ago

This is sick

swamiji 5y ago

this is wonderful - I've been curious about continuations and program state as a language construct. Not sure I understand the redplanet - type system et

fnord77 5y ago

we have a clojure codebase that's about 100k lines. Honestly I'm kinda fed up with it. Certain 3rd party libs we've used have been abandoned. We wrote our own libs for a major framework and it is failing behind. Too many "I'm very clever" functions that are hard to understand and also have subtle bugs.

ashes-of-sol 5y ago
Would you mind sharing the abandoned libs/framework? When you say you have too many "I'm clever" functions, do you mean within code your team wrote, or in the ecosystem at large?
- fnord77 5y ago
  it is a couple of the clojurewerkz libs. the "I'm very clever" defns and macros that I'm referring to were written by our team. Most of the 3rd party libs are pretty good. Except for them being not maintained and not keeping up with the latest versions of databases, etc.
dragandj 5y ago
I wonder if they'd be abandoned if companies using them considered these 3rd party libraries valuable enough to contribute/fund the development.
coneill 5y ago
Seconded. I work in a clojure codebase that were trying to get out of. There's just dead libraries everywhere and stuff that maintained by one person that gets no updates at all. That or we just end up making functional "wrappers" around Java libraries and at that point we might as well just write straight Java. Also yea everyone wants to be so damn smart having macros within macros within macros that no one knows what the original intent of the code is anymore. The repl based development I find also breeds a really bad mentality of forgoing building a deployment process and instead people just repl in and make a bunch of changes and prod rarely matches whats checked into github.
- 5y ago

logistark 5y ago

I would like to ask if you are comfortable of Clojure protocols, because i tend to avoid them. What do you thik about it?

xpe 5y ago
> I would like to ask if you Ask who?
raspasov 5y ago
I've been using Clojure for a while (8+ years) and I don't use them often. Unless you have a specific need (say, very high performance), regular Clojure maps give you most of the benefits without any obvious downsides.

chromanoid 5y ago

> One of the coolest parts of our codebase is the new general purpose language at its foundation. Though the semantics of the language are substantially different than Clojure, it’s defined entirely within Clojure using macros to express the differing behavior. It compiles directly to bytecode using the ASM library. The rest of our system is built using both this language and vanilla Clojure, interoperating seamlessly. Actually this sounds quite horrible.

afro88 5y ago
Imagine starting a job as a closure dev and being told you have to learn a new language with substantially different semantics
- TeMPOraL 5y ago
  Everyone is doing it already, to a large extent. Where "an API" ends and "a programming language" starts is a matter of opinion. It's a fuzzy boundary.
- xpe 5y ago
  Different semantics for different folks. For me, this sounds interesting. I'll defer judgment until I learn more.
- jwhitlark 5y ago
  I think it really comes down to how well the new semantics fit the problem domain. If it's a much better fit than the alternative, it's a net win. But just because you can define new semantics, doesn't make it a good idea. This is Nathan Marz we're talking about though; he's on the short list of people who can both accurately identify the need and create something workable to fill it. If he told me it was necessary for a particular problem domain, I (personally) would think long and hard before I disagreed.
- bpodgursky 5y ago
  I know people who work at RPL, am 1000% confident that anyone who joins knows _exactly_ what they are getting into (and this kind of crazy Nathan Marz rabbit hole is exactly why they signed up).

Xevi 5y ago

> Our codebase consists of 250k lines of Clojure split evenly between source and test code. It’s one of the largest Clojure codebases in the world. Aren't there many Clojure projects out there, or is the language generally not used for large projects? I have a JavaScript frontend written in Vue that is 150k+ lines of code, without any tests. Which would be 25k lines of code more than they have, disregarding their tests. I'm not saying that more code is better, I just found it odd that 250k was considered one of the largest Clojure codebases in the world

chungus 5y ago
In general clojure can be a very terse language. A lot of mileage per line of code, so to speak.
AlexCoventry 5y ago
At least according to Paul Graham, one of the appealing things about Lisp is the economy of expression. Not many Clojurists would brag about how big their code base is, I think. http://www.paulgraham.com/popular.html http://www.paulgraham.com/popular.html

coltnz 5y ago

While I've used all of them, some of their listed libraries are a little dated IMO. Of course this true of almost all mature codebases. For the sake of the less experienced, I'd point them to these substitutions in particular: Compojure: Reitit (which they used in the front end too, so migration maybe in progress) would be my preference for backend routing. Component: Integrant takes Component's ideas, but prefers the flexibility of multimethods operating on plain data to the typed records approach for defining systems. Schema: Once very popular, but superseded by clojure.spec (bigger in scope) and to a lesser degree Malli for data schemas. Potemkin: Avoid, handy for some internal code organisation purposes but hostile to tooling and debugging IMO.

jwr 5y ago
In a lage codebase, the cost of switching is huge, and if older libraries work well, there is often no clear incentive to do so. Clojure in general being very stable and backwards-compatible makes it even more easier to just continue using older libraries. So what if the library is "dated"? If it works well, why not continue using it? (I speak from my own experience)
- coltnz 5y ago
  Totally agree, I didn't say they shouldn't continue using them.

rebelos 5y ago

This is the first time I'm looking closely at Clojure code and it seems wholly antithetical to the practice of software engineering. Not very readable, too many ways to create magical jank, odd coding conventions, and it looks like refactoring and maintenance would be a nightmare. I'm very glad the community has had the collective sense to not adopt this widely.

django77 5y ago
It's weird to dismiss something so easily after seeing it for the first time. Maybe you should give it more time.
cutler 5y ago
By "looking closely at" do you mean you've actually grokked the language and used it to build anything?

dmcclurg 5y ago

I jumped to the comments first, and my initial thought was, “what would Nathan Marz think of all of this...and what is he investing in these days?” I was honestly skeptical, reading all of this clojure-macro criticism (constructive for the most part), but trust was restored when I saw the author. Well done, sir! Excited to see the details.

jwr 5y ago

I develop and maintain a 60k line Clojure+ClojureScript codebase by myself, so I can definitely confirm that Clojure does allow for smaller teams to maintain larger codebases :-) I also fully agree with this: > Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem. I made it a rule to perform integrity checks as early as possible: when creating, accepting or transforming data. I use spec (but schema would work just as well here), and have lots of pre/post conditions in my code. I tend to settle on "simpler" solutions. I use mount, rather than component, because it uses the namespace hierarchy and requires less code and management. I use very few macros, and try to use simpler tools rather than more complex ones. I noticed that these days roughly 30-40% of the code I write deals with integrity checks, anomalies and anomaly handling.

heyzk 5y ago
> I noticed that these days roughly 30-40% of the code I write deals with integrity checks, anomalies and anomaly handling. Love to hear more about this. My approach has been to convert errors into data and have behavior that deals with conveying these errors in different ways, but it's always felt too complex for the task. I've just got a lot of stuff dealing with handling errors in the different environments (jvm / browser / nodejs / rn) and across async and non-async code.

logistark 5y ago

Seeing that there is a need for type checking in Clojure. Has anyone used https://github.com/typedclojure/typedclojure https://github.com/typedclojure/typedclojure in production?

mnming 5y ago

Really a great article! It's extremely rare to find people who truly understand Clojure/Lisp willing to share to this detail. Having said that, the article also exposes some flaws of Clojure that I also found. 1. Clojure overly obsesses with expression and non-procedural style coding. While in reality many large scale Clojure repo deploy their own macro to bring procedural style back (like letlocal in the article). 2. The builtin abstraction tools are almost always too simple to be useful. But those are not big deals IMO. At the same time, I also wonder if this article mention any feature of Clojure that is truly unique to Clojure compared to other Lisp/Scheme languages. I wonder if the article will still make sense if we simply substitute all "Clojure" to "Racket" (obviously I know ecosystem is not comparable).