16 ms·

Why is Jepsen written in Clojure?

469 points by manicennui 3y ago

metadat 3y ago
> Jepsen’s (gosh) about a decade old now I thought Jepsen is more like 12 years old now, going by when I first heard about it circa 2011.
- tomnipotent 3y ago
  The song that inspired the name was released in 2011. Aphyr's "Call Me Maybe" blog series/talk started in 2013, and the commercial audits we've come to love came later.
- dllthomas 3y ago
  From a certain perspective, Jepsen is 38 years old.
- michaelsbradley 3y ago
  less than 15, “about” can go +/-
contrarian1234 3y ago
out of curiosity, does it use core.async or manifold for concurrency? (and what are the tradeoffs)
- stefcoetzee 3y ago
  Seems like `core.async`: https://github.com/search?q=repo%3Ajepsen-io%2Fjepsen+core.async&type=code https://github.com/search?q=repo%3Ajepsen-io%2Fjepsen+core.a... (As an aside: Missionary is a higher-level, more recent option in this area.)
  - contrarian1234 3y ago
    Do you have experience using it? Missionary looks very cool, but I don't think anyone understands how to use it. Other than Electric/Rama (from the same people) I haven't been able to find any projects that use it on Github
    - stefcoetzee 3y ago
      I've only kicked the tires, not used it in anger. The tutorials are fairly accessible: https://cljdoc.org/d/missionary/missionary/b.33/doc/readme/tutorials/hello-task https://cljdoc.org/d/missionary/missionary/b.33/doc/readme/t... In a recent London Clojurians talk, the author Leo Noel explained Electric can be thought of as a frontend for Missionary. Rama is by a different team (RPL), but agreed, their philosophies are similar.
      - contrarian1234 3y ago
        Oh they updated the docs! Then I'll have to give it another look. I tried to make a simple subscription/memoization setup but I couldn't figure it out. They have a lot of keywords/terms (task/flow/etc.) that a bit hard to understand.. and even when you make sense of them isolation it was still a bit unclear how to compose them properly. The resulting code had to specify a lot of things and was more verbose than I had hoped for I'll be honest, part of the problem is I don't work in the web space. So I don't care for (or completely understand) the things they're looking to solve (ex: server/client sync issues). I just want to have simpler state-management and have changes propagate across my program in a way that makes my life easier :) (think like a "notebook" where you update values and everything gets reevaluated where-needed)
- aphyr 3y ago
  Nope! For a few reasons Jepsen sticks very closely to Traditional JVM Threads. Part of that is because Jepsen isn't hyper-concurrent. Most concurrency errors manifest in just a handful of client threads, and the JVM will run ~10K threads without too many issues. Another part is that I spend a lot of time debugging and profiling. Core.async turns stacktraces into Swiss cheese--it has to, to perform its implicit continuation-passing magic. But that can make it really hard to figure out Who Called What When, and Why Is This Process Slow. Standard blocking IO calls are also something that my profiler (YourKit) can analyze well. Finally, I invest a lot of time into tuning Jepsen for speed, and manage the handoff of data between threads carefully. In a system where actors were handing off state constantly core.async would be a bigger help, but there's really only a couple shapes for concurrent handoff in Jepsen, and they're addressed by either j.u.c. queues, forkjoin-style structured concurrency, or Weird Transactional Threadpool stuff. For a little more on this, take a look at https://github.com/aphyr/dom-top https://github.com/aphyr/dom-top or https://github.com/jepsen-io/history/blob/main/src/jepsen/history/task.clj https://github.com/jepsen-io/history/blob/main/src/jepsen/hi...
leoszliard 3y ago
Also, Clojure's fun
- resonious 3y ago
  This is underrated. I know it sounds unprofessional, but funness is also the reason Basecamp and Ruby on Rails was built in Ruby. Rails dominated web dev in the 2010s and tons of super successful startups were built on it. So while many folks like to scoff at engineers "playing with toys", a lot of good things we have now used to be toys (edit: Linux is another good example of this)
- scary-size 3y ago
  I think that captures perfectly! I've actually written about that sentiment in the past [1]. The tricky thing with a very "fun" language is that you assume that you can solve everything with it. You end up writing a lot of code, when, in the end, no code would have been an acceptable solution. [1] https://franz.hamburg/writing/clojure-makes-happy.html https://franz.hamburg/writing/clojure-makes-happy.html
  - wavemode 3y ago
    lol sounds like you need to ditch paredit for parinfer
qsort 3y ago
Sort of a meta comment -- the article starts saying that people keep asking "why Clojure". Why such an emphasis on what language stuff is written in? Why does it even matter? Sounds like it was a good engineering choice given the constraints of the problem, get over yourself ffs.
- simongray 3y ago
  It's probably because the author keeps getting comments from other developers about his choice of language and he's tired of explaining why. There was a HN submission yesterday about another project of his (also in Clojure) and many of the comments were "Why is this made in Clojure?". Unfortunately, a large segment of developers think there is no value in venturing beyond C-style languages. Another dimension in recent years is the static type checking cult, whose adherents must warn everyone else in the cult whenever they spot any kind of dynamically typed programming language, e.g. causing a separate thread about the lack of static type checking in any comment section about a Clojure project.
  - eyelidlessness 3y ago
    > Another dimension in recent years is the static type checking cult, whose adherents must warn everyone else in the cult whenever they spot any kind of dynamically typed programming language Shun the nonbeliever!
  - Scarbutt 3y ago
    The blog post does mentioned the lack of a static type system as a drawback. Although it sounds that what he really wants (cause of his prototyping remarks) is gradual typing similar to typescript.
    - aphyr 3y ago
      Clojure has a gradual type system--core.typed--that I've used extensively. It's... it's very cool, but for my uses it seems to be more trouble than it's worth. I keep coming back to it every few years, though.
whalesalad 3y ago
Clojure’s biggest issue is definitely the ecosystem. As far as the bell curve goes for accessible and usable tooling it’s usually either on the far left end and terrible but easily accessible (but doesn’t follow the functional paradigms well) or it’s on the far right end -extremely esoteric, LOTR quote in the readme, everything is data to the n-th degree, only accessible to dragons who can write emacs plugins blindfolded and upside down. The language itself is amazing, and Java interop is much better than Aphyr makes it sound. If there was a rails or Django for Clojure for instance (or better yet, a Phoenix) it would be an easier sell and methinks more widely adopted. The juice needs to be worth the squeeze - particularly when you’re convincing a team to jump from Algol style imperative languages to lisp. But at this time you need to reinvent the wheel on virtually any project, aside from fundamentals like http routing (which is also a rough scene) But the fact that literally any maven package can be used is tremendous. Plus it compiles to a jar, so anywhere a jar will run - so will clojure.
- silcoon 3y ago
  You should definitely check out Calva, a wonderful plugin for VSCode which bring the REPL and much more (live documentation) for an easy way to use Clojure. I'm using it everyday at work. https://calva.io https://calva.io
  - civilitty 3y ago
    Situation: There are three competing Clojure development environments. Three!? Ridiculous! We need to develop one universal Clojure environment that covers everyone's use cases. Situation: There are fourteen competing Clojure development environments.
    - pharmakom 3y ago
      This comment has been repeated many times. I think there is even an XKCD. Anyway, it’s not very helpful. If the 3 existing solutions are all deficient in some way then a new standard may be a good idea. And besides that, we may not realize that what we have is deficient until someone explores the space a bit. Edit: Please forgive my tone, I’m weary from seeing this repeated so many times and you have unfairly taken the brunt of that!
silcoon 3y ago
For all of you new to Clojure, check out this interactive tutorial: https://tryclojure.org https://tryclojure.org
- chii 3y ago
  https://4clojure.oxal.org/ https://4clojure.oxal.org/ that is an even better interactive set of lessons starting from easy to intermediate to hard.
rtpg 3y ago
Last time I spent a couple klines trying to write some clojure (+ clojurescript), the big thing was that most of the tooling had some really nasty error states. Python has some of this (ctrl+C out of most Python programs and get that stack trace with a bunch of inscrutible stuff), but given I was just starting out it was tough for me to move forwards. I do think the language + ecosystem has so many innovative and interesting ideas built in. Definitely worth at least reading the history of Clojure paper and internalizing most of the lessons. This is going to be anathema to everyone but I think there's a nice little ecosystem space for a scripting language that is as serious about performance and code ergonomics as Clojure... but without the parens[0] (and I could do without the nil-punning, but at least there's a bit of a foundation there) [0]: yes I know that (f x y) is nicer than f(x, y). My problem is more when it comes to things like let forms introducing indentation (I shouldn't have to pay an indent + paren cost when trying to give a value a name!) and other things that just feel like line noise when coming from a language like Python, that just uses the indentation I'm going to put in my code anyways as information
- chii 3y ago
  > let forms introducing indentation this is a symptom of using let forms too copiously, rather than using a functional style, where there's very few requirements to attach names to intermediate arguments being passed between functions. Often, you would use threading macros (the -> and ->> stuff) to implicitly pass arguments from one function to the next, or define transformations which are themselves just pure functions (so you would define it at the top level). It's understandable that some interop require a lot of let forms - mainly those that mutate an argument in a chain of procedures (looking at you, java jwt libraries...you know who you are).
  - skrebbel 3y ago
    Well-named intermediary variables often help tremendously with readability though. A language discouraging their use seems like the wrong trade-off decision to me.
    - chii 3y ago
tieway59 3y ago
[flagged]
- koudelka 3y ago
  Honestly though, bless Kyle for truly being himself on the internet.
- jrflowers 3y ago
  This is a good point. We should draw more attention to his social media account because maybe people might read this without taking the time to click through
jwr 3y ago
As a heavy user of Clojure (my SaaS is written in Clojure/ClojureScript) for 10 years or so, I would fully agree with what Kyle wrote. In addition, things that I found very valuable: * most of my domain code is in cljc files, so these get compiled both server-side and client-side and I get a huge boost from that * transducers and transducer pipelines are an under-appreciated feature, I get huge mileage out of them, because of performance, composability and reusability * stability and longevity: Clojure was created by a very experienced system designer and programmer (Rich Hickey) and it shows. Backwards compatibility is hugely important, and I can focus on my application rather than rely on tools that are a moving target. It's not an accident that an average "Clojure programmer" (I dislike these kinds of tags, but let's run with it here) has 12+ years of experience and earns a salary in the top quartile at least.
- drikerf 3y ago
  Similar story. I'm also running my SaaS on Clojure and maintenance has been a dream compared to other languages. Things rarely break and are easy to upgrade. Developer experience is fantastic too with HMR, REPL and cljc files. It's fun to write Clojure code :)
  - weikju 3y ago
    HMR?
    - G4BB3R 3y ago
      "Hot Module Replacement (HMR) is a feature in some programming languages and frameworks that allows developers to update modules or parts of the application in real-time while the application is running, without requiring a full page refresh. This is particularly useful during development as it speeds up the development process and helps maintain the application state."
      - weikju 3y ago
        Hmm I’d rather hear more from the GP (assuming it’s the same concept) and specific tools being used than a generic GPT sounding reply. Thanks though at least I learned an acronym (no snark).
koito17 3y ago
In short: - being on JVM means Jepsen can use JDBC drivers to interact with databases - Clojure was designed to facilitate concurrent programming (via software transactional memory and persistent data structures) - Clojure's interactivity allows quick prototyping - Clojure's community has a strong emphasis on backwards compatibility - macros allow a great deal of code reuse - while there are some big drawbacks (niche language, no strong static typing, ...), it isnt an issue in practice since Jepsen is only worked on by 1-3 people at a time I agree with most of these points, but it seems overly simplistic of an explanation. I really wanted to see what other languages were used for prototyping and why they were deemed insufficient. For instance, Haskell is mentioned but the most treatment it gets is that it was too "dogmatic" for writing Jepsen. There was no comparison on how easy it is to interact with databases from Haskell as opposed to Clojure. Moreover, Jepsen was prototyped "in a few different languages", but they are not compared to Clojure, let alone mentioned.
- travisjungroth 3y ago
  Based on the title and intro, I think part of the point of this post was to have something to link to for a “good enough” answer to this common question. I’m guessing he wrote this relatively quickly. It was a choice made over ten years ago. Trying to recreate the decision would likely be rather time consuming, prone to mistakes, and really only invite more discussion.
- deleted 3y ago
  [deleted]
- fulafel 3y ago
  Indeed the protoyping takeaways would be interesting. It's quite rare in real world apps to prototype a system in several languages to pick one. From the text, to me it seems more likely to me that there were several prototypes for other reasons, and they might not be functionally or architecturally equivalent enough for direct language comparisons.
- aphyr 3y ago
  DB vendors generally don't write Haskell clients, which put a serious damper on my "test lots of databases for a living" project. Love Haskell as a language, but it's just not a popular target, and I wasn't brave enough to go for C interop. As I touched on in the post, Haskell is also rigorous about side effects--I've found it difficult to, say, inject side-effectual performance monitoring into an otherwise pure piece of code so I can profile it. Maybe Haskell experts are better at this, or the story is better now. There was a Ruby version of Jepsen too. That enjoyed better library support and language flexibility, but it suffered from terrible performance, a standard library built for mutability, and a janky thread model. There was, at the time, essentially no way to reliably time out a Ruby thread. That was a dealbreaker all in itself. I wrote some of Jepsen in Erlang too, but abandoned the project quickly. It's got a really good concurrency story. A VM designed for concurrency and failure with bounded reduction queues--honestly this is something I miss in the JVM. I love Erlang as a language (and I think some of my Erlang might still be in prod!) but it also suffered from a lack of client support, not being well-suited for high-speed mutable state, and generally terrible string manipulation. There's a lot of strings in Jepsen. ;-)
dsign 3y ago
This is what happens: - Dev Foor Barry needs to solve problem X. Mr. Barry thinks about the problem for a long time, picks a technology stack YZ that is suitable to the solution and to his experience, and goes to work. - Some time later, Mr. Barry meets dev Alice Mallory. She didn't know X was a problem, and she didn't know Mr. Barry either. But less than a minute into their conversation and being a mutual acquaintance, Ms. Mallory is convinced that technology stack YZ is the WRONG tool for the job. She can't stop herself from asking "Foor, why YZ? Wouldn't it have been better to use KY?" I've been involved in this ploy (sometimes as Mr. Barry, and sometimes as Ms. Mallory) any number of times, for any number of technologies, from mainstream ones to more obscure ones. For the last few years, I've been trying to act more as Steve: "Use whatever you want." I have to admit though, that kind of acceptance is a difficult, uphill spiritual path. And I have met plenty of architects in their sixties and seventies who saw the hill and took a wide berth, and keep acting like Mallory.
- deleted 3y ago
  [deleted]
ReleaseCandidat 3y ago
I guess Jepsen is useful and stuff, but the _really_ important contributions are the "interviews": https://aphyr.com/posts/340-reversing-the-technical-interview https://aphyr.com/posts/340-reversing-the-technical-intervie... https://aphyr.com/posts/341-hexing-the-technical-interview https://aphyr.com/posts/341-hexing-the-technical-interview https://aphyr.com/posts/342-typing-the-technical-interview https://aphyr.com/posts/342-typing-the-technical-interview https://aphyr.com/posts/353-rewriting-the-technical-interview https://aphyr.com/posts/353-rewriting-the-technical-intervie... https://aphyr.com/posts/354-unifying-the-technical-interview https://aphyr.com/posts/354-unifying-the-technical-interview
- deleted 3y ago
  [deleted]
- roughly 3y ago
  I really appreciate Aphyr for being one of the only people I've seen to fully integrate their entire selves into their public persona and accomplish success. The Jepsen reports on any given system are effectively canonical documentation for a product, and Kyle's work and writing on concurrency overall is fantastic. At the same time, there's no "suit and tie" Aphyr out there - you read the blog, you get the weird; you follow them on social media, well, you get the Full Aphyr. I respect that they've been willing to put their whole person out there and have been so goddamn good at what they do that they've been successful at it - it's genuinely inspiring.
727564797069706 3y ago
Clojure does many things right and I like it for it a lot. One thing that I see slowing down real teams building real web stuff with Clojure is the near-dogmatic desire to compose (dare I say hack) everything together from tons of libraries that each have their own understanding of how things should be. I bet Clojure would be much more popular and nicer to work with if the community would _also_ (i.e. in addition to the current approach) have actively-maintained, batteries-included, generally usable web framework(s). (Is Biff it?) So that there would be alternative path to choose for us who like the language, but are not fans of the build-your-own-monster approach. Disclaimer: I have been spoiled by Ruby on Rails
- gleenn 3y ago
  I commented above but check out Luminus or Kit.
- poidos 3y ago
  I like Biff the most out of the ones I’ve tried.
- Capricorn2481 3y ago
  Biff is very batteries included, but because it uses a couple defaults that are not particularly mainstream (XTDB instead of SQL, HTMX/Hyperscript) I imagine newcomers will still have a hard time in it. Learning Clojure, AND graph databases, and potentially HTMX is a bit much. Still, a lot of it is just setup for you. Auth is thoughtful and easy and the server config files are well commented. There are deploy scripts to setup a server in linux and SSH into the production REPL. And if you want to use XTDB, the way Biff has setup document validation is really nice. I also like using it more than an app running on ClojureScript, personally. There are a lot of ClojureScript wrappers around React, and it's almost a meme. As a react developer, I have not found any of them easier than React. I find them kind of convoluted and un-ergonomic. However, HTMX/Hyperscript works really well with Clojure, because their HTML syntax is just arrays and objects that are easy to manipulate. It's the first time that the whole "code is data" proverb repeated by Clojure enthusiasts has rung true for me.
  - 3y ago
dig1 3y ago
> Clojure does have significant drawbacks. It has a small engineering community and no (broadly-accepted, successful) static typing system. I tried to dabble with OCaml multiple times in the past, and what I didn't like was the overly strict type system. I spent more time trying to please the compiler than writing the code. With Clojure (Common Lisp and C), I have more freedom to bend the compiler (type system) to do what I want it to do. Sure, there are pros and cons to that, but I don't want distractions when I write the code. It may be my style, but I meet numerous people who think the same. I believe (static) type systems are a matter of preference, and another camp thinks otherwise. However, it is always lovely to have opposite camps around because we learn a lot from each other. > Working with JVM primitives can be frustrating without dropping to Java; I do this on occasion. Rich (Clojure author) mentioned multiple times that Clojure is a hosted language, so it is OK to jump into Java when you have to (or javascript). IMHO, this is one of the most powerful features of the Clojure ecosystem because it is way more pragmatic. I've seen numerous attempts to re-implement the same thing repeatedly in your favorite language just because you don't want to touch "a library written in That Other Filthy language because it is slow/insecure/not-flexible/you-name-it". This was/is very frequent in Common Lisp but also in the Rust communities.
- tmtvl 3y ago
  Having type checking is indeed a matter of preference. I personally quite like having the language protect me from making silly mistakes, which is why I use type declarations liberally in my Common Lisp code. With SBCL doing its best to ensure they hold it has probably saved me minutes of headaches each project. Also CL is very pragmatic and nobody bats an eye at, for example, woo being based on a C library.
- Karrot_Kream 3y ago
  > I believe (static) type systems are a matter of preference, and another camp thinks otherwise. It's funny, when I led high reliability teams at a Big Tech, I wanted static strict typing everywhere done yesterday. Every production problem of ours was related to small, subtle issues. Upgrading the runtime leading to JSON key reordering breaking some ancient client that was hand-unrolling JSON. Some library deep within our stack started returning HTTP codes as ints and not strings which broke some error parsing of ours. And every outage was a big deal as the scale we operated at meant that if even a small percentage of customers were affected, we'd have hundreds of problem reports. Now that I'm working at a tiny startup things couldn't be more different. We chose to go with a static language because that's where the mindshare seems to be, but I am missing working in a dynamic language. Because of the speed with which we're shipping our code quality is really bad. In the interests of time, instead of dealing with the type system and refactoring things cleanly, we'll just make multiple copies of the same object structure with small differences and use them all over the place. We copy/paste the same logic in 10 different places instead of isolating the functionality and referring to it in different places, because the shapes of types make things a bit tricky to make generic. I find that different problem spaces lend themselves to different solutions. Are you getting paged in the middle of the night for subtle bugs? Then strict static typing is probably for you. Are you iterating quickly and can problems be solved by someone sending you an email or Slack? Dynamic typing is probably better. The trick is to not accumulate too much debt in the dynamic paradigm in case you see growth to port things over to a static paradigm. > I've seen numerous attempts to re-implement the same thing repeatedly in your favorite language just because you don't want to touch "a library written in That Other Filthy language because it is slow/insecure/not-flexible/you-name-it". This was/is very frequent in Common Lisp but also in the Rust communities. I think languages with lots of enthusiasts just end up with these attitudes unfortunately.
snoopsnopp 3y ago
[flagged]
- raspasov 3y ago
  https://www.youtube.com/watch?v=msptE3ezksE https://www.youtube.com/watch?v=msptE3ezksE
  - snoopsnopp 3y ago
    [flagged]
kaliszad 3y ago
Can fully agree that Clojure(Script) really is a blessing and a great choice for many projects smallish or big. Frankly, Clojure is the first language I did professional programming in, with a little bit of experience in C, Python, PowerShell, Java, Bash, SQL before to solve university assignments or to help with systems administration/ some analytics of traffic dumps etc. We program OrgPad in Clojure+ClojureScript and it was a good decision, even as we have more than 100kloc of code. It is approachable and if I write a bit of code, I am fairly certain that it will do what I imagine. I didn't have that to that degree not even in Python, PowerShell or Bash, it is a completely different league, quite possibly game. The REPL workflow is really nice, my colleague did a video about it: https://www.youtube.com/watch?v=4igO7Qbyj9o https://www.youtube.com/watch?v=4igO7Qbyj9o Nowadays, even some functions/ short tasks or small projects can be served by Babashka/ nbb in places where you would often consider switching from Bash to Python.
jgrodziski 3y ago
We also write about our choice of Clojure for application development here: https://defsquare.io/blog/why-we-bet-on-clojure https://defsquare.io/blog/why-we-bet-on-clojure
scotty79 3y ago
> The error messages are terrible. I have no apologetics for this. ;-) Could something be done about this? I tried to learn Clojure but I bumped into that fairly early on and gave up. Rust was way more approachable for me because good error messages form learning dialog between the learner and the language.
- phyzome 3y ago
  It's sad to hear this is still an issue. For those who don't know, Clojure's error messages... aren't. They take the form of a stack trace from the compiler's internals, and that whole stack trace just gets barfed out onto the terminal. There's an exception message but it sometimes just isn't that helpful. I know there was an effort a while back to improve the situation, but last I saw (2019) I believe things were only marginally improved. It's a shame because other than that it's really quite a fantastic language.
  - smw 3y ago
    Isn't that true in most common languages? Python and Ruby and Java and Go (panics) will all spit out the stacktrace?
    - phyzome 3y ago
      This is about compile-time errors, not runtime errors.
    - scotty79 3y ago
      Clojure spits out Java stack trace if you make syntax error. It's equivalent of Python barfing segfault when you put tab in the wrong place.
      - fulafel 3y ago
        This doesn't happen with the normal REPL (or invoking with clj -X) after a couple of first examples that came to mind: user=> (defn foo [[a b] (+ a b)) Syntax error reading source at (REPL:14:26). Unmatched delimiter: ) user=> (defn foo [a b] a/b) Syntax error compiling at (REPL:1:1). No such namespace: a user=> It will give you a stack trace of the compiler when asked following a syntax error though, with a disclaimer not to confuse it with your own code: user=> (pst) Note: The following stack trace applies to the reader or compiler, your code was not executed. CompilerException Syntax error compiling at (1:1). #:clojure.error{:phase :compile-syntax-check, :line 1, :column 1, :source "NO_SOURCE_PATH"} clojure.lang.Compiler.analyze (Compiler.java:6808) clojure.lang.Compiler.analyze (Compiler.java:6745) [...] Possibly you were using tooling that always tried to present the stack trace without discerning between kinds of evaluation errors?
38529977thrw 3y ago
It would have been worthwhile for them to mention why not use Java. I mean, grep for Java and JVM and that short blog lights up! It was good that they did stress the (very) small team size for the product as well. This was an aesthetic choice and I wish he had stressed that. Java remains an excellent choice for concurrent systems, with a commonly accepted static type system, and all "java.util.concurrent" and the rest of JVM are native and the language and tools do scale to even monstrous sized "teams".
- aphyr 3y ago
  I've been a Java programmer for ~18 years! A nontrivial portion of Jepsen is written in Java too. :-) Many of the points I touched on in the post are direct critiques of Java, but maybe I should be explicit. An obvious factor is size--when I've written the same project in both languages, Clojure usually winds up 5-10 times smaller. That's definitely improving as Java adds streams, lambda expressions, and so on, but it's still a good deal more verbose. That gets in the way of the kind of exploratory programming I do, especially at the REPL, and it's harder for me to read and maintain a giant codebase. I appreciate having fewer nouns, in general: Clojure emphasizes a uniform way of traversing and transforming data structures, and in Java every single class has its own way of representing its data, and many of those ways are bad. The standard collections library is full of mutability and thread safety pitfalls. Printed representations of datatypes are verbose. It's not particularly good at dealing with highly polymorphic data structure transformation, and it's not a particularly good static type system either--a good part of my Java Brain (TM) is devoted to knowing the mechanisms of erasure, memory layout, and when unsafe casts are actually the right choice. Functions are sort of becoming first-class with method refs and functional interfaces, but it's nowhere near the convenience and flexibility of a Lisp-1. Accessing the compiler at runtime is a pain in the ass. No hygenic macro system makes things like custom iteration or compositional error handling a pain. There's no analogue to Clojure's protocols, which are a fantastic tool for "Hey, compiler, I want a monomorphic-cached type-dispatch polymorphic call site here for a type hierarchy I don't control". Etc etc. There are things I love about Java. Automated refactoring is easier, and I generally like the level of IDE support. Nearly anything involving primitives, I drop to Java. Ditto, APIs that require annotations. Interface specification is more rigorous. Etc etc. That's why I write in both languages. But most of Jepsen is in Clojure for good reasons. :-)
swiftlyTyped 3y ago
I love clojure, but over time I've learned the painful lesson that at the end of the day, for complex modern applications, the ecosystem is all. That said Clojure ecosystem isn't too bad
barrenko 3y ago
There is no Rails for Clojure because Clojure is Rails for the right kind of software architect. Clojure is a language that beat the crap out of me but in a positive way. I don't use it, but learned a lot about what is possible.
yayitswei 3y ago
Love that Rich jumped in in the comments!
asylteltine 3y ago
[flagged]
- tmtvl 3y ago
  While I agree that square brackets are abominable, beauty is in the eye of the beholder. I'm sure that Clojure users find them a more elegant solution than the classic number sign followed by round brackets like Lisps use. And hey, something like Rust code can be quite ugly at times but you can hardly deny its efficiency.
  - asylteltine 3y ago
    Rust is also ugly but it’s very Cpp like. It’s less ugly than clojure. Go is the most elegant language imo very powerful but yet not ugly.
    - za3faran 3y ago
      I wouldn't use elegant and powerful to refer to golang.
- roughly 3y ago
  Well then have I got the article for you
- finnjohnsen2 3y ago
  Only to the untrained eye. Once you learn it, it is beautiful. It is the simplest language you’ll ever learn, as has very very little syntax.
avbanks 3y ago
"Almost every database has a JVM client, typically written in Java, and Clojure has decent Java interop." Curious to know in what ways could Java interop be better in Clojure?
- Capricorn2481 3y ago
  I'm not the most knowledgeable on this, but in general, Clojure has smart ways to convert data types from Java types to Clojure ones. Just like Clojure lets you use functions on a lot of different data types (mapping over vectors, objects, etc), it lets you do the same with Java collections. Instantiating objects and using their methods is pretty terse and easy when using macros like -> (def method-result (-> (SomeClass.) (.someMethod "arg1")))
- aphyr 3y ago
  Clojure's Java interop story is generally really good. Method calls are predictable and uniform. The collection types generally work both ways, rather than being a completely independent type hierarchy (as a former Scala programmer, what a breath of fresh air). Functions work as Runnable, Callable, Comparator, etc. Reify, deftype, and proxy are sufficient for maybe 98% of APIs I've worked with. There are, however, those rare exceptions! For instance, some Java APIs literally won't work without annotations. There's just no way (last I checked) to express those in Clojure. I don't remember exactly the nature of this problem, but I've had occasional issues with deeply overloaded argument dispatch before. Maybe it was something like "The Java API expects an object x which responds to x.foo(long l) and also x.foo(int i) differently", and there was no way to convince deftype to emit the right kind of class. AOT is... it's weird. You want ahead-of-time compilation when, say, you want to write a Java class that uses datatypes defined in Clojure. But AOT, at least in Leiningen, tends to break stuff in truly inscrutable ways. My professional opinion is that it's haunted.
  - seancorfield 3y ago
    There's a lot of cool Java interop stuff coming in Clojure 1.12 that might help here (Alpha 6 should drop "soon").
jrvarela56 3y ago
I have been acquainted with Clojure for a few years now. Haven't been able to 'break in' despite several attempts. My main conclusion is this: the language/workflow make developers way more productive compared to other modern stacks (I do Python/Ruby/TS on my day to day). The biggest, unsolved problem is http://www.winestockwebdesign.com/Essays/Lisp_Curse.html http://www.winestockwebdesign.com/Essays/Lisp_Curse.html tl;dr It is so easy to implement things in Lisp so you end up with a myriad of half-baked solutions and no long-term support of dependencies.
- iLemming 3y ago
  I'm not sure if the Lisp Curse can in fact be easily applied to Clojure. Clojure projects, for some reason (in my opinion), don't suffer as much as projects in other Lisps - such as Common Lisp and Emacs Lisp. Maybe it's because of the hosted nature of the language, there's usually a good set of libraries to choose from for Java and Javascript, and my guess is that this would hold true for ClojureDart and ClojureCLR as well. You can typically find some Clojure/Clojurescript wrappers if you want to interact with a JVM or Javascript library. But if a wrapper doesn't do what you need, you can always use interop directly. Also, Clojurists tend to adhere to some good stylistic defaults. They try not to overcomplicate things without good reasons. Furthermore, the nature of the language forces you to keep things small and simple.
  - jrvarela56 3y ago
    I think having to mention the other ecosystems shows some aspect of the problem. Maybe unrelated (but seemed like a symptom): In my case putting together a web framework was not trivial, as there was no equivalent to Django or Rails. Instead it seemed I was going to have to put together every piece of the stack - I loved it but it was not productive. Deciding between integrant/component, ring/compojure/reitit, etc was easy once you took a bit of time to read code and try it out but the furthest from 'just run it and keep going by tweaking'. I haven't given up yet, will give it another try this xmas break. I do believe there's huge productivity gains in a repl-driven clojure workflow.
masijo 3y ago
Clojure is my favorite language by far, it feels natural to me. That said, the two things which I truly miss from other languages are: understandable error messages (as mentioned by other people) and typing hints. I've had to refactor large code at work and it's been a pain. It's hard to know when a function will throw an exception or just return nil, it's hard to know if a core function will return a vector, a sequence, or another type of collection, and so on and on. There are some projects that try to address this (spec, malli, typed clojure) but I haven't found them to be a good replacement to something among the lines of python/typescript typing.
- Spivak 3y ago
  Maybe I'm just spoiled by modern languages that have this and the ones that don't are bolting it on (Python/TS) but how do you put up with it? Any language that can't tell me the what goes in / comes out of a function feels borderline worthless for a codebase more than a few files. Trying to infer types by looking at what the current code appears to be able to do the objects is just pain for its own sake.
  - cushpush 3y ago
    Actually, Clojure has typehinting! It looks like ^String or ^Int but usually it is not required because the compiler is very good at determining things. You can use the Clojure function (typeof? the-object-goes-here) to get the type. It is not a type-less language, it is a language of heterogenous collections and therefore lax type rules because they gotta go into the same collections together. The collections can be built in creative ways, so using fundamental elements to make larger nested collections is direct and possible. We can always ask what the type of an item is, and we can introspect contents, it's not as cumbersome as some people make it out to be via hn comment.
  - kennytilton 3y ago
    I have written a couple of 80kloc Common Lisp apps. Not a problem. But we all are used to what we are used to, and anything different offends.
ARandomerDude 3y ago
> Clojure...[has] no (broadly-accepted, successful) static typing system. I've used Clojure exclusively for 7 years and actively work on a Clojure + ClojureScript codebase of more than 500K LOC. The lack of a robust type system is my number one frustration. Many people point out that Clojure is expressive, lets you go fast, and generally just gets out of the way. I agree. It also lets you have some nasty runtime errors that can be hard to reproduce and debug. If there were a broadly-accepted, successful static typing system Clojure would be amazing. As it is, I think it's pretty decent if you like Ruby or Python and want more power. At this point I'm wondering if Go is the way.
- erichocean 3y ago
  You can integrate another JVM language like Kotlin into your Clojure project in places where you have measured that static typing would help.
  - joshlemer 3y ago
    How do people measure whether static typing would help or not in any particular section of code?
    - erichocean 3y ago
      > How do people measure whether static typing would help or not in any particular section of code? Whether or not you are getting bug reports in production that static typing would have caught prior to deployment *merely by adding types* to your existing functions/implementations, and where adding trivial-to-write dynamic type checks/assertions is too onerous. Dynamic type assertions are nice because they progressively insert static typing (they are caught by unit test == compile time) into dynamic languages, without having the full overhead of a static type checker. As a bonus, the technique can be used for assertions that are still research areas for static typing. Separately, there are a lot of way language-supported static typing can help by completing changing the nature of the solution to a problem—see Haskell for what that looks like. In my experience though, if I'm going to go through the trouble of adopting static typing everywhere, I just bite the bullet and go to full model checking with something like Alloy or TLA+. Different developers have different experiences, it's very dependent on the kind of code you write.
amai 3y ago
Being a Lisp dialect for the JVM Clojure has readability issues: https://tonsky.me/blog/readable-clojure/ https://tonsky.me/blog/readable-clojure/ That alone will make sure Clojure will always be a niche programming language and ensure job safety for the few ones who can read it. But nowadays Clojure brings no advantages anymore over Java. I wouldn't start any new project using Clojure.
- j-pb 3y ago
  I'd consider your point more if you could actually write the name correctly.
  - amai 3y ago
    Fixed it. It seems the autocorrection on smartphones doesn’t like Clojure either.
    - j-pb 3y ago
      You're missing a/the crucial point about Clojure btw. It's not about the functional programming stuff, of which Java has added a lot in recent versions, but it's actually about the immutable/persistent data-structures, of which Java has added none afaik. Value types are simply not the same thing. The structural sharing you get with Clojure, means that you can have immutable data that contains billions of entries. This allows you to solve really hard problems that are almost impossible to get right any other way, undo-redo, search algorithms that exploit lattices, idempotent network operations.
      - amai 3y ago
        "the core JDK package java.util.concurrent includes CopyOnWriteArrayList and CopyOnWriteArraySet which are persistent structures, implemented using copy-on-write techniques...Fully persistent collections are available in third-party libraries" https://en.m.wikipedia.org/wiki/Persistent_data_structure#Java https://en.m.wikipedia.org/wiki/Persistent_data_structure#Ja... So if you need that in Java you can simply use 3rd-party libraries like https://pcollections.org/ https://pcollections.org/ or https://github.com/GlenKPeterson/Paguro https://github.com/GlenKPeterson/Paguro. There is no need to completely change your programming language to Clojure and suffer from all that readability issues: https://softwareengineering.stackexchange.com/questions/197232/how-to-write-readable-clojure-code https://softwareengineering.stackexchange.com/questions/1972...
fuzztester 3y ago
sorry guys, i don't know clojure, so i'll write this script in my favorite programming language: #/usr/bin/english if clojure is mentioned, can rich hickey be far behind? and if rich is mentioned, can this video of his be far behind? methinks not. lo, and behold, said video: https://youtu.be/f84n5oFoZBc?si=Fy-1QcLkIiXVnqPo https://youtu.be/f84n5oFoZBc?si=Fy-1QcLkIiXVnqPo
effnorwood 3y ago
[dead]

469 points by manicennui 3y ago

> Jepsen’s (gosh) about a decade old now I thought Jepsen is more like 12 years old now, going by when I first heard about it circa 2011.

tomnipotent 3y ago
The song that inspired the name was released in 2011. Aphyr's "Call Me Maybe" blog series/talk started in 2013, and the commercial audits we've come to love came later.
dllthomas 3y ago
From a certain perspective, Jepsen is 38 years old.
michaelsbradley 3y ago
less than 15, “about” can go +/-

contrarian1234 3y ago

out of curiosity, does it use core.async or manifold for concurrency? (and what are the tradeoffs)

stefcoetzee 3y ago
Seems like `core.async`: https://github.com/search?q=repo%3Ajepsen-io%2Fjepsen+core.async&type=code https://github.com/search?q=repo%3Ajepsen-io%2Fjepsen+core.a... (As an aside: Missionary is a higher-level, more recent option in this area.)
- contrarian1234 3y ago
  Do you have experience using it? Missionary looks very cool, but I don't think anyone understands how to use it. Other than Electric/Rama (from the same people) I haven't been able to find any projects that use it on Github
  - stefcoetzee 3y ago
    I've only kicked the tires, not used it in anger. The tutorials are fairly accessible: https://cljdoc.org/d/missionary/missionary/b.33/doc/readme/tutorials/hello-task https://cljdoc.org/d/missionary/missionary/b.33/doc/readme/t... In a recent London Clojurians talk, the author Leo Noel explained Electric can be thought of as a frontend for Missionary. Rama is by a different team (RPL), but agreed, their philosophies are similar.
    - contrarian1234 3y ago
      Oh they updated the docs! Then I'll have to give it another look. I tried to make a simple subscription/memoization setup but I couldn't figure it out. They have a lot of keywords/terms (task/flow/etc.) that a bit hard to understand.. and even when you make sense of them isolation it was still a bit unclear how to compose them properly. The resulting code had to specify a lot of things and was more verbose than I had hoped for I'll be honest, part of the problem is I don't work in the web space. So I don't care for (or completely understand) the things they're looking to solve (ex: server/client sync issues). I just want to have simpler state-management and have changes propagate across my program in a way that makes my life easier :) (think like a "notebook" where you update values and everything gets reevaluated where-needed)
aphyr 3y ago
Nope! For a few reasons Jepsen sticks very closely to Traditional JVM Threads. Part of that is because Jepsen isn't hyper-concurrent. Most concurrency errors manifest in just a handful of client threads, and the JVM will run ~10K threads without too many issues. Another part is that I spend a lot of time debugging and profiling. Core.async turns stacktraces into Swiss cheese--it has to, to perform its implicit continuation-passing magic. But that can make it really hard to figure out Who Called What When, and Why Is This Process Slow. Standard blocking IO calls are also something that my profiler (YourKit) can analyze well. Finally, I invest a lot of time into tuning Jepsen for speed, and manage the handoff of data between threads carefully. In a system where actors were handing off state constantly core.async would be a bigger help, but there's really only a couple shapes for concurrent handoff in Jepsen, and they're addressed by either j.u.c. queues, forkjoin-style structured concurrency, or Weird Transactional Threadpool stuff. For a little more on this, take a look at https://github.com/aphyr/dom-top https://github.com/aphyr/dom-top or https://github.com/jepsen-io/history/blob/main/src/jepsen/history/task.clj https://github.com/jepsen-io/history/blob/main/src/jepsen/hi...

leoszliard 3y ago

Also, Clojure's fun

resonious 3y ago
This is underrated. I know it sounds unprofessional, but funness is also the reason Basecamp and Ruby on Rails was built in Ruby. Rails dominated web dev in the 2010s and tons of super successful startups were built on it. So while many folks like to scoff at engineers "playing with toys", a lot of good things we have now used to be toys (edit: Linux is another good example of this)
scary-size 3y ago
I think that captures perfectly! I've actually written about that sentiment in the past [1]. The tricky thing with a very "fun" language is that you assume that you can solve everything with it. You end up writing a lot of code, when, in the end, no code would have been an acceptable solution. [1] https://franz.hamburg/writing/clojure-makes-happy.html https://franz.hamburg/writing/clojure-makes-happy.html
- wavemode 3y ago
  lol sounds like you need to ditch paredit for parinfer

qsort 3y ago

Sort of a meta comment -- the article starts saying that people keep asking "why Clojure". Why such an emphasis on what language stuff is written in? Why does it even matter? Sounds like it was a good engineering choice given the constraints of the problem, get over yourself ffs.

simongray 3y ago
It's probably because the author keeps getting comments from other developers about his choice of language and he's tired of explaining why. There was a HN submission yesterday about another project of his (also in Clojure) and many of the comments were "Why is this made in Clojure?". Unfortunately, a large segment of developers think there is no value in venturing beyond C-style languages. Another dimension in recent years is the static type checking cult, whose adherents must warn everyone else in the cult whenever they spot any kind of dynamically typed programming language, e.g. causing a separate thread about the lack of static type checking in any comment section about a Clojure project.
- eyelidlessness 3y ago
  > Another dimension in recent years is the static type checking cult, whose adherents must warn everyone else in the cult whenever they spot any kind of dynamically typed programming language Shun the nonbeliever!
- Scarbutt 3y ago
  The blog post does mentioned the lack of a static type system as a drawback. Although it sounds that what he really wants (cause of his prototyping remarks) is gradual typing similar to typescript.
  - aphyr 3y ago
    Clojure has a gradual type system--core.typed--that I've used extensively. It's... it's very cool, but for my uses it seems to be more trouble than it's worth. I keep coming back to it every few years, though.

whalesalad 3y ago

Clojure’s biggest issue is definitely the ecosystem. As far as the bell curve goes for accessible and usable tooling it’s usually either on the far left end and terrible but easily accessible (but doesn’t follow the functional paradigms well) or it’s on the far right end -extremely esoteric, LOTR quote in the readme, everything is data to the n-th degree, only accessible to dragons who can write emacs plugins blindfolded and upside down. The language itself is amazing, and Java interop is much better than Aphyr makes it sound. If there was a rails or Django for Clojure for instance (or better yet, a Phoenix) it would be an easier sell and methinks more widely adopted. The juice needs to be worth the squeeze - particularly when you’re convincing a team to jump from Algol style imperative languages to lisp. But at this time you need to reinvent the wheel on virtually any project, aside from fundamentals like http routing (which is also a rough scene) But the fact that literally any maven package can be used is tremendous. Plus it compiles to a jar, so anywhere a jar will run - so will clojure.

silcoon 3y ago
You should definitely check out Calva, a wonderful plugin for VSCode which bring the REPL and much more (live documentation) for an easy way to use Clojure. I'm using it everyday at work. https://calva.io https://calva.io
- civilitty 3y ago
  Situation: There are three competing Clojure development environments. Three!? Ridiculous! We need to develop one universal Clojure environment that covers everyone's use cases. Situation: There are fourteen competing Clojure development environments.
  - pharmakom 3y ago
    This comment has been repeated many times. I think there is even an XKCD. Anyway, it’s not very helpful. If the 3 existing solutions are all deficient in some way then a new standard may be a good idea. And besides that, we may not realize that what we have is deficient until someone explores the space a bit. Edit: Please forgive my tone, I’m weary from seeing this repeated so many times and you have unfairly taken the brunt of that!

silcoon 3y ago

For all of you new to Clojure, check out this interactive tutorial: https://tryclojure.org https://tryclojure.org

chii 3y ago
https://4clojure.oxal.org/ https://4clojure.oxal.org/ that is an even better interactive set of lessons starting from easy to intermediate to hard.

rtpg 3y ago

Last time I spent a couple klines trying to write some clojure (+ clojurescript), the big thing was that most of the tooling had some really nasty error states. Python has some of this (ctrl+C out of most Python programs and get that stack trace with a bunch of inscrutible stuff), but given I was just starting out it was tough for me to move forwards. I do think the language + ecosystem has so many innovative and interesting ideas built in. Definitely worth at least reading the history of Clojure paper and internalizing most of the lessons. This is going to be anathema to everyone but I think there's a nice little ecosystem space for a scripting language that is as serious about performance and code ergonomics as Clojure... but without the parens[0] (and I could do without the nil-punning, but at least there's a bit of a foundation there) [0]: yes I know that (f x y) is nicer than f(x, y). My problem is more when it comes to things like let forms introducing indentation (I shouldn't have to pay an indent + paren cost when trying to give a value a name!) and other things that just feel like line noise when coming from a language like Python, that just uses the indentation I'm going to put in my code anyways as information

chii 3y ago
> let forms introducing indentation this is a symptom of using let forms too copiously, rather than using a functional style, where there's very few requirements to attach names to intermediate arguments being passed between functions. Often, you would use threading macros (the -> and ->> stuff) to implicitly pass arguments from one function to the next, or define transformations which are themselves just pure functions (so you would define it at the top level). It's understandable that some interop require a lot of let forms - mainly those that mutate an argument in a chain of procedures (looking at you, java jwt libraries...you know who you are).
- skrebbel 3y ago
  Well-named intermediary variables often help tremendously with readability though. A language discouraging their use seems like the wrong trade-off decision to me.
  - chii 3y ago

tieway59 3y ago

[flagged]

koudelka 3y ago
Honestly though, bless Kyle for truly being himself on the internet.
jrflowers 3y ago
This is a good point. We should draw more attention to his social media account because maybe people might read this without taking the time to click through

jwr 3y ago

As a heavy user of Clojure (my SaaS is written in Clojure/ClojureScript) for 10 years or so, I would fully agree with what Kyle wrote. In addition, things that I found very valuable: * most of my domain code is in cljc files, so these get compiled both server-side and client-side and I get a huge boost from that * transducers and transducer pipelines are an under-appreciated feature, I get huge mileage out of them, because of performance, composability and reusability * stability and longevity: Clojure was created by a very experienced system designer and programmer (Rich Hickey) and it shows. Backwards compatibility is hugely important, and I can focus on my application rather than rely on tools that are a moving target. It's not an accident that an average "Clojure programmer" (I dislike these kinds of tags, but let's run with it here) has 12+ years of experience and earns a salary in the top quartile at least.

drikerf 3y ago
Similar story. I'm also running my SaaS on Clojure and maintenance has been a dream compared to other languages. Things rarely break and are easy to upgrade. Developer experience is fantastic too with HMR, REPL and cljc files. It's fun to write Clojure code :)
- weikju 3y ago
  HMR?
  - G4BB3R 3y ago
    "Hot Module Replacement (HMR) is a feature in some programming languages and frameworks that allows developers to update modules or parts of the application in real-time while the application is running, without requiring a full page refresh. This is particularly useful during development as it speeds up the development process and helps maintain the application state."
    - weikju 3y ago
      Hmm I’d rather hear more from the GP (assuming it’s the same concept) and specific tools being used than a generic GPT sounding reply. Thanks though at least I learned an acronym (no snark).

koito17 3y ago

In short: - being on JVM means Jepsen can use JDBC drivers to interact with databases - Clojure was designed to facilitate concurrent programming (via software transactional memory and persistent data structures) - Clojure's interactivity allows quick prototyping - Clojure's community has a strong emphasis on backwards compatibility - macros allow a great deal of code reuse - while there are some big drawbacks (niche language, no strong static typing, ...), it isnt an issue in practice since Jepsen is only worked on by 1-3 people at a time I agree with most of these points, but it seems overly simplistic of an explanation. I really wanted to see what other languages were used for prototyping and why they were deemed insufficient. For instance, Haskell is mentioned but the most treatment it gets is that it was too "dogmatic" for writing Jepsen. There was no comparison on how easy it is to interact with databases from Haskell as opposed to Clojure. Moreover, Jepsen was prototyped "in a few different languages", but they are not compared to Clojure, let alone mentioned.

travisjungroth 3y ago
Based on the title and intro, I think part of the point of this post was to have something to link to for a “good enough” answer to this common question. I’m guessing he wrote this relatively quickly. It was a choice made over ten years ago. Trying to recreate the decision would likely be rather time consuming, prone to mistakes, and really only invite more discussion.
deleted 3y ago
[deleted]
fulafel 3y ago
Indeed the protoyping takeaways would be interesting. It's quite rare in real world apps to prototype a system in several languages to pick one. From the text, to me it seems more likely to me that there were several prototypes for other reasons, and they might not be functionally or architecturally equivalent enough for direct language comparisons.
aphyr 3y ago
DB vendors generally don't write Haskell clients, which put a serious damper on my "test lots of databases for a living" project. Love Haskell as a language, but it's just not a popular target, and I wasn't brave enough to go for C interop. As I touched on in the post, Haskell is also rigorous about side effects--I've found it difficult to, say, inject side-effectual performance monitoring into an otherwise pure piece of code so I can profile it. Maybe Haskell experts are better at this, or the story is better now. There was a Ruby version of Jepsen too. That enjoyed better library support and language flexibility, but it suffered from terrible performance, a standard library built for mutability, and a janky thread model. There was, at the time, essentially no way to reliably time out a Ruby thread. That was a dealbreaker all in itself. I wrote some of Jepsen in Erlang too, but abandoned the project quickly. It's got a really good concurrency story. A VM designed for concurrency and failure with bounded reduction queues--honestly this is something I miss in the JVM. I love Erlang as a language (and I think some of my Erlang might still be in prod!) but it also suffered from a lack of client support, not being well-suited for high-speed mutable state, and generally terrible string manipulation. There's a lot of strings in Jepsen. ;-)

dsign 3y ago

This is what happens: - Dev Foor Barry needs to solve problem X. Mr. Barry thinks about the problem for a long time, picks a technology stack YZ that is suitable to the solution and to his experience, and goes to work. - Some time later, Mr. Barry meets dev Alice Mallory. She didn't know X was a problem, and she didn't know Mr. Barry either. But less than a minute into their conversation and being a mutual acquaintance, Ms. Mallory is convinced that technology stack YZ is the WRONG tool for the job. She can't stop herself from asking "Foor, why YZ? Wouldn't it have been better to use KY?" I've been involved in this ploy (sometimes as Mr. Barry, and sometimes as Ms. Mallory) any number of times, for any number of technologies, from mainstream ones to more obscure ones. For the last few years, I've been trying to act more as Steve: "Use whatever you want." I have to admit though, that kind of acceptance is a difficult, uphill spiritual path. And I have met plenty of architects in their sixties and seventies who saw the hill and took a wide berth, and keep acting like Mallory.

deleted 3y ago
[deleted]

ReleaseCandidat 3y ago

I guess Jepsen is useful and stuff, but the _really_ important contributions are the "interviews": https://aphyr.com/posts/340-reversing-the-technical-interview https://aphyr.com/posts/340-reversing-the-technical-intervie... https://aphyr.com/posts/341-hexing-the-technical-interview https://aphyr.com/posts/341-hexing-the-technical-interview https://aphyr.com/posts/342-typing-the-technical-interview https://aphyr.com/posts/342-typing-the-technical-interview https://aphyr.com/posts/353-rewriting-the-technical-interview https://aphyr.com/posts/353-rewriting-the-technical-intervie... https://aphyr.com/posts/354-unifying-the-technical-interview https://aphyr.com/posts/354-unifying-the-technical-interview

deleted 3y ago
[deleted]
roughly 3y ago
I really appreciate Aphyr for being one of the only people I've seen to fully integrate their entire selves into their public persona and accomplish success. The Jepsen reports on any given system are effectively canonical documentation for a product, and Kyle's work and writing on concurrency overall is fantastic. At the same time, there's no "suit and tie" Aphyr out there - you read the blog, you get the weird; you follow them on social media, well, you get the Full Aphyr. I respect that they've been willing to put their whole person out there and have been so goddamn good at what they do that they've been successful at it - it's genuinely inspiring.

727564797069706 3y ago

Clojure does many things right and I like it for it a lot. One thing that I see slowing down real teams building real web stuff with Clojure is the near-dogmatic desire to compose (dare I say hack) everything together from tons of libraries that each have their own understanding of how things should be. I bet Clojure would be much more popular and nicer to work with if the community would _also_ (i.e. in addition to the current approach) have actively-maintained, batteries-included, generally usable web framework(s). (Is Biff it?) So that there would be alternative path to choose for us who like the language, but are not fans of the build-your-own-monster approach. Disclaimer: I have been spoiled by Ruby on Rails

gleenn 3y ago
I commented above but check out Luminus or Kit.
poidos 3y ago
I like Biff the most out of the ones I’ve tried.
Capricorn2481 3y ago
Biff is very batteries included, but because it uses a couple defaults that are not particularly mainstream (XTDB instead of SQL, HTMX/Hyperscript) I imagine newcomers will still have a hard time in it. Learning Clojure, AND graph databases, and potentially HTMX is a bit much. Still, a lot of it is just setup for you. Auth is thoughtful and easy and the server config files are well commented. There are deploy scripts to setup a server in linux and SSH into the production REPL. And if you want to use XTDB, the way Biff has setup document validation is really nice. I also like using it more than an app running on ClojureScript, personally. There are a lot of ClojureScript wrappers around React, and it's almost a meme. As a react developer, I have not found any of them easier than React. I find them kind of convoluted and un-ergonomic. However, HTMX/Hyperscript works really well with Clojure, because their HTML syntax is just arrays and objects that are easy to manipulate. It's the first time that the whole "code is data" proverb repeated by Clojure enthusiasts has rung true for me.
- 3y ago

dig1 3y ago

> Clojure does have significant drawbacks. It has a small engineering community and no (broadly-accepted, successful) static typing system. I tried to dabble with OCaml multiple times in the past, and what I didn't like was the overly strict type system. I spent more time trying to please the compiler than writing the code. With Clojure (Common Lisp and C), I have more freedom to bend the compiler (type system) to do what I want it to do. Sure, there are pros and cons to that, but I don't want distractions when I write the code. It may be my style, but I meet numerous people who think the same. I believe (static) type systems are a matter of preference, and another camp thinks otherwise. However, it is always lovely to have opposite camps around because we learn a lot from each other. > Working with JVM primitives can be frustrating without dropping to Java; I do this on occasion. Rich (Clojure author) mentioned multiple times that Clojure is a hosted language, so it is OK to jump into Java when you have to (or javascript). IMHO, this is one of the most powerful features of the Clojure ecosystem because it is way more pragmatic. I've seen numerous attempts to re-implement the same thing repeatedly in your favorite language just because you don't want to touch "a library written in That Other Filthy language because it is slow/insecure/not-flexible/you-name-it". This was/is very frequent in Common Lisp but also in the Rust communities.

tmtvl 3y ago
Having type checking is indeed a matter of preference. I personally quite like having the language protect me from making silly mistakes, which is why I use type declarations liberally in my Common Lisp code. With SBCL doing its best to ensure they hold it has probably saved me minutes of headaches each project. Also CL is very pragmatic and nobody bats an eye at, for example, woo being based on a C library.
Karrot_Kream 3y ago
> I believe (static) type systems are a matter of preference, and another camp thinks otherwise. It's funny, when I led high reliability teams at a Big Tech, I wanted static strict typing everywhere done yesterday. Every production problem of ours was related to small, subtle issues. Upgrading the runtime leading to JSON key reordering breaking some ancient client that was hand-unrolling JSON. Some library deep within our stack started returning HTTP codes as ints and not strings which broke some error parsing of ours. And every outage was a big deal as the scale we operated at meant that if even a small percentage of customers were affected, we'd have hundreds of problem reports. Now that I'm working at a tiny startup things couldn't be more different. We chose to go with a static language because that's where the mindshare seems to be, but I am missing working in a dynamic language. Because of the speed with which we're shipping our code quality is really bad. In the interests of time, instead of dealing with the type system and refactoring things cleanly, we'll just make multiple copies of the same object structure with small differences and use them all over the place. We copy/paste the same logic in 10 different places instead of isolating the functionality and referring to it in different places, because the shapes of types make things a bit tricky to make generic. I find that different problem spaces lend themselves to different solutions. Are you getting paged in the middle of the night for subtle bugs? Then strict static typing is probably for you. Are you iterating quickly and can problems be solved by someone sending you an email or Slack? Dynamic typing is probably better. The trick is to not accumulate too much debt in the dynamic paradigm in case you see growth to port things over to a static paradigm. > I've seen numerous attempts to re-implement the same thing repeatedly in your favorite language just because you don't want to touch "a library written in That Other Filthy language because it is slow/insecure/not-flexible/you-name-it". This was/is very frequent in Common Lisp but also in the Rust communities. I think languages with lots of enthusiasts just end up with these attitudes unfortunately.

snoopsnopp 3y ago

[flagged]

raspasov 3y ago
https://www.youtube.com/watch?v=msptE3ezksE https://www.youtube.com/watch?v=msptE3ezksE
- snoopsnopp 3y ago
  [flagged]

kaliszad 3y ago

Can fully agree that Clojure(Script) really is a blessing and a great choice for many projects smallish or big. Frankly, Clojure is the first language I did professional programming in, with a little bit of experience in C, Python, PowerShell, Java, Bash, SQL before to solve university assignments or to help with systems administration/ some analytics of traffic dumps etc. We program OrgPad in Clojure+ClojureScript and it was a good decision, even as we have more than 100kloc of code. It is approachable and if I write a bit of code, I am fairly certain that it will do what I imagine. I didn't have that to that degree not even in Python, PowerShell or Bash, it is a completely different league, quite possibly game. The REPL workflow is really nice, my colleague did a video about it: https://www.youtube.com/watch?v=4igO7Qbyj9o https://www.youtube.com/watch?v=4igO7Qbyj9o Nowadays, even some functions/ short tasks or small projects can be served by Babashka/ nbb in places where you would often consider switching from Bash to Python.

jgrodziski 3y ago

We also write about our choice of Clojure for application development here: https://defsquare.io/blog/why-we-bet-on-clojure https://defsquare.io/blog/why-we-bet-on-clojure

scotty79 3y ago

> The error messages are terrible. I have no apologetics for this. ;-) Could something be done about this? I tried to learn Clojure but I bumped into that fairly early on and gave up. Rust was way more approachable for me because good error messages form learning dialog between the learner and the language.

phyzome 3y ago
It's sad to hear this is still an issue. For those who don't know, Clojure's error messages... aren't. They take the form of a stack trace from the compiler's internals, and that whole stack trace just gets barfed out onto the terminal. There's an exception message but it sometimes just isn't that helpful. I know there was an effort a while back to improve the situation, but last I saw (2019) I believe things were only marginally improved. It's a shame because other than that it's really quite a fantastic language.
- smw 3y ago
  Isn't that true in most common languages? Python and Ruby and Java and Go (panics) will all spit out the stacktrace?
  - phyzome 3y ago
    This is about compile-time errors, not runtime errors.
  - scotty79 3y ago
    Clojure spits out Java stack trace if you make syntax error. It's equivalent of Python barfing segfault when you put tab in the wrong place.
    - fulafel 3y ago
      This doesn't happen with the normal REPL (or invoking with clj -X) after a couple of first examples that came to mind: user=> (defn foo [[a b] (+ a b)) Syntax error reading source at (REPL:14:26). Unmatched delimiter: ) user=> (defn foo [a b] a/b) Syntax error compiling at (REPL:1:1). No such namespace: a user=> It will give you a stack trace of the compiler when asked following a syntax error though, with a disclaimer not to confuse it with your own code: user=> (pst) Note: The following stack trace applies to the reader or compiler, your code was not executed. CompilerException Syntax error compiling at (1:1). #:clojure.error{:phase :compile-syntax-check, :line 1, :column 1, :source "NO_SOURCE_PATH"} clojure.lang.Compiler.analyze (Compiler.java:6808) clojure.lang.Compiler.analyze (Compiler.java:6745) [...] Possibly you were using tooling that always tried to present the stack trace without discerning between kinds of evaluation errors?

38529977thrw 3y ago

It would have been worthwhile for them to mention why not use Java. I mean, grep for Java and JVM and that short blog lights up! It was good that they did stress the (very) small team size for the product as well. This was an aesthetic choice and I wish he had stressed that. Java remains an excellent choice for concurrent systems, with a commonly accepted static type system, and all "java.util.concurrent" and the rest of JVM are native and the language and tools do scale to even monstrous sized "teams".

aphyr 3y ago
I've been a Java programmer for ~18 years! A nontrivial portion of Jepsen is written in Java too. :-) Many of the points I touched on in the post are direct critiques of Java, but maybe I should be explicit. An obvious factor is size--when I've written the same project in both languages, Clojure usually winds up 5-10 times smaller. That's definitely improving as Java adds streams, lambda expressions, and so on, but it's still a good deal more verbose. That gets in the way of the kind of exploratory programming I do, especially at the REPL, and it's harder for me to read and maintain a giant codebase. I appreciate having fewer nouns, in general: Clojure emphasizes a uniform way of traversing and transforming data structures, and in Java every single class has its own way of representing its data, and many of those ways are bad. The standard collections library is full of mutability and thread safety pitfalls. Printed representations of datatypes are verbose. It's not particularly good at dealing with highly polymorphic data structure transformation, and it's not a particularly good static type system either--a good part of my Java Brain (TM) is devoted to knowing the mechanisms of erasure, memory layout, and when unsafe casts are actually the right choice. Functions are sort of becoming first-class with method refs and functional interfaces, but it's nowhere near the convenience and flexibility of a Lisp-1. Accessing the compiler at runtime is a pain in the ass. No hygenic macro system makes things like custom iteration or compositional error handling a pain. There's no analogue to Clojure's protocols, which are a fantastic tool for "Hey, compiler, I want a monomorphic-cached type-dispatch polymorphic call site here for a type hierarchy I don't control". Etc etc. There are things I love about Java. Automated refactoring is easier, and I generally like the level of IDE support. Nearly anything involving primitives, I drop to Java. Ditto, APIs that require annotations. Interface specification is more rigorous. Etc etc. That's why I write in both languages. But most of Jepsen is in Clojure for good reasons. :-)

swiftlyTyped 3y ago

I love clojure, but over time I've learned the painful lesson that at the end of the day, for complex modern applications, the ecosystem is all. That said Clojure ecosystem isn't too bad

barrenko 3y ago

There is no Rails for Clojure because Clojure is Rails for the right kind of software architect. Clojure is a language that beat the crap out of me but in a positive way. I don't use it, but learned a lot about what is possible.

yayitswei 3y ago

Love that Rich jumped in in the comments!

asylteltine 3y ago

[flagged]

tmtvl 3y ago
While I agree that square brackets are abominable, beauty is in the eye of the beholder. I'm sure that Clojure users find them a more elegant solution than the classic number sign followed by round brackets like Lisps use. And hey, something like Rust code can be quite ugly at times but you can hardly deny its efficiency.
- asylteltine 3y ago
  Rust is also ugly but it’s very Cpp like. It’s less ugly than clojure. Go is the most elegant language imo very powerful but yet not ugly.
  - za3faran 3y ago
    I wouldn't use elegant and powerful to refer to golang.
roughly 3y ago
Well then have I got the article for you
finnjohnsen2 3y ago
Only to the untrained eye. Once you learn it, it is beautiful. It is the simplest language you’ll ever learn, as has very very little syntax.

avbanks 3y ago

"Almost every database has a JVM client, typically written in Java, and Clojure has decent Java interop." Curious to know in what ways could Java interop be better in Clojure?

Capricorn2481 3y ago
I'm not the most knowledgeable on this, but in general, Clojure has smart ways to convert data types from Java types to Clojure ones. Just like Clojure lets you use functions on a lot of different data types (mapping over vectors, objects, etc), it lets you do the same with Java collections. Instantiating objects and using their methods is pretty terse and easy when using macros like -> (def method-result (-> (SomeClass.) (.someMethod "arg1")))
aphyr 3y ago
Clojure's Java interop story is generally really good. Method calls are predictable and uniform. The collection types generally work both ways, rather than being a completely independent type hierarchy (as a former Scala programmer, what a breath of fresh air). Functions work as Runnable, Callable, Comparator, etc. Reify, deftype, and proxy are sufficient for maybe 98% of APIs I've worked with. There are, however, those rare exceptions! For instance, some Java APIs literally won't work without annotations. There's just no way (last I checked) to express those in Clojure. I don't remember exactly the nature of this problem, but I've had occasional issues with deeply overloaded argument dispatch before. Maybe it was something like "The Java API expects an object x which responds to x.foo(long l) and also x.foo(int i) differently", and there was no way to convince deftype to emit the right kind of class. AOT is... it's weird. You want ahead-of-time compilation when, say, you want to write a Java class that uses datatypes defined in Clojure. But AOT, at least in Leiningen, tends to break stuff in truly inscrutable ways. My professional opinion is that it's haunted.
- seancorfield 3y ago
  There's a lot of cool Java interop stuff coming in Clojure 1.12 that might help here (Alpha 6 should drop "soon").

jrvarela56 3y ago

I have been acquainted with Clojure for a few years now. Haven't been able to 'break in' despite several attempts. My main conclusion is this: the language/workflow make developers way more productive compared to other modern stacks (I do Python/Ruby/TS on my day to day). The biggest, unsolved problem is http://www.winestockwebdesign.com/Essays/Lisp_Curse.html http://www.winestockwebdesign.com/Essays/Lisp_Curse.html tl;dr It is so easy to implement things in Lisp so you end up with a myriad of half-baked solutions and no long-term support of dependencies.

iLemming 3y ago
I'm not sure if the Lisp Curse can in fact be easily applied to Clojure. Clojure projects, for some reason (in my opinion), don't suffer as much as projects in other Lisps - such as Common Lisp and Emacs Lisp. Maybe it's because of the hosted nature of the language, there's usually a good set of libraries to choose from for Java and Javascript, and my guess is that this would hold true for ClojureDart and ClojureCLR as well. You can typically find some Clojure/Clojurescript wrappers if you want to interact with a JVM or Javascript library. But if a wrapper doesn't do what you need, you can always use interop directly. Also, Clojurists tend to adhere to some good stylistic defaults. They try not to overcomplicate things without good reasons. Furthermore, the nature of the language forces you to keep things small and simple.
- jrvarela56 3y ago
  I think having to mention the other ecosystems shows some aspect of the problem. Maybe unrelated (but seemed like a symptom): In my case putting together a web framework was not trivial, as there was no equivalent to Django or Rails. Instead it seemed I was going to have to put together every piece of the stack - I loved it but it was not productive. Deciding between integrant/component, ring/compojure/reitit, etc was easy once you took a bit of time to read code and try it out but the furthest from 'just run it and keep going by tweaking'. I haven't given up yet, will give it another try this xmas break. I do believe there's huge productivity gains in a repl-driven clojure workflow.

masijo 3y ago

Clojure is my favorite language by far, it feels natural to me. That said, the two things which I truly miss from other languages are: understandable error messages (as mentioned by other people) and typing hints. I've had to refactor large code at work and it's been a pain. It's hard to know when a function will throw an exception or just return nil, it's hard to know if a core function will return a vector, a sequence, or another type of collection, and so on and on. There are some projects that try to address this (spec, malli, typed clojure) but I haven't found them to be a good replacement to something among the lines of python/typescript typing.

Spivak 3y ago
Maybe I'm just spoiled by modern languages that have this and the ones that don't are bolting it on (Python/TS) but how do you put up with it? Any language that can't tell me the what goes in / comes out of a function feels borderline worthless for a codebase more than a few files. Trying to infer types by looking at what the current code appears to be able to do the objects is just pain for its own sake.
- cushpush 3y ago
  Actually, Clojure has typehinting! It looks like ^String or ^Int but usually it is not required because the compiler is very good at determining things. You can use the Clojure function (typeof? the-object-goes-here) to get the type. It is not a type-less language, it is a language of heterogenous collections and therefore lax type rules because they gotta go into the same collections together. The collections can be built in creative ways, so using fundamental elements to make larger nested collections is direct and possible. We can always ask what the type of an item is, and we can introspect contents, it's not as cumbersome as some people make it out to be via hn comment.
- kennytilton 3y ago
  I have written a couple of 80kloc Common Lisp apps. Not a problem. But we all are used to what we are used to, and anything different offends.

ARandomerDude 3y ago

> Clojure...[has] no (broadly-accepted, successful) static typing system. I've used Clojure exclusively for 7 years and actively work on a Clojure + ClojureScript codebase of more than 500K LOC. The lack of a robust type system is my number one frustration. Many people point out that Clojure is expressive, lets you go fast, and generally just gets out of the way. I agree. It also lets you have some nasty runtime errors that can be hard to reproduce and debug. If there were a broadly-accepted, successful static typing system Clojure would be amazing. As it is, I think it's pretty decent if you like Ruby or Python and want more power. At this point I'm wondering if Go is the way.

erichocean 3y ago
You can integrate another JVM language like Kotlin into your Clojure project in places where you have measured that static typing would help.
- joshlemer 3y ago
  How do people measure whether static typing would help or not in any particular section of code?
  - erichocean 3y ago
    > How do people measure whether static typing would help or not in any particular section of code? Whether or not you are getting bug reports in production that static typing would have caught prior to deployment *merely by adding types* to your existing functions/implementations, and where adding trivial-to-write dynamic type checks/assertions is too onerous. Dynamic type assertions are nice because they progressively insert static typing (they are caught by unit test == compile time) into dynamic languages, without having the full overhead of a static type checker. As a bonus, the technique can be used for assertions that are still research areas for static typing. Separately, there are a lot of way language-supported static typing can help by completing changing the nature of the solution to a problem—see Haskell for what that looks like. In my experience though, if I'm going to go through the trouble of adopting static typing everywhere, I just bite the bullet and go to full model checking with something like Alloy or TLA+. Different developers have different experiences, it's very dependent on the kind of code you write.

amai 3y ago

Being a Lisp dialect for the JVM Clojure has readability issues: https://tonsky.me/blog/readable-clojure/ https://tonsky.me/blog/readable-clojure/ That alone will make sure Clojure will always be a niche programming language and ensure job safety for the few ones who can read it. But nowadays Clojure brings no advantages anymore over Java. I wouldn't start any new project using Clojure.

j-pb 3y ago
I'd consider your point more if you could actually write the name correctly.
- amai 3y ago
  Fixed it. It seems the autocorrection on smartphones doesn’t like Clojure either.
  - j-pb 3y ago
    You're missing a/the crucial point about Clojure btw. It's not about the functional programming stuff, of which Java has added a lot in recent versions, but it's actually about the immutable/persistent data-structures, of which Java has added none afaik. Value types are simply not the same thing. The structural sharing you get with Clojure, means that you can have immutable data that contains billions of entries. This allows you to solve really hard problems that are almost impossible to get right any other way, undo-redo, search algorithms that exploit lattices, idempotent network operations.
    - amai 3y ago
      "the core JDK package java.util.concurrent includes CopyOnWriteArrayList and CopyOnWriteArraySet which are persistent structures, implemented using copy-on-write techniques...Fully persistent collections are available in third-party libraries" https://en.m.wikipedia.org/wiki/Persistent_data_structure#Java https://en.m.wikipedia.org/wiki/Persistent_data_structure#Ja... So if you need that in Java you can simply use 3rd-party libraries like https://pcollections.org/ https://pcollections.org/ or https://github.com/GlenKPeterson/Paguro https://github.com/GlenKPeterson/Paguro. There is no need to completely change your programming language to Clojure and suffer from all that readability issues: https://softwareengineering.stackexchange.com/questions/197232/how-to-write-readable-clojure-code https://softwareengineering.stackexchange.com/questions/1972...

fuzztester 3y ago

sorry guys, i don't know clojure, so i'll write this script in my favorite programming language: #/usr/bin/english if clojure is mentioned, can rich hickey be far behind? and if rich is mentioned, can this video of his be far behind? methinks not. lo, and behold, said video: https://youtu.be/f84n5oFoZBc?si=Fy-1QcLkIiXVnqPo https://youtu.be/f84n5oFoZBc?si=Fy-1QcLkIiXVnqPo

effnorwood 3y ago

[dead]