6 ms·

I tried switching to Clojure, and while I love the purity, simplicity, and logic of it, two things always bothered me: 1) Immutable data structures are always

I tried switching to Clojure, and while I love the purity, simplicity, and logic of it, two things always bothered me: 1) Immutable data structures are always going to be slower than their mutable cousins. 2) "Clojure code is beautiful" should be changed to "Your OWN Clojure code is beautiful". When I finished writing a compact piece of logic or data transformation, I was often struck with the beauty of it. When I tried to read someone ELSE's Clojure code, however, I couldn't even begin to make sense of it. I am ever open to being proved wrong, however. Any Clojure programmers reading this, please reply with some code that is readable, elegant, and performant, to provide a counterpoint to my pessimism.

lukev 14y ago
As someone who has breathed Clojure for the past three years, I very rarely come across code that is truly difficult to understand. It does tend to be more information-dense than other languages (especially Java) but that just means I only have to understand a couple hundred lines split into 3 files instead of thousands of lines across dozens of files. That said, there is definitely some nasty Clojure code out there, but I'm not sure that's avoidable in any language. Fortunately, the conventional wisdom to only use macros when necessary has started to catch on and that's made the situation a good deal better.
- weareconvo 14y ago
  > As someone who has breathed Clojure for the past three years, I very rarely come across code that is truly difficult to understand. I'm not really arguing that Clojure code is "truly" difficult to understand, if that is taken to mean reasoning about its performance and effects. Rather, I'm talking about the fact that idiomatic Clojure code encourages writing functions upon functions upon functions, which continually builds up layers of abstraction that inherently make code more and more difficult to read. Example: console.log("Hiya."); console.log("How's it "); console.log("going?"); vs. printFirstString(); printSecondString(); printThirdString(); function printFirstString() { ...etc }
  - lukev 14y ago
    I don't know anyone who writes code like that. Unnecessary wrapping of functions is an antipattern. Idomatic Clojure for what you just wrote is: (println "Hiya.") (println "How's it ") (println "going?") If a Clojureist wanted to get fancy and build an abstraction (as they probably would) they'd probably write a debug function instead of using 'println' directly: (log/debug "Hiya.") (log/debug "How's it ") Which is, granted, an abstraction, but isn't any harder to read.
    - vorg 14y ago
      Or define a function called `printlns`. Then call... (printlns "Hiya." "How's it " "going?")
      - lukev 14y ago
        Actually the default `println` already does that. ;)
        vorg 14y ago
        `println` inserts a space between its arguments. I was showing we could define one that inserts a newline instead.
  - rayiner 14y ago
    I could say the same thing about Java. Clojure/Lisp code just tends to have functions that do mostly what their names indicate. Java code tends to be broken up into billions of classes, and understanding an algorithm tends to involve trying to collect together all the pieces of it that are spread out as methods in a bunch of classes. Take something like code generation from an AST. In Clojure, you might do it in a single file with multimethods. In Java, the code generation algorithm would be spread out over dozens of different AST classes, each with a visitor method.
DigitalJack 14y ago
Clojure has some very clever immutable data structures based on Phil Bagwell's work [1] that help mitigate the slow down. I think you are totally right with regard to reading other people's clojure. Sometimes it's easy and pleasant, sometimes inscrutable. I think the issue at hand is that it is so easy to go the DSL route in clojure. I call that an issue because when you invent a DSL to solve your problem, you've essentially created a new language that only you understand. So now, if I'm reading your code, I have to understand raw clojure, and then I have to try to figure out your DSL. [1] http://lampwww.epfl.ch/papers/idealhashtrees.pdf http://lampwww.epfl.ch/papers/idealhashtrees.pdf
- weareconvo 14y ago
  > Clojure has some very clever immutable data structures based on Phil Bagwell's work [1] that help mitigate the slow down. This is true, and I've read some of the papers describing said structures and was blown away by the cleverness. However (and I don't have a source here, so please forgive me if I'm just totally wrong) I have watched many of Rich Hickey's hour+ long talks, and in one of them, I distinctly remember him saying that his vectors were "Within an order of magnitude of mutable vectors performance-wise, which is good enough for me". I was... put off by that. EDIT: Now that I think about it, it might have been Maps that he was talking about. I are not good at remember.
  - DigitalJack 14y ago
    I totally understand. One thing to watch out for though is "premature optimization" as I've heard it called here on HN. It may very well be that you need that order of magnitude, but if you don't then you might be missing out something you might have really enjoyed otherwise.
  - lukev 14y ago
    So that was a pretty early talk. I don't have benchmark numbers in hand, but I'm reasonably sure that the performance gap isn't that wide any more. Also, there's a new(er) feature called transients that can allow you to use mutable data structures where needed for performance without changing the structure of your code: (http://clojure.org/transients http://clojure.org/transients). And you can drop down to mutable Java data structures, if you need to: they are fully supported by Clojure, just not the default. In general, Clojure's approach is to be thread safe and purely functional by default, but it's not too difficult at all to drop out of that for particular parts of your code you've identified as bottlenecks.
pdrummond 14y ago
I also struggle reading Clojure code that isn't my own. It reminds me of a quote by Brenton Ashworth on the subject: http://goo.gl/yCIWI http://goo.gl/yCIWI
- weareconvo 14y ago
  It seems like that mentality would make building a community of any kind difficult, as programming communities (companies, shared libraries, etc) are all about being able to read and improve one another's code.
  - jeremyjh 14y ago
    I interpret this more as saying that it is ok if there is a barrier to entry. If it takes you six months of programming Clojure before you can easily read idiomatic code written by other people for example - maybe he's ok with that.
hueyp 14y ago
Reading other Clojure code is weird. I agree that it can be hard to get an eye for other peoples code, but the flip side is that everything is just functions and data. When I decided to go look at ring or compojure it was incredible how simple the code really is. I've looked at plenty of other clojure libraries and I have this revelation every time. Its awesome. This is in comparison to reading an OO library in a language I'm much more familiar with but where inheritance / mixins mean you have to dig through many files (often not obvious which ones) to understand a piece of code.
chc 14y ago
> Immutable data structures are always going to be slower than their mutable cousins. Not always. It depends on usage. Mutable data structures can actually entail more work if the data needs to be accessed from more than one place. Immutable data structures can be shared and reused much more freely. (For example, adding an item to an array you got from somebody else means either the caller or the callee needs to have copied the whole array, while adding an item to an immutable list you got from somebody else means creating a single cons cell.) At any rate, even if a destructive solution would be faster, the functional solution will probably still be faster in Clojure than the destructive solution would be in Ruby. And if not, you can still do something destructive — it's just not the default. > "Clojure code is beautiful" should be changed to "Your OWN Clojure code is beautiful". When I finished writing a compact piece of logic or data transformation, I was often struck with the beauty of it. When I tried to read someone ELSE's Clojure code, however, I couldn't even begin to make sense of it. I believe this is largely a matter of experience (not completely — some code just is hard to read!). Most people get that feeling when they're working with a relatively unfamiliar language. I find that the more familiar I become with Clojure, the better all other Clojure programmers seem to become.
- weareconvo 14y ago
  > I find that the more familiar I become with Clojure, the better all other Clojure programmers seem to become. That could be. However, I believe it's more to do with the fact that idiomatic Clojure encourages continual definition of nested layers of abstraction. Which makes the code beautiful from a certain viewpoint. However, it's also very similar to essentially writing a new language and adding features to it. If you buy this comparison, then reading your own Clojure code is like reading code written in a language you wrote, which is tailored to your own ideas about aesthetics and interfaces. Which is great. However, that would then mean that in order to read anyone else's Clojure code, you would have to learn a new programming language every single time. I would buy that this gets easier the more you do it, because my second programming language was definitely easier to learn than my first.
  - chc 14y ago
    That's true to some degree, but for well-written code, I don't think it's necessarily as severe as that description makes it sound. Any Ruby on Rails codebase is also chock-full of pseudo-DSLs (controllers, scopes, formats, etc.), but you don't hear the same complaints about Rails code being unreadable very often. It's possible to go overboard and turn your codebase into the Clojure equivalent of Negaposi, but from my (somewhat limited) experience, it's not exactly endemic.
sedachv 14y ago
> 1) Immutable data structures are always going to be slower than their mutable cousins. This would be ok, if immutable data structures weren't also much harder to reason about, implement, and actually use in your algorithms. Nothing is stopping you from doing functional programming and using immutable data structures in an imperative language (the reverse is very much not true), but why would you bother if it's easier to think about (and prove properties of, if you're into that thing) mutable data structures? People recommend Okasaki's Purely Functional Data Structures all the time, but the main result of that dissertation aren't the clever immutable data structures, it's the fact that Okasaki was the first person to come up with a somewhat workable way to do asymptotic runtime analysis of immutable data structures. IMO it's not pretty.
- DanWaterworth 14y ago
  I don't understand why you think that immutable data structures are more difficult to reason about and prove properties of. Perhaps you would enlighten me.
  - masklinn 14y ago
    Likely at least in part because they've been studied far less than mutable/imperative data structures, and thus there are significantly less tools for reasoning about them. Plus the most efficient functional datastctures rely on lazy evaluation, adding a layer of complexity to reasoning about their behavior. In fact, in his 10-years PFDS retrospective Okasaki noted exactly that: > In 1993, I ran across an interesting paper by Tyng-Ruey Chuang and Benjamin Goldberg, showing how to implement efficient purely functional queues and double-ended queues. Their result was quite nice, but rather complicated, so I started looking for a simpler approach. Eventually, I came up a much simpler approach based on lazy evaluation, leading to my first paper in this area. (When I say simpler here, I mean simpler to implement—the new structure was actually harder to analyze than Chuang and Goldberg's.) or > , I went into a frenzy, playing with different variations, and a few hours later I came up with an approach again based on lazy evaluation. I was sure that this approach worked, but I had no idea how to prove it. (Again, this was a case where the implementation was simple, but the analysis was a bear.) [...] The whole way home I thought about nothing but how to analyze my data structure. I knew it had something to do with amortization, but the usual techniques of amortization weren't working.
pron 14y ago
> Immutable data structures are always going to be slower than their mutable cousins. True in general, but only if you just care about single core performance. A data structure that allows your algorithm to scale gracefully to more cores is actually more performant than a data structure that's faster for a single thread, but is extremely hard to scale in concurrent settings. Clojure sacrifices single-thread performance for multithreaded scaling, and it's the right tradeoff given current hardware trends (for example, consider the design of Clojure's reducers, and how it focuses on multi-core scaling). My main concern with immutable data structures (and most data structures designed to work in concurrent settings, actually), is their heavy reliance on garbage collection, and on old-gen garbage collection in particular. Old-gen GC still has some ways to go, at least in HotSpot.
bkirkbri 14y ago
Interesting. One of the things I've come to love most about Clojure is that I find it easy to read and reason about code I did not write. Your comment demonstrates that is not universally true.
elarkin 14y ago
Have you used transients in Clojure? http://clojure.org/transients http://clojure.org/transients They allow mutation as long as it's localized.

lukev 14y ago

As someone who has breathed Clojure for the past three years, I very rarely come across code that is truly difficult to understand. It does tend to be more information-dense than other languages (especially Java) but that just means I only have to understand a couple hundred lines split into 3 files instead of thousands of lines across dozens of files. That said, there is definitely some nasty Clojure code out there, but I'm not sure that's avoidable in any language. Fortunately, the conventional wisdom to only use macros when necessary has started to catch on and that's made the situation a good deal better.

weareconvo 14y ago
> As someone who has breathed Clojure for the past three years, I very rarely come across code that is truly difficult to understand. I'm not really arguing that Clojure code is "truly" difficult to understand, if that is taken to mean reasoning about its performance and effects. Rather, I'm talking about the fact that idiomatic Clojure code encourages writing functions upon functions upon functions, which continually builds up layers of abstraction that inherently make code more and more difficult to read. Example: console.log("Hiya."); console.log("How's it "); console.log("going?"); vs. printFirstString(); printSecondString(); printThirdString(); function printFirstString() { ...etc }
- lukev 14y ago
  I don't know anyone who writes code like that. Unnecessary wrapping of functions is an antipattern. Idomatic Clojure for what you just wrote is: (println "Hiya.") (println "How's it ") (println "going?") If a Clojureist wanted to get fancy and build an abstraction (as they probably would) they'd probably write a debug function instead of using 'println' directly: (log/debug "Hiya.") (log/debug "How's it ") Which is, granted, an abstraction, but isn't any harder to read.
  - vorg 14y ago
    Or define a function called `printlns`. Then call... (printlns "Hiya." "How's it " "going?")
    - lukev 14y ago
      Actually the default `println` already does that. ;)
      - vorg 14y ago
        `println` inserts a space between its arguments. I was showing we could define one that inserts a newline instead.
- rayiner 14y ago
  I could say the same thing about Java. Clojure/Lisp code just tends to have functions that do mostly what their names indicate. Java code tends to be broken up into billions of classes, and understanding an algorithm tends to involve trying to collect together all the pieces of it that are spread out as methods in a bunch of classes. Take something like code generation from an AST. In Clojure, you might do it in a single file with multimethods. In Java, the code generation algorithm would be spread out over dozens of different AST classes, each with a visitor method.

DigitalJack 14y ago

Clojure has some very clever immutable data structures based on Phil Bagwell's work [1] that help mitigate the slow down. I think you are totally right with regard to reading other people's clojure. Sometimes it's easy and pleasant, sometimes inscrutable. I think the issue at hand is that it is so easy to go the DSL route in clojure. I call that an issue because when you invent a DSL to solve your problem, you've essentially created a new language that only you understand. So now, if I'm reading your code, I have to understand raw clojure, and then I have to try to figure out your DSL. [1] http://lampwww.epfl.ch/papers/idealhashtrees.pdf http://lampwww.epfl.ch/papers/idealhashtrees.pdf

weareconvo 14y ago
> Clojure has some very clever immutable data structures based on Phil Bagwell's work [1] that help mitigate the slow down. This is true, and I've read some of the papers describing said structures and was blown away by the cleverness. However (and I don't have a source here, so please forgive me if I'm just totally wrong) I have watched many of Rich Hickey's hour+ long talks, and in one of them, I distinctly remember him saying that his vectors were "Within an order of magnitude of mutable vectors performance-wise, which is good enough for me". I was... put off by that. EDIT: Now that I think about it, it might have been Maps that he was talking about. I are not good at remember.
- DigitalJack 14y ago
  I totally understand. One thing to watch out for though is "premature optimization" as I've heard it called here on HN. It may very well be that you need that order of magnitude, but if you don't then you might be missing out something you might have really enjoyed otherwise.
- lukev 14y ago
  So that was a pretty early talk. I don't have benchmark numbers in hand, but I'm reasonably sure that the performance gap isn't that wide any more. Also, there's a new(er) feature called transients that can allow you to use mutable data structures where needed for performance without changing the structure of your code: (http://clojure.org/transients http://clojure.org/transients). And you can drop down to mutable Java data structures, if you need to: they are fully supported by Clojure, just not the default. In general, Clojure's approach is to be thread safe and purely functional by default, but it's not too difficult at all to drop out of that for particular parts of your code you've identified as bottlenecks.

pdrummond 14y ago

I also struggle reading Clojure code that isn't my own. It reminds me of a quote by Brenton Ashworth on the subject: http://goo.gl/yCIWI http://goo.gl/yCIWI

weareconvo 14y ago
It seems like that mentality would make building a community of any kind difficult, as programming communities (companies, shared libraries, etc) are all about being able to read and improve one another's code.
- jeremyjh 14y ago
  I interpret this more as saying that it is ok if there is a barrier to entry. If it takes you six months of programming Clojure before you can easily read idiomatic code written by other people for example - maybe he's ok with that.

hueyp 14y ago

Reading other Clojure code is weird. I agree that it can be hard to get an eye for other peoples code, but the flip side is that everything is just functions and data. When I decided to go look at ring or compojure it was incredible how simple the code really is. I've looked at plenty of other clojure libraries and I have this revelation every time. Its awesome. This is in comparison to reading an OO library in a language I'm much more familiar with but where inheritance / mixins mean you have to dig through many files (often not obvious which ones) to understand a piece of code.

chc 14y ago

> Immutable data structures are always going to be slower than their mutable cousins. Not always. It depends on usage. Mutable data structures can actually entail more work if the data needs to be accessed from more than one place. Immutable data structures can be shared and reused much more freely. (For example, adding an item to an array you got from somebody else means either the caller or the callee needs to have copied the whole array, while adding an item to an immutable list you got from somebody else means creating a single cons cell.) At any rate, even if a destructive solution would be faster, the functional solution will probably still be faster in Clojure than the destructive solution would be in Ruby. And if not, you can still do something destructive — it's just not the default. > "Clojure code is beautiful" should be changed to "Your OWN Clojure code is beautiful". When I finished writing a compact piece of logic or data transformation, I was often struck with the beauty of it. When I tried to read someone ELSE's Clojure code, however, I couldn't even begin to make sense of it. I believe this is largely a matter of experience (not completely — some code just is hard to read!). Most people get that feeling when they're working with a relatively unfamiliar language. I find that the more familiar I become with Clojure, the better all other Clojure programmers seem to become.

weareconvo 14y ago
> I find that the more familiar I become with Clojure, the better all other Clojure programmers seem to become. That could be. However, I believe it's more to do with the fact that idiomatic Clojure encourages continual definition of nested layers of abstraction. Which makes the code beautiful from a certain viewpoint. However, it's also very similar to essentially writing a new language and adding features to it. If you buy this comparison, then reading your own Clojure code is like reading code written in a language you wrote, which is tailored to your own ideas about aesthetics and interfaces. Which is great. However, that would then mean that in order to read anyone else's Clojure code, you would have to learn a new programming language every single time. I would buy that this gets easier the more you do it, because my second programming language was definitely easier to learn than my first.
- chc 14y ago
  That's true to some degree, but for well-written code, I don't think it's necessarily as severe as that description makes it sound. Any Ruby on Rails codebase is also chock-full of pseudo-DSLs (controllers, scopes, formats, etc.), but you don't hear the same complaints about Rails code being unreadable very often. It's possible to go overboard and turn your codebase into the Clojure equivalent of Negaposi, but from my (somewhat limited) experience, it's not exactly endemic.

sedachv 14y ago

> 1) Immutable data structures are always going to be slower than their mutable cousins. This would be ok, if immutable data structures weren't also much harder to reason about, implement, and actually use in your algorithms. Nothing is stopping you from doing functional programming and using immutable data structures in an imperative language (the reverse is very much not true), but why would you bother if it's easier to think about (and prove properties of, if you're into that thing) mutable data structures? People recommend Okasaki's Purely Functional Data Structures all the time, but the main result of that dissertation aren't the clever immutable data structures, it's the fact that Okasaki was the first person to come up with a somewhat workable way to do asymptotic runtime analysis of immutable data structures. IMO it's not pretty.

DanWaterworth 14y ago
I don't understand why you think that immutable data structures are more difficult to reason about and prove properties of. Perhaps you would enlighten me.
- masklinn 14y ago
  Likely at least in part because they've been studied far less than mutable/imperative data structures, and thus there are significantly less tools for reasoning about them. Plus the most efficient functional datastctures rely on lazy evaluation, adding a layer of complexity to reasoning about their behavior. In fact, in his 10-years PFDS retrospective Okasaki noted exactly that: > In 1993, I ran across an interesting paper by Tyng-Ruey Chuang and Benjamin Goldberg, showing how to implement efficient purely functional queues and double-ended queues. Their result was quite nice, but rather complicated, so I started looking for a simpler approach. Eventually, I came up a much simpler approach based on lazy evaluation, leading to my first paper in this area. (When I say simpler here, I mean simpler to implement—the new structure was actually harder to analyze than Chuang and Goldberg's.) or > , I went into a frenzy, playing with different variations, and a few hours later I came up with an approach again based on lazy evaluation. I was sure that this approach worked, but I had no idea how to prove it. (Again, this was a case where the implementation was simple, but the analysis was a bear.) [...] The whole way home I thought about nothing but how to analyze my data structure. I knew it had something to do with amortization, but the usual techniques of amortization weren't working.

pron 14y ago

> Immutable data structures are always going to be slower than their mutable cousins. True in general, but only if you just care about single core performance. A data structure that allows your algorithm to scale gracefully to more cores is actually more performant than a data structure that's faster for a single thread, but is extremely hard to scale in concurrent settings. Clojure sacrifices single-thread performance for multithreaded scaling, and it's the right tradeoff given current hardware trends (for example, consider the design of Clojure's reducers, and how it focuses on multi-core scaling). My main concern with immutable data structures (and most data structures designed to work in concurrent settings, actually), is their heavy reliance on garbage collection, and on old-gen garbage collection in particular. Old-gen GC still has some ways to go, at least in HotSpot.

bkirkbri 14y ago

Interesting. One of the things I've come to love most about Clojure is that I find it easy to read and reason about code I did not write. Your comment demonstrates that is not universally true.

elarkin 14y ago

Have you used transients in Clojure? http://clojure.org/transients http://clojure.org/transients They allow mutation as long as it's localized.