9 ms·

I was at this talk and I disagree with his fundamental statement that simple + simple = simple. I program in Ruby one of the biggest problems beginners make is

I was at this talk and I disagree with his fundamental statement that simple + simple = simple. I program in Ruby one of the biggest problems beginners make is not creating complex data structures where they are needed. Instead they pass around hashes of hashes of hashes. Why? Hashes are simple, they're easy to understand and work with. Unfortunately this initial simplicity introduces unexpected complexity, things like deep dup and deep merge are now needed. Every part of your system must now know the hash structure you're using and often reproduce logic in many places like dealing with optional keys that may be missing. By not isolating the complexity to one part of the app it must now be shared by the entire app. simple + simple !(always)= simple. If I had a time machine and could make one change in an open source project it would be to go back and remove the design choice of putting using hashes as internal config objects in Rails. It makes understanding all the possible edge cases almost impossible unless you're really familiar with the project. The second is the claim of speed that this "simplicity" buys you. I agree that functional programming is extremely fast when it's parallelized and it's extremely easy to make a functional program parallelizable. When we're dealing with sequential tasks, mutating data is much faster than allocating new data structures. I think clojure helps with the speed of allocating new immutable structures by using copy on write strategies behind the scenes. I think Rich is an extremely smart and very accomplished programmer. I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at. To me we if we're not embracing and explore all a new concept/language/paradigm strengths and weaknesses, we're not growing by being exposed to that thing.

jules 11y ago
> Unfortunately this initial simplicity introduces unexpected complexity, things like deep dup and deep merge are now needed. Not if your data structures are immutable, which is exactly what he is advocating.
- hellofunk 11y ago
  Exactly; the whole idea of "deep dup" doesn't even exist in Clojure. It's a non-issue. Some languages go so far as to encourage actually serializing and then reading back in a data structure as a form of doing a deep duplication! (I've done this in Objective-C, based on advice in popular books.) This is an extreme example of the pitfalls of mutable data in many circumstances.
- schneems 11y ago
  Excellent point, I honestly hadn't considered that. I've spent more time dealing with functional programming than in actual languages with immutable structures. Thanks for pointing that out. I think my other point still stands, things like optional keys, or missing or required keys still must be shared throughout the app. Everything that touches that hash needs to know its structure.
  - j-pb 11y ago
    Yeah, but that structure is also easy to explore and understand. And in case you want some security you can use libraries like prismatic/schema, that allow you to declaratively describe your expected data-structures akin to a type system. The web is build on json and not corba because plain maps and vectors are far easier to work with than domain specific objects.
  - calibraxis 11y ago
    There is a discussion of deep vs shallow hashes. Take Datomic or Datascript for instance. Very shallow datastructures, with a defined schema. Indexes to make this fast. There’s tradeoffs. Shallow datastructures are under-used. But OOP objects share fundamental flaws with deep hashes. (And add some more.)
  - lorddoig 11y ago
    Having to know the structure of a hash/map/dict is strictly superior to having to know the methods and fields of an object. For any given task the two are equivalent, except that one is easily modified/reused/passed to other callers without creating an extremely brittle class hierarchy you're only going to have to rearchitect when your requirements change. As an IBM researcher looking at Watson waaaay before Watson actually existed: "OOP requires you to treat all extrinsic information as if it were intrinsic to the object itself." This is tantamount to saying you must have a complete and consistent programme design before you write a single line. Which is crazy (and probably the reason UML exists.) But don't take my word for it, think it through, it's the only conclusion you can reach, I think.
nightski 11y ago
You are sort of arguing a strawman. First you say that hashes are simple, but then go on to argue how they are not. Maybe they were not simple in the first place? Simplicity does not mean easy. It was "easy" to use a hash. Simplicity can actually be incredibly hard to design properly. I think this is what Rich alludes to in the video. Edit: It's one of the first thing he says in the video actually - "It's about the interleaving, not the cardinality."
- nilliams 11y ago
  Actually, parent gets it and is neither arguing a strawman nor mistakenly labelling hashes as simple rather than easy. Rich absolutely argues that basic data structures like hashes are simple -- something that is generally agreed upon. They are simpler than 'objects' because you can use basic comparison operators on them, and you can operate on them with higher-order functions etc. The question which parent is digging at is whether a larger, complex app, that leans heavily on hashes actually results in an app that is on the whole simpler, i.e. does 'a simple thing plus a simple thing equal a simple thing'. This is something I've heard discussed well on the Ruby Rogues podcast -- I think that either David Brady or Josh Susser may have a good blog post on the subject of 'simple + simple != simple' but I'm struggling to track it down. Whilst I love Rich Hickey's talks I do find myself coming round to the same conclusion as parent -- if you forgive my possibly incorrect interpretation of their argument -- that the idea that simple data structures are simpler is fairly useless if programmers use that fact naively. PS. Everyone should watch the linked talk, it's brilliant, and one of my favourite programming talks ever. I recommend it to every programmer I meet.
  - schneems 11y ago
    I think you nailed my original intent quite well, thanks for the post.
  - deleted 11y ago
    [deleted]
  - nightski 11y ago
    Why ignore the message of simplicity in abstractions and design only to nitpick on hashes? I am not a fan of using hashes either. I also disagree with Rich's stance on static typing. But still think the video has a great message that is being lost here.
nickbauman 11y ago
> I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at. This is a wonderful tenet in general: let's think about what X is bad at, rather than what it's good for. While I'm not much of a Rubyist (I've hacked a little in my time, but I fall on the Python side of the divide), working with trees (hashes of hashes are a kind of tree) has never been easier than in Clojure in my experience. I suspect this is true of any lisp.
- sanderjd 11y ago
  Could you expand on how Clojure makes it easier to deal with the frustrations your parent comment mentions (deep merges, shared reliance on hash structure, reproduced logic, and handling of optional values)? I've only used Clojure a bit, but I'm not sure how it solves any of these problems (which I also find frustrating in Ruby).
  - prospero 11y ago
    Immutability is a big part of it, as is the wide array of operators for maps and map-like data structures.
  - hellofunk 11y ago
    Hickey has stated elsewhere that optional values don't really make sense in a dynamically-typed language.
  - nickbauman 11y ago
    Persistent data structures are part of it, for sure. But on a higher level. On a lower level, if you look at how and assoc-in works in Clojure, the scales will fall from your eyes (they certainly fell from mine). Consider: (def g {:x "a" :y {:k 71}}) user=> (assoc-in g [:y :k] 72) {:y {:k 72}, :x "a"} With this you can walk and mod deeply nested datastructures with ease (and others can reason about it well).
    - aaron-lebo 11y ago
akurilin 11y ago
Your comment seems more about Clojure's type system and fundamental data structures than about functional programming in general. e.g. Haskell solves some/most of the problems you mention, while introducing different ones and presenting other trade-offs to consider.
sheepmullet 11y ago
There is actually a lot of complexity to hashes in Ruby. The two major sources of complexity are the fact they are mutable and the fact that keys are mutable but hashes are computed when the object is first added. So if you construct a valid hash, then call three functions passing them the hash, you have no idea if the hash is still valid for the second and third functions without reading the code for all the preceding functions. What this tends to mean in practice is you are always having to check all over the place that your hash is valid. They might look the same as clojure hashes but in practice they are used very differently.
- schneems 11y ago
  > There is actually a lot of complexity to hashes in Ruby. I agree here > So if you construct a valid hash, then call three functions passing them the hash, you have no idea if the hash is still valid for the second and third functions without reading the code for all the preceding functions. I don't find that to be true, even in the Rails codebase. You pass a hash to a function (method) and expect that it won't be mutated. If you're writing a method and you mutate a hash, you're expected to dup the argument so it won't be mutated. This is the convention. There are times when hashes are mutated, but generally that's reserved for methods who's purpose are to mutate its arguments. Where Rails gets into trouble in its functional passing of hashes isn't in the mutability of the hashes, it's in the composability of the functions. I've never written about this problem before, so i'm not sure I have a great example but it's definitely a problem. One example in rails is `url_for` it takes a hash argument. The problem is that there's multiple `url_for` methods that all do slightly different things. Some are needed for generating links in email, while some work in your views, and others are designed for you to use programmatically outside of a view context. One of the hardest things about this method is that one `url_for` can call another `url_for`. Since we can never be guaranteed the order of the calls it is really it makes things like having default values, or optional keys. You have to replicate logic in different functions since some may never be called in the order you might expect. This significantly impacts our ability to refactor which in turn impacts our ability to make performance improvements. I recently did a bunch of perf work in https://engineering.heroku.com/blogs/2015-08-06-patching-rails-performance/ https://engineering.heroku.com/blogs/2015-08-06-patching-rai... and some of my biggest perf improvements were getting rid of duping and merging of hashes. If Ruby had an immutable and performance efficient hash then it would have helped a bunch, however I don't think it would make the general awfulness that is an entirely hash based API to a very complex action (such as url_for) that significantly better to work with.
hellofunk 11y ago
>I think clojure helps with the speed of allocating new immutable structures by using copy on write strategies behind the scenes. No, it's not copy on write in the sense that COW means in other languages. Data in clojure is persistent, so usually even an altered piece of data is not actually copied, only the tiny bit that changed is added, if necessary. This is very different than, for example, Swift that implements copy on write, but the whole data structure is copied even if the change is minor. Additionally, you say: >When we're dealing with sequential tasks, mutating data is much faster than allocating new data structures That would be true in cases where the entire data piece is new; but so much of what you do in any language involves interative changes over existing data, and again because of clojure's persistence, new allocations are often not happening at all in many cases. ...However, I do agree with most of your other points :)
- schneems 11y ago
  I was introduced to http://hypirion.com/musings/understanding-persistent-vector-pt-1 http://hypirion.com/musings/understanding-persistent-vector-... not too long ago on HN. I don't use clojure so I was paraphrasing my somewhat limited understanding. Thanks for the clarification. That part about speed was more about my general frustration with the "immutable is fast" and "mutable is slow" meme that I hear too frequently. While that can be the case, it isn't always 100% true. Your language and how you use it can play a huge part. I'm hoping to learn more about clojure data structures in the coming months, so much good stuff.
  - j-pb 11y ago
    My guess would be that immutable clojure is still faster than say mutable ruby though. I don't think anybody is arguing that trie lookup is faster than array lookup. But when it comes to passing data between parts of the system, zero cost copies are much faster than copying buffers or objects.
rubiquity 11y ago
> I think Rich is an extremely smart and very accomplished programmer. I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at. As someone who used to swear by Ruby but now prefers FP langs, part of learning FP is as much about learning what FP is good at as it is about learning what FP isn't good at. Given FP's proclamation for declarative style and immutability, clearly FP isn't good at domains where imperative styles are important or really, really, really[0] CPU intensive work where mutability is needed. Although, thanks to compiler research (which is mostly done by FP language users, ahem) languages like OCaml are bridging this gap when you take a look at projects like MirageOS. You get to write your code in a declarative, immutable style that then gets compiled to very fast native code. It's having your cake and eating it, too. As far as your original complaint of everything in Ruby either being a hash of hashes or a class that just wraps a hash of hashes, I completely agree. That is what drove me away from the language. I'd rather just blow that hash + class relationship up completely. Not to mention, in ML-dialect languages you can use features like Enum/Sum/Product types to define the configuration/syntax of your programs which is a lot better than reading arbitrary keys in a hash, as you have stated. 0 - No, I mean really, really, really, really. "I think I need C to do this fast enough for my use case" is the "I need Cassandra for my 2GB database of 'Big Data'" of the FP world.
erichmond 11y ago
Using data structures without a schema is user error, and isn't the fault of any particular language. The rest of your points have some merit. :)
jdc 11y ago
> Every part of your system must now know the hash structure you're using and often reproduce logic in many places like dealing with optional keys that may be missing. Why not just write accessors and mutators for your hashes? Also, Python hash a function dict.get(key, default). Does Ruby not have this?

jules 11y ago

> Unfortunately this initial simplicity introduces unexpected complexity, things like deep dup and deep merge are now needed. Not if your data structures are immutable, which is exactly what he is advocating.

hellofunk 11y ago
Exactly; the whole idea of "deep dup" doesn't even exist in Clojure. It's a non-issue. Some languages go so far as to encourage actually serializing and then reading back in a data structure as a form of doing a deep duplication! (I've done this in Objective-C, based on advice in popular books.) This is an extreme example of the pitfalls of mutable data in many circumstances.
schneems 11y ago
Excellent point, I honestly hadn't considered that. I've spent more time dealing with functional programming than in actual languages with immutable structures. Thanks for pointing that out. I think my other point still stands, things like optional keys, or missing or required keys still must be shared throughout the app. Everything that touches that hash needs to know its structure.
- j-pb 11y ago
  Yeah, but that structure is also easy to explore and understand. And in case you want some security you can use libraries like prismatic/schema, that allow you to declaratively describe your expected data-structures akin to a type system. The web is build on json and not corba because plain maps and vectors are far easier to work with than domain specific objects.
- calibraxis 11y ago
  There is a discussion of deep vs shallow hashes. Take Datomic or Datascript for instance. Very shallow datastructures, with a defined schema. Indexes to make this fast. There’s tradeoffs. Shallow datastructures are under-used. But OOP objects share fundamental flaws with deep hashes. (And add some more.)
- lorddoig 11y ago
  Having to know the structure of a hash/map/dict is strictly superior to having to know the methods and fields of an object. For any given task the two are equivalent, except that one is easily modified/reused/passed to other callers without creating an extremely brittle class hierarchy you're only going to have to rearchitect when your requirements change. As an IBM researcher looking at Watson waaaay before Watson actually existed: "OOP requires you to treat all extrinsic information as if it were intrinsic to the object itself." This is tantamount to saying you must have a complete and consistent programme design before you write a single line. Which is crazy (and probably the reason UML exists.) But don't take my word for it, think it through, it's the only conclusion you can reach, I think.

nightski 11y ago

You are sort of arguing a strawman. First you say that hashes are simple, but then go on to argue how they are not. Maybe they were not simple in the first place? Simplicity does not mean easy. It was "easy" to use a hash. Simplicity can actually be incredibly hard to design properly. I think this is what Rich alludes to in the video. Edit: It's one of the first thing he says in the video actually - "It's about the interleaving, not the cardinality."

nilliams 11y ago
Actually, parent gets it and is neither arguing a strawman nor mistakenly labelling hashes as simple rather than easy. Rich absolutely argues that basic data structures like hashes are simple -- something that is generally agreed upon. They are simpler than 'objects' because you can use basic comparison operators on them, and you can operate on them with higher-order functions etc. The question which parent is digging at is whether a larger, complex app, that leans heavily on hashes actually results in an app that is on the whole simpler, i.e. does 'a simple thing plus a simple thing equal a simple thing'. This is something I've heard discussed well on the Ruby Rogues podcast -- I think that either David Brady or Josh Susser may have a good blog post on the subject of 'simple + simple != simple' but I'm struggling to track it down. Whilst I love Rich Hickey's talks I do find myself coming round to the same conclusion as parent -- if you forgive my possibly incorrect interpretation of their argument -- that the idea that simple data structures are simpler is fairly useless if programmers use that fact naively. PS. Everyone should watch the linked talk, it's brilliant, and one of my favourite programming talks ever. I recommend it to every programmer I meet.
- schneems 11y ago
  I think you nailed my original intent quite well, thanks for the post.
- deleted 11y ago
  [deleted]
- nightski 11y ago
  Why ignore the message of simplicity in abstractions and design only to nitpick on hashes? I am not a fan of using hashes either. I also disagree with Rich's stance on static typing. But still think the video has a great message that is being lost here.

nickbauman 11y ago

> I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at. This is a wonderful tenet in general: let's think about what X is bad at, rather than what it's good for. While I'm not much of a Rubyist (I've hacked a little in my time, but I fall on the Python side of the divide), working with trees (hashes of hashes are a kind of tree) has never been easier than in Clojure in my experience. I suspect this is true of any lisp.

sanderjd 11y ago
Could you expand on how Clojure makes it easier to deal with the frustrations your parent comment mentions (deep merges, shared reliance on hash structure, reproduced logic, and handling of optional values)? I've only used Clojure a bit, but I'm not sure how it solves any of these problems (which I also find frustrating in Ruby).
- prospero 11y ago
  Immutability is a big part of it, as is the wide array of operators for maps and map-like data structures.
- hellofunk 11y ago
  Hickey has stated elsewhere that optional values don't really make sense in a dynamically-typed language.
- nickbauman 11y ago
  Persistent data structures are part of it, for sure. But on a higher level. On a lower level, if you look at how and assoc-in works in Clojure, the scales will fall from your eyes (they certainly fell from mine). Consider: (def g {:x "a" :y {:k 71}}) user=> (assoc-in g [:y :k] 72) {:y {:k 72}, :x "a"} With this you can walk and mod deeply nested datastructures with ease (and others can reason about it well).
  - aaron-lebo 11y ago

akurilin 11y ago

Your comment seems more about Clojure's type system and fundamental data structures than about functional programming in general. e.g. Haskell solves some/most of the problems you mention, while introducing different ones and presenting other trade-offs to consider.

sheepmullet 11y ago

There is actually a lot of complexity to hashes in Ruby. The two major sources of complexity are the fact they are mutable and the fact that keys are mutable but hashes are computed when the object is first added. So if you construct a valid hash, then call three functions passing them the hash, you have no idea if the hash is still valid for the second and third functions without reading the code for all the preceding functions. What this tends to mean in practice is you are always having to check all over the place that your hash is valid. They might look the same as clojure hashes but in practice they are used very differently.

schneems 11y ago
> There is actually a lot of complexity to hashes in Ruby. I agree here > So if you construct a valid hash, then call three functions passing them the hash, you have no idea if the hash is still valid for the second and third functions without reading the code for all the preceding functions. I don't find that to be true, even in the Rails codebase. You pass a hash to a function (method) and expect that it won't be mutated. If you're writing a method and you mutate a hash, you're expected to dup the argument so it won't be mutated. This is the convention. There are times when hashes are mutated, but generally that's reserved for methods who's purpose are to mutate its arguments. Where Rails gets into trouble in its functional passing of hashes isn't in the mutability of the hashes, it's in the composability of the functions. I've never written about this problem before, so i'm not sure I have a great example but it's definitely a problem. One example in rails is `url_for` it takes a hash argument. The problem is that there's multiple `url_for` methods that all do slightly different things. Some are needed for generating links in email, while some work in your views, and others are designed for you to use programmatically outside of a view context. One of the hardest things about this method is that one `url_for` can call another `url_for`. Since we can never be guaranteed the order of the calls it is really it makes things like having default values, or optional keys. You have to replicate logic in different functions since some may never be called in the order you might expect. This significantly impacts our ability to refactor which in turn impacts our ability to make performance improvements. I recently did a bunch of perf work in https://engineering.heroku.com/blogs/2015-08-06-patching-rails-performance/ https://engineering.heroku.com/blogs/2015-08-06-patching-rai... and some of my biggest perf improvements were getting rid of duping and merging of hashes. If Ruby had an immutable and performance efficient hash then it would have helped a bunch, however I don't think it would make the general awfulness that is an entirely hash based API to a very complex action (such as url_for) that significantly better to work with.

hellofunk 11y ago

>I think clojure helps with the speed of allocating new immutable structures by using copy on write strategies behind the scenes. No, it's not copy on write in the sense that COW means in other languages. Data in clojure is persistent, so usually even an altered piece of data is not actually copied, only the tiny bit that changed is added, if necessary. This is very different than, for example, Swift that implements copy on write, but the whole data structure is copied even if the change is minor. Additionally, you say: >When we're dealing with sequential tasks, mutating data is much faster than allocating new data structures That would be true in cases where the entire data piece is new; but so much of what you do in any language involves interative changes over existing data, and again because of clojure's persistence, new allocations are often not happening at all in many cases. ...However, I do agree with most of your other points :)

schneems 11y ago
I was introduced to http://hypirion.com/musings/understanding-persistent-vector-pt-1 http://hypirion.com/musings/understanding-persistent-vector-... not too long ago on HN. I don't use clojure so I was paraphrasing my somewhat limited understanding. Thanks for the clarification. That part about speed was more about my general frustration with the "immutable is fast" and "mutable is slow" meme that I hear too frequently. While that can be the case, it isn't always 100% true. Your language and how you use it can play a huge part. I'm hoping to learn more about clojure data structures in the coming months, so much good stuff.
- j-pb 11y ago
  My guess would be that immutable clojure is still faster than say mutable ruby though. I don't think anybody is arguing that trie lookup is faster than array lookup. But when it comes to passing data between parts of the system, zero cost copies are much faster than copying buffers or objects.

rubiquity 11y ago

> I think Rich is an extremely smart and very accomplished programmer. I think functional programming is really good at some things, I don't feel like many people talk about the things it's not good at. As someone who used to swear by Ruby but now prefers FP langs, part of learning FP is as much about learning what FP is good at as it is about learning what FP isn't good at. Given FP's proclamation for declarative style and immutability, clearly FP isn't good at domains where imperative styles are important or really, really, really[0] CPU intensive work where mutability is needed. Although, thanks to compiler research (which is mostly done by FP language users, ahem) languages like OCaml are bridging this gap when you take a look at projects like MirageOS. You get to write your code in a declarative, immutable style that then gets compiled to very fast native code. It's having your cake and eating it, too. As far as your original complaint of everything in Ruby either being a hash of hashes or a class that just wraps a hash of hashes, I completely agree. That is what drove me away from the language. I'd rather just blow that hash + class relationship up completely. Not to mention, in ML-dialect languages you can use features like Enum/Sum/Product types to define the configuration/syntax of your programs which is a lot better than reading arbitrary keys in a hash, as you have stated. 0 - No, I mean really, really, really, really. "I think I need C to do this fast enough for my use case" is the "I need Cassandra for my 2GB database of 'Big Data'" of the FP world.

erichmond 11y ago

Using data structures without a schema is user error, and isn't the fault of any particular language. The rest of your points have some merit. :)

jdc 11y ago

> Every part of your system must now know the hash structure you're using and often reproduce logic in many places like dealing with optional keys that may be missing. Why not just write accessors and mutators for your hashes? Also, Python hash a function dict.get(key, default). Does Ruby not have this?