7 ms·

ICFP report on the use of Haskell at Google

50 points by palsecam 16y ago

edifice 16y ago
Good article. Mirrors my own experience in three ways: 1. Immutable datastructures make it a lot easier to reason about data flow and to write correct code. 2. Static typing helps write cleaner, more explicit APIs. 3. Programming effectively in a functional style is more difficult than banging out imperative code. I wonder why they didn't try Scala. With Google's extensive Java infrastructure that seems like a more logical choice. Certainly the problems they discuss with Haskell's string handling and debugging support wouldn't be issues in Scala.
- iustin 16y ago
  As I tried to explain in the paper, this started as an experiment, and not "let's write this in Haskell and use it in production". But aside from that, the cluster balancer runs in just 5MB of RAM (RSS). I'm not familiar with Scala, but I doubt any JVM can run in just that; and the way we deploy this software, we like it to use as few memory as possible.
  - edifice 16y ago
    That's true. Scala can be pretty fast but it needs a lot of ram. Thanks for taking the time to write this up. Detailed descriptions of real-world applications of FP languages are always welcome.
- runT1ME 16y ago
  >3. Programming effectively in a functional style is more difficult than banging out imperative code. As someone who is just starting to learn functional programming, do you believe this is because "we've" all been taught to think imperatively, and functional is just new to us? Or is it something inherent in functional style coding?
  - edifice 16y ago
    Good question. My opinion is that thinking in terms of recursive transformations of immutable datastructures is just fundamentally more difficult than the imperative equivalents. My earliest exposure to recursive algorithms was in AP computer science in high school and I remember a lot of the students really struggled with them, both those with previous imperative experience and without. This extra effort can pay off over time though because it makes the overall program easier to understand and incurs fewer bugs than the imperative style. You can also avoid a lot of this if you express your functions in terms of other elemental sequence functions (map, reduce etc) instead of explicit recursion. Doubtless others will disagree though. Maybe we don't all have the same mental strengths and weaknesses.
    - jemfinch 16y ago
      Chalk me up as someone who disagrees. Personally, I find recursive transformations of immutable structures far easier to reason about than iterative modifications to mutable structures. At least in some cases, I think that's the expected result: consider the difficulty of reversing a singly linked list in-place and reversing it applicatively.
      - chromatic 16y ago
        Perhaps the difficulty is in designing the appropriate data structures and identifying the correct transformation functions. In other words, designing a good FP program is difficult just like designing a good OO program is difficult.
      - edifice 16y ago
        It definitely depends on the algorithm. It also depends on the language. A lot of the difficulty of learning Haskell, for example, is in the type system. A dynamically typed FP lang like Clojure has a shallower learning curve.
        iustin 16y ago
        It is true, but it's also unfortunate—I think a lot of Haskell's strengths come from its type system. So what makes Haskell worth learning also makes it hard to learn Haskell :)
jrockway 16y ago
Weird. It seems like they didn't really know Haskell going into this; not having heard of "Either String" and not knowing that you can get a stack trace when you run your code in ghci.
- iustin 16y ago
  I thought that is clear in the paper—I didn't have (almost) any experience with Haskell before, indeed. And the stack trace example is not related to ghci. You run software in production, and it crashes. What do you get?
  - jrockway 16y ago
    I write my software in Haskell because it doesn't crash in production. But then again, before I deploy, I ensure that there are unit tests.
strlen 16y ago
When I worked at Yahoo in an operations role ("production engineering", Yahoo's equivalent of Google's SRE), my team did something similar, we used OCaml in combination with Perl for one task: developing an a mini-language language for describing large clusters of machines (ranges of machines, querying and describing metadata such memberships of machine in a cluster, designating special nodes, descriptions of clusters). Although we ended up switching to C and lex/yacc for this mini-language (due to compiler issues after the switch to 64-bit OS, no idea if these compiler issues were fixed) it was a great experience. I've since moved on to a software engineering role, but there's something to be said about the freedom (e.g., picking your tools) that exists when you're writing software that only you are responsible for.
nickik 16y ago
Would be intressting to know why Haskell? Did already know a little or did do a comparison of more then one FP Language.

50 points by palsecam 16y ago

Good article. Mirrors my own experience in three ways: 1. Immutable datastructures make it a lot easier to reason about data flow and to write correct code. 2. Static typing helps write cleaner, more explicit APIs. 3. Programming effectively in a functional style is more difficult than banging out imperative code. I wonder why they didn't try Scala. With Google's extensive Java infrastructure that seems like a more logical choice. Certainly the problems they discuss with Haskell's string handling and debugging support wouldn't be issues in Scala.

iustin 16y ago
As I tried to explain in the paper, this started as an experiment, and not "let's write this in Haskell and use it in production". But aside from that, the cluster balancer runs in just 5MB of RAM (RSS). I'm not familiar with Scala, but I doubt any JVM can run in just that; and the way we deploy this software, we like it to use as few memory as possible.
- edifice 16y ago
  That's true. Scala can be pretty fast but it needs a lot of ram. Thanks for taking the time to write this up. Detailed descriptions of real-world applications of FP languages are always welcome.
runT1ME 16y ago
>3. Programming effectively in a functional style is more difficult than banging out imperative code. As someone who is just starting to learn functional programming, do you believe this is because "we've" all been taught to think imperatively, and functional is just new to us? Or is it something inherent in functional style coding?
- edifice 16y ago
  Good question. My opinion is that thinking in terms of recursive transformations of immutable datastructures is just fundamentally more difficult than the imperative equivalents. My earliest exposure to recursive algorithms was in AP computer science in high school and I remember a lot of the students really struggled with them, both those with previous imperative experience and without. This extra effort can pay off over time though because it makes the overall program easier to understand and incurs fewer bugs than the imperative style. You can also avoid a lot of this if you express your functions in terms of other elemental sequence functions (map, reduce etc) instead of explicit recursion. Doubtless others will disagree though. Maybe we don't all have the same mental strengths and weaknesses.
  - jemfinch 16y ago
    Chalk me up as someone who disagrees. Personally, I find recursive transformations of immutable structures far easier to reason about than iterative modifications to mutable structures. At least in some cases, I think that's the expected result: consider the difficulty of reversing a singly linked list in-place and reversing it applicatively.
    - chromatic 16y ago
      Perhaps the difficulty is in designing the appropriate data structures and identifying the correct transformation functions. In other words, designing a good FP program is difficult just like designing a good OO program is difficult.
    - edifice 16y ago
      It definitely depends on the algorithm. It also depends on the language. A lot of the difficulty of learning Haskell, for example, is in the type system. A dynamically typed FP lang like Clojure has a shallower learning curve.
      - iustin 16y ago
        It is true, but it's also unfortunate—I think a lot of Haskell's strengths come from its type system. So what makes Haskell worth learning also makes it hard to learn Haskell :)

jrockway 16y ago

Weird. It seems like they didn't really know Haskell going into this; not having heard of "Either String" and not knowing that you can get a stack trace when you run your code in ghci.

iustin 16y ago
I thought that is clear in the paper—I didn't have (almost) any experience with Haskell before, indeed. And the stack trace example is not related to ghci. You run software in production, and it crashes. What do you get?
- jrockway 16y ago
  I write my software in Haskell because it doesn't crash in production. But then again, before I deploy, I ensure that there are unit tests.

strlen 16y ago

When I worked at Yahoo in an operations role ("production engineering", Yahoo's equivalent of Google's SRE), my team did something similar, we used OCaml in combination with Perl for one task: developing an a mini-language language for describing large clusters of machines (ranges of machines, querying and describing metadata such memberships of machine in a cluster, designating special nodes, descriptions of clusters). Although we ended up switching to C and lex/yacc for this mini-language (due to compiler issues after the switch to 64-bit OS, no idea if these compiler issues were fixed) it was a great experience. I've since moved on to a software engineering role, but there's something to be said about the freedom (e.g., picking your tools) that exists when you're writing software that only you are responsible for.

nickik 16y ago

Would be intressting to know why Haskell? Did already know a little or did do a comparison of more then one FP Language.