25 ms·

Haskell in the Large [pdf]

187 points by kryptiskt 12y ago

taeric 12y ago
"Your choice of programming language can have a real effect on these results over time." Probably more accurate to say "Choice of programming team can have a real effect on these results over time." I want to like Haskell. However, I have given up on wanting to dislike Java. To the point that it is painful to read such things as "you can write Java in any language."
- the_af 12y ago
  To be fair, that's one sentence in the second to last slide of 42 slides. The presentation, at least what I can see in the slides, is hardly Java bashing :) There are 3 mentions of Java in the whole presentation, only one of which has negative connotations: the one you quoted.
  - tel 12y ago
    And to be extra fair "you can write Java in any language" is much more a dig at some of the bad patterns which Java has become known for culturally than anything about the language itself!
  - taeric 12y ago
    Yeah, the Java bashing I could (and, admittedly, should) have ignored. It is more the crediting the language for the success that I would rather focus on. It is a fairly strong assertion at the end, that I feel needs more support. Back when fewer teams were writing Java applications, I feel similar advantages were felt for it. Of course, I still pine for lisp, so I can not claim to have no biases. (Well, that and MMIX)
    - kyllo 12y ago
      The entire Java community pines for lisp, whether they know it or not. That's why Apache Ant was invented, it's Java's manifestation of Greenspun's Tenth Rule.
      - taeric 12y ago
        Is it confined to the Java community? Seems a bit more widespread, honestly.
        kyllo 12y ago
        Well yeah, that's why the original Greenspun's tenth rule cited "Any sufficiently complicated C or Fortran program..." Those who do not learn from history are doomed to repeat it. That is the only logical explanation for why XML even exists.
- stonemetal 12y ago
  I read that rather differently. I see it as a poke at management that wants to make the move because of hype not a real understanding of what it will take, and not put the effort in training\hiring devs who have as he puts it "have taste and guidance".
olenhad 12y ago
I wonder what's the primary reason for their "Mu" compiler adopting a "strict-ish" evaluation strategy.
- tel 12y ago
  A pretty common idea in Haskell-land is that "the next Haskell would be strict". Laziness was once a dreamy ideal execution strategy, but thanks to Haskell the practical tradeoff is better understood. At the same time (as the slides note) the strict/lazy divide is hardly decided! As many people who would rather Haskell be strict are willing to defend laziness to the death for being the key reason Haskell is so composable. [0] So Lennart Augustsson—the author of the first standard's compliant Haskell compiler, I believe—wrote up Mu and made it strict. There is probably a justification in SC's case for why it was an experiment worth trying. [0] At the very least it's indisputable that laziness forced Haskell's hand with purity and that ended up being an enormous win in everyone's opinion.
  - the_af 12y ago
    This is interesting, because the well-known paper "Why functional programming matters" identifies two key aspects of FP: higher-order functions and lazy evaluation. I wonder if John Hughes has reviewed his opinion on this, or if the FP community thinks the paper is no longer an accurate insight into FP... In particular, I'm thinking of the last paragraph of Hughes' conclusions: "It is also relevant to the present controversy over lazy evaluation. Some believe that functional languages should be lazy, others believe they should not. Some compromise and provide only lazy lists, with a special syntax for constructing them (as, for example, in SCHEME [AS86]). This paper provides further evidence that lazy evaluation is too important to be relegated to second-class citizenship. It is perhaps the most powerful glue functional programmers possess. One should not obstruct access to such a vital tool." Maybe it turned out that practical evidence has shown that lazy evaluation wasn't as important for modularity as Hughes thought, or at least that its drawbacks have been found unacceptable in practice?
    - tel 12y ago
melling 12y ago
Has anyone used both Haskell and OCaml (or F#?)? How do they compare in practice? I've been wanting to put some time into a functional language and I've been debating between Haskell and OCaml. Because of F#, OCaml seems like it might be the more practical language (i.e. direct job opportunities). However, excluding F#, Haskell does seem to much more popular than OCaml.
- codygman 12y ago
  How do they compare in practice might be too broad a question, I'm willing to bet you'd get more responses if you asked for comparisons in certain use cases.
- tel 12y ago
  Either one is great. Short of it is that OCaml and Haskell will both teach you a lot of the same things. OCaml has better modularity features (namely, genuine modules which are replicated almost nowhere else) and Haskell has better purity features (namely... purity). It turns out that sophisticated functional programming essentially stresses that good modularity and good purity are both killer features and techniques and they should be in heavy use all of the time. In OCaml you'll have an easy time expressing the exact modularity concerns you find important, but will struggle a bit to manage purity explicitly. In Haskell you'll find it beautiful to express pure and typed-effectful computation as much as needed, but will struggle a bit with expressing modularity as you'd like. Haskell also has the whole typeclass system which is really, really interesting and valuable for many kinds of expressiveness. It's sort of an oddball feature, but interesting to see the impact of. Personally, I find strict purity more important than expressive modularity. But if anyone could really put them together in a way that worked it'd be great. I'm not so sure such a language exists today, though.
  - thinkpad20 12y ago
    > Personally, I find strict purity more important than expressive modularity. But if anyone could really put them together in a way that worked it'd be great. I'm not so sure such a language exists today, though. There is Ur, which has both ML-style modules and Haskell-style type classes. It's also pure and strict. And it has advanced type-level programming features and record types. There's a lot to like. There are a few problems with it, though. It's mostly a one-man project. The compiler is immense, complex and almost entirely devoid of documentation (it's pretty amazing actually, like 1 comment per 1000 lines or so), so contributing is hard. It's very difficult, if not impossible, to use the language for anything but web development: there's no way that I know to execute a program directly, rather than as a sub-program of the ur/web framework. The standard library is tiny and doesn't even include `print`. But even with those warts it has enough good things about it that it's worth checking out.
spopejoy 12y ago
“Make illegal states unrepresentable” Types pay off the most on large systems. Architectural requirements captured formally. This more than ANYTHING else is why I want to move enterprisey app code to Haskell. Having worked on numerous ginormous enterprisey systems -- which are usually doing pretty straightforward things, just at scale, and needing to be maintained by non-brilliant developers -- I can say pretty securely that north of 95% of the invariants could be lifted into the type system, making run-time errors a thing of the past. Also, in most frameworky big systems, you WANT to stop devs from "just doing IO" or pretty much doing anything without a strong contract around it. Monad stacks do wonders for circumscribing your computational context in an app. I wonder why they rewrote Aeson though ... perhaps before it was mature? Aeson's pretty awesome.
- carterschonwald 12y ago
  It might have to do with the way the default generics for aesons fromjson and tojson instances for record types are defined? Or maybe the lack of incremental streaming read write support in the aeson data model? Or maybe they wanted better date or integer support? It'd be pretty easy to ask don or Lennart I'd imagine. I have to say that I definitely enjoy using haskell when faced with dirty complicated data, which I'm told happens a lot in larger organizations.
- eru 12y ago
  > Monad stacks do wonders for circumscribing your computational context in an app. Wait until you discover the power of Applicative Functors. They are more restricted in their operations than Monads, so they compose better and allow for analysis.
  - davdar 12y ago
    I'm curious, what do you mean they compose better and allow for analysis [better than monads]?
    - gergoerdi 12y ago
      You can determine a lot more about a computation described only in terms of an applicative functor as you can from a monadic one, without running any of its effects (i.e. "statically"). It's easy to see why when you consider the opaqueness of the function in the right-hand side of a bind. Check out http://gergo.erdi.hu/blog/2012-12-01-static_analysis_with_applicatives/ http://gergo.erdi.hu/blog/2012-12-01-static_analysis_with_ap... for an example.
dkarapetyan 12y ago
I see. So all you need are compiler/interpreter experts that can turn any problem into a interpreter/compiler problem and you're golden. I'm not saying the approach is not worthwhile but how exactly does this generalize to other workplaces where there is no critical mass of such experts? I mean they have their own compiler for Haskell for Pete's sake. I would also like to know how many of the core team members have PhDs and MScs. We can check off Don and Lennart. Maybe Don is really measuring the effects of PhDs in language/compiler design on how to manage complexity? Alternatively, Facebook has been experimenting with Haskell and OCaml. Seeing their case studies would be valuable as well.
- tel 12y ago
  I think "interpreter/compiler" problems arise way more frequently than people expect. It's just they're often not called that until they're being tackled by interpreter/compiler experts. In particular, you can think of things like message passing, interpreter/command patterns, and anything which uses reflection as being at least a little like an AST/interpreter pattern if you look at it in the right light. Then, if you have a team of compiler writers they can make a big win there.
  - dkarapetyan 12y ago
    I completely agree. When viewed from the right angle almost everything is a combination of some "VM bytecode" and a "compiler" targeting that bytecode. Greenspun's tenth rule applies to anything large enough. That's not what I'm getting at. What I'm getting at is that when you have a bunch of problem solvers well versed in programming language design and theory the programming language at that point is no longer relevant. Standard Chartered doesn't have all that software because Haskell somehow gave them superpowers. Standard Chartered has all that software because a bunch of PhDs chose Haskell to write it in and along the way Lennart wrote another compiler because why not, the man is good at it. So instead of being a case study in how to apply a functional language to solve problems this is really a presentation about what kind of people you want to hire to build systems, i.e. PhDs, MScs, and MDs (apparently). Imagine if the title of this post was "PHP in the large". Would anyone take it seriously? Does anyone really believe PHP is a great language to build large systems in? No, everyone would immediately jump on the fact that all of it is running on HHVM and there are bunch of smart folks optimizing the hell out of it. Same here. The metapoint here is really a quote by Rich Hickey from a David Nolen talk/blog post - "Not everything is awesome". There are few hidden gems in the presentation when concessions are made by including Any type and re-inventing a lot of Erlang style process management. I wonder why more trading companies don't leverage Erlang and Dialyzer? The relevant post for the quote: http://swannodette.github.io/2015/01/09/life-with-dynamic-typing/ http://swannodette.github.io/2015/01/09/life-with-dynamic-ty....
michaelochurch 12y ago
I'm really happy to hear about Standard Chartered's success with this, and I want to know more. This is really promising stuff. My current company is looking into our "next generation" platform for when our datasets exceed what we can do in R. R may not be the best language, but it's great for exploratory data science, has the best or the only library out there for some ML purposes, and we've done a lot of things to make it production-worthy (on small- and medium-data). We'll probably need to involve something else, down the road, when our data sets get larger than what fits on one box. The leading candidates are Haskell, Clojure, and Scala (Scala because of Spark). I'll have to evaluate the languages fairly and relative to our needs, but I hope Haskell wins for a number of reasons, including the fact that Chicago + Haskell is an unfilled niche and we'd attract a ton of talent. For those who've taken Haskell this far into production: have you encountered any negatives? Are there any times when you think it might be better not to have a strong type system? To me, the biggest drawback of Haskell isn't anything intrinsic to the language, but the amount of stuff it forces a person to learn. For me, that's a fun challenge... but trying to convince 110 programmers to use a language that forces I/O into a set of types (loosely) called a monad seems like an epic task. Clojure has the advantage of being simple and beautiful once you get past the parentheses. Haskell is demanding and frustrating for the first 6 months (and pays off handsomely later on, but this can make it a hard sell). Also, how does the Any type in Mu (if anyone familiar with it is here) differ from Data.Dynamic?
- ajtulloch 12y ago
  I'm sorry - I love Haskell, and it's a great choice for a lot of domains, but it's a poor choice for numeric/scientific computation. The ecosystem just isn't there yet. Here's a recent r/haskell thread discussing this: http://www.reddit.com/r/haskell/comments/2rsxrb/is_haskell_a_good_programming_language_for/ http://www.reddit.com/r/haskell/comments/2rsxrb/is_haskell_a...
- tel 12y ago
  Exploratory data analysis isn't killer in Haskell. I think types can help a lot here (god knows the amount of pain that R/Numpy spaghetti has inflicted) but the jury is still out for how to do it well. Row types seem like an important technology which gets a little bolted on to Haskell---both exciting and unclear at the moment. https://acowley.github.io/Frames/ https://acowley.github.io/Frames/ On the other hand, if you've got a solid data pipeline you're scaling then there are grand tools for streaming data in Haskell. You're in the complete sweet spot for strong typing. You can also drop down into C easily when speed is an issue. Matrix libraries exist for the intrepid user. HMatrix is BSD now so you can at least touch LAPACK.
saosebastiao 12y ago
I've toyed with Haskell, ultimately moving on to the Ocaml/F# camp. There are only two things that I miss from Haskell without an appropriate equivalent or easy workaround: Type Classes, and Higher Kinded Types. This big roadblock that everyone claims with Haskell, Monads, didn't give me any problems at all...even if it didn't make any sense to resort to them to do something as trivial as IO. What really turned me off more than anything was the combination of the academic focus of the community combined with this weird culture that I could only describe as a cleverness competition. I'm a pragmatist, and it is obvious that real software can be created with Haskell, but it doesn't really come through in the tutorials or books. There is a lot of "Look, this is really cool!", but rarely a follow up with "and this is why it matters!". Everything seemed to revolve around cool tricks with no practical concern behind them, and then in the comments you inevitably find comments from other Haskellers claiming to do it just a little bit more cleverly using <<lenses, GADTs, zippers, continuations, or some other obscure abstraction>>. Ultimately it was Ocaml and F# that taught me all of the really important lessons of the ML family, despite the relative lack of learning resources out there. It was there that I found the benefits of making illegal states unrepresentable, expressive pattern matching, type inference, composition over inheritance, etc. It is a shame, because as languages and runtimes, they probably are inferior to Haskell. Ocaml does fine with concurrency, but parallelism is a disaster. Its class system, its one major distinction from SML, is mostly considered a code smell. Its stance on operator overloading requires me to keep a table of operators handy, instead of the usual intuitive ones. And F# has the sane operators and parallelism story but doesn't have SML/Ocaml Functors, and is still a Windows-first ecosystem (Mono has gotten a lot better, but its still a kludge).
- Ironballs 12y ago
  The Windows-first status of F# is changing. There have been major improvements in this recently, most notably, the whole compiler being open sourced and its development moved to github[1], and .NET is going to be available on Linux[2]. This is not reality yet, but it will be during this year. [1] https://github.com/microsoft/visualfsharp https://github.com/microsoft/visualfsharp [2] http://www.hanselman.com/blog/AnnouncingNET2015NETasOpenSourceNETonMacandLinuxandVisualStudioCommunity.aspx http://www.hanselman.com/blog/AnnouncingNET2015NETasOpenSour...
jongraehl 12y ago
I wonder why they don't just use GHC with some customization to make the default eval strategy less lazy. I understand why someone might prefer more predictable (strict) evals by default.
- tel 12y ago
  Lennart Augustsson is a skilled compiler writer, especially for Haskell, to say the least. Mu does more than just strictify Haskell—the code portability stuff is pretty unique as well, for instance.
- eru 12y ago
  Starting from scratch, building on GHC might have made sense. There were historical reasons for going with their own compiler. (Not sure, how much I can say.) And Lennart pulled it off.
- tome 12y ago
  They were targeting an already existing runtime for on older in-house functional language, and the runtime only supported strict evaluation.
thu 12y ago
Lazy vs strict evaluation: Lennart, the author of the Mu compiler, has written about it: http://augustss.blogspot.co.uk/2011/05/more-points-for-lazy-evaluation-in.html http://augustss.blogspot.co.uk/2011/05/more-points-for-lazy-...
boothead 12y ago
This An awful lot of data mining and analysis is best done with relational algebra Is an interesting statement. Would anyone care to elaborate?
- tome 12y ago
  I think it means that a plethora operations you perform on sets, lists, maps, hash maps and indeed collections of all kinds are just specific implementations of the general "relational algebra" operations.

187 points by kryptiskt 12y ago

"Your choice of programming language can have a real effect on these results over time." Probably more accurate to say "Choice of programming team can have a real effect on these results over time." I want to like Haskell. However, I have given up on wanting to dislike Java. To the point that it is painful to read such things as "you can write Java in any language."

the_af 12y ago
To be fair, that's one sentence in the second to last slide of 42 slides. The presentation, at least what I can see in the slides, is hardly Java bashing :) There are 3 mentions of Java in the whole presentation, only one of which has negative connotations: the one you quoted.
- tel 12y ago
  And to be extra fair "you can write Java in any language" is much more a dig at some of the bad patterns which Java has become known for culturally than anything about the language itself!
- taeric 12y ago
  Yeah, the Java bashing I could (and, admittedly, should) have ignored. It is more the crediting the language for the success that I would rather focus on. It is a fairly strong assertion at the end, that I feel needs more support. Back when fewer teams were writing Java applications, I feel similar advantages were felt for it. Of course, I still pine for lisp, so I can not claim to have no biases. (Well, that and MMIX)
  - kyllo 12y ago
    The entire Java community pines for lisp, whether they know it or not. That's why Apache Ant was invented, it's Java's manifestation of Greenspun's Tenth Rule.
    - taeric 12y ago
      Is it confined to the Java community? Seems a bit more widespread, honestly.
      - kyllo 12y ago
        Well yeah, that's why the original Greenspun's tenth rule cited "Any sufficiently complicated C or Fortran program..." Those who do not learn from history are doomed to repeat it. That is the only logical explanation for why XML even exists.
stonemetal 12y ago
I read that rather differently. I see it as a poke at management that wants to make the move because of hype not a real understanding of what it will take, and not put the effort in training\hiring devs who have as he puts it "have taste and guidance".

olenhad 12y ago

I wonder what's the primary reason for their "Mu" compiler adopting a "strict-ish" evaluation strategy.

tel 12y ago
A pretty common idea in Haskell-land is that "the next Haskell would be strict". Laziness was once a dreamy ideal execution strategy, but thanks to Haskell the practical tradeoff is better understood. At the same time (as the slides note) the strict/lazy divide is hardly decided! As many people who would rather Haskell be strict are willing to defend laziness to the death for being the key reason Haskell is so composable. [0] So Lennart Augustsson—the author of the first standard's compliant Haskell compiler, I believe—wrote up Mu and made it strict. There is probably a justification in SC's case for why it was an experiment worth trying. [0] At the very least it's indisputable that laziness forced Haskell's hand with purity and that ended up being an enormous win in everyone's opinion.
- the_af 12y ago
  This is interesting, because the well-known paper "Why functional programming matters" identifies two key aspects of FP: higher-order functions and lazy evaluation. I wonder if John Hughes has reviewed his opinion on this, or if the FP community thinks the paper is no longer an accurate insight into FP... In particular, I'm thinking of the last paragraph of Hughes' conclusions: "It is also relevant to the present controversy over lazy evaluation. Some believe that functional languages should be lazy, others believe they should not. Some compromise and provide only lazy lists, with a special syntax for constructing them (as, for example, in SCHEME [AS86]). This paper provides further evidence that lazy evaluation is too important to be relegated to second-class citizenship. It is perhaps the most powerful glue functional programmers possess. One should not obstruct access to such a vital tool." Maybe it turned out that practical evidence has shown that lazy evaluation wasn't as important for modularity as Hughes thought, or at least that its drawbacks have been found unacceptable in practice?
  - tel 12y ago

melling 12y ago

Has anyone used both Haskell and OCaml (or F#?)? How do they compare in practice? I've been wanting to put some time into a functional language and I've been debating between Haskell and OCaml. Because of F#, OCaml seems like it might be the more practical language (i.e. direct job opportunities). However, excluding F#, Haskell does seem to much more popular than OCaml.

codygman 12y ago
How do they compare in practice might be too broad a question, I'm willing to bet you'd get more responses if you asked for comparisons in certain use cases.
tel 12y ago
Either one is great. Short of it is that OCaml and Haskell will both teach you a lot of the same things. OCaml has better modularity features (namely, genuine modules which are replicated almost nowhere else) and Haskell has better purity features (namely... purity). It turns out that sophisticated functional programming essentially stresses that good modularity and good purity are both killer features and techniques and they should be in heavy use all of the time. In OCaml you'll have an easy time expressing the exact modularity concerns you find important, but will struggle a bit to manage purity explicitly. In Haskell you'll find it beautiful to express pure and typed-effectful computation as much as needed, but will struggle a bit with expressing modularity as you'd like. Haskell also has the whole typeclass system which is really, really interesting and valuable for many kinds of expressiveness. It's sort of an oddball feature, but interesting to see the impact of. Personally, I find strict purity more important than expressive modularity. But if anyone could really put them together in a way that worked it'd be great. I'm not so sure such a language exists today, though.
- thinkpad20 12y ago
  > Personally, I find strict purity more important than expressive modularity. But if anyone could really put them together in a way that worked it'd be great. I'm not so sure such a language exists today, though. There is Ur, which has both ML-style modules and Haskell-style type classes. It's also pure and strict. And it has advanced type-level programming features and record types. There's a lot to like. There are a few problems with it, though. It's mostly a one-man project. The compiler is immense, complex and almost entirely devoid of documentation (it's pretty amazing actually, like 1 comment per 1000 lines or so), so contributing is hard. It's very difficult, if not impossible, to use the language for anything but web development: there's no way that I know to execute a program directly, rather than as a sub-program of the ur/web framework. The standard library is tiny and doesn't even include `print`. But even with those warts it has enough good things about it that it's worth checking out.

spopejoy 12y ago

“Make illegal states unrepresentable” Types pay off the most on large systems. Architectural requirements captured formally. This more than ANYTHING else is why I want to move enterprisey app code to Haskell. Having worked on numerous ginormous enterprisey systems -- which are usually doing pretty straightforward things, just at scale, and needing to be maintained by non-brilliant developers -- I can say pretty securely that north of 95% of the invariants could be lifted into the type system, making run-time errors a thing of the past. Also, in most frameworky big systems, you WANT to stop devs from "just doing IO" or pretty much doing anything without a strong contract around it. Monad stacks do wonders for circumscribing your computational context in an app. I wonder why they rewrote Aeson though ... perhaps before it was mature? Aeson's pretty awesome.

carterschonwald 12y ago
It might have to do with the way the default generics for aesons fromjson and tojson instances for record types are defined? Or maybe the lack of incremental streaming read write support in the aeson data model? Or maybe they wanted better date or integer support? It'd be pretty easy to ask don or Lennart I'd imagine. I have to say that I definitely enjoy using haskell when faced with dirty complicated data, which I'm told happens a lot in larger organizations.
eru 12y ago
> Monad stacks do wonders for circumscribing your computational context in an app. Wait until you discover the power of Applicative Functors. They are more restricted in their operations than Monads, so they compose better and allow for analysis.
- davdar 12y ago
  I'm curious, what do you mean they compose better and allow for analysis [better than monads]?
  - gergoerdi 12y ago
    You can determine a lot more about a computation described only in terms of an applicative functor as you can from a monadic one, without running any of its effects (i.e. "statically"). It's easy to see why when you consider the opaqueness of the function in the right-hand side of a bind. Check out http://gergo.erdi.hu/blog/2012-12-01-static_analysis_with_applicatives/ http://gergo.erdi.hu/blog/2012-12-01-static_analysis_with_ap... for an example.

dkarapetyan 12y ago

I see. So all you need are compiler/interpreter experts that can turn any problem into a interpreter/compiler problem and you're golden. I'm not saying the approach is not worthwhile but how exactly does this generalize to other workplaces where there is no critical mass of such experts? I mean they have their own compiler for Haskell for Pete's sake. I would also like to know how many of the core team members have PhDs and MScs. We can check off Don and Lennart. Maybe Don is really measuring the effects of PhDs in language/compiler design on how to manage complexity? Alternatively, Facebook has been experimenting with Haskell and OCaml. Seeing their case studies would be valuable as well.

tel 12y ago
I think "interpreter/compiler" problems arise way more frequently than people expect. It's just they're often not called that until they're being tackled by interpreter/compiler experts. In particular, you can think of things like message passing, interpreter/command patterns, and anything which uses reflection as being at least a little like an AST/interpreter pattern if you look at it in the right light. Then, if you have a team of compiler writers they can make a big win there.
- dkarapetyan 12y ago
  I completely agree. When viewed from the right angle almost everything is a combination of some "VM bytecode" and a "compiler" targeting that bytecode. Greenspun's tenth rule applies to anything large enough. That's not what I'm getting at. What I'm getting at is that when you have a bunch of problem solvers well versed in programming language design and theory the programming language at that point is no longer relevant. Standard Chartered doesn't have all that software because Haskell somehow gave them superpowers. Standard Chartered has all that software because a bunch of PhDs chose Haskell to write it in and along the way Lennart wrote another compiler because why not, the man is good at it. So instead of being a case study in how to apply a functional language to solve problems this is really a presentation about what kind of people you want to hire to build systems, i.e. PhDs, MScs, and MDs (apparently). Imagine if the title of this post was "PHP in the large". Would anyone take it seriously? Does anyone really believe PHP is a great language to build large systems in? No, everyone would immediately jump on the fact that all of it is running on HHVM and there are bunch of smart folks optimizing the hell out of it. Same here. The metapoint here is really a quote by Rich Hickey from a David Nolen talk/blog post - "Not everything is awesome". There are few hidden gems in the presentation when concessions are made by including Any type and re-inventing a lot of Erlang style process management. I wonder why more trading companies don't leverage Erlang and Dialyzer? The relevant post for the quote: http://swannodette.github.io/2015/01/09/life-with-dynamic-typing/ http://swannodette.github.io/2015/01/09/life-with-dynamic-ty....

michaelochurch 12y ago

I'm really happy to hear about Standard Chartered's success with this, and I want to know more. This is really promising stuff. My current company is looking into our "next generation" platform for when our datasets exceed what we can do in R. R may not be the best language, but it's great for exploratory data science, has the best or the only library out there for some ML purposes, and we've done a lot of things to make it production-worthy (on small- and medium-data). We'll probably need to involve something else, down the road, when our data sets get larger than what fits on one box. The leading candidates are Haskell, Clojure, and Scala (Scala because of Spark). I'll have to evaluate the languages fairly and relative to our needs, but I hope Haskell wins for a number of reasons, including the fact that Chicago + Haskell is an unfilled niche and we'd attract a ton of talent. For those who've taken Haskell this far into production: have you encountered any negatives? Are there any times when you think it might be better not to have a strong type system? To me, the biggest drawback of Haskell isn't anything intrinsic to the language, but the amount of stuff it forces a person to learn. For me, that's a fun challenge... but trying to convince 110 programmers to use a language that forces I/O into a set of types (loosely) called a monad seems like an epic task. Clojure has the advantage of being simple and beautiful once you get past the parentheses. Haskell is demanding and frustrating for the first 6 months (and pays off handsomely later on, but this can make it a hard sell). Also, how does the Any type in Mu (if anyone familiar with it is here) differ from Data.Dynamic?

ajtulloch 12y ago
I'm sorry - I love Haskell, and it's a great choice for a lot of domains, but it's a poor choice for numeric/scientific computation. The ecosystem just isn't there yet. Here's a recent r/haskell thread discussing this: http://www.reddit.com/r/haskell/comments/2rsxrb/is_haskell_a_good_programming_language_for/ http://www.reddit.com/r/haskell/comments/2rsxrb/is_haskell_a...
tel 12y ago
Exploratory data analysis isn't killer in Haskell. I think types can help a lot here (god knows the amount of pain that R/Numpy spaghetti has inflicted) but the jury is still out for how to do it well. Row types seem like an important technology which gets a little bolted on to Haskell---both exciting and unclear at the moment. https://acowley.github.io/Frames/ https://acowley.github.io/Frames/ On the other hand, if you've got a solid data pipeline you're scaling then there are grand tools for streaming data in Haskell. You're in the complete sweet spot for strong typing. You can also drop down into C easily when speed is an issue. Matrix libraries exist for the intrepid user. HMatrix is BSD now so you can at least touch LAPACK.

saosebastiao 12y ago

I've toyed with Haskell, ultimately moving on to the Ocaml/F# camp. There are only two things that I miss from Haskell without an appropriate equivalent or easy workaround: Type Classes, and Higher Kinded Types. This big roadblock that everyone claims with Haskell, Monads, didn't give me any problems at all...even if it didn't make any sense to resort to them to do something as trivial as IO. What really turned me off more than anything was the combination of the academic focus of the community combined with this weird culture that I could only describe as a cleverness competition. I'm a pragmatist, and it is obvious that real software can be created with Haskell, but it doesn't really come through in the tutorials or books. There is a lot of "Look, this is really cool!", but rarely a follow up with "and this is why it matters!". Everything seemed to revolve around cool tricks with no practical concern behind them, and then in the comments you inevitably find comments from other Haskellers claiming to do it just a little bit more cleverly using <<lenses, GADTs, zippers, continuations, or some other obscure abstraction>>. Ultimately it was Ocaml and F# that taught me all of the really important lessons of the ML family, despite the relative lack of learning resources out there. It was there that I found the benefits of making illegal states unrepresentable, expressive pattern matching, type inference, composition over inheritance, etc. It is a shame, because as languages and runtimes, they probably are inferior to Haskell. Ocaml does fine with concurrency, but parallelism is a disaster. Its class system, its one major distinction from SML, is mostly considered a code smell. Its stance on operator overloading requires me to keep a table of operators handy, instead of the usual intuitive ones. And F# has the sane operators and parallelism story but doesn't have SML/Ocaml Functors, and is still a Windows-first ecosystem (Mono has gotten a lot better, but its still a kludge).

Ironballs 12y ago
The Windows-first status of F# is changing. There have been major improvements in this recently, most notably, the whole compiler being open sourced and its development moved to github[1], and .NET is going to be available on Linux[2]. This is not reality yet, but it will be during this year. [1] https://github.com/microsoft/visualfsharp https://github.com/microsoft/visualfsharp [2] http://www.hanselman.com/blog/AnnouncingNET2015NETasOpenSourceNETonMacandLinuxandVisualStudioCommunity.aspx http://www.hanselman.com/blog/AnnouncingNET2015NETasOpenSour...

jongraehl 12y ago

I wonder why they don't just use GHC with some customization to make the default eval strategy less lazy. I understand why someone might prefer more predictable (strict) evals by default.

tel 12y ago
Lennart Augustsson is a skilled compiler writer, especially for Haskell, to say the least. Mu does more than just strictify Haskell—the code portability stuff is pretty unique as well, for instance.
eru 12y ago
Starting from scratch, building on GHC might have made sense. There were historical reasons for going with their own compiler. (Not sure, how much I can say.) And Lennart pulled it off.
tome 12y ago
They were targeting an already existing runtime for on older in-house functional language, and the runtime only supported strict evaluation.

thu 12y ago

Lazy vs strict evaluation: Lennart, the author of the Mu compiler, has written about it: http://augustss.blogspot.co.uk/2011/05/more-points-for-lazy-evaluation-in.html http://augustss.blogspot.co.uk/2011/05/more-points-for-lazy-...

boothead 12y ago

This An awful lot of data mining and analysis is best done with relational algebra Is an interesting statement. Would anyone care to elaborate?

tome 12y ago
I think it means that a plethora operations you perform on sets, lists, maps, hash maps and indeed collections of all kinds are just specific implementations of the general "relational algebra" operations.