16 ms·

Haskell in Production: 12 Months Building ApiEngine

136 points by lelf 13y ago

brotchie 13y ago
Awesome. Is there a video recording of your presentation anywhere online? What strategies did you use to combat the lack of stack traces, especially in production? I was happy to see that you are fellow Brisbanites. I'm a bit out of the "local loop", is there much of a local Brisbane start-up scene?
- alberich 13y ago
  I think i'm missing something. On slide 4 it says (the code) it has 0 exceptions and is robust, but then on slide 7 it says that (the system) it crashes without stack traces? Is that right? What did the OP meant by being robust, when the code crashes unexpectedly?
  - Guvante 13y ago
    Likely that is referring to the fact that when it crashes, it crashes in a way that can be interesting to debug. I took it as a comment about development vs one about production.
    - emilv 13y ago
      "can be interesting to debug" sounds scary :-)
      - dons 13y ago
        Lazy languages don't have equivalent call stacks as strict languages. However, GHC lets you get a stack trace via profiling. E.g. +RTS -xc flag gives a stack trace.
        enigmo 13y ago
        The actual evaluation stack (re: GHC w/ DWARF symbols) is also quite useful for debugging and doesn't need a profiled executable to work.
  - alipang 13y ago
    In Haskell one typically eschews exceptions in favor of for instance the Error monad, as you don't get a typical stack trace in a lazily evaluated language. Probably they have experienced some exception during development, but have had no exeptions in productin.
  - bermanoid 13y ago
    The cynic in me says that unless Haskell has a magical "do the right thing in every situation, with automatic full knowledge of your application's business requirements" operator, if there's no specific logic in the code that responds to errors, they're just ignoring them (or else they're building a very bare bones app - any nontrivial piece of software will have potential error conditions out of its control that it has to react to in very specific ways that depend on the app itself). In which case I could produce an equally "robust" Java program by wrapping everything in a try/catch and not logging anything. Except that in Java, or just about any other language, with a single method call I could obtain that elusive stack trace that OP is sorely missing. Which, in real applications (which, to be fair, I've never been masochistic enough to write in Haskell), I always do, and it significantly improves our ability to track down the remaining unanticipated conditions that our error handling logic hasn't already handled in well defined ways. And that sort of work should consume most of anyone's time spent developing, because the happy path is the easiest piece of the puzzle - engineers handle all code paths, hackers handle the most important ones, and hacks only handle the best case scenarios. I'm not really getting what Haskell has helped here, apart from making the article more upvote-able. That said, the presentation slides don't give me even a vague sense what the real thrust of the presentation was, so I'm probably missing the substance. I take issue with this being posted without any additional context, but I'll grant the benefit of the doubt and assume that there may actually have been something worth listening to if we didn't just have the slides to look at.
    - DanWaterworth 13y ago
      Haskell doesn't need a magical "do the right thing" operator. What it does instead is constantly nag you, saying, "but you haven't considered this case". This causes you to think about you code in much more depth and generally leads to good results without the need to constantly debug. I find that things written in Haskell that compile generally work first time more often than the law of averages would seem to allow.
    - tel 13y ago
      Exception in Haskell is more specific than in other languages. There are many ways to have "unexpected circumstances" such as Maybe, Either, Error, Mplus, Alternative, and the transformer variants thereof. Exceptions refer explicitly to "asynchronous" exceptions—the kind in most other languages, that imply global changes in control flow that are difficult to reason about. People tend to avoid this behavior except in rare circumstances. That said, it's easy to replicate locally with the Cont monad. These local reflows are easier to reason about as well. You can even build it atop delimited continuations for more control. These are great for covering early stopping in searches, for instance.
  - gtani 13y ago
    I think s7 was emphasizing issues getting what strict/eagerly evaluated languages refer to as stacktraces http://hackage.haskell.org/trac/ghc/ticket/3693 http://hackage.haskell.org/trac/ghc/ticket/3693 http://www.haskell.org/wikiupload/6/6c/Hiw2012-simon-marlow.pdf http://www.haskell.org/wikiupload/6/6c/Hiw2012-simon-marlow....
    - gtani 13y ago
      [ed] oops, nevermind, see dons' post above re:"+RTS -xc", probably shd mark the trac ticket superceded or something
tome 13y ago
Reddit discussion here: http://www.reddit.com/r/haskell/comments/1etssw/haskell_in_production_slides/ http://www.reddit.com/r/haskell/comments/1etssw/haskell_in_p...
laumars 13y ago
I appreciate the need some people feel to make modern-looking web pages, but the format of this really is just a triumph of style of substance: * I have no idea how far through the content I am * The lack of verbosity means I'm constantly having to apply guesswork to their statements (ie an explained point will tell the readers why Haskell gave them better communication. A slide show like that with just headings mean I have to guess the reason Haskell had those benefits) * It's scales terribly (at least on Opera, some of the content fell off screen - so I had to maximize the browser. But this might just have been an Opera bug. In either case, this wouldn't have been a problem with a more traditional blog layout). * The navigation requires guesswork * And most annoying of all, it doesn't actually save the reader any time by displaying the content like that than it would have done with blog post as the jarring navigation presented with slides require the reader to pause before each page to understand how to read the slide, and then pause after each page to apply the aforementioned guesswork. With a blog post, the information is presented in a more natural communicative way which makes it quicker to digest the information. It's a great pity I'm having to pick post a negative comment about the presentation of the submission rather than comment about the point being raised - particularly as I've been very interested in Haskell lately. But I found even my interest in the subject wasn't enough for me to put up with this slide show layout and thus I ended up closing the browser tab before the end. :(
- aidos 13y ago
  In terms of how far through the content - there seems to be a little marker (in yellow) that runs along the bottom of the screen. I suspect, depending on the browser, YMMV.
  - laumars 13y ago
    That may have been chopped of by Opera then. Thanks for the correction though :)
- tome 13y ago
  It didn't work at all on my (admittedly old) version of Firefox. I suspect the author made the slideshow to display on his/her own system, and generously shared it publically as an afterthought.
sejje 13y ago
Why wouldn't the landing page say "use arrows to navigate?" I spent a good little while trying to figure out what I was missing.
- kbenson 13y ago
  Took me a second until I noticed that the scroll wheel seemed to take me to a different page. As soon as I determined it was a slideshow, I tried the arrow keys, as they work on every HTML slideshow I've seen in the last 5 years.
- verroq 13y ago
  Scroll wheel works.
stonemetal 13y ago
Does anyone have some insight to what is going on in the Z monad bit? It looks like an Either monad with a bunch of error states instead of just the one.
- boothead 13y ago
  Z isn't a monad from what I can see... ZZ is the monad. It's an IO action that returns a ZResult (which is either a value or an error state as you said) and has access to an environment for logging, a database connection and a configuration. I'm not a Haskell expert though - so don't take that as truth until confirmed by someone else :-)
- tel 13y ago
  There are two independent parts. The Z monad is nothing more than a ReaderT monad giving access to some config and environment information. Since it's a transformer they can execute actions in higher monads (like IO) pretty easily from Z, but it can also be used in pure code if you just need the configuration information. Parallel to that is the ZResultT monad. It's built on the ZResult monad (it should be easy enough to write a Monad instance for it anyway) which looks like an Identity monad with some "Z-specific" error handling layered in. ZResultT just lets you have a transformer stack interleaved with the ZResults. Then you combine those two stacks in ZZ which gives you both "Z-related" configuration and "Z-related" error states. It looks like the whole thing is specialized to make it easy to build functions pure functions which only need a handful of the effects and then weave them together into the ZZ monad which is likely to be run at the top level, right below Yesod.
hcarvalhoalves 13y ago
The fact I read this and have absolutely no idea about what's going on makes me compelled to learn more about Haskell. Is it really that alien (I know Scheme)?
- chc 13y ago
  It's pretty weird. It's a very strongly statically typed lazily evaluated pure functional language with a type system that is actually Turing-complete.
  - njs12345 13y ago
    The type system isn't Turing-complete by default, you need to enable a fair number of extensions: http://www.haskell.org/haskellwiki/Type_SK http://www.haskell.org/haskellwiki/Type_SK
- njs12345 13y ago
  Haskell is definitely pretty far away from almost any other language (I can't think of any others which are lazy by default, for example). Learn You a Haskell is probably the best resource if you want to take a look, although it seems to be down right now: http://learnyouahaskell.com/ http://learnyouahaskell.com/
- gtani 13y ago
  Not alien, but not like ordering a sausage mcmuffin for breakfast. You'll recognize ideas taken from haskell (list comprehensions, especially) in other languages but the function definitions (multiple pattern matching clauses/"equations"), may be alien, unless you've looked at erlang, F#, ocaml, scala or some such. You could look at these to get why http://www.haskell.org/pipermail/haskell-cafe/2012-June/101558.html http://www.haskell.org/pipermail/haskell-cafe/2012-June/1015... https://www.fpcomplete.com/page/case-studies https://www.fpcomplete.com/page/case-studies http://elaforge.blogspot.com/2011/07/what-haskell-doesnt-have.html http://elaforge.blogspot.com/2011/07/what-haskell-doesnt-hav... http://haskell.cs.yale.edu/?post_type=publication&p=112 http://haskell.cs.yale.edu/?post_type=publication&p=112 http://www.jerf.org/iri/post/2908 http://www.jerf.org/iri/post/2908 (or a few dozen tutorials: http://haskell.org/haskellwiki/Learning_Haskell http://haskell.org/haskellwiki/Learning_Haskell ----------- here's my attempt at organizing the learning process (having lots of places to ask questions (SO, reddit, beginners list, G+) is great http://www.reddit.com/r/haskell/comments/1b7axp/haskell_homework_helptutor_wanted/c94hvr1 http://www.reddit.com/r/haskell/comments/1b7axp/haskell_home...
salimmadjd 13y ago
Not only I have no clue about the slides, but I went to their site (apiengine) and I have no idea what their site does. Interestingly, both the slides and their site (apiengine) has clean and beautiful designs, however neither one of them are a bit informative.
- DevMonkey 13y ago
  He He... Their site is hard to read and understand and so is Haskell at first glance.

136 points by lelf 13y ago

Awesome. Is there a video recording of your presentation anywhere online? What strategies did you use to combat the lack of stack traces, especially in production? I was happy to see that you are fellow Brisbanites. I'm a bit out of the "local loop", is there much of a local Brisbane start-up scene?

alberich 13y ago
I think i'm missing something. On slide 4 it says (the code) it has 0 exceptions and is robust, but then on slide 7 it says that (the system) it crashes without stack traces? Is that right? What did the OP meant by being robust, when the code crashes unexpectedly?
- Guvante 13y ago
  Likely that is referring to the fact that when it crashes, it crashes in a way that can be interesting to debug. I took it as a comment about development vs one about production.
  - emilv 13y ago
    "can be interesting to debug" sounds scary :-)
    - dons 13y ago
      Lazy languages don't have equivalent call stacks as strict languages. However, GHC lets you get a stack trace via profiling. E.g. +RTS -xc flag gives a stack trace.
      - enigmo 13y ago
        The actual evaluation stack (re: GHC w/ DWARF symbols) is also quite useful for debugging and doesn't need a profiled executable to work.
- alipang 13y ago
  In Haskell one typically eschews exceptions in favor of for instance the Error monad, as you don't get a typical stack trace in a lazily evaluated language. Probably they have experienced some exception during development, but have had no exeptions in productin.
- bermanoid 13y ago
  The cynic in me says that unless Haskell has a magical "do the right thing in every situation, with automatic full knowledge of your application's business requirements" operator, if there's no specific logic in the code that responds to errors, they're just ignoring them (or else they're building a very bare bones app - any nontrivial piece of software will have potential error conditions out of its control that it has to react to in very specific ways that depend on the app itself). In which case I could produce an equally "robust" Java program by wrapping everything in a try/catch and not logging anything. Except that in Java, or just about any other language, with a single method call I could obtain that elusive stack trace that OP is sorely missing. Which, in real applications (which, to be fair, I've never been masochistic enough to write in Haskell), I always do, and it significantly improves our ability to track down the remaining unanticipated conditions that our error handling logic hasn't already handled in well defined ways. And that sort of work should consume most of anyone's time spent developing, because the happy path is the easiest piece of the puzzle - engineers handle all code paths, hackers handle the most important ones, and hacks only handle the best case scenarios. I'm not really getting what Haskell has helped here, apart from making the article more upvote-able. That said, the presentation slides don't give me even a vague sense what the real thrust of the presentation was, so I'm probably missing the substance. I take issue with this being posted without any additional context, but I'll grant the benefit of the doubt and assume that there may actually have been something worth listening to if we didn't just have the slides to look at.
  - DanWaterworth 13y ago
    Haskell doesn't need a magical "do the right thing" operator. What it does instead is constantly nag you, saying, "but you haven't considered this case". This causes you to think about you code in much more depth and generally leads to good results without the need to constantly debug. I find that things written in Haskell that compile generally work first time more often than the law of averages would seem to allow.
  - tel 13y ago
    Exception in Haskell is more specific than in other languages. There are many ways to have "unexpected circumstances" such as Maybe, Either, Error, Mplus, Alternative, and the transformer variants thereof. Exceptions refer explicitly to "asynchronous" exceptions—the kind in most other languages, that imply global changes in control flow that are difficult to reason about. People tend to avoid this behavior except in rare circumstances. That said, it's easy to replicate locally with the Cont monad. These local reflows are easier to reason about as well. You can even build it atop delimited continuations for more control. These are great for covering early stopping in searches, for instance.
- gtani 13y ago
  I think s7 was emphasizing issues getting what strict/eagerly evaluated languages refer to as stacktraces http://hackage.haskell.org/trac/ghc/ticket/3693 http://hackage.haskell.org/trac/ghc/ticket/3693 http://www.haskell.org/wikiupload/6/6c/Hiw2012-simon-marlow.pdf http://www.haskell.org/wikiupload/6/6c/Hiw2012-simon-marlow....
  - gtani 13y ago
    [ed] oops, nevermind, see dons' post above re:"+RTS -xc", probably shd mark the trac ticket superceded or something

tome 13y ago

Reddit discussion here: http://www.reddit.com/r/haskell/comments/1etssw/haskell_in_production_slides/ http://www.reddit.com/r/haskell/comments/1etssw/haskell_in_p...

laumars 13y ago

I appreciate the need some people feel to make modern-looking web pages, but the format of this really is just a triumph of style of substance: * I have no idea how far through the content I am * The lack of verbosity means I'm constantly having to apply guesswork to their statements (ie an explained point will tell the readers why Haskell gave them better communication. A slide show like that with just headings mean I have to guess the reason Haskell had those benefits) * It's scales terribly (at least on Opera, some of the content fell off screen - so I had to maximize the browser. But this might just have been an Opera bug. In either case, this wouldn't have been a problem with a more traditional blog layout). * The navigation requires guesswork * And most annoying of all, it doesn't actually save the reader any time by displaying the content like that than it would have done with blog post as the jarring navigation presented with slides require the reader to pause before each page to understand how to read the slide, and then pause after each page to apply the aforementioned guesswork. With a blog post, the information is presented in a more natural communicative way which makes it quicker to digest the information. It's a great pity I'm having to pick post a negative comment about the presentation of the submission rather than comment about the point being raised - particularly as I've been very interested in Haskell lately. But I found even my interest in the subject wasn't enough for me to put up with this slide show layout and thus I ended up closing the browser tab before the end. :(

aidos 13y ago
In terms of how far through the content - there seems to be a little marker (in yellow) that runs along the bottom of the screen. I suspect, depending on the browser, YMMV.
- laumars 13y ago
  That may have been chopped of by Opera then. Thanks for the correction though :)
tome 13y ago
It didn't work at all on my (admittedly old) version of Firefox. I suspect the author made the slideshow to display on his/her own system, and generously shared it publically as an afterthought.

sejje 13y ago

Why wouldn't the landing page say "use arrows to navigate?" I spent a good little while trying to figure out what I was missing.

kbenson 13y ago
Took me a second until I noticed that the scroll wheel seemed to take me to a different page. As soon as I determined it was a slideshow, I tried the arrow keys, as they work on every HTML slideshow I've seen in the last 5 years.
verroq 13y ago
Scroll wheel works.

stonemetal 13y ago

Does anyone have some insight to what is going on in the Z monad bit? It looks like an Either monad with a bunch of error states instead of just the one.

boothead 13y ago
Z isn't a monad from what I can see... ZZ is the monad. It's an IO action that returns a ZResult (which is either a value or an error state as you said) and has access to an environment for logging, a database connection and a configuration. I'm not a Haskell expert though - so don't take that as truth until confirmed by someone else :-)
tel 13y ago
There are two independent parts. The Z monad is nothing more than a ReaderT monad giving access to some config and environment information. Since it's a transformer they can execute actions in higher monads (like IO) pretty easily from Z, but it can also be used in pure code if you just need the configuration information. Parallel to that is the ZResultT monad. It's built on the ZResult monad (it should be easy enough to write a Monad instance for it anyway) which looks like an Identity monad with some "Z-specific" error handling layered in. ZResultT just lets you have a transformer stack interleaved with the ZResults. Then you combine those two stacks in ZZ which gives you both "Z-related" configuration and "Z-related" error states. It looks like the whole thing is specialized to make it easy to build functions pure functions which only need a handful of the effects and then weave them together into the ZZ monad which is likely to be run at the top level, right below Yesod.

hcarvalhoalves 13y ago

The fact I read this and have absolutely no idea about what's going on makes me compelled to learn more about Haskell. Is it really that alien (I know Scheme)?

chc 13y ago
It's pretty weird. It's a very strongly statically typed lazily evaluated pure functional language with a type system that is actually Turing-complete.
- njs12345 13y ago
  The type system isn't Turing-complete by default, you need to enable a fair number of extensions: http://www.haskell.org/haskellwiki/Type_SK http://www.haskell.org/haskellwiki/Type_SK
njs12345 13y ago
Haskell is definitely pretty far away from almost any other language (I can't think of any others which are lazy by default, for example). Learn You a Haskell is probably the best resource if you want to take a look, although it seems to be down right now: http://learnyouahaskell.com/ http://learnyouahaskell.com/
gtani 13y ago
Not alien, but not like ordering a sausage mcmuffin for breakfast. You'll recognize ideas taken from haskell (list comprehensions, especially) in other languages but the function definitions (multiple pattern matching clauses/"equations"), may be alien, unless you've looked at erlang, F#, ocaml, scala or some such. You could look at these to get why http://www.haskell.org/pipermail/haskell-cafe/2012-June/101558.html http://www.haskell.org/pipermail/haskell-cafe/2012-June/1015... https://www.fpcomplete.com/page/case-studies https://www.fpcomplete.com/page/case-studies http://elaforge.blogspot.com/2011/07/what-haskell-doesnt-have.html http://elaforge.blogspot.com/2011/07/what-haskell-doesnt-hav... http://haskell.cs.yale.edu/?post_type=publication&p=112 http://haskell.cs.yale.edu/?post_type=publication&p=112 http://www.jerf.org/iri/post/2908 http://www.jerf.org/iri/post/2908 (or a few dozen tutorials: http://haskell.org/haskellwiki/Learning_Haskell http://haskell.org/haskellwiki/Learning_Haskell ----------- here's my attempt at organizing the learning process (having lots of places to ask questions (SO, reddit, beginners list, G+) is great http://www.reddit.com/r/haskell/comments/1b7axp/haskell_homework_helptutor_wanted/c94hvr1 http://www.reddit.com/r/haskell/comments/1b7axp/haskell_home...

salimmadjd 13y ago

Not only I have no clue about the slides, but I went to their site (apiengine) and I have no idea what their site does. Interestingly, both the slides and their site (apiengine) has clean and beautiful designs, however neither one of them are a bit informative.

DevMonkey 13y ago
He He... Their site is hard to read and understand and so is Haskell at first glance.