6 ms·

What I still do not know and always wished to know when I learned Haskell was how to write efficient code easily. I wrote huge projects with thousands of lines

What I still do not know and always wished to know when I learned Haskell was how to write efficient code easily. I wrote huge projects with thousands of lines knowing nothing about C++ execution model and had insanely fast code (that could have been made even faster - but I was not that kind of expert) but with Haskell I have to be an expert to really write code that is as performant as something that would take me much less time to write in C/C++. Just writing simple efficient matrix multiplication is a pain. It took me a couple of days to write a working quicksort. I couldn't find any resources that provide a very serious introduction to optimizing Haskell code. I found out way too late in my adventures with Haskell that Monad Transformers and similar abstractions have a significant runtime overhead and aren't free, I thought I was just playing with types that won't get in the way when the code compiles. No one seems to cover this aspect of Haskell.

coolsunglasses 11y ago
Don't do something that generates thunks in a long-running loop which'll hold references to them. (This applies to JS...virtually any language with GC and first-class functions as well). This is pretty easy to fix and identify once you know what you're doing. Don't use String. Use Text for Text, ByteString for raw bytes. Related to String: don't use lists for large datasets unless you're intentionally modeling and reasoning about your data as an infinite space. Use Vector if it's finite, fits in memory, and you're going to comprehend it all at once. Vector gives you cache-friendliness as well. As a safe default: lazy in the spine, strict in the leaves. Use streaming libraries (Pipes, Conduit) for processing large datasets. I did a test with a csv parsing library and compared the non-streaming (default) interface and the streaming interface provided by pipes-csv for summing columns in a ~6mb CSV. No streaming used 30mb of heap. Naive streaming used 10mb. Pipes used 600kb. monad-control and mtl (monad transformer libraries) are pretty fast and can be used without much (any?) worry in 99.999% of circumstances. Use attoparsec and be mindful of backtracking anytime you're doing perf sensitive parsing. If you hit a limit, you may need to use a parser generator but this is almost never the case. If you need to parse something huge...use a streaming parser. Sometimes CPS transformation can be a huge boon. Read what Bryan O'Sullivan was written about this for the parsing side of it. Kmett has good examples of CPS transformation for performance in his libraries as well. Use async. Default to using TVars (STM containers) for correctness until you know what specific properties you need. If you get perf sensitive but still want transactions, turn the scope of your transactions into cells of a data structure. Cf. http://hackage.haskell.org/package/stm-containers http://hackage.haskell.org/package/stm-containers when you have composable concurrency abstractions, it can become just yet another data structures problem. If you're writing a (very) hot loop, you'll probably end up looking to avoid boxing (just like Java) and touching the heap (just like Java, C, C++). Don Stewart has written good, thorough examples of this. Resources: Anything Don Stewart has ever written Kmett's libraries and blog Bryan O'Sullivan's libraries (particularly attoparsec and aeson) and blog Johan Tibell is the strictness perf honcho. Check his libraries for ideas if you're making something strict. Don Stewart has written along these lines too. http://book.realworldhaskell.org/ http://book.realworldhaskell.org/ is out of date but the stuff on perf and debugging are some of the best in the book. My own book http://haskellbook.com/ http://haskellbook.com/ will explain how to reason about performance and laziness, though it's primarily a practical beginner's book. Focus WRT perf will be more on understanding the foundations, how things evaluate, how the runtime works so you can absorb all the other information as easily as possible. Koalafications: Every backend I work on in ad tech, including the public facing adserver, is written in Haskell. Our adserver latencies range 12-17ms with some pretty gentle 99th percentiles. The average DSP we talk to has response times in the low hundreds (100-400).
- agumonkey 11y ago
  How out of date is RWH ?
  - coolsunglasses 11y ago
    Depends on the chapter and how upset you get when you need to tweak something to make it work :) Mostly the stuff touching libraries will break but if you know where to look on Hackage you can fix it.
  - psibi 11y ago
    This answer[0] explains various parts of RWH which is outdated. [0] http://stackoverflow.com/a/23733494/1651941 http://stackoverflow.com/a/23733494/1651941
- imakesnowflakes 11y ago
  Apologies, if this is a bit off topic. I am a freelance computer programmer from India. I have been teaching Haskell myself for the past 3-4 months. I have stopped for a while, because I am hard pressed to find a job where I can work remotely. Can you look at some code I have written and let me know if I can put Haskell as something I can put in my resume? If yes, what is the best way to put it. Here is the code. 1. A simple neural network with back propogation algorithm that recognizes alphabets letters on a 8x8 grid. https://bitbucket.org/sras/haskell-stuff/src/default/snn.hs?fileviewer=file-view-default https://bitbucket.org/sras/haskell-stuff/src/default/snn.hs?... 2. A port (partial) of my side project web application in PHP/MySQL to Haskell using Spock framework. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee?at=default https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... What are my chances of getting a remote Haskell job? Based on my experience, What kind of position/pay can I expect to get? If it matters, I have 9 years of professional experience in developing web applications.
  - coolsunglasses 11y ago
    I don't really have time to do a proper code review, but here are some thoughts from glancing. It's petty, but the visual appearance of your code will affect how a client evaluates your knowledge of Haskell. You could use stylish-haskell and hindent to help clean things up. We use a (tiny) Pipes project to tech screen candidates because it's usually stuff like Pipes/Conduit that'll make somebody thrash - not straight-forward stuff in IO. Ability to overcome or preexisting experience is what we're looking for. That doesn't necessarily apply to other companies hiring/contracting with Haskellers of course. I don't know what your focus is, but Haskell isn't _too_ much different from other PLs in that a lot of the work is web, so I'd recommend getting comfortable in Yesod & Snap. Nothing wrong with Spock/Scotty, but a lot of apps will be written in them. There is more numerical work to be had, but if you just want to get work that is Haskell, whatever that may be, that's where you'll want to get comfy. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Domain.hs?at=default&fileviewer=file-view-default#Domain.hs-56:64 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... looks Maybe Monad'ish. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Domain.hs?at=default&fileviewer=file-view-default#Domain.hs-80:88 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... you can kill duplication like this by having a getOne variant of your query function. Many db clients provide this. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Types.hs?at=default&fileviewer=file-view-default https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... this is what I mean by petty cleanup. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Domain.hs?at=default&fileviewer=file-view-default#Domain.hs-37 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... Use a quasiquoter so you can write the query naturally, but inline in the code. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Utils.hs?at=default&fileviewer=file-view-default#Utils.hs-45 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... this whole datatype is kinda suspect and you need newtypes for the Text soup. See Bloodhound below for how I use newtypes to make the types more useful. If you'd like example code, my Elasticsearch client is pretty reasonable I think: https://github.com/bitemyapp/bloodhound https://github.com/bitemyapp/bloodhound The projects you've linked look non-trivial and pretty cool, just needs refinement that comes from experience and improvement. Beyond that, seek out sticking points that trip up beginning/intermediate Haskell programmers like getting comfortable in mature web frameworks, with monad transformers, and with a streaming library or two. (Pipes and Conduit) I'd recommend contributing to an existing library as well, if you have the time. I can't guarantee you'll find a Haskell gig if you do all this. If you're serious about finding one, your best bets are consulting with one that already uses Haskell (start keeping a spreadsheet of companies using Haskell and whom you've contacted) or join an early stage company open to transition. I don't recommend the latter until you are _very_ comfortable in Haskell and can teach it. I hope this helped, good luck! Please ping me at my email (it's on my github) if you have further questions but keep in mind the book has me very busy so I might be slow to reply.

coolsunglasses 11y ago

Don't do something that generates thunks in a long-running loop which'll hold references to them. (This applies to JS...virtually any language with GC and first-class functions as well). This is pretty easy to fix and identify once you know what you're doing. Don't use String. Use Text for Text, ByteString for raw bytes. Related to String: don't use lists for large datasets unless you're intentionally modeling and reasoning about your data as an infinite space. Use Vector if it's finite, fits in memory, and you're going to comprehend it all at once. Vector gives you cache-friendliness as well. As a safe default: lazy in the spine, strict in the leaves. Use streaming libraries (Pipes, Conduit) for processing large datasets. I did a test with a csv parsing library and compared the non-streaming (default) interface and the streaming interface provided by pipes-csv for summing columns in a ~6mb CSV. No streaming used 30mb of heap. Naive streaming used 10mb. Pipes used 600kb. monad-control and mtl (monad transformer libraries) are pretty fast and can be used without much (any?) worry in 99.999% of circumstances. Use attoparsec and be mindful of backtracking anytime you're doing perf sensitive parsing. If you hit a limit, you may need to use a parser generator but this is almost never the case. If you need to parse something huge...use a streaming parser. Sometimes CPS transformation can be a huge boon. Read what Bryan O'Sullivan was written about this for the parsing side of it. Kmett has good examples of CPS transformation for performance in his libraries as well. Use async. Default to using TVars (STM containers) for correctness until you know what specific properties you need. If you get perf sensitive but still want transactions, turn the scope of your transactions into cells of a data structure. Cf. http://hackage.haskell.org/package/stm-containers http://hackage.haskell.org/package/stm-containers when you have composable concurrency abstractions, it can become just yet another data structures problem. If you're writing a (very) hot loop, you'll probably end up looking to avoid boxing (just like Java) and touching the heap (just like Java, C, C++). Don Stewart has written good, thorough examples of this. Resources: Anything Don Stewart has ever written Kmett's libraries and blog Bryan O'Sullivan's libraries (particularly attoparsec and aeson) and blog Johan Tibell is the strictness perf honcho. Check his libraries for ideas if you're making something strict. Don Stewart has written along these lines too. http://book.realworldhaskell.org/ http://book.realworldhaskell.org/ is out of date but the stuff on perf and debugging are some of the best in the book. My own book http://haskellbook.com/ http://haskellbook.com/ will explain how to reason about performance and laziness, though it's primarily a practical beginner's book. Focus WRT perf will be more on understanding the foundations, how things evaluate, how the runtime works so you can absorb all the other information as easily as possible. Koalafications: Every backend I work on in ad tech, including the public facing adserver, is written in Haskell. Our adserver latencies range 12-17ms with some pretty gentle 99th percentiles. The average DSP we talk to has response times in the low hundreds (100-400).

agumonkey 11y ago
How out of date is RWH ?
- coolsunglasses 11y ago
  Depends on the chapter and how upset you get when you need to tweak something to make it work :) Mostly the stuff touching libraries will break but if you know where to look on Hackage you can fix it.
- psibi 11y ago
  This answer[0] explains various parts of RWH which is outdated. [0] http://stackoverflow.com/a/23733494/1651941 http://stackoverflow.com/a/23733494/1651941
imakesnowflakes 11y ago
Apologies, if this is a bit off topic. I am a freelance computer programmer from India. I have been teaching Haskell myself for the past 3-4 months. I have stopped for a while, because I am hard pressed to find a job where I can work remotely. Can you look at some code I have written and let me know if I can put Haskell as something I can put in my resume? If yes, what is the best way to put it. Here is the code. 1. A simple neural network with back propogation algorithm that recognizes alphabets letters on a 8x8 grid. https://bitbucket.org/sras/haskell-stuff/src/default/snn.hs?fileviewer=file-view-default https://bitbucket.org/sras/haskell-stuff/src/default/snn.hs?... 2. A port (partial) of my side project web application in PHP/MySQL to Haskell using Spock framework. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee?at=default https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... What are my chances of getting a remote Haskell job? Based on my experience, What kind of position/pay can I expect to get? If it matters, I have 9 years of professional experience in developing web applications.
- coolsunglasses 11y ago
  I don't really have time to do a proper code review, but here are some thoughts from glancing. It's petty, but the visual appearance of your code will affect how a client evaluates your knowledge of Haskell. You could use stylish-haskell and hindent to help clean things up. We use a (tiny) Pipes project to tech screen candidates because it's usually stuff like Pipes/Conduit that'll make somebody thrash - not straight-forward stuff in IO. Ability to overcome or preexisting experience is what we're looking for. That doesn't necessarily apply to other companies hiring/contracting with Haskellers of course. I don't know what your focus is, but Haskell isn't _too_ much different from other PLs in that a lot of the work is web, so I'd recommend getting comfortable in Yesod & Snap. Nothing wrong with Spock/Scotty, but a lot of apps will be written in them. There is more numerical work to be had, but if you just want to get work that is Haskell, whatever that may be, that's where you'll want to get comfy. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Domain.hs?at=default&fileviewer=file-view-default#Domain.hs-56:64 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... looks Maybe Monad'ish. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Domain.hs?at=default&fileviewer=file-view-default#Domain.hs-80:88 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... you can kill duplication like this by having a getOne variant of your query function. Many db clients provide this. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Types.hs?at=default&fileviewer=file-view-default https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... this is what I mean by petty cleanup. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Domain.hs?at=default&fileviewer=file-view-default#Domain.hs-37 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... Use a quasiquoter so you can write the query naturally, but inline in the code. https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc30f95d7be9afc58ec061aee/Babloos/Utils.hs?at=default&fileviewer=file-view-default#Utils.hs-45 https://bitbucket.org/sras/babloos-hs/src/06bffa288865af1fc3... this whole datatype is kinda suspect and you need newtypes for the Text soup. See Bloodhound below for how I use newtypes to make the types more useful. If you'd like example code, my Elasticsearch client is pretty reasonable I think: https://github.com/bitemyapp/bloodhound https://github.com/bitemyapp/bloodhound The projects you've linked look non-trivial and pretty cool, just needs refinement that comes from experience and improvement. Beyond that, seek out sticking points that trip up beginning/intermediate Haskell programmers like getting comfortable in mature web frameworks, with monad transformers, and with a streaming library or two. (Pipes and Conduit) I'd recommend contributing to an existing library as well, if you have the time. I can't guarantee you'll find a Haskell gig if you do all this. If you're serious about finding one, your best bets are consulting with one that already uses Haskell (start keeping a spreadsheet of companies using Haskell and whom you've contacted) or join an early stage company open to transition. I don't recommend the latter until you are _very_ comfortable in Haskell and can teach it. I hope this helped, good luck! Please ping me at my email (it's on my github) if you have further questions but keep in mind the book has me very busy so I might be slow to reply.