5 ms·

Distributed Systems in Haskell

171 points by philippelh 10y ago

wyager 10y ago
Author here. Glad to see this on HN! Happy to answer any questions.
- state_less 10y ago
  Thanks for the writeup! I am wanting to learn Haskell and this inspires confidence that one can develop larger projects using a more rigorous, less error prone approach. I like it.
- domlebo70 10y ago
  This was a great post. Really love how you walked through the example code.
- cyknus 10y ago
  Why Haskell? What made you choose Haskell?
  - wyager 10y ago
    For the first project, curiosity. For subsequent projects, the desire to repeat our experience with the first project.
- geekingfrog 10y ago
  Great post. How much knowledge and experience of haskell did you have before starting this project?
  - wyager 10y ago
    I'd been dabbling with it for a year or two. Probably my most significant previous use was in my second-semester computer architecture class as a substitute for Verilog. Using Christiaan Baaij's Cλash project, one can transform a (large) subset of Haskell into VHDL/Verilog/SystemVerilog and slap it on an FPGA. There's a rundown of that project on my blog too. My project partner had never really used Haskell before, so he was learning as we went. By the end of the semester he was writing code no problem. "Immersion learning" seemed to work quite well here.
- navaati 10y ago
  Shouldn't the type of the monadic serverStep be "Message -> m ()" to match the type of the explicit "serverStep :: Config -> State -> Message -> (State, [Message])" ? Aka, serverStep isn't a monadic object, it's a Kleisli arrow.
  - wyager 10y ago
    You're right, that's a typo. I even wrote it out correctly later. I'll fix that. Thanks!
gnusouth 10y ago
Did you run into any bugs or strange beahviour in Cloud Haskell? I looked into using it a while ago and came to the (possibly hasty) conclusion that it's not stable enough to build a large project on. Thanks!
- wyager 10y ago
  None. It was rock solid; never a hiccup. It's evolving, but it seems quite well written and correct. As with many Haskell libraries in "beta", it works perfectly even though it's not finalized.
T-R 10y ago
Distributed Programming was one of the things flagged "Immature" in State of the Haskell Ecosystem [1]. Just out of curiosity, was that a motivation behind the project, or do you have any other thoughts on that post-project? [1] https://github.com/Gabriel439/post-rfc/blob/master/sotu.md#distributed-programming https://github.com/Gabriel439/post-rfc/blob/master/sotu.md#d...
- wyager 10y ago
  It was definitely a factor. When we decided to use Haskell on the first project, we were mostly just curious how viable it would be for us. If the current state of Distributed Programming in Haskell is "Immature", then I can't wait to see "Mature" :)
selmat 10y ago
Very interesting structure of explanation. I like your short sumarization (tl;dr) at beginning and "why?" reasoning. I think i will copy it in my next book :-)
boothead 10y ago
Excellent write up. One point though: watch out for WriterT and RWST leaking memory: http://stackoverflow.com/questions/25280852/space-leak-in-pipes-with-rwst http://stackoverflow.com/questions/25280852/space-leak-in-pi... It's often better to just include the writer part you need in a larder State type (lenses make this really seamless)/
- tikhonj 10y ago
  That's a good point, but it was also mentioned in the article, with a link to a useful mailing list thread[1] on the matter. It seems to be a matter of laziness. If you're writing some small piece of state like an Int, this can be a real problem. If you're writing out a lazy data structure like a list (as happens in the article), I think the extra laziness should be fine or even desired. (I've never used Writer in practice, so take my comments with a grain of salt!) [1]: http://comments.gmane.org/gmane.comp.lang.haskell.libraries/18980 http://comments.gmane.org/gmane.comp.lang.haskell.libraries/...
  - boothead 10y ago
    It was? I still can't see it... AFAIR the problem with WriterT was the use of mappend within is always lazy, regardless of the strictness of the w in WriterT w m a.
- wyager 10y ago
  Yep, I mentioned this at the bottom of "monad it".

171 points by philippelh 10y ago

wyager 10y ago

Author here. Glad to see this on HN! Happy to answer any questions.

state_less 10y ago
Thanks for the writeup! I am wanting to learn Haskell and this inspires confidence that one can develop larger projects using a more rigorous, less error prone approach. I like it.
domlebo70 10y ago
This was a great post. Really love how you walked through the example code.
cyknus 10y ago
Why Haskell? What made you choose Haskell?
- wyager 10y ago
  For the first project, curiosity. For subsequent projects, the desire to repeat our experience with the first project.
geekingfrog 10y ago
Great post. How much knowledge and experience of haskell did you have before starting this project?
- wyager 10y ago
  I'd been dabbling with it for a year or two. Probably my most significant previous use was in my second-semester computer architecture class as a substitute for Verilog. Using Christiaan Baaij's Cλash project, one can transform a (large) subset of Haskell into VHDL/Verilog/SystemVerilog and slap it on an FPGA. There's a rundown of that project on my blog too. My project partner had never really used Haskell before, so he was learning as we went. By the end of the semester he was writing code no problem. "Immersion learning" seemed to work quite well here.
navaati 10y ago
Shouldn't the type of the monadic serverStep be "Message -> m ()" to match the type of the explicit "serverStep :: Config -> State -> Message -> (State, [Message])" ? Aka, serverStep isn't a monadic object, it's a Kleisli arrow.
- wyager 10y ago
  You're right, that's a typo. I even wrote it out correctly later. I'll fix that. Thanks!

gnusouth 10y ago

Did you run into any bugs or strange beahviour in Cloud Haskell? I looked into using it a while ago and came to the (possibly hasty) conclusion that it's not stable enough to build a large project on. Thanks!

wyager 10y ago
None. It was rock solid; never a hiccup. It's evolving, but it seems quite well written and correct. As with many Haskell libraries in "beta", it works perfectly even though it's not finalized.

T-R 10y ago

Distributed Programming was one of the things flagged "Immature" in State of the Haskell Ecosystem [1]. Just out of curiosity, was that a motivation behind the project, or do you have any other thoughts on that post-project? [1] https://github.com/Gabriel439/post-rfc/blob/master/sotu.md#distributed-programming https://github.com/Gabriel439/post-rfc/blob/master/sotu.md#d...

wyager 10y ago
It was definitely a factor. When we decided to use Haskell on the first project, we were mostly just curious how viable it would be for us. If the current state of Distributed Programming in Haskell is "Immature", then I can't wait to see "Mature" :)

selmat 10y ago

Very interesting structure of explanation. I like your short sumarization (tl;dr) at beginning and "why?" reasoning. I think i will copy it in my next book :-)

boothead 10y ago

Excellent write up. One point though: watch out for WriterT and RWST leaking memory: http://stackoverflow.com/questions/25280852/space-leak-in-pipes-with-rwst http://stackoverflow.com/questions/25280852/space-leak-in-pi... It's often better to just include the writer part you need in a larder State type (lenses make this really seamless)/

tikhonj 10y ago
That's a good point, but it was also mentioned in the article, with a link to a useful mailing list thread[1] on the matter. It seems to be a matter of laziness. If you're writing some small piece of state like an Int, this can be a real problem. If you're writing out a lazy data structure like a list (as happens in the article), I think the extra laziness should be fine or even desired. (I've never used Writer in practice, so take my comments with a grain of salt!) [1]: http://comments.gmane.org/gmane.comp.lang.haskell.libraries/18980 http://comments.gmane.org/gmane.comp.lang.haskell.libraries/...
- boothead 10y ago
  It was? I still can't see it... AFAIR the problem with WriterT was the use of mappend within is always lazy, regardless of the strictness of the w in WriterT w m a.
wyager 10y ago
Yep, I mentioned this at the bottom of "monad it".