6 ms·
ugh first time I touched a haskell codebase it was crashing in prod and nobody knew why real 'this never happens' energy
by awinter-py 3y ago
ugh first time I touched a haskell codebase it was crashing in prod and nobody knew why
real 'this never happens' energy
- kccqzy 3y agoThere's a big difference between crashing as in segfaults and crashing as in I should have handled an exception but I chose not to. I can trigger a crash by dividing by zero and not handling it in almost any language. "Nobody knew why" could mean bad code, communication breakdown or a host of things. You can make code puzzling in any language.
- suntipa 3y agoLazy evaluation also implies that Haskell programs can crash based on their input. That doesn't imply big input, simply that the input caused a space leak the author hadn't considered. Try selling that to your boss who runs a mission-critical app.
- kccqzy 3y agoSo? I tested my Python app on a list with ten elements and it works fine. But in production I unexpectedly received a list with a million elements! It crashes! Or more likely, it repeatedly receives a short list but still runs out of memory due to a memory leak I hadn't considered. Python doesn't prevent a bad software engineer from writing bad code. Try to sell that.
- suntipa 3y agoLazy eval opens the floodgates for complex space leaks. Suppose you have a complicated algorithm that runs until it reaches mathematical convergence. In a lazy language like Haskell, those cycles of iteration/recursion can accumulate thunks and crash. Thunk accumulation and crashing can depend on input that you can't control or know in advance. How do I know this? Because our big Haskell app died this way on live customer data and became a fiasco for my employer. We dumped Haskell and never looked back.
- kccqzy 3y agoI don't doubt your experience, but in my experience judicious use of bang patterns and StrictData are enough to prevent those issues. It's no different from debugging and avoiding memory leaks in other languages. It just takes different skill sets that are more difficult to find.
- suntipa 3y agoBang and StrictData are fine locally but things get enormously worse when that data flows into deeply-layered libraries that I don't own. Then, I need to understand and modify the evaluation behavior of a deeply-layered external software stack, including its crazy type-level magic, all before a looming deadline. Seriously, how is this good for my sanity or career?
- posed 3y agoWhat’s your product if you don't mind sharing? Curious.
- suntipa 3y agoThe machinery is large-scale ML and discrete optimization over large/complex data structures, designed for a specific vertical industry. These algorithms run for hours and lazy eval was an engineering disaster. Our team had experienced Haskell developers who fought complicated space leaks night and day. 0/10 would not recommend.
- valcron1000 3y agoAnd what are you using now?
- suntipa 3y agoThe heavy numerical work is in julia, python, R, and a little custom c++ where necessary. Team productivity has gone way up.
- awinter-py 3y agoyes but haskell code is inherently puzzling and you are supposed to get safety in exchange
- nequo 3y agoWhat turned out to be the reason for the crashes in production?
- awinter-py 3y agounknown
- tome 3y agoThen it's a stretch to conclude that the problem was Haskell!