4 ms·
Lazy eval opens the floodgates for complex space leaks. Suppose you have a complicated algorithm that runs until it reaches mathematical convergence. In a lazy
by suntipa 3y ago
Lazy eval opens the floodgates for complex space leaks. Suppose you have a complicated algorithm that runs until it reaches mathematical convergence. In a lazy language like Haskell, those cycles of iteration/recursion can accumulate thunks and crash. Thunk accumulation and crashing can depend on input that you can't control or know in advance.
How do I know this? Because our big Haskell app died this way on live customer data and became a fiasco for my employer. We dumped Haskell and never looked back.
- kccqzy 3y agoI don't doubt your experience, but in my experience judicious use of bang patterns and StrictData are enough to prevent those issues. It's no different from debugging and avoiding memory leaks in other languages. It just takes different skill sets that are more difficult to find.
- suntipa 3y agoBang and StrictData are fine locally but things get enormously worse when that data flows into deeply-layered libraries that I don't own. Then, I need to understand and modify the evaluation behavior of a deeply-layered external software stack, including its crazy type-level magic, all before a looming deadline. Seriously, how is this good for my sanity or career?
- posed 3y agoWhat’s your product if you don't mind sharing? Curious.
- suntipa 3y agoThe machinery is large-scale ML and discrete optimization over large/complex data structures, designed for a specific vertical industry. These algorithms run for hours and lazy eval was an engineering disaster. Our team had experienced Haskell developers who fought complicated space leaks night and day. 0/10 would not recommend.
- valcron1000 3y agoAnd what are you using now?
- suntipa 3y agoThe heavy numerical work is in julia, python, R, and a little custom c++ where necessary. Team productivity has gone way up.