3 ms·
One Haskell resource leak I've encountered a couple of times has to do with opening large numbers of files combined with non-strict semantics. By default Haskel
by whatgoodisaroad 13y ago
One Haskell resource leak I've encountered a couple of times has to do with opening large numbers of files combined with non-strict semantics. By default Haskell will open IO handles but not consume them until the contents are needed, and thus, not close them. To read the contents of many files in a directory, the result is opening thousands of concurrent file handles and exhausting the OS's IO handle pool. The solution is to add strictness annotations to force evaluation and relinquish the handles, which isn't fun and isn't pretty.
- mightybyte 13y agoThis problem is being addressed by a number of packages like conduit, pipes, and (at a lower level) io-streams. These are second-generation solutions to the problem that was pioneered by the iteratee and enumerator packages.
- shadytrees 13y agoSeconding. I've used conduit before, and it was a delight to use something so carefully designed. The blog posts about conduit are in themselves an insight into how to think with Haskell. http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit http://www.yesodweb.com/blog/2013/10/simpler-conduit-core http://www.yesodweb.com/blog/2013/10/simpler-conduit-core
- jfischoff 13y agoThe library in question was using unsafePerformIO to open a file handle. It was just a bug.
- dllthomas 13y agoThere's apparently a particular instance you're referring to? But the general issue can be encountered with lazy IO without any use of unsafePerformIO. There has been a lot of discussion about this around the various enumerator-like libraries - in particular, Snoyman has many posts about ensuring timely release of resources.