3 ms·
> never see memory issues In contrast, I see them all the time (damn you, Pandas). Beyond out-of-memory errors, generators can be vastly more compute-efficien
by xapata 6y ago
> never see memory issues
In contrast, I see them all the time (damn you, Pandas). Beyond out-of-memory errors, generators can be vastly more compute-efficient because of the CPU cache.
- Groxx 6y agoPandas I'd definitely categorize as being niche :) A moderately-sized one, but still a niche. And there you use whatever performs the way you need it, yeah - there are special constraints on the system, and pandas is built around dealing with them.
- hansvm 6y agoHaha, that one hits close to home. Iirc Pandas was the author's first big project, and memory use is one of the key things he wants to improve if he gives it another go. It's still an impressive piece of software.