5 ms·
10x reflects both experience and expert recommendations. You may recognize the author [1]: > Nowadays, my rule of thumb for pandas is that you should have 5 to
by ihnorton 6y ago
10x reflects both experience and expert recommendations. You may recognize the author [1]:
> Nowadays, my rule of thumb for pandas is that you should have 5 to 10 times as much RAM as the size of your dataset
[1] https://wesmckinney.com/blog/apache-arrow-pandas-internals/ https://wesmckinney.com/blog/apache-arrow-pandas-internals/
- willseth 6y agoI don't doubt Wes' upper bound for Pandas OOTB, without optimization. The context was web applications. If you're seeing 10x on a web app, either something is wrong or you probably shouldn't be using Pandas.
- mlyle 6y agoI think most of us use Pandas for data exploration and one-offs, so it is completely reasonable to discuss what our likely use of RAM is going to be in this circumstance. Web application using Pandas and "highly optimized web application" would seem to be nearly disjoint sets...