3 ms·
> If it's a million items, pandas is a great solution. Possibly not even then, it depends on how much you're doing and I feel like the topic at hand might be a
by Izkata 5y ago
> If it's a million items, pandas is a great solution.
Possibly not even then, it depends on how much you're doing and I feel like the topic at hand might be around that tipping point. We have some rather slow code that, profiling it, turned out to spend something like 60-70% of its time just converting between python types and native types when moving data in and out of the dataframe.
- rbanffy 5y agoTrue. If there are millions of different “things” conversion times will end up dominating. If they are just a handful, then the libraries will be able to do a lot more work with parallel operations and converting the output will be very quick