4 ms·
Data science consulting for business things. Most datasets < 1M rows
by oneoff786 4y ago
Data science consulting for business things. Most datasets < 1M rows
- eskaytwo 4y agoThose are small datasets. Also numpy != just operations on a matrix, but as mentioned above, proper vectorization can have profound speed differences for larger datasets.
- oneoff786 4y agoThey are indeed small. But most real world problems operate on small data. And if it’s bigger you probably ought to be using a different toolset than Python altogether. In my experience it is exceedingly rare to find a situation where you have > 1M rows of data and need to do tabular data manipulation, and it’s not coming from a managed database sort of setup where the manipulation could be better done in the database.