3 ms·
Well, this is surely useful for some use cases. But for reapeated analyses that 50 min loading time is a no go and there is certainly big room for improvements.
by haddr 12y ago
Well, this is surely useful for some use cases. But for reapeated analyses that 50 min loading time is a no go and there is certainly big room for improvements.
For instance, using R with data.table to process ~4GB file would result in few minutes spent on loading and processing, while giving a very similar flexibility of REPL and scripting. There should a similar way to do it in python i guess...
- maxerickson 12y agoThe loading time is probably because of the way they are using the ORM: https://github.com/pydata/pandas/issues/8953 https://github.com/pydata/pandas/issues/8953 I guess it also isn't a great approach to do row based filtering using a data frame.
- blumkvist 12y agoAll in all, excellent showcase of the product and the team behind it.
- Lofkin 12y agohttp://blaze.pydata.org/en/latest/ http://blaze.pydata.org/en/latest/