3 ms·

I think it's great people like data.table's performance but I don't think performance is a compelling reason to be prescriptive about data.table. It's a reason

I think it's great people like data.table's performance but I don't think performance is a compelling reason to be prescriptive about data.table. It's a reason data.table may be appealing to some. People can try them and decide for themselves if the performance is worth the trouble. In many cases the performance difference with dplyr is hardly noticeable and IMO readability of dplyr becomes the more meaningful advantage. I have plenty of projects with tables with millions of rows, and I rarely find dplyr to be slow enough to get frustrated about. And even then I'd rather endure, or maybe use dtplyr, or maybe use data.table for a small performance-critical part of the code, than make a wholesale switch.

nojito 4y ago
Performance is probably the last reason I use datatable. I just enjoy having to read much less code when compared to tidyverse and SQL. Plus the syntax difference between tidyverse and dt are overstated. https://atrebas.github.io/post/2019-03-03-datatable-dplyr/ https://atrebas.github.io/post/2019-03-03-datatable-dplyr/
- civilized 4y ago
  I find data.table code as obscure as base R if not more so. Purported brevity (always illustrated only on very simple problems) doesn't compensate me enough for the obscurity. I could get used to it, I'm sure. I could also get used to doing everything in base R, pandas, or Clojure. Doesn't mean it's a good use of my time. I get the sense from how some people blog that basic "filter + group by + summarize" type pipelines on giant datasets are all they really do. data.table may be a great choice for that kind of thing. But I use dplyr as a tool of thought. I can solve complex, novel data processing and analytics problems in dplyr more precisely and efficiently than I can explain in words. Instead of having an idea and then having to think about how to code it, the ideas translate into dplyr in an obvious and automatic way. All that said, data.table's authors have worked much harder to provide advanced functionality, most notably non-equijoins, that it looks like the tidyverse may never catch up to. So if you need that kind of thing a lot, you need data.table or maybe DuckDB.

nojito 4y ago

Performance is probably the last reason I use datatable. I just enjoy having to read much less code when compared to tidyverse and SQL. Plus the syntax difference between tidyverse and dt are overstated. https://atrebas.github.io/post/2019-03-03-datatable-dplyr/ https://atrebas.github.io/post/2019-03-03-datatable-dplyr/

civilized 4y ago
I find data.table code as obscure as base R if not more so. Purported brevity (always illustrated only on very simple problems) doesn't compensate me enough for the obscurity. I could get used to it, I'm sure. I could also get used to doing everything in base R, pandas, or Clojure. Doesn't mean it's a good use of my time. I get the sense from how some people blog that basic "filter + group by + summarize" type pipelines on giant datasets are all they really do. data.table may be a great choice for that kind of thing. But I use dplyr as a tool of thought. I can solve complex, novel data processing and analytics problems in dplyr more precisely and efficiently than I can explain in words. Instead of having an idea and then having to think about how to code it, the ideas translate into dplyr in an obvious and automatic way. All that said, data.table's authors have worked much harder to provide advanced functionality, most notably non-equijoins, that it looks like the tidyverse may never catch up to. So if you need that kind of thing a lot, you need data.table or maybe DuckDB.