4 ms·
It looks like both Dplython and pandas-ply are missing one of (what I think is) the core value propositions of dplyr: the ability to use the same abstractions o
by CurtHagenlocher 11y ago
It looks like both Dplython and pandas-ply are missing one of (what I think is) the core value propositions of dplyr: the ability to use the same abstractions on local data and on remote data, with execution against the remote source happening lazily such that the entire table doesn't need to be downloaded in order to run a filter locally.
(Of course, I may be biased in that I work on a commercial product which also has this characteristic.)
- TheLogothete 11y agoWhat product? edit: After some stalking, I see you work on PowerQuery. Great product! Improving the SQL editing capabilities would make it much better!
- tanlermin 11y agoBlaze for python does this
- capybara 11y agoDplython developer here... Definitely a big and important next step. I wanted to get some feedback on the in-memory case first.
- tanlermin 11y agoBlaze has already spent alot of time on this usecase and seems quite promising for a whole host of other reasons. Perhaps it would be best to combine forces? Also see this: https://github.com/blaze/blaze/pull/484 https://github.com/blaze/blaze/pull/484
- capybara 11y agoThe idea of putting the column names into the namespace is very interesting, and blaze looks pretty promising overall as a way to connect to big data sources.