2 ms·
I may be wrong. It happens a lot! But I think Arrow's vision encompasses compute, not just a data format and data interface. https://www.slideshare.net/wesm/py
by awaythrowact 5y ago
I may be wrong. It happens a lot! But I think Arrow's vision encompasses compute, not just a data format and data interface.
https://www.slideshare.net/wesm/pycon-colombia-2020-python-for-data-analysis-past-present-and-future https://www.slideshare.net/wesm/pycon-colombia-2020-python-f...
Slide 43: The "Arrow C++ Platform" encompasses a "Multi-core Work Scheduler" and a "Query Engine"
Slide 38: "It would be more productive (long-term) to have a reusable computational foundation for data frames"
Again, I agree that, today, it's more data format, and the shared compute stuff is more a vision.
EDIT: See also https://ursalabs.org/tech/ https://ursalabs.org/tech/
- BadInformatics 5y agoFor sure, I didn't mean to imply they weren't looking at compute too! https://github.com/apache/arrow-datafusion https://github.com/apache/arrow-datafusion is another example of the shared compute vision. What I was trying to point out is that (at least for Arrow core) they seem to eschew FFI and generating shared libraries in favour of from scratch implementations in other compiled languages and direct bindings in interpreted ones.