3 ms·
Why this hybrid dataframe? [1]: > [Arrow] adoption will take time, and most people are probably more comfortable seeing NumPy arrays. Therefore a Vaex version
by sradman 6y ago
Why this hybrid dataframe? [1]:
> [Arrow] adoption will take time, and most people are probably more comfortable seeing NumPy arrays. Therefore a Vaex version 4 a DataFrame can hold both NumPy arrays and Apache Arrow arrays to make the transition period easier.
There seems to be agreement that Apache Arrow is the future of dataframes across ML ecosystems. I didn't realize this transition impacted NumPy arrays in addition to Pandas dataframes in Python.
[1] https://vaex.io/blog/a-hybrid-apache-arrow-numpy-dataframe-with-vaex-version-4#why-hybrid https://vaex.io/blog/a-hybrid-apache-arrow-numpy-dataframe-w...
- nomel 6y ago> There seems to be agreement that Apache Arrow is the future of dataframes across ML ecosystems Not, until there is proper multidimensional array support.