3 ms·
Polars has a python front end, which I have used. All the work happens in Rust but the queries can be specified in python. The data is stored using Apache Arrow
by josh-sematic 3y ago
Polars has a python front end, which I have used. All the work happens in Rust but the queries can be specified in python. The data is stored using Apache Arrow format so there is no copy required for the same data to be accessible in both Rust and python.
- nonethewiser 3y agoOh, that's cool. I did not realize this. I realize this isn't novel but I do not really understand how a library written in one language is used by another - some sort of bindings that the library handles I guess.
- dleeftink 3y agoThe magic of well-defined APIs! If you're interested in mixing different DS backends and kernels in a single notebook, check out Quarto: [1]: https://quarto.org/ https://quarto.org/
- wodenokoto 3y agoBoth R and pythons pandas uses code written in C and Fortran to do the actual calculations when you ask them to manipulate data.
- movpasd 3y agoIt looks like polars is using PyO3, which are Rust bindings for Python. Python's reference implementation is in C, so I imagine it's interacting with that API [0] through FFI. Common Python extension modules (as these are called) are compiled as dynamically linked libraries and binaries (or compilation instructions) are included in Python wheels. [0] https://docs.python.org/3/extending/extending.html https://docs.python.org/3/extending/extending.html