4 ms·
The fact that storing data in binary format gives you more speed is exactly why you'd want to use Feather. It means you don't have to translate things into an u
by rspeer 11y ago
The fact that storing data in binary format gives you more speed is exactly why you'd want to use Feather. It means you don't have to translate things into an unspecified .bin format. This is the answer to the original question about "why not use CSV".
I don't know about R, but in Python, operations on objects such as NumPy arrays and Pandas DataFrames are all implemented using fast C code, and so is Feather. You can be concerned about speed.
- roel_v 11y agoYes, of course, and I'd much rather use something like it; and when it will support matrices with more than 2 dimensions, I will (or at least, I will look into it). In cases where I need more robust storage I already use HDF5 or NetCDF but they're a PITA to work with. Of course in-memory operations can be implemented efficiently, R does that too. But Python needs to parse CSV into numbers just like everybody else, and even if it's done in C underneath, it'll still be 'slow' (for some values of that word).