3 ms·
I'm similarly confused. It seems to be a family of table encodings that sacrifice encoding simplicity for compactness. PyArrow implements parquet files for exam
by cookguyruffles 5y ago
I'm similarly confused. It seems to be a family of table encodings that sacrifice encoding simplicity for compactness. PyArrow implements parquet files for example, but also I think feather? PR for this project is a mess. Front page should be a bullet point list of deliverables rather than aspirations nobody understands
- vletal 5y agoCome on, it's on the top of the front page https://arrow.apache.org/ https://arrow.apache.org/
- cookguyruffles 5y agoYou don't understand, I'm already a user of PyArrow and it doesn't match that page at all. It handles Parquet and Feather, right? They don't appear on the home page. Clicking the "specifications" link instead starts to talk about Flatbuffers. What's going on?
- papercrane 5y agoArrow is the in-memory format, PyArrow supports loading and saving that data as Parquet and Feather formatted files.
- nerdponx 5y agoParquet and Feather are on-disk file formats. Arrow is an in-memory format. Parquet is not Arrow, but they work well together, in that one can easily be (de)serialized to the other. Feather uses the Arrow IPC format internally.
- Tostino 5y agoTheir FAQ page at least answered some of that for me. Have the same feeling. http://arrow.apache.org/faq http://arrow.apache.org/faq