Agreed. Im 80% sure CSV is best for his use case. Maybe even JSON line files. I think that he's working with regular data (a few hundred MB) because he's consid

Agreed. Im 80% sure CSV is best for his use case. Maybe even JSON line files. I think that he's working with regular data (a few hundred MB) because he's considering memory mapping it. I have a rule that no-one is allowed to use the words big,small,fast or slow. You must quantify. I've met too many people who think that 100MB is big data or that 1 Gbps is a fast internet connection.