IMO, the answer is pretty clear here. Use Arrow with some form of compression. Most of the other options the Julia ecosystem offers are just in case you don't h

IMO, the answer is pretty clear here. Use Arrow with some form of compression. Most of the other options the Julia ecosystem offers are just in case you don't have the luxury of controlling your data format (ie because you are reading data from someone else). Also, did you look at using ZSTD? In my experience, it provides a really good compression, and is pretty fast also.