When working on a genome project I saw files up to 150gb, and as I understand it these are compressed csv files.its a lot of storage, especially if you unrl thi

When working on a genome project I saw files up to 150gb, and as I understand it these are compressed csv files.its a lot of storage, especially if you unrl this and index it further for performance reasons. Also the more you compress, the longer the processing time. Some jobs took days, so there was a trade-off. I'm not a genome expert btw, just a devops nerd on a related project.