Author here! A few others have pointed this out, but to restate: in our situation, compression was costing us almost nothing. 60TB of logs was 60 days worth of

Author here! A few others have pointed this out, but to restate: in our situation, compression was costing us almost nothing. 60TB of logs was 60 days worth of retention, so 1TB a day. That means we process about 11MB/s on average, peaking at 100MB/s. A single CPU core can manage 100MB/s of compression, so if you assume we compress and decompress in multiple places, let's say we're paying about 4 CPU cores constantly for this. That's a pretty worse case scenario, and it would cost us $37.50/month on GCP for those cores, in order to save about 100x that amount. The takeaway (for me at least) is that compressing an in-flight transfer is almost always worthwhile if you're working in the cloud and the transfer will eventually be stored somewhere. The economics of CPU vs the total amount of data storage cost is a no brainer. Hope that makes sense!