They do compress for most log delivery types like load balancers, cdn, cloudtrail, etc. and it makes a huge difference over raw logs - compression ratios are in

They do compress for most log delivery types like load balancers, cdn, cloudtrail, etc. and it makes a huge difference over raw logs - compression ratios are in the 90s. One AWS specific trick is individual log files below some size threshold are passed through raw, so you end up with a mix of compressed and uncompressed files in S3, and the only reliable way to distinguish between them is to receive them and look for a zlib header - you can’t depend on the file name or any other metadata to tell you ahead of time if an individual file is compressed or not. (I think Cloudtrail does set metadata correctly for uncompressed files, but other types do not, best to spend the time developing a abstraction that deals with both)