Even with chunks it's prohibitively expensive at this scale, and you then need some intermediate system batching the inbound data either to disk (then why use S

Even with chunks it's prohibitively expensive at this scale, and you then need some intermediate system batching the inbound data either to disk (then why use S3 instead of a database) or in memory (more expensive, more risk of data loss).