Not everything is physical access though. Suppose someone comes up with a way to force S3 to read random storage blocks, or to make a virtualized storage device

Not everything is physical access though. Suppose someone comes up with a way to force S3 to read random storage blocks, or to make a virtualized storage device read past its boundaries in the underlying storage, or to intercept another VM's ring buffer in the hypervisor. It's an entirely different scenario to read plaintext vs read something encrypted. Even if all it protects is against the scenario of the cloud provider forgetting to wipe a disk, that's worth it.