That isn't true; a git repository isn't just a single file. It is one (or many) pack and object files, and it's possible to have a repository by using each push

That isn't true; a git repository isn't just a single file. It is one (or many) pack and object files, and it's possible to have a repository by using each push (which may be multiple commits) to store a new pack file rather than deleting the old one. The only time you would need to delete/replace is during a GC operation which can be scheduled periodically to compact the representation. The JGit driver supports pushing to S3 in this way, and I have implemented a custom backend to hosting git repositories in an S3 like service (which in turn is based on how Google hosts git repositories at scale)