You might be interested in Gazette (https://github.com/gazette/core https://github.com/gazette/core). It presents durable, append-only, transactional byte strea

You might be interested in Gazette (https://github.com/gazette/core https://github.com/gazette/core). It presents durable, append-only, transactional byte streams ("journals") while delegating all actual storage to S3/GCS/Azure etc. Brokers themselves are fully ephemeral. Written journal content ends up in predictably-named "fragments" in a configured bucket path, where each fragment holds only raw written bytes (eg, if you write JSONL messages, your fragments are also just JSONL). The broker is pure Go, and Etcd is used for distributed consensus. The project has been around for years (as closed source), but still lacks a bunch of polish as an open-source offering. Working on that...