3 ms·
> So is the protocol sort of like a big public kafka topic? yes, at least the firehose is that, and anyone can subscribe or run their own https://atproto.com/
by verdverm 1y ago
> So is the protocol sort of like a big public kafka topic?
yes, at least the firehose is that, and anyone can subscribe or run their own
https://atproto.com/articles/atproto-for-distsys-engineers https://atproto.com/articles/atproto-for-distsys-engineers
> to make sure you don't miss updates
Yes, the firehose and jetstreams have a cursor, though with limited time window depending on the instance. (1-3h -> 72h)
If you want the entire history you do what's called a backfill
- maelito 1y agoHi, thanks for these info. I have a web app that produces atproto records, app.cartes.comment. My server listens to Jetstream and builds its DB based on this. But what about if my server goes down for say 4 days ? How can I rebuild my DB to catch up those days ? It looks like Jetstream cannot work for this. My need is simple : give me all the records of the collection on the whole atmosphere. How to do this ?