3 ms·
> I would much rather a solution where I can offload old events into some other storage system At large enough scale, it becomes mandatory to separate the even
by bob1029 2y ago
> I would much rather a solution where I can offload old events into some other storage system
At large enough scale, it becomes mandatory to separate the event bus from the system of record. I witnessed this in semiconductor manufacturing. There was a gigantic transaction table of finalized things, and all of the detail event data was on a separate messaging bus/system with its own log files. The raw event data was on the order of 300-400 megabytes per hour per mfg line.
Even with this separation, the high level table was still very scary to use. An unconstrained "SELECT *" would invariably result in a visit from the on-call DBA.
- AndyNemmity 2y agoI don't even thing the scale has to be that large. You can't hold the event bus, and the system of record. I was going to post about this very topic, but you accurately pointed it out.