I wrote a program that ran on that type of computer once. It was real-time log analysis. There was a lot of data coming in, and there was a need for a globall

I wrote a program that ran on that type of computer once. It was real-time log analysis. There was a lot of data coming in, and there was a need for a globally-consistent datastore for the results. Solution? Put it all in RAM. If the single replica crashes, you can just read all the inputs again. Some extra capacity existed for that, and some other system already stored the raw logs to disk durably. The terabyte of in-memory data just made queries tolerable enough to run every few seconds and display in alert/chart form. I didn't keep the system in this state for very long, but for version 1.0 it was just the thing -- idea to production in a short period of time. Eventually it did move to a more distributed system, as log volume (and usefulness) increased, and we had more time to deal with the details. It was mostly nice to not have to reprocess data after releases -- I could do them in the middle of the day without anyone caring. My biggest worry when writing this was that 40Gbps of network bandwidth wouldn't be enough, but it was fine in the early stages. 40Gbps is a lot of data. I'm not sure I'd say it's a great sign that you need a single beefy machine to run something, but it's a tradeoff worth considering. I found the distributed system version easier to operate, but it did constrain what sort of features you could add. I think we got it right, but it's easy to code yourself into a corner when you have encoded assumptions deep into the system. Best to avoid that until you're sure your assumptions are right. Avery wrote up some details of the system: https://apenwarr.ca/log/20190216 https://apenwarr.ca/log/20190216 His idea, but I wrote most of the code ;)