> You're using the file system to dump static rows of data That's what MySQL, PostgreSQL, SQL Server, and Oracle all do. They write to a logfile (called the "w

> You're using the file system to dump static rows of data That's what MySQL, PostgreSQL, SQL Server, and Oracle all do. They write to a logfile (called the "write ahead log") then periodically (and concurrently) process it into working sets that are checkpointed (checked) in much the same way. It's a lot slower because they don't know what is actually important in the statement except what they can deduce from analysis. Whilst that analysis is slow, they do this so that structural concerns can be handed off to a data expert (often called a DBA), since most programmers have no fucking clue how to work with data. That can work for small data, but it doesn't scale past around the 5bn inserts/day mark currently, without some very special processing strategies, and even then, you don't get close to 50bn. > All your data munging is on memory-sized blocks at program-level. That is literally all a computer can do. If you think otherwise, I think you need a more remedial education than the one I've been providing. > What you're doing here is incredibly simplistic. It's not proper resiliency, it's not scalable to more complex problems, and it's not scalable to larger workloads. An mmap'ed table and an actual database are different things. Yes, except for everything you said, nothing you said is true in the way that you meant it. Google.com does not query a "database" that looks any different from the one I'm describing; Bigtable was based on Arthur's work. So was Apache's Kafaka and Amazon's Kinesis. Stream processing is undoubtedly the future, but it started here: https://pdfs.semanticscholar.org/15ec/7dd999f291a38b3e7f455f39f242820c7939.pdf https://pdfs.semanticscholar.org/15ec/7dd999f291a38b3e7f455f... Not only does this strategy get used for some of the hardest problems and biggest workloads, it's quite possibly the only strategy that can be used for some of these problems. Resiliency? Simplistic? I'm not even sure you know what those words mean. Putting "proper" in front of it is just weasel words...