3 ms·

I think your second point is vastly overstated for 99% of companies and their use cases. There are precious few situations that can't be handled decently well

I think your second point is vastly overstated for 99% of companies and their use cases. There are precious few situations that can't be handled decently well by a simple RDBMS, and a simple ETL process into an analytical data store for reporting. Almost everything that exist fits one of three buckets: 1) CRUD app, 2) reporting tool, 3) business process app. And almost all of this is for internal use only. It's really only a handful of enterprise things that need massive scalability and performance, and consumer-oriented stuff (much of which is pretty niche and also doesn't need massive global scalability).

benrutter 3y ago
I think you're right and I probably came over a bit strong in how likely I made out companies would be to use that level of scale. The point I was trying to make was more that because using something like RDBMS (maybe + VMs or something for data science) has a hard ceiling in terms of scale, even if companies are really unlikely to hit that ceiling, there is a weird incentive structure. The costs of databricks or snowflake are really high, but probably not enough to bankrupt a company or division. If you're a small to midsize company and you realise you can't compute on your data, that could cause much more serious issues. Even if those issues are only feasible 1/100 times, it's a pretty scary risk to expect a CTO to ignore.