4 ms·

> These paradigms get thought up by people who don't run big systems This is explicitly for people that don't run big systems – "the 99%". > The best model fo

> These paradigms get thought up by people who don't run big systems This is explicitly for people that don't run big systems – "the 99%". > The best model for random devs who just want to scale up their jobs is to model what the dev is using for dev. That doesn't seem like it will work if "random devs" aren't writing their code to even be capable of running on multiple nodes. You're also ignoring points made in the post: > Many more casual users of data processing platforms (the 99% from the paper title) don’t want to spend hours understanding a complex stack of e.g. Spark, Scala, HDFS, Yarn and the JVM. Even the choice of instance types (70 on AWS at the time the paper was written), cluster sizes, regions and so on can be bewildering. AFAIK, there's nothing that will automatically pick a stack, instance type, etc. based on a random dev's laptop (or desktop).

peterwwillis 9y ago
It is for people who run big systems because the title includes the words "distributed computing". They might as well have called this paper "building skyscrapers in your back yard" (which is a pretty good name for a paper I think!). If you don't write your app to be capable of running on multiple nodes, you aren't doing distributed computing - you're doing concurrent computing. If there's nothing out there that picks a stack for you, that could certainly be automated based on a Q&A for the dev. But it's arguable that this decision should not be automated, because this decision directly impacts how the app works.