3 ms·

This was actually for a few of my post-Google projects. At Google we would just use MapReduce regardless of how inappropriate it was because of the difficulty

This was actually for a few of my post-Google projects. At Google we would just use MapReduce regardless of how inappropriate it was because of the difficulty of standing up an RDBMS on Google's cloud infrastructure, plus CPU cycles were basically free for engineers. Maybe it's different now that internal projects are encouraged to use Google Cloud; Cloud SQL + Compute Engine works just fine for this. And this is definitely not a new pattern - if you wanna go back in time, this is how things often worked in the mainframe era. My point - and IIUIC the point of the article - is that people often reach for flavor-of-the-month heavyweight frameworks when they can easily adapt tools they're already using to solve the problem within hours. (Another great example of this is the single-process in-RAM architecture used by PlentyOfFish, StackOverflow, Mailinator, and Hacker News when they were solo-developer projects, where you store all state in RAM and run the whole site on a single box, periodically dumping out checkpoints to disk in case the process crashes. This can scale to thousands of requests per second - that's millions of queries per day - and is a lot simpler to program than 3-tier architectures where everything runs through a data abstraction layer and you spend 90+% of the code shuffling data around.)

gspetr 7y ago
> CPU cycles were basically free for engineers. So what happened to a guy who had the bright idea to do CPU mining on Google's infra?
- WesternStar 7y ago
  He was politely asked to stop. I believe it was folding at home(?) if I remember the story I heard correctly.
rhizome 7y ago
>people often reach for flavor-of-the-month heavyweight frameworks when they can easily adapt tools they're already using to solve the problem within hours. Absolutely, which is why I think it's important to call concepts the same thing they called it in the olden times if you know it's an old thing!

gspetr 7y ago

> CPU cycles were basically free for engineers. So what happened to a guy who had the bright idea to do CPU mining on Google's infra?

WesternStar 7y ago
He was politely asked to stop. I believe it was folding at home(?) if I remember the story I heard correctly.

rhizome 7y ago

>people often reach for flavor-of-the-month heavyweight frameworks when they can easily adapt tools they're already using to solve the problem within hours. Absolutely, which is why I think it's important to call concepts the same thing they called it in the olden times if you know it's an old thing!