2 ms·

> Why pay $230k a year for a "scalability/reliability engineer", when I could pay exponentially less to Google or AWS and not have to worry about anything? I t

by mbesto 12y ago

> Why pay $230k a year for a "scalability/reliability engineer", when I could pay exponentially less to Google or AWS and not have to worry about anything? I think you're massively undervaluing the complexity of systems at scale. Let me give you a more concrete example. In Facebook and LinkedIn there is a feature that says "You may know this person". It's a small box in the upper right corner and looks like the smallest feature ever. But the underlying complexity of building such a feature is easily millions of dollars worth of man hours. Just that small feature has probably single-handedly grown those companies by > 1x factor. You don't get that simply by "throwing it at AWS". I appreciate your enthusiasm for this topic, but I think the lens at which you see the startup world is being blurred by media and survivor bias. There is so much under the tip of the iceberg that 99% of the people here will never see.

no_future 12y ago
I have to disagree with you here. Elasticsearch has a "more like this" feature built into it, which would give you pretty much the desired functionality you're describing that Facebook and LinkedIn have. It's a free and open source technology that I've installed and maintained on my servers. All you have to do is feed it some JSON, which you can modify as you choose to bias it to your exact specifications. Google and AWS offer cloud search and database features which already have this type of functionality implemented for you - all you have to do is wire it to your app. I'm not saying that these problems were trivial to solve in the first place, but THEY HAVE ALREADY BEEN SOLVED BY GOOGLE AND AMAZON AND FACEBOOK AND A MILLION OTHER COMPANIES' ENGINEERS, and made available to you in a nice and easy to use package with a bow on top. See here: https://cloud.google.com/products/bigquery/ https://cloud.google.com/products/bigquery/ https://cloud.google.com/products/cloud-datastore/ https://cloud.google.com/products/cloud-datastore/ http://aws.amazon.com/cloudsearch/ http://aws.amazon.com/cloudsearch/

by mbesto 12y ago

no_future 12y ago

I have to disagree with you here. Elasticsearch has a "more like this" feature built into it, which would give you pretty much the desired functionality you're describing that Facebook and LinkedIn have. It's a free and open source technology that I've installed and maintained on my servers. All you have to do is feed it some JSON, which you can modify as you choose to bias it to your exact specifications. Google and AWS offer cloud search and database features which already have this type of functionality implemented for you - all you have to do is wire it to your app. I'm not saying that these problems were trivial to solve in the first place, but THEY HAVE ALREADY BEEN SOLVED BY GOOGLE AND AMAZON AND FACEBOOK AND A MILLION OTHER COMPANIES' ENGINEERS, and made available to you in a nice and easy to use package with a bow on top. See here: https://cloud.google.com/products/bigquery/ https://cloud.google.com/products/bigquery/ https://cloud.google.com/products/cloud-datastore/ https://cloud.google.com/products/cloud-datastore/ http://aws.amazon.com/cloudsearch/ http://aws.amazon.com/cloudsearch/