3 ms·

Bottling the Magic Behind Google and Facebook

15 points by peter123 18y ago

mdasen 18y ago
So, as a business model, I get this feeling that it won't fly. Hadoop is a great tool if you need to deal with data at the level that a top 50 site would need to. But at that point, wouldn't you be able to hire someone full-time with a six-figure salary? Consulting tends to work better for firms that can't afford to hire someone as staff - and those firms tend not to be of he size that would need such a scalable data architecture. Heck, even sites like Flickr (39th most trafficked), Wikipedia (7th), Wordpress.com (36th), and Craigslist (47th) are still able to run off MySQL. It just seems like businesses of that size would want staff and smaller businesses wouldn't have the data to justify it.
- boorad 18y ago
  I can only guess how they're going to do, business model-wise, but big data doesn't necessarily mean big companies or web properties as you suggest. Small startups can do compelling things if they make sense out of big data. It's a tedious and often impossible job now for just about everyone. Companies displaying competence will do quite well for at least the beginning phase of this movement, and have customers both big and small. My gut says that it will become commonplace to need to handle tera or petabyte-sized data. 640K turned out to be a wee bit shy of our memory needs. Traditional RDBMS's will turn out to be a wee bit shy of our analysis needs.
- inerte 18y ago
  There are a lot of big business that don't have a strong web presence in traffic. While the first example that it comes to mind is Wal-Mart, the anecdote I am going to share is from there, so they probably don't "need" Hadoop, but maybe it'll make their analysis faster and more accurate. It's said that after Katrina, sales of peanut butter skyrocketed. I can't find a proper link right now to confirm this story, but it doesn't need exactness, because the theme is simply: There's a lot of data out there, how can we make sense of it?

15 points by peter123 18y ago

So, as a business model, I get this feeling that it won't fly. Hadoop is a great tool if you need to deal with data at the level that a top 50 site would need to. But at that point, wouldn't you be able to hire someone full-time with a six-figure salary? Consulting tends to work better for firms that can't afford to hire someone as staff - and those firms tend not to be of he size that would need such a scalable data architecture. Heck, even sites like Flickr (39th most trafficked), Wikipedia (7th), Wordpress.com (36th), and Craigslist (47th) are still able to run off MySQL. It just seems like businesses of that size would want staff and smaller businesses wouldn't have the data to justify it.

boorad 18y ago
I can only guess how they're going to do, business model-wise, but big data doesn't necessarily mean big companies or web properties as you suggest. Small startups can do compelling things if they make sense out of big data. It's a tedious and often impossible job now for just about everyone. Companies displaying competence will do quite well for at least the beginning phase of this movement, and have customers both big and small. My gut says that it will become commonplace to need to handle tera or petabyte-sized data. 640K turned out to be a wee bit shy of our memory needs. Traditional RDBMS's will turn out to be a wee bit shy of our analysis needs.
inerte 18y ago
There are a lot of big business that don't have a strong web presence in traffic. While the first example that it comes to mind is Wal-Mart, the anecdote I am going to share is from there, so they probably don't "need" Hadoop, but maybe it'll make their analysis faster and more accurate. It's said that after Katrina, sales of peanut butter skyrocketed. I can't find a proper link right now to confirm this story, but it doesn't need exactness, because the theme is simply: There's a lot of data out there, how can we make sense of it?