4 ms·
What do people use Hadoop for? Are you asking because you genuinely don't know? Hadoop lets me pick up PBs of (typically) record-format data and stick it on a
by growse 12y ago
What do people use Hadoop for? Are you asking because you genuinely don't know?
Hadoop lets me pick up PBs of (typically) record-format data and stick it on a bunch of cheap servers that are already standard items in my datacenter catalogue and are well-understood by my datacenter/ops teams. I can then open up any Java IDE, write a program now I can process that data. Up until this point, I've not really needed much specialist knowledge, perhaps with the exception of understanding the map/reduce paradigm which is a fairly basic programming construct anyway.
There's lots of companies out there who have data problems that are bigger than a few TB, and they're bored of shovelling money at Oracle so they can have a fancy, expensive database when their data isn't actually relational to begin with.
- KaiserPro 12y agoThats the thing, So I get that it gives you network aware Fork() Its just the scheduling is a bit, well, lacking. In my datacenter, we run dirty cheap iron. The only thing special are the core switches(lots and lots of ten gig ports, to cope with the petabytes that we shove across it daily) we use standard linux tools, sharded posix file systems with a decent task dispatcher that understands dependencies.