36 ms·
In HPC they are. CUDA wins because it is wonderfully fast. There is a huge benifit for implementing it, even if it requires lots of work to get efficient MPI,
by KaiserPro 13y ago
In HPC they are.
CUDA wins because it is wonderfully fast. There is a huge benifit for implementing it, even if it requires lots of work to get efficient MPI, because the rewards are literally massive.
Hadoop uses REST ffs. You're never going to get low latency with REST. Its a nasty hack in the world of web, let alone HPC where there really isn't any excuse for using HTTP for something its really not designed to do.
But the question is this:
what do people actually use Hadoop for?
- growse 13y agoWhat do people use Hadoop for? Are you asking because you genuinely don't know? Hadoop lets me pick up PBs of (typically) record-format data and stick it on a bunch of cheap servers that are already standard items in my datacenter catalogue and are well-understood by my datacenter/ops teams. I can then open up any Java IDE, write a program now I can process that data. Up until this point, I've not really needed much specialist knowledge, perhaps with the exception of understanding the map/reduce paradigm which is a fairly basic programming construct anyway. There's lots of companies out there who have data problems that are bigger than a few TB, and they're bored of shovelling money at Oracle so they can have a fancy, expensive database when their data isn't actually relational to begin with.
- KaiserPro 13y agoThats the thing, So I get that it gives you network aware Fork() Its just the scheduling is a bit, well, lacking. In my datacenter, we run dirty cheap iron. The only thing special are the core switches(lots and lots of ten gig ports, to cope with the petabytes that we shove across it daily) we use standard linux tools, sharded posix file systems with a decent task dispatcher that understands dependencies.