2 ms·
I'd be interested to know how you can assert that Hadoop is a 'complete waste of hosts for most companies'. Also, don't underestimate the many, many people succ
by growse 10y ago
I'd be interested to know how you can assert that Hadoop is a 'complete waste of hosts for most companies'. Also, don't underestimate the many, many people successfully running Spark on YARN at scale. Hadoop is actually quite helpful to some workloads.
- dkersten 10y agoMost companies simply don't have the data volume to make Hadoop worthwhile. You can process tens of TB in an RDBMS on a beefy machine cheaper than a Hadoop cluster. Hadoop is slow, but on huge data volume the overheads are dwarfed by the parallelism gained. Most companies don't have huge volume though. For example recently I saw someone propose using Hadoop for a sub-TB dataset...