4 ms·
Spark still uses Hadoop underneath. While Hadoop mapreduce uses the disk, spark uses memory for faster processing. And I am not sure anything is going to kill s
by ganeshkrishnan 9y ago
Spark still uses Hadoop underneath. While Hadoop mapreduce uses the disk, spark uses memory for faster processing. And I am not sure anything is going to kill spark soon. Once a library gains critical mass, it's harder to replace it in existing systems
- doug1001 9y agoSpark can read from and write to HDFS, and YARN (the resource negotiator in Hadoop v 2) is one of the cluster managers supported by Spark. but you can run a Spark cluster without YARN--eg, with Mesos--or with the built-in manager provided in the Spark distro. Likewise, your Spark cluster doesn't need to read or write to HDFS.