3 ms·
You could but I would do thorough real-world benchmarks. EMRFS has dozens of optimisations for Spark/Hadoop workloads e.g. S3 select, partitioning pruning, opt
by threeseed 6y ago
You could but I would do thorough real-world benchmarks.
EMRFS has dozens of optimisations for Spark/Hadoop workloads e.g. S3 select, partitioning pruning, optimised committers etc and since EMR is a core product it is continually being improved. Using HopsFS would negate all of that.