2 ms·
We have also seen this. The huge memory capacities now offered in the cloud makes single node processing very capable with entire TB datasets fitting into RAM,
by manigandham 8y ago
We have also seen this. The huge memory capacities now offered in the cloud makes single node processing very capable with entire TB datasets fitting into RAM, and running quickly enough to offset the hourly cost. Spark clusters are more efficient though for longer-running or continuous background processes.
- riku_iki 8y agoAnd spark also has some other goodies: integration with storages, monitoring, fault tolerance, common algorithms implemented.