3 ms·

Now an experiment with 4 gpu nodes. Horizontal scaling matters as well, which is not shown here. Furthermore, using less than 16gb of data.. why would you use

Now an experiment with 4 gpu nodes. Horizontal scaling matters as well, which is not shown here. Furthermore, using less than 16gb of data.. why would you use these frameworks in the first place? It is common knowledge that a grep and sort on 100gb is faster than Hadoop, however 100tb is a different story

felipe_aramburu 8y ago
Uncoordinated distributed is coming soon. We have been building this tool with distribution in mind and right now are finishing up our first distributed version of BlazingSQL which we will be releasing just before GTC ~March 15th. We have already distributed using dask which you can do RIGHT now with blazingsql but we felt it would not be fair to make a comparison between distributed blazingsql which is user managed and something like spark which happens under the hood. We will be releasing distributed numbers early March where we can show differences between BlazingSQL and larger spark clusters where the distribution is handled by the engine instead of the user.