4 ms·
Nice! Always thought this would be cool. Few thought questions though: how do you get things like consistent hashing? Spark for example can shuffle data somewha
by eranation 8y ago
Nice! Always thought this would be cool. Few thought questions though: how do you get things like consistent hashing? Spark for example can shuffle data somewhat efficiently by sending the data to the right node / getting data from the right node by the hash key, right? How in a serverless stateless world you call a specific Serverless function instance? Assuming it can’t be done, arent you losing performance gained by data locality? Eg data has to be saved in a massive and efficient key value store? Isn’t it much slower than spark’s in memory / data locality (bring the compute to the data and not vice versa). Would love to see benchmarks on this. This is the future IMHO... well done.