I believe the answer to that question is yes. I've raved about Spark many times on HN before. We're using it and absolutely love it. Not sure why the author th

I believe the answer to that question is yes. I've raved about Spark many times on HN before. We're using it and absolutely love it. Not sure why the author thinks there's no built-in support for iterations. The support is there, native scala/java. But you might have to collect to the driver/master to check for convergence, for example. Their simple logistic regression example doesn't check for convergence, but hard codes the number of iterations. http://spark.incubator.apache.org/examples.html http://spark.incubator.apache.org/examples.html Some quirks that I've learned. When doing groupbys or other join operations, it helps to specify the number of partitions -- actually this is true in general, but more so for join-based operations, otherwise you can run into memory/gc issues. The equivalent is of course, setting the number of reducers to something which ensure you won't run out of heap space. Because spark serializes the closures which transform your data, if you don't cache (at least via disk, using persist), then when iterating over an RDD or re-using it, you'll just waste cycles. Beyond that, as an experienced scala developer, I've found that Spark feels incredibly natural. And being able to run things on the repl cannot be appreciated enough.