Oh yeah, there's stuff you can do, but again it takes tuning and ideally it shouldn't be so annoying. I definitely get the frustration, but waiting for sklearn

Oh yeah, there's stuff you can do, but again it takes tuning and ideally it shouldn't be so annoying. I definitely get the frustration, but waiting for sklearn for hours to fit a fraction of the observations that spark can do is also frustrating, in a different way. > run on a single node yeah, Spark is pretty crap here, but it's not really one of their main use cases, to be fair.