4 ms·
Spark is vastly different. Tensorflow focuses more on numerical computing and is a low level tool. Spark is more focused on "counting at scale with a function
by agibsonccc 10y ago
Spark is vastly different. Tensorflow focuses more on numerical computing and is a low level tool.
Spark is more focused on "counting at scale with a functional DSL". Hence its focus on things like ETL and columnar processing ala dataframes.
As far as spark doing "deep learning" what you should mean here is: "libraries in the ecosystem leverage spark as a data access layer for doing the real numerical compute"
Spark can count things with functional programming. It's not meant for heavy numerical operations. They are working on this where they can but you really can't beat a gpu or good ole simd instructions on hardware.