4 ms·
-- Functional programming does not work well in programs which handle a lot of states Apache Spark is written in Scala. It can handle extremely large amounts
by century19 10y ago
-- Functional programming does not work well in programs which handle a lot of states
Apache Spark is written in Scala. It can handle extremely large amounts of state across large clusters of machines. Is the argument here that Scala is not purely functional? Or what am I missing?
- paulddraper 10y agoBy "lots of state" I don't think he meant "lots of data", but rather data that get changed a lot: Q&A forum, CRUD application, etc. Functional paradigms can still be used (e.g. HackerNews, though let's be honest...I love it but it's not the most challenging feature set). But FP isn't as natural of a fit as the problems chosen to introduce it.
- century19 10y agoNot saying your wrong about what he meant, but Spark can also keep state across the Cluster, not just data. Check out the "UpdateStateByKey Operation" in the following page. http://spark.apache.org/docs/latest/streaming-programming-guide.html http://spark.apache.org/docs/latest/streaming-programming-gu...
- teirce 10y agoI haven't taken a look at the Spark codebase, but it is very easy to write Scala code that is not purely functional.