3 ms·
Spark uses off heap memory via JNI in order to avoid JVM memory management drawbacks. It hides it behind a heavy abstraction, which makes it plausible to use su
by krcz 7y ago
Spark uses off heap memory via JNI in order to avoid JVM memory management drawbacks. It hides it behind a heavy abstraction, which makes it plausible to use such solution for data processing pipelines, for the smaller applications not so much.
But for most things that is not that much a problem - I'm not saying that JVM memory model makes Scala unusable, just that Spark is not a great example here.