3 ms·
https://issues.apache.org/jira/browse/SPARK-139 https://issues.apache.org/jira/browse/SPARK-139 https://issues.apache.org/jira/browse/SPARK-1476 https://issues
by snnn 10y ago
https://issues.apache.org/jira/browse/SPARK-139 https://issues.apache.org/jira/browse/SPARK-139
https://issues.apache.org/jira/browse/SPARK-1476 https://issues.apache.org/jira/browse/SPARK-1476
https://issues.apache.org/jira/browse/SPARK-6235 https://issues.apache.org/jira/browse/SPARK-6235
You'll hit this bug when your model size is larger than 2GB.
BTW, Recomputation of RDDs may result in duplicated accumulator updates. So please do not use accumulator in your trainer for gradient summation. They know that, but they said they will not fix that.
https://issues.apache.org/jira/browse/SPARK-732 https://issues.apache.org/jira/browse/SPARK-732
https://issues.apache.org/jira/browse/SPARK-5490 https://issues.apache.org/jira/browse/SPARK-5490