3 ms·
edit: local[1] has been updated to local[N], thank you for the update! Ok thanks, I didn't know that's what "local[1]" did, so the more relevant comparison wou
by optimali 10y ago
edit: local[1] has been updated to local[N], thank you for the update!
Ok thanks, I didn't know that's what "local[1]" did, so the more relevant comparison would be with --master local[30]?
The algorithm took around 500 seconds to train on the NETFLIX dataset on a SINGLE processor, which is good for data as large as 1 billion ratings.
- this is from the sequential portion of the test, the parallel portion is the next section.
- ViralBShah 10y agoThat was a typo in the blog post. If you look at the graph, with more cores, spark gets faster as does Julia. The typo is now fixed.
- shipman05 10y agoThanks for the update. The typo had me misinterpreting things. Now it makes more sense. Assuming you're part of the team? Keep up the good work.