4 ms·
The point was that the concurrency model of these ML libraries is not dependent on a CPU for backprop (ideally), so discussing language level concurrency as an
by codesushi42 7y ago
The point was that the concurrency model of these ML libraries is not dependent on a CPU for backprop (ideally), so discussing language level concurrency as an advantage is irrelevant. The parallelism is not implemented at the language level, nor does it rely on language constructs for synchronizing shared memory. The parallelism happens at the hardware level (e.g. with SIMD). And in the model architecture (e.g. with convolutions).
So the advantages you posited in your lengthy diatribe are meaningless, because you failed to take into account the problem domain, along with utterly failing to grasp what gets calculated during backprop. Plus, backprop doesn't get calculated in Python.
The argument about functional vs imperative languages is old, tired and is not relevant to the subject. Please at least learn some fundamentals about NNs and ML frameworks before contributing a long, uninformed response.
- K0SM0S 7y ago> the concurrency model of these ML libraries is not dependent on a CPU Gotcha, my bad. > Please at least learn some fundamentals about NNs and ML frameworks before contributing a long, uninformed response. My apologies. It's not like I enjoy wasting people's time, starting with mine. It was an honest mistake, point taken, and thanks for explaining. If you have a somewhat 'definitive' book or chapter/resource to share on "understanding concurrency"... please do so. I'm eager to learn, and I'd wager many reading such a thread.
- codesushi42 7y agoSee Udacity's course on parallel programming: https://www.youtube.com/watch?v=F620ommtjqk&list=PLGvfHSgImk4aweyWlhBXNF6XISY3um82_ https://www.youtube.com/watch?v=F620ommtjqk&list=PLGvfHSgImk... I don't think it touches upon ML, but it is relevant for understanding the difference between GPUs and CPUs. You can go through the intro to backprop lectures on deeplearning.ai for ML.