You are somewhat right. You can't manipulate the _same_ memory address concurrently, but normally in multi-threaded programs your not doing that. Consider a tra

You are somewhat right. You can't manipulate the _same_ memory address concurrently, but normally in multi-threaded programs your not doing that. Consider a traditional matrix multiplication program. You will pass each row/column pair to a core each pair occupying its own memory space ( or an address to a common memory pool that's read only), the core will perform the multiplication then it will pass that back to the parent to aggregate the result. The limiting factor in that situation is. Either the speed of the aggregating core ( you can sometimes parallelize that away by having multiple aggregation steps ) , or the amount of time it take to get all the data to the cores.