4 ms·
Data races on normal memory access are undefined behaviour in many languages (e.g. C, C++, Rust), so I assume that you mean races using atomic loads with relaxe
by CodesInChaos 6y ago
Data races on normal memory access are undefined behaviour in many languages (e.g. C, C++, Rust), so I assume that you mean races using atomic loads with relaxed ordering.
One practical example of using those is Hogwild, which is an implementation of stochastic gradient descent which skips synchronization on individual parameter updates, assuming that it'll converge even if some writes are lost or some reads are stale.
https://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent https://papers.nips.cc/paper/4390-hogwild-a-lock-free-approa...