4 ms·
GPUs are mainly good at doing matrix multiplication and other math with little branching. Compilers have a lot of if/else statements; training neural networks d
by the_pwner224 6y ago
GPUs are mainly good at doing matrix multiplication and other math with little branching. Compilers have a lot of if/else statements; training neural networks does not. IIRC (and I may be wrong) the architecture is something like many GPU cores share things like the code cache and branch prediction, so if the different cores have to execute different code paths there is a big performance decrease.
Edit: another reply to this top level comment: https://news.ycombinator.com/item?id=23788755 https://news.ycombinator.com/item?id=23788755
- Symmetry 6y agoAlso GPUs really like predictable memory access patterns so that they can load the data that'll be needed next ahead of time into registers or dedicated caches explicitly. They don't have cache hierarchy features a CPU has to minimize the average disruption from only partially predictable memory accesses and they don't have out of order windows to paper over the occasional low level cache miss.
- FpUser 6y agoNot to counter the main argument but CPU loves it some predictable patterns as well. pattern like the one below will execute up to 10 times faster if array is sorted due to data locality and branch prediction sum: float a: float array[some_huge_number] fill_array(a, random(1)) for f in a if a[i]>0.5 sum=sum+f