GPUs are good for algorithms with simple control flow (take conventional rendering where you basically does simple processing of huge amounts of vertices and pi

GPUs are good for algorithms with simple control flow (take conventional rendering where you basically does simple processing of huge amounts of vertices and pixels). If you have an algorithm with complex control flow (e.g. ray tracing, most problems in science) you will have to code really hard to achieve even 50% of peak. You can usually do something with an algorithm which is good for GPU but not optimal overall. So you have 30x speedup on GPU compared to CPUs but the scaling of this algorithm is worse then its more complex variant on CPU (e.g. O(n) vs O(log(n))). In the end many of these speedups are just marketing. Moreover GPUs have problems with double precision floating point numbers which are a must for most scientific problems (though sometimes you can utilize mixed precision approaches -> estimate in single precision, then get a correction in double). Another problem is having ECC (error correction). You don't want to run a computation for a week to find out that it crashed because of data corruption. Latest generation of Teslas has ECC, consumer cards don't.