This is actually very subtle. Intel's OpenCL stack is currently atrocious, but if they ever make it decent, it will be able to vectorize. The problem is that

This is actually very subtle. Intel's OpenCL stack is currently atrocious, but if they ever make it decent, it will be able to vectorize. The problem is that vectorization is only part of the equation and many (likely most) applications today are limited in some way by memory and/or communication rather than by vectorization. Meanwhile, vectorization is generally at odds with memory bandwidth. CPU cache, shared memory on a GPU, and registers on a GPU are limited and vectorization usually leads to less effective use of these precious resources. Additionally, many kernels have inconvenient dependencies (e.g., an 30-term equation of state, a 70-species reaction mechanism) that force extremely low occupancy for typical GPU implementations (e.g., one thread per quadrature point). If you try to make the parallelism finer grained to reduce the register demands (break the quadrature point into several parts), you either need physics-dependent decomposition within a thread block or multiple kernel invocations (higher latency and needs to reload from memory). All of these things are counter-productive for strong scaling. Recall that a lot of applications are run near the limit of strong scaling and that CPU network latencies are in the microsecond range while GPU kernel launch overheads start at 20 microseconds (and you likely need several of them per inter-node communication). While it's true that there are kernels that are easier to optimize for a GPU, I still think most claims of huge speedup are due to neglecting the CPU implementation. See "Debunking the 100x GPU vs CPU Myth" and other papers on this. And amidst all of that, there are a lot of applications that utterly fail on GPUs, despite having lots of parallelism, just not at quite the right granularity.