I can see it. I mean, just multithreading embarrassingly parallel loops could often offer drastic speedups - the reason it's not generally done at the compiler

I can see it. I mean, just multithreading embarrassingly parallel loops could often offer drastic speedups - the reason it's not generally done at the compiler level is due to overhead problems (by the time you're done creating threads / running them / cleaning them up / doing context switches and memory stalls you might as well have done the problem the straightforward way). If you're doing things at the hardware level you can get substantially less overhead. In other words, don't think of it as "this can always multithread" - think of it along the lines of "this can pick up on common programming idioms and emit corresponding SIMD instructions". On a related note: this sort of thing is one of the reasons I wish that instead of SSE we had something along the lines of a forall {chunk of code} instruction, (with the limitations that the code is executed in an unspecified order, and that the code cannot branch). Explicitly hardcoding the number of parallel data streams is... shortsighted.