Each CUDA "core" is actually a lane in a 16-wide SIMD processor, so it has 20 CPUs in the traditional sense. (Intel CPUs have 4-way SIMD with SSE) Only one "pro

Each CUDA "core" is actually a lane in a 16-wide SIMD processor, so it has 20 CPUs in the traditional sense. (Intel CPUs have 4-way SIMD with SSE) Only one "program" (kernel) could run on the older GPUs in parallel, on newer GPUs you can have have all the "CPUs" running different programs, but instruction cache space is quite limited.