4 ms·
Yes, but it’s meant to do ML inference, which can be parallelized decently. On those workloads, you can use GPUs, which are also composed of thousands of “wimpy
by mirker 5y ago
Yes, but it’s meant to do ML inference, which can be parallelized decently. On those workloads, you can use GPUs, which are also composed of thousands of “wimpy” cores.
- R0b0t1 5y agoA full CPU is useful for decision intensive or time series intensive data. Normal ML inference is not necessarily either of those. You could have more complicated neurons (or just make normal compute tiles which they may be doing).
- goldenkey 5y agoI thought the same thing back in 2015 considering the way GPUs supposedly handle branches with warps. However, my stock trading simulator ran way better on 3 GTX Titans rather than the Intel "Knights Many Cores" Phi preview I had exclusively been able to obtain. I was excited because it had something like 100 pentium 4 cores on it, and was supposed to be much faster than a GPU for logical code. Dissapointment set in when the GPU stomped it performance wise. I still don't even understand why but I do know now that the whole "GPUs can't handle branching performantly" is a bit overstated. Intel discontinued their Phi which I can only gander was due to its lack of competitiveness.
- sdenton4 5y agoA standard way to handle branching in gpu code is with masking, like so (where x is a vector, and broadcasting is implied): M = x > 0 y = M * f(x) + (1-M) * g(x) So you end up evaluating both sides of the decision branch and adding the results. But this is fine if you've got a dumb number of cores. And often traditional cpus wind up evaluating both branches anyway.
- monocasa 5y ago> And often traditional cpus wind up evaluating both branches anyway. That's actually really overstated. Evaluating both sides isn't really something CPUs tend to do, but instead predict one path and roll back on mispredict. This is because the out of order hardware isn't a tree fundamentally, but generally better thought of a ring buffer where uncommitted state is what's between the head and tail. Storing diverging paths is incredibly expensive there. I'm not going to say something as strong as "it's never been done", but but I certainly don't know of an general purpose CPU arch that'll compute both sides of an architectural branch, instead relying an making good predictions down one instruction stream, then rolling back and restarting when it's clear you mispredicted.
- jasonwatkinspdx 5y agoIt's not even about the expense of implementing diverging paths in hardware. This concept, of exploring like a tree vs a path was explored under the name Disjoint Eager Execution. You know what killed it? Branch predictors. In a world where branch predictors are maybe only 75% effective, DEE could make sense. We live in a world where branch predictors are far better than that. So it just isn't worth speculating off the predicted most likely path.
- monocasa 5y agoWhat killed it was more the effectiveness of Tomasulo style out of order machinery, and the real problem not being control hazards, but data hazards. DEE was thought of in a day where memory was about as fast as the processor. That's why it's always being compared with cores like R3000.
- R0b0t1 5y agoYours isn't really the problem a massive core system is for. I truly don't know what is, but they are out there.
- monocasa 5y agoSort of. GPU "cores" in the CPU space would be called SIMD lanes. Apples to apples GPU cores using the CPU terminology would put an Nvidia 3060 at 28 cores and a 3090 at 82 cores.