5 ms·
True that, but I think in a very short amount of time, using dedicated general purpose GPUs just for inferencing is going to be mega overkill. If there's dedic
by LarsDu88 2y ago
True that, but I think in a very short amount of time, using dedicated general purpose GPUs just for inferencing is going to be mega overkill.
If there's dedicated inferencing silicon (like say the thing created by Groq), all those GPUs will be power sucking liabilities, and then the REAL singularity superintelligence level training can begin.
- campers 2y agoEtched is another dedicated inference hardware company that recently announced their product. It only works for transformer based models, but is ~20x faster than a H100