3 ms·
> but that also involves an exponential increase in cost Not necessarily. For training, yes. But there surely will be custom hardware specialized for the infer
by xpl 2y ago
> but that also involves an exponential increase in cost
Not necessarily. For training, yes. But there surely will be custom hardware specialized for the inference of "1-bit" quantized models, which could be orders of magnitude more energy-effective than GPUs.
Hell, you probably can even bake trained models in silicon. Maybe instead of digital logic, use analog circuits. Photonics?
I believe there's so much room for improvement and I feel that GPUs are just the "training wheels" (lol, quite literally) — there will be exponentially more effective hardware for running LLMs.
- metadat 2y agoLLMs aren't even relevant in Kurzweil's Singularity vision, they are just a current trend. They may pan out, or something else will.
- sottol 2y agoEven asic will imo only give us one step-function jump. there will be a huge jump and then it is mostly back to moore's law-ish. maybe the gradient will be a bit steeper but i doubt it.
- saulpw 2y agoIf blockchain is any indication, ASICs give two jumps (2 orders of magnitude) over GPUs. But yeah, then further jumps will have to be via parallelism; can we put 100s or 1000s of these ASICs in a single computer?