3 ms·
Agree, but much less than 10 years. Now that the transformer is establishing itself as the model, we’ll see dedicated HW acceleration for transformers, encoders
by brookst 4y ago
Agree, but much less than 10 years. Now that the transformer is establishing itself as the model, we’ll see dedicated HW acceleration for transformers, encoders, decoders, etc. I will eat my hat* if we don’t see local inference for 200B+ parameters within 5 years.
* I don’t own a hat
- senttoschool 4y agoWhich chip company do you think will benefit from the move to transformers?
- flakeoil 4y agoNo-one in particular. It will be integrated in the processors (the intels/AMDs on PC, the Mx on Macs and the Qualcomms, Mediateks and Samsungs on phones). Not much different from video and graphics accelerators being integrated today, or DSP focused instructions in the instruction set. They just have to do it to stay relevant.
- chii 4y agoi would imagine these models would be considered trade secrets, and that the models (esp. good ones that take a lot of resources to have trained) would not leave the data center. Your access to such models would be dictated by an api instead of locally run.
- jononor 4y agoNot sure about 200B+ models on phone hardware in 10 years. But I think we'll be able to deliver the same predictive performance with models 1/10 the size and those will fit. That is what happened with CNN vision models over the last 8 years.