3 ms·
We're likely way too early to call this one. Based on current hardware trends, GPT-4 will run on your phone within 6-10 years. Right now, folks are feeling out
by lumost 3y ago
We're likely way too early to call this one. Based on current hardware trends, GPT-4 will run on your phone within 6-10 years. Right now, folks are feeling out the edges of what makes sense and what doesn't make sense at extreme R&D and opex expense. In 10 years, we'd expect the winners of today to still be winners and have great margins due to declining compute costs.
Granted, if you are spending 10x the future value of the product your are offering ... then even a 10x decline in compute costs won't get you where you need to be.
- riku_iki 3y ago> Based on current hardware trends, GPT-4 will run on your phone within 6-10 years they quantized model from 16 bits to 4 bits which was low hanging fruit, and looks like they can't quantize it anymore to 2 bits..
- lumost 3y agoCPU/GPUs are general purpose, if enough workload demand exists specialized Transformer cores will be designed. Likewise, its not at all clear that current O(N^2) self-attention is the ideal setup for larger context lengths. All to say, I'd believe we have another 8-10x algorithmic improvement in inference costs over the next 10 years. In addition to whatever Moore's law brings.
- westurner 3y agoMobile TPUs/NPUs: Pixel 6+ phones have TPUs (in addition to CPUs and an iGPU/dGPU). Tensor Processing Unit > Products > Google Tensor https://en.wikipedia.org/wiki/Tensor_Processing_Unit https://en.wikipedia.org/wiki/Tensor_Processing_Unit TensorFlow lite; tflite: https://www.tensorflow.org/lite https://www.tensorflow.org/lite From https://github.com/hollance/neural-engine https://github.com/hollance/neural-engine : > The Apple Neural Engine (or ANE) is a type of NPU, which stands for Neural Processing Unit. From https://github.com/basicmi/AI-Chip https://github.com/basicmi/AI-Chip : > A list of ICs and IPs for AI, Machine Learning and Deep Learning