4 ms·
So in 50 years the equivalent of a gpt4 training cluster from today's datacenters will fit in a cheap cable, and it will run over 100 times faster than a full c
by daxfohl 3y ago
So in 50 years the equivalent of a gpt4 training cluster from today's datacenters will fit in a cheap cable, and it will run over 100 times faster than a full cluster today.
- FredPret 3y agoComputronium
- ko27 3y agoYeap, that's how exponential growth works. It just never stops.
- daxfohl 3y agoI'd say even more impressive is that, given an ARM Cortex X4 runs at almost 100x the clock speed, up to 14 cores, has caches, branch prediction, floating point, specialized instructions, gpu, etc., it's probably 100000 times faster than the CYPD4225. So given GPT4 training took ~100 days, it means a typical cell phone equivalent in 50 years will be able to train a GPT4 equivalent from scratch in under a second.
- daxfohl 3y agoWhat hardware giveth, software taketh away. JS frameworks of the 2070s will find some reason to fully retrain an LLM on each keypress.