2 ms·
So you believe humans spend more energy on prediction, relative to computers? Isn't that because personal computers are not powerful enough to train big models,
by esafak 2y ago
So you believe humans spend more energy on prediction, relative to computers? Isn't that because personal computers are not powerful enough to train big models, and most people have no desire to? It is more economically efficient to socialize the cost of training, as is done. Are you thinking of a distributed training, where we split the work and cost? That could happen when robots become more widespread.
- bobosha 2y agoThe human brain operates at just 25W of power—less than the monitor you're likely using right now—whereas AI models like ChatGPT consume nearly 1GWh every 24 hours! As I discuss in the paper, predictive coding suggests that the brain actively generates predictions and compares them to incoming sensory data (vision, hearing, etc.), prioritizing anomalies. Its efficiency stems from a hierarchical memory system that continuously updates only the "deltas"—the differences that matter. Embracing this approach could lead to a paradigm shift, enabling the development of significantly more energy-efficient AI in the future.