Non paywall: https://archive.ph/XwWTi https://archive.ph/XwWTi Imo, it is true that the current architecture is hitting the limit. We need a breakthrough on th

Non paywall: https://archive.ph/XwWTi https://archive.ph/XwWTi Imo, it is true that the current architecture is hitting the limit. We need a breakthrough on the scale of the transistor to get past this problem. We know it is possible though. Every single human is proof that high performance AI can be run with less energy than a laptop. We just need a dedicated architecture for their working mechanisms the same way a transistor is the embodiment of 1 and 0. Unfortunately, in terms of understanding intelligence and how they work, I don't think we have made any significant advance in the last few decades. Maybe with better tools at probing how the LLM work, we can get some new insights.