4 ms·
LLM based chatbots are already more useful than all of crypto combined. Whereas crypto wastes energy as "proof of work" AI models try to be as energy-efficient
by gizmo 4y ago
LLM based chatbots are already more useful than all of crypto combined. Whereas crypto wastes energy as "proof of work" AI models try to be as energy-efficient as possible. The energy needed to train these LLMs is also 10,000 times less, so let's not act like they're in the same universe.
Moreover, training the base layer of a LLM has to be done only once (for each generation) and subsequent training for domain specific purposes can happen on top of the base layer. This means these AI startups are not going to require an unlimited number of GPUs.
Another big difference is that fundamental AI insights are what drives these AI innovations. Throwing more training data at a neural network is not what makes ChatGPT different from older generations of chatbots (although ChatGPT certainly does benefit from its huge training data set). Transformers for example are a new development, first introduced in a 2017 paper "Attention is all you need".
It's unclear if LLMs will turn out to be a big deal. Maybe this will turn out just to be another AI summer. Too early to tell. But cynicism is unwarranted here.
- gessha 4y ago> Another big difference is that fundamental AI insights are what drives these AI innovations. Bigger models, more compute, more data, better compute have driven a lot of the performance of ML post-2013. For ChatGPT specifically, the secret sauce is the addition of human-vetted conversations [1]. In general fundamental AI insights come when - (1) somebody does something that works incredibly well and others try to explain it. Transformers are arguably an example of this although there’s a ton of attention literature before that. - (2) a ton of people make incremental steps towards a goal and at some point it becomes actually useful. ChatGPT is an example of this one. The point I’m trying to make is fundamental AI is boring, slow, rarely drives AI innovations and that’s not a bad thing. What drives AI innovations is really good UI and good data both of which are hard to nail down. [1] https://openai.com/blog/chatgpt/#methods https://openai.com/blog/chatgpt/#methods
- incrudible 4y agoWritten by an LLM.