2 ms·
This is correct. Current LLMs work by predicting the next word based on a bunch of preceding words. In other words, they are autocomplete. You can often form a
by immibis 3y ago
This is correct. Current LLMs work by predicting the next word based on a bunch of preceding words. In other words, they are autocomplete. You can often form a valid sentence on your phone if you click on any text field and then press the automatic suggestions several times.
Transformer-based LLMs are interesting because they are such good version of autocomplete that they can, for example, complete a news article about scientists discovering unicorns, using just the first sentence (this was one of the first public demonstrations of GPT-2). But fundamentally they are still just auto-complete.
- astrange 3y agoLLM chat models are not autocomplete. They can recognize and respond to user text, which is not the same thing as completing it. If you prompted GPT-2 with a question you'd get another question, not an answer.
- IanCal 3y agoThey are autocomplete, they're just completing something in the form Assistant: ... User: ... Assistant: ... And the output is stopped when they start generating the equivalent of "User: " and the reins are handed back to you. This isn't a problem, autocomplete at the level of "what would a person say next" is outrageously powerful, but it is how they're working afaik.