3 ms·
That's dodging my question. I know theyre just predicting the next word, but that doesn't explain how that works so amazingly well in this case. There's a few s
by fnqi8ckfek 2y ago
That's dodging my question. I know theyre just predicting the next word, but that doesn't explain how that works so amazingly well in this case. There's a few steps missing.
If anything this seems like an ad for an AI. When I try these things they can even solve leetcode, for which they actually have the solution in the training set.
- baggy_trough 2y agoTo predict the next word, you need some model of the universe.
- dartos 2y agoThe intuition I’ve been building around LLMs is that they’re like search systems. They encode their training data in their weights and do a kind of search on it given a prompt. If leetcode problems are in their training data, then they’ll of course answer the question with ease.
- aiven 2y agoEvery solvable problem is solved by using known information, patterns, context, etc. We (and LLMs) are using some model of the universe and trying to coordinate it in a way that will help us solve some task. The difference between us and "old" LLMs is that we can generate new information/patterns/etc., immediately add it to our model, and use it to solve more complex problems. New LLMs such as o1-o3 are also capable of thinking over and over again and producing new information (in the current context) and trying to apply it to the current task that might not be solvable with just the information that it was trained on. (This is my understanding, I’m not ml engineer)