3 ms·
To be fair parent pointing out they're purely statistical machines predicting next token is incorrect anyways. They are essentially next token predictors after
by fennecfoxy 2y ago
To be fair parent pointing out they're purely statistical machines predicting next token is incorrect anyways.
They are essentially next token predictors after first training, but then instruct models are fine tuned on reasoning and Q/A scenarios, afaik early research has determined that this isn't just pure parroting, that it does actually result in some logic in there as well.
People also have to remember the training for these is super shallow at the moment, when compared with what humans go through in our lifespans as well as our millions of years of evolution (as humans).
- globnomulous 2y agoWhat you're saying doesn't contradict what I wrote. I said models are trained. You're saying they're trained and fine tuned -- i.e. continue to be trained. I also didn't say they do any kind of parroting or that logic doesn't take place. I'm saying, rather, that the models do what they're taught to do, and what they're taught to do are computations that give us a result that looks like reasoning, just the way I could use 3ds max as a teenager to generate on my computer screen an output that looked like a cube. There was never an actual cube in my computer when I did that. To say that the model is reasoning because what it does resembles reasoning is no different from saying there was an actual cube somewhere in my computer every time I rendered one.