4 ms·
>As a simple example I recommend trying to have it multiply two large numbers, it does a decent job guestimating but obviously can't do the recursive steps requ
by sebzim4500 4y ago
>As a simple example I recommend trying to have it multiply two large numbers, it does a decent job guestimating but obviously can't do the recursive steps required to generate a correct answer.
Is it worse than the average human at mental arithmetic? I haven't done detailed benchmarks, but for my usecase it only gets maths wrong once it gets way past the range that I could do myself.
Personally, I find the fact that it can't say "I don't know" to be much more offputting than the dodgy maths.
- ftxbro 4y ago> Personally, I find the fact that it can't say "I don't know" to be much more offputting This is a problem with their PPO "finishing school" not with their base model. Check Figure 8 in https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf its base model has an uncannily well calibrated confidence in the correctness of its decisions. Basically, their PPO is for training it to talk to normies, and normies don't like uncertainty. Reinforcement learning using Proximal Policy Optimization (PPO): They optimize a policy against the reward model using reinforcement learning. For each new prompt sampled from the dataset, the policy generates an output. The reward model calculates a reward for the output, and the reward is used to update the policy using the PPO algorithm.
- sebzim4500 4y agoTrue, but even with the base model it is hard to go from logits to a confidence level for freeform answers. For multiple choice questions it is easy, since you just look at the probabilities for each of the allowable tokens in the answer (which is presumably what they did in that benchmark) but I have no idea how to do it more generally.