4 ms·

Can I ask why it's effective to tell it to get it's math correct? Two sub-questions: 1) I thought LLMs worked by predicting the most likely next token. Why wo

Can I ask why it's effective to tell it to get it's math correct? Two sub-questions: 1) I thought LLMs worked by predicting the most likely next token. Why would telling it to check it's math actually make it's predictions more accurate? 2) Why would asking it to check it's math change the results it produces? Shouldn't it just do the same thing a second time and produce answers that are just as (in)accurate as it did the first time?

mintplant 4y ago
My intuition is: the AI is trying to predict a likely response to the input query given what it's seen in the dataset it's been trained on. That dataset will include instances where math has been done incorrectly, so the AI producing a response with incorrect math isn't inherently "wrong" from its perspective. However, queries containing reminders to be careful with the math are more likely to be followed by responses with correct numbers.
- hansworst 4y ago
  So it’s pretending to be an average human, and the average human gets it wrong a lot? I wonder if you can improve coding results by telling it to act as an expert programmer?
  - sharemywin 4y ago
    I told it to sound 100% more human when writing a blog post and it moves the score with the AI detectors. Then I told it to sound even more human and it does even better.
  - the_gipsy 4y ago
    It's not "pretending". As the comment said, the dataset has something like "paths" and that prompt narrows it down to those that (hopefully) are related to "getting the math right".
  - kaetemi 4y ago
    I noticed as well that by default the output is effectively average. The right context bias does help with getting better quality output.
    - sharemywin 4y ago
      You can also have it check it's self after the fact. which is the advantage of this over GPT3.
  - thecupisblue 4y ago
    Imagine you are navigating a tiny robot on mars, or let's say a giant land region with hills and valleys. Saying it "split it evenly" and giving it numbers is something like saying "find me the hill next to this valley". This will lead you to the valley on the left, then to the tiny hill right behind it. The hill right behind it in this example is where all the weights about splitting stuff evenly are hanging around. Giving it "split evenly and make sure the math is correct" is like saying "find me the the hill next to this valley, but make sure it's the tallest". So it will lead you to the same valley, but then to a different hill, because of adding "tallest" it will lead you to the intersection of where all the weights about splitting stuff evenly and correct math were found. I.e. there is other stuff you can engineer about - saying stuff like "Write the result step by step and check for correctness." afterwards will navigate it to the same hill but then towards the "all math tests are here" hill which is where step by step calculations weights are and it will find it's new home there, giving you a more detailed output and more chances to be right since the prediction is easier - basically it "splits" the task into smaller, easier to predict chunks, like you split ugly code into small testable functions to understand it.
visarga 4y ago
Yes, it's weird this happens. It's all because this is a conditional model. So it can be influenced by anything in the prompt, including the persona it is assuming. But it is not deterministic. For each token it picks at random, from a probability distribution generated by the neural network. You can make it deterministic if you set temperature=0. In general it is stochastic, if you sample again with the same prompt you get a different answer.

mintplant 4y ago

My intuition is: the AI is trying to predict a likely response to the input query given what it's seen in the dataset it's been trained on. That dataset will include instances where math has been done incorrectly, so the AI producing a response with incorrect math isn't inherently "wrong" from its perspective. However, queries containing reminders to be careful with the math are more likely to be followed by responses with correct numbers.

hansworst 4y ago
So it’s pretending to be an average human, and the average human gets it wrong a lot? I wonder if you can improve coding results by telling it to act as an expert programmer?
- sharemywin 4y ago
  I told it to sound 100% more human when writing a blog post and it moves the score with the AI detectors. Then I told it to sound even more human and it does even better.
- the_gipsy 4y ago
  It's not "pretending". As the comment said, the dataset has something like "paths" and that prompt narrows it down to those that (hopefully) are related to "getting the math right".
- kaetemi 4y ago
  I noticed as well that by default the output is effectively average. The right context bias does help with getting better quality output.
  - sharemywin 4y ago
    You can also have it check it's self after the fact. which is the advantage of this over GPT3.
- thecupisblue 4y ago
  Imagine you are navigating a tiny robot on mars, or let's say a giant land region with hills and valleys. Saying it "split it evenly" and giving it numbers is something like saying "find me the hill next to this valley". This will lead you to the valley on the left, then to the tiny hill right behind it. The hill right behind it in this example is where all the weights about splitting stuff evenly are hanging around. Giving it "split evenly and make sure the math is correct" is like saying "find me the the hill next to this valley, but make sure it's the tallest". So it will lead you to the same valley, but then to a different hill, because of adding "tallest" it will lead you to the intersection of where all the weights about splitting stuff evenly and correct math were found. I.e. there is other stuff you can engineer about - saying stuff like "Write the result step by step and check for correctness." afterwards will navigate it to the same hill but then towards the "all math tests are here" hill which is where step by step calculations weights are and it will find it's new home there, giving you a more detailed output and more chances to be right since the prediction is easier - basically it "splits" the task into smaller, easier to predict chunks, like you split ugly code into small testable functions to understand it.

visarga 4y ago

Yes, it's weird this happens. It's all because this is a conditional model. So it can be influenced by anything in the prompt, including the persona it is assuming. But it is not deterministic. For each token it picks at random, from a probability distribution generated by the neural network. You can make it deterministic if you set temperature=0. In general it is stochastic, if you sample again with the same prompt you get a different answer.