3 ms·

GPT4 can double-check to an extent. I gave it a sequence of 67 letter As and asked it to count them. It said "100", I said "recount": 98, recount, 69, recount,

GPT4 can double-check to an extent. I gave it a sequence of 67 letter As and asked it to count them. It said "100", I said "recount": 98, recount, 69, recount, 67, recount, 67, recount, 67, recount, 67. It converged to the correct count and stayed there. This is quite a different scenario though, tangential to your [correct] point.

kordlessagain 3y ago
The example of asking it things like counting or sequences isn't a great one because it's been solved by asking it to "translate" to code and then run the code. I took this up as a challenge a while back with a similar line of reasoning on Reddit (that it couldn't do such a thing) and ended up implementing it in my AI web shell thing. heavy-magpie|> I am feeling excited. system=> History has been loaded. pastel-mature-herring~> !calc how many Ns are in nnnnnnnnnnnnnnnnnnnn heavy-magpie|> Writing code. // filename: synth_num_ns.js // version: 0.1.1 // description: calculate number of Ns var num_ns = 'nnnnnnnnnnnnnnnnnnnn'; var num_Ns = num_ns.length; Sidekick("There are " + num_Ns + " Ns in " + num_ns + "."); heavy-magpie|> There are 20 Ns in nnnnnnnnnnnnnnnnnnnn.
- mikestew 3y ago
  Shouldn’t the answer be zero?
  - suddenclarity 3y ago
    Thanks. I'll steal this and refer to it in the future as an example of ambiguous project orders.
- mcv 3y ago
  Oh god, it's even worse at naming things than people are.
einpoklum 3y ago
But would GPT4 actually check something it had not checked the first time? Remember, telling the truth is not a consideration for it (and probably isn't even modeled), just saying something that would typically be said in similar circumstances.
- simonw 3y ago
  Only in as much as there's an element of randomness to the way GPT responds to a prompt - so you can re-run effectively the same prompt and get a different result depending on the outcome of several hundred billion floating point calculations with a random seed thrown in.

kordlessagain 3y ago

The example of asking it things like counting or sequences isn't a great one because it's been solved by asking it to "translate" to code and then run the code. I took this up as a challenge a while back with a similar line of reasoning on Reddit (that it couldn't do such a thing) and ended up implementing it in my AI web shell thing. heavy-magpie|> I am feeling excited. system=> History has been loaded. pastel-mature-herring~> !calc how many Ns are in nnnnnnnnnnnnnnnnnnnn heavy-magpie|> Writing code. // filename: synth_num_ns.js // version: 0.1.1 // description: calculate number of Ns var num_ns = 'nnnnnnnnnnnnnnnnnnnn'; var num_Ns = num_ns.length; Sidekick("There are " + num_Ns + " Ns in " + num_ns + "."); heavy-magpie|> There are 20 Ns in nnnnnnnnnnnnnnnnnnnn.

mikestew 3y ago
Shouldn’t the answer be zero?
- suddenclarity 3y ago
  Thanks. I'll steal this and refer to it in the future as an example of ambiguous project orders.
mcv 3y ago
Oh god, it's even worse at naming things than people are.

einpoklum 3y ago

But would GPT4 actually check something it had not checked the first time? Remember, telling the truth is not a consideration for it (and probably isn't even modeled), just saying something that would typically be said in similar circumstances.

simonw 3y ago
Only in as much as there's an element of randomness to the way GPT responds to a prompt - so you can re-run effectively the same prompt and get a different result depending on the outcome of several hundred billion floating point calculations with a random seed thrown in.