3 ms·

> they can maybe do 5+5 because it shows up somewhere in the data, but then they can't do 3792 + 29382 I'd say you're underselling modern AIs. GPT-3 can come c

> they can maybe do 5+5 because it shows up somewhere in the data, but then they can't do 3792 + 29382 I'd say you're underselling modern AIs. GPT-3 can come close, as can GPT-J (which is publicly available and therefore perhaps easier to prove), even if they don't quite get the right answer 100% of the time. I gave GPT-J the following prompt (which you can try yourself at https://6b.eleuther.ai/ https://6b.eleuther.ai/ just wait a few min for an answer) What is 1 + 1? 2. What is 3 + 2? 5. What is 6984 + 120? 7104. What is 3792 + 29382? And it responded with 33374 What is -0.084435 + -0.3? -0.384435 What is -4.1 + -0.4? -4.5 ... [more of the same omitted] Note that it got almost the right answer! Close enough that if it was a human I would be tempted to call it human error. GPT-3 can do even better. EDIT: On another try GPT-J got the answer! Not bad (and GPT-3 is an even more powerful model). 33174 -3 + -91.4 -94.4 What is -1.6 + 13.81? 12.21

tuatoru 5y ago
Prompt:- Jon has six mangoes, and he needs ten rupees. Only Alice wants to buy any mangoes. Alice wants to buy three. How much does Jon sell them for? Response:- Jon has six mangoes, and he needs ten rupees. Only Alice wants to buy any mangoes. Alice wants to buy three. How much does Jon sell them for? 100 rupees. Q: What is the smallest number that can be written as 1/2 x 5 x 7? A: 6,550 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11? A: 6,550,000 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13? A: 6,550,000,000 Q: How many ways can you make 50 if you use five different colored marbles? A: 10 Q: How many ways can you make 75 using only three different colored marbles? A: 12 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13? A: 6,550,000,000 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13 x 15? A: 6,550,000,000,000 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13 x 15 x 17? A: 6,550,000,000
- tuatoru 5y ago
  Commentary. The first line of the answer is part of an appropriate answer for this linguistic formula given as a prompt. An appropriate response answers the question, and (ideally) supplies a brief warrant for the answer. (A warrant is a reasoned argument as to why the answer is correct.) The rest of the answer is just noise. Not much evidence of linguistic understanding, despite the enormous corpus ingested by the tool.
hackinthebochs 5y ago
You have to be careful when analyzing GPT-3 and derivatives due to BPE encoding. It's accuracy greatly improves when adding commas to strings of numbers: https://www.gwern.net/GPT-3#fn21 https://www.gwern.net/GPT-3#fn21

tuatoru 5y ago

Prompt:- Jon has six mangoes, and he needs ten rupees. Only Alice wants to buy any mangoes. Alice wants to buy three. How much does Jon sell them for? Response:- Jon has six mangoes, and he needs ten rupees. Only Alice wants to buy any mangoes. Alice wants to buy three. How much does Jon sell them for? 100 rupees. Q: What is the smallest number that can be written as 1/2 x 5 x 7? A: 6,550 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11? A: 6,550,000 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13? A: 6,550,000,000 Q: How many ways can you make 50 if you use five different colored marbles? A: 10 Q: How many ways can you make 75 using only three different colored marbles? A: 12 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13? A: 6,550,000,000 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13 x 15? A: 6,550,000,000,000 Q: What is the smallest number that can be written as 1/2 x 5 x 7 x 11 x 13 x 15 x 17? A: 6,550,000,000

tuatoru 5y ago
Commentary. The first line of the answer is part of an appropriate answer for this linguistic formula given as a prompt. An appropriate response answers the question, and (ideally) supplies a brief warrant for the answer. (A warrant is a reasoned argument as to why the answer is correct.) The rest of the answer is just noise. Not much evidence of linguistic understanding, despite the enormous corpus ingested by the tool.

hackinthebochs 5y ago

You have to be careful when analyzing GPT-3 and derivatives due to BPE encoding. It's accuracy greatly improves when adding commas to strings of numbers: https://www.gwern.net/GPT-3#fn21 https://www.gwern.net/GPT-3#fn21