3 ms·

GPT-4 is pretty amazing but I, too, feel this is being overhyped. For me, a sobering example is how OpenAI does math (eg [1]). Specifically, the model clearly d

by jmyeet 3y ago

GPT-4 is pretty amazing but I, too, feel this is being overhyped. For me, a sobering example is how OpenAI does math (eg [1]). Specifically, the model clearly doesn't really understand multiplication and "learns" it from training data. This tends to get the first few and last few digits right for a simple multiplication with 6-7 digit numbers. Now you can solve that with plugins (eg training the model to recognize math problems and have access to a calculator) so it's a solvable problem but you realize there's an extremely long tail of such problems. It goes to show that GPT-4 isn't "magic" and we still have a long way to go. [1]: https://www.reddit.com/r/OpenAI/comments/12donja/gpt4_and_math/ https://www.reddit.com/r/OpenAI/comments/12donja/gpt4_and_ma...

seanhunter 3y ago
Most of the time when people find a maths problem that they can trick the model into getting wrong, it's also possible to get the model to give the correct answer with better prompting. A trick that's worth knowing is just to ask the model to give each step in the solution and explain as it goes. This gives the model "time to think" and leads to better results.
- usaar333 3y ago
  Pretty sure you can't get GPT-4 to do 8 digit multiplication with any prompt. For what it's worth, I'm not even sure if chain of thought provides much value to GPT-4. The RLHF it went through seems to have encouraged more logical thinking already.
  - morkalork 3y ago
    Now you have me wondering if you could prompt it to solve it step by step, using the same method an elementary student would do on paper.
    - seanhunter 3y ago
      You can. One thing that I've found fun is to prompt it for some maths problems without solutions, then I provide solutions and for any that I get wrong ask it to explain my mistakes.
  - seanhunter 3y ago
    The deeplearning.com prompt engineering course contains an example where GPT4 solves a problem with chain of thought and fails without. The prompt is Determine if the student's solution is correct or not. Question: I'm building a solar power installation and I need help working out the financials. - Land costs $100 / square foot - I can buy solar panels for $250 / square foot - I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot What is the total cost for the first year of operations as a function of the number of square feet. Student's Solution: Let x be the size of the installation in square feet. Costs: 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 100x Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000 GPT-4 will say the student is correct even though the answer includes an error (they should be multiplying by 10x not 100x). If you change the prompt to include chain of thought as follows it notices the error Your task is to determine if the student's solution is correct or not. To solve the problem do the following: - First, work out your own solution to the problem. - Then compare your solution to the student's solution \ and evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you have done the problem yourself. Use the following format: Question: ``` question here ``` Student's solution: ``` student's solution here ``` Actual solution: ``` steps to work out the solution and your solution here ``` Is the student's solution the same as actual solution \ just calculated: ``` yes or no ``` Student grade: ``` correct or incorrect ``` Question: ``` I'm building a solar power installation and I need help \ working out the financials. - Land costs $100 / square foot - I can buy solar panels for $250 / square foot - I negotiated a contract for maintenance that will cost \ me a flat $100k per year, and an additional $10 / square \ foot What is the total cost for the first year of operations \ as a function of the number of square feet. ``` Student's solution: ``` Let x be the size of the installation in square feet. Costs: 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 100x Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000 ``` Actual solution: As I say this same prompt "fools" GPT3.5 and 4.
  - seanhunter 3y ago
    Btw I think you're not correct about the 8 digit multiplication. I tried: please could you multiply 12345678 by 87654321. Give your answer in steps. Each step should calculate one column at a time from right to left remembering to carry any digits which overflowed in a previous step. At the end add a final step adding the values up. And it gave me the following, which I believe to be correct: Sure, I'll be doing a long multiplication here, step by step, column by column from right to left. For brevity, I'll write the numbers to be multiplied as: A: 12345678 B: 87654321 Step 1: Multiply the 1st digit of B by A: 1 * 12345678 = 12345678 Step 2: Multiply the 2nd digit of B by A, remember to shift one position to the left: 2 * 12345678 = 24691356 (shift one position to the left, i.e., append a 0 at the end) Step 3: Multiply the 3rd digit of B by A, remember to shift two positions to the left: ... skip some of the answer because you get the gist.... 7 * 12345678 = 86419746 (shift six positions to the left, i.e., append six 0s at the end) Step 8: Multiply the 8th digit of B by A, remember to shift seven positions to the left: 8 * 12345678 = 98765424 (shift seven positions to the left, i.e., append seven 0s at the end) Now, the last step is to add all these results together: 12345678 246913560 3703703400 49382712000 617283900000 7407406800000 86419746000000 987654240000000 = 1082152022374638 So, the result of 12345678 multiplied by 87654321 is 1082152022374638.
    - usaar333 3y ago
      You might be getting lucky because that sequence is trained on (it's not very random). I repeated this experiment with bing and verified it could get your solution. But it failed on more random input: please could you multiply 92631924 by 41920318. Give your answer in steps. Each step should calculate one column at a time from right to left remembering to carry any digits which overflowed in a previous step. At the end add a final step adding the values up. Got 3880341653910192 with intermediate steps being broken.
ly3xqhl8g9 3y ago
Large language models and the transformer architecture are just the ALGOL 60 [1] of the search for general synthetic competence (intelligence is too finicky, competence is being given a task and fulfilling it successfully, in time, on budget, optimizing along the way). ChatGPT + Wolfram or other plugins are great for making transformers mathy, but the plugins must also be end-to-end machine learning architectures. To continue the high-level programming language analogy, we will probably see the C-level neural architecture in a few years once we get more embedded data (competence obtained through correlation physical object ↔ concept) with feedback loops forcing algorithms to operate under energy-restrictions (unlike the current architectures, more of a proof of concept, being careless about their energy). [1] https://en.wikipedia.org/wiki/ALGOL_60 https://en.wikipedia.org/wiki/ALGOL_60

by jmyeet 3y ago

seanhunter 3y ago

Most of the time when people find a maths problem that they can trick the model into getting wrong, it's also possible to get the model to give the correct answer with better prompting. A trick that's worth knowing is just to ask the model to give each step in the solution and explain as it goes. This gives the model "time to think" and leads to better results.

usaar333 3y ago
Pretty sure you can't get GPT-4 to do 8 digit multiplication with any prompt. For what it's worth, I'm not even sure if chain of thought provides much value to GPT-4. The RLHF it went through seems to have encouraged more logical thinking already.
- morkalork 3y ago
  Now you have me wondering if you could prompt it to solve it step by step, using the same method an elementary student would do on paper.
  - seanhunter 3y ago
    You can. One thing that I've found fun is to prompt it for some maths problems without solutions, then I provide solutions and for any that I get wrong ask it to explain my mistakes.
- seanhunter 3y ago
  The deeplearning.com prompt engineering course contains an example where GPT4 solves a problem with chain of thought and fails without. The prompt is Determine if the student's solution is correct or not. Question: I'm building a solar power installation and I need help working out the financials. - Land costs $100 / square foot - I can buy solar panels for $250 / square foot - I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot What is the total cost for the first year of operations as a function of the number of square feet. Student's Solution: Let x be the size of the installation in square feet. Costs: 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 100x Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000 GPT-4 will say the student is correct even though the answer includes an error (they should be multiplying by 10x not 100x). If you change the prompt to include chain of thought as follows it notices the error Your task is to determine if the student's solution is correct or not. To solve the problem do the following: - First, work out your own solution to the problem. - Then compare your solution to the student's solution \ and evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you have done the problem yourself. Use the following format: Question: ``` question here ``` Student's solution: ``` student's solution here ``` Actual solution: ``` steps to work out the solution and your solution here ``` Is the student's solution the same as actual solution \ just calculated: ``` yes or no ``` Student grade: ``` correct or incorrect ``` Question: ``` I'm building a solar power installation and I need help \ working out the financials. - Land costs $100 / square foot - I can buy solar panels for $250 / square foot - I negotiated a contract for maintenance that will cost \ me a flat $100k per year, and an additional $10 / square \ foot What is the total cost for the first year of operations \ as a function of the number of square feet. ``` Student's solution: ``` Let x be the size of the installation in square feet. Costs: 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 100x Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000 ``` Actual solution: As I say this same prompt "fools" GPT3.5 and 4.
- seanhunter 3y ago
  Btw I think you're not correct about the 8 digit multiplication. I tried: please could you multiply 12345678 by 87654321. Give your answer in steps. Each step should calculate one column at a time from right to left remembering to carry any digits which overflowed in a previous step. At the end add a final step adding the values up. And it gave me the following, which I believe to be correct: Sure, I'll be doing a long multiplication here, step by step, column by column from right to left. For brevity, I'll write the numbers to be multiplied as: A: 12345678 B: 87654321 Step 1: Multiply the 1st digit of B by A: 1 * 12345678 = 12345678 Step 2: Multiply the 2nd digit of B by A, remember to shift one position to the left: 2 * 12345678 = 24691356 (shift one position to the left, i.e., append a 0 at the end) Step 3: Multiply the 3rd digit of B by A, remember to shift two positions to the left: ... skip some of the answer because you get the gist.... 7 * 12345678 = 86419746 (shift six positions to the left, i.e., append six 0s at the end) Step 8: Multiply the 8th digit of B by A, remember to shift seven positions to the left: 8 * 12345678 = 98765424 (shift seven positions to the left, i.e., append seven 0s at the end) Now, the last step is to add all these results together: 12345678 246913560 3703703400 49382712000 617283900000 7407406800000 86419746000000 987654240000000 = 1082152022374638 So, the result of 12345678 multiplied by 87654321 is 1082152022374638.
  - usaar333 3y ago
    You might be getting lucky because that sequence is trained on (it's not very random). I repeated this experiment with bing and verified it could get your solution. But it failed on more random input: please could you multiply 92631924 by 41920318. Give your answer in steps. Each step should calculate one column at a time from right to left remembering to carry any digits which overflowed in a previous step. At the end add a final step adding the values up. Got 3880341653910192 with intermediate steps being broken.

ly3xqhl8g9 3y ago

Large language models and the transformer architecture are just the ALGOL 60 [1] of the search for general synthetic competence (intelligence is too finicky, competence is being given a task and fulfilling it successfully, in time, on budget, optimizing along the way). ChatGPT + Wolfram or other plugins are great for making transformers mathy, but the plugins must also be end-to-end machine learning architectures. To continue the high-level programming language analogy, we will probably see the C-level neural architecture in a few years once we get more embedded data (competence obtained through correlation physical object ↔ concept) with feedback loops forcing algorithms to operate under energy-restrictions (unlike the current architectures, more of a proof of concept, being careless about their energy). [1] https://en.wikipedia.org/wiki/ALGOL_60 https://en.wikipedia.org/wiki/ALGOL_60