3 ms·

Honestly this doesn't surprise me. I don't have GPT-4 access, but I fed ChatGPT a simple calculus question as a sanity check. Question: Calculate the integral

Honestly this doesn't surprise me. I don't have GPT-4 access, but I fed ChatGPT a simple calculus question as a sanity check. Question: Calculate the integral of x arctan x dx. Show your work. Answer: We can solve this integral using integration by parts: Let u = arctan x and dv = x dx, then we have: du/dx = 1/(1+x^2) v = (1/2)x^2 Using the integration by parts formula, we get: ∫ x arctan x dx = uv - ∫ v du/dx dx = (1/2)x^2 arctan x - (1/2)∫ x^2/(1+x^2) dx To solve the second integral, we can use substitution. Letting u = 1+x^2, then du/dx = 2x, and dx = du/(2x). Substituting into the integral, we get: ∫ x^2/(1+x^2) dx = (1/2) ∫ du/u = (1/2) ln|u| + C = (1/2) ln|1+x^2| + C Putting it all together, we get: ∫ x arctan x dx = (1/2)x^2 arctan x - (1/2) [(1/2) ln|1+x^2|] + C = (1/2)x^2 arctan x - (1/4) ln|1+x^2| + C Therefore, the integral of x arctan x dx is (1/2)x^2 arctan x - (1/4) ln|1+x^2| + C, where C is the constant of integration. --- This is wrong, although it gets pretty damn close (see if you can spot the mistake.) I was an exceptionally kind grader when I taught and Landsburg seems like an exceptionally picky one. But either way ChatGPT wouldn't pass calculus 2, either.

MuffinFlavored 4y ago
I think the real power will be when LLMs are trained to formulate the response "math detected, let me attach a low score to anything other than 'pipe user input into wolfram alpha and then use output'" Same for "let me call my XYZ plugin to do XYZ specialized task" to fill the gaps where LLMs fall short (most places when it comes to accuracy?). once ChatGPT can access the web/do basic curl/wget requests, then maybe visually inspect an image or a PDF, it's game on for "ok, AI is serious now".
- jasfi 4y ago
  Exactly this. LLMs shouldn't attempt math, they should pass it to a solver.
  - MuffinFlavored 4y ago
    i kind of wonder if in hindsight, not thinking in this pattern (pass math to a solver, etc.) was an oversight by... OpenAI? is that who/what is responsible for widespread popularity of LLMs? asking ChatGPT says LLMs got popular when GPT-2 was released by OpenAI in 2019
hgsgm 4y ago
Did it lose an x in the middle?
JCharante 4y ago
Would you consider it fair if ChatGPT answered using the WolframAlpha plugin? https://user-images.githubusercontent.com/13973198/231566582-6ee98a69-77c3-42ca-927a-a2b4192d7c09.png https://user-images.githubusercontent.com/13973198/231566582...
- hgsgm 4y ago
  Can you try it again without WA? I'm curious how often ChatGPT makes mistakes because its temperature makes it sometimes randomly give wrong answers intentionally, to be generatively creative.
  - JCharante 4y ago
    GPT3.5-turbo We can solve this integral by using integration by parts. Let u = arctan x and dv = x dx. Then we have: du/dx = 1/(1+x^2) v = (1/2)x^2 Using the formula for integration by parts, we get: ∫x arctan x dx = x (arctan x) - ∫(1/2)x^2 / (1+x^2) dx We can simplify the integral on the right-hand side by using the substitution u = 1+x^2, du/dx = 2x: ∫(1/2)x^2 / (1+x^2) dx = (1/2) ∫du/u = (1/2) ln|1+x^2| + C Putting it all together, we have: ∫x arctan x dx = x (arctan x) - (1/2) ln|1+x^2| + C where C is the constant of integration. GPT-4: To calculate the integral of x * arctan(x) dx, we'll use integration by parts, which states: ∫u dv = uv - ∫v du First, we need to choose our u and dv: u = arctan(x) dv = x dx Now, we differentiate u and integrate dv: du = (1 / (1 + x^2)) dx v = (1/2) x^2 Now, apply the integration by parts formula: ∫x arctan(x) dx = (1/2) x^2 arctan(x) - ∫(1/2) x^2 * (1 / (1 + x^2)) dx Let's simplify the second integral: ∫(1/2) x^2 * (1 / (1 + x^2)) dx = (1/2) ∫x^2 / (1 + x^2) dx Now, let's use a substitution method to solve this integral: Let z = x^2 + 1, so dz = 2x dx. Then, x^2 = z - 1 and x dx = (1/2) dz. We can rewrite the integral in terms of z: (1/2) ∫(z - 1) / z dz = (1/2) ∫(z / z - 1 / z) dz = (1/2) ∫(1 - 1/z) dz Now, integrate with respect to z: (1/2) [z - ln(|z|)] + C = (1/2) [(x^2 + 1) - ln(|x^2 + 1|)] + C Now, plug this back into our integration by parts expression: ∫x arctan(x) dx = (1/2) x^2 arctan(x) - (1/2) [(x^2 + 1) - ln(|x^2 + 1|)] + C So the final result is: (1/2) x^2 arctan(x) - (1/2) x^2 - (1/2) ln(|x^2 + 1|) + C
    - hgsgm 4y ago

MuffinFlavored 4y ago

I think the real power will be when LLMs are trained to formulate the response "math detected, let me attach a low score to anything other than 'pipe user input into wolfram alpha and then use output'" Same for "let me call my XYZ plugin to do XYZ specialized task" to fill the gaps where LLMs fall short (most places when it comes to accuracy?). once ChatGPT can access the web/do basic curl/wget requests, then maybe visually inspect an image or a PDF, it's game on for "ok, AI is serious now".

jasfi 4y ago
Exactly this. LLMs shouldn't attempt math, they should pass it to a solver.
- MuffinFlavored 4y ago
  i kind of wonder if in hindsight, not thinking in this pattern (pass math to a solver, etc.) was an oversight by... OpenAI? is that who/what is responsible for widespread popularity of LLMs? asking ChatGPT says LLMs got popular when GPT-2 was released by OpenAI in 2019

hgsgm 4y ago

Did it lose an x in the middle?

JCharante 4y ago

Would you consider it fair if ChatGPT answered using the WolframAlpha plugin? https://user-images.githubusercontent.com/13973198/231566582-6ee98a69-77c3-42ca-927a-a2b4192d7c09.png https://user-images.githubusercontent.com/13973198/231566582...

hgsgm 4y ago
Can you try it again without WA? I'm curious how often ChatGPT makes mistakes because its temperature makes it sometimes randomly give wrong answers intentionally, to be generatively creative.
- JCharante 4y ago
  GPT3.5-turbo We can solve this integral by using integration by parts. Let u = arctan x and dv = x dx. Then we have: du/dx = 1/(1+x^2) v = (1/2)x^2 Using the formula for integration by parts, we get: ∫x arctan x dx = x (arctan x) - ∫(1/2)x^2 / (1+x^2) dx We can simplify the integral on the right-hand side by using the substitution u = 1+x^2, du/dx = 2x: ∫(1/2)x^2 / (1+x^2) dx = (1/2) ∫du/u = (1/2) ln|1+x^2| + C Putting it all together, we have: ∫x arctan x dx = x (arctan x) - (1/2) ln|1+x^2| + C where C is the constant of integration. GPT-4: To calculate the integral of x * arctan(x) dx, we'll use integration by parts, which states: ∫u dv = uv - ∫v du First, we need to choose our u and dv: u = arctan(x) dv = x dx Now, we differentiate u and integrate dv: du = (1 / (1 + x^2)) dx v = (1/2) x^2 Now, apply the integration by parts formula: ∫x arctan(x) dx = (1/2) x^2 arctan(x) - ∫(1/2) x^2 * (1 / (1 + x^2)) dx Let's simplify the second integral: ∫(1/2) x^2 * (1 / (1 + x^2)) dx = (1/2) ∫x^2 / (1 + x^2) dx Now, let's use a substitution method to solve this integral: Let z = x^2 + 1, so dz = 2x dx. Then, x^2 = z - 1 and x dx = (1/2) dz. We can rewrite the integral in terms of z: (1/2) ∫(z - 1) / z dz = (1/2) ∫(z / z - 1 / z) dz = (1/2) ∫(1 - 1/z) dz Now, integrate with respect to z: (1/2) [z - ln(|z|)] + C = (1/2) [(x^2 + 1) - ln(|x^2 + 1|)] + C Now, plug this back into our integration by parts expression: ∫x arctan(x) dx = (1/2) x^2 arctan(x) - (1/2) [(x^2 + 1) - ln(|x^2 + 1|)] + C So the final result is: (1/2) x^2 arctan(x) - (1/2) x^2 - (1/2) ln(|x^2 + 1|) + C
  - hgsgm 4y ago