3 ms·

I actually just sent out an API request for a particular application that I believe GPT-3 should be capable. I'm a university mathematics instructor. I find th

I actually just sent out an API request for a particular application that I believe GPT-3 should be capable. I'm a university mathematics instructor. I find that a lot of students struggle with doing proofs and I think GPT-3 can help with that. A proof is essentially a sequence of logical inferences. I believe that, when given a proof written in natural language GPT-3 should be capable of detecting the logical implications made within. From there, it would be possible to correct student's mistakes or help them improve their answer.

heyitsguay 6y ago
Really? This seems like the exact sort of thing GPT-3 would be very bad at. The statements aren't the product of an internally consistent logical model, they are statistically plausible word sequences learned in a really clever way from a huge corpus of text. That said, before dismissing it sight unseen there are effective and quick ways to test its powers with mathematics. You could prime an instance with the axioms defining a group, or a topology, and ask it to produce proofs of varying difficulty from intro abstract algebra or topology textbooks. Whatever the answer it would make an interesting blog post. What happens when you ask it to generate repeated outputs for the same prompt? What happens if you ask it to prove an untrue proposition?
- james_s_tayler 6y ago
  So perhaps it could produce convincing looking proofs that are ultimately nonsensical. It could generate math problems and have students try them out. Might be a bit of fun.
microtherion 6y ago
That's probably precisely an area you would NOT want to work with GPT-3. As Julian Togelius put it: > GPT-3 often performs like a clever student who hasn't done their reading trying to bullshit their way through an exam. Some well-known facts, some half-truths, and some straight lies, strung together in what first looks like a smooth narrative. https://twitter.com/togelius/status/1284131360857358337?s=20 https://twitter.com/togelius/status/1284131360857358337?s=20

heyitsguay 6y ago

Really? This seems like the exact sort of thing GPT-3 would be very bad at. The statements aren't the product of an internally consistent logical model, they are statistically plausible word sequences learned in a really clever way from a huge corpus of text. That said, before dismissing it sight unseen there are effective and quick ways to test its powers with mathematics. You could prime an instance with the axioms defining a group, or a topology, and ask it to produce proofs of varying difficulty from intro abstract algebra or topology textbooks. Whatever the answer it would make an interesting blog post. What happens when you ask it to generate repeated outputs for the same prompt? What happens if you ask it to prove an untrue proposition?

james_s_tayler 6y ago
So perhaps it could produce convincing looking proofs that are ultimately nonsensical. It could generate math problems and have students try them out. Might be a bit of fun.

microtherion 6y ago

That's probably precisely an area you would NOT want to work with GPT-3. As Julian Togelius put it: > GPT-3 often performs like a clever student who hasn't done their reading trying to bullshit their way through an exam. Some well-known facts, some half-truths, and some straight lies, strung together in what first looks like a smooth narrative. https://twitter.com/togelius/status/1284131360857358337?s=20 https://twitter.com/togelius/status/1284131360857358337?s=20