5 ms·
People don't understand what exponential improvement means. GPT-3 is a 175B with B parameter model. Another few rounds of doubling and we could be seeing models
by distant_hat 6y ago
People don't understand what exponential improvement means. GPT-3 is a 175B with B parameter model. Another few rounds of doubling and we could be seeing models spit out short stories and novellas.
- ur-whale 6y agoYet, the darn thing still can't reason.
- Enginerrrd 6y agoTo be fair, it wasn't trained to. To be really fair, I think you need to more precisely define what you mean by "reason". It absolutely CAN reason by some measures.
- jbay808 6y agoFive years ago we would have said "the darn thing still can't write a cohesive paragraph".
- wnoise 6y agoThat's still true. It can write a paragraph that's usually grammatical, and can stay on topic, but it's missing things like facts, or even the ability to remember which side of an argument it's taken previously.
- jbay808 6y agoI think over the course of several paragraphs that's true, but within one it tends to be pretty good.
- distant_hat 6y agoPeople look at the present and feel like that's all that's there. You don't look at a 2-year old and say he barely makes any intelligible sounds yet.
- visarga 6y agoThey "can't reason" but they can solve symbolic integration and differential equations (https://arxiv.org/pdf/2006.06462.pdf https://arxiv.org/pdf/2006.06462.pdf), beat us at all board games, and more recently get to be almost impossible to tell apart from human writing (GPT3).
- deleted 6y ago[deleted]
- derivativethrow 6y agoHave you read the GPT-3 paper? The authors are pretty forthright about how the GPT techniques likely won't scale much beyond 175B parameters.