4 ms·

The first two versions generated content based on a prompt that was coherent for a sentence or a paragraph. GPT-3 generates an entire article that's largely co

The first two versions generated content based on a prompt that was coherent for a sentence or a paragraph. GPT-3 generates an entire article that's largely coherent (though still lacking nuanced meaning). Each version of GPT has exponentially increasing parameter count. Raw compute seems to be winning out here. Basically for this application they aren't seeming to hit diminishing returns for increasing parameters. Now people are wondering what GPT-4 can do. If you can write human-level coherent articles or if it passes the turing test or something it's gonna trigger an arms race for governments to obtain this capability.

brunoluiz 6y ago
Thanks for the explanation ;) > Each version of GPT has exponentially increasing parameter count What are parameters in this context? > Now people are wondering what GPT-4 can do. If you can write human-level coherent articles or if it passes the turing test or something [...] Well, when I first heard about "text generation" last year what I first raised to my friends was "surely this will make fake news even worse". But well, any technology come with a bright and dark side. I try to not be pessimistic in these moments hehe
- vsskanth 6y ago
  parameters here are just the size of the neural net of the model. Each unit of computation has its own parameter that needs to be fit. Bigger model = more layers or units of computations = more parameters.
op03 6y ago
This is like watching parents talking about their kid develop over time and getting excited over their evolving writing ability. Doesn't mean we get Turing or Gandhi at the end of the story. Or that there is any control over what is produced. To produce those 2 nature had to iterate over 100 billion humans.
nprz 6y ago
Disagree with the largely coherent statement. The output I've read from GPT-3 is still quite confusing, for example this[0] blog post demonstrates some of it's capabilities. Reading through the article it generates is very confusing. Statements are made and then later contradicted, elements that do not actually exist on the page are referenced. And there just doesn't seem to be an actual cohesive narrative. All the sentences are grammatically correct and make sense on their own, but the thought connecting the sentences making some sort of larger point just isn't there. And this is after the author generated 10 different articles and picked the most intelligible one. [0] https://maraoz.com/2020/07/18/openai-gpt3/ https://maraoz.com/2020/07/18/openai-gpt3/

brunoluiz 6y ago

Thanks for the explanation ;) > Each version of GPT has exponentially increasing parameter count What are parameters in this context? > Now people are wondering what GPT-4 can do. If you can write human-level coherent articles or if it passes the turing test or something [...] Well, when I first heard about "text generation" last year what I first raised to my friends was "surely this will make fake news even worse". But well, any technology come with a bright and dark side. I try to not be pessimistic in these moments hehe

vsskanth 6y ago
parameters here are just the size of the neural net of the model. Each unit of computation has its own parameter that needs to be fit. Bigger model = more layers or units of computations = more parameters.

op03 6y ago

This is like watching parents talking about their kid develop over time and getting excited over their evolving writing ability. Doesn't mean we get Turing or Gandhi at the end of the story. Or that there is any control over what is produced. To produce those 2 nature had to iterate over 100 billion humans.

nprz 6y ago

Disagree with the largely coherent statement. The output I've read from GPT-3 is still quite confusing, for example this[0] blog post demonstrates some of it's capabilities. Reading through the article it generates is very confusing. Statements are made and then later contradicted, elements that do not actually exist on the page are referenced. And there just doesn't seem to be an actual cohesive narrative. All the sentences are grammatically correct and make sense on their own, but the thought connecting the sentences making some sort of larger point just isn't there. And this is after the author generated 10 different articles and picked the most intelligible one. [0] https://maraoz.com/2020/07/18/openai-gpt3/ https://maraoz.com/2020/07/18/openai-gpt3/