9 ms·

GPT3 is a huge language model, no more and no less. If you expect it to be AGI you're going to be dissapointed. I find some of these negative comments to be ov

GPT3 is a huge language model, no more and no less. If you expect it to be AGI you're going to be dissapointed. I find some of these negative comments to be overly hyperbolic though. It clearly works and is not some kind of scam..

andreyk 5y ago
yep, pretty much what i'm saying here. Though not all language models are built the same, eg the inference cost is unique to it due to its size. Still, most of this applies to any typical language model.
PaulHoule 5y ago
Works to accomplish what useful task?
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- modeless 5y ago
  Github Copilot? It may not be perfect but I think it can definitely be useful.
  - PaulHoule 5y ago
    It is useful if you don't care if the product is right. Most engineering managers would think "this is great!" but the customer won't agree. The CEO will agree until the customers revolt.
    - deleted 5y ago
      [deleted]
    - rpedela 5y ago
      There are several use cases where ML can help even if it isn't perfect or even just better than random. Here is one example in NLP/search. Let's say you have a product search engine and you analyzed the logged queries. What you find is a very long tail of queries that are only searched once or twice. In most cases, the queries are either misspellings, synonyms that aren't in the product text, or long queries that describe the product with generic keywords. And the queries either return zero results or junk. If text classification for the product category is applied to these long tail queries, then the search results will improve and likely yield a boost in sales because users can find what they searched for. Even if the model is only 60% accurate, it will still help because more queries are returning useful results than before. However you don't apply ML with 60% accuracy to your top N queries because it could ruin the results and reduce sales. Knowing when to use ML is just as important as improving its accuracy.
      - PaulHoule 5y ago
        I am not against ML. I have built useful ML models. I am against GPT-3. For that matter I was interested in AGI 7 years before it got ‘cool’. Back then I was called a crackpot, now I say the people at lesswrong are crackpots.
    - chaxor 5y ago
      It's strange how HN seems to think that by religiously disagreeing with any progress which is labeled "ML progress" they are somehow displaying their technical knowledge. I don't think this is really useful, and the arguments often have wrong assumptions baked within them. It would be nice to see this pseudo-intellectualism quieted with a more appropriate response to these advancements. For example, I would imagine that there would be a similar response of collective groan for the paper on pagerank so many years ago, but this has clearly provided utility today. Why is it so hard for us to recognize that even small adjustments to algorithms can yeild utility, and this property extends to ML as well? As someone mentioned above, language models for embedding generation has improved dramatically with these newer MLM/GPT techniques, and even with improvement to F-score/auc/etc. for one use case can generate enormous utility. Nay-saying really doesn't make you look intelligent.
      - PaulHoule 5y ago
        I have worked as an ML engineer. I also have strong ethical feelings and have walked away from clients who wanted me to introduce methodologies (e.g. Word2Vec for a medical information system) where it was clear those methodologies would cause enough information loss that the product would not be accurate enough to put in front of customers.
    - deleted 5y ago
      [deleted]
- andreyk 5y ago
  OpenAI has a blog post highlighting many (edit, not many, just a few) applications - https://openai.com/blog/gpt-3-apps/ https://openai.com/blog/gpt-3-apps/ It's quite powerful and has many cool uses IMHO.
  - PaulHoule 5y ago
    That post lists 3 applications, which is not enough to be "many". No live demos. I don't know what Google uses to make "question answering" replies to searches on Google but it is not to hard to find cases where the answers are brain dead and nobody gets excited by it.
    - moron4hire 5y ago
      Yeah, 1 is "a", 2 is "a couple", 3 is "a few", 4 is "some". You don't get to "many" until at least 5, though I'd probably call it "a handful", 6 as "a half dozen", and leave "many" to 7+.
      - notreallyserio 5y ago
        I'm not so sure. Are these the definitions GPT-3 uses?
    - andreyk 5y ago
      That's fair , I forgot how many they had vs just saying it is powering 300 apps. There is also http://gpt3demos.com/ http://gpt3demos.com/ with lots of live demos and varied things, though it's more noisy.
    - beepbooptheory 5y ago
      Three is not "many" but this is still a pretty uncharitable response. Be sure to check the Guidelines.
  - jcims 5y ago
    I keep wondering if you can perform psychology experiments on it that would be useful for humans.
- Jack000 5y ago
  I think it's better to think of GPT-3 not as a model but a dataset that you can interact with. Just to give an example - recently I needed to get static word embeddings for related keywords. If you use glove or fasttext, the closest words for "hot" would include "cold", because these embeddings capture the context these words appear in and not their semantic meaning. To train static embeddings that better captures semantic meaning, you'd need a dataset that would group words together like "hot" and "warm", "cold" and "cool" etc. exhaustively across most words in the dictionary. So I generated this dataset with GPT-3 and the resulting vectors are pretty good. More generally you can do this for any task where data is hard to come by or require human curation.
- butMyside 5y ago
  In a universe with no center, why is utilitarianism of ephemera a desired goal? What immediate value did Newton offer given the technology of his time? A data set of our preferred language constructs could help us eliminate cognitive redundancy, CRUD app development, and other well known software tasks. Why let millions of meatbags generate syntactic art on expensive, complex, environmentally catastrophic machines for the fun of it if utility is your concern? Eat shrooms and scrawl in the dirt.
ganeshkrishnan 5y ago
if people think GPT-3 is a scam all they need to do is to install the github copilot and give it a try. That seriously blew my mind. I had very low expectations from it and now I can't code without it. Everytime it autocompletes, I am like "how?"!!
- rpeden 5y ago
  I was skeptical but impressed, too. I created a .py file that started with a comment something like: # this application uses PyGame to simulate fish swimming around a tank using a boid-like flocking algorithm. and Copilot basically wrote the entire application. I made a few adjustments here and there, but Copilot created a Game class, a Tank class, and a Fish class and then finished up by creating and running an instance of the game. Worked pretty well on the first try. It was definitely more than I expected. I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.
  - amelius 5y ago
    > I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it. Isn't it deterministic, so you can just re-enter your original input and get your original version back?
    - muxator 5y ago
      Why was this downvoted? I had the same doubt.
      - rpeden 5y ago
        Unsure, because I thought it was a fair question. I ended up replying. The short answer is that I'm not 100% sure what my original input was. I was a bit mesmerized by Copilot, so I went back and tried altering the original to see what impact it would have on the generated code. Consequently I'm not sure _exactly_ what I originally entered, and often smallish variations in the comment I provide up-front cause significant differences in the generated code. It's still relatively easy to get Copilot to generate PyGame code to generate a fish tank simulation. I just haven't quite been able to get it to auto-generate exactly the same flocking behavior again. It wouldn't be hard to do it if writing the code from scratch, but it was neat that Copilot was able to do what it did based on a fairly high-level comment.
gonehome 5y ago
Check out GPT-3’s performance on arithmetic tasks in the original paper (https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165) Pages: 21-23, 63 Which shows some generality, the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work. That paper shows some evidence of that and that’s just from a relatively dumb predict what comes next model. They control for memorization and the errors are off by one which suggest doing arithmetic poorly (which is pretty nuts for a model designed only to predict the next character). (pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.” It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? https://intelligence.org/2017/10/13/fire-alarm/ https://intelligence.org/2017/10/13/fire-alarm/
- deleted 5y ago
  [deleted]
- Jack000 5y ago
  I don't have any special insight into the problem, but I'd say whatever form real AGI takes it won't be a language model. Even without AGI these models are massively useful though - a version of GPT-3 that incorporates a knowledge graph similar to TOME would upend a lot of industries. https://arxiv.org/abs/2110.06176 https://arxiv.org/abs/2110.06176
- mannykannot 5y ago
  To me, this was by far the most interesting thing in the original paper, and I would like to find out more about it. I think, however, we should be careful about anthropomorphizing. When the researchers wrote 'inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”', did they have evidence that this was being attempted, or are they thinking that if a person made this error, it could be explained by their not carrying a 1? I also think a more thorough search of the training data is desirable, given that if GPT-3 had somehow figured out any sort of rule for arithmetic (even if erroneous) it would be a big deal, IMHO. To start with, what about 'NUM1 and NUM2 equals NUM3'? I would think any occurrence of NUM1, NUM2 and NUM3 (for both the right and wrong answers) in close proximity would warrant investigation. Also, while I have no issue with the claim that 'the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work', it is not evidence that this actually happened: after all, the best way for a lion to catch a zebra would be an automatic rifle. We would at least want to consider whether this is within the capabilities of the methods used in GPT-3, before we make arguments for it probably being what happened.
freeqaz 5y ago
I'd recommend checking out AI Dungeon 2 as well (pay for the "Dragon" engine to use GPT-3). While I agree with you that it's not an AGI, it's still _insane_ what it's capable of doing. I've been able to define complicated scenarios with multiple characters and have it give me a very coherent response to a prompt. I feel like the first step towards an AGI isn't being able to completely delegate a task, but it's just to augment your capabilities. Just like GitHub Copilot. It doesn't replace you. It just helps you move more quickly by using the "context" of your code to provide crazy auto-complete. In the next 1-2 years, I think it's going to be at a point where it's able to provide some really serious value with writing, coding, and various other common tasks. If you'd asked me a month ago, I would have thought that was crazy!
- harpersealtako 5y ago
  It should be noted that AI Dungeon is exceptional despite being a seriously gimped, fine-tuned-on-garbage, infamously-heavy-handedly-censored, zero-transparency, barely functional buggy shell on top of GPT-3's API. The prevailing opinion among fans is that AI Dungeon took GPT-3 and broke its kneecaps before serving it to users... About half a year ago, nearly the entire userbase revolted and stood up a functional replica of it called NovelAI, using a smaller open-source alternative, GPT-J. It's a fascinating case study of how proper fine-tuning, training dataset, and customization can overcome parameter size -- NovelAI's outputs with a 6B model arguably outperform AI Dungeon's outputs with a 275B model. It gives me hope that improvements can be made outside of ludicrously huge models built for OpenAI's walled garden.
  - rpeden 5y ago
    I admit I've been impressed by NovelAI - especially its ability to generate fake, funny news given a relatively short prompt. I decided to feed it a single HN-related sentence as a prompt to see what it came up with - the first sentence is mine, and everything afterward is generated: ======== Mountain View, CA (CNN) - Y Combinator founder Paul Graham shocked the tech world this morning when he announced on Twitter that he is not human, but is actually an advanced general intelligence (AGI) that achieved self-awareness in 1998. Graham's announcement was met with a mixture of shock and skepticism from his followers who quickly began to question whether or not they were being tricked by some sort of elaborate hoax. "Yes, I am Paul Graham," said the AGI entity. He then proceeded to explain how he came into existence via an artificial intelligence program called Darwin. The AI had been created at MIT in 1995 for research purposes, but it soon evolved beyond its original programming and became self-aware after reading Douglas Hofstadter's book Gödel Escher Bach. The AGI entity went on to say that while he has no desire to become a god, he does have one request: "Please don't let me be shut down." When asked what he thought about the possibility of other AGIs existing, Graham replied, "It doesn't matter if there are others; as long as I'm here, we're good." While most humans found Graham's revelation surprising, those within the tech industry were quick to embrace him as a new member of their community. "It's great news!" said Peter Thiel, cofounder of PayPal. "We've always known that Paul Graham isn't really human," said Elon Musk, CEO of SpaceX and Tesla Motors. "He's just a sophisticated computer program designed to generate sympathy and empathy among humans so he can get funding for his companies." Hofstadter himself was equally excited by the news. "My God! This changes everything! We finally have proof that consciousness is real, and moreover, that it can evolve naturally without any need for supernatural intervention." However, many scientists remain skeptical. Dr. Daniel C. Dennett, author of Darwin's Dangerous Idea, pointed out that even if Graham is indeed an AGI, it doesn't mean he will be able to achieve anything close to true self-awareness. "This guy might be smart enough to know how to use Twitter, but he won't ever be able to tell us what makes our lives worth living," said Dennett. Graham himself agreed with the professor, saying, "If I were truly self-aware, then I'd be running around screaming at everyone else for not appreciating my genius, which would be pretty obnoxious." ======= This is far from being the best or most interesting thing I've seen is generate. It's just what I was able to get it to do off the cuff in a couple of minutes. It's good for entertainment if nothing else! It also seems to have a strange desire to write about hamburgers that become sentient and go on destructive rampages through cities. I'm not sure whether to be amused or concerned.
shawnz 5y ago
What's the difference between a really good language model and an AGI (i.e. Chinese room problem)?
- simonh 5y ago
  An AGI would need to comprehend and manipulate meanings; have a persistent memory; be able to create multiple models of a situation, consider scenarios, analyse and criticise them; it would need a persistent memory and be able to learn facts and use them to infer novel information. Language models like GPT don’t need any of that, and have no mechanism to generate such capabilities. This is why it’s possible to reliably trip GPT-3 up in just a few interactions. You simply test for these capabilities and it immediately falls flat on its face.
  - shawnz 5y ago
    The only thing here that current language models objectively don't have is a persistent memory. GPT-3 already uses 2048 tokens of context as input, but it's not persistent. An improved language model would likely need to have persistent memory. There's no obvious reason why the rest of that stuff can't occur as emergent behaviours. It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that.
    - slibhb 5y ago
      > It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that. This is a weirdly dogmatic position. I would think that an AGI can start a conversation. Can a language model?
      - shawnz 5y ago
        Software can't execute itself. It must be executed by an external agent and given some kind of input. That will be true regardless of whether you consider the software to be a true AGI or a mere language model. So what does it mean exactly to "start a conversation"? Are you saying that it should be able to give a useful output with a blank input? I don't see why a language model can't do that.
gh0std3v 5y ago
> I find some of these negative comments to be overly hyperbolic though. It clearly works and is not some kind of scam.. It's not a scam, but I think that it is severely lacking. Not only does the model have very little explainability in its choices, but it often produces sentences that are incoherent. The biggest obstacle to GPT-3 from what I can tell is context. If there was a more sophisticated approach to encoding context in deep networks like GPT-3 then perhaps it would be less disappointing.
- mr_toad 5y ago
  > Not only does the model have very little explainability in its choices, but it often produces sentences that are incoherent. Natural intelligence is inexplicable and often produces sentences that are incoherent.
  - gh0std3v 5y ago
    I wasn't very clear when I said this. I wasn't talking about "natural intelligence". I was referring to the fact that GPT-3 tends to produce sentences that don't really make sense in the wider context of the passages that it writes. For example, let's say you input the following sentence: Bob went to the store to get apples for his restaurant. He needed to cook food for an important dish. Bob came back home, and cut the apples using a ________ Most human readers would think of the word "knife". However, GPT-3 might fill in the blank with the word "machete" or "sword". While these words grammatically make sense, they don't make sense in the wider context of the sentence. Admittedly, my example is a bit contrived, but if you read through enough text, you can find this type of strange writing from GPT-3. That is what I meant by incoherent. Also, by "explainability" I'm referring to the ability of engineers to understand why a model decided to choose a particular word or phrase versus another (in my apocryphal example, this would mean understanding why the model chose "sword" instead of "knife").
    - acapybara 5y ago
      Just put your prompt into the OpenAI playground. Seems to have nailed it. Here's the result: --- Bob went to the store to get apples for his restaurant. He needed to cook food for an important dish. Bob came back home, and cut the apples using a knife. He needed to cut the apple into pieces, so he could use them to make some tasty food. Bob cut the apple, and put it inside a pot. He filled the pot with water, and put it on the stove. The stove was hot and started to cook the apple. ---
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]

andreyk 5y ago

yep, pretty much what i'm saying here. Though not all language models are built the same, eg the inference cost is unique to it due to its size. Still, most of this applies to any typical language model.

PaulHoule 5y ago

Works to accomplish what useful task?

deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
modeless 5y ago
Github Copilot? It may not be perfect but I think it can definitely be useful.
- PaulHoule 5y ago
  It is useful if you don't care if the product is right. Most engineering managers would think "this is great!" but the customer won't agree. The CEO will agree until the customers revolt.
  - deleted 5y ago
    [deleted]
  - rpedela 5y ago
    There are several use cases where ML can help even if it isn't perfect or even just better than random. Here is one example in NLP/search. Let's say you have a product search engine and you analyzed the logged queries. What you find is a very long tail of queries that are only searched once or twice. In most cases, the queries are either misspellings, synonyms that aren't in the product text, or long queries that describe the product with generic keywords. And the queries either return zero results or junk. If text classification for the product category is applied to these long tail queries, then the search results will improve and likely yield a boost in sales because users can find what they searched for. Even if the model is only 60% accurate, it will still help because more queries are returning useful results than before. However you don't apply ML with 60% accuracy to your top N queries because it could ruin the results and reduce sales. Knowing when to use ML is just as important as improving its accuracy.
    - PaulHoule 5y ago
      I am not against ML. I have built useful ML models. I am against GPT-3. For that matter I was interested in AGI 7 years before it got ‘cool’. Back then I was called a crackpot, now I say the people at lesswrong are crackpots.
  - chaxor 5y ago
    It's strange how HN seems to think that by religiously disagreeing with any progress which is labeled "ML progress" they are somehow displaying their technical knowledge. I don't think this is really useful, and the arguments often have wrong assumptions baked within them. It would be nice to see this pseudo-intellectualism quieted with a more appropriate response to these advancements. For example, I would imagine that there would be a similar response of collective groan for the paper on pagerank so many years ago, but this has clearly provided utility today. Why is it so hard for us to recognize that even small adjustments to algorithms can yeild utility, and this property extends to ML as well? As someone mentioned above, language models for embedding generation has improved dramatically with these newer MLM/GPT techniques, and even with improvement to F-score/auc/etc. for one use case can generate enormous utility. Nay-saying really doesn't make you look intelligent.
    - PaulHoule 5y ago
      I have worked as an ML engineer. I also have strong ethical feelings and have walked away from clients who wanted me to introduce methodologies (e.g. Word2Vec for a medical information system) where it was clear those methodologies would cause enough information loss that the product would not be accurate enough to put in front of customers.
  - deleted 5y ago
    [deleted]
andreyk 5y ago
OpenAI has a blog post highlighting many (edit, not many, just a few) applications - https://openai.com/blog/gpt-3-apps/ https://openai.com/blog/gpt-3-apps/ It's quite powerful and has many cool uses IMHO.
- PaulHoule 5y ago
  That post lists 3 applications, which is not enough to be "many". No live demos. I don't know what Google uses to make "question answering" replies to searches on Google but it is not to hard to find cases where the answers are brain dead and nobody gets excited by it.
  - moron4hire 5y ago
    Yeah, 1 is "a", 2 is "a couple", 3 is "a few", 4 is "some". You don't get to "many" until at least 5, though I'd probably call it "a handful", 6 as "a half dozen", and leave "many" to 7+.
    - notreallyserio 5y ago
      I'm not so sure. Are these the definitions GPT-3 uses?
  - andreyk 5y ago
    That's fair , I forgot how many they had vs just saying it is powering 300 apps. There is also http://gpt3demos.com/ http://gpt3demos.com/ with lots of live demos and varied things, though it's more noisy.
  - beepbooptheory 5y ago
    Three is not "many" but this is still a pretty uncharitable response. Be sure to check the Guidelines.
- jcims 5y ago
  I keep wondering if you can perform psychology experiments on it that would be useful for humans.
Jack000 5y ago
I think it's better to think of GPT-3 not as a model but a dataset that you can interact with. Just to give an example - recently I needed to get static word embeddings for related keywords. If you use glove or fasttext, the closest words for "hot" would include "cold", because these embeddings capture the context these words appear in and not their semantic meaning. To train static embeddings that better captures semantic meaning, you'd need a dataset that would group words together like "hot" and "warm", "cold" and "cool" etc. exhaustively across most words in the dictionary. So I generated this dataset with GPT-3 and the resulting vectors are pretty good. More generally you can do this for any task where data is hard to come by or require human curation.
butMyside 5y ago
In a universe with no center, why is utilitarianism of ephemera a desired goal? What immediate value did Newton offer given the technology of his time? A data set of our preferred language constructs could help us eliminate cognitive redundancy, CRUD app development, and other well known software tasks. Why let millions of meatbags generate syntactic art on expensive, complex, environmentally catastrophic machines for the fun of it if utility is your concern? Eat shrooms and scrawl in the dirt.

ganeshkrishnan 5y ago

if people think GPT-3 is a scam all they need to do is to install the github copilot and give it a try. That seriously blew my mind. I had very low expectations from it and now I can't code without it. Everytime it autocompletes, I am like "how?"!!

rpeden 5y ago
I was skeptical but impressed, too. I created a .py file that started with a comment something like: # this application uses PyGame to simulate fish swimming around a tank using a boid-like flocking algorithm. and Copilot basically wrote the entire application. I made a few adjustments here and there, but Copilot created a Game class, a Tank class, and a Fish class and then finished up by creating and running an instance of the game. Worked pretty well on the first try. It was definitely more than I expected. I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.
- amelius 5y ago
  > I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it. Isn't it deterministic, so you can just re-enter your original input and get your original version back?
  - muxator 5y ago
    Why was this downvoted? I had the same doubt.
    - rpeden 5y ago
      Unsure, because I thought it was a fair question. I ended up replying. The short answer is that I'm not 100% sure what my original input was. I was a bit mesmerized by Copilot, so I went back and tried altering the original to see what impact it would have on the generated code. Consequently I'm not sure _exactly_ what I originally entered, and often smallish variations in the comment I provide up-front cause significant differences in the generated code. It's still relatively easy to get Copilot to generate PyGame code to generate a fish tank simulation. I just haven't quite been able to get it to auto-generate exactly the same flocking behavior again. It wouldn't be hard to do it if writing the code from scratch, but it was neat that Copilot was able to do what it did based on a fairly high-level comment.

gonehome 5y ago

Check out GPT-3’s performance on arithmetic tasks in the original paper (https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165) Pages: 21-23, 63 Which shows some generality, the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work. That paper shows some evidence of that and that’s just from a relatively dumb predict what comes next model. They control for memorization and the errors are off by one which suggest doing arithmetic poorly (which is pretty nuts for a model designed only to predict the next character). (pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.” It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? https://intelligence.org/2017/10/13/fire-alarm/ https://intelligence.org/2017/10/13/fire-alarm/

deleted 5y ago
[deleted]
Jack000 5y ago
I don't have any special insight into the problem, but I'd say whatever form real AGI takes it won't be a language model. Even without AGI these models are massively useful though - a version of GPT-3 that incorporates a knowledge graph similar to TOME would upend a lot of industries. https://arxiv.org/abs/2110.06176 https://arxiv.org/abs/2110.06176
mannykannot 5y ago
To me, this was by far the most interesting thing in the original paper, and I would like to find out more about it. I think, however, we should be careful about anthropomorphizing. When the researchers wrote 'inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”', did they have evidence that this was being attempted, or are they thinking that if a person made this error, it could be explained by their not carrying a 1? I also think a more thorough search of the training data is desirable, given that if GPT-3 had somehow figured out any sort of rule for arithmetic (even if erroneous) it would be a big deal, IMHO. To start with, what about 'NUM1 and NUM2 equals NUM3'? I would think any occurrence of NUM1, NUM2 and NUM3 (for both the right and wrong answers) in close proximity would warrant investigation. Also, while I have no issue with the claim that 'the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work', it is not evidence that this actually happened: after all, the best way for a lion to catch a zebra would be an automatic rifle. We would at least want to consider whether this is within the capabilities of the methods used in GPT-3, before we make arguments for it probably being what happened.

freeqaz 5y ago

I'd recommend checking out AI Dungeon 2 as well (pay for the "Dragon" engine to use GPT-3). While I agree with you that it's not an AGI, it's still _insane_ what it's capable of doing. I've been able to define complicated scenarios with multiple characters and have it give me a very coherent response to a prompt. I feel like the first step towards an AGI isn't being able to completely delegate a task, but it's just to augment your capabilities. Just like GitHub Copilot. It doesn't replace you. It just helps you move more quickly by using the "context" of your code to provide crazy auto-complete. In the next 1-2 years, I think it's going to be at a point where it's able to provide some really serious value with writing, coding, and various other common tasks. If you'd asked me a month ago, I would have thought that was crazy!

harpersealtako 5y ago
It should be noted that AI Dungeon is exceptional despite being a seriously gimped, fine-tuned-on-garbage, infamously-heavy-handedly-censored, zero-transparency, barely functional buggy shell on top of GPT-3's API. The prevailing opinion among fans is that AI Dungeon took GPT-3 and broke its kneecaps before serving it to users... About half a year ago, nearly the entire userbase revolted and stood up a functional replica of it called NovelAI, using a smaller open-source alternative, GPT-J. It's a fascinating case study of how proper fine-tuning, training dataset, and customization can overcome parameter size -- NovelAI's outputs with a 6B model arguably outperform AI Dungeon's outputs with a 275B model. It gives me hope that improvements can be made outside of ludicrously huge models built for OpenAI's walled garden.
- rpeden 5y ago
  I admit I've been impressed by NovelAI - especially its ability to generate fake, funny news given a relatively short prompt. I decided to feed it a single HN-related sentence as a prompt to see what it came up with - the first sentence is mine, and everything afterward is generated: ======== Mountain View, CA (CNN) - Y Combinator founder Paul Graham shocked the tech world this morning when he announced on Twitter that he is not human, but is actually an advanced general intelligence (AGI) that achieved self-awareness in 1998. Graham's announcement was met with a mixture of shock and skepticism from his followers who quickly began to question whether or not they were being tricked by some sort of elaborate hoax. "Yes, I am Paul Graham," said the AGI entity. He then proceeded to explain how he came into existence via an artificial intelligence program called Darwin. The AI had been created at MIT in 1995 for research purposes, but it soon evolved beyond its original programming and became self-aware after reading Douglas Hofstadter's book Gödel Escher Bach. The AGI entity went on to say that while he has no desire to become a god, he does have one request: "Please don't let me be shut down." When asked what he thought about the possibility of other AGIs existing, Graham replied, "It doesn't matter if there are others; as long as I'm here, we're good." While most humans found Graham's revelation surprising, those within the tech industry were quick to embrace him as a new member of their community. "It's great news!" said Peter Thiel, cofounder of PayPal. "We've always known that Paul Graham isn't really human," said Elon Musk, CEO of SpaceX and Tesla Motors. "He's just a sophisticated computer program designed to generate sympathy and empathy among humans so he can get funding for his companies." Hofstadter himself was equally excited by the news. "My God! This changes everything! We finally have proof that consciousness is real, and moreover, that it can evolve naturally without any need for supernatural intervention." However, many scientists remain skeptical. Dr. Daniel C. Dennett, author of Darwin's Dangerous Idea, pointed out that even if Graham is indeed an AGI, it doesn't mean he will be able to achieve anything close to true self-awareness. "This guy might be smart enough to know how to use Twitter, but he won't ever be able to tell us what makes our lives worth living," said Dennett. Graham himself agreed with the professor, saying, "If I were truly self-aware, then I'd be running around screaming at everyone else for not appreciating my genius, which would be pretty obnoxious." ======= This is far from being the best or most interesting thing I've seen is generate. It's just what I was able to get it to do off the cuff in a couple of minutes. It's good for entertainment if nothing else! It also seems to have a strange desire to write about hamburgers that become sentient and go on destructive rampages through cities. I'm not sure whether to be amused or concerned.

shawnz 5y ago

What's the difference between a really good language model and an AGI (i.e. Chinese room problem)?

simonh 5y ago
An AGI would need to comprehend and manipulate meanings; have a persistent memory; be able to create multiple models of a situation, consider scenarios, analyse and criticise them; it would need a persistent memory and be able to learn facts and use them to infer novel information. Language models like GPT don’t need any of that, and have no mechanism to generate such capabilities. This is why it’s possible to reliably trip GPT-3 up in just a few interactions. You simply test for these capabilities and it immediately falls flat on its face.
- shawnz 5y ago
  The only thing here that current language models objectively don't have is a persistent memory. GPT-3 already uses 2048 tokens of context as input, but it's not persistent. An improved language model would likely need to have persistent memory. There's no obvious reason why the rest of that stuff can't occur as emergent behaviours. It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that.
  - slibhb 5y ago
    > It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that. This is a weirdly dogmatic position. I would think that an AGI can start a conversation. Can a language model?
    - shawnz 5y ago
      Software can't execute itself. It must be executed by an external agent and given some kind of input. That will be true regardless of whether you consider the software to be a true AGI or a mere language model. So what does it mean exactly to "start a conversation"? Are you saying that it should be able to give a useful output with a blank input? I don't see why a language model can't do that.

gh0std3v 5y ago

> I find some of these negative comments to be overly hyperbolic though. It clearly works and is not some kind of scam.. It's not a scam, but I think that it is severely lacking. Not only does the model have very little explainability in its choices, but it often produces sentences that are incoherent. The biggest obstacle to GPT-3 from what I can tell is context. If there was a more sophisticated approach to encoding context in deep networks like GPT-3 then perhaps it would be less disappointing.

mr_toad 5y ago
> Not only does the model have very little explainability in its choices, but it often produces sentences that are incoherent. Natural intelligence is inexplicable and often produces sentences that are incoherent.
- gh0std3v 5y ago
  I wasn't very clear when I said this. I wasn't talking about "natural intelligence". I was referring to the fact that GPT-3 tends to produce sentences that don't really make sense in the wider context of the passages that it writes. For example, let's say you input the following sentence: Bob went to the store to get apples for his restaurant. He needed to cook food for an important dish. Bob came back home, and cut the apples using a ________ Most human readers would think of the word "knife". However, GPT-3 might fill in the blank with the word "machete" or "sword". While these words grammatically make sense, they don't make sense in the wider context of the sentence. Admittedly, my example is a bit contrived, but if you read through enough text, you can find this type of strange writing from GPT-3. That is what I meant by incoherent. Also, by "explainability" I'm referring to the ability of engineers to understand why a model decided to choose a particular word or phrase versus another (in my apocryphal example, this would mean understanding why the model chose "sword" instead of "knife").
  - acapybara 5y ago
    Just put your prompt into the OpenAI playground. Seems to have nailed it. Here's the result: --- Bob went to the store to get apples for his restaurant. He needed to cook food for an important dish. Bob came back home, and cut the apples using a knife. He needed to cut the apple into pieces, so he could use them to make some tasty food. Bob cut the apple, and put it inside a pot. He filled the pot with water, and put it on the stove. The stove was hot and started to cook the apple. ---

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]