3 ms·

AGI is usually defined as anything a normal human could do. Examples of some things that GPT is currently lacking: 1) Understanding images and video 2) Learni

by ipsum2 4y ago

AGI is usually defined as anything a normal human could do. Examples of some things that GPT is currently lacking: 1) Understanding images and video 2) Learning and remembering things outside the 2048 token context window 3) Interacting with the environment (either through a computer, or in the real world) 4) Doing basic math (1662 + 239 = ?) and logic

scotty79 4y ago
> 4) Doing basic math (1662 + 239 = ?) and logic Doing math. Period. There are only two things brains do. Math and bullshitting. Transformers have second part covered but the other part is a completely different beast. I think AGI will be achieved once ML can translate natural language to math, do math then output results as natural language correct in the context of a problem.
wcoenen 4y ago
> Learning and remembering things outside the 2048 token context window It is probably possible to do this with fine tuning. Once the context window is full, it might be possible to convert the content in there into additional training examples and feed those to a fine tuning process, which then retrains the model. (The OpenAI API for fine tuning is here: https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning) It would be a bit like sleeping. Whenever the context window fills up, the model would have to go offline for a while to move memories from its context window (short term) to its network weights (long term).
- dr_dshiv 4y ago
  So an 18 hour long token window is acceptable as AGI, I like it.
bbor 4y ago
I’d argue “learning and remembering” is the core issue here, and the other things you’ve mentioned are just ways humans use that ability. I took a “Knowledge Based AI” course and it really opened my eyes to how central a knowledge system is to cognition. I wouldn’t call what LLMs have knowledge since it lacks structure, though I’m very much a novice on the exact details there. Well actually #3 isn’t great bc what exactly would you call taking input from a human and displaying output, much less e.g. controlling a smart home? Seems like LLMs are plenty smart enough to interact with things.
dr_dshiv 4y ago
Nice list. 1. Understanding— provided this isn’t based on consciousness but benchmarks — appears present in text, but yes, GPT3 can’t do images or video yet 2. Is there some number of tokens that would be AGI? Of course it can remember things it has already been trained on… 3. In what way is interacting with a human not interacting with the environment? (Eg, you can ask it to tell you the steps to make a pbj sandwich and troubleshoot) 4. Basic math and logic. Ok. So, once AI can do images, video, math, logic and remember 20000 tokens, then we have AGI?

by ipsum2 4y ago

scotty79 4y ago

> 4) Doing basic math (1662 + 239 = ?) and logic Doing math. Period. There are only two things brains do. Math and bullshitting. Transformers have second part covered but the other part is a completely different beast. I think AGI will be achieved once ML can translate natural language to math, do math then output results as natural language correct in the context of a problem.

wcoenen 4y ago

> Learning and remembering things outside the 2048 token context window It is probably possible to do this with fine tuning. Once the context window is full, it might be possible to convert the content in there into additional training examples and feed those to a fine tuning process, which then retrains the model. (The OpenAI API for fine tuning is here: https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning) It would be a bit like sleeping. Whenever the context window fills up, the model would have to go offline for a while to move memories from its context window (short term) to its network weights (long term).

dr_dshiv 4y ago
So an 18 hour long token window is acceptable as AGI, I like it.

bbor 4y ago

I’d argue “learning and remembering” is the core issue here, and the other things you’ve mentioned are just ways humans use that ability. I took a “Knowledge Based AI” course and it really opened my eyes to how central a knowledge system is to cognition. I wouldn’t call what LLMs have knowledge since it lacks structure, though I’m very much a novice on the exact details there. Well actually #3 isn’t great bc what exactly would you call taking input from a human and displaying output, much less e.g. controlling a smart home? Seems like LLMs are plenty smart enough to interact with things.

dr_dshiv 4y ago

Nice list. 1. Understanding— provided this isn’t based on consciousness but benchmarks — appears present in text, but yes, GPT3 can’t do images or video yet 2. Is there some number of tokens that would be AGI? Of course it can remember things it has already been trained on… 3. In what way is interacting with a human not interacting with the environment? (Eg, you can ask it to tell you the steps to make a pbj sandwich and troubleshoot) 4. Basic math and logic. Ok. So, once AI can do images, video, math, logic and remember 20000 tokens, then we have AGI?