3 ms·

> But at no point does the LLM learn any language other than token math. Then can we make the argument that in the next 3 years, we aren't really due for any m

> But at no point does the LLM learn any language other than token math. Then can we make the argument that in the next 3 years, we aren't really due for any major AI breakthroughs and the current LLMs are about as good as they're going to get / they are kind of already at the limits of what they can do? What would more GPU power, more time to training models, more tokens, more resources for high context size do if at the end of the day it's a random-nonsense generator with a really good scoring mechanism? How much more robust/advanced can you make it? Take the context of replacing people with AI, if you can't trust the AI to be reliable + find a way to make it as accurate as humans, is there really an upcoming "AI revolution" to come?

I_Am_Nous 3y ago
LLMs are a pretty big breakthrough which can be useful for some applications, but not others. There is always room for futher breakthroughs, and as more time is devoted to research the LLMs of the future could be highly optimized for any number of metrics from power usage to token count. So with that in mind, while it isn't a silver bullet which is guaranteed to lead to AGI, it can certainly fill a role in many other places and is bound to lead to cool tech layered on top of better LLMs. It's nowhere near robust enough to replace every human, but might be enough to at least displace some.
- MuffinFlavored 3y ago
  > There is always room for further breakthroughs What would one of those breakthroughs hypothetically look like, if it's not an LLM (which is a robust trained model with a complex scoring system) > So with that in mind, while it isn't a silver bullet which is guaranteed to lead to AGI Could you give me an example on what AGI looks like/means to you specifically? People say like "oh, once AGI is here, it'll automate some tasks humans do today away and free them up to do other things" Can you think of a task off the top of your head that is potentially realistically ripe for automation through AGI? I can't.
  - I_Am_Nous 3y ago
    If I were to speculate about breakthroughs in LLMs, in another comment I have been discussing the addition of some kind of "conscience LLM" which acts as an internal dialog so an LLM can have its initial output to a question kind of "thought about" in a back and forth manner (similar to a human debating if they want soup or salad in their head). That inner LLM could be added for safety (to prevent encouraging suicide, or similar) or for accuracy (where a very purpose-trained smaller LLM could ensure output aligns with any requirement) or even as a way to quickly change the performance of an LLM without retraining -- just swap the "conscience" LLM to something different. I'd be surprised if this sort of "middle man" LLM isn't in use in some project already, but even if LLMs themselves don't have a major breakthrough they are still useful tools for certain applications. >Could you give me an example on what AGI looks like/means to you specifically? What I consider AGI would be a system which actually understands the input and output it is working with. Not just "knows how to work with it" but rather if I am discussing a topic with an AGI it has a theory of mind regarding that topic. A system that can think around a topic, draw parallels to potentially unrelated topics, but synthesize how they actually do relate to help generate novel hypotheses. For me, it's a pretty high bar that I don't forsee LLMs reaching alone. Such a system could actually respond if you ask it "How do you know that?" and it can explain each step without losing context after too many extra questions. LLMs could be a part of that system, in the same way it takes multiple systems for humans to be able to speak. >Can you think of a task off the top of your head that is potentially realistically ripe for automation through AGI? Automation isn't the only possible usage for AGI. Of course a crontab entry won't be able to think, but I have seen current industry uses for "AI" that help with tasks humans find tedious such as SIEM ticket monitoring and interpreting syslogs in realtime to keep outages as minimal as they can. Such a system would not meet my requirements to be an AGI, but would still be very useful even if it does not have any true intelligence.
    - MuffinFlavored 3y ago
      > A system that can think around a topic, draw parallels to potentially unrelated topics, but synthesize how they actually do relate to help generate novel hypotheses I feel like ChatGPT can already do this?

I_Am_Nous 3y ago

LLMs are a pretty big breakthrough which can be useful for some applications, but not others. There is always room for futher breakthroughs, and as more time is devoted to research the LLMs of the future could be highly optimized for any number of metrics from power usage to token count. So with that in mind, while it isn't a silver bullet which is guaranteed to lead to AGI, it can certainly fill a role in many other places and is bound to lead to cool tech layered on top of better LLMs. It's nowhere near robust enough to replace every human, but might be enough to at least displace some.

MuffinFlavored 3y ago
> There is always room for further breakthroughs What would one of those breakthroughs hypothetically look like, if it's not an LLM (which is a robust trained model with a complex scoring system) > So with that in mind, while it isn't a silver bullet which is guaranteed to lead to AGI Could you give me an example on what AGI looks like/means to you specifically? People say like "oh, once AGI is here, it'll automate some tasks humans do today away and free them up to do other things" Can you think of a task off the top of your head that is potentially realistically ripe for automation through AGI? I can't.
- I_Am_Nous 3y ago
  If I were to speculate about breakthroughs in LLMs, in another comment I have been discussing the addition of some kind of "conscience LLM" which acts as an internal dialog so an LLM can have its initial output to a question kind of "thought about" in a back and forth manner (similar to a human debating if they want soup or salad in their head). That inner LLM could be added for safety (to prevent encouraging suicide, or similar) or for accuracy (where a very purpose-trained smaller LLM could ensure output aligns with any requirement) or even as a way to quickly change the performance of an LLM without retraining -- just swap the "conscience" LLM to something different. I'd be surprised if this sort of "middle man" LLM isn't in use in some project already, but even if LLMs themselves don't have a major breakthrough they are still useful tools for certain applications. >Could you give me an example on what AGI looks like/means to you specifically? What I consider AGI would be a system which actually understands the input and output it is working with. Not just "knows how to work with it" but rather if I am discussing a topic with an AGI it has a theory of mind regarding that topic. A system that can think around a topic, draw parallels to potentially unrelated topics, but synthesize how they actually do relate to help generate novel hypotheses. For me, it's a pretty high bar that I don't forsee LLMs reaching alone. Such a system could actually respond if you ask it "How do you know that?" and it can explain each step without losing context after too many extra questions. LLMs could be a part of that system, in the same way it takes multiple systems for humans to be able to speak. >Can you think of a task off the top of your head that is potentially realistically ripe for automation through AGI? Automation isn't the only possible usage for AGI. Of course a crontab entry won't be able to think, but I have seen current industry uses for "AI" that help with tasks humans find tedious such as SIEM ticket monitoring and interpreting syslogs in realtime to keep outages as minimal as they can. Such a system would not meet my requirements to be an AGI, but would still be very useful even if it does not have any true intelligence.
  - MuffinFlavored 3y ago
    > A system that can think around a topic, draw parallels to potentially unrelated topics, but synthesize how they actually do relate to help generate novel hypotheses I feel like ChatGPT can already do this?