7 ms·

Yann is respected in the field, and he has some good ideas in that paper and others. But he not advancing the state of the art on LLMs. Ie, he talked about a wo

Yann is respected in the field, and he has some good ideas in that paper and others. But he not advancing the state of the art on LLMs. Ie, he talked about a world model and actor/critic model, but it's other people who are exploring LLM driven world/actor/critic models. If you're not reading papers published in the last months, you're not going to see how quickly things are improving, compared to a year ago. Here's just one example: the incredible explosion in context length. A year ago GPT4 would hallucinate critical details from a 10 page 5000 word essay. Today, Gemini 1.5 can read a 1000 page 5 million word textbook, and recall facts with 99.5% accuracy. This is already an AGI level feat. Now it just needs adaptors, memory modules. AGI doesn't imply it can self improve. It's just generally intelligent and capable of many tasks. Off the shelf LLMs with accessory modules will be enough to build AGI capability and change the nature of work and knowledge.

ein0p 3y ago
Yes, we’re now pretty good at associative memory. But associative memory is but a small part of the whole picture. Read the paper, clouds might part a little and enthusiasm might subside. I quite literally work on LLMs as my day job. They alone are not going to somehow get us to AGI
- barfbagginus 3y ago
  Please tell me which sections of the paper you feel suggest that LLMs + extensions cannot achieve AGI. I have read the paper 3 times in the last year, but I'm not seeing it. There's basically no reason you wouldn't be able to implement LeCunn's entire JEPA architecture using LLM analogues. Associative memory is just one of many huge improvements we could discuss And before you object that LeCunn's methods do online learning with energy methods, let me point to RAG systems that also do online learning - and better than anything Lecunn has published. I own a company where I build new computational geometry codes. And I'm increasingly using LLMs to research, plan, implement, test, and debug my codes. Not only that, but I am an autistic person who increasingly uses LLMs to overcome my social and communication difficulties. GPT is far more empathic and constructive with it's suggestions than any friend, therapist, or advisor I have worked with or have access to. So again it already is AGI for the purpose of helping me process my often complex and baffling emotions. Nobody's saying pure LLMs will be AGI. But pure LLMs working together in novel configurations and using specific components already have that capability. Memory, empathy, planning, agency, theory of mind, etc. You name it, I'll show you a paper where LLMs demonstrate ir. GPT4 by itself does these things better than most people. These capabilities will be packaged into off the shelf components. They will be integrated into post-llm systems that merge these capabilities into a system with the most of the capabilities of a conscious human. That'll be AGI. But what we already have is already AGI as well. You're just not giving it credit.
  - staticman2 3y ago
    I was confused why you would say LLM's have agency, but I see there's something like dozens of potential definitions of agency, much like there's probably dozens of potential definitions of AGI.
    - barfbagginus 3y ago
      Here's the most simple definition of agency: Agency is the ability to independently make and execute decisions in an environment. This typically involves three components: 1. Perception: The ability to observe the environment 2. Cognition: the ability to process perceptions and arrive at decisions 3. Action: the ability to act on the environment LLMs can process many kinds of perceptions and make decisions about them - often as well as a human. That means that giving agency to an LLM is often just a matter of giving it an environment it can perceive and act on. There are of course different levels of agency. A General Intelligence is a higher level agent that can handle many different environments and actions. LLMs have two main challenges here. There are environments they can't connect to. And many environments require complex reasoning that LLMs can't do on their own. There are two low hanging approaches here; 1. Adaptors. These are perception and action adaptors that can connect LLMs to many environments. Work here includes multimodal llms, LLM driven robots, single LLM agents, and communities of LLM agents 2. Accelerators: These help LLMs reason about complex environments like math, code, optimal planning, and knowledge bases. Work here includes tool generating LLMs (Toolformer, LLM+P), RAG engines, LLM-driven mixtures of experts etc. Conclusion: Agency is easy Overall LLMs have the cognitive horsepower to be agents in many environments. All they need is efficient and general adaptors and tools. We can modify LLMs to better work with these components, like multimodal LLMs or mixtures of experts. Or we can plug and play different adapters and tools to increase the level of agency, like tool use and RAG. Future generations of AI will likely integrate these features in much more effective ways, achieving higher levels of agency. But even as it stands, LLMs have very real agent capabilities, needing only tools and adaptors.
      - staticman2 3y ago
        I'd say that's a pretty useless definition of agency, it's not clear why a thermostat wouldn't meet that low bar, and with that overly generous definition we now we need a new term "super agency" or whatever to talk about what other people mean by agency.
dkjaudyeqooe 3y ago
> This is already an AGI level feat. This is a fundamental misunderstanding of what AGI is, in the sense that any of these "feats" do not prove AGI in any way. If you can't define it rigorously you can't prove it or detect it. We're just inundated with some version of these. Even those who should be least credulous are swallowing the hype, hook line and sinker.
- barfbagginus 3y ago
  You seem to argue that AGI is undefinable. But it's not hard to define. And once you have my definition, you realize there's nothing to hype. Let's first agree that GI is general intelligence - the ability to make effective observations and statements about general situations. You and I are biological GIs, even though there are some situations that we won't be useful in. In general, we are useful. GPT4 is also a GI, and it's artificial. That's an AGI. There's no hook line or sinker to swallow: 1. Anyone telling you about secret God like AGI is bullshiting you. 2. Anyone claiming gpt4 is not generally useful is also bullshiting you If you are not convinced about item 2, I can show you examples of GPT4 being generally useful for a wide variety of tasks, including soft skills like business planning, emotional analysis, and conflict resolution.

ein0p 3y ago

Yes, we’re now pretty good at associative memory. But associative memory is but a small part of the whole picture. Read the paper, clouds might part a little and enthusiasm might subside. I quite literally work on LLMs as my day job. They alone are not going to somehow get us to AGI

barfbagginus 3y ago
Please tell me which sections of the paper you feel suggest that LLMs + extensions cannot achieve AGI. I have read the paper 3 times in the last year, but I'm not seeing it. There's basically no reason you wouldn't be able to implement LeCunn's entire JEPA architecture using LLM analogues. Associative memory is just one of many huge improvements we could discuss And before you object that LeCunn's methods do online learning with energy methods, let me point to RAG systems that also do online learning - and better than anything Lecunn has published. I own a company where I build new computational geometry codes. And I'm increasingly using LLMs to research, plan, implement, test, and debug my codes. Not only that, but I am an autistic person who increasingly uses LLMs to overcome my social and communication difficulties. GPT is far more empathic and constructive with it's suggestions than any friend, therapist, or advisor I have worked with or have access to. So again it already is AGI for the purpose of helping me process my often complex and baffling emotions. Nobody's saying pure LLMs will be AGI. But pure LLMs working together in novel configurations and using specific components already have that capability. Memory, empathy, planning, agency, theory of mind, etc. You name it, I'll show you a paper where LLMs demonstrate ir. GPT4 by itself does these things better than most people. These capabilities will be packaged into off the shelf components. They will be integrated into post-llm systems that merge these capabilities into a system with the most of the capabilities of a conscious human. That'll be AGI. But what we already have is already AGI as well. You're just not giving it credit.
- staticman2 3y ago
  I was confused why you would say LLM's have agency, but I see there's something like dozens of potential definitions of agency, much like there's probably dozens of potential definitions of AGI.
  - barfbagginus 3y ago
    Here's the most simple definition of agency: Agency is the ability to independently make and execute decisions in an environment. This typically involves three components: 1. Perception: The ability to observe the environment 2. Cognition: the ability to process perceptions and arrive at decisions 3. Action: the ability to act on the environment LLMs can process many kinds of perceptions and make decisions about them - often as well as a human. That means that giving agency to an LLM is often just a matter of giving it an environment it can perceive and act on. There are of course different levels of agency. A General Intelligence is a higher level agent that can handle many different environments and actions. LLMs have two main challenges here. There are environments they can't connect to. And many environments require complex reasoning that LLMs can't do on their own. There are two low hanging approaches here; 1. Adaptors. These are perception and action adaptors that can connect LLMs to many environments. Work here includes multimodal llms, LLM driven robots, single LLM agents, and communities of LLM agents 2. Accelerators: These help LLMs reason about complex environments like math, code, optimal planning, and knowledge bases. Work here includes tool generating LLMs (Toolformer, LLM+P), RAG engines, LLM-driven mixtures of experts etc. Conclusion: Agency is easy Overall LLMs have the cognitive horsepower to be agents in many environments. All they need is efficient and general adaptors and tools. We can modify LLMs to better work with these components, like multimodal LLMs or mixtures of experts. Or we can plug and play different adapters and tools to increase the level of agency, like tool use and RAG. Future generations of AI will likely integrate these features in much more effective ways, achieving higher levels of agency. But even as it stands, LLMs have very real agent capabilities, needing only tools and adaptors.
    - staticman2 3y ago
      I'd say that's a pretty useless definition of agency, it's not clear why a thermostat wouldn't meet that low bar, and with that overly generous definition we now we need a new term "super agency" or whatever to talk about what other people mean by agency.

dkjaudyeqooe 3y ago

> This is already an AGI level feat. This is a fundamental misunderstanding of what AGI is, in the sense that any of these "feats" do not prove AGI in any way. If you can't define it rigorously you can't prove it or detect it. We're just inundated with some version of these. Even those who should be least credulous are swallowing the hype, hook line and sinker.

barfbagginus 3y ago
You seem to argue that AGI is undefinable. But it's not hard to define. And once you have my definition, you realize there's nothing to hype. Let's first agree that GI is general intelligence - the ability to make effective observations and statements about general situations. You and I are biological GIs, even though there are some situations that we won't be useful in. In general, we are useful. GPT4 is also a GI, and it's artificial. That's an AGI. There's no hook line or sinker to swallow: 1. Anyone telling you about secret God like AGI is bullshiting you. 2. Anyone claiming gpt4 is not generally useful is also bullshiting you If you are not convinced about item 2, I can show you examples of GPT4 being generally useful for a wide variety of tasks, including soft skills like business planning, emotional analysis, and conflict resolution.