4 ms·

It looks like Andrej's definition of "agent" here is an entity that can replace a human employee entirely - from the first few minutes of the conversation: Whe

by simonw 1y ago

It looks like Andrej's definition of "agent" here is an entity that can replace a human employee entirely - from the first few minutes of the conversation: When you’re talking about an agent, or what the labs have in mind and maybe what I have in mind as well, you should think of it almost like an employee or an intern that you would hire to work with you. For example, you work with some employees here. When would you prefer to have an agent like Claude or Codex do that work? Currently, of course they can’t. What would it take for them to be able to do that? Why don’t you do it today? The reason you don’t do it today is because they just don’t work. They don’t have enough intelligence, they’re not multimodal enough, they can’t do computer use and all this stuff. They don’t do a lot of the things you’ve alluded to earlier. They don’t have continual learning. You can’t just tell them something and they’ll remember it. They’re cognitively lacking and it’s just not working. It will take about a decade to work through all of those issues.

bbor 1y ago
Quite telling -- thanks for the insightful comment as always, Simon. Didn't know that, even though I've been discussing this on and off all day on Reddit. He's a smart man with well-reasoned arguments, but I think he's also a bit poisoned by working at such a huge org, with all the constraints that comes with. Like, this: You can’t just tell them something and they’ll remember it. It might take a decade to work through this issue if you just want to put a single LLM in a single computer and have it be a fully-fledged human, sure. And since he works at a company making some of the most advanced LLMs in the world, that perspective makes sense! But of course that's not how it's actually going to be (/already is). LLMs are a necessary part of AGI(/"agents") due to their ability to avoid the Frame Problem[1], but they're far from the only needed thing. We're pretty dang good at "remembering things" with computers already, and connecting that with LLM ensembles isn't going to take anywhere close to 10 years. Arguably, we're already doing it pretty darn well in unified systems[2]... If anyone's unfamiliar and finds my comment interesting, I highly recommend Minsky's work on the Society of Mind, which handled this topic definitively over 20 years ago. Namely; A short summary of "Connectionism and Society of Mind" for laypeople at DARPA: https://apps.dtic.mil/sti/tr/pdf/ADA200313.pdf https://apps.dtic.mil/sti/tr/pdf/ADA200313.pdf A description of the book itself, available via Amazon in 48h or via PDF: https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind By far my favorite paper on the topic of connectionist+symbolist syncreticism, though a tad long: https://www.mit.edu/~dxh/marvin/web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html https://www.mit.edu/~dxh/marvin/web.media.mit.edu/~minsky/pa... [1] https://plato.stanford.edu/entries/frame-problem/ https://plato.stanford.edu/entries/frame-problem/ [2] https://github.com/modelcontextprotocol/servers/tree/main/src/memory https://github.com/modelcontextprotocol/servers/tree/main/sr...
- erichocean 1y ago
  > You can’t just tell them something and they’ll remember it. I find it fascinating that this is the problem people consistently think we're a decade away on. If you can't do this, you don't have employee-like AI agents, you have AI-enhanced scripting. It's basically the first thing you have to be able to do to credibly replace an actual human employee.
eddiewithzato 1y ago
Because that's the definition that is leading to all these investments, the promise that very soon they will reach it. If Altman said plainly that LLMs will never reach that stage, there would be a lot less investment into the industry.
- aik 1y ago
  Hard disagree. You don’t need AGI to transform countless workflows within companies, current LLMs can do it. A lot of the current investments are to help with the demand with current generation LLMs (and use cases we know will keep opening up with incremental improvements). Are you aware of how intensely all the main companies that host leading models (azure, aws, etc) are throttling usage due to not enough data center capacity? (Eg. At my company we have 100x more demand than we can get capacity for, and we’re barely getting started. We have a roadmap with 1000x+ the current demand and we’re a relatively small company.) AGI would be more impactful of course, and some use cases aren’t possible until we have it, but that doesn’t diminish the value of current AI.
  - Culonavirus 1y ago
    Oh look, people with skin in the AI game insist AI is not a massive bubble. More news at 11.
    - aik 1y ago
      We’re a regular old SaaS company that has figured out how to add massive value using AI. I am making no statements about valuations and bubbles. I’m actually guessing there is some bubble / overhype. That doesn’t mean it isn’t still incredibly valuable.
      - rhetocj23 1y ago
        Link? And explain in detail, incrementally, what you have done so we can analyse it for ourselves?
      - player1234 1y ago
sarchertech 1y ago
He’s not just talking about agents good enough to replace workers. He’s talking about whether agents are currently useful at all. >Overall, the models are not there. I feel like the industry is making too big of a jump and is trying to pretend like this is amazing, and it’s not. It’s slop. They’re not coming to terms with it, and maybe they’re trying to fundraise or something like that. I’m not sure what’s going on, but we’re at this intermediate stage. The models are amazing. They still need a lot of work. For now, autocomplete is my sweet spot. But sometimes, for some types of code, I will go to an LLM agent. >They kept trying to mess up the style. They’re way too over-defensive. They make all these try-catch statements. They keep trying to make a production code base, and I have a bunch of assumptions in my code, and it’s okay. I don’t need all this extra stuff in there. So I feel like they’re bloating the code base, bloating the complexity, they keep misunderstanding, they’re using deprecated APIs a bunch of times. It’s a total mess. It’s just not net useful. I can go in, I can clean it up, but it’s not net useful.
- consumer451 1y ago
  I am just some shmoe, but I agree with that assessment. My biggest take-away is that we got super lucky. At least now we have a slight chance to prepare for the potential economic and social impacts.
  - Bengalilol 1y ago
    I am thinking the same. And we should start considering on what makes us humans and how we can valorize our common ground.
    - tablatom 1y ago
      This. I believe it’s the most important question in the world right now. I’ve been thinking long and hard about this from an entirely practical perspective and have surprised myself that the answer seems to be our capacity to love. The idea is easily dismissed as romantic but when I say I’m being practical I really mean it. I’m writing about it here https://giftcommunity.substack.com/ https://giftcommunity.substack.com/
ambicapter 1y ago
Do you have a comment? Most of what you've said here is a quote.
- simonw 1y ago
  This is part of my wider hobby of collecting definitions of "agents" - you can see more in my collection here: https://simonwillison.net/tags/agent-definitions/ https://simonwillison.net/tags/agent-definitions/ In this case the specific definition matters because the title of the HN submission is "it will take a decade to work through the issues with agents."

by simonw 1y ago

bbor 1y ago

Quite telling -- thanks for the insightful comment as always, Simon. Didn't know that, even though I've been discussing this on and off all day on Reddit. He's a smart man with well-reasoned arguments, but I think he's also a bit poisoned by working at such a huge org, with all the constraints that comes with. Like, this: You can’t just tell them something and they’ll remember it. It might take a decade to work through this issue if you just want to put a single LLM in a single computer and have it be a fully-fledged human, sure. And since he works at a company making some of the most advanced LLMs in the world, that perspective makes sense! But of course that's not how it's actually going to be (/already is). LLMs are a necessary part of AGI(/"agents") due to their ability to avoid the Frame Problem[1], but they're far from the only needed thing. We're pretty dang good at "remembering things" with computers already, and connecting that with LLM ensembles isn't going to take anywhere close to 10 years. Arguably, we're already doing it pretty darn well in unified systems[2]... If anyone's unfamiliar and finds my comment interesting, I highly recommend Minsky's work on the Society of Mind, which handled this topic definitively over 20 years ago. Namely; A short summary of "Connectionism and Society of Mind" for laypeople at DARPA: https://apps.dtic.mil/sti/tr/pdf/ADA200313.pdf https://apps.dtic.mil/sti/tr/pdf/ADA200313.pdf A description of the book itself, available via Amazon in 48h or via PDF: https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind By far my favorite paper on the topic of connectionist+symbolist syncreticism, though a tad long: https://www.mit.edu/~dxh/marvin/web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html https://www.mit.edu/~dxh/marvin/web.media.mit.edu/~minsky/pa... [1] https://plato.stanford.edu/entries/frame-problem/ https://plato.stanford.edu/entries/frame-problem/ [2] https://github.com/modelcontextprotocol/servers/tree/main/src/memory https://github.com/modelcontextprotocol/servers/tree/main/sr...

erichocean 1y ago
> You can’t just tell them something and they’ll remember it. I find it fascinating that this is the problem people consistently think we're a decade away on. If you can't do this, you don't have employee-like AI agents, you have AI-enhanced scripting. It's basically the first thing you have to be able to do to credibly replace an actual human employee.

eddiewithzato 1y ago

Because that's the definition that is leading to all these investments, the promise that very soon they will reach it. If Altman said plainly that LLMs will never reach that stage, there would be a lot less investment into the industry.

aik 1y ago
Hard disagree. You don’t need AGI to transform countless workflows within companies, current LLMs can do it. A lot of the current investments are to help with the demand with current generation LLMs (and use cases we know will keep opening up with incremental improvements). Are you aware of how intensely all the main companies that host leading models (azure, aws, etc) are throttling usage due to not enough data center capacity? (Eg. At my company we have 100x more demand than we can get capacity for, and we’re barely getting started. We have a roadmap with 1000x+ the current demand and we’re a relatively small company.) AGI would be more impactful of course, and some use cases aren’t possible until we have it, but that doesn’t diminish the value of current AI.
- Culonavirus 1y ago
  Oh look, people with skin in the AI game insist AI is not a massive bubble. More news at 11.
  - aik 1y ago
    We’re a regular old SaaS company that has figured out how to add massive value using AI. I am making no statements about valuations and bubbles. I’m actually guessing there is some bubble / overhype. That doesn’t mean it isn’t still incredibly valuable.
    - rhetocj23 1y ago
      Link? And explain in detail, incrementally, what you have done so we can analyse it for ourselves?
    - player1234 1y ago

sarchertech 1y ago

He’s not just talking about agents good enough to replace workers. He’s talking about whether agents are currently useful at all. >Overall, the models are not there. I feel like the industry is making too big of a jump and is trying to pretend like this is amazing, and it’s not. It’s slop. They’re not coming to terms with it, and maybe they’re trying to fundraise or something like that. I’m not sure what’s going on, but we’re at this intermediate stage. The models are amazing. They still need a lot of work. For now, autocomplete is my sweet spot. But sometimes, for some types of code, I will go to an LLM agent. >They kept trying to mess up the style. They’re way too over-defensive. They make all these try-catch statements. They keep trying to make a production code base, and I have a bunch of assumptions in my code, and it’s okay. I don’t need all this extra stuff in there. So I feel like they’re bloating the code base, bloating the complexity, they keep misunderstanding, they’re using deprecated APIs a bunch of times. It’s a total mess. It’s just not net useful. I can go in, I can clean it up, but it’s not net useful.

consumer451 1y ago
I am just some shmoe, but I agree with that assessment. My biggest take-away is that we got super lucky. At least now we have a slight chance to prepare for the potential economic and social impacts.
- Bengalilol 1y ago
  I am thinking the same. And we should start considering on what makes us humans and how we can valorize our common ground.
  - tablatom 1y ago
    This. I believe it’s the most important question in the world right now. I’ve been thinking long and hard about this from an entirely practical perspective and have surprised myself that the answer seems to be our capacity to love. The idea is easily dismissed as romantic but when I say I’m being practical I really mean it. I’m writing about it here https://giftcommunity.substack.com/ https://giftcommunity.substack.com/

ambicapter 1y ago

Do you have a comment? Most of what you've said here is a quote.

simonw 1y ago
This is part of my wider hobby of collecting definitions of "agents" - you can see more in my collection here: https://simonwillison.net/tags/agent-definitions/ https://simonwillison.net/tags/agent-definitions/ In this case the specific definition matters because the title of the HN submission is "it will take a decade to work through the issues with agents."