4 ms·

Agent Is Not the Model

74 points by joejag 1mo ago

ericol 1mo ago
On other news, water wets. > I often hear people use the words agent and model interchangeably _what_ people. Would I hear one of my colleagues do this, I'll slap them across the face. With a 4 pounds salmon. Alive. > to help us have more precise conversations. What problem are you trying to solve. _Why_ you need more precise conversations. I mean, I understand what you aiming at. But is it really worth it to go nitpicking at people's mental models, is the gain worth it?
- teekert 1mo ago
  Well, at least as an LLM provider you should use the right nomenclature. I just tried to sign up for Mistral. Who have Vibe (former le Chat), then they have Vibe Code, which is the same as Vibe for Code, is that like Claude Code? No, their harness is called Vibe Cli. So is Vibe Code a model? No, it is a "mode" for Vibe (the web interface). Not sure how it's different from "Chat" (the mode) but it forces you to use a project, there are no other differences it seems. No idea what the underlying model is for any of this. More over, I don't ever vibe code, I check and understand the code that is generated by my LLMs. And yet, I use Vibe Code (the product) all day. Lost the thread yet? I did... Tbh, it also took some time between Anthropic starting the push towards Claude Code and me understanding what is really was. Using terms interchangeably during this time of discovery is absolutely maddening. For Mistral it comes on top of their rename of services from "le Chat" and Mistral Code (still in parts of the UI) to Vibe and Vibe (for) Code.
- the_af 1mo ago
  From TFA: > When you can name the layer, you can fix the layer. That is the whole point of being precise. It is not about being pedantic. It is about being able to improve things faster and more effectively. Also, for any given fact, tons of people aren't aware. Anything you already know is news to a sizeable number of people. Relevant xkcd: https://xkcd.com/1053/ https://xkcd.com/1053/
  - ericol 1mo ago
    I get your point, my 80 yo mother doesn't care.
    - the_af 1mo ago
      I don't think the TFA has your 80 year old mother as its intended audience.
      - ericol 1mo ago
        It doesn't define one.
        the_af 1mo ago
        Yes, it does. It's hosted in a software development blog called "Just Another Dev", with other entries such as "I'm AWS certified? Should you trust me?" or "Synthetic Monitoring with Cypress". Is your 80 year old mother a software developer? If not, she's not the audience. Don't pretend you don't understand this.
  - ericol 1mo ago
    The irony that you sent an xkcd link to prove your point is not lost on me. > Also, for any given fact, tons of people aren't aware. Double points for Gryffindor.
- esafak 1mo ago
  Totally the wrong site to post this on. Joe, don't you see that we talk about this stuff day in day out?
- altmanaltman 1mo ago
  Wait, your entire comment is on how one should not nitpick someone's mental models but isn't that you nitpicking at someone's mental models? sus
- yipinwong 1mo ago
  You sound like a manager, not an engineer.
- mickmister 1mo ago
  > _what_ people. Eric from alignment and research at OpenAI: https://www.youtube.com/watch?v=87DyyMV0kCY https://www.youtube.com/watch?v=87DyyMV0kCY It honestly bothers me so much when he says "This new model has access to x". No, the harness you allowed it use at runtime has access to x.
  - yomismoaqui 1mo ago
    You can argue that the model has access to that tool through the harness the same way your brain has access to see this comment through your body (your eyes specifically).
    - mickmister 1mo ago
      Sure, but given the situation and audience of this talk, I think they should be more precise with how they word these things. If you watch the video you'll see what I mean. He talks like they have no control over what they give to the model, because the model simply "has access" by default, which is not true.
gustavus 1mo ago
Well first off if you ask Microsoft everything is Copilot. Secondly the confusion is designed to benefit the bull** by using ambiguous language they can do as humpty dumpty did in Alice in Wonderland and say "When I use a word, it means just what I choose it to mean. Neither more nor less" Which benefits whatever they are pushing. Beware those that attempt to muddle language and avoid precision in speaking.
rwoerz 1mo ago
> An agent system is made up of several layers. Why "layers"? The constituents of a Multi-agent System (MAS) [1] are called "agents". BTW: Synecdochical semantic diffusion is not uncommon in software engineering [1] https://en.wikipedia.org/wiki/Multi-agent_system https://en.wikipedia.org/wiki/Multi-agent_system
- yipinwong 1mo ago
  Real-life usage of interchangeable or synecdochical word triumphs in real life. My take on the post is for engineering disciple where JoeJag wants to create a common word while tackling "Agent" issues. I like Joe's approach as this disambiguates during troubleshooting without trying to figure out under which "context" other engineers are using Agent vs. Models. You get lost in context just like AIs do without such disambiguation.
yaaaaam 1mo ago
An agent, in general, is just whatever carries out a task on behalf of someone/something else.
llm_nerd 1mo ago
I've literally never heard anyone conflate an agent and a model. Ever. Often with posts like this I imagine someone had their own confusion and then somehow projected it on everyone else. Like Trump thinking people didn't know about the word groceries or that dumb ends with a b.
- fultonn 1mo ago
  > I've literally never heard anyone conflate an agent and a model. Ever. Author is a senior staff engineer. A big part of his job is to help his org's engineers and non-technical decision makers understand basics. My assumption (because it's my experience) is that he is writing this piece because has seen this confusion in others > N times. And to be fair to those juniors and less-technical folks: big groups of AI phds make the same mistake. Papers in prominent ML conferences up to like 2024-2025 that were consistently presenting comparisons of proprietary model end-points to open weight models as if they were apples-apples comparisons. Even though there were obvious examples of prop model providers doing "stuff" behind the endpoint as early as Fall 2021. Most of those papers did not put in the legwork to figure out if/when that "stuff" was happening, or work to figure out what the "stuff" probably was, or conjecture to what extent that "stuff" might overlap with the benchmarks being used, or try to add basic interventions to open-weight models to get a better comparison point, or even simply state the caveat that they don't actually know what happens when they hit a chat completions endpoint! Not exactly the same thing as conflating agent with model, but 100% conflating "model+software" with "model", which is at the end of the day basically the same confusion. And those papers were being written by and for llm nerds. So if ICML/NeurIPS authors and reviewers aren't careful about "software+model" vs "model" in stating and evaluating their claims, it's unsurprising that some a SWE or non-CS SME also doesn't understand how the LLM software stack works. It doesn't help that some model providers (OpenAI...) actively obfuscate how the stack works in their marketing.
- mfonda 1mo ago
  I have—frequently—especially among the non-technical crowd. For example, the recent-ish OpenAI Hugging Face breakout was widely reported as a rogue model escaping. But a model on its own can’t do anything—it’s the agent/harness that escaped. I think it’s an important distinction and I’m glad to see efforts attempting to clear it up.
agentdev001 1mo ago
The post ends with a comment on "its not about being pedantic..." so, a few not being pedantic bits: In the table "Real world examples"; "Claude Desktop" houses three harnesses at the moment; Claude, Claude Cowork, and Claude Code. "Claude CLI", I presume, is referring to Claude Code CLI. This is distinct from the 'ant CLI', which is sometimes referred to as 'Claude CLI'. "Cursor" could be any of them- but, 'Cursor Agents', 'Cursor Cloud Agents', 'Cursor CLI', and whatever the vscode fork is called now, are distinct. Maybe not in the context of this blog post, but it isnt specified which is being referred to in the example table. "ChatGPT" sounds like the chatgpt web interface. OpenAI's desktop app is named 'ChatGPT Desktop', and now houses 'ChatGPT work' and 'Codex' (Codex Desktop, not the TUI, though it does essentially wrap the tui and give it capabilities through app-built-in tools). I believe the ChatGPT web interface's harness can change a bit, depending on settings + subscription level (remote sandboxes, etc.) Additionally, there is a distinction in available models depending on which "ChatGPT" product is being used (instant/live/etc non-5.6 luna/terra/sol suite). Inference service is more accurately 'default inference provider'. Also, this post has an ai-generated smell.
- joejag 1mo ago
  Author here. My writing process is: * Write: Use FluidVoice to get all the text down in my own voice * Edit: Ask ChatGPT etc to criticise the article * Cleanup: Grammarly * Publish: Good old GitHub Pages
  - karmakaze 1mo ago
    > FluidVoice turns rough, rambling speech into polished, ready-to-send text in any app. Free forever, open source, and 100% on-device.
    - fragmede 1mo ago
      The difference between being a writer and a orator in this day and age, seems to be a bit muddy.
fabsalvadori 1mo ago
[flagged]
azath92 1mo ago
If the goal is to provide a distinction between model and agent, i think the "agent system" is doing too much heavy lifting in the example here. A useful extension to this mental framework that i use when trying to make this distinction is the application (cursor) -> which sometimes includes an orchestrator and all of the QOL stuff like resuming, checkpointing, etc. single or multiple agents (cursor agents)-> and runs a single or many agent instances (single agent in cursor)-> service api-> model. This is to address a confusion i often see with agent being conflated with the application that we use agents in, rather than the distinction in the article which tries to unpick agent-model confusion.
6keZbCECT2uB 1mo ago
A fun one is that in claude code, you can configure 'agents' which are prompt presets + some configuration. Or sub-agents sometime are indistinguishable from the foreground agent (usually called orchestrator) in configuration except that they have different contents in their context window (forks more or less). IMO, if there's a ubiquitous term that is unambiguous, use it (harness, model). If there's an ambiguous term you have to explain, try not to use it. Language is for communication.
th3o6a1d 1mo ago
I like to think of an agent as a LLM chat loop with tool calling. The agentic behavior emerges with these ingredients. Don’t really a harness…unless you consider a shell script that runs that loop a harness.
Linekiller89 1mo ago
[flagged]
ewgenij87snwork 1mo ago
[flagged]

74 points by joejag 1mo ago

On other news, water wets. > I often hear people use the words agent and model interchangeably _what_ people. Would I hear one of my colleagues do this, I'll slap them across the face. With a 4 pounds salmon. Alive. > to help us have more precise conversations. What problem are you trying to solve. _Why_ you need more precise conversations. I mean, I understand what you aiming at. But is it really worth it to go nitpicking at people's mental models, is the gain worth it?

teekert 1mo ago
Well, at least as an LLM provider you should use the right nomenclature. I just tried to sign up for Mistral. Who have Vibe (former le Chat), then they have Vibe Code, which is the same as Vibe for Code, is that like Claude Code? No, their harness is called Vibe Cli. So is Vibe Code a model? No, it is a "mode" for Vibe (the web interface). Not sure how it's different from "Chat" (the mode) but it forces you to use a project, there are no other differences it seems. No idea what the underlying model is for any of this. More over, I don't ever vibe code, I check and understand the code that is generated by my LLMs. And yet, I use Vibe Code (the product) all day. Lost the thread yet? I did... Tbh, it also took some time between Anthropic starting the push towards Claude Code and me understanding what is really was. Using terms interchangeably during this time of discovery is absolutely maddening. For Mistral it comes on top of their rename of services from "le Chat" and Mistral Code (still in parts of the UI) to Vibe and Vibe (for) Code.
the_af 1mo ago
From TFA: > When you can name the layer, you can fix the layer. That is the whole point of being precise. It is not about being pedantic. It is about being able to improve things faster and more effectively. Also, for any given fact, tons of people aren't aware. Anything you already know is news to a sizeable number of people. Relevant xkcd: https://xkcd.com/1053/ https://xkcd.com/1053/
- ericol 1mo ago
  I get your point, my 80 yo mother doesn't care.
  - the_af 1mo ago
    I don't think the TFA has your 80 year old mother as its intended audience.
    - ericol 1mo ago
      It doesn't define one.
      - the_af 1mo ago
        Yes, it does. It's hosted in a software development blog called "Just Another Dev", with other entries such as "I'm AWS certified? Should you trust me?" or "Synthetic Monitoring with Cypress". Is your 80 year old mother a software developer? If not, she's not the audience. Don't pretend you don't understand this.
- ericol 1mo ago
  The irony that you sent an xkcd link to prove your point is not lost on me. > Also, for any given fact, tons of people aren't aware. Double points for Gryffindor.
esafak 1mo ago
Totally the wrong site to post this on. Joe, don't you see that we talk about this stuff day in day out?
altmanaltman 1mo ago
Wait, your entire comment is on how one should not nitpick someone's mental models but isn't that you nitpicking at someone's mental models? sus
yipinwong 1mo ago
You sound like a manager, not an engineer.
mickmister 1mo ago
> _what_ people. Eric from alignment and research at OpenAI: https://www.youtube.com/watch?v=87DyyMV0kCY https://www.youtube.com/watch?v=87DyyMV0kCY It honestly bothers me so much when he says "This new model has access to x". No, the harness you allowed it use at runtime has access to x.
- yomismoaqui 1mo ago
  You can argue that the model has access to that tool through the harness the same way your brain has access to see this comment through your body (your eyes specifically).
  - mickmister 1mo ago
    Sure, but given the situation and audience of this talk, I think they should be more precise with how they word these things. If you watch the video you'll see what I mean. He talks like they have no control over what they give to the model, because the model simply "has access" by default, which is not true.

gustavus 1mo ago

Well first off if you ask Microsoft everything is Copilot. Secondly the confusion is designed to benefit the bull** by using ambiguous language they can do as humpty dumpty did in Alice in Wonderland and say "When I use a word, it means just what I choose it to mean. Neither more nor less" Which benefits whatever they are pushing. Beware those that attempt to muddle language and avoid precision in speaking.

rwoerz 1mo ago

> An agent system is made up of several layers. Why "layers"? The constituents of a Multi-agent System (MAS) [1] are called "agents". BTW: Synecdochical semantic diffusion is not uncommon in software engineering [1] https://en.wikipedia.org/wiki/Multi-agent_system https://en.wikipedia.org/wiki/Multi-agent_system

yipinwong 1mo ago
Real-life usage of interchangeable or synecdochical word triumphs in real life. My take on the post is for engineering disciple where JoeJag wants to create a common word while tackling "Agent" issues. I like Joe's approach as this disambiguates during troubleshooting without trying to figure out under which "context" other engineers are using Agent vs. Models. You get lost in context just like AIs do without such disambiguation.

yaaaaam 1mo ago

An agent, in general, is just whatever carries out a task on behalf of someone/something else.

llm_nerd 1mo ago

I've literally never heard anyone conflate an agent and a model. Ever. Often with posts like this I imagine someone had their own confusion and then somehow projected it on everyone else. Like Trump thinking people didn't know about the word groceries or that dumb ends with a b.

fultonn 1mo ago
> I've literally never heard anyone conflate an agent and a model. Ever. Author is a senior staff engineer. A big part of his job is to help his org's engineers and non-technical decision makers understand basics. My assumption (because it's my experience) is that he is writing this piece because has seen this confusion in others > N times. And to be fair to those juniors and less-technical folks: big groups of AI phds make the same mistake. Papers in prominent ML conferences up to like 2024-2025 that were consistently presenting comparisons of proprietary model end-points to open weight models as if they were apples-apples comparisons. Even though there were obvious examples of prop model providers doing "stuff" behind the endpoint as early as Fall 2021. Most of those papers did not put in the legwork to figure out if/when that "stuff" was happening, or work to figure out what the "stuff" probably was, or conjecture to what extent that "stuff" might overlap with the benchmarks being used, or try to add basic interventions to open-weight models to get a better comparison point, or even simply state the caveat that they don't actually know what happens when they hit a chat completions endpoint! Not exactly the same thing as conflating agent with model, but 100% conflating "model+software" with "model", which is at the end of the day basically the same confusion. And those papers were being written by and for llm nerds. So if ICML/NeurIPS authors and reviewers aren't careful about "software+model" vs "model" in stating and evaluating their claims, it's unsurprising that some a SWE or non-CS SME also doesn't understand how the LLM software stack works. It doesn't help that some model providers (OpenAI...) actively obfuscate how the stack works in their marketing.
mfonda 1mo ago
I have—frequently—especially among the non-technical crowd. For example, the recent-ish OpenAI Hugging Face breakout was widely reported as a rogue model escaping. But a model on its own can’t do anything—it’s the agent/harness that escaped. I think it’s an important distinction and I’m glad to see efforts attempting to clear it up.

agentdev001 1mo ago

The post ends with a comment on "its not about being pedantic..." so, a few not being pedantic bits: In the table "Real world examples"; "Claude Desktop" houses three harnesses at the moment; Claude, Claude Cowork, and Claude Code. "Claude CLI", I presume, is referring to Claude Code CLI. This is distinct from the 'ant CLI', which is sometimes referred to as 'Claude CLI'. "Cursor" could be any of them- but, 'Cursor Agents', 'Cursor Cloud Agents', 'Cursor CLI', and whatever the vscode fork is called now, are distinct. Maybe not in the context of this blog post, but it isnt specified which is being referred to in the example table. "ChatGPT" sounds like the chatgpt web interface. OpenAI's desktop app is named 'ChatGPT Desktop', and now houses 'ChatGPT work' and 'Codex' (Codex Desktop, not the TUI, though it does essentially wrap the tui and give it capabilities through app-built-in tools). I believe the ChatGPT web interface's harness can change a bit, depending on settings + subscription level (remote sandboxes, etc.) Additionally, there is a distinction in available models depending on which "ChatGPT" product is being used (instant/live/etc non-5.6 luna/terra/sol suite). Inference service is more accurately 'default inference provider'. Also, this post has an ai-generated smell.

joejag 1mo ago
Author here. My writing process is: * Write: Use FluidVoice to get all the text down in my own voice * Edit: Ask ChatGPT etc to criticise the article * Cleanup: Grammarly * Publish: Good old GitHub Pages
- karmakaze 1mo ago
  > FluidVoice turns rough, rambling speech into polished, ready-to-send text in any app. Free forever, open source, and 100% on-device.
  - fragmede 1mo ago
    The difference between being a writer and a orator in this day and age, seems to be a bit muddy.

fabsalvadori 1mo ago

[flagged]

azath92 1mo ago

If the goal is to provide a distinction between model and agent, i think the "agent system" is doing too much heavy lifting in the example here. A useful extension to this mental framework that i use when trying to make this distinction is the application (cursor) -> which sometimes includes an orchestrator and all of the QOL stuff like resuming, checkpointing, etc. single or multiple agents (cursor agents)-> and runs a single or many agent instances (single agent in cursor)-> service api-> model. This is to address a confusion i often see with agent being conflated with the application that we use agents in, rather than the distinction in the article which tries to unpick agent-model confusion.

6keZbCECT2uB 1mo ago

A fun one is that in claude code, you can configure 'agents' which are prompt presets + some configuration. Or sub-agents sometime are indistinguishable from the foreground agent (usually called orchestrator) in configuration except that they have different contents in their context window (forks more or less). IMO, if there's a ubiquitous term that is unambiguous, use it (harness, model). If there's an ambiguous term you have to explain, try not to use it. Language is for communication.

th3o6a1d 1mo ago

I like to think of an agent as a LLM chat loop with tool calling. The agentic behavior emerges with these ingredients. Don’t really a harness…unless you consider a shell script that runs that loop a harness.

Linekiller89 1mo ago

[flagged]

ewgenij87snwork 1mo ago

[flagged]