3 ms·

I put the agents in quotes because anthropic actually talks more about what they call "workflows". And imo this is where the real value of LLMs currently lies,

I put the agents in quotes because anthropic actually talks more about what they call "workflows". And imo this is where the real value of LLMs currently lies, workflow automation. They also say that using LangChain and other frameworks is mostly unnecessary and does more harm than good. They instead argue to use some simple patterns, directly on the API level. Not dis-similar to the old-school Gang of Four software engineering patterns. Really like this post as a guidance for how to actually build useful tools with LLMs. Keep it simple, stupid.

curious_cat_163 2y ago
Indeed. Very clarifying. I would just posit that they do make a distinction between workflows and agents
Philpax 2y ago
Aren't you editorialising by doing so?
- jascha_eng 2y ago
  I guess a little. I really liked the read though, it put in words what I couldn't and I was curious if others felt the same. However the post was posted here yesterday and didn't really have a lot of traction. I thought this was partially because of the term agentic, which the community seems a bit fatigued by. So I put it in quotes to highlight that Anthropic themselves deems it a little vague and hopefully spark more interest. I don't think it messes with their message too much? Honestly it didn't matter anyways, without second chance pooling this post would have been lost again (so thanks Daniel!)
rybosome 2y ago
I felt deeply vindicated by their assessment of these frameworks, in particular LangChain. I've built and/or worked on a few different LLM-based workflows, and LangChain definitely makes things worse in my opinion. What it boils down to is that we are still coming to understand the right patterns of development for how to develop agents and agentic workflows. LangChain made choices about how to abstract things that are not general or universal enough to be useful.
- wahnfrieden 2y ago
  The article does not mention the LangChain framework. LangGraph is a different framework, have you tried it?
  - rybosome 2y ago
    Yes, our previous lead dev built a lot of our infra using LangGraph. I've been slowly ripping it out since assuming ownership of this part of the codebase. I've been replacing LangGraph with simple primitives, relying on native Python constructs, etc. For example, instead of building this verbose graph of computation with LangGraph, you can just...call functions in the order you want them. Or declare them async, add them to a list, then await the resolution of all of them. For a time I was maintaining a spreadsheet of all the refactor PRs, and I had a cumulative reduction of over 1,000 lines of code from these changes. Eventually I stopped keeping track. ^ That's a 1K LOC reduction with no functionality changes. I feel pretty strongly that LangChain/LangGraph are a net negative for our use case.
    - temp_praneshp 2y ago
      So all you achieved from a spreadsheet's worth of PRs was a 1k LoC reduction?
      - herecomethefuzz 2y ago
        If you do any software engineering all, you would know that a 1k LoC reduction to achieve the same functionality at the same/better performance is non-trivial.
        wahnfrieden 2y ago
        They’re questioning whether it was a valuable use of time, not whether a spreadsheet of PRs was time-consuming which is apparent
    - ahmadawais 2y ago
ankit219 2y ago
Deploying in production, the current agentic systems do not really work well. Workflow automation does. The reason is very native to LLMs, but also incredibly basic. Every agentic system starts with planning and reasoning module, where an LLM evaluates the task given and plans about how to accomplish that task, before moving on to next steps. When an agent is given a task, they inevitably come up with different plans on different tries due to inherent nature of LLMs. Most companies like this step to be predictable, and they end up removing it from the system and doing it manually. Thus turning it into a workflow automation vs an agentic system. I think this is what people actually mean when they want to deploy agents in production. LLMs are great at automation*, not great at problem solving. Examples I have seen - customer support (you want predictability), lead mining, marketing copy generation, code flows and architecture, product specs generation, etc. The next leap for AI systems is going to be whether they can solve challenging problems at companies - being the experts vs the doing the task they are assigned. They should really be called agents, not the current ones.
Kydlaw 2y ago
In fact they are mentioning LangGraph (the agent framework from the LangChain company). Imo LangGraph is a much more thoughtful and better built piece of software than the LangChain framework. As I said, they already mention LangGraph in the article, so the Anthropic's conclusions still hold (i.e. KISS). But this thread is going in the wrong direction when talking about LangChain
- jascha_eng 2y ago
  I'm lumping them all in the same category tbh. They say to just use the model libraries directly or a thin abstraction layer (like litellm maybe?) if you want to keep flexibility to change models easily.

curious_cat_163 2y ago

Indeed. Very clarifying. I would just posit that they do make a distinction between workflows and agents

Philpax 2y ago

Aren't you editorialising by doing so?

jascha_eng 2y ago
I guess a little. I really liked the read though, it put in words what I couldn't and I was curious if others felt the same. However the post was posted here yesterday and didn't really have a lot of traction. I thought this was partially because of the term agentic, which the community seems a bit fatigued by. So I put it in quotes to highlight that Anthropic themselves deems it a little vague and hopefully spark more interest. I don't think it messes with their message too much? Honestly it didn't matter anyways, without second chance pooling this post would have been lost again (so thanks Daniel!)

rybosome 2y ago

I felt deeply vindicated by their assessment of these frameworks, in particular LangChain. I've built and/or worked on a few different LLM-based workflows, and LangChain definitely makes things worse in my opinion. What it boils down to is that we are still coming to understand the right patterns of development for how to develop agents and agentic workflows. LangChain made choices about how to abstract things that are not general or universal enough to be useful.

wahnfrieden 2y ago
The article does not mention the LangChain framework. LangGraph is a different framework, have you tried it?
- rybosome 2y ago
  Yes, our previous lead dev built a lot of our infra using LangGraph. I've been slowly ripping it out since assuming ownership of this part of the codebase. I've been replacing LangGraph with simple primitives, relying on native Python constructs, etc. For example, instead of building this verbose graph of computation with LangGraph, you can just...call functions in the order you want them. Or declare them async, add them to a list, then await the resolution of all of them. For a time I was maintaining a spreadsheet of all the refactor PRs, and I had a cumulative reduction of over 1,000 lines of code from these changes. Eventually I stopped keeping track. ^ That's a 1K LOC reduction with no functionality changes. I feel pretty strongly that LangChain/LangGraph are a net negative for our use case.
  - temp_praneshp 2y ago
    So all you achieved from a spreadsheet's worth of PRs was a 1k LoC reduction?
    - herecomethefuzz 2y ago
      If you do any software engineering all, you would know that a 1k LoC reduction to achieve the same functionality at the same/better performance is non-trivial.
      - wahnfrieden 2y ago
        They’re questioning whether it was a valuable use of time, not whether a spreadsheet of PRs was time-consuming which is apparent
  - ahmadawais 2y ago

ankit219 2y ago

Deploying in production, the current agentic systems do not really work well. Workflow automation does. The reason is very native to LLMs, but also incredibly basic. Every agentic system starts with planning and reasoning module, where an LLM evaluates the task given and plans about how to accomplish that task, before moving on to next steps. When an agent is given a task, they inevitably come up with different plans on different tries due to inherent nature of LLMs. Most companies like this step to be predictable, and they end up removing it from the system and doing it manually. Thus turning it into a workflow automation vs an agentic system. I think this is what people actually mean when they want to deploy agents in production. LLMs are great at automation*, not great at problem solving. Examples I have seen - customer support (you want predictability), lead mining, marketing copy generation, code flows and architecture, product specs generation, etc. The next leap for AI systems is going to be whether they can solve challenging problems at companies - being the experts vs the doing the task they are assigned. They should really be called agents, not the current ones.

Kydlaw 2y ago

In fact they are mentioning LangGraph (the agent framework from the LangChain company). Imo LangGraph is a much more thoughtful and better built piece of software than the LangChain framework. As I said, they already mention LangGraph in the article, so the Anthropic's conclusions still hold (i.e. KISS). But this thread is going in the wrong direction when talking about LangChain

jascha_eng 2y ago
I'm lumping them all in the same category tbh. They say to just use the model libraries directly or a thin abstraction layer (like litellm maybe?) if you want to keep flexibility to change models easily.