4 ms·
Morden software consists of separation of responsibilities between services and a higher plane orchestrating the data flow for business logic. If you believe t
by abshkbh 1y ago
Morden software consists of separation of responsibilities between services and a higher plane orchestrating the data flow for business logic.
If you believe there is value in fuzzy tasks being done by LLMs then from that it follows that having separate "agent" services with a higher order orchestrator would be required. Each calling LLMs on their own inside.
- simonw 1y agoI don't buy it. Why would I want my LLM to talk to some other LLM and introduce even more space for weird, non-deterministic bugs when I could have my LLM call a deterministic API to achieve the same thing?
- octopoc 1y agoIsn't an agent just a system prompt + specific tools? Why not just switch out the system prompt and tools in the same context?
- IanCal 1y agoNo. You have anything else that modifies the context, tools, model, and most importantly perhaps the iteration that controls what's going on with those other values.
- IanCal 1y agoWhy do you assume there's a deterministic API doing the same thing?
- simonw 1y agoBecause if a company built an LLM that can perform actions, they almost certainly did that by building an API first for it to use as a tool.
- IanCal 1y agoBut so much more besides that, including the model itself, RAG, the agentic workflow control, moderation, etc. There's also a huge factor of maintenance, that's a key reason why companies have different internal and external APIs - they don't just open up everything internal and hand the code for managing all of that to you. Interface design is really important. Not to mention the cost being a factor here - who pays for which part.
- simonw 1y agoOffering up an LLM-fronted "agent" for people to send their LLMs to talk to feels a whole lot more expensive and complicated to me than operating a traditional API endpoint.
- IanCal 1y agoA traditional API endpoint wrapping an agent? That's pretty much what this is but as a standard so we don't need to build thousands of them.
- phillipcarter 1y agoBecause there often is?