4 ms·

Thanks for sharing this. I'm actually starting to explore integrating an agent into one of my SaaS solutions, based on a client request. To be honest, my exper

by ednite 1y ago

Thanks for sharing this. I'm actually starting to explore integrating an agent into one of my SaaS solutions, based on a client request. To be honest, my experience with agents is still pretty limited, so I’d really appreciate any advice, especially around best practices or a roadmap for implementation. The goal is to build something that can learn and reflect the company’s culture, answer situational questions like “what to do in this case,” assist with document error checking, and generally serve as a helpful internal assistant. All of this came from the client's desire to have a tool that aligns with their internal knowledge and workflows. Is something like this feasible in terms of quality and reliability? And beyond hallucinations, are there major security or roadblocks concerns I should be thinking about?

eric-burel 1y ago
The complexity of an agent may range from something relatively simple to whatever level of complexity you want. So your projet sounds doable but you'll have to run some exploration to get proper answers. Regarding reliability, quality and security, it is as important to learn how to observe an agent system than learning how to implement an agent system. An agent/LLM-based solution is proven to work only if you observe that it actually works, experiments, tests and monitoring are not optional like eg in web development. As for security concerns, you'd want to take a look at the OWASP top 10 for LLMs: https://owasp.org/www-project-top-10-for-large-language-model-applications/ https://owasp.org/www-project-top-10-for-large-language-mode... LLMs/agents indeed have their own new set of vulnerabilities.
- ednite 1y ago
  That’s sound advice, really appreciate the link. Regarding your point about continuous monitoring, that’s actually the first thing I mentioned to the client. It’s still highly experimental and needs to be observed, corrected, and tweaked constantly, kind of like teaching a child, where feedback and reinforcement are key. I may share my experience with the HN community down the line. Thanks again!
abelanger 1y ago
I'm a big fan of https://github.com/humanlayer/12-factor-agents https://github.com/humanlayer/12-factor-agents because I think it gets at the heart of engineering these systems for usage in your app rather than a completely unconstrained demo or MCP-based solution. In particular you can reduce most concerns around security and reliability when you treat your LLM call as a library method with structured output (Factor 4) and own your own control flow (Factor 8). There should never be a case where your agent is calling a tool with unconstrained input.
- ednite 1y ago
  I guess I’ve got some reading and research ahead of me. I definitely would rather support the idea of treating LLM calls more like structured library functions, rather than letting them run wild. Definitely bookmarking this for reference. Appreciate you sharing it.
ursaguild 1y ago
Ingesting documents and using natural language to search your org docs with an internal assistant sounds more like a good use case for RAG[1]. Agents are best when you need to autonomously plan and execute a series of actions[2]. You can combine the two but knowing when depends on the use case. I really like the OpenAI approach and how they outlined the thought process of when and how to use agents. [1] https://www.willowtreeapps.com/craft/retrieval-augmented-generation https://www.willowtreeapps.com/craft/retrieval-augmented-gen... [2] https://www.willowtreeapps.com/craft/building-ai-agents-with-plan-and-execute https://www.willowtreeapps.com/craft/building-ai-agents-with...
- ednite 1y ago
  Interesting, and thanks for explanations. In this case, the agent would also need to learn from new events, like project lessons learned, for example. Just curious: can a RAG[1] system actually learn from new situations over time in this kind of setup, or is it purely pulling from what's already there?
  - mousetree 1y ago
    You can ingest new documents and data into the RAG system as you need
  - ursaguild 1y ago
    Especially with a client, consider the word choices around "learning". When using llms, agents, or rag, the system isn't learning (yet) but making a decision based on the context you provide. Most models are a fixed snapshot. If you provide up to date information, it will be able to give you an output based on that. "Learning" happens when initially training the llm or arguably when fine-tuning. Neither of which are needed for your use case as presented.
    - ednite 1y ago
      Thanks for the clarification, really appreciate it. It helps frame things more precisely. In my case, there will be a large amount of initial data fed into the system as context. But the client also expects the agent to act more like a smart assistant or teacher, one that can respond to new, evolving scenarios. Without getting into too much detail, imagine I feed the system an instruction like: “Box A and Box B should fit into Box 1 with at least 1" clearance.” Later, a user gives the agent Box A, Box B, and now adds Box D and E, and asks it to fit everything into Box 1, which is too small. The expected behavior would be that the agent infers that an additional Box 2 is needed to accommodate everything. So I understand this isn't "learning" in the training sense, but rather pattern recognition and contextual reasoning based on prior examples and constraints. Basically, I should be saying "contextual reasoning" instead of "learning." Does that framing make sense?
trevinhofmann 1y ago
Others have given some decent advice based on your comment, but would you be interested in a ~30 minute (video) call to dive a bit deeper so I can give more tailored suggestions?
glompers 1y ago
Provided the goal is solid internal culture at the company, the company would have an automatic dart thrower that reliably hits a target that is the wrong target altogether. Supposing you hired a consultant to be "culture keeper" for this company -- and she or he said, "I'm just going to reason about context by treating this culture as a body of text" -- you would instantly assume that they didn't have skin in the game and didn't understand how culture actually grows and accretes, let alone monitoring and validating eventual quality or reliability. We can't read about what rules apply in some foreign culture's situations and then remotely prescribe what to do socially in a foreign culture we've never set foot in. We can't accurately anticipate even the second-order effects of our recommendations in that situation. We simply have to participate first. It would be better for this to be a role that involves someone inside of the company who does participate in navigating the culture themselves so that they make accurate observations from experience. A person trustworthy enough to steward this culture would also necessarily be trustworthy enough not to alarm the chief of HR. Based on my model of how work works, from experience, I am wondering if they imagine they want this sensitive role filled with a nonhuman 'trusted' advisor so that it can't ever become a social shadow power center within the firm. Or maybe they don't want to admit that modeling culture is beyond the reach of their matter-of-fact internal process models and simulations, and they're just wishfully hoping you can abstract away all of the soft elements without producing social fever dreams or ever having to develop a costly true soft element model. But then you absolutely abstract away where the rubber meets the road! That's quite a roadblock, to be honest with you.
- ednite 1y ago
  Oh, your response basically captures hours of conversation I’ve had with that and many other clients. I completely agree, no AI tool can replace years of experience within a company. Trying to model that too literally risks creating misunderstandings, or worse, damaging trust and reputation. It definitely will never be a replacement for HR or top executive thinking. At best, I’ll be proposing something much lighter, more like a glorified internal search tool for real user examples. To be honest, I’m still all figuring it out. Best case: a helpful resource guide. Worst case: it adds no real value. The tricky part is, if I don’t provide something, even just a prototype, they’re already looking at other consultants who’ll happily promise the moon. And that’s my bigger concern: if I’m not involved, someone else might introduce a half-baked solution that interferes with the SaaS I’ve already built for them. So now I’m in a position where I need to put together a clear, honest demo that shows what this tech can and can’t do, just to avoid further complications down the line. Ironically, this all started when another “AI expert” sold them the idea. I’ve been saying the same thing all along, we’re not quite there yet. Maybe one day, but not now. I also get that businesses want to take full advantage of this tech when it’s pitched as a money-saving opportunity, the pressure to act fast is real. I wonder how many other devs and consultants are facing similar situations?

by ednite 1y ago

eric-burel 1y ago

The complexity of an agent may range from something relatively simple to whatever level of complexity you want. So your projet sounds doable but you'll have to run some exploration to get proper answers. Regarding reliability, quality and security, it is as important to learn how to observe an agent system than learning how to implement an agent system. An agent/LLM-based solution is proven to work only if you observe that it actually works, experiments, tests and monitoring are not optional like eg in web development. As for security concerns, you'd want to take a look at the OWASP top 10 for LLMs: https://owasp.org/www-project-top-10-for-large-language-model-applications/ https://owasp.org/www-project-top-10-for-large-language-mode... LLMs/agents indeed have their own new set of vulnerabilities.

ednite 1y ago
That’s sound advice, really appreciate the link. Regarding your point about continuous monitoring, that’s actually the first thing I mentioned to the client. It’s still highly experimental and needs to be observed, corrected, and tweaked constantly, kind of like teaching a child, where feedback and reinforcement are key. I may share my experience with the HN community down the line. Thanks again!

abelanger 1y ago

I'm a big fan of https://github.com/humanlayer/12-factor-agents https://github.com/humanlayer/12-factor-agents because I think it gets at the heart of engineering these systems for usage in your app rather than a completely unconstrained demo or MCP-based solution. In particular you can reduce most concerns around security and reliability when you treat your LLM call as a library method with structured output (Factor 4) and own your own control flow (Factor 8). There should never be a case where your agent is calling a tool with unconstrained input.

ednite 1y ago
I guess I’ve got some reading and research ahead of me. I definitely would rather support the idea of treating LLM calls more like structured library functions, rather than letting them run wild. Definitely bookmarking this for reference. Appreciate you sharing it.

ursaguild 1y ago

Ingesting documents and using natural language to search your org docs with an internal assistant sounds more like a good use case for RAG[1]. Agents are best when you need to autonomously plan and execute a series of actions[2]. You can combine the two but knowing when depends on the use case. I really like the OpenAI approach and how they outlined the thought process of when and how to use agents. [1] https://www.willowtreeapps.com/craft/retrieval-augmented-generation https://www.willowtreeapps.com/craft/retrieval-augmented-gen... [2] https://www.willowtreeapps.com/craft/building-ai-agents-with-plan-and-execute https://www.willowtreeapps.com/craft/building-ai-agents-with...

ednite 1y ago
Interesting, and thanks for explanations. In this case, the agent would also need to learn from new events, like project lessons learned, for example. Just curious: can a RAG[1] system actually learn from new situations over time in this kind of setup, or is it purely pulling from what's already there?
- mousetree 1y ago
  You can ingest new documents and data into the RAG system as you need
- ursaguild 1y ago
  Especially with a client, consider the word choices around "learning". When using llms, agents, or rag, the system isn't learning (yet) but making a decision based on the context you provide. Most models are a fixed snapshot. If you provide up to date information, it will be able to give you an output based on that. "Learning" happens when initially training the llm or arguably when fine-tuning. Neither of which are needed for your use case as presented.
  - ednite 1y ago
    Thanks for the clarification, really appreciate it. It helps frame things more precisely. In my case, there will be a large amount of initial data fed into the system as context. But the client also expects the agent to act more like a smart assistant or teacher, one that can respond to new, evolving scenarios. Without getting into too much detail, imagine I feed the system an instruction like: “Box A and Box B should fit into Box 1 with at least 1" clearance.” Later, a user gives the agent Box A, Box B, and now adds Box D and E, and asks it to fit everything into Box 1, which is too small. The expected behavior would be that the agent infers that an additional Box 2 is needed to accommodate everything. So I understand this isn't "learning" in the training sense, but rather pattern recognition and contextual reasoning based on prior examples and constraints. Basically, I should be saying "contextual reasoning" instead of "learning." Does that framing make sense?

trevinhofmann 1y ago

Others have given some decent advice based on your comment, but would you be interested in a ~30 minute (video) call to dive a bit deeper so I can give more tailored suggestions?

glompers 1y ago

Provided the goal is solid internal culture at the company, the company would have an automatic dart thrower that reliably hits a target that is the wrong target altogether. Supposing you hired a consultant to be "culture keeper" for this company -- and she or he said, "I'm just going to reason about context by treating this culture as a body of text" -- you would instantly assume that they didn't have skin in the game and didn't understand how culture actually grows and accretes, let alone monitoring and validating eventual quality or reliability. We can't read about what rules apply in some foreign culture's situations and then remotely prescribe what to do socially in a foreign culture we've never set foot in. We can't accurately anticipate even the second-order effects of our recommendations in that situation. We simply have to participate first. It would be better for this to be a role that involves someone inside of the company who does participate in navigating the culture themselves so that they make accurate observations from experience. A person trustworthy enough to steward this culture would also necessarily be trustworthy enough not to alarm the chief of HR. Based on my model of how work works, from experience, I am wondering if they imagine they want this sensitive role filled with a nonhuman 'trusted' advisor so that it can't ever become a social shadow power center within the firm. Or maybe they don't want to admit that modeling culture is beyond the reach of their matter-of-fact internal process models and simulations, and they're just wishfully hoping you can abstract away all of the soft elements without producing social fever dreams or ever having to develop a costly true soft element model. But then you absolutely abstract away where the rubber meets the road! That's quite a roadblock, to be honest with you.

ednite 1y ago
Oh, your response basically captures hours of conversation I’ve had with that and many other clients. I completely agree, no AI tool can replace years of experience within a company. Trying to model that too literally risks creating misunderstandings, or worse, damaging trust and reputation. It definitely will never be a replacement for HR or top executive thinking. At best, I’ll be proposing something much lighter, more like a glorified internal search tool for real user examples. To be honest, I’m still all figuring it out. Best case: a helpful resource guide. Worst case: it adds no real value. The tricky part is, if I don’t provide something, even just a prototype, they’re already looking at other consultants who’ll happily promise the moon. And that’s my bigger concern: if I’m not involved, someone else might introduce a half-baked solution that interferes with the SaaS I’ve already built for them. So now I’m in a position where I need to put together a clear, honest demo that shows what this tech can and can’t do, just to avoid further complications down the line. Ironically, this all started when another “AI expert” sold them the idea. I’ve been saying the same thing all along, we’re not quite there yet. Maybe one day, but not now. I also get that businesses want to take full advantage of this tech when it’s pitched as a money-saving opportunity, the pressure to act fast is real. I wonder how many other devs and consultants are facing similar situations?