6 ms·

Lessons from interviews on deploying AI Agents in production

107 points by advikipedia 1y ago

advikipedia 1y ago
We recently spoke with 30+ startup founders and 40+ enterprise practitioners who are building and deploying agentic AI systems across industries like financial services, healthcare, cybersecurity, and developer tooling. A few patterns emerged that might be relevant to anyone working on applied AI or automation: - The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance. - Incremental deployment beats ambition. Successful teams focus on narrow, verifiable use cases that deliver measurable ROI and build user trust before scaling autonomy. - Enterprise adoption is uneven. Many companies have “some agents” in production, but most use them with strong human oversight. The fully autonomous cases remain rare. - Pricing is unresolved. Hybrid models dominate; pure outcome-based pricing is uncommon due to attribution and monitoring challenges. Infrastructure is mostly homegrown. Over half of surveyed startups build their own agentic stacks, citing limited flexibility in existing frameworks. The article also includes detailed case studies, commentary on autonomy vs. accuracy trade-offs, and what’s next for ambient and proactive agents. If you’re building in this space, the full report is free here: https://mmc.vc/research/state-of-agentic-ai-founders-edition/ https://mmc.vc/research/state-of-agentic-ai-founders-edition... Would be interested to hear how others on HN are thinking about real-world deployment challenges — especially around trust, evaluation, and scaling agentic systems.
- Etheryte 1y ago
  Perhaps I simply don't understand what you mean, but it sounds like the first point could be rephrased in some way. To me, workflow integration and data privacy sound very much like technical blockers.
  - barrenko 1y ago
    But if you define them as non-technical related blockers agents are just swell.
  - refactor_master 1y ago
    Consider this simple example: Storing all your sensitive user data in one centralized location (e.g. a US server) would be great for any kind of analytics and modeling to tap into, and is technically very easy to do, but it also violates virtually every country's data privacy laws. So then you have to set up siloed servers around the world, deal with data governance, legal stuff, etc. Sure, it then becomes a technical challenge to work around those limits, but that may be cost/time prohibitive.
    - 1718627440 1y ago
      That sounds more like, that you can solve the problem, when it would have other requirements.
      - refactor_master 1y ago
        If you ask Silicon Valley, any organizational problem can be a technical problem if you try hard enough.
  - IanCal 1y ago
    There are two sides to workflow integration. One is technical (it’s a hassle to connect things to a specific system because you’d need to deal with the api or there is no api) The other isn’t, because it’s figuring out how and where to use these new tools in an existing workflow. Maybe you could design something from scratch but you have lots of business processes right now, how do you smoothly modify that? Where does it make sense? Frankly understanding what the systems can and can’t do takes at least some time even if only because the field is moving so fast (I worked with a small local firm who I was able to help by showing them the dramatic improvements in transcription quality vs cost recently - people here are more used to whisper and the like but it’s not as common knowledge how and where you can use these things).
  - advikipedia 1y ago
    More than the "actual" problem, the "perception" of the problem is worse. Workflow integration is more to do with users having to rethink their workflows, their roles, and how they work with AI. As for data privacy concerns, even where startups have taken measures to overcome the problems, very often enterprises still remain concerned (making this more of a perception problem than an actual problem). That's why I focused on the non-technical aspect of it!
    - DrScientist 1y ago
      When I see vendors complain about workflow and integration issues, it's because the vendors software is written around an expectation of a certain workflow and integration points and they find out in reality every customer does it slightly differently. Some key challenges around workflow are that while the fundamental white-board task flow is the same, different companies may distribute those tasks between people and over time in different ways. Workflow is about flowing the task and associated information between people - not just doing the tasks. Same goes for integration - the timing of when certain necessary information might be available again not uniform and timing concerns are often missed on the high level whiteboard. Here's a classic example of ignoring timing issues. https://www.harrowell.org.uk/blog/2017/03/19/universal-credit-the-history-of-an-it-project-failure/ https://www.harrowell.org.uk/blog/2017/03/19/universal-credi...
- baxtr 1y ago
  > The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance. What does that even mean? Are you trying to say that the problem isn’t that the AI models are bad — it’s that it’s hard to get people to use them naturally in their daily work?
  - Arnechos 1y ago
    For example where I work business users required model output to be 100% correct, which wasn't possible, so they decided to stick to old manual workflow.
    - sigwinch 1y ago
      That’s our definition of a process: when your objective is well-defined, a process is guaranteed to succeed. Not everything is a process. And sometimes people mistake what the desired success must be. For example, a piece of surgical equipment might not have features guaranteeing profitability.
- thatjoeoverthr 1y ago
  Honestly, just sad seeing AI posts on HN now.
  - ChrisMarshallNY 1y ago
    I’m not sure this is the case, here (although it’s always a possibility, sadly). It just looks like the highly-polished marketing copy I’ve read, all my career. It’s entirely possible that it was edited by AI (a task that I have found useful), but I think that it’s actually a fairly important (to the firm) paper, and was likely originally written by their staff (or a consultant), and carefully edited. I do feel as if it’s a promotional effort, but HN often features promotional material, if it is of interest to our community.
- woeirua 1y ago
  Lack of employee trust in these systems is caused by model (under)performance. There's a HUGE disconnect between the C-suite right now and the people on the ground using these models. Anyone who builds something with the models would tell you that they can't be trusted.
veegee 1y ago
[dead]
jakozaur 1y ago
I talked to some enterprises and saw similar patterns: 1. Agentic AI systems are hard to measure and evaluate methodologically. 2. Quote from Salesforce analyst day: "it's been so easy to build a killer demo, but why has it been so hard to get agents that actually deliver the goods.” 3. Unfortunately, small errors tend to compound over time, which means most systems need a human in the loop as of 2025. 4. A lot of enterprise buyers feel the huge potential (and FOMO), yet ROI is still unclear as of 2025. MIT report "State of AI in business 2025": Despite $30–40 billion in enterprise investment into GenAI, 95% of organizations are not seeing profit and loss impact.
- thatjoeoverthr 1y ago
  The Salesforce analyst quote is ChatGPT, though.
  - jakozaur 1y ago
    Source: https://seekingalpha.com/article/4830274-salesforce-inc-crm-analyst-investor-day-transcript https://seekingalpha.com/article/4830274-salesforce-inc-crm-...
    - thatjoeoverthr 1y ago
      [flagged]
donatj 1y ago
I've seen companies including my own pouring lots of money into AI. Outside of "replacing developers", I am genuinely curious what have people done that's actually useful? We've got a sort of "business intelligence" AI they poured a lot of time and money into, and I don't think anyone really uses it because it makes stuff up. I'm sure there are things. I just haven't seen them. I would love to hear concrete examples. The cynic in me says I wouldn't want something with the error aptitude and truth telling of a small child taking any sort of important action on my behalf.
- sigwinch 1y ago
  We had a lack of (digital images of) training cases in emergency surgery. You’d prefer to give ML experience with many rare cases, but must resort to style transfer. Humans can do this, but variously problematic and you’re taking them away from necessary work.
- Balgair 1y ago
  Two use cases with my corpo are: 1) Enhanced User guides/manuals. We sell some very complicated and expensive instruments. As such, making them work is quite hard. One of our biggest expenses is our engineers that go out, physically, to help customers. Company policy is that the first visit is always free. These customers can be very remote (Deep sea oil platforms, Australian outback, quite nice ski country ;) , etc). Often their issue is simple but they can also be very complex. We have phone trees, email, texts, iridium phones, etc. to talk customers through things to avoid these first visits and then hep them afterwards. So adding in AI chatbots is a natural way to help out. People don't feel quite the same 'shame' in asking really dumb questions to a chatbot that they do to a real person. So, to make these chatbots smarter, we use some of this AI mumbo-jumbo (RAG), to help them out. So far, it seems successful and the customer and engineers like the enhanced/AI manuals. 2) Making said manuals We support 35 languages and many regulatory environments. Our instruments are all compliant with whatever version of a government agency you've got (modulo a lot of time, money, ITAR regulations). As such, making all that paper (manuals, compliance docs, contracts, etc) takes a lot of time and effort and has to pass the legal tests too. So AI is really helpful with it. Most of the work for these large stacks of paper is essentially boilerplate, but all subtly different so that literal copy-pasting doesn't get you quite that far. AI systems have been able to, last I checked, get that team about 5x faster, as it cuts out ~85% of the process and drudgery. Since these documents get hauled into courts, they can't just be blindly AI made, and a human always has to go over everything with a sharp eye still, but AI helps out there a bit too. Last lunch I had with them, they were saying that they were actually working on their burn-down charts now and not just going from panic to panic. As in, they could actually do their jobs.
arisAlexis 1y ago
All of these are temporal. It's the only thing we can be sure of. Very limited value in these "lessons" 6 months ahead.
- layer8 1y ago
  Do you mean “temporary”?
reedf1 1y ago
What you actually need in most business cases is a 100% auditable, explainable and deterministic workflow. While AI is strictly deterministic - it is technically chaotic. Introducing this in large customer pipelines or in intensive data applications means that even if the AI only does something a bit off 99%, 99.9% or 99.99% you will see large spurious error rates in your workflow. Worst of all these will be difficult to explain - or maybe even purposely hidden, as I have seen some agents attempt to do.
- enraged_camel 1y ago
  This comment is a bit strange. >> While AI is strictly deterministic - it is technically chaotic AI is neither deterministic nor chaotic. It is nondeterministic because it works based on probability, which means that for open-ended contexts it can be unpredictable. But properly engineered agentic AI workflows can drastically reduce and even completely eliminate the unpredictability. Having proper guardrails such as well-defined prompts, validations and fallbacks in place can help ensure mistakes made by AIs don't result in errors in your system.
  - CjHuber 1y ago
    Are they? I mean I wouldn't say they are strictly deterministic, but with a temperature and topk of 0 and topp of 1 you can at least get them to be deterministic if I'm correct. In my experience if you need a higher temp than 0 in a prompt that is supposed to be within a pipeline, you need to optimize your prompt rather than introduce non determinism. Still of course that doesn't mean some inputs won't give unexpected outputs.
    - flufluflufluffy 1y ago
      In the hard, logically rigorous sense of the word, yes they are deterministic. Computers are deterministic machines. Everything that runs on a computer is deterministic. If that wasn’t the case, computers wouldn’t work. Of course I am considering the idealized version of a computer that is immune to environmental disturbances (a stray cosmic ray striking just the right spot and flipping a bit, somebody yanking out a RAM card, etc etc). LLMs are computation, they are very complex, but they are deterministic. If you run one on the same device, in the same state, with exactly the same input parameters multiple times, you will always get the same result. This is the case for every possible program. Most of the time, we don’t run them with exactly the same input parameters, or we run them on different devices, or some part of the state of the system has changed between runs, which could all potentially result in a different outcome (which, incidentally, is also the case for every possible program).
roxolotl 1y ago
> The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance. These, outside of employee resistance are technical problems. The insistence they aren’t seems to be the root of the misunderstanding around these tools. The reality is that “computers that speak English” are, at face value, incredibly impressive. But there’s nothing inherent to said systems that makes them easier to integrate with than computers which speak C. In fact I’d argue it’s harder because natural languages are significantly less precise. Communication and integration is incredibly challenging because you’re trying to transfer states between systems. When you let “the machine carry a larger share of the burden,” as Dijkstra described of the presumed benefit of natural language programming but actual downside[0], you’re also forfeiting a large amount of control. It is for the same reason that word problems are considered more challenging than equations in math class. With natural languages the states being communicated are much less clear than with formal languages and much of the burden assumed to be transferred to the machine is returned in the form of an increase in required specificity and preciseness of which formal languages already solve for. None of this is to say these tools aren’t useful nor that they cannot be deployed successfully. It is instead to say that the seduction of computers which speak English is more exactly that. These tools are incredibly easy to use to impress, and much more challenging to use to extract value. 0: https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667.html https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...
- IanCal 1y ago
  I think I disagree here. The integration parts aren’t natural language issues but connecting systems and how to put these things in your workflow. For example. I have a bunch of off the cuff questions and problems and tasks. I want to have these in one place and have that trigger a conversation with ChatGPT, which shows the results in the first place but can be continued easily. Before it was added the other week, I could track issues in linear and I could have codex write the code to solve them but only by manually moving text from one place to another, manually moving the tickets, checking on progress, click g buttons to open prs - all of that is integration hassle but none is about the model itself. I think now with GitHub copilot I could just assign a task.
jgoode19 1y ago
[dead]
sanskarix 1y ago
[dead]
- advikipedia 1y ago
  That's precisely what we found in our research as well! We outlined it in our observations too (excerpt below): The most successful deployment strategies we’ve seen started with: simple and specific use cases with clear value drivers, that were low risk yet medium impact; weren’t majorly disruptive to existing workflows; preferably automating a task that the human user dislikes (or was outsourced); the output of the workflow can be easily/quickly verified by the human for accuracy or suitability; and demonstrated clear ROI quickly Given the current levels of technological development, AI Agents work best when narrowly applied to very specific tasks and operating under a specific context. For instance, we’ve seen this in healthcare with revenue cycle management processes (claim and denial management) that health systems were already outsourcing to third-party providers. The land-and-expand strategy for AI agents is very different to traditional SaaS. Given enterprises are increasingly under pressure from the C-Suite to incorporate AI into their work, there are plenty of opportunities for startups to “land” but it’s much harder to “expand” – and not only that, it’s taking much longer to expand even when they want to expand, because it’s a use case by use case rollout. Much like the iconic Volkswagen ad, sometimes it’s better to “Think Small” and build trust first, rather than attempt too many use cases (and excessively complex use cases) right off the bat.
  - DrScientist 1y ago
    > Much like the iconic Volkswagen ad, sometimes it’s better to “Think Small” and build trust firsy Poor choice of example for building trust - Volkswagen: - lie big on emissions/fuel efficiency performance.
OBELISK_ASI 1y ago
[dead]
sanskarix 1y ago
[dead]
pmarreck 1y ago
"Working with AI is like being a mentor for a monkey pissing into its own mouth. Using agentic pipelines is doing the same, but now it's 5 monkeys pissing into each others' mouths in a Roman fountain kind of way." Except it's worse than that because we'll all end up having to do it anyway, because the overall velocity of emitted working code will be faster, and productivity > all.
chmod775 1y ago
It took me a while to realize that the cringe AI hype bro is just wearing a tie this time. Unsubstantiated fluff with anonymous sources that wants to disguise itself as legitimate research. Maybe at least these charts are based on real data - albeit self-reported by AI startups likely talking to their investors. Either way it's useless unless hopping on this train is a past time of yours or you make a living taking investors - to poor to fund an OpenAI, but just rich enough to fund someone eating OpenAI's scraps - for their money. It has been how many years of people trying to create businesses around chatgpt prompts? I think we need to bring bullying back. This is getting ridiculous.
throwaway_2494 1y ago
Prediction: Non-determinism will become acceptable in areas we used to expect accuracy. For example we will accept 'probabilistic bookkeeping' because it's cheaper than requiring ledgers to balance to the penny. But this leeway won't be equally applied. Powerful institutions like banks will use “probabilistic models” to decide they probably don’t owe you that refund, but if they decide you owe them money, they will still hold you to every cent. Nondeterminism for the powerful, determinism for everyone else. Yay!
- creaghpatr 1y ago
  Agree with your prediction in many cases but not bookkeeping. The whole point of bookkeeping is to balance the ledger to the penny so people would toss bookkeeping altogether before accepting a 'probabilistic' output. Agents will be used to accelerate the recon process though, or maybe they will become advanced enough to provide a (correct) deterministic result.
  - skeeter2020 1y ago
    So the worst outcome? We will still demand deterministic bookkeeping, but everyone will attempt to "optimize" using non-deterministic tools and assitance? Kind of feels like the US/Canada tax codes of today...
    - suncemoje 1y ago
      That’s where “human-in-the-loop” becomes a necessity, which _adds_ a step as opposed to removing one
- nicbou 1y ago
  A bunch of people are going to become "acceptable risk" and "cost of doing business". Companies will opt to get it 95% right for cheap over getting it 100% right, and for 95% of the population, it will be good enough.
- rolandog 1y ago
  Don't forget that this will likely be paired with rubber-stamping one-sided arguments maskerading as quality control processes where some maliciously-biased oversampling of (probably paid-for) good reviews takes place in order to consistently reach the conclusion that there's nothing wrong going on.
gizajob 1y ago
I know some people in business who won’t even delegate to very competent humans who have worked closely alongside them for years. And now we have to believe that letting AI agents autonomously roam without oversight is going to be acceptable to people like that.
datadrivenangel 1y ago
"AI agents can reason toward a goal, make dynamic decisions on the fly, and learn or improve over time" I've yet to see an 'agentic' setup that actually learns or improves over time. There are many techniques for this, but I don't see them used.
- htrp 1y ago
  > There are many techniques for this, but I don't see them used Why do you think that is?
deleted 1y ago
[deleted]

107 points by advikipedia 1y ago

We recently spoke with 30+ startup founders and 40+ enterprise practitioners who are building and deploying agentic AI systems across industries like financial services, healthcare, cybersecurity, and developer tooling. A few patterns emerged that might be relevant to anyone working on applied AI or automation: - The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance. - Incremental deployment beats ambition. Successful teams focus on narrow, verifiable use cases that deliver measurable ROI and build user trust before scaling autonomy. - Enterprise adoption is uneven. Many companies have “some agents” in production, but most use them with strong human oversight. The fully autonomous cases remain rare. - Pricing is unresolved. Hybrid models dominate; pure outcome-based pricing is uncommon due to attribution and monitoring challenges. Infrastructure is mostly homegrown. Over half of surveyed startups build their own agentic stacks, citing limited flexibility in existing frameworks. The article also includes detailed case studies, commentary on autonomy vs. accuracy trade-offs, and what’s next for ambient and proactive agents. If you’re building in this space, the full report is free here: https://mmc.vc/research/state-of-agentic-ai-founders-edition/ https://mmc.vc/research/state-of-agentic-ai-founders-edition... Would be interested to hear how others on HN are thinking about real-world deployment challenges — especially around trust, evaluation, and scaling agentic systems.

Etheryte 1y ago
Perhaps I simply don't understand what you mean, but it sounds like the first point could be rephrased in some way. To me, workflow integration and data privacy sound very much like technical blockers.
- barrenko 1y ago
  But if you define them as non-technical related blockers agents are just swell.
- refactor_master 1y ago
  Consider this simple example: Storing all your sensitive user data in one centralized location (e.g. a US server) would be great for any kind of analytics and modeling to tap into, and is technically very easy to do, but it also violates virtually every country's data privacy laws. So then you have to set up siloed servers around the world, deal with data governance, legal stuff, etc. Sure, it then becomes a technical challenge to work around those limits, but that may be cost/time prohibitive.
  - 1718627440 1y ago
    That sounds more like, that you can solve the problem, when it would have other requirements.
    - refactor_master 1y ago
      If you ask Silicon Valley, any organizational problem can be a technical problem if you try hard enough.
- IanCal 1y ago
  There are two sides to workflow integration. One is technical (it’s a hassle to connect things to a specific system because you’d need to deal with the api or there is no api) The other isn’t, because it’s figuring out how and where to use these new tools in an existing workflow. Maybe you could design something from scratch but you have lots of business processes right now, how do you smoothly modify that? Where does it make sense? Frankly understanding what the systems can and can’t do takes at least some time even if only because the field is moving so fast (I worked with a small local firm who I was able to help by showing them the dramatic improvements in transcription quality vs cost recently - people here are more used to whisper and the like but it’s not as common knowledge how and where you can use these things).
- advikipedia 1y ago
  More than the "actual" problem, the "perception" of the problem is worse. Workflow integration is more to do with users having to rethink their workflows, their roles, and how they work with AI. As for data privacy concerns, even where startups have taken measures to overcome the problems, very often enterprises still remain concerned (making this more of a perception problem than an actual problem). That's why I focused on the non-technical aspect of it!
  - DrScientist 1y ago
    When I see vendors complain about workflow and integration issues, it's because the vendors software is written around an expectation of a certain workflow and integration points and they find out in reality every customer does it slightly differently. Some key challenges around workflow are that while the fundamental white-board task flow is the same, different companies may distribute those tasks between people and over time in different ways. Workflow is about flowing the task and associated information between people - not just doing the tasks. Same goes for integration - the timing of when certain necessary information might be available again not uniform and timing concerns are often missed on the high level whiteboard. Here's a classic example of ignoring timing issues. https://www.harrowell.org.uk/blog/2017/03/19/universal-credit-the-history-of-an-it-project-failure/ https://www.harrowell.org.uk/blog/2017/03/19/universal-credi...
baxtr 1y ago
> The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance. What does that even mean? Are you trying to say that the problem isn’t that the AI models are bad — it’s that it’s hard to get people to use them naturally in their daily work?
- Arnechos 1y ago
  For example where I work business users required model output to be 100% correct, which wasn't possible, so they decided to stick to old manual workflow.
  - sigwinch 1y ago
    That’s our definition of a process: when your objective is well-defined, a process is guaranteed to succeed. Not everything is a process. And sometimes people mistake what the desired success must be. For example, a piece of surgical equipment might not have features guaranteeing profitability.
thatjoeoverthr 1y ago
Honestly, just sad seeing AI posts on HN now.
- ChrisMarshallNY 1y ago
  I’m not sure this is the case, here (although it’s always a possibility, sadly). It just looks like the highly-polished marketing copy I’ve read, all my career. It’s entirely possible that it was edited by AI (a task that I have found useful), but I think that it’s actually a fairly important (to the firm) paper, and was likely originally written by their staff (or a consultant), and carefully edited. I do feel as if it’s a promotional effort, but HN often features promotional material, if it is of interest to our community.
woeirua 1y ago
Lack of employee trust in these systems is caused by model (under)performance. There's a HUGE disconnect between the C-suite right now and the people on the ground using these models. Anyone who builds something with the models would tell you that they can't be trusted.

veegee 1y ago

[dead]

jakozaur 1y ago

I talked to some enterprises and saw similar patterns: 1. Agentic AI systems are hard to measure and evaluate methodologically. 2. Quote from Salesforce analyst day: "it's been so easy to build a killer demo, but why has it been so hard to get agents that actually deliver the goods.” 3. Unfortunately, small errors tend to compound over time, which means most systems need a human in the loop as of 2025. 4. A lot of enterprise buyers feel the huge potential (and FOMO), yet ROI is still unclear as of 2025. MIT report "State of AI in business 2025": Despite $30–40 billion in enterprise investment into GenAI, 95% of organizations are not seeing profit and loss impact.

thatjoeoverthr 1y ago
The Salesforce analyst quote is ChatGPT, though.
- jakozaur 1y ago
  Source: https://seekingalpha.com/article/4830274-salesforce-inc-crm-analyst-investor-day-transcript https://seekingalpha.com/article/4830274-salesforce-inc-crm-...
  - thatjoeoverthr 1y ago
    [flagged]

donatj 1y ago

I've seen companies including my own pouring lots of money into AI. Outside of "replacing developers", I am genuinely curious what have people done that's actually useful? We've got a sort of "business intelligence" AI they poured a lot of time and money into, and I don't think anyone really uses it because it makes stuff up. I'm sure there are things. I just haven't seen them. I would love to hear concrete examples. The cynic in me says I wouldn't want something with the error aptitude and truth telling of a small child taking any sort of important action on my behalf.

sigwinch 1y ago
We had a lack of (digital images of) training cases in emergency surgery. You’d prefer to give ML experience with many rare cases, but must resort to style transfer. Humans can do this, but variously problematic and you’re taking them away from necessary work.
Balgair 1y ago
Two use cases with my corpo are: 1) Enhanced User guides/manuals. We sell some very complicated and expensive instruments. As such, making them work is quite hard. One of our biggest expenses is our engineers that go out, physically, to help customers. Company policy is that the first visit is always free. These customers can be very remote (Deep sea oil platforms, Australian outback, quite nice ski country ;) , etc). Often their issue is simple but they can also be very complex. We have phone trees, email, texts, iridium phones, etc. to talk customers through things to avoid these first visits and then hep them afterwards. So adding in AI chatbots is a natural way to help out. People don't feel quite the same 'shame' in asking really dumb questions to a chatbot that they do to a real person. So, to make these chatbots smarter, we use some of this AI mumbo-jumbo (RAG), to help them out. So far, it seems successful and the customer and engineers like the enhanced/AI manuals. 2) Making said manuals We support 35 languages and many regulatory environments. Our instruments are all compliant with whatever version of a government agency you've got (modulo a lot of time, money, ITAR regulations). As such, making all that paper (manuals, compliance docs, contracts, etc) takes a lot of time and effort and has to pass the legal tests too. So AI is really helpful with it. Most of the work for these large stacks of paper is essentially boilerplate, but all subtly different so that literal copy-pasting doesn't get you quite that far. AI systems have been able to, last I checked, get that team about 5x faster, as it cuts out ~85% of the process and drudgery. Since these documents get hauled into courts, they can't just be blindly AI made, and a human always has to go over everything with a sharp eye still, but AI helps out there a bit too. Last lunch I had with them, they were saying that they were actually working on their burn-down charts now and not just going from panic to panic. As in, they could actually do their jobs.

arisAlexis 1y ago

All of these are temporal. It's the only thing we can be sure of. Very limited value in these "lessons" 6 months ahead.

layer8 1y ago
Do you mean “temporary”?

reedf1 1y ago

What you actually need in most business cases is a 100% auditable, explainable and deterministic workflow. While AI is strictly deterministic - it is technically chaotic. Introducing this in large customer pipelines or in intensive data applications means that even if the AI only does something a bit off 99%, 99.9% or 99.99% you will see large spurious error rates in your workflow. Worst of all these will be difficult to explain - or maybe even purposely hidden, as I have seen some agents attempt to do.

enraged_camel 1y ago
This comment is a bit strange. >> While AI is strictly deterministic - it is technically chaotic AI is neither deterministic nor chaotic. It is nondeterministic because it works based on probability, which means that for open-ended contexts it can be unpredictable. But properly engineered agentic AI workflows can drastically reduce and even completely eliminate the unpredictability. Having proper guardrails such as well-defined prompts, validations and fallbacks in place can help ensure mistakes made by AIs don't result in errors in your system.
- CjHuber 1y ago
  Are they? I mean I wouldn't say they are strictly deterministic, but with a temperature and topk of 0 and topp of 1 you can at least get them to be deterministic if I'm correct. In my experience if you need a higher temp than 0 in a prompt that is supposed to be within a pipeline, you need to optimize your prompt rather than introduce non determinism. Still of course that doesn't mean some inputs won't give unexpected outputs.
  - flufluflufluffy 1y ago
    In the hard, logically rigorous sense of the word, yes they are deterministic. Computers are deterministic machines. Everything that runs on a computer is deterministic. If that wasn’t the case, computers wouldn’t work. Of course I am considering the idealized version of a computer that is immune to environmental disturbances (a stray cosmic ray striking just the right spot and flipping a bit, somebody yanking out a RAM card, etc etc). LLMs are computation, they are very complex, but they are deterministic. If you run one on the same device, in the same state, with exactly the same input parameters multiple times, you will always get the same result. This is the case for every possible program. Most of the time, we don’t run them with exactly the same input parameters, or we run them on different devices, or some part of the state of the system has changed between runs, which could all potentially result in a different outcome (which, incidentally, is also the case for every possible program).

roxolotl 1y ago

> The main blockers aren’t technical. Most founders pointed to workflow integration, employee trust, and data privacy as the toughest challenges — not model performance. These, outside of employee resistance are technical problems. The insistence they aren’t seems to be the root of the misunderstanding around these tools. The reality is that “computers that speak English” are, at face value, incredibly impressive. But there’s nothing inherent to said systems that makes them easier to integrate with than computers which speak C. In fact I’d argue it’s harder because natural languages are significantly less precise. Communication and integration is incredibly challenging because you’re trying to transfer states between systems. When you let “the machine carry a larger share of the burden,” as Dijkstra described of the presumed benefit of natural language programming but actual downside[0], you’re also forfeiting a large amount of control. It is for the same reason that word problems are considered more challenging than equations in math class. With natural languages the states being communicated are much less clear than with formal languages and much of the burden assumed to be transferred to the machine is returned in the form of an increase in required specificity and preciseness of which formal languages already solve for. None of this is to say these tools aren’t useful nor that they cannot be deployed successfully. It is instead to say that the seduction of computers which speak English is more exactly that. These tools are incredibly easy to use to impress, and much more challenging to use to extract value. 0: https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667.html https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

IanCal 1y ago
I think I disagree here. The integration parts aren’t natural language issues but connecting systems and how to put these things in your workflow. For example. I have a bunch of off the cuff questions and problems and tasks. I want to have these in one place and have that trigger a conversation with ChatGPT, which shows the results in the first place but can be continued easily. Before it was added the other week, I could track issues in linear and I could have codex write the code to solve them but only by manually moving text from one place to another, manually moving the tickets, checking on progress, click g buttons to open prs - all of that is integration hassle but none is about the model itself. I think now with GitHub copilot I could just assign a task.

jgoode19 1y ago

[dead]

sanskarix 1y ago

[dead]

advikipedia 1y ago
That's precisely what we found in our research as well! We outlined it in our observations too (excerpt below): The most successful deployment strategies we’ve seen started with: simple and specific use cases with clear value drivers, that were low risk yet medium impact; weren’t majorly disruptive to existing workflows; preferably automating a task that the human user dislikes (or was outsourced); the output of the workflow can be easily/quickly verified by the human for accuracy or suitability; and demonstrated clear ROI quickly Given the current levels of technological development, AI Agents work best when narrowly applied to very specific tasks and operating under a specific context. For instance, we’ve seen this in healthcare with revenue cycle management processes (claim and denial management) that health systems were already outsourcing to third-party providers. The land-and-expand strategy for AI agents is very different to traditional SaaS. Given enterprises are increasingly under pressure from the C-Suite to incorporate AI into their work, there are plenty of opportunities for startups to “land” but it’s much harder to “expand” – and not only that, it’s taking much longer to expand even when they want to expand, because it’s a use case by use case rollout. Much like the iconic Volkswagen ad, sometimes it’s better to “Think Small” and build trust first, rather than attempt too many use cases (and excessively complex use cases) right off the bat.
- DrScientist 1y ago
  > Much like the iconic Volkswagen ad, sometimes it’s better to “Think Small” and build trust firsy Poor choice of example for building trust - Volkswagen: - lie big on emissions/fuel efficiency performance.

OBELISK_ASI 1y ago

[dead]

sanskarix 1y ago

[dead]

pmarreck 1y ago

"Working with AI is like being a mentor for a monkey pissing into its own mouth. Using agentic pipelines is doing the same, but now it's 5 monkeys pissing into each others' mouths in a Roman fountain kind of way." Except it's worse than that because we'll all end up having to do it anyway, because the overall velocity of emitted working code will be faster, and productivity > all.

chmod775 1y ago

It took me a while to realize that the cringe AI hype bro is just wearing a tie this time. Unsubstantiated fluff with anonymous sources that wants to disguise itself as legitimate research. Maybe at least these charts are based on real data - albeit self-reported by AI startups likely talking to their investors. Either way it's useless unless hopping on this train is a past time of yours or you make a living taking investors - to poor to fund an OpenAI, but just rich enough to fund someone eating OpenAI's scraps - for their money. It has been how many years of people trying to create businesses around chatgpt prompts? I think we need to bring bullying back. This is getting ridiculous.

throwaway_2494 1y ago

Prediction: Non-determinism will become acceptable in areas we used to expect accuracy. For example we will accept 'probabilistic bookkeeping' because it's cheaper than requiring ledgers to balance to the penny. But this leeway won't be equally applied. Powerful institutions like banks will use “probabilistic models” to decide they probably don’t owe you that refund, but if they decide you owe them money, they will still hold you to every cent. Nondeterminism for the powerful, determinism for everyone else. Yay!

creaghpatr 1y ago
Agree with your prediction in many cases but not bookkeeping. The whole point of bookkeeping is to balance the ledger to the penny so people would toss bookkeeping altogether before accepting a 'probabilistic' output. Agents will be used to accelerate the recon process though, or maybe they will become advanced enough to provide a (correct) deterministic result.
- skeeter2020 1y ago
  So the worst outcome? We will still demand deterministic bookkeeping, but everyone will attempt to "optimize" using non-deterministic tools and assitance? Kind of feels like the US/Canada tax codes of today...
  - suncemoje 1y ago
    That’s where “human-in-the-loop” becomes a necessity, which _adds_ a step as opposed to removing one
nicbou 1y ago
A bunch of people are going to become "acceptable risk" and "cost of doing business". Companies will opt to get it 95% right for cheap over getting it 100% right, and for 95% of the population, it will be good enough.
rolandog 1y ago
Don't forget that this will likely be paired with rubber-stamping one-sided arguments maskerading as quality control processes where some maliciously-biased oversampling of (probably paid-for) good reviews takes place in order to consistently reach the conclusion that there's nothing wrong going on.

gizajob 1y ago

I know some people in business who won’t even delegate to very competent humans who have worked closely alongside them for years. And now we have to believe that letting AI agents autonomously roam without oversight is going to be acceptable to people like that.

datadrivenangel 1y ago

"AI agents can reason toward a goal, make dynamic decisions on the fly, and learn or improve over time" I've yet to see an 'agentic' setup that actually learns or improves over time. There are many techniques for this, but I don't see them used.

htrp 1y ago
> There are many techniques for this, but I don't see them used Why do you think that is?

deleted 1y ago

[deleted]