4 ms·

I think a very direct answer to this is pointing to the hot water Cursor found itself in after an AI customer support agent made stuff up. - Do you want an LLM

I think a very direct answer to this is pointing to the hot water Cursor found itself in after an AI customer support agent made stuff up. - Do you want an LLM salesbot to close a deal your company isn't actually able to fulfill? - Do you think your company will use AI more intelligently and reliably than the people who made a popular LLM coding system?

ghaff 1y ago
A human sales rep would never confuse selling with installing :-)
- heelix 1y ago
  What is the difference between software and car sales? The dealership rep knows when they are lying.
  - ghaff 1y ago
    Im not saying that good sales reps actively lie but, with software especially, there are often features in the pipeline or that only somewhat work to a degree the sales rep may not even be aware of.
soulofmischief 1y ago
I want my AI support rep to have access to data and documents that it can vector/text search and forward to the user. I want my AI support rep to create tasks to engage my team, with all the relevant data linked to it. It should be able to automatically schedule things and ask a human for confirmation. It should be able to elevate communication to a human in the loop, using whatever mediums of communication makes sense given staff availability and workload. At no point is it allowed to answer any questions unless the answers are constrained and probably directly quoted from a cited and linked section in our documentation. In general, it should never confirm or deny things, it should never try to close things or acknowledge the content of any of the user's communication other than to call tools and surface public information which might be relevant to their request. Most of this is a software architecture problem. The LLM is just there to provide an intuitive and extremely powerful natural language interface for search and tool calling. A little bit of glue between different systems, both internal and external.
- semi-extrinsic 1y ago
  > In general, it should never confirm or deny things, it should never try to close things or acknowledge the content of any of the user's communication other than to call tools and surface public information which might be relevant to their request. If you were an end user of such a system, would you be happy?
  - soulofmischief 1y ago
    As long as it addressed my needs by either pointing me to the correct documentation, or elevating me to a human, then yes, of course I'd be happy with that. It's an incremental stop along the way to truly reliable agentic systems which we can trust with important things.
bastardoperator 1y ago
If the CEO already thinks AI can do everything, I think their answer to that is yes and yes.
- AIPedant 1y ago
  You would think that Cursor's leadership would be aware of other cases where LLM customer support went awry - e.g. that Canadian airline whose chatbot promised a bereavement discount, ending with a judge ordering them to honor the chatbot's BS. I suspect Cursor told themselves that they are super-smart AI experts who would never make an amateur mistake like the airline, they will use prompt engineering + RAG. With this, it will be unpossible that the LLM could make a mistake.

ghaff 1y ago

A human sales rep would never confuse selling with installing :-)

heelix 1y ago
What is the difference between software and car sales? The dealership rep knows when they are lying.
- ghaff 1y ago
  Im not saying that good sales reps actively lie but, with software especially, there are often features in the pipeline or that only somewhat work to a degree the sales rep may not even be aware of.

soulofmischief 1y ago

I want my AI support rep to have access to data and documents that it can vector/text search and forward to the user. I want my AI support rep to create tasks to engage my team, with all the relevant data linked to it. It should be able to automatically schedule things and ask a human for confirmation. It should be able to elevate communication to a human in the loop, using whatever mediums of communication makes sense given staff availability and workload. At no point is it allowed to answer any questions unless the answers are constrained and probably directly quoted from a cited and linked section in our documentation. In general, it should never confirm or deny things, it should never try to close things or acknowledge the content of any of the user's communication other than to call tools and surface public information which might be relevant to their request. Most of this is a software architecture problem. The LLM is just there to provide an intuitive and extremely powerful natural language interface for search and tool calling. A little bit of glue between different systems, both internal and external.

semi-extrinsic 1y ago
> In general, it should never confirm or deny things, it should never try to close things or acknowledge the content of any of the user's communication other than to call tools and surface public information which might be relevant to their request. If you were an end user of such a system, would you be happy?
- soulofmischief 1y ago
  As long as it addressed my needs by either pointing me to the correct documentation, or elevating me to a human, then yes, of course I'd be happy with that. It's an incremental stop along the way to truly reliable agentic systems which we can trust with important things.

bastardoperator 1y ago

If the CEO already thinks AI can do everything, I think their answer to that is yes and yes.

AIPedant 1y ago
You would think that Cursor's leadership would be aware of other cases where LLM customer support went awry - e.g. that Canadian airline whose chatbot promised a bereavement discount, ending with a judge ordering them to honor the chatbot's BS. I suspect Cursor told themselves that they are super-smart AI experts who would never make an amateur mistake like the airline, they will use prompt engineering + RAG. With this, it will be unpossible that the LLM could make a mistake.