4 ms·

What is a business problem you solved with LLMs that you couldn't solve as efficiently without them?

by cloudking 3y ago

thomasahle 3y ago
Customer support.
- endisneigh 3y ago
  do you have an example? from my experience most customers support related issues are self inflicted (e.g. not allowing the customer to self service all tasks and instead gatekeeping it to some support rep) I do not see how a LLM will help in a way that was not achievable before.
  - thomasahle 3y ago
    Previous chatbots were too dumb to handle said tasks. LLMs may be able to understand the internal APIs and perform the task, without the company having to design a UI for self service.
- deleted 3y ago
  [deleted]
- hammyhavoc 3y ago
  This depends on so many variables, including complexity of the average problem, the obscurity of the average problem, having documentation or a knowledgebase with a known solution. ChatGPT hallucinates so many things that don't exist, e.g., buttons or menu options in programs that don't exist, not considering file compatibility issues etc. I could go on for hours. If the average problem is "I got locked out of my account" or basic and common stuff like this that just warrants giving someone a link or telling them to reboot their router, sure, maybe it'll be better than dealing with a human being in x out of y occurrences, hallucination not withstanding. If it's something more complex like needing an NGINX configuration when a company has only ever considered Apache htaccess in the past, the customer is probably seven kinds of fucked. I wasted days trying to get something other than nonsense out of ChatGPT for an NGINX config, even going so far as to feed it documentation and the lines it would need for implementing URL rewrites. It kept hallucinating things that didn't exist in the documentation, and was a complete waste of time. Even after correcting it umpteen times, it still gave the same response. There's no reasoning applied. Is there potential? Sure. But it isn't replacing human beings in a lot of cases. And for even more cases, just using a search function on a knowledgebase or search engine will yield more accurate information than trusting it isn't hallucinating. I have wasted far more time than I've saved due to hallucinations. If all customer support does is search the problem on a knowledgebase, sure, it makes sense, because the support agent is already just doing what the LLM does if they can't apply logic and reasoning to the query. But why not just access the knowledgebase directly and not risk hallucinations?
  - thomasahle 3y ago
    Sure, it's not as good as a human rep, but it's still better than the dumb chatbots we are forced to interact with now. May be better than searching a poorly updated knowledgebase too.
    - hammyhavoc 3y ago
      It might be better than most reps considering not infrequently, the big corps hire the cheapest labour and quality of English is a major problem in my experiences. Especially if all the rep does is query something in a knowledgebase. I disagree very strongly over the knowledgebase being inferior to the LLM data set. The data sets of LLMs are not stored in a human-readable format. The data sets of LLMs are likely a worse means of storing data because the outputs are prone to hallucination, and if an output can only be rendered via an LLM itself, well, therein lies your problem. You can't trust that it's right and not hallucinating if you can't read the data it is using.
tester457 3y ago
Inefficient use of time is a business problem. LLMs save work hours we would usually spend on mundane tasks. Regular office workers can make automations, devs can enhance documentation with LLMs, LLM powered search and spreadsheets, these all save time. Businesses that don't take advantage of this will be left in the dust. In a few years not using it will be like not allowing your employees to use Google, because there is fake news on the internet. The time efficiency gained will surpass the hallucinations, just like how people for the most part have learned to use the internet for their benefit despite all of the garbage on the net.
- cloudking 3y ago
  Totally agree that they save time, I think that may be the primary problem LLMs solve.
alach11 3y ago
I work in O&G so I can give a few domain-specific examples I've worked on just in the last two weeks: - Smart pre-job safety app that looks across past observations and incidents to make job-specific recommendations about safety risks - Automatically summarize detailed hourly rig reports into daily summary reports. This is an ongoing savings of $1.2 million per year in labor - Review rig reports for evidence of formation gas for a study on casing integrity. This was a one-off savings of 400 engineer hours - Perform clustering and topic modeling of employee HR development goals and schedule training classes based on the areas of greatest demand - Custom chatbot to answer questions about how to use SAP. We have ~50 training PPTs that nobody reads. The chatbot answers questions using the documents What's amazing is there's all this low-hanging fruit to be captured and spinning up these solutions takes mere hours.

What is a business problem you solved with LLMs that you couldn't solve as efficiently without them?

thomasahle 3y ago

Customer support.

endisneigh 3y ago
do you have an example? from my experience most customers support related issues are self inflicted (e.g. not allowing the customer to self service all tasks and instead gatekeeping it to some support rep) I do not see how a LLM will help in a way that was not achievable before.
- thomasahle 3y ago
  Previous chatbots were too dumb to handle said tasks. LLMs may be able to understand the internal APIs and perform the task, without the company having to design a UI for self service.
deleted 3y ago
[deleted]
hammyhavoc 3y ago
This depends on so many variables, including complexity of the average problem, the obscurity of the average problem, having documentation or a knowledgebase with a known solution. ChatGPT hallucinates so many things that don't exist, e.g., buttons or menu options in programs that don't exist, not considering file compatibility issues etc. I could go on for hours. If the average problem is "I got locked out of my account" or basic and common stuff like this that just warrants giving someone a link or telling them to reboot their router, sure, maybe it'll be better than dealing with a human being in x out of y occurrences, hallucination not withstanding. If it's something more complex like needing an NGINX configuration when a company has only ever considered Apache htaccess in the past, the customer is probably seven kinds of fucked. I wasted days trying to get something other than nonsense out of ChatGPT for an NGINX config, even going so far as to feed it documentation and the lines it would need for implementing URL rewrites. It kept hallucinating things that didn't exist in the documentation, and was a complete waste of time. Even after correcting it umpteen times, it still gave the same response. There's no reasoning applied. Is there potential? Sure. But it isn't replacing human beings in a lot of cases. And for even more cases, just using a search function on a knowledgebase or search engine will yield more accurate information than trusting it isn't hallucinating. I have wasted far more time than I've saved due to hallucinations. If all customer support does is search the problem on a knowledgebase, sure, it makes sense, because the support agent is already just doing what the LLM does if they can't apply logic and reasoning to the query. But why not just access the knowledgebase directly and not risk hallucinations?
- thomasahle 3y ago
  Sure, it's not as good as a human rep, but it's still better than the dumb chatbots we are forced to interact with now. May be better than searching a poorly updated knowledgebase too.
  - hammyhavoc 3y ago
    It might be better than most reps considering not infrequently, the big corps hire the cheapest labour and quality of English is a major problem in my experiences. Especially if all the rep does is query something in a knowledgebase. I disagree very strongly over the knowledgebase being inferior to the LLM data set. The data sets of LLMs are not stored in a human-readable format. The data sets of LLMs are likely a worse means of storing data because the outputs are prone to hallucination, and if an output can only be rendered via an LLM itself, well, therein lies your problem. You can't trust that it's right and not hallucinating if you can't read the data it is using.

tester457 3y ago

Inefficient use of time is a business problem. LLMs save work hours we would usually spend on mundane tasks. Regular office workers can make automations, devs can enhance documentation with LLMs, LLM powered search and spreadsheets, these all save time. Businesses that don't take advantage of this will be left in the dust. In a few years not using it will be like not allowing your employees to use Google, because there is fake news on the internet. The time efficiency gained will surpass the hallucinations, just like how people for the most part have learned to use the internet for their benefit despite all of the garbage on the net.

cloudking 3y ago
Totally agree that they save time, I think that may be the primary problem LLMs solve.

alach11 3y ago

I work in O&G so I can give a few domain-specific examples I've worked on just in the last two weeks: - Smart pre-job safety app that looks across past observations and incidents to make job-specific recommendations about safety risks - Automatically summarize detailed hourly rig reports into daily summary reports. This is an ongoing savings of $1.2 million per year in labor - Review rig reports for evidence of formation gas for a study on casing integrity. This was a one-off savings of 400 engineer hours - Perform clustering and topic modeling of employee HR development goals and schedule training classes based on the areas of greatest demand - Custom chatbot to answer questions about how to use SAP. We have ~50 training PPTs that nobody reads. The chatbot answers questions using the documents What's amazing is there's all this low-hanging fruit to be captured and spinning up these solutions takes mere hours.