6 ms·

> I'd love to know what are some of the hidden challenges to making a useful product with agents? One thing that is still confusing to me, is that we've been b

> I'd love to know what are some of the hidden challenges to making a useful product with agents? One thing that is still confusing to me, is that we've been building products with machine learning pretty heavily for a decade now and somehow abandoned all that we have learned about the process now that we're building "AI". The biggest thing any ML practitioner realizes when they step out of a research setting is that for most tasks accuracy has to be very high for it be productizable. You can do handwritten digit recognition with 90% accuracy? Sounds pretty good, but if you need to turn that into recognizing a 12 digit account number you now have a 70% chance of getting at least one digit incorrect. This means a product worthy digit classifier needs to be much higher accuracy. Go look at some of the LLM benchmarks out there, even in these happy cases it's rare to see any LLM getting above 90%. Then consider you want to chain these calls together to create proper agent based workflows. Even with 90% accuracy in each task, chain 3 of these together and you're down to 0.9 x 0.9 x 0.9 = 0.73, 73% accuracy. This is by far this biggest obstacle towards seeing more useful products built with agents. There are cases where lower accuracy results are acceptable, but most people don't even consider this before embarking on their journey to build an AI product/agent.

chenxi9649 3y ago
Yea that's a good point. Now I am curious, what are some tasks that can accept a model that is at 80% as good as a human, but is 100x cheaper?(or, 100x faster?)
- Aerbil313 3y ago
  A ton of tasks. Call centers to start with (they already do[1]), with human fallback. 1: In my country, after ChatGPT launched last year, when you call customer support you are now prompted to “just say in a few words” what you want instead of going through tap-this-number menus (they exist as a fallback) and I believe the backend is an LLM. The user flow and voice recordings are still programmatically determined though, but I can easily see one streamlined model calling APIs and whatnot, handling it all.
  - salad-tycoon 3y ago
    I speak English clearly. These things always tell me to repeat what I said. Never once has it ever worked for me. I want to throw my phone at the wall. Also I think this has been around for longer than chatgpt. It is often accompanied by a fake keyboard clicking noise.
    - sofixa 3y ago
      > I speak English clearly. These things always tell me to repeat what I said. Never once has it ever worked for me. I want to throw my phone at the wall Now imagine how well it works for people with non-"native" accents (even for native, I'd guess that a good Scouse/Glaswegian/Kiwi accent might confuse the hell of those systems as well). It's a disaster and I hate those.
      - int_19h 3y ago
        There's a general problem across the tech industry with replacing existing simple and reliable (in a sense of conveying the user's intention) interfaces like physical buttons with stuff that is supposedly "more natural" like speech recognition or swipe gestures that in practice has a much higher error rate. See also: replacing physical buttons with convoluted swipe gestures on mobile devices in the never-ending quest to make screen as large as possible. When was this ever a user ask? I feel sometimes like the present UX design is one large LLM-like hallucination.
  - chenxi9649 3y ago
    Yea that's pretty cool too, I heard some restaurants are also doing a 100% voice LLM to take orders. Transcription, specifically Whisper, is one of those ML models where the accuracy is basically on-par with humans. So, I really expect a lot more to come out of real time voice/LLM integrations.(the ChatGPT voice thing is a good glimpse, but very janky)
    - allanwind 3y ago
      "A firm providing AI drive-thru tech to fast food chains actually relies on human workers to take orders 70% of the time": https://www.businessinsider.com/ai-drive-thru-tech-relies-on-workers-70-of-time-2023-12?op=1 https://www.businessinsider.com/ai-drive-thru-tech-relies-on...
- geoduck14 3y ago
  Scan a menu, look for the different entrees, identify the most probable ingredients, determine health content. Then: allow people to search for food based allergies, food aversions, calories. Generate pictures of what the food might look like, display the pics next to the food to make it more likely a user will buy that food.
  - charleslmunger 3y ago
    Until one of your customers' children eats a peanut that the AI didn't infer would be an ingredient, and dies. Generating fake pictures also seems like it would be more ordinary false advertising.
    - red-iron-pine 3y ago
      also requires the resto or manufacturer to list all ingredients, and most already list potential allergens. but yeah liability would scare me, esp. because without it you can put the liability squarely on the restaurant or in some cases the person ordering / asking / buying.
- tetha 3y ago
  Similar to the sibling comment, helpdesk ticket routing. The volume of helpdesk tickets large enterprises deal with is very easily and vastly underestimated. If you can even route 30% away from the central triage with 90+% accuracy and drop everything else back to the central triage... you suddenly safe 2 FTEs in that spot in some places. And increase customer satisfaction for most of those tickets because they get resolved faster. Or, as much as people hate it, chatbots as a customer front. Yes, everyone here as an expert in a lot of tech has had terrible experiences with chatbots. Please mark your hate with the word "Lemon" in the comments. But decently implemented chatbots with a few systems behind them can resolve staggering amounts of simple problems from non-techies without human interaction from the company deploying them. It remains important to eventually escalate to humans - including the history from all of these interactions to avoid frustrations, sure. Or, ticket/request preprocessing. Remember how spelling that 10 digit account number to a call center agent hard of hearing sucks? Those 4 retries because of you not using a better way to communicate that number also costs the company. Now, you can push a few of these retries into an AI system. If you mail them, an AI system can try to extract information like account numbers, intent, start of the problem, problem descriptions and such into dedicated fields to make the support agents faster. Companies are certainly overdoing it at the moment, I'm not denying that. But a lot of the support/helpdesk pre-screening can be automated with current AI/ML capabilities very decently. Especially if you learn to recognize and navigate it.
  - Karrot_Kream 3y ago
    The last company I worked at eventually became a Big Tech. In the beginning though, we used to ask all engineers to pair with customer service folks to deal with ticket triage. When we got a bit bigger, we used to have rotations where eng would pair with customer service folks. Being on the other side of that was very eye-opening for all eng. Many used to come in with the same bias that you see on this site, that how dare you be routed to some automated service and how inhumane the service is. On the other side you see competent CS agents absolutely swamped with low-level questions that were often literally answered in docs and FAQ pages. I think getting transformer-based triage models correct can unlock tons of value.
  - Sabinus 3y ago
- deleted 3y ago
  [deleted]
- vintermann 3y ago
  Well, an old one is OCR, especially handwritten OCR. I'm doing genealogy. There is SO MUCH old handwritten material that is never transcribed, and which requires special expertise to read (old and exotic handwriting styles) and interpret (place names, writing conventions, abbreviations). It doesn't have to be perfect. It's not as if the actual data in there is perfect. It just has to be in a form where I can search it, ideally with named entities mapped. Quality - like deciphering the writing on scrolls buried in volcanic ash in Herculaneum - gets all the attention. But what I really want is quantity - I want to be able to search through those 5000 pages of 200 year old mildly damaged cadastral records in dense handwriting. I want to relieve the army of kind retirees who currently transcribe these sorts of documents one by one based on their own needs.
  - jazzyjackson 3y ago
    bruh what are the retirees supposed to do once you've automated their hobby
Qwero 3y ago
I use already a few ai tools even without perfect accuracy. And a LLM who only needs to call to a few API calls isn't hard. Very little need perfect accuracy and for that we still have classical software.
- chenxi9649 3y ago
  curious to know, which tools do you use and how do you use em?
  - Qwero 3y ago
    I use copilot for coding. CharGPT for writing emails to bigger audiences, Gramma correction. Image generator for fun. LLM for Feature Extraction from random text like a website or PDF (llama).
- skywhopper 3y ago
  You use them successfully because your human mind can filter out the junk. It would only take one inaccurate API call that charges your credit card $10k or sells your car for 10 cents to cause a lot of damage to your life.
  - qeternity 3y ago
    Which is why even with classical software, most of us don't have APIs where that's all it takes.
    - ltengelis 3y ago
      Frankly I think you've said it all here - a properly designed API + a well designed LLM interface on top of that enables non-technical people to do things they otherwise couldn't.
mFixman 3y ago
> You can do handwritten digit recognition with 90% accuracy? Sounds pretty good, but if you need to turn that into recognizing a 12 digit account number you now have a 70% chance of getting at least one digit incorrect. You are assuming that the probability of failure is independent, which couldn't be further from the truth. If a digit recogniser can recognise one of your "hard" handwritten digits, such as a 4 or a 9, it will likely be able to recognise all of them. The same happens with AI agents. They are not good at some tasks, but really really food at others.
- jhbadger 3y ago
  And the US Post Office and other postal services have been using this tech to sort letters for several decades now (although postal codes with both letters and numbers like Canada's are harder). It was viewed as the "killer app" for ML in the 1990s.
  - PheonixPharts 3y ago
    This thread is an object lesson in the point I'm making: people have forgotten everything we've learned about making ML based products. Parent comment doesn't understand the concept of expectation, and this comment is apparently unfamiliar with the fact that SotA for digit recognition [0] has been much higher than 90% even in the 90s. 90% accuracy for digit recognition is what you get if you use logistic regression as your model. My point was that numbers that look good in terms of research often aren't close to go enough for the real world. It's not that 90% works for zip codes, it's that in the 90s accuracy was closer to 99%. You have validated my point rather than rejected it. 0. https://en.wikipedia.org/wiki/MNIST_database https://en.wikipedia.org/wiki/MNIST_database
- allanwind 3y ago
  The "food" typo is just too good to ignore in this context.
spenczar5 3y ago
> The biggest thing any ML practitioner realizes when they step out of a research setting is that for most tasks accuracy has to be very high for it be productizable. I think that ChatGPT's success might be partly attributable to its chat interface. For whatever reason, a lot of people - including me! - are much more forgiving of inconsistencies, slip-ups, and inaccuracies when in a conversational format. Kind of like how you might forgive a real human for making a mistake in conversation. I don't think that's necessarily good, and might not have much connection to attempts to build new non-conversational products on top of LLMs, but maybe it has some explanatory power for the current situation.
- startupsfail 3y ago
  There are several reasons to forget: - copilots are useful - chat is entertaining and useful - future tech is coming - investment money
- dougb5 3y ago
  I don't know if I'm more forgiving of inaccuracies in a conversational interface, but I'm way less likely to notice them in the first place. Especially since the current crop of RLHF'd models are so eager to please that they say nearly everything with high confidence.
  - __loam 3y ago
    Yeah I think a lot of people think RLHF is a tool for increasing accuracy but it really is training it to be convincing.
    - godelski 3y ago
      Which should be rather obvious if you understand that it's basically a GAN. You have a discriminative model who's objective function is based on Justice Potter's description of porn: I know it when I see it. If you ask what types of errors might emerge from such a formulation I think it becomes rather obvious. Which isn't to say that the tools aren't useful. But I have to add this fact because many people conflate any criticism with being dismissive of the technology. But technology is about progress, not completion. Gotta balance criticism and optimism. Optimism drives you and criticism directs you.
ianbicking 3y ago
One interesting thing about LLMs is that they can actually recover (and without error loops). You can have a step that doesn't work right, and a later step can use its common-sense knowledge to ignore some of the missing results, conflicting information, etc. One of the problems with developing with LLMs is that the machine will often cover up bugs! You think it's giving sub-par results but actually you've given it conflicting or incomplete instructions. Another opportunity is that you can have less steps or more shared context. One interesting thing about Whisper is that it's not just straight speech recognition but can also be prompted and given context to understand what sort of thing the speech may be about, increasing its accuracy considerably. LLM Vision models also do this with things like OCR. This might not help it with the individual digits in an account number, but it does help with distinguishing an account number from a street address on a check. Or to take another old-style ML technique, you probably shouldn't be doing sentiment analysis in some pipeline, because you don't need to: instead you should step back and look at the purpose of the sentiment analysis and see if you can connect that purpose directly with the original text. All that said, you definitely can write pipelines with compounding errors. We haven't collectively learned how to factor problems and engineer these systems with LLMs yet. Among the things I think we have to do is connect the tools more directly with user intention (effectively flatting another error-inducing part of the pipeline), and make the pipelines collaborative with users. This is more complex and distinctly not autonomous, but then hopefully you are addressing a broader problem or doing so in a more complete way.
LarsDu88 3y ago
The first think any ML practictioner realizes is that accuracy is about the single worst performance metric you can use for most real-world tasks, lol
- altdataseller 3y ago
  Can you explain this? Why is that?
  - LarsDu88 3y ago
    Let me sell you an amazing cancer test my friend. It's 99.5 percent accurate. It works through an incredibly novel mathematical technique. You simply go to the patient and tell them they don't have cancer. Since 99.5 percent of people don't have cancer, this classifier is 99.5 percent accurate. Completely bullshit classifier for a completely bullshit metric. Use sensitivity/specificity or precision/sensitivity instead
rozap 3y ago
This has been a perfect description of my experience doing this. I had written some code to go through reasonably complex web onboarding flows and it basically played out exactly like you predicted in your comment. In addition, I've been working with some vendors that have been trying to do the same thing and they're finding that it works out just like you describe. The handwritten automations have performed better and the issues are reproducible, so even when there are issues, there's some sense of forward progress as you fix them. With handing it all over to an agent, it really feels like running around in circles. I think there's probably something here, but it's less trivial than just tossing a webpage at chatGPT and hoping for the best.
jptoor 3y ago
You're 100% right - but I do think there are more lower accuracy cases than I initially expected, *especially* if you assume a human-in-the-loop. Still 10x better than status quo. Ex. Content generation + zero-shot classification/mapping are powerful, and with a human in the loop (somewhat) responsible for accuracy, they can move much faster.
gremlinsinc 3y ago
> You can do handwritten digit recognition with 90% accuracy? Sounds pretty good, but if you need to turn that into recognizing a 12 digit account number you now have a 70% chance of getting at least one digit incorrect. This means a product worthy digit classifier needs to be much higher accuracy. Language is essential for human civilization, so are tools. We wouldn't get far without either. maybe a language model can understand what it needs to do but not how to do it, so you give it a tool. Humans can get pretty far without 100 percent accuracy, we can get a lot from AI models before they reach 100 percent, but being that at some point AI will be able to improve itself even remake itself daily with 2x the abilities, 100 percent or at least 99.7 percent is attainable. Right now I can take any YouTube video summarize it and turn it into a podcast, short form videos, and a blog post. There's definitely a lot of marketing uses right now for ai agents. If you think about embodied AI, it's only as good as it's body. if it doesn't have good grippers it will struggle to pick things up. Also with a lot of things, accuracy is subjective one person might think ad copy is great and maybe their manager thinks it's shit. One person could give it a 100 percent score and another a 70 percent. My point is we're so close here, and it's already amazing technology and we can augment failures by creating larger toolboxes.
usgroup 3y ago
I'm not sure this argument is in any way specific to LLMs, and the space for their application is still enormous. Search results, ad targeting, recommendation systems, anomaly detection, content flagging, and so on, are all systems using machine learning with a high false positive rate. Up until fairly recently many systems used non-LLM models for making decisions based on natural language. Their performance would have been far worse but they still did useful work. Examples would include content policy enforcement, semantic search and so on. There are very many cases where a system will make an automated decision on a heuristic or random basis for lack of better options. ML improved those decision points and spawned new ones. LLMs improve a subset of those decision points and spawn new ones.
krallistic 3y ago
> The biggest thing any ML practitioner realizes when they step out of a research setting is that for most tasks accuracy has to be very high for it be productizable. You can do handwritten digit recognition with 90% accuracy? It's way more nuanced than this. Of course, you need a decent "accuracy" (not necessarily the metric), but in many business cases, you don't need high accuracy. But you need a solid process: you can catch errors later, you can cross references etc, you need to failsafe, you need to have post-mortem error handling, etc... I shipped stuff (classical ML) that was nothing more than "a biased coin flip," but that still generates value ($) due to the process around it.
greenie_beans 3y ago
> There are cases where lower accuracy results are acceptable, but most people don't even consider this before embarking on their journey to build an AI product/agent. what do you think would help people consider this before going down that path?
renonce 3y ago
The last widely used AI tool was facial recognition, a technology widely used in fields such as company clock-ins, access control, surveillance, and more, and it is so trusted that facial recognition is often the sole method for clocking in. These facial recognition systems can maintain an extremely high accuracy rate for every entry and exit of thousands of people in a database every day. Now when will LLMs achieve such accuracy?
- bitnasty 3y ago
  They have… they write language and they are good at it. The problem is language is not reality, proper language does not mean truth or fact. The models predict what is most likely to come next, not what is most likely to reflect reality.
  - nomel 3y ago
    > what is most likely to come next > not what is most likely to reflect reality. Shouldn't there be a strong statistical correlation between the two? And, isn't that, fundamentally, more about intent of the training? If I train a model to predict what comes next in reality, it's through next word prediction, but it is predicting what reflects reality the best.

chenxi9649 3y ago

Yea that's a good point. Now I am curious, what are some tasks that can accept a model that is at 80% as good as a human, but is 100x cheaper?(or, 100x faster?)

Aerbil313 3y ago
A ton of tasks. Call centers to start with (they already do[1]), with human fallback. 1: In my country, after ChatGPT launched last year, when you call customer support you are now prompted to “just say in a few words” what you want instead of going through tap-this-number menus (they exist as a fallback) and I believe the backend is an LLM. The user flow and voice recordings are still programmatically determined though, but I can easily see one streamlined model calling APIs and whatnot, handling it all.
- salad-tycoon 3y ago
  I speak English clearly. These things always tell me to repeat what I said. Never once has it ever worked for me. I want to throw my phone at the wall. Also I think this has been around for longer than chatgpt. It is often accompanied by a fake keyboard clicking noise.
  - sofixa 3y ago
    > I speak English clearly. These things always tell me to repeat what I said. Never once has it ever worked for me. I want to throw my phone at the wall Now imagine how well it works for people with non-"native" accents (even for native, I'd guess that a good Scouse/Glaswegian/Kiwi accent might confuse the hell of those systems as well). It's a disaster and I hate those.
    - int_19h 3y ago
      There's a general problem across the tech industry with replacing existing simple and reliable (in a sense of conveying the user's intention) interfaces like physical buttons with stuff that is supposedly "more natural" like speech recognition or swipe gestures that in practice has a much higher error rate. See also: replacing physical buttons with convoluted swipe gestures on mobile devices in the never-ending quest to make screen as large as possible. When was this ever a user ask? I feel sometimes like the present UX design is one large LLM-like hallucination.
- chenxi9649 3y ago
  Yea that's pretty cool too, I heard some restaurants are also doing a 100% voice LLM to take orders. Transcription, specifically Whisper, is one of those ML models where the accuracy is basically on-par with humans. So, I really expect a lot more to come out of real time voice/LLM integrations.(the ChatGPT voice thing is a good glimpse, but very janky)
  - allanwind 3y ago
    "A firm providing AI drive-thru tech to fast food chains actually relies on human workers to take orders 70% of the time": https://www.businessinsider.com/ai-drive-thru-tech-relies-on-workers-70-of-time-2023-12?op=1 https://www.businessinsider.com/ai-drive-thru-tech-relies-on...
geoduck14 3y ago
Scan a menu, look for the different entrees, identify the most probable ingredients, determine health content. Then: allow people to search for food based allergies, food aversions, calories. Generate pictures of what the food might look like, display the pics next to the food to make it more likely a user will buy that food.
- charleslmunger 3y ago
  Until one of your customers' children eats a peanut that the AI didn't infer would be an ingredient, and dies. Generating fake pictures also seems like it would be more ordinary false advertising.
  - red-iron-pine 3y ago
    also requires the resto or manufacturer to list all ingredients, and most already list potential allergens. but yeah liability would scare me, esp. because without it you can put the liability squarely on the restaurant or in some cases the person ordering / asking / buying.
tetha 3y ago
Similar to the sibling comment, helpdesk ticket routing. The volume of helpdesk tickets large enterprises deal with is very easily and vastly underestimated. If you can even route 30% away from the central triage with 90+% accuracy and drop everything else back to the central triage... you suddenly safe 2 FTEs in that spot in some places. And increase customer satisfaction for most of those tickets because they get resolved faster. Or, as much as people hate it, chatbots as a customer front. Yes, everyone here as an expert in a lot of tech has had terrible experiences with chatbots. Please mark your hate with the word "Lemon" in the comments. But decently implemented chatbots with a few systems behind them can resolve staggering amounts of simple problems from non-techies without human interaction from the company deploying them. It remains important to eventually escalate to humans - including the history from all of these interactions to avoid frustrations, sure. Or, ticket/request preprocessing. Remember how spelling that 10 digit account number to a call center agent hard of hearing sucks? Those 4 retries because of you not using a better way to communicate that number also costs the company. Now, you can push a few of these retries into an AI system. If you mail them, an AI system can try to extract information like account numbers, intent, start of the problem, problem descriptions and such into dedicated fields to make the support agents faster. Companies are certainly overdoing it at the moment, I'm not denying that. But a lot of the support/helpdesk pre-screening can be automated with current AI/ML capabilities very decently. Especially if you learn to recognize and navigate it.
- Karrot_Kream 3y ago
  The last company I worked at eventually became a Big Tech. In the beginning though, we used to ask all engineers to pair with customer service folks to deal with ticket triage. When we got a bit bigger, we used to have rotations where eng would pair with customer service folks. Being on the other side of that was very eye-opening for all eng. Many used to come in with the same bias that you see on this site, that how dare you be routed to some automated service and how inhumane the service is. On the other side you see competent CS agents absolutely swamped with low-level questions that were often literally answered in docs and FAQ pages. I think getting transformer-based triage models correct can unlock tons of value.
- Sabinus 3y ago
deleted 3y ago
[deleted]
vintermann 3y ago
Well, an old one is OCR, especially handwritten OCR. I'm doing genealogy. There is SO MUCH old handwritten material that is never transcribed, and which requires special expertise to read (old and exotic handwriting styles) and interpret (place names, writing conventions, abbreviations). It doesn't have to be perfect. It's not as if the actual data in there is perfect. It just has to be in a form where I can search it, ideally with named entities mapped. Quality - like deciphering the writing on scrolls buried in volcanic ash in Herculaneum - gets all the attention. But what I really want is quantity - I want to be able to search through those 5000 pages of 200 year old mildly damaged cadastral records in dense handwriting. I want to relieve the army of kind retirees who currently transcribe these sorts of documents one by one based on their own needs.
- jazzyjackson 3y ago
  bruh what are the retirees supposed to do once you've automated their hobby

Qwero 3y ago

I use already a few ai tools even without perfect accuracy. And a LLM who only needs to call to a few API calls isn't hard. Very little need perfect accuracy and for that we still have classical software.

chenxi9649 3y ago
curious to know, which tools do you use and how do you use em?
- Qwero 3y ago
  I use copilot for coding. CharGPT for writing emails to bigger audiences, Gramma correction. Image generator for fun. LLM for Feature Extraction from random text like a website or PDF (llama).
skywhopper 3y ago
You use them successfully because your human mind can filter out the junk. It would only take one inaccurate API call that charges your credit card $10k or sells your car for 10 cents to cause a lot of damage to your life.
- qeternity 3y ago
  Which is why even with classical software, most of us don't have APIs where that's all it takes.
  - ltengelis 3y ago
    Frankly I think you've said it all here - a properly designed API + a well designed LLM interface on top of that enables non-technical people to do things they otherwise couldn't.

mFixman 3y ago

> You can do handwritten digit recognition with 90% accuracy? Sounds pretty good, but if you need to turn that into recognizing a 12 digit account number you now have a 70% chance of getting at least one digit incorrect. You are assuming that the probability of failure is independent, which couldn't be further from the truth. If a digit recogniser can recognise one of your "hard" handwritten digits, such as a 4 or a 9, it will likely be able to recognise all of them. The same happens with AI agents. They are not good at some tasks, but really really food at others.

jhbadger 3y ago
And the US Post Office and other postal services have been using this tech to sort letters for several decades now (although postal codes with both letters and numbers like Canada's are harder). It was viewed as the "killer app" for ML in the 1990s.
- PheonixPharts 3y ago
  This thread is an object lesson in the point I'm making: people have forgotten everything we've learned about making ML based products. Parent comment doesn't understand the concept of expectation, and this comment is apparently unfamiliar with the fact that SotA for digit recognition [0] has been much higher than 90% even in the 90s. 90% accuracy for digit recognition is what you get if you use logistic regression as your model. My point was that numbers that look good in terms of research often aren't close to go enough for the real world. It's not that 90% works for zip codes, it's that in the 90s accuracy was closer to 99%. You have validated my point rather than rejected it. 0. https://en.wikipedia.org/wiki/MNIST_database https://en.wikipedia.org/wiki/MNIST_database
allanwind 3y ago
The "food" typo is just too good to ignore in this context.

spenczar5 3y ago

> The biggest thing any ML practitioner realizes when they step out of a research setting is that for most tasks accuracy has to be very high for it be productizable. I think that ChatGPT's success might be partly attributable to its chat interface. For whatever reason, a lot of people - including me! - are much more forgiving of inconsistencies, slip-ups, and inaccuracies when in a conversational format. Kind of like how you might forgive a real human for making a mistake in conversation. I don't think that's necessarily good, and might not have much connection to attempts to build new non-conversational products on top of LLMs, but maybe it has some explanatory power for the current situation.

startupsfail 3y ago
There are several reasons to forget: - copilots are useful - chat is entertaining and useful - future tech is coming - investment money
dougb5 3y ago
I don't know if I'm more forgiving of inaccuracies in a conversational interface, but I'm way less likely to notice them in the first place. Especially since the current crop of RLHF'd models are so eager to please that they say nearly everything with high confidence.
- __loam 3y ago
  Yeah I think a lot of people think RLHF is a tool for increasing accuracy but it really is training it to be convincing.
  - godelski 3y ago
    Which should be rather obvious if you understand that it's basically a GAN. You have a discriminative model who's objective function is based on Justice Potter's description of porn: I know it when I see it. If you ask what types of errors might emerge from such a formulation I think it becomes rather obvious. Which isn't to say that the tools aren't useful. But I have to add this fact because many people conflate any criticism with being dismissive of the technology. But technology is about progress, not completion. Gotta balance criticism and optimism. Optimism drives you and criticism directs you.

ianbicking 3y ago

One interesting thing about LLMs is that they can actually recover (and without error loops). You can have a step that doesn't work right, and a later step can use its common-sense knowledge to ignore some of the missing results, conflicting information, etc. One of the problems with developing with LLMs is that the machine will often cover up bugs! You think it's giving sub-par results but actually you've given it conflicting or incomplete instructions. Another opportunity is that you can have less steps or more shared context. One interesting thing about Whisper is that it's not just straight speech recognition but can also be prompted and given context to understand what sort of thing the speech may be about, increasing its accuracy considerably. LLM Vision models also do this with things like OCR. This might not help it with the individual digits in an account number, but it does help with distinguishing an account number from a street address on a check. Or to take another old-style ML technique, you probably shouldn't be doing sentiment analysis in some pipeline, because you don't need to: instead you should step back and look at the purpose of the sentiment analysis and see if you can connect that purpose directly with the original text. All that said, you definitely can write pipelines with compounding errors. We haven't collectively learned how to factor problems and engineer these systems with LLMs yet. Among the things I think we have to do is connect the tools more directly with user intention (effectively flatting another error-inducing part of the pipeline), and make the pipelines collaborative with users. This is more complex and distinctly not autonomous, but then hopefully you are addressing a broader problem or doing so in a more complete way.

LarsDu88 3y ago

The first think any ML practictioner realizes is that accuracy is about the single worst performance metric you can use for most real-world tasks, lol

altdataseller 3y ago
Can you explain this? Why is that?
- LarsDu88 3y ago
  Let me sell you an amazing cancer test my friend. It's 99.5 percent accurate. It works through an incredibly novel mathematical technique. You simply go to the patient and tell them they don't have cancer. Since 99.5 percent of people don't have cancer, this classifier is 99.5 percent accurate. Completely bullshit classifier for a completely bullshit metric. Use sensitivity/specificity or precision/sensitivity instead

rozap 3y ago

This has been a perfect description of my experience doing this. I had written some code to go through reasonably complex web onboarding flows and it basically played out exactly like you predicted in your comment. In addition, I've been working with some vendors that have been trying to do the same thing and they're finding that it works out just like you describe. The handwritten automations have performed better and the issues are reproducible, so even when there are issues, there's some sense of forward progress as you fix them. With handing it all over to an agent, it really feels like running around in circles. I think there's probably something here, but it's less trivial than just tossing a webpage at chatGPT and hoping for the best.

jptoor 3y ago

You're 100% right - but I do think there are more lower accuracy cases than I initially expected, *especially* if you assume a human-in-the-loop. Still 10x better than status quo. Ex. Content generation + zero-shot classification/mapping are powerful, and with a human in the loop (somewhat) responsible for accuracy, they can move much faster.

gremlinsinc 3y ago

> You can do handwritten digit recognition with 90% accuracy? Sounds pretty good, but if you need to turn that into recognizing a 12 digit account number you now have a 70% chance of getting at least one digit incorrect. This means a product worthy digit classifier needs to be much higher accuracy. Language is essential for human civilization, so are tools. We wouldn't get far without either. maybe a language model can understand what it needs to do but not how to do it, so you give it a tool. Humans can get pretty far without 100 percent accuracy, we can get a lot from AI models before they reach 100 percent, but being that at some point AI will be able to improve itself even remake itself daily with 2x the abilities, 100 percent or at least 99.7 percent is attainable. Right now I can take any YouTube video summarize it and turn it into a podcast, short form videos, and a blog post. There's definitely a lot of marketing uses right now for ai agents. If you think about embodied AI, it's only as good as it's body. if it doesn't have good grippers it will struggle to pick things up. Also with a lot of things, accuracy is subjective one person might think ad copy is great and maybe their manager thinks it's shit. One person could give it a 100 percent score and another a 70 percent. My point is we're so close here, and it's already amazing technology and we can augment failures by creating larger toolboxes.

usgroup 3y ago

I'm not sure this argument is in any way specific to LLMs, and the space for their application is still enormous. Search results, ad targeting, recommendation systems, anomaly detection, content flagging, and so on, are all systems using machine learning with a high false positive rate. Up until fairly recently many systems used non-LLM models for making decisions based on natural language. Their performance would have been far worse but they still did useful work. Examples would include content policy enforcement, semantic search and so on. There are very many cases where a system will make an automated decision on a heuristic or random basis for lack of better options. ML improved those decision points and spawned new ones. LLMs improve a subset of those decision points and spawn new ones.

krallistic 3y ago

> The biggest thing any ML practitioner realizes when they step out of a research setting is that for most tasks accuracy has to be very high for it be productizable. You can do handwritten digit recognition with 90% accuracy? It's way more nuanced than this. Of course, you need a decent "accuracy" (not necessarily the metric), but in many business cases, you don't need high accuracy. But you need a solid process: you can catch errors later, you can cross references etc, you need to failsafe, you need to have post-mortem error handling, etc... I shipped stuff (classical ML) that was nothing more than "a biased coin flip," but that still generates value ($) due to the process around it.

greenie_beans 3y ago

> There are cases where lower accuracy results are acceptable, but most people don't even consider this before embarking on their journey to build an AI product/agent. what do you think would help people consider this before going down that path?

renonce 3y ago

The last widely used AI tool was facial recognition, a technology widely used in fields such as company clock-ins, access control, surveillance, and more, and it is so trusted that facial recognition is often the sole method for clocking in. These facial recognition systems can maintain an extremely high accuracy rate for every entry and exit of thousands of people in a database every day. Now when will LLMs achieve such accuracy?

bitnasty 3y ago
They have… they write language and they are good at it. The problem is language is not reality, proper language does not mean truth or fact. The models predict what is most likely to come next, not what is most likely to reflect reality.
- nomel 3y ago
  > what is most likely to come next > not what is most likely to reflect reality. Shouldn't there be a strong statistical correlation between the two? And, isn't that, fundamentally, more about intent of the training? If I train a model to predict what comes next in reality, it's through next word prediction, but it is predicting what reflects reality the best.