3 ms·

> I'm curious what those things are. > At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry

> I'm curious what those things are. > At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more) That’s a list of things that gets clicks in the popular press. Some solutions I love: Recording a video and creating a transcript from it. Then edit the transcript and the video gets edited. Translating (dubbing) video and changing the lips of the speaker to match where they should be for the new audio. Scanning invoices for mistakes (I know entire businesses build just in this one thing). Understanding edge cases. So many things have some nice hard and fast rules that can fail, and an ML system of some sort can figure it out so the system can move on. I do this for processing SEC data, and soon for web scraping (like they change the html but visually it’s kinda the same, and the ai system can figure it, give me a new html selector, and back in business).

surfingdino 2y ago
You may want to familiarise yourself with https://www.vice.com/en/article/gvy4gb/one-mans-david-and-goliath-battle-to-get-xerox-to-fix-a-major-bug https://www.vice.com/en/article/gvy4gb/one-mans-david-and-go...
vitus 2y ago
> That’s a list of things that gets clicks in the popular press. Are you saying they're nonissues in practice? I agree that most of those points have shown up in the news, but they're also things that I have personally observed when interacting with LLMs. Of your (and sibling commenters') cited use cases, I see a number of scenarios where AI is used to perform a quick first pass, and a human then refines that output (transcript generation, scanning invoices, iterating on transformations for syntax trees, etc). That's great that it works for you. My worry here is that you might heuristically observe that it worked perfectly 20 times in a row, then decide to remove that human check even as it admits more errors than is acceptable for your use case. > Scanning invoices for mistakes This is one of those cases where I would like to better understand the false negatives. If a human reviews the output, then okay, false positives are easy enough to override. But how bad is a false negative? Is it just unnecessary expenses to the company, or does it expose them to liability? > Translating (dubbing) video and changing the lips of the speaker to match where they should be for the new audio. This is useful in itself, but surely you too can see the potential for abuse? (This is literally putting words in someone else's mouth.)
- sroussey 2y ago
  > This is one of those cases where I would like to better understand the false negatives. If a human reviews the output, then okay, false positives are easy enough to override. But how bad is a false negative? Is it just unnecessary expenses to the company, or does it expose them to liability? In the companies I know about, these invoices requesting overpayment just got paid. So, worst case, it’s the same. But best case there is way way way more money to save than the cost of the service.
- sroussey 2y ago
  > This is useful in itself, but surely you too can see the potential for abuse? (This is literally putting words in someone else's mouth.) If I was a famous actor, I would demand it. I don’t want people hearing different voices for different movies. I want them to hear my voice. And I’d want it to be authentic. Seeing lips move to the wrong words does not help make a connection. As for abuse, sure. Not sure anything has had worse abuse than database technology. There should definitely be an avenue for the government to shutdown any database instance anywhere (like California is doing with AI). I would have shutdown data broker databases long ago.

surfingdino 2y ago

You may want to familiarise yourself with https://www.vice.com/en/article/gvy4gb/one-mans-david-and-goliath-battle-to-get-xerox-to-fix-a-major-bug https://www.vice.com/en/article/gvy4gb/one-mans-david-and-go...

vitus 2y ago

> That’s a list of things that gets clicks in the popular press. Are you saying they're nonissues in practice? I agree that most of those points have shown up in the news, but they're also things that I have personally observed when interacting with LLMs. Of your (and sibling commenters') cited use cases, I see a number of scenarios where AI is used to perform a quick first pass, and a human then refines that output (transcript generation, scanning invoices, iterating on transformations for syntax trees, etc). That's great that it works for you. My worry here is that you might heuristically observe that it worked perfectly 20 times in a row, then decide to remove that human check even as it admits more errors than is acceptable for your use case. > Scanning invoices for mistakes This is one of those cases where I would like to better understand the false negatives. If a human reviews the output, then okay, false positives are easy enough to override. But how bad is a false negative? Is it just unnecessary expenses to the company, or does it expose them to liability? > Translating (dubbing) video and changing the lips of the speaker to match where they should be for the new audio. This is useful in itself, but surely you too can see the potential for abuse? (This is literally putting words in someone else's mouth.)

sroussey 2y ago
> This is one of those cases where I would like to better understand the false negatives. If a human reviews the output, then okay, false positives are easy enough to override. But how bad is a false negative? Is it just unnecessary expenses to the company, or does it expose them to liability? In the companies I know about, these invoices requesting overpayment just got paid. So, worst case, it’s the same. But best case there is way way way more money to save than the cost of the service.
sroussey 2y ago
> This is useful in itself, but surely you too can see the potential for abuse? (This is literally putting words in someone else's mouth.) If I was a famous actor, I would demand it. I don’t want people hearing different voices for different movies. I want them to hear my voice. And I’d want it to be authentic. Seeing lips move to the wrong words does not help make a connection. As for abuse, sure. Not sure anything has had worse abuse than database technology. There should definitely be an avenue for the government to shutdown any database instance anywhere (like California is doing with AI). I would have shutdown data broker databases long ago.