4 ms·

Boy, I can’t wait for the foundation of my house to disappear because the LLM mis-classified a drilling report as non-hazardous. What’s the deal here with liab

Boy, I can’t wait for the foundation of my house to disappear because the LLM mis-classified a drilling report as non-hazardous. What’s the deal here with liability and accountability? That’s a serious problem when considering using these for anything other than toy problems.

bongodongobob 2y ago
You don't actually think the LLM is reviewing those 30k documents do you? You tell it to write a program (which is easy to audit) to pull the info from the PDFs or whatever. I don't get why this crowd is so goddamn unimaginative with LLMs.
- goatlover 2y ago
  Because I've heard of enough lazy uses of LLMs to be suspicious. Auditing the program means being sure that the info pulled from those documents is reviewed properly. Also, a complete lack of regard for other people's privacy.
  - bongodongobob 2y ago
    No idea where privacy enters in here.
- svieira 2y ago
  > You tell it to write a program (which is easy to audit) to pull the info from the PDFs Wherein you discover that unless you ask it to consider the fact that PDFs are ... very hard to parse [1] [2] you get something that misses whole blocks of text or turns them into something they aren't and the rest of the program misses chunks of the document. [1]: https://news.ycombinator.com/item?id=22473263 https://news.ycombinator.com/item?id=22473263 [2]: https://web.archive.org/web/20200303102734/https://www.filingdb.com/pdf-text-extraction https://web.archive.org/web/20200303102734/https://www.filin...
  - bongodongobob 2y ago
    Why are you expecting they are all very different? They're all likely very similar.
    - svieira 2y ago
      Because presuming that all of them are produced by the same utility is a _presumption_. They could be - but they could also be produced by many different vendors using many different methods all of them simply conforming to the specification "a PDF with HIGH LEVEL DESCRIPTION OF THE DATA".
jakderrida 2y ago
>Boy, I can’t wait for the foundation of my house to disappear because the LLM mis-classified a drilling report as non-hazardous LMAO! It's so hilarious that people like you forget that the alternative is relying on bureaucracies managed by people that get things wrong more often and are both too lazy and too stubborn to process your application to review your drilling report again. If using both human-level and AI-level analysis is cheaper and much more accurate (but still imperfect), I'm willing to settle for a better system than oppose all change and die holding out for a perfect system.
- JBorrow 2y ago
  What are 'people like me'? It's not like I know nothing about large language models, I just think using them for civil engineering is a bad idea...

bongodongobob 2y ago

You don't actually think the LLM is reviewing those 30k documents do you? You tell it to write a program (which is easy to audit) to pull the info from the PDFs or whatever. I don't get why this crowd is so goddamn unimaginative with LLMs.

goatlover 2y ago
Because I've heard of enough lazy uses of LLMs to be suspicious. Auditing the program means being sure that the info pulled from those documents is reviewed properly. Also, a complete lack of regard for other people's privacy.
- bongodongobob 2y ago
  No idea where privacy enters in here.
svieira 2y ago
> You tell it to write a program (which is easy to audit) to pull the info from the PDFs Wherein you discover that unless you ask it to consider the fact that PDFs are ... very hard to parse [1] [2] you get something that misses whole blocks of text or turns them into something they aren't and the rest of the program misses chunks of the document. [1]: https://news.ycombinator.com/item?id=22473263 https://news.ycombinator.com/item?id=22473263 [2]: https://web.archive.org/web/20200303102734/https://www.filingdb.com/pdf-text-extraction https://web.archive.org/web/20200303102734/https://www.filin...
- bongodongobob 2y ago
  Why are you expecting they are all very different? They're all likely very similar.
  - svieira 2y ago
    Because presuming that all of them are produced by the same utility is a _presumption_. They could be - but they could also be produced by many different vendors using many different methods all of them simply conforming to the specification "a PDF with HIGH LEVEL DESCRIPTION OF THE DATA".

jakderrida 2y ago

>Boy, I can’t wait for the foundation of my house to disappear because the LLM mis-classified a drilling report as non-hazardous LMAO! It's so hilarious that people like you forget that the alternative is relying on bureaucracies managed by people that get things wrong more often and are both too lazy and too stubborn to process your application to review your drilling report again. If using both human-level and AI-level analysis is cheaper and much more accurate (but still imperfect), I'm willing to settle for a better system than oppose all change and die holding out for a perfect system.

JBorrow 2y ago
What are 'people like me'? It's not like I know nothing about large language models, I just think using them for civil engineering is a bad idea...