5 ms·

Will it really though? So far I've seen most of the "revolutionise" claims to be mainly hot air and marketing. It's possible that LLMs will suddenly make a lea

Will it really though? So far I've seen most of the "revolutionise" claims to be mainly hot air and marketing. It's possible that LLMs will suddenly make a leap in reliability and usability (e.g. much higher context window without corresponding massive increases in memory usage). But I have yet to see it. So far it's great at some specific usecases. Interacting with humans, rewriting or making up text. Summarising. A hit & miss at everything else. Don't get me wrong, I love AI tech and I'm heavily experimenting with it (both at work and at home with local models). But as with most hyped technologies I find the benefits far overblown in marketing stories. Our leadership jumped on Microsoft Copilot (the one for Office 365 because they have tens of different copilots :) ) like a pack of hungry wolves afraid to miss the boat. And the result was.... kinda meh. It's kinda promising and impresses with simple play school stuff ("make me a presentation about home safety") and totally and utterly fails when you try to do anything serious work related. Sooo many times I get "Sorry I can't do this right now", "Sorry I need more training for this", "I can't do this for you but this is how you can do it yourself!" or it does something but like totally wrong. Meanwhile we have a bunch of MS training people running around evangelising and telling us how great everything is and making excuses for everything that goes wrong :) You can almost see them breathe a sigh of relief every time something works as it should. That's not what we were promised. Maybe it will get there, but I don't see it happening tomorrow to be honest. LLMs were an impressive leap but their achilles heels have become clear and it's proving difficult to overcome them. I'm really enjoying surfing the knife's edge of technology (as I was and still am with metaverse) but I don't yet see this as a game changer except in a few specific industries. People editing text for a living certainly have a need to worry. I also wonder what will happen with future AI training. Now that more and more websites are filled with AI-generated content that is often at best "mediocre", and considering future AI models will be trained on that, will they be able to improve their accuracy or struggle to maintain it?

alach11 2y ago
I use LLMs extensively in my field to automate all sorts of tasks. Need to classify a million PDF documents for cheap? Write a prompt and submit a batch job. Need to read 30,000 drilling reports to automatically scan for hazards? Done in 60 minutes. These are tasks that would have taken months of development or millions of dollars in manual effort before. It's not just hype.
- JBorrow 2y ago
  Boy, I can’t wait for the foundation of my house to disappear because the LLM mis-classified a drilling report as non-hazardous. What’s the deal here with liability and accountability? That’s a serious problem when considering using these for anything other than toy problems.
  - bongodongobob 2y ago
    You don't actually think the LLM is reviewing those 30k documents do you? You tell it to write a program (which is easy to audit) to pull the info from the PDFs or whatever. I don't get why this crowd is so goddamn unimaginative with LLMs.
    - goatlover 2y ago
      Because I've heard of enough lazy uses of LLMs to be suspicious. Auditing the program means being sure that the info pulled from those documents is reviewed properly. Also, a complete lack of regard for other people's privacy.
      - bongodongobob 2y ago
        No idea where privacy enters in here.
    - svieira 2y ago
      > You tell it to write a program (which is easy to audit) to pull the info from the PDFs Wherein you discover that unless you ask it to consider the fact that PDFs are ... very hard to parse [1] [2] you get something that misses whole blocks of text or turns them into something they aren't and the rest of the program misses chunks of the document. [1]: https://news.ycombinator.com/item?id=22473263 https://news.ycombinator.com/item?id=22473263 [2]: https://web.archive.org/web/20200303102734/https://www.filingdb.com/pdf-text-extraction https://web.archive.org/web/20200303102734/https://www.filin...
      - 2y ago

alach11 2y ago

I use LLMs extensively in my field to automate all sorts of tasks. Need to classify a million PDF documents for cheap? Write a prompt and submit a batch job. Need to read 30,000 drilling reports to automatically scan for hazards? Done in 60 minutes. These are tasks that would have taken months of development or millions of dollars in manual effort before. It's not just hype.

JBorrow 2y ago
Boy, I can’t wait for the foundation of my house to disappear because the LLM mis-classified a drilling report as non-hazardous. What’s the deal here with liability and accountability? That’s a serious problem when considering using these for anything other than toy problems.
- bongodongobob 2y ago
  You don't actually think the LLM is reviewing those 30k documents do you? You tell it to write a program (which is easy to audit) to pull the info from the PDFs or whatever. I don't get why this crowd is so goddamn unimaginative with LLMs.
  - goatlover 2y ago
    Because I've heard of enough lazy uses of LLMs to be suspicious. Auditing the program means being sure that the info pulled from those documents is reviewed properly. Also, a complete lack of regard for other people's privacy.
    - bongodongobob 2y ago
      No idea where privacy enters in here.
  - svieira 2y ago
    > You tell it to write a program (which is easy to audit) to pull the info from the PDFs Wherein you discover that unless you ask it to consider the fact that PDFs are ... very hard to parse [1] [2] you get something that misses whole blocks of text or turns them into something they aren't and the rest of the program misses chunks of the document. [1]: https://news.ycombinator.com/item?id=22473263 https://news.ycombinator.com/item?id=22473263 [2]: https://web.archive.org/web/20200303102734/https://www.filingdb.com/pdf-text-extraction https://web.archive.org/web/20200303102734/https://www.filin...
    - 2y ago