7 ms·
Further, the problem with this example is it relies on a comparison against public data. Most of these AI start failing pretty hard when you ask it to do the s
by SkyPuncher 3y ago
Further, the problem with this example is it relies on a comparison against public data.
Most of these AI start failing pretty hard when you ask it to do the same task on something completely novel to it (like a company document). Sometimes they'll get it right. Other times, they'll spit out gibberish that's clearly some generic answer.
- nomel 3y ago> Most of these AI This is as meaningful as saying most of the hominids can't count. You can't usefully generalize AI models with the rate of change that exists right now. Any statements/comparisons about AI has to contain specific models and versions, otherwise it's increasingly irrelevant noise.
- jiggawatts 3y agoEvery time someone has said "LLMs can't do X", I tried X in GPT 4 and it could do it. They usually try free LLMs like Bard or GPT 3 and assume that the results generalise.
- dmix 3y agoI'd imagine working with an entire company document would require a lot more hand holding and investment in prompt engineering. You can definitely get better results if you add much more context of what you're expecting and how the LLM should do it. Treating these LLMs as just simple Q&A machines is usually not enough unless you're doing simple stuff.
- robotresearcher 3y agoAsking to spot the difference between a given document and an unseen document is impossible.
- lkbm 3y agoA couple years ago, I read Superfudge by Judy Blume, a book originally published in 1980. In it, the protagonist writes a letter to Santa: "Please bring me one or more of the following items. A clock-radio, a remote-controlled model airplane, a laptop computer, an MP3 player and six CD's." I didn't need to have seen this book before to know this wasn't in the original 1980s text. Similarly, if I were reading the Great Gatsby for the first time, and it identified a character as a software engineer, I would notice.
- drusepth 3y agoI think there are plenty of humans who wouldn't notice, though. And probably plenty of AI implementations that would notice.
- bugglebeetle 3y agoAre we now aspiring to ABAI (Artificial Below-Average Intelligence)?
- cosmojg 3y agoI regret to inform you that "average intelligence" is a far lower bar than you might think.
- EGreg 3y agoThe AI would not only notice, it would also notice 300 other things that are far more subtle haha
- 1024core 3y agoAs the joke goes: look around you and see how dumb the average person is. And now imagine, half the people in your city/country/Earth are dumber than this person.