4 ms·

AI isn't the most efficient way to retrieve accurate information or to generate quality content, it's the most efficient way to retrieve plausible information a

by caseyy 2y ago

AI isn't the most efficient way to retrieve accurate information or to generate quality content, it's the most efficient way to retrieve plausible information and generate poor quality art and code. And that's really interesting. There is a group of people that trust LLMs for all searches but most tech-literate people know they hallucinate, and understand roughly how an LLM works and that hallucinations are indistinguishable from the truth to the model. And so we still Google things and read books. The same applies to AI-generated art, music, voiceovers, and code. They are technically impressive, but rarely meet client expectations in any industry. And so most artists and programmers still produce the work themselves. AI is fantastic for low-quality content, and this explosion of AI is emblematic of how people often lack a filter for quality of information they put in their brains. This isn't new; YouTube has always had more low-quality content than high-quality pieces, probably 1000:1 if not a higher contrast. However, AI shows the scale like no tech has shown it to us before (and maybe this was known by some people at Google or Bing but not so publicly). Almost everyone consumes AI work — blogs, movie posters, art, music, and code. The market is massive but all AI does is reproduce the existing work it was loss-minimized on and the loss of quality is mathematically necessary unless the model is not a neural net, but a database of literally the entire dataset itself that is being searched. So this is very interesting. When did we become so okay with feeding ourselves a fast food diet of content? Isn't that causing more harm than good? The scale of this phenomenon is truly (and literally) industrial. I myself can hardly imagine what the modern life would look like if people turned to the internet, let's say, only to look up facts and information they already seek. We are so far from that. And AI both illustrates it, and exacerbates it.

tibbar 2y ago
I see people frame generative AI this way a lot, as if they're strictly worse than alternatives in all dimensions (inaccurate, low quality). But I think there's several important dimension where they're strictly better, including flexibility, interactivity, and accessibility. I can ask an LLM a very specific question about my specific development environment, codebase, and tools and get a specific answer back that works. If it doesn't work, I can chat with the LLM about what happened to make changes. I don't even reach for LLMs that often compared to most engineers these days (maybe a couple times a month), but when I do, it's basically always the fastest and best way to deal with my particular situation.
- deleted 2y ago
  [deleted]
- bruce511 2y ago
  I concur. I'm notca huge user of AI, but it does make a good search tool when my search term is verbose. For example I was looking for the name of a windows API command. I "knew" the command must exist, but didn't gave a clue what it was called. Asked Cgpt for an example program, and there's the name of the API. (Which I can then Google for docs.) I also had a complicated-to-ask question about sun movement which it explained to me, along with site links to actual data. I'm not using it as a Google replacement, but more of a Google supplement when the question is long-winded to write.
- caseyy 2y ago
  I think a lot of people drastically underestimate how much the LLMs mislead the user with plausible-sounding answers. I would say that whatever % of queries you found them not to give you correct answers, the actual percentage is materially higher. In my work niche, which at least 10,000 other software engineers do each day, ChatGPT 4 and 4o almost never give correct answers. Usually, they are misleading. I do find myself trying the LLMs when I'm faced with a challenging problem, but in that scenario they have not been helpful once. Granted, there are areas of work that are much more popular and LLMs will be more helpful there. But these are individual scenarios, and we can find many where LLMs are fantastic and many where LLMs are awful if we wanted to cherry-pick. Overall, the quality of the content is lower than what a human professional would do in their area of work, including flexibility, interactivity, and accessibility. This extends to books written by professionals and lectures given by them, as well as work carried out by them. It applies to home carpentry as much as it does to neurosurgery. The more specific the knowledge has to be, the more this is true. The more generic, the less. But all-in-all, I think it's still very evident humans produce higher quality knowledge and content. And any quantized model of that content and knowledge will be unable to reproduce it at the same fidelity or quality. With that said, I hope I expressed this enough — I do see your point in some circumstances. Just not overall.
  - tibbar 2y ago

by caseyy 2y ago

tibbar 2y ago

I see people frame generative AI this way a lot, as if they're strictly worse than alternatives in all dimensions (inaccurate, low quality). But I think there's several important dimension where they're strictly better, including flexibility, interactivity, and accessibility. I can ask an LLM a very specific question about my specific development environment, codebase, and tools and get a specific answer back that works. If it doesn't work, I can chat with the LLM about what happened to make changes. I don't even reach for LLMs that often compared to most engineers these days (maybe a couple times a month), but when I do, it's basically always the fastest and best way to deal with my particular situation.

deleted 2y ago
[deleted]
bruce511 2y ago
I concur. I'm notca huge user of AI, but it does make a good search tool when my search term is verbose. For example I was looking for the name of a windows API command. I "knew" the command must exist, but didn't gave a clue what it was called. Asked Cgpt for an example program, and there's the name of the API. (Which I can then Google for docs.) I also had a complicated-to-ask question about sun movement which it explained to me, along with site links to actual data. I'm not using it as a Google replacement, but more of a Google supplement when the question is long-winded to write.
caseyy 2y ago
I think a lot of people drastically underestimate how much the LLMs mislead the user with plausible-sounding answers. I would say that whatever % of queries you found them not to give you correct answers, the actual percentage is materially higher. In my work niche, which at least 10,000 other software engineers do each day, ChatGPT 4 and 4o almost never give correct answers. Usually, they are misleading. I do find myself trying the LLMs when I'm faced with a challenging problem, but in that scenario they have not been helpful once. Granted, there are areas of work that are much more popular and LLMs will be more helpful there. But these are individual scenarios, and we can find many where LLMs are fantastic and many where LLMs are awful if we wanted to cherry-pick. Overall, the quality of the content is lower than what a human professional would do in their area of work, including flexibility, interactivity, and accessibility. This extends to books written by professionals and lectures given by them, as well as work carried out by them. It applies to home carpentry as much as it does to neurosurgery. The more specific the knowledge has to be, the more this is true. The more generic, the less. But all-in-all, I think it's still very evident humans produce higher quality knowledge and content. And any quantized model of that content and knowledge will be unable to reproduce it at the same fidelity or quality. With that said, I hope I expressed this enough — I do see your point in some circumstances. Just not overall.
- tibbar 2y ago