4 ms·

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of hav

by sho_hn 1y ago

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense of certainty. They have almost the opposite effect on me. Even with knowledge from books or articles I've learned to multi-source and question things, and my mind treats the LLMs as a less reliable averaging of sources.

deadbabe 1y ago
If you feel dumber, it’s because you’re using the LLM to do raw work instead of using it for research. It should be a google/stackoverflow replacement, not a really powerful intellisense. You should feel no dumber than using google to investigate questions.
- Insanity 1y ago
  I don't think this is entirely accurate. If you look at this: https://www.media.mit.edu/publications/your-brain-on-chatgpt/ https://www.media.mit.edu/publications/your-brain-on-chatgpt..., it shows that search engines do engage your brain _more_ than LLM usage. So you'll remember more through search engine use (and crawling the web 'manually') than by just prompting a chatbot.
- pessimizer 1y ago
  I find that it is terrible for research, and hallucinates 25% to 90% of its references. If you tell it to find something and give it a detailed description of what you're looking for, it will pretend like it has verified that that thing exists, and give you a bulletpoint lecture about why it is such an effective and interesting thing that 1) you didn't ask for, and 2) is really it parroting your description back to you with embellishments. I thought I was going to be able to use LLMs primarily for research, because I have read an enormous number of things (books, papers) in my life, and I can't necessarily find them again when they would be useful. Trying to track them down through LLMs is rarely successful and always agonizing, like pulling teeth that are constantly lying to you. A surprising outcome is that I often get so frustrated by the LLM and so detailed in how I'm complaining about its stupid responses that I remind myself of something that allows me to find the reference on my own. I have to suspect that people who find it useful for research are researching things that are easily discoverable through many other means. Those are not the things that are interesting. I totally find it useful to find something in software docs that I'm too lazy to look up myself, but it's literally saving me 10 minutes.
Insanity 1y ago
I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" will slowly fade away and people will blindly trust them.
- aabhay 1y ago
  There’s also the fact that both Wikipedia and LLMs are non-stationary. The quality of wikipedia has grown immensely since its inception and LLMs will get more accurate (if not explicitly “smarter”)
  - derektank 1y ago
    I'm not entirely convinced that the quality of Wikipedia has improved substantially in the last decade.
    - vablings 1y ago
      Care to provide any counter-examples? Please make it know if you end up using Wikipedia for your source of if Wikipedia's quality has changed
      - pessimizer 1y ago
        How in the world would you supply a counter-example for "the quality of Wikipedia has/hasn't improved substantially in the last decade"? I also can't even read the second sentence. I think there are typos there, but there's no mental correction I can do to make it coherent for me.
        jancsika 1y ago
        I can't think of a better accidental metric than that! I'll go ahead and speculate that the number of incoherent sentences per article has gone down substantially over the last decade, probably due to the relevant tooling getting better over the same period.
freejazz 1y ago
> Is anyone really cocksure on the basis of LLM received knowledge? Yeah, the stupid.
code_for_monkey 1y ago
unfortunately im like you and we are in the minority. The manager class loves the llm and doesnt seem to consider its flaws like that.
lukan 1y ago
Nah, I feel smart to use it in a smart way to get stuff done faster than before.
everdrive 1y ago
This captures my experience quite well. I can "get a lot more done," but it's not really me doing the things, and I feel like a bit of a fraud. And as the workday and the workweek roll on, I find myself needing to force myself to look things up and experiment rather than just asking the LLM. It's quite clear that for most people LLMs will make the more dependent. People with better discipline I think will really benefit in big ways, and you'll see this become a new luxury belief; the disciplined geniuses around us will genuinely be perplexed why people are saying that LLMs have made them less capable, much in the same way they wonder why people can't just limit their drug use recreationally.
- Brendinooo 1y ago
  >it's not really me doing the things, and I feel like a bit of a fraud I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here? My car has an automatic transmission, am I a fraud because the machine is shifting gears for me? My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually? Spell check caught a word, am I a fraud because I didn't look it up in a dictionary?
  - everdrive 1y ago
    I've been thinking about that comparison as well. A common fantasy is that civilization will collapse and the guy who knows how to hunt and start a fire will really excel. In practice, this never happens and he's sort of left behind unless he also has other skills relevant to the modern world. And, for instance, I have barely any knowledge of how my computer works, but it's a tool I use to do my job. (and to have fun at home.) Why are these different than using LLMs? I think at least for me the distinction is whether or not something enables me to perform a task, or whether it's just doing the task for me. If I had to write my own OS and word processor just to write a letter, it'd never happen. The fact that the computer does this for me facilitates my task. I could write the letter by hand, but doing it in a word processor is way better. Especially if I want to print multiple copies of the letter. But for LLMs, my task might be something like "setting up apache is easy, but I've never done it so just tell me how do it so I don't fumble through learning and make it take way longer." The task was setting up Apache. The task was assigned to me, but I didn't really do it. There wasn't necessarily some higher level task that I merely needed Apache for. Apache was the whole task! And I didn't do it! Now, this will not be the case for all LLM-enabled tasks, but I think this distinction speaks to my experience. In the previous word processor example, the LLM would just write my document for me. It doesn't allow me to write my document more efficiently. It's efficient, only in the sense that I no longer need to actually do it myself, except for maybe to act as an editor. (and most people don't even do much of that work) My skill in writing either atrophies or never fully develops since I don't actually need to spend any time doing it or thinking about it. In a perfect world, I use self-discipline to have the LLM show me how to set up Apache, then take notes, and then research, and then set it up manually in subsequent runs; I'd have benefited from learning the task much more quickly than if I'd done it alone, but also used my self-discipline to make sure I actually really learned something and developed expertise as well. My argument is that most people will not succeed in doing this, and will just let the LLM think for them.
agumonkey 1y ago
Most of the time it feels like a crutch to me. There has been a few moments where it unlocked deep motivation (by having a feel for the size of a solution based on chatgpt output) and one time a research project where any crazy idea I threw, it would imagine what it would entail in terms of semantics and then I was inspired even more. The jury is Still out on what value these things will bring
- borroka 1y ago
  I have found them to be very useful. Here are just a few examples of how I have used them. My fountain pen stopped working, so I tried the common solutions recommended, but they did not solve the problem. Claude told me to try using a mixture of window cleaner and water. It worked! (The solution must have been in the corpus used to train Claude.) I switched from W2 to consulting, but I didn't know anything about taxes. ChatGPT gave me the right recommendation, saving me hours of research. I wanted to evaluate the quality of some shirts I have, so I used ChatGPT to estimate stitches per inch and the quality of the buttonholes and other details. I planned my diet using ChatGPT. I had it calculate calories, macros, and deficit. Could I have done it without an LLM? Of course, but it made the planning much faster.
  - agumonkey 1y ago
    hmm true, for personal investigation / learning, chatgpt became a better google especially large, old, convoluted domains where you want to be able to quickly map where things are, it's indeed a massive time saver
rsynnott 1y ago
> Is anyone really cocksure on the basis of LLM received knowledge? Some people certainly seem to be. You see this a lot on webforums; someone spews a lot of confident superficially plausible-looking nonsense, then when someone points out that it is nonsense, they say they got it from a magic robot. I think this is particularly common for non-tech people, who are more likely to believe that the magic robots are actually intelligent.
jayd16 1y ago
Well I think your experience is, if not in the minority, at least not the overwhelming majority. Lots of folks think it's amazing and greatly empowers them.
nathan_compton 1y ago
Yeah, everything I get out of the AI stinks of wrongness, even when its not materially wrong. There is a flimsiness to everything.
bootsmann 1y ago
> Is anyone really cocksure on the basis of LLM received knowledge? I work for a company with an open source product and the number of support requests we get from people who ask the chatbot to do their config and then end up with something nonfunctioning is quite significant. Goes up to users complaining our api is down because the chatbot hallucinated the endpoint.
- threetonesun 1y ago
  LLMs do love to make up endpoints and parameters, but I have found that ones with web access are pretty good at copy/pasting configs if they can find them, so it might be worth a few minutes of exploring what people are actually finding that's causing it to make up an endpoint. I have not (yet!) seen an instance where making something easier for LLMs to parse didn't also help human comprehension.
- tiltowait 1y ago
  I work in DevSecOps, and devs sometimes come to us with AI-slop summaries and writeups about our own tooling. Any time I see emojis in a message, I know I'm about to have a laugh.
doctorpangloss 1y ago
Nothing stops you from spending the paltry $60 to find out how ridiculously good coding agents are. It’s only a matter of time for other problems.
chamomeal 1y ago
If I use LLMs too much I swear I can feel my brain powering down. Same feeling as playing a really grindy/mindless game
collingreen 1y ago
I agree but I've personally seen some egregious examples of people who are not only extremely confident in their new "knowledge" and "ability" but simultaneously think everyone else is extremely stupid. It's been absolutely wild to watch people paste chatgpt output and claim they wrote it, over and over again, even though every time I actually read it and ask a few "what does this mean" questions they have no idea and simply ask chatgpt then confidently say the response. It's so bad it's like a pathology; I wouldn't believe it if I hadn't seen it with my own eyes. Something is happening here. Hopefully it's just revealing something that was already there in society and it isn't something new.

by sho_hn 1y ago

deadbabe 1y ago

If you feel dumber, it’s because you’re using the LLM to do raw work instead of using it for research. It should be a google/stackoverflow replacement, not a really powerful intellisense. You should feel no dumber than using google to investigate questions.

Insanity 1y ago
I don't think this is entirely accurate. If you look at this: https://www.media.mit.edu/publications/your-brain-on-chatgpt/ https://www.media.mit.edu/publications/your-brain-on-chatgpt..., it shows that search engines do engage your brain _more_ than LLM usage. So you'll remember more through search engine use (and crawling the web 'manually') than by just prompting a chatbot.
pessimizer 1y ago
I find that it is terrible for research, and hallucinates 25% to 90% of its references. If you tell it to find something and give it a detailed description of what you're looking for, it will pretend like it has verified that that thing exists, and give you a bulletpoint lecture about why it is such an effective and interesting thing that 1) you didn't ask for, and 2) is really it parroting your description back to you with embellishments. I thought I was going to be able to use LLMs primarily for research, because I have read an enormous number of things (books, papers) in my life, and I can't necessarily find them again when they would be useful. Trying to track them down through LLMs is rarely successful and always agonizing, like pulling teeth that are constantly lying to you. A surprising outcome is that I often get so frustrated by the LLM and so detailed in how I'm complaining about its stupid responses that I remind myself of something that allows me to find the reference on my own. I have to suspect that people who find it useful for research are researching things that are easily discoverable through many other means. Those are not the things that are interesting. I totally find it useful to find something in software docs that I'm too lazy to look up myself, but it's literally saving me 10 minutes.

Insanity 1y ago

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" will slowly fade away and people will blindly trust them.

aabhay 1y ago
There’s also the fact that both Wikipedia and LLMs are non-stationary. The quality of wikipedia has grown immensely since its inception and LLMs will get more accurate (if not explicitly “smarter”)
- derektank 1y ago
  I'm not entirely convinced that the quality of Wikipedia has improved substantially in the last decade.
  - vablings 1y ago
    Care to provide any counter-examples? Please make it know if you end up using Wikipedia for your source of if Wikipedia's quality has changed
    - pessimizer 1y ago
      How in the world would you supply a counter-example for "the quality of Wikipedia has/hasn't improved substantially in the last decade"? I also can't even read the second sentence. I think there are typos there, but there's no mental correction I can do to make it coherent for me.
      - jancsika 1y ago
        I can't think of a better accidental metric than that! I'll go ahead and speculate that the number of incoherent sentences per article has gone down substantially over the last decade, probably due to the relevant tooling getting better over the same period.

freejazz 1y ago

> Is anyone really cocksure on the basis of LLM received knowledge? Yeah, the stupid.

code_for_monkey 1y ago

unfortunately im like you and we are in the minority. The manager class loves the llm and doesnt seem to consider its flaws like that.

lukan 1y ago

Nah, I feel smart to use it in a smart way to get stuff done faster than before.

everdrive 1y ago

This captures my experience quite well. I can "get a lot more done," but it's not really me doing the things, and I feel like a bit of a fraud. And as the workday and the workweek roll on, I find myself needing to force myself to look things up and experiment rather than just asking the LLM. It's quite clear that for most people LLMs will make the more dependent. People with better discipline I think will really benefit in big ways, and you'll see this become a new luxury belief; the disciplined geniuses around us will genuinely be perplexed why people are saying that LLMs have made them less capable, much in the same way they wonder why people can't just limit their drug use recreationally.

Brendinooo 1y ago
>it's not really me doing the things, and I feel like a bit of a fraud I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here? My car has an automatic transmission, am I a fraud because the machine is shifting gears for me? My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually? Spell check caught a word, am I a fraud because I didn't look it up in a dictionary?
- everdrive 1y ago
  I've been thinking about that comparison as well. A common fantasy is that civilization will collapse and the guy who knows how to hunt and start a fire will really excel. In practice, this never happens and he's sort of left behind unless he also has other skills relevant to the modern world. And, for instance, I have barely any knowledge of how my computer works, but it's a tool I use to do my job. (and to have fun at home.) Why are these different than using LLMs? I think at least for me the distinction is whether or not something enables me to perform a task, or whether it's just doing the task for me. If I had to write my own OS and word processor just to write a letter, it'd never happen. The fact that the computer does this for me facilitates my task. I could write the letter by hand, but doing it in a word processor is way better. Especially if I want to print multiple copies of the letter. But for LLMs, my task might be something like "setting up apache is easy, but I've never done it so just tell me how do it so I don't fumble through learning and make it take way longer." The task was setting up Apache. The task was assigned to me, but I didn't really do it. There wasn't necessarily some higher level task that I merely needed Apache for. Apache was the whole task! And I didn't do it! Now, this will not be the case for all LLM-enabled tasks, but I think this distinction speaks to my experience. In the previous word processor example, the LLM would just write my document for me. It doesn't allow me to write my document more efficiently. It's efficient, only in the sense that I no longer need to actually do it myself, except for maybe to act as an editor. (and most people don't even do much of that work) My skill in writing either atrophies or never fully develops since I don't actually need to spend any time doing it or thinking about it. In a perfect world, I use self-discipline to have the LLM show me how to set up Apache, then take notes, and then research, and then set it up manually in subsequent runs; I'd have benefited from learning the task much more quickly than if I'd done it alone, but also used my self-discipline to make sure I actually really learned something and developed expertise as well. My argument is that most people will not succeed in doing this, and will just let the LLM think for them.

agumonkey 1y ago

Most of the time it feels like a crutch to me. There has been a few moments where it unlocked deep motivation (by having a feel for the size of a solution based on chatgpt output) and one time a research project where any crazy idea I threw, it would imagine what it would entail in terms of semantics and then I was inspired even more. The jury is Still out on what value these things will bring

borroka 1y ago
I have found them to be very useful. Here are just a few examples of how I have used them. My fountain pen stopped working, so I tried the common solutions recommended, but they did not solve the problem. Claude told me to try using a mixture of window cleaner and water. It worked! (The solution must have been in the corpus used to train Claude.) I switched from W2 to consulting, but I didn't know anything about taxes. ChatGPT gave me the right recommendation, saving me hours of research. I wanted to evaluate the quality of some shirts I have, so I used ChatGPT to estimate stitches per inch and the quality of the buttonholes and other details. I planned my diet using ChatGPT. I had it calculate calories, macros, and deficit. Could I have done it without an LLM? Of course, but it made the planning much faster.
- agumonkey 1y ago
  hmm true, for personal investigation / learning, chatgpt became a better google especially large, old, convoluted domains where you want to be able to quickly map where things are, it's indeed a massive time saver

rsynnott 1y ago

> Is anyone really cocksure on the basis of LLM received knowledge? Some people certainly seem to be. You see this a lot on webforums; someone spews a lot of confident superficially plausible-looking nonsense, then when someone points out that it is nonsense, they say they got it from a magic robot. I think this is particularly common for non-tech people, who are more likely to believe that the magic robots are actually intelligent.

jayd16 1y ago

Well I think your experience is, if not in the minority, at least not the overwhelming majority. Lots of folks think it's amazing and greatly empowers them.

nathan_compton 1y ago

Yeah, everything I get out of the AI stinks of wrongness, even when its not materially wrong. There is a flimsiness to everything.

bootsmann 1y ago

> Is anyone really cocksure on the basis of LLM received knowledge? I work for a company with an open source product and the number of support requests we get from people who ask the chatbot to do their config and then end up with something nonfunctioning is quite significant. Goes up to users complaining our api is down because the chatbot hallucinated the endpoint.

threetonesun 1y ago
LLMs do love to make up endpoints and parameters, but I have found that ones with web access are pretty good at copy/pasting configs if they can find them, so it might be worth a few minutes of exploring what people are actually finding that's causing it to make up an endpoint. I have not (yet!) seen an instance where making something easier for LLMs to parse didn't also help human comprehension.
tiltowait 1y ago
I work in DevSecOps, and devs sometimes come to us with AI-slop summaries and writeups about our own tooling. Any time I see emojis in a message, I know I'm about to have a laugh.

doctorpangloss 1y ago

Nothing stops you from spending the paltry $60 to find out how ridiculously good coding agents are. It’s only a matter of time for other problems.

chamomeal 1y ago

If I use LLMs too much I swear I can feel my brain powering down. Same feeling as playing a really grindy/mindless game

collingreen 1y ago

I agree but I've personally seen some egregious examples of people who are not only extremely confident in their new "knowledge" and "ability" but simultaneously think everyone else is extremely stupid. It's been absolutely wild to watch people paste chatgpt output and claim they wrote it, over and over again, even though every time I actually read it and ask a few "what does this mean" questions they have no idea and simply ask chatgpt then confidently say the response. It's so bad it's like a pathology; I wouldn't believe it if I hadn't seen it with my own eyes. Something is happening here. Hopefully it's just revealing something that was already there in society and it isn't something new.