5 ms·

A friend of mine was trying to tell me how much different Ruby 3 was from Ruby 2, he's never even worked with Ruby, and I said that it's not nearly as big as th

A friend of mine was trying to tell me how much different Ruby 3 was from Ruby 2, he's never even worked with Ruby, and I said that it's not nearly as big as the difference between Python 2 and 3. He sent me this hilariously wrong transcript where the bot tried to claim that Ruby 3 changed the class keyword to something else. Not a single one of the bullet points was anywhere even close to being true.

simonw 3y ago
Do you know which bot he was using? That's the kind of question I would expect to see wild differences between LLMs - ChatGPT 3.5 v.s. ChatGPT 4 for example I would expect to produce very different levels of quality. That said, it's a bad idea to try to use the output of ANY LLM as a primary source in this kind of conversation. Distrust and verify!
- Zambyte 3y ago
  > That said, it's a bad idea to try to use the output of ANY LLM as a primary source in this kind of conversation. Distrust and verify! People used to say the same thing about the web. It seems like people are unfortunately very bad at being skeptical right now.
  - paulryanrogers 3y ago
    People who said the web was untrustworthy were being reductive. IME late 90s high school, they taught us the Internet is like any source, you have to rely on data or at least expert sources and cite them.
    - wolverine876 3y ago
      > IME late 90s high school, they taught us the Internet is like any source, you have to rely on data or at least expert sources and cite them. Do you see people doing that now? I don't. People overwhelmingly believe, repeat, or just blithely fabricate nonsense - the latter really is amazing to see. In a recent HN thread someone confidently wrote a few paragraphs explaining some physiology. That sat up there for maybe a day, and it was easy to believe. Then a researcher in the field explained it, and the first person was very wrong (the researcher was very polite).
  - hnfong 3y ago
    The important point IMHO isn't about being skeptical, but having the ability to verify the claims, or if not, at least having a good heuristic to detect bullshit (eg. by checking whether it contradicts with something you already know). Just being skeptical without some way to allow good information to flow through the filter makes one close-minded.
- CuriouslyC 3y ago
  It helps if you ask the bot to note reputable people in the field and reference original sources for anything significantly outside of common knowledge. That gets you leads for further information and also reduces hallucination somewhat too.
CharlesW 3y ago
I have a friend who also relishes sharing these kinds of antidotes, and seems to have decided that AI-augmented coding is a bridge too far. Personally, I worry that this is a cut-off-your-nose-to-spite-your-face response, and that he'll be unhirable sooner than he expects.
- vinceguidry 3y ago
  I wouldn't have any of these anecdotes if this friend didn't keep pushing horrible 'bot-takes' on me. My issue isn't with what GPT tools are actually useful or not useful for, they're just tools, but with idiots who think typing in a query to save 5 minutes of Googling and actually reading results to get a quick overview of a topic, especially on a topic that I have some familiarity with. He seems to think the bot is smarter and more useful than actual human knowledge. Oh and the other problem I have with it is that you have to wade through endless pages of gpt-trash when you do try to use a search engine to get an answer to a question now.
  - CharlesW 3y ago
    > Oh and the other problem I have with it is that you have to wade through endless pages of gpt-trash when you do try to use a search engine to get an answer to a question now. True, and most people don't realize how bad it's going to get. We're still in the honeymoon period where most content on the internet is human-created, but machine-generated content will be the vast majority of online content by the end of the decade.

simonw 3y ago

Do you know which bot he was using? That's the kind of question I would expect to see wild differences between LLMs - ChatGPT 3.5 v.s. ChatGPT 4 for example I would expect to produce very different levels of quality. That said, it's a bad idea to try to use the output of ANY LLM as a primary source in this kind of conversation. Distrust and verify!

Zambyte 3y ago
> That said, it's a bad idea to try to use the output of ANY LLM as a primary source in this kind of conversation. Distrust and verify! People used to say the same thing about the web. It seems like people are unfortunately very bad at being skeptical right now.
- paulryanrogers 3y ago
  People who said the web was untrustworthy were being reductive. IME late 90s high school, they taught us the Internet is like any source, you have to rely on data or at least expert sources and cite them.
  - wolverine876 3y ago
    > IME late 90s high school, they taught us the Internet is like any source, you have to rely on data or at least expert sources and cite them. Do you see people doing that now? I don't. People overwhelmingly believe, repeat, or just blithely fabricate nonsense - the latter really is amazing to see. In a recent HN thread someone confidently wrote a few paragraphs explaining some physiology. That sat up there for maybe a day, and it was easy to believe. Then a researcher in the field explained it, and the first person was very wrong (the researcher was very polite).
- hnfong 3y ago
  The important point IMHO isn't about being skeptical, but having the ability to verify the claims, or if not, at least having a good heuristic to detect bullshit (eg. by checking whether it contradicts with something you already know). Just being skeptical without some way to allow good information to flow through the filter makes one close-minded.
CuriouslyC 3y ago
It helps if you ask the bot to note reputable people in the field and reference original sources for anything significantly outside of common knowledge. That gets you leads for further information and also reduces hallucination somewhat too.

CharlesW 3y ago

I have a friend who also relishes sharing these kinds of antidotes, and seems to have decided that AI-augmented coding is a bridge too far. Personally, I worry that this is a cut-off-your-nose-to-spite-your-face response, and that he'll be unhirable sooner than he expects.

vinceguidry 3y ago
I wouldn't have any of these anecdotes if this friend didn't keep pushing horrible 'bot-takes' on me. My issue isn't with what GPT tools are actually useful or not useful for, they're just tools, but with idiots who think typing in a query to save 5 minutes of Googling and actually reading results to get a quick overview of a topic, especially on a topic that I have some familiarity with. He seems to think the bot is smarter and more useful than actual human knowledge. Oh and the other problem I have with it is that you have to wade through endless pages of gpt-trash when you do try to use a search engine to get an answer to a question now.
- CharlesW 3y ago
  > Oh and the other problem I have with it is that you have to wade through endless pages of gpt-trash when you do try to use a search engine to get an answer to a question now. True, and most people don't realize how bad it's going to get. We're still in the honeymoon period where most content on the internet is human-created, but machine-generated content will be the vast majority of online content by the end of the decade.