4 ms·

> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. Count me in the camp that doesn't and wishes fewer people did. I just a

> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. Count me in the camp that doesn't and wishes fewer people did. I just asked ChatGPT what it knows about me, and apparently I am known for my work in the field of AI. I've apparently contributed to multiple AI projects. I have a globally-unique name and have done absolutely nothing of the sort. I don't dispute that there are good uses for it right now, but people taking this stuff at face value for anything important is terrifying.

airstrike 2y ago
"You're holding it wrong." You're likely to get more value out of it if you try to do something useful with it rather than asking it to answer a question you know ex ante it cannot know the answer for.
- stouset 2y ago
  I expect it to provide information to me on topics I don’t already know. If it can’t do so accurately on topics I do know, what on earth would make me believe it’s any better on everything else? I had absolutely zero reason to believe it would mischaracterize me in advance of asking that question. In fact I presumed that would be an easy gimme: simply Googling me is enough to provide a pretty accurate assessment even before following any of the links.
  - airstrike 2y ago
    > I expect it to provide information to me on topics I don’t already know. Presumably you already know about yourself, yet that was the example you chose to pick. It comes across as you basically leading the witness just to be able to say say "gotcha! I knew you'd suck". It's not a search engine, although you can ask some models/chatbots to search the web. Again, you're holding it wrong. Instead of trying to confirm your bias that LLMs are bad, try to find discomforming evidence. “The human understanding when it has once adopted an opinion, draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects.” — Francis Bacon
    - stouset 2y ago
      > Presumably you already know about yourself, yet that was the example you chose to pick. It comes across as you basically leading the witness just to be able to say say "gotcha! I knew you'd suck". With all due respect, how the hell else am I supposed to evaluate its ability to answer accurately on topics without testing it first on topics I know about? > Instead of trying to confirm your bias that LLMs are bad, try to find discomforming evidence. I wasn't trying to confirm my bias. I literally thought that with all the information OpenAI has collected throughout the web they can probably give me a pretty good summary of the publicly available information about me. They probably even have stuff that's a bit more obscure or difficult to find. This seemed like an extremely easy way for me to be impressed. I was trying to find disconforming evidence. Yet again I was thoroughly disappointed. If spot-checking AI with softball questions is considered a "gotcha", that is an extremely damning comment on the current state of things.
      - airstrike 2y ago
        But that's not a softball question. You're asking your microwave oven to toast some bread and complaining it comes out hot but not toasted. It's not a toaster, just like an LLM is not meant to be used as a search tool although some integration exists. Ask it some programming question, to draft an email, to create a marketing campaign, to help you dig deeper into a research topic, I don't know.. things an LLM was built for.
        stouset 2y ago
        Summarizing the publicly-available information on a person or topic is what LLMs are supposedly built for.
        joquarky 2y ago
        The summarizing feature of LLMs requires that you provide the input you want to be summarized.
        dragonwriter 2y ago
  - dragonwriter 2y ago
    > I expect it to provide information to me Yeah, that's something language models are exceptionally bad at, and, yeah, ChatGPT is more than an LLM it is a UI around an LLM core that has some other facilities for data retrieval, but, still way out of its area of strength (unfortunately, that seems to be a lot of what people want LLM-centered products like ChatGPT to be.)
lightandlight 2y ago
> I just asked ChatGPT what it knows about me, and apparently I am known for my work in the field of AI. [..] people taking this stuff at face value for anything important is terrifying. Yeah, I'd be more comfortable if it could say "I don't know" instead of generating bullshit. Simon Willison's article Think of language models like ChatGPT as a “calculator for words”[1] helped me think about this topic better: > If you ask an LLM a question, it will answer it—no matter what the question! Using them as an alternative to a search engine such as Google is one of the most obvious applications—and for a lot of queries this works just fine. It’s also going to quickly get you into trouble. All of this is why I only every use LLMs for things that I can verify. [1]: https://simonwillison.net/2023/Apr/2/calculator-for-words/ https://simonwillison.net/2023/Apr/2/calculator-for-words/

airstrike 2y ago

"You're holding it wrong." You're likely to get more value out of it if you try to do something useful with it rather than asking it to answer a question you know ex ante it cannot know the answer for.

stouset 2y ago
I expect it to provide information to me on topics I don’t already know. If it can’t do so accurately on topics I do know, what on earth would make me believe it’s any better on everything else? I had absolutely zero reason to believe it would mischaracterize me in advance of asking that question. In fact I presumed that would be an easy gimme: simply Googling me is enough to provide a pretty accurate assessment even before following any of the links.
- airstrike 2y ago
  > I expect it to provide information to me on topics I don’t already know. Presumably you already know about yourself, yet that was the example you chose to pick. It comes across as you basically leading the witness just to be able to say say "gotcha! I knew you'd suck". It's not a search engine, although you can ask some models/chatbots to search the web. Again, you're holding it wrong. Instead of trying to confirm your bias that LLMs are bad, try to find discomforming evidence. “The human understanding when it has once adopted an opinion, draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects.” — Francis Bacon
  - stouset 2y ago
    > Presumably you already know about yourself, yet that was the example you chose to pick. It comes across as you basically leading the witness just to be able to say say "gotcha! I knew you'd suck". With all due respect, how the hell else am I supposed to evaluate its ability to answer accurately on topics without testing it first on topics I know about? > Instead of trying to confirm your bias that LLMs are bad, try to find discomforming evidence. I wasn't trying to confirm my bias. I literally thought that with all the information OpenAI has collected throughout the web they can probably give me a pretty good summary of the publicly available information about me. They probably even have stuff that's a bit more obscure or difficult to find. This seemed like an extremely easy way for me to be impressed. I was trying to find disconforming evidence. Yet again I was thoroughly disappointed. If spot-checking AI with softball questions is considered a "gotcha", that is an extremely damning comment on the current state of things.
    - airstrike 2y ago
      But that's not a softball question. You're asking your microwave oven to toast some bread and complaining it comes out hot but not toasted. It's not a toaster, just like an LLM is not meant to be used as a search tool although some integration exists. Ask it some programming question, to draft an email, to create a marketing campaign, to help you dig deeper into a research topic, I don't know.. things an LLM was built for.
      - stouset 2y ago
        Summarizing the publicly-available information on a person or topic is what LLMs are supposedly built for.
        joquarky 2y ago
        The summarizing feature of LLMs requires that you provide the input you want to be summarized.
        dragonwriter 2y ago
- dragonwriter 2y ago
  > I expect it to provide information to me Yeah, that's something language models are exceptionally bad at, and, yeah, ChatGPT is more than an LLM it is a UI around an LLM core that has some other facilities for data retrieval, but, still way out of its area of strength (unfortunately, that seems to be a lot of what people want LLM-centered products like ChatGPT to be.)

lightandlight 2y ago

> I just asked ChatGPT what it knows about me, and apparently I am known for my work in the field of AI. [..] people taking this stuff at face value for anything important is terrifying. Yeah, I'd be more comfortable if it could say "I don't know" instead of generating bullshit. Simon Willison's article Think of language models like ChatGPT as a “calculator for words”[1] helped me think about this topic better: > If you ask an LLM a question, it will answer it—no matter what the question! Using them as an alternative to a search engine such as Google is one of the most obvious applications—and for a lot of queries this works just fine. It’s also going to quickly get you into trouble. All of this is why I only every use LLMs for things that I can verify. [1]: https://simonwillison.net/2023/Apr/2/calculator-for-words/ https://simonwillison.net/2023/Apr/2/calculator-for-words/