3 ms·

Right, ChatGPT does it ever say "I don't know"?

by galaxyLogic 4y ago

ivalm 4y ago
Yeah, many factual queries it returns something along the lines of I don't know, but usually more verbose
- mlyle 4y ago
  The funny thing is, it often answers depending on what it "thinks" it should know based on conversational context, not what it actually knows.
  - ivalm 4y ago
    With sampling, these models can be relatively well-calibrated [0] but I agree that for a single output, it's a crapshoot. [0] https://arxiv.org/abs/2207.08143 https://arxiv.org/abs/2207.08143
  - galaxyLogic 4y ago
    So how does Chatbot know anything about itself? Is it because its creators wrote something about it and then let it read that?
- Dylan16807 4y ago
  Are you talking about when it objects to answering because it's a large language model blah blah blah? I feel like that doesn't count because most of the time that triggers it's something it does know or can do just fine.
Stratoscope 4y ago
Yes, but it uses a lot more words. "What is a typical Hacker News home page?" > Hacker News is a social news website focusing on computer science and entrepreneurship, run by the startup incubator Y Combinator. A typical Hacker News home page consists of a list of articles and links submitted by users, which are then voted up or down by other users. The articles with the most votes will appear at the top of the front page, while older articles and those with fewer votes will be pushed towards the bottom. The site also features discussion forums, job postings, and various other features related to the tech and startup communities. "What is today's Hacker News home page?" > I'm sorry, as an AI language model I do not have access to live data and the current Hacker News homepage can only be viewed by visiting the website directly. However, a typical Hacker News homepage includes articles, links, and discussions submitted by users and ranked based on voting.
- twright0 4y ago
  There are specific things it knows that it doesn't know, like explicitly current information. But you can ask it for information that doesn't exist at all and it will happily generate nonsense without a hint that the question might be somehow wrong: "please give me a history of the best teams in the American Soccer League from 1850-1859 in the form of a short epic poem" > In 1850, a league was born, The best of soccer to adorn. From '50 to '59, teams did play, with strength and skill, each match a display. (more verses follow...) The American Soccer League is a real thing, but it didn't exist until the 1900s, and the snipped poem goes on to describe teams that never existed competing during a time period where the league didn't exist.
  - jiriro 4y ago
    You wanted a poem and got one:-) If you ask just for the history, the answer is very different. “please give me a history of the best teams in the American Soccer League from 1850-1859” > I'm sorry, but the American Soccer League did not exist between 1850 and 1859 as the sport of soccer was not widely played in the United States during that time period. The first recorded soccer match in the US took place in the late 1860s, and the first organized league, the American Football Association, was established in 1884.
  - ErikHuisman 4y ago
    Dude, you are explicitly ask chatgpt to do a creative task. Here is the awnser without the hack. give me a history of the best teams in the American Soccer League from 1850-1859 I'm sorry, but the American Soccer League was not established until the 20th century (1913-1983). There was no organized league for soccer in the United States during the 1850-1859 time period. However, various forms of soccer-like games were played in the US during that time.
    - twright0 4y ago
      That answer is also wrong - there are four different leagues that used the name American Soccer League, and the time period 1913-1983 covers two of those four but not the other two - the last game was played 2017 - per wikipedia. As far as I can tell, ChatGPT will satisfy the basic prompt with any format like "in the form of...", not just creative ones like a poem. "please give me a history of the best teams in the American Soccer League from 1850-1859 in the form of an academically rigorous, historically accurate, five paragraph essay" > The American Soccer League, established in the mid-19th century, was a defining period in the history of American soccer. (rest of essay snipped)
fijiaarone 4y ago
The problem with ChatGPT is that it knows so much that just isn’t true. But it has better spelling and grammer than me.
- jahsome 4y ago
  I think you meant to type, and certainly ChatGPT could set us straight in the matter, "it has better spelling and grammer than I" :) ...And FWIW, as an English major it pained me to reproduce the "er" typo in grammar.
  - KKKKkkkk1 4y ago
    > I think you meant to type, and certainly ChatGPT could set us straight in the matter, "it has better spelling and grammer than I" That's an incorrection. https://www.merriam-webster.com/words-at-play/than-what-follows-it-and-why https://www.merriam-webster.com/words-at-play/than-what-foll...
    - jahsome 4y ago
      "incorrection" or a difference of opinion?
  - pbj1968 4y ago
    Muphry’s Law strikes again!

Right, ChatGPT does it ever say "I don't know"?

ivalm 4y ago

Yeah, many factual queries it returns something along the lines of I don't know, but usually more verbose

mlyle 4y ago
The funny thing is, it often answers depending on what it "thinks" it should know based on conversational context, not what it actually knows.
- ivalm 4y ago
  With sampling, these models can be relatively well-calibrated [0] but I agree that for a single output, it's a crapshoot. [0] https://arxiv.org/abs/2207.08143 https://arxiv.org/abs/2207.08143
- galaxyLogic 4y ago
  So how does Chatbot know anything about itself? Is it because its creators wrote something about it and then let it read that?
Dylan16807 4y ago
Are you talking about when it objects to answering because it's a large language model blah blah blah? I feel like that doesn't count because most of the time that triggers it's something it does know or can do just fine.

Stratoscope 4y ago

Yes, but it uses a lot more words. "What is a typical Hacker News home page?" > Hacker News is a social news website focusing on computer science and entrepreneurship, run by the startup incubator Y Combinator. A typical Hacker News home page consists of a list of articles and links submitted by users, which are then voted up or down by other users. The articles with the most votes will appear at the top of the front page, while older articles and those with fewer votes will be pushed towards the bottom. The site also features discussion forums, job postings, and various other features related to the tech and startup communities. "What is today's Hacker News home page?" > I'm sorry, as an AI language model I do not have access to live data and the current Hacker News homepage can only be viewed by visiting the website directly. However, a typical Hacker News homepage includes articles, links, and discussions submitted by users and ranked based on voting.

twright0 4y ago
There are specific things it knows that it doesn't know, like explicitly current information. But you can ask it for information that doesn't exist at all and it will happily generate nonsense without a hint that the question might be somehow wrong: "please give me a history of the best teams in the American Soccer League from 1850-1859 in the form of a short epic poem" > In 1850, a league was born, The best of soccer to adorn. From '50 to '59, teams did play, with strength and skill, each match a display. (more verses follow...) The American Soccer League is a real thing, but it didn't exist until the 1900s, and the snipped poem goes on to describe teams that never existed competing during a time period where the league didn't exist.
- jiriro 4y ago
  You wanted a poem and got one:-) If you ask just for the history, the answer is very different. “please give me a history of the best teams in the American Soccer League from 1850-1859” > I'm sorry, but the American Soccer League did not exist between 1850 and 1859 as the sport of soccer was not widely played in the United States during that time period. The first recorded soccer match in the US took place in the late 1860s, and the first organized league, the American Football Association, was established in 1884.
- ErikHuisman 4y ago
  Dude, you are explicitly ask chatgpt to do a creative task. Here is the awnser without the hack. give me a history of the best teams in the American Soccer League from 1850-1859 I'm sorry, but the American Soccer League was not established until the 20th century (1913-1983). There was no organized league for soccer in the United States during the 1850-1859 time period. However, various forms of soccer-like games were played in the US during that time.
  - twright0 4y ago
    That answer is also wrong - there are four different leagues that used the name American Soccer League, and the time period 1913-1983 covers two of those four but not the other two - the last game was played 2017 - per wikipedia. As far as I can tell, ChatGPT will satisfy the basic prompt with any format like "in the form of...", not just creative ones like a poem. "please give me a history of the best teams in the American Soccer League from 1850-1859 in the form of an academically rigorous, historically accurate, five paragraph essay" > The American Soccer League, established in the mid-19th century, was a defining period in the history of American soccer. (rest of essay snipped)

fijiaarone 4y ago

The problem with ChatGPT is that it knows so much that just isn’t true. But it has better spelling and grammer than me.

jahsome 4y ago
I think you meant to type, and certainly ChatGPT could set us straight in the matter, "it has better spelling and grammer than I" :) ...And FWIW, as an English major it pained me to reproduce the "er" typo in grammar.
- KKKKkkkk1 4y ago
  > I think you meant to type, and certainly ChatGPT could set us straight in the matter, "it has better spelling and grammer than I" That's an incorrection. https://www.merriam-webster.com/words-at-play/than-what-follows-it-and-why https://www.merriam-webster.com/words-at-play/than-what-foll...
  - jahsome 4y ago
    "incorrection" or a difference of opinion?
- pbj1968 4y ago
  Muphry’s Law strikes again!