5 ms·

There was a lot of stories like "Webb captures it's fist ever picture of an exoplanet" [eg]. My guess is that it's digesting those and not understanding that t

There was a lot of stories like "Webb captures it's fist ever picture of an exoplanet" [eg]. My guess is that it's digesting those and not understanding that the "it's" in that sentence is critical. Here is a prior example of an exoplanet picture: https://esahubble.org/images/heic0821a/ https://esahubble.org/images/heic0821a/ [eg] https://blogs.nasa.gov/webb/2022/09/01/nasas-webb-takes-its-first-ever-direct-image-of-distant-world/ https://blogs.nasa.gov/webb/2022/09/01/nasas-webb-takes-its-...

lucb1e 4y ago
Did you mean "its" such as in <https://news.ycombinator.com/item?id=34359839 https://news.ycombinator.com/item?id=34359839>? Given your statement of this being critical... :) (Advice I also gave at work today: just don't use contractions and the right spelling will usually be obvious. In an informal setting, it's more tempting, but that's the way to easily check yourself.)
- advisedwang 4y ago
  Ha, I initially wrote "its" then got nervous I was wrong, overthought it and did get it wrong.
  - panarky 4y ago
    Maybe it's okay if the AI gets its grammar wrong sometimes, as long as it's less wrong than humans?
- dwringer 4y ago
  I like that this thread points out even humans have difficulty with that construction sometimes. We're trying to hold Google's language model to a higher standard than humans in this case I think. I remember "learning" thousands of bits of trivia like that from people who had misinterpreted something they read and misstated it in such a way. Of course Google has already been putting often-incorrect summaries/factoids in its search infoboxes for a few years now.
  - somenameforme 4y ago
    It's not a matter of "its vs it's" in this case, but the very existence of the word in the sentence: "NASA’s Webb Takes Its First-Ever Direct Image of Distant World" It doesn't matter if one misspells its. You know what it means and it largely defines this sentence. A failure to parse such a relatively simple construct doesn't bode well.
    - dwringer 4y ago
      You're of course correct, but perhaps I should have focused more on the concept of skipping words when reading, misremembering, and "reading what you want to read" - those traits are extremely common if not universal at some level in human readers as well.
  - pohuing 4y ago
    Well yeah, when I make a tool I want it to do its job correctly. If it doesn't I throw it out. If a human keeps messing up I do the same. A human messing up confidently in their interview probably won't even get hired...
    - dwringer 4y ago
      Sure but this seems analogous to creating a claw hammer then showing off how it can be used to drive screws, then saying the hammer isn't doing its job correctly when the screws aren't driven properly. I think chat technology like this is an incredible tool, but I don't think it's being judged fairly: I don't think its usefulness is as some kind of oracle or advisor expected to provide correct or logical information. That seems so orthogonal (if not diametrically opposed) from its actual function that it really feels like we're being trolled by things like "Galactica". But I'm much more (cautiously) optimistic about the potential use for the technology in web search, which has never been logical or "correct" and has always required critical thinking on the part of its users. Perhaps there should be more of a disclaimer that the things it says are not and cannot be construed to be factual, no matter how verisimilitudinous.
jeffbee 4y ago
I can't tell from the tweet why the Bard response is wrong. Is it because some other instrument has taken an image of an exoplanet, or because no instrument has ever done so? ChatGPT seems to believe it is the latter.
- techsupporter 4y ago
  Another instrument took an image of an exoplanet, in 2005. https://exoplanets.nasa.gov/resources/300/2m1207b-first-image-of-an-exoplanet/ https://exoplanets.nasa.gov/resources/300/2m1207b-first-imag...
  - duckmysick 4y ago
    Your LLM needs fine-tuning, the linked article says it's 2004: > 2M1207b is the first exoplanet directly imaged and the first discovered orbiting a brown dwarf. It was imaged the first time by the VLT in 2004.
- denlekke 4y ago
  i'm not a grammar expert but i think as a layperson there is a little bit of ambiguity in how a reader could interpret Bard's response. If i were writing i would probably specify "first ever picture". I think a lot of times words like "the first" are relative and context specific which the Bard answer lacks.
  - shkkmo 4y ago
    There is no ambiguity. We don't call Neil Armstrong "the first ever man on the moon". "The first" has a clear meaning and making any qualification of that implicit is either deceptive or bad writing.
    - noonesays 4y ago
      > We don't call Neil Armstrong "the first ever man on the moon". https://www.scout75.com/apollo-11.html#/ https://www.scout75.com/apollo-11.html#/ > On July 20, 1969 America landed the first ever man on the moon in Neil Armstrong.
est 4y ago
My thoughts exactly. natural language semantics is imperfect and human reasoning is weird. Let's not mistake LLM models as a single source of absolute truth, but a funny & bullshitting assistant who happens to read and vaguely remembers much information.

lucb1e 4y ago

Did you mean "its" such as in <https://news.ycombinator.com/item?id=34359839 https://news.ycombinator.com/item?id=34359839>? Given your statement of this being critical... :) (Advice I also gave at work today: just don't use contractions and the right spelling will usually be obvious. In an informal setting, it's more tempting, but that's the way to easily check yourself.)

advisedwang 4y ago
Ha, I initially wrote "its" then got nervous I was wrong, overthought it and did get it wrong.
- panarky 4y ago
  Maybe it's okay if the AI gets its grammar wrong sometimes, as long as it's less wrong than humans?
dwringer 4y ago
I like that this thread points out even humans have difficulty with that construction sometimes. We're trying to hold Google's language model to a higher standard than humans in this case I think. I remember "learning" thousands of bits of trivia like that from people who had misinterpreted something they read and misstated it in such a way. Of course Google has already been putting often-incorrect summaries/factoids in its search infoboxes for a few years now.
- somenameforme 4y ago
  It's not a matter of "its vs it's" in this case, but the very existence of the word in the sentence: "NASA’s Webb Takes Its First-Ever Direct Image of Distant World" It doesn't matter if one misspells its. You know what it means and it largely defines this sentence. A failure to parse such a relatively simple construct doesn't bode well.
  - dwringer 4y ago
    You're of course correct, but perhaps I should have focused more on the concept of skipping words when reading, misremembering, and "reading what you want to read" - those traits are extremely common if not universal at some level in human readers as well.
- pohuing 4y ago
  Well yeah, when I make a tool I want it to do its job correctly. If it doesn't I throw it out. If a human keeps messing up I do the same. A human messing up confidently in their interview probably won't even get hired...
  - dwringer 4y ago
    Sure but this seems analogous to creating a claw hammer then showing off how it can be used to drive screws, then saying the hammer isn't doing its job correctly when the screws aren't driven properly. I think chat technology like this is an incredible tool, but I don't think it's being judged fairly: I don't think its usefulness is as some kind of oracle or advisor expected to provide correct or logical information. That seems so orthogonal (if not diametrically opposed) from its actual function that it really feels like we're being trolled by things like "Galactica". But I'm much more (cautiously) optimistic about the potential use for the technology in web search, which has never been logical or "correct" and has always required critical thinking on the part of its users. Perhaps there should be more of a disclaimer that the things it says are not and cannot be construed to be factual, no matter how verisimilitudinous.

jeffbee 4y ago

I can't tell from the tweet why the Bard response is wrong. Is it because some other instrument has taken an image of an exoplanet, or because no instrument has ever done so? ChatGPT seems to believe it is the latter.

techsupporter 4y ago
Another instrument took an image of an exoplanet, in 2005. https://exoplanets.nasa.gov/resources/300/2m1207b-first-image-of-an-exoplanet/ https://exoplanets.nasa.gov/resources/300/2m1207b-first-imag...
- duckmysick 4y ago
  Your LLM needs fine-tuning, the linked article says it's 2004: > 2M1207b is the first exoplanet directly imaged and the first discovered orbiting a brown dwarf. It was imaged the first time by the VLT in 2004.
denlekke 4y ago
i'm not a grammar expert but i think as a layperson there is a little bit of ambiguity in how a reader could interpret Bard's response. If i were writing i would probably specify "first ever picture". I think a lot of times words like "the first" are relative and context specific which the Bard answer lacks.
- shkkmo 4y ago
  There is no ambiguity. We don't call Neil Armstrong "the first ever man on the moon". "The first" has a clear meaning and making any qualification of that implicit is either deceptive or bad writing.
  - noonesays 4y ago
    > We don't call Neil Armstrong "the first ever man on the moon". https://www.scout75.com/apollo-11.html#/ https://www.scout75.com/apollo-11.html#/ > On July 20, 1969 America landed the first ever man on the moon in Neil Armstrong.

est 4y ago

My thoughts exactly. natural language semantics is imperfect and human reasoning is weird. Let's not mistake LLM models as a single source of absolute truth, but a funny & bullshitting assistant who happens to read and vaguely remembers much information.