6 ms·

I tried the two US presidents having the same parents one, and while it understood the intent, it got caught up in being adamant that Joe Biden won the election

by 9rx 1y ago

I tried the two US presidents having the same parents one, and while it understood the intent, it got caught up in being adamant that Joe Biden won the election in 2024 and anything I do to try and tell it otherwise is dismissed as being false and expresses quite definitely that I need to do proper research with legitimate sources.

tankenmate 1y ago
chat log please?
- 9rx 1y ago
  https://dpaste.org/zOev0 https://dpaste.org/zOev0
  - rafaelmn 1y ago
    Is the knowledge cutoff for this thing so stale or is this just bad performance on recent data ?
    - 9rx 1y ago
      It is painful to read, I know, but if you make it towards the end it admits that its knowledge cutoff was prior to the election and that it doesn't know who won. Yet, even then, it still remains adamant that Biden won.
    - dragonwriter 1y ago
      The knowledge cutoff is before the 2024 election (which was, after all, just 9 months ago), June 2024 (I believe this is consistent with the current versions of GPT-4o and -4.1), after Biden had secured the nomination. It is very clear in that chat logs (which include reasoning traces) that the model knew that, knew what the last election it knew about was, and answered correctly based on its cut off initially. Under pressure to answer about an election that was not within its knowledge window it then confabulated a Biden 2024 victory, which it dug in on after being contradicted with a claim that, based on the truth at the time of its knowledge cutoff, was unambiguously false ("Joe Biden did not run") He, in fact, did run for reelection, but withdrew after having secured enough delegates to win the nomination by a wide margin on July 21. Confabulation (called "hallucination" in AI circles, but it is more like human confabulation than hallucination) when pressed for answers on questions for which it lacks grounding remains an unsolved AI problem. Unsolved, but mitigated by providing it grounding independent of its knowledge cutoff, e.g., by tools like web browsing (which GPT-OSS is specifically trained for, but that training does no good if its not hooked into a framework which provides it the tools.)
      - jlaternman 1y ago
        I like that term much better, confabulation. I’ve come to think of it as it relies on an inherent trust in the fact that whatever process it uses to produce a coherent response (which I don’t think the LLM can really analyze after the fact) is inherently a truth-making process, since it trusts inherently its training data and considers that the basis of all its responses. Something along those lines. We might do something similar at times as humans, it feels similar to how some people get trapped in lies and almost equate what they have said as true with having the quality of truth as a result of them having claimed it as true (pathological liars can demonstrate this kind of thinking).
        9rx 1y ago
        > since it trusts inherently its training data and considers that the basis of all its responses. Doesn't that make "hallucination" the better term? The LLM is "seeing" something in the data that isn't actually reflected in reality. Whereas "confabulation" would imply that LLMs are creating data out of "thin air", which leaves the training data to be immaterial. Both words, as they have been historically used, need to be stretched really far to fit an artificial creation that bears no resemblance to what those words were used to describe, so, I mean, any word is as good as any other at that point, but "hallucination" requires less stretching. So I am curious about why you like "confabulation" much better. Perhaps it simply has a better ring to your ear? But, either way, these pained human analogies have grown tired. It is time to call it what it really is: Snorfleblat.
  - lucumo 1y ago
    I've never found the Socratic method to work well on any model I've tried it with. They always seem to get stuck justifying their previous answers. We expect them to answer the question and re-reason the original question with the new information, because that's what a human would do. Maybe next time I'll try to be explicit about that expectation when I try the Socratic method.
  - FergusArgyll 1y ago
    incredible
freehorse 1y ago
I mean I would hardly blame the specific model, Anthropic has a specific mention in their system prompts on trump winning. For some reason llms get confused with this one.
- deleted 1y ago
  [deleted]
- jari_mustonen 1y ago
  It's the political bias in the training material. No surprise there.
  - regularfry 1y ago
    More likely is that there's a lot of source material having to very stridently assert that Trump didn't win in 2020, and it's generalising to a later year. That's not political bias.
    - ben_w 1y ago
      It's also extremely weird that Trump did win in 2024. If I'd been in a coma from Jan 1 2024 to today, and woke up to people saying Trump was president again, I'd think they were pulling my leg or testing my brain function to see if I'd become gullible.
      - cpursley 1y ago
        You’re in a bubble. It was no surprise to folks who touch grass on the regular.
        ben_w 1y ago
        > You’re in a bubble. Sure, all I have to go on from the other side of the Atlantic is the internet. So in that regard, kinda like the AI. One of the big surprises from the POV of me in Jan 2024, is that I would have anticipated Trump being in prison and not even available as an option for the Republican party to select as a candidate for office, and that even if he had not gone to jail that the Republicans would not want someone who behaved as he did on Jan 6 2021.
mark_l_watson 1y ago
I think the lesson is: smaller models hallucinate more, so only use them in your applications where you load up large prompts with specific data to reason about. Then even the small Google gemma3n 4B model can be amazingly useful. I use the SOTA models from Google and OpenAI mostly for getting feedback on ideas, helping me think through designs, and sometimes for coding. Your question is clearly best answered using a large commercial model with a web search tool. That said, integrating a local model with a home built interface to something like the Brave search API can be effective but I no longer make the effort.
- 9rx 1y ago
  > think the lesson is: smaller models hallucinate more The interesting part isn't the hallucination, but the sheer unwillingness to take in new information.
  - dragonwriter 1y ago
    Might have dug in less on the confabulation about information outside of its knowledge cutoff if the new information weren't offered with support from a user “hallucination” about information within its knowledge cutoff. More detail: https://news.ycombinator.com/item?id=44809145 https://news.ycombinator.com/item?id=44809145
    - 9rx 1y ago
      > Might have dug in less... The digging in at all is what is interesting. Like an earlier comment alluded to, the presumptive value of these tools is being able to feed it your own information where that information is to be considered authoritative. > More detail: [...] "He, in fact, did run for reelection" A slow walk, maybe. He was in no condition to run. That is why he ultimately dropped out. But, really, that statement is just a silly game of semantics. "Run", when used in hindsight, often implies completion. This LLM model even says so too. If a model doesn't understand nuance, that is also interesting.
aaroninsf 1y ago
Have we considered the possibility that maybe it knows something we don't.

by 9rx 1y ago

tankenmate 1y ago

chat log please?

9rx 1y ago
https://dpaste.org/zOev0 https://dpaste.org/zOev0
- rafaelmn 1y ago
  Is the knowledge cutoff for this thing so stale or is this just bad performance on recent data ?
  - 9rx 1y ago
    It is painful to read, I know, but if you make it towards the end it admits that its knowledge cutoff was prior to the election and that it doesn't know who won. Yet, even then, it still remains adamant that Biden won.
  - dragonwriter 1y ago
    The knowledge cutoff is before the 2024 election (which was, after all, just 9 months ago), June 2024 (I believe this is consistent with the current versions of GPT-4o and -4.1), after Biden had secured the nomination. It is very clear in that chat logs (which include reasoning traces) that the model knew that, knew what the last election it knew about was, and answered correctly based on its cut off initially. Under pressure to answer about an election that was not within its knowledge window it then confabulated a Biden 2024 victory, which it dug in on after being contradicted with a claim that, based on the truth at the time of its knowledge cutoff, was unambiguously false ("Joe Biden did not run") He, in fact, did run for reelection, but withdrew after having secured enough delegates to win the nomination by a wide margin on July 21. Confabulation (called "hallucination" in AI circles, but it is more like human confabulation than hallucination) when pressed for answers on questions for which it lacks grounding remains an unsolved AI problem. Unsolved, but mitigated by providing it grounding independent of its knowledge cutoff, e.g., by tools like web browsing (which GPT-OSS is specifically trained for, but that training does no good if its not hooked into a framework which provides it the tools.)
    - jlaternman 1y ago
      I like that term much better, confabulation. I’ve come to think of it as it relies on an inherent trust in the fact that whatever process it uses to produce a coherent response (which I don’t think the LLM can really analyze after the fact) is inherently a truth-making process, since it trusts inherently its training data and considers that the basis of all its responses. Something along those lines. We might do something similar at times as humans, it feels similar to how some people get trapped in lies and almost equate what they have said as true with having the quality of truth as a result of them having claimed it as true (pathological liars can demonstrate this kind of thinking).
      - 9rx 1y ago
        > since it trusts inherently its training data and considers that the basis of all its responses. Doesn't that make "hallucination" the better term? The LLM is "seeing" something in the data that isn't actually reflected in reality. Whereas "confabulation" would imply that LLMs are creating data out of "thin air", which leaves the training data to be immaterial. Both words, as they have been historically used, need to be stretched really far to fit an artificial creation that bears no resemblance to what those words were used to describe, so, I mean, any word is as good as any other at that point, but "hallucination" requires less stretching. So I am curious about why you like "confabulation" much better. Perhaps it simply has a better ring to your ear? But, either way, these pained human analogies have grown tired. It is time to call it what it really is: Snorfleblat.
- lucumo 1y ago
  I've never found the Socratic method to work well on any model I've tried it with. They always seem to get stuck justifying their previous answers. We expect them to answer the question and re-reason the original question with the new information, because that's what a human would do. Maybe next time I'll try to be explicit about that expectation when I try the Socratic method.
- FergusArgyll 1y ago
  incredible

freehorse 1y ago

I mean I would hardly blame the specific model, Anthropic has a specific mention in their system prompts on trump winning. For some reason llms get confused with this one.

deleted 1y ago
[deleted]
jari_mustonen 1y ago
It's the political bias in the training material. No surprise there.
- regularfry 1y ago
  More likely is that there's a lot of source material having to very stridently assert that Trump didn't win in 2020, and it's generalising to a later year. That's not political bias.
  - ben_w 1y ago
    It's also extremely weird that Trump did win in 2024. If I'd been in a coma from Jan 1 2024 to today, and woke up to people saying Trump was president again, I'd think they were pulling my leg or testing my brain function to see if I'd become gullible.
    - cpursley 1y ago
      You’re in a bubble. It was no surprise to folks who touch grass on the regular.
      - ben_w 1y ago
        > You’re in a bubble. Sure, all I have to go on from the other side of the Atlantic is the internet. So in that regard, kinda like the AI. One of the big surprises from the POV of me in Jan 2024, is that I would have anticipated Trump being in prison and not even available as an option for the Republican party to select as a candidate for office, and that even if he had not gone to jail that the Republicans would not want someone who behaved as he did on Jan 6 2021.

mark_l_watson 1y ago

I think the lesson is: smaller models hallucinate more, so only use them in your applications where you load up large prompts with specific data to reason about. Then even the small Google gemma3n 4B model can be amazingly useful. I use the SOTA models from Google and OpenAI mostly for getting feedback on ideas, helping me think through designs, and sometimes for coding. Your question is clearly best answered using a large commercial model with a web search tool. That said, integrating a local model with a home built interface to something like the Brave search API can be effective but I no longer make the effort.

9rx 1y ago
> think the lesson is: smaller models hallucinate more The interesting part isn't the hallucination, but the sheer unwillingness to take in new information.
- dragonwriter 1y ago
  Might have dug in less on the confabulation about information outside of its knowledge cutoff if the new information weren't offered with support from a user “hallucination” about information within its knowledge cutoff. More detail: https://news.ycombinator.com/item?id=44809145 https://news.ycombinator.com/item?id=44809145
  - 9rx 1y ago
    > Might have dug in less... The digging in at all is what is interesting. Like an earlier comment alluded to, the presumptive value of these tools is being able to feed it your own information where that information is to be considered authoritative. > More detail: [...] "He, in fact, did run for reelection" A slow walk, maybe. He was in no condition to run. That is why he ultimately dropped out. But, really, that statement is just a silly game of semantics. "Run", when used in hindsight, often implies completion. This LLM model even says so too. If a model doesn't understand nuance, that is also interesting.

aaroninsf 1y ago

Have we considered the possibility that maybe it knows something we don't.