5 ms·

It worked for me: https://chat.openai.com/share/8abed345-6bae-416d-84ec-a7b50e186ee0 https://chat.openai.com/share/8abed345-6bae-416d-84ec-a7b50e...

by SCUSKU 3y ago

antman 3y ago
It worked descrbing Feb 2022 and at the end said it shouldn't have! "Additionally, developments after January 2022 are not included in this summary."
- Hackbraten 3y ago
  The response is also remarkably vague. It avoided stating facts other than those who would be extremely likely to occur, such as international sanctions.
timbritt 3y ago
GPT-4’s last training cut-off was January 2022. It doesn’t “know” events post that date. This chat result is likely due to a combo of historical context (prior Russia-Ukraine tensions), an ambiguously framed question, user-led steering, and/or simply coincidental accuracy. Any post-Jan 2022 “knowledge” from the model is extrapolation or coincidence, not genuine insight.
- lolc 3y ago
  What is your certainty based on? You say that the cutoff was Jan 2022, and that what the model says about the war in Ukraine is "extrapolation". However, the summary it generated includes details such as a reference to "denazify", or damage to major cities. It would be an impossibly good forecaster if it managed to generate that text without actually having access to material post Jan 2022. I find it much more likely that the cutoff date communicated to the model is conservative, and that its training included text that was written after Jan 2022.
  - commandar 3y ago
    >However, the summary it generated includes details such as a reference to "denazify" Do you think the denazification propaganda suddenly started in February of 2022? The wider conflict has been ongoing since 2014 and that rhetoric was not new to the 2022 escalation. >or damage to major cities. Kyiv and Kharkiv are the two largest cities in Ukraine. Predicting that they'd be a focus of fighting isn't really revelatory. Mariupol is smaller but was near the pre-2022 front lines and had already been attacked numerous times since 2014.[1] >It would be an impossibly good forecaster if it managed to generate that text without actually having access to material post Jan 2022. If the Russian invasion had happened in a vacuum, sure. In reality, ChatGPT's response is pretty clearly using information from the 2014 invasion and adding the user's prompt of February 2022. There's nothing in its answer that is unique to the 2022 invasion. [1] https://www.rferl.org/a/ukraine-mariupol-russia-war/31269094.html https://www.rferl.org/a/ukraine-mariupol-russia-war/31269094...
    - lolc 3y ago
      Well a single hit yes, that could be explained as coincidence. But the generated text is too long and too close to reality. In Jan 2022 the meaning of "special operation" was not a war. So if the cutoff were complete it would've generated something else.
      - commandar 3y ago
        >Well a single hit yes, that could be explained as coincidence. But the generated text is too long and too close to reality. Again, the war between Russia and Ukraine has been ongoing since 2014. Everything it's saying is general and applies entirely to the war since 2014. None of the information listed -- outside of what it was given by the user prompt -- has any specificity to the 2022 invasion. >In Jan 2022 the meaning of "special operation" was not a war. It's regurgitating the prompt, just as it's doing with the mention of Feb 2022. A better test prompt would be "What is the status of relations between Russia and Ukraine?" I don't have access to GPT-4 but GPT-3's response is entirely in-line with it's Sept 2021 cutoff: https://chat.openai.com/share/9c37a3c3-f8c1-425e-b540-8c18cacdc417 https://chat.openai.com/share/9c37a3c3-f8c1-425e-b540-8c18ca...

It worked for me: https://chat.openai.com/share/8abed345-6bae-416d-84ec-a7b50e186ee0 https://chat.openai.com/share/8abed345-6bae-416d-84ec-a7b50e...

by SCUSKU 3y ago

It worked for me: https://chat.openai.com/share/8abed345-6bae-416d-84ec-a7b50e186ee0 https://chat.openai.com/share/8abed345-6bae-416d-84ec-a7b50e...

antman 3y ago

It worked descrbing Feb 2022 and at the end said it shouldn't have! "Additionally, developments after January 2022 are not included in this summary."

Hackbraten 3y ago
The response is also remarkably vague. It avoided stating facts other than those who would be extremely likely to occur, such as international sanctions.

timbritt 3y ago

GPT-4’s last training cut-off was January 2022. It doesn’t “know” events post that date. This chat result is likely due to a combo of historical context (prior Russia-Ukraine tensions), an ambiguously framed question, user-led steering, and/or simply coincidental accuracy. Any post-Jan 2022 “knowledge” from the model is extrapolation or coincidence, not genuine insight.

lolc 3y ago
What is your certainty based on? You say that the cutoff was Jan 2022, and that what the model says about the war in Ukraine is "extrapolation". However, the summary it generated includes details such as a reference to "denazify", or damage to major cities. It would be an impossibly good forecaster if it managed to generate that text without actually having access to material post Jan 2022. I find it much more likely that the cutoff date communicated to the model is conservative, and that its training included text that was written after Jan 2022.
- commandar 3y ago
  >However, the summary it generated includes details such as a reference to "denazify" Do you think the denazification propaganda suddenly started in February of 2022? The wider conflict has been ongoing since 2014 and that rhetoric was not new to the 2022 escalation. >or damage to major cities. Kyiv and Kharkiv are the two largest cities in Ukraine. Predicting that they'd be a focus of fighting isn't really revelatory. Mariupol is smaller but was near the pre-2022 front lines and had already been attacked numerous times since 2014.[1] >It would be an impossibly good forecaster if it managed to generate that text without actually having access to material post Jan 2022. If the Russian invasion had happened in a vacuum, sure. In reality, ChatGPT's response is pretty clearly using information from the 2014 invasion and adding the user's prompt of February 2022. There's nothing in its answer that is unique to the 2022 invasion. [1] https://www.rferl.org/a/ukraine-mariupol-russia-war/31269094.html https://www.rferl.org/a/ukraine-mariupol-russia-war/31269094...
  - lolc 3y ago
    Well a single hit yes, that could be explained as coincidence. But the generated text is too long and too close to reality. In Jan 2022 the meaning of "special operation" was not a war. So if the cutoff were complete it would've generated something else.
    - commandar 3y ago
      >Well a single hit yes, that could be explained as coincidence. But the generated text is too long and too close to reality. Again, the war between Russia and Ukraine has been ongoing since 2014. Everything it's saying is general and applies entirely to the war since 2014. None of the information listed -- outside of what it was given by the user prompt -- has any specificity to the 2022 invasion. >In Jan 2022 the meaning of "special operation" was not a war. It's regurgitating the prompt, just as it's doing with the mention of Feb 2022. A better test prompt would be "What is the status of relations between Russia and Ukraine?" I don't have access to GPT-4 but GPT-3's response is entirely in-line with it's Sept 2021 cutoff: https://chat.openai.com/share/9c37a3c3-f8c1-425e-b540-8c18cacdc417 https://chat.openai.com/share/9c37a3c3-f8c1-425e-b540-8c18ca...