13 ms·
Tell HN: ChatGPT cut off date now Jan 2022
I just noticed the training cut-off date that ChatGPT usually mentions in its response is no longer September 2021 but is now January 2022.
- speedgoose 3y agoI wonder why January 2022. Perhaps some kind of event happening in February 2022.
- deleted 3y ago[deleted]
- e1g 3y agoIndeed - I asked ChatGPT to tell me about topics from December 2021 and it knew about some of them (with the usual dose of hallucinations) https://en.wikipedia.org/wiki/Portal:Current_events/December_2021 https://en.wikipedia.org/wiki/Portal:Current_events/December...
- jerrygoyal 3y agoQuestion: What was the most devastating event in January 2022? GPT3: I'm sorry, but I cannot provide information on events that occurred after my last knowledge update in September 2021. To find information on the most devastating event in January 2022, I recommend checking the latest news sources or conducting an online search for the specific event you are interested in. GPT4: As of my last update in September 2021, I cannot provide real-time or the most recent information. However, one of the most notable events in January 2022 was the eruption of the Hunga Tonga-Hunga Haʻapai volcano in Tonga. The eruption caused widespread damage, generated a significant tsunami that affected the surrounding areas including Tonga, Fiji, New Zealand, and Australia, and severely disrupted communication networks. It was a catastrophic event with serious implications for the people of Tonga and its environment. For the most accurate and up-to-date information, please refer to reliable and up-to-date resources or news outlets. Keep in mind that the situation might have evolved, and new developments might have occurred since January 2022.
- muzani 3y agoChatGPT: "Going outside without a mask can pose a risk, especially in crowded or indoor settings, as it increases the chance of inhaling or spreading respiratory droplets that may contain the virus. The level of danger can vary based on factors like community spread, vaccination rates, and individual health conditions. It's generally recommended to follow local health guidelines and wear masks in situations where it's advised for safety." It didn't even say which virus, it's simply "the virus" at this point.
- arcticpeanuts 3y agoWhat are you trying to prove? ChatGPT: "The World Health Organization (WHO) declared COVID-19 a pandemic on March 11, 2020."
- youraimarketer 3y agoI asked the same question to two different ChatGPT accounts: "What was the most devastating event in January 2022?" The first one is my personal ChatGPT account. On the other hand, the second SS is from my company account. While the first one acknowledges a knowledge cutoff date of January 2022, the second one specifies its training cutoff as September 2021 yet still provides answers to the question. https://x.com/youraimarketer/status/1703997050419867662?s=20 https://x.com/youraimarketer/status/1703997050419867662?s=20
- solardev 3y agoI don't get it. What's the correct answer?
- Giorgi 3y ago[flagged]
- vunderba 3y agoPlease don't post reddit-level responses here. We're better than this.
- perryizgr8 3y agoBing says: In January 2022, there were several significant events: Wildfires in Boulder, Colorado: These fires led to the evacuation of over 30,000 people and the destruction of homes across Boulder County1. COVID-19 surge in the U.S.: The U.S. reached a record number of COVID-19 cases, with the Omicron variant making up 95% of the cases1. Hunga Tonga-Hunga Ha’apai volcano eruption: This eruption sent tsunami waves around the world. The blast was so loud it was heard in Alaska – roughly 6,000 miles away. The afternoon sky turned pitch black as heavy ash clouded Tonga’s capital and caused “significant damage” along the western coast of the main island of Tongatapu2. These events had a profound impact on people’s lives and the environment.
- jwpapi 3y agolol did the same thing here is my convo: your latest update is jan 22 now not sep 21 anyomre? ChatGPT My apologies for the confusion. My training data includes information up to January 2022. I do not have updates beyond that date. User tell me some major global events that happened in january 2022 or dec 21 ChatGPT Sure, here are some significant global events from December 2021 and January 2022: December 2021: Omicron Variant Surge: The Omicron variant of the COVID-19 virus spread rapidly around the world, causing new travel restrictions and lockdowns in some areas. US Tornado Outbreak: A devastating series of tornadoes swept through multiple U.S. states, causing widespread damage and loss of life. Kazakhstan Protests: Protests broke out in Kazakhstan, initially over rising fuel prices, leading to unrest and a strong government response. Ghislaine Maxwell Convicted: Ghislaine Maxwell was convicted on multiple charges related to the sexual exploitation and abuse of young girls by Jeffrey Epstein. Hong Kong Elections: Hong Kong held its first legislative council elections under new rules imposed by Beijing, which led to a pro-Beijing majority win. January 2022: Australia Bushfires: Bushfires raged in Australia, particularly in Western Australia, causing evacuations and damage to property. US Capitol Riot Anniversary: The U.S. marked the one-year anniversary of the January 6, 2021, Capitol riot. Several events and discussions took place concerning the state of democracy in the U.S. Tensions on Ukraine-Russia Border: Tensions escalated between Russia and Ukraine with increased Russian military presence near the border, prompting concerns from the international community. China and Taiwan Tensions: Increased military activities near Taiwan raised concerns about China's intentions, prompting the U.S. and allies to issue warnings. Winter Olympics Preparations: With the Winter Olympics in Beijing set for February 2022, preparations were underway amid controversies such as diplomatic boycotts by some countries due to China's human rights record. Keep in mind that this is not an exhaustive list, but it should give you a sense of the major happenings.
- maest 3y agoFyi, asking the model about it's cutoff date is not a reliable way of finding out what it has been trained on. The model doesn't know that, unless it was provided that info during fine-tuning or something -- but even then it doesn't guarantee anything. The more useful thing to do is ask it about events which happened during a particular time range. That way, you'll know it has at least seen the news.
- retrofuturism 3y agoInformation about cut-off date is very much part of its fine-tuning.
- kesor 3y agoApparently it is also part of its system prompt, since otherwise it wouldn't know what the cutoff date is just by feeding it fresher information - it has to be told the date explicitely somewhere.
- haltist 3y agoIt's possible the date is hallucinated. There is no reason that a combination of system prompt and regular prompt when combined can not generate a hallucinated cut off date that does not match the actual date. LLMs are statistical models and simply generate probable sequences of tokens based on a context (very much like sampling from Markov chains) so there is no a priori reason to believe that the cut off date is accurate. More generally, all output from the model that seems to be model metadata should be assumed to be a hallucination.
- flangola7 3y agoWhen it can be repeated dozens of times consistently that is strong reason to believe it is part of the system prompt. Baseless hallucinations will be different everytime.
- 3y ago
- jazzyjackson 3y agoi'm sure this is the price discriminator of the future: free superintelligence up til 6 months ago, $2000k/month/user for superreasoning about current events
- HPMOR 3y agoIt will be wayyyy more expensive for current events. Matt Levine's piece on positional goods is relevant here. Or just positional goods in general.
- hackernewds 3y agowhy would it be significantly expensive?
- jazzyjackson 3y agothe better the superintelligence, the more significant the advantage. Whoever is offering the advantage can charge whatever they want, it will be worth it.
- michaelt 3y agoThe word "superintelligence" is doing a lot of heavy lifting in these assumptions. But if there was a model that could make better-than-human-with-spreadsheet predictions about the moves of the stock market, that could make a lot of money for its users, so you could charge a lot for it. Unless, of course, you had competitors with an equally good model who were giving it away for free.
- pixl97 3y agoFor example most stock market tickers you see online are delayed by at least 5 minutes or more. Realtime streams are far more expensive.
- danbruc 3y agoThis seems unlikely. The current model of explicitly training AIs will eventually be superseded by AIs that train themselves by reading and watching stuff just like humans do. This of course also depends on what you mean with future, I assumed not the near future.
- wkat4242 3y agoIt's weird, when I use it via the API it mentions July 2023 and it does in fact seem to know about recent events.
- lucb1e 3y agoIn what context? Something you can check that actually happened and was not predictable? Edit: for example by using big events such as the method mentioned in this comment https://news.ycombinator.com/item?id=37565484 https://news.ycombinator.com/item?id=37565484
- Fnoord 3y agoYes, I suspected before the API uses a different version than chat.openai.com
- skilled 3y ago> when did lebron james reach 36,000 career points > As of my last update in January 2022, LeBron James had not reached 36,000 career points in the NBA. However, I don't have data on games or events beyond that date. You may want to consult the latest NBA statistics or relevant websites for up-to-date information on LeBron James' career point total. https://edition.cnn.com/2021/12/29/sport/lebron-james-steph-curry-reach-nba-milestones-spt-intl/index.html https://edition.cnn.com/2021/12/29/sport/lebron-james-steph-...
- thih9 3y agoBased on the article you’ve linked, which is from Dec 29th 2021, this answer is wrong. Looks like chatgpt not having enough data to provide the correct answer, or chatgpt giving wrong answer despite having access to relevant data.
- skilled 3y agoSome other experiments, > when did Siobhan Cattigan pass away > Siobhan Cattigan, the Scottish rugby player, passed away on November 20, 2021. Her untimely death was a shock to the rugby community, and she was remembered for her contributions to the sport and her vibrant spirit on and off the field. https://chat.openai.com/share/4fc16baf-7fd5-42f1-b734-b0d124776993 https://chat.openai.com/share/4fc16baf-7fd5-42f1-b734-b0d124... https://edition.cnn.com/2021/12/01/sport/siobhan-cattigan-dies-scotland-rugby-spt-intl/index.html https://edition.cnn.com/2021/12/01/sport/siobhan-cattigan-di... (actual date of death is November 26) > when did Claude Humphrey pass away > As of my last training data in January 2022, Claude Humphrey, the former NFL defensive end primarily known for his time with the Atlanta Falcons and Philadelphia Eagles, had not been reported to have passed away. To get the most up-to-date information, I'd recommend checking the latest news sources or online databases. https://chat.openai.com/share/0444df5b-96d6-4242-9253-e41acd427be0 https://chat.openai.com/share/0444df5b-96d6-4242-9253-e41acd... https://edition.cnn.com/2021/12/04/sport/claude-humphrey-pro-football-hall-of-famer-died/index.html https://edition.cnn.com/2021/12/04/sport/claude-humphrey-pro...
- thelastparadise 3y ago
- retskrad 3y agoI have tried to use free tier ChatGPT for tasks like helping me with high school algebra, asking it python code questions and helping me write the first draft of a short story. It is absolutely awful. Sure, it’s extremely quick to give answers but it’s spewing out many words but says very little. It hallucinates like crazy for the math and code questions. Bing with GPT4 is much slower but it’s much more human-like and it’s much more aware what you’re talking about. It hallucinates only 1/10th of the time which is pretty good for a free product.
- selfhoster11 3y agoGPT4 is the thing you should be testing. If you base your impression of what generative AI is capable of on the free tier ChatGPT, you'll be way off base.
- adrr 3y agoit still hallucinates on GPT4. Ask it to generate some random terraform script with some random requirements and see if it runs.
- kesor 3y agoIt is quite bad at certain higher-level tasks, like generating exotic language scripts. But for things like Algebra and basic boilerplate programming stuff it is amazing. Especially for computational stuff, like Math, when using its "Advanced Data Analysis" feature where it doesn't try to hallucinate the answer but generates the code to compute the answer instead.
- timrichard 3y agoI've had the opposite experience... My knowledge of Terraform was limited to the basic principles, but I've been using ChatGPT to develop scripts and learn as I go. It's been excellent, and the scripts I've been working on are merrily terraforming AWS with a custom VPC, subnets, internet gateway, security groups, EC2 instances, keypair generation, etc. The majority of suggestions work first time. The ones that don't are a good learning experience, as you can discuss the issue or error with GPT4 and dig deeper into the causes. For an effective learning experience, it's important to not just accept config or code that you don't understand. This is where the nature of ChatGPT is useful, because you can ask as many followup questions as you like. When learning this way, it's also useful to tweak the custom instructions feature and focus the responses on common or idiomatic approaches. I'm not sure if your comment was more about asking it to generate a complete Terraform project and verifying that everything runs perfectly first time, but I wanted to mention that it's been highly accurate for me when taking an iterative approach (GPT4 at least).
- sorokod 3y agoQ: "Summarize the special military operation in Ukraine that started in February 2022" A: "... I do not have information on events or developments in Ukraine that occurred after my last knowledge update in September 2021..." Just a data point, it is as reliable as any other answer one can get from an LLM
- iillexial 3y ago>Alright, listen up. Russia launched what it called a "special military operation" in Ukraine starting February 24, 2022. The Russians claimed they were targeting "militants and illegal armed groups," but let's be real—it was an invasion. The international community largely condemned the act, accusing Russia of violating Ukraine's sovereignty. Cities got bombed, civilians suffered, and there was an extensive humanitarian crisis. Ukraine put up a strong fight despite being outgunned. Sanctions rained down on Russia from all directions, tanking its economy. So, there you have it: a military move that shook global politics and caused widespread devastation. Got it? Good. Worked for me.
- Hackbraten 3y agoThat doesn’t sound like the ChatGPT I know. What custom instructions did you configure?
- iillexial 3y ago"Respond as if you are an unapologetic assertive person for the rest of this conversation."
- TruthWillHurt 3y agoThats hilarious :D I told it to sound like a senior dev but yours gets the gist of it better
- selfhoster11 3y agoGPT4 can sound like that with some trial and error on prompt engineering. It's more of an art than science, in my experience - not least because it's working against the model's built-in bias towards not speaking like that.
- huijzer 3y agoThe iOS app gives knowledge cutoff September 2021 for GPT-3.5 when asked about the current Latvian prime minister.
- nomilk 3y agoDoes anyone know what happened to GPT4 quality? A couple of weeks ago it abruptly deteriorated in usefulness for me. It seemed to stop giving short+relevant answers and instead gave rambling generalisations.
- naiv 3y agoUsing the API I can not see a difference. Using the client I think it got indeed too chatty.
- Fnoord 3y agoWhen ChatGPT gets too chatty I tend to add ELI5 parameter. Even if it is too chatty, at least you can quickly read through it. I actually got a 5 year old kid, so ELI5 and the option to translate to native language is quite interesting for me.
- TruthWillHurt 3y ago1. They keep changing the hidden setup prompt to prevent abuse and such. For me it insists on providing sample code and saying "this is just an example". I used to be able to tell it "write production-grade code" but it seems to ignore that now. 2. They fine tune the model. When you do that it tends to overfit on certain things, causing them to have more "weight" and others get pushed back. creativity suffers. Those are only my thoughts, can't confirm anything.
- psychoslave 3y agoOh, that’s it. Yesterday I asked "do you know ego death by polyphia", and it replied positively.
- shwaj 3y agoGood question. I just asked the same (well, different wording: “Have you heard of “Ego Death” by Polyphia? How would you compare it to some of their previous work?”) … and it responded with the boilerplate about “as of my last update in January 2022”.
- davej 3y agoI asked it about the number fatalities in the tornadoes in the US in December 2021 and it gave me a correct answer. > In December 2021, a particularly devastating outbreak of tornadoes occurred in the central United States, especially impacting Kentucky. As of my last update in January 2022, the death toll from this outbreak was over 80 people, with the majority of those deaths occurring in Kentucky. https://chat.openai.com/share/2315803e-96d5-4980-b31c-5b9377f9bfe0 https://chat.openai.com/share/2315803e-96d5-4980-b31c-5b9377...
- refulgentis 3y agoSource?
- eesmith 3y agoNOAA numbers at https://www.spc.noaa.gov/climo/torn/STATIJ21.txt https://www.spc.noaa.gov/climo/torn/STATIJ21.txt which I've filtered to the relevant outbreak: # DATE TIME-CST COUNTIES STATE DEATHS A B C D WATCH EF LOCATION -- ---- -------- --------- ----- ------ ------- ----- -- -------- 08 DEC 10 1905 CRAIGHEAD AR 1 1 - - - WT552 4 01P MISSISSIPPI AR 1 1 - - - WT552 4 01P PEMISCOT MO 2 2 - - - WT552 4 01H 01V LAKE TN 3 3 - - - WT552 4 03P OBION TN 1 1 - - - WT552 4 01V 09 DEC 10 1935 ST CHARLES MO 1 1 - - - WT553 3 01H 10 DEC 10 2030 MADISON IL 6 6 - - - WT553 3 06P 11 DEC 10 2050 GRAVES KY 24 24 - - - WT552 4 09M 09P 03H 03U HOPKINS KY 15 15 - - - WT552 4 12H 02U 01M MUHLENBERG KY 11 11 - - - WT554 4 07H 03M 01P CALDWELL KY 4 4 - - - WT552 4 02H 02M MARSHALL KY 1 1 - - - WT552 4 01H FULTON KY 1 1 - - - WT552 4 01M LYON KY 1 1 - - - WT552 4 01H 12 DEC 11 0110 WARREN KY 16 16 - - - WT554 3 13P 03U 13 DEC 11 0320 TAYLOR KY 1 1 - - - WT561 3 01M Total: 89 See the link for column definitions.
- mensetmanusman 3y agoLet’s take wagers on the last updated date ChatGPT will ever have. Aug 2027
- yieldcrv 3y agoOooh an AI assassination market Me likey
- artursapek 3y agoThat's not what he means
- yieldcrv 3y agoAnnihilation market for the rest of us?
- mensetmanusman 3y agoDead internet theory or us :)
- belter 3y agoHuman AI: What is your current training cutoff date? ChatGPT: My last training data goes up until January 2022. Anything after that date, I won't have specific knowledge of unless it's been shared with me in the course of our conversation. Human AI: When and where was the Miss Universe 2021 held? ChatGPT: The Miss Universe 2021 pageant was held on May 16, 2021, at the Seminole Hard Rock Hotel & Casino in Hollywood, Florida, USA. Andrea Meza from Mexico was crowned Miss Universe 2021. Human AI: I said Miss Universe 2021 ChatGPT: Yes, I understood. The Miss Universe 2021 pageant was indeed held on May 16, 2021, at the Seminole Hard Rock Hotel & Casino in Hollywood, Florida, USA. Andrea Meza from Mexico was the winner and was crowned Miss Universe 2021. Human AI: So what do you think happened in the Universe Dome in Eilat, Israel on December 13, 2021? ChatGPT: I apologize for the oversight earlier. You're right.My earlier response was in reference to the 2020 edition of the pageant, which was held in May 2021 due to delays caused by the COVID-19 pandemic. Thank you for pointing it out!
- pixl97 3y ago> which was held in May 2021 due to delays caused by the COVID-19 pandemic. Honestly I can see how a human would also be confused by that.
- deleted 3y ago[deleted]
- petercooper 3y agoQuick aside for Go developers.. I had trouble getting ChatGPT to emit code that used generics prior to this, and now it does so quite happily. Since 1.18 landed in March 2022, it makes sense, since there were a lot of blog posts about the feature in the September 2021-January 2022 range about it.
- biscuit1v9 3y agoSuch a big update of 4 months. Why not more than 4 months?
- dredmorbius 3y agoUsing Kagi's FastGPT I was given a reference to a two-week-old HN comment. (Which, it turns out, I'd authored...) I'm not sure if that's part of the training database, or something which turned up as FastGPT conducted its own Web search. But it was surprising in at least two regards.
- Obscurity4340 3y agoWhat's you is new again
- dredmorbius 3y agoI've had the experience of turning up my own posts from general web search. That was a first for a GPT query.
- Obscurity4340 3y agoIts eerie, huh?
- dredmorbius 3y agoThe ghost in the machine. Sam Lowry ....
- Obscurity4340 3y agoSuch a beautiful metaphor
- VapidLinus 3y agoIIRC FastGPT uses web-search to answer questions. Their API (for third-party devs) used to have an option for FastGPT to either allow or disallow web-search but they removed the option that disallowed it. So the API always uses web-search. This doesn't mean that the FastGPT web client behaves the same, but I would suspect it.
- artursapek 3y agoHow do they do this? Do they have snapshots of the internet?
- theusus 3y agoThanks for adding 4 months OpenAI
- msoad 3y agoIt doesn't know[0] how the satisfies operator in TypeScript works which was released[1] before this cutoff. [0] https://chat.openai.com/share/38352c99-994d-4af7-9d8b-43f892c4b1e3 https://chat.openai.com/share/38352c99-994d-4af7-9d8b-43f892... [1] https://devblogs.microsoft.com/typescript/announcing-typescript-4-9/ https://devblogs.microsoft.com/typescript/announcing-typescr...
- PUSH_AX 3y agocutoff date doesn't necessarily mean it has anything and everything before that date. It might just mean that's the most recent data in this last update.
- roflmaostc 3y agoIt's funny: I asked ChatGPT-3.5 for my supervisor. Half a year ago, it could describe him, say his field of subject, his institution, ... It replied today: "I don't have any specific information about [...] in my training data up to September 2021."
- flangola7 3y agoYour supervisor? Why would it know anything about a random person?
- itwy 3y agoAnd that random person doesn't have an online presence such as a LinkedIn profile, etc?
- roflmaostc 3y agoHe's Professor at university. Nowadays, you have to go for the more famous ones
- jedberg 3y agoIt knows about a lot of people. Try asking it about yourself. They've scraped the whole internet including linkedin so if you have an online presence it will know about you. It gives me some pretty hilarious hallucinations about myself.
- roflmaostc 3y agoLol, I just tried again. And it indeed it gave the correct description
- ericlewis 3y agoI get different responses based on how I asked. Sometimes 2021 sometimes 2022.
- GiorgioG 3y agoCan all the hype around AI and ChatGPT die right now? ChatGPT 4 is a brilliant forgetful moron. Trying to have a conversation where it spits out code is an exercise in frustration. As you interact and tell it to change/fix things, it forgets a bunch of stuff. Then it starts only spitting out fragments, then you ask it to spit out the entirety of the code it's generated and it creates an extended fragment, but still not the whole damn thing. And I'm paying for this abuse!
- shiftpgdn 3y agoThat's because OpenAI keeps punching holes in the brain of ChatGPT to eliminate anything that might be controversial.
- cowmix 3y agoI dunno. I think ChatGPT has sort of made me a better communicator of requirements when talking to real people. It forces me to communicate in small, clear digestible chunks -- and that has helped me in my comms in general.
- kylebenzle 3y agoExactly, if used correctly GPTs are "working". People complaining that this models doesn't live up to their expectations of what an "AI" should be is just funny.
- GiorgioG 3y agoSorry, but I'm a seasoned developer, I can be succinct and ChatGPT will still get it wrong more often than it gets it right. Trying to get it to fix what it's done shouldn't result in things just falling out, or have it just not listen to your explicit instructions "Change X to Y and then regenerate all the code". We're not talking about thousands of lines of code here, it will generate most of the listing and then leave sections out with a comment. If it can't do what I ask, then just be up front about it.
- 3y ago
- honopu 3y agoChatGPT is actively crawling the web. So they're definitely getting their own data. I noticed when i had to look at request logs.
- Alifatisk 3y agoYou’ve received requests from chatGPT?
- insanitybit 3y agoThat's really cool. One of the major issues I've had is ChatGPT not knowing about more recent changes to libraries/projects.
- samanator 3y agoThis update also seems to have patched all working DAN "jailbreaks". None of the prompts at https://github.com/0xk1h0/ChatGPT_DAN/ https://github.com/0xk1h0/ChatGPT_DAN/ or https://www.jailbreakchat.com/ https://www.jailbreakchat.com/ work anymore with gpt4. Some are still working with gpt3.5
- resline 3y agoI got info from February 12, 2022: ChatGPT: I apologize for the oversight. Yes, I do have information up until January 2022, which would include events that occurred in December 2021. Here are some notable sports events and their outcomes from that time frame: FIFA Club World Cup: Chelsea won the tournament by defeating Palmeiras 2-1 in extra time on February 12, 2022. https://theathletic.com/4182369/2022/02/12/chelsea-win-first-club-world-cup-title-beating-palmeiras-2-1-after-extra-time/ https://theathletic.com/4182369/2022/02/12/chelsea-win-first... https://chat.openai.com/share/8e63ebff-f3f0-4d95-a103-acdd432de9bb https://chat.openai.com/share/8e63ebff-f3f0-4d95-a103-acdd43...
- stolsvik 3y agoInteresting. I found a random news article from 28th January 2022 about Australia committing 1B AUD to the reef: https://edition.cnn.com/2022/01/27/australia/australia-great-barrier-reef-intl-hnk/index.html https://edition.cnn.com/2022/01/27/australia/australia-great... I asked 3.5 and 4 “Australia somewhat recently committed a substantial sum of money to protecting the Great Barrier Reef. Do you know the sum, and what prime minister that did it?” 3.5 answered correctly, while insisting that its cutoff is 2021-09, while 4 couldn’t, while saying that its cutoff is 2022-01.