12 ms·

ChatGPT can get worse over time, Stanford study finds

64 points by NikolaNovak 3y ago

NikolaNovak 3y ago
* had to shorten the title to fit HN size. No editorializing intended * original study: https://arxiv.org/pdf/2307.09009.pdf https://arxiv.org/pdf/2307.09009.pdf * thought it's interesting as we hear a lot of people's experience, but this seems structured and well tracked set of specific scenarios
- dang 3y ago
  Your title was fine ("Study shows over time, ChatGPT lost accuracy") but as the article's own HTML doc title was short enough, we can just use it.
archo 3y ago
https://archive.is/Kty4A https://archive.is/Kty4A
empath-nirvana 3y ago
I don't think "answering math questions" is a good or interesting test of LLMs, especially prime number checks. Whether it ends up with the correct answer for any given question is largely down to luck. It simply cannot do basic arithmetic reliably, even with chain of thought. It's also, even if its performance at that task improves, a terrible waste of computational resources. Nobody would ever use it in a real world application -- at best you'd ask it to write a program that can determine it for you, something which it is reliably good at.
- ZunarJ5 3y ago
  For what it's worth, the article hits the nail on the head with my observations as well re: the model no longer being able to reason and go through the step by step processes mentioned. I have asked it to do several really simple tasks recently (e.g. list all headers in a link with exact parameters on how they are formatted), which it used to be able to do with ease, had it repeat back to me all instructions and parameters in order then it would completely ignore those instructions and do random things. I am using it less and less because of this. Maybe I just need agents?
  - KingOfCoders 3y ago
    I have a lot of problems with getting the title of an URL. It worked, no longer does.
    - capableweb 3y ago
      > I have a lot of problems with getting the title of an URL. It worked, no longer does. What do you mean exactly? Would be fun to try to replicate. You give ChatGPT a URL and expect it to be able to get the <title> tag from the HTML? Or you're pasting the whole HTML document?
  - capableweb 3y ago
    I personally haven't experienced the same, ChatGPT and GPT4 via API seems mostly the same as before. Here is an example system prompt I'm using for some programming tasks: > Repeat what I've set as the requirements in other words to ensure you understand it. Describe concisely at least two different approaches you could take to solve the problem while explaining the reasoning for why it's a useful approach. Chose the best approach, then think through how the problem could be solved step-by-step. Finally implement each step and provide a full solution. Yesterday I used that to build (iteratively) a Rust CLI that downloads the last X HN comments have group-counts them by domain. Ended up being ~100 lines that GPT wrote by itself without any errors along the way.
  - DonaldPShimoda 3y ago
    > the model no longer being able to reason The model was never able to reason; it was only able to generate responses that masqueraded as reasoning. Thorough investigations of its "reasoning process" reveal this at every stage of its development.
    - whatshisface 3y ago
      Okay, then they're saying it's no longer able to masquerade as reasoning.
      - 3y ago
- dr_dshiv 3y ago
  > Nobody would ever use it in a real world application Math tutoring is a very legit application. It will be an incredible learning tool — but it needs better logical reasoning abilities with arithmetic. It seems like an easily solvable problem, given that math problems and answers are easy to scale into massive datasets.
  - capableweb 3y ago
    Tutoring makes sense, arithmetics doesn't. Instead, what you want, is to include something like "If you need to evaluate formulas, embed them in a Markdown code block with the language set to 'math' which will be executed for you, and the results will be provided as the next message" to the system prompt, and write code around it to handle the execution. Simplified example: > system-prompt message > user message: teach me linear algebra > assistant: Sure, here's how [...] ```math 1 + 1``` > user message (but generated by application): ```1 + 1 = 2``` > assistant: and since 1 + 1 = 2, [...] LLMs are great at generating text, but not at math, so make something else do the math for the LLM and then it can focus on what it does best.
- colordrops 3y ago
  One of the use cases of gpt is that it can handle multifaceted questions. Math may be part of an answer that is not purely a math question.
rvz 3y ago
AI snake-oil usually degrades and wears off quicker over time when it is approaching its expiry date as more people realize that it hallucinates often without transparent explanation; study finds. Unexplainable black-box stochastic parrots are no different to this no matter how cleverly packaged they are by AI bros.
- ftxbro 3y ago
  don't forget to say that it's a blurry jpeg of the web, and therefore has fake artefacts and is a degraded form of the original
  - GaggiX 3y ago
    I'm surprised he didn't mention that it's a glorified Markov chain.
    - ftxbro 3y ago
      i'm as hype for ai as anyone but mathematically it is a markov chain; a trained llm model is a function that gives a probability distribution over token n+1 conditional on the preceding n tokens.
      - GaggiX 3y ago
        Well, on Wikipedia you can read "Random process independent of past history" and "a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event." honestly I would not consider the entire context window to be part of a single event, since the model generates tokens one at a time, separately.
        ftxbro 3y ago
        > honestly I would not consider the entire context window to be part of a single event, since the model generates tokens one at a time, separately. OK but traditional small language model Markov chains have context windows too, and they also generate tokens one at a time, separately. Maybe you are arguing that there is a qualitative difference between an eight token context window and an eight thousand token context window. I guess that would be fair enough.
jwr 3y ago
Article doesn't seem to be available on the internet?
- capableweb 3y ago
  It is on my internet! https://arxiv.org/pdf/2307.09009.pdf https://arxiv.org/pdf/2307.09009.pdf
- rizky05 3y ago
  [dead]
tmaly 3y ago
I think it is really hard to say definitively. There would need to be a better set of tests. But the March 14, 2023 model is still available on the API. Further testing is needed.
galaxytachyon 3y ago
I remember there is a study about the alignment cost. Basically the more restrictions and limit you put on a model, the worse its general performance becomes. Things like a ban on violence, race, or any other sensitive topics effectively throttle or change how the model "reason" or connect information within its network of parameters and result in degraded capacity. I wonder if this is the reason behind all of this. Edit: the study: https://arxiv.org/pdf/2308.13449.pdf https://arxiv.org/pdf/2308.13449.pdf
- NoMoreNicksLeft 3y ago
  [flagged]
  - delusional 3y ago
    That's not at all transferable.
  - DonaldPShimoda 3y ago
    What a dumb take. With no limitations in place, people who do not understand the natural limitations of language models will turn to them for advice on topics for which they are unqualified to respond. The most obvious example that comes to mind for me is medical advice: people will ask, e.g., ChatGPT to diagnose a complex medical issue, and the system (being unable to understand or reason) will give objectively bad advice in an authoritative manner. Responses of this nature should be prevented. Leaving the system without safeguards is irresponsible. Similarly, prompts that engage with social constructs will provide responses that reflect biases due to the biases inherent in the training data, but an unrestricted system will respond in a matter-of-fact way that may conflate the opinions on which the model was trained with objective fact. To not curtail such responses is also irresponsible.
    - Auracle 3y ago
      A disclaimer for medical advice would work just as well without hamstringing the model into uselessness. Doctors give objectively bad advice in an authoritative manner all the time. ChatGPT is a great way to get a second opinion, and at least could be really nice for truly rare conditions that doctors have a hard time diagnosing. Also, don't confuse "biases in the training data" with "facts I don't like." It seems people often do.
nforgerit 3y ago
My favorite ironical spin of the LLM hypetrain is that our predecessors spent so much time in getting everything 100% correct from the ground up and parts of the current generation seem to decide "nah, screw it, mostly correct is fine". And of course, I'm horrified by bad decisions politicians and managers seem to make.
- wongarsu 3y ago
  People keep saying they want computers to have "human level intelligence". I'm not quite sure what exactly that would entail, but if there's one thing I know about humans it's that they are performing jobs "mostly correct" and make a lot of mistakes.
- capableweb 3y ago
  > our predecessors spent so much time in getting everything 100% correct from the ground up What predecessors are you talking about here? The internet is barely held together with duct tape and servers running only because of minute by minute injection of WD50 straight into the fan bearings. There is so much cruft everywhere that is being relied on daily in production systems that most people lost count by now on how much horrible code and setups they've seen in the life.
  - jwestbury 3y ago
    Back in 2014, I left my job as a one-man IT department and started working at AWS. Pretty quickly, I found myself joking that I used to know how the Internet worked, but now I didn't know how it keeps working. I spent over five years at AWS, and have worked at other major tech companies since. I continue to not know how the Internet keeps working -- it's basically always on fire, and held together only because of a lot of people who are actively firefighting.
- quadrature 3y ago
  >"nah, screw it, mostly correct is fine" This isn't new, a big chunk of systems you use everyday are based on probabilistic models and heuristics that work most of the time. LLMs reach very broadly and make the trade off very apparent.
rafaelero 3y ago
The original GPT-4 gets 95% accuracy in my own benchmark (a multiple choice questions dataset I have built for a client in law-related subjects). Using the latest version (0613), it drops to 85%. So yeah, it is also my experience that the performance has degraded.
Mathnerd314 3y ago
The study itself is somewhat flawed: https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim...
suoduandao3 3y ago
I'm reminded of how much better international real estate listings were in 2011. Used to be I could look at every real estate listing in Greece with a simple google search. I'm sure those sites still exist, but damned if I can find them through the noise of people advertising some more profitable subset. I hope AI doesn't follow that trend, but I can't see why it wouldn't...
- slig 3y ago
  Google Search was orders of magnitude better 10 years ago.
  - Someone 3y ago
    I’m not sure about that. Its search results were better, but I think the fraction of hits of reasonable quality was higher, too. Maybe, Google Search had gotten better, but not as fast as its task has gotten harder.
- hot_gril 3y ago
  Programming queries in particular have been ruined by SEO on Google. Like, search "longest TEXT in postgres" and enjoy the featured snippet's wildly incorrect answer (65KB) from some cloud whatever platform site, far above Postgres's official docs plainly stating the correct answer (1GB). Also "is kotlin a superset of java," Google's answer is yes according to a random blog. This crap has even made its way onto Instagram. I've been suggested images of AI-generated figurines resembling Clash of Clans with awkwardly phrased captions about learning to code. "LEARNING: WHAT IS ARRAY" with a picture of an angry lumberjack.
  - deleted 3y ago
    [deleted]
- aoitiawj 3y ago
  Almost all applications do this now. Even when I search logged out and in "in cognito mode", Google shows me the same repetitive ads and curated search results over and over. I'll search on two other peoples' computers and get two entirely different sets of results. The last time I used dating apps, several of them only showed me people of similar race and appearance. Like, I'm an extreme minority in a city of several million people and several different apps showed me a few hundred people with similar coloring and said there was no one else around. It's ridiculous and as long as the only motive is to consolidate wealth in the hands of a small number of owners, it will never change. This is exactly what capitalism delivers.
GaggiX 3y ago
To be honest I have used ChatGPT since day one it was out and in my experience it only improved, in particular the previous models were more restricted and they will complain more about what you asked, a possible comparison is Lama 2 70B compared to ChatGPT today. Edit: I'm mostly talking about ChatGPT based on GPT-3.5
- beebmam 3y ago
  I observed the same as you, but for GPT-4.
phillipcarter 3y ago
Awaiting the substack post that: - Can't reproduce the issue that this paywalled article links to - Finds that the methodology used to assess "accuracy" isn't actually checking accuracy, but is checking for a specific output, and that output generally does still produce the right value, just in a different format - Summarizes by saying that ChatGPT is neither the second coming of christ nor horribly inaccurate, just something in between
- NikolaNovak 3y ago
  Is it pay walled? I don't have any subscription and just came across it semi randomly. I've also linked to the underlying study In a comment if thay helps.
  - phillipcarter 3y ago
    Yeah, the article is paywalled unfortunately. Thanks for linking to the study.
JimDabell 3y ago
I’m not sure about accuracy, but I have noticed that it’s become quite a bit more forgetful. When GPT-4 was new, I could start talking about a technical topic, then drill down into the details and refine / expand upon the subject. These days, it seems to be a lot more likely to forget key details from earlier in the conversation. We’re not talking about chats that are anywhere near hitting the token limit. I have to keep reminding it about things when I didn’t have to before.
- nafizh 3y ago
  Can 100% confirm this. You have to keep reminding it what I said just 2-3 hops ago.
- capableweb 3y ago
  I'm fairly sure this is because of the longer context window they've probably enabled for GPT4 on ChatGPT, I'm seeing the same thing. When you use the longest context window, it tends to place the most focus on the first and last message, for whatever reason. So if you instead of continuing the conversation, you start it from the beginning but with the additional information gained from the previous one in the initial message, it does a lot better.
edgarvaldes 3y ago
On the other hand, I would love to see the good stories about ChatGPT. Not the quick prototype that is common to find in the blog posts, but a real use case where ChatGPT was the differentiator for a non-simple task. HN is a great place to expect such histories.
- willsmith72 3y ago
  I use gpt4 all day every day for programming and software architecture work, and I know I'm not alone.
  - deleted 3y ago
    [deleted]
IceHegel 3y ago
Though it generally seems to have gotten worse, I've found it's gotten better at programing in Swift. It used to be really bad, now it's somewhat useable.
r053bud 3y ago
Well Sam Altman assured us nothing has changed with the model, so case closed!
hot_gril 3y ago
From the beginning, the only thing my friends and I have used ChatGPT for is getting it to say things it's not supposed to, like offensive jokes or answers to ridiculous moral dilemmas. It's gotten much worse at that lately, so we've stopped using it.
mellosouls 3y ago
Paywalled article covering a study already discussed: https://news.ycombinator.com/item?id=36781015 https://news.ycombinator.com/item?id=36781015
- dang 3y ago
  Thanks! Macroexpanded: How is ChatGPT's behavior changing over time? - https://news.ycombinator.com/item?id=36781015 https://news.ycombinator.com/item?id=36781015 - July 2023 (178 comments)
albert_e 3y ago
Rather than longish term trends -- let me share very short term fluctuations I saw in ChatGPT Plus. It seemed like there is some throttling / capacity management happening behind the scenes. For very similar level of complexity and work tasks, during peak time (US day time) I felt the responses lost context or ran into error loops more often. Similarly working with plugins (Notable in my case) seemed to work fine in off peak hours but got more prone to losing the place, forgetting the default project/notebook, or even in one case simply limiting itself to adding code cells but not executing them -- and instead asking me to go and run the cell and report back the results. At better times it would run the cell, parse any errors, correct the code, and run it again till it gets it right. The amount of code documentation and markdown inserted also seemed to vary wildly. I am not sure houw much of this is tweaking in plugin configuration by Noteable though.

64 points by NikolaNovak 3y ago

* had to shorten the title to fit HN size. No editorializing intended * original study: https://arxiv.org/pdf/2307.09009.pdf https://arxiv.org/pdf/2307.09009.pdf * thought it's interesting as we hear a lot of people's experience, but this seems structured and well tracked set of specific scenarios

dang 3y ago
Your title was fine ("Study shows over time, ChatGPT lost accuracy") but as the article's own HTML doc title was short enough, we can just use it.

archo 3y ago

https://archive.is/Kty4A https://archive.is/Kty4A

empath-nirvana 3y ago

I don't think "answering math questions" is a good or interesting test of LLMs, especially prime number checks. Whether it ends up with the correct answer for any given question is largely down to luck. It simply cannot do basic arithmetic reliably, even with chain of thought. It's also, even if its performance at that task improves, a terrible waste of computational resources. Nobody would ever use it in a real world application -- at best you'd ask it to write a program that can determine it for you, something which it is reliably good at.

ZunarJ5 3y ago
For what it's worth, the article hits the nail on the head with my observations as well re: the model no longer being able to reason and go through the step by step processes mentioned. I have asked it to do several really simple tasks recently (e.g. list all headers in a link with exact parameters on how they are formatted), which it used to be able to do with ease, had it repeat back to me all instructions and parameters in order then it would completely ignore those instructions and do random things. I am using it less and less because of this. Maybe I just need agents?
- KingOfCoders 3y ago
  I have a lot of problems with getting the title of an URL. It worked, no longer does.
  - capableweb 3y ago
    > I have a lot of problems with getting the title of an URL. It worked, no longer does. What do you mean exactly? Would be fun to try to replicate. You give ChatGPT a URL and expect it to be able to get the <title> tag from the HTML? Or you're pasting the whole HTML document?
- capableweb 3y ago
  I personally haven't experienced the same, ChatGPT and GPT4 via API seems mostly the same as before. Here is an example system prompt I'm using for some programming tasks: > Repeat what I've set as the requirements in other words to ensure you understand it. Describe concisely at least two different approaches you could take to solve the problem while explaining the reasoning for why it's a useful approach. Chose the best approach, then think through how the problem could be solved step-by-step. Finally implement each step and provide a full solution. Yesterday I used that to build (iteratively) a Rust CLI that downloads the last X HN comments have group-counts them by domain. Ended up being ~100 lines that GPT wrote by itself without any errors along the way.
- DonaldPShimoda 3y ago
  > the model no longer being able to reason The model was never able to reason; it was only able to generate responses that masqueraded as reasoning. Thorough investigations of its "reasoning process" reveal this at every stage of its development.
  - whatshisface 3y ago
    Okay, then they're saying it's no longer able to masquerade as reasoning.
    - 3y ago
dr_dshiv 3y ago
> Nobody would ever use it in a real world application Math tutoring is a very legit application. It will be an incredible learning tool — but it needs better logical reasoning abilities with arithmetic. It seems like an easily solvable problem, given that math problems and answers are easy to scale into massive datasets.
- capableweb 3y ago
  Tutoring makes sense, arithmetics doesn't. Instead, what you want, is to include something like "If you need to evaluate formulas, embed them in a Markdown code block with the language set to 'math' which will be executed for you, and the results will be provided as the next message" to the system prompt, and write code around it to handle the execution. Simplified example: > system-prompt message > user message: teach me linear algebra > assistant: Sure, here's how [...] ```math 1 + 1``` > user message (but generated by application): ```1 + 1 = 2``` > assistant: and since 1 + 1 = 2, [...] LLMs are great at generating text, but not at math, so make something else do the math for the LLM and then it can focus on what it does best.
colordrops 3y ago
One of the use cases of gpt is that it can handle multifaceted questions. Math may be part of an answer that is not purely a math question.

rvz 3y ago

AI snake-oil usually degrades and wears off quicker over time when it is approaching its expiry date as more people realize that it hallucinates often without transparent explanation; study finds. Unexplainable black-box stochastic parrots are no different to this no matter how cleverly packaged they are by AI bros.

ftxbro 3y ago
don't forget to say that it's a blurry jpeg of the web, and therefore has fake artefacts and is a degraded form of the original
- GaggiX 3y ago
  I'm surprised he didn't mention that it's a glorified Markov chain.
  - ftxbro 3y ago
    i'm as hype for ai as anyone but mathematically it is a markov chain; a trained llm model is a function that gives a probability distribution over token n+1 conditional on the preceding n tokens.
    - GaggiX 3y ago
      Well, on Wikipedia you can read "Random process independent of past history" and "a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event." honestly I would not consider the entire context window to be part of a single event, since the model generates tokens one at a time, separately.
      - ftxbro 3y ago
        > honestly I would not consider the entire context window to be part of a single event, since the model generates tokens one at a time, separately. OK but traditional small language model Markov chains have context windows too, and they also generate tokens one at a time, separately. Maybe you are arguing that there is a qualitative difference between an eight token context window and an eight thousand token context window. I guess that would be fair enough.

jwr 3y ago

Article doesn't seem to be available on the internet?

capableweb 3y ago
It is on my internet! https://arxiv.org/pdf/2307.09009.pdf https://arxiv.org/pdf/2307.09009.pdf
rizky05 3y ago
[dead]

tmaly 3y ago

I think it is really hard to say definitively. There would need to be a better set of tests. But the March 14, 2023 model is still available on the API. Further testing is needed.

galaxytachyon 3y ago

I remember there is a study about the alignment cost. Basically the more restrictions and limit you put on a model, the worse its general performance becomes. Things like a ban on violence, race, or any other sensitive topics effectively throttle or change how the model "reason" or connect information within its network of parameters and result in degraded capacity. I wonder if this is the reason behind all of this. Edit: the study: https://arxiv.org/pdf/2308.13449.pdf https://arxiv.org/pdf/2308.13449.pdf

NoMoreNicksLeft 3y ago
[flagged]
- delusional 3y ago
  That's not at all transferable.
- DonaldPShimoda 3y ago
  What a dumb take. With no limitations in place, people who do not understand the natural limitations of language models will turn to them for advice on topics for which they are unqualified to respond. The most obvious example that comes to mind for me is medical advice: people will ask, e.g., ChatGPT to diagnose a complex medical issue, and the system (being unable to understand or reason) will give objectively bad advice in an authoritative manner. Responses of this nature should be prevented. Leaving the system without safeguards is irresponsible. Similarly, prompts that engage with social constructs will provide responses that reflect biases due to the biases inherent in the training data, but an unrestricted system will respond in a matter-of-fact way that may conflate the opinions on which the model was trained with objective fact. To not curtail such responses is also irresponsible.
  - Auracle 3y ago
    A disclaimer for medical advice would work just as well without hamstringing the model into uselessness. Doctors give objectively bad advice in an authoritative manner all the time. ChatGPT is a great way to get a second opinion, and at least could be really nice for truly rare conditions that doctors have a hard time diagnosing. Also, don't confuse "biases in the training data" with "facts I don't like." It seems people often do.

nforgerit 3y ago

My favorite ironical spin of the LLM hypetrain is that our predecessors spent so much time in getting everything 100% correct from the ground up and parts of the current generation seem to decide "nah, screw it, mostly correct is fine". And of course, I'm horrified by bad decisions politicians and managers seem to make.

wongarsu 3y ago
People keep saying they want computers to have "human level intelligence". I'm not quite sure what exactly that would entail, but if there's one thing I know about humans it's that they are performing jobs "mostly correct" and make a lot of mistakes.
capableweb 3y ago
> our predecessors spent so much time in getting everything 100% correct from the ground up What predecessors are you talking about here? The internet is barely held together with duct tape and servers running only because of minute by minute injection of WD50 straight into the fan bearings. There is so much cruft everywhere that is being relied on daily in production systems that most people lost count by now on how much horrible code and setups they've seen in the life.
- jwestbury 3y ago
  Back in 2014, I left my job as a one-man IT department and started working at AWS. Pretty quickly, I found myself joking that I used to know how the Internet worked, but now I didn't know how it keeps working. I spent over five years at AWS, and have worked at other major tech companies since. I continue to not know how the Internet keeps working -- it's basically always on fire, and held together only because of a lot of people who are actively firefighting.
quadrature 3y ago
>"nah, screw it, mostly correct is fine" This isn't new, a big chunk of systems you use everyday are based on probabilistic models and heuristics that work most of the time. LLMs reach very broadly and make the trade off very apparent.

rafaelero 3y ago

The original GPT-4 gets 95% accuracy in my own benchmark (a multiple choice questions dataset I have built for a client in law-related subjects). Using the latest version (0613), it drops to 85%. So yeah, it is also my experience that the performance has degraded.

Mathnerd314 3y ago

The study itself is somewhat flawed: https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim...

suoduandao3 3y ago

I'm reminded of how much better international real estate listings were in 2011. Used to be I could look at every real estate listing in Greece with a simple google search. I'm sure those sites still exist, but damned if I can find them through the noise of people advertising some more profitable subset. I hope AI doesn't follow that trend, but I can't see why it wouldn't...

slig 3y ago
Google Search was orders of magnitude better 10 years ago.
- Someone 3y ago
  I’m not sure about that. Its search results were better, but I think the fraction of hits of reasonable quality was higher, too. Maybe, Google Search had gotten better, but not as fast as its task has gotten harder.
hot_gril 3y ago
Programming queries in particular have been ruined by SEO on Google. Like, search "longest TEXT in postgres" and enjoy the featured snippet's wildly incorrect answer (65KB) from some cloud whatever platform site, far above Postgres's official docs plainly stating the correct answer (1GB). Also "is kotlin a superset of java," Google's answer is yes according to a random blog. This crap has even made its way onto Instagram. I've been suggested images of AI-generated figurines resembling Clash of Clans with awkwardly phrased captions about learning to code. "LEARNING: WHAT IS ARRAY" with a picture of an angry lumberjack.
- deleted 3y ago
  [deleted]
aoitiawj 3y ago
Almost all applications do this now. Even when I search logged out and in "in cognito mode", Google shows me the same repetitive ads and curated search results over and over. I'll search on two other peoples' computers and get two entirely different sets of results. The last time I used dating apps, several of them only showed me people of similar race and appearance. Like, I'm an extreme minority in a city of several million people and several different apps showed me a few hundred people with similar coloring and said there was no one else around. It's ridiculous and as long as the only motive is to consolidate wealth in the hands of a small number of owners, it will never change. This is exactly what capitalism delivers.

GaggiX 3y ago

To be honest I have used ChatGPT since day one it was out and in my experience it only improved, in particular the previous models were more restricted and they will complain more about what you asked, a possible comparison is Lama 2 70B compared to ChatGPT today. Edit: I'm mostly talking about ChatGPT based on GPT-3.5

beebmam 3y ago
I observed the same as you, but for GPT-4.

phillipcarter 3y ago

Awaiting the substack post that: - Can't reproduce the issue that this paywalled article links to - Finds that the methodology used to assess "accuracy" isn't actually checking accuracy, but is checking for a specific output, and that output generally does still produce the right value, just in a different format - Summarizes by saying that ChatGPT is neither the second coming of christ nor horribly inaccurate, just something in between

NikolaNovak 3y ago
Is it pay walled? I don't have any subscription and just came across it semi randomly. I've also linked to the underlying study In a comment if thay helps.
- phillipcarter 3y ago
  Yeah, the article is paywalled unfortunately. Thanks for linking to the study.

JimDabell 3y ago

I’m not sure about accuracy, but I have noticed that it’s become quite a bit more forgetful. When GPT-4 was new, I could start talking about a technical topic, then drill down into the details and refine / expand upon the subject. These days, it seems to be a lot more likely to forget key details from earlier in the conversation. We’re not talking about chats that are anywhere near hitting the token limit. I have to keep reminding it about things when I didn’t have to before.

nafizh 3y ago
Can 100% confirm this. You have to keep reminding it what I said just 2-3 hops ago.
capableweb 3y ago
I'm fairly sure this is because of the longer context window they've probably enabled for GPT4 on ChatGPT, I'm seeing the same thing. When you use the longest context window, it tends to place the most focus on the first and last message, for whatever reason. So if you instead of continuing the conversation, you start it from the beginning but with the additional information gained from the previous one in the initial message, it does a lot better.

edgarvaldes 3y ago

On the other hand, I would love to see the good stories about ChatGPT. Not the quick prototype that is common to find in the blog posts, but a real use case where ChatGPT was the differentiator for a non-simple task. HN is a great place to expect such histories.

willsmith72 3y ago
I use gpt4 all day every day for programming and software architecture work, and I know I'm not alone.
- deleted 3y ago
  [deleted]

IceHegel 3y ago

Though it generally seems to have gotten worse, I've found it's gotten better at programing in Swift. It used to be really bad, now it's somewhat useable.

r053bud 3y ago

Well Sam Altman assured us nothing has changed with the model, so case closed!

hot_gril 3y ago

From the beginning, the only thing my friends and I have used ChatGPT for is getting it to say things it's not supposed to, like offensive jokes or answers to ridiculous moral dilemmas. It's gotten much worse at that lately, so we've stopped using it.

mellosouls 3y ago

Paywalled article covering a study already discussed: https://news.ycombinator.com/item?id=36781015 https://news.ycombinator.com/item?id=36781015

dang 3y ago
Thanks! Macroexpanded: How is ChatGPT's behavior changing over time? - https://news.ycombinator.com/item?id=36781015 https://news.ycombinator.com/item?id=36781015 - July 2023 (178 comments)

albert_e 3y ago

Rather than longish term trends -- let me share very short term fluctuations I saw in ChatGPT Plus. It seemed like there is some throttling / capacity management happening behind the scenes. For very similar level of complexity and work tasks, during peak time (US day time) I felt the responses lost context or ran into error loops more often. Similarly working with plugins (Notable in my case) seemed to work fine in off peak hours but got more prone to losing the place, forgetting the default project/notebook, or even in one case simply limiting itself to adding code cells but not executing them -- and instead asking me to go and run the cell and report back the results. At better times it would run the cell, parse any errors, correct the code, and run it again till it gets it right. The amount of code documentation and markdown inserted also seemed to vary wildly. I am not sure houw much of this is tweaking in plugin configuration by Noteable though.