15 ms·

How is ChatGPT's behavior changing over time?

289 points by tim_sw 3y ago

caffeinatedwo 3y ago
It's "Open" AI ;-)
Der_Einzige 3y ago
Never let OpenAI or Google employees (in regards to search) gaslight you into believing that things aren't being "enshittified". Glad to see scholarly evidence of this coming out.
- EGreg 3y ago
  Profit motive corrupts Absolute profit motive corrupts absolutely
- quickthrower2 3y ago
  Honest enshittification is good but rare. Like a cloud telling you what specific CPU SKU is in the hardware (not just 1 vcore). They should say what you are paying for in terms of technicals, either spill the beans on the architecture or have some SLAs on how good the thing is.
  - opportune 3y ago
    I work in cloud-land, the problem with exposing every little technical detail is that 1. part of the point of purchasing a cloud product is to abstract that away 2. coding to the implementation instead of the API creates huge headaches for everybody because there are often good reasons (make things more efficient) to change the implementation 3. sadly, there are “features” such as overcommit which you don’t want to know about as a customer. While some of those are anti-customer on their face, the “we shouldn’t commit to implementation details and instead just to our API/feature surface” may seem anti-customer while actually making things much better for customers. When implementations are supported in perpetuity development grinds to a standstill. Even overcommit ends up helping customers - cloud companies can offer lower sticker prices. In the case of both regular Public cloud and OpenAI - if you need stability beyond publicly supported APIs you are probably not a good fit as a customer and should instead find a company willing to commit to lower level implementations (eg bare metal, truly major/minor versioned software) or roll-your-own.
    - quickthrower2 3y ago
      That is fine, then you need some kinds of comparable metric that stays the same over time. FLOPS maybe? Ideally the same across clouds, or more likely cloud-specific. Or even re pin it every 5 years. E.g. a 2020-CPU, 2025-CPU etc. We need a "horse power" for the cloud I guess!
deleted 3y ago
[deleted]
svaha1728 3y ago
>June version added “‘python and “‘ before and after the code snippet. Second, it also generated a few more comments. While a small change, the extra triple quotes render the code not executable. These are to represent markdown code blocks, which probably helped their front end developers, but hinders copy-pasta
- QuantumG 3y ago
  Outputting that on the API is kinda dumb, but yeah, hardly something worth publishing.
dudeinhawaii 3y ago
I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quotes). I'd be more interested in changes to the quality and performance of the code generated, not whether it can be easily copy/pasted from the page. I could not replicate that result (but could replicate the other math issues). I've also never seen the triple quotes in results so it's unclear if there was a temporary presentation bug.
- sanxiyn 3y ago
  Or OpenAI can stop being stupid and adopt LLaMA-like tokenization, which special cases numbers and tokenize them into individual digits.
  - jinay 3y ago
    Are LLaMA-based models better at math because of this digit-based tokenization?
  - bugglebeetle 3y ago
    Isn’t the tokenization tied to the training of the model?
    - colobas 3y ago
      The model learns an embedding table, where (roughly) each row is used as the model’s internal representation of each token. The numbers in that table are learned. What isn’t learned is the map from token (i.e. combination of characters/byte-pairs) to row-index in the embedding table. That is given by tokenization EDIT: removed redundant bit
- ehnto 3y ago
  I think knowing if the code can be used verbatim is actually the more important part practically speaking. That is the actually useful part. Quality is important to humans, because humans have to read it, but correctness is what people using ChatGPT for code actually need. So long as the quality and performance is good enough, then it will be useful. Performance is such a nuanced topic that you need very context aware devs anyway, and I think a general purpose LLM is never going to have that kind of awareness.
ramesh31 3y ago
OpenAI is going to be left in the dust by (actual) open models. Llama 2 is already reaching GPT-3 levels, and can run inference on consumer hardware. Crazy how fast that flipped.
- nomel 3y ago
  What enables this? There's a huge gap between GPT-3.5 and 4, put there by a massive amount of money, from my understanding. To compete, with open source projects being less well funded, I would assume that orders of magnitude improvements in training cost would be required. What do you see driving this, and who do you see paying for it? If Meta, or anyone else, gets something that beats GPT-4, I would naively assume they would monetize it. If someopen source effort manages to beat GPT-4, then I assume some well funded, profit seeking, entity will dump orders of magnitude more money into whatever enabled the open source offerings to compete. I don't think GPT-4 is the pinnacle of OpenAI, or that their funding will run dry, so they shouldn't be viewed as a static entity.
  - ramesh31 3y ago
    >Or, are you suggesting that GPT-4 is the pinnacle of OpenAI, or that their funding will run dry? My bet is that Meta has pivoted almost entirely to this space with their R&D in the last six months. Llama 2 is spectacular. And with its' success, there will undoubtedly be more. They also happen to have access to limitless amounts of compute, cash, and engineering that puts OpenAI to shame. This could finally be their chance to create a platform for real. And the open source community and startups will benefit off of that.
    - deleted 3y ago
      [deleted]
    - nomel 3y ago
      What motive do you see for them releasing these models for free, especially with the massive increase in cost associated with catching up?
ttul 3y ago
I tried the prime number problem and GPT-4 nailed it. I’m not sure whether they are testing things correctly… “ Sure, let's go step by step. A prime number is a number greater than 1 that has no positive divisors other than 1 and itself. This means if we can find any other number (excluding 1 and the number itself) that divides 17077, then it is not a prime number. Let's start by checking divisibility by 2. Since 17077 is an odd number (its last digit is 7), it is not divisible by 2. For the next step, we can look at divisibility by 3. We add up the digits: 1+7+0+7+7 = 22. Because 22 is not divisible by 3, 17077 is also not divisible by 3. The smallest prime number greater than 3 is 5. Checking divisibility by 5, we see that 17077 does not end in 0 or 5, so it is not divisible by 5. The smallest prime number greater than 5 is 7. To check divisibility by 7, there is no simple rule like with 2, 3, and 5, so we would perform the actual division. But we can stop here because I just recalled a rule that we can only check up to the square root of the number and the square root of 17077 is approximately 130.68. So if there were factors other than 1 and 17077, we would have found them by now. Therefore, 17077 is a prime number.”
- moonchild 3y ago
  > GPT-4 nailed it Did it? Did you read what it wrote?
  - kevinventullo 3y ago
    It certainly nails the cadence of a rigorous proof. It just needs to learn to append “the rest is left as an exercise to the reader” to the end of its output.
    - deleted 3y ago
      [deleted]
  - pedrosorio 3y ago
    I’ve seen so many of these on Twitter since chat GPT came out. > Look at this amazing capability, it gets it perfectly! (screenshot of interaction with clear flaws/outright wrong answer)
blobbers 3y ago
A little surprised this sort of thing passes for a publish-able paper. Isn't it the equivalent to saying "Here's the top 10 results for google searching golden retrievers March 2023, and here's the top 10 results from June 2023. We see that google is returning even cuter animals today. Unfortunately though, one of the results linked to a page full of cats." I'm sure openai has a list of standard questions that it tracks the responses it is getting over a period of time with perfect knowledge / version timing of their own releases. This does not seem like valid research / publication.
- ineedasername 3y ago
  No, the methodology listed in the paper shows their efforts were significantly more extensive than you represent here. As one example, their test of prime numbers went through 500 randomly selected primes, also evaluating the chain of thought in responses along with the success rate of this classification task. I’m not sure how you arrived at your impression, but it does not match the contents of the paper that I saw with a quick scan of each section.
  - blobbers 3y ago
    Not to seem sarcastic, but 500 randomly selected primes doesn't change my mind. I read the paper, and dumbed it down to dogs and cats. Do you really feel like the efforts were significant or meaningful?
    - ineedasername 3y ago
      >I read the paper, and dumbed it down to dogs and cats. I could read the Pricicipia, or anything, and dumb it down to dogs and cats, but that reduction to the dumb would be a failing all my own. As for the test-- a classification task benchmarked with 500 examples is a fairly decent test of a system's capabilities, be it LLM or a traditional machine learning model. And again, this was only one of a variety of tasks. While it's certainly no Principia, I don't know how you get from there to dogs and cats, nor have you explained your reasoning on how you found the path between the two.
swyx 3y ago
Previous commentary I know of from OpenAI staff: Logan: The API does not just change without us telling you. The models are static there. https://twitter.com/OfficialLoganK/status/1663934947931897857 https://twitter.com/OfficialLoganK/status/166393494793189785... may 31 Peter: No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one. https://twitter.com/jlowin/status/1679660938415177731 https://twitter.com/jlowin/status/1679660938415177731 july 14 either the models are static, or they are being improved continuously and there have been unforeseen regressions. only one can be true at any point in time. was this policy changed in the last 1.5 months?
- flangola7 3y ago
  You can call a specific version of the model. It's ones of the API values. The latter person is referring to the "gpt-4" which the documentation states will update and change without warning.
- airgapstopgap 3y ago
  Not really. They have a way of squaring this circle, by changing their inference code. Speculative sampling [1] would still make their first claim a lie – sure, there'd still be the original GPT-4 model, plus a smaller draft worker. But early exit decoding [2] allows you to get almost as good results for much cheaper from exactly the same checkpoint. We know that this line of research for large-scale inference is going strong [3] so it stands to reason that OpenAI, with their wealth of talent focused on GPT-4 throughput&inference [4], large contexts and aggressive pricing policy, would also develop something like that. And of course it's "smarter" that way – in a very deceptive sense of the word. 1. https://arxiv.org/abs/2302.01318 https://arxiv.org/abs/2302.01318 2. https://arxiv.org/abs/2207.07061 https://arxiv.org/abs/2207.07061 3. https://arxiv.org/abs/2307.02628 https://arxiv.org/abs/2307.02628 4. https://openai.com/contributions/gpt-4 https://openai.com/contributions/gpt-4
  - BoorishBears 3y ago
    I don't get why you're jumping to cloak and daggers style operations: OpenAI would not kneecap their commercial offering by randomly changing how it works. At the end of the day 99% of the confusion comes from people using the web interface, which undoubtedly does change much more often than the API versions they share. The web app they host isn't a simple API wrapper, it does summarization, has some sort of system prompt, and calls the moderation API. That's undoubtedly being updated all the time.
JimmyRuska 3y ago
The pre-prompting length will grow more and more as more liabilities in the responses are uncovered. I would imagine the more the pre-prompting grows the more attention is diverted to the rules rather than the user prompt, and the less reasoning available. I wonder if they'll start using LLMs while ingesting new data. eg asking the LLM if the content is helpful, cites sources, respectful, positive, not-thin content, common or often duplicated content, etc etc, before each content import.
randomwalker 3y ago
This paper is being misinterpreted. The degradations reported are somewhat peculiar to the authors' task selection and evaluation method and can easily result from fine tuning rather than intentionally degrading GPT-4's performance for cost saving reasons. They report 2 degradations: code generation & math problems. In both cases, they report a behavior change (likely fine tuning) rather than a capability decrease (possibly intentional degradation). The paper confuses these a bit: they mostly say behavior, including in the title, but the intro says capability in a couple of places. Code generation: the change they report is that the newer GPT-4 adds non-code text to its output. They don't evaluate the correctness of the code. They merely check if the code is directly executable. So the newer model's attempt to be more helpful counted against it. Math problems (primality checking): to solve this the model needs to do chain of thought. For some weird reason, the newer model doesn't seem to do so when asked to think step by step (but the current ChatGPT-4 does, as you can easily check). The paper doesn't say that the accuracy is worse conditional on doing CoT. The other two tasks are visual reasoning and answering sensitive questions. On the former, they report a slight improvement. On the latter, they report that the filters are much more effective — unsurprising since we know that OpenAI has been heavily tweaking these. In short, everything in the paper is consistent with fine tuning. It is possible that OpenAI is gaslighting everyone by denying that they degraded performance for cost saving purposes — but if so, this paper doesn't provide evidence of it. Still, it's a fascinating study of the unintended consequences of model updates.
- AbrahamParangi 3y ago
  In my opinion the more likely thing is that OpenAI is gaslighting people that the finetuning is improving the model when it likely mostly improves safety at some cost to capability. I'd bet this is measured against a set of evals and it looks like it performs well BUT I'd also bet the evals are asymmetrically good at detecting "unsafe" or jailbreak behavior and bad at detecting reduced general cognitive flexibility. The obvious avenue to degradation is that the "HR personality" is much more strictly applied and the resistance to being jailbroken is also in some sense an inability to think.
  - kuchenbecker 3y ago
s3p 3y ago
Commented this on a submission that was a duplicate of this one, but: When GPT's details were allegedly leaked [0] I read about the mixture of experts and wondered if this explains my recent distaste for GPT. Lately I have been using Anthropic's offering (mainly since it is free and has a 100k context window) and I've been surprised at just how well it reasons and understands what I'm asking. It responds like GPT 4 used to, and speaks to me with nuance I haven't seen since Bing released their original chatbot. I still find GPT 4 better if I want to fine-tune the model and make it adopt a persona-- Claude will almost always refuse. Either way, I'm a bit confused with the way GPT 4 has been changing over time. It seems the team made significant changes to the model quality in favor of performance. Whether accuracy or performance is more important is up for debate, but something is clearly changing. [0] https://archive.ph/2RQ8X https://archive.ph/2RQ8X
- swyx 3y ago
  reminder that that "leak" was by a twitter grifter that accessed and reposted wholesale without attribution the contents of a paid newsletter and had the gall to chargeback the credit card. dont encourage him or link to him.
barrkel 3y ago
Irritatingly, OpenAI reps deny any change in model capabilities over time. It's more likely that as the models are optimized for cost and performance, their in-house evaluation metrics don't cover everything customers are interested in. Meanwhile, the probabilistic nature of LLM output means there is plausible deniability.
- NLPaep 3y ago
  Source? They’ve only denied the API model changing. Not the website.
  - barrkel 3y ago
    https://twitter.com/npew/status/1679538687854661637?t=3CLXlTzccRu-NthzF9ZXOw&s=19 https://twitter.com/npew/status/1679538687854661637?t=3CLXlT...
    - blowski 3y ago
      (Tweet contents) > No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one. > Current hypothesis: When you use it more heavily, you start noticing issues you didn't see before. I don’t see how it supports your argument. Your comment says “they deny making changes to GPT-4”, and the tweet says “we are making incremental improvements to GPT-4”.
      - esperent 3y ago
        It's important to read company marketing statements as if you were a lawyer. We haven't made GPT-4 dumber != We haven't made ChatGPT(4) dumber. Personal hypothesis is that they have made a few changes to ChatGPT recently - possibly quantization, and almost certainly some tweaks to make it give shorter/less detailed answers. But by the nature of a probibalistic tool being run by a secretive company, it's hard to say for sure. Maybe I and everyone else complaining have just started to get unlucky answers.
deleted 3y ago
[deleted]
- blowski 3y ago
  Hyperbole, much? Are we at that stage of the hype curve already?
metanonsense 3y ago
With those GPT4 architecture infos leaked a few weeks ago, my personal theory is that they use mixture-of-experts routing for scaling the system. For instance, they could say: from 0 to 50% system load use a mixture of top-6 models for inference, from 60 to 80% use top-4 models, above 80% use top-2 models. This could make ChatGPT look dumber or smarter depending on the time of the day and system usage. Naturally, the variance of inference quality would spread and people would experience regressions in quality with some probability. They could also use these parameters over time to become more cost efficient, in general, or reserve GPUs (i.e. allocating more expert networks) for some higher-margin use-cases.
- metanonsense 3y ago
  They could even persist the inference parameterization, e.g. "use top-8 for 'compute the sum of 42 and 17'", for every prompt ever seen and reuse it later to prevent people doing "poor man's regression testing" and make the model look more stable.
- ineedasername 3y ago
  I’m guessing that some tasks relegated to a single one of those experts are tasks that previously benefited from those experts’ capabilities being integrated in a single model. Sort of like having a problem and then asking both a biologist & a chemist for their assessment. But if asked a single biochemist they could more readily provide an answer that synthesized both more comprehensively, even if you got both the biologist and the chemist in the same room with each other.
spiritplumber 3y ago
And even the lecturers acquiesced when they found that a lecture on the sea was none the less stimulating when compiled out of other lectures that had already been delivered on the same subject. “Beware of first-hand ideas!” exclaimed one of the most advanced of them. “First-hand ideas do not really exist. They are but the physical impressions produced by love and fear, and on this gross foundation who could erect a philosophy? Let your ideas be second-hand, and if possible tenth-hand, for then they will be far removed from that disturbing element — direct observation. Do not learn anything about this subject of mine — the French Revolution. Learn instead what I think that Enicharmon thought Urizen thought Gutch thought Ho-Yung thought Chi-Bo-Sing thought Lafcadio Hearn thought Carlyle thought Mirabeau said about the French Revolution. Through the medium of these ten great minds, the blood that was shed at Paris and the windows that were broken at Versailles will be clarified to an idea which you may employ most profitably in your daily lives. But be sure that the intermediates are many and varied, for in history one authority exists to counteract another. Urizen must counteract the scepticism of Ho-Yung and Enicharmon, I must myself counteract the impetuosity of Gutch. You who listen to me are in a better position to judge about the French Revolution than I am. Your descendants will be even in a better position than you, for they will learn what you think I think, and yet another intermediate will be added to the chain. And in time” — his voice rose — “there will come a generation that had got beyond facts, beyond impressions, a generation absolutely colourless, a generation ‘seraphically free From taint of personality,’ which will see the French Revolution not as it happened, nor as they would like it to have happened, but as it would have happened, had it taken place in the days of the Machine.” E M Forster, "The Machine Stops", 1909
high_5 3y ago
Since the system appears to be dynamic to certain extent (beyond training) this will be a permanent problem. The feedback that large/public GPT systems are given and retrained upon will cause ups and downs, depending on the mood of the society. But I believe this entanglement of people dependent on LLMs and vice versa will go pear shaped until full Idiocracy scenario is reached. As George Carlin supposedly said "never underestimate the power of stupid people in large groups".
Aulig 3y ago
Just yesterday I gave ChatGPT a summarization task and it performed horribly. I even tried multiple times and got the identical answer. Then I gave the identical prompt to gpt-3.5-turbo via the API and I immediately got the expected good answer.
- nomel 3y ago
  Which gpt-3.5-turbo API? Legacy completion or chat?
  - gopikl 3y ago
    Turbo is always chat.
dontupvoteme 3y ago
one things for sure, it really does not want to output entire code blocks now, opting heavily to just give the delta.
courseofaction 3y ago
OpenAI just lie through their teeth lol.
ChemSpider 3y ago
For _medical_ questions, the public version of ChatGPT has become useless. Even a simple question such as "What are the typical side effects of XYZ" is answered with a gigantic disclaimer and then often a fancy version of "I am not sure". Yeah, that much I knew already. It used to be better in the past.
pknerd 3y ago
Am I the only lucky person who never invested a dime for GPT4?
amelius 3y ago
How is Copilot's behavior changing over time? And has anyone successfully used Copilot to do simple but tedious things like generate library bindings for various languages? I suspect that Copilot would be good at this kind of thing because it is a relatively simple task and there is a lot of training-code available.
- AndroidKitKat 3y ago
  I use Copilot primarily as a tab-complete bot, essentially only accepting its input if that's what I was going to type. It is correct mayb ~95% of the time, but depending on the programming language it will try to suggest a large block of code that I don't really trust (in my usage, it loves to spit out Python but in my Java codebase it sticks to boilerplate or finishing my line for me).
DrNosferatu 3y ago
Could they just be moving to a more aggressive quantization so they can save money? PS: Which is obviously ripping off paying users. Competition is the answer.
AbrahamParangi 3y ago
The main thing that ChatGPT has gotten better at is rejecting jailbreaks and refusing to go off the reservation. It has been demonstrated that "safety" trades off against "capability". I'm sure OpenAI has evaluations that demonstrate the improvements they've been making to "safety" have not come at the cost of capability, but I'd bet those evaluations are wrong (by being insufficient). It also wouldn't surprise me if the tradeoff between "safety" and capability is just intrinsic. You can have your model talk like HR or think like a mad scientist, but not both equally well.
- spaceman_2020 3y ago
  I've cancelled my pro subscription. You can't ask me to pay money and also expect me to spend hours Googling around to find prompts for basic queries. I would also like to not be treated like an idiot. Every time I query anything related to health or medicine, chatGPT will give me a short generic answer, then add two paragraphs of warnings about how I should just seek a medical professional's help. As if I'm the kind of moron who will blindly follow whatever some AI tool tells me and not actually go to a doctor if there's something wrong with me. This organization seems like it's being run by (scared) lawyers.
  - copperx 3y ago
    I'm nostalgic about the first public version of GPT-4. It did have some sparks of AGI. The interface was very ergonomic because it assumed correctly many things about the user's request and intentions. Now I find I have to write very long prompts and explanations to steer it correctly, as if training a brand new employee. It's really tiring. And I'm not even talking about the accuracy of the responses. It's sad, but I understand why they did it. It was too much power in the hands of people.
T3RMINATED 3y ago
[dead]
tagyro 3y ago
I read the paper and while I agree that maybe math is not GPT-4 strongest point, I have noticed the same degradation in quality. And as I know there will be many commenters asking, here's my experience. Four months ago I released a mobile app that wraps the "Open"AI API and allows a more private use. The app also includes 15 domains with over 150+ editable prompts, specifically crafted to help users get the most out of GPT. I initially crafted these prompts in English but given I wanted to allow also non-English speakers to make use of them, I started to translate them in Italian and German. I wrote a small script that took each English prompt and, with some more prompt-fu, translated them to these languages, using GPT-4. As I'm fairly fluent in both Italian and German, I was able to verify the quality of these translations. Well, around end of May, when I first noticed some weird answers from GPT, I ran my script again and (surprise!) the quality of the translations is visible inferior. On a different note, I noticed that from one day to the other, any prompt would get an empty reply in the app, only to discover that "Open"AI has also made a subtle change in the json format of the API response that made broke the parser in the app. Admittedly, 150 phrases is not a huge sample size and I could/should have used the default json parser. I don't trust "Open"AI and will, as soon as it's feasible, change to a different model.
- nmfisher 3y ago
  This tracks with my experience too. For the past few years, I've been using "give me the pinyin for these Chinese characters" as a rough guide for how well a given model would perform. When GPT4 was released, it absolutely nailed this task; every word was 100% correct, every single time. For reference, even GPT3.5 wasn't great at this. I repeated this last week and it's still getting the phonetic content right, it's now often getting the tones wrong (something which I literally never saw on release). Perhaps changing the prompt would deliver the correct results, but that's not really the point. Something has pretty clearly changed such that old prompts no longer delivering the same results.
python273 3y ago
https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim... > The math questions were of the form “Is 17077 prime”? They picked 500 numbers, but all of them were prime! > The June version of GPT-3.5 and the March version of GPT-4 almost always conclude that the number is prime regardless of whether it is prime or composite. The other two models do the opposite. But the paper only tested prime numbers, and hence concluded that GPT-3.5’s performance improved while GPT-4’s degraded.

289 points by tim_sw 3y ago

It's "Open" AI ;-)

Never let OpenAI or Google employees (in regards to search) gaslight you into believing that things aren't being "enshittified". Glad to see scholarly evidence of this coming out.

EGreg 3y ago
Profit motive corrupts Absolute profit motive corrupts absolutely
quickthrower2 3y ago
Honest enshittification is good but rare. Like a cloud telling you what specific CPU SKU is in the hardware (not just 1 vcore). They should say what you are paying for in terms of technicals, either spill the beans on the architecture or have some SLAs on how good the thing is.
- opportune 3y ago
  I work in cloud-land, the problem with exposing every little technical detail is that 1. part of the point of purchasing a cloud product is to abstract that away 2. coding to the implementation instead of the API creates huge headaches for everybody because there are often good reasons (make things more efficient) to change the implementation 3. sadly, there are “features” such as overcommit which you don’t want to know about as a customer. While some of those are anti-customer on their face, the “we shouldn’t commit to implementation details and instead just to our API/feature surface” may seem anti-customer while actually making things much better for customers. When implementations are supported in perpetuity development grinds to a standstill. Even overcommit ends up helping customers - cloud companies can offer lower sticker prices. In the case of both regular Public cloud and OpenAI - if you need stability beyond publicly supported APIs you are probably not a good fit as a customer and should instead find a company willing to commit to lower level implementations (eg bare metal, truly major/minor versioned software) or roll-your-own.
  - quickthrower2 3y ago
    That is fine, then you need some kinds of comparable metric that stays the same over time. FLOPS maybe? Ideally the same across clouds, or more likely cloud-specific. Or even re pin it every 5 years. E.g. a 2020-CPU, 2025-CPU etc. We need a "horse power" for the cloud I guess!

deleted 3y ago

[deleted]

svaha1728 3y ago

>June version added “‘python and “‘ before and after the code snippet. Second, it also generated a few more comments. While a small change, the extra triple quotes render the code not executable. These are to represent markdown code blocks, which probably helped their front end developers, but hinders copy-pasta

QuantumG 3y ago
Outputting that on the API is kinda dumb, but yeah, hardly something worth publishing.

dudeinhawaii 3y ago

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quotes). I'd be more interested in changes to the quality and performance of the code generated, not whether it can be easily copy/pasted from the page. I could not replicate that result (but could replicate the other math issues). I've also never seen the triple quotes in results so it's unclear if there was a temporary presentation bug.

sanxiyn 3y ago
Or OpenAI can stop being stupid and adopt LLaMA-like tokenization, which special cases numbers and tokenize them into individual digits.
- jinay 3y ago
  Are LLaMA-based models better at math because of this digit-based tokenization?
- bugglebeetle 3y ago
  Isn’t the tokenization tied to the training of the model?
  - colobas 3y ago
    The model learns an embedding table, where (roughly) each row is used as the model’s internal representation of each token. The numbers in that table are learned. What isn’t learned is the map from token (i.e. combination of characters/byte-pairs) to row-index in the embedding table. That is given by tokenization EDIT: removed redundant bit
ehnto 3y ago
I think knowing if the code can be used verbatim is actually the more important part practically speaking. That is the actually useful part. Quality is important to humans, because humans have to read it, but correctness is what people using ChatGPT for code actually need. So long as the quality and performance is good enough, then it will be useful. Performance is such a nuanced topic that you need very context aware devs anyway, and I think a general purpose LLM is never going to have that kind of awareness.

ramesh31 3y ago

OpenAI is going to be left in the dust by (actual) open models. Llama 2 is already reaching GPT-3 levels, and can run inference on consumer hardware. Crazy how fast that flipped.

nomel 3y ago
What enables this? There's a huge gap between GPT-3.5 and 4, put there by a massive amount of money, from my understanding. To compete, with open source projects being less well funded, I would assume that orders of magnitude improvements in training cost would be required. What do you see driving this, and who do you see paying for it? If Meta, or anyone else, gets something that beats GPT-4, I would naively assume they would monetize it. If someopen source effort manages to beat GPT-4, then I assume some well funded, profit seeking, entity will dump orders of magnitude more money into whatever enabled the open source offerings to compete. I don't think GPT-4 is the pinnacle of OpenAI, or that their funding will run dry, so they shouldn't be viewed as a static entity.
- ramesh31 3y ago
  >Or, are you suggesting that GPT-4 is the pinnacle of OpenAI, or that their funding will run dry? My bet is that Meta has pivoted almost entirely to this space with their R&D in the last six months. Llama 2 is spectacular. And with its' success, there will undoubtedly be more. They also happen to have access to limitless amounts of compute, cash, and engineering that puts OpenAI to shame. This could finally be their chance to create a platform for real. And the open source community and startups will benefit off of that.
  - deleted 3y ago
    [deleted]
  - nomel 3y ago
    What motive do you see for them releasing these models for free, especially with the massive increase in cost associated with catching up?

ttul 3y ago

I tried the prime number problem and GPT-4 nailed it. I’m not sure whether they are testing things correctly… “ Sure, let's go step by step. A prime number is a number greater than 1 that has no positive divisors other than 1 and itself. This means if we can find any other number (excluding 1 and the number itself) that divides 17077, then it is not a prime number. Let's start by checking divisibility by 2. Since 17077 is an odd number (its last digit is 7), it is not divisible by 2. For the next step, we can look at divisibility by 3. We add up the digits: 1+7+0+7+7 = 22. Because 22 is not divisible by 3, 17077 is also not divisible by 3. The smallest prime number greater than 3 is 5. Checking divisibility by 5, we see that 17077 does not end in 0 or 5, so it is not divisible by 5. The smallest prime number greater than 5 is 7. To check divisibility by 7, there is no simple rule like with 2, 3, and 5, so we would perform the actual division. But we can stop here because I just recalled a rule that we can only check up to the square root of the number and the square root of 17077 is approximately 130.68. So if there were factors other than 1 and 17077, we would have found them by now. Therefore, 17077 is a prime number.”

moonchild 3y ago
> GPT-4 nailed it Did it? Did you read what it wrote?
- kevinventullo 3y ago
  It certainly nails the cadence of a rigorous proof. It just needs to learn to append “the rest is left as an exercise to the reader” to the end of its output.
  - deleted 3y ago
    [deleted]
- pedrosorio 3y ago
  I’ve seen so many of these on Twitter since chat GPT came out. > Look at this amazing capability, it gets it perfectly! (screenshot of interaction with clear flaws/outright wrong answer)

blobbers 3y ago

A little surprised this sort of thing passes for a publish-able paper. Isn't it the equivalent to saying "Here's the top 10 results for google searching golden retrievers March 2023, and here's the top 10 results from June 2023. We see that google is returning even cuter animals today. Unfortunately though, one of the results linked to a page full of cats." I'm sure openai has a list of standard questions that it tracks the responses it is getting over a period of time with perfect knowledge / version timing of their own releases. This does not seem like valid research / publication.

ineedasername 3y ago
No, the methodology listed in the paper shows their efforts were significantly more extensive than you represent here. As one example, their test of prime numbers went through 500 randomly selected primes, also evaluating the chain of thought in responses along with the success rate of this classification task. I’m not sure how you arrived at your impression, but it does not match the contents of the paper that I saw with a quick scan of each section.
- blobbers 3y ago
  Not to seem sarcastic, but 500 randomly selected primes doesn't change my mind. I read the paper, and dumbed it down to dogs and cats. Do you really feel like the efforts were significant or meaningful?
  - ineedasername 3y ago
    >I read the paper, and dumbed it down to dogs and cats. I could read the Pricicipia, or anything, and dumb it down to dogs and cats, but that reduction to the dumb would be a failing all my own. As for the test-- a classification task benchmarked with 500 examples is a fairly decent test of a system's capabilities, be it LLM or a traditional machine learning model. And again, this was only one of a variety of tasks. While it's certainly no Principia, I don't know how you get from there to dogs and cats, nor have you explained your reasoning on how you found the path between the two.

swyx 3y ago

Previous commentary I know of from OpenAI staff: Logan: The API does not just change without us telling you. The models are static there. https://twitter.com/OfficialLoganK/status/1663934947931897857 https://twitter.com/OfficialLoganK/status/166393494793189785... may 31 Peter: No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one. https://twitter.com/jlowin/status/1679660938415177731 https://twitter.com/jlowin/status/1679660938415177731 july 14 either the models are static, or they are being improved continuously and there have been unforeseen regressions. only one can be true at any point in time. was this policy changed in the last 1.5 months?

flangola7 3y ago
You can call a specific version of the model. It's ones of the API values. The latter person is referring to the "gpt-4" which the documentation states will update and change without warning.
airgapstopgap 3y ago
Not really. They have a way of squaring this circle, by changing their inference code. Speculative sampling [1] would still make their first claim a lie – sure, there'd still be the original GPT-4 model, plus a smaller draft worker. But early exit decoding [2] allows you to get almost as good results for much cheaper from exactly the same checkpoint. We know that this line of research for large-scale inference is going strong [3] so it stands to reason that OpenAI, with their wealth of talent focused on GPT-4 throughput&inference [4], large contexts and aggressive pricing policy, would also develop something like that. And of course it's "smarter" that way – in a very deceptive sense of the word. 1. https://arxiv.org/abs/2302.01318 https://arxiv.org/abs/2302.01318 2. https://arxiv.org/abs/2207.07061 https://arxiv.org/abs/2207.07061 3. https://arxiv.org/abs/2307.02628 https://arxiv.org/abs/2307.02628 4. https://openai.com/contributions/gpt-4 https://openai.com/contributions/gpt-4
- BoorishBears 3y ago
  I don't get why you're jumping to cloak and daggers style operations: OpenAI would not kneecap their commercial offering by randomly changing how it works. At the end of the day 99% of the confusion comes from people using the web interface, which undoubtedly does change much more often than the API versions they share. The web app they host isn't a simple API wrapper, it does summarization, has some sort of system prompt, and calls the moderation API. That's undoubtedly being updated all the time.

JimmyRuska 3y ago

The pre-prompting length will grow more and more as more liabilities in the responses are uncovered. I would imagine the more the pre-prompting grows the more attention is diverted to the rules rather than the user prompt, and the less reasoning available. I wonder if they'll start using LLMs while ingesting new data. eg asking the LLM if the content is helpful, cites sources, respectful, positive, not-thin content, common or often duplicated content, etc etc, before each content import.

randomwalker 3y ago

This paper is being misinterpreted. The degradations reported are somewhat peculiar to the authors' task selection and evaluation method and can easily result from fine tuning rather than intentionally degrading GPT-4's performance for cost saving reasons. They report 2 degradations: code generation & math problems. In both cases, they report a behavior change (likely fine tuning) rather than a capability decrease (possibly intentional degradation). The paper confuses these a bit: they mostly say behavior, including in the title, but the intro says capability in a couple of places. Code generation: the change they report is that the newer GPT-4 adds non-code text to its output. They don't evaluate the correctness of the code. They merely check if the code is directly executable. So the newer model's attempt to be more helpful counted against it. Math problems (primality checking): to solve this the model needs to do chain of thought. For some weird reason, the newer model doesn't seem to do so when asked to think step by step (but the current ChatGPT-4 does, as you can easily check). The paper doesn't say that the accuracy is worse conditional on doing CoT. The other two tasks are visual reasoning and answering sensitive questions. On the former, they report a slight improvement. On the latter, they report that the filters are much more effective — unsurprising since we know that OpenAI has been heavily tweaking these. In short, everything in the paper is consistent with fine tuning. It is possible that OpenAI is gaslighting everyone by denying that they degraded performance for cost saving purposes — but if so, this paper doesn't provide evidence of it. Still, it's a fascinating study of the unintended consequences of model updates.

AbrahamParangi 3y ago
In my opinion the more likely thing is that OpenAI is gaslighting people that the finetuning is improving the model when it likely mostly improves safety at some cost to capability. I'd bet this is measured against a set of evals and it looks like it performs well BUT I'd also bet the evals are asymmetrically good at detecting "unsafe" or jailbreak behavior and bad at detecting reduced general cognitive flexibility. The obvious avenue to degradation is that the "HR personality" is much more strictly applied and the resistance to being jailbroken is also in some sense an inability to think.
- kuchenbecker 3y ago

s3p 3y ago

Commented this on a submission that was a duplicate of this one, but: When GPT's details were allegedly leaked [0] I read about the mixture of experts and wondered if this explains my recent distaste for GPT. Lately I have been using Anthropic's offering (mainly since it is free and has a 100k context window) and I've been surprised at just how well it reasons and understands what I'm asking. It responds like GPT 4 used to, and speaks to me with nuance I haven't seen since Bing released their original chatbot. I still find GPT 4 better if I want to fine-tune the model and make it adopt a persona-- Claude will almost always refuse. Either way, I'm a bit confused with the way GPT 4 has been changing over time. It seems the team made significant changes to the model quality in favor of performance. Whether accuracy or performance is more important is up for debate, but something is clearly changing. [0] https://archive.ph/2RQ8X https://archive.ph/2RQ8X

swyx 3y ago
reminder that that "leak" was by a twitter grifter that accessed and reposted wholesale without attribution the contents of a paid newsletter and had the gall to chargeback the credit card. dont encourage him or link to him.

barrkel 3y ago

Irritatingly, OpenAI reps deny any change in model capabilities over time. It's more likely that as the models are optimized for cost and performance, their in-house evaluation metrics don't cover everything customers are interested in. Meanwhile, the probabilistic nature of LLM output means there is plausible deniability.

NLPaep 3y ago
Source? They’ve only denied the API model changing. Not the website.
- barrkel 3y ago
  https://twitter.com/npew/status/1679538687854661637?t=3CLXlTzccRu-NthzF9ZXOw&s=19 https://twitter.com/npew/status/1679538687854661637?t=3CLXlT...
  - blowski 3y ago
    (Tweet contents) > No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one. > Current hypothesis: When you use it more heavily, you start noticing issues you didn't see before. I don’t see how it supports your argument. Your comment says “they deny making changes to GPT-4”, and the tweet says “we are making incremental improvements to GPT-4”.
    - esperent 3y ago
      It's important to read company marketing statements as if you were a lawyer. We haven't made GPT-4 dumber != We haven't made ChatGPT(4) dumber. Personal hypothesis is that they have made a few changes to ChatGPT recently - possibly quantization, and almost certainly some tweaks to make it give shorter/less detailed answers. But by the nature of a probibalistic tool being run by a secretive company, it's hard to say for sure. Maybe I and everyone else complaining have just started to get unlucky answers.

deleted 3y ago

[deleted]

blowski 3y ago
Hyperbole, much? Are we at that stage of the hype curve already?

metanonsense 3y ago

With those GPT4 architecture infos leaked a few weeks ago, my personal theory is that they use mixture-of-experts routing for scaling the system. For instance, they could say: from 0 to 50% system load use a mixture of top-6 models for inference, from 60 to 80% use top-4 models, above 80% use top-2 models. This could make ChatGPT look dumber or smarter depending on the time of the day and system usage. Naturally, the variance of inference quality would spread and people would experience regressions in quality with some probability. They could also use these parameters over time to become more cost efficient, in general, or reserve GPUs (i.e. allocating more expert networks) for some higher-margin use-cases.

metanonsense 3y ago
They could even persist the inference parameterization, e.g. "use top-8 for 'compute the sum of 42 and 17'", for every prompt ever seen and reuse it later to prevent people doing "poor man's regression testing" and make the model look more stable.
ineedasername 3y ago
I’m guessing that some tasks relegated to a single one of those experts are tasks that previously benefited from those experts’ capabilities being integrated in a single model. Sort of like having a problem and then asking both a biologist & a chemist for their assessment. But if asked a single biochemist they could more readily provide an answer that synthesized both more comprehensively, even if you got both the biologist and the chemist in the same room with each other.

spiritplumber 3y ago

And even the lecturers acquiesced when they found that a lecture on the sea was none the less stimulating when compiled out of other lectures that had already been delivered on the same subject. “Beware of first-hand ideas!” exclaimed one of the most advanced of them. “First-hand ideas do not really exist. They are but the physical impressions produced by love and fear, and on this gross foundation who could erect a philosophy? Let your ideas be second-hand, and if possible tenth-hand, for then they will be far removed from that disturbing element — direct observation. Do not learn anything about this subject of mine — the French Revolution. Learn instead what I think that Enicharmon thought Urizen thought Gutch thought Ho-Yung thought Chi-Bo-Sing thought Lafcadio Hearn thought Carlyle thought Mirabeau said about the French Revolution. Through the medium of these ten great minds, the blood that was shed at Paris and the windows that were broken at Versailles will be clarified to an idea which you may employ most profitably in your daily lives. But be sure that the intermediates are many and varied, for in history one authority exists to counteract another. Urizen must counteract the scepticism of Ho-Yung and Enicharmon, I must myself counteract the impetuosity of Gutch. You who listen to me are in a better position to judge about the French Revolution than I am. Your descendants will be even in a better position than you, for they will learn what you think I think, and yet another intermediate will be added to the chain. And in time” — his voice rose — “there will come a generation that had got beyond facts, beyond impressions, a generation absolutely colourless, a generation ‘seraphically free From taint of personality,’ which will see the French Revolution not as it happened, nor as they would like it to have happened, but as it would have happened, had it taken place in the days of the Machine.” E M Forster, "The Machine Stops", 1909

high_5 3y ago

Since the system appears to be dynamic to certain extent (beyond training) this will be a permanent problem. The feedback that large/public GPT systems are given and retrained upon will cause ups and downs, depending on the mood of the society. But I believe this entanglement of people dependent on LLMs and vice versa will go pear shaped until full Idiocracy scenario is reached. As George Carlin supposedly said "never underestimate the power of stupid people in large groups".

Aulig 3y ago

Just yesterday I gave ChatGPT a summarization task and it performed horribly. I even tried multiple times and got the identical answer. Then I gave the identical prompt to gpt-3.5-turbo via the API and I immediately got the expected good answer.

nomel 3y ago
Which gpt-3.5-turbo API? Legacy completion or chat?
- gopikl 3y ago
  Turbo is always chat.

dontupvoteme 3y ago

one things for sure, it really does not want to output entire code blocks now, opting heavily to just give the delta.

courseofaction 3y ago

OpenAI just lie through their teeth lol.

ChemSpider 3y ago

For _medical_ questions, the public version of ChatGPT has become useless. Even a simple question such as "What are the typical side effects of XYZ" is answered with a gigantic disclaimer and then often a fancy version of "I am not sure". Yeah, that much I knew already. It used to be better in the past.

pknerd 3y ago

Am I the only lucky person who never invested a dime for GPT4?

amelius 3y ago

How is Copilot's behavior changing over time? And has anyone successfully used Copilot to do simple but tedious things like generate library bindings for various languages? I suspect that Copilot would be good at this kind of thing because it is a relatively simple task and there is a lot of training-code available.

AndroidKitKat 3y ago
I use Copilot primarily as a tab-complete bot, essentially only accepting its input if that's what I was going to type. It is correct mayb ~95% of the time, but depending on the programming language it will try to suggest a large block of code that I don't really trust (in my usage, it loves to spit out Python but in my Java codebase it sticks to boilerplate or finishing my line for me).

DrNosferatu 3y ago

Could they just be moving to a more aggressive quantization so they can save money? PS: Which is obviously ripping off paying users. Competition is the answer.

AbrahamParangi 3y ago

The main thing that ChatGPT has gotten better at is rejecting jailbreaks and refusing to go off the reservation. It has been demonstrated that "safety" trades off against "capability". I'm sure OpenAI has evaluations that demonstrate the improvements they've been making to "safety" have not come at the cost of capability, but I'd bet those evaluations are wrong (by being insufficient). It also wouldn't surprise me if the tradeoff between "safety" and capability is just intrinsic. You can have your model talk like HR or think like a mad scientist, but not both equally well.

spaceman_2020 3y ago
I've cancelled my pro subscription. You can't ask me to pay money and also expect me to spend hours Googling around to find prompts for basic queries. I would also like to not be treated like an idiot. Every time I query anything related to health or medicine, chatGPT will give me a short generic answer, then add two paragraphs of warnings about how I should just seek a medical professional's help. As if I'm the kind of moron who will blindly follow whatever some AI tool tells me and not actually go to a doctor if there's something wrong with me. This organization seems like it's being run by (scared) lawyers.
- copperx 3y ago
  I'm nostalgic about the first public version of GPT-4. It did have some sparks of AGI. The interface was very ergonomic because it assumed correctly many things about the user's request and intentions. Now I find I have to write very long prompts and explanations to steer it correctly, as if training a brand new employee. It's really tiring. And I'm not even talking about the accuracy of the responses. It's sad, but I understand why they did it. It was too much power in the hands of people.

T3RMINATED 3y ago

[dead]

tagyro 3y ago

I read the paper and while I agree that maybe math is not GPT-4 strongest point, I have noticed the same degradation in quality. And as I know there will be many commenters asking, here's my experience. Four months ago I released a mobile app that wraps the "Open"AI API and allows a more private use. The app also includes 15 domains with over 150+ editable prompts, specifically crafted to help users get the most out of GPT. I initially crafted these prompts in English but given I wanted to allow also non-English speakers to make use of them, I started to translate them in Italian and German. I wrote a small script that took each English prompt and, with some more prompt-fu, translated them to these languages, using GPT-4. As I'm fairly fluent in both Italian and German, I was able to verify the quality of these translations. Well, around end of May, when I first noticed some weird answers from GPT, I ran my script again and (surprise!) the quality of the translations is visible inferior. On a different note, I noticed that from one day to the other, any prompt would get an empty reply in the app, only to discover that "Open"AI has also made a subtle change in the json format of the API response that made broke the parser in the app. Admittedly, 150 phrases is not a huge sample size and I could/should have used the default json parser. I don't trust "Open"AI and will, as soon as it's feasible, change to a different model.

nmfisher 3y ago
This tracks with my experience too. For the past few years, I've been using "give me the pinyin for these Chinese characters" as a rough guide for how well a given model would perform. When GPT4 was released, it absolutely nailed this task; every word was 100% correct, every single time. For reference, even GPT3.5 wasn't great at this. I repeated this last week and it's still getting the phonetic content right, it's now often getting the tones wrong (something which I literally never saw on release). Perhaps changing the prompt would deliver the correct results, but that's not really the point. Something has pretty clearly changed such that old prompts no longer delivering the same results.

python273 3y ago

https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim... > The math questions were of the form “Is 17077 prime”? They picked 500 numbers, but all of them were prime! > The June version of GPT-3.5 and the March version of GPT-4 almost always conclude that the number is prime regardless of whether it is prime or composite. The other two models do the opposite. But the paper only tested prime numbers, and hence concluded that GPT-3.5’s performance improved while GPT-4’s degraded.