5 ms·

> Compare with drug trials: Adderall only differs from regular amphetamine in the relative concentration of enantiomers, and the entire value of the drug is in

> Compare with drug trials: Adderall only differs from regular amphetamine in the relative concentration of enantiomers, and the entire value of the drug is in the measurements. Drug trials may be expected to be somewhat reproducible. What I don't get is how it can even be called research if it cannot be expected to be reproducible at all! GPT is a closed source/weights, proprietary product that changes every couple of weeks or so. How can you expect a prompt to do the same for a reasonable length of time for the research to be even rudimentarily reproducible? And if it's not reproducible, what is it actually worth? I don't think much. Could as well have been a fault in the research setup or a fake.

IanCal 3y ago
> GPT is a closed source/weights, proprietary product that changes every couple of weeks or so. Do you have any evidence that the weights for versioned models are being changed without notifications?
- starbugs 3y ago
  > Do you have any evidence that the weights for versioned models are being changed without notifications? I think in a real scientific process, it's upon those who claim that they are not to provide the evidence.
  - IanCal 3y ago
    I'm sorry, but that's entirely ridiculous. You're mangling up a concept of burden of proof here. You can easily see this because it can be flipped around easily - you made a claim that they are being changed, even every few weeks! Should it really be on me to show that your very specific claim is false? Aside - but even if the model weights did change, that wouldn't stop research being possible. Otherwise no drug trial could be replicated because you couldn't get the exact same participants at the exact same age.
    - dartos 3y ago
      Apples and oranges comparison. You couldn’t get the same participants, but you could get the same drugs. If you could get identical participants, that wouldn’t be very helpful since humans are so varied. But for GPT based papers, what you’re actually testing could change without you knowing. There’s no way to know if a paper is reproducible at all. If you can’t reproduce results, is it really research, or just show and tell?
      - mewpmewp2 3y ago
        With API you can choose versions fixed to a date. Are you suggesting that OpenAI is lying about these being fixed to a date? Why would they lie about it? The whole point of these versions is so that when you build on top of that it would keep working as you expect.
        hnfong 3y ago
        I'm not saying they lie about it, but as a hypothetical there could be many reasons to lie. - realizing their model leaks confidential information against malicious prompts - copyright claims against them forcing them to remove bits of data from the training set - serious "alignment" bugs that need to be fixed - vastly improved optimization techniques that slightly affect results in 0.1% of the cases If updating the model would save the company a couple hundred million dollars, they might want to do it. And in some of the cases, I can imagine they have an incentive to keep the update low key.
      - IanCal 3y ago
        > If you can’t reproduce results, is it really research, or just show and tell? You can't start with a statement about clinical trials not being perfectly reproducible and that's fine, then say this. > what you’re actually testing could change without you knowing If people are lying about an extremely important part of their product, which they have little reason to. But then this applies to pretty much everything. Starting with the assumption that people are lying about everything and nothing is as it seems may technically make things more reproducible but it's going to require unbelievable effort for very little return. > There’s no way to know if a paper is reproducible at all. This is a little silly because these models are available extremely easily and at a pay-as-you-go pricing. And again, it requires an assumption that openai is lying about a specific feature of a product.
awestroke 3y ago
You can select a static snapshot that presumably does not change, if you use the API
- nradov 3y ago
  Is that a contractual guarantee, or more of a "trust us" kind of thing?
- starbugs 3y ago
  > You can select a static snapshot that presumably does not change, if you use the API Sorry, I won't blindly believe a company who are cynical enough to call themselves "OpenAI", then publish a commercial closed source/weights model for profit. Evidence that they do not change without notice or it didn't happen. Better even, provide the source and weights for research purposes. These models could be pulled at every instant if the company sees fit or ceases to exist.
  - cqqxo4zV46cp 3y ago
    Yeah, here it comes. In these conversations you don’t need to ask very many “why”s before it just turns out that the antagonist (you) has an axe to grind about OpenAI, and has added that the their misplaced sense of expertise with regard to the typical standards of proof in academic publications.
    - starbugs 3y ago
      > Yeah, here it comes. In these conversations you don’t need to ask very many “why”s before it just turns out that the antagonist (you) has an axe to grind about OpenAI, and has added that the their misplaced sense of expertise with regard to the typical standards of proof in academic publications. Seems to have hit hard? I would find it borderline acceptable being offended by a user whose name has obviously been generated using a password generator if you could at least provide some substance to the discussion. Just labeling someone and questioning their competence based on your hurt feelings is a bit low. Please improve.
sebzim4500 3y ago
If I book telescope time and capture a supernova then no one will ever be able to reproduce my raw results because it has already happened. I don't see why OpenAI pulling old model snapshots is any different.
- kjkjadksj 3y ago
  But you know the parameters of your telescope at least. If openai wants to update all the time, fine, but then they should be like how every other piece of research software works, where you can list what exact version of software you used and pull that version yourself if need be.
  - IanCal 3y ago
    Stability is the purpose of the versioned models.
- starbugs 3y ago
  > If I book telescope time and capture a supernova then no one will ever be able to reproduce my raw results because it has already happened. I don't see why OpenAI pulling old model snapshots is any different. That's why you capture multiple of them and verify your data statistically?
  - sebzim4500 3y ago
    And ideally if someone is proposing new prompting techniques they should test it across both the most capable models (which are unfortunately proprietary) and the best open models. The problem is that what works on small LLMs does not necessarily scale to larger ones. See page 35 of [1] for example. A researcher only using the models of a few years ago (where the open models had <1B parameters) could come to a completely incorrect conclusion: that language models are incapable of generalising facts learned in one language to another. [1] https://arxiv.org/pdf/2308.03296.pdf https://arxiv.org/pdf/2308.03296.pdf
- arter 3y ago
  Not always but we can reproduce your findings in the future - credit to gravitational lensing causing some light paths to years longer to reach us.

IanCal 3y ago

> GPT is a closed source/weights, proprietary product that changes every couple of weeks or so. Do you have any evidence that the weights for versioned models are being changed without notifications?

starbugs 3y ago
> Do you have any evidence that the weights for versioned models are being changed without notifications? I think in a real scientific process, it's upon those who claim that they are not to provide the evidence.
- IanCal 3y ago
  I'm sorry, but that's entirely ridiculous. You're mangling up a concept of burden of proof here. You can easily see this because it can be flipped around easily - you made a claim that they are being changed, even every few weeks! Should it really be on me to show that your very specific claim is false? Aside - but even if the model weights did change, that wouldn't stop research being possible. Otherwise no drug trial could be replicated because you couldn't get the exact same participants at the exact same age.
  - dartos 3y ago
    Apples and oranges comparison. You couldn’t get the same participants, but you could get the same drugs. If you could get identical participants, that wouldn’t be very helpful since humans are so varied. But for GPT based papers, what you’re actually testing could change without you knowing. There’s no way to know if a paper is reproducible at all. If you can’t reproduce results, is it really research, or just show and tell?
    - mewpmewp2 3y ago
      With API you can choose versions fixed to a date. Are you suggesting that OpenAI is lying about these being fixed to a date? Why would they lie about it? The whole point of these versions is so that when you build on top of that it would keep working as you expect.
      - hnfong 3y ago
        I'm not saying they lie about it, but as a hypothetical there could be many reasons to lie. - realizing their model leaks confidential information against malicious prompts - copyright claims against them forcing them to remove bits of data from the training set - serious "alignment" bugs that need to be fixed - vastly improved optimization techniques that slightly affect results in 0.1% of the cases If updating the model would save the company a couple hundred million dollars, they might want to do it. And in some of the cases, I can imagine they have an incentive to keep the update low key.
    - IanCal 3y ago
      > If you can’t reproduce results, is it really research, or just show and tell? You can't start with a statement about clinical trials not being perfectly reproducible and that's fine, then say this. > what you’re actually testing could change without you knowing If people are lying about an extremely important part of their product, which they have little reason to. But then this applies to pretty much everything. Starting with the assumption that people are lying about everything and nothing is as it seems may technically make things more reproducible but it's going to require unbelievable effort for very little return. > There’s no way to know if a paper is reproducible at all. This is a little silly because these models are available extremely easily and at a pay-as-you-go pricing. And again, it requires an assumption that openai is lying about a specific feature of a product.

awestroke 3y ago

You can select a static snapshot that presumably does not change, if you use the API

nradov 3y ago
Is that a contractual guarantee, or more of a "trust us" kind of thing?
starbugs 3y ago
> You can select a static snapshot that presumably does not change, if you use the API Sorry, I won't blindly believe a company who are cynical enough to call themselves "OpenAI", then publish a commercial closed source/weights model for profit. Evidence that they do not change without notice or it didn't happen. Better even, provide the source and weights for research purposes. These models could be pulled at every instant if the company sees fit or ceases to exist.
- cqqxo4zV46cp 3y ago
  Yeah, here it comes. In these conversations you don’t need to ask very many “why”s before it just turns out that the antagonist (you) has an axe to grind about OpenAI, and has added that the their misplaced sense of expertise with regard to the typical standards of proof in academic publications.
  - starbugs 3y ago
    > Yeah, here it comes. In these conversations you don’t need to ask very many “why”s before it just turns out that the antagonist (you) has an axe to grind about OpenAI, and has added that the their misplaced sense of expertise with regard to the typical standards of proof in academic publications. Seems to have hit hard? I would find it borderline acceptable being offended by a user whose name has obviously been generated using a password generator if you could at least provide some substance to the discussion. Just labeling someone and questioning their competence based on your hurt feelings is a bit low. Please improve.

sebzim4500 3y ago

If I book telescope time and capture a supernova then no one will ever be able to reproduce my raw results because it has already happened. I don't see why OpenAI pulling old model snapshots is any different.

kjkjadksj 3y ago
But you know the parameters of your telescope at least. If openai wants to update all the time, fine, but then they should be like how every other piece of research software works, where you can list what exact version of software you used and pull that version yourself if need be.
- IanCal 3y ago
  Stability is the purpose of the versioned models.
starbugs 3y ago
> If I book telescope time and capture a supernova then no one will ever be able to reproduce my raw results because it has already happened. I don't see why OpenAI pulling old model snapshots is any different. That's why you capture multiple of them and verify your data statistically?
- sebzim4500 3y ago
  And ideally if someone is proposing new prompting techniques they should test it across both the most capable models (which are unfortunately proprietary) and the best open models. The problem is that what works on small LLMs does not necessarily scale to larger ones. See page 35 of [1] for example. A researcher only using the models of a few years ago (where the open models had <1B parameters) could come to a completely incorrect conclusion: that language models are incapable of generalising facts learned in one language to another. [1] https://arxiv.org/pdf/2308.03296.pdf https://arxiv.org/pdf/2308.03296.pdf
arter 3y ago
Not always but we can reproduce your findings in the future - credit to gravitational lensing causing some light paths to years longer to reach us.