4 ms·

I’d say the two most relevant models right now are Stable Diffusion and GPT-4. Stable Diffusion is open in all senses of the word. GPT-4 is closed but the tec

I’d say the two most relevant models right now are Stable Diffusion and GPT-4. Stable Diffusion is open in all senses of the word. GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. GPT-like things like LLaMA are, well, available. Direct, individually reproducible, these tools and their related academic publications qualify as computer science. Nullius in verba certainly applies.

latexr 4y ago
> GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. Requiring a credit card to verify scientific claims is a large deterrent. And in addition one needs to provide them with a phone number, which many people will not be comfortable with when considering their CEO’s history.¹ Even ignoring the above, using the output of ChatGPT is definitely not the same as having “access to any of the data or code used in the research”, which is the author’s complaint and what the original commenter quoted. As for the other cases, we could only properly assess them if we discussed specific studies which I don’t think we’re going to do in the limited context of HN comments. Thank you for taking the time to explain your view. ¹ https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
- pwillia7 4y ago
  I had to give apple my credit card to install xcode... e: and I'll never forgive them for it
  - throwaway1777 4y ago
    You actually don’t have to pay to use Xcode. You only need to pay to build for real devices and submit to the App Store.
    - jhbadger 4y ago
      I think the OP means is that even for free (as in beer) Apple software you still have to have a credit card attached to your account (or at least you used to), even if it isn't actually charged, because of the way the App Store works.
      - pwillia7 4y ago
        correct
- taeric 4y ago
  This is silly, though. Are NASA and friends not advancing science? You need way more than a credit card to replicate their work. I would prefer more open advances, but hard not to feel gate keeper for a term to claim they aren't advancing the science.
  - ethbr0 4y ago
    There's a difference between capital/operational expenditures to reproduce work, and directly paying the owner of the work. The former is science. The latter is a commercial product. Stretching their business model to its most charitable interpretation of "open", OpenAI is essentially offering "Here's the cost for the lab and labor for the work you want to do. We'll do it and send you the results, but won't give you any of our lab procedures or intermediate data."
    - taeric 4y ago
      I mean, sorta? Science is advanced in many ways. Not all are open. Some for good reasons. Some not.
      - ethbr0 4y ago
        Fair. But there's a difference between "openly accessible scientific information" and "trade secrets". The US military has pushed state of the art in many fields, but I don't think anyone would say it pushed stealth composite science forward with the B-2/F-22/F-35 programs, because all the details are classified. The risk is that non-shared research is eventually lost.
        taeric 4y ago
        Also mostly fair. I think the catch is Open AI is risking turning "open" into a weasel word? That is, still science, but hardly open. Which is fine, but then don't call yourself open?
        4y ago
- williamcotton 4y ago
  Arguing that academic papers for AI are snake oil by using one CEO’s previous and unrelated venture as an attack on his character is not convincing. The methods for reproducing the LLMs in question have been published. Much like with the LHC, one hindrance to reproducing claims is the cost of building the machine. Unlike the LHC, mere mortals can interact with the machine and draw their own conclusions of the claims published by the academic journals. This the the foundation of the issues you raise with regards to science in the 21st century: the cost of reproducing a claim is at odds with “taking no one’s word”. The leak of the LLaMA models changes this dynamic. It is now possible to run the models locally and with the same datasets published in the journals. This is hardly egalitarian but science has always had a material cost. Competition from both public and private sources almost guarantees that these models will be cheap and abundant in not too long of a time.
  - latexr 4y ago
    > Arguing that academic papers for AI are snake oil by using one CEO’s previous and unrelated venture as an attack on his character is not convincing. That’s a wild extrapolation and an absolute mischaracterisation of what I wrote. Please respond to what people write, not what you think they believe. I only claimed many people are wary of giving OpenAI their phone number due to their CEO. I did not say AI papers are snake oil. On the contrary, I explicitly said I don’t know and have respectfully asked for clarifications. Also, Worldcoin is neither a previous venture nor unrelated. They just pivoted to collect your eyeballs to prove you’re not an AI, “solving” the problem they created. https://worldcoin.org/blog/engineering/humanness-in-the-age-of-ai https://worldcoin.org/blog/engineering/humanness-in-the-age-...
HarHarVeryFunny 4y ago
Models like GPT-4 are highly functional, so they are not snake oil in that sense, but they are certainly not scientifically reproducible (or understood for that matter). In fact there's generally a reproducibility crisis in machine learning, since things like models, datasets, training techniques, hyper-parameters are often only documented in the vaguest of terms. Reproducibility doesn't mean I have a black box and when I push the button (after swiping my credit card) the light comes on as expected. Reproducibility in science means I can reproduce the experiment from scratch and get the same results. Practically nothing is known about GPT-4. OpenAI's "GPT-4 Technical Report" doesn't say much more than the fact that it's "Transformer-based". https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf
HybridCurve 4y ago
>GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card No, you really can't reproduce it this way. Science is not a carnival show. This is why the term snake oil was appropriate.

latexr 4y ago

> GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. Requiring a credit card to verify scientific claims is a large deterrent. And in addition one needs to provide them with a phone number, which many people will not be comfortable with when considering their CEO’s history.¹ Even ignoring the above, using the output of ChatGPT is definitely not the same as having “access to any of the data or code used in the research”, which is the author’s complaint and what the original commenter quoted. As for the other cases, we could only properly assess them if we discussed specific studies which I don’t think we’re going to do in the limited context of HN comments. Thank you for taking the time to explain your view. ¹ https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...

pwillia7 4y ago
I had to give apple my credit card to install xcode... e: and I'll never forgive them for it
- throwaway1777 4y ago
  You actually don’t have to pay to use Xcode. You only need to pay to build for real devices and submit to the App Store.
  - jhbadger 4y ago
    I think the OP means is that even for free (as in beer) Apple software you still have to have a credit card attached to your account (or at least you used to), even if it isn't actually charged, because of the way the App Store works.
    - pwillia7 4y ago
      correct
taeric 4y ago
This is silly, though. Are NASA and friends not advancing science? You need way more than a credit card to replicate their work. I would prefer more open advances, but hard not to feel gate keeper for a term to claim they aren't advancing the science.
- ethbr0 4y ago
  There's a difference between capital/operational expenditures to reproduce work, and directly paying the owner of the work. The former is science. The latter is a commercial product. Stretching their business model to its most charitable interpretation of "open", OpenAI is essentially offering "Here's the cost for the lab and labor for the work you want to do. We'll do it and send you the results, but won't give you any of our lab procedures or intermediate data."
  - taeric 4y ago
    I mean, sorta? Science is advanced in many ways. Not all are open. Some for good reasons. Some not.
    - ethbr0 4y ago
      Fair. But there's a difference between "openly accessible scientific information" and "trade secrets". The US military has pushed state of the art in many fields, but I don't think anyone would say it pushed stealth composite science forward with the B-2/F-22/F-35 programs, because all the details are classified. The risk is that non-shared research is eventually lost.
      - taeric 4y ago
        Also mostly fair. I think the catch is Open AI is risking turning "open" into a weasel word? That is, still science, but hardly open. Which is fine, but then don't call yourself open?
        4y ago
williamcotton 4y ago
Arguing that academic papers for AI are snake oil by using one CEO’s previous and unrelated venture as an attack on his character is not convincing. The methods for reproducing the LLMs in question have been published. Much like with the LHC, one hindrance to reproducing claims is the cost of building the machine. Unlike the LHC, mere mortals can interact with the machine and draw their own conclusions of the claims published by the academic journals. This the the foundation of the issues you raise with regards to science in the 21st century: the cost of reproducing a claim is at odds with “taking no one’s word”. The leak of the LLaMA models changes this dynamic. It is now possible to run the models locally and with the same datasets published in the journals. This is hardly egalitarian but science has always had a material cost. Competition from both public and private sources almost guarantees that these models will be cheap and abundant in not too long of a time.
- latexr 4y ago
  > Arguing that academic papers for AI are snake oil by using one CEO’s previous and unrelated venture as an attack on his character is not convincing. That’s a wild extrapolation and an absolute mischaracterisation of what I wrote. Please respond to what people write, not what you think they believe. I only claimed many people are wary of giving OpenAI their phone number due to their CEO. I did not say AI papers are snake oil. On the contrary, I explicitly said I don’t know and have respectfully asked for clarifications. Also, Worldcoin is neither a previous venture nor unrelated. They just pivoted to collect your eyeballs to prove you’re not an AI, “solving” the problem they created. https://worldcoin.org/blog/engineering/humanness-in-the-age-of-ai https://worldcoin.org/blog/engineering/humanness-in-the-age-...

HarHarVeryFunny 4y ago

Models like GPT-4 are highly functional, so they are not snake oil in that sense, but they are certainly not scientifically reproducible (or understood for that matter). In fact there's generally a reproducibility crisis in machine learning, since things like models, datasets, training techniques, hyper-parameters are often only documented in the vaguest of terms. Reproducibility doesn't mean I have a black box and when I push the button (after swiping my credit card) the light comes on as expected. Reproducibility in science means I can reproduce the experiment from scratch and get the same results. Practically nothing is known about GPT-4. OpenAI's "GPT-4 Technical Report" doesn't say much more than the fact that it's "Transformer-based". https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf

HybridCurve 4y ago

>GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card No, you really can't reproduce it this way. Science is not a carnival show. This is why the term snake oil was appropriate.