4 ms·
> for almost all of the relevant AI and ML work The word “relevant” is an important qualifier which may be doing a lot of lifting. Who defines what is “relevan
by latexr 4y ago
> for almost all of the relevant AI and ML work
The word “relevant” is an important qualifier which may be doing a lot of lifting. Who defines what is “relevant”? How do you interpret it for this particular case?
I haven‘t looked at the matter myself, but the author did provide a source for their claim¹. Do you have any specific information on why it is wrong? It is not my intention to put you on the spot, I’m trying to understand if your point is based on data or gut feeling so I can myself get a better understanding of the validity of each argument.
¹ https://www.baldurbjarnason.com/2023/beware-of-ai-snake-oil/#fn:7 https://www.baldurbjarnason.com/2023/beware-of-ai-snake-oil/...
- williamcotton 4y agoI’d say the two most relevant models right now are Stable Diffusion and GPT-4. Stable Diffusion is open in all senses of the word. GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. GPT-like things like LLaMA are, well, available. Direct, individually reproducible, these tools and their related academic publications qualify as computer science. Nullius in verba certainly applies.
- latexr 4y ago> GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. Requiring a credit card to verify scientific claims is a large deterrent. And in addition one needs to provide them with a phone number, which many people will not be comfortable with when considering their CEO’s history.¹ Even ignoring the above, using the output of ChatGPT is definitely not the same as having “access to any of the data or code used in the research”, which is the author’s complaint and what the original commenter quoted. As for the other cases, we could only properly assess them if we discussed specific studies which I don’t think we’re going to do in the limited context of HN comments. Thank you for taking the time to explain your view. ¹ https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
- pwillia7 4y agoI had to give apple my credit card to install xcode... e: and I'll never forgive them for it
- throwaway1777 4y agoYou actually don’t have to pay to use Xcode. You only need to pay to build for real devices and submit to the App Store.
- taeric 4y agoThis is silly, though. Are NASA and friends not advancing science? You need way more than a credit card to replicate their work. I would prefer more open advances, but hard not to feel gate keeper for a term to claim they aren't advancing the science.
- ethbr0 4y agoThere's a difference between capital/operational expenditures to reproduce work, and directly paying the owner of the work. The former is science. The latter is a commercial product. Stretching their business model to its most charitable interpretation of "open", OpenAI is essentially offering "Here's the cost for the lab and labor for the work you want to do. We'll do it and send you the results, but won't give you any of our lab procedures or intermediate data."
- HarHarVeryFunny 4y agoModels like GPT-4 are highly functional, so they are not snake oil in that sense, but they are certainly not scientifically reproducible (or understood for that matter). In fact there's generally a reproducibility crisis in machine learning, since things like models, datasets, training techniques, hyper-parameters are often only documented in the vaguest of terms. Reproducibility doesn't mean I have a black box and when I push the button (after swiping my credit card) the light comes on as expected. Reproducibility in science means I can reproduce the experiment from scratch and get the same results. Practically nothing is known about GPT-4. OpenAI's "GPT-4 Technical Report" doesn't say much more than the fact that it's "Transformer-based". https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf
- HybridCurve 4y ago>GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card No, you really can't reproduce it this way. Science is not a carnival show. This is why the term snake oil was appropriate.