4 ms·

The GPT4 "paper" that's been talked about so much is little more than a marketing press release. I certainly disagree that it's the case for the whole field of

The GPT4 "paper" that's been talked about so much is little more than a marketing press release. I certainly disagree that it's the case for the whole field of ML. But the AI that's in the news right now is LLMs with OpenAI in the lead who have done their utmost to imped reproducible research and even research built on top of their already released tools. It's been a trend that started with Google years ago and their increasingly large models which are unreproducible for most of the scientific community and often not released (if I recall correctly).

brookst 3y ago
Extrapolation from one GPT4 paper to all papers from all sources is a huge mistake. It's funny the article makes its own overbroad claims without data or references, in the name of asserting that's all that's going on in AI.
- latexr 3y ago
  > without data or references Can you clarify what you mean? There are almost two dozen footnotes pointing to references and papers.
  - brookst 3y ago
    I mean, there is no data support in the claim that this pattern is essentially universal in AI research. Citing a few instances doesn’t prove the point. I’m open to the idea that AI research is generally more closed than other areas of research, but there is no quantitative analysis of how open AI research is, or how open other areas of research are. pick any other area of research, and I will happily find you some footnotes citations that support exactly these claims.
    - latexr 3y ago
      > Citing a few instances doesn’t prove the point. I haven’t read every linked reference, but for a relatively short article there are more than a few. It's certainly more annotated than the average article I find on HN. Shouldn’t the quality of the sources matter most? And if what matters is the number, how many would be enough in this case? What would prove the point? > or how open other areas of research are. Other areas of research are irrelevant to the point. Even if other areas suck, that doesn’t excuse this one sucking too. > pick any other area of research, and I will happily find you some footnotes citations that support exactly these claims. Even if that is true, it doesn’t make the argument automatically false or irrelevant for any particular case. First you objected the author had no data or references. Now you’re saying that for any area of research you can find references to prove the point they’re making. Which is it? Is the problem that they didn’t provide enough sources, or that the sources aren’t believable?
      - brookst 3y ago
        I’m sorry I was apparently hard to follow. I meant, the author cites anecdotes (“here’s a paper that’s not open!”) rather than data (“of the 500 most-cited papers in the AI space, 82% were not open, compared to only 36% of papers in anthropology being non-open”). And, I meant if you pick any other discipline that produces thousands of academic papers a year, I will be able to find 20 that don’t count as open. Anger often makes it hard to understand things.

brookst 3y ago

Extrapolation from one GPT4 paper to all papers from all sources is a huge mistake. It's funny the article makes its own overbroad claims without data or references, in the name of asserting that's all that's going on in AI.

latexr 3y ago
> without data or references Can you clarify what you mean? There are almost two dozen footnotes pointing to references and papers.
- brookst 3y ago
  I mean, there is no data support in the claim that this pattern is essentially universal in AI research. Citing a few instances doesn’t prove the point. I’m open to the idea that AI research is generally more closed than other areas of research, but there is no quantitative analysis of how open AI research is, or how open other areas of research are. pick any other area of research, and I will happily find you some footnotes citations that support exactly these claims.
  - latexr 3y ago
    > Citing a few instances doesn’t prove the point. I haven’t read every linked reference, but for a relatively short article there are more than a few. It's certainly more annotated than the average article I find on HN. Shouldn’t the quality of the sources matter most? And if what matters is the number, how many would be enough in this case? What would prove the point? > or how open other areas of research are. Other areas of research are irrelevant to the point. Even if other areas suck, that doesn’t excuse this one sucking too. > pick any other area of research, and I will happily find you some footnotes citations that support exactly these claims. Even if that is true, it doesn’t make the argument automatically false or irrelevant for any particular case. First you objected the author had no data or references. Now you’re saying that for any area of research you can find references to prove the point they’re making. Which is it? Is the problem that they didn’t provide enough sources, or that the sources aren’t believable?
    - brookst 3y ago
      I’m sorry I was apparently hard to follow. I meant, the author cites anecdotes (“here’s a paper that’s not open!”) rather than data (“of the 500 most-cited papers in the AI space, 82% were not open, compared to only 36% of papers in anthropology being non-open”). And, I meant if you pick any other discipline that produces thousands of academic papers a year, I will be able to find 20 that don’t count as open. Anger often makes it hard to understand things.