3 ms·

It is a _paper_, but it's not science, since GTP-4 is closed source and thus not reproducible in a lab. If OpenAI disappears tomorrow, papers are GTP-4 will li

It is a _paper_, but it's not science, since GTP-4 is closed source and thus not reproducible in a lab. If OpenAI disappears tomorrow, papers are GTP-4 will likely be of little to no value, which is another tell of a non-scientific exploration. (note: not all explorations are scientific, and that is great! Science is just one of many tools for exploring lived reality.)

MattRix 3y ago
That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not. The idea that science has to happen in a lab is of course absurd as well.
- dncornholio 3y ago
  I compare a paper on a GPT-4 prompt to a tutorial on how to use Photoshop. It's not science IMO.
- nicce 3y ago
  > That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not. The idea that science has to happen in a lab is of course absurd as well. The main point here is that anyone can likely start studing those endangered species and try to reproduce the results while in GPT4 it is not possible at all. The lab point is related to fact that we are talking about the software here.
  - roguas 3y ago
    Whats not possible? Do they ban people for exploring sota model that they offer?
    - nicce 3y ago
      That is not reproducible research. To reproduce the research, you need training data, source code and all the parameters. With black-boxed generated models, there is no way to tell how they have been actually generated. It has no value for science to how to improve it further.
- mmcwilliams 3y ago
  In the case of an endangered species a biologist would still have access to take samples from it and inspect it. Science doesn't have to happen in a lab but it's questionable to call something science when it involves hitting a black box endpoint which can change the underlying models and behaviors at a whim.
EForEndeavour 3y ago
What is science? Can you not apply it to artifacts whose inner workings are hidden? As as aside, I was surprised at the repeated misspelling of GPT-4 and take it as a heuristic that this comment was likely written by a real human :)
- mensetmanusman 3y ago
  Aha, didn't know Guanosine-5'-triphosphate (GTP) (a purine nucleoside triphosphate) was part of my ios dictionary, good catch!

MattRix 3y ago

That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not. The idea that science has to happen in a lab is of course absurd as well.

dncornholio 3y ago
I compare a paper on a GPT-4 prompt to a tutorial on how to use Photoshop. It's not science IMO.
nicce 3y ago
> That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not. The idea that science has to happen in a lab is of course absurd as well. The main point here is that anyone can likely start studing those endangered species and try to reproduce the results while in GPT4 it is not possible at all. The lab point is related to fact that we are talking about the software here.
- roguas 3y ago
  Whats not possible? Do they ban people for exploring sota model that they offer?
  - nicce 3y ago
    That is not reproducible research. To reproduce the research, you need training data, source code and all the parameters. With black-boxed generated models, there is no way to tell how they have been actually generated. It has no value for science to how to improve it further.
mmcwilliams 3y ago
In the case of an endangered species a biologist would still have access to take samples from it and inspect it. Science doesn't have to happen in a lab but it's questionable to call something science when it involves hitting a black box endpoint which can change the underlying models and behaviors at a whim.

EForEndeavour 3y ago

What is science? Can you not apply it to artifacts whose inner workings are hidden? As as aside, I was surprised at the repeated misspelling of GPT-4 and take it as a heuristic that this comment was likely written by a real human :)

mensetmanusman 3y ago
Aha, didn't know Guanosine-5'-triphosphate (GTP) (a purine nucleoside triphosphate) was part of my ios dictionary, good catch!