4 ms·

> claiming human authorship for a proof generated entirely by an AI system would misrepresent both the system’s contribution and the nature of genuine human int

by zkmon 2mo ago

> claiming human authorship for a proof generated entirely by an AI system would misrepresent both the system’s contribution and the nature of genuine human intellectual work. AI has no self-awareness. It's a tool. When you assemble a furniture using a screw driver, the torque force interacts with the molecular forces inside the metal and miraculously it transfers the force to the screw though a clever geometry design, communicating the force to the screw to turn it in a certain way. Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down.

NitpickLawyer 2mo ago
A better analogy would be a manufactured object, say 3d printed for simplicity. The 3d printer is given an input, and an object manifests itself after some time. We say that the creator of the object is the person turning on the machine, sending the data, and collecting the object. Not the machine itself.
- cure_42 2mo ago
  I'd say the creator is the one who created the 3d model, not the one who pushed the print button.
  - NitpickLawyer 2mo ago
    (let's assume that)My 3dprinter is special. It has a bunch of values + an algorithm (i.e. a neural network) that takes input as tokens and outputs a printed object.
  - dgellow 2mo ago
    I would say „I made this gadget with my 3d printer, but the designer is someone else (I found the model online)“. The intent, the drive, the action comes from the human
    - traes 2mo ago
      "I made this proof myself, but the designer is someone else" is an extremely unconvincing claim to ownership.
      - dgellow 2mo ago
        Almost as if a proof isn’t the same as a 3d print. It’s just not a good analogy
    - cure_42 2mo ago
      The action came from the machine. The drive could have come from anywhere, maybe it was you maybe it was someone else. The intent seems largely irrelevant to who made it. If I commission an artwork, the intent came from me but I sure as heck didn't make it.
  - samatman 2mo ago
    True story: I have a moisture issue in my furnace, such that it needs vacuuming out. This involved detaching a length of tubing, but that puts stress on said tubing, sometimes knocks other things out of alignment, and involves completing the seal between the wetvac and the tubing with my hand. I also have a 3D printer. I also have a ChatGPT subscription, and some OpenSCAD chops. I came up with a part which would go into the top of the down tube to the drainage pump, and mostly-seal the down tube itself, with an opening on the side to vacuum out the moisture. This was purely prooompted, I took some measurements, printed bits of the part, refined the shape, and you know what? It works! I can stick it down there, turn on the (very loud) wet vac, and go upstairs. On a 1.5Ah battery it sucks for a bit less than ten minutes, which turns out to be plenty of time. So: who made that? Don't care. I'm waking up warm at night. Also: me, obviously. ChatGPT doesn't have a fucking furnace.
    - cure_42 2mo ago
      Would have said cool story but you're an ass, so. Cool story bro
esikich 2mo ago
Your brain also is physical. Electrochemical gradients flow between physical molecular constructs. Isn't it just chemistry? Do you attribute it to physics or some whole-is-greater-than-the-parts idea?
naasking 2mo ago
> AI has no self-awareness What is your mechanistic model of self awareness that yields this conclusion? > It's a tool Does your model suggest that tools can't have self awareness?
- deleted 2mo ago
  [deleted]
- perching_aix 2mo ago
  Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to, and influences future choices. This implies statefulness, which models are intentionally not at inference time (*). (*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in, I fail to identify (**) a representation of its own state in these bodies of text that it'd be meticulously maintaining. I further fail to identify how it could be hidden or maintained, considering I control like half of it. The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line"). You'll sometimes catch models mixing up who's who and how many who-s there even are for example. (**) I did wish for something hidden though, so maybe it's just concealed? The same way people can encode a lot more of their emotional and mental state than normal into text if they read and write a lot of it, I'm aware of research that suggested the same for LLMs, albeit I cannot cite it. Maybe those phrasing signatures are just alien to me and will never pop out. Either way, I'd expect researchers to stumble upon this during interpretability studies, and either they haven't, they have but it wasn't popsci adopted, or they're keeping awfully tight lipped about it. If you know of anything like this, your turn now, would be happy to learn. I do wonder how reasonable it is to expect e.g. a single maintained identity though. Maybe it isn't? (*) Another way to hack around this of course is to just precompute some internal "self-awareness states" and hop around between them. Probably the closest to what the models are actually "doing".
  - ben_w 2mo ago
raincole 2mo ago
Sorry, OpenAI's take is correct here. If you're not convinced, here is how they prompted LLM: https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98d31/cdc_prompt.pdf https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98... [0] A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt. [0]: Not one of the proofs in the linked article, but from OpenAI too.
- yaqubroli 2mo ago
  The human provides the intention and the ability to appreciate the output. Tools do “heavy lifting” all the time, but we still primarily credit the humans who use them precisely because they made the choice to use them. Provability is just going the way of computation. John Napier had to manually compute logarithm tables over decades and was recognised for his work; now that same work could be performed by a 10 year old with a calculator in an evening.
  - esikich 2mo ago
    What gives the intention and ability to the human?
  - oklahomasports 2mo ago
    Are you playing dumb? Using power tools to build furniture is very different than using an ai robot to carve a statue or whatever.
- zkmon 2mo ago
  When you use a crane to do the "heavy lifting" for construction work, do you give full credit to the cranes?
  - raincole 2mo ago
    Read the prompts in the PDF I link and see if your analogy makes sense in this context :)
ben_w 2mo ago
> Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down. When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it. I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".

by zkmon 2mo ago

NitpickLawyer 2mo ago

A better analogy would be a manufactured object, say 3d printed for simplicity. The 3d printer is given an input, and an object manifests itself after some time. We say that the creator of the object is the person turning on the machine, sending the data, and collecting the object. Not the machine itself.

cure_42 2mo ago
I'd say the creator is the one who created the 3d model, not the one who pushed the print button.
- NitpickLawyer 2mo ago
  (let's assume that)My 3dprinter is special. It has a bunch of values + an algorithm (i.e. a neural network) that takes input as tokens and outputs a printed object.
- dgellow 2mo ago
  I would say „I made this gadget with my 3d printer, but the designer is someone else (I found the model online)“. The intent, the drive, the action comes from the human
  - traes 2mo ago
    "I made this proof myself, but the designer is someone else" is an extremely unconvincing claim to ownership.
    - dgellow 2mo ago
      Almost as if a proof isn’t the same as a 3d print. It’s just not a good analogy
  - cure_42 2mo ago
    The action came from the machine. The drive could have come from anywhere, maybe it was you maybe it was someone else. The intent seems largely irrelevant to who made it. If I commission an artwork, the intent came from me but I sure as heck didn't make it.
- samatman 2mo ago
  True story: I have a moisture issue in my furnace, such that it needs vacuuming out. This involved detaching a length of tubing, but that puts stress on said tubing, sometimes knocks other things out of alignment, and involves completing the seal between the wetvac and the tubing with my hand. I also have a 3D printer. I also have a ChatGPT subscription, and some OpenSCAD chops. I came up with a part which would go into the top of the down tube to the drainage pump, and mostly-seal the down tube itself, with an opening on the side to vacuum out the moisture. This was purely prooompted, I took some measurements, printed bits of the part, refined the shape, and you know what? It works! I can stick it down there, turn on the (very loud) wet vac, and go upstairs. On a 1.5Ah battery it sucks for a bit less than ten minutes, which turns out to be plenty of time. So: who made that? Don't care. I'm waking up warm at night. Also: me, obviously. ChatGPT doesn't have a fucking furnace.
  - cure_42 2mo ago
    Would have said cool story but you're an ass, so. Cool story bro

esikich 2mo ago

Your brain also is physical. Electrochemical gradients flow between physical molecular constructs. Isn't it just chemistry? Do you attribute it to physics or some whole-is-greater-than-the-parts idea?

naasking 2mo ago

> AI has no self-awareness What is your mechanistic model of self awareness that yields this conclusion? > It's a tool Does your model suggest that tools can't have self awareness?

deleted 2mo ago
[deleted]
perching_aix 2mo ago
Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to, and influences future choices. This implies statefulness, which models are intentionally not at inference time (*). (*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in, I fail to identify (**) a representation of its own state in these bodies of text that it'd be meticulously maintaining. I further fail to identify how it could be hidden or maintained, considering I control like half of it. The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line"). You'll sometimes catch models mixing up who's who and how many who-s there even are for example. (**) I did wish for something hidden though, so maybe it's just concealed? The same way people can encode a lot more of their emotional and mental state than normal into text if they read and write a lot of it, I'm aware of research that suggested the same for LLMs, albeit I cannot cite it. Maybe those phrasing signatures are just alien to me and will never pop out. Either way, I'd expect researchers to stumble upon this during interpretability studies, and either they haven't, they have but it wasn't popsci adopted, or they're keeping awfully tight lipped about it. If you know of anything like this, your turn now, would be happy to learn. I do wonder how reasonable it is to expect e.g. a single maintained identity though. Maybe it isn't? (*) Another way to hack around this of course is to just precompute some internal "self-awareness states" and hop around between them. Probably the closest to what the models are actually "doing".
- ben_w 2mo ago

raincole 2mo ago

Sorry, OpenAI's take is correct here. If you're not convinced, here is how they prompted LLM: https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98d31/cdc_prompt.pdf https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98... [0] A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt. [0]: Not one of the proofs in the linked article, but from OpenAI too.

yaqubroli 2mo ago
The human provides the intention and the ability to appreciate the output. Tools do “heavy lifting” all the time, but we still primarily credit the humans who use them precisely because they made the choice to use them. Provability is just going the way of computation. John Napier had to manually compute logarithm tables over decades and was recognised for his work; now that same work could be performed by a 10 year old with a calculator in an evening.
- esikich 2mo ago
  What gives the intention and ability to the human?
- oklahomasports 2mo ago
  Are you playing dumb? Using power tools to build furniture is very different than using an ai robot to carve a statue or whatever.
zkmon 2mo ago
When you use a crane to do the "heavy lifting" for construction work, do you give full credit to the cranes?
- raincole 2mo ago
  Read the prompts in the PDF I link and see if your analogy makes sense in this context :)

ben_w 2mo ago

> Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down. When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it. I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".