9 ms·

I will throw in a random story here about chat gpt 4.0. I'm not commenting on this article directly, just a somewhat related anecdote. I was using chatgpt to he

I will throw in a random story here about chat gpt 4.0. I'm not commenting on this article directly, just a somewhat related anecdote. I was using chatgpt to help me write some android opengl rendering code. OpenGL can be very esoteric and I haven't touched it for at least 10 years. Everything was going great and I had a working example, so I decided to look online for some example code to verify I was doing things correctly, and not making any glaring mistakes. It was then that I found an exact line by line copy of what chat gpt had given me. This was before it had the ability to google things, and the code predated openAI. It had even brought across spelling errors in the variables, the only thing it changed was it translated the comments from Spanish to English. I had always been under the impression that chat gpt just learned from sources, and then gave you a new result based roughly on its sources. I think some of the confounding variables here were, 1. this was a very specific use case and not many examples existed, and 2. all opengl code looks similar, to a point. The worst part was, there was no license provided for the code or the repo, so it was not legal for me to take the code wholesale like that. I am now much more cautious about asking chat gpt for code, I only have it give me direction now, and no longer use 'sample code' that it produces.

vundercind 2y ago
> I had always been under the impression that chat gpt just learned from sources, and then gave you a new result based roughly on its sources. I think some of the confounding variables here were, 1. If I’ve understood the transformer paper correctly, these things probabilistically guess words based on what they’ve been trained on, with the probabilities weighted dynamically by the prompt, by what they’ve already generated, and by what they “think” they might generate for the next few tokens (they look ahead somewhat), with another set of probability-weight adjustments applied to all that by a statistical guess at which tokens or words are most-significant or important. None of that would prevent them from spitting out exactly what they’ve seen in training data. Keeping them from doing that a lot requires introducing “noise” to all the statistics stuff above, and maybe a gate after generation that tries to check if what’s been generated is too similar to training data and forces another run (maybe with more noise) if it is, similar to how they prevent them from saying racist stuff or whatever.
- devmor 2y ago
  You have understood correctly. What LLMs are, at least in their current state, is not fundamentally different from a simple markov chain generator. Technically speaking, it is of course, far more complex. There is some incredible vector math and token rerouting going on; But in terms of how you get output from input - it's still "how often have I seen x in relation to y" at the core level. They do not learn, they do not think, they do not reason. They are probability engines. If anyone tells you their LLM is not, it has just been painted over in snake oil to appear otherwise.
  - ben_w 2y ago
    For all that I agree there's a big pile of snake oil in this area, I disagree with you overall. Having played with Markov models since I was a kid*, LLMs are really not just that. All that stuff you acknowledge but then gloss over, that is the actual learning, which tells it which previous tokens are relevant and how much attention to pay to them. This learning creates a world model which is functionally (barely but functionally) performing something approximately reasoning. Statistics and probability is the mechanism it uses to do this, but that alone doesn't make it a Markov chain, those are a very specific thing that's a lot more limited. For example: consider a context window of 128k tokens where each token has 64k possible discrete values. If implemented as a Markov chain, this would need a transition matrix of (2^16)^(2^17) by (2^16) entries (unless I've gotten one of the numbers backwards, but you get the idea regardless). This is too many, and because it is too many you have to create a function to approximate those transitions. But even then, that only works as a Markov chain if it's a deterministic function, and the actual behaviour is not deterministic due to the temperature setting not (usually) being zero. * Commodore 64 user guide aged 6 or so, so I didn't really understand it at the time but that's what it was
    - ipaddr 2y ago
      I have the original guide here. Do you remember where you spotted this? Before the bouncing ball sprite demo?
codedokode 2y ago
I remember similar news about ML services that generate mnusic: they are able to reproduce melodies and lyric from copyrighted songs (if you find a way around filters on song or artist title) and even producer tags in Hip-Hop tracks. All this latest ML growth is built on massive copyright violations.
- tomp 2y ago
  It’s not a copyright violation. Maybe not myself, but many averagely-talented artists can draw Mickey Mouse. They might even draw one for me if I ask! Or I can just find it on Google… (technically my computer is producing it on the screen…) That in itself is not a copyright violation. But if I use it, in a commercial manner, then it becomes a copyright violation. Producing copyrighted things isn’t illegal. It’s on the user to not use copyrighted things, in a way that’s illegal (not fair use or licenced).
  - BobaFloutist 2y ago
    But the AI is a commercial product, that is able to produce copyrighted material. How is that not using copyrighted material for commercial purposes?
    - amanaplanacanal 2y ago
      A cassette recorder is a commercial product that is able to produce copyrighted material. If the user directs it to.
      - skydhash 2y ago
        There was no media used as the material for the cassete recorder.
  - kelnos 2y ago
    So what? Most uses when we're talking about code or artwork are going to involve someone taking the generated result and publishing it somewhere. > But if I use it, in a commercial manner, then it becomes a copyright violation. No, that's incorrect. Commercial use has nothing to do with it. Any act of distribution, regardless of whether or not it's for commercial or personal use, regardless of whether you charge $100,000 or $0, falls under copyright law.
kreyenborgi 2y ago
This has happened quite a few times with me as well, both with chatgpt and phind (phind in particular is often basically stackoverflow with a few variable names changed).
ars 2y ago
One way that can happen is if your prompt and context are so specific, that this copied code is the only thing that matches. This would also imply that this specific question is a rare one, with few examples online for it to train on.
ativzzz 2y ago
> there was no license provided for the code or the repo Interesting - I assume any code that's not licensed as "free to use for whatever purpose I want"
- _rend 2y ago
  Not at all: unless a license is provided, the code is fully protected under copyright and you have _no_ rights to copy it or use it in _any_ way you want (unless falling under "fair use" clauses for the jurisdiction you're in/the author is in).
- skybrian 2y ago
  Uh, no, this is not how copyright law works.
- johnnyanmac 2y ago
  no license[0] is the default fallback when nothing is provided. Realistically, it's "use at your own risk", because someone who doesn't license may not even be aware of what others do with it (or you fallback to whatever rules of the platform you posted on). https://choosealicense.com/no-permission/ https://choosealicense.com/no-permission/
- paulddraper 2y ago
  The opposite
johnnyanmac 2y ago
>I think some of the confounding variables here were, 1. this was a very specific use case and not many examples existed, and 2. all opengl code looks similar, to a point. Yeah, that's why I wouldn't trust AI right now with much except the most basic renderin boilerplate. I'd be so brazen to wager that 90% of the most vauable rendering recipes are prorietary code within some real time rendering studio. Of the remaining, half of that is in some text book and may or may not even be available to scrape online. LLM's still needs a training set, and I'm not convinced the information even exists to be scraped on the public internet for this kind of stuff (if years of googling has taught me anything).
nox101 2y ago
What were you asking it to do? If it's less than 15 lines, tell me and I'll write the solution myself. You can check how close I get. My point being that lots of OpenGL is practically boilerplate.
MLskynetio 2y ago
I do not understand your problem. Do you wanna life in fear as a software developer because you did the same thing as others? Even if the problem has basically not an unlimited way of doing it? Do you want a certain amout of code being (c) and patent? I personally don't. I see an advantage of limited patents for complex magical algorithms were someone was really sitting there and solving a hard problem to reap the benefits for a short period of time, but otherwise no. I do not want to check every code block for some (c) or patent.
rlpb 2y ago
If there's only one way to do it, or a developer familiar with the field would independently come up with the same way of doing it, then the "copyrightability" of the result comes into question. This doesn't stop you getting yourself a legal headache though, of course.

vundercind 2y ago

> I had always been under the impression that chat gpt just learned from sources, and then gave you a new result based roughly on its sources. I think some of the confounding variables here were, 1. If I’ve understood the transformer paper correctly, these things probabilistically guess words based on what they’ve been trained on, with the probabilities weighted dynamically by the prompt, by what they’ve already generated, and by what they “think” they might generate for the next few tokens (they look ahead somewhat), with another set of probability-weight adjustments applied to all that by a statistical guess at which tokens or words are most-significant or important. None of that would prevent them from spitting out exactly what they’ve seen in training data. Keeping them from doing that a lot requires introducing “noise” to all the statistics stuff above, and maybe a gate after generation that tries to check if what’s been generated is too similar to training data and forces another run (maybe with more noise) if it is, similar to how they prevent them from saying racist stuff or whatever.

devmor 2y ago
You have understood correctly. What LLMs are, at least in their current state, is not fundamentally different from a simple markov chain generator. Technically speaking, it is of course, far more complex. There is some incredible vector math and token rerouting going on; But in terms of how you get output from input - it's still "how often have I seen x in relation to y" at the core level. They do not learn, they do not think, they do not reason. They are probability engines. If anyone tells you their LLM is not, it has just been painted over in snake oil to appear otherwise.
- ben_w 2y ago
  For all that I agree there's a big pile of snake oil in this area, I disagree with you overall. Having played with Markov models since I was a kid*, LLMs are really not just that. All that stuff you acknowledge but then gloss over, that is the actual learning, which tells it which previous tokens are relevant and how much attention to pay to them. This learning creates a world model which is functionally (barely but functionally) performing something approximately reasoning. Statistics and probability is the mechanism it uses to do this, but that alone doesn't make it a Markov chain, those are a very specific thing that's a lot more limited. For example: consider a context window of 128k tokens where each token has 64k possible discrete values. If implemented as a Markov chain, this would need a transition matrix of (2^16)^(2^17) by (2^16) entries (unless I've gotten one of the numbers backwards, but you get the idea regardless). This is too many, and because it is too many you have to create a function to approximate those transitions. But even then, that only works as a Markov chain if it's a deterministic function, and the actual behaviour is not deterministic due to the temperature setting not (usually) being zero. * Commodore 64 user guide aged 6 or so, so I didn't really understand it at the time but that's what it was
  - ipaddr 2y ago
    I have the original guide here. Do you remember where you spotted this? Before the bouncing ball sprite demo?

codedokode 2y ago

I remember similar news about ML services that generate mnusic: they are able to reproduce melodies and lyric from copyrighted songs (if you find a way around filters on song or artist title) and even producer tags in Hip-Hop tracks. All this latest ML growth is built on massive copyright violations.

tomp 2y ago
It’s not a copyright violation. Maybe not myself, but many averagely-talented artists can draw Mickey Mouse. They might even draw one for me if I ask! Or I can just find it on Google… (technically my computer is producing it on the screen…) That in itself is not a copyright violation. But if I use it, in a commercial manner, then it becomes a copyright violation. Producing copyrighted things isn’t illegal. It’s on the user to not use copyrighted things, in a way that’s illegal (not fair use or licenced).
- BobaFloutist 2y ago
  But the AI is a commercial product, that is able to produce copyrighted material. How is that not using copyrighted material for commercial purposes?
  - amanaplanacanal 2y ago
    A cassette recorder is a commercial product that is able to produce copyrighted material. If the user directs it to.
    - skydhash 2y ago
      There was no media used as the material for the cassete recorder.
- kelnos 2y ago
  So what? Most uses when we're talking about code or artwork are going to involve someone taking the generated result and publishing it somewhere. > But if I use it, in a commercial manner, then it becomes a copyright violation. No, that's incorrect. Commercial use has nothing to do with it. Any act of distribution, regardless of whether or not it's for commercial or personal use, regardless of whether you charge $100,000 or $0, falls under copyright law.

kreyenborgi 2y ago

This has happened quite a few times with me as well, both with chatgpt and phind (phind in particular is often basically stackoverflow with a few variable names changed).

ars 2y ago

One way that can happen is if your prompt and context are so specific, that this copied code is the only thing that matches. This would also imply that this specific question is a rare one, with few examples online for it to train on.

ativzzz 2y ago

> there was no license provided for the code or the repo Interesting - I assume any code that's not licensed as "free to use for whatever purpose I want"

_rend 2y ago
Not at all: unless a license is provided, the code is fully protected under copyright and you have _no_ rights to copy it or use it in _any_ way you want (unless falling under "fair use" clauses for the jurisdiction you're in/the author is in).
skybrian 2y ago
Uh, no, this is not how copyright law works.
johnnyanmac 2y ago
no license[0] is the default fallback when nothing is provided. Realistically, it's "use at your own risk", because someone who doesn't license may not even be aware of what others do with it (or you fallback to whatever rules of the platform you posted on). https://choosealicense.com/no-permission/ https://choosealicense.com/no-permission/
paulddraper 2y ago
The opposite

johnnyanmac 2y ago

>I think some of the confounding variables here were, 1. this was a very specific use case and not many examples existed, and 2. all opengl code looks similar, to a point. Yeah, that's why I wouldn't trust AI right now with much except the most basic renderin boilerplate. I'd be so brazen to wager that 90% of the most vauable rendering recipes are prorietary code within some real time rendering studio. Of the remaining, half of that is in some text book and may or may not even be available to scrape online. LLM's still needs a training set, and I'm not convinced the information even exists to be scraped on the public internet for this kind of stuff (if years of googling has taught me anything).

nox101 2y ago

What were you asking it to do? If it's less than 15 lines, tell me and I'll write the solution myself. You can check how close I get. My point being that lots of OpenGL is practically boilerplate.

MLskynetio 2y ago

I do not understand your problem. Do you wanna life in fear as a software developer because you did the same thing as others? Even if the problem has basically not an unlimited way of doing it? Do you want a certain amout of code being (c) and patent? I personally don't. I see an advantage of limited patents for complex magical algorithms were someone was really sitting there and solving a hard problem to reap the benefits for a short period of time, but otherwise no. I do not want to check every code block for some (c) or patent.

rlpb 2y ago

If there's only one way to do it, or a developer familiar with the field would independently come up with the same way of doing it, then the "copyrightability" of the result comes into question. This doesn't stop you getting yourself a legal headache though, of course.