6 ms·

I worry (And I assume this is either dismissed or a core concern in the field because it's so obvious) that the coming generations of models will use a lot of i

I worry (And I assume this is either dismissed or a core concern in the field because it's so obvious) that the coming generations of models will use a lot of input from existing models. Will we be chasing "untainted" input (pre-2022 data) just like we chase low-background steel (pre-WW2) for sensitive machinery? Should we be "tagging" generated data NOW when it's published in large chunks like images, long texts?

mahathu 4y ago
Low-background steel is actually pre-nuke tests. That's what they scavenge sunken ships a lot for.
- alkonaut 4y ago
  Yes it should really be "pre end of WW2" (The first nuclear test was 1945).
NovaDudely 4y ago
I love that you use the same Steel analogy I came to. Data sets from pre-2023 are going to be useful in determining the validity of future data sets - if that is possible.
- enkid 4y ago
  What's the use case for such validation?
- brookst 4y ago
  Why didn’t we do that before AI? If that’s useful now, surely it would be useful to know what knowledge pre-dates humans learning and repeating it?
  - falcor84 4y ago
    Well, isn't this pretty much the definition that historians use for "primary sources"?
anothernewdude 4y ago
A lot of AI training data is bootstrapped from weaker models.
currymj 4y ago
there's active work on watermarking LM outputs. here's one example: https://huggingface.co/spaces/tomg-group-umd/lm-watermarking https://huggingface.co/spaces/tomg-group-umd/lm-watermarking
lmarcos 4y ago
The immediate benefactor of tagging non-ai generated data would be OpenAI.
- alkonaut 4y ago
  Sure. But it would be a lot easier to tag output like images as "AI generated" in a small handful of generators like StableDiffusion and similar, than making the opposite and telling people to tag their non-ai-generated content as non-ai-gemerated.
- brookst 4y ago
  Also people who want to make a quick buck by selling AI-generated data and marking it as non-AI. Sounds super cheap and high ROI.
dr_dshiv 4y ago
I would expect that smart people generate data with GPT that is more and better than smart people alone. I really don’t see the issue. Especially since smart people + GPT + empirical data (think GPT plus surveys and sensors) will be a HUGE source of intelligence.
- mihaaly 4y ago
  Will not-so-smart people be banned from using GPT?
  - bryanrasmussen 4y ago
    they will have to pay more. And I'm already broke :(
  - lonelyasacloud 4y ago
    > Will not-so-smart people be banned from using GPT? If we're consuming information filtered and noise reduced by very smart AI's how much does it matter?
    - mihaaly 4y ago
      I responded to (from the parent comment): >> I would expect that smart people generate data with GPT [...] Also: > filtered and noise reduced by *very smart* AI Are you sure you know what GPT is?
  - deelly 4y ago
    Of course no! Only people that can't afford ChatGPT will be out of luck.
    - falcor84 4y ago
      I'd agree if it weren't the case that ChatGPT is the fastest growing free AI service in the world. Indeed I cannot think of a single worse example to "The future is already here, it's just not very evenly distributed" - everyone who wants to use ChatGPT can, what more could you expect?
mihaaly 4y ago
What I only can hope about is the natural selection of useful data throughout time. Garbage is all around, people generate it on mass scale with modern technology nowadays, but the useful portion may survive the test of time, prevail. Those using 'good' data over 'bad' can gain benefits that could lead to the not self harming population using good data, suppressing bad ones. Mostly, of course.
jgrahamc 4y ago
I literally made a Tumblr called https://lowbackgroundsteel.ai https://lowbackgroundsteel.ai the other day :-)
baxuz 4y ago
Don Hertzfeldt's Simpsons couch gag seems like an end result of AI training AI. https://www.youtube.com/watch?v=6i2l-LQ-dXI https://www.youtube.com/watch?v=6i2l-LQ-dXI
- ddalex 4y ago
  I mean, the singularity proponents see this as the golden grail, with the condition that N+1 AI trainee is slightly smarter then the N AI trainer.
roenxi 4y ago
It is a safer bet that someone will hook a neural net up to a camera on someone's hat, train from reality, then re-derive art from that. Then AIs will use the same methods as humans to maintain their grounding in reality. The problem with the current crop of AI is they pretty clearly don't understand that they are taking a 2D projection of a 3D concept. That'll be fixed and there is a good chance it'll be fixed with video footage because that is how humans do it.
version_five 4y ago
First thing I thought reading your comment was "this guy's repeating the low background steel analogy that was posted the other day". I don't know if that's actually true, and it doesn't matter, my point is the internet is already an echo chamber. GPT is trained on reddit and HN and other forums where people say the same stuff over and over again anyway. It's not like the pre-gpt internet is some pristine pool of new ideas. As long as someone somewhere is injecting some new stuff in, and they will be, it doesn't matter that there will be more automatically generated crap, there already was
- banannaise 4y ago
  Ignoring the effects of volume is a common and fatal mistake. "This effect already exists" is not incompatible with "Multiplying the impact of this effect by orders of magnitude will cause it to become a massive problem." Or, put another way: A slightly degraded signal-to-noise ratio isn't really a problem, but once it degrades far enough, the whole stream becomes impossible to decode, and worthless.
  - falcor84 4y ago
    My Information Theory is rusty (and wasn't too strong to begin with), but didn't Shannon prove that we can decode a signal given arbitrarily bad noise, but that the more noise, the lower the channel capacity?
    - banannaise 4y ago
      Yep, pretty much. But therein lies the problem: filtering out garbage writing is something done by humans, whose limited time, attention, and effort all make for a pretty tight bottleneck in the first place. Degrade that channel capacity, and our ability to find texts of value becomes pretty close to nil. edit: Also, when the noise comes in the form of competing signals rather than random noise, that becomes much, much more difficult, but we're also outside my area of expertise at this point.
  - version_five 4y ago
getoffmycase 4y ago
I work in a field where the training data is extremely sparse because labeled data is incredibly time consuming and labor intensive to come by. There’s ways to automate labeling, but I don’t really trust those methods fully cause I’ve seen how the sausage is made, so to say. Everybody and their dog is trying to do machine learning, but I question how useful or real the results are because I know that the data underlying the models is of questionable quality. I don’t know what’s next for the field, but it’ll probably be some high profile disaster.
johnmw 4y ago
I am guessing in the future they would just run simulations to generate the untainted training data pre 2022. ...oh wait.

mahathu 4y ago

Low-background steel is actually pre-nuke tests. That's what they scavenge sunken ships a lot for.

alkonaut 4y ago
Yes it should really be "pre end of WW2" (The first nuclear test was 1945).

NovaDudely 4y ago

I love that you use the same Steel analogy I came to. Data sets from pre-2023 are going to be useful in determining the validity of future data sets - if that is possible.

enkid 4y ago
What's the use case for such validation?
brookst 4y ago
Why didn’t we do that before AI? If that’s useful now, surely it would be useful to know what knowledge pre-dates humans learning and repeating it?
- falcor84 4y ago
  Well, isn't this pretty much the definition that historians use for "primary sources"?

anothernewdude 4y ago

A lot of AI training data is bootstrapped from weaker models.

currymj 4y ago

there's active work on watermarking LM outputs. here's one example: https://huggingface.co/spaces/tomg-group-umd/lm-watermarking https://huggingface.co/spaces/tomg-group-umd/lm-watermarking

lmarcos 4y ago

The immediate benefactor of tagging non-ai generated data would be OpenAI.

alkonaut 4y ago
Sure. But it would be a lot easier to tag output like images as "AI generated" in a small handful of generators like StableDiffusion and similar, than making the opposite and telling people to tag their non-ai-generated content as non-ai-gemerated.
brookst 4y ago
Also people who want to make a quick buck by selling AI-generated data and marking it as non-AI. Sounds super cheap and high ROI.

dr_dshiv 4y ago

I would expect that smart people generate data with GPT that is more and better than smart people alone. I really don’t see the issue. Especially since smart people + GPT + empirical data (think GPT plus surveys and sensors) will be a HUGE source of intelligence.

mihaaly 4y ago
Will not-so-smart people be banned from using GPT?
- bryanrasmussen 4y ago
  they will have to pay more. And I'm already broke :(
- lonelyasacloud 4y ago
  > Will not-so-smart people be banned from using GPT? If we're consuming information filtered and noise reduced by very smart AI's how much does it matter?
  - mihaaly 4y ago
    I responded to (from the parent comment): >> I would expect that smart people generate data with GPT [...] Also: > filtered and noise reduced by *very smart* AI Are you sure you know what GPT is?
- deelly 4y ago
  Of course no! Only people that can't afford ChatGPT will be out of luck.
  - falcor84 4y ago
    I'd agree if it weren't the case that ChatGPT is the fastest growing free AI service in the world. Indeed I cannot think of a single worse example to "The future is already here, it's just not very evenly distributed" - everyone who wants to use ChatGPT can, what more could you expect?

mihaaly 4y ago

What I only can hope about is the natural selection of useful data throughout time. Garbage is all around, people generate it on mass scale with modern technology nowadays, but the useful portion may survive the test of time, prevail. Those using 'good' data over 'bad' can gain benefits that could lead to the not self harming population using good data, suppressing bad ones. Mostly, of course.

jgrahamc 4y ago

I literally made a Tumblr called https://lowbackgroundsteel.ai https://lowbackgroundsteel.ai the other day :-)

baxuz 4y ago

Don Hertzfeldt's Simpsons couch gag seems like an end result of AI training AI. https://www.youtube.com/watch?v=6i2l-LQ-dXI https://www.youtube.com/watch?v=6i2l-LQ-dXI

ddalex 4y ago
I mean, the singularity proponents see this as the golden grail, with the condition that N+1 AI trainee is slightly smarter then the N AI trainer.

roenxi 4y ago

It is a safer bet that someone will hook a neural net up to a camera on someone's hat, train from reality, then re-derive art from that. Then AIs will use the same methods as humans to maintain their grounding in reality. The problem with the current crop of AI is they pretty clearly don't understand that they are taking a 2D projection of a 3D concept. That'll be fixed and there is a good chance it'll be fixed with video footage because that is how humans do it.

version_five 4y ago

First thing I thought reading your comment was "this guy's repeating the low background steel analogy that was posted the other day". I don't know if that's actually true, and it doesn't matter, my point is the internet is already an echo chamber. GPT is trained on reddit and HN and other forums where people say the same stuff over and over again anyway. It's not like the pre-gpt internet is some pristine pool of new ideas. As long as someone somewhere is injecting some new stuff in, and they will be, it doesn't matter that there will be more automatically generated crap, there already was

banannaise 4y ago
Ignoring the effects of volume is a common and fatal mistake. "This effect already exists" is not incompatible with "Multiplying the impact of this effect by orders of magnitude will cause it to become a massive problem." Or, put another way: A slightly degraded signal-to-noise ratio isn't really a problem, but once it degrades far enough, the whole stream becomes impossible to decode, and worthless.
- falcor84 4y ago
  My Information Theory is rusty (and wasn't too strong to begin with), but didn't Shannon prove that we can decode a signal given arbitrarily bad noise, but that the more noise, the lower the channel capacity?
  - banannaise 4y ago
    Yep, pretty much. But therein lies the problem: filtering out garbage writing is something done by humans, whose limited time, attention, and effort all make for a pretty tight bottleneck in the first place. Degrade that channel capacity, and our ability to find texts of value becomes pretty close to nil. edit: Also, when the noise comes in the form of competing signals rather than random noise, that becomes much, much more difficult, but we're also outside my area of expertise at this point.
- version_five 4y ago

getoffmycase 4y ago

I work in a field where the training data is extremely sparse because labeled data is incredibly time consuming and labor intensive to come by. There’s ways to automate labeling, but I don’t really trust those methods fully cause I’ve seen how the sausage is made, so to say. Everybody and their dog is trying to do machine learning, but I question how useful or real the results are because I know that the data underlying the models is of questionable quality. I don’t know what’s next for the field, but it’ll probably be some high profile disaster.

johnmw 4y ago

I am guessing in the future they would just run simulations to generate the untainted training data pre 2022. ...oh wait.