4 ms·

And companies that can generate the training data at scale will be the highest value ones. These LLMs are the shiniest thing in AI/ML right now b/c the data th

And companies that can generate the training data at scale will be the highest value ones. These LLMs are the shiniest thing in AI/ML right now b/c the data they use for training is already freely available and massive in scale via the internet. There's an entire universe of data that hasn't been collected, curated, and leveraged in the right way yet: - The DNA sequences of all living things - The DNA sequences of all of humanity - The CAD files for all manufactured goods - The motion data for humans working manual jobs - The words spoken by multiple peoples across their entire lifetime

Teever 4y ago
To me the end game of this all is a self replicating machine. I'm watching this whole thing with curiosity, waiting for the first image model to be trained on schematics and circuit diagrams. I want to be able to ask an AI to give me the schematics of the machine it runs on, with instructions on how to build that machine, by hand if need be. And then I want to ask it how to build a humanoid robot that can do the same work, and I'll build the humanoid robot and let it build the machine that the AI runs on. I'm not sure what happens then. I think that might be the end of the economy as we know it. Or perhaps the real start of an economy, and everything that we've experience before then had just been a crude approximation of what a real economic system could be.
- em-bee 4y ago
  the self replicating machines will optimize to become smaller and more efficient until everything turns into gray goo.
  - Teever 4y ago
    Perhaps. I'm more interested in using them to build large objects like rockets to facilitate asteroid mining and the construction of Von Neumann probes.
    - ClumsyPilot 4y ago
      Junt make sure you dont become dissasembked for construction material
    - allturtles 4y ago
      Why would they do that for you? Don't you think that intelligent machines will have their own interests?
      - Teever 4y ago
        Why do any of the machines that we make do anything? Because we designed them that way.
      - numpad0 4y ago
        Perhaps because it'd want to survive and to multiply...?
        luma 4y ago
        How are you and your goals involved in that machine being able to survive and multiply? You are made of carbon which that machine might put to better use.
        Teever 4y ago
        How have you not been hit by a car yet? Cars move very fast, and are made of steel while you move very slow and are made of carbon.
        luma 4y ago
        ...are you suggesting cars can make conscious decisions? If not, then what exactly is your point?
- lofaszvanitt 4y ago
  Don't kid yourself that they will let the average joe have that ability.
  - deleted 4y ago
    [deleted]
  - Teever 4y ago
    I think the cat is out of the bag. Once people.know that these things can be made, they will seek to do it themselves How do you think that 'they' will stop that? At best they can do a holding pattern that keeps people a couple years away, but I mean like, what are they going to do, ban semi conductor fabrication?
    - lofaszvanitt 4y ago
      Imagine people making weapons for themselves and the like by using AI. It will be regulated like hell.
      - Teever 4y ago
        How do you enforce the regulations? Are you going to invade the countries that chose to regulate it differently? What if they have nukes? If people get away with illicit cannabis operations they can get away with illicit AI operations.
        ChatGTP 4y ago
        I’m not sure you want to find out but it probably goes something like taking your computers away from you, using government controlled AI to monitor your movements etc...or, restrictions on the classes of device that people can access. So a high-end graphics card will be treated like a weapon. I actually can see this kind of thing happening sooner than we think, thanks to ideas like yours. The government will team up with corporations and they will do the “research” and provide for us and we will be at their mercy with regards to technology we can access. People will actually be asking for this pretty soon once the general public get wind of the dangers we are facing, it’s really not hard to believe. Look at covid lockdowns and times that hysteria by 10x. Nice dystopia…this might end a lot more like 1984 or blade runner rather than some crazy wild amazing sci-fi movie. Edit: I actually think this is already in play, see America, Taiwan and Netherlands restricting China's access to high-end semi-conductor production. I think there you can actually see the writing is already on the wall and where it's going from here.
- pixl97 4y ago
  And this is why we don't let people build the paperclip optimizer. Making a machine that could do what you say above probably isn't that hard. Making the machine that could do the above without destroying the world is impossible.
  - Teever 4y ago
    How are you going to stop me?
    - pixl97 4y ago
      I mean, in general in the world if you live in a country with a rule of law and you said something like "Hey, I'm working on a project that's going to kill a lot of people" some nice little armed men would show up at your door and arrest you and put you in a concrete and iron cage. But in the same sense I do agree, because that same group that controls the armed men will say to themselves "wow, we should build an AI to monitor everybody on the planet" and then we're in the same position where we've created the world eating monster, but now with even more authoritarianism.
- LarsDu88 4y ago
  I think its good not to overthink this idea of a self-replicating machine. In the GPT-4 paper, the system was given some access to some cloud apis and was able to do things like requisition more resources from the cloud. I imagine the first stage of "self-replication" for these LLMs or their successors is going to be like ordering CPUs from Amazon rather than setting up silicon fabs. You need to get far beyond single human level intelligence and effort to start building chips and we are probably not even there yet!
juujian 4y ago
That sounds like it would be ripe for abuse. If companies get $$$ for big samples, there is an incentive to fake it. And garbage in=garbage out. Quality of AI/ML models might even degrade if the "raw material" is contaminated. Pre-GPT 3.5 data will be really valuable.
0xcde4c3db 4y ago
After playing around with Stable Diffusion for a while, I'm convinced that big media conglomerates like Disney and Warner Bros. Discovery are going to have a huge advantage here because they're in a position to pipe absolutely massive media libraries into models that are big enough to legally become "copies" of the involved works. While some of the angry rhetoric from artists whose work has been used to train models may be overblown, it's far from a stretch to suppose that companies like OpenAI and Stability AI, if they were being sloppy, could overfit to copyrighted training data to the point that the models and some outputs are infringing. I suspect they're aware of this risk. For instance, the differences in consistency and detail between a batch of SD outputs using "American Gothic painting" vs. "Dark Side of the Moon album cover" are very striking; the former is reproduced with remarkable fidelity and consistency, while the latter was clearly observed but has a much more abstract relationship to its iconic source image. I don't think that difference is accidental; I think they went to some effort to ensure that only public domain images were so firmly encoded into the model. That sort of thing is not a problem for entities that own the involved copyrights or have a such a broad license that they might as well own them (and, even failing that, are in a strong position to demand such concessions from artists going forward).
- CuriouslyC 4y ago
  These technologies aren't really going to help big rights holders nearly as much as they're going to help small players and indies. Big rights holders already have efficient pipelines, automation and well tuned art departments. They can already crank out movies that are so effects laden as to be visually overloading non-stop. These tools might let them achieve similar or slightly better quality for less, but it's not going to fundamentally change anything. For example, if Avatar 2 VFX cost 200 million, maybe it'd come down to 20 million, but the other production costs would still be significant. On the other hand, small indies can now do respectable VFX without resorting to CGI, needing expensive makeup/props or green screens. They could film in a field with actors in Halloween costumes, then place them in locations built using simple geometry in blender, and then AI will texture, style and light everything, modulate voices, etc. It'll be a game changer in terms of the types of films people will be able to make at the <100k price point.
koeng 4y ago
What about the SRA or Genbank? Those are pretty readily collected DNA databases. You can get curation from uniprot or embl or pdb or kegg or rhea.
vishal0123 4y ago
I think the biggest value would be products tracking users, just as it currently is, like Google, Facebook etc. If assuming AI could learn from low quality data like humans could now, these companies have a huge dataset consisting of multiple GBs per living person that is available to them now including (verified) human written texts, search history, browsing history, video call logs/transcripts, translation transcripts etc. In future company could even pay users to get access to their keyboard and mic to get the data that is verified to be human.
sva_ 4y ago
This comment read like the intro to some kind of dystopian movie.

Teever 4y ago

To me the end game of this all is a self replicating machine. I'm watching this whole thing with curiosity, waiting for the first image model to be trained on schematics and circuit diagrams. I want to be able to ask an AI to give me the schematics of the machine it runs on, with instructions on how to build that machine, by hand if need be. And then I want to ask it how to build a humanoid robot that can do the same work, and I'll build the humanoid robot and let it build the machine that the AI runs on. I'm not sure what happens then. I think that might be the end of the economy as we know it. Or perhaps the real start of an economy, and everything that we've experience before then had just been a crude approximation of what a real economic system could be.

em-bee 4y ago
the self replicating machines will optimize to become smaller and more efficient until everything turns into gray goo.
- Teever 4y ago
  Perhaps. I'm more interested in using them to build large objects like rockets to facilitate asteroid mining and the construction of Von Neumann probes.
  - ClumsyPilot 4y ago
    Junt make sure you dont become dissasembked for construction material
  - allturtles 4y ago
    Why would they do that for you? Don't you think that intelligent machines will have their own interests?
    - Teever 4y ago
      Why do any of the machines that we make do anything? Because we designed them that way.
    - numpad0 4y ago
      Perhaps because it'd want to survive and to multiply...?
      - luma 4y ago
        How are you and your goals involved in that machine being able to survive and multiply? You are made of carbon which that machine might put to better use.
        Teever 4y ago
        How have you not been hit by a car yet? Cars move very fast, and are made of steel while you move very slow and are made of carbon.
        luma 4y ago
        ...are you suggesting cars can make conscious decisions? If not, then what exactly is your point?
lofaszvanitt 4y ago
Don't kid yourself that they will let the average joe have that ability.
- deleted 4y ago
  [deleted]
- Teever 4y ago
  I think the cat is out of the bag. Once people.know that these things can be made, they will seek to do it themselves How do you think that 'they' will stop that? At best they can do a holding pattern that keeps people a couple years away, but I mean like, what are they going to do, ban semi conductor fabrication?
  - lofaszvanitt 4y ago
    Imagine people making weapons for themselves and the like by using AI. It will be regulated like hell.
    - Teever 4y ago
      How do you enforce the regulations? Are you going to invade the countries that chose to regulate it differently? What if they have nukes? If people get away with illicit cannabis operations they can get away with illicit AI operations.
      - ChatGTP 4y ago
        I’m not sure you want to find out but it probably goes something like taking your computers away from you, using government controlled AI to monitor your movements etc...or, restrictions on the classes of device that people can access. So a high-end graphics card will be treated like a weapon. I actually can see this kind of thing happening sooner than we think, thanks to ideas like yours. The government will team up with corporations and they will do the “research” and provide for us and we will be at their mercy with regards to technology we can access. People will actually be asking for this pretty soon once the general public get wind of the dangers we are facing, it’s really not hard to believe. Look at covid lockdowns and times that hysteria by 10x. Nice dystopia…this might end a lot more like 1984 or blade runner rather than some crazy wild amazing sci-fi movie. Edit: I actually think this is already in play, see America, Taiwan and Netherlands restricting China's access to high-end semi-conductor production. I think there you can actually see the writing is already on the wall and where it's going from here.
pixl97 4y ago
And this is why we don't let people build the paperclip optimizer. Making a machine that could do what you say above probably isn't that hard. Making the machine that could do the above without destroying the world is impossible.
- Teever 4y ago
  How are you going to stop me?
  - pixl97 4y ago
    I mean, in general in the world if you live in a country with a rule of law and you said something like "Hey, I'm working on a project that's going to kill a lot of people" some nice little armed men would show up at your door and arrest you and put you in a concrete and iron cage. But in the same sense I do agree, because that same group that controls the armed men will say to themselves "wow, we should build an AI to monitor everybody on the planet" and then we're in the same position where we've created the world eating monster, but now with even more authoritarianism.
LarsDu88 4y ago
I think its good not to overthink this idea of a self-replicating machine. In the GPT-4 paper, the system was given some access to some cloud apis and was able to do things like requisition more resources from the cloud. I imagine the first stage of "self-replication" for these LLMs or their successors is going to be like ordering CPUs from Amazon rather than setting up silicon fabs. You need to get far beyond single human level intelligence and effort to start building chips and we are probably not even there yet!

juujian 4y ago

That sounds like it would be ripe for abuse. If companies get $$$ for big samples, there is an incentive to fake it. And garbage in=garbage out. Quality of AI/ML models might even degrade if the "raw material" is contaminated. Pre-GPT 3.5 data will be really valuable.

0xcde4c3db 4y ago

After playing around with Stable Diffusion for a while, I'm convinced that big media conglomerates like Disney and Warner Bros. Discovery are going to have a huge advantage here because they're in a position to pipe absolutely massive media libraries into models that are big enough to legally become "copies" of the involved works. While some of the angry rhetoric from artists whose work has been used to train models may be overblown, it's far from a stretch to suppose that companies like OpenAI and Stability AI, if they were being sloppy, could overfit to copyrighted training data to the point that the models and some outputs are infringing. I suspect they're aware of this risk. For instance, the differences in consistency and detail between a batch of SD outputs using "American Gothic painting" vs. "Dark Side of the Moon album cover" are very striking; the former is reproduced with remarkable fidelity and consistency, while the latter was clearly observed but has a much more abstract relationship to its iconic source image. I don't think that difference is accidental; I think they went to some effort to ensure that only public domain images were so firmly encoded into the model. That sort of thing is not a problem for entities that own the involved copyrights or have a such a broad license that they might as well own them (and, even failing that, are in a strong position to demand such concessions from artists going forward).

CuriouslyC 4y ago
These technologies aren't really going to help big rights holders nearly as much as they're going to help small players and indies. Big rights holders already have efficient pipelines, automation and well tuned art departments. They can already crank out movies that are so effects laden as to be visually overloading non-stop. These tools might let them achieve similar or slightly better quality for less, but it's not going to fundamentally change anything. For example, if Avatar 2 VFX cost 200 million, maybe it'd come down to 20 million, but the other production costs would still be significant. On the other hand, small indies can now do respectable VFX without resorting to CGI, needing expensive makeup/props or green screens. They could film in a field with actors in Halloween costumes, then place them in locations built using simple geometry in blender, and then AI will texture, style and light everything, modulate voices, etc. It'll be a game changer in terms of the types of films people will be able to make at the <100k price point.

koeng 4y ago

What about the SRA or Genbank? Those are pretty readily collected DNA databases. You can get curation from uniprot or embl or pdb or kegg or rhea.

vishal0123 4y ago

I think the biggest value would be products tracking users, just as it currently is, like Google, Facebook etc. If assuming AI could learn from low quality data like humans could now, these companies have a huge dataset consisting of multiple GBs per living person that is available to them now including (verified) human written texts, search history, browsing history, video call logs/transcripts, translation transcripts etc. In future company could even pay users to get access to their keyboard and mic to get the data that is verified to be human.

sva_ 4y ago

This comment read like the intro to some kind of dystopian movie.