5 ms·

Yep - had this discussion today. The problem is that building high quality modern 'AI' tools requires a metric assload of data. Right now everyone is running ar

by thadt 4y ago

Yep - had this discussion today. The problem is that building high quality modern 'AI' tools requires a metric assload of data. Right now everyone is running around scraping the Internet indiscriminately for that data and building the best things that they can. The next step is that the creators of said training data want their cut, and the lawyers step in and shake out the industry for the next few years. Precedent is set, laws are made and content creators get to license their data for training these AI models. Once those laws are in place, the moat is dug - and large corporations with access to capital vacuum up good training data in bulk. The whole time loudly proclaiming that they're 'empowering creators' and 'rewarding original thinkers.' Now the large corporations have the largest and best datasets tied up and neat, deep moat. Any startup wanting to challenge the incumbents could have significantly better models, but without access to the same quality and volume of training data, they'll be at a massive disadvantage.

mountainriver 4y ago
Eh not really, we aren’t far off from AI being able to use browsers just like humans and operate in the world like us. There are no reasonable laws that will be able to prevent this
- chii 4y ago
  > There are no reasonable laws that will be able to prevent this any law can be legislated, if it can be put down as text. It may be considered unreasonable, but if the majority of people _do_ agree, then it could be done.
- anon291 4y ago
  Yeah there are... There is no fair use for AI. Fair use is about human rights to their own thoughts. AI does not have that. The law is simple... You have to have a license to the data to train a model with that data. If you don't... Get ready to be sued.
  - biscottigelato 4y ago
    Becareful of regulatory capture tho. If the one with AI headstart can capture a huge swath of value, said value can be used to lobby and bribe legislators or the executives of the government. There's no good way to put the AI genie back in the bottle, or make sure it will not create catastrophic undesired consequences. Only way is forward, even if there is no good path in sight.
  - flangola7 4y ago
    > The law is simple... You have to have a license to the data to train a model with that data. If you don't... Get ready to be sued. This won't be the case everywhere. UK has actually already explicitly ruled the opposite
- andsoitis 4y ago
  > operate in the world like us To operate in the real world like humans you need to have motivation. What is an AI motivated by?
  - Animats 4y ago
    Its goal function.
    - andsoitis 4y ago
      Sure, but what would it be?
Kranar 4y ago
Yeah I posted this in another thread but your claim is simply untrue. There is little to no need for anyone to scrape the web themselves, CommonCrawl does it for free and provides access to all that data for free, they are a charity similar to The Internet Archive: https://commoncrawl.org/ https://commoncrawl.org/ As for using that data to train a model, I've done it at my own company and it's not that expensive, certainly not the astronomically expensive figures people seem to think it is. Maybe a hobbyist can't do it for themselves, but my small company managed to get pretty good results training on our existing hardware. It's hard to give concrete figures because we manage our own hardware instead of using AWS, but all in we're talking on the order of 100k, as opposed to what I've heard people on HN think it costs, as if it's a $10 million+ enterprise.
- anon291 4y ago
  Yeah and the model will be useless when the courts determine it's illegal to use it commercially without having properly licensed the data. Remember... Bits have color and AI models are a rainbow.
  - deleted 4y ago
    [deleted]
  - joenot443 4y ago
    “The courts” don’t have rules which apply globally for this sort of thing. My understanding is that even if the models were made illegal to host in America, there’s no shortage of other countries much looser copyright laws where one could hypothetically host such a thing. There are clearnet sites which host entire movies and games and there have been since the dawn of the internet, wouldn’t you expect those would be bigger targets to go after if the powers were there? All that to say, I’m not bullish on the future global leverage of American copyright law, especially when it comes to AI models.
    - piva00 4y ago
      But courts have jurisdiction over their consumer markets, if a court in the US says it's illegal to use non-licenced data for training models and any product doing so can't be used/sold in the US you have lost access to a massive consumer market. Case in point: GDPR, you can vacuum as much private data as you wish from non-EU citizens, the moment you touch EU-citizen data you must abide by the GDPR or you don't have access to a market of 400+ million of some of the wealthiest countries in the world. You don't need global jurisdiction when your economical power is large enough.
htag 4y ago
I wonder how correct this is. There are two growth factors here. One is the amount of data required to train a premium model is increasing. The second is that the amount of data being generated is increasing. I think the future you described is possible. I also think it's possible that corporations are overpaying for data that will become stale after several years and new fresh data is generated in such great quantity/quality that access to it is not capital intensive.
anigbrowl 4y ago
high quality modern 'AI' tools requires a metric assload of data There seems to be a lot of different ideas about 'quality'. A lot of people want a better-than-search tool that knows vast amounts and can be assigned tasks. I'm personally much more interested in a reasoning machine, I don't need it to include the entire contents of the New York Public Library.
dchuk 4y ago
Something I’ve been pondering lately: with the release of all of these content generating language models lately, what will happen to the “scrape and train” process going forward? Aren’t we about to realize an infinite loop problem basically where the training sets will be full of ai generated content that isn’t actually useful for training? Seems like we almost need pre-ai scraped datasets, almost like how it’s sometimes very useful to have metals that were created before the first atom bombs (I don’t know too much about this, just that there’s a definite land in the radiated sand around that timeframe).

by thadt 4y ago

mountainriver 4y ago

Eh not really, we aren’t far off from AI being able to use browsers just like humans and operate in the world like us. There are no reasonable laws that will be able to prevent this

chii 4y ago
> There are no reasonable laws that will be able to prevent this any law can be legislated, if it can be put down as text. It may be considered unreasonable, but if the majority of people _do_ agree, then it could be done.
anon291 4y ago
Yeah there are... There is no fair use for AI. Fair use is about human rights to their own thoughts. AI does not have that. The law is simple... You have to have a license to the data to train a model with that data. If you don't... Get ready to be sued.
- biscottigelato 4y ago
  Becareful of regulatory capture tho. If the one with AI headstart can capture a huge swath of value, said value can be used to lobby and bribe legislators or the executives of the government. There's no good way to put the AI genie back in the bottle, or make sure it will not create catastrophic undesired consequences. Only way is forward, even if there is no good path in sight.
- flangola7 4y ago
  > The law is simple... You have to have a license to the data to train a model with that data. If you don't... Get ready to be sued. This won't be the case everywhere. UK has actually already explicitly ruled the opposite
andsoitis 4y ago
> operate in the world like us To operate in the real world like humans you need to have motivation. What is an AI motivated by?
- Animats 4y ago
  Its goal function.
  - andsoitis 4y ago
    Sure, but what would it be?

Kranar 4y ago

Yeah I posted this in another thread but your claim is simply untrue. There is little to no need for anyone to scrape the web themselves, CommonCrawl does it for free and provides access to all that data for free, they are a charity similar to The Internet Archive: https://commoncrawl.org/ https://commoncrawl.org/ As for using that data to train a model, I've done it at my own company and it's not that expensive, certainly not the astronomically expensive figures people seem to think it is. Maybe a hobbyist can't do it for themselves, but my small company managed to get pretty good results training on our existing hardware. It's hard to give concrete figures because we manage our own hardware instead of using AWS, but all in we're talking on the order of 100k, as opposed to what I've heard people on HN think it costs, as if it's a $10 million+ enterprise.

anon291 4y ago
Yeah and the model will be useless when the courts determine it's illegal to use it commercially without having properly licensed the data. Remember... Bits have color and AI models are a rainbow.
- deleted 4y ago
  [deleted]
- joenot443 4y ago
  “The courts” don’t have rules which apply globally for this sort of thing. My understanding is that even if the models were made illegal to host in America, there’s no shortage of other countries much looser copyright laws where one could hypothetically host such a thing. There are clearnet sites which host entire movies and games and there have been since the dawn of the internet, wouldn’t you expect those would be bigger targets to go after if the powers were there? All that to say, I’m not bullish on the future global leverage of American copyright law, especially when it comes to AI models.
  - piva00 4y ago
    But courts have jurisdiction over their consumer markets, if a court in the US says it's illegal to use non-licenced data for training models and any product doing so can't be used/sold in the US you have lost access to a massive consumer market. Case in point: GDPR, you can vacuum as much private data as you wish from non-EU citizens, the moment you touch EU-citizen data you must abide by the GDPR or you don't have access to a market of 400+ million of some of the wealthiest countries in the world. You don't need global jurisdiction when your economical power is large enough.

htag 4y ago

I wonder how correct this is. There are two growth factors here. One is the amount of data required to train a premium model is increasing. The second is that the amount of data being generated is increasing. I think the future you described is possible. I also think it's possible that corporations are overpaying for data that will become stale after several years and new fresh data is generated in such great quantity/quality that access to it is not capital intensive.

anigbrowl 4y ago

high quality modern 'AI' tools requires a metric assload of data There seems to be a lot of different ideas about 'quality'. A lot of people want a better-than-search tool that knows vast amounts and can be assigned tasks. I'm personally much more interested in a reasoning machine, I don't need it to include the entire contents of the New York Public Library.

dchuk 4y ago

Something I’ve been pondering lately: with the release of all of these content generating language models lately, what will happen to the “scrape and train” process going forward? Aren’t we about to realize an infinite loop problem basically where the training sets will be full of ai generated content that isn’t actually useful for training? Seems like we almost need pre-ai scraped datasets, almost like how it’s sometimes very useful to have metals that were created before the first atom bombs (I don’t know too much about this, just that there’s a definite land in the radiated sand around that timeframe).