4 ms·

The usefulness of AI depends on training data availability. The reason OpenAI et al. were able to surprise everyone is thanks to taking everyone by surprise and

The usefulness of AI depends on training data availability. The reason OpenAI et al. were able to surprise everyone is thanks to taking everyone by surprise and using their data for training without consent. As the public is catching on[0], what we may get is not some insanely genius AI but a fragmented, private web where no one is stupid enough to publish original content in the open anymore (given all incentives, psychological and financial, have been destroyed) and models choking on themselves having nothing to be trained on except their own output. This is my reasoning for giving higher probability to it being a peak (or very near it). There will be cool, actually useful instances of AI within specialized niches, which it could well transform, but otherwise everyone will go about their life normally. [0] https://twitter.com/sonnyrossdraws/status/1610002959043121160 https://twitter.com/sonnyrossdraws/status/161000295904312116...

foruhar 4y ago
Taking data without consent is a real issue. There is still lots of data out there that is free of copyright. I'd be curious to see a model that is trained solely on public domain data (perhaps with an option to include creative commons-compliant data). I think there is plenty of knowledge that is in the free and clear to make a very useful LL and/or stable diffusion model. We may miss out on Wegovy and air fryer reviews, articles on the how to beat the stock market with Numpy, and manga art styles yet there is plenty of a few decades ago that would make for a useful "AI." Even Steamboat Willie may soon be in play.
- anileated 4y ago
  Yeah, dated content would be the only reliable training data.
- pixl97 4y ago
  Eh, you're just switching problems with the 'consent' model. I'm very much in the camp that the corpus of human knowledge is not some companies IP, this just pushes ownership further into hands of large and well monied companies and further baits patent/IP trolls to lock up collective knowledge.
  - nightski 4y ago
    I look at it like it's some companies IP in the way oil/gas companies sell earth's resources. It takes a lot of work to transform raw crude into usable product, similarly OpenAI and others put a ton of money/resources/work into transforming that knowledge into a workable model.
    - freejazz 4y ago
      Great point, no issues with oil and gas companies as a business model
    - pixl97 4y ago
      Again this gets particularly messy. With oil there is very strong chain of possession. I can't copy your raw oil at little to no cost, and for the most part the next barrel of oil I pump out of the ground is not made of pieces of the past barrel of oil I pumped out of the ground. Each barrel of oil is a wholly separate entity. If I make all past oil disappear, you still have your barrel. Information is not like that at all. It is far more often a continuum of large bits of the past with small changes that redefine it's usage. If I took all bits of past knowledge out of your IP set, you'd be left with something useless in incomplete in almost every case. Trying to treat IP like a physical artifact leads to a multitude of failures.
    - anileated 4y ago
      1. In your analogy a company is processing resources that humans didn’t create. Dead biomass from ages ago is not a fruit of your work, though note that even then you would expect extracting those resources from the ground to be taxed and proceeds to benefit you if you happen to be living on said ground. 2. Unlike petrol vs. raw oil, LLM output is not necessarily “better” than its source material. Indeed there’re plenty of authors that did both extensive work and eloquently written about it, so when LLM is asked a question on the topic their work is among the very few sources—when I am talking about LLM attributing output, I mostly mean instances like these (not when LLM aggregates some really common knowledge). The danger I described is when people are no longer motivated to do such work in the open, assuming it’s then scraped and monetized by LLM operators—or worse, undetectably modified by LLM designers to inject sentiment the original author didn’t subscribe to.

foruhar 4y ago

Taking data without consent is a real issue. There is still lots of data out there that is free of copyright. I'd be curious to see a model that is trained solely on public domain data (perhaps with an option to include creative commons-compliant data). I think there is plenty of knowledge that is in the free and clear to make a very useful LL and/or stable diffusion model. We may miss out on Wegovy and air fryer reviews, articles on the how to beat the stock market with Numpy, and manga art styles yet there is plenty of a few decades ago that would make for a useful "AI." Even Steamboat Willie may soon be in play.

anileated 4y ago
Yeah, dated content would be the only reliable training data.
pixl97 4y ago
Eh, you're just switching problems with the 'consent' model. I'm very much in the camp that the corpus of human knowledge is not some companies IP, this just pushes ownership further into hands of large and well monied companies and further baits patent/IP trolls to lock up collective knowledge.
- nightski 4y ago
  I look at it like it's some companies IP in the way oil/gas companies sell earth's resources. It takes a lot of work to transform raw crude into usable product, similarly OpenAI and others put a ton of money/resources/work into transforming that knowledge into a workable model.
  - freejazz 4y ago
    Great point, no issues with oil and gas companies as a business model
  - pixl97 4y ago
    Again this gets particularly messy. With oil there is very strong chain of possession. I can't copy your raw oil at little to no cost, and for the most part the next barrel of oil I pump out of the ground is not made of pieces of the past barrel of oil I pumped out of the ground. Each barrel of oil is a wholly separate entity. If I make all past oil disappear, you still have your barrel. Information is not like that at all. It is far more often a continuum of large bits of the past with small changes that redefine it's usage. If I took all bits of past knowledge out of your IP set, you'd be left with something useless in incomplete in almost every case. Trying to treat IP like a physical artifact leads to a multitude of failures.
  - anileated 4y ago
    1. In your analogy a company is processing resources that humans didn’t create. Dead biomass from ages ago is not a fruit of your work, though note that even then you would expect extracting those resources from the ground to be taxed and proceeds to benefit you if you happen to be living on said ground. 2. Unlike petrol vs. raw oil, LLM output is not necessarily “better” than its source material. Indeed there’re plenty of authors that did both extensive work and eloquently written about it, so when LLM is asked a question on the topic their work is among the very few sources—when I am talking about LLM attributing output, I mostly mean instances like these (not when LLM aggregates some really common knowledge). The danger I described is when people are no longer motivated to do such work in the open, assuming it’s then scraped and monetized by LLM operators—or worse, undetectably modified by LLM designers to inject sentiment the original author didn’t subscribe to.