5 ms·

Imagine saying that someone should pay for the right to make open source software, before you’ve made anything, or proven any kind of market value. In a world w

Imagine saying that someone should pay for the right to make open source software, before you’ve made anything, or proven any kind of market value. In a world with power law distributions, the vast majority of people will not come close to even millions in profit. At one point I was hoping just to run my own GPT API to self fund my measly $7k/mo burn rate. Then I could live life happy as a clam doing research all day in my little shell. Such a dream is completely impossible when rightsholders demand everyone pay for training data, rather than sharing in the profits of the result. It’s backwards. The actual market profit is what matters. But authors are frothing at the mouth, upset that their work is being used at all without their explicit consent. And some of those authors, like Sarah Silverman and more recently former governor Mike Huckabee, are going after LLM companies and demanding injunctions preventing them from using books at all, under any circumstance, unless they go through authors. Good luck going through 100,000 different authors. The net result is that we’re headed for a world where the YouTube of LLMs will dominate the space, and no one else will be able to run their own platforms. I’ve done everything humanly possible to try to steer us away from that outcome — you like using stable diffusion yourself? Imagine if no one released stable diffusion in the first place. But things are not looking good. Lawsuit against Eleuther for books3, courtesy of Huckabee: https://news.ycombinator.com/item?id=37962244 https://news.ycombinator.com/item?id=37962244 Example of authors being furious, though thankfully with an exception for academic curiosity: https://news.ycombinator.com/item?id=37949012 https://news.ycombinator.com/item?id=37949012 Twitter is completely merciless. Digging up some examples to illustrate the point… Here’s one: look through the replies of someone who released a 3D dataset. https://x.com/mattdeitke/status/1678855859089326080?s=61&t=jQbmCk1JqL7depzFWJNuPA https://x.com/mattdeitke/status/1678855859089326080?s=61&t=j... "Eat shit Matt" (5O likes) "Hope you enjoy the storm" "you make me sick" and on and on. I can tell you firsthand that this isn’t unusual; it’s the default reaction now.

stavros 3y ago
The good thing about LLMs (at least when compared to YouTube) is that "all" you need to train an LLM is processing power, so it's easier now (and will be much easier in the future) to do it illegally. Much easier than running a YouTube competitor illegally for any length of time.
vasco 3y ago
> Such a dream is completely impossible when rightsholders demand everyone pay for training data, rather than sharing in the profits of the result. It’s backwards. The actual market profit is what matters I like the way you think, can I also get free books at the store if I'm just reading them for learning and will only make money with the knowledge later, or if I promise that I won't put it in practice? I did a similar deal in university but had to return books to the library every week so I like your plan better.
- pwagland 3y ago
  One big difference with your university analogy is that the university _did_ pay for the resource, often way more than you would as a private citizen, since it is for sharing purposes. Even worse, for e-books, libraries are often limited to how often they can lend the book out, and this was done since physical books "degrade" over time, and so have a limited lend-life, the idea is that the same should hold true for digital books as well… I get the point behind your sarcasm of the first sentence, since that is the essence of the problem.
- TeMPOraL 3y ago
  Well yeah, people sometimes read whole books in a store. Though this use case is done better in facilities that specifically support it - libraries. If you really need to take the books back with you, that's what the library copy machine is for.
sensanaty 3y ago
Why do you deserve to reap the benefits of their labour while not giving them a cent? Just saying "but open source!" Isn't a valid argument, everyone who wants access to someone else's work should be forced to pay, this includes the current big players that have so far been circumventing paying people for their work. It's attitudes like these from the AI sycophants that makes people despise the technology and anyone involved in any of it. I don't give a shit how difficult not having free unfettered access to everyone else's work makes it for you. I don't care that we're supposedly holding the world back (from infinite machine-generated spam made for the sole purpose of making everyone else's lives maximally miserable).
- sillysaurusx 3y ago
  In general, the goal of labor is to enrich the society you live in. Whether it’s capitalism or socialism, the theory is the same: the richer your society, the happier you’ll be, even if you’re poor. Objectively, the poorest in America live far better than the poorest in poor countries. Then there is the harm argument. What is the harm to authors of letting AI models do this? Models aren’t replacing authors any time soon. And even if they were, you won’t be able to stop it; precisely zero times in history have people stood against a wave of new technology. Technology wins 10 out of 10 times in the long run. I am skeptical that authors deserve anything for merely having a transformative use of their work. But I’m also sympathetic. The key is for authors to get a slice of the actual market value. Right now authors are shutting everyone down merely for using their work at all, with no consideration of whether anyone is even profiting. Yet even when authors deserve a cent, how do you decide which authors get how many cents? Do all profits go to copyright holders, as with YouTube? Then there’s no room to run a service. Is half fair? Before or after company expenses? And whatever is allocated, is it distributed evenly among all 100,000 authors, whether they’re Stephen King or just an enthusiastic contributor to AO3? $1M in profits distributed to 100,000 authors is a whopping $10 per author. That’ll buy them a Happy Meal and fries. It’s far more lucrative to make up imaginary damages and sue them in court. It’s immensely complicated, and it’s not at all clear that it’s evil or even exploitative to train AI models on other people’s work.
  - sensanaty 3y ago

stavros 3y ago

The good thing about LLMs (at least when compared to YouTube) is that "all" you need to train an LLM is processing power, so it's easier now (and will be much easier in the future) to do it illegally. Much easier than running a YouTube competitor illegally for any length of time.

vasco 3y ago

> Such a dream is completely impossible when rightsholders demand everyone pay for training data, rather than sharing in the profits of the result. It’s backwards. The actual market profit is what matters I like the way you think, can I also get free books at the store if I'm just reading them for learning and will only make money with the knowledge later, or if I promise that I won't put it in practice? I did a similar deal in university but had to return books to the library every week so I like your plan better.

pwagland 3y ago
One big difference with your university analogy is that the university _did_ pay for the resource, often way more than you would as a private citizen, since it is for sharing purposes. Even worse, for e-books, libraries are often limited to how often they can lend the book out, and this was done since physical books "degrade" over time, and so have a limited lend-life, the idea is that the same should hold true for digital books as well… I get the point behind your sarcasm of the first sentence, since that is the essence of the problem.
TeMPOraL 3y ago
Well yeah, people sometimes read whole books in a store. Though this use case is done better in facilities that specifically support it - libraries. If you really need to take the books back with you, that's what the library copy machine is for.

sensanaty 3y ago

Why do you deserve to reap the benefits of their labour while not giving them a cent? Just saying "but open source!" Isn't a valid argument, everyone who wants access to someone else's work should be forced to pay, this includes the current big players that have so far been circumventing paying people for their work. It's attitudes like these from the AI sycophants that makes people despise the technology and anyone involved in any of it. I don't give a shit how difficult not having free unfettered access to everyone else's work makes it for you. I don't care that we're supposedly holding the world back (from infinite machine-generated spam made for the sole purpose of making everyone else's lives maximally miserable).

sillysaurusx 3y ago
In general, the goal of labor is to enrich the society you live in. Whether it’s capitalism or socialism, the theory is the same: the richer your society, the happier you’ll be, even if you’re poor. Objectively, the poorest in America live far better than the poorest in poor countries. Then there is the harm argument. What is the harm to authors of letting AI models do this? Models aren’t replacing authors any time soon. And even if they were, you won’t be able to stop it; precisely zero times in history have people stood against a wave of new technology. Technology wins 10 out of 10 times in the long run. I am skeptical that authors deserve anything for merely having a transformative use of their work. But I’m also sympathetic. The key is for authors to get a slice of the actual market value. Right now authors are shutting everyone down merely for using their work at all, with no consideration of whether anyone is even profiting. Yet even when authors deserve a cent, how do you decide which authors get how many cents? Do all profits go to copyright holders, as with YouTube? Then there’s no room to run a service. Is half fair? Before or after company expenses? And whatever is allocated, is it distributed evenly among all 100,000 authors, whether they’re Stephen King or just an enthusiastic contributor to AO3? $1M in profits distributed to 100,000 authors is a whopping $10 per author. That’ll buy them a Happy Meal and fries. It’s far more lucrative to make up imaginary damages and sue them in court. It’s immensely complicated, and it’s not at all clear that it’s evil or even exploitative to train AI models on other people’s work.
- sensanaty 3y ago