5 ms·
This is the https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox and it's what always happens in these cases.
by Erethon 2y ago
This is the https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox and it's what always happens in these cases.
- ben_w 2y agoIt does happen, but not always. For example, food got cheaper and consumption has increased to the extent that obesity is a major problem, but this is much less than you might conclude from the degree to which productivity has increased per farmer. For image generation, the energy needed to create an image is rapidly approaching the energy cost of a human noticing that they've seen an image — once it gets cheap enough (and good enough) to have it replace game rendering engines, we can't really spend meaningfully more on it. (Probably. By that point they may be good enough to be trainers for other AI, or we might not need any better AI — impossible to know at this point). For text generation, difficult to tell because e.g. source code and legal code have a lot of text.
- _heimdall 2y agoFood may be a bit of an outlier, the number of consumers won't change quickly in response and each person can only eat so much. When it comes to converting electricity into images and text, there really is no upper bound in sight. People are happy to load the internet up with as much content as they can churn out.
- ben_w 2y agoYou don't see an end at the level of everyone having it at 60fps (or so) in each eye?
- _heimdall 2y agoI'm not quite sure what you mean, 60fps would have something to do with output displays but nothing to do with the content. There's no upper bound to how much content people would have LLMs make, whether that content is being consumed on cell phone screens or some kind of in-eye display.
- ben_w 2y agoIf you generate a new image 60 times per second, that's reasonably described as "60 fps", this is how the output of video game engines has been described for at least 25 years*. If everyone's doing that all day every day on each eye, that's a reasonable guess of an upper bound: you as a human cannot actually consume more even if you make it. GANs can already do that speed, but any given GAN is a specialist AI and not a general model; diffusion models are general, but they're slower (best generation speed I've seen is 4-5 frames per second on unknown hardware). LLMs aren't really suited to doing images at all, but can control other models (this is what ChatGPT does when "it makes an image" — it calls out to DALL•E). * how long I've been paying attention to that, not a detailed historical analysis
- _heimdall 2y agoSure, I supposed you could calculate a limit by looking at how many human eyes there are, how many frames per second they can see, and max resolution visible. That still isn't actually a limit on how many images could be made, only how many could be consumed. That said, if we got to such a massive scale I'd expect us to hit other limits first (electricity available, best produced, storage space, network transmission, etc.). Or did I totally misunderstand your example here? I may have misread it completely, if so sorry about that!
- ben_w 2y ago> Sure, I supposed you could calculate a limit by looking at how many human eyes there are, how many frames per second they can see, and max resolution visible. That still isn't actually a limit on how many images could be made, only how many could be consumed. Sure, absolutely. But I can say the same of food, which is why I drew the analogy between them previously. > That said, if we got to such a massive scale I'd expect us to hit other limits first (electricity available, best produced, storage space, network transmission, etc.). Difficult to guess when the quality isn't yet at the right threshold: GANs are already this speed on phone hardware*, so we're not bounded on that specific combination with available electrical energy; on the other hand, 2 years ago I was seeing images for about 3 kJ, which is in the region of hundreds of kilowatts for 2 eyes at 60 fps, which is absolutely going to be a problem… if they were limited to that hardware and with that model (though both are moving targets, I'd be very surprised if the unknown hardware that I've seen doing 4-5 fps was burning 12-15 kW, but it's not strictly speaking impossible it really was that power hungry). * Specifically: on an iPhone 11, BlazeStyleGAN model was generating images in 12.14 ms, which is just over 82 fps — https://research.google/blog/mediapipe-facestylizer-on-device-real-time-few-shot-face-stylization/ https://research.google/blog/mediapipe-facestylizer-on-devic...
- debugnik 2y agoYou're assuming people will create content to consume it, and not just to spam various platforms, competing for attention. Most of it might only be ever consumed by crawlers, if at all.
- UltraSane 2y agoThe ultimate end-game for image-gen AI is a closed-loop system where a computer can monitor sexual arousal levels and generate the most arousing porn possible for the subject. This would be VERY addictive. Unless people can just become completely immune to all pornographic stimuli.
- ben_w 2y agoI'd say the Matrix rather than that; most of us have a refractory period where that would at best do nothing and at worst be actively undesirable.
- wongarsu 2y agoIf we assume that text and images are made for human consumption then there is a limit in how much we can consume. In fact I doubt there is much room for our society's per-person media consumption to increase. There is obviously room for growth in fewer people seeing the same content, and room for some "waste" (i.e. content nobody ever sees). The upper bound (ignoring waste) would be if everybody only saw and read content that nobody else has ever seen and will ever see. But if we assume society continues to function as it does the real limit will be a lot lower. Now maybe waste is a bigger issue with content than with food. I'm not sure. Both have some nonzero cost to waste. It might depend on how content is distributed
- _heimdall 2y agoIts an interesting question for sure. Anecdotally it seems to me like there's a ton of content thrown online that is rarely, if ever, consumed. From bot-generated blog posts to social media posts, surely some of it is never seen or viewed only a few times before it gets buried and never seen again. Market dynamics should push people to stop generating that content if they don't enough value to justify the cost. In practice, though, it hasn't seemed to happen yet and we must be pass a threshold where there's more content created online than we could ever value. It'd make for an interesting study, but short of having verifiable data I have to assume we'll continue increasing the rate at which content is created whether the value is there or not.
- UltraSane 2y agoYandex image search works really well at finding similar images but it also leads you to some very strange parts of the internet that are exactly what you are describing: bot generated pages that almost no one reads.
- ben_w 2y agoMm. I'd would say that text is capable of being extremely useful even when no human reads it, because of source code, maths proofs, etc. But I'm curious: 238 wpm * 0.75 words per token * 16 (waking) hours per day * 83 years * $10.00 / 1M output tokens (current API cost for 4o without batching) means the current cost of making as many tokens as a human can read in a lifetime is $92,300: https://www.wolframalpha.com/input?i=238+words+per+minute+%2F+0.75+words+per+token+*+16+hours+per+day+*+83+years+*+%2410.00+%2F+1e6+tokens https://www.wolframalpha.com/input?i=238+words+per+minute+%2... With these numbers, a well-written project with even a billion lines of code would be a rounding error even if only a thousand people used any specific such software and none of that was ever shared with what other people wanted to get done.
- AcerbicZero 2y agoI think you're missing the broader analogy here; Cheap LLMs == LLMs everywhere. Cheap food == People everywhere. I'm no Malthusian, but the paradox holds here pretty well.
- ben_w 2y agoThe population indeed went up, and at the same time the fertility rate is declining. What Malthus was expecting is that more food would just lead to more people on the knife-edge of famine, and we're wildly far from that in most of the world. (What is paradoxical is that the USA is simultaneously very rich, has high obesity, and somehow manages to also have a huge problem with kids going hungry). The very specific point I'm claiming is that the increased consumption isn't always unbounded.
- AcerbicZero 2y agoThe fertility rate trends are missing the core point here. Your obesity and hunger examples actually reinforce the Jevons paradox - when a resource becomes cheap enough, we find ways to use it even beyond what seems rational. But more importantly, you're still not getting the original Malthusian comparison: Malthus wasn't predicting that cheaper food would make people eat more (obesity) - he was predicting that cheaper food would lead to more total people. Similarly, cheaper AI won't just make individual AIs consume more - it means AI will be deployed everywhere possible. The parallel is about multiplication of instances, not increased individual consumption.
- hhdhdbdb 2y agoWhy is fertility declining? I posit we are hitting non-food constraints. Political ones. Land use constraints. If you build millions of homes fertility will go up.
- ben_w 2y agoThere are millions of empty homes in this world. I'd assume environmental, but there's also more subtle answers than will fit in a comment box — whatever the cause, it has to be near-global. China's building loads more houses, still has a fertility decline.
- skybrian 2y agoImage generation isn't cheap enough until we have sites that work like Google Image search, filling the page with image variations nearly instantly and available for free.
- ben_w 2y agoWe're not a huge distance from that already. https://arxiv.org/abs/2408.14837 https://arxiv.org/abs/2408.14837 Also TIL this is generated at 20 frames per second, the best I've used myself was "only" 4-5; does anyone know the performance and power consumption of a Google TPU?
- hhdhdbdb 2y agoBitcoin is a pure exanple thay shows the limit to energy consumption is how much money people have to throw at it. And if that money is thrown into generating more energy it is a cycle. There is no stomach size and human reproduction constraints. We can waste power as quickly as we can generate more. The only hope is to generate this power greenly.