4 ms·

We're really at the cusp of gen AI and we've barely scratched the surface. Two Reddit threads really highlight this. - ~10 years ago: https://www.reddit.com/r

We're really at the cusp of gen AI and we've barely scratched the surface. Two Reddit threads really highlight this. - ~10 years ago: https://www.reddit.com/r/StableDiffusion/comments/y9zxj1/you_think_thats_insane_heres_what_an_aigenerated/ https://www.reddit.com/r/StableDiffusion/comments/y9zxj1/you... - Today: https://www.reddit.com/r/StableDiffusion/comments/1f0b45f/flux_is_a_gamechanger_for_character_wardrobe/ https://www.reddit.com/r/StableDiffusion/comments/1f0b45f/fl... The upgrade in throughput from GPT-4 to GPT-4o and GPT-4o Mini actually unlocked use cases for the startup I'm at. People that think demand for GPU compute capacity is going to decrease are probably wrong in the same way that people who thought the demand for faster processors and more RAM would wane were wrong. We are just barely at the start of finding the use cases and how to eat those GPU cycles. > The need for specialist hardware, he observed, is a sign of the "brute force" phase of AI, in which programming techniques are yet to be refined and powerful hardware is needed. "If you cannot find the elegant way of programming … it [the AI application] dies," he added. The thing is that even if there is an elegant and efficient programmatic/algorithmic solution, having more and faster hardware only makes it better and pushes the limits even more.

dartos 2y ago
> We're really at the cusp of gen AI and we've barely scratched the surface. What makes you say that? I don’t really see a trend of AI generated content getting better, just more players in the space. I think we’re at the peak of AI gen, I doubt we’ll see much improvement in quality (it’s already pretty good and it seems like all the low hanging fruit is gone), just more specialized models. Maybe some better tooling to give artists more control having seen it grow more and more since 2016 when GANs started making fairly realistic human faces, this seems like the end goal already.
- CharlieDigital 2y ago
  > I don’t really see a trend of AI generated content getting better You see that second link as the endpoint? That there's nowhere to go from there? How about you can have a holodeck type experience with Apple Vision Pro? Literally generate any scenario you want? Download generated scenarios and customize it however you want in real time? Entire animation workflows changed from animating models to using voice and text to describe scenes and actions. Lowering the barrier of digital film making to the same level and ease of use as photo editing apps today -- even easier. You really think that the second link is the peak of gen AI? You really think that nothing else and no more major industry shifts are going to happen when gen AI gets cheaper, faster, algorithms get better, and hardware gets more powerful?
  - abletonlive 2y ago
    Some people are just pessimistic rather than unimaginative. It seems like a miserable way to live life but it's fairly common IME.
    - dartos 2y ago
      “I think you’re a downer so you must be miserable” is a really great thing to add to the conversation. Thank you.
      - abletonlive 2y ago
        Well the other commentator presumed people were just unimaginative. I was giving an alternative option.
    - dukeyukey 2y ago
      People are burned by crypto, metaverse, web3, and all the other stuff the tech industry came out with over the last few years that crashed and burned. Optimism is great and underrated, but you can't sum up the same enthusiasm for everything. And this is coming from someone thoroughly bullish on AI!
      - CharlieDigital 2y ago
        I think there's a distinction here because there's very tangible output from gen AI (content) and we can see it getting better, more advanced, more capable, and more realistic. The applications are obvious: film making, content creation, teaching, etc. This is in contrast to crypto which was/is quite abstract (as is money in the first place) and the metaverse which required investing hundreds of dollars in specialized hardware. In contrast, our world is surrounded by visual content so the applications and utility of gen AI seems far more obvious for the layperson.
        pona-a 2y ago
        And what we do see is terrible; bland art, more spam and political astroturfing than ever in human history, bad code, and ignorant lessons, all to the tune of a PR campaign to shift the Overton window towards praising incompetence and denigrating hard work. The only real accomplishments of LLMs were how good the proposed use-cases sound on paper under competent implementation, and a theoretical solution to unstructured data parsing that's still too heavy to be worth a tiny bump in performance. Do you want to live in a future where all human thought has been replaced by its surface level reproductions, made by big tech stuffing copyrighted works into a GPU farm with near-zero human labor? We both know it won't benefit you and me, our role is merely transitory in bootstrapping their self-improvement under the guise of a paid product, nor had the relationship between us and these tools been in any shape collaborative in the first place. This fantasy targets the owner class, which can finally dream of labor decoupled from the laborer, the work simply costing no more than the price of electricity, all without the demands for livable compensation or following best practice. Even if the LLMs gained above-human performance in all domains of knowledge shortly followed by institution of a universal basic income, their invention will still have only been a force of stagnation, learned intellectual helpless, and overconsumption.
- HarHarVeryFunny 2y ago
  From a machine learning perspective, "generative AI" embraces any generative method, so includes diffusion-based image generation as well as text-generating LLMs, but in the world of C-suite execs "GenAI" really refers to LLMs which they dream will replace developers, customer service agents, etc, etc.
  - CharlieDigital 2y ago
    Even if it's just text, the point is that LLMs are currently still really bad. Some models we still have to correct it for outputting Markdown code fences for JSON (we just brute force and string replace it). We have to give it minute details or fine-tune it on a large enough sample set to get the results we want. Even just LLMs right now still have a ways to go in terms of how good they are at working without needing precise instructions and micro-corrections and throughput.
  - dartos 2y ago
    Until AI became a buzzword it was considered a subfield of ML. Decision trees, for example, were not considered AI.
- marcosdumay 2y ago
  Both yours and the GP's are weird things to believe about a problem. You seem to be talking about a technical solution, but naming it after the problem.
Vox_Leone 2y ago
I agree. While GPUs have been indispensable for training and deploying large language models, their role in the field of computer vision is becoming increasingly critical. The expanding applications, increasing complexity of models, real-time processing needs, growing dataset sizes, and other factors, seem to indicate that the demand for GPUs is likely to rise in the mid term.
- CharlieDigital 2y ago
  I think if you take a device like the Apple Vision Pro or Meta Quest: imagine that you can procedurally generate any experience like a holodeck from just a few instructions and fine tuning the generated world. We're not even close. Imagine YouTube except you can fully generate shorts. Realistic, 2D animated, 3D, whatever your imagination desires. Imagine how that changes storytelling and content creation. More GPUs, please.
  - dghlsakjg 2y ago
    I'm not super up on the latest gaming, but I imagine that procedurally generated in-game characters that you can have a voice conversation with are just around the corner. The ability to have a conversation with a character with a back story is definitely going to be an interesting addition in the bery near future.
    - CharlieDigital 2y ago
      Google DeepMind has a fully procedurally generated Doom: https://gamengen.github.io/ https://gamengen.github.io/
  - jononor 2y ago
    Driven by companies like Facebook and Meta, optimizing purely for engagement and ad viewing time and spending on content+products. The Matrix is ever closer. Neuralink already prepping gen 2.
- ActorNightly 2y ago
  The interesting thing is that at some point, its very possible that LLM (or rather just transformer based large models) are going to be used to create ASIC or at least used to program much cheaper FPGAs for the specific applications, so in fact, GPU use will still decline.
fwip 2y ago
By "gen AI" do you mean "generative" or "general?" They're very different statements.
- CharlieDigital 2y ago
  "General" is typically referred to AGI[0]. "Generative" is typically referred to as gen AI. This is pretty standard nomenclature. [0] https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
janwas 2y ago
I acknowledge Jevon and that demand may increase. But the original point was actually that specialized HW will be outcompeted by newer algorithms on more flexible HW. I have seen this play out several times, for example with brute force Sum of Abs Difference instructions that no video codec uses anymore, because a sparse search is faster than brute force HW. Personal opinion: this will happen again. Why should we focus on how to fill GPU cycles, rather than invent the future of how to do better?

dartos 2y ago

> We're really at the cusp of gen AI and we've barely scratched the surface. What makes you say that? I don’t really see a trend of AI generated content getting better, just more players in the space. I think we’re at the peak of AI gen, I doubt we’ll see much improvement in quality (it’s already pretty good and it seems like all the low hanging fruit is gone), just more specialized models. Maybe some better tooling to give artists more control having seen it grow more and more since 2016 when GANs started making fairly realistic human faces, this seems like the end goal already.

CharlieDigital 2y ago
> I don’t really see a trend of AI generated content getting better You see that second link as the endpoint? That there's nowhere to go from there? How about you can have a holodeck type experience with Apple Vision Pro? Literally generate any scenario you want? Download generated scenarios and customize it however you want in real time? Entire animation workflows changed from animating models to using voice and text to describe scenes and actions. Lowering the barrier of digital film making to the same level and ease of use as photo editing apps today -- even easier. You really think that the second link is the peak of gen AI? You really think that nothing else and no more major industry shifts are going to happen when gen AI gets cheaper, faster, algorithms get better, and hardware gets more powerful?
- abletonlive 2y ago
  Some people are just pessimistic rather than unimaginative. It seems like a miserable way to live life but it's fairly common IME.
  - dartos 2y ago
    “I think you’re a downer so you must be miserable” is a really great thing to add to the conversation. Thank you.
    - abletonlive 2y ago
      Well the other commentator presumed people were just unimaginative. I was giving an alternative option.
  - dukeyukey 2y ago
    People are burned by crypto, metaverse, web3, and all the other stuff the tech industry came out with over the last few years that crashed and burned. Optimism is great and underrated, but you can't sum up the same enthusiasm for everything. And this is coming from someone thoroughly bullish on AI!
    - CharlieDigital 2y ago
      I think there's a distinction here because there's very tangible output from gen AI (content) and we can see it getting better, more advanced, more capable, and more realistic. The applications are obvious: film making, content creation, teaching, etc. This is in contrast to crypto which was/is quite abstract (as is money in the first place) and the metaverse which required investing hundreds of dollars in specialized hardware. In contrast, our world is surrounded by visual content so the applications and utility of gen AI seems far more obvious for the layperson.
      - pona-a 2y ago
        And what we do see is terrible; bland art, more spam and political astroturfing than ever in human history, bad code, and ignorant lessons, all to the tune of a PR campaign to shift the Overton window towards praising incompetence and denigrating hard work. The only real accomplishments of LLMs were how good the proposed use-cases sound on paper under competent implementation, and a theoretical solution to unstructured data parsing that's still too heavy to be worth a tiny bump in performance. Do you want to live in a future where all human thought has been replaced by its surface level reproductions, made by big tech stuffing copyrighted works into a GPU farm with near-zero human labor? We both know it won't benefit you and me, our role is merely transitory in bootstrapping their self-improvement under the guise of a paid product, nor had the relationship between us and these tools been in any shape collaborative in the first place. This fantasy targets the owner class, which can finally dream of labor decoupled from the laborer, the work simply costing no more than the price of electricity, all without the demands for livable compensation or following best practice. Even if the LLMs gained above-human performance in all domains of knowledge shortly followed by institution of a universal basic income, their invention will still have only been a force of stagnation, learned intellectual helpless, and overconsumption.
HarHarVeryFunny 2y ago
From a machine learning perspective, "generative AI" embraces any generative method, so includes diffusion-based image generation as well as text-generating LLMs, but in the world of C-suite execs "GenAI" really refers to LLMs which they dream will replace developers, customer service agents, etc, etc.
- CharlieDigital 2y ago
  Even if it's just text, the point is that LLMs are currently still really bad. Some models we still have to correct it for outputting Markdown code fences for JSON (we just brute force and string replace it). We have to give it minute details or fine-tune it on a large enough sample set to get the results we want. Even just LLMs right now still have a ways to go in terms of how good they are at working without needing precise instructions and micro-corrections and throughput.
- dartos 2y ago
  Until AI became a buzzword it was considered a subfield of ML. Decision trees, for example, were not considered AI.
marcosdumay 2y ago
Both yours and the GP's are weird things to believe about a problem. You seem to be talking about a technical solution, but naming it after the problem.

Vox_Leone 2y ago

I agree. While GPUs have been indispensable for training and deploying large language models, their role in the field of computer vision is becoming increasingly critical. The expanding applications, increasing complexity of models, real-time processing needs, growing dataset sizes, and other factors, seem to indicate that the demand for GPUs is likely to rise in the mid term.

CharlieDigital 2y ago
I think if you take a device like the Apple Vision Pro or Meta Quest: imagine that you can procedurally generate any experience like a holodeck from just a few instructions and fine tuning the generated world. We're not even close. Imagine YouTube except you can fully generate shorts. Realistic, 2D animated, 3D, whatever your imagination desires. Imagine how that changes storytelling and content creation. More GPUs, please.
- dghlsakjg 2y ago
  I'm not super up on the latest gaming, but I imagine that procedurally generated in-game characters that you can have a voice conversation with are just around the corner. The ability to have a conversation with a character with a back story is definitely going to be an interesting addition in the bery near future.
  - CharlieDigital 2y ago
    Google DeepMind has a fully procedurally generated Doom: https://gamengen.github.io/ https://gamengen.github.io/
- jononor 2y ago
  Driven by companies like Facebook and Meta, optimizing purely for engagement and ad viewing time and spending on content+products. The Matrix is ever closer. Neuralink already prepping gen 2.
ActorNightly 2y ago
The interesting thing is that at some point, its very possible that LLM (or rather just transformer based large models) are going to be used to create ASIC or at least used to program much cheaper FPGAs for the specific applications, so in fact, GPU use will still decline.

fwip 2y ago

By "gen AI" do you mean "generative" or "general?" They're very different statements.

CharlieDigital 2y ago
"General" is typically referred to AGI[0]. "Generative" is typically referred to as gen AI. This is pretty standard nomenclature. [0] https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

janwas 2y ago

I acknowledge Jevon and that demand may increase. But the original point was actually that specialized HW will be outcompeted by newer algorithms on more flexible HW. I have seen this play out several times, for example with brute force Sum of Abs Difference instructions that no video codec uses anymore, because a sparse search is faster than brute force HW. Personal opinion: this will happen again. Why should we focus on how to fill GPU cycles, rather than invent the future of how to do better?