5 ms·

Ok, so on the generative model modality landscape I'm now aware of: - speech - images - audio samples - text - code - 3d models I've seen basic attempts

by TOMDM 4y ago

Ok, so on the generative model modality landscape I'm now aware of: - speech - images - audio samples - text - code - 3d models I've seen basic attempts at music and video, and based on everything else we've seen getting good results there seems to be mostly a matter of scaling. What content generation modalities are left? Will all corporate generation of these fall to progressively larger models, leaving a (relatively) niche "Made by humans!" industry in it's wake?

r1chardnl 4y ago
Management positions replaced by a supervising AI
- monkeydust 4y ago
  Don't see why not, especially for low impact, high frequency decisions. Some AI guided assistance with option to automate. The next level for auto complete I guess.
  - nuclearnice3 4y ago
    The future is already here -- it's just not evenly distributed. https://www.retailbiz.com.au/latest-news/inside-amazons-ai-powered-supply-chain/ https://www.retailbiz.com.au/latest-news/inside-amazons-ai-p...
- TOMDM 4y ago
  I wonder what your input data would need to be for a competant AI in this space. Finance, goals, delivery timelines, capabilities of the team, employee availability, which employees work well with each other, office politics, regulatory constraints...
  - Filligree 4y ago
    It doesn’t need to work as well as current management, just well enough to be cheaper. Perhaps not even that, factoring in loyalty.
- thanatos519 4y ago
  The problem is finding a good training set.
Keyframe 4y ago
Animation, at least the ‘background one’.
christiangenco 4y ago
HN comments.
shadowfoxx 4y ago
Well, you're gonna need folks to sift through all the generated images and curate the results into something coherent. Taste is still a thing after-all.
- TOMDM 4y ago
  Visions of the future where the consumer has to pick apart "human curated", "human assembled" and "human made" much the same way we do now for cage free and free range eggs at the grocer.
- gersh 4y ago
  That would be the same as curating social media feeds.
snek_case 4y ago
It's not just a matter of different modalities. It's still a matter of sophistication. The end game is endless generative music or video streaming customized to your preferences. Being able to describe a story, or having the AI model take a guess at what you mind find interesting/entertaining and generating a whole TV show or movie for you to watch. Or generating background music while you work and automatically adjusting to your tastes as well as adjusting if you're finding it hard to concentrate or you need to take a call.
- tetris11 4y ago
  Except it wont be, will it. Such things were promised for the internet and we had maybe a good 10 years or so before corps caught up and told us what to watch through their channels. I imagine this being much of the same: AI trained on corp-approved training sets to give suggestions to your preferences that they want. Sure, you could spin up your own and buy a machine that trains on it's own training data, but watch how no one will do that because of the cost, or the diminishing access to untainted resources.
  - suby 4y ago
    It's going to be akin to content creation on youtube, perhaps even just people using youtube as their distribution medium. Anyone can make a youtube video but we don't see everyone creating content. We should see a proliferation of the tech such that lots of small (even one-man) studios pop up pumping out high quality content, but the content is released on a schedule similar to how youtube videos are now. Your preferences come into play through your suggested watch list, it'll be populated from this pre-created media based on whatever preference machine learning algorithm the distribution platform (youtube?) decides. The feedback through watch-metrics will then be used by these micro studios to decide what to create next. It's basically what already happens now with youtube content creation, but the quality of what people will produce will be better than hollywood content movies / tv shows, and the pace of release will be much quicker. Not everyone needs to be training and generating their own content in order for your content preferences to be absolutely saturated with things you'd enjoy watching.
  - 4y ago
thanatos519 4y ago
I'm waiting for the "literal video" generator, which writes and sings new lyrics describing what is happening in the video.
eezurr 4y ago
Music will continue to be made by humans because of strong copyright law. Its illegal to sample (and distribute) > 0 seconds of recorded music. If that makes it into any ai generated music, its game over if you distribute it. Source on sampling: the head audio engineer at Juilliard School of Music.
- dinobones 4y ago
  Why would being a "head audio engineer" at Julliard give you any credibility on AI generated music and sampling/copyright law? Lol.
  - eezurr 4y ago
    Because they work with/teach electronic music/sampling in addition to recording classical acoustic music.
- Geee 4y ago
  It's not sampling. Sampling is copying & pasting, but that doesn't happen, technically. Just like stable diffusion doesn't copy any artworks. AI learns from previous works, but doesn't copy them. It's quite similar to how humans learn and make adaptations based on other work.
  - eezurr 4y ago
    While technically true, the unspoken premise of my argument is that it can and will output distinctive samples derived from the source. E.g. you cant change the pitch, tempo, and add other effects and call the output your own, legally. Its a landmine
    - Baloo 4y ago
      Do you mean it's a minefield* :P Not sure it's as much of a problem as you're making it seem though.. likely there can be enough iteration that it would not be distinguishable. Also lots of mainstream artists ignore copyright anyway (Kanye West etc). Plus you could just use training data that is from public domain stuff if it was really such an issue, there is decades worth of music that are outside of copyright law.
brnaftr361 4y ago
To me these models look worthless. They'd be useless for anything other than BG props with high DOF and complementary lighting, you can see on the rear windows in particular there are artifacts from the topology. If you hit most of that shit with a light from the side it would look horrendous. You can get away with a lot, but I think this is too much. I think future iterations could be promising, but this definitely isn't challenging any pipeline I'm aware of.
- smoldesu 4y ago
  To be fair, these models are no worse than the ones the iPhone makes with LIDAR. It's pretty impressive for being generated from a single static image.
bcjordan 4y ago
Real-time interactive 3D scenes. Imagine Myst, but the 60fps visuals can be anything you can imagine, and anything you can think of can be done within it. Imagine GTA but it doesn't take a AAAA team 5-10 years to flesh out the content to make everything interactable. At some point we'll have a universal Holodeck more powerful than our imaginations, able to simulate slices of our experience better than our dreams. It'll be fun as heck
- bzzzt 4y ago
  At some point we'll have fusion and/or fully self-driving cars. Still the AI only can replicate learned ideas, so you can't do anything that hasn't been done before. Generated content will look 99% there but be just a bit off. Besides that, even if you could do 'anything' in a game, what's the point if nothing you do has meaning?

by TOMDM 4y ago

r1chardnl 4y ago

Management positions replaced by a supervising AI

monkeydust 4y ago
Don't see why not, especially for low impact, high frequency decisions. Some AI guided assistance with option to automate. The next level for auto complete I guess.
- nuclearnice3 4y ago
  The future is already here -- it's just not evenly distributed. https://www.retailbiz.com.au/latest-news/inside-amazons-ai-powered-supply-chain/ https://www.retailbiz.com.au/latest-news/inside-amazons-ai-p...
TOMDM 4y ago
I wonder what your input data would need to be for a competant AI in this space. Finance, goals, delivery timelines, capabilities of the team, employee availability, which employees work well with each other, office politics, regulatory constraints...
- Filligree 4y ago
  It doesn’t need to work as well as current management, just well enough to be cheaper. Perhaps not even that, factoring in loyalty.
thanatos519 4y ago
The problem is finding a good training set.

Keyframe 4y ago

Animation, at least the ‘background one’.

christiangenco 4y ago

HN comments.

shadowfoxx 4y ago

Well, you're gonna need folks to sift through all the generated images and curate the results into something coherent. Taste is still a thing after-all.

TOMDM 4y ago
Visions of the future where the consumer has to pick apart "human curated", "human assembled" and "human made" much the same way we do now for cage free and free range eggs at the grocer.
gersh 4y ago
That would be the same as curating social media feeds.

snek_case 4y ago

It's not just a matter of different modalities. It's still a matter of sophistication. The end game is endless generative music or video streaming customized to your preferences. Being able to describe a story, or having the AI model take a guess at what you mind find interesting/entertaining and generating a whole TV show or movie for you to watch. Or generating background music while you work and automatically adjusting to your tastes as well as adjusting if you're finding it hard to concentrate or you need to take a call.

tetris11 4y ago
Except it wont be, will it. Such things were promised for the internet and we had maybe a good 10 years or so before corps caught up and told us what to watch through their channels. I imagine this being much of the same: AI trained on corp-approved training sets to give suggestions to your preferences that they want. Sure, you could spin up your own and buy a machine that trains on it's own training data, but watch how no one will do that because of the cost, or the diminishing access to untainted resources.
- suby 4y ago
  It's going to be akin to content creation on youtube, perhaps even just people using youtube as their distribution medium. Anyone can make a youtube video but we don't see everyone creating content. We should see a proliferation of the tech such that lots of small (even one-man) studios pop up pumping out high quality content, but the content is released on a schedule similar to how youtube videos are now. Your preferences come into play through your suggested watch list, it'll be populated from this pre-created media based on whatever preference machine learning algorithm the distribution platform (youtube?) decides. The feedback through watch-metrics will then be used by these micro studios to decide what to create next. It's basically what already happens now with youtube content creation, but the quality of what people will produce will be better than hollywood content movies / tv shows, and the pace of release will be much quicker. Not everyone needs to be training and generating their own content in order for your content preferences to be absolutely saturated with things you'd enjoy watching.
- 4y ago

thanatos519 4y ago

I'm waiting for the "literal video" generator, which writes and sings new lyrics describing what is happening in the video.

eezurr 4y ago

Music will continue to be made by humans because of strong copyright law. Its illegal to sample (and distribute) > 0 seconds of recorded music. If that makes it into any ai generated music, its game over if you distribute it. Source on sampling: the head audio engineer at Juilliard School of Music.

dinobones 4y ago
Why would being a "head audio engineer" at Julliard give you any credibility on AI generated music and sampling/copyright law? Lol.
- eezurr 4y ago
  Because they work with/teach electronic music/sampling in addition to recording classical acoustic music.
Geee 4y ago
It's not sampling. Sampling is copying & pasting, but that doesn't happen, technically. Just like stable diffusion doesn't copy any artworks. AI learns from previous works, but doesn't copy them. It's quite similar to how humans learn and make adaptations based on other work.
- eezurr 4y ago
  While technically true, the unspoken premise of my argument is that it can and will output distinctive samples derived from the source. E.g. you cant change the pitch, tempo, and add other effects and call the output your own, legally. Its a landmine
  - Baloo 4y ago
    Do you mean it's a minefield* :P Not sure it's as much of a problem as you're making it seem though.. likely there can be enough iteration that it would not be distinguishable. Also lots of mainstream artists ignore copyright anyway (Kanye West etc). Plus you could just use training data that is from public domain stuff if it was really such an issue, there is decades worth of music that are outside of copyright law.

brnaftr361 4y ago

To me these models look worthless. They'd be useless for anything other than BG props with high DOF and complementary lighting, you can see on the rear windows in particular there are artifacts from the topology. If you hit most of that shit with a light from the side it would look horrendous. You can get away with a lot, but I think this is too much. I think future iterations could be promising, but this definitely isn't challenging any pipeline I'm aware of.

smoldesu 4y ago
To be fair, these models are no worse than the ones the iPhone makes with LIDAR. It's pretty impressive for being generated from a single static image.

bcjordan 4y ago

Real-time interactive 3D scenes. Imagine Myst, but the 60fps visuals can be anything you can imagine, and anything you can think of can be done within it. Imagine GTA but it doesn't take a AAAA team 5-10 years to flesh out the content to make everything interactable. At some point we'll have a universal Holodeck more powerful than our imaginations, able to simulate slices of our experience better than our dreams. It'll be fun as heck

bzzzt 4y ago
At some point we'll have fusion and/or fully self-driving cars. Still the AI only can replicate learned ideas, so you can't do anything that hasn't been done before. Generated content will look 99% there but be just a bit off. Besides that, even if you could do 'anything' in a game, what's the point if nothing you do has meaning?