5 ms·

> how much are GenAI companies willing to invest to get eyeball reflections right This isn't how it works. As the models are improved, they learn more about re

> how much are GenAI companies willing to invest to get eyeball reflections right This isn't how it works. As the models are improved, they learn more about reality largely on their own. Except for glaringly obvious problems (like hands, deformed limbs, etc) the improvements are really just giving the models techniques for more accurately replicating features from reasoning data. There's nobody that's like "today we're working on fingernails" or "today we're making hair physics work better": it's about making the model understand and replicate the features already present in the training dataset.

sangnoir 2y ago
> This isn't how it works. As the models are improved, they learn more about reality largely on their own. AI models aren't complete blackboxes to the people who develop them: there is careful thought behind the architecture, dataset selection and model evaluation. Assuming that you can take an existing model and simply throw more compute at it will automatically result in higher fidelity illumination modeling takes almost religious levels of faith. If moar hardware is all you need, Nvidia would have the best models in every category right now. Perhaps someone ought to write the sequel to Fred Brooks' book amd name it "The Mythical GPU-Cluster-Month". FWIW, Google has AI-based illumination adjustment in Google Photos where one can add virtual lights - so specialized models for lighting already exist. However, I'm very cynical about a generic mixed model incidentally gaining those capabilities without specific training for it. When dealing with exponential requirements (training data, training time, GPUs, model weight size), you'll run out of resources in short order.
- devbent 2y ago
  > If moar hardware is all you need, Nvidia would have the best models in every category right now. Nvidia is making boatloads of money right now selling GPUs to companies that think they will be making boatloads of money in the future. Nvidia has the better end of things at this very moment in time.
- bastawhiz 2y ago
  What you're refuting isn't what I said. I'm making the point that nobody is encoding all of the individual features of the human form and reality into their models through code or model design. You build a model by making it capable of observing details and then letting it observe the details of your training data. Nobody is spending time getting the reflections in the eyeballs working well, that comes as an emergent property of a model that's able to identify and replicate that. That doesn't mean it's a black box, it means that it's built in a general way so the researchers don't need to care about every facet of reality.
- dragonwriter 2y ago
  > Assuming that you can take an existing model and simply throw more compute at it will automatically result in higher fidelity illumination modeling takes almost religious levels of faith. Seems an odd response to a poster who said “as the models are improved...”; the way the models are improved isn't just additional training to existing models, its updated model architectures.
janalsncm 2y ago
No, it’s a valid point, which I didn’t interpret as literally “we’re working on eyeballs today” but rather “we’re scaling up these imperfect methods to a trillion dollar GPU cluster”, the latter of which is genuinely something people talk about. The models will learn to mimic more and more of the long tail of the distribution of training data, which to us looks like an emergent understanding. So there’s a theoretical amount of data you could provide for them to memorize physical laws. The issue is practical. There isn’t enough data out there to learn the long tail. If neural nets genuinely understood the world they would be getting 100% on ARC.
- bastawhiz 2y ago
  You don't need a trillion dollar GPU to accomplish this stuff. Models from two years ago look incredibly different than the ones today, not just because they're bigger but because they're more sophisticated. And the data from two years ago looks a lot like the data today: messy and often poorly annotated. Even if we added no new compute and capped the resources used for inference and training, models would produce higher fidelity results over time due to improvements in the architecture.
jhanschoo 2y ago
I don't know where you get this opinion from as it doesn't match the landscape that I'm witnessing. Around the huge names are many companies and business units fine-tuning foundational models on their private datasets for the specific domains they are interested in. I can think of scenarios where someone is interested in training models to generate images with accurate reflections in specific settings.
- viraptor 2y ago
  There could be edge cases, but fine tuning doesn't normally concentrate on a single specific feature. With positive and negative examples you could definitely train the eyes, but it's not what people usually do. Fine tuning is widely used to provide a specific style, clothes, or other larger scale elements.
  - dragonwriter 2y ago
    > There could be edge cases, but fine tuning doesn't normally concentrate on a single specific feature. Normally is being strained here: yes, most finetuning isn't for things like this, but quite a substantial minority is for more accurate rendering of some narrow specific feature, especially ones typically identified as signature problems of AI image gen; publicly identifying this as a way to visually distinguish AI gens makes it more likely that fine tuning effort will be directed at addressing it.
- bastawhiz 2y ago
  > I don't know where you get this opinion from as it doesn't match the landscape that I'm witnessing. I'm an engineer at an AI company that makes text/image to video models, but go off
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - deleted 2y ago

sangnoir 2y ago

> This isn't how it works. As the models are improved, they learn more about reality largely on their own. AI models aren't complete blackboxes to the people who develop them: there is careful thought behind the architecture, dataset selection and model evaluation. Assuming that you can take an existing model and simply throw more compute at it will automatically result in higher fidelity illumination modeling takes almost religious levels of faith. If moar hardware is all you need, Nvidia would have the best models in every category right now. Perhaps someone ought to write the sequel to Fred Brooks' book amd name it "The Mythical GPU-Cluster-Month". FWIW, Google has AI-based illumination adjustment in Google Photos where one can add virtual lights - so specialized models for lighting already exist. However, I'm very cynical about a generic mixed model incidentally gaining those capabilities without specific training for it. When dealing with exponential requirements (training data, training time, GPUs, model weight size), you'll run out of resources in short order.

devbent 2y ago
> If moar hardware is all you need, Nvidia would have the best models in every category right now. Nvidia is making boatloads of money right now selling GPUs to companies that think they will be making boatloads of money in the future. Nvidia has the better end of things at this very moment in time.
bastawhiz 2y ago
What you're refuting isn't what I said. I'm making the point that nobody is encoding all of the individual features of the human form and reality into their models through code or model design. You build a model by making it capable of observing details and then letting it observe the details of your training data. Nobody is spending time getting the reflections in the eyeballs working well, that comes as an emergent property of a model that's able to identify and replicate that. That doesn't mean it's a black box, it means that it's built in a general way so the researchers don't need to care about every facet of reality.
dragonwriter 2y ago
> Assuming that you can take an existing model and simply throw more compute at it will automatically result in higher fidelity illumination modeling takes almost religious levels of faith. Seems an odd response to a poster who said “as the models are improved...”; the way the models are improved isn't just additional training to existing models, its updated model architectures.

janalsncm 2y ago

No, it’s a valid point, which I didn’t interpret as literally “we’re working on eyeballs today” but rather “we’re scaling up these imperfect methods to a trillion dollar GPU cluster”, the latter of which is genuinely something people talk about. The models will learn to mimic more and more of the long tail of the distribution of training data, which to us looks like an emergent understanding. So there’s a theoretical amount of data you could provide for them to memorize physical laws. The issue is practical. There isn’t enough data out there to learn the long tail. If neural nets genuinely understood the world they would be getting 100% on ARC.

bastawhiz 2y ago
You don't need a trillion dollar GPU to accomplish this stuff. Models from two years ago look incredibly different than the ones today, not just because they're bigger but because they're more sophisticated. And the data from two years ago looks a lot like the data today: messy and often poorly annotated. Even if we added no new compute and capped the resources used for inference and training, models would produce higher fidelity results over time due to improvements in the architecture.

jhanschoo 2y ago

I don't know where you get this opinion from as it doesn't match the landscape that I'm witnessing. Around the huge names are many companies and business units fine-tuning foundational models on their private datasets for the specific domains they are interested in. I can think of scenarios where someone is interested in training models to generate images with accurate reflections in specific settings.

viraptor 2y ago
There could be edge cases, but fine tuning doesn't normally concentrate on a single specific feature. With positive and negative examples you could definitely train the eyes, but it's not what people usually do. Fine tuning is widely used to provide a specific style, clothes, or other larger scale elements.
- dragonwriter 2y ago
  > There could be edge cases, but fine tuning doesn't normally concentrate on a single specific feature. Normally is being strained here: yes, most finetuning isn't for things like this, but quite a substantial minority is for more accurate rendering of some narrow specific feature, especially ones typically identified as signature problems of AI image gen; publicly identifying this as a way to visually distinguish AI gens makes it more likely that fine tuning effort will be directed at addressing it.
bastawhiz 2y ago
> I don't know where you get this opinion from as it doesn't match the landscape that I'm witnessing. I'm an engineer at an AI company that makes text/image to video models, but go off
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- deleted 2y ago