4 ms·

Why are specialized CAD making LLM models not showing up? In future are we going to have same model for everything? from programming to creative writing to CADs

Why are specialized CAD making LLM models not showing up? In future are we going to have same model for everything? from programming to creative writing to CADs?

xnx 5mo ago
> In future are we going to have same model for everything? A model that knows more in general, will often be better at specific tasks. e.g. If you ask a model to "make a program that estimates the annual production of a solar installation", it needs to have been trained on a lot more than just Python code.
- lifty 5mo ago
  You might combine a general world model with a python coding model in that case. Not sure if it's better, just saying.
  - lukeschlather 5mo ago
    What's the difference between a "general world model combined with a python coding model" and a multimodal LLM?
- JumpCrisscross 5mo ago
  > A model that knows more in general, will often be better at specific tasks Is this your hypothesis or broad conclusion among AI experts?
  - xnx 5mo ago
    I'm not an expert, but I believe it's an accepted phenomenon that https://en.wikipedia.org/wiki/Transfer_learning https://en.wikipedia.org/wiki/Transfer_learning is more broadly applicable than expected. I think there was a paper that showed an AI trained on multiple games outperformed one trained on a single game. (Maybe it was https://deepmind.google/blog/sima-2-an-agent-that-plays-reasons-and-learns-with-you-in-virtual-3d-worlds/ https://deepmind.google/blog/sima-2-an-agent-that-plays-reas... ?)
embedding-shape 5mo ago
If you have a model that only know how to model CAD but also doesn't know history, and was trained on visual language of said history, how is it supposed to be able to model the Pantheon in the first place? It'd only be able to model exactly what you can describe with text, or even worse, exactly what it'd be able to visually extract from images via the vision encoders, for "vision models", but it'd be a far cry from what you see in this blogpost, would be my guess.
coderenegade 5mo ago
There are good information theoretic reasons to suspect that general models will be better than specialized ones, because knowledge and skills often overlap different areas, sometimes in surprising and unintuitive ways. And yes, I'm aware that that statement might seem to fly in the face of much of the past two years of industry development, where specialized models have been in vogue. I think they'll settle to being appropriate for low cost "good enough" applications, but I'm less convinced they'll have anywhere near the fidelity of larger frontier models.

Why are specialized CAD making LLM models not showing up? In future are we going to have same model for everything? from programming to creative writing to CADs?

xnx 5mo ago

> In future are we going to have same model for everything? A model that knows more in general, will often be better at specific tasks. e.g. If you ask a model to "make a program that estimates the annual production of a solar installation", it needs to have been trained on a lot more than just Python code.

lifty 5mo ago
You might combine a general world model with a python coding model in that case. Not sure if it's better, just saying.
- lukeschlather 5mo ago
  What's the difference between a "general world model combined with a python coding model" and a multimodal LLM?
JumpCrisscross 5mo ago
> A model that knows more in general, will often be better at specific tasks Is this your hypothesis or broad conclusion among AI experts?
- xnx 5mo ago
  I'm not an expert, but I believe it's an accepted phenomenon that https://en.wikipedia.org/wiki/Transfer_learning https://en.wikipedia.org/wiki/Transfer_learning is more broadly applicable than expected. I think there was a paper that showed an AI trained on multiple games outperformed one trained on a single game. (Maybe it was https://deepmind.google/blog/sima-2-an-agent-that-plays-reasons-and-learns-with-you-in-virtual-3d-worlds/ https://deepmind.google/blog/sima-2-an-agent-that-plays-reas... ?)

embedding-shape 5mo ago

If you have a model that only know how to model CAD but also doesn't know history, and was trained on visual language of said history, how is it supposed to be able to model the Pantheon in the first place? It'd only be able to model exactly what you can describe with text, or even worse, exactly what it'd be able to visually extract from images via the vision encoders, for "vision models", but it'd be a far cry from what you see in this blogpost, would be my guess.

coderenegade 5mo ago

There are good information theoretic reasons to suspect that general models will be better than specialized ones, because knowledge and skills often overlap different areas, sometimes in surprising and unintuitive ways. And yes, I'm aware that that statement might seem to fly in the face of much of the past two years of industry development, where specialized models have been in vogue. I think they'll settle to being appropriate for low cost "good enough" applications, but I'm less convinced they'll have anywhere near the fidelity of larger frontier models.