4 ms·
Your argument just assumes there is no latent structure that can be exploited. That's a big assumption.
by naasking 9mo ago
Your argument just assumes there is no latent structure that can be exploited. That's a big assumption.
- logicchains 9mo agoIt's a necessary assumption for the universal approximation property; if you assume some structure then your LLM can no longer solve problems that don't fit into that structure as effectively.
- naasking 9mo agoBut language does have structure, as does logic and reasoning. Universal approximation is great when you don't know the structure and want to brute force search to find an approximate solution. That's not optimal by any stretch of the imagination though.
- direwolf20 9mo agoNeural nets are structured as matrix multiplication, yet, they are universal approximators.
- noosphr 9mo agoYou're missing the non-linear activations.